Magemaker by SlashML

Deploy open source AI models to AWS, GCP, and Azure in minutes.
📚 Documentation »

Join our Discord

Table of Contents

1. About Magemaker
2. Getting Started
   • Prerequisites
   • Installation
3. Using Magemaker
4. What we're working on next
5. Known issues
6. Contributing
7. License
8. Contact

About Magemaker

Magemaker is a Python tool that simplifies the process of deploying an open source AI model to your own cloud. Instead of spending hours digging through documentation, Magemaker lets you deploy Hugging Face models directly to AWS SageMaker, Google Cloud Vertex AI, or Azure Machine Learning.

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

• Python 3.11+ (except 3.12)
• Cloud provider account(s):
  • AWS for SageMaker
  • GCP for Vertex AI
  • Azure for Azure ML
• Cloud CLI tools:
  • AWS CLI (optional)s
  • Google Cloud SDK for GCP
  • Azure CLI for Azure
• Hugging Face account (for access to models)

Installing the package

pip install magemaker

Run it by simply doing the following:

magemaker

The first time you run this command, it will ask you for a cloud provider flag. You can specify it using magemaker --cloud gcp. The possible options for cloud provider flags are gcp|aws|azure|all, where all will configure all providers. If this is your first time running it, it will take some time to configure it. You'll be guided through setting up your chosen cloud provider(s).

Using Magemaker

Deploying models from dropdown

When you run magemaker command it will give you an interactive menu to deploy models. You can choose from a dropdown of models to deploy.

Deploy using a yaml file

We recommend deploying through a yaml file for reproducibility and IAC. Deploy via YAML files by passing the --deploy option:

magemaker --deploy .magemaker_config/model-config.yaml

Example YAML files for deploying facebook/opt-125m:

AWS (SageMaker):

deployment: !Deployment
  destination: aws
  endpoint_name: facebook--opt-125m
  instance_count: 1
  instance_type: ml.m5.xlarge

models:
- !Model
  id: facebook/opt-125m
  source: huggingface
  task: text-generation

GCP (Vertex AI):

deployment: !Deployment
  destination: gcp
  endpoint_name: test-endpoint-12
  accelerator_count: 1
  instance_type: g2-standard-12
  accelerator_type: NVIDIA_L4

models:
- !Model
  id: facebook/opt-125m
  source: huggingface

Azure ML:

deployment: !Deployment
  destination: azure
  endpoint_name: facebook--opt-125m
  instance_count: 1
  instance_type: Standard_DS3_v2

models:
- !Model
  id: facebook/opt-125m
  source: huggingface
  task: text-generation

Deactivating models

⚠️ Any model endpoints you spin up will run continuously unless you deactivate them! Make sure to delete endpoints you're no longer using to avoid unnecessary charges.

What we're working on next

• More robust error handling for various edge cases
• Verbose logging
• Enabling / disabling autoscaling
• Enhanced multi-cloud support features

Known issues

• Querying within Magemaker currently only works with text-based models
• Deleting a model is not instant, it may show up briefly after deletion
• Deploying the same model within the same minute will break

License

Distributed under the Apache 2.0 License. See LICENSE for more information.

Contact

You can reach us through:

• Email: support@slashml.com
• Discord: Join our community
• Documentation: magemaker.slashml.com

We'd love to hear from you! We're excited to learn how we can make this more valuable for the community and welcome any and all feedback and suggestions.