MosaicML

Overview

MosaicML, acquired by Databricks in 2023, is an enterprise platform for the development and deployment of large language models (LLMs). Originally founded in 2021, its focus has been on addressing the computational and operational challenges associated with training and fine-tuning LLMs. The platform is designed to allow organizations to build and deploy their own custom LLMs on private infrastructure, providing control over data and model artifacts.

The service facilitates the training of LLMs from scratch or fine-tuning of existing open-source models. It includes features intended to optimize training efficiency, such as advanced data loading, distributed training protocols, and inference acceleration techniques. This approach aims to reduce the overall cost and time required for LLM development projects. MosaicML is integrated into the broader Databricks Lakehouse Platform, enabling users to manage their LLM workflows alongside their existing data and AI initiatives.

MosaicML's offerings cater to developers and technical buyers who require custom LLM solutions that can be deployed within their own environments, addressing concerns related to data privacy and intellectual property. The platform supports a range of use cases, from developing domain-specific chatbots to enhancing enterprise search and knowledge management systems. Its compliance certifications, including SOC 2 Type II, GDPR, and HIPAA, are intended to meet regulatory requirements for sensitive data operations Databricks LLM Training.

The platform also provides access to the MPT (MosaicML Pretrained Transformer) family of models, which are open-source, commercially usable LLMs. These models are designed to be efficient for training and inference, providing a flexible base for further customization. For instance, the MPT-7B models offer capabilities comparable to other open-source models, and the MPT-30B series provides larger parameter counts for more complex tasks Hugging Face MosaicML models. The focus on open-source models aligns with a trend towards greater transparency and customizability in enterprise AI applications, as highlighted by industry discussions on AI model ownership and deployment strategies a16z on AI Model Ownership.

Key features

• LLM Training Platform: Provides tools and infrastructure for training large language models from initial datasets, including distributed training capabilities.
• Model Fine-tuning: Supports fine-tuning of pre-trained open-source models, enabling adaptation to specific datasets and tasks without starting training from scratch.
• Private LLM Deployment: Facilitates the deployment of LLMs within an organization's private cloud or on-premises infrastructure, offering data governance and security controls.
• Cost Optimization for LLMs: Includes techniques and optimizations aimed at reducing the computational resources and time required for LLM training and inference.
• MPT Model Family: Offers access to MosaicML Pretrained Transformer (MPT) models, which are commercially usable, open-source LLMs designed for efficient deployment and customization.
• Integration with Databricks Lakehouse: Seamlessly integrates with the Databricks platform, allowing users to leverage existing data pipelines and machine learning workflows for LLM development.
• Experiment Tracking and Management: Provides features to track training runs, monitor model performance, and manage different versions of models and datasets.
• Compliance and Security Features: Adheres to industry compliance standards such as SOC 2 Type II, GDPR, and HIPAA, supporting secure handling of sensitive data.

Pricing

MosaicML, as part of Databricks, follows a custom enterprise pricing model. Specific costs are determined based on usage, enterprise requirements, and the scope of services needed for LLM development and deployment.

Common integrations

• Databricks Lakehouse Platform: Deep integration with Databricks for data ingestion, processing, and ML workflow management Databricks LLM Documentation.
• Cloud Providers (AWS, Azure, GCP): Underlying infrastructure support for deploying and scaling LLM workloads on major cloud platforms.
• Hugging Face: Compatibility with Hugging Face models and libraries for leveraging a wide range of open-source LLMs and tools Hugging Face MosaicML Models.
• MLflow: Utilizes MLflow for experiment tracking, model registry, and managing the LLM development lifecycle.

Alternatives

• Hugging Face: Offers a platform for building, training, and deploying ML models, with a strong focus on open-source NLP models and a large community.
• AWS SageMaker: A fully managed service that helps developers and data scientists build, train, and deploy machine learning models quickly.
• Google Cloud Vertex AI: A managed machine learning platform that allows users to train and deploy ML models and AI applications.

Getting started

To begin using MosaicML integrated within Databricks, users typically start by setting up a Databricks workspace and configuring access to the LLM training and deployment features. The following Python code snippet illustrates a conceptual basic setup for loading a pre-trained MPT model and preparing it for fine-tuning or inference within a Databricks environment. This example assumes the necessary Databricks and MosaicML libraries are installed and accessible in the environment.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Define the MosaicML MPT model to load
model_name = "mosaicml/mpt-7b"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

# Move model to GPU if available
if torch.cuda.is_available():
    model.to("cuda")
    print("Model moved to GPU.")
else:
    print("GPU not available, model loaded on CPU.")

# Example: Generate text (simple inference)
# For actual training/fine-tuning, you would set up a training loop
# with a dataset, optimizer, and specific Databricks/MosaicML utilities.

prompt = "Write a short story about an AI assistant that discovers art."
input_ids = tokenizer.encode(prompt, return_tensors="pt")

if torch.cuda.is_available():
    input_ids = input_ids.to("cuda")

output = model.generate(input_ids, max_length=100, num_return_sequences=1)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

print("\n--- Generated Text ---")
print(generated_text)

# In a Databricks notebook, you would typically integrate with
# Databricks MLflow for logging and managing experiments.
# For example:
# import mlflow
# with mlflow.start_run():
#     mlflow.log_param("model_name", model_name)
#     mlflow.log_text(generated_text, "generated_story.txt")