Tromero Python API library

The Tromero Python API Library is the official Python client for Tromero’s API platform, providing a convenient way for interacting with the REST APIs and enables easy integrations with Python 3.6+ applications with easy to use synchronous and asynchronous clients.

Installation

To install Tromero Python Library from PyPI, simply run:

pip install --upgrade tromero

Setting up API Key

✨ You will need to create an account with Tromero.ai to obtain a Tromero API Key. ✨

Using the client

First, import the TailorAI class from the AITailor package:

from tromero import Tromero

Initializing the Client

Initialize the TailorAI client using your API keys, which should be stored securely and preferably as environment variables:

client = TailorAI(tromero_key="your-tromero-key", save_data_default=True)

If you have a preference over the location of the Models, you can specify that in the client

client = TailorAI(tromero_key="your-tromero-key", save_data_default=True, location="uk")

There are different model availability in different regions, so by selecting a region you may be limiting the choice of base models. The client parameter for location takes priority over the settings on the Tromero platform.

If you require openai models you need to specify an openai key.

client = TailorAI(api_key="your-openai-key", tromero_key="your-tromero-key", save_data_default=True)

Usage – Python Client

response = client.chat.completions.create(
    model="gpt-4-turbo-preview",
    messages=[
        {"role": "user", "content": prompt},
    ],
    )

And for your trained model

response = client.chat.completions.create(
    model="your-model-name",
    messages=[
        {"role": "user", "content": prompt},
    ],
    save_data=False
    )

Json formatting

Tromero Tailor supports JSON response formatting, allowing you to specify the expected structure of the response using a JSON schema. Formatting works for models you have trained on tromero.

To utilize JSON formatting, you need to define a schema that describes the structure of the JSON object. The schema should conform to the JSON Schema standard. Here is an example schema:

schema = {
    'type': 'object',
    'properties': {
        'name': {'type': 'string'},
        'age': {'type': 'integer'}
    }
}

response = client.chat.completions.create(
    model="llama-3.1-70b-instruct",
    messages=[
        {"role": "user", "content": "Please provide your name and age."},
    ],
    guided_schema=improved_schema
)

Streaming

Tromero Tailor AI supports streaming responses, which allows you to receive and process data incrementally as it's generated.

Enabling Streaming

To enable streaming in your API calls, simply pass the parameter stream=True in your request. This tells the API to return the response incrementally, rather than waiting for the complete response to be ready.

Here's an example of how to initiate a streaming request:

response = client.chat.completions.create(
    model="gpt-4-turbo-preview",
    messages=[
        {"role": "user", "content": "Please describe the streaming process."},
    ],
    stream=True
)

Once you have initiated a streaming request, you can process the response as it arrives. Each chunk of the response will contain part of the data, and you can handle it within a loop. This is similar to how streaming is handled in the OpenAI API, making it familiar for users transitioning or using both services.

Here’s how you can access and process each chunk of the streamed response:

for chunk in response:
    chunk_message = chunk.choices[0].delta.content

Fallback Models

Tromero Tailor AI supports the specification of fallback models to ensure robustness and continuity of service, even when your primary model might encounter issues. You can configure a fallback model, which can be either a Tromero-hosted model or an OpenAI model, to be used in case the primary model fails.

Configuring a Fallback Model

To set up a fallback model, you simply specify the fallback_model parameter in your API calls. This parameter allows you to define an alternative model that the system should switch to in the event that the primary model fails to generate a response. The fallback model can be any other model that you have access to, whether selfhosted, hosted by Tromero or available through OpenAI.

Here’s an example of how to specify a fallback model in your API calls:

response = client.chat.completions.create(
    model="your-primary-model-name",
    fallback_model="gpt-4o"  # Fallback model
    messages=[
        {"role": "user", "content": "Please provide details about our new product."},
    ],
)

Saving Data for Fine-Tuning

To save data for future fine-tuning with Tromero, you must set save_data=True when initializing the TailorAI client. When save_data is true, Tromero will handle the formatting and saving of data automatically. Here’s how to initialize the client with data saving enabled:

client = TailorAI(api_key="your-openai-key", tromero_key="your-tromero-key", save_data=True)

Using Tags for Data Management

Tags help you sort and separate data when it comes to fine-tuning. By setting tags, you can easily manage and categorize the data collected during your interactions. You can pass tags in the create call as shown below:

response = client.chat.completions.create(
    model="your-primary-model-name",
    fallback_model="gpt-4o",  # Fallback model
    messages=[
        {"role": "user", "content": "Please provide details about our new product."},
    ],
    tags=["super_cool_model", "version1"]
)

By utilizing tags, you can ensure that your data is organized effectively, making the fine-tuning process more efficient and streamlined.

Utility Functions/Cli

In addition to the core functionalities, the Tromero package provides utility functions and a command-line interface (CLI) to manage your models and data on Tromero. These utilities are designed to help you seamlessly handle tasks such as training, deploying, and monitoring your models, as well as managing the data used for training.

The Functions

Models

List all the models you have on tromero

Python

client.tromero_models.list()

CLI

tromero models list

Deploy model

Python

client.tromero_models.deploy("{model_name}")

CLI

tromero models deploy --model_name '{model_name}'  

Uneploy model

Python

client.tromero_models.undeploy("{model_name}")

CLI

tromero models undeploy --model_name '{model_name}'  

Get model info

Python

client.tromero_models.get_info("{model_name}")

CLI

tromero models get_info --model_name '{model_name}'  

Data

List all the tags in your data

Python

client.data.get_tags()

CLI

tromero data get_tags 

Upload data

When uploading data to Tromero, you need to ensure that the file is in the correct format, specifically a JSONL file as specified on the Tromero website. The data will be tagged with the provided tags, making it easier to organize, sort, and train models later on.

Python

client.data.upload('{file_path}', ['tag1', 'tag2'])

CLI

tromero data upload --file_path='{file_path}' --tags tag1,tag2 

Tromero also provides an option to enhance your data by generating synthetic versions for improved training. You can enable this feature by passing the make_synthetic_version argument.

Python

client.data.upload('{file_path}', ['tag1', 'tag2'], make_synthetic_version=True)

CLI

tromero data upload --file_path='{file_path}' --tags tag1,tag2 --make_synthetic_version True

Datasets

Create Dataset From File

A dataset in Tromero is a collection of grouped data that can be used for future training purposes. By organizing your data into datasets, you can easily manage and fine-tune models based on specific subsets of your data.

Python

client.datasets.create_from_file(
    file_path='{file_path}', 
    name='your-dataset-name', 
    description='A brief description of your dataset', 
    tags=['tag1', 'tag2']
)

CLI

tromero datasets create_from_file --file_path='{file_path}' --name='your-dataset-name' --description='A brief description of your dataset' --tags tag1,tag2

In this example, {file_path} should be replaced with the path to your JSONL file. The name parameter assigns a name to your dataset, while the description provides a brief summary of the dataset's content or purpose. The tags help in categorizing and managing your data effectively.

Create Dataset From Tags

You can create a dataset in Tromero by grouping together previously uploaded data that shares specific tags. This allows you to efficiently organize and manage your data based on common themes or characteristics, making it easier to use for future training. Python

client.datasets.create_from_tags(
    name='your-dataset-name', 
    description='A brief description of your dataset', 
    tags=['tag1', 'tag2']
)

CLI

tromero datasets create_from_tags --name='your-dataset-name' --description='A brief description of your dataset' --tags tag1,tag2

List your datasets

client.datasets.list()

CLI

tromero datasets list

Fine Tuning Jobs

Create a fine tuning job

parameters = {
    'epoch': 1,
    'tags': ['tag1', 'tag2'],
    'custom_dataset': 'your-dataset-name'
}

response = client.fine_tuning_jobs.create(
    model_name='{model_name}',
    base_model='llama-3.1-70B-instruct',
    parameters=parameters
)

CLI

tromero fine_tuning_jobs create --base_model llama-3.1-70B-instruct --model_name {model_name} --parameters '{"epoch": 1}'

Get fine tuning job metrics

response = client.fine_tuning_jobs.get_metrics("{model_name}")

CLI

tromero fine_tuning_jobs get_metrics --model_name {model_name}