Sutro

https://docs.sutro.sh
Admin
Sutro is a managed, scalable, and cost-effective platform for batch and offline inference with LLMs,...
Tokens:39,075
Snippets:112
Trust Score:7.1
Update:1 year ago
Show doc for...
Tokens:
Raw
### CLI API Key Setup

Source: https://docs.sutro.sh/python_sdk/Setting_Up

Instructions for setting the Sutro API key using the command-line interface (CLI) via the `sutro login` command.

```bash
sutro login
```

--------------------------------

### Python SDK API Key Setup

Source: https://docs.sutro.sh/python_sdk/Setting_Up

Demonstrates how to set the API key for the Sutro Python SDK using the `set_api_key` method. This is essential for authenticating with the Sutro API.

```python
set_api_key(self, api_key: str)

Set the API key for the Sutro API. **Parameters:**  
`api_key` (str): The API key to set. **Returns:** None
```

--------------------------------

### Sutro Python SDK Setup

Source: https://docs.sutro.sh/available_models

Instructions for setting up and using the Sutro Python SDK. This typically involves installation and basic configuration for interacting with Sutro services.

```python
pip install sutro

import sutro

# Initialize Sutro client (example)
client = sutro.Client(api_key='YOUR_API_KEY')
```

--------------------------------

### Infer with File Input (Python)

Source: https://docs.sutro.sh/quickstart

Perform inference using a CSV file as input. Requires specifying the column containing the input data and a system prompt. This example uses the `sutro` library.

```python
import sutro as so
results = so.infer(
    inputs='my_file.csv', 
    column='reviews', 
    system_prompt='Classify the review as positive, neutral, or negative.',
)
print(results)
```

--------------------------------

### Batch Inference API Example (Python)

Source: https://docs.sutro.sh/quickstart

Perform batch inference by sending a POST request to the Sutro API. Data should be preprocessed into a list or array and sent in the JSON body. Requires an API key for authorization.

```python
import requests
import json
user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]
params = {
    "model": "llama-3.1-8b",
    "inputs": user_reviews,
    "system_prompt": "Classify the review as positive, neutral, or negative.",
}
headers = {
    "Authorization": "Key <YOUR_API_KEY>",
    "Content-Type": "application/json"
}
response = requests.post("https://api.sutro.sh/batch-inference", json=params, headers=headers)
results = response.json()
```

--------------------------------

### Sutro Python SDK Setup

Source: https://docs.sutro.sh/index

Instructions and guidance on setting up the Sutro Python SDK for batch and offline inference with LLMs. This section covers installation and initial configuration.

```python
from sutro import Sutro

# Example usage (assuming setup is complete)
sutro = Sutro()
result = sutro.run_inference(...) 

```

--------------------------------

### Sutro Batch API Reference

Source: https://docs.sutro.sh/quickstart/index

Provides an overview of the Sutro Batch API, focusing on running batch inference jobs. This documentation outlines the general process and links to more detailed information.

```APIDOC
Endpoint: /batch/inference
Method: POST

Description:
Submits a batch inference job to the Sutro API. This endpoint is used for processing large datasets or multiple inference requests simultaneously.

Request Body:
- `model_id` (string, required): The identifier of the model to use for inference.
- `inputs` (array of strings or objects, required): The data to be processed by the model.
- `system_prompt` (string, optional): A prompt to guide the model's behavior.
- `output_schema` (object, optional): A schema to structure the model's output (e.g., Pydantic model or JSON schema).
- `dataset_id` (string, optional): If processing a pre-registered dataset.

Response:
- `job_id` (string): An identifier for the submitted batch job.
- `status` (string): The initial status of the job (e.g., 'queued', 'processing').

Example Usage:
POST /batch/inference
{
  "model_id": "claude-3-opus-20240229",
  "inputs": [
    "What is the capital of France?",
    "Translate 'hello' to Spanish."
  ],
  "system_prompt": "Respond concisely."
}

Related Endpoints:
- GET /jobs/{job_id}: Retrieve the status and results of a specific batch job.
- GET /jobs: List all submitted batch jobs.
```

--------------------------------

### Python SDK Inference with Structured Outputs

Source: https://docs.sutro.sh/quickstart/index

Illustrates how to achieve structured outputs from the Sutro Python SDK by using the `output_schema` parameter with a Pydantic model. This guides the LLM to produce output in a specific format, such as JSON.

```python
import sutro as so
from pydantic import BaseModel
so.set_api_key("sk_******") # this can be skipped if set via the CLI with `sutro login`
user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]
class ReviewClassification(BaseModel):
    classification: str
results = so.infer(user_reviews, system_prompt="Classify the review as positive, neutral, or negative.", output_schema=ReviewClassification)
print(results)
```

```python
[
    {"classification": "positive"},
    {"classification": "neutral"},
    {"classification": "negative"}
]
```

--------------------------------

### Basic Inference with Sutro Python SDK

Source: https://docs.sutro.sh/quickstart

Demonstrates how to perform basic inference using the Sutro Python SDK. It shows setting the API key, providing input data (user reviews), and a system prompt for classification. The output preserves the original order of the inputs.

```python
import sutro as so
so.set_api_key("sk_******") # this can be skipped if set via the CLI with `sutro login`
user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]
results = so.infer(user_reviews, system_prompt="Classify the review as positive, neutral, or negative.")
print(results)
```

```python
# Expected Output:
# [
#     'This review is classified as positive. The reviewer expressed a strong affection for the product ("I loved the product") and mentioned two specific benefits: it was "easy to use" and had a "great user interface".',
#     'This review is neutral.',
#     'This review is negative.'
# ]
```

--------------------------------

### Basic Python SDK Inference

Source: https://docs.sutro.sh/quickstart/index

Demonstrates a basic inference task using the Sutro Python SDK. It shows how to set the API key, pass a list of texts, and provide a system prompt for classification. The output preserves the original order of the inputs.

```python
import sutro as so
so.set_api_key("sk_******") # this can be skipped if set via the CLI with `sutro login`
user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]
results = so.infer(user_reviews, system_prompt="Classify the review as positive, neutral, or negative.")
print(results)
```

```python
[
    'This review is classified as positive. The reviewer expressed a strong affection for the product ("I loved the product") and mentioned two specific benefits: it was "easy to use" and had a "great user interface".',
    'This review is neutral.',
    'This review is negative.'
]
```

--------------------------------

### Sutro CLI Commands

Source: https://docs.sutro.sh/quickstart

Manage and monitor Sutro jobs using the command-line interface. Commands include listing jobs, checking job status, and retrieving job results.

```bash
sutro jobs
```

```bash
sutro status <job_id>
```

```bash
sutro results <job_id>
```

--------------------------------

### Structured Output Inference with Sutro Python SDK

Source: https://docs.sutro.sh/quickstart

Illustrates how to obtain structured outputs from the Sutro Python SDK by providing a Pydantic model or JSON schema via the `output_schema` parameter. This guides the LLM to produce output in a specific format, such as a JSON object with a 'classification' field.

```python
import sutro as so
from pydantic import BaseModel
so.set_api_key("sk_******") # this can be skipped if set via the CLI with `sutro login`
user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]
class ReviewClassification(BaseModel):
    classification: str
results = so.infer(user_reviews, system_prompt="Classify the review as positive, neutral, or negative.", output_schema=ReviewClassification)
print(results)
```

```python
# Expected Output:
# [
#     {"classification": "positive"},
#     {"classification": "neutral"},
#     {"classification": "negative"}
# ]
```

--------------------------------

### Python SDK - Setting Up

Source: https://docs.sutro.sh/concepts/cost-estimates

Instructions for setting up the Sutro Python SDK, likely involving installation and initial configuration.

```python
from sutro import Sutro

sutro = Sutro(api_key='YOUR_API_KEY')

```

--------------------------------

### Sutro Batch Inference API

Source: https://docs.sutro.sh/quickstart

Endpoint for performing batch inference. Accepts a JSON payload containing model, inputs, and system prompt. Requires an API key for authentication.

```APIDOC
POST https://api.sutro.sh/batch-inference

Request Body:
{
  "model": "string",
  "inputs": "list[string] | string",
  "column": "string (optional)",
  "system_prompt": "string"
}

Headers:
  Authorization: Key <YOUR_API_KEY>
  Content-Type: application/json

Response:
  {
    "results": "list[any]"
  }
```

--------------------------------

### Manage Jobs using Sutro CLI

Source: https://docs.sutro.sh/quickstart/index

Interact with Sutro jobs using the command-line interface. View all jobs, check the status of a specific job, or retrieve job results.

```bash
sutro jobs
```

```bash
sutro status <job_id>
```

```bash
sutro results <job_id>
```

--------------------------------

### Batch Inference via API

Source: https://docs.sutro.sh/quickstart/index

Perform batch inference by sending a POST request to the Sutro API. Data should be preprocessed into a list or array and sent in the JSON body.

```python
import requests
import json
user_reviews = [
    "I loved the product! It was easy to use and had a great user interface.",
    "The product was okay, but the customer support could be better.",
    "I had a terrible experience with the product. It didn't work as advertised and customer service was unhelpful."
]
params = {
    "model": "llama-3.1-8b",
    "inputs": user_reviews,
    "system_prompt": "Classify the review as positive, neutral, or negative.",
}
headers = {
    "Authorization": "Key <YOUR_API_KEY>",
    "Content-Type": "application/json"
}
response = requests.post("https://api.sutro.sh/batch-inference", json=params, headers=headers)
results = response.json()
```

--------------------------------

### Infer with Files using Python SDK

Source: https://docs.sutro.sh/quickstart/index

Perform inference using data from files (CSV, Parquet, TXT) with the Sutro Python SDK. Specify the input file and the relevant column for text data.

```python
import sutro as so
results = so.infer(
    inputs='my_file.csv', 
    column='reviews', 
    system_prompt='Classify the review as positive, neutral, or negative.',
)
print(results)
```

--------------------------------

### Sutro Python SDK Setup

Source: https://docs.sutro.sh/introduction

Instructions for setting up and using the Sutro Python SDK for batch and offline inference with LLMs. This SDK simplifies the process of interacting with the Sutro platform.

```python
from sutro import Sutro

sutro = Sutro(api_key='YOUR_API_KEY')

# Example usage:
# result = sutro.run_inference(...) 

```

--------------------------------

### Install Sutro Python SDK and CLI

Source: https://docs.sutro.sh/installation

Installs the Sutro Python SDK and CLI using pip. This is the primary method for setting up the command-line interface and programmatic access to Sutro's features.

```bash
pip install sutro
```

--------------------------------

### Run Production Jobs with Python SDK

Source: https://docs.sutro.sh/quickstart/index

Submit inference jobs to production (priority 1) using the Sutro Python SDK. This returns a job ID that can be used to track progress and retrieve results.

```python
import sutro as so
job_id = so.infer(inputs='my_file.csv', column='reviews', system_prompt='Classify the review as positive, neutral, or negative.', job_priority=1)
```

```python
import time
import sutro as so
job_id = so.infer(
    inputs="my_file.csv",
    column="reviews",
    system_prompt="Classify the review as positive, neutral, or negative.",
    job_priority=1,
)
results = so.await_job_completion(job_id)
```

--------------------------------

### Production Job with Priority (Python)

Source: https://docs.sutro.sh/quickstart

Submit an inference job with a production priority (priority 1). This returns a job ID immediately, which can be used to await job completion and retrieve results.

```python
import sutro as so
job_id = so.infer(
    inputs='my_file.csv', 
    column='reviews', 
    system_prompt='Classify the review as positive, neutral, or negative.', 
    job_priority=1
)

import time
results = so.await_job_completion(job_id)
```

--------------------------------

### Basic System Prompt Example

Source: https://docs.sutro.sh/concepts/system-prompts

Demonstrates how to use system prompts in Sutro for batch inference. This example shows classifying customer support tickets by providing a specific instruction set to the AI model to ensure consistent task orientation.

```python
import sutro as so
customer_support_conversations = ... # (code to retrieve customer support conversations from a database)
json_schema = {
    "type": "object",
    "properties": {
        "was_handled_properly": {
            "type": "boolean"
        }
    }
}
system_prompt = """
You are a overseeing customer support agents. 
Your job is to review dialogues between customers and customer support agents, and ensure that they were handled properly. 
You will be provided a dialogue, and you should respond with True if the dialogue was handled properly, and False otherwise."""
results = so.infer(
    inputs=dialogues,
    system_prompt=system_prompt,
    output_schema=json_schema
)
print(results)
```

--------------------------------

### List Jobs - Python Example

Source: https://docs.sutro.sh/batch_api_reference/List-jobs

Demonstrates how to fetch and process a list of jobs using the Sutro API in Python. It shows how to make a GET request, handle the JSON response, and iterate through job details.

```Python
import requests
response = requests.get(
    'https://api.sutro.sh/list-jobs/',
    headers={
        'Authorization': 'Key YOUR_SUTRO_API_KEY'
    }
)
result = response.json()
print(f"Message: {result['message']}")
print(f"Total jobs: {len(result['jobs'])}")
for job in result['jobs']:
    print(f"Job ID: {job['job_id']}")
    print(f"Status: {job['status']}")
    print(f"Model: {job['model']}")
    print(f"Input Count: {job['input_count']}")
    print(f"Created: {job['created_at']}")
    if job['completed_at']:
        print(f"Completed: {job['completed_at']}")
    print("---")
```

--------------------------------

### List Jobs - Node.js Example

Source: https://docs.sutro.sh/batch_api_reference/List-jobs

Provides a Node.js example for interacting with the Sutro List Jobs API. It uses the 'requests' library to make the API call and logs job information from the response.

```Node.js
const axios = require('axios');

async function listJobs() {
    try {
        const response = await axios.get('https://api.sutro.sh/list-jobs/', {
            headers: {
                'Authorization': 'Key YOUR_SUTRO_API_KEY'
            }
        });
        const result = response.data;
        console.log(`Message: ${result.message}`);
        console.log(`Total jobs: ${result.jobs.length}`);
        result.jobs.forEach(job => {
            console.log(`Job ID: ${job.job_id}`);
            console.log(`Status: ${job.status}`);
            console.log(`Model: ${job.model}`);
            console.log(`Input Count: ${job.input_count}`);
            console.log(`Created: ${job.created_at}`);
            if (job.completed_at) {
                console.log(`Completed: ${job.completed_at}`);
            }
            console.log("---");
        });
    } catch (error) {
        console.error('Error fetching jobs:', error);
    }
}

listJobs();
```

--------------------------------

### Basic Reasoning Model Example

Source: https://docs.sutro.sh/concepts/reasoning-models

Demonstrates how to use Sutro's reasoning models to solve problems step-by-step. It shows how to format input, specify a reasoning model, and process the output which includes both the reasoning trace and the final answer.

```python
import sutro as so
problems = [
    "What is 15% of 240?",
    "Explain why the sky appears blue"
]
results = so.infer(
    problems,
    model="qwen-3-14b-thinking",
    system_prompt="Solve this problem step by step"
)
# Each result contains both reasoning and final answer
for result in results:
    print(f"Reasoning: {result['reasoning_content']}")
    print(f"Answer: {result['content']}")
```

--------------------------------

### Check Job Status and Await Completion

Source: https://docs.sutro.sh/examples/dataset-inference-example

Shows how to submit an inference job and then check its status. It includes an example of a single status check and using `await_job_completion` to poll for updates and retrieve results once the job is finished.

```python
import time
import sutro as so
job_id = so.infer(
    inputs="my_file.csv",
    column="reviews",
    system_prompt="Classify the review as positive, neutral, or negative.",
    job_priority=1,
)
# Single check
status = so.get_job_status(job_id)
print(status)
# >> STARTING
# Use `await_job_completion` (blocking) to poll for status updates and
# retrieve results once they're ready
results = so.await_job_completion(job_id)
```

--------------------------------

### List Jobs - cURL Example

Source: https://docs.sutro.sh/batch_api_reference/List-jobs

Illustrates how to call the Sutro List Jobs API using cURL. This command-line example shows the necessary headers, including the Authorization key, to retrieve job data.

```cURL
curl -X GET \
  'https://api.sutro.sh/list-jobs/' \
  -H 'Authorization: Key YOUR_SUTRO_API_KEY'
```

--------------------------------

### Download Inference Results

Source: https://docs.sutro.sh/examples/dataset-inference-example

Demonstrates how to download the results of a completed batch inference job. It shows how to specify the dataset ID and an output path, and then how to read and inspect the downloaded parquet files.

```python
import sutro as so
dataset_id = "dataset-8be.."
# download the results, which will be in the output_dir directory, and will be named the same as the files in the dataset.
so.download_from_dataset(dataset_id, output_path="output_dir")
# you can inspect the inference results
df = pl.read_parquet("output_dir/*.parquet")
for row in df.iter_rows(named=True):
    print(f"prompt: {row['prompt']}, label: {row['inference_result']['label']}")
    break
```

--------------------------------

### List Datasets cURL Example

Source: https://docs.sutro.sh/datasets_api_reference/List-datasets

Provides a cURL command to list datasets, demonstrating how to authenticate and make the request.

```cURL
curl --request POST \
  --url https://api.sutro.sh/list-datasets \
  --header 'Authorization: <authorization>'
```

--------------------------------

### Submit Batch Inference Job with Dataset

Source: https://docs.sutro.sh/examples/dataset-inference-example

Demonstrates how to create a dataset, upload files, and submit a batch inference job using the Sutro Python SDK. It includes setting the API key, creating a dataset, uploading parquet files, defining a system prompt and output schema, and initiating the inference job with a specified model and priority.

```python
from import sutro as so
so.set_api_key("sk_******") # you can skip this if you set your key via `sutro login`
# create a dataset
dataset_id = so.create_dataset()
# Note: both files must have the same schema
so.upload_to_dataset(dataset_id, ["file1.parquet", "file2.parquet"])
system_prompt = "Label the following text as positive or negative."
json_schema = {
    "type": "object",
    "properties": {
        "label": {"type": "string", "enum": ["positive", "negative"]}
    },
    "required": ["label"]
}
# submit a batch inference job
so.infer(dataset_id, column="prompt", model="llama-3.2-3b", system_prompt=system_prompt, output_schema=json_schema, job_priority=1)
```

--------------------------------

### Fetch Datasets using Node.js

Source: https://docs.sutro.sh/datasets_api_reference/List-datasets

Provides an example of how to retrieve a list of datasets using the Sutro API with Node.js. It shows how to make a POST request and handle the response.

```Node.js
const fetch = require('node-fetch');

async function listDatasets() {
    const response = await fetch('https://api.sutro.sh/list-datasets', {
        method: 'POST',
        headers: {
            'Authorization': 'Key YOUR_SUTRO_API_KEY',
            'Content-Type': 'application/json'
        }
    });
    const result = await response.json();
    console.log(`Found ${result.datasets.length} datasets:`);
    result.datasets.forEach(dataset => {
        console.log(`Dataset ID: ${dataset.dataset_id}`);
        console.log(`Name: ${dataset.name}`);
        console.log(`Created: ${dataset.created_at}`);
        console.log(`Files: ${dataset.file_count}`);
        console.log(`Size: ${dataset.total_size_bytes} bytes`);
        console.log("---");
    });
}

listDatasets();
```

--------------------------------

### List Dataset Files - cURL Example

Source: https://docs.sutro.sh/datasets_api_reference/List-files

Provides a cURL command to interact with the Sutro API for listing dataset files. This example includes the necessary headers for authentication and content type, along with the JSON payload containing the dataset ID.

```cURL
curl -X POST \
  https://api.sutro.sh/list-dataset-files \
  -H 'Authorization: Key YOUR_SUTRO_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "dataset_id": "dataset_12345"
}'
```

--------------------------------

### cURL Example for Retrieving Results

Source: https://docs.sutro.sh/batch_api_reference/Job-results

Example of how to call the Sutro API to retrieve batch inference job results using cURL.

```curl
curl --request POST \
  --url https://api.sutro.sh/job-results/ \
  --header 'Authorization: <authorization>' \
  --header 'Content-Type: application/json' \
  --data '{ \
  "job_id": "<string>", \
  "include_inputs": true, \
  "include_cumulative_logprobs": true \
}'
```

--------------------------------

### Fetch Datasets using cURL

Source: https://docs.sutro.sh/datasets_api_reference/List-datasets

Illustrates how to list datasets from the Sutro API using a cURL command. This example includes setting the authorization header.

```cURL
curl -X POST \
  https://api.sutro.sh/list-datasets \
  -H 'Authorization: Key YOUR_SUTRO_API_KEY' \
  -H 'Content-Type: application/json'
```

--------------------------------

### Run Batch Inference with cURL

Source: https://docs.sutro.sh/batch_api_reference/running-batch-inference

Example of how to send a batch inference request to the Sutro API using cURL. This command-line example demonstrates setting the Authorization header and providing the JSON payload for the inference job.

```cURL
curl -X POST https://api.sutro.sh/batch-inference \
-H "Authorization: Key YOUR_SUTRO_API_KEY" \
-H "Content-Type: application/json" \
-d '{ 
  "inputs": [
    "What is the capital of France?",
    "Explain quantum computing in simple terms",
    "Write a haiku about programming"
  ],
  "model": "llama-3.1-8b",
  "system_prompt": "You are a helpful assistant.",
  "job_priority": 0
}'
```

--------------------------------

### Python Example: Reasoning Model with Pydantic Structured Output

Source: https://docs.sutro.sh/concepts/reasoning-models

Demonstrates how to use a Pydantic model for structured output with Sutro's reasoning models. The `output_schema` is applied to the `content` field, while `reasoning_content` captures the model's thought process.

```python
import sutro as so
from pydantic import BaseModel

class MathSolution(BaseModel):
    numerical_answer: float
    unit: str
    is_exact: bool

problems = [
    "A car travels 120 miles in 2.5 hours. What is its average speed?"
]

results = so.infer(
    problems,
    model="qwen-3-32b-thinking",
    system_prompt="Solve this physics problem",
    output_schema=MathSolution
)

# Example Result format:
# {
#     "reasoning_content": "To find average speed, I need to divide distance by time...",
#     "content": {
#         "numerical_answer": 48.0,
#         "unit": "miles per hour",
#         "is_exact": true
#     }
# }
```

--------------------------------

### List Datasets API Reference

Source: https://docs.sutro.sh/datasets_api_reference/List-datasets

This section details the API endpoint for listing all datasets. It includes the HTTP method, URL, required headers, and an example of the response structure.

```APIDOC
Datasets API
# Listing Datasets
List all datasets
POST
/list-datasets
Headers
Authorization: string (required) - Your Sutro API key using Key authentication scheme. Format: `Key YOUR_API_KEY` Example: `Authorization: Key sk_live_abc123...`
```

--------------------------------

### List Dataset Files - Python Example

Source: https://docs.sutro.sh/datasets_api_reference/List-files

Demonstrates how to use the Sutro API to list files in a dataset using Python's requests library. It shows how to set up the request headers, send the POST request with the dataset ID, and process the JSON response to display file information.

```Python
import requests
response = requests.post(
    'https://api.sutro.sh/list-dataset-files',
    headers={
        'Authorization': 'Key YOUR_SUTRO_API_KEY',
        'Content-Type': 'application/json'
    },
    json={
        'dataset_id': 'dataset_12345'
    }
)
result = response.json()
print(f"Found {len(result['files'])} files in dataset:")
for i, file in enumerate(result['files'], 1):
    print(f"{i}. {file['file_name']}")
    print(f"   File ID: {file['file_id']}")
    print(f"   Uploaded: {file['uploaded_at']}")
    print(f"   Size: {file['size_bytes']} bytes")
    print(f"   Rows: {file['row_count']}")
    print("---")
```

--------------------------------

### Create Dataset API

Source: https://docs.sutro.sh/datasets_api_reference/creating-datasets

This section details the API endpoint for creating a new internal dataset. It includes the HTTP method, URL, required headers, and an example of the expected JSON response. The Authorization header is mandatory for authentication.

```APIDOC
GET /create-dataset

Headers:
  Authorization: string (required) - Your Sutro API key using Key authentication scheme. Format: `Key YOUR_API_KEY`. Example: `Authorization: Key sk_live_abc123...`

Response:
A JSON object containing the dataset ID.
  dataset_id: string - The unique identifier for the created dataset. Example: `ds_abc123xyz`
```

```cURL
curl --request GET \
  --url https://api.sutro.sh/create-dataset \
  --header 'Authorization: <authorization>'
```

```Python
import requests
response = requests.get(
    'https://api.sutro.sh/create-dataset',
    headers={
        'Authorization': 'Key YOUR_SUTRO_API_KEY',
        'Accept': 'application/json'
    }
)
result = response.json()
print(f"Dataset created: {result['dataset_id']}")
```

--------------------------------

### cURL Request to List All Batch Jobs

Source: https://docs.sutro.sh/batch_api_reference/List-jobs

Example cURL command to list all batch inference jobs. Replace `<authorization>` with your actual Sutro API key.

```bash
curl --request GET \
  --url https://api.sutro.sh/list-jobs/ \
  --header 'Authorization: <authorization>'
```

--------------------------------

### Run Batch Inference with Python

Source: https://docs.sutro.sh/batch_api_reference/running-batch-inference

Example of how to send a batch inference request to the Sutro API using Python's requests library. It demonstrates setting the Authorization header, Content-Type, and the JSON payload with inputs and model configuration.

```Python
import requests
response = requests.post(
'https://api.sutro.sh/batch-inference',
headers={
'Authorization': 'Key YOUR_SUTRO_API_KEY',
'Content-Type': 'application/json'
},
json={
'inputs': [
'What is the capital of France?',
'Explain quantum computing in simple terms',
'Write a haiku about programming'
],
'model': 'llama-3.1-8b',
'system_prompt': 'You are a helpful assistant.',
'job_priority': 0
}
)
result = response.json()
print(f"Job created: {result['job_id']}")
```

--------------------------------

### Run Batch Inference with Node.js

Source: https://docs.sutro.sh/batch_api_reference/running-batch-inference

Example of how to send a batch inference request to the Sutro API using Node.js. It shows how to configure the request with necessary headers and a JSON body containing inference parameters.

```Node.js
const axios = require('axios');

async function runBatchInference() {
  try {
    const response = await axios.post('https://api.sutro.sh/batch-inference', {
      inputs: [
        'What is the capital of France?',
        'Explain quantum computing in simple terms',
        'Write a haiku about programming'
      ],
      model: 'llama-3.1-8b',
      system_prompt: 'You are a helpful assistant.',
      job_priority: 0
    }, {
      headers: {
        'Authorization': 'Key YOUR_SUTRO_API_KEY',
        'Content-Type': 'application/json'
      }
    });
    console.log(`Job created: ${response.data.job_id}`);
  } catch (error) {
    console.error('Error creating job:', error.response ? error.response.data : error.message);
  }
}

runBatchInference();
```

--------------------------------

### Get Job Status

Source: https://docs.sutro.sh/batch_api_reference/Job-status

Retrieves the status of a batch inference job using the Sutro SH API. This example shows how to make a GET request with an API key and parse the response to display job details.

```Python
import requests
job_id = "batch_job_12345"
response = requests.get(
    f'https://api.sutro.sh/job-status/{job_id}',
    headers={
        'Authorization': 'Key YOUR_SUTRO_API_KEY'
    }
)
result = response.json()
print(f"Job Status: {result['job_status']}")
print(f"Message: {result['message']}")
if result['job_status'] == 'failed':
    print(f"Failure Reason: {result['metadata'].get('failure_reason', 'Unknown')}")

```

--------------------------------

### Sutro Python SDK - Setting Up

Source: https://docs.sutro.sh/concepts/data-retention

Instructions for setting up and configuring the Sutro Python SDK for interacting with Sutro services.

```python
from sutro import Sutro

# Initialize the Sutro client with your API key
sutro = Sutro(api_key="YOUR_API_KEY")

# You can also set the API key as an environment variable:
# export SUTRO_API_KEY='YOUR_API_KEY'
sutro = Sutro()

# Example: Listing available models
models = sutro.list_models()
for model in models:
    print(f"Model Name: {model['name']}, Provider: {model['provider']}")

# Example: Uploading a dataset
# with open("my_data.jsonl", "rb") as f:
#     dataset = sutro.upload_dataset(file=f, name="my-dataset")
# print(f"Dataset uploaded with ID: {dataset['id']}")
```

--------------------------------

### Sutro Batch API Reference

Source: https://docs.sutro.sh/introduction

Reference documentation for the Sutro Batch API, detailing how to run batch inference jobs. This includes information on job submission and monitoring.

```APIDOC
POST /v1/batch/inference

Description:
  Submits a batch inference job to the Sutro platform.

Request Body:
  {
    "model_name": "string",       // Required: The name of the model to use
    "dataset_id": "string",       // Required: The ID of the dataset to process
    "output_path": "string",      // Optional: Path to store results
    "sampling_parameters": {      // Optional: Parameters for controlling inference
      "temperature": 0.7,
      "max_tokens": 100
    }
  }

Response:
  {
    "job_id": "string",           // The ID of the submitted job
    "status": "string"            // The initial status of the job (e.g., "queued")
  }

Error Conditions:
  - Invalid API key
  - Missing required parameters
  - Model not found
  - Dataset not found

```

--------------------------------

### Command Line Interface (CLI) - Account Management

Source: https://docs.sutro.sh/concepts/cost-estimates

Details on managing your Sutro account using the command-line interface, potentially including login and configuration commands.

```bash
sutro login --email YOUR_EMAIL --password YOUR_PASSWORD
sutro config set api_key YOUR_API_KEY

```

--------------------------------

### Get Job Status

Source: https://docs.sutro.sh/command_line_interface/Job_Commands

Retrieves the status of a job using its unique job ID.

```bash
sutro jobs status <job_id>
```

--------------------------------

### Sutro SDK Dataset Management

Source: https://docs.sutro.sh/python_sdk/datasets

Provides reference documentation for dataset operations using the Sutro Python SDK. This includes methods for creating, listing, uploading, and downloading files within datasets.

```APIDOC
Sutro SDK Dataset Methods:

create_dataset(self)
  - Creates a new internal dataset.
  - Returns: dict: A dictionary containing the dataset ID.

list_datasets(self)
  - Lists all datasets.
  - Returns: list: A list of dataset IDs.

list_dataset_files(self, dataset_id: str)
  - Lists all files in a specified dataset.
  - Parameters:
    - dataset_id (str): The ID of the dataset to list files from.
  - Returns: list: A list of file names in the dataset.

upload_to_dataset(self, dataset_id: List[str] | str = None, file_paths: List[str] | str = None)
  - Uploads files to a dataset. If dataset_id is not provided, files are uploaded to a new dataset.
  - Parameters:
    - dataset_id (Union[List[str], str], optional): The ID of the dataset to upload files to. Defaults to None.
    - file_paths (Union[List[str], str], optional): A list of file paths to upload. Defaults to None.
  - Returns: list: A list of file names in the dataset.

download_from_dataset(self, dataset_id: str, files: List[str] | str = None, output_path: str = None)
  - Downloads files from a dataset. If no file name is provided, all files in the dataset are downloaded.
  - Parameters:
    - dataset_id (str): The ID of the dataset to download files from.
    - files (Union[List[str], str], optional): The name(s) of the file(s) to download. Defaults to None.
    - output_path (str, optional): The directory to save downloaded files. Defaults to the current working directory.
  - Returns: None
```

--------------------------------

### Sutro CLI General Commands

Source: https://docs.sutro.sh/command_line_interface/Dataset_Commands

These commands provide access to Sutro documentation and general help information directly from the command line.

```bash
sutro docs

```

```bash
sutro 

```

```bash
sutro --help

```

--------------------------------

### Get Quotas

Source: https://docs.sutro.sh/python_sdk/Batch_Inference

Retrieves the current usage quotas for the account. This method returns a list containing quota information for each priority level, including row and token quotas.

```APIDOC
get_quotas(self) -> list

  Get your current quotas.

  Returns:
    list: A list of quotas, one for each priority level. Contains row_quota and token_quota for each priority level.
```

--------------------------------

### Sutro Batch API Reference

Source: https://docs.sutro.sh/available_models

Reference for the Sutro Batch API, detailing how to run batch inference jobs. This includes information on request formats, parameters, and expected responses.

```APIDOC
POST /v1/batch/inference

Request Body:
{
  "model_id": "string",
  "dataset_id": "string",
  "output_dataset_id": "string",
  "sampling_parameters": {
    "temperature": "number",
    "top_p": "number"
  }
}

Response:
{
  "job_id": "string",
  "status": "string"
}
```

--------------------------------

### Upload File to Dataset (Python)

Source: https://docs.sutro.sh/datasets_api_reference/Uploading

Provides a Python example using the requests library to upload a parquet file to a Sutro dataset. It includes setting the Authorization header, specifying the dataset ID, and handling the file upload.

```python
import requests
# Upload a parquet file
with open('data.parquet', 'rb') as file:
    response = requests.post(
        'https://api.sutro.sh/upload-to-dataset',
        headers={
            'Authorization': 'Key YOUR_SUTRO_API_KEY'
        },
        json={
            'dataset_id': 'dataset_12345'
        },
        files={
            'file': file
        }
    )
result = response.json()
if 'file_id' in result:
    print(f"File uploaded successfully: {result['file_id']}")
else:
    print(f"Upload failed: {result.get('error', 'Unknown error')}")
```

--------------------------------

### Sutro Batch API Reference

Source: https://docs.sutro.sh/index

Reference documentation for the Sutro Batch API, detailing how to run batch inference jobs. This includes information on job submission, monitoring, and retrieving results.

```APIDOC
POST /v1/batch/inference

Description:
  Submits a batch inference job.

Request Body:
  {
    "model_name": "string",
    "dataset_id": "string",
    "output_dataset_id": "string",
    "sampling_parameters": { ... },
    "system_prompt": "string"
  }

Response:
  {
    "job_id": "string",
    "status": "string"
  }

Error Codes:
  400: Invalid request payload
  401: Authentication failed
  500: Internal server error

Related Endpoints:
  GET /v1/batch/jobs/{job_id} - Retrieves the status of a batch job.
  GET /v1/batch/jobs/{job_id}/results - Retrieves the results of a batch job.
```

--------------------------------

### Fetch Datasets using Python

Source: https://docs.sutro.sh/datasets_api_reference/List-datasets

Demonstrates how to fetch a list of datasets using the Sutro API with Python's requests library. It includes setting the API key in the headers and processing the JSON response.

```Python
import requests
response = requests.post(
    'https://api.sutro.sh/list-datasets',
    headers={
        'Authorization': 'Key YOUR_SUTRO_API_KEY'
    }
)
result = response.json()
print(f"Found {len(result['datasets'])} datasets:")
for dataset in result['datasets']:
    print(f"Dataset ID: {dataset['dataset_id']}")
    print(f"Name: {dataset['name']}")
    print(f"Created: {dataset['created_at']}")
    print(f"Files: {dataset['file_count']}")
    print(f"Size: {dataset['total_size_bytes']} bytes")
    print("---")
```

--------------------------------

### Sutro Batch API Reference - Running Batch Inference

Source: https://docs.sutro.sh/concepts/random-seeds

Provides reference documentation for running batch inference using the Sutro API. This section details how to interact with the API for various inference tasks.

```APIDOC
POST /v1/batch/inference

Request Body:
{
  "model": "string",
  "inputs": [
    {
      "prompt": "string",
      "parameters": {
        "temperature": "number",
        "max_tokens": "integer",
        "random_seed": "integer"
      }
    }
  ],
  "random_seed_per_input": "boolean"
}

Response Body:
{
  "results": [
    {
      "output": "string",
      "logprobs": "object"
    }
  ]
}

Parameters:
- model (string, required): The name of the model to use for inference.
- inputs (array of objects, required): A list of input prompts and their parameters.
  - prompt (string, required): The input prompt for the model.
  - parameters (object, optional): Parameters for controlling the generation process.
    - temperature (number, optional): Controls randomness. Lower values make output more focused and deterministic.
    - max_tokens (integer, optional): The maximum number of tokens to generate.
    - random_seed (integer, optional): A seed for reproducibility. If `random_seed_per_input` is true, this can be set per input.
- random_seed_per_input (boolean, optional): If true, a random seed will be generated for each input, allowing for diverse outputs.

Example Usage:
curl -X POST https://api.sutro.sh/v1/batch/inference \
-H "Authorization: Bearer YOUR_API_KEY" \
-d '{ 
  "model": "sutro/claude-3-opus",
  "inputs": [
    { "prompt": "Generate a synthetic customer review for a coffee shop." },
    { "prompt": "Generate a synthetic customer review for a bookstore." }
  ],
  "random_seed_per_input": true
}'
```
Tokens:
Raw