### Execute Example Script

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Makes an example Python script executable and then runs it. This is used to test or demonstrate functionality within the `examples/` directory.

```shell
#!/bin/bash
$ chmod +x examples/<your-example>.py
# run the example against your api
$ ./examples/<your-example>.py
```

--------------------------------

### Build and Install Wheel

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Builds a distributable wheel file for the package and then installs it using pip. This is an alternative to installing directly from source or git.

```shell
#!/bin/bash
$ uv build
# or
$ python -m build

$ pip install ./path-to-wheel-file.whl
```

--------------------------------

### Run Examples

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Adds and executes example scripts located in the `examples/` directory. Examples can be made executable and run directly. The `uv run python` prefix in the shebang line ensures the script runs within the uv-managed environment.

```python
# add an example to examples/<your-example>.py

#!/usr/bin/env -S uv run python
…
```

--------------------------------

### Install Together Python Library

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Installs the Together Python library from the production repository using pip. This command is used for initial setup or to get the latest version directly from GitHub.

```shell
pip install git+ssh://git@github.com/togethercomputer/together-py.git
```

--------------------------------

### Install from Git

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Installs the together-py package directly from its GitHub repository using pip. This is useful for using the latest development version or contributing to the project.

```shell
#!/bin/bash
$ pip install git+ssh://git@github.com/togethercomputer/together-py.git
```

--------------------------------

### Run Mock Server for Tests

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Starts a mock server using Prism against the OpenAPI specification. This is a prerequisite for running most of the project's tests, allowing for isolated testing.

```shell
# you will need npm installed
$ npx prism mock path/to/your/openapi.yml
```

--------------------------------

### Install Dependencies with uv

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Installs project dependencies using the uv package manager. It either runs a bootstrap script or syncs all extras. This method automatically provisions the correct Python version.

```shell
#!/bin/bash
$ ./scripts/bootstrap
```

```shell
#!/bin/bash
$ uv sync --all-extras
```

--------------------------------

### Install Dependencies with pip

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Installs project dependencies using pip if uv is not used. It requires ensuring the correct Python version is specified and a virtual environment is created prior to running the command.

```shell
#!/bin/bash
$ pip install -r requirements-dev.lock
```

--------------------------------

### Run Project Tests

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Executes the project's test suite. This command typically relies on a mock server being set up beforehand to validate the API interactions.

```shell
#!/bin/bash
$ ./scripts/test
```

--------------------------------

### Install Together Python Library with aiohttp support

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Installs the Together Python library with optional aiohttp support for enhanced asynchronous performance. This command ensures the necessary dependencies for `aiohttp` integration are included.

```shell
pip install 'together[aiohttp] @ git+ssh://git@github.com/togethercomputer/together-py.git'
```

--------------------------------

### Activate Virtual Environment

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Manually activates the Python virtual environment created by uv. Once activated, the `uv run` prefix can be omitted when executing Python scripts.

```shell
#!/bin/bash
$ source .venv/bin/activate

# now you can omit the `uv run` prefix
$ python script.py
```

--------------------------------

### Publish Release Manually

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Publishes the package to PyPI manually by running a script. This requires setting the PYPI_TOKEN environment variable with appropriate credentials.

```shell
#!/bin/bash
$ bin/publish-pypi
```

--------------------------------

### Lint Code

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Runs the linter (ruff) to check for code quality issues and style guide violations. This command helps maintain code consistency across the project.

```shell
#!/bin/bash
$ ./scripts/lint
```

--------------------------------

### Format Code

Source: https://github.com/togethercomputer/together-py/blob/main/CONTRIBUTING.md

Applies automatic code formatting using black and ruff to fix style issues. This command ensures the codebase adheres to the project's formatting standards.

```shell
#!/bin/bash
$ ./scripts/format
```

--------------------------------

### Async Chat Completion with aiohttp Backend

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Demonstrates using the `AsyncTogether` client with `aiohttp` as the HTTP backend for improved concurrency. This requires installing `aiohttp` and instantiating the client with `DefaultAioHttpClient()`.

```python
import os
import asyncio
from together import DefaultAioHttpClient
from together import AsyncTogether


async def main() -> None:
    async with AsyncTogether(
        api_key=os.environ.get("TOGETHER_API_KEY"),  # This is the default and can be omitted
        http_client=DefaultAioHttpClient(),
    ) as client:
        chat_completion = await client.chat.completions.create(
            messages=[
                {
                    "role": "user",
                    "content": "Say this is a test!",
                }
            ],
            model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
        )
        print(chat_completion.choices)


asyncio.run(main())
```

--------------------------------

### Manage HTTP client resources with context manager (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

This Python example shows how to manage the lifecycle of the HTTP client using a context manager. By using `with Together() as client:`, the underlying HTTP connections are automatically closed when exiting the block, ensuring proper resource management.

```python
from together import Together

with Together() as client:
  # make requests here
  ...
```

--------------------------------

### Configure Request Timeouts for API Calls (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Provides examples of configuring request timeouts for API calls using the Together AI Python SDK. It covers setting a default timeout for all requests, using a granular `httpx.Timeout` object, and overriding the timeout for individual requests.

```python
from together import Together
import httpx

# Configure the default for all requests:
client = Together(
    # 20 seconds (default is 1 minute)
    timeout=20.0,
)

# More granular control:
client = Together(
    timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=2.0),
)

# Override per-request:
client.with_options(timeout=5.0).chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
)
```

--------------------------------

### Stream API response content using context manager (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

This Python example illustrates how to stream API response content using `.with_streaming_response`. It requires a context manager and allows reading the response body incrementally via methods like `.iter_lines()`. This is useful for large responses where eager loading is not desired.

```python
with client.chat.completions.with_streaming_response.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
) as response:
    print(response.headers.get("X-My-Header"))

    for line in response.iter_lines():
        print(line)
```

--------------------------------

### Handle API Errors with Together AI SDK (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Illustrates how to handle potential API errors when interacting with the Together AI Python SDK. It specifically catches `APIConnectionError`, `RateLimitError`, and general `APIStatusError`, providing example error messages and accessing error details.

```python
import together
from together import Together

client = Together()

try:
    client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "Say this is a test",
            }
        ],
        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    )
except together.APIConnectionError as e:
    print("The server could not be reached")
    print(e.__cause__)  # an underlying Exception, likely raised within httpx.
except together.RateLimitError as e:
    print("A 429 status code was received; we should back off a bit.")
except together.APIStatusError as e:
    print("Another non-200-range status code was received")
    print(e.status_code)
    print(e.response)
```

--------------------------------

### Async Chat Completions with Together Python API

Source: https://context7.com/togethercomputer/together-py/llms.txt

Utilizes asynchronous operations for concurrent API calls, enhancing performance in async applications. This example demonstrates fetching chat completions asynchronously. It requires setting up an AsyncTogether client and running within an async function.

```python
import os
import asyncio
from together import AsyncTogether

client = AsyncTogether(
    api_key=os.environ.get("TOGETHER_API_KEY")
)

async def main():
    try:
        chat_completion = await client.chat.completions.create(
            messages=[
                {
                    "role": "user",
                    "content": "Explain quantum computing in simple terms."
                }
            ],
            model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
            max_tokens=256
        )
        print(chat_completion.choices[0].message.content)
    except Exception as e:
        print(f"Error: {e}")
    finally:
        await client.close()

asyncio.run(main())
```

--------------------------------

### Get Together Python Package Version

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Determine the currently installed version of the Together Python package. This is useful for debugging or ensuring compatibility with specific features.

```python
import together

print(together.__version__)
```

--------------------------------

### CLI command for Together files - Help (Bash)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

This Bash command displays the help information for the `together files` subcommand, providing an overview of its available options and usage.

```bash
together files --help
```

--------------------------------

### Get Evaluation Job Status - Python

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Retrieves the status of a specific evaluation job using its ID. Returns an EvalStatusResponse object.

```python
from together.types import EvalStatusResponse

# Assuming 'client' is an initialized TogetherClient instance
# eval_id: str = "your-eval-id"
# eval_status: EvalStatusResponse = client.evals.status(eval_id)
```

--------------------------------

### Configure HTTP client with proxies and transports (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

This Python code demonstrates how to configure the underlying `httpx` client for the Together library. It shows how to set custom `base_url`, proxies, and transports, allowing for advanced network configurations. The `DefaultHttpxClient` is used for this purpose.

```python
import httpx
from together import Together, DefaultHttpxClient

client = Together(
    # Or use the `TOGETHER_BASE_URL` env var
    base_url="http://my.test.server.example.com:8083",
    http_client=DefaultHttpxClient(
        proxy="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)
```

--------------------------------

### Fine-tuning Job Management with Together CLI

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Manage fine-tuning jobs for AI models using the Together CLI. This section covers creating, listing, retrieving details, managing events, canceling, and downloading fine-tuned models. Requires specifying model names and file identifiers.

```bash
# Get help for fine-tuning commands
together fine-tuning --help

# Create a new fine-tune job
together fine-tuning create \
  --model togethercomputer/llama-2-7b-chat \
  --training-file file-711d8724-b3e3-4ae2-b516-94841958117d

# List all fine-tune jobs
together fine-tuning list

# Retrieve details for a specific fine-tune job
together fine-tuning retrieve ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# List events for a fine-tune job
together fine-tuning list-events ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# List checkpoints for a fine-tune job
together fine-tuning list-checkpoints ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# Cancel a running fine-tune job
together fine-tuning cancel ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# Download fine-tuned model weights
together fine-tuning download ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# Delete fine-tuned model weights
together fine-tuning delete ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
```

--------------------------------

### Hardware API

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Provides information about available hardware resources for model deployment.

```APIDOC
## GET /hardware

### Description
Lists available hardware configurations.

### Method
GET

### Endpoint
/hardware

### Parameters
#### Query Parameters
- **params** (object) - Optional - Parameters for filtering the list of hardware.

### Response
#### Success Response (200)
- **HardwareListResponse** (object) - A list of available hardware configurations.

#### Response Example
```json
{
  "data": [
    {
      "name": "A100",
      "gpu_memory_gb": 40,
      "num_gpus": 1
    }
  ]
}
```
```

--------------------------------

### Create and Manage Fine-Tuning Jobs with Together AI Python SDK

Source: https://context7.com/togethercomputer/together-py/llms.txt

Covers the process of creating, monitoring, listing events for, and downloading fine-tuned models. It uses the 'together' library and includes error handling for job creation and monitoring loop that checks status updates and sleeps for 10 seconds between checks. It also handles downloading the model weights.

```python
import time
from together import Together

client = Together()

# Create fine-tuning job
try:
    file_id = "file-bf72b951-fa1a-41af-a152-fe385dca0201"
    fine_tune_model = client.fine_tuning.create(
        model="meta-llama/Meta-Llama-3-8B",
        training_file=file_id,
        hyperparameters={
            "n_epochs": 3,
            "learning_rate": 5e-5,
            "batch_size": 4
        }
    )
    print(f"Fine-tuning job created: {fine_tune_model.id}")
    fine_tune_id = fine_tune_model.id
except together.APIError as e:
    print(f"Failed to create fine-tuning job: {e}")
    exit(1)

# Monitor progress
while True:
    fine_tune_status = client.fine_tuning.retrieve(fine_tune_id)
    print(f"Status: {fine_tune_status.status}")

    if fine_tune_status.status == "completed":
        print(f"Training completed! Model: {fine_tune_status.output_name}")
        break
    elif fine_tune_status.status == "failed":
        print("Training failed")
        break
    else:
        time.sleep(10)

# List events
model_events = client.fine_tuning.list_events(fine_tune_id)
for event in model_events.data:
    print(f"[{event.created_at}] {event.type}: {event.message}")

# Download model weights
downloaded = client.fine_tuning.content(ft_id=fine_tune_id)
with open("fine_tuned_model.tar.gz", "wb") as f:
    f.write(downloaded.content)
```

--------------------------------

### Model Listing and Uploading with Together CLI

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Interact with AI models available on the Together AI platform. This includes listing all available models and uploading new models. Ensure models are properly sourced before uploading.

```bash
# Get help for model commands
together models --help

# List all available models
together models list

# Upload a new model
together models upload --model-name my-org/my-model --model-source s3-or-hugging-face
```

--------------------------------

### Enable Logging with TOGETHER_LOG Environment Variable (Shell)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Demonstrates how to enable logging for the Together AI Python SDK by setting the `TOGETHER_LOG` environment variable in a shell environment. It shows how to set it to 'info' for standard logging or 'debug' for more verbose output.

```shell
$ export TOGETHER_LOG=info
```

--------------------------------

### Synchronous Chat Completion with Together Python API

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Demonstrates how to create a synchronous chat completion using the Together Python client. It initializes the client with an API key (preferably from an environment variable) and sends a user message to a specified model.

```python
import os
from together import Together

client = Together(
    api_key=os.environ.get("TOGETHER_API_KEY"),  # This is the default and can be omitted
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test!",
        }
    ],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
)
print(chat_completion.choices)
```

--------------------------------

### Batch Processing with Together AI Python SDK

Source: https://context7.com/togethercomputer/together-py/llms.txt

Demonstrates how to manage batch jobs for efficient processing of multiple requests. Includes creating a batch job with an input file, listing existing jobs, retrieving status, and cancelling jobs if necessary. Handles API errors during batch operations.

```python
from together import Together

client = Together()

try:
    # Create batch job
    batch_job = client.batches.create(
        input_file_id="file-abc123",
        endpoint="/chat/completions",
        completion_window="24h"
    )
    print(f"Batch job created: {batch_job.id}")

    # List all batch jobs
    batches = client.batches.list()
    for batch in batches.data:
        print(f"- {batch.id}: {batch.status}")

    # Retrieve specific batch
    batch_status = client.batches.retrieve(batch_job.id)
    print(f"Status: {batch_status.status}")
    print(f"Completed: {batch_status.request_counts.completed}/{batch_status.request_counts.total}")

    # Cancel if needed
    if batch_status.status == "in_progress":
        cancelled = client.batches.cancel(batch_job.id)
        print(f"Batch cancelled: {cancelled.status}")

except together.APIError as e:
    print(f"Batch processing error: {e}")
```

--------------------------------

### Advanced HTTP Configuration

Source: https://context7.com/togethercomputer/together-py/llms.txt

Customize HTTP client with proxies, custom transports, and connection settings.

```APIDOC
## Advanced HTTP Configuration

### Description
Customize HTTP client with proxies, custom transports, and connection settings.

### Usage
Pass a configured `DefaultHttpxClient` instance to the `Together` constructor.

### Parameters
- **api_key** (string) - Your Together API key.
- **base_url** (string) - The base URL for the API.
- **timeout** (float) - Request timeout in seconds.
- **max_retries** (integer) - Maximum number of retries for failed requests.
- **http_client** (DefaultHttpxClient) - A configured HTTP client instance.
  - **proxy** (string) - URL for the proxy server.
  - **transport** (httpx.HTTPTransport) - Custom HTTP transport configuration.
    - **local_address** (string) - Local address to bind to.
    - **retries** (integer) - Number of retries for the transport layer.

### Example
```python
import httpx
from together import Together, DefaultHttpxClient

client = Together(
    api_key="your-api-key",
    base_url="https://api.together.xyz",
    timeout=30.0,
    max_retries=3,
    http_client=DefaultHttpxClient(
        proxy="http://proxy.example.com:8080",
        transport=httpx.HTTPTransport(
            local_address="0.0.0.0",
            retries=3
        )
    )
)

# Use the customized client
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"
)
print(response.choices[0].message.content)

# Close connections when done
client.close()
```
```

--------------------------------

### List Hardware - Python

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Retrieves a list of available hardware types, optionally with filtering parameters. Returns a HardwareListResponse object.

```python
from together.types import HardwareListResponse

# Assuming 'client' is an initialized TogetherClient instance
# params would be a dictionary conforming to HardwareListParams
# e.g., params = {"provider": "aws", ...}
# hardware_list: HardwareListResponse = client.hardware.list(**params)
```

--------------------------------

### Manage Files for Fine-Tuning with Together AI Python SDK

Source: https://context7.com/togethercomputer/together-py/llms.txt

Demonstrates file management operations such as uploading, listing, retrieving metadata, and deleting files. These files are intended for use with fine-tuning models. It utilizes the 'together' library and handles potential API and Not-Found errors.

```python
from together import Together

client = Together()

# Upload a file
try:
    with open("training_data.jsonl", "rb") as f:
        uploaded_file = client.files.create(
            file=f,
            purpose="fine-tune"
        )
    print(f"File uploaded: {uploaded_file.id}")
except together.APIError as e:
    print(f"Upload failed: {e}")

# List all files
files = client.files.list()
print(f"Total files: {len(files.data)}")
for file in files.data:
    print(f"- {file.id}: {file.filename} ({file.bytes} bytes)")

# Retrieve specific file metadata
if files.data and files.data[0]:
    file_id = files.data[0].id
    file_data = client.files.retrieve(file_id)
    print(f"File details: {file_data.filename}, created at {file_data.created_at}")

# Delete a file
try:
    delete_response = client.files.delete(file_id)
    print(f"File deleted: {delete_response.deleted}")
except together.NotFoundError:
    print("File not found")
```

--------------------------------

### Create Dedicated Endpoint with Autoscaling (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Demonstrates creating a dedicated endpoint with autoscaling configuration using the Together AI Python SDK. It utilizes TypedDict for nested autoscaling parameters and Pydantic models for the response object.

```python
from together import Together

client = Together()

dedicated_endpoint = client.endpoints.create(
    autoscaling={
        "max_replicas": 5,
        "min_replicas": 2,
    },
    hardware="1x_nvidia_a100_80gb_sxm",
    model="meta-llama/Llama-3-8b-chat-hf",
)
print(dedicated_endpoint.autoscaling)
```

--------------------------------

### CLI command for Together files - Upload (Bash)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

This Bash command uploads a specified JSONL file to the Together API. It requires the filename as an argument.

```bash
together files upload example.jsonl
```

--------------------------------

### Audio API - Speech

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Endpoints for text-to-speech synthesis.

```APIDOC
## POST /audio/speech

### Description
Synthesizes speech from the provided text.

### Method
POST

### Endpoint
/audio/speech

### Parameters

#### Request Body
- **model** (string) - Required - The model to use for speech synthesis.
- **input** (string) - Required - The text to synthesize.
- **voice** (string) - Optional - The voice to use for synthesis.

### Request Example
```json
{
  "model": "tts-1",
  "input": "Hello, world!",
  "voice": "alloy"
}
```

### Response
#### Success Response (200)
- **audio_stream** (BinaryAPIResponse) - The synthesized audio stream.

#### Response Example
(Binary audio data)
```

--------------------------------

### Asynchronous Chat Completion with Together Python API

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Shows how to perform asynchronous chat completions using the `AsyncTogether` client. This approach is suitable for applications requiring high concurrency, utilizing `asyncio` and `await` for non-blocking API calls.

```python
import os
import asyncio
from together import AsyncTogether

client = AsyncTogether(
    api_key=os.environ.get("TOGETHER_API_KEY"),  # This is the default and can be omitted
)


async def main() -> None:
    chat_completion = await client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "Say this is a test!",
            }
        ],
        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    )
    print(chat_completion.choices)


asyncio.run(main())
```

--------------------------------

### CLI command for Together files - Check (Bash)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

This Bash command checks the syntax and validity of a JSONL file intended for use with the Together API. It takes the filename as an argument.

```bash
together files check example.jsonl
```

--------------------------------

### Code Interpreter API

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Provides functionality to execute code within a secure sandbox environment.

```APIDOC
## POST /tci/execute

### Description
Executes code using the Together Code Interpreter.

### Method
POST

### Endpoint
/tci/execute

### Parameters
#### Query Parameters
None

#### Request Body
- **params** (object) - Required - Parameters for the code execution request. See `together.types.code_interpreter.execute_params` for details.

### Request Example
```json
{
  "code": "print('Hello from Code Interpreter!')"
}
```

### Response
#### Success Response (200)
- **ExecuteResponse** (object) - The result of the code execution. See `together.types.execute_response.ExecuteResponse` for details.

#### Response Example
```json
{
  "stdout": "Hello from Code Interpreter!\n",
  "stderr": "",
  "error": null,
  "logs": []
}
```
```

--------------------------------

### File Operations with Together CLI

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Manage files within the Together AI platform using the CLI. This includes checking, uploading, listing, retrieving metadata and content, and deleting files. Ensure files are correctly formatted before uploading.

```bash
# Help for files command
together files --help

# Check a local file
together files check example.jsonl

# Upload a local file
together files upload example.jsonl

# List all files in the remote storage
together files list

# Retrieve metadata for a specific file
together files retrieve file-6f50f9d1-5b95-416c-9040-0799b2b4b894

# Retrieve the content of a specific file
together files retrieve-content file-6f50f9d1-5b95-416c-9040-0799b2b4b894

# Delete a remote file
together files delete file-6f50f9d1-5b95-416c-9040-0799b2b4b894
```

--------------------------------

### Completions API

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Generates text completions for a given prompt. This is suitable for tasks like text generation, summarization, and more.

```APIDOC
## POST /completions

### Description
Creates a text completion request. This endpoint generates text based on a provided prompt.

### Method
POST

### Endpoint
/completions

### Parameters
#### Query Parameters
None

#### Request Body
- **params** (object) - Required - Parameters for the completion request. See `together.types.completion_create_params` for details.

### Request Example
```json
{
  "model": "model_name",
  "prompt": "Write a short story about a robot."
}
```

### Response
#### Success Response (200)
- **Completion** (object) - The generated text completion. See `together.types.completion.Completion` for details.

#### Response Example
```json
{
  "id": "cmpl-123",
  "object": "text_completion",
  "created": 1677652288,
  "model": "model_name",
  "choices": [
    {
      "text": "Once upon a time, in a futuristic city, lived a robot named Bolt...",
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 50,
    "total_tokens": 60
  }
}
```
```

--------------------------------

### List Batch Jobs - Python

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Retrieves a list of all batch jobs. Returns a BatchListResponse object.

```python
from together.types import BatchListResponse

# Assuming 'client' is an initialized TogetherClient instance
# batch_list: BatchListResponse = client.batches.list()
```

--------------------------------

### Fine-Tuning API

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Manages the fine-tuning process for models. This includes creating, retrieving, listing, canceling, and managing fine-tuning jobs.

```APIDOC
## GET /fine-tunes/{id}

### Description
Retrieves the details of a specific fine-tuning job.

### Method
GET

### Endpoint
/fine-tunes/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the fine-tuning job.

### Response
#### Success Response (200)
- **FinetuneResponse** (object) - Details of the fine-tuning job. See `together.types.finetune_response.FinetuneResponse` for details.

#### Response Example
```json
{
  "id": "ft-123",
  "object": "fine-tune",
  "model": "base-model-name",
  "status": "succeeded",
  "created_at": 1677652288,
  "fine_tuned_model": "my-custom-model"
}
```
```

```APIDOC
## GET /fine-tunes

### Description
Lists all fine-tuning jobs associated with the account.

### Method
GET

### Endpoint
/fine-tunes

### Parameters
None

### Response
#### Success Response (200)
- **FineTuningListResponse** (object) - A list of fine-tuning job objects. See `together.types.fine_tuning_list_response.FineTuningListResponse` for details.

#### Response Example
```json
{
  "object": "list",
  "data": [
    {
      "id": "ft-123",
      "object": "fine-tune",
      "model": "base-model-name",
      "status": "succeeded"
    }
    // ... more fine-tuning jobs
  ]
}
```
```

```APIDOC
## DELETE /fine-tunes/{id}

### Description
Deletes a specific fine-tuning job.

### Method
DELETE

### Endpoint
/fine-tunes/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the fine-tuning job to delete.
#### Request Body
- **params** (object) - Optional - Parameters for the delete request. See `together.types.fine_tuning_delete_params.FineTuningDeleteParams` for details.

### Response
#### Success Response (200)
- **FineTuningDeleteResponse** (object) - Confirmation of the fine-tuning job deletion. See `together.types.fine_tuning_delete_response.FineTuningDeleteResponse` for details.

#### Response Example
```json
{
  "id": "ft-123",
  "object": "fine-tune.deleted",
  "deleted": true
}
```
```

```APIDOC
## POST /fine-tunes/{id}/cancel

### Description
Cancels a fine-tuning job that is currently in progress.

### Method
POST

### Endpoint
/fine-tunes/{id}/cancel

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the fine-tuning job to cancel.

### Response
#### Success Response (200)
- **FineTuningCancelResponse** (object) - Confirmation of the cancellation. See `together.types.fine_tuning_cancel_response.FineTuningCancelResponse` for details.

#### Response Example
```json
{
  "id": "ft-123",
  "object": "fine-tune",
  "status": "canceled"
}
```
```

```APIDOC
## GET /finetune/download

### Description
Downloads the content related to a fine-tuning job.

### Method
GET

### Endpoint
/finetune/download

### Parameters
#### Query Parameters
- **params** (object) - Required - Parameters for the download request. See `together.types.fine_tuning_content_params.FineTuningContentParams` for details.

### Response
#### Success Response (200)
- **BinaryAPIResponse** - The raw content of the file.

#### Response Example
Binary file content (e.g., logs, model weights).
```

```APIDOC
## POST /fine-tunes/estimate-price

### Description
Estimates the price for a fine-tuning job based on provided parameters.

### Method
POST

### Endpoint
/fine-tunes/estimate-price

### Parameters
#### Request Body
- **params** (object) - Required - Parameters for estimating the price. See `together.types.fine_tuning_estimate_price_params.FineTuningEstimatePriceParams` for details.

### Response
#### Success Response (200)
- **FineTuningEstimatePriceResponse** (object) - The estimated price details. See `together.types.fine_tuning_estimate_price_response.FineTuningEstimatePriceResponse` for details.

#### Response Example
```json
{
  "estimated_price": 10.50,
  "currency": "USD"
}
```
```

```APIDOC
## GET /fine-tunes/{id}/checkpoints

### Description
Lists all checkpoints for a specific fine-tuning job.

### Method
GET

### Endpoint
/fine-tunes/{id}/checkpoints

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the fine-tuning job.

### Response
#### Success Response (200)
- **FineTuningListCheckpointsResponse** (object) - A list of checkpoint objects. See `together.types.fine_tuning_list_checkpoints_response.FineTuningListCheckpointsResponse` for details.

#### Response Example
```json
{
  "object": "list",
  "data": [
    {
      "id": "chkpt-123",
      "created_at": 1677652288
    }
    // ... more checkpoints
  ]
}
```
```

```APIDOC
## GET /fine-tunes/{id}/events

### Description
Lists all events for a specific fine-tuning job.

### Method
GET

### Endpoint
/fine-tunes/{id}/events

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the fine-tuning job.

### Response
#### Success Response (200)
- **FineTuningListEventsResponse** (object) - A list of event objects. See `together.types.fine_tuning_list_events_response.FineTuningListEventsResponse` for details.

#### Response Example
```json
{
  "object": "list",
  "data": [
    {
      "id": "evt-123",
      "event": "start",
      "created_at": 1677652288
    }
    // ... more events
  ]
}
```
```

--------------------------------

### Context Manager Usage

Source: https://context7.com/togethercomputer/together-py/llms.txt

Use context managers for automatic resource cleanup with the Together client.

```APIDOC
## Context Manager Usage

### Description
Use context managers for automatic resource cleanup.

### Usage
Instantiate the `Together` client within a `with` statement to ensure HTTP connections are automatically closed.

### Example
```python
from together import Together

with Together() as client:
    response = client.chat.completions.create(
        messages=[{"role": "user", "content": "What is AI?"}],
        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"
    )
    print(response.choices[0].message.content)
# HTTP connections automatically closed here
```
```

--------------------------------

### Create Evaluation Job - Python

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Creates a new evaluation job for model assessment. Requires evaluation creation parameters. Returns an EvalCreateResponse object.

```python
from together.types import EvalCreateResponse

# Assuming 'client' is an initialized TogetherClient instance
# params would be a dictionary conforming to EvalCreateParams
# e.g., params = {"name": "my-eval", "model": "model-name", "dataset": "dataset-name", ...}
# eval_create_response: EvalCreateResponse = client.evals.create(**params)
```

--------------------------------

### Jobs API

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Endpoints for managing jobs.

```APIDOC
## GET /jobs/{jobId}

### Description
Retrieves a specific job by its ID.

### Method
GET

### Endpoint
/jobs/{jobId}

### Parameters
#### Path Parameters
- **jobId** (string) - Required - The ID of the job to retrieve.

### Request Example
None

### Response
#### Success Response (200)
- **id** (string) - The ID of the job.
- **status** (string) - The status of the job.
- **result** (any) - The result of the job (if completed).

#### Response Example
```json
{
  "id": "job_abc123",
  "status": "completed",
  "result": {
    "output": "Job completed successfully."
  }
}
```

## GET /jobs

### Description
Lists all jobs.

### Method
GET

### Endpoint
/jobs

### Parameters

#### Query Parameters
None

### Request Example
None

### Response
#### Success Response (200)
- **data** (array) - A list of job objects.

#### Response Example
```json
{
  "data": [
    {
      "id": "job_abc123",
      "status": "completed"
    },
    {
      "id": "job_def456",
      "status": "processing"
    }
  ]
}
```
```

--------------------------------

### Fine-Tuning API

Source: https://context7.com/togethercomputer/together-py/llms.txt

Create and manage fine-tuning jobs to customize models on your own data.

```APIDOC
## Fine-Tuning

### Description
Create and manage fine-tuning jobs to customize models on your own data.

### Method
POST

### Endpoint
/fine_tuning/jobs

### Parameters
#### Request Body
- **model** (string) - Required - The base model to fine-tune (e.g., "meta-llama/Meta-Llama-3-8B").
- **training_file** (string) - Required - The ID of the file containing training data.
- **hyperparameters** (object) - Optional - Hyperparameter settings for the fine-tuning job.
  - **n_epochs** (integer) - Optional - Number of training epochs.
  - **learning_rate** (float) - Optional - Learning rate for training.
  - **batch_size** (integer) - Optional - Batch size for training.

### Request Example
```json
{
  "model": "meta-llama/Meta-Llama-3-8B",
  "training_file": "file-bf72b951-fa1a-41af-a152-fe385dca0201",
  "hyperparameters": {
    "n_epochs": 3,
    "learning_rate": 5e-5,
    "batch_size": 4
  }
}
```

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the fine-tuning job.
- **status** (string) - The current status of the fine-tuning job (e.g., "pending", "running", "completed", "failed").
- **output_name** (string) - The name of the fine-tuned model if the job is completed.

#### Response Example
```json
{
  "id": "ftjob-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "status": "completed",
  "output_name": "ft:meta-llama/Meta-Llama-3-8B:my-org:my-model:12345"
}
```

## Retrieve Fine-Tuning Job

### Method
GET

### Endpoint
/fine_tuning/jobs/{fine_tune_id}

### Parameters
#### Path Parameters
- **fine_tune_id** (string) - Required - The ID of the fine-tuning job to retrieve.

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the fine-tuning job.
- **status** (string) - The current status of the fine-tuning job.
- **output_name** (string) - The name of the fine-tuned model if the job is completed.

#### Response Example
```json
{
  "id": "ftjob-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "status": "completed",
  "output_name": "ft:meta-llama/Meta-Llama-3-8B:my-org:my-model:12345"
}
```

## List Fine-Tuning Events

### Method
GET

### Endpoint
/fine_tuning/jobs/{fine_tune_id}/events

### Parameters
#### Path Parameters
- **fine_tune_id** (string) - Required - The ID of the fine-tuning job to list events for.

### Response
#### Success Response (200)
- **data** (array) - An array of event objects.
  - **created_at** (string) - Timestamp of the event.
  - **type** (string) - Type of the event.
  - **message** (string) - Description of the event.

#### Response Example
```json
{
  "data": [
    {
      "created_at": "2023-01-01T12:00:00Z",
      "type": "log",
      "message": "Training started."
    }
  ]
}
```

## Download Fine-Tuning Model Content

### Method
GET

### Endpoint
/fine_tuning/jobs/{ft_id}/content

### Parameters
#### Path Parameters
- **ft_id** (string) - Required - The ID of the fine-tuning job.

### Response
#### Success Response (200)
- **content** (binary) - The binary content of the fine-tuned model weights (tar.gz).

#### Response Example
(Binary content, not a JSON example)
```

--------------------------------

### List Availability Zones - Python

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Retrieves a list of available zones for clusters. Returns an EndpointListAvzonesResponse object.

```python
from together.types import EndpointListAvzonesResponse

# Assuming 'client' is an initialized TogetherClient instance
# avzones_response: EndpointListAvzonesResponse = client.endpoints.list_avzones()
```

--------------------------------

### Streaming Chat Completion (Asynchronous)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Demonstrates asynchronous streaming of chat completion responses using the `AsyncTogether` client. Similar to the synchronous version, `stream=True` enables SSE, processed asynchronously with `async for`.

```python
from together import AsyncTogether

client = AsyncTogether()

stream = await client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    stream=True,
)
async for chat_completion in stream:
    print(chat_completion.choices)
```

--------------------------------

### Context Manager Usage for Together AI Python SDK

Source: https://context7.com/togethercomputer/together-py/llms.txt

Illustrates using a context manager (`with Together() as client:`) for the Together AI client. This ensures that HTTP connections are automatically closed upon exiting the block, simplifying resource management.

```python
from together import Together

with Together() as client:
    response = client.chat.completions.create(
        messages=[{"role": "user", "content": "What is AI?"}],
        model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"
    )
    print(response.choices[0].message.content)
# HTTP connections automatically closed here
```

--------------------------------

### Configure Automatic Retries for API Requests (Python)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Shows how to configure the automatic retry mechanism for API requests using the Together AI Python SDK. It demonstrates setting a default for all requests by initializing the client with `max_retries` and overriding it for a specific request using `with_options`.

```python
from together import Together

# Configure the default for all requests:
client = Together(
    # default is 2
    max_retries=0,
)

# Or, configure per-request:
client.with_options(max_retries=5).chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
)
```

--------------------------------

### Videos API

Source: https://github.com/togethercomputer/together-py/blob/main/api.md

Endpoints for video creation and retrieval.

```APIDOC
## POST /videos

### Description
Creates a new video job.

### Method
POST

### Endpoint
/videos

### Parameters

#### Request Body
- **prompt** (string) - Required - The text prompt for video generation.
- **input** (string) - Optional - Path to a video file to use as input.

### Request Example
```json
{
  "prompt": "A drone flying over a futuristic city"
}
```

### Response
#### Success Response (200)
- **id** (string) - The ID of the created video job.
- **status** (string) - The status of the video job.

#### Response Example
```json
{
  "id": "vidjob_abcdef123",
  "status": "processing"
}
```

## GET /videos/{id}

### Description
Retrieves a specific video job by its ID.

### Method
GET

### Endpoint
/videos/{id}

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the video job to retrieve.

### Request Example
None

### Response
#### Success Response (200)
- **id** (string) - The ID of the video job.
- **status** (string) - The current status of the video job.
- **url** (string) - The URL of the generated video (if available).

#### Response Example
```json
{
  "id": "vidjob_abcdef123",
  "status": "succeeded",
  "url": "https://example.com/video.mp4"
}
```
```

--------------------------------

### Streaming Chat Completion (Synchronous)

Source: https://github.com/togethercomputer/together-py/blob/main/README.md

Illustrates how to receive streaming responses for chat completions using the synchronous `Together` client. The `stream=True` parameter enables Server-Sent Events (SSE), allowing incremental output.

```python
from together import Together

client = Together()

stream = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    stream=True,
)
for chat_completion in stream:
    print(chat_completion.choices)
```