### Install Development Dependencies with Poetry

Source: https://github.com/togethercomputer/together-python/blob/main/CONTRIBUTING.md

Installs Together development requirements, including those for running the project, examples, linting, and tests, using Poetry. It also sets up pre-commit for automated formatting and linting.

```Bash
poetry install --with quality,tests
pre-commit install
```

--------------------------------

### Install Together Python Library

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Installs or upgrades the Together Python library using pip. This command ensures you have the latest version of the client for interacting with the Together API.

```shell
pip install --upgrade together
```

--------------------------------

### Async Completions with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Provides an example of performing asynchronous completions using the Together AI Python library. It sends multiple prompts concurrently and prints the generated text.

```python
import asyncio
from together import AsyncTogether

async_client = AsyncTogether()
prompts = [
    "Write a Next.js component with TailwindCSS for a header component.",
    "Write a python function for the fibonacci sequence",
]

async def async_chat_completion(prompts):
    tasks = [
        async_client.completions.create(
            model="codellama/CodeLlama-34b-Python-hf",
            prompt=prompt,
        )
        for prompt in prompts
    ]
    responses = await asyncio.gather(*tasks)

    for response in responses:
        print(response.choices[0].text)

asyncio.run(async_chat_completion(prompts))
```

--------------------------------

### Code Completion with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

An example of using the Together AI Python library to generate code completions. It specifies a code model and a prompt for code generation.

```python
from together import Together

client = Together()

response = client.completions.create(
    model="codellama/CodeLlama-34b-Python-hf",
    prompt="Write a Next.js component with TailwindCSS for a header component.",
    max_tokens=200,
)
print(response.choices[0].text)
```

--------------------------------

### Image Generation with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

An example of using the Together AI Python library to generate images. It specifies a prompt, model, and generation steps, then prints the base64 encoded image data.

```python
from together import Together

client = Together()

response = client.images.generate(
    prompt="space robots",
    model="stabilityai/stable-diffusion-xl-base-1.0",
    steps=10,
    n=4,
)
print(response.data[0].b64_json)
```

--------------------------------

### CLI: Fine-tuning Operations

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This section outlines bash commands for managing fine-tuning jobs via the Together AI CLI. It covers getting help, creating a fine-tuning job, listing jobs, retrieving job details, listing job events, and canceling a running job.

```bash
# Help
together fine-tuning --help

# Create fine-tune job
together fine-tuning create \
  --model togethercomputer/llama-2-7b-chat \
  --training-file file-711d8724-b3e3-4ae2-b516-94841958117d

# List fine-tune jobs
together fine-tuning list

# Retrieve fine-tune job details
together fine-tuning retrieve ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# List fine-tune job events
together fine-tuning list-events ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b

# Cancel running job
together fine-tuning cancel ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
```

--------------------------------

### Get Embeddings with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Demonstrates how to generate embeddings for a list of texts using the Together AI Python library. It includes a helper function to process texts and retrieve embeddings.

```python
from typing import List
from together import Together

client = Together()

def get_embeddings(texts: List[str], model: str) -> List[List[float]]:
    texts = [text.replace("\n", " ") for text in texts]
    outputs = client.embeddings.create(model=model, input = texts)
    return [outputs.data[i].embedding for i in range(len(texts))]

input_texts = ['Our solar system orbits the Milky Way galaxy at about 515,000 mph']
embeddings = get_embeddings(input_texts, model='togethercomputer/m2-bert-80M-8k-retrieval')

print(embeddings)
```

--------------------------------

### CLI: File Operations

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This section provides bash commands for managing files with the Together AI CLI. It includes commands for getting help, checking a file, uploading a file, listing files, retrieving file metadata, retrieving file content, and deleting a file.

```bash
# Help
together files --help

# Check file
together files check example.jsonl

# Upload file
together files upload example.jsonl

# List files
together files list

# Retrieve file metadata
together files retrieve file-6f50f9d1-5b95-416c-9040-0799b2b4b894

# Retrieve file content
together files retrieve-content file-6f50f9d1-5b95-416c-9040-0799b2b4b894

# Delete remote file
together files delete file-6f50f9d1-5b95-416c-9040-0799b2b4b894
```

--------------------------------

### Create Chat Completion with Text

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Generates a text-based chat completion using the Together Python client. This example demonstrates sending a simple text message to a specified model and printing the response.

```python
from together import Together

client = Together()

# Simple text message
response = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages=[{"role": "user", "content": "tell me about new york"}],
)
print(response.choices[0].message.content)
```

--------------------------------

### Run Integration Tests with Make

Source: https://github.com/togethercomputer/together-python/blob/main/CONTRIBUTING.md

Executes integration tests for the project using the 'make integration_tests' command. Note that this requires an API key and may incur charges; it's recommended to let the CI system handle these tests.

```Bash
make integration_tests
```

--------------------------------

### Run Unit Tests with Make

Source: https://github.com/togethercomputer/together-python/blob/main/CONTRIBUTING.md

Executes the unit tests for the project using the 'make tests' command. This is a crucial step to ensure code correctness before submitting changes.

```Bash
make tests
```

--------------------------------

### Format Code with Make

Source: https://github.com/togethercomputer/together-python/blob/main/CONTRIBUTING.md

Runs the code formatting process using the 'make format' command. This ensures code adheres to the project's styling standards before submission.

```Bash
make format
```

--------------------------------

### Together CLI: List Available Models

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Lists all available models that can be used with the Together CLI. This command helps in discovering and selecting models for various tasks.

```bash
together models list
```

--------------------------------

### Python: Create, List, Retrieve, Cancel, and Download Fine-tuning Jobs

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This Python code illustrates how to manage fine-tuning jobs using the Together AI SDK. It includes creating a new fine-tuning job with specified parameters, listing all fine-tuning jobs, retrieving details of a specific job, canceling a running job, and downloading a fine-tuned model or checkpoint.

```python
from together import Together

client = Together()

client.fine_tuning.create(
  training_file = 'file-d0d318cb-b7d9-493a-bd70-1cfe089d3815',
  model = 'meta-llama/Llama-3.2-3B-Instruct',
  n_epochs = 3,
  n_checkpoints = 1,
  batch_size = "max",
  learning_rate = 1e-5,
  suffix = 'my-demo-finetune',
  wandb_api_key = '1a2b3c4d5e.......',
)
client.fine_tuning.list() # lists all fine-tuned jobs
client.fine_tuning.retrieve(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # retrieves information on finetune event
client.fine_tuning.cancel(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # Cancels a fine-tuning job
client.fine_tuning.list_events(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") #  Lists events of a fine-tune job
client.fine_tuning.download(id="ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b") # downloads compressed fine-tuned model or checkpoint to local disk
```

--------------------------------

### Streaming Completions with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Demonstrates how to receive streaming responses for completions from the Together AI API. This is useful for real-time output display.

```python
from together import Together

client = Together()
stream = client.completions.create(
    model="codellama/CodeLlama-34b-Python-hf",
    prompt="Write a Next.js component with TailwindCSS for a header component.",
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)
```

--------------------------------

### Initialize Together Python Client

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Initializes the Together Python client. It can be initialized with an API key directly or by using the environment variable TOGETHER_API_KEY if set.

```python
from together import Together

client = Together(api_key="xxxxx")
```

--------------------------------

### Together CLI: Model Management Help

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Provides help information for the 'models' subcommand in the Together CLI. This is useful for understanding available model-related operations.

```bash
together models --help
```

--------------------------------

### Add Optional Dependency with Poetry

Source: https://github.com/togethercomputer/together-python/blob/main/CONTRIBUTING.md

Adds a new package as an optional dependency using Poetry. This involves adding it to the main group as optional, then updating the 'pyproject.toml' file and relocking the Poetry file.

```Bash
poetry add --optional [package_name]
poetry lock --no-update
```

--------------------------------

### Download Fine-Tuned Model Weights

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Downloads the fine-tuned model weights for a specific fine-tuning job using its unique identifier. Ensure you have the correct fine-tuning ID.

```bash
together fine-tuning download ft-c66a5c18-1d6d-43c9-94bd-32d756425b4b
```

--------------------------------

### CLI: Text Completions

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This bash command shows how to generate text completions using the Together AI CLI. It includes the prompt, the model to use, the maximum number of tokens, and a stop sequence. Streaming is enabled by default and can be disabled with `--no-stream`.

```bash
together completions \
  "Large language models are " \
  --model meta-llama/Llama-4-Scout-17B-16E-Instruct \
  --max-tokens 512 \
  --stop "."
```

--------------------------------

### Python: List Available Models

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This Python code snippet shows how to retrieve a list of all models supported by Together AI. It initializes the client and then calls the `models.list()` method to fetch the model information.

```python
from together import Together

client = Together()

models = client.models.list()

for model in models:
    print(model)
```

--------------------------------

### CLI: Image Generations

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This bash command demonstrates how to generate images using the Together AI CLI. It specifies the prompt for the image, the model to use, and the number of images to generate. The generated image is opened in the default viewer by default, which can be disabled with `--no-show`.

```bash
together images generate \
  "space robots" \
  --model stabilityai/stable-diffusion-xl-base-1.0 \
  --n 4
```

--------------------------------

### CLI: Chat Completions

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This bash command demonstrates how to perform chat completions using the Together AI CLI. It specifies system and user messages and the model to be used. Streaming is enabled by default and can be disabled with `--no-stream`.

```bash
together chat.completions \
  --message "system" "You are a helpful assistant named Together" \
  --message "user" "What is your name?" \
  --model meta-llama/Llama-4-Scout-17B-16E-Instruct
```

--------------------------------

### Python: Batch Inference Operations

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This Python code demonstrates how to use the batch API for inference jobs. It covers uploading a batch file, creating a batch job, monitoring its status, listing all batch jobs, and retrieving the output file content upon completion.

```python
from together import Together

client = Together()

# Upload the batch file
batch_file = client.files.upload(file="simpleqa_batch_student.jsonl", purpose="batch-api")

# Create the batch job
batch = client.batches.create_batch(file_id=batch_file.id, endpoint="/v1/chat/completions")

# Monitor the batch status
batch_stat = client.batches.get_batch(batch.id)

# List all batches - contains other batches as well
client.batches.list_batches()

# Download the file content if job completed
if batch_stat.status == 'COMPLETED':
    output_response = client.files.retrieve_content(id=batch_stat.output_file_id,
                                                    output="simpleqa_v3_output.jsonl")
```

--------------------------------

### Create Multi-modal Chat Completion with Video

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Generates a multi-modal chat completion that analyzes video content. The API supports video analysis by providing a video URL in the messages content.

```python
from together import Together

client = Together()

# Multi-modal message with text and video
response = client.chat.completions.create(
    model="Qwen/Qwen2.5-VL-72B-Instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's happening in this video?"
                },
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerFun.mp4"
                    }
                }
            ]
        }
    ]
)
print(response.choices[0].message.content)
```

--------------------------------

### Async Chat Completion with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Demonstrates how to perform asynchronous chat completions using the Together AI Python library. It sends multiple messages concurrently and prints the responses.

```python
import asyncio
from together import AsyncTogether

async_client = AsyncTogether()
messages = [
    "What are the top things to do in San Francisco?",
    "What country is Paris in?",
]

async def async_chat_completion(messages):
    async_client = AsyncTogether()
    tasks = [
        async_client.chat.completions.create(
            model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
            messages=[{"role": "user", "content": message}],
        )
        for message in messages
    ]
    responses = await asyncio.gather(*tasks)

    for response in responses:
        print(response.choices[0].message.content)

asyncio.run(async_chat_completion(messages))
```

--------------------------------

### Python: Upload, List, Retrieve, and Delete Files

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

This Python code demonstrates how to interact with the Together AI files API. It covers uploading a file, listing all uploaded files, retrieving a specific file by its ID, retrieving the content of a file, and deleting a file.

```python
from together import Together

client = Together()

client.files.upload(file="somedata.jsonl") # uploads a file
client.files.list() # lists all uploaded files
client.files.retrieve(id="file-d0d318cb-b7d9-493a-bd70-1cfe089d3815") # retrieves a specific file
client.files.retrieve_content(id="file-d0d318cb-b7d9-493a-bd70-1cfe089d3815") # retrieves content of a specific file
client.files.delete(id="file-d0d318cb-b7d9-493a-bd70-1cfe089d3815") # deletes a file
```

--------------------------------

### Fetch Logprobs for Chat Completions

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Shows how to retrieve logprobs (logarithms of token-level generation probabilities) for chat completions. Logprobs indicate the model's confidence in its output, useful for decision-making.

```python
from together import Together

client = Together()

response = client.chat.completions.create(
    model="meta-llama/Llama-3.2-3B-Instruct-Turbo",
    messages=[{"role": "user", "content": "tell me about new york"}],
    logprobs=1
)

response_lobprobs = response.choices[0].logprobs

print(dict(zip(response_lobprobs.tokens, response_lobprobs.token_logprobs)))
```

--------------------------------

### Create Multi-modal Chat Completion with Image

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Generates a multi-modal chat completion that includes both text and an image. The client processes the image content from a provided URL to generate a response.

```python
from together import Together

client = Together()

# Multi-modal message with text and image
response = client.chat.completions.create(
    model="meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's in this image?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/yosemite.png"
                    }
                }
            ]
        }
    ]
)
print(response.choices[0].message.content)
```

--------------------------------

### Set Together API Key Environment Variable

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Sets the TOGETHER_API_KEY environment variable, which is required for authenticating with the Together API. Replace 'xxxxx' with your actual API key obtained from Together.ai.

```shell
export TOGETHER_API_KEY=xxxxx
```

--------------------------------

### Create Multi-modal Chat Completion with Multiple Images

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Generates a multi-modal chat completion that compares multiple images. The API accepts an array of image URLs within the messages content for comparison tasks.

```python
from together import Together

client = Together()

# Multi-modal message with multiple images
response = client.chat.completions.create(
    model="Qwen/Qwen2.5-VL-72B-Instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Compare these two images."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/yosemite.png"
                    }
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/slack.png"
                    }
                }
            ]
        }
    ]
)
print(response.choices[0].message.content)
```

--------------------------------

### Pytest Mark for Dependency

Source: https://github.com/togethercomputer/together-python/blob/main/CONTRIBUTING.md

A Python code snippet demonstrating how to use the '@pytest.mark.requires(package_name)' decorator for tests that depend on a specific package. This is used in conjunction with optional dependencies.

```Python
@pytest.mark.requires(package_name)
```

--------------------------------

### Rerank Documents with Together AI

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Shows how to rerank a list of documents based on a query using the Together AI Python library. It returns the documents sorted by relevance score.

```python
from typing import List
from together import Together

client = Together()

def get_reranked_documents(query: str, documents: List[str], model: str, top_n: int = 3) -> List[str]:
    outputs = client.rerank.create(model=model, query=query, documents=documents, top_n=top_n)
    # sort by relevance score and returns the original docs
    return [documents[i] for i in [x.index for x in sorted(outputs.results, key=lambda x: x.relevance_score, reverse=True)]]

query = "What is the capital of the United States?"
documents = ["New York","Washington, D.C.", "Los Angeles"]

reranked_documents = get_reranked_documents(query, documents, model='Salesforce/Llama-Rank-V1', top_n=1)

print(reranked_documents)
```

--------------------------------

### Stream Chat Completion Response

Source: https://github.com/togethercomputer/together-python/blob/main/README.md

Generates a chat completion and streams the response back in chunks. This is useful for real-time applications where immediate feedback is desired.

```python
import os
from together import Together

client = Together()
stream = client.chat.completions.create(
    model="meta-llama/Llama-4-Scout-17B-16E-Instruct",
    messages=[{"role": "user", "content": "tell me about new york"}],
    stream=True,
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.