### Install Together AI SDK

Source: https://docs.together.ai/docs/together-code-sandbox

Install the SDK using npm. Ensure you have Node.js and npm installed.

```text
npm install @codesandbox/sdk
```

--------------------------------

### Install Together Code Sandbox SDK

Source: https://docs.together.ai/docs/code-execution

Install the SDK using npm. Ensure you have Node.js and npm installed.

```bash
npm install @codesandbox/sdk

```

--------------------------------

### Initialize and Run Real-time Transcriber

Source: https://docs.together.ai/docs/inference/transcription/streaming

Instantiate the RealtimeTranscriber and start the transcription process. Ensure the main function is called to execute the setup.

```javascript
const transcriber = new RealtimeTranscriber();
    await transcriber.run();
  }

  main();
  ```
```

--------------------------------

### Use Examples and References in OpenCode Prompts

Source: https://docs.together.ai/docs/how-to-use-opencode

Enhance your prompts by referencing existing code patterns or files to guide OpenCode's implementation. This helps ensure consistency and adherence to established practices.

```plaintext
Add error handling to the API similar to how it's done in @src/utils/errorHandler.js
```

--------------------------------

### Install uv and Together Python SDK

Source: https://docs.together.ai/docs/pythonv2-migration-guide

Install the uv package manager and then use it to add the Together Python SDK to your project. Pip can also be used for installation.

```bash
# Install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create a new project and enter it
uv init myproject
cd myproject

# Install the Together Python SDK (allowing prereleases)
uv add together

# pip still works as well
pip install together

```

--------------------------------

### Install OpenCode

Source: https://docs.together.ai/docs/how-to-use-opencode

Install OpenCode system-wide using this command. Ensure you have curl installed.

```bash
curl -fsSL https://opencode.ai/install | bash
```

--------------------------------

### Install AutoGen Library

Source: https://docs.together.ai/docs/autogen

Install the AutoGen library using pip. This is the first step to using the framework.

```shell
pip install autogen
```

--------------------------------

### Install Together Library (uv)

Source: https://docs.together.ai/docs/dedicated_containers_image

Install the Together Python library using uv. This is an alternative package installer.

```shell
uv add together
```

--------------------------------

### Install Together AI Node SDK

Source: https://docs.together.ai/docs/ai-search-engine

Installs the Together AI Node.js SDK using npm.

```bash
npm i together-ai
```

--------------------------------

### Configure Template Tasks

Source: https://docs.together.ai/docs/code-execution

Defines setup and development tasks for a CodeSandbox template. Ensures dependencies are installed and the development server starts automatically.

```json
{
    "setupTasks": [
        "npm install"
    ],
    "tasks": {
        "dev-server": {
            "name": "Dev Server",
            "command": "npm run dev",
            "runAtStart": true
        }
    }
}

```

--------------------------------

### Create and Monitor Sandbox Setup Steps

Source: https://docs.together.ai/docs/together-code-sandbox

This snippet demonstrates how to create a new sandbox, retrieve its setup steps, and monitor their progress, including real-time output.

```javascript
const sandbox = await sdk.sandboxes.create()
 
const setupSteps = sandbox.setup.getSteps() 

for (const step of setupSteps) {
  console.log(`Step: ${step.name}`);
  console.log(`Command: ${step.command}`);
  console.log(`Status: ${step.status}`);
 
  const output = await step.open() 

  output.onOutput((output) => {
    console.log(output)
  })
 
  await step.waitUntilComplete()
}
```

--------------------------------

### Track Sandbox Setup Steps with JavaScript

Source: https://docs.together.ai/docs/code-execution

Use this JavaScript code to iterate through the setup steps of a sandbox, log their details, and stream their output. Ensure the 'sdk' object is initialized and available in the scope.

```javascript
const sandbox = await sdk.sandboxes.create()
 
const setupSteps = sandbox.setup.getSteps()
 
for (const step of setupSteps) {
  console.log(`Step: ${step.name}`);
  console.log(`Command: ${step.command}`);
  console.log(`Status: ${step.status}`);
 
  const output = await step.open()
 
  output.onOutput((output) => {
    console.log(output)
  })
 
  await step.waitUntilComplete()
}
```

--------------------------------

### Get Started with Kimi K2 (Python)

Source: https://docs.together.ai/docs/kimi-k2-quickstart

Use this snippet to interact with the Kimi K2 model for chat completions. It streams the response token by token. Ensure you have the 'together' library installed.

```Python
from together import Together

client = Together()
resp = client.chat.completions.create(
    model="moonshotai/Kimi-K2-Instruct-0905",
    messages=[{"role": "user", "content": "Code a hacker news clone"}],
    stream=True,
)
for tok in resp:
    print(tok.choices[0].delta.content, end="", flush=True)
```

--------------------------------

### Set Up Python Virtual Environment with uv

Source: https://docs.together.ai/docs/nanochat-on-instant-clusters

Inside the compute pod, install uv if necessary, create a virtual environment, and install repository dependencies with GPU support.

```bash
# Install uv (if not already installed)
command -v uv &> /dev/null || curl -LsSf https://astral.sh/uv/install.sh | sh

# Create a local virtual environment
[ -d ".venv" ] || uv venv

# Install the repo dependencies with GPU support
uv sync --extra gpu

# Activate the virtual environment
source .venv/bin/activate
```

--------------------------------

### Get Started with Together AI Inference API

Source: https://docs.together.ai/docs

This Python snippet demonstrates how to initialize the Together AI client and make a chat completion request. Ensure you have the 'together' library installed.

```python
from together import Together
client = Together()

completion = client.chat.completions.create(
  model="MiniMaxAI/MiniMax-M3",
  messages=[{"role": "user", "content": "What are the top 3 things to do in New York?"}],
)

print(completion.choices[0].message.content)

```

--------------------------------

### Example Response with Diarization

Source: https://docs.together.ai/docs/inference/transcription/features

This is an example of the JSON response structure when speaker diarization is enabled. It includes speaker IDs, start and end times, transcribed text, and word-level details with speaker attribution.

```JSON
AudioSpeakerSegment(
    id=1,
    speaker_id='SPEAKER_01',
    start=6.268,
    end=30.776,
    text=(
        "Hello. Oh, hey, Justin. How are you doing? ..."
    ),
    words=[
        AudioTranscriptionWord(
            word='Hello.',
            start=6.268,
            end=11.314,
            id=0,
            speaker_id='SPEAKER_01'
        ),
        AudioTranscriptionWord(
            word='Oh,',
            start=11.834,
            end=11.894,
            id=1,
            speaker_id='SPEAKER_01'
        ),
        AudioTranscriptionWord(
            word='hey,',
            start=11.914,
            end=11.995,
            id=2,
            speaker_id='SPEAKER_01'
        ),
        ...
    ]
)
```

--------------------------------

### Deploy Example Worker

Source: https://docs.together.ai/docs/containers-quickstart

Navigate to the example worker directory and execute the `tg beta jig deploy` command to build the Docker image, push it to Together's registry, and create a deployment.

```shell
cd examples/hello-world
tg beta jig deploy
```

--------------------------------

### Python: Multiple Function Calling Example

Source: https://docs.together.ai/docs/inference/function-calling/single-call

Provide a list of tools to the model and have it select the best one for the user's request. This example defines functions for getting weather and stock prices.

```python
import json
from together import Together

client = Together()

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                    },
                },
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "get_current_stock_price",
            "description": "Get the current stock price for a given stock symbol",
            "parameters": {
                "type": "object",
                "properties": {
                    "symbol": {
                        "type": "string",
                        "description": "The stock symbol, e.g. AAPL, GOOGL, TSLA",
                    },
                    "exchange": {
                        "type": "string",
                        "description": "The stock exchange (optional)",
                        "enum": ["NYSE", "NASDAQ", "LSE", "TSX"],
                    },
                },
                "required": ["symbol"],
            },
        },
    },
]

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct-Turbo",
    messages=[
        {
            "role": "user",
            "content": "What's the current price of Apple's stock?",
        },
    ],
    tools=tools,
)

print(
    json.dumps(
        response.choices[0].message.model_dump()["tool_calls"],
        indent=2,
    )
)

```

--------------------------------

### Set up Client and Helper Functions

Source: https://docs.together.ai/docs/conditional-workflows

Initializes the Together client and defines helper functions for running LLM calls, including a function for JSON output.

```Python
import json
from pydantic import ValidationError
from together import Together

client = Together()


def run_llm(user_prompt: str, model: str, system_prompt: str = None):
    messages = []
    if system_prompt:
        messages.append({"role": "system", "content": system_prompt})

    messages.append({"role": "user", "content": user_prompt})

    response = client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=0.7,
        max_tokens=4000,
    )

    return response.choices[0].message.content


def JSON_llm(user_prompt: str, schema, system_prompt: str = None):
    try:
        messages = []
        if system_prompt:
            messages.append({"role": "system", "content": system_prompt})

        messages.append({"role": "user", "content": user_prompt})

        extract = client.chat.completions.create(
            messages=messages,
            model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
            response_format={
                "type": "json_schema",
                "json_schema": {
                    "name": "route_selection",
                    "schema": schema.model_json_schema(),
                },
            },
        )
        return json.loads(extract.choices[0].message.content)

    except ValidationError as e:
        error_message = f"Failed to parse JSON: {e}"
        print(error_message)
```

--------------------------------

### Sprocket Worker SDK Example

Source: https://docs.together.ai/docs/together-deployments

Implement a custom inference worker using the Sprocket SDK. Define setup and predict methods for model loading and inference. This example shows a basic model inference task.

```python
import sprocket


class MyModel(sprocket.Sprocket):
    def setup(self):
        self.model = load_model()

    def predict(self, args: dict) -> dict:
        result = self.model(args["input"])
        return {"output": result}


if __name__ == "__main__":
    sprocket.run(MyModel(), "my-org/my-model")
```

--------------------------------

### Create and Run in Sandbox

Source: https://docs.together.ai/docs/code-execution

This snippet demonstrates how to initialize the SDK with an API key, create a new sandbox, connect to it, and run a command within the sandbox environment.

```APIDOC
## Create and Run in Sandbox

### Description
Initializes the SDK, creates a new sandbox, establishes a connection, and executes a command.

### Method
POST (implicitly through SDK method)

### Endpoint
N/A (SDK method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```typescript
import { CodeSandbox } from "@codesandbox/sdk";

const sdk = new CodeSandbox(process.env.CSB_API_KEY!);

const sandbox = await sdk.sandboxes.create();

const session = await sandbox.connect();

const output = await session.commands.run("echo 'Hello World'");

console.log(output)
```

### Response
#### Success Response
- **output** (string) - The output from the executed command.

#### Response Example
```json
{
  "example": "Hello World"
}
```
```

--------------------------------

### Initialize Together Client with Constructor Parameters

Source: https://docs.together.ai/docs/pythonv2-migration-guide

Example of initializing the Together client with various constructor parameters in the legacy SDK.

```python
client = Together(
    api_key="...",
    base_url="...",
    timeout=30,
    max_retries=3,
    supplied_headers={"X-Custom-Header": "value"},
)

```

--------------------------------

### Get structured JSON output

Source: https://docs.together.ai/docs/quickstart

This example demonstrates how to obtain parseable JSON output by passing a JSON schema via the `response_format` parameter. It includes Python and TypeScript examples using Pydantic and Zod respectively.

```APIDOC
## Get structured JSON output

Pass a [JSON schema](/docs/inference/chat/structured-outputs) via `response_format` to get parseable JSON back:

<CodeGroup>
  ```python Python theme={null}
  from pydantic import BaseModel
  from together import Together

  client = Together()


  class Activity(BaseModel):
      name: str
      neighborhood: str
      why: str


  class Itinerary(BaseModel):
      city: str
      activities: list[Activity]


  response = client.chat.completions.create(
      model="MiniMaxAI/MiniMax-M3",
      messages=[
          {"role": "user", "content": "Suggest 3 things to do in New York."},
      ],
      response_format={
          "type": "json_schema",
          "json_schema": {
              "name": "Itinerary",
              "schema": Itinerary.model_json_schema(),
          },
      },
  )

  itinerary = Itinerary.model_validate_json(response.choices[0].message.content)
  print(itinerary)
  ```

  ```typescript TypeScript theme={null}
  import Together from "together-ai";
  import { z } from "zod";

  const together = new Together();

  const Itinerary = z.object({
    city: z.string(),
    activities: z.array(
      z.object({
        name: z.string(),
        neighborhood: z.string(),
        why: z.string(),
      }),
    ),
  });

  async function main() {
    const response = await together.chat.completions.create({
      model: "MiniMaxAI/MiniMax-M3",
      messages: [
        { role: "user", content: "Suggest 3 things to do in New York." },
      ],
      response_format: {
        type: "json_schema",
        json_schema: { name: "Itinerary", schema: z.toJSONSchema(Itinerary) },
      },
    });

    const itinerary = Itinerary.parse(
      JSON.parse(response.choices[0].message.content!),
    );
    console.log(itinerary);
  }

  main();
  ```
</CodeGroup>
```

--------------------------------

### Setup Client and Helper Functions (Python)

Source: https://docs.together.ai/docs/parallel-workflows

Initializes the Together AI client and defines asynchronous functions for parallel LLM calls with rate limit handling and synchronous JSON schema-based LLM interactions.

```python
import asyncio
import json
import together
from pydantic import ValidationError
from together import AsyncTogether, Together

client = Together()
async_client = AsyncTogether()


# The function below will call the reference LLMs in parallel
async def run_llm_parallel(
    user_prompt: str,
    model: str,
    system_prompt: str = None,
):
    """Run a single LLM call with a reference model."""
    for sleep_time in [1, 2, 4]:
        try:
            messages = []
            if system_prompt:
                messages.append({"role": "system", "content": system_prompt})

            messages.append({"role": "user", "content": user_prompt})

            response = await async_client.chat.completions.create(
                model=model,
                messages=messages,
                temperature=0.7,
                max_tokens=2000,
            )
            break
        except together.error.RateLimitError as e:
            print(e)
            await asyncio.sleep(sleep_time)
    return response.choices[0].message.content


def JSON_llm(user_prompt: str, schema, system_prompt: str = None):
    try:
        messages = []
        if system_prompt:
            messages.append({"role": "system", "content": system_prompt})

        messages.append({"role": "user", "content": user_prompt})

        extract = client.chat.completions.create(
            messages=messages,
            model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
            response_format={
                "type": "json_schema",
                "json_schema": {
                    "name": "response",
                    "schema": schema.model_json_schema(),
                },
            },
        )
        return json.loads(extract.choices[0].message.content)

    except ValidationError as e:
        error_message = f"Failed to parse JSON: {e}"
        print(error_message)

```

--------------------------------

### Add a System Prompt

Source: https://docs.together.ai/docs/quickstart

This example shows how to prepend a system message to the conversation to guide the model's behavior, such as setting its tone or defining constraints.

```APIDOC
## Add a System Prompt

Prepend a `system` message to set the model's tone, role, or constraints:

### Python
```python
from together import Together

client = Together()

response = client.chat.completions.create(
    model="MiniMaxAI/MiniMax-M3",
    messages=[
        {
            "role": "system",
            "content": "You are a concise travel guide. Answer in two sentences or fewer.",
        },
        {
            "role": "user",
            "content": "What are the top 3 things to do in New York?",
        },
    ],
)

print(response.choices[0].message.content)
```

### TypeScript
```typescript
import Together from "together-ai";

const together = new Together();

async function main() {
  const response = await together.chat.completions.create({
    model: "MiniMaxAI/MiniMax-M3",
    messages: [
      {
        role: "system",
        content:
          "You are a concise travel guide. Answer in two sentences or fewer.",
      },
      { role: "user", content: "What are the top 3 things to do in New York?" },
    ],
  });

  console.log(response.choices[0].message.content);
}

main();
```
```

--------------------------------

### List Available Hardware for Adapter Deployment (Python SDK)

Source: https://docs.together.ai/docs/dedicated-endpoints/adapter

Use the Together AI Python SDK to list hardware options for deploying an adapter. This is an alternative to the CLI command.

```python
from together import Together

client = Together()

hw = client.endpoints.list_hardware(model="<ADAPTER_MODEL_NAME>")
for h in hw.data:
    print(h.id)
```

--------------------------------

### Single Reference Image Editing (Python)

Source: https://docs.together.ai/docs/quickstart-flux

Use a single reference image to guide image generation or editing. Ensure the 'together' library is installed.

```python
from together import Together

client = Together()

response = client.images.generate(
    model="black-forest-labs/FLUX.2-pro",
    prompt="Replace the color of the car to blue",
    width=1024,
    height=768,
    reference_images=[
        "https://images.pexels.com/photos/3729464/pexels-photo-3729464.jpeg"
    ],
)

print(response.data[0].url)
```

--------------------------------

### Single Reference Image Editing (TypeScript)

Source: https://docs.together.ai/docs/quickstart-flux

Use a single reference image to guide image generation or editing in TypeScript. Ensure the 'together-ai' library is installed.

```typescript
import Together from "together-ai";

const together = new Together();

async function main() {
  const response = await together.images.generate({
    model: "black-forest-labs/FLUX.2-pro",
    prompt: "Replace the color of the car to blue",
    width: 1024,
    height: 768,
    reference_images: [
      "https://images.pexels.com/photos/3729464/pexels-photo-3729464.jpeg",
    ],
  });

  console.log(response.data[0].url);
}

main();
```

--------------------------------

### Launch Qwen Code

Source: https://docs.together.ai/docs/how-to-use-qwen-code

Navigate to your project directory and launch Qwen Code to start using it with your configured Together AI models.

```bash
cd your-project/
qwen
```

--------------------------------

### Python Real-time Transcriber Setup

Source: https://docs.together.ai/docs/inference/transcription/streaming

Sets up and runs a real-time transcriber using Python. It connects to the API, starts audio capture, and handles transcription messages concurrently.

```python
import asyncio
import base64
import json
import sys

import numpy as np
import websockets


class RealtimeTranscriber:
    def __init__(self):
        self.ws = None
        self.stream = None
        self.audio_buffer = []
        self.is_ready = False

    async def connect(self):
        url = (
            f"wss://api.together.ai/v1/realtime"
            f"?intent=transcription"
            f"&model=openai/whisper-large-v3"
            f"&input_audio_format=pcm_s16le_16000"
            f"&authorization=Bearer {sys.argv[1]}"
        )
        self.ws = await websockets.connect(url)

    async def send_audio(self):
        while True:
            if len(self.audio_buffer) > 0 and self.ws and self.is_ready:
                try:
                    audio_int16 = (
                        np.clip(self.audio_buffer, -1.0, 1.0) * 32767
                    ).astype(np.int16)
                    audio_base64 = base64.b64encode(audio_int16.tobytes()).decode()
                    await self.ws.send(
                        json.dumps(
                            {
                                "type": "input_audio_buffer.append",
                                "audio": audio_base64,
                            }
                        )
                    )
                    self.audio_buffer = []
                except Exception:
                    pass
            await asyncio.sleep(0.01)

    async def receive_transcriptions(self):
        async for message in self.ws:
            data = json.loads(message)
            if data["type"] == "session.created":
                self.is_ready = True
            elif data["type"] == "conversation.item.input_audio_transcription.delta":
                print(f"\x1b[90m{data['delta']}\x1b[0m", end="", flush=True)
            elif data["type"] == "conversation.item.input_audio_transcription.completed":
                print(f"\r\x1b[K\x1b[92m{data['transcript']}\x1b[0m")
            elif data["type"] == "error":
                print(f"\nError: {data['message']}", file=sys.stderr)

    async def close(self):
        # Flush remaining audio
        if len(self.audio_buffer) > 0 and self.ws and self.is_ready:
            try:
                audio_int16 = (
                    np.clip(self.audio_buffer, -1.0, 1.0) * 32767
                ).astype(np.int16)
                audio_base64 = base64.b64encode(audio_int16.tobytes()).decode()
                await self.ws.send(
                    json.dumps(
                        {
                            "type": "input_audio_buffer.append",
                            "audio": audio_base64,
                        }
                    )
                )
            except Exception:
                pass

        if self.ws:
            await self.ws.close()

    async def run(self):
        """Main execution loop."""
        try:
            print("🎤 Together AI Realtime Transcription")
            print("=" * 40)
            print("Connecting...")

            await self.connect()

            print("✓ Connected")
            print("✓ Recording started - speak now\n")

            # Run audio capture and transcription concurrently
            await asyncio.gather(
                self.send_audio(), self.receive_transcriptions()
            )

        except KeyboardInterrupt:
            print("\n\nStopped")
        except Exception as e:
            print(f"Error: {e}", file=sys.stderr)
        finally:
            await self.close()


async def main():
    transcriber = RealtimeTranscriber()
    await transcriber.run()


if __name__ == "__main__":
    asyncio.run(main())

```

--------------------------------

### Deploy and Monitor Dedicated Containers

Source: https://docs.together.ai/docs/dedicated_containers_video

Use the 'tg beta jig deploy' command to build, push, and create a deployment. The '--warmup' flag can reduce cold start latency. Monitor startup with 'tg beta jig logs --follow'.

```shell
# Deploy (builds, pushes, and creates deployment)
tg beta jig deploy

# Or deploy with cache warmup to reduce cold start latency
tg beta jig deploy --warmup

# Monitor startup
tg beta jig logs --follow
```

--------------------------------

### Get Started with Kimi K2 (TypeScript)

Source: https://docs.together.ai/docs/kimi-k2-quickstart

This snippet demonstrates how to use the Kimi K2 model for chat completions in TypeScript, streaming the output. Requires the 'together-ai' package.

```TypeScript
import Together from 'together-ai';
const together = new Together();

const stream = await together.chat.completions.create({
  model: 'moonshotai/Kimi-K2-Instruct-0905',
  messages: [{ role: 'user', content: 'Code a hackernews clone' }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
```

--------------------------------

### Deploy Model with Together CLI

Source: https://docs.together.ai/docs/dedicated_containers_image

Use the `tg beta jig deploy` command to build, push, and create a deployment. Include `--warmup` for cache warmup to reduce cold start latency. Monitor startup with `tg beta jig logs --follow`.

```shell
# Deploy (builds, pushes, and creates deployment)
tg beta jig deploy

# Or deploy with cache warmup to reduce cold start latency
tg beta jig deploy --warmup

# Monitor startup (model download takes a few minutes on first deploy)
tg beta jig logs --follow
```

--------------------------------

### List Available Hardware for a Model

Source: https://docs.together.ai/docs/dedicated-endpoints/manage

Use this command or SDK method to see the hardware options available for a specific model before creating an endpoint. This helps in choosing the most suitable hardware for your needs.

```shell
together endpoints hardware --model Qwen/Qwen3.5-9B-FP8
```

```python
from together import Together

client = Together()

response = client.endpoints.list_hardware(model="Qwen/Qwen3.5-9B-FP8")
for hw in response.data:
    print(hw.id)
```

```typescript
import Together from "together-ai";

const client = new Together();

const response = await client.endpoints.listHardware({
  model: "Qwen/Qwen3.5-9B-FP8",
});
for (const hw of response.data) {
  console.log(hw.id);
}
```

--------------------------------

### Verbose JSON with Segment Timestamps

Source: https://docs.together.ai/docs/inference/transcription/features

Use `response_format='verbose_json'` and `timestamp_granularities='segment'` to get timing information for larger text segments. Iterate through `response.segments` to access start and end times.

```python
from pathlib import Path

response = client.audio.transcriptions.create(
    file=Path("audio.mp3"),
    model="openai/whisper-large-v3",
    response_format="verbose_json",
    timestamp_granularities="segment",
)

## Access segments with timestamps
for segment in response.segments:
    print(
        f"[{segment['start']:.2f}s - {segment['end']:.2f}s]: {segment['text']}"
    )
```

--------------------------------

### Send Image URL with Text Prompt (TypeScript)

Source: https://docs.together.ai/docs/kimi-k2.6-quickstart

This TypeScript example shows how to send an image URL with a text prompt to Kimi-K2.6. Make sure to install the 'together-ai' package.

```typescript
import Together from "together-ai";

const together = new Together();

const response = await together.chat.completions.create({
  model: "moonshotai/Kimi-K2.6",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "What can you see in this image?" },
      { type: "image_url", image_url: { url: "https://huggingface.co/datasets/patrickvonplaten/random_img/resolve/main/yosemite.png" }}
    ]
  }],
  temperature: 0.6,
  top_p: 0.95,
});

console.log(response.choices[0].message.content);
```

--------------------------------

### List Hardware Options for a Model

Source: https://docs.together.ai/docs/dedicated-endpoints/quickstart

Use the Together CLI to discover compatible hardware for a specific model.

```shell
tg endpoints hardware --model Qwen/Qwen3.5-9B-FP8
```

--------------------------------

### Generate Image with FLUX Kontext (TypeScript)

Source: https://docs.together.ai/docs/quickstart-flux-kontext

This TypeScript example demonstrates how to generate an image using the FLUX Kontext model with a prompt and an image URL. Make sure to install the 'together-ai' package.

```typescript
import Together from "together-ai";

const together = new Together();

async function main() {
  const response = await together.images.generate({
    model: "black-forest-labs/FLUX.1-kontext-pro",
    width: 1536,
    height: 1024,
    steps: 28,
    prompt: "make his shirt yellow",
    image_url: "https://github.com/nutlope.png",
  });

  console.log(response.data[0].url);
}

main();
```

--------------------------------

### Parallel Function Calling with Python

Source: https://docs.together.ai/docs/inference/function-calling/parallel

Use this Python snippet to make parallel function calls. Ensure you have the 'together' library installed. The example demonstrates calling a weather function with different locations.

```Python
import json
from together import Together

client = Together()

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct-Turbo",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant that can access external functions. The responses from these function calls will be appended to this dialogue. Please provide responses based on the information from these function calls.",
        },
        {
            "role": "user",
            "content": "What is the current temperature of New York, San Francisco and Chicago?",
        },
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                        },
                    },
                },
            }
        }
    ],
)

print(
    json.dumps(
        response.choices[0].message.model_dump()["tool_calls"],
        indent=2,
    )
)
```

--------------------------------

### Deploy Adapter via CLI/SDK

Source: https://docs.together.ai/docs/dedicated-endpoints/adapter

List available hardware for the adapter and then create the endpoint using the returned hardware ID.

```APIDOC
## Deploy the adapter

Uploaded adapters deploy as dedicated endpoints, the same way as any other model. Use the `model_name` from the upload response as the `model` argument when creating the endpoint.

### List Available Hardware

#### Method
GET

#### Endpoint
`/v1/endpoints/hardware`

#### Query Parameters
- **model** (string) - Required - The model name of the adapter.

### Create Endpoint

#### Method
POST

#### Endpoint
`/v1/endpoints`

#### Request Body
- **display_name** (string) - Required - The desired display name for the endpoint.
- **model** (string) - Required - The model name of the adapter.
- **hardware** (string) - Required - The ID of the hardware to use for the endpoint.
- **autoscaling** (object) - Optional - Autoscaling configuration for the endpoint.
  - **min_replicas** (integer) - Minimum number of replicas.
  - **max_replicas** (integer) - Maximum number of replicas.

### Request Example (CLI)
```shell
together endpoints hardware --model <ADAPTER_MODEL_NAME>

together endpoints create \
  --display-name <ENDPOINT_NAME> \
  --model <ADAPTER_MODEL_NAME> \
  --hardware <HARDWARE_ID>
```

### Request Example (Python SDK)
```python
from together import Together

client = Together()

hw = client.endpoints.list_hardware(model="<ADAPTER_MODEL_NAME>")
for h in hw.data:
    print(h.id)

endpoint = client.endpoints.create(
    model="<ADAPTER_MODEL_NAME>",
    hardware="<HARDWARE_ID>",
    display_name="<ENDPOINT_NAME>",
    autoscaling={"min_replicas": 1, "max_replicas": 1},
)
print(endpoint.id, endpoint.name)
```

### Request Example (TypeScript SDK)
```typescript
import Together from "together-ai";

const client = new Together();

const hw = await client.endpoints.listHardware({
  model: "<ADAPTER_MODEL_NAME>",
});
for (const h of hw.data) {
  console.log(h.id);
}

const endpoint = await client.endpoints.create({
  model: "<ADAPTER_MODEL_NAME>",
  hardware: "<HARDWARE_ID>",
  display_name: "<ENDPOINT_NAME>",
  autoscaling: { min_replicas: 1, max_replicas: 1 },
});
console.log(endpoint.id, endpoint.name);
```
```

--------------------------------

### Set up DSPy Module with Tools

Source: https://docs.together.ai/docs/dspy

Set up a DSPy module, such as ReAct, with a task-specific signature and custom tools. This example includes a Python interpreter and a Wikipedia search tool.

```python
# Configure dspy to use the LLM
dspy.configure(lm=lm)


# Gives the agent access to a python interpreter
def evaluate_math(expression: str):
    return dspy.PythonInterpreter({}).execute(expression)


# Gives the agent access to a wikipedia search tool
def search_wikipedia(query: str):

    results = dspy.ColBERTv2(url="http://20.102.90.50:2017/wiki17_abstracts")(query, k=3)
    return [x["text"] for x in results]


# setup ReAct module with question and math answer signature
react = dspy.ReAct(
    "question -> answer: float",
    tools=[evaluate_math, search_wikipedia],
)

pred = react(
    question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?"
)

print(pred.answer)
```

--------------------------------

### Initialize Together Python Client

Source: https://docs.together.ai/docs/pythonv2-migration-guide

Demonstrates how to initialize the Together Python client using an API key directly or via an environment variable. Also shows how to initialize the asynchronous client.

```python
from together import Together

# Using API key directly
client = Together(api_key="your-api-key")

# Using environment variable (recommended)
client = Together()  # Uses TOGETHER_API_KEY env var

# Async client
from together import AsyncTogether

async_client = AsyncTogether()

```

--------------------------------

### Define a SkyPilot task for GPU workload

Source: https://docs.together.ai/docs/gpu-clusters-api

Create a SkyPilot task file (YAML format) to define resources, setup commands, and the run command for an AI workload. This example specifies 8 H100 GPUs.

```yaml
resources:
  accelerators: H100:8
  cloud: kubernetes

setup: |
  pip install torch transformers

run: |
  python train.py
```

--------------------------------

### List Available Hardware for Adapter Deployment

Source: https://docs.together.ai/docs/dedicated-endpoints/adapter

Before creating a dedicated endpoint for an adapter, list the available hardware options compatible with the adapter's model name using the CLI.

```shell
together endpoints hardware --model <ADAPTER_MODEL_NAME>
```

--------------------------------

### TypeScript Text-to-Speech Request

Source: https://docs.together.ai/docs/inference/text-to-speech/overview

Generates speech audio from text using the Together AI TypeScript client. This example streams the audio response and saves it to an MP3 file. Ensure you have the together-audio skill installed for agent usage.

```typescript
import Together from 'together-ai';

const together = new Together();

async function generateAudio() {
  const res = await together.audio.speech.create({
    input: 'Hello, how are you today?',
    voice: 'tara',
    response_format: 'mp3',
    sample_rate: 24000,
    stream: false,
    model: 'canopylabs/orpheus-3b-0.1-ft',
  });

  if (res.body) {
    console.log(res.body);
    const nodeStream = Readable.from(res.body as ReadableStream);
    const fileStream = createWriteStream('./speech.mp3');

    nodeStream.pipe(fileStream);
  }
}

generateAudio();
```

--------------------------------

### Submit Video Generation Jobs with Requests Library

Source: https://docs.together.ai/docs/dedicated_containers_video

Submit video generation jobs to the managed queue using the `requests` library. This example shows how to send a POST request to submit a job and then poll for its status using GET requests.

```python
import requests
import time

api_key = "your_key_here"
deployment = "sprocket-wan2.1"

# Submit job to queue
response = requests.post(
    "https://api.together.ai/v1/queue/submit",
    headers={"Authorization": f"Bearer {api_key}"},
    json={
        "model": deployment,
        "payload": {
            "prompt": "A cat playing with a ball of yarn",
            "num_inference_steps": 30,
        },
    },
)
job = response.json()
print(f"Job submitted: {job['request_id']}")

# Poll for completion
while True:
    status_response = requests.get(
        f"https://api.together.ai/v1/queue/status?request_id={job['request_id']}&model={deployment}",
        headers={"Authorization": f"Bearer {api_key}"},
    )
    status = status_response.json()

    print(f"Status: {status['status']}")

    if status["status"] == "done":
        print(f"Video URL: {status['outputs']['url']}")
        break
    elif status["status"] == "failed":
        print(f"Job failed: {status.get('error')}")
        break

    time.sleep(5)
```

--------------------------------

### Hello World Sprocket Worker

Source: https://docs.together.ai/docs/containers-quickstart

A minimal Python example of a Sprocket worker that returns a greeting. It defines a 'HelloWorld' class inheriting from 'sprocket.Sprocket' and implements 'setup' and 'predict' methods. The 'predict' method takes a 'name' argument and returns a formatted greeting.

```python
import os
import sprocket


class HelloWorld(sprocket.Sprocket):
    def setup(self) -> None:
        self.greeting = "Hello"

    def predict(self, args: dict) -> dict:
        name = args.get("name", "world")
        return {"message": f"{self.greeting}, {name}!"}


if __name__ == "__main__":
    queue_name = os.environ.get("TOGETHER_DEPLOYMENT_NAME", "hello-world")
    sprocket.run(HelloWorld(), queue_name)
```

--------------------------------

### Model File Structure Example

Source: https://docs.together.ai/docs/custom-models

A typical model directory compatible with Hugging Face's `from_pretrained` should contain these files.

```text
config.json
generation_config.json
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors.index.json
special_tokens_map.json
tokenizer.json
tokenizer_config.json
```

--------------------------------

### End-to-End RAG Pipeline Example

Source: https://docs.together.ai/docs/inference/embeddings/rag

This Python script demonstrates a minimal RAG pipeline. It embeds a small corpus, stores vectors in memory, retrieves top matches using cosine similarity, and uses these to ground a chat completion. Ensure the Together SDK is installed.

```Python
import math
from together import Together

client = Together()

EMBEDDING_MODEL = "intfloat/multilingual-e5-large-instruct"
CHAT_MODEL = "MiniMaxAI/MiniMax-M3"

# A tiny corpus. In a real app, load from your data source and chunk first.
corpus = [
    "Photosynthesis converts sunlight, water, and carbon dioxide into glucose and oxygen, primarily in the chloroplasts of plant leaves.",
    "Mitochondria generate ATP through cellular respiration and are often called the powerhouse of the cell.",
    "Plate tectonics explains the slow movement of Earth's lithospheric plates and accounts for earthquakes and volcanoes.",
    "The water cycle moves water between oceans, atmosphere, and land through evaporation, condensation, and precipitation.",
    "Natural selection favors organisms whose inherited traits improve their chance of surviving and reproducing.",
    "Neural networks are layered computations of weighted sums and nonlinear activations, loosely inspired by biological neurons.",
]


def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    na = math.sqrt(sum(x * x for x in a))
    nb = math.sqrt(sum(x * x for x in b))
    return dot / (na * nb) if na and nb else 0.0


# 1. Embed the corpus once.
doc_embeddings = client.embeddings.create(
    model=EMBEDDING_MODEL, input=corpus
).data
index = list(zip(corpus, [d.embedding for d in doc_embeddings]))


def rag(query: str, top_k: int = 3) -> str:
    # 2. Embed the query.
    q_emb = (
        client.embeddings.create(model=EMBEDDING_MODEL, input=query)
        .data[0]
        .embedding
    )

    # 3. Retrieve top_k by cosine similarity.
    ranked = sorted(index, key=lambda d: cosine(q_emb, d[1]), reverse=True)
    context = "\n\n".join(text for text, _ in ranked[:top_k])

    # 4. Generate an answer grounded in the retrieved context.
    response = client.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "Answer the question using only the context below. "
                    "If the context is insufficient, say so.\n\n"
                    f"Context:\n{context}"
                ),
            },
            {"role": "user", "content": query},
        ],
    )
    return response.choices[0].message.content


print(rag("How do plants make their food?"))
```

--------------------------------

### Launch OpenCode in Project Directory

Source: https://docs.together.ai/docs/how-to-use-opencode

Navigate to your project's root directory and run the opencode command to launch the agent with LSP support.

```bash
cd your-project
opencode
```