Martian

https://docs.withmartian.com
Admin
Martian provides unified access to 200+ AI models through a single API with intelligent routing,...
Tokens:34,836
Snippets:349
Trust Score:8.8
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install Dependencies with uv

Source: https://docs.withmartian.com/k-steering/how-it-works

Install project dependencies using the 'uv sync' command after cloning the repository. This command sets up the environment and installs all required packages.

```bash
uv sync

```

--------------------------------

### Install Aider

Source: https://docs.withmartian.com/integrations/aider

Install Aider using pip. Ensure you have Python and pip installed.

```bash
pip install aider-chat

```

--------------------------------

### Clone the k-steering Repository

Source: https://docs.withmartian.com/k-steering/how-it-works

Clone the official k-steering repository to your local machine to get started.

```bash
git clone https://github.com/withmartian/k-steering.git

```

--------------------------------

### Start OpenCode

Source: https://docs.withmartian.com/integrations/opencode

Navigate to your project directory and run the opencode command to start routing requests through the Martian Gateway. You can switch models using the /model command within OpenCode.

```bash
cd your-project
opencode
```

--------------------------------

### Install OpenCode with cURL

Source: https://docs.withmartian.com/integrations/opencode

Installs OpenCode using a cURL command. Ensure you have OpenCode installed before proceeding with configuration.

```bash
curl -fsSL https://opencode.ai/install | bash
```

--------------------------------

### Install LiteLLM Python SDK

Source: https://docs.withmartian.com/integrations/litellm

Install the LiteLLM Python SDK using pip. This is the first step to using LiteLLM's features for model routing and cost tracking.

```bash
pip install litellm

```

--------------------------------

### Install OpenAI SDK (Python)

Source: https://docs.withmartian.com/integrations/openai-sdk

Install the OpenAI SDK using pip for Python integration.

```python
pip install openai

```

--------------------------------

### Install Martian Provider and Vercel AI SDK

Source: https://docs.withmartian.com/integrations/vercel-ai-sdk

Install the necessary packages for Martian AI SDK provider and Vercel AI SDK using npm.

```bash
npm install @withmartian/ai-sdk-provider ai

```

--------------------------------

### Install Anthropic SDK (Python)

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Install the Anthropic SDK using pip. This is the first step to integrating with the Anthropic API.

```python
pip install anthropic
```

--------------------------------

### Install Codex CLI

Source: https://docs.withmartian.com/integrations/codex

Install the Codex CLI globally using npm.

```bash
npm install -g @openai/codex
```

--------------------------------

### Tool Use Example

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Implement tool use by defining available tools and their schemas. The model can then choose to use these tools to respond to user queries, as shown in this example for a weather lookup.

```python
response = client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=1024,
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather in a location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                }
            }
        }
    ],
    messages=[
        {"role": "user", "content": "What's the weather in San Francisco?"}
    ]
)

for item in response.content:
    if item.type == "tool_use":
        print(f"Tool: {item.name}")
        print(f"Args: {item.input}")
```

--------------------------------

### Using a Specific Model

Source: https://docs.withmartian.com/api-reference/models

This example demonstrates how to use the client to create a chat completion with a specific model. Ensure the client is initialized before use.

```python
response = client.chat.completions.create(
    model="openai/gpt-4.1-nano",
    messages=[{"role": "user", "content": "Hello!"}]
)

```

--------------------------------

### Install LiteLLM Proxy Server

Source: https://docs.withmartian.com/integrations/litellm

Pull the LiteLLM Docker image for the proxy server.

```bash
docker pull ghcr.io/berriai/litellm:main-latest

```

--------------------------------

### Cache Setup and Usage

Source: https://docs.withmartian.com/api-reference/advanced

Demonstrates how to set up a cacheable system prompt and then utilize the cached content in subsequent requests. The first request caches the provided text, and the second request reuses this cache.

```APIDOC
## POST /v1/messages

### Description
This endpoint allows for sending messages to a specified model, with advanced options for system prompts and cache control. It supports caching of system content for subsequent, faster responses.

### Method
POST

### Endpoint
/v1/messages

### Parameters
#### Request Body
- **model** (string) - Required - The identifier of the model to use for generation.
- **max_tokens** (integer) - Optional - The maximum number of tokens to generate in the response.
- **system** (array) - Required - A list of system prompt content. Each item can include:
  - **type** (string) - Required - The type of content, e.g., "text".
  - **text** (string) - Required - The actual text content. For cacheable content, this should be the text to be cached.
  - **cache_control** (object) - Optional - Controls how the content is cached. If present, it must contain:
    - **type** (string) - Required - The type of cache, e.g., "ephemeral".
- **messages** (array) - Required - A list of user messages. Each item should contain:
  - **role** (string) - Required - The role of the message sender (e.g., "user").
  - **content** (string) - Required - The content of the message.

### Request Example
**First Request (Cache Setup):**
```json
{
  "model": "anthropic/claude-opus-4-20250514",
  "max_tokens": 100,
  "system": [
    {
      "type": "text",
      "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on facts, specifications, and other information."
    },
    {
      "type": "text",
      "text": "<the entire contents of Space! : The Universe As You have Never Seen It Before>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "What is the largest planet?"
    }
  ]
}
```

**Second Request (Cache Reuse):**
```json
{
  "model": "anthropic/claude-opus-4-20250514",
  "max_tokens": 100,
  "system": [
    {
      "type": "text",
      "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on facts, specifications, and other information."
    },
    {
      "type": "text",
      "text": "<the entire contents of Space! : The Universe As You have Never Seen It Before>",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "Which planet is closest to the sun?"
    }
  ]
}
```

### Response
#### Success Response
**First Request Response (Cache Creation):**
```json
{
  "id": "msg_XXXxXxXXxXxxXX",
  "type": "message",
  "role": "assistant",
  "model": "anthropic/claude-opus-4-20250514",
  "content": [
    {
      "type": "text",
      "text": "According to the information provided, Jupiter is the largest planet in our solar system..."
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 27,
    "cache_creation_input_tokens": 1279,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 1279,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 45,
    "service_tier": "standard"
  }
}
```

**Second Request Response (Cache Reuse):**
```json
{
  "id": "msg_YYYyYyYYyYyyYY",
  "type": "message",
  "role": "assistant",
  "model": "anthropic/claude-opus-4-20250514",
  "content": [
    {
      "type": "text",
      "text": "Mercury is the planet closest to the Sun, orbiting at an average distance of..."
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 21,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 1279,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 42,
    "service_tier": "standard"
  }
}
```

### Response Example
See **Success Response** section above for detailed examples of responses indicating cache creation and reuse.
```

--------------------------------

### OpenAI Format cURL Example for Tool Use

Source: https://docs.withmartian.com/api-reference/advanced

Use this cURL command to make a request to the OpenAI compatible API for tool use, specifying a function to get weather information.

```bash
curl https://api.withmartian.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "model": "openai/gpt-4.1-nano",
    "max_tokens": 1024,
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string"
              }
            }
          }
        }
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "What's the weather in San Francisco?"
      }
    ] 
  }'
```

--------------------------------

### Start Codex with a Profile

Source: https://docs.withmartian.com/integrations/codex

Navigate to your project directory and start Codex using a specific profile to route requests through the Martian Gateway. You can switch profiles using the --profile flag or the /model command.

```bash
cd your-project
codex --profile gpt-5.2
```

--------------------------------

### Start LiteLLM Proxy Server with Docker

Source: https://docs.withmartian.com/integrations/litellm

Run the LiteLLM proxy server using Docker, mounting the configuration file and exposing the necessary port. Ensure the MARTIAN_API_KEY environment variable is set.

```bash
docker run \
    -v $(pwd)/config.yaml:/app/config.yaml \
    -e MARTIAN_API_KEY=your-martian-api-key \
    -p 4000:4000 \
    ghcr.io/berriai/litellm:main-latest \
    --config /app/config.yaml

# Proxy running on http://0.0.0.0:4000

```

--------------------------------

### Start Claude Code in Project Directory

Source: https://docs.withmartian.com/integrations/claude-code

Navigate to your project directory and start Claude Code to begin routing requests through the Martian Gateway.

```shell
cd your-project
claude

```

--------------------------------

### Configure OpenAI SDK (Python)

Source: https://docs.withmartian.com/integrations/openai-sdk

Configure the OpenAI SDK with Martian Gateway's base URL and API key for basic Python setup.

```python
import openai

client = openai.OpenAI(
    base_url="https://api.withmartian.com/v1",
    api_key="MARTIAN_API_KEY"
)

```

--------------------------------

### Install Martian ARES

Source: https://docs.withmartian.com/ares

Install the Martian ARES library using uv. This command adds the ARES package to your project's dependencies.

```bash
uv add martian-ares
```

--------------------------------

### Python: System Instruction Caching Example

Source: https://docs.withmartian.com/api-reference/advanced

Demonstrates how to use system instruction caching with the Anthropic client. The first request creates the cache, and subsequent identical requests reuse it, as shown by the usage metrics.

```python
import anthropic

anth_client = anthropic.Anthropic(
    base_url="https://api.withmartian.com/v1",
    api_key=MARTIAN_API_KEY
)

# Define a common system instruction that will be used repeatedly
system_instruction = [
    {
        "type": "text",
        "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on facts, specifications, and other information.\n",
    },
    {
        "type": "text",
        "text": "<the entire contents of 'Space! : The Universe As You've Never Seen It Before'>",
        "cache_control": {"type": "ephemeral"}
    }
]

# First interaction - the system instruction will be cached
message1 = anth_client.messages.create(
    model="anthropic/claude-opus-4-20250514",
    max_tokens=100,
    system=system_instruction,
    messages=[{"role": "user", "content": "What is the largest planet?"}],
)

# Check the usage to confirm cache creation
print("First request usage:")
print(f"  cache_creation_input_tokens: {message1.usage.cache_creation_input_tokens}")
print(f"  cache_read_input_tokens: {message1.usage.cache_read_input_tokens}")
print(f"  input_tokens: {message1.usage.input_tokens}")
print(f"  output_tokens: {message1.usage.output_tokens}")

# Output:
# First request usage:
#   cache_creation_input_tokens: 1280
#   cache_read_input_tokens: 0
#   input_tokens: 15
#   output_tokens: 53

# Subsequent interaction - the cached system instruction will be reused
message2 = anth_client.messages.create(
    model="anthropic/claude-opus-4-20250514",
    max_tokens=100,
    system=system_instruction,  # Must be identical
    messages=[{"role": "user", "content": "Which planet is closest to the sun?"}],
)

# Check the usage to confirm cache hit
print("\nSecond request usage:")
print(f"  cache_creation_input_tokens: {message2.usage.cache_creation_input_tokens}")
print(f"  cache_read_input_tokens: {message2.usage.cache_read_input_tokens}")
print(f"  input_tokens: {message2.usage.input_tokens}")
print(f"  output_tokens: {message2.usage.output_tokens}")

# Output:
# Second request usage:
#   cache_creation_input_tokens: 0
#   cache_read_input_tokens: 1280
#   input_tokens: 16
#   output_tokens: 42

```

--------------------------------

### Example Response Structure for /v1/models

Source: https://docs.withmartian.com/api-reference/endpoints

This is a sample JSON response from the /v1/models endpoint. It includes details for each model such as ID, pricing per token, and timestamps.

```json
{
  "data": [
    {
      "id": "openai/gpt-4.1-nano",
      "pricing": {
        "prompt": "0.0000001",
        "completion": "0.0000004",
        "image": "0",
        "request": "0",
        "web_search": "0.01",
        "internal_reasoning": "0"
      },
      "added_at": "2024-05-13T00:00:00+00:00",
      "updated_at": "2025-01-15T10:30:00+00:00",
      "reliability_tier": 1,
      "max_completion_tokens": 32768
    },
    {
      "id": "anthropic/claude-sonnet-4-20250514",
      "pricing": {
        "prompt": "0.000003",
        "completion": "0.000015",
        "image": "0.0048",
        "request": "0",
        "web_search": "0",
        "internal_reasoning": "0"
      },
      "added_at": "2025-05-14T00:00:00+00:00",
      "updated_at": "2025-05-14T00:00:00+00:00",
      "reliability_tier": 1,
      "max_completion_tokens": 64000
    },
    {
      "id": "google/gemini-2.5-flash",
      "pricing": {
        "prompt": "0.0000003",
        "completion": "0.0000025",
        "image": "0",
        "request": "0",
        "web_search": "0",
        "internal_reasoning": "0"
      },
      "added_at": "2024-12-01T00:00:00+00:00",
      "updated_at": "2025-02-20T14:22:00+00:00",
      "reliability_tier": 2,
      "max_completion_tokens": 65536
    }
    // ... more models
  ]
}
```

--------------------------------

### Streaming Responses (Python)

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Implement streaming responses for a more interactive experience. This example iterates through the stream and prints text deltas as they arrive.

```python
stream = client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Write a story about Mars"}],
    stream=True
)

for chunk in stream:
    if chunk.type == "content_block_delta" and chunk.delta.type == "text_delta":
        print(chunk.delta.text, end="")
```

--------------------------------

### Install Claude Code via Curl

Source: https://docs.withmartian.com/integrations/claude-code

Use this command to install Claude Code using curl.

```shell
curl -fsSL https://claude.ai/install.sh | bash

```

--------------------------------

### OpenAI Format Python Example for Tool Use

Source: https://docs.withmartian.com/api-reference/advanced

This Python script demonstrates how to use the OpenAI client library to enable tool use, defining a 'get_weather' function for the model.

```python
import openai

oai_client = openai.OpenAI(
    base_url="https://api.withmartian.com/v1",
    api_key=MARTIAN_API_KEY
)

response = oai_client.chat.completions.create(
    model="openai/gpt-4.1-nano",
    max_tokens=1024,
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {"type": "string"}
                    }
                }
            }
        }
    ],
    messages=[
        {
            "role": "user",
            "content": "What's the weather in San Francisco?"
        }
    ]
)

print(response.choices[0].message.tool_calls)
```

--------------------------------

### Chat Completions (Python httpx Async)

Source: https://docs.withmartian.com/integrations/http-client

Example of making an asynchronous Chat Completions request using Python's `httpx` library.

```APIDOC
## POST /v1/chat/completions (Python httpx Async)

### Description
Demonstrates how to make an asynchronous call to the Chat Completions endpoint using the Python `httpx` library.

### Method
POST

### Endpoint
https://api.withmartian.com/v1/chat/completions

### Parameters
#### Headers
- **Content-Type**: application/json
- **Authorization**: Bearer MARTIAN_API_KEY

#### Request Body
- **model** (string) - Required - The model to use for generating completions.
- **messages** (array) - Required - An array of message objects representing the conversation history.
  - **role** (string) - Required - The role of the message sender (e.g., "user", "assistant").
  - **content** (string) - Required - The content of the message.

### Request Example
```python
import httpx

async def make_request():
    async with httpx.AsyncClient() as client:
        response = await client.post(
            "https://api.withmartian.com/v1/chat/completions",
            headers={
                "Content-Type": "application/json",
                "Authorization": "Bearer MARTIAN_API_KEY"
            },
            json={
                "model": "openai/gpt-4.1-nano",
                "messages": [
                    {"role": "user", "content": "Hello!"}
                ]
            },
            timeout=60.0
        )
        result = response.json()
        return result['choices'][0]['message']['content']
```

### Response
#### Success Response (200)
- **choices** (array) - An array of completion choices.
  - **message** (object) - The message content from the assistant.
    - **role** (string) - The role of the sender (e.g., "assistant").
    - **content** (string) - The generated text content.

#### Response Example
```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Hello there! How can I help you today?"
      }
    }
  ]
}
```
```

--------------------------------

### Chat Completions (Python requests)

Source: https://docs.withmartian.com/integrations/http-client

Example of making a Chat Completions request using Python's `requests` library.

```APIDOC
## POST /v1/chat/completions (Python requests)

### Description
Demonstrates how to call the Chat Completions endpoint using the Python `requests` library.

### Method
POST

### Endpoint
https://api.withmartian.com/v1/chat/completions

### Parameters
#### Headers
- **Content-Type**: application/json
- **Authorization**: Bearer MARTIAN_API_KEY

#### Request Body
- **model** (string) - Required - The model to use for generating completions.
- **messages** (array) - Required - An array of message objects representing the conversation history.
  - **role** (string) - Required - The role of the message sender (e.g., "user", "assistant").
  - **content** (string) - Required - The content of the message.

### Request Example
```python
import requests

url = "https://api.withmartian.com/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer MARTIAN_API_KEY"
}
data = {
    "model": "openai/gpt-4.1-nano",
    "messages": [
        {"role": "user", "content": "What is Olympus Mons?"}
    ]
}

response = requests.post(url, headers=headers, json=data)
result = response.json()
print(result['choices'][0]['message']['content'])
```

### Response
#### Success Response (200)
- **choices** (array) - An array of completion choices.
  - **message** (object) - The message content from the assistant.
    - **role** (string) - The role of the sender (e.g., "assistant").
    - **content** (string) - The generated text content.

#### Response Example
```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Olympus Mons is a large shield volcano on the planet Mars."
      }
    }
  ]
}
```
```

--------------------------------

### Configure Request Timeout

Source: https://docs.withmartian.com/integrations/http-client

Set a timeout for HTTP requests to prevent indefinite waiting. This example sets a 60-second timeout.

```python
import requests

response = requests.post(
    url,
    headers=headers,
    json=data,
    timeout=60  # 60 seconds
)


```

--------------------------------

### Function Calling with Python

Source: https://docs.withmartian.com/resources/examples

Utilize function calling to enable the model to call external tools. This example demonstrates calling a `get_weather` function.

```python
import json
import openai

client = openai.OpenAI(
    base_url="https://api.withmartian.com/v1",
    api_key=MARTIAN_API_KEY
)

# Define available functions
def get_weather(location: str, unit: str = "fahrenheit"):
    """Get the current weather for a location"""
    # This would call a real weather API
    return {
        "location": location,
        "temperature": 72,
        "unit": unit,
        "forecast": "sunny"
    }

# Define the function schema
functions = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"]
                    }
                },
                "required": ["location"]
            }
        }
    }
]

# Make the API call
response = client.chat.completions.create(
    model="openai/gpt-4.1-nano",
    messages=[
        {"role": "user", "content": "What's the weather like in Boston?"}
    ],
    tools=functions,
    tool_choice="auto"
)

# Process function calls
message = response.choices[0].message
if message.tool_calls:
    for tool_call in message.tool_calls:
        if tool_call.function.name == "get_weather":
            args = json.loads(tool_call.function.arguments)
            result = get_weather(**args)
            print(f"Weather in {args['location']}: {result}")

```

--------------------------------

### Example Chat Completions Request (cURL)

Source: https://docs.withmartian.com/api-reference/endpoints

Demonstrates how to make a POST request to the chat completions endpoint using cURL. Ensure you replace $MARTIAN_API_KEY with your actual API key.

```curl
curl https://api.withmartian.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "model": "openai/gpt-4.1-nano",
    "messages": [
      {
        "role": "user",
        "content": "Explain quantum computing in simple terms."
      }
    ],
    "temperature": 0.7,
    "max_completion_tokens": 500
  }'
```

--------------------------------

### Anthropic Format Python Example for Tool Use

Source: https://docs.withmartian.com/api-reference/advanced

This Python script utilizes the Anthropic client library to demonstrate tool use, defining a 'get_weather' tool and processing its output.

```python
import anthropic


anth_client = anthropic.Anthropic(
    base_url="https://api.withmartian.com/v1",
    api_key=MARTIAN_API_KEY
)

response = anth_client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=1024,
    tools=[
        {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                }
            }
        }
    ],
    messages=[
        {
            "role": "user",
            "content": "What's the weather in San Francisco?"
        }
    ]
)

for item in response.content:
    if item.type == "tool_use":
        print("Tool name:", item.name)
        print("Input args:", item.input)
```

--------------------------------

### Model Name Resolution Examples

Source: https://docs.withmartian.com/integrations/vercel-ai-sdk

Demonstrates how the Martian provider handles model name resolution for OpenAI and other providers. Auto-prefixed OpenAI models and pass-through for others.

```typescript
import { generateText } from "ai";
import { martianProvider } from "@withmartian/ai-sdk-provider";

// OpenAI models - auto-prefixed
await generateText({
  model: martianProvider("gpt-4o-mini"), // → openai/gpt-4o-mini
  prompt: "Hello"
});

await generateText({
  model: martianProvider("openai/gpt-4o"), // → openai/gpt-4o
  prompt: "Hello"
});

// Any other model - passed through
await generateText({
  model: martianProvider("anthropic/claude-sonnet-4-20250514"),
  prompt: "Hello"
});

```

--------------------------------

### Anthropic Format cURL Example for Tool Use

Source: https://docs.withmartian.com/api-reference/advanced

This cURL command shows how to invoke the Anthropic compatible API for tool use, defining a 'get_weather' tool with its input schema.

```bash
curl https://api.withmartian.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5-20250929",
    "max_tokens": 1024,
    "tools": [
      {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "input_schema": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string"
            }
          }
        }
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "Use the get_weather tool to check San Francisco weather"
      }
    ]
  }'
```

--------------------------------

### Prompt Caching Example

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Utilize Anthropic's prompt caching feature to reduce costs and latency. The system instruction includes a cacheable context, and subsequent requests with identical instructions benefit from the cache.

```python
system_instruction = [
    {
        "type": "text",
        "text": "You are a helpful assistant."
    },
    {
        "type": "text",
        "text": "<large context document here>",
        "cache_control": {"type": "ephemeral"}
    }
]

# First request - creates cache
message1 = client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=100,
    system=system_instruction,
    messages=[{"role": "user", "content": "Question 1"}]
)

# Second request - uses cache
message2 = client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=100,
    system=system_instruction,  # Must be identical
    messages=[{"role": "user", "content": "Question 2"}]
)
```

--------------------------------

### Basic Message Request (Python)

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Make a simple message request to the Anthropic API using the configured client. This example demonstrates a basic user query and prints the response.

```python
message = client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "What is Olympus Mons?"}
    ]
)

print(message.content[0].text)
```

--------------------------------

### Implement API Call with Retry Logic

Source: https://docs.withmartian.com/resources/examples

Implement retry logic for API calls using the `tenacity` library. This example retries up to 3 times with exponential backoff.

```python
from tenacity import retry, stop_after_attempt, wait_exponential

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=4, max=10)
)
def call_martian_api(messages):
    return client.chat.completions.create(
        model="openai/gpt-4.1-nano",
        messages=messages
    )


```

--------------------------------

### API Response Showing Cache Creation

Source: https://docs.withmartian.com/api-reference/advanced

This is an example response from the first cURL request. It includes usage metadata, specifically 'cache_creation_input_tokens', indicating that the provided content was successfully cached.

```json
{
  "id": "msg_XXXxXxXXxXxxXX",
  "type": "message",
  "role": "assistant",
  "model": "anthropic/claude-opus-4-20250514",
  "content": [
    {
      "type": "text",
      "text": "According to the information provided, Jupiter is the largest planet in our solar system..."
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 27,
    "cache_creation_input_tokens": 1279,
    "cache_read_input_tokens": 0,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 1279,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 45,
    "service_tier": "standard"
  }
}
```

--------------------------------

### Invoke Anthropic Messages API with Python SDK

Source: https://docs.withmartian.com/quickstart

This Python example shows how to use the Anthropic SDK to communicate with the Martian Gateway's messages endpoint. Make sure MARTIAN_API_KEY is set.

```python
import anthropic

# Note: Define MARTIAN_API_KEY as your assigned API key
_client = anthropic.Anthropic(
    base_url="https://api.withmartian.com/v1",
    api_key=MARTIAN_API_KEY
)

response = anth_client.messages.create(
    model="anthropic/claude-3-haiku-20240307",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Write a haiku about Mars (the planet, not the god)."
        }
    ]
)

print(response.content[0].text)
```

--------------------------------

### Set Up Cache with cURL

Source: https://docs.withmartian.com/api-reference/advanced

Use this cURL command to make the first request to the API, which sets up the cache with provided system prompts and content. Observe the 'cache_creation_input_tokens' in the response.

```bash
curl https://api.withmartian.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "model": "anthropic/claude-opus-4-20250514",
    "max_tokens": 100,
    "system": [
      {
        "type": "text",
        "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on facts, specifications, and other information.\n"
      },
      {
        "type": "text",
        "text": "<the entire contents of Space! : The Universe As You have Never Seen It Before>",
        "cache_control": { "type": "ephemeral" }
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "What is the largest planet?"
      }
    ]
  }'
```

--------------------------------

### Initialize K-Steering Model

Source: https://docs.withmartian.com/k-steering/how-it-works

Initialize the KSteering wrapper with the chosen base model name and steering configuration.

```python
steer_model = KSteering(
    model_name=MODEL_NAME,
    steering_config=steering_config,
)

```

--------------------------------

### KSteering

Source: https://docs.withmartian.com/k-steering/core-concepts

The main entry point for training and applying steering mechanisms within the K-Steering framework.

```APIDOC
## KSteering

Main entry point for training and applying steering.

### Parameters

- **model_name** (str) - Required - Hugging Face model identifier.
- **steering_config** (SteeringConfig) - Required - Configuration object defining steering behavior.
```

--------------------------------

### Import K-Steering Modules

Source: https://docs.withmartian.com/k-steering/how-it-works

Import the necessary classes, SteeringConfig and KSteering, from the k_steering library to begin using its functionalities.

```python
from k_steering.steering.config import SteeringConfig
from k_steering.steering.k_steer import KSteering

```

--------------------------------

### Dataset Loading

Source: https://docs.withmartian.com/k-steering/core-concepts

Demonstrates how to load custom datasets for K-Steering using various sources like Hugging Face, CSV, JSON, and DataFrames, along with defining the dataset schema.

```APIDOC
## Dataset Loading

This section shows how to load custom datasets for K-Steering.

### `DatasetSchema`

Defines the schema for your dataset.

- **prompt_column** (str) - Required - The column containing the input question or prompt.
- **category_columns** (list[str]) - Required - A list of column names, where each column represents a steering category.

### `TaskDataset.from_huggingface`

Loads a dataset from a Hugging Face repository.

- **repo_id** (str) - Required - The Hugging Face repository ID.
- **split** (str) - Required - The dataset split to load (e.g., "train").
- **schema** (DatasetSchema) - Required - The schema definition for the dataset.

### `TaskDataset.from_csv`

Loads a dataset from a CSV file.

- **path** (str) - Required - The path to the CSV file.
- **schema** (DatasetSchema) - Required - The schema definition for the dataset.

### `TaskDataset.from_json`

Loads a dataset from a JSON file.

- **path** (str) - Required - The path to the JSON file.
- **schema** (DatasetSchema) - Required - The schema definition for the dataset.

### `TaskDataset.from_dataframe`

Loads a dataset from a pandas DataFrame.

- **df** (pandas.DataFrame) - Required - The DataFrame containing the dataset.
- **schema** (DatasetSchema) - Required - The schema definition for the dataset.

### Example Usage

```python
from k_steering.steering.dataset import DatasetSchema, TaskDataset

schema = DatasetSchema(
    prompt_column="Question",
    category_columns=["Expert", "Casual"],
)

# From Hugging Face
dataset, eval_prompts = TaskDataset.from_huggingface(
    repo_id="your-username/your-dataset",
    split="train",
    schema=schema,
)

# From CSV
dataset, eval_prompts = TaskDataset.from_csv(path="my_dataset.csv", schema=schema)

# From JSON
dataset, eval_prompts = TaskDataset.from_json(path="my_dataset.json", schema=schema)

# From DataFrame
dataset, eval_prompts = TaskDataset.from_dataframe(df=df, schema=schema)
```
```

--------------------------------

### Create .claude Directory for Per-Project Config

Source: https://docs.withmartian.com/integrations/claude-code

Create a .claude directory in your project root to house per-project configuration files.

```shell
mkdir .claude

```

--------------------------------

### Create .env File for Per-Project API Key

Source: https://docs.withmartian.com/integrations/claude-code

For per-project configuration, store your Martian API key in a .env file in the project root and add .env to .gitignore.

```shell
echo "MARTIAN_API_KEY=your-martian-api-key" >> .env
echo ".env" >> .gitignore

```

--------------------------------

### Python Requests Chat Completions

Source: https://docs.withmartian.com/integrations/http-client

Use the `requests` library in Python to make a chat completions request. This example shows how to set up headers and JSON payload for the API.

```python
import requests

url = "https://api.withmartian.com/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer MARTIAN_API_KEY"
}
data = {
    "model": "openai/gpt-4.1-nano",
    "messages": [
        {"role": "user", "content": "What is Olympus Mons?"}
    ]
}

response = requests.post(url, headers=headers, json=data)
result = response.json()
print(result['choices'][0]['message']['content'])
```

--------------------------------

### Configure OpenAI SDK (Node.js/TypeScript)

Source: https://docs.withmartian.com/integrations/openai-sdk

Configure the OpenAI SDK with Martian Gateway's base URL and API key for Node.js/TypeScript.

```javascript
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.withmartian.com/v1',
  apiKey: process.env.MARTIAN_API_KEY,
});

```

--------------------------------

### Configure OpenAI SDK for Martian Gateway

Source: https://docs.withmartian.com/api-reference/authentication

Set up the OpenAI Python SDK to communicate with the Martian Gateway API. Provide the base URL and your API key during client initialization.

```python
import openai

client = openai.OpenAI(
    base_url="https://api.withmartian.com/v1",
    api_key=MARTIAN_API_KEY
)
```

--------------------------------

### Authentication Error Response

Source: https://docs.withmartian.com/api-reference/errors

Example of an authentication error, typically returned with a 401 status code. Ensure your API key is valid and correctly formatted in the Authorization header.

```json
{
  "error": {
    "message": "Invalid API key provided",
    "type": "invalid_request_error",
    "code": "invalid_api_key"
  }
}
```

--------------------------------

### Accessing Models from Different Providers

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Demonstrates how to use the Anthropic SDK to access models from Anthropic, OpenAI, and Google by specifying the correct model identifier in the request.

```python
# Use Anthropic models
message = client.messages.create(
    model="anthropic/claude-sonnet-4-20250514",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)

# Use OpenAI models
message = client.messages.create(
    model="openai/gpt-4.1-nano",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)

# Use Google models
message = client.messages.create(
    model="google/gemini-2.5-flash",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)
```

--------------------------------

### Make Requests to LiteLLM Proxy with OpenAI SDK

Source: https://docs.withmartian.com/integrations/litellm

Use the OpenAI Python SDK to make chat completion requests to the running LiteLLM proxy. Configure the client with the LiteLLM master key and the proxy's base URL.

```python
import openai

client = openai.OpenAI(
    api_key="sk-1234",  # Your LiteLLM master key
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="martian-gpt-5",  # Use the model_name from config
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

```

--------------------------------

### Access Multi-Provider Models (Python)

Source: https://docs.withmartian.com/integrations/openai-sdk

Use the OpenAI SDK to access models from different providers like Anthropic, Google, and Meta.

```python
# Use Anthropic models
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Use Google models
response = client.chat.completions.create(
    model="google/gemini-2.5-flash",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Use Meta models
response = client.chat.completions.create(
    model="meta-llama/llama-3.3-70b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

```

--------------------------------

### Error Handling (Python)

Source: https://docs.withmartian.com/integrations/anthropic-sdk

Implement robust error handling for API requests. This example shows how to catch specific Anthropic exceptions like AuthenticationError, RateLimitError, and general APIError.

```python
import anthropic

try:
    message = client.messages.create(
        model="anthropic/claude-sonnet-4-20250514",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello!"}]
    )
except anthropic.AuthenticationError:
    print("Invalid API key")
except anthropic.RateLimitError:
    print("Rate limit exceeded")
except anthropic.APIError as e:
    print(f"API error: {e}")
```

--------------------------------

### LiteLLM Proxy Configuration File

Source: https://docs.withmartian.com/integrations/litellm

Configure LiteLLM Proxy with Martian Gateway models, including GPT-5, Claude Sonnet 4, and Gemini 2.5 Flash. Set up model names, LiteLLM parameters, and general settings like master key.

```yaml
model_list:
  # GPT-5 via the Martian Gateway with cost optimization
  - model_name: martian-gpt-5
    litellm_params:
      model: openai/openai/gpt-5
      api_base: https://api.withmartian.com/v1
      api_key: "os.environ/MARTIAN_API_KEY"

  # Claude Sonnet 4 via the Martian Gateway with cost optimization
  - model_name: martian-sonnet-4
    litellm_params:
      model: anthropic/anthropic/claude-sonnet-4-20250514
      api_base: https://api.withmartian.com/v1
      api_key: "os.environ/MARTIAN_API_KEY"

  # Gemini 2.5 Flash via the Martian Gateway
  - model_name: martian-gemini-flash
    litellm_params:
      model: openai/google/gemini-2.5-flash
      api_base: https://api.withmartian.com/v1
      api_key: "os.environ/MARTIAN_API_KEY"

general_settings:
  master_key: sk-1234  # Your LiteLLM admin key

```

--------------------------------

### Chat Completion Request using cURL

Source: https://docs.withmartian.com/gateway

Example of how to make a chat completion request to the Martian Gateway API using cURL. Ensure you have your MARTIAN_API_KEY set as an environment variable.

```shell
curl https://api.withmartian.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $MARTIAN_API_KEY" \
  -d '{
    "model": "openai/gpt-4.1-nano",
    "messages": [
      {
        "role": "user",
        "content": "What is Olympus Mons?"
      }
    ]
  }'
```

--------------------------------

### API Response Showing Cache Reuse

Source: https://docs.withmartian.com/api-reference/advanced

This is an example response from the second cURL request. The 'cache_read_input_tokens' in the usage metadata confirms that cached content was successfully reused, indicating efficiency gains.

```json
{
  "id": "msg_YYYyYyYYyYyyYY",
  "type": "message",
  "role": "assistant",
  "model": "anthropic/claude-opus-4-20250514",
  "content": [
    {
      "type": "text",
      "text": "Mercury is the planet closest to the Sun, orbiting at an average distance of..."
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 21,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 1279,
    "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 0
    },
    "output_tokens": 42,
    "service_tier": "standard"
  }
}
```

--------------------------------

### cURL Messages (Anthropic Format) Request

Source: https://docs.withmartian.com/integrations/http-client

Send a request to the Martian Gateway API using the Anthropic message format via cURL. This example demonstrates how to specify the model and content.

```bash
curl https://api.withmartian.com/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer MARTIAN_API_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-4-20250514",
    "max_tokens": 1024,
    "messages": [
      {
        "role": "user",
        "content": "What is Olympus Mons?"
      }
    ]
  }'
```
Tokens:
Raw