### SDK Integrations

Source: https://docs.llmgateway.io/quick-start

Examples of how to integrate with LLM Gateway using various SDKs.

```APIDOC
## SDK Integrations

### ai-sdk.ts
```typescript
import { llmgateway } from "@llmgateway/ai-sdk-provider";
import { generateText } from "ai";

const { text } = await generateText({
	model: llmgateway("openai/gpt-4o"),
	prompt: "Write a vegetarian lasagna recipe for 4 people.",
});
```

### vercel-ai-sdk.ts
```typescript
import { createOpenAI } from "@ai-sdk/openai";

const llmgateway = createOpenAI({
	baseURL: "https://api.llmgateway.io/v1",
	apiKey: process.env.LLM_GATEWAY_API_KEY!,
});

const completion = await llmgateway.chat({
	model: "gpt-4o",
	messages: [{ role: "user", content: "Hello, how are you?" }],
});

console.log(completion.choices[0].message.content);
```

### openai-sdk.ts
```typescript
import OpenAI from "openai";

const openai = new OpenAI({
	baseURL: "https://api.llmgateway.io/v1",
	apiKey: process.env.LLM_GATEWAY_API_KEY,
});

const completion = await openai.chat.completions.create({
	model: "gpt-4o",
	messages: [{ role: "user", content: "Hello, how are you?" }],
});

console.log(completion.choices[0].message.content);
```
```

--------------------------------

### Get Chat Completion from LLMGateway.io API using Rust

Source: https://docs.llmgateway.io/quick-start

This Rust example uses the reqwest and tokio crates to perform an asynchronous POST request to the LLMGateway.io API. It includes setting headers, sending JSON data, and deserializing the JSON response.

```rust
use reqwest::Client;
use serde_json::json;
use std::env;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new();
    let api_key = env::var("LLM_GATEWAY_API_KEY")?;

    let response = client
        .post("https://api.llmgateway.io/v1/chat/completions")
        .header("Content-Type", "application/json")
        .header("Authorization", format!("Bearer {}", api_key))
        .json(&json!({
            "model": "gpt-4o",
            "messages": [
                {"role": "user", "content": "Hello, how are you?"}
            ]
        }))
        .send()
        .await?;

    let result: serde_json::Value = response.json().await?;
    println!("{}", result["choices"][0]["message"]["content"]);

    Ok(())
}
```

--------------------------------

### Get Chat Completion from LLMGateway.io API using TypeScript

Source: https://docs.llmgateway.io/quick-start

This example demonstrates how to fetch a chat completion from the LLMGateway.io API using TypeScript's fetch API. It configures POST request with necessary headers and a JSON body, then logs the content of the first message choice.

```typescript
const response = await fetch('https://api.llmgateway.io/v1/chat/completions', {method: 'POST',headers: {  'Content-Type': 'application/json',  'Authorization': `Bearer ${process.env.LLM_GATEWAY_API_KEY}`},body: JSON.stringify({  model: 'gpt-4o',  messages: [    { role: 'user', content: 'Hello, how are you?' }  ]})});if (!response.ok) {throw new Error(`HTTP error! status: ${response.status}`);}const data = await response.json();console.log(data.choices[0].message.content);
```

--------------------------------

### Generate Text using LLM Gateway SDK (TypeScript)

Source: https://docs.llmgateway.io/quick-start

This TypeScript code snippet demonstrates using the `@llmgateway/ai-sdk-provider` and `ai` libraries to generate text. It configures the model using `llmgateway` and specifies the prompt. The `generateText` function is called to get the completion. Dependencies include '@llmgateway/ai-sdk-provider' and 'ai'.

```typescript
import { llmgateway } from "@llmgateway/ai-sdk-provider";
import { generateText } from "ai";

const { text } = await generateText({
	model: llmgateway("openai/gpt-4o"),
	prompt: "Write a vegetarian lasagna recipe for 4 people.",
});
```

--------------------------------

### Get Chat Completion from LLMGateway.io API using Java

Source: https://docs.llmgateway.io/quick-start

This Java code snippet utilizes the HttpClient to send a POST request to the LLMGateway.io API. It constructs the request with appropriate headers and a JSON body, then prints the response body.

```java
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.URI;

String apiKey = System.getenv("LLM_GATEWAY_API_KEY");
String requestBody = """{\"model\": \"gpt-4o\",\"messages\": [{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]}"""

HttpRequest request = HttpRequest.newBuilder()
    .uri(URI.create("https://api.llmgateway.io/v1/chat/completions"))
    .header("Content-Type", "application/json")
    .header("Authorization", "Bearer " + apiKey)
    .POST(HttpRequest.BodyPublishers.ofString(requestBody))
    .build();

HttpResponse<String> response = HttpClient.newHttpClient().send(request, HttpResponse.BodyHandlers.ofString());

System.out.println(response.body());
```

--------------------------------

### Get Chat Completion from LLMGateway.io API using Python

Source: https://docs.llmgateway.io/quick-start

This Python script uses the requests library to make a POST request to the LLMGateway.io API. It sets the required headers and JSON payload, then prints the content of the chat completion.

```python
import requests
import os

response = requests.post('https://api.llmgateway.io/v1/chat/completions',
                           headers={'Content-Type': 'application/json',
                                    'Authorization': f'Bearer {os.getenv("LLM_GATEWAY_API_KEY")}'},
                           json={'model': 'gpt-4o',
                                 'messages': [{'role': 'user', 'content': 'Hello, how are you?'}]}
                          )
response.raise_for_status()
print(response.json()['choices'][0]['message']['content'])
```

--------------------------------

### Make a POST Request to LLMGateway.io API using cURL

Source: https://docs.llmgateway.io/quick-start

This snippet shows how to send a POST request to the LLMGateway.io API using cURL to get a chat completion. It includes setting the Content-Type and Authorization headers, and a JSON payload with the model and messages.

```curl
curl -X POST https://api.llmgateway.io/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
-d '{"model": "gpt-4o","messages": [  {"role": "user", "content": "Hello, how are you?"}]}'
```

--------------------------------

### Chat Completions API

Source: https://docs.llmgateway.io/quick-start

This endpoint allows you to interact with various language models to generate chat completions.

```APIDOC
## POST /v1/chat/completions

### Description
This endpoint allows you to send a chat message and receive a response from a language model. It supports different models and message formats.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - The ID of the model to use for generating the chat completion.
- **messages** (array) - Required - A list of messages comprising the conversation so far. Each message is an object with 'role' and 'content'.
  - **role** (string) - Required - The role of the author of the message ('user' or 'assistant').
  - **content** (string) - Required - The content of the message.

### Request Example
```json
{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "user",
      "content": "Hello, how are you?"
    }
  ]
}
```

### Response
#### Success Response (200)
- **choices** (array) - A list of chat completion choices.
  - **message** (object) - The message content and role.
    - **role** (string) - The role of the author of the message ('assistant').
    - **content** (string) - The content of the assistant's message.

#### Response Example
```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "I'm doing well, thank you for asking! How can I help you today?"
      }
    }
  ]
}
```
```

--------------------------------

### Deploy LLMGateway Unified Image with Docker Compose

Source: https://docs.llmgateway.io/self-host

This set of commands downloads the Docker Compose file for the unified LLMGateway image and its example environment configuration. After copying and editing the .env file with your settings, you can start the service in detached mode. It is recommended to update the version tag in the compose file to the latest release.

```bash
# Download the compose file
curl -O https://raw.githubusercontent.com/theopenco/llmgateway/main/infra/docker-compose.unified.yml
curl -O https://raw.githubusercontent.com/theopenco/llmgateway/main/.env.example

# Configure environment
cp .env.example .env
# Edit .env with your configuration

# Start the service
docker compose -f docker-compose.unified.yml up -d
```

--------------------------------

### Chat Completions API

Source: https://docs.llmgateway.io/quick-start

This endpoint allows you to send messages to the chat model and receive a completion.

```APIDOC
## POST /v1/chat/completions

### Description
Sends messages to the chat model and receives a completion.

### Method
POST

### Endpoint
`https://api.llmgateway.io/v1/chat/completions`

### Parameters
#### Request Body
- **model** (string) - Required - The model to use for chat completions (e.g., "gpt-4o").
- **messages** (array) - Required - A list of message objects, where each object has a `role` (e.g., "user", "system", "assistant") and `content` (string).

### Request Example
```json
{
  "model": "gpt-4o",
  "messages": [
    {"role": "user", "content": "Hello, how are you?"}
  ]
}
```

### Response
#### Success Response (200)
- **choices** (array) - A list of completion choices. Each choice contains a `message` object with `role` and `content`.

#### Response Example
```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "I am doing well, thank you for asking!"
      }
    }
  ]
}
```
```

--------------------------------

### Example: Tracking Costs in Code

Source: https://docs.llmgateway.io/features/cost-breakdown

Provides a practical JavaScript example demonstrating how to programmatically access and log the cost breakdown information from API responses.

```APIDOC
## Example: Tracking Costs in Code

Here's an example of how to track costs programmatically using the cost breakdown feature:

```javascript
import OpenAI from "openai";

const client = new OpenAI({
	apiKey: process.env.LLM_GATEWAY_API_KEY,
	baseURL: "https://api.llmgateway.io/v1",
});

async function trackCosts() {
	const response = await client.chat.completions.create({
		model: "gpt-4o",
		messages: [{ role: "user", content: "Hello!" }],
	});

	const usage = response.usage as any;

	if (usage.cost_usd_total !== undefined) {
		console.log(`Request cost: $${usage.cost_usd_total.toFixed(6)}`);
		console.log(`  Input: $${usage.cost_usd_input.toFixed(6)}`);
		console.log(`  Output: $${usage.cost_usd_output.toFixed(6)}`);

		if (usage.cost_usd_cached_input > 0) {
			console.log(`  Cached: $${usage.cost_usd_cached_input.toFixed(6)}`);
		}
	}

	return response;
}
```
```

--------------------------------

### Create a Next.js API Route for Chat Completions

Source: https://docs.llmgateway.io/quick-start

This Next.js API route handles POST requests to the /api/chat endpoint. It parses the incoming message, sends it to the LLMGateway.io API, and returns the response or an error.

```typescript
// app/api/chat/route.ts
import { NextRequest, NextResponse } from "next/server";

export async function POST(request: NextRequest) {
  const { message } = await request.json();
  const response = await fetch('https://api.llmgateway.io/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': `Bearer ${process.env.LLM_GATEWAY_API_KEY}`
    },
    body: JSON.stringify({
      model: 'gpt-4o',
      messages: [{ role: 'user', content: message }]
    })
  });

  if (!response.ok) {
    return NextResponse.json({ error: 'Failed to get response' }, { status: response.status });
  }

  const data = await response.json();
  return NextResponse.json({message: data.choices[0].message.content});
}

// Usage in component:
// const response = await fetch('/api/chat', {
// method: 'POST',
// headers: { 'Content-Type': 'application/json' },
// body: JSON.stringify({ message: 'Hello, how are you?' })
// });
```

--------------------------------

### Export LLM Gateway API Key (Shell)

Source: https://docs.llmgateway.io/quick-start

This snippet shows how to export your LLM Gateway API key as an environment variable in a shell. This is a common first step to authenticate your requests to the LLM Gateway API.

```bash
export LLM_GATEWAY_API_KEY="llmgtwy_XXXXXXXXXXXXXXXX"
```

--------------------------------

### Build a Chat Component in React with LLMGateway.io API

Source: https://docs.llmgateway.io/quick-start

This React component uses the useState hook to manage response and loading states. It includes a sendMessage function that makes a POST request to the LLMGateway.io API and displays the received message.

```javascript
import { useState } from 'react'
function ChatComponent() {
const [response, setResponse] = useState('');
const [loading, setLoading] = useState(false);
const sendMessage = async () => {
setLoading(true);
try {
const res = await fetch('https://api.llmgateway.io/v1/chat/completions', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${process.env.REACT_APP_LLM_GATEWAY_API_KEY}`
},
body: JSON.stringify({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Hello, how are you?' }]
})
});
    if (!res.ok) {
      throw new Error(`HTTP error! status: ${res.status}`);
    }
    const data = await res.json();
    setResponse(data.choices[0].message.content);
  } catch (error) {
    console.error('Error:', error);
  } finally {
    setLoading(false);
  }
};
return (
<div>
<button onClick={sendMessage} disabled={loading}>
	{loading ? "Sending..." : "Send Message"}
</button>
{response && <p>{response}</p>}
</div>
);
}
export default ChatComponent;
```

--------------------------------

### Enable Reasoning in Chat Completions Request (cURL)

Source: https://docs.llmgateway.io/features/reasoning

This example demonstrates how to make a POST request to the LLM Gateway API to get reasoning output. It includes the model, messages, and the 'reasoning_effort' parameter set to 'medium'. The 'Authorization' and 'Content-Type' headers are also required.

```cURL
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5-preview",
    "messages": [
      {
        "role": "user",
        "content": "What is 2/3 + 1/4 + 5/6?"
      }
    ],
    "reasoning_effort": "medium"
  }'
```

--------------------------------

### Caching Usage Example - Development and Testing

Source: https://docs.llmgateway.io/features/caching

Demonstrates how to use LLM Gateway's caching for development and testing. Repeated identical prompts are cached, incurring costs only on the first request.

```javascript
const response = await client.chat.completions.create({
	model: "gpt-4o",
	messages: [{ role: "user", content: "Explain quantum computing" }],
});
```

--------------------------------

### Caching Usage Example - Batch Processing

Source: https://docs.llmgateway.io/features/caching

Shows how LLM Gateway caching optimizes batch processing. Duplicate items within a batch are served from the cache, reducing redundant calls.

```javascript
for (const item of items) {
	const response = await client.chat.completions.create({
		model: "gpt-4o",
		messages: [{ role: "user", content: `Classify: ${item}` }],
	});
}
```

--------------------------------

### Configure Image Generation for Z.AI Models

Source: https://docs.llmgateway.io/features/image-generation

This example shows how to configure image generation for Z.AI models, specifying image size using WIDTHxHEIGHT format. It utilizes the `image_config` parameter with options relevant to Z.AI's CogView-4 model.

```curl
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ \
    "model": "zai/cogview-4", \
    "messages": [ \
      { \
        "role": "user", \
        "content": "Generate an image of a futuristic city skyline" \
      } \
    ], \
    "image_config": { \
      "image_size": "1024x1024" \
    } \
  }'
```

--------------------------------

### Using API Keys

Source: https://docs.llmgateway.io/features/api-keys

This section details how to use your API key to authenticate your requests to the LLM Gateway API. It provides an example using curl.

```APIDOC
## Using API Keys

Once you have an API key, use it in the `Authorization` header of your requests:

### Request Example
```bash
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer llmgtwy_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{ 
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

### Request Headers
- **Authorization** (string): `Bearer YOUR_API_KEY` - Your LLM Gateway API key.
- **Content-Type** (string): `application/json` - Specifies the request body format.
```

--------------------------------

### Send Multi-tenant Metadata (cURL)

Source: https://docs.llmgateway.io/features/metadata

Example of sending metadata for a multi-tenant application using cURL. It includes headers for tenant ID, user ID, application version, and feature, enabling detailed usage tracking and cost analysis per segment.

```bash
curl -X POST https://api.llmgateway.io/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "X-LLMGateway-Tenant-ID: acme-corp" \
  -H "X-LLMGateway-User-ID: user-12345" \
  -H "X-LLMGateway-App-Version: 2.1.4" \
  -H "X-LLMGateway-Feature: chat-assistant" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Summarize this document..."
      }
    ]
  }'
```

--------------------------------

### Configure Image Generation for Google Models

Source: https://docs.llmgateway.io/features/image-generation

This example shows how to customize image generation for Google models by specifying aspect ratio and image size. It uses the chat completions API and includes an `image_config` object within the request.

```curl
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ \
    "model": "gemini-3-pro-image-preview", \
    "messages": [ \
      { \
        "role": "user", \
        "content": "Generate an image of a mountain landscape at sunset" \
      } \
    ], \
    "image_config": { \
      "aspect_ratio": "16:9", \
      "image_size": "4K" \
    } \
  }'
```

--------------------------------

### Auto Routing to Free Models Only

Source: https://docs.llmgateway.io/features/routing

This example demonstrates how to restrict auto routing to only utilize free models by setting the `free_models_only` parameter to `true`. This is useful for managing costs, but will result in an error if no suitable free models are available.

```curl
# Auto route to free models only
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello!"}],
    "free_models_only": true
  }'
```

--------------------------------

### Route to a Specific Provider (cURL)

Source: https://docs.llmgateway.io/features/routing

This example shows how to explicitly route a request to a specific provider, such as OpenAI or CloudRift, by prefixing the model name with the provider's identifier. If the specified provider has low uptime, LLMGateway will attempt to route to an alternative provider.

```curl
# Use OpenAI specifically
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Use the CloudRift provider specifically
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cloudrift/deepseek-v3",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

--------------------------------

### Make Requests to Custom OpenAI-Compatible Providers with cURL

Source: https://docs.llmgateway.io/features/custom-providers

This example demonstrates how to make a POST request to the LLM Gateway API to interact with a configured custom OpenAI-compatible provider. It includes setting the authorization header, content type, and the request payload with model and messages.

```curl
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mycompany/custom-gpt-4",
    "messages": [
      {
        "role": "user",
        "content": "Hello from my custom provider!"
      }
    ]
  }'
```

--------------------------------

### Chat Completion with OpenAI SDK (TypeScript)

Source: https://docs.llmgateway.io/quick-start

This TypeScript code snippet demonstrates using the official OpenAI SDK to interact with LLM Gateway. It configures the `OpenAI` client with the LLM Gateway's base URL and API key, then uses `openai.chat.completions.create` to send a chat message. The content of the response is printed to the console. Dependencies include 'openai'.

```typescript
import OpenAI from "openai";

const openai = new OpenAI({
	baseURL: "https://api.llmgateway.io/v1",
	apiKey: process.env.LLM_GATEWAY_API_KEY,
});

const completion = await openai.chat.completions.create({
	model: "gpt-4o",
	messages: [{ role: "user", content: "Hello, how are you?" }],
});

console.log(completion.choices[0].message.content);
```

--------------------------------

### Send Chat Completion Request (Go)

Source: https://docs.llmgateway.io/quick-start

This Go code snippet demonstrates how to send a chat completion request to the LLM Gateway API. It sets up the request body, marshals it to JSON, creates an HTTP POST request, sets necessary headers including authentication, and sends the request using an HTTP client. The response body is closed using defer. Dependencies include the 'net/http', 'encoding/json', 'bytes', 'fmt', and 'os' packages.

```go
package main

import (
  "bytes"
  "encoding/json"
  "fmt"
  "net/http"
  "os"
)

type ChatRequest struct {
  Model string `json:"model"`
  Messages []Message `json:"messages"`
}

type Message struct {
  Role string `json:"role"`
  Content string `json:"content"`
}

func main() {
  apiKey := os.Getenv("LLM_GATEWAY_API_KEY")
  requestBody := ChatRequest{
      Model: "gpt-4o",
      Messages: []Message{{Role: "user", Content: "Hello, how are you?"}},
  }
  jsonData, _ := json.Marshal(requestBody)
  req, _ := http.NewRequest("POST", "https://api.llmgateway.io/v1/chat/completions", bytes.NewBuffer(jsonData))
  req.Header.Set("Content-Type", "application/json")
  req.Header.Set("Authorization", "Bearer "+apiKey)
  client := &http.Client{}
  resp, _ := client.Do(req)
  defer resp.Body.Close()
  fmt.Println("Response received")
}
```

--------------------------------

### Chat Completion with Vercel AI SDK (TypeScript)

Source: https://docs.llmgateway.io/quick-start

This TypeScript code snippet shows how to perform a chat completion using the Vercel AI SDK with LLM Gateway. It configures `createOpenAI` with the LLM Gateway's base URL and API key, then uses the `chat` method to send messages. The completion result is logged to the console. Dependencies include '@ai-sdk/openai'.

```typescript
import { createOpenAI } from "@ai-sdk/openai";

const llmgateway = createOpenAI({
	baseURL: "https://api.llmgateway.io/v1",
	apiKey: process.env.LLM_GATEWAY_API_KEY!,
});

const completion = await llmgateway.chat({
	model: "gpt-4o",
	messages: [{ role: "user", content: "Hello, how are you?" }],
});

console.log(completion.choices[0].message.content);
```

--------------------------------

### Example Cached Response Usage Data

Source: https://docs.llmgateway.io/features/caching

Provides an example of the usage data returned for a cached response. Cached responses indicate zero or minimal token usage, signifying no inference costs.

```json
{
	"usage": {
		"prompt_tokens": 0,
		"completion_tokens": 0,
		"total_tokens": 0,
		"cost_usd_total": 0
	}
}
```

--------------------------------

### Anthropic Response Format Example

Source: https://docs.llmgateway.io/features/anthropic-endpoint

Example of the response format received from LLMGateway's Anthropic-compatible endpoint. It follows Anthropic's message structure, including message ID, type, role, model, content, stop reason, and usage statistics.

```json
{
	"id": "msg_abc123",
	"type": "message",
	"role": "assistant",
	"model": "gpt-5",
	"content": [
		{
			"type": "text",
			"text": "Hello! I'm doing well, thank you for asking. How can I help you today?"
		}
	],
	"stop_reason": "end_turn",
	"stop_sequence": null,
	"usage": {
		"input_tokens": 13,
		"output_tokens": 20
	}
}
```

--------------------------------

### GET /v1/models

Source: https://docs.llmgateway.io/v1_models

Lists all available models, with options to include deactivated or exclude deprecated models.

```APIDOC
## GET /v1/models

### Description
Lists all available models. You can filter the results to include or exclude deactivated and deprecated models.

### Method
GET

### Endpoint
https://api.llmgateway.io/v1/models

### Query Parameters
- **include_deactivated** (string) - Optional - Include deactivated models in the response.
- **exclude_deprecated** (string) - Optional - Exclude deprecated models from the response.

### Response
#### Success Response (200)
- **data** (array) - An array of model objects.
  - **id** (string) - The unique identifier for the model.
  - **name** (string) - The name of the model.
  - **aliases** (array) - An array of alternative names for the model.
  - **created** (integer) - Timestamp of when the model was created.
  - **description** (string) - A description of the model.
  - **family** (string) - The model family it belongs to.
  - **architecture** (object) - Details about the model's architecture.
    - **input_modalities** (array) - List of input modalities supported (e.g., "text").
    - **output_modalities** (array) - List of output modalities supported (e.g., "text").
    - **tokenizer** (string) - The tokenizer used by the model.
  - **top_provider** (object) - Information about the top provider for this model.
    - **is_moderated** (boolean) - Indicates if the provider is moderated.
  - **providers** (array) - An array of provider details for the model.
    - **providerId** (string) - The ID of the provider.
    - **modelName** (string) - The model name as used by the provider.
    - **pricing** (object) - Pricing information from the provider.
      - **prompt** (string) - Price per prompt token.
      - **completion** (string) - Price per completion token.
      - **image** (string) - Price per image token.
    - **streaming** (boolean) - Whether the provider supports streaming.
    - **vision** (boolean) - Whether the provider supports vision capabilities.
    - **cancellation** (boolean) - Whether the provider supports cancellation.
    - **tools** (boolean) - Whether the provider supports tools.
    - **parallelToolCalls** (boolean) - Whether the provider supports parallel tool calls.
    - **reasoning** (boolean) - Whether the provider supports reasoning features.
    - **stability** (string) - The stability level of the provider's service.
  - **pricing** (object) - General pricing information for the model.
    - **prompt** (string) - Price per prompt token.
    - **completion** (string) - Price per completion token.
    - **image** (string) - Price per image token.
    - **request** (string) - Price per request.
    - **input_cache_read** (string) - Price for reading from input cache.
    - **input_cache_write** (string) - Price for writing to input cache.
    - **web_search** (string) - Price for web search usage.
    - **internal_reasoning** (string) - Price for internal reasoning.
  - **context_length** (integer) - The maximum context length of the model.
  - **per_request_limits** (object) - Limits per request.
    - **property1** (string)
    - **property2** (string)
  - **supported_parameters** (array) - List of supported parameters for the model.
  - **json_output** (boolean) - Whether the model supports JSON output.
  - **structured_outputs** (boolean) - Whether the model supports structured outputs.
  - **free** (boolean) - Indicates if the model is free to use.
  - **deprecated_at** (string) - Timestamp when the model was deprecated.
  - **deactivated_at** (string) - Timestamp when the model was deactivated.
  - **stability** (string) - The overall stability of the model.

#### Response Example
```json
{
  "data": [
    {
      "id": "string",
      "name": "string",
      "aliases": [
        "string"
      ],
      "created": 0,
      "description": "string",
      "family": "string",
      "architecture": {
        "input_modalities": [
          "text"
        ],
        "output_modalities": [
          "text"
        ],
        "tokenizer": "string"
      },
      "top_provider": {
        "is_moderated": true
      },
      "providers": [
        {
          "providerId": "string",
          "modelName": "string",
          "pricing": {
            "prompt": "string",
            "completion": "string",
            "image": "string"
          },
          "streaming": true,
          "vision": true,
          "cancellation": true,
          "tools": true,
          "parallelToolCalls": true,
          "reasoning": true,
          "stability": "stable"
        }
      ],
      "pricing": {
        "prompt": "string",
        "completion": "string",
        "image": "string",
        "request": "string",
        "input_cache_read": "string",
        "input_cache_write": "string",
        "web_search": "string",
        "internal_reasoning": "string"
      },
      "context_length": 0,
      "per_request_limits": {
        "property1": "string",
        "property2": "string"
      },
      "supported_parameters": [
        "string"
      ],
      "json_output": true,
      "structured_outputs": true,
      "free": true,
      "deprecated_at": "string",
      "deactivated_at": "string",
      "stability": "stable"
    }
  ]
}
```
```

--------------------------------

### X-Source Header Example

Source: https://docs.llmgateway.io/features/source

Include the X-Source header with your domain name in your requests to LLM Gateway to attribute usage.

```APIDOC
## POST /v1/chat/completions

### Description
This endpoint is used for chat completions. Including the `X-Source` header allows for source attribution in public usage statistics.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Headers
- **Content-Type** (string) - Required - Specifies the format of the request body, usually `application/json`.
- **Authorization** (string) - Required - Bearer token for authentication.
- **X-Source** (string) - Optional - Your domain name (e.g., `example.com`, `https://example.com`) for source attribution.

#### Request Body
- **model** (string) - Required - The model to use for the chat completion (e.g., `gpt-4o`).
- **messages** (array) - Required - An array of message objects representing the conversation history.
  - **role** (string) - Required - The role of the message sender (`user`, `assistant`, `system`).
  - **content** (string) - Required - The content of the message.

### Request Example
```json
{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "user",
      "content": "Hello, how are you?"
    }
  ]
}
```

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the response.
- **object** (string) - Type of object returned, e.g., `chat.completion`.
- **created** (integer) - Timestamp of creation.
- **model** (string) - The model used for the completion.
- **choices** (array) - An array of completion choices.
  - **index** (integer) - Index of the choice.
  - **message** (object) - The message content.
    - **role** (string) - Role of the sender (`assistant`).
    - **content** (string) - The generated response content.
  - **logprobs** (any) - Log probability information (may be null).
  - **finish_reason** (string) - The reason the model stopped generating tokens (e.g., `stop`).
- **usage** (object) - Usage statistics for the request.
  - **prompt_tokens** (integer) - Number of tokens in the prompt.
  - **completion_tokens** (integer) - Number of tokens in the completion.
  - **total_tokens** (integer) - Total number of tokens used.

#### Response Example
```json
{
  "id": "chatcmpl-12345",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I am doing well, thank you for asking!"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 15,
    "total_tokens": 25
  }
}
```
```

--------------------------------

### Send Chat Completion Request (Ruby)

Source: https://docs.llmgateway.io/quick-start

This Ruby code snippet illustrates how to send a chat completion request to the LLM Gateway API. It utilizes the built-in `Net::HTTP` and `JSON` libraries to construct and send a POST request. The API key is fetched from the environment, and the request headers and JSON body are set. It handles potential HTTP errors by checking the response code and then parses the JSON response to extract and print the message content. Dependencies include 'net/http', 'json', and 'uri'.

```ruby
require 'net/http'
require 'json'
require 'uri'
uri = URI('https://api.llmgateway.io/v1/chat/completions')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Post.new(uri)
request['Content-Type'] = 'application/json'
request['Authorization'] = "Bearer #{ENV['LLM_GATEWAY_API_KEY']}"
request.body = {model: 'gpt-4o',messages: [{ role: 'user', content: 'Hello, how are you?' }]}.to_json
response = http.request(request)
if response.code != '200'
  raise "HTTP Error: #{response.code}"
end
result = JSON.parse(response.body)
puts result['choices'][0]['message']['content']
```

--------------------------------

### Health Check Endpoint (cURL)

Source: https://docs.llmgateway.io/health

Demonstrates how to perform a health check on the LLM Gateway using cURL. This is a basic GET request to the root endpoint.

```bash
curl -X GET "https://api.llmgateway.io"
```

--------------------------------

### Send Chat Completion Request (PHP)

Source: https://docs.llmgateway.io/quick-start

This PHP code snippet shows how to make a chat completion request to the LLM Gateway API. It retrieves the API key from environment variables, prepares the request data, sets up HTTP context options including headers and the JSON payload, and uses `file_get_contents` to send the request. It then decodes the JSON response and echoes the content of the first choice's message. This code relies on PHP's stream context functionality.

```php
<?php
$apiKey = $_ENV['LLM_GATEWAY_API_KEY'];
$data = ['model' => 'gpt-4o',
'messages' => [['role' => 'user', 'content' => 'Hello, how are you?']]];
$options = [
  'http' => [
      'header' => [
          'Content-Type: application/json',
          'Authorization: Bearer ' . $apiKey
      ],
      'method' => 'POST',
      'content' => json_encode($data)
  ]
];
$context = stream_context_create($options);
$response = file_get_contents('https://api.llmgateway.io/v1/chat/completions',false,$context);
if ($response === FALSE) {
  throw new Exception('Request failed');
}
$result = json_decode($response, true);
echo $result['choices'][0]['message']['content'];
?>
```

--------------------------------

### Deploy LLMGateway Separate Services with Docker Compose

Source: https://docs.llmgateway.io/self-host

This sequence clones the LLMGateway repository, copies and configures the environment file, and then starts the separate services using a specific Docker Compose file. This approach offers more flexibility in managing individual components. Remember to update image version tags to the latest releases.

```bash
# Clone the repository
git clone https://github.com/theopenco/llmgateway.git
cd llmgateway

# Configure environment
cp .env.example .env
# Edit .env with your configuration

# Start the services
docker compose -f infra/docker-compose.split.yml up -d
```

--------------------------------

### Caching Usage Example - Chatbots with Common Questions

Source: https://docs.llmgateway.io/features/caching

Illustrates using LLM Gateway's caching for common questions in chatbots. Frequently asked questions are efficiently served from the cache.

```javascript
const faqs = [
	"What are your business hours?",
	"How do I reset my password?",
	"What is your return policy?",
];
```

--------------------------------

### Send Request with X-Source Header (cURL)

Source: https://docs.llmgateway.io/features/source

This code snippet demonstrates how to include the X-Source header in a cURL request to LLM Gateway. The header specifies the domain making the request, which is used for public usage statistics. Ensure you replace '$LLM_GATEWAY_API_KEY' and 'example.com' with your actual API key and domain.

```bash
curl -X POST https://api.llmgateway.io/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -H "X-Source: example.com" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Hello, how are you?"
      }
    ]
  }'
```

--------------------------------

### Create Chat Completion (cURL)

Source: https://docs.llmgateway.io/v1_chat_completions

Example using cURL to send a POST request to the chat completions endpoint. It includes the necessary headers and a JSON body with a model and a user message.

```bash
curl -X POST "https://api.llmgateway.io/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'
```

--------------------------------

### Run Unified Docker Image for LLMGateway

Source: https://docs.llmgateway.io/self-host

This command runs a single Docker container that includes all LLMGateway services (UI, API, Gateway, Database, Redis). It maps necessary ports and mounts a volume for persistent data. Ensure to use the latest version tag from the GitHub releases instead of 'latest'.

```bash
docker run -d \
  --name llmgateway \
  --restart unless-stopped \
  -p 3002:3002 \
  -p 3003:3003 \
  -p 3005:3005 \
  -p 3006:3006 \
  -p 4001:4001 \
  -p 4002:4002 \
  -v ~/llmgateway_data:/var/lib/postgresql/data \
  -e AUTH_SECRET=your-secret-key-here \
  ghcr.io/theopenco/llmgateway-unified:latest
```

--------------------------------

### List Available Models via API (Python)

Source: https://docs.llmgateway.io/v1_models

This Python script uses the 'requests' library to make a GET request to the LLM Gateway API for listing models. It prints the JSON response.

```python
import requests

url = "https://api.llmgateway.io/v1/models"
response = requests.get(url)

if response.status_code == 200:
    print(response.json())
else:
    print(f"Error: {response.status_code}")
```

--------------------------------

### IAM Rules - Overview

Source: https://docs.llmgateway.io/features/api-keys

An introduction to IAM rules, explaining their purpose in providing fine-grained access control for models, providers, and pricing.

```APIDOC
## IAM Rules - Overview

IAM (Identity Access Management) rules provide fine-grained access control over what models, providers, and pricing tiers an API key can access.

### Description
Configure rules to restrict or allow access to specific LLM models, providers, and pricing tiers based on your security requirements.
```

--------------------------------

### List Available Models via API (Go)

Source: https://docs.llmgateway.io/v1_models

This Go program makes an HTTP GET request to the LLM Gateway API to retrieve a list of models. It handles the response and prints the JSON data.

```go
package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
)

func main() {
	resp, err := http.Get("https://api.llmgateway.io/v1/models")
	if err != nil {
		fmt.Printf("Error making request: %s\n", err)
		return
	}
	defer resp.Body.Close()

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		fmt.Printf("Error reading response body: %s\n", err)
		return
	}

	fmt.Println(string(body))
}
```

--------------------------------

### Extract Base64 Content from Data URL (TypeScript)

Source: https://docs.llmgateway.io/features/image-generation

Extracts the base64-encoded content from a given data URL string. If the input string does not start with 'data:', it returns an empty string, indicating invalid input.

```typescript
/**
 * Extracts base64-only content from a data URL.
 * Returns empty string if the input is not a valid data URL.
 */
export function extractBase64FromDataUrl(dataUrl: string): string {
	if (!dataUrl.startsWith("data:")) {
		return "";
	}

	const comma = dataUrl.indexOf(",");
	return comma >= 0 ? dataUrl.slice(comma + 1) : "";
}

```

--------------------------------

### Implement Budget Monitoring with LLM Gateway (TypeScript)

Source: https://docs.llmgateway.io/features/cost-breakdown

This TypeScript example illustrates how to implement a budget monitoring system using LLM Gateway's cost tracking feature. It maintains a running total of `totalSpent` and throws an error if a predefined `BUDGET_LIMIT` is exceeded. The cost of each API request is retrieved from the `cost_usd_total` field in the `usage` object of the response and added to the `totalSpent`.

```typescript
let totalSpent = 0;
const BUDGET_LIMIT = 10.0; // $10 budget

async function makeRequest(messages: Message[]) {
	const response = await client.chat.completions.create({
		model: "gpt-4o",
		messages,
	});

	const cost = (response.usage as any).cost_usd_total || 0;
	totalSpent += cost;

	if (totalSpent > BUDGET_LIMIT) {
		throw new Error(`Budget exceeded: $${totalSpent.toFixed(2)}`);
	}

	return response;
}
```