Kilo AI Gateway

https://kilo.ai/docs/gateway
Admin
Kilo AI Gateway is a unified, OpenAI-compatible API that provides access to hundreds of AI models...
Tokens:13,478
Snippets:118
Trust Score:8.2
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install Dependencies

Source: https://kilo.ai/docs/gateway/quickstart

Install the required AI SDK and OpenAI provider packages.

```bash
npm install ai @ai-sdk/openai dotenv
```

--------------------------------

### Install OpenAI SDK (TypeScript/JavaScript)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Install the OpenAI SDK using npm. This is a prerequisite for using the SDK.

```bash
npm install openai

```

--------------------------------

### Install OpenAI SDK (Python)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Install the OpenAI SDK using pip. This is a prerequisite for using the SDK.

```bash
pip install openai

```

--------------------------------

### Install Vercel AI SDK and OpenAI integration

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Install the necessary packages for using the Vercel AI SDK with OpenAI compatibility.

```bash
npm install ai @ai-sdk/openai

```

--------------------------------

### List Models Response Example

Source: https://kilo.ai/docs/gateway/api-reference

This JSON response shows an example of the data returned when listing available models. It includes model ID, object type, creation timestamp, owner, name, context length, and pricing details.

```json
{
  "data": [
    {
      "id": "anthropic/claude-sonnet-4.5",
      "object": "model",
      "created": 1739000000,
      "owned_by": "anthropic",
      "name": "Claude Sonnet 4.5",
      "context_length": 200000,
      "pricing": {
        "prompt": "0.000003",
        "completion": "0.000015"
      }
    }
  ]
}
```

--------------------------------

### Install Pi Provider Extension

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Install the Kilo-maintained Pi provider extension to access Kilo Gateway models from the Pi coding agent. Run `/login kilo` in Pi to connect your account or use free models without signing in.

```bash
pi install git:github.com/Kilo-Org/kilo-pi-provider

```

--------------------------------

### Model allow list configuration

Source: https://kilo.ai/docs/gateway/usage-and-billing

Examples of entries for restricting model access using exact matches or wildcard patterns.

```text
# Examples of allow list entries
anthropic/claude-sonnet-4.5      # Specific model
anthropic/*                       # All Anthropic models
openai/gpt-5.2                   # Specific OpenAI model
```

--------------------------------

### FIM Completion Example Request

Source: https://kilo.ai/docs/gateway/api-reference

An example cURL command to perform a Fill-in-the-Middle completion. It includes the model, prompt, suffix, and maximum tokens. Note that FIM completions are restricted to Mistral models.

```bash
curl -X POST "https://api.kilo.ai/api/fim/completions" \
  -H "Authorization: Bearer $KILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistralai/codestral-2508",
    "prompt": "def fibonacci(n):\n    if n <= 1:\n        return n\n    ",
    "suffix": "\n\nprint(fibonacci(10))",
    "max_tokens": 200,
    "stream": false
  }'
```

--------------------------------

### OpenAI SDK Integration (Python)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Configure the OpenAI SDK to use the Kilo Gateway by setting the base_url. Examples show non-streaming and streaming chat completion requests.

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("KILO_API_KEY"),
    base_url="https://api.kilo.ai/api/gateway",
)

# Non-streaming
response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.5",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain quantum entanglement simply."},
    ],
)

print(response.choices[0].message.content)

# Streaming
stream = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.5",
    messages=[
        {"role": "user", "content": "Write a poem about the ocean."},
    ],
    stream=True,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

```

--------------------------------

### Request Body for Tool Calling

Source: https://kilo.ai/docs/gateway/api-reference

Example of a request body that includes tool definitions and specifies tool_choice for enabling function/tool calling capabilities.

```json
{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [{ "role": "user", "content": "What's the weather in San Francisco?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City name"
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

```

--------------------------------

### Raw SSE Stream Format

Source: https://kilo.ai/docs/gateway/streaming

Example of the raw data format returned by the gateway during a streaming session.

```text
data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"anthropic/claude-sonnet-4.5","choices":[{"index":0,"delta":{"role":"assistant","content":"Once"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"anthropic/claude-sonnet-4.5","choices":[{"index":0,"delta":{"content":" upon"},"finish_reason":null}]}

data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1234567890,"model":"anthropic/claude-sonnet-4.5","choices":[{"index":0,"delta":{"content":" a"},"finish_reason":null}]}

data: [DONE]
```

--------------------------------

### OpenAI SDK Integration (TypeScript/JavaScript)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Configure the OpenAI SDK to use the Kilo Gateway by setting the baseURL. Examples show non-streaming and streaming chat completion requests.

```typescript
import OpenAI from "openai"

const client = new OpenAI({
  apiKey: process.env.KILO_API_KEY,
  baseURL: "https://api.kilo.ai/api/gateway",
})

// Non-streaming
const response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-4.5",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Explain quantum entanglement simply." },
  ],
})

console.log(response.choices[0].message.content)

// Streaming
const stream = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-4.5",
  messages: [{ role: "user", content: "Write a poem about the ocean." }],
  stream: true,
})

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content
  if (content) process.stdout.write(content)
}

```

--------------------------------

### Streaming text with tool calling using Vercel AI SDK

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Implement streaming text generation with tool calling capabilities using the Vercel AI SDK. This example defines a getWeather tool for location-based weather information. Ensure your KILO_API_KEY is set.

```typescript
import { streamText, tool } from "ai"
import { createOpenAI } from "@ai-sdk/openai"
import { z } from "zod"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})

const result = streamText({
  model: kilo.chat("anthropic/claude-sonnet-4.5"),
  prompt: "What is the weather in San Francisco?",
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location",
      parameters: z.object({
        location: z.string().describe("City name"),
      }),
      execute: async ({ location }) => {
        return { temperature: 72, condition: "sunny" }
      },
    }),
  },
})

for await (const textPart of result.textStream) {
  process.stdout.write(textPart)
}

```

--------------------------------

### Example cURL Request for Chat Completion

Source: https://kilo.ai/docs/gateway/api-reference

Demonstrates how to make a POST request to the chat completions endpoint using cURL, including setting the API key, content type, and request body.

```shell
curl -X POST "https://api.kilo.ai/api/gateway/chat/completions" \
  -H "Authorization: Bearer $KILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is quantum computing?"}
    ],
    "max_tokens": 500,
    "temperature": 0.7
  }'

```

--------------------------------

### GET https://api.kilo.ai/api/gateway/models

Source: https://kilo.ai/docs/gateway/models-and-providers

Retrieves the full list of available AI models supported by the Kilo AI Gateway.

```APIDOC
## GET https://api.kilo.ai/api/gateway/models

### Description
Retrieves the full list of available AI models supported by the Kilo AI Gateway.

### Method
GET

### Endpoint
https://api.kilo.ai/api/gateway/models
```

--------------------------------

### Executing a request with mode header via cURL

Source: https://kilo.ai/docs/gateway/models-and-providers

Example of using the x-kilocode-mode header to influence routing behavior in the frontier tier.

```bash
curl -X POST "https://api.kilo.ai/api/gateway/chat/completions" \
  -H "Authorization: Bearer $KILO_API_KEY" \
  -H "x-kilocode-mode: plan" \
  -H "Content-Type: application/json" \
  -d '{"model": "kilo-auto/efficient", "messages": [{"role": "user", "content": "Design a database schema"}]}'
```

--------------------------------

### Example Chat Completion Response

Source: https://kilo.ai/docs/gateway/api-reference

Illustrates a typical JSON response for a non-streaming chat completion, showing the generated message, model details, and token usage.

```json
{
  "id": "gen-abc123",
  "object": "chat.completion",
  "created": 1739000000,
  "model": "anthropic/claude-sonnet-4.5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Quantum computing is a type of computation that uses quantum mechanics..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 150,
    "total_tokens": 175
  }
}

```

--------------------------------

### Streaming Chat Completions Request (cURL)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This example demonstrates how to make a streaming POST request to the chat completions endpoint using cURL. The `-N` flag ensures tokens are received as they are generated.

```APIDOC
## POST /api/gateway/chat/completions (Streaming)

### Description
Sends a streaming request to the chat completions endpoint. Tokens are returned as they are generated.

### Method
POST

### Endpoint
https://api.kilo.ai/api/gateway/chat/completions

### Parameters
#### Query Parameters
- **stream** (boolean) - Optional - Set to `true` to enable streaming.

### Request Body
- **model** (string) - Required - The model to use for completions.
- **messages** (array) - Required - An array of message objects, each with a 'role' and 'content'.

### Request Example
```json
{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [
    {"role": "user", "content": "Write a short story about AI."}
  ],
  "stream": true
}
```

### Response
#### Success Response (200)
- **choices** (array) - An array of completion choices. Each chunk may contain partial content.
  - **delta** (object)
    - **content** (string) - The generated content chunk.

### Response Example (Streaming Chunks)
```json
{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "anthropic/claude-sonnet-4.5",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant"
      },
      "finish_reason": null
    }
  ]
}
{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "anthropic/claude-sonnet-4.5",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "Once upon a time,"
      },
      "finish_reason": null
    }
  ]
}
```
```

--------------------------------

### Tool Call Response Example

Source: https://kilo.ai/docs/gateway/api-reference

This JSON structure represents a typical response from a tool call, indicating the assistant's message content and any tool calls it has made, including their ID, type, function name, and arguments.

```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"San Francisco\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}
```

--------------------------------

### Non-streaming Chat Completions Request (cURL)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This example shows how to make a non-streaming POST request to the chat completions endpoint using cURL.

```APIDOC
## POST /api/gateway/chat/completions

### Description
Sends a non-streaming request to the chat completions endpoint.

### Method
POST

### Endpoint
https://api.kilo.ai/api/gateway/chat/completions

### Request Body
- **model** (string) - Required - The model to use for completions.
- **messages** (array) - Required - An array of message objects, each with a 'role' and 'content'.

### Request Example
```json
{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [
    {"role": "user", "content": "What is the capital of France?"}
  ]
}
```

### Response
#### Success Response (200)
- **choices** (array) - An array of completion choices.
  - **message** (object)
    - **content** (string) - The generated content.

### Response Example
```json
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "anthropic/claude-sonnet-4.5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 7,
    "total_tokens": 17
  }
}
```
```

--------------------------------

### Context Length Error Response

Source: https://kilo.ai/docs/gateway/api-reference

An example of an error response when a request exceeds the model's context window. It specifies the model's context limit and the approximate number of tokens in the request.

```json
{
  "error": {
    "message": "This request exceeds the model's context window of 200000 tokens. Your request contains approximately 250000 tokens.",
    "code": 400
  }
}
```

--------------------------------

### Initialize Project

Source: https://kilo.ai/docs/gateway/quickstart

Create a new directory and initialize a Node.js project.

```bash
mkdir my-ai-app
cd my-ai-app
npm init -y
```

--------------------------------

### Initialize Kilo AI SDK with API Key

Source: https://kilo.ai/docs/gateway/authentication

Configure the Kilo AI SDK using the OpenAI-compatible client with your API key.

```typescript
import { createOpenAI } from "@ai-sdk/openai"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})
```

--------------------------------

### Configure Environment Variables

Source: https://kilo.ai/docs/gateway/quickstart

Set the Kilo API key in a .env file.

```text
KILO_API_KEY=your_api_key_here
```

--------------------------------

### Basic streaming text with Vercel AI SDK

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Demonstrates basic text streaming from a specified model using the Vercel AI SDK and Kilo Gateway. Ensure your KILO_API_KEY is set in the environment.

```typescript
import { streamText } from "ai"
import { createOpenAI } from "@ai-sdk/openai"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})

const result = streamText({
  model: kilo.chat("anthropic/claude-sonnet-4.5"),
  prompt: "Write a haiku about programming.",
})

for await (const textPart of result.textStream) {
  process.stdout.write(textPart)
}

```

--------------------------------

### Use OpenAI SDK

Source: https://kilo.ai/docs/gateway/quickstart

Interact with the Kilo AI Gateway using the OpenAI SDK by overriding the base URL.

```javascript
import OpenAI from "openai"

const client = new OpenAI({
  apiKey: process.env.KILO_API_KEY,
  baseURL: "https://api.kilo.ai/api/gateway",
})

const response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-4.5",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
})

console.log(response.choices[0].message.content)
```

--------------------------------

### Run Node.js Script

Source: https://kilo.ai/docs/gateway/quickstart

Execute the created index.mjs script.

```bash
node index.mjs
```

--------------------------------

### Integrate Kilo Gateway with Vercel AI SDK

Source: https://kilo.ai/docs/gateway

Configure the OpenAI provider to point to the Kilo Gateway base URL for model streaming.

```typescript
import { streamText } from "ai"
import { createOpenAI } from "@ai-sdk/openai"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})

const result = streamText({
  model: kilo.chat("anthropic/claude-sonnet-4.5"),
  prompt: "Why is the sky blue?",
})
```

--------------------------------

### Stream with OpenAI SDK

Source: https://kilo.ai/docs/gateway/streaming

Use the standard OpenAI SDK configured with the Kilo Gateway base URL to stream completions.

```typescript
import OpenAI from "openai"

const client = new OpenAI({
  apiKey: process.env.KILO_API_KEY,
  baseURL: "https://api.kilo.ai/api/gateway",
})

const stream = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-4.5",
  messages: [{ role: "user", content: "Write a short story" }],
  stream: true,
})

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content
  if (content) {
    process.stdout.write(content)
  }
}
```

--------------------------------

### Stream Text with Vercel AI SDK

Source: https://kilo.ai/docs/gateway/quickstart

Create and run a script to stream model responses using the Vercel AI SDK.

```javascript
import { streamText } from "ai"
import { createOpenAI } from "@ai-sdk/openai"
import "dotenv/config"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})

async function main() {
  const result = streamText({
    model: kilo.chat("anthropic/claude-sonnet-4.5"),
    prompt: "Invent a new holiday and describe its traditions.",
  })

  for await (const textPart of result.textStream) {
    process.stdout.write(textPart)
  }

  console.log()
  console.log("Token usage:", await result.usage)
  console.log("Finish reason:", await result.finishReason)
}

main().catch(console.error)
```

--------------------------------

### Make Request with cURL

Source: https://kilo.ai/docs/gateway/quickstart

Perform a direct POST request to the Kilo AI Gateway chat completions endpoint.

```bash
curl -X POST "https://api.kilo.ai/api/gateway/chat/completions" \
  -H "Authorization: Bearer $KILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [
      {
        "role": "user",
        "content": "Why is the sky blue?"
      }
    ],
    "stream": false
  }'
```

--------------------------------

### Chat Completions Request (Go)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This Go code snippet demonstrates how to make a POST request to the chat completions endpoint.

```APIDOC
## POST /api/gateway/chat/completions (Go)

### Description
This Go program sends a POST request to the chat completions endpoint to get a response.

### Method
POST

### Endpoint
https://api.kilo.ai/api/gateway/chat/completions

### Request Body
- **model** (string) - Required - The model to use for completions.
- **messages** (array) - Required - An array of message objects, each with a 'role' and 'content'.

### Request Example
```go
package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
)

func main() {
    body := map[string]interface{}{
        "model": "anthropic/claude-sonnet-4.5",
        "messages": []map[string]string{
            {"role": "user", "content": "Why is the sky blue?"},
        },
    }

    jsonBody, _ := json.Marshal(body)

    req, _ := http.NewRequest("POST",
        "https://api.kilo.ai/api/gateway/chat/completions",
        bytes.NewBuffer(jsonBody))

    req.Header.Set("Authorization", "Bearer "+os.Getenv("KILO_API_KEY"))
    req.Header.Set("Content-Type", "application/json")

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    respBody, _ := io.ReadAll(resp.Body)
    fmt.Println(string(respBody))
}
```
```

--------------------------------

### Send a chat completion request with Go

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This Go program sends a chat completion request to the Kilo AI Gateway. It marshals a JSON body, sets necessary headers including the API key from an environment variable, and prints the response.

```go
package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
)

func main() {
    body := map[string]interface{}{
        "model": "anthropic/claude-sonnet-4.5",
        "messages": []map[string]string{
            {"role": "user", "content": "Why is the sky blue?"},
        },
    }

    jsonBody, _ := json.Marshal(body)

    req, _ := http.NewRequest("POST",
        "https://api.kilo.ai/api/gateway/chat/completions",
        bytes.NewBuffer(jsonBody))

    req.Header.Set("Authorization", "Bearer "+os.Getenv("KILO_API_KEY"))
    req.Header.Set("Content-Type", "application/json")

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    respBody, _ := io.ReadAll(resp.Body)
    fmt.Println(string(respBody))
}
```

--------------------------------

### Next.js API route for streaming text with Vercel AI SDK

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Set up a Next.js API route to handle streaming text requests using the Vercel AI SDK and Kilo Gateway. This route accepts JSON messages and returns a data stream response. Ensure your KILO_API_KEY is set.

```typescript
import { streamText } from "ai"
import { createOpenAI } from "@ai-sdk/openai"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})

export async function POST(request: Request) {
  const { messages } = await request.json()

  const result = streamText({
    model: kilo.chat("anthropic/claude-sonnet-4.5"),
    messages,
  })

  return result.toDataStreamResponse()
}

```

--------------------------------

### Chat Completions Request (Ruby)

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This Ruby script shows how to send a POST request to the chat completions endpoint.

```APIDOC
## POST /api/gateway/chat/completions (Ruby)

### Description
This Ruby script sends a POST request to the chat completions endpoint and parses the JSON response.

### Method
POST

### Endpoint
https://api.kilo.ai/api/gateway/chat/completions

### Request Body
- **model** (string) - Required - The model to use for completions.
- **messages** (array) - Required - An array of message objects, each with a 'role' and 'content'.

### Request Example
```ruby
require 'net/http'
require 'json'

uri = URI('https://api.kilo.ai/api/gateway/chat/completions')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

request = Net::HTTP::Post.new(uri)
request['Authorization'] = "Bearer #{ENV['KILO_API_KEY']}"
request['Content-Type'] = 'application/json'
request.body = {
  model: 'anthropic/claude-sonnet-4.5',
  messages: [
    { role: 'user', content: 'Why is the sky blue?' }
  ]
}.to_json

response = http.request(request)
result = JSON.parse(response.body)
puts result['choices'][0]['message']['content']
```
```

--------------------------------

### Retrieve available models

Source: https://kilo.ai/docs/gateway/models-and-providers

Fetch the complete list of supported models using the gateway models endpoint.

```http
GET https://api.kilo.ai/api/gateway/models
```

--------------------------------

### POST /chat/completions

Source: https://kilo.ai/docs/gateway/api-reference

This is the primary endpoint for interacting with AI models to create chat completions. It supports various parameters for controlling generation, streaming, and tool usage.

```APIDOC
## POST /chat/completions

### Description
Create a chat completion. This is the primary endpoint for interacting with AI models.

### Method
POST

### Endpoint
https://api.kilo.ai/api/gateway/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - Model ID (e.g., "anthropic/claude-sonnet-4.5")
- **messages** (Message[]) - Required - Array of conversation messages
- **stream** (boolean) - Optional - Enable SSE streaming (default: false)
- **max_tokens** (number) - Optional - Maximum tokens to generate
- **temperature** (number) - Optional - Sampling temperature (0-2)
- **top_p** (number) - Optional - Nucleus sampling (0-1)
- **stop** (string | string[]) - Optional - Stop sequences
- **frequency_penalty** (number) - Optional - Frequency penalty (-2 to 2)
- **presence_penalty** (number) - Optional - Presence penalty (-2 to 2)
- **tools** (Tool[]) - Optional - Available tools/functions
- **tool_choice** (ToolChoice) - Optional - Tool selection strategy
- **response_format** (ResponseFormat) - Optional - Structured output
- **user** (string) - Optional - End-user identifier for safety
- **seed** (number) - Optional - Deterministic sampling seed

### Request Example
```json
{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is quantum computing?"}
  ],
  "max_tokens": 500,
  "temperature": 0.7
}
```

### Response (non-streaming)
#### Success Response (200)
- **id** (string) - Unique identifier for the completion
- **object** (string) - Type of object, "chat.completion"
- **created** (number) - Timestamp of creation
- **model** (string) - The model used for completion
- **choices** (Array) - Array of completion choices
  - **index** (number) - Index of the choice
  - **message** (object) - The generated message
    - **role** (string) - Role of the message sender ("assistant")
    - **content** (string | null) - The content of the message
    - **tool_calls** (ToolCall[]) - Optional - Tool calls made by the assistant
  - **finish_reason** (string) - Reason for finishing generation ("stop", "length", "tool_calls", "content_filter")
- **usage** (object) - Token usage statistics
  - **prompt_tokens** (number) - Tokens in the prompt
  - **completion_tokens** (number) - Tokens in the completion
  - **total_tokens** (number) - Total tokens used

#### Response Example
```json
{
  "id": "gen-abc123",
  "object": "chat.completion",
  "created": 1739000000,
  "model": "anthropic/claude-sonnet-4.5",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Quantum computing is a type of computation that uses quantum mechanics..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 25,
    "completion_tokens": 150,
    "total_tokens": 175
  }
}
```

### Response (streaming)
When `stream: true`, the response is a series of SSE events:
```json
type ChatCompletionChunk = {
  id: string
  object: "chat.completion.chunk"
  created: number
  model: string
  choices: Array<{
    index: number
    delta: {
      role?: "assistant"
      content?: string
      tool_calls?: ToolCall[]
    }
    finish_reason: string | null
  }>
  // Only in the final chunk
  usage?: {
    prompt_tokens: number
    completion_tokens: number
    total_tokens: number
  }
}
```
```

--------------------------------

### Tool Calling

Source: https://kilo.ai/docs/gateway/api-reference

The Kilo AI Gateway supports function/tool calling, enabling models to interact with external tools. This includes automatic repair for common issues.

```APIDOC
## Tool Calling

### Description
The gateway supports function/tool calling with automatic repair for common issues like duplicate tool calls and orphan cleanup.

### Request with tools
```json
{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [{ "role": "user", "content": "What's the weather in San Francisco?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City name"
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}
```
```

--------------------------------

### LangChain Integration with Kilo AI Gateway

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Use `ChatOpenAI` with Kilo's base URL for LangChain integrations. Ensure your Kilo API key is set as an environment variable.

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-4.5",
    api_key=os.getenv("KILO_API_KEY"),
    base_url="https://api.kilo.ai/api/gateway",
)

response = llm.invoke("Explain photosynthesis in simple terms.")
print(response.content)

```

--------------------------------

### Stream with Vercel AI SDK

Source: https://kilo.ai/docs/gateway/streaming

Use the Vercel AI SDK to handle SSE parsing and stream text generation.

```typescript
import { streamText } from "ai"
import { createOpenAI } from "@ai-sdk/openai"

const kilo = createOpenAI({
  baseURL: "https://api.kilo.ai/api/gateway",
  apiKey: process.env.KILO_API_KEY,
})

const result = streamText({
  model: kilo.chat("anthropic/claude-sonnet-4.5"),
  prompt: "Write a short story about a robot.",
})

for await (const textPart of result.textStream) {
  process.stdout.write(textPart)
}

// Access usage data after streaming completes
const usage = await result.usage
console.log("Tokens used:", usage)
```

--------------------------------

### LangChain.js Integration with Kilo AI Gateway

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Use `ChatOpenAI` with Kilo's base URL for LangChain.js integrations. Ensure your Kilo API key is set as an environment variable.

```javascript
import { ChatOpenAI } from "@langchain/openai"

const model = new ChatOpenAI({
  modelName: "anthropic/claude-sonnet-4.5",
  openAIApiKey: process.env.KILO_API_KEY,
  configuration: {
    baseURL: "https://api.kilo.ai/api/gateway",
  },
})

const response = await model.invoke("Explain photosynthesis in simple terms.")
console.log(response.content)

```

--------------------------------

### List Models API

Source: https://kilo.ai/docs/gateway/api-reference

Retrieve a list of all available models that can be used with the API.

```APIDOC
## GET /models

### Description
Retrieve the list of available models.

### Method
GET

### Endpoint
/models

### Parameters
No parameters required.

### Response
#### Success Response (200)
Returns an OpenAI-compatible model list.

### Response Example
```json
{
  "data": [
    {
      "id": "anthropic/claude-sonnet-4.5",
      "object": "model",
      "created": 1739000000,
      "owned_by": "anthropic",
      "name": "Claude Sonnet 4.5",
      "context_length": 200000,
      "pricing": {
        "prompt": "0.000003",
        "completion": "0.000015"
      }
    }
  ]
}
```
```

--------------------------------

### Send a chat completion request with Ruby

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This Ruby script sends a chat completion request to the Kilo AI Gateway using the Net::HTTP library. It constructs the JSON payload, sets authorization and content-type headers, and parses the response to display the message content. Ensure your API key is set as an environment variable.

```ruby
require 'net/http'
require 'json'

uri = URI('https://api.kilo.ai/api/gateway/chat/completions')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

request = Net::HTTP::Post.new(uri)
request['Authorization'] = "Bearer #{ENV['KILO_API_KEY']}"
request['Content-Type'] = 'application/json'
request.body = {
  model: 'anthropic/claude-sonnet-4.5',
  messages: [
    { role: 'user', content: 'Why is the sky blue?' }
  ]
}.to_json

response = http.request(request)
result = JSON.parse(response.body)
puts result['choices'][0]['message']['content']
```

--------------------------------

### Authentication Headers

Source: https://kilo.ai/docs/gateway/authentication

Standard headers required for authenticating requests to the Kilo AI Gateway.

```APIDOC
## Authentication Headers

### Description
Requests to the Kilo AI Gateway require specific headers for authentication and context. 

### Headers
- **Authorization** (string) - Required (unless using free models) - Format: `Bearer <your_api_key>`
- **Content-Type** (string) - Required - Must be `application/json`
- **X-KiloCode-OrganizationId** (string) - Optional - Used for requests on behalf of an organization
- **X-KiloCode-TaskId** (string) - Optional - Task identifier for prompt cache keying
- **X-KiloCode-Version** (string) - Optional - Client version string
- **x-kilocode-mode** (string) - Optional - Mode hint for `kilo-auto` model routing
```

--------------------------------

### Send a streaming chat completion request with cURL

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

This snippet demonstrates how to send a streaming request for chat completions. The `-N` flag disables buffering, allowing you to receive tokens as they are generated. Ensure your API key is set as an environment variable.

```bash
curl -N -X POST "https://api.kilo.ai/api/gateway/chat/completions" \
  -H "Authorization: Bearer $KILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [
      {"role": "user", "content": "Write a short story about AI."}
    ],
    "stream": true
  }'
```

--------------------------------

### Enable Streaming in Request Body

Source: https://kilo.ai/docs/gateway/streaming

Set the stream parameter to true in the JSON request body to initiate a streaming response.

```json
{
  "model": "anthropic/claude-sonnet-4.5",
  "messages": [{ "role": "user", "content": "Write a short story" }],
  "stream": true
}
```

--------------------------------

### Send a non-streaming chat completion request with cURL

Source: https://kilo.ai/docs/gateway/sdks-and-frameworks

Use this snippet to send a standard, non-streaming request for chat completions to the Kilo AI Gateway. Ensure your API key is set as an environment variable.

```bash
curl -X POST "https://api.kilo.ai/api/gateway/chat/completions" \
  -H "Authorization: Bearer $KILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'
```

--------------------------------

### POST /api/gateway/chat/completions

Source: https://kilo.ai/docs/gateway/models-and-providers

Sends a chat completion request to the gateway, supporting specific models or auto-routing tiers.

```APIDOC
## POST /api/gateway/chat/completions

### Description
Sends a chat completion request to the gateway. You can specify a model ID or use a 'kilo-auto/*' tier for dynamic routing.

### Method
POST

### Endpoint
https://api.kilo.ai/api/gateway/chat/completions

### Parameters
#### Headers
- **Authorization** (string) - Required - Bearer token for authentication.
- **x-kilocode-mode** (string) - Optional - Specifies the mode for 'kilo-auto/frontier' routing (e.g., plan, general, architect, orchestrator, ask, debug, build, explore, code).
- **Content-Type** (string) - Required - Must be 'application/json'.

#### Request Body
- **model** (string) - Required - The model ID or auto-tier (e.g., 'kilo-auto/frontier', 'anthropic/claude-opus-4.7').
- **messages** (array) - Required - List of message objects with 'role' and 'content'.

### Request Example
{
  "model": "kilo-auto/frontier",
  "messages": [{ "role": "user", "content": "Help me design a database schema" }]
}
```

--------------------------------

### Pre-streaming Error Response

Source: https://kilo.ai/docs/gateway/streaming

Standard JSON error format returned if an error occurs before streaming begins.

```json
{
  "error": {
    "message": "Insufficient balance",
    "code": 402
  }
}
```

--------------------------------

### Insufficient balance error response

Source: https://kilo.ai/docs/gateway/usage-and-billing

The JSON response returned with HTTP 402 status when an account has insufficient credits to perform a request.

```json
{
  "error": {
    "message": "Insufficient balance. Please add credits to continue.",
    "code": 402,
    "metadata": {
      "buyCreditsUrl": "https://app.kilo.ai/credits"
    }
  }
}
```

--------------------------------

### List Providers API

Source: https://kilo.ai/docs/gateway/api-reference

Retrieve a list of all available providers that can be used with the API.

```APIDOC
## GET /providers

### Description
Retrieve the list of available providers.

### Method
GET

### Endpoint
/providers

### Parameters
No parameters required.

### Response
#### Success Response (200)
Returns a list of available providers.
```

--------------------------------

### Specify a model in streamText

Source: https://kilo.ai/docs/gateway/models-and-providers

Use the kilo.chat helper to define the model within a streamText request.

```javascript
const result = streamText({
  model: kilo.chat("anthropic/claude-sonnet-4.6"),
  prompt: "Hello!",
})
```

--------------------------------

### Error Handling

Source: https://kilo.ai/docs/gateway/api-reference

Information about common error codes and response formats.

```APIDOC
## Error Codes

### Description
Details on HTTP status codes and their corresponding error descriptions.

### Error Response Format
```json
{
  "error": {
    "message": "Human-readable error description",
    "code": 400
  }
}
```

### Error Codes Table
HTTP Status| Description  
---|---
400| Bad request -- invalid parameters or model ID  
401| Unauthorized -- invalid or missing API key  
402| Insufficient balance -- add credits to continue  
403| Forbidden -- model not allowed by organization policy  
429| Rate limited -- too many requests  
500| Internal server error  
502| Provider error -- upstream provider returned an error  
503| Service unavailable -- provider temporarily unavailable  

### Context Length Errors
If your request exceeds the model's context window, you'll receive a descriptive error:
```json
{
  "error": {
    "message": "This request exceeds the model's context window of 200000 tokens. Your request contains approximately 250000 tokens.",
    "code": 400
  }
}
```

### Info
When the gateway receives a 402 (Payment Required) from an upstream provider, it returns 503 to the client to avoid exposing internal billing details.
```

--------------------------------

### FIM Completions API

Source: https://kilo.ai/docs/gateway/api-reference

This endpoint allows you to generate code completions using the Fill-in-the-Middle (FIM) approach, powered by Mistral Codestral models.

```APIDOC
## POST /api/fim/completions

### Description
Generate fill-in-the-middle completions for code generation.

### Method
POST

### Endpoint
/api/fim/completions

### Parameters
#### Request Body
- **model** (string) - Required - Must be a Mistral model (e.g., "mistralai/codestral-2508")
- **prompt** (string) - Required - Code before the cursor
- **suffix** (string) - Optional - Code after the cursor
- **max_tokens** (number) - Optional - Maximum tokens (capped at 1000)
- **temperature** (number) - Optional - Sampling temperature
- **stop** (string[]) - Optional - Stop sequences
- **stream** (boolean) - Optional - Whether to stream the response

### Request Example
```json
{
  "model": "mistralai/codestral-2508",
  "prompt": "def fibonacci(n):\n    if n <= 1:\n        return n\n    ",
  "suffix": "\n\nprint(fibonacci(10))",
  "max_tokens": 200,
  "stream": false
}
```

### Info
FIM completions are limited to Mistral models (model IDs starting with `mistralai/`). BYOK is supported with the `codestral` key type.
```

--------------------------------

### Chat Completion Response Schema (Streaming)

Source: https://kilo.ai/docs/gateway/api-reference

Defines the structure of a streaming chat completion response using Server-Sent Events (SSE). Includes delta updates for content and tool calls, and final usage information.

```typescript
type ChatCompletionChunk = {
  id: string
  object: "chat.completion.chunk"
  created: number
  model: string
  choices: Array<{
    index: number
    delta: {
      role?: "assistant"
      content?: string
      tool_calls?: ToolCall[]
    }
    finish_reason: string | null
  }>
  // Only in the final chunk
  usage?: {
    prompt_tokens: number
    completion_tokens: number
    total_tokens: number
  }
}

```

--------------------------------

### Requesting a model via JSON payload

Source: https://kilo.ai/docs/gateway/models-and-providers

Basic structure for specifying a model in a chat completion request.

```json
{
  "model": "kilo-auto/frontier",
  "messages": [{ "role": "user", "content": "Help me design a database schema" }]
}
```

--------------------------------

### Mid-stream Error Event

Source: https://kilo.ai/docs/gateway/streaming

Errors occurring during an active stream are sent as an SSE event with a finish_reason of error.

```text
data: {"error":{"message":"Provider disconnected","code":502},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}
```

--------------------------------

### Chat Completion Request Body Schema

Source: https://kilo.ai/docs/gateway/api-reference

Defines the structure for chat completion requests, including model selection, messages, streaming options, generation parameters, and tool calling configurations.

```typescript
type ChatCompletionRequest = {
  // Required
  model: string // Model ID (e.g., "anthropic/claude-sonnet-4.5")
  messages: Message[] // Array of conversation messages

  // Streaming
  stream?: boolean // Enable SSE streaming (default: false)

  // Generation parameters
  max_tokens?: number // Maximum tokens to generate
  temperature?: number // Sampling temperature (0-2)
  top_p?: number // Nucleus sampling (0-1)
  stop?: string | string[] // Stop sequences
  frequency_penalty?: number // Frequency penalty (-2 to 2)
  presence_penalty?: number // Presence penalty (-2 to 2)

  // Tool calling
  tools?: Tool[] // Available tools/functions
  tool_choice?: ToolChoice // Tool selection strategy

  // Structured output
  response_format?: ResponseFormat

  // Other
  user?: string // End-user identifier for safety
  seed?: number // Deterministic sampling seed
}

```

--------------------------------

### Specify a model in raw API request

Source: https://kilo.ai/docs/gateway/models-and-providers

Pass the model identifier directly in the JSON body of a raw API request.

```json
{
  "model": "anthropic/claude-sonnet-4.6",
  "messages": [{ "role": "user", "content": "Hello!" }]
}
```
Tokens:
Raw