LLMBase

https://llmbase.ai/docs/
Admin
LLMBase is an OpenAI-compatible inference API for EU-hosted open models, letting developers switch...
Tokens:33,212
Snippets:249
Trust Score:7.1
Update:1 day ago
Show doc for...
Tokens:
Raw
### Full chat completion example using OpenAI SDK

Source: https://llmbase.ai/docs/inference/chat-completions/

JavaScript example using the OpenAI SDK to create a chat completion. It sets the base URL to https://api.llmbase.ai/v1 and uses the LLMBASE_API_KEY environment variable. The request includes system and user messages, temperature 0.7, and max_tokens 100, then logs the assistant's content.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Explain recursion in one sentence." },
  ],
  temperature: 0.7,
  max_tokens: 100,
});

console.log(response.choices[0].message.content);
```

--------------------------------

### GET /v1/models?metadata=true

Source: https://llmbase.ai/docs/inference/structured-outputs/

Check the live list of models and their supported features to determine if structured output is available.

```APIDOC
## [GET] /v1/models?metadata=true

### Description
Check the live list of models and their supported features to determine if structured output is available.

### Method
GET

### Endpoint
/v1/models?metadata=true

### Parameters
#### Query Parameters
- **metadata** (boolean) - Required - Set to `true` to include metadata about supported features.

### Request Example
```bash
curl "https://api.llmbase.ai/v1/models?metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **data** (array) - List of model objects.
- **object** (string) - The object type, e.g., `list`.

#### Response Example
```json
{
  "object": "list",
  "data": [
    {
      "id": "model-id",
      "object": "model",
      "created": 1677652288,
      "owned_by": "llmbase",
      "supported_features": ["json_mode", "structured_outputs"]
    }
  ]
}
```
```

--------------------------------

### Non-streaming response example

Source: https://llmbase.ai/docs/inference/chat-completions/

Example JSON response from a non-streaming chat completion request. Shows the structure including id, object, created, model, choices, and usage details. The finish_reason is 'stop' and usage includes prompt_tokens_details with cached_tokens.

```json
{
  "id": "chatcmpl-a1b2c3d4e5f6a1b2c3d4e5f6",
  "object": "chat.completion",
  "created": 1741000000,
  "model": "<model-id-from-/v1/models>",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I can help you with a wide range of tasks..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 48,
    "total_tokens": 60,
    "prompt_tokens_details": {
      "cached_tokens": 8
    }
  }
}
```

--------------------------------

### OpenAI SDK client setup

Source: https://llmbase.ai/docs/authentication/

Constructs an OpenAI client with the LLMBase base URL and API key. The SDK automatically adds the Authorization: Bearer header to every request.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY, // never hard-code keys
});
```

--------------------------------

### GET /v1/models

Source: https://llmbase.ai/docs/openai-compatibility/

Lists available models for OpenAI SDK model discovery. Supports optional metadata and filtering parameters.

```APIDOC
## GET /v1/models

### Description
Lists available models for OpenAI SDK model discovery. Supports optional metadata and filtering parameters.

### Method
GET

### Endpoint
/v1/models

### Parameters
#### Query Parameters
- **metadata** (boolean) - Optional - When true, returns detailed model metadata including pricing, modalities, and capabilities.
- **filter** (string) - Optional - Filters models by type. Use `agents` to get agent-compatible models.

### Response
#### Success Response (200)
- **data** (array) - List of model objects.
- **data[].id** (string) - Model identifier.
- **data[].supported_features** (array) - List of supported features (e.g., `tools`, `response_format`, `reasoning`, `logprobs`).
- **data[].supported_parameters** (array) - List of supported parameters.
- **data[].pricing** (object) - Pricing information (when metadata=true).
- **data[].modalities** (array) - Supported input/output modalities (when metadata=true).
- **data[].capabilities** (object) - Model capabilities (when metadata=true).

#### Response Example
{
  "data": [
    {
      "id": "model-id",
      "supported_features": ["tools", "response_format", "reasoning"],
      "supported_parameters": ["temperature", "top_p"],
      "pricing": {},
      "modalities": ["text"],
      "capabilities": {}
    }
  ]
}
```

--------------------------------

### OpenAI SDK chat agent example

Source: https://llmbase.ai/docs/agents/setup/

Use the OpenAI SDK with a custom base URL and the LLMBASE_CHAT_AGENT_KEY environment variable. Lists models, picks the first model ID, and creates a chat completion. Requires the OpenAI SDK and the environment variable set.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://llmbase.ai/api/v1/agents",
  apiKey: process.env.LLMBASE_CHAT_AGENT_KEY,
});

const models = await client.models.list();
const model = models.data[0]?.id;

const response = await client.chat.completions.create({
  model,
  messages: [
    { role: "user", content: "Draft a short product update." },
  ],
});

console.log(response.choices[0]?.message?.content);
```

--------------------------------

### GET /v1/balance

Source: https://llmbase.ai/docs/authentication/

Check the live API balance for an inference API key. Returns the balance in USD cents.

```APIDOC
## GET /v1/balance

### Description
Check the live API balance for an inference API key. Returns the balance in USD cents.

### Method
GET

### Endpoint
https://api.llmbase.ai/v1/balance

### Parameters
#### Headers
- **Authorization** (string) - Required - Bearer token with a valid `llmbase_...` API key.

### Request Example
```
curl https://api.llmbase.ai/v1/balance \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **balance_usd_cents** (integer) - The current prepaid balance in USD cents.

#### Response Example
```
{
  "balance_usd_cents": 500
}
```

```

--------------------------------

### GET /v1/model-metadata

Source: https://llmbase.ai/docs/inference/model-discovery/

Compatibility alias for the same rich metadata format as /v1/models?metadata=true. New clients should prefer /v1/models?metadata=true.

```APIDOC
## GET /v1/model-metadata

### Description
Compatibility alias for the same rich metadata format as /v1/models?metadata=true. New clients should prefer /v1/models?metadata=true.

### Method
GET

### Endpoint
/v1/model-metadata

### Parameters
None

### Response
#### Success Response (200)
Same response structure as GET /v1/models?metadata=true.

#### Response Example
```json
{
  "data": [
    {
      "id": "model-id",
      "context_length": 128000,
      "max_output_length": 4096,
      "input_modalities": ["text"],
      "output_modalities": ["text"],
      "supported_input_image_media_types": [],
      "pricing": {
        "prompt": 0.01,
        "completion": 0.02,
        "input_cache_read": 0.005
      },
      "supported_parameters": ["temperature", "top_p", "max_tokens"],
      "supported_features": ["tools", "json_mode"],
      "supported_reasoning_efforts": ["high", "max"],
      "lifecycle": {
        "removal_date": "2025-12-31",
        "replacement_ids": ["new-model-id"]
      }
    }
  ]
}
```
```

--------------------------------

### 401 error response

Source: https://llmbase.ai/docs/authentication/

Example error response returned when a missing or invalid key is used.

```json
{
  "error": {
    "message": "Invalid API key",
    "type": "invalid_request_error",
    "code": "invalid_api_key"
  }
}
```

--------------------------------

### Using the OpenAI SDK with a dedicated endpoint

Source: https://llmbase.ai/docs/dedicated-inference/

Example of using the OpenAI SDK with a dedicated endpoint. Set the baseURL to the endpoint's /v1 path and provide your LLMBase API key. The model ID should be obtained from /v1/models.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<endpoint-id>-inference.llmbase.cloud/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Hello!" }],
});
```

--------------------------------

### GET /api/v1/agents/models

Source: https://llmbase.ai/docs/inference/model-discovery/

Returns the current chat/tool-capable models available for subscription-backed agents. Use this endpoint at runtime instead of maintaining a static allowlist.

```APIDOC
## GET /api/v1/agents/models

### Description
Returns the current chat/tool-capable models available for subscription-backed agents. Retrieve it at runtime instead of maintaining a static OpenClaw or Hermes allowlist.

### Method
GET

### Endpoint
https://llmbase.ai/api/v1/agents/models

### Authentication
Bearer token (use a `llmbase_chat_...` key)

### Request Example
```
curl https://llmbase.ai/api/v1/agents/models \
  -H "Authorization: Bearer $LLMBASE_CHAT_AGENT_KEY"
```

### Response
Returns the current list of chat/tool-capable models. Models not returned by this endpoint cannot be used with `llmbase_chat_...` keys.

### Notes
- Use a `llmbase_...` key and `https://api.llmbase.ai/v1` when you need direct inference billing or a model outside the agent-compatible list.
```

--------------------------------

### Chat messages array

Source: https://llmbase.ai/docs/inference/chat-completions/

Example of a conversation history array with system, user, and assistant messages. At least one message is required.

```json
[
  { "role": "system",    "content": "You are a helpful assistant." },
  { "role": "user",      "content": "Summarise this article: ..." },
  { "role": "assistant", "content": "Here is a summary: ..." },
  { "role": "user",      "content": "Make it shorter." }
]
```

--------------------------------

### List agent-compatible models with curl

Source: https://llmbase.ai/docs/inference/model-discovery/

Sends a GET request to the agent API endpoint `https://llmbase.ai/api/v1/agents/models` with a Bearer token. This endpoint returns chat/tool-capable models for subscription-backed agents. Models not returned here cannot be used with `llmbase_chat_...` keys.

```bash
curl https://llmbase.ai/api/v1/agents/models \
  -H "Authorization: Bearer $LLMBASE_CHAT_AGENT_KEY"
```

--------------------------------

### GET /v1/models

Source: https://llmbase.ai/docs/inference/model-discovery/

Returns all available models in the OpenAI models format. Send the same Bearer header used for other API requests.

```APIDOC
## GET /v1/models

### Description
Returns all available models in the OpenAI models format.

### Method
GET

### Endpoint
https://api.llmbase.ai/v1/models

### Authentication
Bearer token (same as other API requests)

### Request Example
```
curl https://api.llmbase.ai/v1/models \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **object** (string) - Always "list"
- **data** (array) - List of model objects
  - **id** (string) - Model identifier in format "provider/model-id"
  - **object** (string) - Always "model"
  - **created** (integer) - Creation timestamp
  - **owned_by** (string) - Provider name
  - **name** (string) - Display name
  - **description** (string) - Model description
  - **context_length** (integer) - Context window size

#### Response Example
```
{
  "object": "list",
  "data": [
    {
      "id": "provider/model-id",
      "object": "model",
      "created": 1700000000,
      "owned_by": "provider",
      "name": "Provider: Model",
      "description": "Model description.",
      "context_length": 1048576
    }
  ]
}
```

### OpenAI SDK Usage
```
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const models = await client.models.list();
for (const model of models.data) {
  console.log(model.id);
}
```
```

--------------------------------

### GET /models

Source: https://llmbase.ai/docs/agents/

Retrieves the list of models available to a chat agent key. The model list may differ from the direct inference catalog.

```APIDOC
## GET /models

### Description
Retrieves the list of models available to a chat agent key. The model list may differ from the direct inference catalog.

### Method
GET

### Endpoint
https://llmbase.ai/api/v1/agents/models

### Authentication
Authorization: Bearer <LLMBASE_CHAT_AGENT_KEY>

### Response
#### Success Response (200)
Returns a list of available models.

#### Error Response (403)
- **error** (string) - `chat_pro_required` if the account is not on Pro or does not have an active Business seat.
```

--------------------------------

### List models with curl

Source: https://llmbase.ai/docs/inference/model-discovery/

Sends a GET request to the direct inference endpoint `https://api.llmbase.ai/v1/models` with a Bearer token. The response is a list of model objects in the OpenAI models format.

```bash
curl https://api.llmbase.ai/v1/models \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

```json
{
  "object": "list",
  "data": [
    {
      "id": "provider/model-id",
      "object": "model",
      "created": 1700000000,
      "owned_by": "provider",
      "name": "Provider: Model",
      "description": "Model description.",
      "context_length": 1048576
    }
  ]
}
```

--------------------------------

### GET /v1/balance

Source: https://llmbase.ai/docs/openai-compatibility/

Retrieves the LLMBase prepaid inference balance.

```APIDOC
## GET /v1/balance

### Description
Retrieves the LLMBase prepaid inference balance.

### Method
GET

### Endpoint
/v1/balance

### Response
#### Success Response (200)
- **balance** (number) - The prepaid inference balance.

#### Response Example
{
  "balance": 100.00
}
```

--------------------------------

### Python: Set up OpenAI client with LLMBase base URL

Source: https://llmbase.ai/docs/openai-compatibility/

Use this Python snippet to configure the OpenAI SDK to point to LLMBase's API endpoint. Set the base_url to https://api.llmbase.ai/v1 and use an llmbase_... inference key. The model ID should be obtained from /v1/models.

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmbase.ai/v1",
    api_key=os.environ["LLMBASE_API_KEY"],
)

response = client.chat.completions.create(
    model="<model-id-from-/v1/models>",
    messages=[{"role": "user", "content": "Write a short deployment checklist."}],
)

print(response.choices[0].message.content)
```

--------------------------------

### List models with OpenAI SDK

Source: https://llmbase.ai/docs/inference/model-discovery/

Uses the OpenAI SDK with a custom base URL to list models. Requires the `LLMBASE_API_KEY` environment variable.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const models = await client.models.list();
for (const model of models.data) {
  console.log(model.id);
}
```

--------------------------------

### GET /v1/models?metadata=true

Source: https://llmbase.ai/docs/inference/rate-limits/

Retrieve model metadata including context_length and max_output_length fields to size requests appropriately.

```APIDOC
## GET /v1/models?metadata=true

### Description
Retrieves model metadata including context and output limits. Use the `context_length` and `max_output_length` fields to size requests and avoid exceeding model limits.

### Method
GET

### Endpoint
/v1/models?metadata=true

### Query Parameters
- **metadata** (boolean) - Required - Set to `true` to include model metadata fields.

### Response
#### Success Response (200)
- **context_length** (integer) - The maximum context length for the model.
- **max_output_length** (integer) - The maximum output length for the model.

#### Response Example
{
  "data": [
    {
      "id": "model-id",
      "context_length": 8192,
      "max_output_length": 2048
    }
  ]
}
```

--------------------------------

### JavaScript: Set up OpenAI client with LLMBase base URL

Source: https://llmbase.ai/docs/openai-compatibility/

Use this JavaScript snippet to configure the OpenAI SDK to point to LLMBase's API endpoint. Set the baseURL to https://api.llmbase.ai/v1 and use an llmbase_... inference key. The model ID should be obtained from /v1/models.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Write a short deployment checklist." }],
});

console.log(response.choices[0]?.message.content);
```

--------------------------------

### Multimodal user message

Source: https://llmbase.ai/docs/inference/chat-completions/

Example of a user message with multimodal content as an array of parts, including text and an image URL.

```json
{
  "role": "user",
  "content": [
    { "type": "text", "text": "What is in this image?" },
    { "type": "image_url", "image_url": { "url": "https://example.com/image.jpg" } }
  ]
}
```

--------------------------------

### Basic request with prompt_cache_key

Source: https://llmbase.ai/docs/inference/prompt-caching/

Use this JSON body to send a request with a stable prompt_cache_key. Keep the cacheable prefix stable and put changing instructions later in the message list. Short prompts may not be cached.

```json
{
  "model": "<model-id-from-/v1/models>",
  "prompt_cache_key": "workspace-123-agent",
  "messages": [
    { "role": "system", "content": "You are a coding agent for this repository..." },
    { "role": "user", "content": "Continue from the last task." }
  ]
}
```

--------------------------------

### OpenAI SDK: Stream chat completions in JavaScript

Source: https://llmbase.ai/docs/inference/streaming/

Uses the OpenAI SDK with a custom `baseURL` to stream chat completions. Requires the `LLMBASE_API_KEY` environment variable. The `for await` loop writes each delta content chunk to stdout.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Write a short poem." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

--------------------------------

### Function tool calling with OpenAI SDK

Source: https://llmbase.ai/docs/inference/tools/

Shows the full workflow: define a weather tool, send a chat completion with `tools`, handle the tool call, and send the result back as a `tool` message. Requires a model with `supported_features: ["tools"]` and an API key in `LLMBASE_API_KEY`.

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const weatherTool = {
  type: "function",
  function: {
    name: "get_current_weather",
    parameters: {
      type: "object",
      properties: {
        city: { type: "string" },
        unit: { type: "string", enum: ["celsius", "fahrenheit"] },
      },
      required: ["city", "unit"],
    },
  },
} as const;

const first = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "What is the weather in Berlin?" }],
  tools: [weatherTool],
});

const toolCall = first.choices[0]?.message.tool_calls?.[0];
if (toolCall) {
  const final = await client.chat.completions.create({
    model: "<model-id-from-/v1/models>",
    messages: [
      { role: "user", content: "What is the weather in Berlin?" },
      first.choices[0].message,
      {
        role: "tool",
        tool_call_id: toolCall.id,
        content: JSON.stringify({ city: "Berlin", temperature: 14, unit: "celsius" }),
      },
    ],
    tools: [weatherTool],
  });

  console.log(final.choices[0]?.message.content);
}
```

--------------------------------

### GET /v1/models

Source: https://llmbase.ai/docs/inference/fallback/

Returns a list of public model IDs that can be used with the Inference API. Use this endpoint to select a model that advertises the matching capability for your request.

```APIDOC
## [GET] /v1/models

### Description
Returns a list of public model IDs that can be used with the Inference API. Use this endpoint to select a model that advertises the matching capability for your request.

### Method
GET

### Endpoint
/v1/models

### Parameters
#### Query Parameters
- **metadata** (boolean) - Optional - When set to `true`, returns model metadata including capabilities.

### Response
#### Success Response (200)
- **models** (array) - List of public model IDs.

### Usage Notes
- Before sending images, tools, structured output, logprobs, or reasoning controls, choose a model that advertises the matching capability in `/v1/models?metadata=true`.
```

--------------------------------

### GET /v1/models?metadata=true

Source: https://llmbase.ai/docs/inference/model-discovery/

Retrieves rich model metadata for all models, including context length, pricing, supported parameters, and features. This is the preferred endpoint for model discovery.

```APIDOC
## GET /v1/models?metadata=true

### Description
Retrieves rich model metadata for all models, including context length, pricing, supported parameters, and features. This is the preferred endpoint for model discovery.

### Method
GET

### Endpoint
/v1/models?metadata=true

### Parameters
#### Query Parameters
- **metadata** (boolean) - Required - Set to `true` to include rich metadata in the response.

### Request Example
```
curl "https://api.llmbase.ai/v1/models?metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **data** (array) - List of model objects. Each object includes:
  - **id** (string) - Stable LLMBase model ID used in API requests.
  - **context_length** (integer) - Maximum input + output context window.
  - **max_output_length** (integer) - Maximum generated tokens for one response.
  - **input_modalities** (array) - Supported input modalities (e.g., text, image).
  - **output_modalities** (array) - Supported output modalities.
  - **supported_input_image_media_types** (array) - Accepted inline image MIME types when image input is supported.
  - **pricing** (object) - Pricing details:
    - **prompt** (number) - USD per input token.
    - **completion** (number) - USD per output token.
    - **input_cache_read** (number) - Cached-input token price when prompt-cache reads are supported.
  - **supported_parameters** (array) - Parameters the model honors (e.g., `temperature`, `top_p`, `max_tokens`, `logprobs`, `top_logprobs`, `reasoning_effort`).
  - **supported_features** (array) - Higher-level features such as `tools`, `json_mode`, `structured_outputs`, `reasoning`, or `logprobs`.
  - **supported_reasoning_efforts** (array) - Optional model-specific `reasoning_effort` values.
  - **lifecycle** (object) - Optional deprecation metadata including `removal_date` and `replacement_ids`.

#### Response Example
```json
{
  "data": [
    {
      "id": "model-id",
      "context_length": 128000,
      "max_output_length": 4096,
      "input_modalities": ["text"],
      "output_modalities": ["text"],
      "supported_input_image_media_types": [],
      "pricing": {
        "prompt": 0.01,
        "completion": 0.02,
        "input_cache_read": 0.005
      },
      "supported_parameters": ["temperature", "top_p", "max_tokens"],
      "supported_features": ["tools", "json_mode"],
      "supported_reasoning_efforts": ["high", "max"],
      "lifecycle": {
        "removal_date": "2025-12-31",
        "replacement_ids": ["new-model-id"]
      }
    }
  ]
}
```
```

--------------------------------

### curl - Check Balance and Model Metadata

Source: https://llmbase.ai/docs/quickstart/

Check prepaid balance and model metadata before running batch jobs or agent loops. Use the LLMBASE_API_KEY environment variable.

```bash
curl https://api.llmbase.ai/v1/balance \
  -H "Authorization: Bearer $LLMBASE_API_KEY"

curl https://api.llmbase.ai/v1/model-metadata \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

--------------------------------

### Python - OpenAI SDK Chat Completion

Source: https://llmbase.ai/docs/quickstart/

Use the OpenAI SDK in Python to make a chat completion request. Set the base_url to https://api.llmbase.ai/v1 and use the LLMBASE_API_KEY environment variable.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmbase.ai/v1",
    api_key=os.environ["LLMBASE_API_KEY"],
)

response = client.chat.completions.create(
    model="<model-id-from-/v1/models>",
    messages=[{"role": "user", "content": "Hello! What can you do?"}],
)

print(response.choices[0].message.content)
```
Tokens:
Raw