### Full chat completion example using OpenAI SDK

Source: https://llmbase.ai/docs/inference/chat-completions/

JavaScript example using the OpenAI SDK to create a chat completion. It sets the base URL to https://api.llmbase.ai/v1 and uses the LLMBASE_API_KEY environment variable. The request includes system and user messages, temperature 0.7, and max_tokens 100, then logs the assistant's content.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Explain recursion in one sentence." },
  ],
  temperature: 0.7,
  max_tokens: 100,
});

console.log(response.choices[0].message.content);
```

--------------------------------

### GET /v1/models?metadata=true

Source: https://llmbase.ai/docs/inference/structured-outputs/

Check the live list of models and their supported features to determine if structured output is available.

```APIDOC
## [GET] /v1/models?metadata=true

### Description
Check the live list of models and their supported features to determine if structured output is available.

### Method
GET

### Endpoint
/v1/models?metadata=true

### Parameters
#### Query Parameters
- **metadata** (boolean) - Required - Set to `true` to include metadata about supported features.

### Request Example
```bash
curl "https://api.llmbase.ai/v1/models?metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **data** (array) - List of model objects.
- **object** (string) - The object type, e.g., `list`.

#### Response Example
```json
{
  "object": "list",
  "data": [
    {
      "id": "model-id",
      "object": "model",
      "created": 1677652288,
      "owned_by": "llmbase",
      "supported_features": ["json_mode", "structured_outputs"]
    }
  ]
}
```
```

--------------------------------

### Non-streaming response example

Source: https://llmbase.ai/docs/inference/chat-completions/

Example JSON response from a non-streaming chat completion request. Shows the structure including id, object, created, model, choices, and usage details. The finish_reason is 'stop' and usage includes prompt_tokens_details with cached_tokens.

```json
{
  "id": "chatcmpl-a1b2c3d4e5f6a1b2c3d4e5f6",
  "object": "chat.completion",
  "created": 1741000000,
  "model": "<model-id-from-/v1/models>",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I can help you with a wide range of tasks..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 48,
    "total_tokens": 60,
    "prompt_tokens_details": {
      "cached_tokens": 8
    }
  }
}
```

--------------------------------

### OpenAI SDK client setup

Source: https://llmbase.ai/docs/authentication/

Constructs an OpenAI client with the LLMBase base URL and API key. The SDK automatically adds the Authorization: Bearer header to every request.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY, // never hard-code keys
});
```

--------------------------------

### GET /v1/models

Source: https://llmbase.ai/docs/openai-compatibility/

Lists available models for OpenAI SDK model discovery. Supports optional metadata and filtering parameters.

```APIDOC
## GET /v1/models

### Description
Lists available models for OpenAI SDK model discovery. Supports optional metadata and filtering parameters.

### Method
GET

### Endpoint
/v1/models

### Parameters
#### Query Parameters
- **metadata** (boolean) - Optional - When true, returns detailed model metadata including pricing, modalities, and capabilities.
- **filter** (string) - Optional - Filters models by type. Use `agents` to get agent-compatible models.

### Response
#### Success Response (200)
- **data** (array) - List of model objects.
- **data[].id** (string) - Model identifier.
- **data[].supported_features** (array) - List of supported features (e.g., `tools`, `response_format`, `reasoning`, `logprobs`).
- **data[].supported_parameters** (array) - List of supported parameters.
- **data[].pricing** (object) - Pricing information (when metadata=true).
- **data[].modalities** (array) - Supported input/output modalities (when metadata=true).
- **data[].capabilities** (object) - Model capabilities (when metadata=true).

#### Response Example
{
  "data": [
    {
      "id": "model-id",
      "supported_features": ["tools", "response_format", "reasoning"],
      "supported_parameters": ["temperature", "top_p"],
      "pricing": {},
      "modalities": ["text"],
      "capabilities": {}
    }
  ]
}
```

--------------------------------

### OpenAI SDK chat agent example

Source: https://llmbase.ai/docs/agents/setup/

Use the OpenAI SDK with a custom base URL and the LLMBASE_CHAT_AGENT_KEY environment variable. Lists models, picks the first model ID, and creates a chat completion. Requires the OpenAI SDK and the environment variable set.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://llmbase.ai/api/v1/agents",
  apiKey: process.env.LLMBASE_CHAT_AGENT_KEY,
});

const models = await client.models.list();
const model = models.data[0]?.id;

const response = await client.chat.completions.create({
  model,
  messages: [
    { role: "user", content: "Draft a short product update." },
  ],
});

console.log(response.choices[0]?.message?.content);
```

--------------------------------

### GET /v1/balance

Source: https://llmbase.ai/docs/authentication/

Check the live API balance for an inference API key. Returns the balance in USD cents.

```APIDOC
## GET /v1/balance

### Description
Check the live API balance for an inference API key. Returns the balance in USD cents.

### Method
GET

### Endpoint
https://api.llmbase.ai/v1/balance

### Parameters
#### Headers
- **Authorization** (string) - Required - Bearer token with a valid `llmbase_...` API key.

### Request Example
```
curl https://api.llmbase.ai/v1/balance \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **balance_usd_cents** (integer) - The current prepaid balance in USD cents.

#### Response Example
```
{
  "balance_usd_cents": 500
}
```

```

--------------------------------

### GET /v1/model-metadata

Source: https://llmbase.ai/docs/inference/model-discovery/

Compatibility alias for the same rich metadata format as /v1/models?metadata=true. New clients should prefer /v1/models?metadata=true.

```APIDOC
## GET /v1/model-metadata

### Description
Compatibility alias for the same rich metadata format as /v1/models?metadata=true. New clients should prefer /v1/models?metadata=true.

### Method
GET

### Endpoint
/v1/model-metadata

### Parameters
None

### Response
#### Success Response (200)
Same response structure as GET /v1/models?metadata=true.

#### Response Example
```json
{
  "data": [
    {
      "id": "model-id",
      "context_length": 128000,
      "max_output_length": 4096,
      "input_modalities": ["text"],
      "output_modalities": ["text"],
      "supported_input_image_media_types": [],
      "pricing": {
        "prompt": 0.01,
        "completion": 0.02,
        "input_cache_read": 0.005
      },
      "supported_parameters": ["temperature", "top_p", "max_tokens"],
      "supported_features": ["tools", "json_mode"],
      "supported_reasoning_efforts": ["high", "max"],
      "lifecycle": {
        "removal_date": "2025-12-31",
        "replacement_ids": ["new-model-id"]
      }
    }
  ]
}
```
```

--------------------------------

### 401 error response

Source: https://llmbase.ai/docs/authentication/

Example error response returned when a missing or invalid key is used.

```json
{
  "error": {
    "message": "Invalid API key",
    "type": "invalid_request_error",
    "code": "invalid_api_key"
  }
}
```

--------------------------------

### Using the OpenAI SDK with a dedicated endpoint

Source: https://llmbase.ai/docs/dedicated-inference/

Example of using the OpenAI SDK with a dedicated endpoint. Set the baseURL to the endpoint's /v1 path and provide your LLMBase API key. The model ID should be obtained from /v1/models.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<endpoint-id>-inference.llmbase.cloud/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Hello!" }],
});
```

--------------------------------

### GET /v1/models?metadata=true

Source: https://llmbase.ai/docs/inference/rate-limits/

Retrieve model metadata including context_length and max_output_length fields to size requests appropriately.

```APIDOC
## GET /v1/models?metadata=true

### Description
Retrieves model metadata including context and output limits. Use the `context_length` and `max_output_length` fields to size requests and avoid exceeding model limits.

### Method
GET

### Endpoint
/v1/models?metadata=true

### Query Parameters
- **metadata** (boolean) - Required - Set to `true` to include model metadata fields.

### Response
#### Success Response (200)
- **context_length** (integer) - The maximum context length for the model.
- **max_output_length** (integer) - The maximum output length for the model.

#### Response Example
{
  "data": [
    {
      "id": "model-id",
      "context_length": 8192,
      "max_output_length": 2048
    }
  ]
}
```

--------------------------------

### GET /v1/balance

Source: https://llmbase.ai/docs/openai-compatibility/

Retrieves the LLMBase prepaid inference balance.

```APIDOC
## GET /v1/balance

### Description
Retrieves the LLMBase prepaid inference balance.

### Method
GET

### Endpoint
/v1/balance

### Response
#### Success Response (200)
- **balance** (number) - The prepaid inference balance.

#### Response Example
{
  "balance": 100.00
}
```

--------------------------------

### GET /api/v1/agents/models

Source: https://llmbase.ai/docs/inference/model-discovery/

Returns the current chat/tool-capable models available for subscription-backed agents. Use this endpoint at runtime instead of maintaining a static allowlist.

```APIDOC
## GET /api/v1/agents/models

### Description
Returns the current chat/tool-capable models available for subscription-backed agents. Retrieve it at runtime instead of maintaining a static OpenClaw or Hermes allowlist.

### Method
GET

### Endpoint
https://llmbase.ai/api/v1/agents/models

### Authentication
Bearer token (use a `llmbase_chat_...` key)

### Request Example
```
curl https://llmbase.ai/api/v1/agents/models \
  -H "Authorization: Bearer $LLMBASE_CHAT_AGENT_KEY"
```

### Response
Returns the current list of chat/tool-capable models. Models not returned by this endpoint cannot be used with `llmbase_chat_...` keys.

### Notes
- Use a `llmbase_...` key and `https://api.llmbase.ai/v1` when you need direct inference billing or a model outside the agent-compatible list.
```

--------------------------------

### Chat messages array

Source: https://llmbase.ai/docs/inference/chat-completions/

Example of a conversation history array with system, user, and assistant messages. At least one message is required.

```json
[
  { "role": "system",    "content": "You are a helpful assistant." },
  { "role": "user",      "content": "Summarise this article: ..." },
  { "role": "assistant", "content": "Here is a summary: ..." },
  { "role": "user",      "content": "Make it shorter." }
]
```

--------------------------------

### List agent-compatible models with curl

Source: https://llmbase.ai/docs/inference/model-discovery/

Sends a GET request to the agent API endpoint `https://llmbase.ai/api/v1/agents/models` with a Bearer token. This endpoint returns chat/tool-capable models for subscription-backed agents. Models not returned here cannot be used with `llmbase_chat_...` keys.

```bash
curl https://llmbase.ai/api/v1/agents/models \
  -H "Authorization: Bearer $LLMBASE_CHAT_AGENT_KEY"
```

--------------------------------

### GET /v1/models

Source: https://llmbase.ai/docs/inference/model-discovery/

Returns all available models in the OpenAI models format. Send the same Bearer header used for other API requests.

```APIDOC
## GET /v1/models

### Description
Returns all available models in the OpenAI models format.

### Method
GET

### Endpoint
https://api.llmbase.ai/v1/models

### Authentication
Bearer token (same as other API requests)

### Request Example
```
curl https://api.llmbase.ai/v1/models \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **object** (string) - Always "list"
- **data** (array) - List of model objects
  - **id** (string) - Model identifier in format "provider/model-id"
  - **object** (string) - Always "model"
  - **created** (integer) - Creation timestamp
  - **owned_by** (string) - Provider name
  - **name** (string) - Display name
  - **description** (string) - Model description
  - **context_length** (integer) - Context window size

#### Response Example
```
{
  "object": "list",
  "data": [
    {
      "id": "provider/model-id",
      "object": "model",
      "created": 1700000000,
      "owned_by": "provider",
      "name": "Provider: Model",
      "description": "Model description.",
      "context_length": 1048576
    }
  ]
}
```

### OpenAI SDK Usage
```
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const models = await client.models.list();
for (const model of models.data) {
  console.log(model.id);
}
```
```

--------------------------------

### GET /models

Source: https://llmbase.ai/docs/agents/

Retrieves the list of models available to a chat agent key. The model list may differ from the direct inference catalog.

```APIDOC
## GET /models

### Description
Retrieves the list of models available to a chat agent key. The model list may differ from the direct inference catalog.

### Method
GET

### Endpoint
https://llmbase.ai/api/v1/agents/models

### Authentication
Authorization: Bearer <LLMBASE_CHAT_AGENT_KEY>

### Response
#### Success Response (200)
Returns a list of available models.

#### Error Response (403)
- **error** (string) - `chat_pro_required` if the account is not on Pro or does not have an active Business seat.
```

--------------------------------

### Multimodal user message

Source: https://llmbase.ai/docs/inference/chat-completions/

Example of a user message with multimodal content as an array of parts, including text and an image URL.

```json
{
  "role": "user",
  "content": [
    { "type": "text", "text": "What is in this image?" },
    { "type": "image_url", "image_url": { "url": "https://example.com/image.jpg" } }
  ]
}
```

--------------------------------

### List models with curl

Source: https://llmbase.ai/docs/inference/model-discovery/

Sends a GET request to the direct inference endpoint `https://api.llmbase.ai/v1/models` with a Bearer token. The response is a list of model objects in the OpenAI models format.

```bash
curl https://api.llmbase.ai/v1/models \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

```json
{
  "object": "list",
  "data": [
    {
      "id": "provider/model-id",
      "object": "model",
      "created": 1700000000,
      "owned_by": "provider",
      "name": "Provider: Model",
      "description": "Model description.",
      "context_length": 1048576
    }
  ]
}
```

--------------------------------

### GET /v1/models?metadata=true

Source: https://llmbase.ai/docs/inference/model-discovery/

Retrieves rich model metadata for all models, including context length, pricing, supported parameters, and features. This is the preferred endpoint for model discovery.

```APIDOC
## GET /v1/models?metadata=true

### Description
Retrieves rich model metadata for all models, including context length, pricing, supported parameters, and features. This is the preferred endpoint for model discovery.

### Method
GET

### Endpoint
/v1/models?metadata=true

### Parameters
#### Query Parameters
- **metadata** (boolean) - Required - Set to `true` to include rich metadata in the response.

### Request Example
```
curl "https://api.llmbase.ai/v1/models?metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
- **data** (array) - List of model objects. Each object includes:
  - **id** (string) - Stable LLMBase model ID used in API requests.
  - **context_length** (integer) - Maximum input + output context window.
  - **max_output_length** (integer) - Maximum generated tokens for one response.
  - **input_modalities** (array) - Supported input modalities (e.g., text, image).
  - **output_modalities** (array) - Supported output modalities.
  - **supported_input_image_media_types** (array) - Accepted inline image MIME types when image input is supported.
  - **pricing** (object) - Pricing details:
    - **prompt** (number) - USD per input token.
    - **completion** (number) - USD per output token.
    - **input_cache_read** (number) - Cached-input token price when prompt-cache reads are supported.
  - **supported_parameters** (array) - Parameters the model honors (e.g., `temperature`, `top_p`, `max_tokens`, `logprobs`, `top_logprobs`, `reasoning_effort`).
  - **supported_features** (array) - Higher-level features such as `tools`, `json_mode`, `structured_outputs`, `reasoning`, or `logprobs`.
  - **supported_reasoning_efforts** (array) - Optional model-specific `reasoning_effort` values.
  - **lifecycle** (object) - Optional deprecation metadata including `removal_date` and `replacement_ids`.

#### Response Example
```json
{
  "data": [
    {
      "id": "model-id",
      "context_length": 128000,
      "max_output_length": 4096,
      "input_modalities": ["text"],
      "output_modalities": ["text"],
      "supported_input_image_media_types": [],
      "pricing": {
        "prompt": 0.01,
        "completion": 0.02,
        "input_cache_read": 0.005
      },
      "supported_parameters": ["temperature", "top_p", "max_tokens"],
      "supported_features": ["tools", "json_mode"],
      "supported_reasoning_efforts": ["high", "max"],
      "lifecycle": {
        "removal_date": "2025-12-31",
        "replacement_ids": ["new-model-id"]
      }
    }
  ]
}
```
```

--------------------------------

### GET /v1/models

Source: https://llmbase.ai/docs/inference/fallback/

Returns a list of public model IDs that can be used with the Inference API. Use this endpoint to select a model that advertises the matching capability for your request.

```APIDOC
## [GET] /v1/models

### Description
Returns a list of public model IDs that can be used with the Inference API. Use this endpoint to select a model that advertises the matching capability for your request.

### Method
GET

### Endpoint
/v1/models

### Parameters
#### Query Parameters
- **metadata** (boolean) - Optional - When set to `true`, returns model metadata including capabilities.

### Response
#### Success Response (200)
- **models** (array) - List of public model IDs.

### Usage Notes
- Before sending images, tools, structured output, logprobs, or reasoning controls, choose a model that advertises the matching capability in `/v1/models?metadata=true`.
```

--------------------------------

### GET /v1/models?filter=agents&metadata=true

Source: https://llmbase.ai/docs/inference/model-discovery/

Retrieves model metadata for agent-capable models only. Useful for agents and SaaS products where model lists should update automatically as LLMBase adds new eligible models.

```APIDOC
## GET /v1/models?filter=agents&metadata=true

### Description
Retrieves model metadata for agent-capable models only. Useful for agents and SaaS products where model lists should update automatically as LLMBase adds new eligible models.

### Method
GET

### Endpoint
/v1/models?filter=agents&metadata=true

### Parameters
#### Query Parameters
- **filter** (string) - Required - Set to `agents` to filter for agent-capable models.
- **metadata** (boolean) - Required - Set to `true` to include rich metadata.

### Request Example
```
curl "https://api.llmbase.ai/v1/models?filter=agents&metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

### Response
#### Success Response (200)
Same response structure as GET /v1/models?metadata=true, but only includes agent-capable models.

#### Response Example
```json
{
  "data": [
    {
      "id": "agent-model-id",
      "context_length": 128000,
      "max_output_length": 4096,
      "input_modalities": ["text"],
      "output_modalities": ["text"],
      "supported_input_image_media_types": [],
      "pricing": {
        "prompt": 0.01,
        "completion": 0.02,
        "input_cache_read": 0.005
      },
      "supported_parameters": ["temperature", "top_p", "max_tokens"],
      "supported_features": ["tools", "json_mode"],
      "supported_reasoning_efforts": ["high", "max"],
      "lifecycle": {
        "removal_date": "2025-12-31",
        "replacement_ids": ["new-model-id"]
      }
    }
  ]
}
```
```

--------------------------------

### POST /v1/chat/completions

Source: https://llmbase.ai/docs/inference/fallback/

Every chat completion request must use a public model ID returned by GET /v1/models. If the selected model is temporarily unavailable, the API returns an OpenAI-compatible error. Treat availability errors as temporary: retry with exponential backoff and avoid sending duplicate requests in parallel.

```APIDOC
## [POST] /v1/chat/completions

### Description
Creates a chat completion using a public model ID. If the selected model is temporarily unavailable, the API returns an OpenAI-compatible error.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - Public model ID returned by GET /v1/models.

### Response
#### Success Response (200)
- **OpenAI-compatible response shape** - The response follows the normal OpenAI-compatible response shape.

#### Error Response (502)
- **error** (object) - Structured error object indicating temporary model unavailability.

#### Error Response (429)
- **error** (object) - Structured error object indicating the request should be retried later.

### Error Handling
- Inspect the HTTP status and structured `error` object.
- A `502` server error can indicate temporary model unavailability.
- A `429` response indicates that the request should be retried later.
- Honor `Retry-After` when present and otherwise use bounded exponential backoff.
```

--------------------------------

### Streamed response: data lines and [DONE] terminator

Source: https://llmbase.ai/docs/inference/streaming/

Example of the Server-Sent Events stream returned by the curl request. Each event is a JSON-encoded `chat.completion.chunk`. The stream ends with `data: [DONE]`. Token usage is included in the final chunk.

```text
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1741000000,"model":"<model-id-from-/v1/models>","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1741000000,"model":"<model-id-from-/v1/models>","choices":[{"index":0,"delta":{"content":"1"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1741000000,"model":"<model-id-from-/v1/models>","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":9,"total_tokens":19}}

data: [DONE]
```

--------------------------------

### Python: Set up OpenAI client with LLMBase base URL

Source: https://llmbase.ai/docs/openai-compatibility/

Use this Python snippet to configure the OpenAI SDK to point to LLMBase's API endpoint. Set the base_url to https://api.llmbase.ai/v1 and use an llmbase_... inference key. The model ID should be obtained from /v1/models.

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmbase.ai/v1",
    api_key=os.environ["LLMBASE_API_KEY"],
)

response = client.chat.completions.create(
    model="<model-id-from-/v1/models>",
    messages=[{"role": "user", "content": "Write a short deployment checklist."}],
)

print(response.choices[0].message.content)
```

--------------------------------

### List models with OpenAI SDK

Source: https://llmbase.ai/docs/inference/model-discovery/

Uses the OpenAI SDK with a custom base URL to list models. Requires the `LLMBASE_API_KEY` environment variable.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const models = await client.models.list();
for (const model of models.data) {
  console.log(model.id);
}
```

--------------------------------

### JavaScript: Set up OpenAI client with LLMBase base URL

Source: https://llmbase.ai/docs/openai-compatibility/

Use this JavaScript snippet to configure the OpenAI SDK to point to LLMBase's API endpoint. Set the baseURL to https://api.llmbase.ai/v1 and use an llmbase_... inference key. The model ID should be obtained from /v1/models.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Write a short deployment checklist." }],
});

console.log(response.choices[0]?.message.content);
```

--------------------------------

### Basic request with prompt_cache_key

Source: https://llmbase.ai/docs/inference/prompt-caching/

Use this JSON body to send a request with a stable prompt_cache_key. Keep the cacheable prefix stable and put changing instructions later in the message list. Short prompts may not be cached.

```json
{
  "model": "<model-id-from-/v1/models>",
  "prompt_cache_key": "workspace-123-agent",
  "messages": [
    { "role": "system", "content": "You are a coding agent for this repository..." },
    { "role": "user", "content": "Continue from the last task." }
  ]
}
```

--------------------------------

### OpenAI SDK: Stream chat completions in JavaScript

Source: https://llmbase.ai/docs/inference/streaming/

Uses the OpenAI SDK with a custom `baseURL` to stream chat completions. Requires the `LLMBASE_API_KEY` environment variable. The `for await` loop writes each delta content chunk to stdout.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Write a short poem." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

--------------------------------

### Function tool calling with OpenAI SDK

Source: https://llmbase.ai/docs/inference/tools/

Shows the full workflow: define a weather tool, send a chat completion with `tools`, handle the tool call, and send the result back as a `tool` message. Requires a model with `supported_features: ["tools"]` and an API key in `LLMBASE_API_KEY`.

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const weatherTool = {
  type: "function",
  function: {
    name: "get_current_weather",
    parameters: {
      type: "object",
      properties: {
        city: { type: "string" },
        unit: { type: "string", enum: ["celsius", "fahrenheit"] },
      },
      required: ["city", "unit"],
    },
  },
} as const;

const first = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "What is the weather in Berlin?" }],
  tools: [weatherTool],
});

const toolCall = first.choices[0]?.message.tool_calls?.[0];
if (toolCall) {
  const final = await client.chat.completions.create({
    model: "<model-id-from-/v1/models>",
    messages: [
      { role: "user", content: "What is the weather in Berlin?" },
      first.choices[0].message,
      {
        role: "tool",
        tool_call_id: toolCall.id,
        content: JSON.stringify({ city: "Berlin", temperature: 14, unit: "celsius" }),
      },
    ],
    tools: [weatherTool],
  });

  console.log(final.choices[0]?.message.content);
}
```

--------------------------------

### curl - Check Balance and Model Metadata

Source: https://llmbase.ai/docs/quickstart/

Check prepaid balance and model metadata before running batch jobs or agent loops. Use the LLMBASE_API_KEY environment variable.

```bash
curl https://api.llmbase.ai/v1/balance \
  -H "Authorization: Bearer $LLMBASE_API_KEY"

curl https://api.llmbase.ai/v1/model-metadata \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

--------------------------------

### Python - OpenAI SDK Chat Completion

Source: https://llmbase.ai/docs/quickstart/

Use the OpenAI SDK in Python to make a chat completion request. Set the base_url to https://api.llmbase.ai/v1 and use the LLMBASE_API_KEY environment variable.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llmbase.ai/v1",
    api_key=os.environ["LLMBASE_API_KEY"],
)

response = client.chat.completions.create(
    model="<model-id-from-/v1/models>",
    messages=[{"role": "user", "content": "Hello! What can you do?"}],
)

print(response.choices[0].message.content)
```

--------------------------------

### Hermes configuration for LLMBase custom provider

Source: https://llmbase.ai/docs/agents/setup/

Configure Hermes as a named custom provider 'llmbase' with the agent API base URL. A bare 'custom' provider with only OPENAI_API_KEY may not send the key to non-OpenAI hosts. Load the current agent model list first, then choose a returned model with the tool, context, and cost profile your agent loop needs. Hermes uses 'reasoning_effort: medium' by default; LLMBase accepts that default for reasoning-capable models.

```yaml
model:
  provider: llmbase
  model: <model-id-from-agent-models>
  base_url: https://llmbase.ai/api/v1/agents
  api_key: llmbase_chat_...
  api_mode: chat_completions

custom_providers:
  - name: llmbase
    base_url: https://llmbase.ai/api/v1/agents
    api_key: llmbase_chat_...
    api_mode: chat_completions
    model: <model-id-from-agent-models>
```

--------------------------------

### Extract text from an image using OpenAI SDK

Source: https://llmbase.ai/docs/ocr/

Sends an image as a data URI to the vision-capable model 'qwen/qwen3.8-27b' via the chat completions API. The image_url.url accepts either a data URI or a publicly reachable https:// URL. Requires the LLMBASE_API_KEY environment variable.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "qwen/qwen3.8-27b",
  temperature: 0,
  messages: [
    {
      role: "user",
      content: [
        {
          type: "text",
          text: "Extract all readable text from this image. Return only the extracted text, preserving line breaks.",
        },
        {
          type: "image_url",
          image_url: { url: "data:image/png;base64,<base64-encoded-image>" },
        },
      ],
    },
  ],
});

console.log(response.choices[0].message.content);
```

--------------------------------

### Node.js - OpenAI SDK Chat Completion

Source: https://llmbase.ai/docs/quickstart/

Use the OpenAI SDK in Node.js to make a chat completion request. Set the baseURL to https://api.llmbase.ai/v1 and use the LLMBASE_API_KEY environment variable.

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llmbase.ai/v1",
  apiKey: process.env.LLMBASE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Hello! What can you do?" }],
});

console.log(response.choices[0].message.content);
```

--------------------------------

### Cost estimation formula

Source: https://llmbase.ai/docs/inference/prompt-caching/

Use this formula to estimate cost. When pricing.input_cache_read is missing, cached input tokens use the normal input-token price.

```text
cost = input_tokens * pricing.prompt
     + cached_input_tokens * pricing.input_cache_read
     + output_tokens * pricing.completion
```

--------------------------------

### Prompt layout for cache hits

Source: https://llmbase.ai/docs/inference/prompt-caching/

Use this message layout to maximize cache hits by keeping a long static prefix and placing the short task at the end. Avoid putting changing content before the static prompt.

```json
[
  { "role": "system", "content": "Long product policy, coding rules, or workspace instructions..." },
  { "role": "user", "content": "Current short task goes here." }
]
```

--------------------------------

### Typical non-streaming response with reasoning trace

Source: https://llmbase.ai/docs/inference/reasoning/

The response includes the final answer in content and the reasoning trace in reasoning_content, with a compatibility alias reasoning.

```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "17 * 23 = 391.",
        "reasoning_content": "Compute 17 * 20 = 340 and 17 * 3 = 51, then add them.",
        "reasoning": "Compute 17 * 20 = 340 and 17 * 3 = 51, then add them."
      }
    }
  ]
}
```

--------------------------------

### Filter models programmatically with JavaScript fetch

Source: https://llmbase.ai/docs/inference/model-discovery/

Fetches model metadata and filters for models that support tools and have input cache read pricing. Uses the LLMBASE_API_KEY environment variable.

```javascript
const res = await fetch("https://api.llmbase.ai/v1/models?metadata=true", {
  headers: { Authorization: `Bearer ${process.env.LLMBASE_API_KEY}` },
});

const { data } = await res.json();
const toolModels = data.filter((model) =>
  model.supported_features?.includes("tools") &&
  model.pricing?.input_cache_read
);

console.log(toolModels.map((model) => model.id));
```

--------------------------------

### curl balance request and response

Source: https://llmbase.ai/docs/authentication/

Checks the live API balance using curl. The response shows the balance in USD cents.

```bash
curl https://api.llmbase.ai/v1/balance \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

```json
{
  "balance_usd_cents": 500
}
```

--------------------------------

### curl: Stream chat completions with stream: true

Source: https://llmbase.ai/docs/inference/streaming/

Sends a chat completion request with `"stream": true` to receive tokens as Server-Sent Events. Requires an API key stored in the `LLMBASE_API_KEY` environment variable. The response is a sequence of `data: {...}` lines terminated by `data: [DONE]`.

```bash
curl https://api.llmbase.ai/v1/chat/completions \
  -H "Authorization: Bearer $LLMBASE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-from-/v1/models>",
    "messages": [{ "role": "user", "content": "Count to 5." }],
    "stream": true
  }'
```

--------------------------------

### Read model metadata with curl

Source: https://llmbase.ai/docs/inference/prompt-caching/

Use this curl command to read token prices from the model metadata endpoint before running a workload. It requires an API key in the LLMBASE_API_KEY environment variable.

```bash
curl "https://api.llmbase.ai/v1/models?metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

--------------------------------

### Base URL

Source: https://llmbase.ai/docs/quickstart/

The base URL for the LLMBase API.

```text
https://api.llmbase.ai
```

--------------------------------

### POST /v1/chat/completions

Source: https://llmbase.ai/docs/inference/chat-completions/

Creates a chat completion. The endpoint supports various request options for different use cases, including streaming, response formatting, tool use, and confidence scoring. It returns a non-streaming response with choices and usage details.

```APIDOC
## POST /v1/chat/completions

### Description
Creates a chat completion. The endpoint supports various request options for different use cases, including streaming, response formatting, tool use, and confidence scoring. It returns a non-streaming response with choices and usage details.

### Method
POST

### Endpoint
/v1/chat/completions

### Parameters
#### Request Body
- **messages** (array) - Required - The conversation messages.
- **stream** (boolean) - Optional - Whether to stream the response.
- **max_tokens** (integer) - Optional - Maximum number of tokens to generate.
- **response_format** (object) - Optional - Format for the response (e.g., JSON).
- **temperature** (number) - Optional - Sampling temperature.
- **tools** (array) - Optional - List of tools the model can call.
- **tool_choice** (string/object) - Optional - Controls tool selection.
- **prompt_cache_key** (string) - Optional - Key for prompt caching.
- **logprobs** (boolean) - Optional - Whether to return log probabilities.
- **top_logprobs** (integer) - Optional - Number of top log probabilities to return.

### Request Example
```json
{
  "model": "<model-id-from-/v1/models>",
  "messages": [
    { "role": "system", "content": "You are a concise assistant." },
    { "role": "user", "content": "Explain recursion in one sentence." }
  ],
  "temperature": 0.7,
  "max_tokens": 100
}
```

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the completion.
- **object** (string) - Always "chat.completion".
- **created** (integer) - Unix timestamp of creation.
- **model** (string) - Model used.
- **choices** (array) - List of completion choices.
  - **index** (integer) - Index of the choice.
  - **message** (object) - The assistant message.
    - **role** (string) - Role of the message (e.g., "assistant").
    - **content** (string) - The generated content.
  - **finish_reason** (string) - Reason for finishing (e.g., "stop").
- **usage** (object) - Token usage details.
  - **prompt_tokens** (integer) - Number of prompt tokens.
  - **completion_tokens** (integer) - Number of completion tokens.
  - **total_tokens** (integer) - Total tokens.
  - **prompt_tokens_details** (object) - Details about prompt tokens.
    - **cached_tokens** (integer) - Number of cached tokens.

#### Response Example
```json
{
  "id": "chatcmpl-a1b2c3d4e5f6a1b2c3d4e5f6",
  "object": "chat.completion",
  "created": 1741000000,
  "model": "<model-id-from-/v1/models>",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I can help you with a wide range of tasks..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 48,
    "total_tokens": 60,
    "prompt_tokens_details": {
      "cached_tokens": 8
    }
  }
}
```

### finish_reason Values
| Value | Meaning |
| --- | --- |
| `stop` | Model finished naturally |
| `length` | `max_tokens` limit reached |
| `content_filter` | Response was filtered |
| `tool_calls` | Model called one or more tools |

```

--------------------------------

### Node.js - Streaming Chat Completion

Source: https://llmbase.ai/docs/quickstart/

Stream chat completion tokens using the OpenAI SDK in Node.js. Add "stream": true to receive tokens as Server-Sent Events.

```javascript
const stream = await client.chat.completions.create({
  model: "<model-id-from-/v1/models>",
  messages: [{ role: "user", content: "Write a haiku about inference." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

--------------------------------

### POST /v1/chat/completions

Source: https://llmbase.ai/docs/authentication/

Send a chat completion request to the LLMBase inference API. Requires a valid Bearer token and a model ID from /v1/models.

```APIDOC
## POST /v1/chat/completions

### Description
Send a chat completion request to the LLMBase inference API. Requires a valid Bearer token and a model ID from /v1/models.

### Method
POST

### Endpoint
https://api.llmbase.ai/v1/chat/completions

### Parameters
#### Headers
- **Authorization** (string) - Required - Bearer token with a valid `llmbase_...` API key.
- **Content-Type** (string) - Required - Must be `application/json`.

#### Request Body
- **model** (string) - Required - The model ID from `/v1/models`.
- **messages** (array) - Required - An array of message objects, each with `role` and `content`.

### Request Example
```
curl https://api.llmbase.ai/v1/chat/completions \
  -H "Authorization: Bearer $LLMBASE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "<model-id-from-/v1/models>", "messages": [{ "role": "user", "content": "Hi" }] }'
```

```

--------------------------------

### Authentication header for dedicated endpoints

Source: https://llmbase.ai/docs/dedicated-inference/

Use your LLMBase API key in the Authorization header, the same as the Serverless Inference API.

```text
Authorization: Bearer <LLMBASE_API_KEY>
```

--------------------------------

### Fetch agent-capable models with curl

Source: https://llmbase.ai/docs/inference/model-discovery/

Adds the agents filter to the /v1/models endpoint to retrieve only agent-capable models. Useful for agents and SaaS products where model lists should update automatically.

```bash
curl "https://api.llmbase.ai/v1/models?filter=agents&metadata=true" \
  -H "Authorization: Bearer $LLMBASE_API_KEY"
```

--------------------------------

### Chat completions endpoint URL

Source: https://llmbase.ai/docs/dedicated-inference/

The standard OpenAI-compatible chat completions interface exposed by the dedicated endpoint. Use this URL for API requests.

```text
POST https://<endpoint-id>-inference.llmbase.cloud/v1/chat/completions
```