Bifrost

https://docs.getbifrost.ai/quickstart/
Admin
Bifrost is a blazing-fast HTTP API gateway that unifies access to various AI providers (like OpenAI,...
Tokens:60,968
Snippets:355
Trust Score:4
Update:8 months ago
Show doc for...
Tokens:
Raw
### Configure Providers via API

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This section details how to add and configure different AI model providers using the API. Examples include OpenAI, Anthropic, and vLLM.

```APIDOC
## POST /api/providers

### Description
Adds or updates a model provider configuration.

### Method
POST

### Endpoint
`/api/providers`

### Parameters
#### Request Body
- **provider** (string) - Required - The name of the AI provider (e.g., "openai", "anthropic", "vllm-local").
- **keys** (array) - Required - A list of API keys or authentication credentials for the provider.
  - **name** (string) - Required - A unique name for the key.
  - **value** (string) - Required - The actual key value or reference (e.g., environment variable).
  - **models** (array) - Optional - A list of specific models supported by this key.
  - **weight** (number) - Optional - A weight for load balancing or priority (defaults to 1.0).
- **network_config** (object) - Optional - Network-related configurations for the provider.
  - **base_url** (string) - Optional - The base URL for the provider's API endpoint (e.g., for self-hosted models).
  - **default_request_timeout_in_seconds** (integer) - Optional - The default timeout for requests to this provider.
- **custom_provider_config** (object) - Optional - Custom configurations specific to the provider type.
  - **base_provider_type** (string) - Optional - The underlying provider type if this is a wrapper (e.g., "openai").
  - **allowed_requests** (object) - Optional - Specifies which API endpoints are allowed for this provider.
    - **chat_completion** (boolean) - Optional - Whether chat completion is allowed.
    - **chat_completion_stream** (boolean) - Optional - Whether streaming chat completion is allowed.

### Request Example
```json
{
    "provider": "openai",
    "keys": [
        {
            "name": "openai-key-1",
            "value": "env.OPENAI_API_KEY",
            "models": [],
            "weight": 1.0
        }
    ]
}
```

### Response
#### Success Response (200)
- **message** (string) - Confirmation message indicating the provider was added successfully.

#### Response Example
```json
{
  "message": "Provider openai added successfully."
}
```
```

--------------------------------

### Google Vertex AI Configuration using config.json

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

An example of how to configure Google Vertex AI within a `config.json` file, specifying provider details, API keys, and model settings.

```APIDOC
## config.json Structure for Vertex AI

### Description
This JSON structure defines the configuration for the Google Vertex AI provider within the `config.json` file. It includes details for API keys, supported models, and specific Vertex AI settings like project ID, region, and authentication credentials.

### Method
N/A (File Configuration)

### Endpoint
N/A (File Configuration)

### Parameters
#### Request Body (config.json content)
- **providers** (object) - Top-level object for provider configurations.
  - **vertex** (object) - Configuration specific to the Vertex AI provider.
    - **keys** (array) - A list of authentication keys and model configurations for Vertex AI.
      - **name** (string) - Required - A unique name for this key configuration.
      - **value** (string) - Required - The API key for the provider (can be an environment variable reference like "env.VERTEX_API_KEY").
      - **models** (array of strings) - Required - A list of model identifiers supported by this key.
      - **weight** (number) - Optional - A weighting factor for load balancing.
      - **vertex_key_config** (object) - Optional - Specific configuration for Vertex AI.
        - **project_id** (string) - Required - Your Google Cloud project ID (can be an environment variable reference like "env.VERTEX_PROJECT_ID").
        - **region** (string) - Required - The Google Cloud region for Vertex AI (e.g., "us-central1").
        - **auth_credentials** (string) - Optional - Path to or content of service account credentials JSON (can be an environment variable reference like "env.VERTEX_CREDENTIALS").
        - **deployments** (object) - Optional - Aliases for custom fine-tuned model deployments.
          - **model_alias** (string) - The deployment alias for a custom model.

### Request Example
```json
{
    "providers": {
        "vertex": {
            "keys": [
                {
                    "name": "vertex-key-1",
                    "value": "env.VERTEX_API_KEY",
                    "models": ["gemini-pro", "gemini-pro-vision"],
                    "weight": 1.0,
                    "vertex_key_config": {
                        "project_id": "env.VERTEX_PROJECT_ID",
                        "region": "us-central1",
                        "auth_credentials": "env.VERTEX_CREDENTIALS",
                        "deployments": {
                            "fine-tuned-gemini-2.5-pro": "123456789"
                        }
                    }
                }
            ]
        }
    }
}
```

### Response
N/A (This is a configuration file, not an API endpoint response.)
```

--------------------------------

### Send Dynamic Headers Per Request

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This example shows how to send dynamic headers with individual requests using the `x-bf-eh-*` prefix. These headers are automatically propagated to the provider after stripping the prefix. This is useful for request-specific metadata. The input is a cURL command with custom headers.

```bash
curl --location 'http://localhost:8080/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'x-bf-eh-user-id: user-123' \
--header 'x-bf-eh-tracking-id: trace-456' \
--data '{ \
    "model": "openai/gpt-4o-mini", \
    "messages": [ \
        {"role": "user", "content": "Hello!"} \
    ] \
}'
```

--------------------------------

### Install Qwen Code using npm

Source: https://docs.getbifrost.ai/quickstart/gateway/cli-agents

Installs the Qwen Code package globally using npm. This command requires Node.js and npm to be installed on your system.

```bash
npm install -g @qwen-code/qwen-code
```

--------------------------------

### Install and Configure Codex CLI (Bash)

Source: https://docs.getbifrost.ai/quickstart/gateway/cli-agents

Instructions for installing and configuring the Codex CLI using npm. It includes setting up environment variables for the OpenAI API key and base URL, which are handled by Bifrost when using virtual keys. After setup, the `codex` command can be run to start the CLI.

```bash
# Install Codex CLI
npm install -g @openai/codex

# Configure Environment Variables
export OPENAI_API_KEY=dummy-key  # Handled by Bifrost (only set when using virtual keys)
export OPENAI_BASE_URL=http://localhost:8080/openai

# Run Codex
codex
```

--------------------------------

### Configuration via config.json

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Configure model-specific API keys for the OpenAI provider using the `config.json` file.

```APIDOC
## Configuration via config.json

### Description
Configure model-specific API keys for the OpenAI provider by modifying the `config.json` file. This allows for persistent configuration of different keys for different models.

### Method
File Configuration

### Endpoint
N/A (File-based configuration)

### Parameters
#### Request Body (config.json structure)
- **providers** (object) - Required - Contains configurations for different model providers.
  - **openai** (object) - Required - Configuration for the OpenAI provider.
    - **keys** (array) - Required - A list of key configurations for OpenAI models.
      - **name** (string) - Required - A unique name for the API key.
      - **value** (string) - Required - The actual API key or environment variable reference (e.g., "env.OPENAI_API_KEY").
      - **models** (array of strings) - Required - A list of model identifiers this key is associated with.
      - **weight** (number) - Optional - A weight for load balancing if multiple keys are assigned to the same models.

### Request Example
```json
{
    "providers": {
        "openai": {
            "keys": [
                {
                    "name": "openai-key-1",
                    "value": "env.OPENAI_API_KEY",
                    "models": ["gpt-4o", "gpt-4o-mini"],
                    "weight": 1.0
                },
                {
                    "name": "openai-key-2",
                    "value": "env.OPENAI_API_KEY_PREMIUM",
                    "models": ["o1-preview", "o1-mini"],
                    "weight": 1.0
                }
            ]
        }
    }
}
```

### Response
N/A (Configuration is applied on service restart or reload)
```

--------------------------------

### Using config.json for Custom Base URL

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Demonstrates how to configure a custom base URL for a provider by modifying the `config.json` file.

```APIDOC
## config.json Structure

### Description
This file allows for the configuration of various API providers, including setting custom network configurations like base URLs.

### Method
N/A (File Configuration)

### Endpoint
N/A (File Configuration)

### Parameters
#### Request Body (config.json structure)
- **providers** (object) - Required - Contains configurations for different providers.
  - **[provider_name]** (object) - Required - Configuration for a specific provider (e.g., "openai").
    - **keys** (array) - Required - A list of API keys for the provider.
      - **name** (string) - Required - The name of the key.
      - **value** (string) - Required - The API key value or environment variable.
      - **models** (array) - Optional - A list of models associated with this key.
      - **weight** (number) - Optional - The weight of this key for load balancing.
    - **network_config** (object) - Optional - Network-specific configurations.
      - **base_url** (string) - Required if overriding - The custom base URL for the provider.

### Request Example
```json
{
    "providers": {
        "openai": {
            "keys": [
                {
                    "name": "openai-key-1",
                    "value": "env.OPENAI_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "network_config": {
                "base_url": "http://localhost:8000/v1"
            }
        }
    }
}
```

### Response
N/A (File Configuration)
```

--------------------------------

### Install and Configure Gemini CLI (Bash)

Source: https://docs.getbifrost.ai/quickstart/gateway/cli-agents

Steps to install and configure the Gemini CLI using npm. This involves setting environment variables for the Gemini API key and the Google Gemini base URL, which Bifrost manages. The guide also mentions selecting 'Use Gemini API Key' during the CLI prompt for authentication.

```bash
# Install Gemini CLI
npm install -g @google/gemini-cli

# Configure Environment Variables
export GEMINI_API_KEY=dummy-key  # Handled by Bifrost (only set when using virtual keys)
export GOOGLE_GEMINI_BASE_URL=http://localhost:8080/genai

# Run Gemini CLI
gemini
```

--------------------------------

### POST /api/providers

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Configure model-specific API keys for the OpenAI provider using the API.

```APIDOC
## POST /api/providers

### Description
Configure model-specific API keys for the OpenAI provider. This allows you to assign different keys to different models, managing access and billing separately.

### Method
POST

### Endpoint
`/api/providers`

### Parameters
#### Request Body
- **provider** (string) - Required - The name of the model provider (e.g., "openai").
- **keys** (array) - Required - A list of key configurations.
  - **name** (string) - Required - A unique name for the API key.
  - **value** (string) - Required - The actual API key or environment variable reference (e.g., "env.OPENAI_API_KEY").
  - **models** (array of strings) - Required - A list of model identifiers this key is associated with.
  - **weight** (number) - Optional - A weight for load balancing if multiple keys are assigned to the same models.

### Request Example
```json
{
    "provider": "openai",
    "keys": [
        {
            "name": "openai-key-1",
            "value": "env.OPENAI_API_KEY",
            "models": ["gpt-4o", "gpt-4o-mini"],
            "weight": 1.0
        },
        {
            "name": "openai-key-2",
            "value": "env.OPENAI_API_KEY_PREMIUM",
            "models": ["o1-preview", "o1-mini"],
            "weight": 1.0
        }
    ]
}
```

### Response
#### Success Response (200)
- **message** (string) - A confirmation message indicating successful configuration.

#### Response Example
```json
{
    "message": "Provider configuration updated successfully."
}
```
```

--------------------------------

### Install and Configure Claude Code with Bifrost

Source: https://docs.getbifrost.ai/quickstart/gateway/cli-agents

These bash commands demonstrate how to install Claude Code globally using npm and configure environment variables to point its base URL to Bifrost. This allows Claude Code to leverage Bifrost's universal model access and advanced features.

```bash
npm install -g @anthropic-ai/claude-code
```

```bash
export ANTHROPIC_API_KEY=dummy-key  # Handled by Bifrost (only set when using virtual keys)
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic
```

```bash
claude
```

--------------------------------

### Configure HTTP and SOCKS5 Proxies via API

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Demonstrates how to configure HTTP proxy for OpenAI and authenticated SOCKS5 proxy for Anthropic using cURL commands. This is useful for corporate environments or regional access requirements.

```bash
# HTTP proxy for OpenAI
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{ 
    "provider": "openai",
    "keys": [
        {
            "name": "openai-key-1",
            "value": "env.OPENAI_API_KEY",
            "models": [],
            "weight": 1.0
        }
    ],
    "proxy_config": {
        "type": "http",
        "url": "http://localhost:8000"
    }
}'

# SOCKS5 proxy with authentication for Anthropic
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{ 
    "provider": "anthropic",
    "keys": [
        {
            "name": "anthropic-key-1",
            "value": "env.ANTHROPIC_API_KEY",
            "models": [],
            "weight": 1.0
        }
    ],
    "proxy_config": {
        "type": "socks5",
        "url": "http://localhost:8000",
        "username": "user",
        "password": "password"
    }
}'
```

--------------------------------

### Configure Provider Concurrency and Buffer Size in config.json

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This JSON configuration file allows you to define custom concurrency and buffer sizes for AI model providers. The example shows settings for OpenAI and Anthropic, specifying worker concurrency and queue buffer size. This method is useful for persistent configurations.

```json
{
    "providers": {
        "openai": {
            "keys": [
                {
                    "name": "openai-key-1",
                    "value": "env.OPENAI_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "concurrency_and_buffer_size": {
                "concurrency": 100,
                "buffer_size": 500
            }
        },
        "anthropic": {
            "keys": [
                {
                    "name": "anthropic-key-1",
                    "value": "env.ANTHROPIC_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "concurrency_and_buffer_size": {
                "concurrency": 25,
                "buffer_size": 100
            }
        }
    }
}
```

--------------------------------

### Configure Google Vertex AI via config.json

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This JSON configuration file outlines the setup for Google Vertex AI. It mirrors the API configuration, specifying the provider, API key, models, weights, and Vertex AI specific configurations like project ID, region, and authentication credentials. This is useful for static configuration management.

```json
{
    "providers": {
        "vertex": {
            "keys": [
                {
                    "name": "vertex-key-1",
                    "value": "env.VERTEX_API_KEY",
                    "models": ["gemini-pro", "gemini-pro-vision"],
                    "weight": 1.0,
                    "vertex_key_config": {
                        "project_id": "env.VERTEX_PROJECT_ID",
                        "region": "us-central1",
                        "auth_credentials": "env.VERTEX_CREDENTIALS",
                        "deployments": {
                            "fine-tuned-gemini-2.5-pro": "123456789"
                        }
                    }
                }
            ]
        }
    }
}
```

--------------------------------

### Run Qwen Code CLI

Source: https://docs.getbifrost.ai/quickstart/gateway/cli-agents

Executes the Qwen Code command-line interface. This command assumes that Qwen Code has been installed and configured correctly.

```bash
qwen
```

--------------------------------

### Making Requests to Bifrost AI

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This section demonstrates how to send a request to a specific model provider (e.g., OpenAI's GPT-4o Mini) using the Bifrost API. Bifrost automatically formats the request according to the provider's specifications.

```APIDOC
## POST /v1/chat/completions

### Description
Sends a chat completion request to a configured model provider.

### Method
POST

### Endpoint
`/v1/chat/completions`

### Parameters
#### Query Parameters
None

#### Request Body
- **model** (string) - Required - The identifier of the model to use (e.g., `"openai/gpt-4o-mini"`).
- **messages** (array) - Required - A list of message objects, where each object has a `role` (string) and `content` (string).

### Request Example
```json
{
    "model": "openai/gpt-4o-mini",
    "messages": [
        {"role": "user", "content": "Hello!"}
    ]
}
```

### Response
#### Success Response (200)
Details of the chat completion response from the model provider.

#### Response Example
```json
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hi there! How can I help you today?"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 20,
    "total_tokens": 30
  }
}
```
```

--------------------------------

### Configure Providers via config.json

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This section shows how to configure multiple AI model providers by editing the `config.json` file.

```APIDOC
## config.json Structure

### Description
Defines the structure for the `config.json` file to configure AI model providers.

### Method
File Configuration

### Endpoint
N/A

### Parameters
#### Request Body
- **providers** (object) - Required - Contains configurations for all AI providers.
  - **[provider_name]** (object) - Required - Configuration for a specific provider (e.g., "openai", "anthropic").
    - **keys** (array) - Required - A list of API keys or authentication credentials for the provider.
      - **name** (string) - Required - A unique name for the key.
      - **value** (string) - Required - The actual key value or reference (e.g., environment variable).
      - **models** (array) - Optional - A list of specific models supported by this key.
      - **weight** (number) - Optional - A weight for load balancing or priority (defaults to 1.0).
    - **network_config** (object) - Optional - Network-related configurations for the provider.
      - **base_url** (string) - Optional - The base URL for the provider's API endpoint (e.g., for self-hosted models).
      - **default_request_timeout_in_seconds** (integer) - Optional - The default timeout for requests to this provider.
    - **custom_provider_config** (object) - Optional - Custom configurations specific to the provider type.
      - **base_provider_type** (string) - Optional - The underlying provider type if this is a wrapper (e.g., "openai").
      - **allowed_requests** (object) - Optional - Specifies which API endpoints are allowed for this provider.
        - **chat_completion** (boolean) - Optional - Whether chat completion is allowed.
        - **chat_completion_stream** (boolean) - Optional - Whether streaming chat completion is allowed.

### Request Example
```json
{
    "providers": {
        "openai": {
            "keys": [
                {
                    "name": "openai-key",
                    "value": "env.OPENAI_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ]
        },
        "anthropic": {
            "keys": [
                {
                    "name": "anthropic-key",
                    "value": "env.ANTHROPIC_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ]
        },
        "vllm-local": {
            "keys": [
                {
                    "name": "vllm-key",
                    "value": "dummy",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "network_config": {
                "base_url": "http://vllm-endpoint:8000",
                "default_request_timeout_in_seconds": 60
            },
            "custom_provider_config": {
                "base_provider_type": "openai",
                "allowed_requests": {
                    "chat_completion": true,
                    "chat_completion_stream": true
                }
            }
        }
    }
}
```

### Response
N/A (This is a configuration file, not an API endpoint.)
```

--------------------------------

### Configure HTTP and SOCKS5 Proxies in config.json

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Shows the JSON structure for configuring HTTP proxy for OpenAI and authenticated SOCKS5 proxy for Anthropic within a `config.json` file. This method is suitable for persistent configurations.

```json
{
    "providers": {
        "openai": {
            "keys": [
                {
                    "name": "openai-key-1",
                    "value": "env.OPENAI_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "proxy_config": {
                "type": "http",
                "url": "http://localhost:8000"
            }
        },
        "anthropic": {
            "keys": [
                {
                    "name": "anthropic-key-1",
                    "value": "env.ANTHROPIC_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "proxy_config": {
                "type": "socks5",
                "url": "http://localhost:8000",
                "username": "user",
                "password": "password"
            }
        }
    }
}
```

--------------------------------

### Provider-Specific Authentication

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Configure enterprise cloud providers like Azure, AWS Bedrock, and Google Vertex with platform-specific authentication details beyond standard API keys.

```APIDOC
## Provider-Specific Authentication

### Description
Enterprise cloud providers require additional configuration beyond API keys. Configure Azure, AWS Bedrock, and Google Vertex with platform-specific authentication details.

### Azure
Azure supports two authentication methods:

(Further details for Azure authentication methods would follow here, e.g., Service Principal, Managed Identity, etc.)

### AWS Bedrock
(Details for AWS Bedrock authentication would follow here, e.g., IAM roles, access keys.)

### Google Vertex AI
(Details for Google Vertex AI authentication would follow here, e.g., Service Account keys, impersonation.)
```

--------------------------------

### Raw Response Structure

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

When 'Include Raw Response' is enabled, the original provider response is appended to the output under the `extra_fields.raw_response` key.

```APIDOC
## Response Structure with Raw Response

### Description
When the `send_back_raw_response` flag is enabled for a provider, the API response will include an `extra_fields` object containing the original response from the provider under the `raw_response` key.

### Endpoint
(Applies to any endpoint that returns provider data, e.g., /chat/completions)

### Response Example (Success)
```json
{
    "choices": [...],
    "usage": {...},
    "extra_fields": {
        "provider": "openai",
        "raw_response": {
            // Original OpenAI response content here
        }
    }
}
```

### Notes
- The structure of `raw_response` will vary depending on the specific provider used.
- This feature is primarily for debugging and advanced use cases where direct access to provider-specific details is required.
```

--------------------------------

### Enable Raw Response

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This endpoint allows you to configure providers to include the raw response from the original provider. This is useful for debugging and accessing provider-specific metadata.

```APIDOC
## POST /api/providers

### Description
Configures a provider to include the raw response from the original provider in the API output. This is useful for debugging and accessing provider-specific metadata.

### Method
POST

### Endpoint
/api/providers

### Parameters
#### Request Body
- **provider** (string) - Required - The name of the provider (e.g., "openai").
- **keys** (array) - Required - A list of API keys for the provider.
  - **name** (string) - Required - The name of the key.
  - **value** (string) - Required - The value of the API key (can be an environment variable).
  - **models** (array) - Optional - A list of models associated with this key.
  - **weight** (number) - Optional - The weight of this key for load balancing.
- **send_back_raw_response** (boolean) - Optional - If true, the raw provider response will be included.

### Request Example
```json
{
    "provider": "openai",
    "keys": [
        {
            "name": "openai-key-1",
            "value": "env.OPENAI_API_KEY",
            "models": [],
            "weight": 1.0
        }
    ],
    "send_back_raw_response": true
}
```

### Response
#### Success Response (200)
- **message** (string) - A confirmation message indicating the provider configuration was updated.

#### Response Example
```json
{
    "message": "Provider configuration updated successfully."
}
```

### Error Handling
- **400 Bad Request**: If the request body is invalid or missing required fields.
- **500 Internal Server Error**: If there is a server-side error during configuration.
```

--------------------------------

### Send Back Raw Request

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Configure Bifrost to include the original request sent to the provider alongside Bifrost's response. This is useful for debugging request transformations and verifying what was actually sent to the provider.

```APIDOC
## Send Back Raw Request

### Description
Include the original request sent to the provider alongside Bifrost's response. Useful for debugging request transformations and verifying what was actually sent to the provider.

### Method
POST

### Endpoint
`/api/providers`

### Parameters
#### Request Body
- **provider** (string) - Required - The name of the provider (e.g., "openai").
- **keys** (array) - Required - A list of API keys for the provider.
  - **name** (string) - Required - The name of the key.
  - **value** (string) - Required - The value of the key (e.g., "env.OPENAI_API_KEY").
  - **models** (array) - Optional - List of models associated with this key.
  - **weight** (number) - Optional - The weight of this key for load balancing.
- **send_back_raw_request** (boolean) - Required - Set to `true` to include the raw request.

### Request Example
```json
{
    "provider": "openai",
    "keys": [
        {
            "name": "openai-key-1",
            "value": "env.OPENAI_API_KEY",
            "models": [],
            "weight": 1.0
        }
    ],
    "send_back_raw_request": true
}
```

### Response
#### Success Response (200)
- **choices** (array) - The response choices from the provider.
- **usage** (object) - Usage statistics.
- **extra_fields** (object) - Additional fields.
  - **provider** (string) - The name of the provider.
  - **raw_request** (object) - The original request sent to the provider.

#### Response Example
```json
{
    "choices": [...],
    "usage": {...},
    "extra_fields": {
        "provider": "openai",
        "raw_request": {
            // Original request sent to OpenAI here
        }
    }
}
```

<Tip>
You can enable both `send_back_raw_request` and `send_back_raw_response` together to see the complete request-response cycle for debugging purposes.
</Tip>
```

--------------------------------

### Configure OpenAI Retries via API

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This snippet demonstrates how to configure network settings for OpenAI, specifically setting up retry behavior with exponential backoff. It requires the provider name and network configuration parameters.

```bash
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{ \
    "provider": "openai", \
    "keys": [ \
        { \
            "name": "openai-key-1", \
            "value": "env.OPENAI_API_KEY", \
            "models": [], \
            "weight": 1.0 \
        } \
    ], \
    "network_config": { \
        "max_retries": 5, \
        "retry_backoff_initial_ms": 1, \
        "retry_backoff_max_ms": 10000 \
    } \
}'
```

--------------------------------

### Environment Variables for API Keys

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

Configure API keys for various providers by setting environment variables. Bifrost supports direct key values or references to environment variables using the `env.` prefix. Sensitive data is automatically redacted.

```APIDOC
## Environment Variables

### Description
Set environment variables to securely provide API keys for different AI providers. Bifrost can reference these variables or accept direct key values.

### Usage
Export your API keys as environment variables:

```bash
export OPENAI_API_KEY="your-openai-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"
export MISTRAL_API_KEY="your-mistral-api-key"
export CEREBRAS_API_KEY="your-cerebras-api-key"
export GROQ_API_KEY="your-groq-api-key"
export COHERE_API_KEY="your-cohere-api-key"
```

### Referencing Environment Variables
When configuring providers, use the `env.` prefix to reference environment variables:

```json
{
  "value": "env.VARIABLE_NAME"
}
```

### Direct Key Values
Alternatively, provide keys directly:

```json
{
  "value": "sk-proj-xxxxxxxxx"
}
```

### Security
All sensitive data, including API keys, is automatically redacted in GET requests and UI responses.
```

--------------------------------

### Bifrost Go Application Example

Source: https://docs.getbifrost.ai/quickstart/go-sdk/setting-up

A complete Go program demonstrating how to initialize Bifrost, define an account interface for provider configuration, and make a chat completion request to OpenAI. This example shows the basic structure for interacting with AI models via Bifrost.

```go
package main

import (
    "context"
    "fmt"
    "os"

    "github.com/maximhq/bifrost/core"
    "github.com/maximhq/bifrost/core/schemas"
)

type MyAccount struct{}

// Account interface needs to implement these 3 methods
func (a *MyAccount) GetConfiguredProviders() ([]schemas.ModelProvider, error) {
    return []schemas.ModelProvider{schemas.OpenAI}, nil
}

func (a *MyAccount) GetKeysForProvider(ctx *context.Context, provider schemas.ModelProvider) ([]schemas.Key, error) {
    if provider == schemas.OpenAI {
    return []schemas.Key{{ 
        Value:  os.Getenv("OPENAI_API_KEY"),
            Models: []string{}, // Keep Models empty to use any model
            Weight: 1.0,
        }}, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    if provider == schemas.OpenAI {
        // Return default config (can be customized for advanced use cases)
        return &schemas.ProviderConfig{
                NetworkConfig:            schemas.DefaultNetworkConfig,
                ConcurrencyAndBufferSize: schemas.DefaultConcurrencyAndBufferSize,
        }, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}

// Main function implement to initialize bifrost and make a request
func main() {
	client, initErr := bifrost.Init(context.Background(), schemas.BifrostConfig{
		Account: &MyAccount{},
	})
	if initErr != nil {
		panic(initErr)
	}
	defer client.Shutdown()

	messages := []schemas.ChatMessage{
		{
            Role:    schemas.ChatMessageRoleUser,
            Content: &schemas.ChatMessageContent{
                ContentStr: schemas.Ptr("Hello, Bifrost!"),
            },
        },
	}

	response, err := client.ChatCompletionRequest(context.Background(), &schemas.BifrostChatRequest{
		Provider: schemas.OpenAI,
		Model:    "gpt-4o-mini",
		Input:    messages,
	})

	if err != nil {
		panic(err)
	}

    fmt.Println("Response:", *response.Choices[0].Message.Content.ContentStr)
}
```

--------------------------------

### Configure Retries for OpenAI Provider in Go

Source: https://docs.getbifrost.ai/quickstart/go-sdk/provider-configuration

Configures retry behavior for handling temporary failures and rate limits for the OpenAI provider. This example sets up exponential backoff with up to 5 retries, starting with 1ms delay and capping at 10 seconds.

```go
func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
	switch provider {
	case schemas.OpenAI:
		return &schemas.ProviderConfig{
			NetworkConfig: schemas.NetworkConfig{
				MaxRetries:          5,
				RetryBackoffInitial: 1 * time.Millisecond,
				RetryBackoffMax:     10 * time.Second,
			},
			ConcurrencyAndBufferSize: schemas.DefaultConcurrencyAndBufferSize,
		}, nil
	}
	return nil, fmt.Errorf("provider %s not supported", provider)
}
```

--------------------------------

### Connect to MCP Server for External Tools with Go

Source: https://docs.getbifrost.ai/quickstart/go-sdk/tool-calling

Illustrates connecting to a Model Context Protocol (MCP) server using a Go client to grant AI models access to external services. This example shows how to configure the MCP client, specify connection details, and define which tools to execute from the server.

```go
client, initErr := bifrost.Init(context.Background(), schemas.BifrostConfig{
	Account: &MyAccount{},
	MCPConfig: &schemas.MCPConfig{
		ClientConfigs: []schemas.MCPClientConfig{
			// Sample youtube-mcp server
			{
				Name:             "youtube-mcp",
				ConnectionType:   schemas.MCPConnectionTypeHTTP,
				ConnectionString: schemas.Ptr("http://your-youtube-mcp-url"),
				ToolsToExecute: []string{"*"}, // Allow all tools from this client
			},
		},
	},
})
if initErr != nil {
	panic(initErr)
}
defer client.Shutdown()

response, err := client.ChatCompletionRequest(context.Background(), &schemas.BifrostChatRequest{
	Provider: schemas.OpenAI,
	Model:    "gpt-4o-mini",
	Input: []schemas.ChatMessage{
		{
			Role: schemas.ChatMessageRoleUser,
			Content: &schemas.ChatMessageContent{
				ContentStr: schemas.Ptr("What do you see when you search for 'bifrost' on youtube?"),
			},
		},
	},
})

if err != nil {
	panic(err)
}

if response.Choices[0].Message.ChatAssistantMessage != nil && response.Choices[0].Message.ChatAssistantMessage.ToolCalls != nil {
	for _, toolCall := range response.Choices[0].Message.ChatAssistantMessage.ToolCalls {
		fmt.Printf("Tool call in response - %s: %s\n", *toolCall.ID, *toolCall.Function.Name)
		fmt.Printf("Tool call arguments - %s\n", toolCall.Function.Arguments)
	}
}

```

--------------------------------

### Specify Language for Transcription (Go)

Source: https://docs.getbifrost.ai/quickstart/go-sdk/multimodal

Improves transcription accuracy by specifying the source language of the audio. This is particularly useful for non-English audio or when dealing with specialized vocabulary. The example shows how to set the 'Language' parameter to 'es' for Spanish audio. It also includes an optional 'Prompt' to guide the transcription model.

```go
response, err := client.TranscriptionRequest(context.Background(), &schemas.BifrostTranscriptionRequest{
	Provider: schemas.OpenAI,
	Model:    "whisper-1",
	Input: &schemas.TranscriptionInput{
		File: audioData,
	},
	Params: &schemas.TranscriptionParameters{
		Language: schemas.Ptr("es"), // Spanish
		Prompt:   schemas.Ptr("This is a Spanish audio recording about technology."),
	},
})
```

--------------------------------

### Set Custom Concurrency and Buffer Size for OpenAI and Anthropic in Go

Source: https://docs.getbifrost.ai/quickstart/go-sdk/provider-configuration

Fine-tunes performance by adjusting worker concurrency and queue sizes per provider. This example gives OpenAI higher limits (100 workers, 500 queue) for high throughput, while Anthropic gets conservative limits to respect their rate limits.

```go
func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    switch provider {
    case schemas.OpenAI:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.DefaultNetworkConfig,
            ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
                MaxConcurrency: 100, // Max number of concurrent requests (no of workers)
                BufferSize:     500, // Max number of requests in the buffer (queue size)
            },
        }, nil
    case schemas.Anthropic:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.DefaultNetworkConfig,
            ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
                MaxConcurrency: 25,
                BufferSize:     100,
            },
        }, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}
```

--------------------------------

### Stream Chat Responses with cURL

Source: https://docs.getbifrost.ai/quickstart/gateway/streaming

This example shows how to get streaming chat completions from the Bifrost API. Using cURL, a POST request is made to the `/v1/chat/completions` endpoint. The request includes the model, a list of messages, and `stream: true`. The SSE response delivers partial content chunks in `data:` lines, allowing for real-time display of the AI's response, concluding with `[DONE]`.

```bash
curl --location 'http://localhost:8080/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{ \
    "model": "openai/gpt-4o-mini", \
    "messages": [ \
        {"role": "user", "content": "Tell me a story about a robot learning to paint"} \
    ], \
    "stream": true \
}'
```

--------------------------------

### Configure Google Vertex AI via API

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This snippet demonstrates how to configure Google Vertex AI using a cURL command. It specifies the provider as 'vertex' and includes essential keys such as API key, project ID, region, and authentication credentials. It also shows how to define model weights and deployments for fine-tuned models.

```bash
curl --location 'http://localhost:8080/api/providers' \
--header 'Content-Type: application/json' \
--data '{ \
    "provider": "vertex", \
    "keys": [ \
        { \
            "name": "vertex-key-1", \
            "value": "env.VERTEX_API_KEY", \
            "models": ["gemini-pro", "gemini-pro-vision"], \
            "weight": 1.0, \
            "vertex_key_config": { \
                "project_id": "env.VERTEX_PROJECT_ID", \
                "region": "us-central1", \
                "auth_credentials": "env.VERTEX_CREDENTIALS", \
                "deployments": { \
                    "fine-tuned-gemini-2.5-pro": "123456789" \
                } \
            } \
        } \
    ] \
}'
```

--------------------------------

### Run Bifrost Go Application

Source: https://docs.getbifrost.ai/quickstart/go-sdk/setting-up

Executes the Go application that integrates Bifrost. This command compiles and runs the `main.go` file, initiating the Bifrost client and sending a request to the configured AI provider. The expected output shows a response from the AI.

```bash
go run main.go
# Output: Response: Hello! I'm Bifrost, your AI model gateway...
```

--------------------------------

### OpenAI TTS Voice Configuration Example (Go)

Source: https://docs.getbifrost.ai/quickstart/go-sdk/streaming

Provides an example of how to specify a voice for text-to-speech synthesis using the Bifrost client with OpenAI's TTS models. This snippet shows the `VoiceConfig` structure and how to set a specific voice like 'nova'.

```go
// Different voice example
VoiceConfig: schemas.SpeechVoiceInput{
    Voice: bifrost.Ptr("nova"),
},

```

--------------------------------

### Install Bifrost Go Package

Source: https://docs.getbifrost.ai/quickstart/go-sdk/setting-up

Installs the core Bifrost package for Go applications. This is the first step to integrating Bifrost's unified AI interface into your project. Ensure you have Go installed and initialized your project with `go mod init`.

```bash
go mod init my-bifrost-app
go get github.com/maximhq/bifrost/core
```

--------------------------------

### Configure Custom Base URL via JSON Config

Source: https://docs.getbifrost.ai/quickstart/gateway/provider-configuration

This JSON configuration shows how to specify a custom base URL for an OpenAI provider within a `config.json` file. This method is suitable for persistent configurations and integrates seamlessly with Bifrost AI's setup. The `network_config` object contains the `base_url` parameter.

```json
{
    "providers": {
        "openai": {
            "keys": [
                {
                    "name": "openai-key-1",
                    "value": "env.OPENAI_API_KEY",
                    "models": [],
                    "weight": 1.0
                }
            ],
            "network_config": {
                "base_url": "http://localhost:8000/v1"
            }
        }
    }
}
```

--------------------------------

### Define and Use Custom Calculator Tool with Go

Source: https://docs.getbifrost.ai/quickstart/go-sdk/tool-calling

Demonstrates defining a custom tool schema for a calculator and using it in a chat completion request with a Go client. It shows how to structure the tool definition, including its name, description, and parameters, and how to process tool calls from the AI's response.

```go
// Define a tool for the calculator
calculatorTool := schemas.ChatTool{
	Type: schemas.ChatToolTypeFunction,
	Function: &schemas.ChatToolFunction{
		Name:        "calculator",
		Description: schemas.Ptr("A calculator tool"),
		Parameters: &schemas.ToolFunctionParameters{
			Type: "object",
			Properties: map[string]interface{}{
				"operation": map[string]interface{}{
					"type":        "string",
					"description": "The operation to perform",
					"enum":        []string{"add", "subtract", "multiply", "divide"},
				},
				"a": map[string]interface{}{
					"type":        "number",
					"description": "The first number",
				},
				"b": map[string]interface{}{
					"type":        "number",
					"description": "The second number",
				},
			},
			Required: []string{"operation", "a", "b"},
		},
	},
}

response, err := client.ChatCompletionRequest(context.Background(), &schemas.BifrostChatRequest{
	Provider: schemas.OpenAI,
	Model:    "gpt-4o-mini",
	Input: []schemas.ChatMessage{
		{
			Role: schemas.ChatMessageRoleUser,
			Content: &schemas.ChatMessageContent{
				ContentStr: schemas.Ptr("What is 2+2? Use the calculator tool."),
			},
		},
	},
	Params: &schemas.ChatParameters{
		Tools: []schemas.ChatTool{calculatorTool},
	},
})

if err != nil {
	panic(err)
}

if response.Choices[0].Message.ChatAssistantMessage != nil && response.Choices[0].Message.ChatAssistantMessage.ToolCalls != nil {
	for _, toolCall := range response.Choices[0].Message.ChatAssistantMessage.ToolCalls {
		fmt.Printf("Tool call in response - %s: %s\n", *toolCall.ID, *toolCall.Function.Name)
		fmt.Printf("Tool call arguments - %s\n", toolCall.Function.Arguments)
	}
}

```

--------------------------------

### Full Bifrost Setup with Custom Model Overrides (Bash)

Source: https://docs.getbifrost.ai/quickstart/gateway/cli-agents

Provides a complete bash script for setting up Bifrost AI with custom model overrides. This includes setting the ANTHROPIC_API_KEY and ANTHROPIC_BASE_URL, and optionally defining default models for Sonnet, Opus, and Haiku tiers. The configuration is intended to be added to shell startup files like ~/.bashrc or ~/.zshrc for persistence.

```bash
# Core Bifrost configuration
export ANTHROPIC_API_KEY=your-bifrost-virtual-key
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic

# Custom model tiers (optional)
export ANTHROPIC_DEFAULT_SONNET_MODEL="anthropic/claude-sonnet-4-5-20250929"
export ANTHROPIC_DEFAULT_OPUS_MODEL="openai/gpt-5"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="azure/claude-haiku-4-5"

# Start Claude Code
claude
```

--------------------------------

### List MCP Clients via API (Bash)

Source: https://docs.getbifrost.ai/quickstart/gateway/tool-calling

Retrieve a list of all configured MCP clients by sending a GET request to the `/api/mcp/clients` endpoint using `curl`. This is useful for verifying client configurations and ensuring they are correctly registered with the server running at `http://localhost:8080`.

```bash
curl --location 'http://localhost:8080/api/mcp/clients'
```
Tokens:
Raw