### Call Configured Routes (OpenAI Format)

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

This example shows how to call pre-configured routes within BricksLLM, which can handle automatic failover, caching, and load balancing. This specific example uses the OpenAI format for chat completions.

```bash
# Call a configured route (uses OpenAI format)
curl -X POST http://localhost:8002/api/routes/production/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
         "messages": [
            {"role": "system", "content": "You are a helpful assistant."}, 
            {"role": "user", "content": "What is the weather like today?"}
         ]
      }'
```

--------------------------------

### Example Usage: Allowed Chat Completion

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

Example of how to make a successful chat completion request using the created API key. The request is routed through BricksLLM to the OpenAI provider.

```APIDOC
## POST /api/providers/openai/v1/chat/completions

### Description
Forwards a chat completion request to the OpenAI provider using the configured API key and access controls.

### Method
POST

### Endpoint
/api/providers/openai/v1/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - The model to use for chat completion (must be allowed).
- **messages** (array) - Required - An array of message objects representing the conversation.

### Request Example
```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-3.5-turbo",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ]
      }'
```
```

--------------------------------

### Deploy BricksLLM with Docker Compose

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Instructions to deploy BricksLLM locally using Docker Compose, including cloning the repository, starting the services, and updating to the latest version.

```bash
git clone https://github.com/bricks-cloud/BricksLLM-Docker
cd BricksLLM-Docker
docker compose up -d
docker pull luyuanxin1995/bricksllm:latest
```

--------------------------------

### Configure OpenAI Node SDK to use BricksLLM

Source: https://github.com/bricks-cloud/bricksllm/blob/main/README.md

This example shows how to configure the OpenAI Node.js SDK to communicate with the BricksLLM proxy. By setting the `baseURL`, all requests will be redirected to the local BricksLLM instance.

```javascript
// OpenAI Node SDK v4
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: "some-secret-key", // key created earlier
  baseURL: "http://localhost:8002/api/providers/openai/v1", // redirect to us
});
```

--------------------------------

### Example: Unauthorized Model Request (GPT-4)

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

This example shows an attempt to use a model (gpt-4) that was not included in the provider's `allowedModels` list. The request is rejected with a 403 Forbidden error, demonstrating model-based access control.

```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-4",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ]
      }'
```

--------------------------------

### Integrate BricksLLM with OpenAI SDKs

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Demonstrates how to configure the OpenAI SDK in JavaScript and Python to route requests through the BricksLLM proxy. It includes examples for standard chat completion, streaming responses, and embedding generation.

```javascript
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: "my-secret-key",
  baseURL: "http://localhost:8002/api/providers/openai/v1"
});

async function chat() {
  const completion = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "system", content: "You are a helpful assistant." }, { role: "user", content: "Hello!" }],
    user: "user-123"
  });
  console.log(completion.choices[0].message.content);
}
```

```python
from openai import OpenAI

client = OpenAI(
    api_key="my-secret-key",
    base_url="http://localhost:8002/api/providers/openai/v1"
)

completion = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "system", "content": "What is Python?"}],
    user="user-123"
)
print(completion.choices[0].message.content)
```

--------------------------------

### Call Configured Routes for Embeddings

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

This example demonstrates how to use configured routes for embedding requests. It specifies the route and provides the input text to be embedded.

```bash
# Route for embeddings
curl -X POST http://localhost:8002/api/routes/embeddings/ada \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
         "input": "Text to embed using the configured route"
      }'
```

--------------------------------

### Configure Routes with Failover and Caching using BricksLLM API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Shows how to create and manage routes with advanced features like automatic failover between providers, request caching, and retry strategies. This includes examples for setting up primary and failover providers, configuring cache settings, and injecting request parameters.

```bash
# Create a route with OpenAI primary and Azure OpenAI failover
curl -X POST http://localhost:8001/api/routes \
   -H "Content-Type: application/json" \
   -d '{
         "name": "Production Chat Route",
         "path": "/production/chat/completions",
         "retryStrategy": "exponential",
         "cacheConfig": {
            "enabled": true,
            "ttl": "30s"
         },
         "steps": [
            {
               "provider": "openai",
               "model": "gpt-4",
               "retries": 2,
               "timeout": "30s",
               "retryInterval": "1s"
            },
            {
               "provider": "azure",
               "model": "gpt-4",
               "retries": 2,
               "timeout": "30s",
               "params": {
                  "deploymentId": "gpt4-deployment",
                  "apiVersion": "2024-02-15-preview"
               }
            }
         ],
         "keyIds": ["550e8400-e29b-41d4-a716-446655440000"]
      }'
```

```bash
# Create route with request parameter injection
curl -X POST http://localhost:8001/api/routes \
   -H "Content-Type: application/json" \
   -d '{
         "name": "Controlled Output Route",
         "path": "/controlled/chat",
         "cacheConfig": { "enabled": false },
         "steps": [
            {
               "provider": "openai",
               "model": "gpt-4",
               "retries": 1,
               "timeout": "60s",
               "requestParams": {
                  "max_tokens": 500,
                  "temperature": 0.7,
                  "top_p": 0.9
               }
            }
         ],
         "keyIds": ["550e8400-e29b-41d4-a716-446655440000"]
      }'
```

```bash
# List all routes
curl -X GET http://localhost:8001/api/routes
```

```bash
# Get specific route
curl -X GET http://localhost:8001/api/routes/9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb
```

```bash
# Delete route
curl -X DELETE http://localhost:8001/api/routes/9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb
```

--------------------------------

### Example: Unauthorized Path Request (Embeddings)

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

This example demonstrates an attempt to access the embeddings endpoint using an API key restricted to chat completions. The request is denied with a 403 Forbidden error, illustrating path-based access control.

```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/embeddings \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
        "input": "The food was delicious and the waiter...",
        "model": "text-embedding-ada-002",
        "encoding_format": "float"
    }'
```

--------------------------------

### Query Events and Reporting Data with BricksLLM API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Provides examples for querying proxy request events and aggregated usage metrics. This includes filtering events by key ID, time range, user ID, and tags, as well as retrieving time-series reporting data with custom increments and filters.

```bash
# Get events by key ID and time range
curl -X GET "http://localhost:8001/api/events?keyIds=550e8400-e29b-41d4-a716-446655440000&start=1718581437&end=1718667837"
```

```bash
# Get events V2 with advanced filtering
curl -X POST http://localhost:8001/api/v2/events \
   -H "Content-Type: application/json" \
   -d '{
         "keyIds": ["550e8400-e29b-41d4-a716-446655440000"],
         "userIds": ["john@example.com"],
         "tags": ["production"],
         "start": 1718581437,
         "end": 1718667837,
         "limit": 50,
         "offset": 0,
         "actions": ["allowed", "blocked", "redacted"],
         "costOrder": "desc",
         "returnCount": true
      }'
```

```bash
# Get aggregated metrics with time series
curl -X POST http://localhost:8001/api/reporting/events \
   -H "Content-Type: application/json" \
   -d '{
         "keyIds": ["550e8400-e29b-41d4-a716-446655440000"],
         "tags": ["production"],
         "start": 1699847171,
         "end": 1699933571,
         "increment": 3600,
         "filters": ["model", "keyId"]
      }'
```

--------------------------------

### Manage API Keys and Test Requests

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/pii_detection.md

Creates a Bricks API key linked to provider settings and policies. Includes examples of sending requests that trigger PII filtering rules resulting in 403 or 200 status codes.

```bash
curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My Secret Key",
	      "key": "my-secret-key",
	      "tags": ["org-1"],
          "settingIds": ["YOUR_SETTING_ID_FROM_STEP_TWO"],
          "policyId": "YOUR_POLICY_ID_FROM_STEP_THREE"
      }'
```

```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-3.5-turbo",
          "messages": [
              { "role": "user", "content": "My name is Spike." }
          ]
      }'
```

--------------------------------

### Send Request to Configured Route

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/openai_with_azure_openai_failover.md

Sends a chat completion request to the newly created route. This example demonstrates how to authenticate using the Bricks API key and format the request payload.

```bash
curl -X POST http://localhost:8002/api/routes/test/chat/completion \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ]
      }'
```

--------------------------------

### Admin API: List and Update Provider Settings

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Retrieve and modify existing LLM provider configurations using GET and PATCH requests to the Admin Server. Supports listing all settings, filtering by IDs, and performing partial updates on existing configurations.

```bash
# List all provider settings
curl -X GET "http://localhost:8001/api/provider-settings"

# List specific provider settings by IDs
curl -X GET "http://localhost:8001/api/provider-settings?ids=98daa3ae-961d-4253-bf6a-322a32fdca3d"

# Update provider setting (partial update)
curl -X PATCH http://localhost:8001/api/provider-settings/98daa3ae-961d-4253-bf6a-322a32fdca3d \
   -H "Content-Type: application/json" \
   -d '{ "name": "updated-provider-name", "allowedModels": ["gpt-4", "gpt-4-turbo", "gpt-3.5-turbo"] }'
```

--------------------------------

### Get Top Spending Keys - Reporting API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Retrieves a list of top spending keys within a specified time range and tags. Requires start and end timestamps, and allows filtering by tags and limiting the number of results.

```bash
curl -X POST http://localhost:8001/api/reporting/top-keys \
   -H "Content-Type: application/json" \
   -d '{ \
         "start": 1699847171, \
         "end": 1699933571, \
         "tags": ["production"], \
         "limit": 10, \
         "order": "desc" \
      }'
```

--------------------------------

### Example Usage: Forbidden Embedding Request

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

Demonstrates a forbidden request to the embeddings endpoint. This will result in a 403 error because the path is not included in the `allowedPaths` of the API key.

```APIDOC
## POST /api/providers/openai/v1/embeddings

### Description
Attempting to access the embeddings endpoint with a key restricted to chat completions will result in a 403 Forbidden error.

### Method
POST

### Endpoint
/api/providers/openai/v1/embeddings

### Request Example
```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/embeddings \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
        "input": "The food was delicious and the waiter...",
        "model": "text-embedding-ada-002",
        "encoding_format": "float"
    }'
```

### Response
#### Error Response (403)
- **error** (object) - Contains error details.
  - **code** (integer) - The error code (403).
  - **message** (string) - Description of the error.
  - **type** (string) - Type of error.

#### Response Example
```json
{
    "error": {
        "code": 403,
        "message": "[BricksLLM] path is not allowed",
        "type": ""
    }
}
```
```

--------------------------------

### Step 2: Create a Bricks API Key

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

Create a Bricks API key, associating it with the provider setting created in Step 1. Define rate limits, cost limits, and specific allowed paths and methods.

```APIDOC
## PUT /api/key-management/keys

### Description
Creates a new API key with specified configurations for rate limiting, cost control, and access permissions.

### Method
PUT

### Endpoint
/api/key-management/keys

### Parameters
#### Request Body
- **name** (string) - Required - A descriptive name for the API key.
- **key** (string) - Required - The actual API key string.
- **tags** (array) - Optional - A list of tags for organizing keys.
- **settingIds** (array) - Required - A list of provider setting IDs to associate with this key.
- **rateLimitOverTime** (integer) - Optional - The number of requests allowed within the rate limit unit.
- **rateLimitUnit** (string) - Optional - The unit of time for the rate limit (e.g., "m" for minute, "h" for hour).
- **costLimitInUsd** (number) - Optional - The maximum cost allowed in USD.
- **allowedPaths** (array) - Optional - A list of paths and methods that this key is allowed to access.
  - **path** (string) - Required - The endpoint path.
  - **method** (string) - Required - The HTTP method (e.g., "POST").

### Request Example
```json
{
  "name": "My Secret Key",
  "key": "my-secret-key",
  "tags": ["mykey"],
  "settingIds": ["ID_FROM_STEP_ONE"],
  "rateLimitOverTime": 2,
  "rateLimitUnit": "m",
  "costLimitInUsd": 0.25,
  "allowedPaths": [
    {
      "path": "/api/providers/openai/v1/chat/completions",
      "method": "POST"
    }
  ]
}
```
```

--------------------------------

### Create Bricks API Key for vLLM

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/vllm_integration.md

This command creates a Bricks API key, associating it with the vLLM provider settings created in the previous step. It configures rate limiting (2 requests per minute) and a cost limit (0.25 USD). The `settingIds` field must be populated with the ID obtained from the provider creation step.

```bash
curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My vLLM Key",
	      "key": "my-vllm-key",
	      "tags": ["mykey"],
        "settingIds": ["ID_FROM_STEP_ONE"],
        "rateLimitOverTime": 2,
        "rateLimitUnit": "m",
        "costLimitInUsd": 0.25
      }'
```

--------------------------------

### Proxy Request with User ID

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/access_control_using_user_id.md

Example of how to make a proxied request to an AI model endpoint, including the `user` field in the request body to enforce access control.

```APIDOC
## POST /api/providers/{provider}/{version}/chat/completions

### Description
Send a request to an AI model provider through the BricksLLM proxy. The `user` field in the request body is crucial for applying user-specific access controls.

### Method
POST

### Endpoint
/api/providers/{provider}/{version}/chat/completions

### Parameters
#### Path Parameters
- **provider** (string) - Required - The AI provider (e.g., "openai").
- **version** (string) - Required - The API version (e.g., "v1").

#### Request Body
- **model** (string) - Required - The AI model to use (e.g., "gpt-4").
- **messages** (array of objects) - Required - The conversation messages.
  - **role** (string) - Required - The role of the message sender (e.g., "system", "user").
  - **content** (string) - Required - The message content.
- **user** (string) - Required - The user ID defined during user creation, used for access control.

### Request Example (Unauthorized)
```json
{
  "model": "gpt-3.5-turbo",
  "messages": [
    {
      "role": "system",
      "content": "hi"
    }
  ],
  "user": "my-user-id"
}
```

### Response
#### Error Response (401)
- **error** (string) - Description of the error, e.g., "Unauthorized access."

#### Success Response (200)
- **choices** (array) - The model's response choices.
- **usage** (object) - Information about token usage.

#### Response Example (Success)
```json
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 20,
    "total_tokens": 30
  }
}
```
```

--------------------------------

### Step 1: Create a Provider

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

Configure an OpenAI provider with specific model allowances. The response will contain an ID to be used in the next step.

```APIDOC
## PUT /api/provider-settings

### Description
Creates or updates provider settings, including API keys and allowed models.

### Method
PUT

### Endpoint
/api/provider-settings

### Parameters
#### Request Body
- **provider** (string) - Required - The name of the provider (e.g., "openai").
- **setting** (object) - Required - Settings for the provider, including the API key.
  - **apikey** (string) - Required - The API key for the provider.
- **allowedModels** (array) - Optional - A list of models allowed for this provider setting.

### Request Example
```json
{
  "provider": "openai",
  "setting": {
    "apikey": "YOUR_OPENAI_KEY"
  },
  "allowedModels": ["gpt-3.5-turbo"]
}
```

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the provider setting.

#### Response Example
```json
{
  "id": "some-provider-setting-id"
}
```
```

--------------------------------

### Manage Keys and Policies with BricksLLM API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Demonstrates how to associate a policy with a key, list policies by tags, and update existing policies using the BricksLLM API. These operations are crucial for access control and data protection configurations.

```bash
curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
         "name": "PII Protected Key",
         "key": "pii-protected-key",
         "settingIds": ["98daa3ae-961d-4253-bf6a-322a32fdca3d"],
         "policyId": "9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb"
      }'
```

```bash
curl -X GET "http://localhost:8001/api/policies?tags=production"
```

```bash
curl -X PATCH http://localhost:8001/api/policies/9e6e8b27-2ce0-4ef0-bdd7-1ed3916592eb \
   -H "Content-Type: application/json" \
   -d '{
         "config": {
            "rules": {
               "name": "block",
               "address": "block",
               "phone": "allow_but_redact"
            }
         }
      }'
```

--------------------------------

### Get Custom IDs Associated with a Key - Reporting API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Retrieves custom identifiers associated with a given API key. This can be used for custom tracking or integration purposes.

```bash
curl -X GET "http://localhost:8001/api/reporting/custom-ids?keyId=550e8400-e29b-41d4-a716-446655440000"
```

--------------------------------

### Example Usage: Forbidden Model Request

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

Illustrates a forbidden request attempting to use a model (`gpt-4`) that is not allowed by the provider settings associated with the API key. This results in a 403 error.

```APIDOC
## POST /api/providers/openai/v1/chat/completions (with disallowed model)

### Description
Attempting to use a model not specified in the provider settings with the API key will result in a 403 Forbidden error.

### Method
POST

### Endpoint
/api/providers/openai/v1/chat/completions

### Request Example
```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-4",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ]
      }'
```

### Response
#### Error Response (403)
- **error** (object) - Contains error details.
  - **code** (integer) - The error code (403).
  - **message** (string) - Description of the error.
  - **type** (string) - Type of error.

#### Response Example
```json
{
    "error": {
        "code": 403,
        "message": "[BricksLLM] model is not allowed",
        "type": ""
    }
}
```
```

--------------------------------

### Configure Provider Settings

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/access_control_using_user_id.md

Registers an AI provider (e.g., OpenAI) with BricksLLM to obtain a setting ID required for subsequent key and user management.

```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_API_KEY"
          }
      }'
```

--------------------------------

### Get User IDs Associated with a Key - Reporting API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Fetches user IDs linked to a specific API key. This is useful for tracking usage by individual users associated with a particular key.

```bash
curl -X GET "http://localhost:8001/api/reporting/users-ids?keyId=550e8400-e29b-41d4-a716-446655440000"
```

--------------------------------

### Test User Access via Gateway

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/access_control_using_user_id.md

Demonstrates how to route requests through the BricksLLM gateway using the user ID to verify access control policies, such as model restrictions.

```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-3.5-turbo",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ],
          "user": "my-user-id"
    }'
```

```bash
curl -X POST http://localhost:8002/api/providers/openai/v1/chat/completions \
   -H "Authorization: Bearer my-secret-key" \
   -H "Content-Type: application/json" \
   -d '{
          "model": "gpt-4",
          "messages": [
              {
                  "role": "system",
                  "content": "hi"
              }
          ],
          "user": "my-user-id"
    }'
```

--------------------------------

### Configure OpenAI Provider with Allowed Models

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

This step involves creating a provider configuration for OpenAI, specifying allowed models, and setting the API key. The response provides an ID that is used in the next step.

```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_KEY"
          },
          "allowedModels": ["gpt-3.5-turbo"]
      }'
```

--------------------------------

### Configure BricksLLM Environment Variables

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Lists the essential environment variables required to deploy BricksLLM, including database connections, Redis caching, gateway timeouts, and AWS integration for PII detection.

```bash
POSTGRESQL_HOSTS=localhost
POSTGRESQL_USERNAME=postgres
POSTGRESQL_PASSWORD=secret
REDIS_HOSTS=localhost
REDIS_PORT=6379
PROXY_TIMEOUT=600s
AMAZON_REGION=us-west-2
```

--------------------------------

### Deploy BricksLLM Environment

Source: https://github.com/bricks-cloud/bricksllm/blob/main/README.md

Commands to clone the repository and launch the BricksLLM infrastructure locally using Docker Compose, including Postgres and Redis dependencies.

```bash
git clone https://github.com/bricks-cloud/BricksLLM-Docker
cd BricksLLM-Docker
docker compose up -d
```

--------------------------------

### Anthropic Claude Messages API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

This section covers proxying requests to Anthropic's Claude models, supporting both direct API calls and requests routed through AWS Bedrock. It includes examples for the messages API and the legacy completion endpoint.

```bash
# Anthropic Claude messages (direct)
curl -X POST http://localhost:8002/api/providers/anthropic/v1/messages \
   -H "Authorization: Bearer my-anthropic-key" \
   -H "Content-Type: application/json" \
   -H "anthropic-version: 2023-06-01" \
   -d '{
         "model": "claude-3-5-sonnet-20241022",
         "max_tokens": 1024,
         "messages": [
            {"role": "user", "content": "Explain the theory of relativity in simple terms."}
         ]
      }'
```

```bash
# Anthropic via AWS Bedrock
curl -X POST http://localhost:8002/api/providers/bedrock/anthropic/v1/messages \
   -H "Authorization: Bearer my-bedrock-key" \
   -H "Content-Type: application/json" \
   -d '{
         "model": "anthropic.claude-3-sonnet-20240229-v1:0",
         "anthropic_version": "bedrock-2023-05-31",
         "max_tokens": 1024,
         "messages": [
            {"role": "user", "content": "What are the benefits of cloud computing?"}
         ]
      }'
```

```bash
# Legacy Anthropic completion endpoint
curl -X POST http://localhost:8002/api/providers/anthropic/v1/complete \
   -H "Authorization: Bearer my-anthropic-key" \
   -H "Content-Type: application/json" \
   -H "anthropic-version: 2023-06-01" \
   -d '{
         "model": "claude-2.1",
         "prompt": "\n\nHuman: Hello\n\nAssistant:",
         "max_tokens_to_sample": 100
      }'
```

--------------------------------

### Create Azure OpenAI Provider Configuration

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/openai_with_azure_openai_failover.md

Sets up the Azure OpenAI provider with its resource name and API key. Similar to the OpenAI provider, this uses a PUT request and returns a unique ID.

```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"azure",
          "setting": {
                "resourceName": "YOUR_AZURE_RESOURCE_NAME",
                "apikey": "YOUR_AZURE_API_KEY"
          }
      }'
```

--------------------------------

### Azure OpenAI Completions (Legacy)

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

This snippet demonstrates how to make legacy completion requests to Azure OpenAI models through the BricksLLM proxy. It requires an Azure OpenAI deployment and an API key.

```bash
curl -X POST "http://localhost:8002/api/providers/azure/openai/deployments/davinci-deployment/completions?api-version=2024-02-15-preview" \
   -H "Authorization: Bearer my-azure-key" \
   -H "Content-Type: application/json" \
   -d '{
         "prompt": "Once upon a time",
         "max_tokens": 100
      }'
```

--------------------------------

### Create Bricks API Key with Access Restrictions

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/granular_access_control.md

This step creates a Bricks API key with specific access controls, including rate limits, cost limits, and allowed API paths. It uses the `settingId` obtained from the provider configuration step.

```bash
curl -X PUT http://localhost:8001/api/key-management/keys \
   -H "Content-Type: application/json" \
   -d '{
	      "name": "My Secret Key",
	      "key": "my-secret-key",
	      "tags": ["mykey"],
        "settingIds": ["ID_FROM_STEP_ONE"],
        "rateLimitOverTime": 2,
        "rateLimitUnit": "m",
        "costLimitInUsd": 0.25,
        "allowedPaths": [{
            "path": "/api/providers/openai/v1/chat/completions",
            "method": "POST"
        }]
      }'
```

--------------------------------

### Configure LLM Provider Settings

Source: https://github.com/bricks-cloud/bricksllm/blob/main/README.md

Registers an LLM provider (e.g., OpenAI) with the BricksLLM gateway by sending a PUT request to the provider-settings endpoint.

```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_KEY"
          }
      }'
```

--------------------------------

### Create vLLM Provider Settings

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/vllm_integration.md

This command creates provider settings for vLLM by specifying the provider type and its configuration, including the deployment URL and API key. The response contains an ID that is used in subsequent steps. Ensure the URL does not end with a '/'.

```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
          "provider":"vllm",
          "setting": {
             "url": "YOUR_VLLM_DEPLOYMENT_URL",
             "apikey": "YOUR_VLLM_API_KEY"
          }
      }'
```

--------------------------------

### Create OpenAI Provider Configuration

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/openai_with_azure_openai_failover.md

Configures the OpenAI provider settings, including the API key. This step requires a PUT request to the provider-settings endpoint and returns an ID for the created provider.

```bash
curl -X PUT http://localhost:8001/api/provider-settings \
   -H "Content-Type: application/json" \
   -d '{
         "provider":"openai",
          "setting": {
             "apikey": "YOUR_OPENAI_KEY"
          }
      }'
```

--------------------------------

### PUT /api/provider-settings

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/vllm_integration.md

Registers a new vLLM provider configuration in the BricksLLM system.

```APIDOC
## PUT /api/provider-settings

### Description
Registers a new vLLM provider configuration to enable model access through the gateway.

### Method
PUT

### Endpoint
http://localhost:8001/api/provider-settings

### Request Body
- **provider** (string) - Required - Must be "vllm"
- **setting** (object) - Required - Contains "url" and "apikey" for the vLLM deployment.

### Request Example
{
  "provider": "vllm",
  "setting": {
    "url": "YOUR_VLLM_DEPLOYMENT_URL",
    "apikey": "YOUR_VLLM_API_KEY"
  }
}
```

--------------------------------

### POST /api/providers/openai/v1/chat/completions

Source: https://github.com/bricks-cloud/bricksllm/blob/main/README.md

Proxies chat completion requests to OpenAI through the BricksLLM gateway.

```APIDOC
## POST /api/providers/openai/v1/chat/completions

### Description
Routes a standard OpenAI chat completion request through the BricksLLM gateway. The gateway handles authentication and proxying to the underlying provider.

### Method
POST

### Endpoint
http://localhost:8002/api/providers/openai/v1/chat/completions

### Parameters
#### Request Body
- **model** (string) - Required - The ID of the model to use (e.g., gpt-3.5-turbo)
- **messages** (array) - Required - A list of messages comprising the conversation so far

### Request Example
{
  "model": "gpt-3.5-turbo",
  "messages": [
    {
      "role": "system",
      "content": "hi"
    }
  ]
}

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the chat completion
- **choices** (array) - List of completion choices

#### Response Example
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "choices": []
}
```

--------------------------------

### Configure PII Detection Policies

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Create security policies to detect and handle PII data. Supports both predefined rule sets and custom regex-based patterns.

```bash
curl -X POST http://localhost:8001/api/policies -H "Content-Type: application/json" -d '{"name": "PII Protection Policy", "tags": ["production", "compliance"], "config": {"rules": {"name": "block", "address": "allow_but_redact", "phone": "allow_but_redact", "email": "block", "ssn": "block", "credit_debit_number": "block"}}}'

curl -X POST http://localhost:8001/api/policies -H "Content-Type: application/json" -d '{"name": "Custom Pattern Policy", "tags": ["internal"], "regexConfig": {"rules": [{"definition": "INTERNAL-[A-Z]{3}-[0-9]{4}", "action": "block"}, {"definition": "SECRET_[A-Za-z0-9]+", "action": "allow_but_redact"}]}}'
```

--------------------------------

### POST /api/providers/vllm/v1/chat/completions

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/vllm_integration.md

Proxy endpoint to interact with the vLLM model using the BricksLLM managed key.

```APIDOC
## POST /api/providers/vllm/v1/chat/completions

### Description
Routes chat completion requests to the configured vLLM provider through the BricksLLM proxy.

### Method
POST

### Endpoint
http://localhost:8002/api/providers/vllm/v1/chat/completions

### Headers
- **Authorization** (string) - Required - Bearer <your-bricks-key>

### Request Example
{
  "model": "facebook/opt-125m",
  "messages": [
    {
      "role": "system",
      "content": "Where is San Francisco?"
    }
  ]
}
```

--------------------------------

### vLLM Chat and Completions API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

This section shows how to proxy requests to self-hosted vLLM deployments for both chat and standard completion tasks. It requires a running vLLM instance and an API key.

```bash
# vLLM chat completions
curl -X POST http://localhost:8002/api/providers/vllm/v1/chat/completions \
   -H "Authorization: Bearer my-vllm-key" \
   -H "Content-Type: application/json" \
   -d '{
         "model": "facebook/opt-125m",
         "messages": [
            {"role": "user", "content": "Tell me a joke"}
         ]
      }'
```

```bash
# vLLM completions
curl -X POST http://localhost:8002/api/providers/vllm/v1/completions \
   -H "Authorization: Bearer my-vllm-key" \
   -H "Content-Type: application/json" \
   -d '{
         "model": "facebook/opt-125m",
         "prompt": "The meaning of life is",
         "max_tokens": 50
      }'
```

--------------------------------

### Configure OpenAI SDK for vLLM via Bricks

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/vllm_integration.md

This JavaScript code snippet shows how to configure the OpenAI Node.js SDK (v4) to interact with vLLM models through the Bricks API. It involves setting the API key and redirecting the `baseURL` to the Bricks endpoint for vLLM.

```javascript
// OpenAI Node SDK v4
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: "my-vllm-key", // key created earlier
  baseURL: "http://localhost:8002/api/providers/vllm/v1", // redirect to us
});
```

--------------------------------

### Configure AWS Credentials

Source: https://github.com/bricks-cloud/bricksllm/blob/main/cookbook/pii_detection.md

Sets up required AWS credentials for PII detection. This can be done via shell environment variables or within a Docker environment configuration.

```bash
export AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEY
export AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID
```

```yaml
environment:
  - AWS_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_ACCESS_KEY
  - AWS_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY_ID
  - AMAZON_REGION=us-west-2
```

--------------------------------

### List OpenAI Models - BricksLLM API

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Retrieves a list of available OpenAI models accessible through the BricksLLM proxy.

```bash
curl -X GET http://localhost:8002/api/providers/openai/v1/models \
   -H "Authorization: Bearer my-secret-key"
```

--------------------------------

### POST /api/providers/azure/openai/deployments/{deployment}/chat/completions

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Proxies chat completion requests to specific Azure OpenAI deployments.

```APIDOC
## POST /api/providers/azure/openai/deployments/{deployment}/chat/completions

### Description
Proxies chat completion requests to specific Azure OpenAI deployments.

### Method
POST

### Endpoint
/api/providers/azure/openai/deployments/{deployment}/chat/completions

### Parameters
#### Path Parameters
- **deployment** (string) - Required - The Azure deployment name

#### Query Parameters
- **api-version** (string) - Required - Azure API version

### Request Example
{
  "messages": [{"role": "user", "content": "What is machine learning?"}]
}
```

--------------------------------

### Key Management API

Source: https://github.com/bricks-cloud/bricksllm/blob/main/README.md

This endpoint allows you to create and manage API keys for accessing BricksLLM, with options for rate limiting and cost control.

```APIDOC
## PUT /api/key-management/keys

### Description
Creates or updates an API key with specified configurations for rate limiting and cost control.

### Method
PUT

### Endpoint
/api/key-management/keys

### Parameters
#### Request Body
- **name** (string) - Required - A descriptive name for the API key.
- **key** (string) - Required - The actual API key string.
- **tags** (array of strings) - Optional - Tags to categorize the API key.
- **settingIds** (array of strings) - Required - IDs of the provider settings to associate with this key.
- **rateLimitOverTime** (integer) - Optional - The number of requests allowed over the specified time unit.
- **rateLimitUnit** (string) - Optional - The unit of time for the rate limit (e.g., "m" for minute, "h" for hour, "d" for day).
- **costLimitInUsd** (number) - Optional - The maximum amount in USD that can be spent using this key.

### Request Example
```json
{
  "name": "My Secret Key",
  "key": "my-secret-key",
  "tags": ["mykey"],
  "settingIds": ["ID_FROM_STEP_FOUR"],
  "rateLimitOverTime": 2,
  "rateLimitUnit": "m",
  "costLimitInUsd": 0.25
}
```

### Response
#### Success Response (200)
- **message** (string) - A success message indicating the key was created or updated.

#### Response Example
```json
{
  "message": "API key created successfully."
}
```
```

--------------------------------

### Call Configured Routes

Source: https://context7.com/bricks-cloud/bricksllm/llms.txt

Use configured routes for automatic failover, caching, and load balancing across different LLM providers.

```APIDOC
## POST /api/routes/{route_name}/chat/completions

### Description
Calls a configured route for chat completions, leveraging failover, caching, and load balancing.

### Method
POST

### Endpoint
`/api/routes/{route_name}/chat/completions`

### Parameters
#### Path Parameters
- **route_name** (string) - Required - The name of the configured route (e.g., `production`).

#### Request Body
- **messages** (array) - Required - An array of message objects, following the OpenAI format.

### Request Example
```json
{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."}, 
    {"role": "user", "content": "What is the weather like today?"}
  ]
}
```

### Response
#### Success Response (200)
- **choices** (array) - The generated chat completion choices from the route.

#### Response Example
```json
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The weather today is sunny with a high of 75 degrees Fahrenheit."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 15,
    "total_tokens": 35
  }
}
```
```

```APIDOC
## POST /api/routes/{route_name}/embeddings

### Description
Calls a configured route for generating embeddings.

### Method
POST

### Endpoint
`/api/routes/{route_name}/embeddings`

### Parameters
#### Path Parameters
- **route_name** (string) - Required - The name of the configured route for embeddings (e.g., `embeddings`).

#### Request Body
- **input** (string or array) - Required - The text or list of texts to embed.

### Request Example
```json
{
  "input": "Text to embed using the configured route"
}
```

### Response
#### Success Response (200)
- **data** (array) - An array of embedding objects.

#### Response Example
```json
{
  "id": "emb-123",
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "embedding": [
        0.001, -0.002, ..., 0.005
      ],
      "index": 0
    }
  ],
  "model": "text-embedding-ada-002",
  "usage": {
    "prompt_tokens": 5,
    "total_tokens": 5
  }
}
```
```