### Database Migrations with sqlx-cli

Source: https://context7.com/traceloop/hub/llms.txt

Sets up the PostgreSQL database schema for Traceloop Hub's Database Mode using sqlx-cli. This involves installing the tool, setting the database connection URL, creating the database, and running migrations to create necessary tables. A setup script is also provided as an alternative. Verify migrations for schema integrity.

```bash
# Install sqlx-cli
cargo install sqlx-cli --no-default-features --features postgres

# Set database URL
export DATABASE_URL=postgresql://hub:password@localhost:5432/hub

# Create database
createdb hub

# Run migrations
sqlx migrate run

# Verify migrations
sqlx migrate info

# Or use the setup script
chmod +x scripts/setup-db.sh
./scripts/setup-db.sh

# The migrations create tables:
# - providers: LLM provider configurations
# - model_definitions: Model configurations
# - pipelines: Request processing pipelines
# - pipeline_plugins: Pipeline plugin configurations
```

--------------------------------

### SQLX CLI Installation and Migration (PostgreSQL)

Source: https://github.com/traceloop/hub/blob/main/README.md

Commands to install the `sqlx-cli` tool with PostgreSQL support and run database migrations for Traceloop Hub in database mode.

```bash
# Install sqlx-cli
# Run migrations
sqlx migrate run

# Use setup script for complete setup
./scripts/setup-db.sh
```

--------------------------------

### YAML Configuration Example

Source: https://github.com/traceloop/hub/blob/main/README.md

Example `config.yaml` file for Traceloop Hub in YAML mode. This configuration defines providers, models, and pipelines for simple deployments.

```yaml
providers:
  - key: openai
    type: openai
    api_key: sk-...

models:
  - key: gpt-4
    type: gpt-4
    provider: openai

pipelines:
  - name: chat
    type: Chat
    plugins:
      - ModelRouter:
          models: [gpt-4]
```

--------------------------------

### Listing and Managing LLM Providers via Management API (REST)

Source: https://context7.com/traceloop/hub/llms.txt

Demonstrates how to use `curl` to list all available LLM providers, retrieve details for a specific provider, and update an existing provider's configuration through the Traceloop Hub Management API. This includes examples for GET and PUT requests.

```bash
# List all providers
curl http://localhost:8080/api/v1/management/providers

# Get specific provider
curl http://localhost:8080/api/v1/management/providers/550e8400-e29b-41d4-a716-446655440000

# Update provider
curl -X PUT http://localhost:8080/api/v1/management/providers/550e8400-e29b-41d4-a716-446655440000 \
  -H "Content-Type: application/json" \
  -d '{ "enabled": false }'
```

--------------------------------

### Helm Chart Installation (Database Mode)

Source: https://github.com/traceloop/hub/blob/main/README.md

Command to install Traceloop Hub using Helm in Database mode. Enables the management component and configures the PostgreSQL database connection.

```bash
helm install hub ./helm \
  --set management.enabled=true \
  --set management.database.host=postgres \
  --set management.database.existingSecret=postgres-secret
```

--------------------------------

### Helm Installation with Custom Values

Source: https://context7.com/traceloop/hub/llms.txt

Installs the Traceloop Hub Helm chart using a custom values.yaml file. This allows for detailed configuration of deployment settings, including database connections, resource allocation, and ingress rules. Ensure the Helm chart and values.yaml are correctly structured.

```yaml
config:
  mode: database
database:
  host: postgres-primary.database.svc.cluster.local
  existingSecret: hub-database-secret
management:
  enabled: true
  service:
    type: LoadBalancer
gateway:
  replicas: 3
  resources:
    limits:
      cpu: 1000m
      memory: 1Gi
    requests:
      cpu: 500m
      memory: 512Mi
ingress:
  enabled: true
  className: nginx
  hosts:
    - host: hub.example.com
      paths:
        - path: /
          pathType: Prefix
```

```bash
helm install traceloop-hub ./helm -f values.yaml
```

--------------------------------

### Helm Chart Installation (YAML Mode)

Source: https://github.com/traceloop/hub/blob/main/README.md

Command to install Traceloop Hub using Helm in YAML mode. Assumes the Helm chart is available locally.

```bash
helm install hub ./helm
```

--------------------------------

### Database Mode Environment Variables Setup

Source: https://github.com/traceloop/hub/blob/main/README.md

Environment variables required to set up Traceloop Hub in database mode. These include the deployment mode and the PostgreSQL connection string.

```bash
HUB_MODE=database
DATABASE_URL=postgresql://user:pass@host:5432/db
```

--------------------------------

### Starting Traceloop Hub in Database Mode

Source: https://context7.com/traceloop/hub/llms.txt

Instructions for launching Traceloop Hub in database mode using Docker or Cargo. This mode enables dynamic configuration management via a PostgreSQL backend and a management API. It specifies ports for the LLM gateway and management API, along with database connection details and polling intervals.

```bash
# Start Database Mode
docker run -p 3000:3000 -p 8080:8080 \
  -e HUB_MODE=database \
  -e DATABASE_URL=postgresql://hub:password@postgres:5432/hub \
  -e DB_POLL_INTERVAL_SECONDS=30 \
  traceloop/hub

# Or with cargo
HUB_MODE=database \
DATABASE_URL=postgresql://hub:password@localhost:5432/hub \
cargo run

# The gateway polls the database every 30 seconds for config changes
# Port 3000: LLM Gateway (chat/completions, embeddings, etc.)
# Port 8080: Management API (providers, models, pipelines CRUD)
```

--------------------------------

### Creating LLM Providers via Management API (REST)

Source: https://context7.com/traceloop/hub/llms.txt

Examples of using `curl` to interact with the Traceloop Hub Management API to create new LLM provider configurations. It demonstrates creating providers with literal API keys, Kubernetes secret references, and environment variable references, specifying details like name, type, and configuration.

```bash
# Create OpenAI provider
curl -X POST http://localhost:8080/api/v1/management/providers \
  -H "Content-Type: application/json" \
  -d '{ "name": "my-openai", "provider_type": "openai", "config": { "api_key": { "type": "literal", "value": "sk-..." }, "organization_id": "org-..." }, "enabled": true }'

# Response:
# {
#   "id": "550e8400-e29b-41d4-a716-446655440000",
#   "name": "my-openai",
#   "provider_type": "openai",
#   "config": {...},
#   "enabled": true,
#   "created_at": "2025-01-15T10:30:00Z",
#   "updated_at": "2025-01-15T10:30:00Z"
# }

# Create provider with Kubernetes secret reference
curl -X POST http://localhost:8080/api/v1/management/providers \
  -H "Content-Type: application/json" \
  -d '{ "name": "anthropic-prod", "provider_type": "anthropic", "config": { "api_key": { "type": "kubernetes", "secret_name": "anthropic-credentials", "key": "api-key", "namespace": "production" } }, "enabled": true }'

# Create provider with environment variable reference
curl -X POST http://localhost:8080/api/v1/management/providers \
  -H "Content-Type: application/json" \
  -d '{ "name": "azure-openai", "provider_type": "azure", "config": { "api_key": { "type": "environment", "variable_name": "AZURE_OPENAI_KEY" }, "resource_name": "my-azure-resource", "api_version": "2024-02-15-preview" } }'
```

--------------------------------

### YAML Configuration Mode

Source: https://context7.com/traceloop/hub/llms.txt

Configure Traceloop Hub using a static YAML file. This mode is suitable for development and simple deployments. The example shows how to set up multiple LLM providers and define routing pipelines.

```APIDOC
## YAML Configuration Mode

Simple static configuration for development and straightforward deployments.

### Docker Run Command
```bash
docker run -p 3000:3000 \
  -v $(pwd)/config.yaml:/app/config.yaml \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  traceloop/hub
```

### Cargo Run Command
```bash
CONFIG_FILE_PATH=config.yaml cargo run
```

### Example `config.yaml`
```yaml
general:
  trace_content_enabled: true

providers:
  - key: openai
    type: openai
    api_key: ${OPENAI_API_KEY}
    base_url: "https://api.openai.com/v1"

  - key: anthropic
    type: anthropic
    api_key: ${ANTHROPIC_API_KEY}

  - key: azure
    type: azure
    api_key: ${AZURE_API_KEY}
    resource_name: "my-azure-resource"
    api_version: "2024-02-15-preview"

  - key: bedrock
    type: bedrock
    api_key: ""
    region: "us-east-1"
    use_iam_role: true
    inference_profile_id: "us"

  - key: vertexai
    type: vertexai
    api_key: ""
    project_id: "my-gcp-project"
    location: "us-central1"
    credentials_path: "/path/to/service-account.json"

models:
  - key: gpt-4
    type: gpt-4
    provider: openai

  - key: claude-3-5-sonnet
    type: claude-3-5-sonnet-20241022
    provider: anthropic

  - key: bedrock-claude
    type: "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
    provider: bedrock
    model_provider: "anthropic"
    model_version: "v2:0"

  - key: gemini-flash
    type: gemini-1.5-flash
    provider: vertexai

pipelines:
  - name: default
    type: chat
    plugins:
      - logging:
          level: info
      - tracing:
          endpoint: "https://api.traceloop.com/v1/traces"
          api_key: "your-traceloop-api-key"
      - model-router:
          models:
            - gpt-4
            - claude-3-5-sonnet
            - gemini-flash

  - name: embeddings
    type: embeddings
    plugins:
      - model-router:
          models:
            - textembedding-gecko
```
```

--------------------------------

### Deploy Traceloop Hub with Helm

Source: https://context7.com/traceloop/hub/llms.txt

These Helm commands demonstrate how to deploy Traceloop Hub to a Kubernetes cluster. It covers installation in 'YAML mode' with custom configuration and 'Database mode' with specific database connection settings, as well as upgrading an existing deployment.

```bash
# Install in YAML mode
helm install traceloop-hub ./helm \
  --set config.mode=yaml \
  --set config.yaml.content="$(cat config.yaml)"
```

```bash
# Install in Database mode
helm install traceloop-hub ./helm \
  --set config.mode=database \
  --set database.host=postgres.default.svc.cluster.local \
  --set database.port=5432 \
  --set database.name=hub \
  --set database.existingSecret=postgres-credentials \
  --set management.enabled=true \
  --set management.port=8080
```

```bash
# Upgrade deployment
helm upgrade traceloop-hub ./helm \
  --set image.tag=v0.7.1 \
  --set resources.limits.memory=2Gi
```

--------------------------------

### Docker Compose Deployment (YAML & Database Modes)

Source: https://context7.com/traceloop/hub/llms.txt

Sets up a multi-service deployment using Docker Compose, featuring Traceloop Hub in both YAML and Database modes. It includes a PostgreSQL service for the Database Mode, with health checks and volume management for data persistence. Ensure Docker and Docker Compose are installed.

```yaml
version: '3.8'

services:
  # YAML Mode deployment
  hub-yaml:
    image: traceloop/hub:latest
    ports:
      - "3000:3000"
    volumes:
      - ./config.yaml:/app/config.yaml
    environment:
      - RUST_LOG=info
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}

  # Database Mode deployment
  hub-database:
    image: traceloop/hub:latest
    ports:
      - "3001:3000"  # Gateway port
      - "8080:8080"  # Management API port
    environment:
      - HUB_MODE=database
      - DATABASE_URL=postgresql://hub:hubpassword@postgres:5432/hub
      - DB_POLL_INTERVAL_SECONDS=30
      - RUST_LOG=info
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped

  postgres:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=hub
      - POSTGRES_USER=hub
      - POSTGRES_PASSWORD=hubpassword
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U hub"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  postgres_data:
```

```bash
# Start services
docker-compose up -d

# View logs
docker-compose logs -f hub-database

# Stop services
docker-compose down
```

--------------------------------

### GET /api/v1/management/pipelines

Source: https://context7.com/traceloop/hub/llms.txt

Retrieves a list of all configured pipelines.

```APIDOC
## GET /api/v1/management/pipelines

### Description
Retrieves a list of all configured pipelines in the Traceloop Hub.

### Method
GET

### Endpoint
`/api/v1/management/pipelines`

### Response
#### Success Response (200)
- A list of pipeline objects.
```

--------------------------------

### YAML Configuration for Traceloop Hub

Source: https://context7.com/traceloop/hub/llms.txt

This YAML file defines the static configuration for Traceloop Hub, including general settings, provider configurations (OpenAI, Anthropic, Azure, Bedrock, VertexAI), model definitions, and pipeline setups. It's suitable for development and straightforward deployments.

```yaml
# config.yaml
general:
  trace_content_enabled: true

providers:
  - key: openai
    type: openai
    api_key: ${OPENAI_API_KEY}
    base_url: "https://api.openai.com/v1"

  - key: anthropic
    type: anthropic
    api_key: ${ANTHROPIC_API_KEY}

  - key: azure
    type: azure
    api_key: ${AZURE_API_KEY}
    resource_name: "my-azure-resource"
    api_version: "2024-02-15-preview"

  - key: bedrock
    type: bedrock
    api_key: ""
    region: "us-east-1"
    use_iam_role: true
    inference_profile_id: "us"

  - key: vertexai
    type: vertexai
    api_key: ""
    project_id: "my-gcp-project"
    location: "us-central1"
    credentials_path: "/path/to/service-account.json"

models:
  - key: gpt-4
    type: gpt-4
    provider: openai

  - key: claude-3-5-sonnet
    type: claude-3-5-sonnet-20241022
    provider: anthropic

  - key: bedrock-claude
    type: "us.anthropic.claude-3-5-sonnet-20241022-v2:0"
    provider: bedrock
    model_provider: "anthropic"
    model_version: "v2:0"

  - key: gemini-flash
    type: gemini-1.5-flash
    provider: vertexai

pipelines:
  - name: default
    type: chat
    plugins:
      - logging:
          level: info
      - tracing:
          endpoint: "https://api.traceloop.com/v1/traces"
          api_key: "your-traceloop-api-key"
      - model-router:
          models:
            - gpt-4
            - claude-3-5-sonnet
            - gemini-flash

  - name: embeddings
    type: embeddings
    plugins:
      - model-router:
          models:
            - textembedding-gecko

```

--------------------------------

### GET /api/v1/management/model-definitions

Source: https://context7.com/traceloop/hub/llms.txt

Retrieves a list of all model definitions.

```APIDOC
## GET /api/v1/management/model-definitions

### Description
Retrieves a list of all model definitions configured in the Traceloop Hub.

### Method
GET

### Endpoint
`/api/v1/management/model-definitions`

### Response
#### Success Response (200)
- A list of model definition objects.
```

--------------------------------

### GET /api/v1/management/model-definitions/key/{key}

Source: https://context7.com/traceloop/hub/llms.txt

Retrieves a specific model definition by its key.

```APIDOC
## GET /api/v1/management/model-definitions/key/{key}

### Description
Retrieves a specific model definition by its key.

### Method
GET

### Endpoint
`/api/v1/management/model-definitions/key/{key}`

### Parameters
#### Path Parameters
- **key** (string) - Required - The key of the model definition to retrieve.
```

--------------------------------

### Configure OpenTelemetry Tracing in Pipelines

Source: https://github.com/traceloop/hub/blob/main/README.md

This configuration snippet demonstrates how to enable OpenTelemetry tracing within the Traceloop Hub pipelines. It specifies the tracing endpoint (e.g., Jaeger) and an API key for authentication. This setup allows for detailed request tracing and monitoring.

```yaml
pipelines:
  - name: traced-chat
    type: Chat
    plugins:
      - Tracing:
          endpoint: http://jaeger:14268/api/traces
          api_key: your-key
      - ModelRouter:
          models: [gpt-4]
```

--------------------------------

### Cargo Build and Run Traceloop Hub

Source: https://github.com/traceloop/hub/blob/main/README.md

Provides instructions for building and running Traceloop Hub from source using Cargo. Covers both YAML Mode and Database Mode deployments, specifying environment variables for the latter.

```bash
# Clone and build
git clone https://github.com/traceloop/hub.git
cd hub
cargo build --release

# YAML Mode
./target/release/hub

# Database Mode  
HUB_MODE=database DATABASE_URL=postgresql://user:pass@host:5432/db ./target/release/hub
```

--------------------------------

### Rust Build and Run Commands

Source: https://github.com/traceloop/hub/blob/main/README.md

Common Cargo commands for building, testing, formatting, linting, and running Traceloop Hub in development. Includes specific commands for YAML and database modes.

```bash
# Build OSS version
cargo build

# Test
# Format
# Lint
# Run YAML mode
# Run database mode
HUB_MODE=database DATABASE_URL=postgresql://... cargo run
```

--------------------------------

### Google VertexAI Provider Configuration (YAML)

Source: https://github.com/traceloop/hub/blob/main/README.md

YAML configuration snippet for integrating the Google VertexAI provider. Requires project ID and location, and uses service account JSON or API key.

```yaml
providers:
  - key: vertexai
    type: vertexai
    project_id: your-project
    location: us-central1
    # Uses service account JSON or API key
```

--------------------------------

### GET /api/v1/management/model-definitions/{id}

Source: https://context7.com/traceloop/hub/llms.txt

Retrieves a specific model definition by its unique identifier.

```APIDOC
## GET /api/v1/management/model-definitions/{id}

### Description
Retrieves a specific model definition by its unique identifier.

### Method
GET

### Endpoint
`/api/v1/management/model-definitions/{id}`

### Parameters
#### Path Parameters
- **id** (string) - Required - The unique identifier of the model definition to retrieve.
```

--------------------------------

### Docker Deployment of Traceloop Hub

Source: https://github.com/traceloop/hub/blob/main/README.md

Demonstrates how to deploy Traceloop Hub using Docker. Supports both YAML mode for simple static configurations and Database mode for dynamic configurations, requiring specific environment variables and port mappings.

```bash
# YAML Mode (simple deployment)
docker run -p 3000:3000 -v $(pwd)/config.yaml:/app/config.yaml traceloop/hub

# Database Mode (with management API)
docker run -p 3000:3000 -p 8080:8080 \
  -e HUB_MODE=database \
  -e DATABASE_URL=postgresql://user:pass@host:5432/db \
  traceloop/hub
```

--------------------------------

### Running Traceloop Hub with Docker and Cargo

Source: https://context7.com/traceloop/hub/llms.txt

Commands to run Traceloop Hub using Docker or Cargo, demonstrating how to apply YAML configuration files and set environment variables for API keys. This covers both Docker-based and native Rust execution environments.

```bash
# Run with YAML config
docker run -p 3000:3000 \
  -v $(pwd)/config.yaml:/app/config.yaml \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  traceloop/hub

# Or with cargo
CONFIG_FILE_PATH=config.yaml cargo run
```

--------------------------------

### Create Pipelines using cURL

Source: https://context7.com/traceloop/hub/llms.txt

These cURL commands demonstrate the creation of different types of pipelines, such as 'chat' and 'embeddings'. Each pipeline can be configured with various plugins like logging, tracing, and model routing, specifying their types and configurations.

```bash
# Create chat pipeline
curl -X POST http://localhost:8080/api/v1/management/pipelines \
  -H "Content-Type: application/json" \
  -d '{
    "name": "production-chat",
    "pipeline_type": "chat",
    "plugins": [
      {
        "type": "logging",
        "config": {"level": "info"}
      },
      {
        "type": "tracing",
        "config": {
          "endpoint": "https://api.traceloop.com/v1/traces",
          "api_key": "your-key"
        }
      },
      {
        "type": "model-router",
        "config": {
          "models": ["gpt-4-turbo", "claude-3-5-sonnet"]
        }
      }
    ]
  }'
```

```bash
# Create embeddings pipeline
curl -X POST http://localhost:8080/api/v1/management/pipelines \
  -H "Content-Type: application/json" \
  -d '{
    "name": "embeddings-pipeline",
    "pipeline_type": "embeddings",
    "plugins": [
      {
        "type": "model-router",
        "config": {
          "models": ["textembedding-gecko", "text-embedding-3-large"]
        }
      }
    ]
  }'
```

--------------------------------

### Create Model Definition using cURL

Source: https://context7.com/traceloop/hub/llms.txt

These cURL commands show how to create new model definitions, specifying details like key, model type, provider ID, and deployment configurations. Supports various model providers like OpenAI, Azure, and Bedrock.

```bash
# Create model definition
curl -X POST http://localhost:8080/api/v1/management/model-definitions \
  -H "Content-Type: application/json" \
  -d '{
    "key": "gpt-4-turbo",
    "model_type": "gpt-4-turbo-preview",
    "provider_id": "550e8400-e29b-41d4-a716-446655440000",
    "deployment": null,
    "model_provider": null,
    "model_version": null
  }'
```

```bash
# Create Azure model with deployment
curl -X POST http://localhost:8080/api/v1/management/model-definitions \
  -H "Content-Type: application/json" \
  -d '{
    "key": "gpt-4-azure",
    "model_type": "gpt-4",
    "provider_id": "azure-provider-uuid",
    "deployment": "my-gpt4-deployment"
  }'
```

```bash
# Create Bedrock model
curl -X POST http://localhost:8080/api/v1/management/model-definitions \
  -H "Content-Type: application/json" \
  -d '{
    "key": "bedrock-claude",
    "model_type": "anthropic.claude-3-5-sonnet-20241022-v2:0",
    "provider_id": "bedrock-provider-uuid",
    "model_provider": "anthropic",
    "model_version": "v2:0"
  }'
```

--------------------------------

### Environment Variables for Runtime Configuration

Source: https://context7.com/traceloop/hub/llms.txt

Configures the runtime behavior of Traceloop Hub through environment variables. This includes settings for mode, ports, logging, database connections, and provider credentials. These variables are crucial for customizing the application's operation in different environments. Ensure sensitive credentials are handled securely.

```bash
# Gateway configuration
export HUB_MODE=database                              # Mode: yaml or database
export PORT=3000                                       # Gateway port
export RUST_LOG=info                                   # Log level: debug, info, warn, error
export TRACE_CONTENT_ENABLED=true                      # Enable request/response content tracing

# Database mode specific
export DATABASE_URL=postgresql://user:pass@host:5432/db  # PostgreSQL connection
export MANAGEMENT_PORT=8080                            # Management API port
export DB_POLL_INTERVAL_SECONDS=30                     # Config polling interval

# YAML mode specific
export CONFIG_FILE_PATH=config.yaml                    # Path to YAML config

# Provider credentials (referenced in config)
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export AZURE_OPENAI_KEY=...
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...

# Run the gateway
cargo run
```

--------------------------------

### OpenAI Provider Configuration (YAML)

Source: https://github.com/traceloop/hub/blob/main/README.md

YAML configuration snippet for integrating the OpenAI provider with Traceloop Hub. Includes required API key and optional organization ID and base URL.

```yaml
providers:
  - key: openai
    type: openai
    api_key: sk-...
    # Optional
    organization_id: org-...
    base_url: https://api.openai.com/v1
```

--------------------------------

### Azure OpenAI Provider Configuration (YAML)

Source: https://github.com/traceloop/hub/blob/main/README.md

YAML configuration snippet for integrating the Azure OpenAI provider. Includes API key, resource name, and API version.

```yaml
providers:
  - key: azure
    type: azure
    api_key: your-key
    resource_name: your-resource
    api_version: "2023-05-15"
```

--------------------------------

### Text Completions API

Source: https://context7.com/traceloop/hub/llms.txt

A legacy OpenAI-compatible endpoint for generating text completions without chat formatting.

```APIDOC
## POST /api/v1/completions

### Description
Legacy OpenAI completions endpoint for text generation without chat formatting.

### Method
POST

### Endpoint
`/api/v1/completions`

### Parameters
#### Request Body
- **model** (string) - Required - The model to use for text completion.
- **prompt** (string) - Required - The prompt to generate completions for.
- **max_tokens** (integer) - Optional - The maximum number of tokens to generate.
- **temperature** (number) - Optional - Controls randomness. Lower values make the output more deterministic.
- **stop** (array of strings) - Optional - Sequences where the API will stop generating further tokens.

### Request Example
```json
{
  "model": "gpt-3.5-turbo",
  "prompt": "Once upon a time in a distant galaxy",
  "max_tokens": 100,
  "temperature": 0.8,
  "stop": ["\n\n"]
}
```

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the completion.
- **object** (string) - Type of object returned (e.g., 'text_completion').
- **created** (integer) - Timestamp of creation.
- **model** (string) - The model used for the completion.
- **choices** (array) - An array of completion choices.
  - **text** (string) - The generated text.
  - **index** (integer) - Index of the choice.
  - **finish_reason** (string) - The reason the model stopped generating tokens.
- **usage** (object) - Token usage information.
  - **prompt_tokens** (integer) - Number of tokens in the prompt.
  - **completion_tokens** (integer) - Number of tokens in the completion.
  - **total_tokens** (integer) - Total tokens used.

#### Response Example
```json
{
  "id": "cmpl-xyz789",
  "object": "text_completion",
  "created": 1699999999,
  "model": "gpt-3.5-turbo",
  "choices": [{
    "text": ", there lived a brave astronaut who discovered...",
    "index": 0,
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 95,
    "total_tokens": 103
  }
}
```
```

--------------------------------

### Anthropic Provider Configuration (YAML)

Source: https://github.com/traceloop/hub/blob/main/README.md

YAML configuration snippet for integrating the Anthropic provider with Traceloop Hub. Requires the API key.

```yaml
providers:
  - key: anthropic
    type: anthropic
    api_key: sk-ant-...
```

--------------------------------

### Manage Model Definitions using cURL

Source: https://context7.com/traceloop/hub/llms.txt

These cURL commands illustrate how to manage existing model definitions, including listing all models, retrieving a specific model by ID or key, updating a model's configuration, and deleting a model.

```bash
# List all models
curl http://localhost:8080/api/v1/management/model-definitions
```

```bash
# Get model by ID
curl http://localhost:8080/api/v1/management/model-definitions/660e8400-e29b-41d4-a716-446655440001
```

```bash
# Get model by key
curl http://localhost:8080/api/v1/management/model-definitions/key/gpt-4-turbo
```

```bash
# Update model
curl -X PUT http://localhost:8080/api/v1/management/model-definitions/660e8400-e29b-41d4-a716-446655440001 \
  -H "Content-Type: application/json" \
  -d '{
    "model_type": "gpt-4-turbo-2024-04-09"
  }'
```

```bash
# Delete model
curl -X DELETE http://localhost:8080/api/v1/management/model-definitions/660e8400-e29b-41d4-a716-446655440001
```

--------------------------------

### Chat Completions API

Source: https://context7.com/traceloop/hub/llms.txt

Provides an OpenAI-compatible endpoint for chat completions. Supports streaming, tool calls, and reasoning modes, routing requests to configured LLM providers.

```APIDOC
## POST /api/v1/chat/completions

### Description
OpenAI-compatible chat completions endpoint that routes requests to configured LLM providers with support for streaming, tool calls, and reasoning modes.

### Method
POST

### Endpoint
`/api/v1/chat/completions`

### Parameters
#### Request Body
- **model** (string) - Required - The model to use for chat completions.
- **messages** (array) - Required - An array of message objects representing the conversation history.
  - **role** (string) - Required - The role of the author of the message ('system', 'user', 'assistant', 'tool').
  - **content** (string) - Required - The content of the message.
- **stream** (boolean) - Optional - Whether to stream the response.
- **temperature** (number) - Optional - Controls randomness. Lower values make the output more deterministic.
- **max_tokens** (integer) - Optional - The maximum number of tokens to generate.
- **reasoning** (object) - Optional - Configuration for reasoning modes (e.g., for OpenAI o-series models).
  - **effort** (string) - Optional - The effort level for reasoning (e.g., 'high').
  - **exclude** (boolean) - Optional - Whether to exclude reasoning.
- **tools** (array) - Optional - A list of tools the model may call.
  - **type** (string) - Required - The type of tool (e.g., 'function').
  - **function** (object) - Required - The function definition.
    - **name** (string) - Required - The name of the function.
    - **description** (string) - Optional - A description of what the function does.
    - **parameters** (object) - Required - The parameters the function accepts.
- **tool_choice** (string) - Optional - Controls how the model uses tools (e.g., 'auto').

### Request Example
```json
{
  "model": "gpt-4",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"}
  ],
  "temperature": 0.7,
  "max_tokens": 150
}
```

### Response
#### Success Response (200)
- **id** (string) - Unique identifier for the completion.
- **object** (string) - Type of object returned (e.g., 'chat.completion').
- **created** (integer) - Timestamp of creation.
- **model** (string) - The model used for the completion.
- **choices** (array) - An array of completion choices.
  - **index** (integer) - Index of the choice.
  - **message** (object) - The message object.
    - **role** (string) - The role of the author (e.g., 'assistant').
    - **content** (string) - The content of the message.
  - **finish_reason** (string) - The reason the model stopped generating tokens.
- **usage** (object) - Token usage information.
  - **prompt_tokens** (integer) - Number of tokens in the prompt.
  - **completion_tokens** (integer) - Number of tokens in the completion.
  - **total_tokens** (integer) - Total tokens used.

#### Response Example
```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1699999999,
  "model": "gpt-4",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "The capital of France is Paris."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 8,
    "total_tokens": 28
  }
}
```
```

--------------------------------

### POST /api/v1/management/providers - Create Provider

Source: https://context7.com/traceloop/hub/llms.txt

Add a new LLM provider configuration dynamically via the Management API. Supports literal API keys, Kubernetes secret references, and environment variable references.

```APIDOC
## POST /api/v1/management/providers

### Description
Add a new LLM provider configuration dynamically without restart.

### Method
`POST`

### Endpoint
`/api/v1/management/providers`

### Request Body

- **name** (string) - Required - The name of the provider.
- **provider_type** (string) - Required - The type of the provider (e.g., `openai`, `anthropic`, `azure`).
- **config** (object) - Required - Configuration details for the provider.
  - **api_key** (object) - Required - API key configuration.
    - **type** (string) - Required - Type of API key source (`literal`, `kubernetes`, `environment`).
    - **value** (string) - Required if type is `literal` - The actual API key.
    - **secret_name** (string) - Required if type is `kubernetes` - The name of the Kubernetes secret.
    - **key** (string) - Required if type is `kubernetes` - The key within the Kubernetes secret.
    - **namespace** (string) - Optional if type is `kubernetes` - The namespace of the Kubernetes secret.
    - **variable_name** (string) - Required if type is `environment` - The name of the environment variable.
  - **organization_id** (string) - Optional - Organization ID for the provider.
  - **resource_name** (string) - Optional - Resource name for Azure providers.
  - **api_version** (string) - Optional - API version for Azure providers.
- **enabled** (boolean) - Optional - Whether the provider is enabled (defaults to true).

### Request Example (OpenAI with literal key)
```json
{
  "name": "my-openai",
  "provider_type": "openai",
  "config": {
    "api_key": {
      "type": "literal",
      "value": "sk-..."
    },
    "organization_id": "org-..."
  },
  "enabled": true
}
```

### Request Example (Anthropic with Kubernetes secret)
```json
{
  "name": "anthropic-prod",
  "provider_type": "anthropic",
  "config": {
    "api_key": {
      "type": "kubernetes",
      "secret_name": "anthropic-credentials",
      "key": "api-key",
      "namespace": "production"
    }
  },
  "enabled": true
}
```

### Request Example (Azure with environment variable)
```json
{
  "name": "azure-openai",
  "provider_type": "azure",
  "config": {
    "api_key": {
      "type": "environment",
      "variable_name": "AZURE_OPENAI_KEY"
    },
    "resource_name": "my-azure-resource",
    "api_version": "2024-02-15-preview"
  }
}
```

### Response
#### Success Response (200 or 201)
- **id** (string) - Unique identifier for the provider.
- **name** (string) - Name of the provider.
- **provider_type** (string) - Type of the provider.
- **config** (object) - Configuration details.
- **enabled** (boolean) - Whether the provider is enabled.
- **created_at** (string) - Timestamp of creation.
- **updated_at** (string) - Timestamp of last update.

#### Response Example
```json
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "name": "my-openai",
  "provider_type": "openai",
  "config": {
    "api_key": {
      "type": "literal"
    },
    "organization_id": "org-..."
  },
  "enabled": true,
  "created_at": "2025-01-15T10:30:00Z",
  "updated_at": "2025-01-15T10:30:00Z"
}
```
```

--------------------------------

### POST /api/v1/management/model-definitions

Source: https://context7.com/traceloop/hub/llms.txt

Creates a new model definition, which specifies configurations for using AI models.

```APIDOC
## POST /api/v1/management/model-definitions

### Description
Creates a new model definition, which specifies configurations for using AI models. This can include details like model type, provider ID, and deployment information.

### Method
POST

### Endpoint
`/api/v1/management/model-definitions`

### Parameters
#### Request Body
- **key** (string) - Required - A unique key for the model definition.
- **model_type** (string) - Required - The type or name of the AI model.
- **provider_id** (string) - Required - The identifier of the model provider.
- **deployment** (string) - Optional - The deployment name, if applicable (e.g., for Azure).
- **model_provider** (string) - Optional - The name of the model provider (e.g., 'anthropic').
- **model_version** (string) - Optional - The version of the model.

### Request Example
```json
{
  "key": "gpt-4-turbo",
  "model_type": "gpt-4-turbo-preview",
  "provider_id": "550e8400-e29b-41d4-a716-446655440000",
  "deployment": null,
  "model_provider": null,
  "model_version": null
}
```

### Response
#### Success Response (201)
- **id** (string) - The unique identifier of the created model definition.
- **key** (string) - The key of the model definition.
- **model_type** (string) - The type of the model.
- **provider_id** (string) - The provider ID.
- **deployment** (string) - The deployment name, if provided.
- **created_at** (string) - Timestamp of creation.
- **updated_at** (string) - Timestamp of last update.

#### Response Example
```json
{
  "id": "660e8400-e29b-41d4-a716-446655440001",
  "key": "gpt-4-turbo",
  "model_type": "gpt-4-turbo-preview",
  "provider_id": "550e8400-e29b-41d4-a716-446655440000",
  "deployment": null,
  "created_at": "2025-01-15T10:35:00Z",
  "updated_at": "2025-01-15T10:35:00Z"
}
```
```

--------------------------------

### Docker Compose Configuration (Database Mode)

Source: https://github.com/traceloop/hub/blob/main/README.md

Docker Compose configuration for running Traceloop Hub in Database mode. Exposes ports 3000 and 8080, sets environment variables for database connection, and depends on a PostgreSQL service.

```yaml
version: '3.8'
services:
  # Database Mode
  hub-database:
    image: traceloop/hub
    ports:
      - "3000:3000"
      - "8080:8080"
    environment:
      - HUB_MODE=database
      - DATABASE_URL=postgresql://hub:password@postgres:5432/hub
    depends_on:
      - postgres

  postgres:
    image: postgres:15
    environment:
      - POSTGRES_DB=hub
      - POSTGRES_USER=hub
      - POSTGRES_PASSWORD=password
```

--------------------------------

### Text Completions API Request (Bash)

Source: https://context7.com/traceloop/hub/llms.txt

Illustrates how to perform a text completion request using the Traceloop Hub API with cURL. This endpoint is for legacy text generation without chat formatting. The response contains the generated text, usage data, and model details.

```bash
# Text completion request
curl -X POST http://localhost:3000/api/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "prompt": "Once upon a time in a distant galaxy",
    "max_tokens": 100,
    "temperature": 0.8,
    "stop": ["\n\n"]
  }'

# Response:
# {
#   "id": "cmpl-xyz789",
#   "object": "text_completion",
#   "created": 1699999999,
#   "model": "gpt-3.5-turbo",
#   "choices": [{
#     "text": ", there lived a brave astronaut who discovered...",
#     "index": 0,
#     "finish_reason": "stop"
#   }],
#   "usage": {
#     "prompt_tokens": 8,
#     "completion_tokens": 95,
#     "total_tokens": 103
#   }
# }
```