CodeGate

https://github.com/stacklok/codegate
Admin
CodeGate is an agent designed to make AI applications, coding assistants, and agentic frameworks...
Tokens:17,451
Snippets:69
Trust Score:9.7
License:Apache-2.0
Update:1 year ago
Show doc for...
Tokens:
Raw
### Codegate CLI Basic Usage

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Illustrates fundamental commands for starting the Codegate server with various configuration options.

```bash
# Start server with default settings
codegate serve

# Start with custom configuration
codegate serve --port 8989 --host localhost --log-level DEBUG

# Start with custom prompts
codegate serve --prompts my-prompts.yaml

# Start with custom provider URL
codegate serve --vllm-url https://vllm.example.com
```

--------------------------------

### Run UI Development Server (npm)

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to start the UI development server.

```Bash
npm run dev
```

--------------------------------

### Install UI Dependencies (npm)

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to install all dependencies for the UI's local development environment using npm.

```Bash
npm install
```

--------------------------------

### Integration Test Setup

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Steps to set up the environment and run integration tests for Codegate. This includes creating a .env file with necessary API keys and executing test scripts.

```bash
ENV_OPENAI_KEY=<YOUR_KEY>
ENV_VLLM_KEY=<YOUR_KEY>
ENV_ANTHROPIC_KEY=<YOUR_KEY>
```

```bash
poetry run python scripts/import_packages.py
```

```bash
poetry run codegate serve --log-level DEBUG --log-format TEXT
```

```bash
poetry run python tests/integration/integration_tests.py
```

```bash
CODEGATE_PROVIDERS=copilot CA_CERT_FILE=./codegate_volume/certs/ca.crt poetry run python tests/integration/integration_tests.py
```

--------------------------------

### Install Project Dependencies (Poetry)

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to install project dependencies using Poetry, including development dependencies.

```Bash
poetry install --with dev
```

--------------------------------

### Start Codegate Server with Various Configurations

Source: https://github.com/stacklok/codegate/blob/main/docs/cli.md

Demonstrates starting the Codegate server with default settings, specifying host and port, custom logging levels and formats, configuration files, custom prompts, and different LLM endpoint integrations (vLLM, Ollama, LM Studio). These commands allow flexible server setup for different environments and use cases.

```bash
codegate serve
```

```bash
codegate serve --port 8989 --host localhost
```

```bash
codegate serve --log-level DEBUG --log-format TEXT
```

```bash
codegate serve --config my-config.yaml
```

```bash
codegate serve --prompts my-prompts.yaml
```

```bash
codegate serve --vllm-url https://vllm.example.com
```

```bash
codegate serve --ollama-url http://localhost:11434
```

```bash
codegate serve --lm-studio-url https://lmstudio.example.com
```

--------------------------------

### macOS Python Setup with SQLite

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Instructions for setting up Python on macOS with specific SQLite configurations using Homebrew and pyenv, ensuring compatibility with project requirements.

```Bash
# substitute for your version of choice
PYTHON_VERSION=3.12.9
brew install sqlite
LDFLAGS="-L$(brew --prefix sqlite)/lib" CPPFLAGS="-I$(brew --prefix sqlite)/include" PYTHON_CONFIGURE_OPTS="--enable-loadable-sqlite-extensions" pyenv install -v $PYTHON_VERSION
poetry env use $PYTHON_VERSION
```

--------------------------------

### Example Configuration File

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

An example YAML file showcasing various configuration options for CodeGate, including network settings, provider URLs, and certificate paths.

```yaml
port: 8989
proxy_port: 8990
host: localhost
log_level: INFO
log_format: JSON
provider_urls:
  vllm: "https://vllm.example.com"
  openai: "https://api.openai.com/v1"
  anthropic: "https://api.anthropic.com/v1"
  ollama: "http://localhost:11434"
  lm_studio: "http://localhost:1234"
certs_dir: "./certs"
ca_cert: "ca.crt"
ca_key: "ca.key"
server_cert: "server.crt"
server_key: "server.key"
```

--------------------------------

### Build UI for Production (npm)

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to build the UI for production deployment.

```Bash
npm run build
```

--------------------------------

### Run UI Production Build (npm)

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to run the production build of the UI.

```Bash
npm run preview
```

--------------------------------

### Clone CodeGate UI Repository

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Steps to clone the CodeGate UI project repository.

```Bash
git clone https://github.com/stacklok/codegate-ui
cd codegate-ui
```

--------------------------------

### Makefile Development Commands

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Common development tasks managed by the project's Makefile. These commands cover dependency installation, code formatting, linting, testing, security checks, building, and running all checks.

```bash
make install
# install all dependencies
```

```bash
make format
# format code using black and ruff
```

```bash
make lint
# run linting checks
```

```bash
make test
# run tests with coverage
```

```bash
make security
# run security checks
```

```bash
make build
# build distribution packages
```

```bash
make all
# run all checks and build (recommended before committing)
```

--------------------------------

### Clone CodeGate Repository

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Steps to clone the main CodeGate project repository from GitHub.

```Bash
git clone https://github.com/stacklok/codegate.git
cd codegate
```

--------------------------------

### CodeGate Serve Command

Source: https://github.com/stacklok/codegate/blob/main/docs/cli.md

Starts the CodeGate server. Allows configuration of port, host, logging, and various AI provider URLs.

```APIDOC
codegate serve [OPTIONS]

Options:
  --port INTEGER: Port to listen on (default: 8989). Must be between 1 and 65535.
  --host TEXT: Host to bind to (default: localhost).
  --log-level [ERROR|WARNING|INFO|DEBUG]: Set the log level (default: INFO). Case-insensitive.
  --log-format [JSON|TEXT]: Set the log format (default: JSON). Case-insensitive.
  --config FILE: Path to YAML config file. Optional.
  --prompts FILE: Path to YAML prompts file. Optional.
  --vllm-url TEXT: vLLM provider URL (default: http://localhost:8000).
  --openai-url TEXT: OpenAI provider URL (default: https://api.openai.com/v1).
  --anthropic-url TEXT: Anthropic provider URL (default: https://api.anthropic.com/v1).
  --ollama-url TEXT: Ollama provider URL (default: http://localhost:11434).
  --lm-studio-url TEXT: LM Studio provider URL (default: http://localhost:1234).
  --model-base-path TEXT: Base path for loading models.
  --embedding-model TEXT: Name of the model used for embeddings.
  --db-path TEXT: Path to a SQLite DB (default: ./codegate_volume/db/codegate.db).
```

--------------------------------

### Test Custom Prompts in Python

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Demonstrates how to load configuration with custom prompts in a Python test and assert prompt content.

```python
def test_custom_prompts():
    config = Config.load(prompts_path="path/to/test/prompts.yaml")
    assert config.prompts.my_prompt == "Expected prompt text"
```

--------------------------------

### Poetry Environment Management Commands

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Common Poetry commands for managing project dependencies and virtual environments.

```Bash
poetry install
poetry add package-name
poetry add --group dev package-name
poetry remove package-name
poetry update
poetry show
poetry env info
```

--------------------------------

### CLI Configuration for Logging

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Example of configuring the logging system via command-line interface arguments, specifying log level and format for the application.

```bash
codegate serve --log-level DEBUG --log-format TEXT
```

--------------------------------

### Docker Deployment Commands

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Commands for building and running the Codegate Docker image. Includes options for basic usage, pulling pre-built images, mounting volumes for data persistence, and setting environment variables.

```bash
make image-build
```

```docker
docker run -p 8989:8989 -p 9090:9090 codegate:latest
# Basic usage with local image
```

```docker
docker pull ghcr.io/stacklok/codegate:latest
# With pre-built pulled image
```

```docker
docker run --name codegate -d -p 8989:8989 -p 9090:9090 ghcr.io/stacklok/codegate:latest
# With pre-built pulled image and detached mode
```

```docker
docker run --name codegate -d -v /path/to/volume:/app/codegate_volume -p 8989:8989 -p 9090:9090 ghcr.io/stacklok/codegate:latest
# With volume mount for data persistence
```

```docker
docker run -p 8989:8989 -p 9090:9090 -e CODEGATE_OLLAMA_URL=http://1.2.3.4:11434/api ghcr.io/stacklok/codegate:latest
# With environment variable override
```

--------------------------------

### Code Formatting with Black

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to format the project's Python code using Black.

```Bash
poetry run black .
```

--------------------------------

### Serve with Prompts via CLI Flag

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Illustrates how to specify a prompts configuration file directly using the --prompts command-line flag when starting the codegate serve command.

```bash
codegate serve --prompts path/to/prompts.yaml
```

--------------------------------

### Run Unit Tests with Pytest

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to execute the project's unit test suite with coverage reporting using Pytest.

```Bash
poetry run pytest
```

--------------------------------

### Prompts File Format Example

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Defines the structure of a YAML prompts file, showing how to map prompt names to their string content, including support for multi-line prompts.

```yaml
prompt_name: "Prompt text content"

another_prompt: "More prompt text"

multiline_prompt: |
  This is a multi-line prompt.
  It can span multiple lines.
```

--------------------------------

### View Loaded Prompts via CLI

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Commands to display prompts loaded by the application, either default or custom ones specified via CLI.

```bash
# Show default prompts
codegate show-prompts

# Show custom prompts
codegate show-prompts --prompts my-prompts.yaml
```

--------------------------------

### Use Custom Prompts via Config File

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

References a custom prompts file within the main configuration file (`config.yaml`).

```yaml
prompts: "path/to/prompts.yaml"
```

--------------------------------

### Use Custom Prompts via CLI

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Loads custom prompts by specifying the YAML file path using the `--prompts` CLI flag.

```bash
# Via CLI
codegate serve --prompts my-prompts.yaml
```

--------------------------------

### Codegate Configuration Options

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Details on Codegate's configuration system, including priority levels and specific configuration options for server settings, logging, and AI provider endpoints. This section also outlines environment variables for Docker deployment.

```APIDOC
Configuration Priority:
1. CLI arguments
2. Environment variables
3. Config file (YAML)
4. Default values

Configuration Options:
- Port: server port (default: `8989`)
- Host: server host (default: `"localhost"`)
- Log level: logging verbosity level (`ERROR`|`WARNING`|`INFO`|`DEBUG`)
- Log format: log format (`JSON`|`TEXT`)
- Prompts: system prompts configuration
- Provider URLs: AI provider endpoint configuration

Exposed Docker Environment Variables:
- CODEGATE_VLLM_URL: URL for the inference engine (defaults to `https://inference.codegate.ai`)
- CODEGATE_OPENAI_URL: URL for OpenAI inference engine (defaults to `https://api.openai.com/v1`)
- CODEGATE_ANTHROPIC_URL: URL for Anthropic inference engine (defaults to `https://api.anthropic.com/v1`)
- CODEGATE_OLLAMA_URL: URL for OLlama inference engine (defaults to `http://localhost:11434/api`)
- CODEGATE_LM_STUDIO_URL: URL for LM Studio inference engine (defaults to `http://localhost:1234/api`)
- CODEGATE_APP_LOG_LEVEL: Level of debug desired when running the codegate server (defaults to `WARNING`, can be `ERROR`/`WARNING`/`INFO`/`DEBUG`)
- CODEGATE_LOG_FORMAT: Type of log formatting desired when running the codegate server (default to `TEXT`, can be `JSON`/`TEXT`)
```

--------------------------------

### Implement New Provider Class in Python

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Provides a template for creating a new provider by extending the `BaseProvider` class. It includes implementing input/output normalizers and handler functions.

```python
from codegate.providers.base import BaseProvider

class NewProvider(BaseProvider):
    def __init__(self, ...):
        super().__init__(
            InputNormalizer(),
            OutputNormalizer(),
            completion_handler,
            pipeline_processor,
            fim_pipeline_processor
        )

    @property
    def provider_route_name(self) -> str:
        return "provider_name"

    def _setup_routes(self):
        # Implement route setup
        pass
```

--------------------------------

### Configure Provider URLs via Environment Variables

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Configures provider URLs by exporting environment variables. This is useful for dynamic configuration in different deployment environments.

```bash
export CODEGATE_PROVIDER_VLLM_URL=https://vllm.example.com
export CODEGATE_PROVIDER_OPENAI_URL=https://api.openai.com/v1
export CODEGATE_PROVIDER_ANTHROPIC_URL=https://api.anthropic.com/v1
export CODEGATE_PROVIDER_OLLAMA_URL=http://localhost:11434
export CODEGATE_PROVIDER_LM_STUDIO_URL=http://localhost:1234
```

--------------------------------

### Linting with Ruff

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to check the project's Python code for linting errors using Ruff.

```Bash
poetry run ruff check .
```

--------------------------------

### Security Checks with Bandit

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Command to perform security checks on the project's source code using Bandit.

```Bash
poetry run bandit -r src/
```

--------------------------------

### Python Logging with Contextual Data

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Best practice example for logging user actions with relevant context like user ID, action type, and IP address for detailed auditing.

```python
logger.info("User action", extra={
    "user_id": "123",
    "action": "login",
    "ip_address": "192.168.1.1"
})
```

--------------------------------

### Define Custom Prompts in YAML Format

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Specifies the structure for creating custom prompt files using YAML. Each key-value pair represents a named prompt.

```yaml
prompt_name: "Prompt text content"
another_prompt: "More prompt text"
```

--------------------------------

### Use Custom Prompts via Environment Variable

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Sets the path to a custom prompts file using the `CODEGATE_PROMPTS_FILE` environment variable.

```bash
export CODEGATE_PROMPTS_FILE=path/to/prompts.yaml
```

--------------------------------

### Configure Provider URLs via CLI Flags

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Specifies provider URLs directly when running the command-line interface. This offers a quick way to override configurations for specific runs.

```bash
codegate serve --vllm-url https://vllm.example.com --ollama-url http://localhost:11434
```

--------------------------------

### Generate Codegate Certificates

Source: https://github.com/stacklok/codegate/blob/main/docs/cli.md

Provides an example of generating certificates using the Codegate CLI with default settings. This command is typically used for setting up secure communication or authentication within the Codegate ecosystem.

```bash
codegate generate-certs
```

--------------------------------

### Tool Invocation: Get Weather

Source: https://github.com/stacklok/codegate/blob/main/tests/types/anthropic/streaming_messages_simple.txt

Demonstrates the structured invocation of a 'get_weather' tool, showing how its JSON input is constructed incrementally through deltas.

```APIDOC
ToolInvocation_GetWeather:
  - Description: Invokes the 'get_weather' tool with JSON parameters.
  - Method: Tool Use Event Stream
  - Parameters:
    - name: "get_weather"
    - input: JSON object representing weather query parameters.
      - Example partial construction:
        - "{""": """
        - ""foo": """
        - " "bar"""
        - "}""
  - Return Value: Not directly specified in this event stream, but implies tool execution and potential subsequent message content.
  - Related Events: content_block_start (tool_use), content_block_delta (input_json_delta), message_delta (stop_reason: "tool_use").
```

--------------------------------

### Run CodeGate Docker Container

Source: https://github.com/stacklok/codegate/blob/main/README.md

This command starts the CodeGate Docker container. It maps necessary ports (8989, 9090, 8990) for dashboard and API access, mounts a volume for persistent data, and ensures the container restarts automatically. Prerequisites include Docker or Podman.

```bash
docker run --name codegate -d -p 8989:8989 -p 9090:9090 -p 8990:8990 \
  --mount type=volume,src=codegate_volume,dst=/app/codegate_volume \
  --restart unless-stopped ghcr.io/stacklok/codegate:latest
```

--------------------------------

### Configure Provider URLs in YAML

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Sets provider endpoints using a YAML configuration file. This method allows for centralized management of external service URLs.

```yaml
provider_urls:
  vllm: "https://vllm.example.com"
  openai: "https://api.openai.com/v1"
  anthropic: "https://api.anthropic.com/v1"
  ollama: "http://localhost:11434" # /api path added automatically
```

--------------------------------

### Get Weather Tool Invocation

Source: https://github.com/stacklok/codegate/blob/main/tests/types/anthropic/streaming_messages_error.txt

Details the invocation of the 'get_weather' tool, including the construction of its input parameters. This represents a specific API or tool interaction pattern observed.

```APIDOC
Tool: get_weather

Description: Retrieves weather information for a specified location.

Invocation Details:
  - The tool is invoked when weather information is requested.
  - Input parameters are constructed incrementally.

Input Parameters (partial construction observed):
  - location: String. The geographical location for which to fetch weather data. Example: "San Francisco, CA"

Observed Events:
  - content_block_start: Indicates the start of a tool use block.
  - content_block_delta (input_json_delta): Shows the incremental building of the JSON input for the tool.
  - error: An 'overloaded_error' was encountered during processing.
```

--------------------------------

### Codegate AI Provider Details

Source: https://github.com/stacklok/codegate/blob/main/docs/development.md

Information on the AI providers supported by Codegate, including their default URLs, API compatibility, and specific endpoint details. This helps users configure Codegate to work with different LLM services.

```APIDOC
Available Providers:

1. vLLM provider:
   - Default URL: `http://localhost:8000`
   - Supports OpenAI-compatible APIs
   - Automatically adds `/v1` path to base URL
   - Model names are prefixed with `hosted_vllm/`

2. OpenAI provider:
   - Default URL: `https://api.openai.com/v1`
   - Standard OpenAI API implementation

3. Anthropic provider:
   - Default URL: `https://api.anthropic.com/v1`
   - Anthropic Claude API implementation

4. Ollama provider:
   - Default URL: `http://localhost:11434`
   - Endpoints:
     - Native Ollama API: `/ollama/api/chat`
     - OpenAI-compatible: `/ollama/chat/completions`
```

--------------------------------

### Message Lifecycle Events

Source: https://github.com/stacklok/codegate/blob/main/tests/types/anthropic/streaming_messages_simple.txt

Documents the sequence of events that define the lifecycle of a message within the system, from start to stop, including initial metadata and final updates.

```APIDOC
MessageLifecycle:
  - message_start: Marks the beginning of a message, containing initial metadata like model, ID, and usage.
    - data.type: "message_start"
    - data.message.id: Unique identifier for the message.
    - data.message.model: The model used for generating the message.
    - data.message.usage.input_tokens: Number of input tokens.
    - data.message.usage.output_tokens: Number of output tokens.
  - message_delta: Represents incremental updates to a message, such as changes in stop reason or usage.
    - data.type: "message_delta"
    - data.delta.stop_reason: Reason for message termination (e.g., "tool_use").
    - data.usage.output_tokens: Incremental output tokens.
  - message_stop: Signals the completion of the message processing.
    - data.type: "message_stop"
```

--------------------------------

### Compare FIM Configurations: CodeGate vs. Direct Provider

Source: https://github.com/stacklok/codegate/blob/main/docs/debugging_clients.md

Sample JSON configurations are provided to illustrate how to set up CodeGate for tab autocomplete and how to configure a direct connection to a provider. Comparing these configurations helps in diagnosing issues related to request forwarding and response handling.

```json
"tabAutocompleteModel": {
    "title": "CodeGate - Provider",
    "provider": "openai",
    "model": "<model>",
    "apiKey": "<insert-api-key-if-required>",
    "apiBase": "http://localhost:8989/<provider>"
}
```

```json
"tabAutocompleteModel": {
    "title": "Provider",
    "provider": "openai",
    "model": "<model>",
    "apiKey": "<insert-api-key-if-required>",
    "apiBase": "<provider-url>"
}
```

--------------------------------

### Load Configuration from Environment

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Demonstrates how to load configuration settings directly from environment variables using the `Config.from_env` method. This is useful for containerized deployments.

```python
config = Config.from_env()
```

--------------------------------

### Load Configuration from File

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Demonstrates how to load configuration settings from a YAML file using the `Config.from_file` method. This is one of the primary ways to configure the application.

```python
config = Config.from_file("config.yaml")
```

--------------------------------

### Environment Variable Configuration for Logging

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Demonstrates setting logging configuration using environment variables, such as APP_LOG_LEVEL and CODEGATE_LOG_FORMAT, for runtime control.

```bash
export APP_LOG_LEVEL=DEBUG
export CODEGATE_LOG_FORMAT=TEXT
```

--------------------------------

### YAML Configuration for Logging

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Shows how to configure logging parameters like log level and format using a YAML configuration file for persistent settings.

```yaml
log_level: DEBUG
log_format: TEXT
```

--------------------------------

### Access Prompts in Python Code

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Shows how to load the configuration and access defined prompts programmatically within a Python application. Assumes a Config class is available.

```python
config = Config.load()
prompt = config.prompts.prompt_name
```

--------------------------------

### Python Basic Logging with structlog

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Demonstrates basic logging operations using the structlog library in Python, showing how to log messages at different levels (info, debug, error, warning).

```python
import structlog

logger = structlog.get_logger(__name__)

# Different log levels
logger.info("This is an info message")
logger.debug("This is a debug message")
logger.error("This is an error message")
logger.warning("This is a warning message")
```

--------------------------------

### Check CI Locally with act

Source: https://github.com/stacklok/codegate/blob/main/SECURITY.md

This snippet demonstrates how to use the 'act' tool to check Continuous Integration workflows locally. 'act' allows you to run GitHub Actions locally, which is useful for testing CI configurations before pushing changes.

```bash
act
```

--------------------------------

### Set Prompts File via Environment Variable

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Demonstrates setting the CODEGATE_PROMPTS_FILE environment variable to specify the prompts configuration file. This is useful for deployment or overriding defaults.

```bash
export CODEGATE_PROMPTS_FILE=path/to/prompts.yaml
```

--------------------------------

### Configure Prompts in YAML File

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Shows how to define prompts directly within a YAML configuration file or reference an external prompts file. This allows for flexible prompt management.

```yaml
# Option 1: Direct prompts definition
prompts:
  my_prompt: "Custom prompt text"
  another_prompt: "Another prompt text"

# Option 2: Reference to prompts file
prompts: "path/to/prompts.yaml"
```

--------------------------------

### Python Exception Logging with structlog

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Illustrates how to log exceptions and their stack traces using the 'exc_info=True' parameter with structlog in Python, aiding in debugging.

```python
try:
    # Some code that might raise an exception
    raise ValueError("Something went wrong")
except Exception as e:
    logger.error("Error occurred", exc_info=True)
```

--------------------------------

### CLI Flags for Provider URLs

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Configures the base URLs for various AI model providers using command-line arguments. These flags override environment variables and configuration files.

```APIDOC
CLI Flags for Provider URLs:

- `--vllm-url <url>`: vLLM provider URL. Example: `codegate serve --vllm-url https://vllm.example.com`
- `--openai-url <url>`: OpenAI provider URL. Example: `codegate serve --openai-url https://api.openai.com/v1`
- `--anthropic-url <url>`: Anthropic provider URL. Example: `codegate serve --anthropic-url https://api.anthropic.com/v1`
- `--ollama-url <url>`: Ollama provider URL. Example: `codegate serve --ollama-url http://localhost:11434`
- `--lm-studio-url <url>`: LM Studio provider URL. Example: `codegate serve --lm-studio-url http://localhost:1234`

Notes:
- For vLLM, `/v1` path is appended if not present.
- For Ollama, `/api` path is appended if not present.
```

--------------------------------

### CLI Flags for Network Settings

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Configures network parameters like port, proxy port, and host using command-line arguments. These flags override environment variables and configuration files.

```APIDOC
CLI Flags for Network Settings:

- `--port <port>`: Port to listen on (1-65535). Example: `codegate serve --port 8989`
- `--proxy-port <proxy_port>`: Proxy port to listen on (1-65535). Example: `codegate serve --proxy-port 8990`
- `--host <host>`: Host to bind to. Example: `codegate serve --host localhost`
```

--------------------------------

### Display Codegate Prompts

Source: https://github.com/stacklok/codegate/blob/main/docs/cli.md

Shows how to display the default system prompts or prompts loaded from a custom file. This is useful for understanding or modifying the prompts used by the Codegate system.

```bash
codegate show-prompts
```

```bash
codegate show-prompts --prompts my-prompts.yaml
```

--------------------------------

### CodeGate Show Prompts Command

Source: https://github.com/stacklok/codegate/blob/main/docs/cli.md

Displays the system prompts loaded by CodeGate. Allows specifying a custom prompts file.

```APIDOC
codegate show-prompts [OPTIONS]

Options:
  --prompts FILE: Path to YAML prompts file. Optional. If not provided, shows default prompts from prompts/default.yaml.
```

--------------------------------

### Environment Variable Configuration Mappings

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Maps environment variables to specific CodeGate configuration settings, enabling configuration via the environment. These variables are automatically loaded by `Config.from_env()`.

```APIDOC
Environment Variable Mappings:

- CODEGATE_APP_PORT: server port
- CODEGATE_APP_PROXY_PORT: server proxy port
- CODEGATE_APP_HOST: server host
- CODEGATE_APP_LOG_LEVEL: logging level
- CODEGATE_LOG_FORMAT: log format
- CODEGATE_PROMPTS_FILE: path to prompts YAML file
- CODEGATE_PROVIDER_VLLM_URL: vLLM provider URL
- CODEGATE_PROVIDER_OPENAI_URL: OpenAI provider URL
- CODEGATE_PROVIDER_ANTHROPIC_URL: Anthropic provider URL
- CODEGATE_PROVIDER_OLLAMA_URL: Ollama provider URL
- CODEGATE_PROVIDER_LM_STUDIO_URL: LM Studio provider URL
- CODEGATE_CERTS_DIR: directory for certificate files
- CODEGATE_CA_CERT: CA certificate file name
- CODEGATE_CA_KEY: CA key file name
- CODEGATE_SERVER_CERT: server certificate file name
- CODEGATE_SERVER_KEY: server key file name
```

--------------------------------

### CLI Flags for Certificate Configuration

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Configures paths and filenames for TLS/SSL certificates using command-line arguments. These flags override environment variables and configuration files.

```APIDOC
CLI Flags for Certificate Configuration:

- `--certs-dir <dir>`: Directory for certificate files. Example: `codegate serve --certs-dir ./certs`
- `--ca-cert <file>`: CA certificate file name. Example: `codegate serve --ca-cert ca.crt`
- `--ca-key <file>`: CA private key file name. Example: `codegate serve --ca-key ca.key`
- `--server-cert <file>`: Server certificate file name. Example: `codegate serve --server-cert server.crt`
- `--server-key <file>`: Server private key file name. Example: `codegate serve --server-key server.key`
```

--------------------------------

### CodeGate Generate Certs Command

Source: https://github.com/stacklok/codegate/blob/main/docs/cli.md

Generates SSL certificates for the CodeGate server. Allows customization of output directory and certificate names.

```APIDOC
codegate generate-certs [OPTIONS]

Options:
  --certs-out-dir PATH: Directory path where certificates are generated (default: ./codegate_volume/certs).
  --ca-cert-name TEXT: Name for the CA certificate (default: ca.crt).
  --ca-key-name TEXT: Name for the CA key (default: ca.key).
  --server-cert-name TEXT: Name for the server certificate (default: server.crt).
  --server-key-name TEXT: Name for the server key (default: server.key).
  --log-level [ERROR|WARNING|INFO|DEBUG]: Set the log level (default: INFO). Case-insensitive.
  --log-format [JSON|TEXT]: Set the log format (default: JSON). Case-insensitive.
```

--------------------------------

### Python Logging with Extra Context

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Shows how to add custom context to log messages using the 'extra' dictionary in Python's structlog. This allows for richer log data aggregation.

```python
logger.info("Server starting", extra={
    "host": "0",
    "port": 8989,
    "environment": "production"
})
```

--------------------------------

### Text Log Entry Format

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Defines the pattern for plain text formatted log entries, showing timestamp, level, name, and message for human-readable output.

```plain
YYYY-MM-DDThh:mm:ss.mmmZ - LEVEL - NAME - MESSAGE
```

--------------------------------

### Log Levels

Source: https://github.com/stacklok/codegate/blob/main/docs/configuration.md

Specifies the available log levels for the CodeGate application. These are case-insensitive and can be set via configuration file, environment variables, or CLI flags.

```APIDOC
Available Log Levels:

- ERROR
- WARNING
- INFO
- DEBUG
```

--------------------------------

### Content Block Handling

Source: https://github.com/stacklok/codegate/blob/main/tests/types/anthropic/streaming_messages_simple.txt

Details the events related to content blocks, including text content and tool usage, with specific handling for incremental JSON input for tools.

```APIDOC
ContentBlockHandling:
  - content_block_start: Initiates a new content block, specifying its type (e.g., 'text', 'tool_use').
    - data.type: "content_block_start"
    - data.index: The sequential index of the content block.
    - data.content_block.type: Type of content ('text' or 'tool_use').
    - data.content_block.id: Unique identifier for tool use blocks.
    - data.content_block.name: Name of the tool being used.
    - data.content_block.input: Initial input for the tool (can be partial).
  - content_block_delta: Provides incremental updates for a content block.
    - data.type: "content_block_delta"
    - data.index: Index of the content block being updated.
    - data.delta.type: Type of delta (e.g., 'text_delta', 'input_json_delta').
    - data.delta.text: Incremental text content.
    - data.delta.partial_json: Incremental JSON fragment for tool input.
  - content_block_stop: Terminates a content block.
    - data.type: "content_block_stop"
    - data.index: Index of the content block being stopped.
```

--------------------------------

### JSON Log Entry Structure

Source: https://github.com/stacklok/codegate/blob/main/docs/logging.md

Defines the structure for JSON formatted log entries, including timestamp, log level, module, event message, and optional extra fields for custom data.

```json
{
  "timestamp": "YYYY-MM-DDThh:mm:ss.mmmZ",
  "log_level": "LOG_LEVEL",
  "module": "MODULE_NAME",
  "event": "Log message",
  "extra": {
    // Additional fields as you desire
  }
}
```

--------------------------------

### Add console.log for FIM debugging in Continue

Source: https://github.com/stacklok/codegate/blob/main/docs/debugging_clients.md

Add a console.log statement within the streamSse function's loop to print raw stream data. This helps in observing the exact data being processed for FIM requests, aiding in the comparison between CodeGate's output and direct provider responses.

```typescript
console.log('Raw stream data:', value);
```

--------------------------------

### Chat Completion Chunk Data

Source: https://github.com/stacklok/codegate/blob/main/tests/types/openai/streaming_messages.txt

Represents a single chunk of data received during a streaming chat completion API call. It contains partial content updates and metadata. Multiple such chunks are typically received sequentially before the final usage information.

```json
{
  "id": "chatcmpl-B0szUPll9BiFva49CokSsI1pVPjA6",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "content 1"
      }
    }
  ],
  "created": 1739551084,
  "model": "gpt-4o-2024-08-06",
  "service_tier": "default",
  "system_fingerprint": "fp_523b9b6e5f",
  "object": "chat.completion.chunk"
}
```

```json
{
  "id": "chatcmpl-B0szUPll9BiFva49CokSsI1pVPjA6",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "content 2"
      }
    }
  ],
  "created": 1739551084,
  "model": "gpt-4o-2024-08-06",
  "service_tier": "default",
  "system_fingerprint": "fp_523b9b6e5f",
  "object": "chat.completion.chunk"
}
```

--------------------------------

### Ping Event

Source: https://github.com/stacklok/codegate/blob/main/tests/types/anthropic/streaming_messages_simple.txt

A simple event indicating a 'ping' from the system, typically used for health checks or keep-alive mechanisms.

```APIDOC
PingEvent:
  - Description: A keep-alive or status check signal.
  - Event Type: "ping"
  - Data Payload: {"type": "ping"}
```

--------------------------------

### API Stream Completion Usage Data

Source: https://github.com/stacklok/codegate/blob/main/tests/types/openai/streaming_messages.txt

The final chunk in a streaming API response, indicating the end of the stream and providing detailed usage statistics. This includes token counts for prompts, completions, and total usage, along with specific token breakdown details.

```json
{
  "id": "chatcmpl-B0szUPll9BiFva49CokSsI1pVPjA6",
  "choices": [],
  "created": 1739551084,
  "model": "gpt-4o-2024-08-06",
  "service_tier": "default",
  "system_fingerprint": "fp_523b9b6e5f",
  "object": "chat.completion.chunk",
  "usage": {
    "completion_tokens": 394,
    "prompt_tokens": 15675,
    "total_tokens": 16069,
    "completion_tokens_details": {
      "accepted_prediction_tokens": 0,
      "audio_tokens": 0,
      "reasoning_tokens": 0,
      "rejected_prediction_tokens": 0
    },
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 4352
    }
  }
}
```

--------------------------------

### Stream End Signal

Source: https://github.com/stacklok/codegate/blob/main/tests/types/openai/streaming_messages.txt

A special marker indicating that the streaming API response has concluded. This signal is typically sent after all data chunks and the final usage information have been transmitted.

```text
[DONE]
```
Tokens:
Raw