### Development Server Setup

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Starts the ChainForge server with default settings, typically on localhost port 8000. Access the application by opening the specified URL in a web browser.

```bash
# Start with default settings (localhost:8000)
chainforge serve

# Then open http://localhost:8000 in your browser
```

--------------------------------

### Install and Run ChainForge

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Demonstrates the basic commands to install ChainForge using pip and then run the server.

```bash
pip install chainforge
chainforge serve
```

--------------------------------

### Flask Error: Missing Examples Directory

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

JSON error response indicating that the 'examples/' directory is missing. This suggests an incomplete ChainForge package installation.

```json
{
  "error": "Could not find an examples/ directory at path <path>"
}
```

--------------------------------

### Start ChainForge Server with Port Configuration

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Demonstrates how to start the ChainForge server, specifying a custom port using the `--port` flag.

```bash
chainforge serve --port 8000
```

--------------------------------

### Start ChainForge Server

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/00-START-HERE.md

Use this command to start the ChainForge server. Specify the port and host for the server instance.

```bash
chainforge serve --port 8000 --host localhost
```

--------------------------------

### Define Examples Directory

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Specifies the location of example flows included with the package.

```python
EXAMPLES_DIR = os.path.join(
    os.path.dirname(os.path.realpath(__file__)),
    'examples'
)
```

--------------------------------

### Setting Environment Variables for API Keys

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Provides examples of setting environment variables for various API providers (OpenAI, Anthropic, Google, HuggingFace, AWS) before starting the ChainForge server.

```bash
# OpenAI
export OPENAI_API_KEY="sk-..."
export OPENAI_BASE_URL="https://api.openai.com/v1"

# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

# Google
export PALM_API_KEY="..."

# HuggingFace
export HUGGINGFACE_API_KEY="..."

# AWS (for Bedrock)
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="us-east-1"
```

--------------------------------

### Start ChainForge Server

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

The primary command to start the ChainForge server with default options. Open http://localhost:8000 in your browser after starting.

```bash
chainforge serve [options]
```

```bash
chainforge serve
```

--------------------------------

### Basic Custom Provider Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

A minimal custom provider that echoes the prompt and makes an external API call. Ensure you have the necessary libraries like 'requests' installed.

```python
from chainforge.providers import provider

@provider(
    name="My Custom Model",
    emoji="🚀",
    models=["model-v1", "model-v2"]
)
def my_custom_provider(prompt: str, model: Optional[str] = None, 
                       chat_history: Optional[ChatHistory] = None, **kwargs) -> str:
    """
    A minimal custom provider that echoes the prompt.
    """
    import requests
    
    headers = {"Authorization": f"Bearer {kwargs.get('api_key', '')}"}
    
    response = requests.post(
        "https://api.example.com/v1/completions",
        json={
            "prompt": prompt,
            "model": model or "model-v1",
            "temperature": kwargs.get("temperature", 0.7),
            "max_tokens": kwargs.get("max_tokens", 100)
        },
        headers=headers
    )
    
    return response.json()["choices"][0]["text"]
```

--------------------------------

### Serve with Custom Storage Directory

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

Starts the ChainForge server and directs it to use a custom path for storing all its data, including flows, media, and settings. This example shows overriding the default storage location.

```bash
chainforge serve --dir /custom/path/to/chainforge
```

--------------------------------

### Start ChainForge Server

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/README.md

Use the 'chainforge serve' command to start the server. Options include specifying the port, host, enabling encryption, and setting a custom storage directory.

```bash
chainforge serve
```

```bash
chainforge serve --port 3000
```

```bash
chainforge serve --host 0.0.0.0
```

```bash
chainforge serve --secure all
```

```bash
chainforge serve --dir /custom/path
```

--------------------------------

### Flask Error: Example Flow Not Found

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

JSON error response for POST /app/fetchExampleFlow when the requested example flow name does not exist. Verify the example flow name against the available files in the examples/ directory.

```json
{
  "error": "Could not find an example flow named <name>"
}
```

--------------------------------

### Fetch Example Flow

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Retrieves a built-in example flow by its name. The response contains the flow data.

```json
{
  "name": "example_name"
}
```

```json
{
  "data": {}
}
```

--------------------------------

### Install ChainForge

Source: https://github.com/ianarawjo/chainforge/blob/main/README.md

Install ChainForge using pip. Ensure you have Python 3.8 or higher.

```bash
pip install chainforge
```

--------------------------------

### POST /app/fetchExampleFlow

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Retrieves a built-in example flow by its name.

```APIDOC
## POST /app/fetchExampleFlow

### Description
Retrieves a built-in example flow by its name.

### Method
POST

### Endpoint
/app/fetchExampleFlow

### Parameters
#### Request Body
- **name** (string) - Required - The name of the example flow to retrieve.

### Response
#### Success Response (200)
- **data** (object) - Contains the data of the example flow.

#### Response Example
{
  "data": {}
}
```

--------------------------------

### Serve ChainForge

Source: https://github.com/ianarawjo/chainforge/blob/main/README.md

Start the ChainForge local server. Access it at localhost:8000.

```bash
chainforge serve
```

--------------------------------

### Start ChainForge on Custom Host and Port

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

Starts the ChainForge server on a specified port (e.g., 3000) and binds it to a specific host (e.g., 0.0.0.0) to allow external connections.

```bash
chainforge serve --port 3000 --host 0.0.0.0
```

--------------------------------

### Start ChainForge with a Custom Flows Directory

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

Starts the ChainForge server and specifies a custom directory for storing flows and autosaves. This overrides the platform default location.

```bash
chainforge serve --dir /home/user/.chainforge
```

--------------------------------

### Successful Server Startup Output

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Indicates that the ChainForge server has started successfully and is ready to accept connections. The output shows the host and port the server is listening on.

```bash
Serving Flask server on localhost on port 8000...

The server will then be ready to accept connections. Open your browser to http://localhost:8000.
```

--------------------------------

### ChainForge CLI Subparser Implementation

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Illustrates the implementation of the CLI using Python's `argparse` module, specifically showing the setup for the `serve` subcommand.

```python
parser = argparse.ArgumentParser(description='Chainforge command line tool')
subparsers = parser.add_subparsers(dest='serve')
serve_parser = subparsers.add_parser('serve', help='Start Chainforge server')
```

--------------------------------

### Example: Chat-Based Provider

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

An example demonstrating how to implement a chat-based custom provider using the Anthropic API, handling chat history and messages.

```APIDOC
## Example: Chat-Based Provider

```python
from chainforge.providers import provider, ChatHistory
from typing import Optional, List, Dict

@provider(
    name="Chat Provider",
    emoji="💬",
    models=["gpt-4", "gpt-3.5"]
)
def chat_provider(prompt: str,
                  model: Optional[str] = None,
                  chat_history: Optional[ChatHistory] = None,
                  **kwargs) -> str:
    """
    A chat-based custom provider using Claude API.
    """
    from anthropic import Anthropic
    
    client = Anthropic(api_key=kwargs.get("api_key", ""))
    
    # Build messages: history + new prompt
    messages = []
    if chat_history:
        for msg in chat_history:
            messages.append({
                "role": msg["role"],
                "content": msg["content"]
            })
    
    messages.append({"role": "user", "content": prompt})
    
    response = client.messages.create(
        model=model or "claude-3-sonnet-20240229",
        max_tokens=kwargs.get("max_tokens", 1024),
        messages=messages
    )
    
    return response.content[0].text
```
```

--------------------------------

### Complete Ollama Local LLM Provider Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

An example of a complete custom provider that integrates with a local Ollama LLM. It includes settings for base URL, temperature, and top P, and handles API requests and responses.

```python
from chainforge.providers import provider, ChatHistory
from typing import Optional, List
import os
import requests

@provider(
    name="Ollama Local LLM",
    emoji="🦙",
    models=["llama2", "mistral", "neural-chat"],
    rate_limit="sequential",
    settings_schema={
        "settings": {
            "base_url": {
                "type": "string",
                "title": "Ollama Base URL",
                "default": "http://localhost:11434"
            },
            "temperature": {
                "type": "number",
                "title": "Temperature",
                "minimum": 0,
                "maximum": 2,
                "default": 0.7
            },
            "top_p": {
                "type": "number",
                "title": "Top P",
                "minimum": 0,
                "maximum": 1,
                "default": 0.95
            }
        },
        "ui": {
            "base_url": {"ui:widget": "text"},
            "temperature": {"ui:widget": "range"}
        }
    }
)
def ollama_provider(
    prompt: str,
    model: Optional[str] = None,
    chat_history: Optional[ChatHistory] = None,
    **kwargs
) -> str:
    """Local Ollama LLM provider"""
    base_url = kwargs.get("base_url", "http://localhost:11434")
    model = model or "llama2"
    
    url = f"{base_url}/api/generate"
    
    payload = {
        "model": model,
        "prompt": prompt,
        "temperature": kwargs.get("temperature", 0.7),
        "top_p": kwargs.get("top_p", 0.95),
        "stream": False
    }
    
    try:
        response = requests.post(url, json=payload, timeout=300)
        response.raise_for_status()
        return response.json()["response"]
    except Exception as e:
        raise RuntimeError(f"Ollama API error: {str(e)}")
```

--------------------------------

### ResponseInfo __str__() Method Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Demonstrates how to create a ResponseInfo object and print its text content using the __str__() method.

```python
response_info = ResponseInfo(
    text="Hello, world!",
    prompt="Say hello",
    var={"name": "world"},
    meta={"source": "demo"},
    llm="GPT-4"
)
print(response_info)  # Output: "Hello, world!"
```

--------------------------------

### OpenAI GPT-4 LLMSpec Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

An example LLMSpec object configured for OpenAI's GPT-4 model, including temperature and API key.

```typescript
{
  name: "GPT-4",
  emoji: "🤖",
  base_model: "openai",
  model: "gpt-4",
  temp: 0.7,
  key: "sk-...",
  settings: {
    max_tokens: 2000,
    top_p: 1.0
  }
}
```

--------------------------------

### Enable Full Encryption on Startup

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

Starts the ChainForge server with full encryption enabled for all files. This provides maximum security for all data stored by ChainForge.

```bash
chainforge serve --secure all
```

--------------------------------

### Enable Settings Encryption on Startup

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

Starts the ChainForge server with encryption enabled for settings files only. This protects sensitive information like API keys stored in settings.json.

```bash
chainforge serve --secure settings
```

--------------------------------

### Flask Error: Example Flow Parse Error

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

JSON error response for POST /app/fetchExampleFlow when an example flow file is malformed JSON. Check the specified file path for JSON syntax errors.

```json
{
  "error": "Error parsing example flow at <filepath>: <error>"
}
```

--------------------------------

### Ollama Local LLMSpec Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

An example LLMSpec object configured for a local Ollama instance running Llama 2, including the base URL for the Ollama server.

```typescript
{
  name: "Llama 2 (Local)",
  emoji: "🦙",
  base_model: "ollama",
  model: "llama2",
  temp: 0.7,
  settings: {
    base_url: "http://localhost:11434"
  }
}
```

--------------------------------

### Chat-Based Custom Provider Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

An example of a chat-based custom provider that integrates with the Claude API. It constructs messages from chat history and the new prompt.

```python
from chainforge.providers import provider, ChatHistory
from typing import Optional, List, Dict

@provider(
    name="Chat Provider",
    emoji="💬",
    models=["gpt-4", "gpt-3.5"]
)
def chat_provider(prompt: str, 
                  model: Optional[str] = None,
                  chat_history: Optional[ChatHistory] = None,
                  **kwargs) -> str:
    """
    A chat-based custom provider using Claude API.
    """
    from anthropic import Anthropic
    
    client = Anthropic(api_key=kwargs.get("api_key", ""))
    
    # Build messages: history + new prompt
    messages = []
    if chat_history:
        for msg in chat_history:
            messages.append({
                "role": msg["role"],
                "content": msg["content"]
            })
    
    messages.append({"role": "user", "content": prompt})
    
    response = client.messages.create(
        model=model or "claude-3-sonnet-20240229",
        max_tokens=kwargs.get("max_tokens", 1024),
        messages=messages
    )
    
    return response.content[0].text
```

--------------------------------

### Python Example Referencing Model Names

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

Shows how to reference LLM model names as strings within a list of dictionaries in Python, suitable for configuration or selection.

```python
models = [
    {"model": "gpt-4", "name": "GPT-4"},
    {"model": "claude-3-5-sonnet-20240620", "name": "Claude 3.5"},
    {"model": "gemini-2.0-flash", "name": "Gemini"}
]
```

--------------------------------

### Google Gemini LLMSpec Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

An example LLMSpec object configured for Google's Gemini 2.0 Flash model, including temperature, API key, and specific settings like top_k and top_p.

```typescript
{
  name: "Gemini 2.0 Flash",
  emoji: "✨",
  base_model: "google",
  model: "gemini-2.0-flash",
  temp: 1.0,
  key: "api_key_...",
  settings: {
    top_k: 40,
    top_p: 0.95
  }
}
```

--------------------------------

### Local Testing with ResponseInfo

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Provides an example of how to locally test evaluation functions by creating a mock `ResponseInfo` object and calling the evaluator directly. Requires importing necessary components.

```python
from chainforge.flask_app import ResponseInfo, run_over_responses

def my_evaluator(response):
    return len(response.text)

test_resp = ResponseInfo(
    text="Test",
    prompt="Generate",
    var={},
    meta={},
    llm="test"
)

result = my_evaluator(test_resp)
```

--------------------------------

### Example Usage of make_sync_call_async

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Demonstrates how to use `make_sync_call_async` to run a slow synchronous operation asynchronously. Requires `asyncio` and `time` imports.

```python
import asyncio
import time

def slow_operation(x):
    time.sleep(1)
    return x * 2

async def main():
    result = await make_sync_call_async(slow_operation, 5)
    print(result)  # 10

asyncio.run(main())
```

--------------------------------

### Custom Providers

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/_MANIFEST.txt

Guide to creating custom LLM providers in ChainForge, including decorator reference, protocol interface, and configuration.

```APIDOC
## Custom Providers

### Description
This document outlines the process for creating custom LLM providers within ChainForge. It references the `@provider` decorator, the `CustomProviderProtocol` interface, settings schema configuration, and provides complete working examples.

### Key Elements
- **@provider decorator**: Used to register custom providers.
- **CustomProviderProtocol**: Interface defining the contract for custom providers.
- **Settings Schema**: Configuration for provider-specific settings.
- **Chat Message Format**: Specification for message formatting.
- **Rate Limiting**: Options for managing API rate limits.
- **Examples**: Complete, runnable examples are included.
```

--------------------------------

### Create a Custom Provider

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/00-START-HERE.md

Example of creating a custom LLM provider using the `@provider` decorator. This allows integration of custom models into ChainForge.

```python
from chainforge.providers import provider

@provider(
    name="My LLM",
    emoji="🚀",
    models=["v1", "v2"]
)
def my_provider(prompt, model=None, chat_history=None, **kwargs):
    return "response text"
```

--------------------------------

### Throw LLMResponseError Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

Example of how to throw an LLMResponseError when an LLM API call fails. This helps in identifying and handling specific LLM communication problems.

```typescript
throw new LLMResponseError("Failed to connect to OpenAI API: 401 Unauthorized");
```

--------------------------------

### ResponseInfo asMarkdownAST() Method Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Shows how to use the asMarkdownAST() method to parse response text into a Markdown Abstract Syntax Tree (AST) using the mistune library.

```python
response_info = ResponseInfo(
    text="# Heading\nParagraph text",
    prompt="Generate markdown",
    var={},
    meta={},
    llm="GPT-4"
)
ast = response_info.asMarkdownAST()  # Returns mistune AST
```

--------------------------------

### Example Usage of exclude_key

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Demonstrates how to use the exclude_key function to filter a dictionary.

```python
original = {"a": 1, "b": 2, "c": 3}
filtered = exclude_key(original, "b")
# Result: {"a": 1, "c": 3}
```

--------------------------------

### Anthropic Claude 3.5 LLMSpec Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

An example LLMSpec object configured for Anthropic's Claude 3.5 Sonnet model, including temperature and API key.

```typescript
{
  name: "Claude 3.5 Sonnet",
  emoji: "🧠",
  base_model: "anthropic",
  model: "claude-3-5-sonnet-20240620",
  temp: 0.8,
  key: "sk-ant-...",
  settings: {
    max_tokens: 4096
  }
}
```

--------------------------------

### Cache Key Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Illustrates the JSON structure for enabling result caching in the `/app/executepy` endpoint, using a unique `id` for cache invalidation.

```json
{
  "id": "unique_cache_key",
  "code": "...",
  "responses": [...] 
}
```

--------------------------------

### GET /mediaExists/<uid>

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Checks if a media file exists on the server using its unique identifier (UID).

```APIDOC
## GET /mediaExists/<uid>

### Description
Checks if media exists.

### Method
GET

### Endpoint
/mediaExists/<uid>

### Response
#### Success Response (200)
- **exists** (boolean) - True if the media exists, false otherwise.
```

--------------------------------

### TypeScript Example Using LLMSpec and NativeLLM

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

Demonstrates how to import and use the `LLMSpec` type and `NativeLLM` enum in TypeScript to configure an LLM query for OpenAI's GPT-4.

```typescript
import { NativeLLM } from "@/backend/models";

const spec: LLMSpec = {
  name: "GPT-4",
  emoji: "🤖",
  base_model: "openai",
  model: NativeLLM.OpenAI_GPT4,
  temp: 0.7,
  key: apiKey
};
```

--------------------------------

### ChainForge Console Script Entry Point

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Defines the console script entry point for the chainforge package, allowing it to be called directly from the command line after installation.

```python
entry_points={
    "console_scripts": [
        "chainforge = chainforge.app:main",
    ],
}
```

--------------------------------

### GET /api/getConfig/<name>

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Retrieves a named configuration from the server. This endpoint is useful for fetching stored settings or user preferences.

```APIDOC
## GET /api/getConfig/<name>

### Description
Retrieves a named configuration.

### Method
GET

### Endpoint
/api/getConfig/<name>

### Parameters
#### Path Parameters
- **name** (string) - Required - Configuration name (e.g., "settings", "favorites")

### Response
#### Success Response (200)
- Configuration JSON object
```

--------------------------------

### Example of run_over_responses for Word Count Evaluation

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Demonstrates using run_over_responses to count words in LLM responses as an evaluator. Shows how to define a processing function and call the main utility.

```python
from chainforge.flask_app import run_over_responses, ResponseInfo

def count_words(response):
    """Evaluator that counts words"""
    return len(response.text.split())

responses = [
    {
        'prompt': 'Write something',
        'responses': ['Hello world', 'Hi there'],
        'vars': {},
        'metavars': {},
        'llm': 'GPT-4'
    }
]

result = run_over_responses(
    count_words,
    responses,
    scope='response',
    process_type='evaluator'
)

# result[0]['eval_res'] = {
#     'items': [2, 2],
#     'mean': 2.0,
#     'median': 2.0,
#     'stdev': 0.0,
#     'range': (2, 2),
#     'dtype': 'Numeric'
# }
```

--------------------------------

### Example LLMGroup Structure

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/models-and-llm-specs.md

Illustrates a nested structure for organizing LLM models into groups like 'OpenAI' and 'Anthropic', each containing specific LLMSpec objects.

```typescript
[
  {
    group: "OpenAI",
    emoji: "🤖",
    items: [
      { /* GPT-4 spec */ },
      { /* GPT-4o spec */ },
      { /* o1 spec */ }
    ]
  },
  {
    group: "Anthropic",
    emoji: "🧠",
    items: [
      { /* Claude 3.5 Sonnet spec */ },
      { /* Claude 3 Opus spec */ }
    ]
  }
]
```

--------------------------------

### Interact with ProviderRegistry

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

Use the ProviderRegistry class to get, check, and remove registered providers by name, or retrieve all registered providers.

```python
from chainforge.providers import ProviderRegistry

# Get a registered provider by name
provider_obj = ProviderRegistry.get("My Custom Model")

# Get all registered providers
all_providers = ProviderRegistry.get_all()

# Check if a provider is registered
exists = ProviderRegistry.has("My Custom Model")

# Remove a provider
ProviderRegistry.remove("My Custom Model")
```

--------------------------------

### KeyValue Result Structure

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Example structure for a KeyValue evaluation result, demonstrating a list of dictionaries and the specific KeyValue metric type.

```python
eval_res = {
    'items': [
        {'accuracy': 0.95, 'precision': 0.92},
        {'accuracy': 0.88, 'precision': 0.90}
    ],
    'dtype': 'KeyValue_Numeric'      # More specific KV type
}
```

--------------------------------

### Set OpenAI API Key Environment Variable

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

Set the OPENAI_API_KEY environment variable before starting ChainForge. Changing environment variables after startup requires a server restart.

```bash
export OPENAI_API_KEY="sk-..."
chainforge serve
```

--------------------------------

### Test Evaluator Code Locally

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

Run your evaluation logic locally before deploying to ChainForge. This snippet includes a sample `evaluate` function and the necessary setup for testing with `ResponseInfo`.

```python
# Local test
def evaluate(response):
    # Your evaluation logic
    return result

# Create test ResponseInfo
from chainforge.flask_app import ResponseInfo
test_response = ResponseInfo(
    text="Test response",
    prompt="Test prompt",
    var={},
    meta={},
    llm="test"
)

# Test the function
try:
    result = evaluate(test_response)
    print(f"Result: {result}, Type: {type(result)}")
except Exception as e:
    print(f"Error: {e}")
```

--------------------------------

### Call ChainForge API via cURL

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/00-START-HERE.md

Example of how to call the ChainForge API using cURL to execute Python code for evaluation. Requires specifying the endpoint, content type, and a JSON payload with execution details.

```bash
curl -X POST http://localhost:8000/app/executepy \
  -H "Content-Type: application/json" \
  -d '{
    "id": "eval1",
    "code": "def evaluate(response): return len(response.text)",
    "responses": [{...}],
    "scope": "response"
  }'
```

--------------------------------

### POST /app/initCustomProvider

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Initializes a custom LLM provider using user-provided Python code. This allows for flexible integration of custom language models.

```APIDOC
## POST /app/initCustomProvider

### Description
Initializes a custom LLM provider from user-provided Python code.

### Method
POST

### Endpoint
/app/initCustomProvider

### Request Body
- **code** (string) - Required - Python code defining the custom provider. Example:
```python
from chainforge.providers import provider
@provider(...)
def my_provider(prompt, model, **kwargs):
    return response
```
```

--------------------------------

### Flask Error: Runtime Evaluation Error Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

Example JSON error response for /app/executepy when an error occurs during evaluator code execution. Includes details of the exception and optional 'logs' from the execution.

```json
{
  "error": "Error encountered while trying to run \"evaluate\" method:\nTypeError: unsupported operand type(s) for +: 'str' and 'int'",
  "logs": ["Debug print output", "More output"]
}
```

--------------------------------

### Flask Error: Code Compilation Error Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

Example JSON error response for /app/executepy when evaluator code fails to compile. This includes Python syntax errors or issues with required functions/imports.

```json
{
  "error": "Could not compile evaluator code. Error message:\n  File \"<string>\", line 1\n    def evaluate(x\nSyntaxError: invalid syntax"
}
```

--------------------------------

### Define Build and Static Directories

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Sets paths to the compiled React frontend build and its static assets.

```python
BUILD_DIR = os.path.join(
    os.path.dirname(os.path.realpath(__file__)),
    'react-server', 'build'
)
STATIC_DIR = os.path.join(BUILD_DIR, 'static')
```

--------------------------------

### Docker Deployment

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Provides commands for building a Docker image for ChainForge and running it. Supports passing environment variables for API keys and mounting custom directories for data persistence.

```bash
# Build image
docker build -t chainforge .

# Run with environment variables
docker run -p 8000:8000 \
  -e OPENAI_API_KEY=sk-... \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  chainforge

# Or with custom directory
docker run -p 8000:8000 \
  -v /host/chainforge:/data \
  chainforge serve --host 0.0.0.0 --dir /data
```

--------------------------------

### Build ChainForge Docker Image

Source: https://github.com/ianarawjo/chainforge/blob/main/README.md

Build the Docker image for ChainForge. This is used for local deployment.

```bash
docker build -t chainforge .
```

--------------------------------

### GET /api/flowExists/<filename>

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Checks if a flow file with the specified filename exists on the server.

```APIDOC
## GET /api/flowExists/<filename>

### Description
Checks if a flow file with the specified filename exists on the server.

### Method
GET

### Endpoint
/api/flowExists/<filename>

### Parameters
#### Path Parameters
- **filename** (string) - Required - Name of .cforge file (with extension)

### Response
#### Success Response (200)
- **exists** (boolean) - True if the flow file exists, false otherwise.

#### Response Example
{
  "exists": true
}
```

--------------------------------

### ChainForge CLI Help Output

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Displays the help message shown when `chainforge` is called without arguments, outlining available subcommands.

```bash
$ chainforge
usage: chainforge [-h] {serve} ...

Chainforge command line tool

optional arguments:
  -h, --help  show this help message and exit

subcommands:
  {serve}
    serve     Start Chainforge server
```

--------------------------------

### Response for Initializing Custom Provider

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

The response from POST /app/initCustomProvider indicates success and provides details about the initialized provider, including its name, emoji, models, rate limit, and settings schema.

```json
{
  "success": true,
  "provider": {
    "name": "My Provider",
    "emoji": "🔧",
    "models": ["model1", "model2"],
    "rate_limit": "sequential",
    "settings_schema": { ... }
  }
}
```

--------------------------------

### GET /api/flows/<filename>

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Retrieves the content of a specific flow file. The filename should include the .cforge extension.

```APIDOC
## GET /api/flows/<filename>

### Description
Retrieves the content of a specific flow file. The filename should include the .cforge extension.

### Method
GET

### Endpoint
/api/flows/<filename>

### Parameters
#### Path Parameters
- **filename** (string) - Required - Name of .cforge file (with extension)

### Response
#### Success Response (200)
- Raw JSON flow data

#### Status Codes
- 200: Flow retrieved successfully
- 400: File not found or access denied
```

--------------------------------

### Configure Server Host

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Specifies the hostname or IP address for the server to listen on. Use '0.0.0.0' to accept connections from any IP address, enabling remote access.

```bash
chainforge serve [--host HOST]
chainforge serve --host localhost
chainforge serve --host 0.0.0.0
chainforge serve --host 192.168.1.100
```

--------------------------------

### Categorical Result Structure

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Example structure for a categorical evaluation result, showing individual categories and the metric type.

```python
eval_res = {
    'items': ['pass', 'fail', 'pass'],   # Individual categories
    'dtype': 'Categorical'                # Metric type name
}
```

--------------------------------

### Initialize Custom Provider via Backend API

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

The backend API endpoint POST /app/initCustomProvider allows initializing a custom provider by sending its Python code. The response includes provider details if successful.

```json
{
  "code": "@provider(...)\ndef my_provider(prompt, model=None, chat_history=None, **kwargs):\n    return response"
}
```

--------------------------------

### Configuration File Not Found Error Message

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

This JSON indicates that a requested configuration file could not be found.

```json
{
  "error": "Configuration \"<name>\" not found."
}
```

--------------------------------

### GET /media/<uid>

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Retrieves uploaded media by its unique identifier (UID). This endpoint allows access to stored files.

```APIDOC
## GET /media/<uid>

### Description
Retrieves uploaded media by UID.

### Method
GET

### Endpoint
/media/<uid>

### Response
#### Success Response (200)
- File binary data
```

--------------------------------

### GET /api/flows

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Lists all saved flows available on the server. Returns a JSON object containing an array of flow filenames.

```APIDOC
## GET /api/flows

### Description
Lists all saved flows available on the server. Returns a JSON object containing an array of flow filenames.

### Method
GET

### Endpoint
/api/flows

### Response
#### Success Response (200)
- **flows** (array) - An array of strings, where each string is the filename of a saved flow.

#### Response Example
{
  "flows": [
    "flow_name_1.cforge",
    "flow_name_2.cforge"
  ]
}
```

--------------------------------

### Running Multiple Instances

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Launches multiple independent ChainForge server instances, each with its own port, data directory, and security settings. This allows for isolated environments or testing different configurations simultaneously.

```bash
# Instance 1 (port 8000)
chainforge serve --port 8000 --dir /data/instance1

# Instance 2 (port 8001, separate flows)
chainforge serve --port 8001 --dir /data/instance2

# Instance 3 (port 8002, encrypted)
chainforge serve --port 8002 --dir /data/instance3 --secure all
```

--------------------------------

### ChainForge File Structure

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/README.md

Illustrates the directory layout of the ChainForge project. It highlights key files and directories such as the Flask application, provider protocols, security utilities, and the React frontend components.

```tree
chainforge/
├── app.py                          # CLI entry point
├── flask_app.py                    # Flask server (1378 lines)
├── providers/
│   ├── __init__.py
│   └── protocol.py                 # Custom provider protocol
├── security/
│   ├── __init__.py
│   ├── password_utils.py           # Encryption utilities
│   └── secure_save.py              # Encrypted file I/O
├── react-server/
│   ├── src/
│   │   ├── App.tsx                 # React root
│   │   ├── backend/
│   │   │   ├── typing.ts           # Type definitions
│   │   │   ├── models.ts           # Model enum
│   │   │   ├── backend.ts          # Backend client
│   │   │   ├── ai.ts               # AI integrations
│   │   │   ├── cache.ts            # Caching layer
│   │   │   ├── query.ts            # Query builders
│   │   │   └── utils.ts            # Utilities
│   │   ├── components/             # React components
│   │   └── ...
│   └── package.json
└── examples/                       # Example flows
```

--------------------------------

### Set LLM API Keys as Environment Variables

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/README.md

Configure API keys for various LLM providers by exporting them as environment variables before running ChainForge.

```bash
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export AWS_ACCESS_KEY_ID="..."
export AWS_SECRET_ACCESS_KEY="..."
```

--------------------------------

### Initialize Custom Provider Code

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Provides the Python code structure for initializing a custom LLM provider. Uses the `@provider` decorator.

```python
from chainforge.providers import provider
@provider(...)
def my_provider(prompt, model, **kwargs):
    return response
```

--------------------------------

### POST /app/fetchOpenAIEval

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Fetches a pre-converted OpenAI evaluation flow. It checks the local cache first and downloads from GitHub if not found.

```APIDOC
## POST /app/fetchOpenAIEval

### Description
Fetches a pre-converted OpenAI evaluation flow. It checks the local cache first and downloads from GitHub if not found.

### Method
POST

### Endpoint
/app/fetchOpenAIEval

### Parameters
#### Request Body
- **name** (string) - Required - The name of the OpenAI evaluation to fetch.

### Response
#### Success Response (200)
- **data** (object) - Contains the data of the fetched evaluation.

#### Response Example
{
  "data": {}
}
```

--------------------------------

### Run ChainForge as a Python Module

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Illustrates how to invoke ChainForge as a Python module, specifying a custom port.

```bash
python -m chainforge serve --port 8080
```

--------------------------------

### POST /app/fetchEnvironAPIKeys

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Retrieves API keys that are loaded from environment variables. This is useful for securely accessing external services without hardcoding keys.

```APIDOC
## POST /app/fetchEnvironAPIKeys

### Description
Retrieves API keys loaded from environment variables.

### Method
POST

### Endpoint
/app/fetchEnvironAPIKeys

### Response
#### Success Response (200)
- **OpenAI** (string) - OpenAI API Key
- **Anthropic** (string) - Anthropic API Key
- **Google** (string) - Google API Key
- **Azure_OpenAI** (string) - Azure OpenAI API Key
- **HuggingFace** (string) - HuggingFace API Key
- **Together** (string) - Together API Key
- **DeepSeek** (string) - DeepSeek API Key
- **AWS_Access_Key_ID** (string) - AWS Access Key ID
- **AWS_Secret_Access_Key** (string) - AWS Secret Access Key
- **AWS_Region** (string) - AWS Region
```

--------------------------------

### Initialize Custom Provider Backend API

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

API endpoint to initialize a custom provider using Python code from the frontend.

```APIDOC
## Initialize Custom Provider Backend API

### Description
This endpoint allows the frontend to send Python code to initialize a new custom provider within ChainForge.

### Method
POST

### Endpoint
`/app/initCustomProvider`

### Parameters
#### Request Body
- **code** (string) - Required - The Python code defining the custom provider, including the `@provider` decorator and implementation.

### Request Example
```json
{
  "code": "@provider(name=\"My Provider\", emoji=\"🔧\", settings_schema=settings_schema)\ndef my_provider(prompt, model=None, chat_history=None, **kwargs):\n    return \"response\""
}
```

### Response
#### Success Response (200)
- **success** (boolean) - Indicates if the initialization was successful.
- **provider** (object) - Details of the initialized provider.
  - **name** (string) - The name of the provider.
  - **emoji** (string) - The emoji representing the provider.
  - **models** (list of strings) - List of available models.
  - **rate_limit** (string or number) - The rate limiting configuration.
  - **settings_schema** (object) - The schema defining the provider's settings.

#### Response Example
```json
{
  "success": true,
  "provider": {
    "name": "My Provider",
    "emoji": "🔧",
    "models": ["model1", "model2"],
    "rate_limit": "sequential",
    "settings_schema": { ... }
  }
}
```
```

--------------------------------

### Inconsistent Dictionary Keys Error Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

This Python code illustrates an error where evaluation results contain dictionaries with inconsistent keys across different responses.

```python
# Response 1
{"accuracy": 0.95, "precision": 0.92}

# Response 2
{"accuracy": 0.88, "recall": 0.90}  # Different keys!

# Raises error
```

--------------------------------

### ChainForge Main Module Entry

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Shows how the main module in `chainforge/app.py` can be executed directly using the Python interpreter.

```python
from chainforge.app import main

if __name__ == "__main__":
    main()
```

--------------------------------

### GET /api/proxyImage

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Proxies image requests to bypass CORS issues. This endpoint allows fetching images from external URLs without encountering cross-origin restrictions.

```APIDOC
## GET /api/proxyImage

### Description
Proxies image requests to bypass CORS issues.

### Method
GET

### Endpoint
/api/proxyImage

### Query Parameters
- **url** (string) - Required - URL of image to proxy

### Response
#### Success Response (200)
- Image binary data
```

--------------------------------

### Configure Data Directory

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Specifies a custom directory for storing ChainForge data, including flows, media, and settings. The directory will be created if it does not exist.

```bash
chainforge serve [--dir DIR]
chainforge serve --dir /home/user/.chainforge_custom
chainforge serve --dir /var/lib/chainforge
chainforge serve --dir ./my_flows
```

--------------------------------

### Team Server Configuration

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Runs the ChainForge server on a specific port and configures it to accept remote connections. Team members can access the server using the server's IP address and the specified port.

```bash
# Run on port 8000, accept remote connections
chainforge serve --host 0.0.0.0 --port 8000

# Team members can access at http://YOUR_IP:8000
```

--------------------------------

### check_typeof_vals Utility Function Examples

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Demonstrates the usage of the check_typeof_vals function to determine the MetricType of different data arrays, including numeric, categorical, KeyValue, and empty arrays.

```python
from chainforge.flask_app import check_typeof_vals, MetricType

# Numeric values
metric_type = check_typeof_vals([42, 45, 40])
assert metric_type == MetricType.Numeric

# Categorical values
metric_type = check_typeof_vals(['pass', 'fail', 'pass'])
assert metric_type == MetricType.Categorical

# Dict values (KeyValue)
metric_type = check_typeof_vals([
    {'accuracy': 0.95, 'precision': 0.92},
    {'accuracy': 0.88, 'precision': 0.90}
])
assert metric_type == MetricType.KeyValue_Numeric

# Empty array
metric_type = check_typeof_vals([])
assert metric_type == MetricType.Empty
```

--------------------------------

### Run ChainForge Docker Container

Source: https://github.com/ianarawjo/chainforge/blob/main/README.md

Run the ChainForge Docker container and map port 8000. Access it at http://127.0.0.1:8000.

```bash
docker run -p 8000:8000 chainforge
```

--------------------------------

### Numeric Metric Result Structure

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Example structure for a numeric evaluation result, including individual scores, mean, median, standard deviation, range, and data type.

```python
eval_res = {
    'items': [42, 45, 40],           # Individual scores
    'mean': 42.33,                   # Average
    'median': 42,                    # Median
    'stdev': 2.05,                   # Standard deviation
    'range': (40, 45),               # (min, max) tuple
    'dtype': 'Numeric'               # Metric type name
}
```

--------------------------------

### GET /mediaToText/<uid>

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/flask-endpoints.md

Converts uploaded media (e.g., PDF, DOCX) to plain text using the MarkItDown library. This is useful for extracting text content from documents.

```APIDOC
## GET /mediaToText/<uid>

### Description
Converts uploaded media (PDF, DOCX, etc) to text using MarkItDown.

### Method
GET

### Endpoint
/mediaToText/<uid>

### Response
#### Success Response (200)
- **text** (string) - Extracted text content from the media.
```

--------------------------------

### Configure Encryption Mode

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/command-line-interface.md

Enables encryption for stored files. Modes include 'off' (no encryption), 'settings' (encrypts settings file only), and 'all' (encrypts all files). A password is required on startup for encrypted modes.

```bash
chainforge serve [--secure MODE]
chainforge serve --secure off
chainforge serve --secure settings
chainforge serve --secure all
```

--------------------------------

### Custom Provider Error Handling

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/custom-providers.md

Demonstrates how custom provider functions should handle errors. Provider functions should return strings or raise exceptions; ChainForge will catch and display runtime errors.

```python
@provider(name="Error Handler", emoji="⚠️")
def error_handler(prompt, model=None, chat_history=None, **kwargs):
    try:
        response = call_api(prompt)
        if not response:
            raise ValueError("Empty response from API")
        return response
    except Exception as e:
        # ChainForge will catch and display this error
        raise RuntimeError(f"API error: {str(e)}")
```

--------------------------------

### Inconsistent Dictionary Value Types Error Example

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

This Python code demonstrates an error where evaluation results contain dictionaries with inconsistent value types for the same key across different responses.

```python
# Response 1
{"score": 95}  # int

# Response 2
{"score": 0.85}  # float

# Raises error
```

--------------------------------

### ChainForge Settings File Structure

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

The main configuration file for ChainForge, typically settings.json, defines API keys, theme, Python interpreter, default prompt settings, and favorite flows.

```json
{
  "apiKeys": {
    "OpenAI": "sk-...",
    "Anthropic": "sk-ant-...",
    "Azure_OpenAI": "...",
    "HuggingFace": "..."
  },
  "theme": "dark",
  "pythonInterpreter": "flask",
  "defaultPromptSettings": {
    "temperature": 0.7,
    "num_responses": 1
  },
  "favoriteFlows": ["flow1.cforge", "flow2.cforge"],
  "customProviders": []
}
```

--------------------------------

### Call a Python Evaluator via API

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/README.md

This example demonstrates how to call a Python evaluator endpoint using a cURL request. It includes the code for the evaluator and specifies the response scope and process type.

```bash
curl -X POST http://localhost:8000/app/executepy \
  -H "Content-Type: application/json" \
  -d '{
    "id": "unique_id",
    "code": "def evaluate(response): return len(response.text)",
    "responses": [{...}],
    "scope": "response",
    "process_type": "evaluator"
  }'
```

--------------------------------

### Define Settings Filename

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/internal-utilities.md

Specifies the filename for user settings and API keys.

```python
SETTINGS_FILENAME = "settings.json"
```

--------------------------------

### Hijack and Revert Python Print Function

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/api-reference/python-evaluation-api.md

Temporarily redirects Python's built-in print() function to capture output into a list. Use HIJACK_PYTHON_PRINT() to start capturing and REVERT_PYTHON_PRINT() to stop and retrieve the logs.

```python
from chainforge.flask_app import HIJACK_PYTHON_PRINT, REVERT_PYTHON_PRINT

HIJACK_PYTHON_PRINT()
print("Debug message")
print("Another message")
logs = REVERT_PYTHON_PRINT()
print(logs)  # Output: ['Debug message', 'Another message']
```

--------------------------------

### Docker Configuration with Environment Variables

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/configuration.md

When deploying ChainForge with Docker, API keys and other sensitive information should be set as environment variables within the container.

```dockerfile
FROM python:3.10
ENV OPENAI_API_KEY=sk-...
ENV ANTHROPIC_API_KEY=sk-ant-...
RUN pip install chainforge
EXPOSE 8000
CMD ["chainforge", "serve", "--host", "0.0.0.0"]
```

--------------------------------

### Cache Directory Creation Error Message

Source: https://github.com/ianarawjo/chainforge/blob/main/_autodocs/errors.md

This JSON indicates a failure to create the 'oaievals' cache directory, often due to permission denied errors.

```json
{
  "error": "Error creating a new directory 'oaievals' at filepath <path>: <error>"
}
```