### Install Dependencies and Start Server Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Installs project dependencies and starts the LM Arena Bridge server. The server listens on http://localhost:8000 and performs initial scraping for configuration. ```bash pip install -r requirements.txt ``` ```bash python -m src.main ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Install all required Python packages listed in the requirements.txt file. This step is crucial before running the application. ```bash pip install -r requirements.txt ``` -------------------------------- ### Python SDK Streaming Example Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Example of using the Python OpenAI library to interact with the streaming chat completions endpoint. ```APIDOC ## Python SDK Streaming Example ### Description This snippet demonstrates how to use the Python `openai` library to receive streaming responses from the chat completions endpoint. ### Code Example ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/api/v1", api_key="sk-lmab-your-api-key") stream = client.chat.completions.create( model="gemini-2.5-pro-exp-03-25", messages=[{"role": "user", "content": "Explain async/await in Python in 2 sentences." }], stream=True ) for chunk in stream: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) ``` ``` -------------------------------- ### Run LM Arena Bridge Application Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Start the LM Arena Bridge server. This command launches the application, making the API available on localhost:8000. ```bash python -m src.main ``` -------------------------------- ### Verify LMArena Bridge Integration with OpenAI Python Library Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Use the `openai` Python library to interact with the LMArena Bridge and verify its compatibility. This example demonstrates listing available models through the bridge. ```python # Verify full round-trip with the openai Python library from openai import OpenAI client = OpenAI( base_url="http://localhost:8000/api/v1", api_key="sk-lmab-your-api-key" ) # List models models = [m.id for m in client.models.list().data] print("Available:", models[:5]) ``` -------------------------------- ### Clone LM Arena Bridge Repository Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Use this command to clone the project repository to your local machine. Ensure you have Git installed. ```bash git clone https://github.com/CloudWaddie/LMArenaBridge.git ``` -------------------------------- ### Configuration Management Functions Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Python functions for getting and saving the bridge configuration. ```APIDOC ## get_config() ### Description Loads and applies defaults to the `config.json` file. ### Language Python ### Usage ```python from src.config import get_config config = get_config() ``` ## save_config(config, preserve_auth_tokens=False) ### Description Atomically saves the provided configuration object to `config.json`. ### Language Python ### Parameters - **config** (dict) - Required - The configuration object to save. - **preserve_auth_tokens** (bool) - Optional - If True, existing auth tokens are preserved. Defaults to False. ### Usage ```python from src.config import save_config # Example: Add a new auth token and save config = get_config() config["auth_tokens"].append("base64-") save_config(config, preserve_auth_tokens=False) # Example: Change admin password and save config["password"] = "my-secure-password" save_config(config) ``` ``` -------------------------------- ### systemd Commands for LMArena Bridge Service Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Commands to enable, start, and check the status of the LMArena Bridge service managed by systemd. ```bash sudo systemctl enable lmarenabridge sudo systemctl start lmarenabridge sudo systemctl status lmarenabridge ``` -------------------------------- ### systemd Service Configuration for LMArena Bridge Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Set up a systemd service file to manage the LMArena Bridge API. This ensures the service starts on boot, restarts on failure, and runs with specified user and environment settings. ```ini [Unit] Description=LMArena Bridge API After=network.target [Service] Type=simple User=youruser WorkingDirectory=/path/to/lmarenabridge Environment="PATH=/path/to/venv/bin" ExecStart=/path/to/venv/bin/python -m src.main Restart=always RestartSec=10 [Install] WantedBy=multi-user.target ``` -------------------------------- ### Chat Completion API Call Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Example of making a chat completion request to the API. Ensure the 'models' variable is populated with available models. ```python response = client.chat.completions.create( model=models[0], messages=[{"role": "user", "content": "Say hello in French."}] ) print(response.choices[0].message.content) # "Bonjour !" ``` -------------------------------- ### Configure systemd Service for LMArena Bridge Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Create a systemd service file to manage the LMArena Bridge as a long-lived background process. This ensures the bridge automatically restarts if it crashes and starts on system boot. ```ini # /etc/systemd/system/lmarenabridge.service [Unit] Description=LMArena Bridge API After=network.target [Service] Type=simple User=youruser WorkingDirectory=/home/youruser/LMArenaBridge Environment="PATH=/home/youruser/.venv/bin" ExecStart=/home/youruser/.venv/bin/python -m src.main Restart=always RestartSec=10 [Install] WantedBy=multi-user.target ``` -------------------------------- ### Non-streaming Chat Completions Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Send a non-streaming request to the chat completions endpoint to get a response from a specified model. ```APIDOC ## POST /api/v1/chat/completions ### Description Send a non-streaming request to the chat completions endpoint. ### Method POST ### Endpoint /api/v1/chat/completions ### Request Body - **model** (string) - Required - The model to use for completion. - **messages** (array) - Required - An array of message objects representing the conversation history. - **role** (string) - Required - The role of the message sender (e.g., "system", "user", "assistant"). - **content** (string) - Required - The content of the message. ### Request Example ```json { "model": "gpt-4o-2024-11-20", "messages": [ {"role": "system", "content": "You are a concise assistant."}, {"role": "user", "content": "What is 2 + 2?"} ] } ``` ### Response #### Success Response (200) - **id** (string) - Unique identifier for the completion. - **object** (string) - Type of object returned (e.g., "chat.completion"). - **created** (integer) - Timestamp of creation. - **model** (string) - The model used for the completion. - **conversation_id** (string) - Identifier for the conversation. - **choices** (array) - An array of completion choices. - **index** (integer) - Index of the choice. - **message** (object) - The message content. - **role** (string) - Role of the message sender. - **content** (string) - The content of the message. - **finish_reason** (string) - The reason the model stopped generating. - **usage** (object) - Token usage information. - **prompt_tokens** (integer) - Number of tokens in the prompt. - **completion_tokens** (integer) - Number of tokens in the completion. - **total_tokens** (integer) - Total tokens used. ### Response Example ```json { "id": "chatcmpl-550e8400-e29b-41d4-a716-446655440000", "object": "chat.completion", "created": 1705312800, "model": "gpt-4o-2024-11-20", "conversation_id": "a3f9b2c1e8d74012", "choices": [ { "index": 0, "message": {"role": "assistant", "content": "4"}, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 28, "completion_tokens": 1, "total_tokens": 29 } } ``` ``` -------------------------------- ### Python OpenAI Library Streaming Request Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt This Python code snippet shows how to use the OpenAI library to make a streaming chat completion request. It requires the 'openai' library to be installed and configured with the bridge's base URL and API key. ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/api/v1", api_key="sk-lmab-your-api-key") stream = client.chat.completions.create( model="gemini-2.5-pro-exp-03-25", messages=[{"role": "user", "content": "Explain async/await in Python in 2 sentences." }], stream=True ) for chunk in stream: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) ``` -------------------------------- ### Get and Remove Auth Tokens Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Selects the next available authentication token in a round-robin fashion, excluding expired or specified tokens. It falls back to a default ephemeral token if none are available. `remove_auth_token` permanently deletes a token from configuration. ```python from src.auth import get_next_auth_token, remove_auth_token, is_arena_auth_token_expired from fastapi import HTTPException # Pick the next healthy token (excludes tokens that already failed this request) failed = set() try: token = get_next_auth_token(exclude_tokens=failed) print(f"Using token: {token[:30]}...") except HTTPException as e: print(f"No tokens available: {e.detail}") # 503 if all tokens exhausted # Check expiry manually if is_arena_auth_token_expired(token, skew_seconds=60): print("Token expires within 60 seconds, rotating...") failed.add(token) token = get_next_auth_token(exclude_tokens=failed) # Permanently remove a bad token remove_auth_token(token) ``` -------------------------------- ### Navigate to Project Directory Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md After cloning, change your current directory to the project's root folder. This is necessary to run subsequent commands. ```bash cd LMArenaBridge ``` -------------------------------- ### List Available LMArena Models with Python Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Demonstrates how to list available LMArena models using the Python `openai` library, configured to use the LM Arena Bridge as the base URL. ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/api/v1", api_key="sk-lmab-your-api-key") models = client.models.list() for m in models.data: print(m.id, "- ", m.owned_by) ``` -------------------------------- ### Initialize AsyncCamoufox in Non-Headless Mode Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/INFO.md Ensure the browser is initialized with `headless=False` for interactive browser sessions. ```python async with AsyncCamoufox(headless=False, main_world_eval=True) as browser: ``` -------------------------------- ### Configure OpenWebUI OpenAI Connection Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Instructions for setting up OpenWebUI to use the LM Arena Bridge as its backend. Set the API Base URL to the bridge's endpoint. ```bash http://localhost:8000/api/v1 ``` -------------------------------- ### Manage Configuration Programmatically (Python) Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Use `get_config()` to load and `save_config()` to atomically write the bridge's configuration. Handles default values and prevents concurrent write clobbering. ```python from src.config import get_config, save_config config = get_config() # config = { # "password": "admin", # "auth_tokens": ["base64-", "base64-"], # "auth_token": "", # legacy single-token field # "cf_clearance": "GAz...", # automatically managed # "api_keys": [{"name": "Default", "key": "sk-lmab-...", "rpm": 60, "created": 1704236400}], # "usage_stats": {"gpt-4o-2024-11-20": 42}, # "persist_arena_auth_cookie": True, # "camoufox_proxy_window_mode": "hide", # "camoufox_fetch_window_mode": "hide", # "chrome_fetch_window_mode": "hide" # } # Add a new auth token programmatically config["auth_tokens"].append("base64-") save_config(config, preserve_auth_tokens=False) # Change admin password config["password"] = "my-secure-password" save_config(config) # Disable auto-persisting browser session cookies config["persist_arena_auth_cookie"] = False save_config(config) ``` -------------------------------- ### Run Single Test File Command Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/CLAUDE.md Run all tests within a specific test file. ```bash python -m pytest tests/test_stream_403_recaptcha_retries.py -v ``` -------------------------------- ### List Available LMArena Models Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Retrieves a list of available models from LMArena that are compatible with the OpenAI API. Requires an API key for authentication. ```bash curl http://localhost:8000/api/v1/models \ -H "Authorization: Bearer sk-lmab-your-api-key" ``` -------------------------------- ### Configure OpenWebUI to Connect to LMArena Bridge Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Set up OpenWebUI to use the LMArena Bridge as its backend by configuring the API Base URL and API Key in the OpenWebUI settings. The bridge mimics the OpenAI API contract. ```text OpenWebUI Settings → Connections → OpenAI: API Base URL : http://localhost:8000/api/v1 API Key : (any value, e.g. "none" — or a real key if you configured one) ``` -------------------------------- ### Syntax Check Command Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/CLAUDE.md Perform a syntax check on the main Python file using the ast module. ```bash python -c "import ast; ast.parse(open('src/main.py', encoding='utf-8').read()); print('OK')" ``` -------------------------------- ### Run Single Test by Name Command Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/CLAUDE.md Execute a specific test case by its full name within a test file. ```bash python -m pytest tests/test_stream_403_recaptcha_retries.py::TestStream403RecaptchaRetries::test_stream_403_recaptcha_validation_failed_retries_with_fresh_token -v ``` -------------------------------- ### Run All Tests Command Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/CLAUDE.md Execute all tests in the repository using pytest. ```bash python -m pytest tests/ ``` -------------------------------- ### Manage Bridge via Dashboard API (Bash) Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Use curl commands to interact with the admin dashboard API for managing tokens, API keys, and triggering refreshes. Ensure you are logged in first. ```bash # Login (default password: admin) curl -c cookies.txt http://localhost:8000/login \ -d "password=admin" -X POST -L ``` ```bash # Trigger a token + model refresh via the dashboard API curl -b cookies.txt http://localhost:8000/refresh-tokens -X POST -L ``` ```bash # Add a new arena-auth-prod-v1 token curl -b cookies.txt http://localhost:8000/add-auth-token \ -d "new_auth_token=base64-" -X POST -L ``` ```bash # Create a new API key named "MyApp" with 120 RPM curl -b cookies.txt http://localhost:8000/create-key \ -d "name=MyApp&rpm=120" -X POST -L ``` ```bash # Delete an API key curl -b cookies.txt http://localhost:8000/delete-key \ -d "key_id=sk-lmab-your-key-id" -X POST -L ``` -------------------------------- ### List Available Models Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Returns an OpenAI-compatible model list filtered to models that have at least one output capability (text, image, or search) and a known organization. Stealth/anonymous models without an organization are excluded. ```APIDOC ## GET /api/v1/models ### Description Returns an OpenAI-compatible model list filtered to models that have at least one output capability (text, image, or search) and a known organization. Stealth/anonymous models without an organization are excluded. ### Method GET ### Endpoint /api/v1/models ### Parameters #### Query Parameters - **Authorization** (string) - Required - Bearer token for authentication (e.g., "Bearer sk-lmab-your-api-key") ### Response #### Success Response (200) - **object** (string) - Type of the response object, always "list" - **data** (array) - An array of model objects - **id** (string) - The unique identifier for the model - **object** (string) - Type of the object, always "model" - **created** (integer) - Timestamp of model creation - **owned_by** (string) - The organization that owns the model #### Response Example ```json { "object": "list", "data": [ { "id": "gpt-4o-2024-11-20", "object": "model", "created": 1705312800, "owned_by": "OpenAI" }, { "id": "claude-3-5-sonnet-20241022", "object": "model", "created": 1705312800, "owned_by": "Anthropic" }, { "id": "gemini-2.5-pro-exp-03-25", "object": "model", "created": 1705312800, "owned_by": "Google" } ] } ``` ``` -------------------------------- ### Image Input Support Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Send images to vision-capable models by embedding them in the `image_url` content part. ```APIDOC ## Image Input Support ### Description Send images to vision-capable models. Images embedded in the OpenAI `image_url` content part are decoded, uploaded, and attached to the request. Maximum image size is 10 MB; supported MIME types are `image/png`, `image/jpeg`, `image/gif`, `image/webp`, and `image/svg+xml`. ### Request Body (Example using Python SDK) ```python import base64, pathlib from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/api/v1", api_key="sk-lmab-your-api-key") # Load a local image and base64-encode it image_bytes = pathlib.Path("screenshot.png").read_bytes() b64 = base64.b64encode(image_bytes).decode() response = client.chat.completions.create( model="gpt-4o-2024-11-20", # must be a vision-capable model messages=[ { "role": "user", "content": [ {"type": "text", "text": "What is in this image?"}, { "type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"} } ] } ] ) print(response.choices[0].message.content) ``` ``` -------------------------------- ### Anthropic SDK Compatibility with LMArena Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt This Python code demonstrates how to use the Anthropic SDK to interact with the LMArena bridge. The client is configured to point to the bridge's base URL, allowing Anthropic SDK-formatted requests to be translated and processed. ```python import anthropic # Point Anthropic client at the bridge client = anthropic.Anthropic( base_url="http://localhost:8000", api_key="sk-lmab-your-api-key" ) messsage = client.messages.create( model="claude-3-5-sonnet-20241022", max_tokens=1024, system="Answer concisely.", messages=[ {"role": "user", "content": "What is the capital of France?"} ] ) print(message.content[0].text) # "Paris" # Streaming with Anthropic SDK: with client.messages.stream( model="claude-3-5-sonnet-20241022", max_tokens=512, messages=[{"role": "user", "content": "Count to 5."}] ) as stream: for text in stream.text_stream: print(text, end="", flush=True) ``` -------------------------------- ### Enable Debug Mode in LMArena Bridge Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/README.md Set the DEBUG variable to True in src/main.py to enable detailed logging and debugging information. Keep this OFF in production. ```python # In src/main.py DEBUG = True # Set to True for detailed logging ``` -------------------------------- ### Configure Nginx as a Reverse Proxy for LMArena Bridge Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Set up Nginx to proxy requests to the LMArena Bridge, enabling TLS termination and proper header forwarding. The `proxy_buffering off` directive is crucial for real-time Server-Sent Events. ```nginx # /etc/nginx/sites-available/lmarenabridge server { listen 443 ssl; server_name api.yourdomain.com; ssl_certificate /path/to/fullchain.pem; ssl_certificate_key /path/to/privkey.pem; location / { proxy_pass http://127.0.0.1:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering off; # required for SSE streaming proxy_cache off; proxy_read_timeout 600s; } } ``` -------------------------------- ### Streaming Chat Completion Request Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt This cURL command demonstrates how to make a streaming request to the chat completions endpoint. The 'stream': true parameter enables Server-Sent Events (SSE) for real-time responses. Replace 'sk-lmab-your-api-key' with your actual API key. ```bash curl http://localhost:8000/api/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer sk-lmab-your-api-key" \ -d '{ "model": "claude-3-5-sonnet-20241022", "messages": [{"role": "user", "content": "Write a haiku about Python."}], "stream": true }' ``` -------------------------------- ### Debug SSE Stream Endpoint Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Verify client's ability to receive streaming responses using a minimal Server-Sent Events stream. ```APIDOC ## GET /api/v1/_debug/stream ### Description Emits a minimal Server-Sent Events stream to verify client's ability to receive streaming responses. Requires a valid API key. ### Method GET ### Endpoint /api/v1/_debug/stream ### Response #### Success Response (200) Server-Sent Events (SSE) stream. - **data: {"ok":true}** - **data: [DONE]** ### Response Example (SSE) ``` : keep-alive data: {"ok":true} data: [DONE] ``` ``` -------------------------------- ### Calculate Backoff Wait Times Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Provides helper functions for calculating wait times during retries. `get_rate_limit_backoff_seconds` handles upstream 429 responses using the `Retry-After` header, with a maximum cap. `get_general_backoff_seconds` implements a standard exponential backoff for other retry scenarios. ```python from src.constants import get_rate_limit_backoff_seconds, get_general_backoff_seconds # With Retry-After header value "30" print(get_rate_limit_backoff_seconds("30", attempt=0)) # 30 print(get_rate_limit_backoff_seconds("30", attempt=1)) # 30 # Without header — exponential: 5, 10, 20, 40 → capped at 300 print(get_rate_limit_backoff_seconds(None, attempt=0)) # 5 print(get_rate_limit_backoff_seconds(None, attempt=1)) # 10 print(get_rate_limit_backoff_seconds(None, attempt=2)) # 20 print(get_rate_limit_backoff_seconds(None, attempt=3)) # 40 # General backoff (for non-429 retries): 2, 4, 8, 16, 30 (capped) for i in range(5): print(get_general_backoff_seconds(i)) # 2, 4, 8, 16, 30 ``` -------------------------------- ### Userscript Proxy for LMArena Requests (JavaScript) Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt A Tampermonkey userscript skeleton that polls the bridge for jobs, executes the LMArena fetch in the browser, and pushes response chunks back. Requires `proxy_secret` to be set in `config.json`. ```javascript // Tampermonkey userscript skeleton (conceptual) // ==UserScript== // @name LMArena Bridge Proxy // @match https://arena.ai/* // ==/UserScript== const BRIDGE_URL = "http://localhost:8000"; const SECRET = "your-proxy-secret"; // set proxy_secret in config.json async function poll() { const res = await fetch(`${BRIDGE_URL}/api/v1/userscript/poll`, { method: "POST", headers: { "Content-Type": "application/json", "X-Proxy-Secret": SECRET }, body: JSON.stringify({ timeout_seconds: 25 }) }); if (res.status === 204) return; // no job available const { job_id, payload } = await res.json(); // Execute the real LMArena fetch inside this browser tab ... const upstream = await fetch(payload.url, { method: payload.method, body: JSON.stringify(payload.body), headers: payload.headers }); // Push the status code first await fetch(`${BRIDGE_URL}/api/v1/userscript/push`, { method: "POST", headers: { "Content-Type": "application/json", "X-Proxy-Secret": SECRET }, body: JSON.stringify({ job_id, status: upstream.status, upstream_fetch_started: true }) }); // Stream chunks back const reader = upstream.body.getReader(); const lines = []; while (true) { const { done, value } = await reader.read(); if (done) break; lines.push(new TextDecoder().decode(value)); } await fetch(`${BRIDGE_URL}/api/v1/userscript/push`, { method: "POST", headers: { "Content-Type": "application/json", "X-Proxy-Secret": SECRET }, body: JSON.stringify({ job_id, lines, done: true }) }); } setInterval(poll, 1000); ``` -------------------------------- ### Anthropic SDK Compatibility Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Use the Anthropic SDK to interact with LMArena by pointing the client to the bridge URL. ```APIDOC ## POST /api/v1/messages — Anthropic SDK compatibility ### Description Accepts Anthropic SDK-format requests and converts them to OpenAI format for LMArena. This allows code written for the Anthropic Python library to work without modification. ### Method POST ### Endpoint /api/v1/messages ### Parameters #### Request Body (Anthropic SDK format) - **model** (string) - Required - The model to use. - **max_tokens** (integer) - Required - The maximum number of tokens to generate. - **system** (string) - Optional - System prompt. - **messages** (array) - Required - An array of message objects. - **role** (string) - Required - Role of the message sender. - **content** (string or array) - Required - Content of the message. ### Request Example (Non-streaming) ```python import anthropic client = anthropic.Anthropic( base_url="http://localhost:8000", api_key="sk-lmab-your-api-key" ) message = client.messages.create( model="claude-3-5-sonnet-20241022", max_tokens=1024, system="Answer concisely.", messages=[ {"role": "user", "content": "What is the capital of France?"} ] ) print(message.content[0].text) ``` ### Request Example (Streaming) ```python import anthropic client = anthropic.Anthropic( base_url="http://localhost:8000", api_key="sk-lmab-your-api-key" ) with client.messages.stream( model="claude-3-5-sonnet-20241022", max_tokens=512, messages=[{"role": "user", "content": "Count to 5."}] ) as stream: for text in stream.text_stream: print(text, end="", flush=True) ``` ``` -------------------------------- ### Streaming Chat Completions Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Send a streaming request to the chat completions endpoint to receive responses in chunks. ```APIDOC ## POST /api/v1/chat/completions (Streaming) ### Description Send a streaming request to the chat completions endpoint to receive responses in chunks. ### Method POST ### Endpoint /api/v1/chat/completions ### Request Body - **model** (string) - Required - The model to use for completion. - **messages** (array) - Required - An array of message objects representing the conversation history. - **role** (string) - Required - The role of the message sender (e.g., "system", "user", "assistant"). - **content** (string) - Required - The content of the message. - **stream** (boolean) - Optional - Set to true to enable streaming. ### Request Example ```json { "model": "claude-3-5-sonnet-20241022", "messages": [{"role": "user", "content": "Write a haiku about Python."}], "stream": true } ``` ### Response #### Success Response (200) Server-Sent Events (SSE) stream where each chunk is prefixed with `data: `. - **id** (string) - Unique identifier for the completion. - **object** (string) - Type of object returned (e.g., "chat.completion.chunk"). - **choices** (array) - An array of completion choices. - **delta** (object) - The change in the message content. - **role** (string) - Role of the message sender (if applicable). - **content** (string) - The content of the message chunk. ### Response Example (SSE stream) ``` data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":""},"index":0}]} data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"Serpents"},"index":0}]} data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":" of logic"},"index":0}]} data: [DONE] ``` ``` -------------------------------- ### Chat Completions Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt The core endpoint of the bridge. Accepts the standard OpenAI chat completions request body and proxies it to LMArena's internal streaming API. Supports both streaming (`"stream": true`) and non-streaming responses. Manages multi-turn conversation state automatically by hashing the API key + model + first user message to produce a deterministic `conversation_id`. On a 429 response, tokens are rotated and the request is retried with exponential backoff. On reCAPTCHA failures (403), a fresh token is minted and the request retried. System messages are prepended to the first user message since LMArena does not have a native system role. ```APIDOC ## POST /api/v1/chat/completions ### Description The core endpoint of the bridge. Accepts the standard OpenAI chat completions request body and proxies it to LMArena's internal streaming API. Supports both streaming (`"stream": true`) and non-streaming responses. Manages multi-turn conversation state automatically by hashing the API key + model + first user message to produce a deterministic `conversation_id`. On a 429 response, tokens are rotated and the request is retried with exponential backoff. On reCAPTCHA failures (403), a fresh token is minted and the request retried. System messages are prepended to the first user message since LMArena does not have a native system role. ### Method POST ### Endpoint /api/v1/chat/completions ### Parameters #### Query Parameters - **Authorization** (string) - Required - Bearer token for authentication (e.g., "Bearer sk-lmab-your-api-key") #### Request Body - **model** (string) - Required - The ID of the model to use for completion. - **messages** (array) - Required - A list of messages comprising the conversation so far. - **role** (string) - Required - The role of the author of the message ('system', 'user', or 'assistant'). - **content** (string) - Required - The content of the message. - **stream** (boolean) - Optional - Whether to stream back partial message deltas as they become available. - **temperature** (number) - Optional - Controls randomness. Lower values make output more deterministic. - **top_p** (number) - Optional - Controls diversity via nucleus sampling. - **n** (integer) - Optional - How many chat completion choices to generate for each input message. - **stop** (string or array) - Optional - Up to 4 sequences where the API will stop generating further tokens. - **max_tokens** (integer) - Optional - The maximum number of tokens to generate in the completion. - **presence_penalty** (number) - Optional - Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. - **frequency_penalty** (number) - Optional - Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat itself. - **logit_bias** (object) - Optional - A map of token IDs to be biased, specified in JSON format. Use this to pass `logprob` as a JSON object. - **user** (string) - Optional - A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. ### Request Example ```json { "model": "gpt-4o-2024-11-20", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Who won the world series in 2020?"} ], "stream": true } ``` ### Response #### Success Response (200) - **id** (string) - Unique identifier for the completion. - **object** (string) - Type of the object, e.g., "chat.completion" or "chat.completion.chunk". - **created** (integer) - Unix timestamp of creation. - **model** (string) - The model used for the completion. - **choices** (array) - A list of completion choices. - **index** (integer) - Index of the choice. - **message** (object) - The message content. - **role** (string) - Role of the message author ('assistant'). - **content** (string) - The content of the message. - **finish_reason** (string) - The reason the model stopped generating tokens (e.g., "stop", "length"). - **usage** (object) - Usage statistics for the completion. - **prompt_tokens** (integer) - Number of tokens in the prompt. - **completion_tokens** (integer) - Number of tokens in the completion. - **total_tokens** (integer) - Total tokens used. #### Response Example (Non-streaming) ```json { "id": "chatcmpl-123", "object": "chat.completion", "created": 1677652288, "model": "gpt-4o-2024-11-20", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 10, "completion_tokens": 15, "total_tokens": 25 } } ``` #### Response Example (Streaming Chunk) ```json { "id": "chatcmpl-123", "object": "chat.completion.chunk", "created": 1677652288, "model": "gpt-4o-2024-11-20", "choices": [ { "index": 0, "delta": { "role": "assistant", "content": "The Los Angeles" }, "finish_reason": null } ] } ``` ``` -------------------------------- ### Upload Image to LMArena R2 Storage Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Uploads image data to LMArena's Cloudflare R2 bucket. This process involves requesting a signed upload URL, performing a PUT request with the image bytes, and then requesting a signed download URL. Results are cached by MD5 hash for approximately one hour. ```python import asyncio, pathlib from src.main import upload_image_to_lmarena async def main(): image_data = pathlib.Path("photo.jpg").read_bytes() result = await upload_image_to_lmarena( image_data=image_data, mime_type="image/jpeg", filename="photo.jpg" ) if result: key, download_url = result print(f"Uploaded key: {key}") print(f"Download URL: {download_url[:80]}...") # Attach to a chat completions request: attachment = { "name": key, "contentType": "image/jpeg", "url": download_url } else: print("Upload failed — check Next-Action IDs in config.json") asyncio.run(main()) ``` -------------------------------- ### Chat Completions Endpoint Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt The primary endpoint for generating chat completions. It proxies requests to LMArena's internal API and supports streaming responses. Handles conversation state, token rotation, and reCAPTCHA challenges. ```bash # Example usage for POST /api/v1/chat/completions would go here, but is not provided in the source. ``` -------------------------------- ### Manage reCAPTCHA v3 Tokens Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Handles the lifecycle of reCAPTCHA v3 tokens. It prioritizes returning a cached token if available and unexpired. If not, it mints a new token by launching a headless browser. `get_cached_recaptcha_token` only accesses the cache. ```python import asyncio from src.recaptcha import refresh_recaptcha_token, get_cached_recaptcha_token async def main(): # Try cached token first (no browser launch) token = get_cached_recaptcha_token() if token: print(f"Cached reCAPTCHA token: {token[:20]}...") else: # Mint a fresh token via Chrome/Camoufox automation token = await refresh_recaptcha_token(force_new=True) if token: print(f"Fresh reCAPTCHA v3 token: {token[:20]}...") else: print("Failed to mint reCAPTCHA token — bridge may be blocked") asyncio.run(main()) ``` -------------------------------- ### Sending Images to Vision-Capable Models Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt This Python code shows how to send image data to a vision-capable model using the OpenAI client. The image is base64-encoded and included in the 'image_url' content part of the message. Ensure the model supports image input. ```python import base64, pathlib from openai import OpenAI client = OpenAI(base_url="http://localhost:8000/api/v1", api_key="sk-lmab-your-api-key") # Load a local image and base64-encode it image_bytes = pathlib.Path("screenshot.png").read_bytes() b64 = base64.b64encode(image_bytes).decode() response = client.chat.completions.create( model="gpt-4o-2024-11-20", # must be a vision-capable model messages=[ { "role": "user", "content": [ {"type": "text", "text": "What is in this image?"}, { "type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"} } ] } ] ) print(response.choices[0].message.content) ``` -------------------------------- ### Lazy Import Helper Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/CLAUDE.md This helper function ensures lazy importing of the main module, useful for testing to intercept calls. ```python def _m(): from . import main return main ``` -------------------------------- ### Debug SSE Stream Endpoint Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt This cURL command targets the debug SSE stream endpoint to verify client-side SSE reception. It requires a valid API key for authentication. ```bash curl http://localhost:8000/api/v1/_debug/stream \ -H "Authorization: Bearer sk-lmab-your-api-key" ``` -------------------------------- ### Add Rate Limiting Dependency to FastAPI Endpoint Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Inject the `rate_limit_api_key` dependency into a custom FastAPI endpoint to enforce per-key rate limiting. This ensures that API requests are controlled based on configured limits. ```python from fastapi import Depends from src.main import app, rate_limit_api_key # The dependency is already active on /api/v1/chat/completions and /api/v1/models. # To add it to a custom endpoint: @app.get("/my-custom-endpoint") async def my_endpoint(api_key: dict = Depends(rate_limit_api_key)): # api_key = {"name": "MyApp", "key": "sk-lmab-...", "rpm": 120} return {"message": f"Hello, {api_key['name']}!"} # Testing rate limiting behaviour: import time, requests for i in range(65): # exceeds default 60 RPM r = requests.get( "http://localhost:8000/api/v1/models", headers={"Authorization": "Bearer sk-lmab-your-api-key"} ) if r.status_code == 429: retry_after = r.headers.get("Retry-After", "?") print(f"Rate limited on request {i+1}. Retry-After: {retry_after}s") break ``` -------------------------------- ### Verify Server Health Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Checks the operational status of the LM Arena Bridge server. It verifies Cloudflare clearance, model loading, and API key configuration. ```bash curl http://localhost:8000/api/v1/health ``` -------------------------------- ### Non-streaming Chat Completion Request Source: https://context7.com/cloudwaddie/lmarenabridge/llms.txt Use this cURL command to send a non-streaming request to the chat completions endpoint. Ensure you replace 'sk-lmab-your-api-key' with your actual API key. ```bash curl http://localhost:8000/api/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer sk-lmab-your-api-key" \ -d '{ "model": "gpt-4o-2024-11-20", "messages": [ {"role": "system", "content": "You are a concise assistant."}, {"role": "user", "content": "What is 2 + 2?"} ] }' ``` -------------------------------- ### Cloudflare Turnstile Challenge Logic Source: https://github.com/cloudwaddie/lmarenabridge/blob/main/INFO.md This code block replaces the old auth loop. It navigates to arena.ai, checks for the Cloudflare challenge title or Turnstile widget, attempts to click the widget if present, and waits for the page to load. ```python # ... inside get_recaptcha_v3_token ... debug_print(" 🌐 Navigating to arena.ai...") await page.goto("https://arena.ai/", wait_until="domcontentloaded") # --- NEW: Cloudflare/Turnstile Pass-Through --- debug_print(" 🛡️ Checking for Cloudflare Turnstile...") # Allow time for the widget to render if it's going to try: # Check for challenge title or widget presence for _ in range(5): title = await page.title() if "Just a moment" in title: debug_print(" 🔒 Cloudflare challenge active. Attempting to click...") clicked = await click_turnstile(page) if clicked: debug_print(" ✅ Clicked Turnstile.") # Give it time to verify await asyncio.sleep(3) else: # If title is normal, we might still have a widget on the page await click_turnstile(page) break await asyncio.sleep(1) # Wait for the page to actually settle into the main app await page.wait_for_load_state("domcontentloaded") except Exception as e: debug_print(f" ⚠️ Error handling Turnstile: {e}") # ---------------------------------------------- # 1. Wake up the page (Humanize) - Keep this as is debug_print(" 🖱️ Waking up page...") # ... rest of function ... ```