Helicone

https://docs.helicone.ai/
Admin
Helicone is an open-source LLM observability platform that provides an AI Gateway for unified access...
Tokens:296,749
Snippets:2,808
Trust Score:10
Update:3 months ago
Show doc for...
Tokens:
Raw
### Start Jawn Backend and Main Proxy

Source: https://docs.helicone.ai/getting-started/self-host/manual

Set up the environment for the Jawn backend, copy the example environment file, install dependencies, and start the development server.

```bash
nvm use 20
cd valhalla/jawn
cp .env.example .env
yarn && yarn dev
```

--------------------------------

### Start Helicone Web Application

Source: https://docs.helicone.ai/getting-started/self-host/manual

Install dependencies for the Helicone web frontend and start the local development server on port 3000.

```bash
nvm use 20
cp .env.example web/.env
cd web
yarn
yarn dev:local -p 3000
```

--------------------------------

### Basic Usage Example

Source: https://docs.helicone.ai/features/advanced-usage/prompts/sdk

This example demonstrates the basic usage of the HeliconePromptManager to get a compiled prompt with variable substitution and then use it with the OpenAI SDK.

```APIDOC
## Basic Usage Example

### `getPromptBody()`

**Description**: Get compiled prompt with variable substitution.

**Parameters**:

- **prompt_id** (string) - Required - The ID of the prompt to retrieve.
- **model** (string) - Required - The model to use for compilation.
- **inputs** (object) - Required - Key-value pairs for prompt variables.

### Request Example

```typescript
const { body, errors } = await promptManager.getPromptBody({
  prompt_id: "abc123",
  model: "gpt-4o-mini",
  inputs: {
    customer_name: "Alice Johnson",
    product: "AI Gateway"
  }
});
```

### Response

**Success Response**:
- **body** (object) - The compiled prompt body ready for the OpenAI SDK.
- **errors** (array) - An array of validation errors, if any.

**Error Handling**:
- If `errors` array is not empty, validation failed.
```

--------------------------------

### Install Helicone Go Helpers Package

Source: https://docs.helicone.ai/getting-started/integration-method/manual-logger-go

Install the necessary Helicone Go helpers package using the go get command.

```bash
go get github.com/helicone/go-helicone-helpers
```

--------------------------------

### Start the MITM Proxy (Ubuntu/Debian)

Source: https://docs.helicone.ai/tools/mitm-proxy

Installs and starts the Helicone MITM proxy. Configure caching and custom properties using environment variables. Ensure the CA bundle is set for HTTPS requests.

```bash
# Configure Cache
export HELICONE_CACHE_ENABLED="true" # Enable caching

# Configure Custom Properties
# Note: There is a lock file that allows you to configure properties dynamically
export HELICONE_PROPERTY_<PROPERTY_NAME>="<PROPERTY_VALUE>" # Set custom properties

bash -c "$(curl -fsSL https://raw.githubusercontent.com/Helicone/helicone/main/mitmproxy.sh)" -s start
echo "sk-<HELICONE_KEY>" > ~/.helicone/api_key # Alternatively, you can export HELICONE_API_KEY to your path before starting the proxy to bypass this step.

export REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt # This is required for HTTPS requests to work properly.
# Your python script here...

```

--------------------------------

### Install Dependencies

Source: https://docs.helicone.ai/getting-started/integration-method/litellm-proxy

Install the necessary LiteLLM and asyncio libraries using pip.

```bash
pip install litellm asyncio
```

--------------------------------

### Install and Run ngrok

Source: https://docs.helicone.ai/features/webhooks-testing

Instructions for installing ngrok on macOS and starting a tunnel to expose your local development server to the internet. This allows Helicone to send real webhook events to your local machine.

```bash
# Install ngrok (macOS)
brew install ngrok

# Or download from https://ngrok.com/download

# Start tunnel to your local server
ngrok http 8000
```

--------------------------------

### Install @helicone/generate SDK

Source: https://docs.helicone.ai/features/prompts-legacy/generate

Install the SDK using npm. This is the first step before using the generate function.

```bash
npm install @helicone/generate
```

--------------------------------

### Install Node.js and Yarn

Source: https://docs.helicone.ai/getting-started/self-host/manual

Install Node.js version 20 using NVM and globally install Yarn for managing project dependencies.

```bash
nvm install 20
nvm use 20
npm install -g yarn
```

--------------------------------

### Install LiteLLM

Source: https://docs.helicone.ai/getting-started/integration-method/litellm-proxy

Install the LiteLLM library using pip. This is the first step for integrating LiteLLM with Helicone proxies.

```bash
pip install litellm
```

--------------------------------

### Install @helicone/helpers Package

Source: https://docs.helicone.ai/guides/cookbooks/manual-logger-streaming

Install the necessary package for Helicone logging. Choose the command corresponding to your package manager.

```bash
npm install @helicone/helpers
# or
yarn add @helicone/helpers
# or
pnpm add @helicone/helpers
```

--------------------------------

### Start Minio Object Storage

Source: https://docs.helicone.ai/getting-started/self-host/manual

Start the Minio service, which provides S3-compatible object storage for Helicone.

```bash
python3 minio_hcone.py --restart
```

--------------------------------

### Install Required Python Packages

Source: https://docs.helicone.ai/integrations/gemini/vertex/python

Install the Google Cloud AI Platform package using pip.

```bash
pip install google-cloud-aiplatform
```

--------------------------------

### Install Helicone Helpers and OpenAI for Python

Source: https://docs.helicone.ai/features/advanced-usage/prompts/sdk

Install the Helicone helpers and the required OpenAI Python SDK using pip.

```bash
pip install helicone-helpers openai
```

--------------------------------

### Install Necessary Packages

Source: https://docs.helicone.ai/guides/cookbooks/openai-structured-outputs

Install the required Python packages for Pydantic, OpenAI, and environment variable management.

```bash
pip install pydantic openai python-dotenv
```

--------------------------------

### Install Helicone Helpers SDK

Source: https://docs.helicone.ai/gateway/concepts/prompt-caching

Install the necessary SDK to extend OpenAI types for Anthropic's cache control.

```bash
npm install @helicone/helpers
```

--------------------------------

### Install Codex SDK

Source: https://docs.helicone.ai/gateway/integrations/codex

Install the Codex SDK using npm. This is the first step to integrating Codex with Helicone.

```bash
npm install @openai/codex-sdk
```

--------------------------------

### Install Dependencies for Vercel AI Gateway

Source: https://docs.helicone.ai/guides/cookbooks/vercel-ai-gateway

Install the necessary packages for using Vercel AI Gateway and the AI SDK.

```bash
npm install @ai-sdk/gateway ai
```

--------------------------------

### Response Example for Environments

Source: https://docs.helicone.ai/rest/prompts/get-v1prompt-2025-environments

This is an example of the JSON response you can expect when successfully retrieving environment names. The response is an array of strings.

```json
[
    "production",
    "staging",
    "development"
  ]
```

--------------------------------

### Install Helicone Async and Boto3

Source: https://docs.helicone.ai/integrations/bedrock/python

Install the necessary Python packages for Helicone's asynchronous logging and AWS SDK integration.

```bash
pip install helicone-async boto3
```

--------------------------------

### Response Example for Production Version

Source: https://docs.helicone.ai/rest/prompts/post-v1prompt-2025-query-production-version

This is an example of the JSON response you can expect when successfully retrieving the production version of a prompt.

```json
{
    "id": "version_789",
    "model": "gpt-4",
    "prompt_id": "prompt_123",
    "major_version": 2,
    "minor_version": 0,
    "commit_message": "Production-ready version with improved accuracy",
    "created_at": "2024-01-16T16:45:00Z",
    "s3_url": "https://s3.amazonaws.com/bucket/prompt-body.json"
  }
```

--------------------------------

### Install Dependencies for Vercel AI Gateway Demo

Source: https://docs.helicone.ai/guides/cookbooks/vercel-ai-gateway-demo

Install the necessary npm packages for integrating Vercel AI Gateway and Helicone.

```bash
npm install @ai-sdk/gateway ai openai
```

--------------------------------

### Install Helicone Async for Python

Source: https://docs.helicone.ai/getting-started/integration-method/openllmetry

Install the Helicone async package using pip. This is the first step for Python integration.

```bash
pip install helicone-async
```

--------------------------------

### Complete Async Example for AWS Bedrock Integration

Source: https://docs.helicone.ai/integrations/bedrock/javascript

A full example combining all steps: initializing the Helicone logger, configuring the Bedrock client, creating a command, and sending the request. This demonstrates the end-to-end integration.

```javascript
import { HeliconeAsyncLogger } from "@helicone/async";
import * as bedrock from "@aws-sdk/client-bedrock-runtime";
import util from "util";

// Initialize Helicone Logger
const logger = new HeliconeAsyncLogger({
  apiKey: process.env.HELICONE_API_KEY,
  providers: {
    bedrock: bedrock,
  },
  baseUrl: "https://api.helicone.ai/v1/trace/log",
});
logger.init();

// Configure AWS Bedrock client
const client = new bedrock.BedrockRuntimeClient({
  region: process.env.AWS_REGION,
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
});

// Create a command for the Bedrock model
const command = new bedrock.ConverseCommand({
  messages: [
    {
      role: "user",
      content: [{ text: "Why is the unix epoch measured from 1970?" }],
    },
  ],
  modelId: "meta.llama2-13b-chat-v1",
});

// Send the request and handle the response
async function sendRequest() {
  try {
    const data = await client.send(command);
    console.log(
      "Response:\n",
      util.inspect(data, { showHidden: false, depth: null, colors: true })
    );
  } catch (error) {
    console.error("Error:", error);
  }
}

sendRequest();
```

--------------------------------

### Start ClickHouse Database

Source: https://docs.helicone.ai/getting-started/self-host/manual

Initialize and start the ClickHouse time-series database using the provided Python script. Ensure no password is required for this setup.

```bash
python3 -m venv env
source env/bin/activate
pip install tabulate requests minio yarl

# Start Clickhouse (time-series database)
python3 clickhouse/ch_hcone.py --restart --no-password --url http://localhost:18123
```

--------------------------------

### Start Postgres Database and Run Migrations

Source: https://docs.helicone.ai/getting-started/self-host/manual

Run a PostgreSQL database in Docker, set up the necessary environment variables, and apply database migrations using Flyway.

```bash
docker run -d --name helicone-postgres-flyway-test -e POSTGRES_DB=helicone_test -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=testpassword -p 54388:5432 -v postgres_flyway_data:/var/lib/postgresql/data postgres:17.4
flyway migrate -configFiles=supabase/flyway.conf
```

--------------------------------

### Example Response for Get Models

Source: https://docs.helicone.ai/rest/ai-gateway/get-v1models

This is an example of the JSON response structure you can expect when querying the /v1/models endpoint. It includes a list of models with their respective details.

```json
{
  "object": "list",
  "data": [
    {
      "id": "claude-opus-4",
      "object": "model",
      "created": 1747180800,
      "owned_by": "anthropic"
    },
    {
      "id": "gpt-4o",
      "object": "model",
      "created": 1715558400,
      "owned_by": "openai"
    },
    ...
  ]
}
```

--------------------------------

### Install OpenAI Agents and OpenAI SDKs

Source: https://docs.helicone.ai/gateway/integrations/openai-agents

Install the necessary SDKs for OpenAI Agents and OpenAI. Choose the package manager that suits your project.

```bash
npm install @openai/agents openai
# or
pip install openai-agents
```

--------------------------------

### Complete Semantic Kernel Example with Helicone Gateway (C#)

Source: https://docs.helicone.ai/gateway/integrations/semantic-kernel

A full C# example demonstrating how to test multiple AI models (OpenAI, Anthropic, Google) through the Helicone AI Gateway. Ensure the HELICONE_API_KEY environment variable is set.

```csharp
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using DotNetEnv;

// Load environment
Env.Load();
var apiKey = Environment.GetEnvironmentVariable("HELICONE_API_KEY");

if (string.IsNullOrEmpty(apiKey))
{
    Console.WriteLine("❌ HELICONE_API_KEY not found in environment");
    return;
}

Console.WriteLine("🚀 Testing multiple models through Helicone AI Gateway\n");

// Test different models
await TestModel("gpt-4.1-mini", "OpenAI GPT-4.1 Mini");
await TestModel("claude-opus-4-1", "Anthropic Claude Opus 4.1");
await TestModel("gemini-2.5-flash-lite", "Google Gemini 2.5 Flash Lite");

Console.WriteLine("\n✅ All models tested!");
Console.WriteLine("🔍 Check your dashboard: https://us.helicone.ai/dashboard");

async Task TestModel(string modelId, string modelName)
{
    try
    {
        var builder = Kernel.CreateBuilder();

        // Configure with Helicone AI Gateway
        builder.AddOpenAIChatCompletion(
            modelId: modelId,
            apiKey: apiKey,
            endpoint: new Uri("https://ai-gateway.helicone.ai/v1")
        );

        var kernel = builder.Build();
        var chatService = kernel.GetRequiredService<IChatCompletionService>();

        var chatHistory = new ChatHistory();
        chatHistory.AddUserMessage("Say hello in one sentence.");

        Console.Write($
```

--------------------------------

### Get All Models - OpenAPI Specification

Source: https://docs.helicone.ai/rest/models/get-v1public-model-registry-models

This OpenAPI specification defines the GET /v1/public/model-registry/models endpoint. It details the request parameters, response structure, and provides an example of the expected JSON output for a model registry response.

```yaml
get /v1/public/model-registry/models
openapi: 3.0.0
info:
  title: helicone-api
  version: 1.0.0
  license:
    name: MIT
  contact: {}
servers:
  - url: https://api.helicone.ai/
  - url: http://localhost:8585/
security: []
paths:
  /v1/public/model-registry/models:
    get:
      tags:
        - Model Registry
      summary: >-
        Returns a comprehensive list of all AI models with their configurations,
        pricing, and capabilities
      description: Get all available models from the registry
      operationId: GetModelRegistry
      parameters: []
      responses:
        '200':
          description: Complete model registry with models and filter options
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Result_ModelRegistryResponse.string_"
              examples:
                Example 1:
                  value:
                    models:
                      - id: claude-opus-4-1
                        name: 'Anthropic: Claude Opus 4.1'
                        author: anthropic
                        contextLength: 200000
                        endpoints:
                          - provider: anthropic
                            providerSlug: anthropic
                            supportsPtb: true
                            pricing:
                              prompt: 15
                              completion: 75
                              cacheRead: 1.5
                              cacheWrite: 18.75
                        maxOutput: 32000
                        trainingDate: '2025-08-05'
                        description: Most capable Claude model with extended context
                        inputModalities:
                          - null
                        outputModalities:
                          - null
                        supportedParameters:
                          - null
                          - null
                          - null
                          - null
                          - null
                          - null
                          - null
                    total: 150
                    filters:
                      providers:
                        - name: anthropic
                          displayName: Anthropic
                        - name: openai
                          displayName: OpenAI
                        - name: google
                          displayName: Google
                      authors:
                        - anthropic
                        - openai
                        - google
                        - meta
                      capabilities:
                        - audio
                        - image
                        - thinking
                        - caching
                        - reasoning
      security: []
components:
  schemas:
    Result_ModelRegistryResponse.string_:
      anyOf:
        - $ref: "#/components/schemas/ResultSuccess_ModelRegistryResponse_"
        - $ref: "#/components/schemas/ResultError_string_"
    ResultSuccess_ModelRegistryResponse_:
      properties:
        data:
          $ref: "#/components/schemas/ModelRegistryResponse"
        error:
          type: number
          enum:
            - null
          nullable: true
      required:
        - data
        - error
      type: object
      additionalProperties: false
    ResultError_string_:
      properties:
        data:
          type: number
          enum:
            - null
          nullable: true
        error:
          type: string
      required:
        - data
        - error
      type: object
      additionalProperties: false
    ModelRegistryResponse:
      properties:
        models:
          items:
            $ref: "#/components/schemas/ModelRegistryItem"
          type: array
        total:
          type: number
          format: double
        filters:
          properties:
            capabilities:
              items:
                $ref: "#/components/schemas/ModelCapability"
              type: array
            authors:
              items:

```

--------------------------------

### Invoke Model with Azure OpenAI and Helicone (JavaScript)

Source: https://docs.helicone.ai/integrations/azure/langchain

Example of invoking the configured Langchain model to get a response to a prompt. Logs the response to the console.

```javascript
const response = await model.invoke("What is the meaning of life?");
console.log(response);
```

--------------------------------

### Python Example with OpenAI and LemonFox Fallback

Source: https://docs.helicone.ai/getting-started/integration-method/gateway-fallbacks

Demonstrates how to configure Helicone's gateway with fallbacks to OpenAI and LemonFox API. It shows how to set the `Helicone-Fallbacks` header and extract the `Helicone-Fallback-Index` from the response.

```python
url = "https://gateway.helicone.ai/v1/chat/completions"

headers = {
  "Content-Type": "application/json",
  "Helicone-Auth": "Bearer <Helicone KEY>",
  "Helicone-Fallbacks": json.dumps([
      {
          "target-url": "https://api.openai.com",
          "headers": {
            "Authorization": "Bearer <OpenAI Key>",
          },
          "onCodes": [{"from": 400, "to": 500}]
      },
      {
          "target-url": "https://api.lemonfox.ai",
          "headers": {
              "Authorization": "Bearer <LemonFox Key>",
              "Content-Type": "application/json",
          },
          "onCodes": [401, 403],
          "bodyKeyOverride": {
              "model": "zephyr-chat",
          }
      },
  ]),
}
payload = {
  "model": "gpt-4o-mini",
  "messages": [
    {
      "role": "user",
      "content": "What is the meaning of life?"
    }
  ],
  "max_tokens": 300
}

response = requests.post(url, headers=headers, data=json.dumps(payload))
print("Fallback index used: ", response.headers["helicone-fallback-index"])
print(response.status_code)
print(response.text)
```

--------------------------------

### Invoke Model with Azure OpenAI and Helicone (Python)

Source: https://docs.helicone.ai/integrations/azure/langchain

Example of invoking the configured Langchain model in Python to get a response. Prints the response to standard output.

```python
response = model.invoke("What is the meaning of life?")
print(response)
```

--------------------------------

### Get Prompt Inputs using TypeScript

Source: https://docs.helicone.ai/rest/prompts/get-v1prompt-2025-id-promptid-versionid-inputs

This TypeScript example demonstrates how to fetch prompt inputs using the Fetch API. It includes setting the necessary authorization header and parsing the JSON response.

```typescript
const response = await fetch(
    'https://api.helicone.ai/v1/prompt-2025/id/prompt_123/version_456/inputs?requestId=req_789',
    {
      method: 'GET',
      headers: {
        'Authorization': `Bearer ${HELICONE_API_KEY}`,
      },
    }
  );

  const inputs = await response.json();
```

--------------------------------

### Prompt Partial Syntax Examples

Source: https://docs.helicone.ai/features/advanced-usage/prompts

Demonstrates the syntax for referencing messages from other prompts using prompt partials. Specify the prompt ID, message index, and optionally the environment.

```text
{{hcp:abc123:0}}                   // Message 0 from prompt abc123 (production)
{{hcp:abc123:1:staging}}           // Message 1 from prompt abc123 (staging)
{{hcp:xyz789:2:development}}       // Message 2 from prompt xyz789 (development)
```

--------------------------------

### Complete Example with Python Requests

Source: https://docs.helicone.ai/getting-started/integration-method/manual-logger-curl

Use this snippet to log LLM API calls manually. It captures request and response details, timing, and sends them to Helicone. Ensure you have the 'requests' library installed.

```python
import requests
import time
import json

# Record start time
start_time = time.time()
start_ms = int((start_time - int(start_time)) * 1000)

# Make your API call to the LLM provider
llm_response = requests.post(
    "https://your-llm-provider.com/generate",
    json={
        "model": "your-model",
        "prompt": "Tell me a story about dragons"
    },
    headers={"Authorization": "Bearer your-provider-api-key"}
)

# Record end time
end_time = time.time()
end_ms = int((end_time - int(end_time)) * 1000)

# Prepare the Helicone log request
helicone_request = {
    "providerRequest": {
        "url": "custom-model-nopath",
        "json": {
            "model": "your-model",
            "prompt": "Tell me a story about dragons"
        },
        "meta": {
            "Helicone-User-Id": "user-123",
            "Helicone-Session-Id": "session-456"
        }
    },
    "providerResponse": {
        "json": llm_response.json(),
        "status": llm_response.status_code,
        "headers": dict(llm_response.headers)
    },
    "timing": {
        "startTime": {
            "seconds": int(start_time),
            "milliseconds": start_ms
        },
        "endTime": {
            "seconds": int(end_time),
            "milliseconds": end_ms
        }
    }
}

# Log to Helicone
helicone_response = requests.post(
    "https://api.worker.helicone.ai/custom/v1/log",
    json=helicone_request,
    headers={
        "Authorization": "Bearer your-helicone-api-key",
        "Content-Type": "application/json"
    }
)

print(f"Helicone logging status: {helicone_response.status_code}")
```

--------------------------------

### Few-Shot Learning: Style Imitation

Source: https://docs.helicone.ai/guides/prompt-engineering/implement-few-shot-learning

Implement few-shot learning to generate text in a specific style, such as imitating Albert Einstein's quotes. Provide examples of the desired style to guide the model.

```text
Write a motivational quote in the style of Albert Einstein.

Example 1:
"Life is like riding a bicycle. To keep your balance, you must keep moving."

Example 2:
"Imagination is more important than knowledge. Knowledge is limited; imagination encircles the world."

Now, generate a new motivational quote in the style of Albert Einstein.


```

--------------------------------

### LiteLLM Provider Selection and Fallback

Source: https://docs.helicone.ai/gateway/integrations/litellm

Demonstrates automatic routing to the cheapest provider, manual provider selection, and setting up a fallback chain for multiple providers.

```python
import os
from litellm import completion
from dotenv import load_dotenv

load_dotenv()

# Automatic routing (cheapest provider)
response = completion(
    model="helicone/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    api_key=os.getenv("HELICONE_API_KEY")
)

# Manual provider selection
response = completion(
    model="helicone/claude-4.5-sonnet/anthropic",
    messages=[{"role": "user", "content": "Hello!"}],
    api_key=os.getenv("HELICONE_API_KEY")
)

# Multiple provider fallback chain
# Try OpenAI first, then Anthropic if it fails
response = completion(
    model="helicone/gpt-4o/openai,claude-4.5-sonnet/anthropic",
    messages=[{"role": "user", "content": "Hello!"}],
    api_key=os.getenv("HELICONE_API_KEY")
)
```

--------------------------------

### Execute OpenAI Tests with Helicone in GitHub Actions

Source: https://docs.helicone.ai/guides/cookbooks/github-actions

An example of a GitHub Actions workflow step that executes OpenAI tests while Helicone is active. It includes starting the proxy and setting necessary environment variables.

```yaml
# ...Rest of yml
tests:
  steps:
    - name: Execute OpenAI tests
      run: |
        curl -s https://raw.githubusercontent.com/Helicone/helicone/main/mitmproxy.sh | bash -s start
        # Execute your tests here

      env:
        OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        HELICONE_API_KEY: ${{ secrets.HELICONE_API_KEY }}
        REQUESTS_CA_BUNDLE: /etc/ssl/certs/ca-certificates.crt
        HELICONE_CACHE_ENABLED: "true"
        HELICONE_PROPERTY_<KEY>: <VALUE>
```

--------------------------------

### Python: Using Environment Control with Helicone

Source: https://docs.helicone.ai/features/advanced-usage/prompts/sdk

This Python example demonstrates how to specify an environment (e.g., 'staging') when retrieving a prompt using the Helicone Prompt Manager. It also shows how to include messages in the prompt request.

```python
import openai
import os
from helicone_helpers import HeliconePromptManager

client = openai.OpenAI(
    base_url="https://ai-gateway.helicone.ai",
    api_key=os.environ.get("HELICONE_API_KEY")
)

prompt_manager = HeliconePromptManager(
    api_key="your-helicone-api-key"
)

def use_environment_version():
    result = prompt_manager.get_prompt_body({
        "prompt_id": "abc123",
        "environment": "staging",  # Use staging environment
        "model": "gpt-4o-mini",
        "inputs": {
            "user_query": "How does caching work?",
            "context": "technical documentation"
        },
        "messages": [
            {"role": "user", "content": "Follow up question..."}
        ]
    })

    if result["errors"]:
        print("Variable validation failed:", result["errors"])

    return client.chat.completions.create(**result["body"])
```

--------------------------------

### Few-Shot Learning: Email Response Generation

Source: https://docs.helicone.ai/guides/prompt-engineering/implement-few-shot-learning

Use few-shot learning to guide an assistant in drafting professional email responses. Provide examples of customer inquiries and desired responses to establish tone and structure.

```python
You are an assistant helping to draft professional email responses.

Example 1:
Customer Inquiry: "I am interested in your software but have some questions about pricing."
Response: "Dear [Customer Name], thank you for reaching out. I'd be happy to provide more details about our pricing plans..."

Example 2:
Customer Inquiry: "Can I schedule a demo of your product?"
Response: "Hello [Customer Name], we'd be delighted to arrange a demo for you. Please let us know your availability..."

Now, based on the customer's message below, compose an appropriate response.

Customer Inquiry: "I'm experiencing issues with logging into my account. Can you assist?"
Response:


```

--------------------------------

### Example Response with Inputs

Source: https://docs.helicone.ai/rest/request/get-v1request-inputs

This JSON shows a successful response containing the inputs, prompt ID, version ID, and environment for a request made through prompt management.

```json
{
  "data": {
    "inputs": {
      "customer_name": "Sarah",
      "issue": "refund request"
    },
    "prompt_id": "customer-support",
    "version_id": "1c7a86c8-...",
    "environment": "production"
  },
  "error": null
}
```

--------------------------------

### Create and Use a Helicone-Powered Agent

Source: https://docs.helicone.ai/gateway/integrations/vercel-ai-sdk

Set up an agent using Helicone as the model provider and define custom tools for specific tasks. This example includes tools for getting weather information and calculating wind chill, demonstrating multi-tool usage within a single agent prompt.

```typescript
import { createHelicone } from "@helicone/ai-sdk-provider";
import { Experimental_Agent as Agent, tool, jsonSchema, stepCountIs } from "ai";

const helicone = createHelicone({
  apiKey: process.env.HELICONE_API_KEY!
});

const weatherAgent = new Agent({
  model: helicone("claude-4.5-haiku"),
  stopWhen: stepCountIs(5),
  tools: {
    getWeather: tool({
      description: "Get the current weather for a location",
      inputSchema: jsonSchema({
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "The city and state, e.g. San Francisco, CA",
          },
          unit: {
            type: "string",
            enum: ["celsius", "fahrenheit"],
            default: "fahrenheit",
            description: "Temperature unit",
          },
        },
        required: ["location"],
      }),
      execute: async ({ location, unit }) => {
        // Simulate weather API call
        const temp =
          unit === "celsius"
            ? Math.floor(Math.random() * 30 + 5)
            : Math.floor(Math.random() * 86 + 32);
        const conditions = ["sunny", "cloudy", "rainy", "partly cloudy"][
          Math.floor(Math.random() * 4)
        ];
        const result = {
          location,
          temperature: temp,
          unit: unit || "fahrenheit",
          conditions,
          description: `It's ${conditions} in ${location} with a temperature of ${temp}°${unit?.charAt(0).toUpperCase() || "F"}.`,
        };
        console.log(`Result: ${JSON.stringify(result)}`);
        return result;
      },
    }),
    calculateWindChill: tool({
      description: "Calculate wind chill temperature",
      inputSchema: jsonSchema({
        type: "object",
        properties: {
          temperature: {
            type: "number",
            description: "Temperature in Fahrenheit",
          },
          windSpeed: {
            type: "number",
            description: "Wind speed in mph",
          },
        },
        required: ["temperature", "windSpeed"],
      }),
      execute: async ({ temperature, windSpeed }) => {
        const windChill =
          35.74 +
          0.6215 * temperature -
          35.75 * Math.pow(windSpeed, 0.16) +
          0.4275 * temperature * Math.pow(windSpeed, 0.16);
        const result = {
          temperature,
          windSpeed,
          windChill: Math.round(windChill),
          description: `With a temperature of ${temperature}°F and wind speed of ${windSpeed} mph, the wind chill feels like ${Math.round(windChill)}°F.`,
        };
        console.log(`Result: ${JSON.stringify(result)}`);
        return result;
      },
    }),
  },
});

try {
  console.log("🌤️  Asking about weather in multiple cities...\n");

  const result = await weatherAgent.generate({
    prompt:
      "You are a helpful weather assistant. When asked about weather, use the getWeather tool to provide accurate information.\n\nWhat is the weather like in San Francisco, CA and New York, NY? Also, if the wind speed in San Francisco is 15 mph, what would the wind chill feel like?"
  });

  console.log("=== Agent Response ===");
  console.log(result.text);

  console.log("\n=== Usage Statistics ===");
  console.log(`Total tokens: ${result.usage?.totalTokens || "N/A"}`);
  console.log(`Finish reason: ${result.finishReason}`);
  console.log(`Steps taken: ${result.steps?.length || 0}`);

  if (result.steps && result.steps.length > 0) {
    console.log("\n=== Steps Breakdown ===");
    result.steps.forEach((step, index) => {
      console.log(`Step ${index + 1}: ${step.finishReason}`);
      if (step.toolCalls && step.toolCalls.length > 0) {
        console.log(
          `  Tool calls: ${step.toolCalls.map((tc) => tc.toolName).join(", ")}`
        );
        step.toolCalls.forEach((tc, i) => {
          console.log(
            `    Tool ${i + 1}: ${tc.toolName}(${JSON.stringify(tc.input)})`
          );
        });
      }
    });
  }
} catch (error) {
  console.error("❌ Error running agent:", error);
  if (error instanceof Error) {
    console.error("Error details:", error.message);
  }
}

```

--------------------------------

### Implement Custom Retry Logic for API Calls

Source: https://docs.helicone.ai/gateway/concepts/error-handling

This TypeScript example demonstrates how to implement custom retry logic for API calls using the OpenAI SDK. It includes handling for authentication errors, rate limits, and server errors with exponential backoff. Ensure you have the OpenAI SDK installed and your HELICONE_API_KEY configured.

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai",
  apiKey: process.env.HELICONE_API_KEY,
});

async function callWithRetry(maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      const response = await client.chat.completions.create({
        model: "gpt-4o-mini",
        messages: [{ role: "user", content: "Hello!" }],
      });
      return response;
    } catch (error: any) {
      const status = error?.status || 500;

      // Don't retry auth errors or bad requests
      if (status === 401 || status === 403 || status === 400) {
        throw error;
      }

      // Don't retry insufficient credits unless it's the last attempt
      if (status === 429 && i === maxRetries - 1) {
        throw error;
      }

      // Retry transient errors (500, 503) with exponential backoff
      if (status >= 500 || status === 429) {
        await new Promise(resolve => setTimeout(resolve, Math.pow(2, i) * 1000));
        continue;
      }

      throw error;
    }
  }
}
```

--------------------------------

### Basic Prompt Partial Syntax

Source: https://docs.helicone.ai/features/advanced-usage/prompts/overview

Demonstrates the basic syntax for referencing messages from other prompts using prompt partials. Includes examples for different indices and environments.

```text
{{hcp:abc123:0}}
// Message 0 from prompt abc123 (production)
{{hcp:abc123:1:staging}}
// Message 1 from prompt abc123 (staging)
{{hcp:xyz789:2:development}}
// Message 2 from prompt xyz789 (development)
```

--------------------------------

### Complete Semantic Kernel Example with Helicone Gateway (Python)

Source: https://docs.helicone.ai/gateway/integrations/semantic-kernel

A full Python example demonstrating how to test multiple AI models (OpenAI, Anthropic, Google) through the Helicone AI Gateway. Ensure the HELICONE_API_KEY environment variable is set.

```python
import semantic_kernel as sk
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.contents import ChatHistory
import os
import asyncio

# Load environment
helicone_api_key = os.getenv("HELICONE_API_KEY")

if not helicone_api_key:
    print("❌ HELICONE_API_KEY not found in environment")
    exit(1)

print("🚀 Testing multiple models through Helicone AI Gateway\n")

async def test_model(model_id: str, model_name: str):
    try:
        # Create kernel
        kernel = sk.Kernel()

        # Configure with Helicone AI Gateway
        kernel.add_service(
            OpenAIChatCompletion(
                service_id="helicone-gateway",
                ai_model_id=model_id,
                api_key=helicone_api_key,
                endpoint="https://ai-gateway.helicone.ai/v1"
            )
        )

        chat_service = kernel.get_service("helicone-gateway")

        chat_history = ChatHistory()
        chat_history.add_user_message("Say hello in one sentence.")

        print(f"🤖 Testing {model_name}... ", end="")
        response = await chat_service.get_chat_message_content(
            chat_history=chat_history
        )
        print("✅")
        print(f"   Response: {response.content}\n")
    except Exception as ex:
        print("❌")
        print(f"   Error: {str(ex)}\n")

async def main():
    # Test different models
    await test_model("gpt-4.1-mini", "OpenAI GPT-4.1 Mini")
    await test_model("claude-opus-4-1", "Anthropic Claude Opus 4.1")
    await test_model("gemini-2.5-flash-lite", "Google Gemini 2.5 Flash Lite")

    print("\n✅ All models tested!")
    print("🔍 Check your dashboard: https://us.helicone.ai/dashboard")

if __name__ == "__main__":
    asyncio.run(main())
```

--------------------------------

### Get Webhooks

Source: https://docs.helicone.ai/llms.txt

Get all webhooks.

```APIDOC
## GET /v1/webhooks

### Description
Get all webhooks.

### Method
GET

### Endpoint
/v1/webhooks

### Query Parameters
(Details not provided in source)

### Response
#### Success Response (200)
(Details not provided in source)
```

--------------------------------

### Using Prompt Integration

Source: https://docs.helicone.ai/gateway/prompt-integration

This example demonstrates how to use the `prompt_id` and `inputs` parameters with the Helicone AI Gateway to reference a managed prompt, replacing the traditional `messages` approach.

```APIDOC
## Using Prompt Integration

### Description
Replace `messages` with `prompt_id` and `inputs` in your gateway calls to utilize managed prompts.

### Method
```typescript
client.chat.completions.create
```

### Parameters
#### Request Body
- **model** (string) - Required - Any supported model.
- **prompt_id** (string) - Required - The ID of your saved prompt from the Helicone dashboard.
- **environment** (string) - Optional - Which environment version to use: `development`, `staging`, or `production`. Defaults to `production`.
- **inputs** (object) - Required - Variables to fill in your prompt template (e.g., `{"customer_name": "John", "issue_type": "billing"}`).

### Request Example
```typescript
const response = await client.chat.completions.create({
  model: "gpt-4o-mini",
  prompt_id: "customer_support_v2",
  environment: "production",
  inputs: {
    customer_name: "Sarah Johnson",
    issue_type: "billing",
    customer_message: "I was charged twice this month"
  }
});
```

### Response
#### Success Response (200)
- **response** (object) - The LLM completion response.

#### Response Example
```json
{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello Sarah, I understand you were charged twice this month. I can help you with that."
      },
      "logprobs": null
    }
  ],
  "usage": {
    "prompt_tokens": 50,
    "completion_tokens": 20,
    "total_tokens": 70
  }
}
```
```

--------------------------------

### Install LiteLLM and Python Dotenv

Source: https://docs.helicone.ai/gateway/integrations/litellm

Install the necessary Python packages for LiteLLM and environment variable management. Ensure you have Python and pip installed.

```bash
pip install litellm python-dotenv
```

--------------------------------

### Install Required Packages

Source: https://docs.helicone.ai/integrations/gemini/vertex/python

Installs the necessary Python packages for using Helicone and Vertex AI.

```bash
pip install python-dotenv helicone
```

--------------------------------

### Responses API Quick Start

Source: https://docs.helicone.ai/gateway/concepts/responses-api

Use your Helicone API key and the AI Gateway base URL. Then call the OpenAI SDK's responses.create method as usual.

```APIDOC
## Responses API Quick Start

Use your Helicone API key and the AI Gateway base URL. Then call the OpenAI SDK's `responses.create` method as usual.

### Method
POST

### Endpoint
/v1/responses

### Request Body
- **model** (string) - Required - The model to use for generating the response.
- **input** (string) - Required - The input prompt for the model.

### Request Example
```json
{
  "model": "gpt-5",
  "input": "Write a one-sentence bedtime story about a unicorn."
}
```

### Response
#### Success Response (200)
- **output_text** (string) - The generated text output from the model.

### Response Example
```json
{
  "output_text": "Once upon a time, a unicorn with a rainbow mane slept soundly under a starry sky."
}
```
```

--------------------------------

### Install Node Fetch Package

Source: https://docs.helicone.ai/integrations/gemini/api/javascript

Ensure the `node-fetch` package is installed for making HTTP requests.

```bash
npm install node-fetch
```

--------------------------------

### Install Helicone Helpers Package

Source: https://docs.helicone.ai/getting-started/integration-method/manual-logger-python

Install the necessary package for Helicone integration using pip.

```bash
pip install helicone-helpers
```

--------------------------------

### Install LangChain Packages for Python

Source: https://docs.helicone.ai/gateway/integrations/langchain

Install the required LangChain packages for Python using pip.

```bash
pip install langchain-openai langchain-core python-dotenv
```

--------------------------------

### Initialize Groq SDK with Helicone

Source: https://docs.helicone.ai/integrations/groq/javascript

Configure the Groq SDK to use Helicone's services by setting the `baseUrl` and including the `Helicone-Auth` header. Ensure your `GROQ_API_KEY` and `HELICONE_API_KEY` environment variables are set.

```javascript
const groq = new Groq({
  apiKey: process.env.GROQ_API_KEY,
  baseUrl: "https://groq.helicone.ai/openai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});
```

--------------------------------

### Setup User Organization in PostgreSQL

Source: https://docs.helicone.ai/getting-started/self-deploy-docker

These commands are used to set up a user's organization in PostgreSQL. First, retrieve the user ID, then create a new organization, and finally, add the user to that organization.

```bash
# Get your user ID
docker exec -u postgres helicone psql -d helicone_test -c "SELECT id, email FROM \"user\" WHERE email = 'your@email.com';"

# Create organization (save the returned ID)
docker exec -u postgres helicone psql -d helicone_test -c "INSERT INTO organization (name, is_personal) VALUES ('My Org', true) RETURNING id;"

# Add user to organization (replace USER_ID and ORG_ID)
docker exec -u postgres helicone psql -d helicone_test -c "INSERT INTO organization_member (\"user\", organization, org_role) 
   VALUES ('USER_ID', 'ORG_ID', 'admin');"
```

--------------------------------

### Install Helicone AI SDK Provider

Source: https://docs.helicone.ai/gateway/integrations/vercel-ai-sdk

Install the Helicone provider package for your preferred package manager.

```bash
pnpm add @helicone/ai-sdk-provider ai
```

```bash
npm install @helicone/ai-sdk-provider ai
```

```bash
yarn add @helicone/ai-sdk-provider ai
```

```bash
bun add @helicone/ai-sdk-provider ai
```
Tokens:
Raw