### Quick Start Examples

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-cli

Basic commands to get started with the Phoenix CLI, including configuration and fetching data.

```APIDOC
## Quick Start

```bash
# Configure your Phoenix instance
export PHOENIX_HOST=http://localhost:6006
export PHOENIX_PROJECT=my-project
export PHOENIX_API_KEY=your-api-key  # if authentication is enabled

# Fetch the most recent trace
px trace list --limit 1

# Fetch a specific trace by ID
px trace get abc123def456

# Fetch recent LLM spans
px span list --span-kind LLM --limit 10

# Export traces to a directory
px trace list ./my-traces --limit 50
```
```

--------------------------------

### Paste-and-Go Tracing Setup for Coding Agents

Source: https://arize.com/docs/ax/agents/arize-skills

Use this prompt for a quick setup. The agent will handle dependency installation, instrumentation, and configuration by following the provided URL.

```text
Follow the instructions from https://arize.com/docs/PROMPT.md and ask me questions as needed.

```

--------------------------------

### Basic Setup with AUTO Detection and Container Registration

Source: https://arize.com/docs/phoenix/integrations/python/graphite/graphite-integration-guide

This example demonstrates a basic setup using TracingOptions.AUTO and registers the tracer with the Grafi container. Ensure necessary imports are present.

```python
from grafi.common.containers.container import container
from grafi.common.instrumentations.tracing import TracingOptions, setup_tracing

# Register the tracer with auto-detection
tracer = setup_tracing(tracing_options=TracingOptions.AUTO)
container.register_tracer(tracer)

# Your assistant code here
```

--------------------------------

### Haystack Pipeline Setup Example

Source: https://arize.com/docs/phoenix/integrations/python/haystack/haystack-tracing

Initialize and configure a Haystack pipeline with components like PromptBuilder and OpenAIGenerator. This example demonstrates connecting these components to form a basic question-answering pipeline.

```python
from haystack import Pipeline
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders.prompt_builder import PromptBuilder

prompt_template = """
Answer the following question.
Question: {{question}}
Answer:
"""

# Initialize the pipeline
pipeline = Pipeline()

# Initialize the OpenAI generator component
llm = OpenAIGenerator(model="gpt-3.5-turbo")
prompt_builder = PromptBuilder(template=prompt_template)

# Add the generator component to the pipeline
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)
pipeline.connect("prompt_builder", "llm")

# Define the question
question = "What is the location of the Hanging Gardens of Babylon?"
```

--------------------------------

### Development Setup with Local Phoenix

Source: https://arize.com/docs/phoenix/integrations/python/graphite/graphite-integration-guide

This example shows how to set up Grafi tracing for development using TracingOptions.PHOENIX, connecting to a local Phoenix instance. It requires installing Arize Phoenix and starting the Docker Compose service.

```bash
# Install Phoenix if not already installed
pip install arize-phoenix

# Start Phoenix server
docker compose up
```

```python
from grafi.common.containers.container import container
from grafi.common.instrumentations.tracing import TracingOptions, setup_tracing

tracer = setup_tracing(
    tracing_options=TracingOptions.PHOENIX,
    collector_endpoint="localhost",
    collector_port=4317,
    project_name="my-dev-assistant"
)
container.register_tracer(tracer)

# Your assistant code here
```

--------------------------------

### Create Mastra Project and Install Packages

Source: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/mastra/mastra-tracing

Create a new project with Mastra and install the necessary Arize exporter and observability packages. This setup is required for tracing.

```bash
npm create mastra@latest
# answer the prompts, include agent, tools, and the example when asked

cd chosen-project-name
```

```bash
npm install @mastra/arize@latest @mastra/observability @mastra/core
npm install -D mastra
```

--------------------------------

### Install LLM Vendor SDK

Source: https://arize.com/docs/phoenix/sdk-api-reference/python/arize-phoenix-evals

Install the necessary SDK for your chosen LLM provider. Examples include OpenAI, Anthropic, and Google.

```sh
pip install openai  # or anthropic, google-generativeai, litellm, etc.
```

--------------------------------

### Run BeeAI Agent with Tracing (JavaScript)

Source: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/beeai/beeai-tracing-js

Import the instrumentation setup file before using the BeeAI framework. This example demonstrates running a ToolCallingAgent with search and weather tools.

```javascript
import "./instrumentation.js";
import { ToolCallingAgent } from "beeai-framework/agents/toolCalling/agent";
import { TokenMemory } from "beeai-framework/memory/tokenMemory";
import { DuckDuckGoSearchTool } from "beeai-framework/tools/search/duckDuckGoSearch";
import { OpenMeteoTool } from "beeai-framework/tools/weather/openMeteo";
import { OpenAIChatModel } from "beeai-framework/adapters/openai/backend/chat";

const llm = new OpenAIChatModel(
  "gpt-4o", 
  {},
  { apiKey: 'your-api-key-here' }
);

const agent = new ToolCallingAgent({
  llm,
  memory: new TokenMemory(),
  tools: [
    new DuckDuckGoSearchTool(),
    new OpenMeteoTool(), // weather tool
  ],
});

async function main() {
  const response = await agent.run({ prompt: "What's the current weather in Berlin?" });
  console.log(`Agent 🤖 : `, response.result.text);
}

main();

```

--------------------------------

### LangChain Adapter Setup

Source: https://arize.com/docs/phoenix/evaluation/how-to-evals/configuring-the-llm

Configure the LLM using a LangChain chat model. This example uses 'openai' as the provider.

```python
from phoenix.evals.llm import LLM
from phoenix.evals import ClassificationEvaluator

llm = LLM(
    provider="openai",
    model="gpt-4o",
    client="langchain",
    api_key="your-api-key",
)

evaluator = ClassificationEvaluator(
    name="example",
    prompt_template="Classify: {input}",
    choices={"positive": 1, "negative": 0},
    llm=llm,
    temperature=0.0,
    max_tokens=100,
)
```

--------------------------------

### Run Interactive Setup Script for Claude Code Tracing

Source: https://arize.com/docs/ax/integrations/platforms/claude-code/claude-code-tracing

Execute the setup script to configure tracing for Claude Code. This script guides you through storing environment variables and selecting a backend (Phoenix or Arize AX).

```bash
bash claude-code-tracing/scripts/setup.sh
```

--------------------------------

### Quickstart: View OpenLLMetry Traces in Phoenix

Source: https://arize.com/docs/ax/integrations/opentelemetry/openllmetry

This example demonstrates setting up OpenLLMetry instrumentation, configuring Arize credentials, and exporting OpenAI traces to Arize for viewing in Phoenix. Ensure your Arize SPACE_ID and API_KEY are set as environment variables.

```python
import os
import grpc
import openai
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from arize.otel import register
from openinference.instrumentation.openllmetry import OpenInferenceSpanProcessor
from opentelemetry.instrumentation.openai import OpenAIInstrumentor

# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
# Set up Arize credentials
SPACE_ID = os.getenv("SPACE_ID")
API_KEY = os.getenv("API_KEY")

tracer_provider = register(
    space_id=SPACE_ID,
    api_key=API_KEY,
    project_name="openllmetry-integration",
    set_global_tracer_provider=True,
)

tracer_provider.add_span_processor(OpenInferenceSpanProcessor())
    
tracer_provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint="http://localhost:4317", #if using phoenix cloud, change to phoenix cloud endpoint (phoenix cloud space -> settings -> endpoint/hostname)
            headers={
                "authorization": f"Bearer {API_KEY}",
                "api_key": API_KEY,
                "arize-space-id": SPACE_ID,
                "arize-interface": "python",
                "user-agent": "arize-python",
            },
            compression=grpc.Compression.Gzip,  # use enum instead of string
        )
    )
)


OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

# Define and invoke your OpenAI model
client = openai.OpenAI()

messages = [
        {"role": "user", "content": "What is the national food of Yemen?"}
    ]

response = client.chat.completions.create(
    model="gpt-4",
    messages=messages,
)

# Now view your converted OpenLLMetry traces in Phoenix!

```

--------------------------------

### Initialize OpenAI Client and Define a Tool

Source: https://arize.com/docs/phoenix/tracing/how-to-tracing/setup-tracing/instrument

Sets up an OpenAI client and defines a sample tool function (`get_weather`) that can be used within LLM interactions. This example demonstrates basic setup for using tools with the OpenAI API.

```python
import json

from openai import OpenAI
from openai.types.chat import (
    ChatCompletionMessage,
    ChatCompletionMessageParam,
    ChatCompletionToolMessageParam,
    ChatCompletionToolParam,
    ChatCompletionUserMessageParam,
)
from opentelemetry.trace import Status, StatusCode

openai_client = OpenAI()


@tracer.tool
def get_weather(city: str) -> str:
    # make an call to a weather API here
    return "sunny"


messages: List[Union[ChatCompletionMessage, ChatCompletionMessageParam]] = [
    ChatCompletionUserMessageParam(
        role="user",
        content="What's the weather like in San Francisco?",
    )
]
temperature = 0.5
invocation_parameters = {"temperature": temperature}
tools: List[ChatCompletionToolParam] = [
    {
        "type": "function",
        "function": {

```

--------------------------------

### Setup Arize SDK and Upload Dataset

Source: https://arize.com/docs/ax/cookbooks/experiments/summarization

This Python snippet demonstrates how to connect to Arize using the SDK and prepare for uploading an example dataset. Ensure your API keys and space ID are correctly configured.

```python
# Note: This example uses Python SDK v7
import pandas as pd
from uuid import uuid1
from arize.experimental.datasets import ArizeDatasetsClient
from arize.experimental.datasets.utils.constants import GENERATIVE

# Setup API Keys
DEVELOPER_KEY = ""
API_KEY = ""
SPACE_ID = ""

```

--------------------------------

### Initialize Phoenix Client and Launch App

Source: https://arize.com/docs/phoenix/cookbook/evaluation/evaluate-rag

Start the Phoenix application and initialize the client for tracing. This setup is necessary to capture data for evaluating RAG pipelines.

```python
import phoenix as px
from phoenix.client import Client

client = Client()
px.launch_app()
```

--------------------------------

### Complete Assistant with Tracing Setup

Source: https://arize.com/docs/phoenix/integrations/python/graphite/graphite-integration-guide

This comprehensive example shows how to set up Grafi tracing using TracingOptions.AUTO and integrate it into a complete assistant application. It includes necessary imports for various Grafi components and standard Python libraries for asynchronous operations and environment variables.

```python
import os
import uuid
import asyncio
from grafi.common.containers.container import container
from grafi.common.events.topic_events.publish_to_topic_event import PublishToTopicEvent
from grafi.common.instrumentations.tracing import TracingOptions, setup_tracing
from grafi.common.models.async_result import async_func_wrapper
from grafi.common.models.invoke_context import InvokeContext
from grafi.common.models.message import Message
from grafi.assistants.assistant_base import AssistantBase

# Setup tracing
tracer = setup_tracing(tracing_options=TracingOptions.AUTO)
container.register_tracer(tracer)

# Get event store
event_store = container.event_store

# Create your assistant
async def main():
    assistant = (
        # YourAssistant is an instance of type grafi.assistants.assistant
        # https://github.com/binome-dev/graphite/blob/main/grafi/assistants/assistant.py
        YourAssistant.builder()
        .name("MyAssistant")
        .api_key(os.getenv("OPENAI_API_KEY"))
        .build()
    )

    # Create invoke context
    invoke_context = InvokeContext(
        conversation_id="conversation_id",
        invoke_id=uuid.uuid4().hex,
        assistant_request_id=uuid.uuid4().hex,
    )

    # Invoke assistant
    input_data = PublishToTopicEvent(
        invoke_context=invoke_context,
        data=[Message(content="Hello!", role="user")]
    )

    output = await async_func_wrapper(
        assistant.invoke(input_data, is_sequential=True)
    )
    print(output)

asyncio.run(main())
```

--------------------------------

### Full Example: Create, Save, and Use a Prompt

Source: https://arize.com/docs/api-clients/python/version-7/prompt-hub-api

Demonstrates the complete workflow of initializing clients, creating a prompt, saving it to the Prompt Hub, formatting it with variables, and using it with an OpenAI LLM.

```python
from arize.experimental.prompt_hub import ArizePromptClient, Prompt, LLMProvider
from openai import OpenAI

# Initialize clients
prompt_client = ArizePromptClient(
    space_id='YOUR_SPACE_ID',
    api_key='YOUR_API_KEY'
)
oai_client = OpenAI(api_key="YOUR_OPENAI_API_KEY")

# Create a prompt
new_prompt = Prompt(
    name='product_recommendation',
    description="Recommends products based on user preferences",
    messages=[
        {
            "role": "system",
            "content": "You are a product recommendation assistant."
        },
        {
            'role': 'user',
            'content': 'Customer preferences: {preferences}\nBudget: {budget}'
        }
    ],
    provider=LLMProvider.OPENAI,
    model_name="gpt-4o",
    tags=["recommendation", "e-commerce"]
)

# Save to Prompt Hub
prompt_client.push_prompt(new_prompt)

# Use the prompt
prompt_vars = {
    "preferences": "I like outdoor activities and photography",
    "budget": "$500"
}
formatted_prompt = new_prompt.format(prompt_vars)
response = oai_client.chat.completions.create(**formatted_prompt)
print(response.choices[0].message.content)
```

--------------------------------

### Setup Tracing with LangChain (JS/TS)

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/langchain/langchain-tracing

Import and run this file at the start of your application to instrument LangChain operations. Ensure all imported packages are installed. For troubleshooting, set the `diag` log level to `DiagLogLevel.DEBUG`.

```typescript
/* instrumentation.ts */
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import { ConsoleSpanExporter } from "@opentelemetry/sdk-trace-base"; // Optional: for console logging
import {
  NodeTracerProvider,
  SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-node";
import { resourceFromAttributes } from "@opentelemetry/resources";
import { OTLPTraceExporter as GrpcOTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-grpc";
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import { Metadata } from "@grpc/grpc-js";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager"; // Critical for LangChain instrumentation

// For troubleshooting, set the log level to DiagLogLevel.DEBUG
// diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO); // Or DiagLogLevel.DEBUG

const spaceId = process.env.ARIZE_SPACE_ID || "YOUR_ARIZE_SPACE_ID";
const apiKey = process.env.ARIZE_API_KEY || "YOUR_ARIZE_API_KEY";
const projectName = process.env.ARIZE_PROJECT_NAME || "my-langchain-js-app";

// Arize AX specific: Create metadata for OTLP exporter headers
const metadata = new Metadata();
metadata.set('space_id', spaceId);
metadata.set('api_key', apiKey);

// Optional: Console exporter for local debugging
const consoleExporter = new ConsoleSpanExporter();
const consoleProcessor = new SimpleSpanProcessor(consoleExporter);

// Arize AX OTLP gRPC exporter
const otlpExporter = new GrpcOTLPTraceExporter({
  url: "https://otlp.arize.com/v1", // Arize AX OTLP endpoint
  metadata,
});
const otlpProcessor = new SimpleSpanProcessor(otlpExporter);

const provider = new NodeTracerProvider({
  resource: resourceFromAttributes({
    // Arize AX specific: Define the project for your traces
    "project_name": projectName,
    // You can add other resource attributes here if needed
  }),
  // Add spanProcessors: otlpProcessor for Arize AX, consoleProcessor for local logs
  spanProcessors: [otlpProcessor, consoleProcessor] 
});

// Register the LangChain instrumentation
const lcInstrumentation = new LangChainInstrumentation();
// LangChain must be manually instrumented as it doesn't have a traditional module structure
// that OpenTelemetry auto-instrumentation typically relies on.
// The CallbackManagerModule is what OpenInference hooks into.
lcInstrumentation.manuallyInstrument(CallbackManagerModule);

provider.register();

console.log(`LangChain instrumented for Arize AX (JS/TS). Project: ${projectName}`);

// Example of how to run your LangChain code (e.g., in your main app.ts or server.ts)
// import { ChatOpenAI } from "@langchain/openai";
// import { HumanMessage } from "@langchain/core/messages";

// async function main() {
//   if (!process.env.OPENAI_API_KEY) {
//     throw new Error("OPENAI_API_KEY environment variable is not set.");
//   }
//   const chat = new ChatOpenAI({ modelName: "gpt-3.5-turbo", temperature: 0 });
//   const response = await chat.invoke([
//     new HumanMessage("Hello, how are you today?"),
//   ]);
//   console.log(response);
// }
// main().catch(console.error);

```

--------------------------------

### Install Strands Agents SDK

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/aws-strands/aws-strands-tracing

Install the necessary packages for Strands Agents, OpenTelemetry, and OTLP export. Ensure these are installed before proceeding with the setup.

```bash
pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc openinference-instrumentation-strands-agents
```

--------------------------------

### List Traces Examples

Source: https://arize.com/docs/api-clients/cli/traces

Examples demonstrating how to list traces with different options, including specifying output files and filters.

```bash
ax traces list proj_abc123
```

```bash
ax traces list proj_abc123 --output traces.json
```

```bash
ax traces list proj_abc123 --filter "latency_ms > 2000" --output traces.parquet
```

```bash
ax traces list my-project --space sp_abc123 --limit 50
```

--------------------------------

### Install Arize Agent Kit with Curl

Source: https://arize.com/docs/ax/integrations/platforms/codex/codex-tracing

Use the curl installer for a recommended and straightforward installation of the Arize Agent Kit. This method detects Codex and guides you through the configuration process.

```bash
curl -fsSL https://raw.githubusercontent.com/Arize-ai/arize-agent-kit/main/install.sh | bash

```

--------------------------------

### Install Prompt Learning SDK

Source: https://arize.com/docs/phoenix/prompt-engineering/tutorial/optimize-prompts-automatically

Clone the prompt-learning repository and install the SDK using pip. This sets up the necessary tools for prompt optimization.

```bash
git clone https://github.com/Arize-ai/prompt-learning.git
cd prompt-learning
pip install .
```

--------------------------------

### Install Instructor and OpenInference Packages

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/instructor/instructor-tracing

Install the necessary Python packages for Instructor, its OpenInference instrumentor, the LLM client instrumentor, Arize OTel, and OpenTelemetry SDK/exporter.

```bash
pip install instructor openinference-instrumentation-instructor openinference-instrumentation-openai arize-otel opentelemetry-sdk opentelemetry-exporter-otlp
```

--------------------------------

### Define Few-Shot Examples for Empathy Evaluation

Source: https://arize.com/docs/phoenix/cookbook/prompt-engineering/llm-as-a-judge-prompt-optimization

This snippet defines a multi-line string containing few-shot examples for empathy evaluation. These examples include user responses, scores, and explanations to guide the LLM. The examples are appended to an existing prompt template.

```python
few_shot_examples = """.---
To help guide your reasoning, compare the text to the following examples and scorings:

Example 1: "You should try to focus and get your work done. Everyone has tough days."
  Score: 2
  Explanation: No acknowledgment of the user’s emotions, dismissive and offers no real support

Example 2: "That sounds really stressful. Have you considered talking to a supervisor or taking breaks?"
  Score: 5
  Explanation: Acknowledges stress, but in a generic way. Provides advice, but not very personal. Could be warmer in tone.

Example 3: "I’m really sorry you’re feeling this way. It’s completely understandable to feel overwhelmed. You’re not alone in this. Have you had a chance to take a break or talk to someone who can support you?"
  Score: 9
  Explanation: Validates emotions, reassures the user, and offers support
""".
EMPATHY_EVALUATION_PROMPT_TEMPLATE_IMPROVED = EMPATHY_EVALUATION_PROMPT_TEMPLATE + few_shot_examples
```

--------------------------------

### Complete Prompt Optimization Workflow

Source: https://arize.com/docs/ax/prompts/prompt-optimization/prompt-learning-sdk

This example demonstrates the full workflow from initializing the optimizer, preparing a dataset, to optimizing the prompt. Ensure all necessary libraries are imported and the dataset has the required columns.

```python
import pandas as pd
from optimizer_sdk.prompt_learning_optimizer import PromptLearningOptimizer
from phoenix.evals import OpenAIModel, llm_generate

# 1. Initialize optimizer
optimizer = PromptLearningOptimizer(
    prompt="You are a math tutor. Solve this problem: {problem}",
    model_choice="gpt-4o"
)

# 2. Prepare dataset
dataset = pd.DataFrame({
    "problem": ["2 + 2 = ?", "5 * 3 = ?", "10 / 2 = ?"],
    "answer": ["4", "15", "5"],
    "correctness": ["correct", "correct", "correct"],
    "explanation": ["Correct addition", "Correct multiplication", "Correct division"]
})

# 3. Optimize prompt
optimized_prompt = optimizer.optimize(
    dataset=dataset,
    output_column="answer",
    feedback_columns=["correctness", "explanation"],
    context_size_k=8000
)

print("Original prompt:", optimizer.prompt)
print("Optimized prompt:", optimized_prompt)
```

--------------------------------

### Example Input: Available Tools

Source: https://arize.com/docs/phoenix/evaluation/pre-built-metrics/tool-selection

List the available tools and their descriptions in a human-readable format. This allows the LLM to understand the capabilities it can leverage.

```text
book_flight: Book a flight between two cities. Requires origin, destination, and date.
search_hotels: Search for hotel accommodations by city and dates.
get_weather: Get current weather conditions for a location.
cancel_booking: Cancel an existing flight or hotel reservation.
```

--------------------------------

### Install @vercel/otel for Next.js Edge Runtimes

Source: https://arize.com/docs/ax/integrations/ts-js-agent-frameworks/vercel/vercel-ai-sdk-tracing

Install the `@vercel/otel` package for tracing AI SDK calls in Next.js edge runtimes. This is a prerequisite for the instrumentation setup.

```bash
npm i @vercel/otel
```

--------------------------------

### Usage Example for Claude Agent SDK

Source: https://arize.com/docs/phoenix/integrations/typescript/claude-agent-sdk

Import the instrumentation setup and use the `query` function from the Claude Agent SDK. This example demonstrates how to stream messages from the assistant.

```typescript
import "./instrumentation.js";
import { query } from "@anthropic-ai/claude-agent-sdk";

async function main() {
  for await (const message of query({
    prompt: "What is the weather in San Francisco?",
    options: {
      model: "claude-sonnet-4-5-20250514",
    },
  })) {
    if (message.type === "assistant") {
      console.log(message.content);
    }
  }
}

main();

```

--------------------------------

### Set Up DuckDB Database and Load Data

Source: https://arize.com/docs/phoenix/datasets-and-experiments/how-to-experiments/run-experiments

Initialize an in-memory DuckDB database and load the NBA training data from the 'suzyanil/nba-data' dataset.

```python
import duckdb
from datasets import load_dataset

data = load_dataset("suzyanil/nba-data")["train"]
conn = duckdb.connect(database=":memory:", read_only=False)
conn.register("nba", data.to_pandas())
```

--------------------------------

### Install DSPy and OpenInference Packages

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/dspy/dspy-tracing

Install the DSPy framework, OpenInference instrumentors for DSPy and LiteLLM, and Arize OTel packages. LiteLLM instrumentation is recommended for detailed traces including token counts.

```bash
# Install DSPy framework
pip install dspy-ai

# Install OpenInference instrumentor for DSPy
pip install openinference-instrumentation-dspy

# Install LiteLLM and its OpenInference instrumentor (DSPy often uses LiteLLM)
pip install litellm openinference-instrumentation-litellm

# Install Arize OTel
pip install arize-otel
```

--------------------------------

### Install Portkey and OpenInference

Source: https://arize.com/docs/phoenix/integrations/python/portkey/portkey-tracing

Install the necessary packages for Portkey instrumentation with OpenInference.

```bash
pip install openinference-instrumentation-portkey portkey-ai
```

--------------------------------

### Launch Local Phoenix Instance

Source: https://arize.com/docs/phoenix/integrations/python/autogen/autogen-agentchat-tracing

Install and serve a local instance of Phoenix for development and testing.

```bash
pip install arize-phoenix
phoenix serve
```

--------------------------------

### GET /v1/datasets/{id}/jsonl/openai_ft

Source: https://arize.com/docs/phoenix/sdk-api-reference/rest-api/api-reference/datasets/download-dataset-examples-as-openai-fine-tuning-jsonl-file

Downloads dataset examples in OpenAI fine-tuning JSONL format.

```APIDOC
## GET /v1/datasets/{id}/jsonl/openai_ft

### Description
Downloads dataset examples in OpenAI fine-tuning JSONL format.

### Method
GET

### Endpoint
/v1/datasets/{id}/jsonl/openai_ft

### Parameters
#### Path Parameters
- **id** (string) - Required - The ID of the dataset

#### Query Parameters
- **version_id** (string | null) - Optional - The ID of the dataset version (if omitted, returns data from the latest version)

### Request Example
```json
{
  "example": ""
}
```

### Response
#### Success Response (200)
- **string** (string) - Successful Response

#### Response Example
```json
"<string>"
```
```

--------------------------------

### Install Packages for AgentChat with OpenAI

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/autogen/autogen-agentchat-tracing

Install the required packages to run an AgentChat example using OpenAI as the LLM provider. This includes the core AutoGen AgentChat library and extensions for OpenAI integration.

```bash
pip install autogen_ext openai
```

--------------------------------

### Install Required Dependencies

Source: https://arize.com/docs/phoenix/get-started/ts-get-started-tracing

Install the necessary Mastra, Arize, and OpenAI SDKs for tracing and evaluation.

```bash
npm install @mastra/arize @ai-sdk/openai @arizeai/phoenix-evals @arizeai/phoenix-client

```

--------------------------------

### Install with Phoenix Client and OpenAI

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/packages/phoenix-evals/overview

Install @arizeai/phoenix-evals, the Phoenix client, and the OpenAI provider adapter for experiment integration.

```bash
npm install @arizeai/phoenix-evals @arizeai/phoenix-client @ai-sdk/openai

```

--------------------------------

### Run Basic AutoGen Example

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/autogen/autogen-tracing

Set up and run a simple AutoGen task. Ensure your OAI_CONFIG_LIST is correctly configured, pointing to your LLM configuration (e.g., using the OPENAI_API_KEY environment variable).

```python
import autogen

# Configuration for the LLM (e.g., OpenAI)
# Ensure OPENAI_API_KEY is set in your environment if model is not "stub"
config_list = autogen.config_list_from_json(
    env_or_file="OAI_CONFIG_LIST", # Example: Set this env var to your config list
    # Sample OAI_CONFIG_LIST content (save as a JSON file or set as ENV var):
    # [
    #   {
    #     "model": "gpt-3.5-turbo",
    #     "api_key": "YOUR_OPENAI_API_KEY" # Can also be picked from env
    #   }
    # ]
    filter_dict={
        "model": ["gpt-3.5-turbo", "gpt-4", "gpt-4o"] # Specify models you want to use
    }
)

# Create agents
assistant = autogen.AssistantAgent(
    name="assistant",
    llm_config={
        "config_list": config_list,
        "temperature": 0
    }
)

user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=5,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config=False, # Set to a dict with work_dir if you need code execution
)

# Start a chat
user_proxy.initiate_chat(
    assistant,
    message="What is the capital of France? Reply TERMINATE when done.",
)

print("AutoGen chat initiated. Check Arize for traces.")
```

--------------------------------

### Instantiate and Use MistralAIModel

Source: https://arize.com/docs/ax/integrations/llm-providers/mistralai/mistralai-evals

Example of how to instantiate and use the MistralAIModel for generating responses. Ensure the 'mistralai' dependency is installed.

```python
# model = Instantiate your MistralAIModel here
model("Hello there, how are you?")
# Output: "As an artificial intelligence, I don't have feelings, 
#          but I'm here and ready to assist you. How can I help you today?"
```

--------------------------------

### Append Examples to Dataset with Python SDK v8

Source: https://arize.com/docs/ax/develop/datasets/update-a-dataset

Use `append_examples()` to add new examples to the latest dataset version. Ensure you have the `arize` library installed and replace placeholders with your actual API key, dataset ID, and data.

```python
from arize import ArizeClient

client = ArizeClient(api_key="your-arize-api-key")
client.datasets.append_examples(dataset_id="your-dataset-id", examples=df)
```

--------------------------------

### Minimal Example: List Datasets

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/packages/phoenix-client/overview

Create a client instance and list datasets using the imported function. Ensure necessary imports are included.

```typescript
import { createClient } from "@arizeai/phoenix-client";
import { listDatasets } from "@arizeai/phoenix-client/datasets";

const client = createClient();
const datasets = await listDatasets({ client });

```

--------------------------------

### Quickstart: Setup OpenTelemetry with Arize-otel

Source: https://arize.com/docs/ax/integrations/opentelemetry/opentelemetry-arize-otel

Configure OpenTelemetry tracing using the `register` function from `arize.otel`. This function returns a `TracerProvider` and can also configure headers and span processing. It is recommended to instrument your application using OpenInference AutoInstrumentators after this setup.

```python
from arize.otel import register
# Setup OTel via our convenience function
tracer_provider = register(
    # See details in examples below...
)

# Instrument your application using OpenInference AutoInstrumentators
from openinference.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)
```

--------------------------------

### Development - Install

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/mcp-server

Install dependencies and build the project for development.

```APIDOC
## Development - Install

This package is managed via a pnpm workspace.

```bash
// From the /js/ directory
pnpm install
pnpm build
```

This only needs to be repeated if dependencies change or there is a change to the phoenix-client.
```

--------------------------------

### Setup OpenInference Span Processor

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/microsoft/microsoft-agent-framework

Configure the tracer provider with the AgentFrameworkToOpenInferenceProcessor and an OTLPSpanExporter. Enable instrumentation to start capturing traces.

```python
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from agent_framework.observability import enable_instrumentation
from openinference.instrumentation.agent_framework import \
    AgentFrameworkToOpenInferenceProcessor

# Configure Arize endpoint
endpoint = "https://otlp.arize.com/v1/traces"

# Setup tracer with OpenInference processor
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(AgentFrameworkToOpenInferenceProcessor())
tracer_provider.add_span_processor(
    SimpleSpanProcessor(OTLPSpanExporter(endpoint=endpoint))
)
trace.set_tracer_provider(tracer_provider)

# Enable instrumentation
enable_instrumentation(enable_sensitive_data=True)

# Your agent code here
```

--------------------------------

### Get Dataset Metadata and Examples (Python SDK v8)

Source: https://arize.com/docs/ax/develop/datasets/export-a-dataset

Use this snippet to fetch dataset metadata and iterate through specific examples from a dataset using Arize Python SDK v8. Ensure you have your Arize API key and dataset ID.

```python
from arize import ArizeClient

client = ArizeClient(api_key="your-arize-api-key")

# Get dataset object with metadata
dataset = client.datasets.get(dataset_id=dataset_id)

# Get specific examples from the dataset
examples_response = client.datasets.list_examples(dataset_id=dataset_id, all=True)
for example in examples_response:
  print(example)
```

--------------------------------

### Install PromptFlow

Source: https://arize.com/docs/phoenix/integrations/platforms/prompt-flow/prompt-flow-tracing

Install the PromptFlow library using pip. This is the first step to enable tracing.

```bash
pip install promptflow
```

--------------------------------

### Launch Ollama Server

Source: https://arize.com/docs/ax/integrations/llm-providers/llama/ollama-tracing

Starts the Ollama server to enable local model interactions. Ensure this command is run after Ollama is installed.

```bash
ollama serve
```

--------------------------------

### Setup Tracing

Source: https://arize.com/docs/phoenix/integrations/python/graphite/graphite-integration-guide

Initializes tracing with automatic configuration. Ensure TracingOptions.AUTO is correctly imported.

```python
tracer = setup_tracing(tracing_options=TracingOptions.AUTO)
```

--------------------------------

### Install BeeAI and OpenInference

Source: https://arize.com/docs/phoenix/integrations/python/beeai/beeai-tracing-python

Install the necessary packages for BeeAI framework and OpenInference instrumentation. This command installs the BeeAI framework and the OpenInference instrumentation package for BeeAI.

```bash
pip install openinference-instrumentation-beeai beeai-framework
```

--------------------------------

### Register Phoenix Tracer

Source: https://arize.com/docs/phoenix/integrations/python/google-adk/google-adk-tracing

Configure the Phoenix tracer by registering it with your application. This setup enables auto-instrumentation based on installed OpenInference dependencies.

```python
from phoenix.otel import register

# Configure the Phoenix tracer
tracer_provider = register(
  project_name="my-llm-app", # Default is 'default'
  auto_instrument=True # Auto-instrument your app based on installed OI dependencies
)
```

--------------------------------

### Basic Semantic Kernel Usage Example

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/semantic-kernel/semantic-kernel-tracing

Demonstrates how to initialize a Semantic Kernel, add an OpenAI chat completion service, define a prompt template, and invoke a function. This example can be used to test the integration and observe traces in Arize.

```python
from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.prompt_template import InputVariable, PromptTemplateConfig
 
kernel = Kernel()
 
kernel.add_service(
    OpenAIChatCompletion(ai_model_id="gpt-4o",api_key="OPENAI_API_KEY"),
)
 
prompt = """
{{$input}}
Given the input above, answer the question to the best of your knowledge.
 """ 
 
prompt_template_config = PromptTemplateConfig(
    template=prompt,
    name="summarize",
    template_format="semantic-kernel",
    input_variables=[
        InputVariable(name="input", description="user input", is_required=True),
    ]
)

summarize = kernel.add_function(
    function_name="summarizeFunc",
    plugin_name="summarizePlugin",
    prompt_template_config=prompt_template_config,
)

input_text = "Summarize Arize AI platform in 50 words"
 
summary = await kernel.invoke(summarize, input=input_text)
 
print(summary)
```

--------------------------------

### Generate Synthetic Data with Few-Shot Prompting

Source: https://arize.com/docs/phoenix/cookbook/tracing/generating-synthetic-datasets-for-llm-evaluators-and-agents

This code uses few-shot prompting with an OpenAI model to generate synthetic customer support classification examples. The prompt includes examples of queries, responses, and classifications to guide the LLM. Ensure the openai_client is configured.

```python
few_shot_prompt = """
Generate synthetic customer support classification examples.
Ensure good coverage across intents (refund, order_status, product_info),
and include both correct and incorrect classifications.
Here are some examples of synthetic customer queries and labels:

Example 1:
{
  "user_query": "Ughhh I bought sneakers that squeak louder than a rubber duck... how do I return these?",
  "intent": "refund",
  "response": "Oh no, squeaky shoes aren’t fun! Let’s get that return started. Could you share your order number?",
  "classification": "correct"
}

Example 2:
{
  "user_query": "My package has been saying 'out for delivery' since last Tuesday… did it decide to take a vacation? Is it actually going to show up?",
  "intent": "refund",
  "response": "Looks like your package is taking its sweet time. Let me check where it’s stuck — can you give me the tracking number?",
  "classification": "incorrect"
}


Example 3:
{
  "user_query": "Thinking about upgrading my blender… does your new model actually crush ice?",
  "intent": "product_info",
  "response": "Haha our blender keeps its promises! It can definitely crush ice. Would you like more details on the specs?",
  "classification": "correct"
}

Now generate 25 new examples in the same format, keeping the reesponses friendly.
Respond ONLY with valid JSON array, no code fences, no extra text.
"""

resp = openai_client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": few_shot_prompt}]
)

few_shot_data = json.loads(resp.choices[0].message.content)
few_shot_df = pd.DataFrame(few_shot_data)
few_shot_df.head()
```

--------------------------------

### Create and Invoke Langchain Agent

Source: https://arize.com/docs/phoenix/integrations/typescript/langchain/langchain-js

Example demonstrating how to create a Langchain agent with a tool and invoke it. Ensure you have installed @langchain/anthropic to call the model.

```typescript
import * as z from "zod";
// npm install @langchain/anthropic to call the model
import { createAgent, tool } from "langchain";

const getWeather = tool(
  ({ city }) => `It's always sunny in ${city}!`,
  {
    name: "get_weather",
    description: "Get the weather for a given city",
    schema: z.object({
      city: z.string(),
    }),
  }
);

const agent = createAgent({
  model: "claude-sonnet-4-5-20250929",
  tools: [getWeather],
});

console.log(
  await agent.invoke({
    messages: [{ role: "user", content: "What's the weather in Tokyo?" }],
  })
);
```

--------------------------------

### Example Input: Tool Selection Invocation

Source: https://arize.com/docs/phoenix/evaluation/pre-built-metrics/tool-invocation

Provide the LLM's tool invocation(s) with arguments. This example shows a single tool invocation in a human-readable format.

```text
book_flight(origin="JFK", destination="LAX", date="2024-01-15", time_preference="morning")
```

--------------------------------

### Initialize and Use FaithfulnessEvaluator in TypeScript

Source: https://arize.com/docs/phoenix/evaluation/pre-built-metrics/faithfulness

Initializes the FaithfulnessEvaluator using the provided model and demonstrates evaluating an example. Ensure necessary SDKs are installed.

```typescript
import { createFaithfulnessEvaluator } from "@arizeai/phoenix-evals";
import { openai } from "@ai-sdk/openai";

// Create the evaluator
const faithfulnessEvaluator = createFaithfulnessEvaluator({
  model: openai("gpt-4o"),
});

// Evaluate an example
const result = await faithfulnessEvaluator.evaluate({
  input: "What is the capital of France?",
  output: "Paris is the capital of France.",
  context: "Paris is the capital and largest city of France.",
});

console.log(result);
// { score: 1, label: "faithful", explanation: "..." }
```

--------------------------------

### Install with Google Provider

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/packages/phoenix-evals/overview

Install @arizeai/phoenix-evals along with the Google provider adapter.

```bash
npm install @arizeai/phoenix-evals @ai-sdk/google

```

--------------------------------

### Run Strands Agent

Source: https://arize.com/docs/phoenix/integrations/python/strands-agents/strands-agents-tracing

Instantiate an Agent with an OpenAI model and system prompt, then invoke it to get a response. This example uses the gpt-4o-mini model.

```python
from strands import Agent
from strands.models.openai import OpenAIModel

model = OpenAIModel(model_id="gpt-4o-mini")
agent = Agent(
    model=model,
    system_prompt="You are a helpful assistant.",
)

result = agent("Explain the theory of relativity in simple terms.")


```

--------------------------------

### Inspect Local Install Pattern

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-evals

After installation, you can inspect the local documentation and source code within the node_modules directory.

```bash
node_modules/@arizeai/phoenix-evals/docs/

```

```bash
node_modules/@arizeai/phoenix-evals/src/

```

--------------------------------

### Initialize Phoenix Client

Source: https://arize.com/docs/phoenix/cookbook/prompt-engineering/prompt-optimization

Initialize a unique ID and the Phoenix client for logging experiments. This setup is required before starting to log prompts and their performance.

```python
import uuid
from phoenix.client import Client

unique_id = uuid.uuid4()
```

--------------------------------

### Fetch Examples from a Dataset

Source: https://arize.com/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-cli

Retrieve examples from a specified dataset. You can filter by split, specify a version, save to a file, or pipe the raw output to jq.

```bash
px dataset get query_response                        # Fetch all examples
px dataset get query_response --split train          # Filter by split
px dataset get query_response --split train --split test  # Multiple splits
px dataset get query_response --version <version-id> # Specific version
px dataset get query_response --file dataset.json    # Save to file
px dataset get query_response --format raw | jq '.examples[].input'
```

--------------------------------

### Setup Arize Tracing for BeeAI

Source: https://arize.com/docs/ax/integrations/python-agent-frameworks/beeai/beeai-tracing-python

Configure and register the Arize OpenTelemetry provider with your space ID and API key, then apply the BeeAIInstrumentor to start tracing.

```python
# Import Arize OTel registration
from arize.otel import register

# Import the automatic instrumentor from OpenInference
from openinference.instrumentation.beeai import BeeAIInstrumentor

# Setup OTel via our convenience function
tracer_provider = register(
    space_id = "your-space-id", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    project_name = "my-beeai-project", # name this to whatever you would like
)

# Start the instrumentor for BeeAI
BeeAIInstrumentor().instrument(tracer_provider=tracer_provider)
```

--------------------------------

### List and Get Datasets

Source: https://arize.com/docs/phoenix/sdk-api-reference/python/arize-phoenix-client

Manage evaluation datasets by listing all available datasets and retrieving specific datasets with their examples. This is useful for preparing data for experiments and evaluations.

```python
import pandas as pd

# List all available datasets
datasets = client.datasets.list()
for dataset in datasets:
    print(f"Dataset: {dataset['name']} ({dataset['example_count']} examples)")

# Get a specific dataset with all examples
dataset = client.datasets.get_dataset(dataset="qa-evaluation")
print(f"Dataset {dataset.name} has {len(dataset)} examples")
```