### Install Dependencies and Run Examples Source: https://github.com/arize-ai/phoenix/blob/main/js/packages/phoenix-client/README.md To run the provided examples, first install dependencies using `pnpm install`, then execute the desired example script using `pnpx tsx`. ```bash pnpm install pnpx tsx examples/list_datasets.ts # change the file name to run other examples ``` -------------------------------- ### Setup Agent Environment Source: https://github.com/arize-ai/phoenix/blob/main/examples/agents/README.md Install dependencies and start the Phoenix server for trace collection. ```bash cd examples/agents # Option 1: pip pip install -r requirements.txt # Option 2: uv (recommended) uv venv --python 3.10 source .venv/bin/activate uv pip install -r requirements.txt # Start Phoenix phoenix serve ``` -------------------------------- ### Install Dependencies and Run Examples Source: https://github.com/arize-ai/phoenix/blob/main/js/packages/phoenix-evals/README.md Instructions for installing project dependencies using pnpm and running example scripts with tsx. Remember to change the file name to execute different examples. ```bash pnpm install pnpx tsx examples/classifier_example.ts # change the file name to run other examples ``` -------------------------------- ### Basic Setup with Auto-Instrumentation Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/api/register.md Use this for a simple setup where automatic instrumentation of installed libraries is desired. Ensure the phoenix.otel module is imported. ```python from phoenix.otel import register tracer_provider = register(auto_instrument=True) ``` -------------------------------- ### Get Dataset Examples Source: https://github.com/arize-ai/phoenix/blob/main/js/packages/phoenix-cli/README.md Fetch examples from a dataset using various filters and output formats. ```bash px dataset get my-dataset --format raw | jq '.examples[].input' ``` ```bash px dataset get my-dataset --split train --split test ``` ```bash px dataset get my-dataset --version ``` ```bash px dataset get my-dataset --file dataset.json ``` -------------------------------- ### Install OpenAI Go SDK and OpenInference Packages Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/llm-providers/openai/openai-go-sdk.mdx Install the necessary OpenAI Go SDK and OpenInference instrumentation packages using go get. ```bash go get github.com/openai/openai-go go get github.com/Arize-ai/openinference/go/openinference-instrumentation go get github.com/Arize-ai/openinference/go/openinference-instrumentation-openai-go go get go.opentelemetry.io/otel go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp go get go.opentelemetry.io/otel/sdk ``` -------------------------------- ### Install Phoenix and Serve Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/cookbook/tracing/openinference-best-practices.mdx Installs the Arize Phoenix library and starts the Phoenix server. This command should be run in a separate terminal to keep the Phoenix UI accessible. ```bash pip install arize-phoenix phoenix serve ``` -------------------------------- ### Start the Mock LLM Server Source: https://github.com/arize-ai/phoenix/blob/main/scripts/mock-llm-server/README.md Commands to install dependencies, build the dashboard, and launch the server in development or production mode. ```bash # Install dependencies pnpm install # Build the dashboard (first time only) pnpm run build:dashboard # Start the server (development mode with hot reload) pnpm dev # Or start without hot reload pnpm start ``` -------------------------------- ### Get Dataset Examples with Phoenix CLI Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-cli.mdx Fetch examples from a dataset, with options to filter by split, version, save to a file, or format the output. ```bash px dataset get query_response # Fetch all examples px dataset get query_response --split train # Filter by split px dataset get query_response --split train --split test # Multiple splits px dataset get query_response --version # Specific version px dataset get query_response --file dataset.json # Save to file px dataset get query_response --format raw | jq '.examples[].input' ``` -------------------------------- ### Install Anthropic SDK Go and OpenInference Packages Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/llm-providers/anthropic/anthropic-sdk-go.mdx Install the necessary Anthropic Go SDK and OpenInference instrumentation packages using go get. ```bash go get github.com/anthropics/anthropic-sdk-go go get github.com/Arize-ai/openinference/go/openinference-instrumentation go get github.com/Arize-ai/openinference/go/openinference-instrumentation-anthropic-sdk-go go get go.opentelemetry.io/otel go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp go get go.opentelemetry.io/otel/sdk ``` -------------------------------- ### Create a Dataset with Examples (Deno) Source: https://github.com/arize-ai/phoenix/blob/main/js/examples/notebooks/deno_datasets_and_experiments_quickstart.ipynb Creates a new dataset in Phoenix with a unique name and description. It populates the dataset with example data, each containing an ID, input, output, and metadata. The function returns the newly created dataset's ID. ```typescript console.log('Creating dataset examples...'); // Create examples directly as an array const { datasetId } = await createDataset({ name: `quickstart-dataset-${Date.now()}`, description: "Dataset for quickstart example", examples: [ { id: `example-1`, updatedAt: new Date(), input: { question: "What is Paul Graham known for?" }, output: { answer: "Co-founding Y Combinator and writing on startups and techology." }, metadata: { topic: "tech" } }, { id: `example-2`, updatedAt: new Date(), input: { question: "What companies did Elon Musk found?" }, output: { answer: "Tesla, SpaceX, Neuralink, The Boring Company, and co-founded PayPal." }, metadata: { topic: "entrepreneurs" } }, { id: `example-3`, updatedAt: new Date(), input: { question: "What is Moore's Law?" }, output: { answer: "The observation that the number of transistors in a dense integrated circuit doubles about every two years." }, metadata: { topic: "computing" } } ]}) console.log(`examples created`); ``` -------------------------------- ### Install Phoenix SDK Source: https://github.com/arize-ai/phoenix/blob/main/scripts/fixtures/vision.ipynb Installs the Phoenix SDK and its dependencies. Use this command in your environment before running other Phoenix SDK examples. ```python %pip install -Uqqq beautifulsoup ``` -------------------------------- ### Install Dependencies (Bash) Source: https://github.com/arize-ai/phoenix/blob/main/internal_docs/vignettes/sqlalchemy/polymorphic_user/README.md This command installs the necessary Python packages, SQLAlchemy for database interaction and bcrypt for secure password hashing, required to run the example. ```bash pip install sqlalchemy bcrypt ``` -------------------------------- ### Inspect Local Install Pattern Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-otel.mdx After installation, you can inspect the package's documentation and source code locally within the node_modules directory. This allows for offline access to implementation details and examples. ```text node_modules/@arizeai/phoenix-otel/docs/ node_modules/@arizeai/phoenix-otel/src/ ``` ```text node_modules/@arizeai/openinference-core/docs/ node_modules/@arizeai/openinference-core/src/ ``` -------------------------------- ### Generate SQL Query with Few-Shot Examples Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/cookbook/datasets-and-experiments/text2sql.mdx This Python snippet demonstrates how to construct a system prompt with few-shot examples to guide an LLM in generating accurate DuckDB SQL queries. It includes fetching sample data, formatting it into an example row and few-shot examples, and then using these to create a system prompt for the OpenAI client. ```python samples = conn.query("SELECT * FROM movies LIMIT 5").to_df().to_dict(orient="records") example_row = "\n".join( f"{column['column_name']} | {column['column_type']} | {samples[0][column['column_name']]}" for column in columns ) column_header = " | ".join(column["column_name"] for column in columns) few_shot_examples = "\n".join( " | ".join(str(sample[column["column_name"]]) for column in columns) for sample in samples ) system_prompt = ( "You are a SQL expert, and you are given a single table named `movies` with the following columns:\n\n" "Column | Type | Example\n" "-------|------|--------\n" f"{example_row}\n" "\n" "Examples:\n" f"{column_header}\n" f"{few_shot_examples}\n" "\n" "Write a DuckDB SQL query corresponding to the user's request. " "Return just the query text, with no formatting (backticks, markdown, etc.)." ) async def generate_query(input): response = await client.chat.completions.create( model=TASK_MODEL, temperature=0, messages=[ { "role": "system", "content": system_prompt, }, { "role": "user", "content": input, }, ], ) return response.choices[0].message.content print(await generate_query("what are the best sci-fi movies in the 2000s?")) ``` -------------------------------- ### Run Initial Experiment with Prompt Template Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/cookbook/datasets-and-experiments/summarization.mdx Initiates an experiment with a basic prompt template to establish a baseline. Use this to get started and understand initial LLM performance. ```python experiment_results = await px_client.experiments.run_experiment( dataset=dataset, task=task, experiment_name="initial-template", experiment_description="first experiment using a simple prompt template", experiment_metadata={"vendor": "openai", "model": gpt_4o}, evaluators=EVALUATORS, ) ``` -------------------------------- ### Quick Start: Register with Auto-Instrumentation Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-otel/docs/source/index.md Use the register() function for simple setup with automatic instrumentation. This is recommended for tracing AI/ML libraries. ```python from phoenix.otel import register # Simple setup with automatic instrumentation (recommended) tracer_provider = register(auto_instrument=True) ``` -------------------------------- ### Setup Local Phoenix Instance for Development (Bash & Python) Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/python/graphite/graphite-integration-guide.mdx Guides through setting up a local Phoenix instance for development by first installing the `arize-phoenix` package and starting the Docker Compose services. Subsequently, it shows how to configure Python code to connect to this local instance. ```bash # Install Phoenix if not already installed pip install arize-phoenix # Start Phoenix server docker compose up ``` ```python from grafi.common.containers.container import container from grafi.common.instrumentations.tracing import TracingOptions, setup_tracing tracer = setup_tracing( tracing_options=TracingOptions.PHOENIX, collector_endpoint="localhost", collector_port=4317, project_name="my-dev-assistant" ) container.register_tracer(tracer) # Your assistant code here # Visit http://localhost:6006 to view the Phoenix UI. ``` -------------------------------- ### Launch Arize Phoenix Application Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/experiments/datasets_and_experiments_quickstart.ipynb Launches the Arize Phoenix application UI. This is typically the first step after installation to start using the platform. ```python import phoenix as px px.launch_app().view() ``` -------------------------------- ### Get Examples From A Dataset Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/llms.txt Retrieve examples from a specific version of a dataset. ```APIDOC ## GET /v1/datasets/{id}/examples ### Description Retrieve examples at a specific dataset version. ### Method GET ### Endpoint /v1/datasets/{id}/examples ### Parameters #### Path Parameters - **id** (string) - Required - The unique identifier for the dataset. #### Query Parameters - **version** (string) - Optional - The specific dataset version to retrieve examples from. ``` -------------------------------- ### Per-Config Explanation Example in Python Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/evaluation/how-to-evals/code-evaluator-output-shapes.mdx Provides a Python example where specific explanations are given for individual output configurations, with a shared fallback. ```python return { "toxicity": {"score": 0.9, "explanation": "Contains slurs."}, "safety": "fail", "explanation": "Overall content is unsafe.", # only used for safety } ``` -------------------------------- ### Create and Add Examples to a Dataset Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/api/datasets.md Demonstrates creating a new dataset with initial input/output pairs and subsequently adding more examples to an existing dataset. ```python # Create a new dataset dataset = client.datasets.create_dataset( name="qa-dataset", inputs=[ {"question": "What is 2+2?"}, {"question": "What's the capital of France?"}, ], outputs=[{"answer": "4"}, {"answer": "Paris"}] ) # Add more examples later updated = client.datasets.add_examples_to_dataset( dataset="qa-dataset", inputs=[{"question": "Who wrote Hamlet?"}], outputs=[{"answer": "Shakespeare"}] ) ``` -------------------------------- ### Simple Tracer Setup Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-otel/README.md Basic setup for the tracer provider, sending traces to localhost. ```python from phoenix.otel import register # Basic setup - sends to localhost tracer_provider = register(auto_instrument=True) ``` -------------------------------- ### Install Graphite with Uv (Bash) Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/python/graphite/graphite-integration-guide.mdx Illustrates installing Grafi using uv, a fast Python package installer and resolver. ```bash uv pip install grafi ``` -------------------------------- ### Install Graphite with Pip (Bash) Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/python/graphite/graphite-integration-guide.mdx Demonstrates how to install the Grafi package and its core dependencies using pip, a common Python package installer. ```bash pip install grafi ``` -------------------------------- ### Install Prompt Learning SDK (Bash) Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/prompts/phoenix_prompt_tutorial.ipynb This bash script provides instructions for installing the prompt learning SDK. It involves cloning the repository from GitHub, navigating into the directory, and then installing the SDK using pip. ```bash git clone https://github.com/priyanjindal/prompt-learning.git cd prompt-learning pip install . ``` -------------------------------- ### Install Required Libraries Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/integrations/tracing_and_evals_weaviate.ipynb Install the necessary Python packages for Phoenix, Weaviate, and OpenAI integration. This is a prerequisite for the rest of the guide. ```python !pip install -q arize-phoenix weaviate weaviate-client openai openinference-instrumentation-openai ``` -------------------------------- ### px dataset get Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-cli.mdx Fetch examples from a dataset. ```APIDOC ## `px dataset get ` ### Description Fetch examples from a dataset. ### Usage ```bash px dataset get query_response # Fetch all examples px dataset get query_response --split train # Filter by split px dataset get query_response --split train --split test # Multiple splits px dataset get query_response --version # Specific version px dataset get query_response --file dataset.json # Save to file px dataset get query_response --format raw | jq '.examples[].input' ``` ### Options | Option | Description | Default | |--------|-------------|---------| | `--split ` | Filter by split (can be used repeatedly) | — | | `--version ` | Fetch from specific dataset version | latest | | `--file ` | Save to file instead of stdout | stdout | | `--format ` | `pretty`, `json`, or `raw` | `pretty` | | `--endpoint ` | Phoenix API endpoint | From env | | `--api-key ` | Phoenix API key | From env | | `--no-progress` | Disable progress indicators | — | ``` -------------------------------- ### Create a Project Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/api/projects.md Initialize a new project using the AsyncClient. ```python from phoenix.client import AsyncClient async_client = AsyncClient() project = await async_client.projects.create( name="My Project", description="A description of my project", ) print(f"Created project with ID: {project['id']}") ``` -------------------------------- ### Import Phoenix Client and Experiment Utilities (Deno) Source: https://github.com/arize-ai/phoenix/blob/main/js/examples/notebooks/deno_datasets_and_experiments_quickstart.ipynb Imports necessary components from the Phoenix client library for Deno, including client creation, experiment running, and dataset creation utilities. It also imports the OpenAI library for LLM interactions. ```typescript import { createClient } from "npm:@arizeai/phoenix-client@latest"; import { runExperiment, asEvaluator, evaluateExperiment } from "npm:@arizeai/phoenix-client@latest/experiments"; import { createDataset } from "npm:@arizeai/phoenix-client@latest/datasets"; import { OpenAI } from "npm:openai"; ``` -------------------------------- ### Manage Mintlify Documentation Source: https://github.com/arize-ai/phoenix/blob/main/DEVELOPMENT.md Commands to install the Mintlify CLI and launch a local development server for documentation previewing. ```bash npm i -g mint mint dev ``` -------------------------------- ### Initialize SimpleSpanProcessor Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-evals/docs/source/api/processors.md Examples for configuring SimpleSpanProcessor with various endpoints, exporters, and authentication headers. ```python from phoenix.otel import SimpleSpanProcessor processor = SimpleSpanProcessor() ``` ```python processor = SimpleSpanProcessor( endpoint="http://localhost:6006/v1/traces" ) ``` ```python from phoenix.otel import HTTPSpanExporter exporter = HTTPSpanExporter(endpoint="http://localhost:6006/v1/traces") processor = SimpleSpanProcessor(span_exporter=exporter) ``` ```python processor = SimpleSpanProcessor( endpoint="https://my-phoenix.com/v1/traces", headers={"Authorization": "Bearer my-token"} ) ``` -------------------------------- ### Install Dependencies Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/experiments/python_experiments_quickstart.ipynb Installs necessary libraries for the tutorial, including agno, arize-phoenix, openai, and instrumentation packages. ```python !pip install agno arize-phoenix openai openinference-agno openinference-openai ``` -------------------------------- ### Run a Specific Example File Source: https://github.com/arize-ai/phoenix/blob/main/js/packages/phoenix-client/examples/README.md Example of running the 'run_experiment.ts' file using tsx. ```sh tsx run_experiment.ts ``` -------------------------------- ### Install TanStack AI OpenInference and Dependencies Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/typescript/tanstack-ai/tanstack-ai-tracing.mdx Install the necessary packages for TanStack AI tracing. This includes the OpenInference middleware for TanStack AI and an OpenTelemetry setup. ```bash npm install --save @arizeai/openinference-tanstack-ai @tanstack/ai ``` ```bash npm install --save @arizeai/phoenix-otel ``` ```bash npm install --save @opentelemetry/api @opentelemetry/sdk-trace-node @opentelemetry/exporter-trace-otlp-proto ``` ```bash npm install --save @tanstack/ai-openai ``` -------------------------------- ### Create a new project Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/api/projects.md Initializes a new project with a name and optional description. ```python from phoenix.client import Client client = Client() project = client.projects.create( name="My Project", description="A description of my project", ) print(f"Created project with ID: {project['id']}") ``` -------------------------------- ### Load Test Dataset for Few-Shot Examples Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/cookbook/prompt-engineering/prompt-optimization.mdx Loads the test set from the 'jailbreak-classification' dataset and samples 10 examples for few-shot prompting. Ensure the 'datasets' library is installed. ```python from datasets import load_dataset ds_test = load_dataset("jackhhao/jailbreak-classification")[ "test" ] # this time, load in the test set instead of the training set few_shot_examples = ds_test.to_pandas().sample(10) ``` -------------------------------- ### Install Optional Phoenix Development Dependency (Bash) Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/integrations/python/graphite/graphite-integration-guide.mdx Installs the 'arize-phoenix' package, which provides optional development dependencies for local Phoenix tracing. ```bash pip install arize-phoenix ``` -------------------------------- ### Prepare Few-Shot Prompt Template Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/cookbook/datasets-and-experiments/summarization.mdx Prepares a few-shot prompt template by formatting example articles and summaries. This approach uses examples to guide the LLM's output. ```python # examples to include (not included in the uploaded dataset) train_df = hf_ds["train"] .to_pandas() .sample(n=5, random_state=42) .head() .rename(columns={"highlights": "summary"}) example_template = """ ARTICLE ======= {article} SUMMARY ======= {summary} """ examples = "\n".join( [ example_template.format(article=row["article"], summary=row["summary"]) for _, row in train_df.iterrows() ] ) template = """ Summarize the article in two to four sentences. Be concise and include only the most important information, as in the examples below. EXAMPLES ======== {examples} Now summarize the following article. ARTICLE ======= {article} SUMMARY ======= """ template = template.format( examples=examples, article="{article}", ) print(template) ``` -------------------------------- ### Create Dataset with Examples Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/experiments/run_experiments_splits.ipynb Use this to create a new dataset in Phoenix, providing a name and a list of examples. ```python from phoenix.client import AsyncClient dataset = await phoenix_client.datasets.create_dataset( name="triage-dataset", examples=examples, ) ``` -------------------------------- ### Clone and Install Phoenix to Arize Migration Tool Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/resources/phoenix-to-arize-ax-migration.mdx This snippet demonstrates how to clone the migration tool repository from GitHub and install its Python dependencies using pip. Ensure you have Git and pip installed. ```bash git clone https://github.com/Arize-ai/phoenix-to-ax-migration cd phoenix-to-ax-migration pip install -r requirements.txt ``` -------------------------------- ### Install Dependencies and Initialize Phoenix Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/prompts/from_openai.ipynb Installs the necessary Arize Phoenix and OpenAI instrumentation libraries, then launches the Phoenix application server. ```python %pip install -Uqqq arize-phoenix-client arize-phoenix-otel openai requests openinference-instrumentation-openai arize-phoenix import phoenix as px px.launch_app() ``` -------------------------------- ### Install Dependencies Source: https://github.com/arize-ai/phoenix/blob/main/examples/agents/tau_bench_openai_agents/README.md Install the required packages to run the agent and enable OpenInference tracing. ```bash pip install openai-agents openinference-instrumentation-openai-agents arize-phoenix litellm openai ``` -------------------------------- ### Quick Start Commands Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/sdk-api-reference/typescript/arizeai-phoenix-cli.mdx Configure your Phoenix instance using environment variables and execute common commands to fetch traces, list spans, or export data. ```bash # Configure your Phoenix instance export PHOENIX_HOST=http://localhost:6006 export PHOENIX_PROJECT=my-project export PHOENIX_API_KEY=your-api-key # if authentication is enabled # Fetch the most recent trace px trace list --limit 1 # Fetch a specific trace by ID px trace get abc123def456 # Fetch recent LLM spans px span list --span-kind LLM --limit 10 # Export traces to a directory px trace list ./my-traces --limit 50 ``` -------------------------------- ### Run Initial Experiment (Python) Source: https://github.com/arize-ai/phoenix/blob/main/js/examples/notebooks/deno_datasets_and_experiments_quickstart.ipynb Executes an experiment using the `runExperiment` function, passing a client, experiment name, dataset, task, and a list of evaluators. The function returns an experiment object containing its ID and results. This snippet demonstrates how to initiate an experiment with predefined evaluators. ```python console.log('Running initial experiment...'); // Pass dataset directly as the array of examples const experiment = await runExperiment({ client, experimentName: "simple-experiment", dataset: { datasetId, }, task, evaluators: [jaccardSimilarity, accuracy], logger: console, }); console.log('Initial experiment completed with ID:', experiment.id); await Deno.jupyter.md` ### Initial experiment results Experiment ID: `${experiment.id}` ` ``` -------------------------------- ### List and Inspect Datasets Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/index.md Lists all available datasets and prints their names, example counts, and creation times. Also shows how to retrieve a limited number of datasets and get a specific dataset with all its examples. ```python import pandas as pd # List all available datasets datasets = client.datasets.list() for dataset in datasets: print(f"Dataset: {dataset['name']} ({dataset['example_count']} examples)") print(f"Created: {dataset['created_at']}") # Get limited number of datasets for large collections limited_datasets = client.datasets.list(limit=10) # Get a specific dataset with all examples dataset = client.datasets.get_dataset(dataset="qa-evaluation") print(f"Dataset {dataset.name} has {len(dataset)} examples") print(f"Version ID: {dataset.version_id}") # Access dataset properties print(f"Description: {dataset.description}") print(f"Created: {dataset.created_at}") print(f"Updated: {dataset.updated_at}") ``` -------------------------------- ### Run gh-comment-watch Directly Source: https://github.com/arize-ai/phoenix/blob/main/scripts/gh-comment-watch/README.md Navigate to the script directory and run this command to install dependencies, build the UI, migrate the database, and serve the UI and API. Open http://localhost:58736 afterwards. ```bash cd scripts/gh-comment-watch pnpm start ``` -------------------------------- ### Docker Example with Authentication Enabled Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/self-hosting/features/authentication.mdx Deploy Phoenix using Docker with authentication enabled. This example sets the necessary authentication environment variables, including a default admin password for initial setup. ```bash docker run \ -e PHOENIX_ENABLE_AUTH=true \ -e PHOENIX_SECRET=change-me-32chars-min1digit1lower \ -e PHOENIX_DEFAULT_ADMIN_INITIAL_PASSWORD='strong-admin-password' \ -p 6006:6006 arizephoenix/phoenix:latest ``` -------------------------------- ### Run the TypeScript Tutorial with npm Source: https://github.com/arize-ai/phoenix/blob/main/js/examples/apps/experiments-quickstart/README.md Executes the main tutorial script using npm. This command initiates the Phoenix experiments, allowing you to view results and traces in the Phoenix application. Ensure all dependencies are installed and environment variables are configured before running. ```bash npm start ``` -------------------------------- ### Install Arize Phoenix and Dependencies Source: https://github.com/arize-ai/phoenix/blob/main/tutorials/experiments/datasets_and_experiments_quickstart.ipynb Installs the Arize Phoenix library with evaluation capabilities, along with OpenAI and httpx. The httpx version is pinned to ensure compatibility. ```python !pip install "arize-phoenix[evals]" openai 'httpx<0.28' ``` -------------------------------- ### Computer Use Tool Example (Beta) Source: https://github.com/arize-ai/phoenix/blob/main/internal_docs/specs/vendor-specific-tools/anthropic-messages-tool-union-illustration.md Example JSON structure for the `BetaToolComputerUse20251124Param`. Configure display width/height for screenshot resolution and click coordinates. `enable_zoom` may alter model planning for UI navigation. ```json { "type": "computer_20251124", "name": "computer", "display_width_px": 1920, "display_height_px": 1080, "display_number": 0, "enable_zoom": true } ``` -------------------------------- ### Dataset Management Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/README.md Manage evaluation datasets and examples for experiments and evaluation. Supports listing, getting, and creating datasets. ```APIDOC ## List Datasets ### Description Lists all available datasets. ### Method `client.datasets.list()` ### Response Example ```json [ { "name": "dataset-name", "example_count": 100 } ] ``` ## Get Dataset ### Description Retrieves a specific dataset with all its examples. ### Method `client.datasets.get_dataset(dataset: str)` ### Parameters #### Path Parameters - **dataset** (str) - Required - The name of the dataset to retrieve. ### Response Example ```json { "name": "qa-evaluation", "examples": [ {"input": "...", "output": "...", "metadata": {...}} ] } ``` ## Create Dataset from Dictionaries ### Description Creates a new dataset from provided dictionaries for inputs, outputs, and metadata. ### Method `client.datasets.create_dataset(name: str, dataset_description: str, inputs: list, outputs: list, metadata: list)` ### Parameters #### Path Parameters - **name** (str) - Required - The name of the new dataset. - **dataset_description** (str) - Optional - A description for the dataset. - **inputs** (list) - Required - A list of dictionaries, where each dictionary represents an input. - **outputs** (list) - Required - A list of dictionaries, where each dictionary represents an output. - **metadata** (list) - Optional - A list of dictionaries, where each dictionary represents metadata for an example. ### Request Example ```python client.datasets.create_dataset( name="customer-support-qa", dataset_description="Q&A dataset for customer support evaluation", inputs=[ {"question": "How do I reset my password?"}, {"question": "What's your return policy?"}, {"question": "How do I track my order?"} ], outputs=[ {"answer": "You can reset your password by clicking the 'Forgot Password' link on the login page."}, {"answer": "We offer 30-day returns for unused items in original packaging."}, {"answer": "You can track your order using the tracking number sent to your email."} ], metadata=[ {"category": "account", "difficulty": "easy"}, {"category": "policy", "difficulty": "medium"}, {"category": "orders", "difficulty": "easy"} ] ) ``` ## Create Dataset from Pandas DataFrame ### Description Creates a new dataset from a pandas DataFrame, specifying which columns to use for input, output, and metadata. ### Method `client.datasets.create_dataset(name: str, dataframe: pd.DataFrame, input_keys: list, output_keys: list, metadata_keys: list)` ### Parameters #### Path Parameters - **name** (str) - Required - The name of the new dataset. - **dataframe** (pd.DataFrame) - Required - The pandas DataFrame containing the dataset. - **input_keys** (list) - Required - A list of column names to be used as input. - **output_keys** (list) - Required - A list of column names to be used as output. - **metadata_keys** (list) - Optional - A list of column names to be used as metadata. ### Request Example ```python import pandas as pd df = pd.DataFrame({ "prompt": ["Hello", "Hi there", "Good morning"], "response": ["Hi! How can I help?", "Hello! What can I do for you?", "Good morning! How may I assist?"], "sentiment": ["neutral", "positive", "positive"], "length": [20, 25, 30] }) client.datasets.create_dataset( name="greeting-responses", dataframe=df, input_keys=["prompt"], output_keys=["response"], metadata_keys=["sentiment", "length"] ) ``` ``` -------------------------------- ### Sample Few-Shot Examples Source: https://github.com/arize-ai/phoenix/blob/main/docs/phoenix/cookbook/prompt-engineering/chain-of-thought-prompting.mdx Load a dataset and sample examples to be used for few-shot prompting. ```javascript ds = load_dataset("syeddula/math_word_problems")["test"] few_shot_examples = ds.to_pandas().sample(5) few_shot_examples ``` -------------------------------- ### Get a Dataset Source: https://github.com/arize-ai/phoenix/blob/main/packages/phoenix-client/docs/source/api/datasets.md Retrieves a specific dataset by its name and prints the number of examples it contains. Ensure the Client is initialized. ```python from phoenix.client import Client client = Client() # Get a dataset dataset = client.datasets.get_dataset(dataset="my-dataset") print(f"Dataset {dataset.name} has {len(dataset)} examples") ``` -------------------------------- ### MCP Toolset Example (Beta) Source: https://github.com/arize-ai/phoenix/blob/main/internal_docs/specs/vendor-specific-tools/anthropic-messages-tool-union-illustration.md Example JSON structure for the `BetaMCPToolsetParam`. Use `default_config` and `configs` to defer heavy MCP tools until referenced, mirroring `defer_loading` on user tools. ```json { "type": "mcp_toolset", "mcp_server_name": "my-server", "default_config": { "enabled": true, "defer_loading": false }, "configs": { "some_tool": { "enabled": true, "defer_loading": true } } } ``` -------------------------------- ### px dataset get Source: https://github.com/arize-ai/phoenix/blob/main/js/packages/phoenix-cli/README.md Fetches examples from a specified dataset. Supports various output formats and filtering by split or version. ```APIDOC ## px dataset get ### Description Fetch examples from a dataset. ### Usage ```bash px dataset get [--format ] [--split ] [--version ] [--file ] ``` ### Parameters #### Path Parameters - **dataset-identifier** (string) - Required - The identifier of the dataset to fetch. #### Options - **--format** (string) - Optional - Output format: `raw`, `json`, `text`. Defaults to `pretty`. - **--split** (string) - Optional - Specify a dataset split (e.g., `train`, `test`). - **--version** (string) - Optional - Specify a dataset version ID. - **--file** (string) - Optional - Path to a JSON file containing dataset configuration. ### Examples ```bash px dataset get my-dataset --format raw | jq '.examples[].input' px dataset get my-dataset --split train --split test px dataset get my-dataset --version px dataset get my-dataset --file dataset.json ``` ``` -------------------------------- ### Initialize Experiment Client Source: https://github.com/arize-ai/phoenix/blob/main/internal_docs/specs/experiment-runner-background-process/appendix-rate-limiting.md Sets up the experiment class by accessing the rate limiter from the provided streaming client. ```python class Experiment: def __init__(self, client: PlaygroundStreamingClient): self._token_bucket = client.rate_limiter._throttler ```