### Install dependencies Source: https://docs.uselemma.ai/integrations/openai-agents-sdk-python Install the required tracing and SDK packages. ```bash pip install "uselemma-tracing[openai-agents]" openai-agents ``` -------------------------------- ### Install Lemma SDK Source: https://docs.uselemma.ai/getting-started/quickstart Install the required package for your environment. ```bash npm install @uselemma/tracing ``` ```bash pip install uselemma-tracing ``` -------------------------------- ### Install uselemma-tracing and LangChain dependencies Source: https://docs.uselemma.ai/integrations/langchain Install the necessary libraries for uselemma-tracing, LangChain, and the OpenInference LangChain instrumentation. Ensure you have LangChain and OpenAI integrations installed. ```bash pip install uselemma-tracing langchain langchain-openai openinference-instrumentation-langchain ``` -------------------------------- ### Install uselemma-tracing and dependencies Source: https://docs.uselemma.ai/integrations/openai-sdk-python Install the necessary packages for uselemma-tracing, OpenAI, and the OpenAI instrumentation. ```bash pip install uselemma-tracing openai openinference-instrumentation-openai ``` -------------------------------- ### Install dependencies Source: https://docs.uselemma.ai/integrations/anthropic-sdk Install the required tracing and instrumentation packages via npm. ```bash npm install @uselemma/tracing anthropic @opentelemetry/instrumentation @arizeai/openinference-instrumentation-anthropic ``` -------------------------------- ### Install @uselemma/tracing and OpenAI SDK Source: https://docs.uselemma.ai/recipes/manual-instrumentation Install the necessary packages for tracing and OpenAI integration. This is the first step before configuring tracing. ```bash npm install @uselemma/tracing openai ``` ```bash pip install uselemma-tracing openai ``` -------------------------------- ### Install Dependencies Source: https://docs.uselemma.ai/integrations/arize-phoenix Install the required packages for Lemma tracing and OpenTelemetry instrumentation. ```bash npm install @uselemma/tracing @opentelemetry/sdk-trace-node @opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-proto @opentelemetry/instrumentation @arizeai/openinference-instrumentation-openai ``` ```bash pip install uselemma-tracing opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-http ``` -------------------------------- ### Install uselemma-tracing (Python) Source: https://docs.uselemma.ai/tracing/using-a-supported-framework Install the Lemma tracing library for Python projects using pip. ```bash pip install uselemma-tracing ``` -------------------------------- ### Install Lemma Tracing Dependencies Source: https://docs.uselemma.ai/integrations/langfuse Install the required packages for tracing and OpenInference instrumentation. ```bash npm install @uselemma/tracing @opentelemetry/instrumentation @arizeai/openinference-instrumentation-openai ``` -------------------------------- ### Python: Streaming Agent Implementation Source: https://docs.uselemma.ai/recipes/streaming-sse This Python example demonstrates streaming agent behavior using `asyncio.Queue` to manage chunks. The agent is started in the background, and chunks are consumed concurrently. `streaming=True` is essential for the agent wrapper. ```python import asyncio from uselemma_tracing import register_otel, TraceContext, agent register_otel() async def run_agent(input: dict, ctx: TraceContext) -> str: queue: asyncio.Queue = input["queue"] full_response = "" async for chunk in stream_llm(input["user_message"]): full_response += chunk await queue.put(chunk) await queue.put(None) # sentinel — signals end of stream to consumer ctx.complete(full_response) # close span with assembled output return full_response wrapped = agent("my-agent", run_agent, streaming=True) async def handle_request(user_message: str): queue: asyncio.Queue = asyncio.Queue() # Start the agent in the background; consume chunks concurrently agent_task = asyncio.create_task( wrapped({"user_message": user_message, "queue": queue}) ) while True: chunk = await queue.get() if chunk is None: break yield chunk # write to SSE / WebSocket / response stream here await agent_task # ensure span closes and run_id is available _, run_id, _ = agent_task.result() ``` -------------------------------- ### Install OpenInference Instrumentation Dependencies Source: https://docs.uselemma.ai/tracing/adding-provider-instrumentation Install the required packages for your specific provider and language to enable automatic tracing. ```bash npm install @uselemma/tracing openai @opentelemetry/instrumentation @arizeai/openinference-instrumentation-openai ``` ```bash npm install @uselemma/tracing anthropic @opentelemetry/instrumentation @arizeai/openinference-instrumentation-anthropic ``` ```bash pip install uselemma-tracing openai openinference-instrumentation-openai ``` ```bash pip install uselemma-tracing anthropic openinference-instrumentation-anthropic ``` ```bash pip install uselemma-tracing litellm openinference-instrumentation-litellm ``` -------------------------------- ### Install OpenAI Agents SDK and Lemma dependencies Source: https://docs.uselemma.ai/integrations/openai-agents-sdk Install the required packages for agent development and tracing. ```bash npm install @openai/agents @uselemma/tracing zod ``` -------------------------------- ### Prompt Agent with Manual Installation Source: https://docs.uselemma.ai/getting-started/quickstart After manually installing the Lemma AI skill, prompt your agent to use the skill from the specified local path to add tracing to your application. ```txt Use the Lemma AI skill in .cursor/rules/lemma-tracing to add tracing to this application. ``` -------------------------------- ### Tool-calling agent with uselemma-tracing and @tool decorator Source: https://docs.uselemma.ai/integrations/anthropic-sdk-python Example of creating a tool-calling agent using the Anthropic SDK and uselemma-tracing. This snippet defines a tool for getting weather information and an agent that can use this tool to respond to user queries. ```python from uselemma_tracing import agent, tool, TraceContext import anthropic import json client = anthropic.AsyncAnthropic() @tool("get-weather") async def get_weather(city: str) -> dict: return {"city": city, "temperature": "72°F", "condition": "sunny"} TOOLS = [ { "name": "get_weather", "description": "Get current weather for a city", "input_schema": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"], }, } ] async def run_agent(user_message: str, ctx: TraceContext) -> str: messages = [{"role": "user", "content": user_message}] response = await client.messages.create( model="claude-sonnet-4-5", max_tokens=1024, tools=TOOLS, messages=messages, ) while response.stop_reason == "tool_use": tool_uses = [b for b in response.content if b.type == "tool_use"] messages.append({"role": "assistant", "content": response.content}) tool_results = [] for block in tool_uses: result = await get_weather(block.input["city"]) # tool.get-weather span tool_results.append( {"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(result)} ) messages.append({"role": "user", "content": tool_results}) response = await client.messages.create( model="claude-sonnet-4-5", max_tokens=1024, tools=TOOLS, messages=messages, ) text_blocks = [b for b in response.content if b.type == "text"] return text_blocks[0].text if text_blocks else "" weatherAgent = agent("weather-agent", run_agent) res = await weatherAgent("What's the weather in Paris?") ``` -------------------------------- ### Complete Agent Loop Example Source: https://docs.uselemma.ai/recipes/manual-instrumentation This example demonstrates how to integrate tracing for agents, LLM calls, and tools within a complete agent loop. Ensure tracing is imported and registered. ```typescript import "./tracer"; import { agent, llm, tool } from "@uselemma/tracing"; import OpenAI from "openai"; const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }); const generate = llm("gpt-4o", async (prompt: string) => { const res = await client.chat.completions.create({ model: "gpt-4o", messages: [{ role: "user", content: prompt }], }); return res.choices[0]?.message?.content ?? ""; }); const getWeatherTool = tool("get-weather", async (city: string) => { return await getWeather(city); }); ``` -------------------------------- ### Install uselemma-tracing, anthropic, and openinference-instrumentation-anthropic Source: https://docs.uselemma.ai/integrations/anthropic-sdk-python Install the necessary packages for tracing Anthropic API calls. This includes the uselemma-tracing library, the official Anthropic Python SDK, and the OpenInference instrumentation for Anthropic. ```bash pip install uselemma-tracing anthropic openinference-instrumentation-anthropic ``` -------------------------------- ### Install Lemma Tracing Dependency Source: https://docs.uselemma.ai/integrations/vercel-ai-sdk Install the required tracing package via npm. ```bash npm install @uselemma/tracing ``` -------------------------------- ### Install Lemma AI Skill for Agent Source: https://docs.uselemma.ai/getting-started/quickstart Instruct your coding agent to install the Lemma AI skill from GitHub and apply it to your application for automatic tracing. ```txt Install the Lemma AI skill from github.com/uselemma/skills and use it to add tracing to this application. ``` -------------------------------- ### GET /llms.txt Source: https://docs.uselemma.ai/api-reference/traces/list-traces Fetches the complete documentation index for the project. ```APIDOC ## GET /llms.txt ### Description Fetch the complete documentation index to discover all available pages. ### Method GET ### Endpoint https://docs.uselemma.ai/llms.txt ``` -------------------------------- ### Install TypeScript Dependencies for Azure and Lemma Source: https://docs.uselemma.ai/integrations/azure-application-insights Install the necessary npm packages for Azure Application Insights telemetry and Lemma tracing. These are typically already installed if Azure telemetry is configured. ```bash npm install @azure/monitor-opentelemetry-exporter @opentelemetry/sdk-trace-node @opentelemetry/sdk-trace-base ``` ```bash npm install @uselemma/tracing @opentelemetry/instrumentation @arizeai/openinference-instrumentation-openai ``` -------------------------------- ### Configure Node.js tracer Source: https://docs.uselemma.ai/integrations/openai-agents-sdk Initialize the tracer in a standard Node.js environment by creating a setup file and importing it at the application entry point. ```typescript // tracer.ts import { registerOTel } from '@uselemma/tracing'; registerOTel(); ``` ```typescript // index.ts or server.ts import './tracer'; // Must be first! // ... rest of your imports ``` -------------------------------- ### Python Context Manager Usage Source: https://docs.uselemma.ai/tracing/patterns/custom-attributes Example of setting attributes using a context manager in Python. ```APIDOC ## Python context manager In context manager mode, access the span through the yielded run object: ### Method ```python async with agent("support-agent", input=input_data) as run: run.span.set_attribute("lemma.user_id", input_data["user_id"]) run.span.set_attribute("lemma.environment", "production") result = await handle_request(input_data["message"]) run.complete(result) ``` ``` -------------------------------- ### Manual Installation of Lemma AI Skill Source: https://docs.uselemma.ai/getting-started/quickstart Clone the Lemma AI skills repository and copy the tracing skill into your project's rules directory for manual integration. ```bash git clone https://github.com/uselemma/skills.git cp -r skills/lemma-tracing .cursor/rules/lemma-tracing ``` -------------------------------- ### Query Lemma via AI Agent Source: https://docs.uselemma.ai/connections/mcp Example prompt for interacting with the Lemma MCP tools within an AI coding assistant. ```text Use Lemma to show me my last 5 traces, including any errors. ``` -------------------------------- ### Single-turn completion with uselemma-tracing Source: https://docs.uselemma.ai/integrations/anthropic-sdk-python Example of performing a single-turn completion using the Anthropic SDK with uselemma-tracing. This demonstrates how to create an agent that sends a user message and receives a text response, with tracing automatically applied. ```python from uselemma_tracing import agent, TraceContext import anthropic client = anthropic.AsyncAnthropic() async def run_agent(user_message: str, ctx: TraceContext) -> str: message = await client.messages.create( model="claude-haiku-4-5", max_tokens=1024, messages=[{"role": "user", "content": user_message}], ) return message.content[0].text supportAgent = agent("support-agent", run_agent) res = await supportAgent("Explain async/await in one sentence.") print(res.result) print(res.run_id) ``` -------------------------------- ### Install Python Dependencies for Azure and Lemma Source: https://docs.uselemma.ai/integrations/azure-application-insights Install the necessary pip packages for Azure Application Insights telemetry and Lemma integration. The Lemma package is the only new dependency if Azure telemetry is already configured. ```bash pip install azure-monitor-opentelemetry-exporter opentelemetry-api opentelemetry-sdk ``` ```bash pip install uselemma-tracing ``` -------------------------------- ### Streaming responses with uselemma-tracing Source: https://docs.uselemma.ai/integrations/anthropic-sdk-python Example demonstrating how to handle streaming responses from the Anthropic SDK with uselemma-tracing. The code assembles the full text from the stream and then completes the trace span. ```python from uselemma_tracing import agent, TraceContext import anthropic client = anthropic.AsyncAnthropic() async def run_agent(user_message: str, ctx: TraceContext): full_text = "" async with client.messages.stream( model="claude-haiku-4-5", max_tokens=512, messages=[{"role": "user", "content": user_message}], ) as stream: async for text in stream.text_stream: full_text += text ctx.complete(full_text) # store assembled text as output, close span return full_text streamingAgent = agent("streaming-agent", run_agent, streaming=True) res = await streamingAgent("Write a haiku about observability.") ``` -------------------------------- ### Trace a simple LangChain chain Source: https://docs.uselemma.ai/integrations/langchain Example of tracing a basic LangChain chain involving a prompt, an LLM call, and an output parser. Ensure LEMMA_API_KEY and LEMMA_PROJECT_ID environment variables are set. ```python from uselemma_tracing import agent, TraceContext from langchain_openai import ChatOpenAI from langchain_core.prompts import ChatPromptTemplate from langchain_core.output_parsers import StrOutputParser llm = ChatOpenAI(model="gpt-4o-mini") prompt = ChatPromptTemplate.from_template("Answer briefly: {question}") chain = prompt | llm | StrOutputParser() async def run_agent(question: str, ctx: TraceContext) -> str: result = await chain.ainvoke({"question": question}) return result return result qaAgent = agent("qa-agent", run_agent) res = await qaAgent("What is the speed of light?") print(res.result) print(res.run_id) ``` -------------------------------- ### Set Up Tracer Provider for Next.js in TypeScript Source: https://docs.uselemma.ai/integrations/azure-application-insights This instrumentation setup is specifically for Next.js applications, ensuring tracing is correctly registered within the Node.js runtime environment. ```typescript // instrumentation.ts export async function register() { if (process.env.NEXT_RUNTIME === "nodejs") { const { NodeTracerProvider } = await import("@opentelemetry/sdk-trace-node"); const { BatchSpanProcessor } = await import("@opentelemetry/sdk-trace-base"); const { AzureMonitorTraceExporter } = await import("@azure/monitor-opentelemetry-exporter"); const { createLemmaSpanProcessor } = await import("@uselemma/tracing"); const { registerInstrumentations } = await import("@opentelemetry/instrumentation"); const { OpenAIInstrumentation } = await import("@arizeai/openinference-instrumentation-openai"); const tracerProvider = new NodeTracerProvider({ spanProcessors: [ new BatchSpanProcessor( new AzureMonitorTraceExporter({ connectionString: process.env.APPLICATIONINSIGHTS_CONNECTION_STRING, }) ), createLemmaSpanProcessor(), ], }); tracerProvider.register(); registerInstrumentations({ instrumentations: [new OpenAIInstrumentation()], tracerProvider: tracerProvider, }); } } ``` -------------------------------- ### Single-turn completion with uselemma-tracing Source: https://docs.uselemma.ai/integrations/openai-sdk-python Example of making a single-turn chat completion call using the OpenAI SDK, wrapped by uselemma-tracing's agent functionality. This will create a top-level run trace and a child span for the chat completion. ```python from uselemma_tracing import agent, TraceContext from openai import AsyncOpenAI client = AsyncOpenAI() async def run_agent(user_message: str, ctx: TraceContext) -> str: response = await client.chat.completions.create( model="gpt-4o-mini", messages=[{"role": "user", "content": user_message}], ) return response.choices[0].message.content or "" supportAgent = agent("support-agent", run_agent) res = await supportAgent("How do I reset my password?") print(res.result) print(res.run_id) ``` -------------------------------- ### Set Up Tracer Provider in TypeScript Source: https://docs.uselemma.ai/integrations/azure-application-insights Configure the NodeTracerProvider with BatchSpanProcessor for exporting spans to Azure Application Insights and Lemma. This setup is crucial for LLM observability. ```typescript import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base"; import { AzureMonitorTraceExporter } from "@azure/monitor-opentelemetry-exporter"; import { createLemmaSpanProcessor } from "@uselemma/tracing"; import { registerInstrumentations } from "@opentelemetry/instrumentation"; import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai"; const tracerProvider = new NodeTracerProvider({ spanProcessors: [ // Export to Azure Application Insights new BatchSpanProcessor( new AzureMonitorTraceExporter({ connectionString: process.env.APPLICATIONINSIGHTS_CONNECTION_STRING, }) ), // Export to Lemma createLemmaSpanProcessor(), ], }); tracerProvider.register(); registerInstrumentations({ instrumentations: [new OpenAIInstrumentation()], tracerProvider: tracerProvider, }); ``` -------------------------------- ### Example of remainingChildren in logs Source: https://docs.uselemma.ai/tracing/troubleshooting/common-problems Observe the `remainingChildren` value in logs to identify if a child span is stuck open, preventing the run from closing. This indicates a span that was started but never ended. ```text [LEMMA:processor] onEnd: direct child ended { ..., remainingChildren: 1 } [LEMMA:processor] onEnd: direct child ended { ..., remainingChildren: 1 } # remainingChildren never hits 0 — one span is stuck open ``` -------------------------------- ### Set up instrumentation Source: https://docs.uselemma.ai/integrations/openai-agents-sdk-python Initialize the Lemma instrumentation before importing the agents module. ```python # main.py from dotenv import load_dotenv load_dotenv() # Must come before importing openai-agents from uselemma_tracing import instrument_openai_agents instrument_openai_agents() from agents import Agent, Runner ``` -------------------------------- ### Register instrumentation at startup Source: https://docs.uselemma.ai/integrations/anthropic-sdk Configure OTel and Anthropic instrumentation for different runtime environments. ```typescript // instrumentation.ts export async function register() { if (process.env.NEXT_RUNTIME === "nodejs") { const { registerOTel } = await import("@uselemma/tracing"); const { registerInstrumentations } = await import("@opentelemetry/instrumentation"); const { AnthropicInstrumentation } = await import( "@arizeai/openinference-instrumentation-anthropic" ); const provider = registerOTel(); registerInstrumentations({ instrumentations: [new AnthropicInstrumentation()], tracerProvider: provider, }); } } ``` ```typescript // tracer.ts — import this first in your entry point import { registerOTel } from "@uselemma/tracing"; import { registerInstrumentations } from "@opentelemetry/instrumentation"; import { AnthropicInstrumentation } from "@arizeai/openinference-instrumentation-anthropic"; const provider = registerOTel(); registerInstrumentations({ instrumentations: [new AnthropicInstrumentation()], tracerProvider: provider, }); ``` ```typescript // index.ts import "./tracer"; // must be first ``` -------------------------------- ### Configure Environment Variables Source: https://docs.uselemma.ai/getting-started/quickstart Set the API key and project ID required for authentication with the Lemma platform. ```bash export LEMMA_API_KEY="lma_..." export LEMMA_PROJECT_ID="proj_..." ``` -------------------------------- ### Register instrumentation at startup Source: https://docs.uselemma.ai/integrations/openai-sdk Configure OpenTelemetry and OpenAI instrumentation for Next.js or Node.js environments. ```typescript // instrumentation.ts export async function register() { if (process.env.NEXT_RUNTIME === "nodejs") { const { registerOTel } = await import("@uselemma/tracing"); const { registerInstrumentations } = await import("@opentelemetry/instrumentation"); const { OpenAIInstrumentation } = await import( "@arizeai/openinference-instrumentation-openai" ); const provider = registerOTel(); registerInstrumentations({ instrumentations: [new OpenAIInstrumentation()], tracerProvider: provider, }); } } ``` ```typescript // tracer.ts — import this first in your entry point import { registerOTel } from "@uselemma/tracing"; import { registerInstrumentations } from "@opentelemetry/instrumentation"; import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai"; const provider = registerOTel(); registerInstrumentations({ instrumentations: [new OpenAIInstrumentation()], tracerProvider: provider, }); ``` ```typescript // index.ts import "./tracer"; // must be first ``` -------------------------------- ### GET /traces Source: https://docs.uselemma.ai/api-reference/traces/list-traces Retrieves a list of recent traces for a project with optional filtering capabilities. ```APIDOC ## GET /traces ### Description Returns recent traces for a project, optionally filtered by time range, environment, and error status. ### Method GET ### Endpoint /traces ``` -------------------------------- ### Configure Lemma with a fresh TracerProvider Source: https://docs.uselemma.ai/recipes/dual-export Manually build a TracerProvider and register both the Lemma span processor and an existing OTLP exporter. ```typescript import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node"; import { BatchSpanProcessor } from "@opentelemetry/sdk-trace-base"; import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-proto"; import { createLemmaSpanProcessor } from "@uselemma/tracing"; const provider = new NodeTracerProvider({ spanProcessors: [ createLemmaSpanProcessor(), new BatchSpanProcessor( new OTLPTraceExporter({ url: "https://your-collector/v1/traces" }) ), ], }); provider.register(); ``` ```python from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry import trace from uselemma_tracing import create_lemma_span_processor provider = TracerProvider() provider.add_span_processor(create_lemma_span_processor()) provider.add_span_processor( BatchSpanProcessor( OTLPSpanExporter(endpoint="https://your-collector/v1/traces") ) ) trace.set_tracer_provider(provider) ``` -------------------------------- ### Add Lemma to a Fresh Provider (Python) Source: https://docs.uselemma.ai/tracing/patterns/dual-export Initialize a TracerProvider, add the Lemma span processor, and then add a batch span processor with an OTLPSpanExporter. Set the provider using trace.set_tracer_provider(). ```python from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry import trace from uselemma_tracing import create_lemma_span_processor provider = TracerProvider() provider.add_span_processor(create_lemma_span_processor()) provider.add_span_processor( BatchSpanProcessor( OTLPSpanExporter(endpoint="https://your-collector/v1/traces") ) ) trace.set_tracer_provider(provider) ``` -------------------------------- ### Trace an Anthropic agent Source: https://docs.uselemma.ai/getting-started/quickstart Example of wrapping an asynchronous Anthropic client call with a Lemma agent for tracing. ```python from uselemma_tracing import agent, TraceContext import anthropic client = anthropic.AsyncAnthropic() async def run_agent(user_message: str, ctx: TraceContext) -> str: message = await client.messages.create( model="claude-haiku-4-5", max_tokens=256, messages=[{"role": "user", "content": user_message}], ) return message.content[0].text myAgent = agent("my-agent", run_agent) res = await myAgent("What is the capital of France?") print(res.result) print(res.run_id) ``` -------------------------------- ### Tool-calling agent Source: https://docs.uselemma.ai/integrations/anthropic-sdk Example of a traced agent that handles tool definitions and multi-turn tool execution loops. ```typescript import { agent, tool } from "@uselemma/tracing"; import Anthropic from "@anthropic-ai/sdk"; const anthropic = new Anthropic(); const getWeather = tool("get-weather", async (city: string) => { return { city, temperature: "72°F", condition: "sunny" }; }); const weatherAgent = agent("weather-agent", async (input: { message: string }, ctx) => { const toolSpec: Anthropic.Tool[] = [ { name: "get_weather", description: "Get current weather for a city", input_schema: { type: "object", properties: { city: { type: "string" } }, required: ["city"], }, }, ]; const messages: Anthropic.MessageParam[] = [ { role: "user", content: input.message }, ]; let response = await anthropic.messages.create({ model: "claude-sonnet-4-5", max_tokens: 1024, tools: toolSpec, messages, }); while (response.stop_reason === "tool_use") { const toolUseBlocks = response.content.filter((b) => b.type === "tool_use"); messages.push({ role: "assistant", content: response.content }); const toolResults: Anthropic.ToolResultBlockParam[] = []; for (const block of toolUseBlocks) { if (block.type !== "tool_use") continue; const weatherData = await getWeather(block.input.city as string); toolResults.push({ type: "tool_result", tool_use_id: block.id, content: JSON.stringify(weatherData) }); } messages.push({ role: "user", content: toolResults }); response = await anthropic.messages.create({ model: "claude-sonnet-4-5", max_tokens: 1024, tools: toolSpec, messages, }); } const textBlock = response.content.find((b) => b.type === "text"); const text = textBlock?.type === "text" ? textBlock.text : ""; return text; }); const { result, runId } = await weatherAgent({ message: "What's the weather in Paris?" }); ``` -------------------------------- ### Single-turn completion agent Source: https://docs.uselemma.ai/integrations/anthropic-sdk Example of wrapping an Anthropic message creation call within a traced agent. ```typescript import { agent } from "@uselemma/tracing"; import Anthropic from "@anthropic-ai/sdk"; const anthropic = new Anthropic(); const supportAgent = agent("support-agent", async (input: { message: string }) => { const response = await anthropic.messages.create({ model: "claude-haiku-4-5", max_tokens: 1024, messages: [{ role: "user", content: input.message }], }); return response.content[0].type === "text" ? response.content[0].text : ""; }); const { result, runId } = await supportAgent({ message: "Explain async/await in one sentence." }); ``` -------------------------------- ### GET /metrics/{metric_id} Source: https://docs.uselemma.ai/api-reference/metrics/get-metric-details Retrieves detailed configuration, recent events, and monitor summaries for a specific metric. ```APIDOC ## GET /metrics/{metric_id} ### Description Returns metric configuration, recent events, and related monitor summary in one payload. ### Method GET ### Endpoint /metrics/{metric_id} ### Parameters #### Path Parameters - **metric_id** (string/uuid) - Required - Metric identifier. ### Response #### Success Response (200) - **id** (string) - Metric ID - **name** (string) - Metric name - **description** (string|null) - Metric description - **type** (string) - Metric type - **collection_type** (string|null) - Collection type - **collection_config** (object|null) - Collection configuration - **project_id** (string) - Project ID - **project_name** (string) - Project name - **preset** (boolean) - Whether the metric is a preset - **agent_name** (string|null) - Associated agent name - **sampling_rate** (integer|null) - Sampling rate - **created_at** (string|null) - Creation timestamp - **total_events** (integer) - Total event count - **recent_events** (array) - List of recent metric events - **monitor** (object|null) - Linked monitor summary #### Response Example { "id": "uuid", "name": "example_metric", "type": "counter", "project_id": "proj_123", "project_name": "my_project", "preset": false, "total_events": 10, "recent_events": [] } ``` -------------------------------- ### Register Tracing at Startup Source: https://docs.uselemma.ai/integrations/langfuse Configure the tracer provider and register instrumentations during application initialization. ```typescript import { registerOTel } from '@uselemma/tracing'; import { registerInstrumentations } from '@opentelemetry/instrumentation'; import { OpenAIInstrumentation } from '@arizeai/openinference-instrumentation-openai'; const provider = registerOTel(); registerInstrumentations({ instrumentations: [new OpenAIInstrumentation()], tracerProvider: provider, }); ``` -------------------------------- ### GET /metrics/{metric_id}/monitor/evaluations Source: https://docs.uselemma.ai/api-reference/metrics/list-monitor-evaluations Retrieves historical evaluation results for a specific metric's monitor. ```APIDOC ## GET /metrics/{metric_id}/monitor/evaluations ### Description Returns historical evaluation results for the monitor attached to this metric (e.g. threshold checks over time). ### Method GET ### Endpoint /metrics/{metric_id}/monitor/evaluations ### Parameters #### Path Parameters - **metric_id** (string) - Required - Metric whose monitor evaluations to list. #### Query Parameters - **limit** (integer) - Optional - Page size. Maximum value is 200, minimum is 1. Defaults to 50. - **offset** (integer) - Optional - Pagination offset. Minimum value is 0. Defaults to 0. ### Responses #### Success Response (200) - **evaluations** (array) - List of monitor evaluation results. - **total** (integer) - Total number of evaluations available. - **limit** (integer) - The limit used for pagination. - **offset** (integer) - The offset used for pagination. - **has_more** (boolean) - Indicates if there are more results available. #### Error Response (422) - **detail** (array) - Details of the validation error. ### Request Example ```json { "example": "No request body for GET request" } ``` ### Response Example (Success) ```json { "evaluations": [ { "status": "pass", "value": 0.95, "created_at": "2023-10-27T10:00:00Z" } ], "total": 100, "limit": 50, "offset": 0, "has_more": true } ``` ### Response Example (Error) ```json { "detail": [ { "loc": ["query", "limit"], "msg": "ensure this value is less than or equal to 200", "type": "value_error.number.max_value", "input": 250, "ctx": {"limit_value": 200} } ] } ``` ``` -------------------------------- ### Python: Tool-calling agent loop Source: https://docs.uselemma.ai/recipes/tool-calling-agent Use this for agents that loop, where the model calls tools, tools return results, and the model calls again until a stop condition is reached. The run span stays open for the entire loop and closes when the wrapped function returns. Each tool execution gets its own `tool.*` child span, and each LLM call gets its own `llm.*` child span. ```python from uselemma_tracing import register_otel, agent, tool, llm, TraceContext register_otel() @tool("dispatch") async def execute_tool(name: str, args: dict) -> str: return await run_tool(name, args) @llm("gpt-4o") async def call_llm(history: list) -> object: return await call_model(history) async def run_agent(input: dict, ctx: TraceContext) -> str: history = [{"role": "user", "content": input["user_message"]}] while True: response = await call_llm(history) # span: llm.gpt-4o if response.stop_reason == "end_turn": return response.text for tool_call in response.tool_calls: result = await execute_tool(tool_call.name, tool_call.args) # span: tool.dispatch history.append({"role": "tool", "name": tool_call.name, "content": result}) wrapped = agent("my-agent", run_agent) result, run_id, _ = await wrapped({"user_message": "What is the weather in London?"}) ``` -------------------------------- ### TypeScript: Tool-calling agent loop Source: https://docs.uselemma.ai/recipes/tool-calling-agent Use this for agents that loop, where the model calls tools, tools return results, and the model calls again until a stop condition is reached. The run span stays open for the entire loop and closes when the wrapped function returns. Each tool execution gets its own `tool.*` child span, and each LLM call gets its own `llm.*` child span. ```typescript import { registerOTel, agent, tool, llm } from "@uselemma/tracing"; registerOTel(); const executeTool = tool("dispatch", async (name: string, args: Record) => { return await runTool(name, args); }); const callLLM = llm("gpt-4o", async (history: unknown[]) => { return await callModel(history); }); const wrapped = agent("my-agent", async (input: { userMessage: string }) => { const history = [{ role: "user", content: input.userMessage }]; while (true) { const response = await callLLM(history); // span: llm.gpt-4o if (response.stopReason === "end_turn") { return response.text; } for (const toolCall of response.toolCalls) { const result = await executeTool(toolCall.name, toolCall.args); // span: tool.dispatch history.push({ role: "tool", name: toolCall.name, content: String(result) }); } } }); const { result, runId } = await wrapped({ userMessage: "What is the weather in London?" }); ``` -------------------------------- ### GET /incidents/{incident_id} Source: https://docs.uselemma.ai/api-reference/incidents/get-incident-details Retrieves the details of a specific incident, including its timeline, associated monitors or metrics, and metadata. ```APIDOC ## GET /incidents/{incident_id} ### Description Loads a single incident with timeline context, linked metric or monitor references, and metadata. ### Method GET ### Endpoint /incidents/{incident_id} ### Parameters #### Path Parameters - **incident_id** (string) - Required - Incident identifier. ### Responses #### Success Response (200) - **id** (string) - Incident identifier. - **monitor_id** (string) - Identifier for the associated monitor. - **metric_id** (string) - Identifier for the associated metric. - **project_id** (string) - Identifier for the project the incident belongs to. - **status** (string) - Current status of the incident. - **created_at** (string or null) - Timestamp when the incident was created. - **updated_at** (string or null) - Timestamp when the incident was last updated. - **rca** (object or null) - Root Cause Analysis details, if available. - **associated_traces** (array) - A list of associated trace summaries. #### Error Response (422) - **detail** (array) - Contains validation error details. ### Response Example ```json { "id": "a1b2c3d4-e5f6-7890-1234-567890abcdef", "monitor_id": "monitor-123", "metric_id": "metric-456", "project_id": "project-789", "status": "open", "created_at": "2023-10-27T10:00:00Z", "updated_at": "2023-10-27T10:05:00Z", "rca": { "status": "completed", "result": {}, "failing_trace_ids": ["trace-001", "trace-002"] }, "associated_traces": [ { "id": "trace-001", "otel_trace_id": "otel-trace-abc", "project_id": "project-789", "project_name": "My Project", "service_name": "My Service", "created_at": "2023-10-27T09:55:00Z" } ] } ``` ``` -------------------------------- ### Define and Run an Agent in Python Source: https://docs.uselemma.ai/recipes/manual-instrumentation Defines an agent named 'scratch-agent' using Python. It includes LLM and tool wrappers for 'gpt-4o' and 'get-weather' respectively. The agent logic sets user ID, retrieves weather, generates a response, and returns it. Error recording and span lifecycle are handled automatically. ```python import tracer from uselemma_tracing import agent, llm, tool, TraceContext from openai import AsyncOpenAI client = AsyncOpenAI() @llm("gpt-4o") async def generate(prompt: str) -> str: res = await client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": prompt}], ) return res.choices[0].message.content or "" @tool("get-weather") async def get_weather_tool(city: str) -> dict: return await get_weather(city) async def agent_logic(input_data: dict, ctx: TraceContext) -> str: ctx.span.set_attribute("lemma.user_id", input_data["user_id"]) weather = await get_weather_tool(input_data["city"]) # span: tool.get-weather answer = await generate(input_data["message"]) # span: llm.gpt-4o return answer run_agent = agent("scratch-agent", agent_logic) ``` -------------------------------- ### GET /incidents Source: https://docs.uselemma.ai/api-reference/incidents/list-incidents Retrieves a list of monitoring incidents for a specific project, with options to filter by metric, monitor, or status. ```APIDOC ## GET /incidents ### Description Returns incidents for a project with optional filters by metric, monitor, or status. ### Method GET ### Endpoint /incidents ### Parameters #### Query Parameters - **project_id** (string/uuid) - Required - Project whose incidents to list. - **metric_id** (string/uuid) - Optional - If set, only incidents for this metric. - **monitor_id** (string/uuid) - Optional - If set, only incidents raised by this monitor. - **status** (string) - Optional - If set, filter by incident status (e.g. open or resolved). - **limit** (integer) - Optional - Page size (default: 50, max: 200). - **offset** (integer) - Optional - Pagination offset (default: 0). ### Response #### Success Response (200) - **incidents** (array) - List of incident objects. - **total** (integer) - Total count of incidents. - **limit** (integer) - Limit applied to the request. - **offset** (integer) - Offset applied to the request. - **has_more** (boolean) - Indicates if there are more results available. ``` -------------------------------- ### Add Lemma to a Fresh Provider (TypeScript) Source: https://docs.uselemma.ai/tracing/patterns/dual-export Manually build the TracerProvider and add both Lemma's span processor and a standard batch span processor. Ensure the OTLP exporter is configured with the correct URL. ```typescript import { NodeTracerProvider, } from "@opentelemetry/sdk-trace-node"; import { BatchSpanProcessor, } from "@opentelemetry/sdk-trace-base"; import { OTLPTraceExporter, } from "@opentelemetry/exporter-trace-otlp-proto"; import { createLemmaSpanProcessor } from "@uselemma/tracing"; const provider = new NodeTracerProvider({ spanProcessors: [ createLemmaSpanProcessor(), new BatchSpanProcessor( new OTLPTraceExporter({ url: "https://your-collector/v1/traces" }) ), ], }); provider.register(); ``` -------------------------------- ### GET /traces/{trace_id} Source: https://docs.uselemma.ai/api-reference/traces/get-trace-details Retrieves the full details of a specific trace, including spans, attributes, and metadata for debugging and analysis. ```APIDOC ## GET /traces/{trace_id} ### Description Loads a single trace with spans, attributes, and related metadata for debugging or analysis. ### Method GET ### Endpoint /traces/{trace_id} ### Parameters #### Path Parameters - **trace_id** (string, uuid) - Required - Trace identifier. ### Response #### Success Response (200) - **id** (string) - Trace ID - **otel_trace_id** (string|null) - OpenTelemetry trace ID - **project_id** (string) - Project ID - **project_name** (string) - Project name - **service_name** (string|null) - Service name - **created_at** (string|null) - Creation timestamp - **spans** (array) - List of span details - **stats** (object) - Trace statistics #### Response Example { "id": "uuid", "project_id": "uuid", "project_name": "example-project", "spans": [], "stats": {} } ``` -------------------------------- ### GET /metrics/{metric_id}/monitor Source: https://docs.uselemma.ai/api-reference/metrics/get-monitor Retrieves the monitor configuration and current state linked to a specific metric, if one exists. ```APIDOC ## GET /metrics/{metric_id}/monitor ### Description Returns the monitor configuration and current state linked to this metric, if one exists. ### Method GET ### Endpoint /metrics/{metric_id}/monitor ### Parameters #### Path Parameters - **metric_id** (string/uuid) - Required - Metric whose monitor to load. ### Response #### Success Response (200) - **project_id** (string) - Project Id - **project_name** (string) - Project Name - **metric_id** (string) - Metric Id - **metric_name** (string) - Metric Name - **metric_description** (string|null) - Metric Description - **threshold** (number) - Threshold - **aggregation_method** (string) - Aggregation Method - **comparison_operator** (string) - Comparison Operator - **interval_value** (integer) - Interval Value - **interval_unit** (string) - Interval Unit - **created_at** (string|null) - Created At ``` -------------------------------- ### Debug Mode Output Example Source: https://docs.uselemma.ai/tracing/troubleshooting/debug-mode Sample log output generated when debug mode is active, showing trace-wrapper and processor events. ```text [LEMMA:trace-wrapper] span started { agentName: 'my-agent', runId: 'abc-123' } [LEMMA:processor] onStart: top-level run span { spanId: '...', runId: 'abc-123' } [LEMMA:processor] onStart: child span { spanName: 'gen_ai.chat', spanId: '...', runId: 'abc-123' } [LEMMA:processor] onEnd: span ended { spanName: 'gen_ai.chat', spanId: '...', runId: 'abc-123', isTopLevel: false, skipped: false } [LEMMA:trace-wrapper] span ended after fn returned { runId: 'abc-123' } [LEMMA:processor] onEnd: span ended { spanName: 'ai.agent.run', spanId: '...', runId: 'abc-123', isTopLevel: true, skipped: false } [LEMMA:processor] exporting batch { runId: 'abc-123', spanCount: 2, force: false } ``` -------------------------------- ### Register OpenTelemetry and instrument OpenAI SDK Source: https://docs.uselemma.ai/integrations/openai-sdk-python Set up the OpenTelemetry transport and patch the OpenAI package to enable tracing for API calls. This should be done at application startup. ```python from uselemma_tracing import register_otel, instrument_openai register_otel() instrument_openai() # patches the openai package ``` -------------------------------- ### Set Up Tracer Provider in Python Source: https://docs.uselemma.ai/integrations/azure-application-insights Configure the TracerProvider in Python to add span processors for exporting traces to Azure Application Insights and Lemma. This enables LLM observability. ```python import os from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from azure.monitor.opentelemetry.exporter import AzureMonitorTraceExporter from opentelemetry import trace from uselemma_tracing import create_lemma_span_processor, instrument_openai provider = TracerProvider() # Export to Azure Application Insights provider.add_span_processor( BatchSpanProcessor( AzureMonitorTraceExporter( connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"] ) ) ) # Export to Lemma provider.add_span_processor(create_lemma_span_processor()) trace.set_tracer_provider(provider) instrument_openai() ``` -------------------------------- ### Setting Attributes on the Run Span (Python) Source: https://docs.uselemma.ai/tracing/patterns/custom-attributes Example of setting custom attributes on the agent's run span using Python. ```APIDOC ## Setting attributes on the run span Use `ctx.span.setAttribute()` / `ctx.span.set_attribute()` to set attributes at any point inside the wrapped function: ### Method ```python async def run_agent(input_data: dict, ctx: TraceContext) -> str: ctx.span.set_attribute("lemma.user_id", input_data["user_id"]) ctx.span.set_attribute("lemma.session_id", input_data["session_id"]) ctx.span.set_attribute("lemma.environment", "production") ctx.span.set_attribute("lemma.feature", "support_chat") return await handle_request(input_data["message"]) my_agent = agent("support-agent", run_agent) ``` ```