### Installation with Bundler Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Add the gem to your application's Gemfile and run bundle install. ```ruby gem 'llm_tracer' ``` ```bash bundle install ``` -------------------------------- ### Installation manually Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Install the gem directly using the gem command. ```bash gem install llm_tracer ``` -------------------------------- ### Basic Setup Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Initialize OpenTelemetry and create a simple workflow span. ```ruby require 'llm_tracer' # Initialize OpenTelemetry (basic console exporter for development) OpenTelemetry::SDK.configure # Create a simple workflow span LlmTracer.workflow(name: "my_workflow", version: "1.0.0") do |span| # Your workflow logic here puts "Executing workflow: #{span.name}" end ``` -------------------------------- ### OpenAI Integration Example Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md An example of an OpenAI client that uses LlmTracer to trace LLM calls, including adding response details to the span. ```ruby class OpenAIClient def initialize(api_key) @api_key = api_key end def chat_completion(messages, model: "gpt-4", temperature: 0.7) LlmTracer.llm_call( model: model, provider: "openai", prompt: messages.map { |m| "#{m[:role]}: #{m[:content]}" }.join("\n"), temperature: temperature ) do |span| # Make actual API call response = make_api_call(messages, model, temperature) # Add response info to span LlmTracer.tracer.add_llm_response( span, response_model: response[:model], response_provider: "openai", finish_reason: response[:finish_reason], usage: response[:usage] ) response end end end ``` -------------------------------- ### Agent Span Usage Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Example of how to use the Agent Span. ```ruby LlmTracer.agent( name: "agent_name", version: "1.0.0", operation_name: LlmTracer::GenAI::Operations::CREATE_AGENT, system: LlmTracer::GenAI::Systems::OPENAI, conversation_id: "conv_123" ) do |span| # Agent logic end ``` -------------------------------- ### Workflow Span Usage Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Example of how to use the Workflow Span. ```ruby LlmTracer.workflow(name: "workflow_name", version: "1.0.0") do |span| # Workflow logic end ``` -------------------------------- ### Tool Integration Source: https://github.com/chatwoot/llm_tracer/blob/main/IMPLEMENTATION_SUMMARY.md Example showing how to integrate external tools, such as an API call, within a workflow and record its input and output. ```ruby LlmTracer.tool_call(name: "weather_api", input: { city: "SF" }) do |span| weather = weather_api.get_weather("San Francisco") LlmTracer.tracer.add_tool_call_result(span, output: weather) end ``` -------------------------------- ### Simple Workflow Source: https://github.com/chatwoot/llm_tracer/blob/main/IMPLEMENTATION_SUMMARY.md Example of defining a simple workflow with a single span to track a customer support request. ```ruby LlmTracer.workflow(name: "customer_support", version: "1.0.0") do |span| # Handle customer support request span.set_attribute("customer.id", "12345") end ``` -------------------------------- ### Anthropic Integration Example Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md An example of an Anthropic client that uses LlmTracer to trace LLM calls, including adding response details to the span. ```ruby class AnthropicClient def messages(prompt, model: "claude-3-sonnet") LlmTracer.llm_call( model: model, provider: "anthropic", prompt: prompt ) do |span| # Make actual API call response = make_api_call(prompt, model) # Add response info to span LlmTracer.tracer.add_llm_response( span, response_model: response[:model], response_provider: "anthropic", finish_reason: response[:stop_reason], usage: response[:usage] ) response end end end ``` -------------------------------- ### Tool Call Span Usage Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Example of how to use the Tool Call Span. ```ruby LlmTracer.tool_call( name: "tool_name", operation_name: LlmTracer::GenAI::Operations::EXECUTE_TOOL, system: LlmTracer::GenAI::Systems::OPENAI, input: { param1: "value1" }, conversation_id: "conv_123" ) do |span| # Tool execution logic end ``` -------------------------------- ### Complex Multi-Agent Workflow Source: https://github.com/chatwoot/llm_tracer/blob/main/IMPLEMENTATION_SUMMARY.md Example demonstrating a complex workflow involving multiple agents and LLM calls, showcasing nested spans for research, writing, and LLM interaction. ```ruby LlmTracer.workflow(name: "content_generation", version: "2.0.0") do |workflow_span| LlmTracer.agent(name: "researcher", version: "1.0.0") do |researcher_span| # Research phase data = research_topic("AI trends") end LlmTracer.agent(name: "writer", version: "1.5.0") do |writer_span| LlmTracer.llm_call( model: "gpt-4", provider: "openai", prompt: "Write about: #{data}" ) do |llm_span| content = generate_content(prompt) # Add response info... end end end ``` -------------------------------- ### Error Handling Example Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Shows how to add error details to a span when an exception occurs during an LLM call. ```ruby LlmTracer.llm_call(model: "gpt-4", provider: "openai") do |span| begin response = make_llm_call() # Process response rescue => e span.set_attribute("error", true) span.set_attribute("error.message", e.message) span.set_attribute("error.type", e.class.name) raise end end ``` -------------------------------- ### LLM Call Span Usage Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Example of how to use the LLM Call Span. ```ruby LlmTracer.llm_call( model: "gpt-4", provider: "OpenAI", operation_name: LlmTracer::GenAI::Operations::CHAT, system: LlmTracer::GenAI::Systems::OPENAI, prompt: "Your prompt here", temperature: 0.7, max_tokens: 1000, conversation_id: "conv_123", output_type: LlmTracer::GenAI::OutputTypes::TEXT ) do |span| # LLM call logic end ``` -------------------------------- ### Custom Attributes Example Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Demonstrates how to add custom attributes to a span within a workflow. ```ruby LlmTracer.workflow(name: "custom_workflow") do |span| span.set_attribute("business.customer_id", "12345") span.set_attribute("business.priority", "high") span.set_attribute("custom.metric", 42.5) end ``` -------------------------------- ### Nested Spans Example Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Illustrates the creation of nested spans, showing how spans automatically inherit parent context to form a trace hierarchy. ```ruby LlmTracer.workflow(name: "parent_workflow") do |workflow_span| # This span is a child of the workflow span LlmTracer.agent(name: "child_agent") do |agent_span| # This span is a child of the agent span LlmTracer.llm_call(model: "gpt-4", provider: "openai") do |llm_span| # This span is a child of the agent span end end end ``` -------------------------------- ### Simple API Usage Source: https://github.com/chatwoot/llm_tracer/blob/main/IMPLEMENTATION_SUMMARY.md Demonstrates the basic usage of the LlmTracer library for tracing workflows, agents, LLM calls, and tool calls. ```ruby LlmTracer.workflow(name: "workflow_name", version: "1.0.0") do |span| # Workflow logic end LlmTracer.agent(name: "agent_name", version: "1.0.0") do |span| # Agent logic end LlmTracer.llm_call( model: "gpt-4", provider: "openai", prompt: "Your prompt", temperature: 0.7 ) do |span| # LLM call logic end LlmTracer.tool_call(name: "tool_name", input: { param: "value" }) do |span| # Tool execution end ``` -------------------------------- ### Basic Configuration Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Configures OpenTelemetry SDK with a simple span processor that exports spans to the console. ```ruby require 'opentelemetry/sdk' OpenTelemetry::SDK.configure do |c| c.add_span_processor( OpenTelemetry::SDK::Trace::Export::SimpleSpanProcessor.new( OpenTelemetry::SDK::Trace::Export::ConsoleSpanExporter.new ) ) end ``` -------------------------------- ### Standardized System Values Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Predefined constants for all major GenAI systems. ```ruby LlmTracer::GenAI::Systems::OPENAI # OpenAI LlmTracer::GenAI::Systems::ANTHROPIC # Anthropic LlmTracer::GenAI::Systems::GCP_GEMINI # Google Gemini LlmTracer::GenAI::Systems::AWS_BEDROCK # AWS Bedrock LlmTracer::GenAI::Systems::AZURE_AI_OPENAI # Azure OpenAI LlmTracer::GenAI::Systems::COHERE # Cohere LlmTracer::GenAI::Systems::GROQ # Groq LlmTracer::GenAI::Systems::MISTRAL_AI # Mistral AI # ... and many more ``` -------------------------------- ### Custom Resource Attributes Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Configures OpenTelemetry SDK with custom resource attributes for service name, version, and environment. ```ruby OpenTelemetry::SDK.configure do |c| c.resource = OpenTelemetry::SDK::Resources::Resource.create( OpenTelemetry::SemanticConventions::Resource::SERVICE_NAME => "my_llm_app", OpenTelemetry::SemanticConventions::Resource::SERVICE_VERSION => "1.0.0", OpenTelemetry::SemanticConventions::Resource::DEPLOYMENT_ENVIRONMENT => "production" ) # Add span processors... end ``` -------------------------------- ### OTLP Exporter Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Configures OpenTelemetry SDK with a batch span processor using the OTLP exporter, suitable for backends like Jaeger, Zipkin, etc. ```ruby require 'opentelemetry/exporter/otlp' OpenTelemetry::SDK.configure do |c| c.add_span_processor( OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new( OpenTelemetry::Exporter::OTLP::Exporter.new( endpoint: "http://localhost:4317" ) ) ) end ``` -------------------------------- ### Agent Operations with GenAI Standards Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Create and invoke an agent using standardized attributes. ```ruby # Create an agent using standardized attributes LlmTracer.agent( name: "Customer Support Agent", version: "1.0.0", operation_name: LlmTracer::GenAI::Operations::CREATE_AGENT, system: LlmTracer::GenAI::Systems::OPENAI, conversation_id: "conv_123" ) do |agent_span| # Agent creation logic # Invoke the agent LlmTracer.agent( name: "Customer Support Agent", version: "1.0.0", operation_name: LlmTracer::GenAI::Operations::INVOKE_AGENT, system: LlmTracer::GenAI::Systems::OPENAI, conversation_id: "conv_123" ) do |invoke_span| # Agent invocation logic end end ``` -------------------------------- ### Standardized Output Types Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Constants for expected output types. ```ruby LlmTracer::GenAI::OutputTypes::TEXT # Plain text LlmTracer::GenAI::OutputTypes::IMAGE # Image LlmTracer::GenAI::OutputTypes::JSON # JSON object LlmTracer::GenAI::OutputTypes::SPEECH # Speech ``` -------------------------------- ### Tool Call Tracing with GenAI Standards Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Trace a tool or function call using standardized attributes and add the result to the span. ```ruby # Trace a tool or function call using standardized attributes LlmTracer.tool_call( name: "weather_api", operation_name: LlmTracer::GenAI::Operations::EXECUTE_TOOL, system: LlmTracer::GenAI::Systems::OPENAI, input: { city: "San Francisco", country: "US" }, conversation_id: "conv_123" ) do |span| # Execute your tool here result = weather_api.get_weather(city, country) # Add the result to the span LlmTracer.tracer.add_tool_call_result(span, output: result) end ``` -------------------------------- ### Standardized Operation Names Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Constants for standardized GenAI operations. ```ruby LlmTracer::GenAI::Operations::CHAT # Chat completion LlmTracer::GenAI::Operations::CREATE_AGENT # Create GenAI agent LlmTracer::GenAI::Operations::EMBEDDINGS # Embeddings generation LlmTracer::GenAI::Operations::EXECUTE_TOOL # Execute a tool LlmTracer::GenAI::Operations::GENERATE_CONTENT # Multimodal content generation LlmTracer::GenAI::Operations::INVOKE_AGENT # Invoke GenAI agent LlmTracer::GenAI::Operations::TEXT_COMPLETION # Text completion ``` -------------------------------- ### Standardized Attributes Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md All attributes follow the official OpenTelemetry GenAI specification. ```ruby # Common attributes "gen_ai.operation.name" # Operation being performed "gen_ai.system" # GenAI system being used "gen_ai.conversation.id" # Conversation identifier "gen_ai.data_source.id" # Data source identifier "gen_ai.output.type" # Expected output type # Request attributes "gen_ai.request.model" # Model being used "gen_ai.request.provider" # Provider of the model # Response attributes "gen_ai.response.model" # Response model "gen_ai.response.provider" # Response provider # Error attributes "error.type" # Error type "error.message" # Error message ``` -------------------------------- ### LLM Call Tracing with GenAI Standards Source: https://github.com/chatwoot/llm_tracer/blob/main/README.md Trace an LLM API call using standardized attributes and add response information. ```ruby # Trace an LLM API call using standardized attributes LlmTracer.llm_call( model: "gpt-4", provider: "OpenAI", operation_name: LlmTracer::GenAI::Operations::CHAT, system: LlmTracer::GenAI::Systems::OPENAI, prompt: "Generate a creative story", temperature: 0.8, max_tokens: 500, conversation_id: "conv_123", output_type: LlmTracer::GenAI::OutputTypes::TEXT ) do |span| # Make your LLM API call here response = make_llm_call(prompt) # Add response information to the span LlmTracer.tracer.add_llm_response( span, response_model: "gpt-4", response_provider: "OpenAI", finish_reason: "stop", usage: { total_tokens: 150, prompt_tokens: 25, completion_tokens: 125 } ) end ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.