### Complete LLM Builder Configuration Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/builder.md

Demonstrates a comprehensive setup of the LLMBuilder, including backend selection, API key, model, generation parameters, and system prompt. This example shows how to build a configured LLM instance.

```rust
use llm::builder::LLMBuilder;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key("sk-your-key")
        .model("gpt-4")
        .max_tokens(1000)
        .temperature(0.7)
        .system("You are an expert assistant")
        .timeout_seconds(30)
        .build()?;

    Ok(())
}
```

--------------------------------

### Run Example

Source: https://github.com/graniet/llm/blob/main/CONTRIBUTING.md

Execute a specific example from the examples/ directory. Replace 'mistral_example' with the desired example name.

```sh
cargo run --example mistral_example
```

--------------------------------

### Basic API (OpenAI Standard Format) Example

Source: https://github.com/graniet/llm/blob/main/README.md

Demonstrates a basic API example using the OpenAI standard format. This example works with multiple providers including OpenAI, Anthropic, DeepSeek, and Groq.

```rust
use llm::models::api::Api;

#[tokio::main]
async fn main() {
    let api = Api::new("openai"); // Specify provider, e.g., "openai", "anthropic", "deepseek", "groq"
    let prompt = "What is the weather like today?";
    let response = api.chat_completion(prompt).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Basic Tool Calling Example with OpenAI

Source: https://github.com/graniet/llm/blob/main/README.md

Basic tool calling example with OpenAI. This demonstrates how to enable LLMs to use external tools or functions.

```rust
use llm::models::openai::OpenAI;

#[tokio::main]
async fn main() {
    let openai = OpenAI::new();
    let prompt = "What is the current time?";
    let tools = vec![llm::models::openai::Tool {
        type_: "function".to_string(),
        function: llm::models::openai::Function {
            name: "get_current_time".to_string(),
            description: "Gets the current time.".to_string(),
            parameters: "null".to_string(),
        },
    }];
    let response = openai.tool_calling(prompt, &tools).await.unwrap();
    println!("{:?}", response);
}
```

--------------------------------

### Multi-Backend Chaining Example

Source: https://github.com/graniet/llm/blob/main/README.md

Shows how to create multi-step prompt chains for exploring programming language features. This example demonstrates combining different LLM providers for advanced tasks.

```rust
use llm::chain::Chain;
use llm::models::deepseek::DeepSeek;
use llm::models::phind::Phind;

#[tokio::main]
async fn main() {
    let deepseek = DeepSeek::new();
    let phind = Phind::new();

    let mut chain = Chain::new();
    chain.add_step(deepseek.into());
    chain.add_step(phind.into());

    let prompt = "Explain the concept of closures in Rust.";
    let response = chain.run(prompt).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Complete Usage Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/completion.md

Demonstrates how to initialize the LLM client and perform both simple and parameterized text completions.

```rust
use llm::builder::LLMBuilder;
use llm::completion::CompletionRequest;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("gpt-3.5-turbo-instruct")
        .build()?;

    // Simple completion
    let req = CompletionRequest::new("The capital of France is");
    let response = llm.complete(&req).await?;
    println!("Response: {}", response);

    // Completion with parameters
    let req = CompletionRequest::builder("Write a haiku about nature:")
        .max_tokens(100)
        .temperature(0.8)
        .build();
    let response = llm.complete(&req).await?;
    println!("Haiku: {}", response.text().unwrap_or_default());

    Ok(())
}
```

--------------------------------

### REST API Client Example (curl)

Source: https://github.com/graniet/llm/blob/main/_autodocs/OVERVIEW.md

Example of how to call the REST API server using curl for chat completions.

```bash
curl -X POST http://localhost:3000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-..." \
  -d '{"model": "openai:gpt-4", "messages": [...]}'
```

--------------------------------

### FunctionBuilder::build Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Demonstrates the final step of building a Tool definition from a FunctionBuilder.

```rust
pub fn build(self) -> Tool
```

--------------------------------

### FunctionBuilder::new Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Demonstrates how to create a new FunctionBuilder with a specified name for a tool.

```rust
use llm::builder::FunctionBuilder;

let builder = FunctionBuilder::new("get_weather");
```

--------------------------------

### Ollama Local LLM Example

Source: https://github.com/graniet/llm/blob/main/README.md

Example of using local LLMs through Ollama integration. This allows you to run models on your own hardware.

```rust
use llm::models::ollama::Ollama;

fn main() {
    let ollama = Ollama::new("llama3"); // Specify your local model name
    let prompt = "Tell me a joke.";
    let response = ollama.chat_completion(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Basic Evaluation Example

Source: https://github.com/graniet/llm/blob/main/README.md

Basic evaluation example with Anthropic, Phind, and DeepSeek. This allows for comparing the outputs of different LLM providers on the same prompt.

```rust
use llm::evaluation::Evaluation;

#[tokio::main]
async fn main() {
    let mut evaluator = Evaluation::new();
    evaluator.add_provider("anthropic");
    evaluator.add_provider("phind");
    evaluator.add_provider("deepseek");

    let prompt = "Write a short summary of the benefits of renewable energy.";
    let results = evaluator.evaluate(prompt).await.unwrap();
    for result in results {
        println!("Provider: {}, Score: {}, Response: {}", result.provider, result.score, result.response);
    }
}
```

--------------------------------

### Example Authorization Header

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Demonstrates the format for the Authorization header required when an authentication key is set on the server.

```bash
curl -H "Authorization: Bearer sk-my-secret-key" ...
```

--------------------------------

### Complete Tool Calling Cycle Example

Source: https://github.com/graniet/llm/blob/main/README.md

Complete tool calling cycle with JSON schema validation and structured responses. This example covers the entire process from request to validated output.

```rust
use llm::models::google::Google;

#[tokio::main]
async fn main() {
    let google = Google::new();
    let prompt = "Book a flight from London to Paris for tomorrow.";
    let schema = r#"{
        "type": "object",
        "properties": {
            "origin": {"type": "string"},
            "destination": {"type": "string"},
            "date": {"type": "string"}
        }
    }"#;
    let response = google.tool_json_schema_cycle(prompt, schema).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### FunctionBuilder::description Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Shows how to set a description for a function using the FunctionBuilder.

```rust
let builder = FunctionBuilder::new("get_weather")
    .description("Get the current weather for a location");
```

--------------------------------

### DeepSeek Chat Completion Example

Source: https://github.com/graniet/llm/blob/main/README.md

A basic chat completion example using DeepSeek's chat models. Ensure you have the necessary API keys or local setup for DeepSeek.

```rust
use llm::models::deepseek::DeepSeek;

fn main() {
    let deepseek = DeepSeek::new();
    let prompt = "Explain the difference between AI and ML.";
    let response = deepseek.chat_completion(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Simple Chat Workflow

Source: https://github.com/graniet/llm/blob/main/_autodocs/OVERVIEW.md

A basic example of initializing the LLM and performing a single chat turn.

```rust
let llm = LLMBuilder::new()
    .backend(LLMBackend::OpenAI)
    .api_key(std::env::var("OPENAI_API_KEY")?)
    .model("gpt-4")
    .build()?;

let response = llm.chat(&[
    ChatMessage::user().content("What is Rust?").build(),
]).await?;

println!("{}", response.text().unwrap_or_default());
```

--------------------------------

### Complete Function Tool Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Demonstrates building a function with parameters using ParamBuilder and using it in a chat request with an LLM.

```rust
use llm::builder::{FunctionBuilder, ParamBuilder, LLMBuilder};
use llm::chat::ChatMessage;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key("sk-your-key")
        .model("gpt-4")
        .build()?;

    // Define a tool
    let weather_tool = FunctionBuilder::new("get_weather")
        .description("Get the current weather for a location")
        .param(
            ParamBuilder::new("location", "string")
                .description("City name")
                .required(true)
                .build()
        )
        .param(
            ParamBuilder::new("unit", "string")
                .description("Temperature unit")
                .enum_values(vec!["celsius".to_string(), "fahrenheit".to_string()])
                .build()
        )
        .build();

    // Use the tool in a chat request
    let messages = vec![
        ChatMessage::user()
            .content("What's the weather in Paris?")
            .build(),
    ];

    let response = llm.chat_with_tools(&messages, Some(&[weather_tool])).await?;

    if let Some(tools) = response.tool_calls() {
        for tool in tools {
            println!("Called: {}", tool.function.name);
            println!("Arguments: {}", tool.function.arguments);
        }
    }

    Ok(())
}
```

--------------------------------

### Unified Tool Calling Example

Source: https://github.com/graniet/llm/blob/main/README.md

Unified tool calling with selectable provider. Demonstrates multi-turn tool use and tool choice across different LLM providers.

```rust
use llm::models::unified::Unified;

#[tokio::main]
async fn main() {
    let mut unified = Unified::new();
    unified.set_provider("openai"); // or "google", "anthropic", etc.

    let prompt = "Find the weather in New York and then tell me a joke.";
    let response = unified.tool_calling(prompt).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Complete Rust Server Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Sets up an LLM server with multiple backends (OpenAI, Anthropic, Ollama) and exposes an API endpoint. Ensure you have the necessary API keys and local Ollama server running.

```rust
use llm::builder::LLMBuilder;
use llm::chain::{LLMRegistry, LLMRegistryBuilder};
use llm::api::Server;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an LLM registry with multiple backends
    let registry = LLMRegistryBuilder::new()
        .add(
            "openai:gpt-4",
            LLMBuilder::new()
                .backend(llm::builder::LLMBackend::OpenAI)
                .api_key(std::env::var("OPENAI_API_KEY")?)
                .model("gpt-4")
                .build()? 
        )
        .add(
            "anthropic:claude-3",
            LLMBuilder::new()
                .backend(llm::builder::LLMBackend::Anthropic)
                .api_key(std::env::var("ANTHROPIC_API_KEY")?)
                .model("claude-3-sonnet")
                .build()? 
        )
        .add(
            "ollama:mistral",
            LLMBuilder::new()
                .backend(llm::builder::LLMBackend::Ollama)
                .base_url("http://localhost:11434")
                .model("mistral")
                .build()? 
        )
        .build();

    // Create and configure server
    let server = Server::new(registry)
        .with_auth_key("sk-my-api-key");

    // Start server
    println!("Server running on http://127.0.0.1:3000");
    server.run("127.0.0.1:3000").await?;

    Ok(())
}
```

--------------------------------

### X.AI (Grok) Chat Completion Example

Source: https://github.com/graniet/llm/blob/main/README.md

Basic chat completion example using xAI's Grok models. This allows interaction with the Grok language model.

```rust
use llm::models::xai::Xai;

fn main() {
    let xai = Xai::new();
    let prompt = "What are the latest developments in space exploration?";
    let response = xai.chat_completion(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### OpenAI Streaming Chat Example

Source: https://github.com/graniet/llm/blob/main/README.md

Example of OpenAI streaming chat, demonstrating real-time token generation. This is useful for interactive applications where immediate feedback is desired.

```rust
use llm::models::openai::OpenAI;

#[tokio::main]
async fn main() {
    let openai = OpenAI::new();
    let prompt = "Explain the concept of recursion.";
    let mut stream = openai.streaming_chat_completion(prompt).await.unwrap();
    while let Some(token) = stream.next().await {
        print!("{}", token);
    }
    println!();
}
```

--------------------------------

### Deepclaude Pipeline Example

Source: https://github.com/graniet/llm/blob/main/README.md

Basic deepclaude pipeline example with DeepSeek and Claude. This demonstrates a specialized pipeline combining multiple models.

```rust
use llm::pipeline::DeepclaudePipeline;

#[tokio::main]
async fn main() {
    let mut pipeline = DeepclaudePipeline::new();
    pipeline.set_deepseek_model("deepseek-coder");
    pipeline.set_claude_model("claude-3-opus");

    let prompt = "Write a Python script to scrape a website.";
    let response = pipeline.run(prompt).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Complete Multi-Step Chain Example in Rust

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/chains.md

Demonstrates building and executing a multi-step chain for extraction, summarization, and categorization using the LLM library. Ensure the OPENAI_API_KEY environment variable is set.

```rust
use llm::builder::LLMBuilder;
use llm::chain::{ChainStepBuilder, ChainStepMode, PromptChain};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("gpt-4")
        .build()?;

    // Create a multi-step chain: extract -> summarize -> categorize
    let chain = PromptChain::new(&llm)
        // Step 1: Extract key information
        .step(
            ChainStepBuilder::new(
                "extract",
                "Extract the main topics from: {{input}}",
                ChainStepMode::Chat,
            )
            .max_tokens(300)
            .build(),
        )
        // Step 2: Summarize the extracted topics
        .step(
            ChainStepBuilder::new(
                "summarize",
                "Create a brief summary of: {{extract}}",
                ChainStepMode::Chat,
            )
            .max_tokens(200)
            .temperature(0.5)
            .build(),
        )
        // Step 3: Categorize the summary
        .step(
            ChainStepBuilder::new(
                "categorize",
                "Categorize the following into one of: Technology, Science, Business, Other\n{{summarize}}",
                ChainStepMode::Chat,
            )
            .max_tokens(50)
            .temperature(0.0)
            .build(),
        );

    // Execute the chain
    let results = chain.run().await?;

    println!("Extracted Topics: {}", results.get("extract").unwrap_or(&"N/A".to_string()));
    println!("Summary: {}", results.get("summarize").unwrap_or(&"N/A".to_string()));
    println!("Category: {}", results.get("categorize").unwrap_or(&"N/A".to_string()));

    Ok(())
}

```

--------------------------------

### Basic Validator Example with Anthropic

Source: https://github.com/graniet/llm/blob/main/README.md

Basic validator example with Anthropic's Claude model. This demonstrates input validation before sending to the LLM.

```rust
use llm::models::anthropic::Anthropic;

fn main() {
    let anthropic = Anthropic::new();
    let prompt = "Validate this email address: test@example.com";
    // Assuming a validator function is integrated within the Anthropic model call
    let response = anthropic.validator_example(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### X.AI Streaming Chat Example

Source: https://github.com/graniet/llm/blob/main/README.md

Example of X.AI streaming chat, demonstrating real-time token generation. This is useful for applications that need to display responses as they are generated.

```rust
use llm::models::xai::Xai;

#[tokio::main]
async fn main() {
    let xai = Xai::new();
    let prompt = "Summarize the current economic trends.";
    let mut stream = xai.streaming_chat_completion(prompt).await.unwrap();
    while let Some(token) = stream.next().await {
        print!("{}", token);
    }
    println!();
}
```

--------------------------------

### Server::run

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Starts the server and listens for incoming HTTP requests on the specified network address. This method runs indefinitely on success.

```APIDOC
## Server::run

### Description
Starts the server and listens for requests on the specified address. This method runs indefinitely on success.

### Signature
```rust
pub async fn run(self, addr: &str) -> Result<(), LLMError>
```

### Parameters
#### Path Parameters
- **addr** (string) - Required - Address to bind to (e.g., "127.0.0.1:3000")

### Returns
- `Result<(), LLMError>` - Never returns on success (server runs indefinitely).

### Request Example
```rust
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let server = Server::new(registry);
    server.run("127.0.0.1:3000").await?;
    Ok(())
}
```
```

--------------------------------

### Google Gemini Chat Completion Example

Source: https://github.com/graniet/llm/blob/main/README.md

Basic chat completion example using Google Gemini models. This demonstrates interaction with Google's powerful language models.

```rust
use llm::models::google::Google;

fn main() {
    let google = Google::new();
    let prompt = "Explain quantum computing in simple terms.";
    let response = google.chat_completion(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### REST API Server Setup

Source: https://github.com/graniet/llm/blob/main/_autodocs/OVERVIEW.md

Expose LLM capabilities via a REST API server compatible with OpenAI endpoints.

```rust
let registry = LLMRegistryBuilder::new()
    .add("openai:gpt-4", openai_llm)
    .add("anthropic:claude", anthropic_llm)
    .build();

let server = Server::new(registry)
    .with_auth_key("sk-...")
    .run("127.0.0.1:3000").await?;
```

--------------------------------

### Google Gemini Embedding Example

Source: https://github.com/graniet/llm/blob/main/README.md

Basic embedding example using Google Gemini models. This demonstrates how to generate vector embeddings for text.

```rust
use llm::models::google::Google;

fn main() {
    let google = Google::new();
    let text = "This is a sample text for embedding.";
    let embedding = google.embedding(text).unwrap();
    println!("Embedding: {:?}", embedding.embedding);
}
```

--------------------------------

### Complete ChatWithMemory Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/memory.md

Demonstrates setting up and using ChatWithMemory for a multi-turn conversation. It shows how to initialize the LLM, memory, and the chat wrapper, then performs two conversational turns, verifying memory recall.

```rust
use llm::builder::LLMBuilder;
use llm::memory::{ChatWithMemory, ChatWithMemoryConfig, SlidingWindowMemory};
use llm::chat::ChatMessage;
use std::sync::Arc;
use tokio::sync::RwLock;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("gpt-4")
        .build()?;

    // Create memory with 20-message sliding window
    let memory = SlidingWindowMemory::new(20);
    let memory_arc = Arc::new(RwLock::new(memory));
    let provider_arc = Arc::from(llm);

    // Create ChatWithMemory wrapper
    let config = ChatWithMemoryConfig::new(provider_arc, memory_arc);
    let chat = ChatWithMemory::with_config(config);

    // Multi-turn conversation with automatic memory
    let msg1 = ChatMessage::user().content("My name is Alice").build();
    let resp1 = chat.chat(&[msg1]).await?;
    println!("Response 1: {}", resp1.text().unwrap_or_default());

    let msg2 = ChatMessage::user().content("What is my name?").build();
    let resp2 = chat.chat(&[msg2]).await?;
    println!("Response 2 (should remember): {}", resp2.text().unwrap_or_default());

    Ok(())
}
```

--------------------------------

### Google Gemini Streaming Chat Example

Source: https://github.com/graniet/llm/blob/main/README.md

Example of Google Gemini streaming chat, demonstrating real-time token generation. This is useful for creating responsive user interfaces.

```rust
use llm::models::google::Google;

#[tokio::main]
async fn main() {
    let google = Google::new();
    let prompt = "Write a short story about a robot discovering music.";
    let mut stream = google.streaming_chat_completion(prompt).await.unwrap();
    while let Some(token) = stream.next().await {
        print!("{}", token);
    }
    println!();
}
```

--------------------------------

### OpenAI Chat Completion Example

Source: https://github.com/graniet/llm/blob/main/README.md

Demonstrates how to perform a chat completion using the OpenAI backend. Ensure the OPENAI_API_KEY environment variable is set.

```rust
use llm::
    builder::{LLMBackend, LLMBuilder},
    chat::ChatMessage,
;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("OPENAI_API_KEY").unwrap_or("sk-TESTKEY".into());

    let llm = LLMBuilder::new()
        .backend(LLMBackend::OpenAI)
        .api_key(api_key)
        .model("gpt-4.1-nano")
        .max_tokens(512)
        .temperature(0.7)
        .normalize_response(true)
        .build()
        .expect("Failed to build LLM");

    let messages = vec![
        ChatMessage::user()
            .content("Tell me that you love cats")
            .build(),
        ChatMessage::assistant()
            .content("I am an assistant, I cannot love cats but I can love dogs")
            .build(),
        ChatMessage::user()
            .content("Tell me that you love dogs in 2000 chars")
            .build(),
    ];

    match llm.chat(&messages).await {
        Ok(response) => {
            if let Some(text) = response.text() {
                println!("Response: {text}");
            }
            if let Some(usage) = response.usage() {
                println!("  Prompt tokens: {}", usage.prompt_tokens);
                println!("  Completion tokens: {}", usage.completion_tokens);
            } else {
                println!("No usage information available");
            }
        }
        Err(e) => eprintln!("Chat error: {e}"),
    }
    Ok(())
}
```

--------------------------------

### Complete Multi-Agent Example with OpenAI and Anthropic

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/agents.md

Demonstrates the creation and interaction of two agents: a researcher using OpenAI and a reviewer using Anthropic. Agents are configured with specific roles, LLM backends, API keys, models, and memory.

```rust
use llm::agent::AgentBuilder;
use llm::builder::LLMBuilder;
use llm::memory::{SlidingWindowMemory, MessageCondition};
use llm::chat::ChatMessage;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create researcher agent
    let researcher = AgentBuilder::new()
        .role("researcher")
        .llm(
            LLMBuilder::new()
                .backend(llm::builder::LLMBackend::OpenAI)
                .api_key(std::env::var("OPENAI_API_KEY")?)
                .model("gpt-4")
        )
        .memory(SlidingWindowMemory::new(50))
        .on("user", MessageCondition::Any)
        .max_cycles(2)
        .build()?;

    // Create reviewer agent
    let reviewer = AgentBuilder::new()
        .role("reviewer")
        .llm(
            LLMBuilder::new()
                .backend(llm::builder::LLMBackend::Anthropic)
                .api_key(std::env::var("ANTHROPIC_API_KEY")?)
                .model("claude-3-sonnet")
        )
        .memory(SlidingWindowMemory::new(50))
        .on("researcher", MessageCondition::Any)
        .max_cycles(1)
        .single_reply_per_turn(true)
        .build()?;

    // Use the researcher agent
    let msg = ChatMessage::user()
        .content("Explain quantum computing")
        .build();

    let response = researcher.chat(&[msg]).await?;
    println!("Researcher: {}", response.text().unwrap_or_default());

    Ok(())
}
```

--------------------------------

### Chain Multiple LLM Backends Example

Source: https://github.com/graniet/llm/blob/main/README.md

Illustrates chaining multiple LLM backends (e.g., OpenAI, Anthropic, DeepSeek) together in a single workflow. This allows for complex prompt processing and model specialization.

```rust
use llm::chain::Chain;
use llm::models::openai::OpenAI;
use llm::models::anthropic::Anthropic;

#[tokio::main]
async fn main() {
    let openai = OpenAI::new();
    let anthropic = Anthropic::new();

    let mut chain = Chain::new();
    chain.add_step(openai.into());
    chain.add_step(anthropic.into());

    let prompt = "Translate the following English text to French: 'Hello, how are you?'";
    let response = chain.run(prompt).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Create LLM Instance with OpenAI Backend

Source: https://github.com/graniet/llm/blob/main/_autodocs/OVERVIEW.md

Instantiate an LLM client using the LLMBuilder. This example configures it to use the OpenAI backend, retrieves the API key from the environment, and specifies the model.

```rust
let llm = LLMBuilder::new()
    .backend(LLMBackend::OpenAI)
    .api_key(std::env::var("OPENAI_API_KEY")?)
    .model("gpt-4")
    .build()?;
```

--------------------------------

### Embedding Example with OpenAI API

Source: https://github.com/graniet/llm/blob/main/README.md

Basic embedding example using OpenAI's API. This shows how to generate vector embeddings for text using OpenAI's models.

```rust
use llm::models::openai::OpenAI;

fn main() {
    let openai = OpenAI::new();
    let text = "Another sample text for embedding.";
    let embedding = openai.embedding(text).unwrap();
    println!("Embedding: {:?}", embedding.embedding);
}
```

--------------------------------

### Install LLM CLI Tool

Source: https://github.com/graniet/llm/blob/main/README.md

Install the LLM command-line interface tool using Cargo. This tool allows for easy interaction with different LLM models directly from your terminal.

```shell
cargo install llm
```

--------------------------------

### Phind Chat Completion Example

Source: https://github.com/graniet/llm/blob/main/README.md

Basic chat completion example using the Phind model. This is suitable for tasks requiring high-quality code generation and technical explanations.

```rust
use llm::models::phind::Phind;

fn main() {
    let phind = Phind::new();
    let prompt = "Write a Python function to calculate factorial.";
    let response = phind.chat_completion(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### PromptChain Variable Substitution Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/chains.md

Demonstrates how PromptChain uses variable substitution to pass outputs from one step to another. Supports setting step-specific parameters like max_tokens.

```rust
use llm::chain::{ChainStepBuilder, ChainStepMode, PromptChain};

let chain = PromptChain::new(&llm)
    .step(
        ChainStepBuilder::new("extract", "Extract the main idea: {{input}}", ChainStepMode::Chat)
            .max_tokens(500)
            .build()
    )
    .step(
        ChainStepBuilder::new("expand", "Expand on this idea: {{extract}}", ChainStepMode::Chat)
            .max_tokens(1000)
            .build()
    );

let results = chain.run().await?;
let expanded = results.get("expand").unwrap();
```

--------------------------------

### Complete LLM Example with Custom Resilience

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/resilience.md

Demonstrates building an LLM client with custom resilience settings and making a chat request with automatic retries on transient failures. The loop handles successful responses or exhausts all retries.

```rust
use llm::builder::LLMBuilder;
use llm::chat::ChatMessage;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create LLM with custom resilience settings
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::Anthropic)
        .api_key(std::env::var("ANTHROPIC_API_KEY")?)
        .model("claude-3-sonnet")
        .build()?;

    // The LLM is now resilient with default retry settings
    // Requests will automatically retry on transient failures

    let messages = vec![
        ChatMessage::user()
            .content("What is the answer to life, the universe, and everything?")
            .build(),
    ];

    loop {
        match llm.chat(&messages).await {
            Ok(response) => {
                println!("Got response: {}", response.text().unwrap_or_default());
                break;
            }
            Err(e) => {
                eprintln!("Request failed: {}", e);
                // If we get here, all retries have been exhausted
                return Err(Box::new(e));
            }
        }
    }

    Ok(())
}
```

--------------------------------

### Google Gemini Function Calling Example

Source: https://github.com/graniet/llm/blob/main/README.md

Google Gemini function calling example with a complex JSON schema for meeting scheduling. This demonstrates advanced structured data generation for specific tasks.

```rust
use llm::models::google::Google;

#[tokio::main]
async fn main() {
    let google = Google::new();
    let prompt = "Schedule a meeting with John Doe tomorrow at 10 AM about project updates.";
    let schema = r#"{
        "type": "object",
        "properties": {
            "attendees": {"type": "array", "items": {"type": "string"}},
            "time": {"type": "string"},
            "subject": {"type": "string"}
        },
        "required": ["attendees", "time", "subject"]
    }"#;
    let response = google.function_calling(prompt, schema).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### FunctionBuilder::param Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Illustrates adding a parameter to a function definition using ParamBuilder within FunctionBuilder.

```rust
use llm::builder::ParamBuilder;

let builder = FunctionBuilder::new("get_weather")
    .param(ParamBuilder::new("location", "string")
        .description("City name"));
```

--------------------------------

### ParamBuilder::new Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Demonstrates how to create a new ParamBuilder for a 'temperature' parameter of type 'number'.

```rust
use llm::builder::ParamBuilder;

let param = ParamBuilder::new("temperature", "number");
```

--------------------------------

### LLM API Usage Examples

Source: https://github.com/graniet/llm/blob/main/_autodocs/README.md

Demonstrates common patterns for interacting with the LLM API, including simple chat, streaming responses, tool calling, prompt chains, memory integration, agent creation, and setting up a REST API server.

```rust
let response = llm.chat(&messages).await?;
```

```rust
let mut stream = llm.chat_stream(&messages).await?;
```

```rust
let response = llm.chat_with_tools(&messages, Some(&tools)).await?;
```

```rust
let results = PromptChain::new(&llm).step(step1).step(step2).run().await?;
```

```rust
let chat = ChatWithMemory::with_config(config);
```

```rust
let agent = AgentBuilder::new().role("name").llm(config).build()?;
```

```rust
let server = Server::new(registry).run("127.0.0.1:3000").await?;
```

--------------------------------

### Create New LLMBuilder

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/builder.md

Instantiate a new LLMBuilder with default settings. This is the starting point for configuring an LLM provider.

```rust
use llm::builder::LLMBuilder;

let builder = LLMBuilder::new();
```

--------------------------------

### Completing Text with CompletionProvider

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/completion.md

Example of sending a completion request and printing the generated text using the CompletionProvider trait.

```rust
use llm::completion::CompletionRequest;

let req = CompletionRequest::new("Describe quantum computing in simple terms:");
let response = llm.complete(&req).await?;
println!("{}", response.text().unwrap_or_default());
```

--------------------------------

### Evaluate Multiple LLM Providers in Parallel

Source: https://github.com/graniet/llm/blob/main/README.md

Evaluate multiple LLM providers in parallel. This example demonstrates how to efficiently compare LLM performance by running evaluations concurrently.

```rust
use llm::evaluation::Evaluation;

#[tokio::main]
async fn main() {
    let mut evaluator = Evaluation::new();
    evaluator.add_provider("openai");
    evaluator.add_provider("google");
    evaluator.add_provider("ollama");

    let prompt = "Explain the concept of blockchain.";
    let results = evaluator.evaluate_parallel(prompt).await.unwrap();
    for result in results {
        println!("Provider: {}, Score: {}, Response: {}", result.provider, result.score, result.response);
    }
}
```

--------------------------------

### Basic OpenAI Chat Completion

Source: https://github.com/graniet/llm/blob/main/README.md

Demonstrates a basic chat completion example using OpenAI's GPT models. Ensure you have configured your OpenAI API key.

```rust
use llm::models::openai::OpenAI;

fn main() {
    let openai = OpenAI::new();
    let prompt = "What is the capital of France?";
    let response = openai.chat_completion(prompt).unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Remember Message Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/memory.md

Demonstrates how to store a chat message in a `SlidingWindowMemory` instance. Ensure the `SlidingWindowMemory` is initialized with a capacity.

```rust
use llm::memory::SlidingWindowMemory;

let mut memory = SlidingWindowMemory::new(10);
let msg = ChatMessage::user().content("Hello").build();
memory.remember(&msg).await?;
```

--------------------------------

### AgentBuilder for Multi-Agent Systems

Source: https://github.com/graniet/llm/blob/main/_autodocs/OVERVIEW.md

Illustrates setting up agents within a multi-agent system using AgentBuilder. This example defines a 'researcher' and a 'reviewer' agent, specifying their roles, LLM configurations, memory, and message triggers.

```rust
let researcher = AgentBuilder::new()
    .role("researcher")
    .llm(researcher_config)
    .memory(memory)
    .on("user", MessageCondition::Any)  // Respond to user messages
    .build()?;

let reviewer = AgentBuilder::new()
    .role("reviewer")
    .llm(reviewer_config)
    .memory(memory)  // Same memory as researcher
    .on("researcher", MessageCondition::Any)  // Respond to researcher
    .build()?
```

--------------------------------

### Project File Organization

Source: https://github.com/graniet/llm/blob/main/_autodocs/README.md

Illustrates the directory structure of the Graniet LLM project, showing the location of the main README, navigation guides, overview, type references, error handling, configuration, and detailed API documentation subdirectories.

```tree
output/
├── README.md                   ← You are here
├── INDEX.md                    ← Navigation guide
├── OVERVIEW.md                 ← Architecture & concepts
├── types.md                    ← Type reference
├── errors.md                   ← Error handling
├── configuration.md            ← Configuration options
└── api-reference/              ← Detailed API docs
    ├── builder.md              (LLMBuilder)
    ├── chat.md                 (Chat interface)
    ├── completion.md           (Text completion)
    ├── tools.md                (Function calling)
    ├── chains.md               (Prompt chains)
    ├── memory.md               (Memory systems)
    ├── agents.md               (Multi-agent)
    ├── embeddings.md           (Embeddings)
    ├── resilience.md           (Retry/backoff)
    ├── validation.md           (Validation)
    ├── evaluator.md            (Evaluation)
    └── rest-api.md             (REST API)
```

--------------------------------

### Basic Retry/Backoff Wrapper Usage

Source: https://github.com/graniet/llm/blob/main/README.md

Simple retry/backoff wrapper usage. This example demonstrates how to implement robust error handling for LLM API calls by retrying on failure with exponential backoff.

```rust
use llm::resilient::Resilient;

#[tokio::main]
async fn main() {
    let resilient = Resilient::new("openai"); // Specify provider
    let prompt = "Generate a creative story.";

    // The Resilient wrapper automatically handles retries and backoff
    let response = resilient.chat_completion(prompt).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Generate Batch Embeddings

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/embeddings.md

This example shows how to generate embeddings for multiple documents simultaneously. It requires the OPENAI_API_KEY to be set.

```rust
use llm::builder::LLMBuilder;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("text-embedding-3-small")
        .build()?;

    let documents = vec![
        "Machine learning is a subset of artificial intelligence".to_string(),
        "Neural networks are inspired by biological neurons".to_string(),
        "Deep learning uses multiple layers of abstraction".to_string(),
    ];

    let embeddings = llm.embed(documents).await?;

    for (i, embedding) in embeddings.iter().enumerate() {
        println!("Document {}: {} dimensions", i, embedding.len());
    }

    Ok(())
}
```

--------------------------------

### Run REST API Server

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Starts the REST API server and begins listening for incoming HTTP requests on the specified network address. This function runs indefinitely on success.

```rust
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let server = Server::new(registry);
    server.run("127.0.0.1:3000").await?;
    Ok(())
}
```

--------------------------------

### Set System Prompt

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/builder.md

Define the system prompt or initial context provided to the LLM. This guides the AI's behavior and persona.

```rust
let builder = LLMBuilder::new()
    .system("You are a helpful assistant");
```

--------------------------------

### Create a new ChainStepBuilder

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/chains.md

Initialize a ChainStepBuilder with a unique ID, a prompt template, and the desired execution mode. This is the starting point for configuring a chain step.

```rust
use llm::chain::{ChainStepBuilder, ChainStepMode};

let step = ChainStepBuilder::new("step1", "Summarize: {{input}}", ChainStepMode::Chat);
```

--------------------------------

### Build Documentation

Source: https://github.com/graniet/llm/blob/main/CONTRIBUTING.md

Generate and open the project's documentation in your web browser.

```sh
cargo doc --open
```

--------------------------------

### Make a Resilient LLM Request

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/resilience.md

This example demonstrates how to make a basic chat request to the LLM API. The client automatically retries up to 3 times on transient errors.

```rust
use llm::builder::LLMBuilder;
use llm::chat::ChatMessage;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(llm::builder::LLMBackend::OpenAI)
        .api_key(std::env::var("OPENAI_API_KEY")?)
        .model("gpt-4")
        .build()?;

    // This request will retry up to 3 times on failure
    let messages = vec![
        ChatMessage::user().content("Hello").build(),
    ];

    match llm.chat(&messages).await {
        Ok(response) => println!("{}", response.text().unwrap_or_default()),
        Err(e) => eprintln!("Failed after retries: {}", e),
    }

    Ok(())
}
```

--------------------------------

### JavaScript Fetch API to LLM API

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Demonstrates making a POST request to the LLM API using the JavaScript 'fetch' API. This example includes setting request headers and stringifying the JSON body.

```javascript
const url = "http://127.0.0.1:3000/v1/chat/completions";
const headers = {
  "Content-Type": "application/json",
  "Authorization": "Bearer sk-my-api-key"
};
const data = {
  model: "openai:gpt-4",
  messages: [
    { role: "user", content: "Tell me about Rust" }
  ],
  temperature: 0.7,
  max_tokens: 500
};

const response = await fetch(url, {
  method: "POST",
  headers,
  body: JSON.stringify(data)
});

const result = await response.json();
console.log(result.choices[0].message.content);
```

--------------------------------

### Anthropic Streaming Chat Example

Source: https://github.com/graniet/llm/blob/main/README.md

An example of Anthropic streaming chat, demonstrating real-time token generation. This is useful for applications requiring live responses.

```rust
use llm::models::anthropic::Anthropic;

#[tokio::main]
async fn main() {
    let anthropic = Anthropic::new();
    let prompt = "Describe the process of photosynthesis.";
    let mut stream = anthropic.streaming_chat_completion(prompt).await.unwrap();
    while let Some(token) = stream.next().await {
        print!("{}", token);
    }
    println!();
}
```

--------------------------------

### Basic Bedrock Backend Usage

Source: https://github.com/graniet/llm/blob/main/src/backends/aws/quickstart.md

Initialize the Bedrock backend, create a completion request, and print the response text. Ensure AWS credentials are configured.

```rust
use fork_llm::backends::bedrock::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize backend
    let backend = BedrockBackend::new().await?;
    
    // Ask a question
    let request = CompletionRequest::new("What is Rust?")
        .with_model(BedrockModel::ClaudeHaiku3)
        .with_max_tokens(100);
    
    let response = backend.complete(request).await?;
    println!("{}", response.text);
    
    Ok(())
}
```

--------------------------------

### PromptChain::new Method

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/chains.md

Creates a new PromptChain instance. Requires an LLM provider.

```rust
pub fn new(llm: &'a dyn LLMProvider) -> Self
```

```rust
use llm::chain::PromptChain;

let chain = PromptChain::new(&llm_provider);
```

--------------------------------

### Advanced Nested JSON Schemas Example

Source: https://github.com/graniet/llm/blob/main/README.md

Advanced example demonstrating deeply nested JSON schemas with arrays of objects and complex data structures. This is useful for generating intricate data formats.

```rust
use llm::models::google::Google;

#[tokio::main]
async fn main() {
    let google = Google::new();
    let prompt = "Generate a complex user profile with nested addresses and hobbies.";
    let schema = r#"{
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "address": {
                "type": "object",
                "properties": {
                    "street": {"type": "string"},
                    "city": {"type": "string"}
                }
            },
            "hobbies": {"type": "array", "items": {"type": "string"}}
        }
    }"#;
    let response = google.json_schema_nested(prompt, schema).await.unwrap();
    println!("{}", response.content);
}
```

--------------------------------

### Create New Server Instance

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Constructs a new server instance with a given LLM registry. Ensure LLMRegistry is properly initialized before use.

```rust
use llm::chain::LLMRegistry;
use llm::api::Server;

let registry = LLMRegistry::new();
let server = Server::new(registry);
```

--------------------------------

### Get Memory Type

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/memory.md

Returns the specific type of the memory implementation, such as `SlidingWindowMemory` or `ConversationSummaryMemory`.

```rust
fn memory_type(&self) -> MemoryType
```

--------------------------------

### Run Built CLI

Source: https://github.com/graniet/llm/blob/main/CONTRIBUTING.md

Execute the command-line interface binary built in release mode. Use --help for available commands.

```sh
./target/release/llm --help
```

--------------------------------

### ParamBuilder::items Example

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/tools.md

Demonstrates creating a ParamBuilder for an array parameter and specifying its item type.

```rust
use llm::builder::ParamBuilder;

let param = ParamBuilder::new("numbers", "array")
    .items("number");
```

--------------------------------

### Build for Production

Source: https://github.com/graniet/llm/blob/main/CONTRIBUTING.md

Compile the project for production, generating optimized binaries in the target/release/ directory.

```sh
cargo build --release
```

--------------------------------

### Server Constructor

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/rest-api.md

Creates a new server instance with the given LLM registry. The server manages LLM backends and handles HTTP requests.

```APIDOC
## Server::new

### Description
Creates a new server instance with the given LLM registry.

### Signature
```rust
pub fn new(llms: LLMRegistry) -> Self
```

### Parameters
#### Path Parameters
- **llms** (LLMRegistry) - Required - Registry containing LLM backends

### Request Example
```rust
use llm::chain::LLMRegistry;
use llm::api::Server;

let registry = LLMRegistry::new();
let server = Server::new(registry);
```
```

--------------------------------

### Get Memory Event Receiver

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/memory.md

Provides a receiver for `MessageEvent`s, allowing for reactive handling of memory-related events.

```rust
fn get_event_receiver(&self) -> Option<broadcast::Receiver<MessageEvent>>
```

--------------------------------

### Run the Application

Source: https://github.com/graniet/llm/blob/main/src/backends/aws/quickstart.md

Execute the Rust application using Cargo.

```bash
cargo run
```

--------------------------------

### Get Memory Size

Source: https://github.com/graniet/llm/blob/main/_autodocs/api-reference/memory.md

Returns the current number of messages stored in memory. This is a simple count of the messages.

```rust
fn size(&self) -> usize
```