### Rust: Build LLM Client from Multiple Provider Configurations

Source: https://context7.com/kitsunex07/kotoba/llms.txt

This Rust code snippet demonstrates setting up an LLM client using multiple provider configurations (OpenAI, Anthropic, Gemini). It defines `ModelConfig` for each provider, including credentials and default models, then uses `build_client_from_configs` to create a client. The example also shows how to list available provider handles and make a chat request to a specific provider.

```rust
use std::collections::HashMap;
use kotoba_llm::config::{ModelConfig, ProviderKind, Credential, build_client_from_configs};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::types::{ChatRequest, Message, Role, ContentPart, TextContent};
use kotoba_llm::LLMError;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Define multiple provider configurations
    let configs = vec![
        ModelConfig {
            handle: "openai".into(),
            provider: ProviderKind::OpenAiChat,
            credential: Credential::ApiKey {
                header: None,
                key: std::env::var("OPENAI_API_KEY")?
            },
            default_model: Some("gpt-4o-mini".into()),
            base_url: None,
            extra: HashMap::new(),
            patch: None,
        },
        ModelConfig {
            handle: "claude".into(),
            provider: ProviderKind::AnthropicMessages,
            credential: Credential::ApiKey {
                header: None,
                key: std::env::var("ANTHROPIC_API_KEY")?
            },
            default_model: Some("claude-3-5-sonnet-20241022".into()),
            base_url: None,
            extra: HashMap::from([
                ("anthropic-version".into(), serde_json::json!("2023-06-01"))
            ]),
            patch: None,
        },
        ModelConfig {
            handle: "gemini".into(),
            provider: ProviderKind::GoogleGemini,
            credential: Credential::ApiKey {
                header: None,
                key: std::env::var("GEMINI_API_KEY")?
            },
            default_model: Some("gemini-2.0-flash-exp".into()),
            base_url: None,
            extra: HashMap::new(),
            patch: None,
        },
    ];

    // Build client from configurations
    let transport = default_dyn_transport()?;
    let client = build_client_from_configs(&configs, transport)?;

    // List all registered handles
    println!("Available providers: {:?}", client.handles());

    // Use any provider
    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "Hello from configuration!".into(),
            })],
            metadata: None,
        }],
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    let response = client.chat("gemini", request).await?;
    println!("Response model: {:?}", response.model);

    Ok(())
}

```

--------------------------------

### Send Multimodal Requests with Images (Rust)

Source: https://context7.com/kitsunex07/kotoba/llms.txt

This example demonstrates how to construct and send a multimodal chat request that includes both text and an image. It utilizes the `OpenAiChatProvider` and specifies an image source, detail level, and metadata for the image content. The response from the vision model is then printed.

```Rust
use std::sync::Arc;
use kotoba_llm::{LLMClient, LLMError};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::types::{
    ChatRequest, Message, Role, ContentPart, TextContent,
    ImageContent, ImageSource, ImageDetail
};

#[tokio::main]
async fn main() -> Result<(), LLMError> {
    let transport = default_dyn_transport()?;
    let provider = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?,
    ).with_default_model("gpt-4o");

    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(provider))? 
        .build();

    // Create multimodal request with image
    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![
                ContentPart::Text(TextContent {
                    text: "What's in this image?".into(),
                }),
                ContentPart::Image(ImageContent {
                    source: ImageSource::Url {
                        url: "https://example.com/image.jpg".into()
                    },
                    detail: Some(ImageDetail::High),
                    metadata: None,
                }),
            ],
            metadata: None,
        }],
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    let response = client.chat("openai", request).await?;

    for output in response.outputs {
        println!("Vision response: {:?}", output);
    }

    Ok(())
}

```

--------------------------------

### Initialize LLM Client and Send Chat Request in Rust

Source: https://context7.com/kitsunex07/kotoba/llms.txt

Demonstrates how to initialize an LLM client with a specific provider (OpenAI), register it, and send a synchronous chat request. It showcases setting up the transport layer, provider configuration, building the client, constructing a chat request with messages, and processing the response, including token usage. This example requires the `tokio` runtime and `kotoba-llm` crate with relevant features enabled.

```rust
use std::sync::Arc;
use kotoba_llm::{LLMClient, LLMError};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::types::{ChatRequest, Message, Role, ContentPart, TextContent};

#[tokio::main]
async fn main() -> Result<(), LLMError> {
    // Initialize transport and provider
    let transport = default_dyn_transport()?;
    let provider = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?
    ).with_default_model("gpt-4o-mini");

    // Build client with registered provider
    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(provider))?;
        .build();

    // Create a chat request
    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "Explain Rust ownership in one sentence".into(),
            })],
            metadata: None,
        }],
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    // Send synchronous request
    let response = client.chat("openai", request).await?;

    // Process response
    for output in response.outputs {
        println!("Output: {:?}", output);
    }

    if let Some(usage) = response.usage {
        println!("Tokens used: {:?}", usage.total_tokens);
    }

    Ok(())
}

```

--------------------------------

### Define and Execute Tool Calls with Kotoba LLM (Rust)

Source: https://context7.com/kitsunex07/kotoba/llms.txt

Demonstrates how to define a tool schema, send a chat request with the tool, and process the tool calls with structured outputs using the Kotoba LLM library in Rust. It includes setting up the LLM client, defining the tool's input schema, and handling the tool execution results.

```rust
use std::sync::Arc;
use kotoba_llm::{LLMClient, LLMError};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::types::{
    ChatRequest, Message, Role, ContentPart, TextContent,
    ToolDefinition, ToolKind, ToolChoice, OutputItem
};
use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), LLMError> {
    let transport = default_dyn_transport()?;
    let provider = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?)
        .with_default_model("gpt-4o-mini");

    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(provider))?;
        .build();

    // Define tool schema
    let weather_tool = ToolDefinition {
        name: "get_weather".into(),
        description: Some("Get current weather for a location".into()),
        input_schema: Some(json!({
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City name"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"]
                }
            },
            "required": ["location"]
        })),
        kind: ToolKind::Function,
        metadata: None,
    };

    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "What's the weather in Tokyo?".into(),
            })],
            metadata: None,
        }],
        options: Default::default(),
        tools: vec![weather_tool],
        tool_choice: Some(ToolChoice::Auto),
        response_format: None,
        metadata: None,
    };

    let response = client.chat("openai", request).await?;

    // Process tool calls
    for output in response.outputs {
        match output {
            OutputItem::ToolCall { call, index } => {
                println!("Tool called: {}", call.name);
                println!("Arguments: {}", call.arguments);
                println!("Call ID: {:?}", call.id);

                // Execute tool and prepare result
                let result = json!({
                    "temperature": 22,
                    "condition": "sunny",
                    "unit": "celsius"
                });

                println!("Tool result: {}", result);
            }
            OutputItem::Message { message, .. } => {
                println!("Assistant response: {:?}", message);
            }
            _ => {}
        }
    }

    Ok(())
}

```

--------------------------------

### Dynamically Select LLM Provider by Capability (Rust)

Source: https://context7.com/kitsunex07/kotoba/llms.txt

This snippet shows how to register multiple LLM providers (OpenAI and Anthropic), query their capabilities (streaming, tools, image input), and dynamically select a provider based on whether it supports tool usage. It then sends a simple chat request using the selected provider.

```Rust
use std::sync::Arc;
use kotoba_llm::{LLMClient, LLMError};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::provider::anthropic_messages::AnthropicMessagesProvider;
use kotoba_llm::types::{ChatRequest, Message, Role, ContentPart, TextContent};

#[tokio::main]
async fn main() -> Result<(), LLMError> {
    let transport = default_dyn_transport()?;

    // Register multiple providers
    let openai = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?,
    ).with_default_model("gpt-4o-mini");

    let anthropic = AnthropicMessagesProvider::new(
        transport.clone(),
        std::env::var("ANTHROPIC_API_KEY")?,
    ).with_default_model("claude-3-5-sonnet-20241022");

    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(openai))? 
        .register_handle("claude", Arc::new(anthropic))? 
        .build();

    // Check capabilities for specific handle
    let openai_caps = client.capabilities("openai")?;
    println!("OpenAI supports streaming: {}", openai_caps.supports_stream);
    println!("OpenAI supports tools: {}", openai_caps.supports_tools);
    println!("OpenAI supports image input: {}", openai_caps.supports_image_input);

    // Get all handles supporting specific capabilities
    let streaming_handles = client.handles_supporting_stream();
    println!("Providers with streaming: {:?}", streaming_handles);

    let tool_handles = client.handles_supporting_tools();
    println!("Providers with tool support: {:?}", tool_handles);

    // Select provider based on capability
    let handle = if tool_handles.contains(&"openai".to_string()) {
        "openai"
    } else {
        "claude"
    };

    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "Hello!".into(),
            })],
            metadata: None,
        }],
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    let response = client.chat(handle, request).await?;
    println!("Used provider: {}", response.provider.provider);

    Ok(())
}

```

--------------------------------

### Handle LLM Errors in Rust

Source: https://context7.com/kitsunex07/kotoba/llms.txt

This Rust code snippet demonstrates how to initialize an LLM client using the `kotoba-llm` library and handle various potential errors that can occur during an API call. It specifically shows how to match against different `LLMError` variants like `Auth`, `RateLimit`, `TokenLimitExceeded`, `ModelNotFound`, `Validation`, `Transport`, and `Provider` errors, providing custom error messages and recovery suggestions for each case. It utilizes `tokio` for asynchronous operations and standard Rust error handling.

```rust
use std::sync::Arc;
use kotoba_llm::{LLMClient, LLMError};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::types::{ChatRequest, Message, Role, ContentPart, TextContent, ChatOptions};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let transport = default_dyn_transport()?;
    let provider = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?
    ).with_default_model("gpt-4o-mini");

    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(provider))?
        .build();

    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "Hello!".into(),
            })],
            metadata: None,
        }],
        options: ChatOptions {
            max_output_tokens: Some(100),
            temperature: Some(0.7),
            ..Default::default()
        },
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    match client.chat("openai", request).await {
        Ok(response) => {
            println!("Success: {:?}", response.outputs);
        }
        Err(LLMError::Auth { message }) => {
            eprintln!("Authentication failed: {}", message);
            eprintln!("Check your API key");
        }
        Err(LLMError::RateLimit { message, retry_after }) => {
            eprintln!("Rate limited: {}", message);
            if let Some(duration) = retry_after {
                eprintln!("Retry after: {:?}", duration);
            }
        }
        Err(LLMError::TokenLimitExceeded { message, estimated, limit }) => {
            eprintln!("Token limit exceeded: {}", message);
            if let Some(est) = estimated {
                eprintln!("Estimated tokens: {}", est);
            }
            if let Some(lim) = limit {
                eprintln!("Token limit: {}", lim);
            }
        }
        Err(LLMError::ModelNotFound { model, message }) => {
            eprintln!("Model not found: {}", message);
            if let Some(m) = model {
                eprintln!("Requested model: {}", m);
            }
        }
        Err(LLMError::Validation { message }) => {
            eprintln!("Invalid request: {}", message);
        }
        Err(LLMError::Transport { message }) => {
            eprintln!("Network error: {}", message);
            eprintln!("Consider retrying");
        }
        Err(LLMError::Provider { provider, message }) => {
            eprintln!("Provider {} error: {}", provider, message);
        }
        Err(e) => {
            eprintln!("Unexpected error: {}", e);
        }
    }

    Ok(())
}
```

--------------------------------

### Stream Chat Responses in Rust

Source: https://context7.com/kitsunex07/kotoba/llms.txt

This Rust code snippet demonstrates how to stream chat responses incrementally using delta events from the kotoba-llm library. It requires the `kotoba-llm` crate and `tokio` for asynchronous operations. The output is printed to the console as it is received, including message deltas and tool call deltas, and token usage is reported upon completion.

```rust
use std::sync::Arc;
use futures_util::StreamExt;
use kotoba_llm::{LLMClient, LLMError};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::types::{ChatRequest, Message, Role, ContentPart, TextContent, ChatEvent};

#[tokio::main]
async fn main() -> Result<(), LLMError> {
    let transport = default_dyn_transport()?;
    let provider = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?
    ).with_default_model("gpt-4o-mini");

    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(provider))?
        .build();

    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "Write a haiku about Rust programming".into(),
            })],
            metadata: None,
        }],
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    // Stream chat response
    let mut stream = client.stream_chat("openai", request).await?;

    println!("Streaming response:");
    while let Some(chunk_result) = stream.next().await {
        match chunk_result {
            Ok(chunk) => {
                // Process each event in the chunk
                for event in chunk.events {
                    match event {
                        ChatEvent::MessageDelta(delta) => {
                            for content in delta.content {
                                if let kotoba_llm::types::ContentDelta::Text { text } = content {
                                    print!("{}", text);
                                }
                            }
                        }
                        ChatEvent::ToolCallDelta(tool_delta) => {
                            println!("\nTool call: {:?}", tool_delta);
                        }
                        _ => {}
                    }
                }

                // Check if stream is complete
                if chunk.is_terminal {
                    println!("\n\nStream completed");
                    if let Some(usage) = chunk.usage {
                        println!("Total tokens: {:?}", usage.total_tokens);
                    }
                }
            }
            Err(e) => {
                eprintln!("Stream error: {}", e);
                break;
            }
        }
    }

    Ok(())
}

```

--------------------------------

### Implement Retry Logic with Exponential Backoff in Rust

Source: https://context7.com/kitsunex07/kotoba/llms.txt

Shows how to configure and apply exponential backoff retry strategies for LLM requests using the Kotoba LLM library in Rust. This handles transient errors like rate limits by retrying requests with increasing delays.

```rust
use std::sync::Arc;
use kotoba_llm::{LLMClient, LLMError, RetryConfig};
use kotoba_llm::http::reqwest::default_dyn_transport;
use kotoba_llm::provider::openai_chat::OpenAiChatProvider;
use kotoba_llm::types::{ChatRequest, Message, Role, ContentPart, TextContent};

#[tokio::main]
async fn main() -> Result<(), LLMError> {
    let transport = default_dyn_transport()?;
    let provider = OpenAiChatProvider::new(
        transport.clone(),
        std::env::var("OPENAI_API_KEY")?)
        .with_default_model("gpt-4o-mini");

    let client = LLMClient::builder()
        .register_handle("openai", Arc::new(provider))?;
        .build();

    let request = ChatRequest {
        messages: vec![Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent {
                text: "Hello!".into(),
            })],
            metadata: None,
        }],
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    // Configure retry behavior
    let retry_config = RetryConfig {
        max_retries: 3,
        initial_backoff_ms: 1000,
        max_backoff_ms: 10000,
        backoff_multiplier: 2.0,
    };

    // Send request with retry on rate limits and transport errors
    match client.chat_with_retry("openai", request, retry_config).await {
        Ok(response) => {
            println!("Success after retries: {:?}", response.outputs);
            if let Some(usage) = response.usage {
                println!("Tokens: {:?}", usage.total_tokens);
            }
        }
        Err(LLMError::RateLimit { message, retry_after }) => {
            eprintln!("Rate limited: {}", message);
            if let Some(wait) = retry_after {
                eprintln!("Retry after: {:?}", wait);
            }
        }
        Err(e) => {
            eprintln!("Request failed: {}", e);
        }
    }

    Ok(())
}

```

--------------------------------

### Estimate Chat Request Tokens in Rust

Source: https://context7.com/kitsunex07/kotoba/llms.txt

Estimates token counts for a chat request using Kotoba's TokenEstimator. This is useful for managing API costs by pre-checking request sizes against budget limits. It takes a ChatRequest object and returns a TokenEstimate containing total tokens, a breakdown by role, and overhead.

```rust
use kotoba_llm::types::{
    ChatRequest,
    Message,
    Role,
    ContentPart,
    TextContent,
    TokenEstimator,
    ProviderType,
    ChatOptions
};

fn main() {
    // Create estimator for specific provider
    let estimator = TokenEstimator::new(ProviderType::OpenAI);

    // Build request
    let request = ChatRequest {
        messages: vec![
            Message {
                role: Role::system(),
                name: None,
                content: vec![ContentPart::Text(TextContent {
                    text: "You are a helpful assistant specialized in Rust programming.".into(),
                })],
                metadata: None,
            },
            Message {
                role: Role::user(),
                name: None,
                content: vec![ContentPart::Text(TextContent {
                    text: "Explain the difference between String and &str in Rust.".into(),
                })],
                metadata: None,
            },
        ],
        options: ChatOptions {
            temperature: Some(0.7),
            max_output_tokens: Some(500),
            ..Default::default()
        },
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    };

    // Estimate tokens
    let estimate = estimator.estimate_request(&request);

    println!("Total estimated tokens: {}", estimate.total);
    println!("Token breakdown by role:");
    for (role, count) in estimate.by_role {
        println!("  {}: {} tokens", role, count);
    }
    println!("Overhead: {} tokens", estimate.overhead);

    // Budget check before sending
    const MAX_PROMPT_TOKENS: usize = 4000;
    if estimate.total > MAX_PROMPT_TOKENS {
        eprintln!("Warning: Request exceeds token budget!");
        eprintln!("Estimated: {}, Limit: {}", estimate.total, MAX_PROMPT_TOKENS);
    } else {
        println!("Request within budget, safe to send");
    }
}
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.