Kotoba LLM

https://docs.rs/kotoba-llm/latest/
Admin
Kotoba LLM provides unified high-level client abstractions for multi-provider LLM access, enabling...
Tokens:699,166
Snippets:4,174
Trust Score:9.7
Update:9 months ago
Show doc for...
Tokens:
Raw
### Rust: Example Searches for Data Structures

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/types/enum.ContentDelta_search=

Provides examples of search queries for common Rust data structures and type transformations. These examples are useful for understanding how to query and manipulate data within the project.

```rust
// Example searches:
//   * std::vec
//   * u32 -> bool
//   * Option<T>, (T -> U) -> Option<U>
```

--------------------------------

### Arc Allocation Examples

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/type.DynProvider_search=u32+-%3E+bool

Examples demonstrating the usage of Arc with custom allocators for creating and converting raw pointers.

```APIDOC
## Arc Allocation Examples

### Description
Examples demonstrating the usage of Arc with custom allocators for creating and converting raw pointers.

### Method
N/A (Illustrative Examples)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```rust
#![feature(allocator_api)]

use std::sync::Arc;
use std::alloc::System;

let x = Arc::new_in("hello".to_owned(), System);
let (x_ptr, alloc) = Arc::into_raw_with_allocator(x);

unsafe {
    // Convert back to an `Arc` to prevent leak.
    let x = Arc::from_raw_in(x_ptr, System);
    assert_eq!(&*x, "hello");

    // Further calls to `Arc::from_raw(x_ptr)` would be memory-unsafe.
}

// The memory was freed when `x` went out of scope above, so `x_ptr` is now dangling!
```

### Response
#### Success Response (200)
N/A

#### Response Example
```json
{
  "example": "No direct JSON response, code execution demonstrates functionality."
}
```
```

```APIDOC
## Convert Slice to Array with Arc

### Description
Demonstrates converting a slice back into its original array using Arc with a custom allocator.

### Method
N/A (Illustrative Example)

### Endpoint
N/A

### Parameters
N/A

### Request Example
```rust
#![feature(allocator_api)]

use std::sync::Arc;
use std::alloc::System;

let x: Arc<[u32], _> = Arc::new_in([1, 2, 3], System);
let x_ptr: *const [u32] = Arc::into_raw_with_allocator(x).0;

unsafe {
    let x: Arc<[u32; 3], _> = Arc::from_raw_in(x_ptr.cast::<[u32; 3]>(), System);
    assert_eq!(&*x, &[1, 2, 3]);
}
```

### Response
#### Success Response (200)
N/A

#### Response Example
```json
{
  "example": "No direct JSON response, code execution demonstrates functionality."
}
```
```

--------------------------------

### Example Usage of RequestPatch in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/config/struct.RequestPatch_search=

Demonstrates how to create and initialize a RequestPatch instance in Rust. This example shows setting values for URL, body, headers, and fields to remove, illustrating the typical use cases for runtime request modification.

```rust
let patch = RequestPatch {
    url: Some("https://proxy.internal/chat".to_string()),
    body: Some(json!({ "metadata": { "tenant": "blue" } })),
    headers: Some(HashMap::from([
        ("x-trace".to_string(), Some("123".to_string())),
        ("x-remove".to_string(), None),
    ])),
    remove_fields: Some(vec!["options.max_tokens".to_string()])
};
```

--------------------------------

### Example Chat Request Initialization in Rust

Source: https://docs.rs/kotoba-llm/latest/src/kotoba_llm/types/mod.rs

Demonstrates how to initialize a `ChatRequest` object in Rust, including setting up system and user messages with text content. This serves as a basic example for making chat requests.

```rust
/// Chat request shared across all providers.
///
/// A `ChatRequest` mimics the shape of OpenAI/Anthropic messages while normalizing
/// multimodal content and tooling metadata. Populate `messages`, configure
/// [`ChatOptions`], and optionally attach tool definitions or response-format hints
/// before passing it to [`crate::client::LLMClient::chat`].
///
/// # Examples
///
/// ```
/// # use kotoba_llm::types::{ChatRequest, ChatOptions, ContentPart, Message, Role, TextContent};
/// let request = ChatRequest {
///     messages: vec![
///         Message {
///             role: Role::system(),
///             name: None,
///             content: vec![ContentPart::Text(TextContent { text: "You are concise.".into() })],
///             metadata: None,
///         },
///         Message {
///             role: Role::user(),
///             name: None,
///             content: vec![ContentPart::Text(TextContent { text: "Summarize Rust traits.".into() })],
///             metadata: None,
///         },
/// ```
```

--------------------------------

### Rust Common Search Examples

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/http/type.HttpBodyStream_search=

Provides examples of common search queries used in Rust documentation or code exploration tools. These examples illustrate how to search for standard library types, type signatures, and generic function patterns.

```rust
std::vec
```

```rust
u32 -> bool
```

```rust
Option<T>, (T -> U) -> Option<U>
```

--------------------------------

### Example: Building an LLMClient

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/client/struct.LLMClientBuilder_search=

Illustrates the process of building an LLMClient after registering a provider handle. It verifies that the built client contains the expected handles.

```rust
let client = LLMClient::builder()
    .register_handle("primary", Arc::new(DummyProvider))
    .expect("unique handle")
    .build();
assert_eq!(client.handles(), vec!["primary".to_string()])
```

--------------------------------

### Example Usage of ModelConfig (Rust)

Source: https://docs.rs/kotoba-llm/latest/src/kotoba_llm/config.rs_search=u32+-%3E+bool

Illustrates how to create and use the `ModelConfig` struct in Rust. This example shows the instantiation of a `ModelConfig` for an OpenAI chat provider, specifying its handle, provider type, API key credential, and a default model. It also includes an example of setting provider-specific extra configurations like 'service_tier'.

```rust
# use std::collections::HashMap;
# use kotoba_llm::config::{ModelConfig, ProviderKind, Credential};
let cfg = ModelConfig {
    handle: "default-openai".into(),
    provider: ProviderKind::OpenAiChat,
    credential: Credential::ApiKey { header: None, key: "sk-test".into() },
    default_model: Some("gpt-4.1-mini".into()),
    base_url: Some("https://api.openai.com".into()),
    extra: HashMap::from([("service_tier".into(), serde_json::json!("default"))]),
    patch: None,
};
assert_eq!(cfg.handle, "default-openai");

```

--------------------------------

### Example Usage of ChatResponse in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/types/struct.ChatResponse_search=std%3A%3Avec

Demonstrates how to create and use the `ChatResponse` struct in Rust. This example shows the instantiation of a `ChatResponse` with a sample message output and asserts the number of outputs.

```rust
let response = ChatResponse {
    outputs: vec![OutputItem::Message {
        index: 0,
        message: Message {
            role: Role::assistant(),
            name: None,
            content: vec![ContentPart::Text(TextContent { text: "Hello".into() })],
            metadata: None,
        },
    }],
    usage: None,
    finish_reason: None,
    model: Some("gpt-4o-mini".into()),
    provider: ProviderMetadata { provider: "openai_chat".into(), ..Default::default() },
};
assert_eq!(response.outputs.len(), 1);
```

--------------------------------

### LLMClientLike Chat Method Example (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/client/trait.LLMClientLike_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Demonstrates how to use the 'chat' method of the LLMClientLike trait. It shows creating a client, casting it to a trait object, preparing a ChatRequest, and making the call. The example asserts that the result is an error, likely due to a dummy client or configuration.

```rust
let client = build_client();
let obj: &dyn LLMClientLike = &client;
let request = ChatRequest {
    messages: Vec::new(),
    options: Default::default(),
    tools: Vec::new(),
    tool_choice: None,
    response_format: None,
    metadata: None,
};
let result = obj.chat("dummy", request).await;
assert!(result.is_err());
```

--------------------------------

### Register Providers Macro Usage Example

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/macro.register_providers_search=

Demonstrates a practical example of using the `register_providers!` macro to register multiple AI model providers, specifically `openai_chat` and `anthropic_messages`. This single macro call generates the necessary enum variants and functions for these providers.

```rust
use crate::provider::macros::register_providers;

register_providers!(
    (openai_chat, "openai_chat", OpenAiChatProvider, OpenAiChat),
    (anthropic_messages, "anthropic_messages", AnthropicMessagesProvider, AnthropicMessages),
);
```

--------------------------------

### Example Usage of TokenUsage in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/types/struct.TokenUsage_search=

Demonstrates how to create and use a TokenUsage struct in Rust. This example shows initializing the struct with specific token counts and asserting the value of total_tokens. It highlights the use of `Option<u64>` for token counts, allowing for missing values.

```rust
let usage = TokenUsage {
    prompt_tokens: Some(1200),
    completion_tokens: Some(200),
    reasoning_tokens: None,
    total_tokens: Some(1400),
    details: None,
};
assert_eq!(usage.total_tokens, Some(1400));
```

--------------------------------

### Example Usage of ChatRequest in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/types/struct.ChatRequest_search=

This Rust code snippet demonstrates how to create and initialize a `ChatRequest` object. It populates the `messages` field with a system and a user message, and sets a `temperature` option. The example also includes an assertion to verify the number of messages in the request.

```rust
let request = ChatRequest {
    messages: vec![
        Message {
            role: Role::system(),
            name: None,
            content: vec![ContentPart::Text(TextContent { text: "You are concise.".into() })],
            metadata: None,
        },
        Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent { text: "Summarize Rust traits.".into() })],
            metadata: None,
        },
    ],
    options: ChatOptions { temperature: Some(0.3), ..Default::default() },
    tools: Vec::new(),
    tool_choice: None,
    response_format: None,
    metadata: None,
};
assert_eq!(request.messages.len(), 2);
```

--------------------------------

### Example Usage of RequestPatch (Rust)

Source: https://docs.rs/kotoba-llm/latest/src/kotoba_llm/config.rs_search=u32+-%3E+bool

Demonstrates the creation of a `RequestPatch` object in Rust. This example shows how to configure a patch to modify an outgoing request. It includes setting a custom URL, merging a JSON fragment into the request body, adding or removing headers, and specifying fields to be removed from the request body.

```rust
# use std::collections::HashMap;
# use serde_json::json;
# use kotoba_llm::config::RequestPatch;
let patch = RequestPatch {
    url: Some("https://proxy.internal/chat".to_string()),
    body: Some(json!({ "metadata": { "tenant": "blue" } })),
    headers: Some(HashMap::from([
        ("x-trace".to_string(), Some("123".to_string())),
        ("x-remove".to_string(), None),
    ])),
    remove_fields: Some(vec!["options.max_tokens".to_string()]),
};

```

--------------------------------

### Example: Registering a Provider Handle

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/client/struct.LLMClientBuilder_search=

Demonstrates how to use the register_handle method to add a provider to the LLMClientBuilder. It shows asserting the presence of the registered handle after building the client.

```rust
let builder = LLMClient::builder()
    .register_handle("primary", Arc::new(DummyProvider))
    .expect("unique handle");
assert!(builder.build().handles().contains(&"primary".to_string()))
```

--------------------------------

### Example: Registering and Building LLMClient (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/client/struct.LLMClientBuilder_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Demonstrates the usage of LLMClientBuilder to register a provider with a handle and then build the LLMClient. It includes assertions to verify the registration and the handles of the built client.

```rust
let builder = LLMClient::builder()
    .register_handle("primary", Arc::new(DummyProvider))
    .expect("unique handle");
assert!(builder.build().handles().contains(&"primary".to_string()));
```

```rust
let client = LLMClient::builder()
    .register_handle("primary", Arc::new(DummyProvider))
    .expect("unique handle")
    .build();
assert_eq!(client.handles(), vec!["primary".to_string()]);
```

--------------------------------

### Initialize ReqwestTransport with Default Client (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/http/reqwest/struct.ReqwestTransport_search=

Creates a ReqwestTransport using the default reqwest configuration. This is a convenient way to get started without explicit client setup. It can return an LLMError if the reqwest client fails to build.

```rust
use kotoba_llm::http::reqwest::ReqwestTransport;

let transport = ReqwestTransport::default_client().expect("transport");
```

--------------------------------

### Initialize LLMClient with Providers (Rust)

Source: https://docs.rs/kotoba-llm/latest/src/kotoba_llm/client.rs

Demonstrates how to create an LLMClient builder, register providers with unique handles, and build the client instance. This is crucial for setting up the client to route requests to the appropriate backend.

```rust
use std::collections::HashMap;
use std::sync::Arc;
use async_trait::async_trait;
use kotoba_llm::client::LLMClient;
use kotoba_llm::error::LLMError;
use kotoba_llm::provider::{LLMProvider, ChatStream};
use kotoba_llm::types::{CapabilityDescriptor, ChatRequest, ChatResponse};
use futures_util::stream;

struct DummyProvider;

#[async_trait]
impl LLMProvider for DummyProvider {
    async fn chat(&self, _request: ChatRequest) -> Result<ChatResponse, LLMError> {
        Err(LLMError::NotImplemented { feature: "dummy" })
    }
    async fn stream_chat(&self, _request: ChatRequest) -> Result<ChatStream, LLMError> {
        Ok(Box::pin(stream::empty()))
    }
    fn capabilities(&self) -> CapabilityDescriptor { CapabilityDescriptor::default() }
    fn name(&self) -> &'static str { "dummy" }
}

let client = LLMClient::builder()
    .register_handle("dummy", Arc::new(DummyProvider))
    .expect("unique handle")
    .build();

assert_eq!(client.handles(), vec!["dummy".to_string()]);
```

--------------------------------

### Rust Arc::get_mut_unchecked Example

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/http/type.DynHttpTransport_search=

Illustrates the use of the `get_mut_unchecked` function to get a mutable reference to the underlying value of an `Arc` without any checks. This is an unsafe operation and should be used with extreme caution. The code snippets provide examples to demonstrate potential undefined behavior when used improperly.

```Rust
#![feature(get_mut_unchecked)]

use std::sync::Arc;

let mut x = Arc::new(String::new());
unsafe {
    Arc::get_mut_unchecked(&mut x).push_str("foo")
}
assert_eq!(*x, "foo");
```

```Rust
#![feature(get_mut_unchecked)]

use std::sync::Arc;

let x: Arc<str> = Arc::from("Hello, world!");
let mut y: Arc<[u8]> = x.clone().into();
unsafe {
    // this is Undefined Behavior, because x's inner type is str, not [u8]
    Arc::get_mut_unchecked(&mut y).fill(0xff); // 0xff is invalid in UTF-8
}
println!("{}", &*x); // Invalid UTF-8 in a str
```

```Rust
#![feature(get_mut_unchecked)]

use std::sync::Arc;

let x: Arc<&str> = Arc::new("Hello, world!");
{
    let s = String::from("Oh, no!");
    let mut y: Arc<&str> = x.clone();
    unsafe {
        // this is Undefined Behavior, because x's inner type
        // is &'long str, not &'short str
        *Arc::get_mut_unchecked(&mut y) = &s;
    }
}
println!("{}", &*x); // Use-after-free
```

--------------------------------

### Get Underlying Allocator for Arc

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/type.DynProvider_search=

This example demonstrates how to access the allocator used by an `Arc` using the `Arc::allocator` associated function. Note that this is a nightly-only experimental API and requires the `allocator_api` feature.

```rust
#![feature(allocator_api)]
use std::sync::Arc;
use std::alloc::System;

let x = Arc::new_in("hello".to_owned(), System);
let alloc = Arc::allocator(&x);
// `alloc` now holds a reference to the System allocator.
```

--------------------------------

### GoogleGeminiProvider Constructor and Configuration

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/google_gemini/struct.GoogleGeminiProvider

Provides methods for creating and configuring a GoogleGeminiProvider instance. This includes setting up the default endpoint, overriding the base URL, and specifying a default model.

```rust
pub fn new(transport: DynHttpTransport, api_key: impl Into<String>) -> Self
pub fn with_base_url(self, base_url: impl Into<String>) -> Self
pub fn with_default_model(self, model: impl Into<String>) -> Self
```

--------------------------------

### Getting Underlying Allocator with Arc::allocator (Nightly)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/type.DynProvider

This example demonstrates how to retrieve a reference to the underlying allocator used by an `Arc`. This is an experimental, nightly-only API and requires the `allocator_api` feature. It shows the associated function call syntax.

```rust
use std::sync::Arc;
use std::alloc::System;

// Note: This requires the `allocator_api` feature and is nightly-only.
// Example usage assumes an Arc created with a specific allocator.
// let a: Arc<i32, System> = Arc::new_in(5, System);
// let allocator_ref = Arc::allocator(&a);
// assert!(std::any::TypeId::of::<System>() == std::any::TypeId::of_val(allocator_ref));
```

--------------------------------

### OpenAiResponsesProvider Initialization (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/openai_responses/struct.OpenAiResponsesProvider

Demonstrates how to create a new OpenAiResponsesProvider instance. It requires an HTTP transport and an API key. The default OpenAI endpoint is used.

```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "key");
assert_eq!(provider.name(), "openai_responses");
```

--------------------------------

### Get Weak Count of Arc

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/type.DynProvider_search=u32+-%3E+bool

This Rust code snippet shows how to retrieve the number of `Weak` pointers associated with an `Arc` using `Arc::weak_count`. It emphasizes the safety considerations, noting that the weak count can change concurrently. The example demonstrates a deterministic scenario where the count is checked without any thread sharing.

```rust
use std::sync::Arc;

let five = Arc::new(5);
let _weak_five = Arc::downgrade(&five);

// This assertion is deterministic because we haven't shared
// the `Arc` or `Weak` between threads.
assert_eq!(1, Arc::weak_count(&five));
```

--------------------------------

### OpenAiResponsesProvider Initialization and Configuration

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/openai_responses/struct.OpenAiResponsesProvider

This section details how to create and configure an instance of the OpenAiResponsesProvider, including setting the API key, base URL, organization, project, and default model.

```APIDOC
## POST /websites/rs_kotoba-llm/provider/openai_responses

### Description
This endpoint (or rather, this set of Rust methods) provides functionalities to interact with the OpenAI API. It allows for creating a provider instance, configuring it with various settings like base URL, organization, project, and default model, and then using it for chat-based interactions.

### Method
N/A (These are Rust struct methods, not HTTP endpoints in the traditional sense, but represent API interactions.)

### Endpoint
N/A (These are Rust struct methods.)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None (for constructor methods)

### Request Example
```rust
// Example for new()
use kotoba_llm::provider::openai_responses::OpenAiResponsesProvider;
use kotoba_llm::transport::default_dyn_transport;

let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "YOUR_API_KEY");

// Example for with_base_url()
let provider = OpenAiResponsesProvider::new(transport, "YOUR_API_KEY")
    .with_base_url("https://your.proxy.url");

// Example for with_organization()
let provider = OpenAiResponsesProvider::new(transport, "YOUR_API_KEY")
    .with_organization("YOUR_ORG_ID");

// Example for with_project()
let provider = OpenAiResponsesProvider::new(transport, "YOUR_API_KEY")
    .with_project("YOUR_PROJECT_ID");

// Example for with_default_model()
let provider = OpenAiResponsesProvider::new(transport, "YOUR_API_KEY")
    .with_default_model("gpt-4-turbo");
```

### Response
#### Success Response (N/A - These are instantiation and configuration methods)

#### Response Example
N/A
```

--------------------------------

### Get Name of OpenAiResponsesProvider (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/openai_responses/struct.OpenAiResponsesProvider_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Demonstrates how to get the identifier name of the OpenAiResponsesProvider. This name is used for logging and error reporting purposes.

```rust
use kotoba_llm::provider::LLMProvider;

// Assume provider is an instance of OpenAiResponsesProvider
// let provider_name = provider.name();
// assert_eq!(provider_name, "openai_responses");
```

--------------------------------

### GoogleGeminiProvider Initialization

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/google_gemini/struct.GoogleGeminiProvider

APIs for creating and configuring instances of the Google Gemini Provider.

```APIDOC
## POST /provider/google_gemini/new

### Description
Creates a new GoogleGeminiProvider instance targeting the default Google Generative Language endpoint.

### Method
POST

### Endpoint
/provider/google_gemini/new

### Parameters
#### Request Body
- **transport** (DynHttpTransport) - Required - The HTTP transport implementation to use.
- **api_key** (string) - Required - The API key for authenticating with Google Gemini.

### Request Example
```json
{
  "transport": "<http_transport_instance>",
  "api_key": "YOUR_API_KEY"
}
```

### Response
#### Success Response (200)
- **provider_instance** (GoogleGeminiProvider) - The newly created provider instance.

#### Response Example
```json
{
  "provider_instance": "<google_gemini_provider_instance>"
}
```
```

```APIDOC
## PUT /provider/google_gemini/with_base_url

### Description
Overrides the base URL for the Google Gemini provider, useful for pointing at proxies or compatibility layers.

### Method
PUT

### Endpoint
/provider/google_gemini/with_base_url

### Parameters
#### Request Body
- **provider_instance** (GoogleGeminiProvider) - Required - The provider instance to configure.
- **base_url** (string) - Required - The new base URL to use.

### Request Example
```json
{
  "provider_instance": "<google_gemini_provider_instance>",
  "base_url": "http://localhost:8080/v1"
}
```

### Response
#### Success Response (200)
- **updated_provider_instance** (GoogleGeminiProvider) - The provider instance with the updated base URL.

#### Response Example
```json
{
  "updated_provider_instance": "<google_gemini_provider_instance_with_new_url>"
}
```
```

```APIDOC
## PUT /provider/google_gemini/with_default_model

### Description
Sets a default model for the Google Gemini provider. This model will be used when requests do not specify one.

### Method
PUT

### Endpoint
/provider/google_gemini/with_default_model

### Parameters
#### Request Body
- **provider_instance** (GoogleGeminiProvider) - Required - The provider instance to configure.
- **model** (string) - Required - The name of the default model (e.g., `gemini-2.0-flash`).

### Request Example
```json
{
  "provider_instance": "<google_gemini_provider_instance>",
  "model": "gemini-2.0-flash"
}
```

### Response
#### Success Response (200)
- **updated_provider_instance** (GoogleGeminiProvider) - The provider instance with the default model set.

#### Response Example
```json
{
  "updated_provider_instance": "<google_gemini_provider_instance_with_default_model>"
}
```
```

```APIDOC
## POST /provider/google_gemini/from_model_config

### Description
Constructs a GoogleGeminiProvider from a `ModelConfig` object. This is typically used by the provider registration system.

### Method
POST

### Endpoint
/provider/google_gemini/from_model_config

### Parameters
#### Request Body
- **config** (ModelConfig) - Required - The model configuration containing credentials and settings.
- **transport** (DynHttpTransport) - Required - The HTTP transport implementation to use.

### Request Example
```json
{
  "config": {
    "api_key": "YOUR_API_KEY",
    "model_name": "gemini-pro"
    // ... other config fields
  },
  "transport": "<http_transport_instance>"
}
```

### Response
#### Success Response (200)
- **provider_instance** (GoogleGeminiProvider) - The constructed provider instance.

#### Error Response (400)
- **error** (LLMError) - Indicates an issue with authentication or configuration.

#### Response Example
```json
{
  "provider_instance": "<google_gemini_provider_instance>"
}
```
```

--------------------------------

### Rust Arc::get_mut_unchecked - Undefined Behavior Examples

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/type.DynProvider_search=std%3A%3Avec

Presents examples of how `Arc::get_mut_unchecked` can lead to undefined behavior if used incorrectly, specifically when the underlying types of the `Arc` instances differ or when lifetimes are mismatched. These examples emphasize the critical need for type correctness and exclusive access when using the `unchecked` method.

```rust
#![feature(get_mut_unchecked)]

use std::sync::Arc;

let x: Arc<str> = Arc::from("Hello, world!");
let mut y: Arc<[u8]> = x.clone().into();
unsafe {
    // this is Undefined Behavior, because x's inner type is str, not [u8]
    Arc::get_mut_unchecked(&mut y).fill(0xff); // 0xff is invalid in UTF-8
}
println!("{}", &*x); // Invalid UTF-8 in a str
```

```rust
#![feature(get_mut_unchecked)]

use std::sync::Arc;

let x: Arc<&str> = Arc::new("Hello, world!");
{
    let s = String::from("Oh, no!");
    let mut y: Arc<&str> = x.clone();
    unsafe {
        // this is Undefined Behavior, because x's inner type
        // is &'long str, not &'short str
        *Arc::get_mut_unchecked(&mut y) = &s;
    }
}
println!("{}", &*x); // Use-after-free
```

--------------------------------

### Register and Query Tool-Supporting Handles in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/client/struct.LLMClient_search=std%3A%3Avec

Demonstrates how to build an LLMClient by registering handles that either support tools or do not. It then shows how to retrieve and assert the list of handles that specifically support tools.

```rust
let client = LLMClient::builder()
    .register_handle("tools", Arc::new(DummyProvider { tools: true }) as _)
    .expect("tools handle")
    .register_handle("no-tools", Arc::new(DummyProvider { tools: false }) as _)
    .expect("no-tools handle")
    .build();
let mut handles = client.handles_supporting_tools();
handles.sort();
assert_eq!(handles, vec!["tools".to_string()]);
```

--------------------------------

### Get Name of OpenAiChatProvider in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/openai_chat/struct.OpenAiChatProvider

Demonstrates how to get the identifier name of the `OpenAiChatProvider`, which is used for logging and error reporting. The name is typically 'openai_chat'.

```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiChatProvider::new(transport, "key");
assert_eq!(provider.name(), "openai_chat");
```

--------------------------------

### GoogleGeminiProvider Construction and Configuration

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/google_gemini/struct.GoogleGeminiProvider_search=

Methods for creating and configuring the GoogleGeminiProvider.

```APIDOC
## GoogleGeminiProvider

### Description
Google Gemini GenerateContent provider implementation.

### Methods

#### `new(transport: DynHttpTransport, api_key: impl Into<String>) -> Self`
Creates a provider that targets the default Google Generative Language endpoint.

#### `with_base_url(self, base_url: impl Into<String>) -> Self`
Overrides the base URL, making it easier to point at proxies or compatibility layers.

#### `with_default_model(self, model: impl Into<String>) -> Self`
Sets a default model such as `gemini-2.0-flash` when the request omits one.

#### `from_model_config(config: &ModelConfig, transport: DynHttpTransport) -> Result<Self, LLMError>`
Constructs a provider from a `crate::config::ModelConfig`.
This method is used by the macro-driven provider registration system to build providers from declarative configuration.

##### Arguments
- `config` (ModelConfig) - The model configuration containing credentials and settings
- `transport` (DynHttpTransport) - The HTTP transport implementation to use

##### Errors
- `LLMError::Auth` - When credentials are missing or invalid.
```

--------------------------------

### Initialize Google Gemini Provider (Rust)

Source: https://docs.rs/kotoba-llm/latest/src/kotoba_llm/provider/google_gemini/provider.rs

Demonstrates how to create a new instance of the GoogleGeminiProvider, specifying the HTTP transport and API key. Additional methods allow customization of the base URL and default model.

```Rust
use std::collections::HashMap;
use async_trait::async_trait;
use serde::de::DeserializeOwned;
use serde_json::Value;

use crate::config::RequestPatch;
use crate::error::LLMError;
use crate::http::{
    DynHttpTransport,
    HttpResponse,
    HttpStreamResponse,
    post_json_stream_with_headers,
    post_json_with_headers,
};
use crate::provider::{ChatStream, LLMProvider, retry::retry_after_from_headers};
use crate::types::{CapabilityDescriptor, ChatRequest, ChatResponse};

use super::error::parse_gemini_error;
use super::request::build_gemini_body;
use super::response::map_response;
use super::stream::{collect_stream_text, create_stream};
use super::types::GeminiGenerateContentResponse;

const DEFAULT_BASE_URL: &str = "https://generativelanguage.googleapis.com";

/// Google Gemini GenerateContent provider implementation.
pub struct GoogleGeminiProvider {
    pub(crate) transport: DynHttpTransport,
    pub(crate) base_url: String,
    pub(crate) api_key: String,
    pub(crate) default_model: Option<String>,
    pub(crate) request_patch: Option<RequestPatch>,
}

impl GoogleGeminiProvider {
    /// Creates a provider that targets the default Google Generative Language endpoint.
    pub fn new(transport: DynHttpTransport, api_key: impl Into<String>) -> Self {
        Self {
            transport,
            base_url: DEFAULT_BASE_URL.to_string(),
            api_key: api_key.into(),
            default_model: None,
            request_patch: None,
        }
    }

    /// Overrides the base URL, making it easier to point at proxies or compatibility layers.
    pub fn with_base_url(mut self, base_url: impl Into<String>) -> Self {
        self.base_url = base_url.into();
        self
    }

    /// Sets a default model such as `gemini-2.0-flash` when the request omits one.
    pub fn with_default_model(mut self, model: impl Into<String>) -> Self {
        self.default_model = Some(model.into());
        self
    }

    /// Constructs a provider from a [`crate::config::ModelConfig`].
    ///
    /// This method is used by the macro-driven provider registration system to build
    /// providers from declarative configuration.
    ///
    /// # Arguments
    ///
    /// * `config` - The model configuration containing credentials and settings
    /// * `transport` - The HTTP transport implementation to use
    ///
    /// # Errors
    ///
    /// Returns [`LLMError::Auth`] when credentials are missing or invalid.
    pub fn from_model_config(
        config: &crate::config::ModelConfig,
        transport: DynHttpTransport,
    ) -> Result<Self, LLMError> {
        use crate::config::Credential;

        let api_key = match &config.credential {
            Credential::ApiKey { key, .. } => key.clone(),
            Credential::Bearer { token } => token.clone(),
            Credential::ServiceAccount { .. } => {
                return Err(LLMError::Auth {
                    message: "provider google_gemini does not support service account credential"
                        .to_string(),
                });
            }
            Credential::None => {
                return Err(LLMError::Auth {
                    message: "provider google_gemini requires credential".to_string(),
                });
            }
        };

        let mut provider = Self::new(transport, api_key);

        if let Some(base_url) = &config.base_url {
            provider = provider.with_base_url(base_url.clone());
        }

        if let Some(model) = &config.default_model {
            provider = provider.with_default_model(model.clone());
        }

        provider.request_patch = config.patch.clone();

        Ok(provider)
    }

    /// Builds the non-streaming endpoint URL for GenerateContent.
    pub(crate) fn endpoint(&self, model: &str) -> String {
        let base = self.base_url.trim_end_matches('/');
        let model_path = normalize_model(model);
        if base.ends_with("/v1beta") {
            format!("{base}/{model_path}:generateContent")
        } else {
            format!("{base}/v1beta/{model_path}:generateContent")
        }
    }

    /// Builds the streaming endpoint URL (SSE) for GenerateContent.
    pub(crate) fn stream_endpoint(&self, model: &str) -> String {
        let base = self.base_url.trim_end_matches('/');
        let model_path = normalize_model(model);
        if base.ends_with("/v1beta") {
            format!("{base}/{model_path}:streamGenerateContent?alt=sse")
        } else {
            format!("{base}/v1beta/{model_path}:streamGenerateContent?alt=sse")
        }
    }

    fn build_headers(&self) -> HashMap<String, String> {
        // Implementation omitted for brevity
    }

    // Placeholder for normalize_model function, assuming it exists elsewhere
    fn normalize_model(model: &str) -> String {
        model.to_string()
    }
}

```

--------------------------------

### GoogleGeminiProvider Construction and Configuration

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/google_gemini/struct.GoogleGeminiProvider_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

This section outlines methods for creating and configuring the GoogleGeminiProvider, including setting the base URL and default model.

```APIDOC
## GoogleGeminiProvider

### Description
Google Gemini GenerateContent provider implementation.

### Methods

#### `new(transport: DynHttpTransport, api_key: impl Into<String>) -> Self`
Creates a provider that targets the default Google Generative Language endpoint.

#### `with_base_url(self, base_url: impl Into<String>) -> Self`
Overrides the base URL, making it easier to point at proxies or compatibility layers.

#### `with_default_model(self, model: impl Into<String>) -> Self`
Sets a default model such as `gemini-2.0-flash` when the request omits one.

#### `from_model_config(config: &ModelConfig, transport: DynHttpTransport) -> Result<Self, LLMError>`
Constructs a provider from a `crate::config::ModelConfig`. This method is used by the macro-driven provider registration system to build providers from declarative configuration.

##### Arguments
- **config** (ModelConfig) - The model configuration containing credentials and settings.
- **transport** (DynHttpTransport) - The HTTP transport implementation to use.

##### Errors
- **LLMError::Auth** - Returned when credentials are missing or invalid.

### Request Example (from_model_config)
```json
{
  "config": {
    "api_key": "YOUR_API_KEY",
    "model_name": "gemini-pro"
  },
  "transport": "<http_transport_implementation>"
}
```

### Response Example (from_model_config)
```json
{
  "GoogleGeminiProvider": {
    "provider_instance": "<instance_of_google_gemini_provider>"
  }
}
```
```

--------------------------------

### Token Estimator Example (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/types/struct.TokenEstimator_search=

An example demonstrating how to use `TokenEstimator::new` and `estimate_request` to calculate token estimates for a chat request. It includes assertions to verify the estimated total and the presence of role-specific breakdowns.

```rust
let request = ChatRequest {
    messages: vec![
        Message {
            role: Role::system(),
            name: None,
            content: vec![ContentPart::Text(TextContent { text: "You are terse".into() })],
            metadata: None,
        },
        Message {
            role: Role::user(),
            name: None,
            content: vec![ContentPart::Text(TextContent { text: "Explain enums".into() })],
            metadata: None,
        },
    ],
    options: ChatOptions::default(),
    tools: vec![],
    tool_choice: None,
    response_format: None,
    metadata: None,
};
let estimate = TokenEstimator::new(ProviderType::OpenAI).estimate_request(&request);
assert!(estimate.total > 0);
assert!(estimate.by_role.contains_key("system"));
```

--------------------------------

### Create GoogleGeminiProvider Instance

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/google_gemini/struct.GoogleGeminiProvider_search=

Provides methods for creating and configuring instances of `GoogleGeminiProvider`. These include creating a provider targeting the default Google endpoint, overriding the base URL, setting a default model, and constructing a provider from a `ModelConfig`.

```rust
pub fn new(transport: DynHttpTransport, api_key: impl Into<String>) -> Self
```

```rust
pub fn with_base_url(self, base_url: impl Into<String>) -> Self
```

```rust
pub fn with_default_model(self, model: impl Into<String>) -> Self
```

```rust
pub fn from_model_config( config: &ModelConfig, transport: DynHttpTransport, ) -> Result<Self, LLMError>
```

--------------------------------

### Rust Arc::is_unique Example

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/http/type.DynHttpTransport_search=

Shows how to use the `is_unique` function to determine if an `Arc` is the only owner of the underlying data. This function can be used to optimize operations. Examples showcase checking the uniqueness and handling `Weak` references.

```Rust
#![feature(arc_is_unique)]

use std::sync::Arc;

let x = Arc::new(3);
assert!(Arc::is_unique(&x));

let y = Arc::clone(&x);
assert!(!Arc::is_unique(&x));
drop(y);

// Weak references also count, because they could be upgraded at any time.
let z = Arc::downgrade(&x);
assert!(!Arc::is_unique(&x));
```

```Rust
#![feature(arc_is_unique)]

use std::sync::Arc;

let arc = Arc::new(5);
let pointer: *const i32 = &*arc;
assert!(Arc::is_unique(&arc));
assert_eq!(unsafe { *pointer }, 5);
```

--------------------------------

### Define Token Usage Structure Example (Rust)

Source: https://docs.rs/kotoba-llm/latest/src/kotoba_llm/types/mod.rs_search=std%3A%3Avec

Provides an example of the `TokenUsage` structure, which holds optional counts for prompt tokens, completion tokens, and reasoning tokens. This structure is used for tracking resource consumption.

```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TokenUsage {
    /// Number of tokens in the prompt.
    pub prompt_tokens: Option<u32>,
    /// Number of tokens in the completion.
    pub completion_tokens: Option<u32>,
    /// Number of tokens used for reasoning.
    pub reasoning_tokens: Option<u32>,
}

// Example usage:
let usage = TokenUsage {
    prompt_tokens: Some(1200),
    completion_tokens: Some(200),
    reasoning_tokens: None,
};
```

--------------------------------

### Register Providers Macro Syntax Example

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/macro.register_providers_search=

Illustrates the syntax for using the `register_providers!` macro in Rust. It shows how to specify the module name, serde serialization name, provider struct name, and enum variant name for each provider being registered.

```rust
register_providers!(
    (module_name, "serde_name", ProviderType, EnumVariant),
    // ... more providers
);
```

--------------------------------

### OpenAI Responses Provider Initialization and Configuration

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/provider/openai_responses/struct.OpenAiResponsesProvider_search=u32+-%3E+bool

This section covers the creation and configuration of the OpenAiResponsesProvider, including setting base URLs, organization, project, and default models.

```APIDOC
## OpenAiResponsesProvider

### Description
OpenAI Responses provider implementation. Translates `ChatRequest` structures into OpenAI Responses payloads and handles both synchronous and streaming flows, including tool invocation support.

### Methods

#### `new(transport: DynHttpTransport, api_key: impl Into<String>) -> Self`
Creates a provider targeting the default `https://api.openai.com` endpoint.

##### Example
```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "key");
assert_eq!(provider.name(), "openai_responses");
```

#### `with_base_url(self, base_url: impl Into<String>) -> Self`
Overrides the base URL, useful for proxies or gateways.

##### Example
```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "key")
    .with_base_url("https://proxy.local");
assert_eq!(provider.name(), "openai_responses");
```

#### `with_organization(self, organization: impl Into<String>) -> Self`
Sets the optional `OpenAI-Organization` header.

##### Example
```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "key").with_organization("org_123");
assert_eq!(provider.name(), "openai_responses");
```

#### `with_project(self, project: impl Into<String>) -> Self`
Sets the optional `OpenAI-Project` header.

##### Example
```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "key").with_project("proj_alpha");
assert_eq!(provider.name(), "openai_responses");
```

#### `with_default_model(self, model: impl Into<String>) -> Self`
Configures a default model when requests omit one.

##### Example
```rust
let transport = default_dyn_transport().expect("transport");
let provider = OpenAiResponsesProvider::new(transport, "key")
    .with_default_model("gpt-4.1-mini");
assert_eq!(provider.name(), "openai_responses");
```

#### `from_model_config(config: &ModelConfig, transport: DynHttpTransport) -> Result<Self, LLMError>`
Constructs a provider from a `crate::config::ModelConfig`. This method is used by the macro-driven provider registration system to build providers from declarative configuration.

##### Arguments
- `config` (ModelConfig) - The model configuration containing credentials and settings
- `transport` (DynHttpTransport) - The HTTP transport implementation to use

##### Errors
- `LLMError::Auth` - When credentials are missing or invalid.
```

--------------------------------

### LLMClientLike Stream Chat Method Example (Rust)

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/client/trait.LLMClientLike_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Illustrates the usage of the 'stream_chat' method from the LLMClientLike trait. This example shows how to initiate a streaming chat request and asserts that the operation is successful. It utilizes a dummy handle and an empty ChatRequest.

```rust
let client = build_client();
let obj: &dyn LLMClientLike = &client;
let result = obj
    .stream_chat("dummy", ChatRequest {
        messages: Vec::new(),
        options: Default::default(),
        tools: Vec::new(),
        tool_choice: None,
        response_format: None,
        metadata: None,
    })
    .await;
assert!(result.is_ok());
```

--------------------------------

### Create and Initialize RequestPatch in Rust

Source: https://docs.rs/kotoba-llm/latest/kotoba_llm/config/struct.RequestPatch

Demonstrates how to create and initialize a RequestPatch instance in Rust. This example shows setting optional fields for URL, body, headers, and fields to be removed, illustrating a typical use case for runtime request modification.

```rust
let patch = RequestPatch {
    url: Some("https://proxy.internal/chat".to_string()),
    body: Some(json!({ "metadata": { "tenant": "blue" } })),
    headers: Some(HashMap::from([
        ("x-trace".to_string(), Some("123".to_string())),
        ("x-remove".to_string(), None),
    ])),
    remove_fields: Some(vec!["options.max_tokens".to_string()]),
};
```
Tokens:
Raw