LLM

https://docs.rs/llm/1.3.6/llm
Admin
LLM (Rust LLM) is a unified interface for interacting with Large Language Model providers, offering...
Tokens:546,734
Snippets:6,940
Trust Score:9.7
Update:10 months ago
Show doc for...
Tokens:
Raw
### LLMRegistry REST API Serving

Source: https://docs.rs/llm/1.3.6/llm/chain/struct.LLMRegistry_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Details on how to start a REST API server using the `serve` method of LLMRegistry.

```APIDOC
## POST /llm/serve

### Description
Starts a REST API server to serve registered LLM backends.

### Method
`POST`

### Endpoint
`/llm/serve`

### Parameters
#### Query Parameters
*   **addr** (string) - Required - The network address (e.g., `0.0.0.0:8080`) to bind the server to.

#### Request Body
(This endpoint does not require a request body, it uses the `LLMRegistry` instance it's called on.)

### Request Example
*(No request body needed)*

### Response
#### Success Response (200)
*   **message** (string) - Indicates the server has started successfully.

#### Response Example
```json
{
  "message": "LLM server started on 0.0.0.0:8080"
}
```

#### Error Response (e.g., 500)
*   **error** (string) - Description of the error encountered during server startup.
```

--------------------------------

### Example: Setting Up LLM with OpenAI Backend and Sliding Memory Instance

Source: https://docs.rs/llm/1.3.6/llm/builder/struct.LLMBuilder

Shows how to create an `LLMBuilder` using the OpenAI backend and directly provide a `SlidingWindowMemory` instance.

```rust
use llm::builder::{LLMBuilder, LLMBackend};
use llm::memory::SlidingWindowMemory;

let memory = SlidingWindowMemory::new(10);
let builder = LLMBuilder::new()
    .backend(LLMBackend::OpenAI)
    .sliding_memory(memory);
```

--------------------------------

### Example: Setting Up LLM with OpenAI Backend and Sliding Window Memory

Source: https://docs.rs/llm/1.3.6/llm/builder/struct.LLMBuilder

Demonstrates how to initialize an `LLMBuilder` with the OpenAI backend and configure a sliding window memory with a capacity of 10 messages.

```rust
use llm::builder::{LLMBuilder, LLMBackend};
use llm::memory::SlidingWindowMemory;

let memory = SlidingWindowMemory::new(10);
let builder = LLMBuilder::new()
    .backend(LLMBackend::OpenAI)
    .memory(memory);
```

--------------------------------

### Example: Setting Up LLM with OpenAI Backend, Window Size, and Trim Strategy

Source: https://docs.rs/llm/1.3.6/llm/builder/struct.LLMBuilder

Demonstrates configuring an `LLMBuilder` with the OpenAI backend, a sliding window size of 5, and a `Summarize` trim strategy for handling memory overflow.

```rust
use llm::builder::{LLMBuilder, LLMBackend};
use llm::memory::TrimStrategy;

let builder = LLMBuilder::new()
    .backend(LLMBackend::OpenAI)
    .sliding_window_with_strategy(5, TrimStrategy::Summarize);
```

--------------------------------

### Run LLM Server on Address (Rust)

Source: https://docs.rs/llm/1.3.6/llm/api/struct.Server

Asynchronously starts the server, making it listen for incoming requests on a specified network address. Returns a `Result` indicating success or an `LLMError` if the server fails to start.

```rust
pub async fn run(self, addr: &str) -> Result<(), LLMError>
```

--------------------------------

### Example: Using ResilientLLM Wrapper for OpenAI

Source: https://docs.rs/llm/1.3.6/llm/resilient_llm/index

Demonstrates how to configure and use the ResilientLLM wrapper to interact with an OpenAI LLM. It sets up the LLM backend, API key, model, and resilience parameters like attempts and backoff duration. The example shows a simple chat interaction and prints the response.

```rust
use llm::builder::{LLMBackend, LLMBuilder};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let llm = LLMBuilder::new()
        .backend(LLMBackend::OpenAI)
        .api_key(std::env::var("OPENAI_API_KEY").unwrap_or_default())
        .model("gpt-4o-mini")
        .resilient(true)
        .resilient_attempts(3)
        .resilient_backoff(200, 2_000)
        .build()?;

    let msgs = [
        llm::chat::ChatMessage::user().content("Say hi succinctly").build(),
    ];
    let resp = llm.chat(&msgs).await?;
    println!("{}", resp);
    Ok(())
}
```

--------------------------------

### Example: Setting Up LLM with OpenAI Backend and Sliding Window Size

Source: https://docs.rs/llm/1.3.6/llm/builder/struct.LLMBuilder

Illustrates initializing an `LLMBuilder` with the OpenAI backend and setting a sliding window memory that keeps the last 5 messages.

```rust
use llm::builder::{LLMBuilder, LLMBackend};

let builder = LLMBuilder::new()
    .backend(LLMBackend::OpenAI)
    .sliding_window_memory(5); // Keep last 5 messages
```

--------------------------------

### SecretStore Initialization and Management

Source: https://docs.rs/llm/1.3.6/llm/secret_store/struct.SecretStore_search=std%3A%3Avec

Provides methods for creating, setting, getting, and deleting secrets, as well as managing default LLM providers.

```APIDOC
## SecretStore

A secure storage for API keys and other sensitive information. Provides functionality to store, retrieve, and manage secrets in a JSON file located in the user’s home directory.

### `new()`

**Description**: Creates a new SecretStore instance. Initializes the store with the default path (~/.llm/secrets.json) and loads any existing secrets from the file.

**Method**: `pub fn new() -> Result<Self>`

**Returns**: `io::Result<Self>` - A new SecretStore instance or an IO error.

### `set(key: &str, value: &str)`

**Description**: Sets a secret value for the given key.

**Method**: `pub fn set(&mut self, key: &str, value: &str) -> Result<()>`

**Arguments**:
- **key** (*&str*) - The key to store the secret under.
- **value** (*&str*) - The secret value to store.

**Returns**: `io::Result<()>` - Success or an IO error.

### `get(key: &str)`

**Description**: Retrieves a secret value for the given key.

**Method**: `pub fn get(&self, key: &str) -> Option<&String>`

**Arguments**:
- **key** (*&str*) - The key to look up.

**Returns**: `Option<&String>` - The secret value if found, or None.

### `delete(key: &str)`

**Description**: Deletes a secret with the given key.

**Method**: `pub fn delete(&mut self, key: &str) -> Result<()>`

**Arguments**:
- **key** (*&str*) - The key to delete.

**Returns**: `io::Result<()>` - Success or an IO error.

### `set_default_provider(provider: &str)`

**Description**: Sets the default provider for LLM interactions.

**Method**: `pub fn set_default_provider(&mut self, provider: &str) -> Result<()>`

**Arguments**:
- **provider** (*&str*) - The provider string in format “provider:model”.

**Returns**: `io::Result<()>` - Success or an IO error.

### `get_default_provider()`

**Description**: Retrieves the default provider for LLM interactions.

**Method**: `pub fn get_default_provider(&self) -> Option<&String>`

**Returns**: `Option<&String>` - The default provider if set, or None.

### `delete_default_provider()`

**Description**: Deletes the default provider setting.

**Method**: `pub fn delete_default_provider(&mut self) -> Result<()>`

**Returns**: `io::Result<()>` - Success or an IO error.

```

--------------------------------

### GET /tools

Source: https://docs.rs/llm/1.3.6/llm/backends/xai/struct.XAI_search=std%3A%3Avec

Retrieves the list of available tools.

```APIDOC
## GET /tools

### Description
Retrieves a list of tools supported by the LLM provider.

### Method
GET

### Endpoint
/tools

### Response
#### Success Response (200)
- **tools** (Option<array[Tool]>) - An optional list of available tools.

#### Response Example
```json
[
  {"name": "calculator", "description": "Performs calculations"}
]
```
```

--------------------------------

### Google Gemini API Chat Example in Rust

Source: https://docs.rs/llm/1.3.6/llm/backends/google/index

Demonstrates how to use the Google Gemini API client for chat functionality. This example initializes a Google client with specific parameters like API key, max tokens, and temperature, then sends a user message and prints the response. It requires the 'llm' crate and 'tokio' for asynchronous operations.

```rust
use llm::backends::google::Google;
use llm::chat::{ChatMessage, ChatRole, ChatProvider};

#[tokio::main]
async fn main() {
let client = Google::new(
    "your-api-key",
    None, // Use default model
    Some(1000), // Max tokens
    Some(0.7), // Temperature
    None, // Default timeout
    None, // No system prompt
    None, // Default top_p
    None, // Default top_k
    None, // No JSON schema
    None, // No tools
);

let messages = vec![
    ChatMessage::user().content("Hello!").build()
];

let response = client.chat(&messages).await.unwrap();
println!("{response}");
}
```

--------------------------------

### LLMRegistry Constructor and Methods

Source: https://docs.rs/llm/1.3.6/llm/chain/struct.LLMRegistry_search=

Provides core functionalities for the LLMRegistry, including creating a new instance, inserting LLM backends, retrieving backends by ID, and starting a REST API server to serve LLMs. The `serve` method is asynchronous and returns a Result.

```rust
pub fn new() -> Self

pub fn insert(&mut self, id: impl Into<String>, llm: Box<dyn LLMProvider>)

pub fn get(&self, id: &str) -> Option<&dyn LLMProvider>

pub async fn serve(self, addr: impl Into<String>) -> Result<(), LLMError>
```

--------------------------------

### GET /models

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Retrieves the list of available models.

```APIDOC
## GET /models

### Description
Asynchronously retrieves the list of available models ID’s from the provider.

### Method
GET

### Endpoint
/models

### Parameters
#### Query Parameters
- **request** (ModelListRequest) - Optional - Filters or parameters for listing models.

### Request Example
(No specific request body, but optional query parameters can be used)

### Response
#### Success Response (200)
- **model_list** (Box<dyn ModelListResponse>) - The response containing the list of models.

#### Response Example
(Example response depends on the ModelListResponse implementation)
```json
{
  "models": [{"id": "gpt-3.5-turbo"}, {"id": "gpt-4"}]
}
```
```

--------------------------------

### OpenRouter Constructor Functions (Rust)

Source: https://docs.rs/llm/1.3.6/llm/backends/openrouter/type.OpenRouter_search=

Provides two primary methods for creating an OpenRouter instance: `with_config` for direct configuration and `new` within the OpenAICompatibleProvider trait implementation, both allowing detailed setup of the provider.

```rust
impl OpenRouter {
    pub fn with_config(
        api_key: impl Into<String>,
        base_url: Option<String>,
        model: Option<String>,
        max_tokens: Option<u32>,
        temperature: Option<f32>,
        timeout_seconds: Option<u64>,
        system: Option<String>,
        top_p: Option<f32>,
        top_k: Option<u32>,
        tools: Option<Vec<Tool>>,
        tool_choice: Option<ToolChoice>,
        extra_body: Option<Value>,
        _embedding_encoding_format: Option<String>,
        _embedding_dimensions: Option<u32>,
        reasoning_effort: Option<String>,
        json_schema: Option<StructuredOutputFormat>,
        parallel_tool_calls: Option<bool>,
        normalize_response: Option<bool>,
    ) -> Self
}

impl<T: OpenAIProviderConfig> OpenAICompatibleProvider<T> {
    pub fn new(
        api_key: impl Into<String>,
        base_url: Option<String>,
        model: Option<String>,
        max_tokens: Option<u32>,
        temperature: Option<f32>,
        timeout_seconds: Option<u64>,
        system: Option<String>,
        top_p: Option<f32>,
        top_k: Option<u32>,
        tools: Option<Vec<Tool>>,
        tool_choice: Option<ToolChoice>,
        reasoning_effort: Option<String>,
        json_schema: Option<StructuredOutputFormat>,
        voice: Option<String>,
        extra_body: Option<Value>,
        parallel_tool_calls: Option<bool>,
        normalize_response: Option<bool>,
        embedding_encoding_format: Option<String>,
        embedding_dimensions: Option<u32>,
    ) -> Self
}
```

--------------------------------

### Get Type ID (Blanket Implementation)

Source: https://docs.rs/llm/1.3.6/llm/backends/xai/struct.XAI_search=u32+-%3E+bool

Gets the `TypeId` of `self`. This is a blanket implementation for any type `T` that is 'static and sized.

```rust
fn type_id(&self) -> TypeId
where T: 'static + ?Sized
```

--------------------------------

### OpenAI Client Initialization

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Demonstrates how to create a new OpenAI client instance with various configuration options.

```APIDOC
## Function `new`

### Description
Creates a new OpenAI client with the specified configuration.

### Method
`pub fn new(
    api_key: impl Into<String>,
    base_url: Option<String>,
    model: Option<String>,
    max_tokens: Option<u32>,
    temperature: Option<f32>,
    timeout_seconds: Option<u64>,
    system: Option<String>,
    top_p: Option<f32>,
    top_k: Option<u32>,
    embedding_encoding_format: Option<String>,
    embedding_dimensions: Option<u32>,
    tools: Option<Vec<Tool>>,
    tool_choice: Option<ToolChoice>,
    normalize_response: Option<bool>,
    reasoning_effort: Option<String>,
    json_schema: Option<StructuredOutputFormat>,
    voice: Option<String>,
    extra_body: Option<Value>,
    enable_web_search: Option<bool>,
    web_search_context_size: Option<String>,
    web_search_user_location_type: Option<String>,
    web_search_user_location_approximate_country: Option<String>,
    web_search_user_location_approximate_city: Option<String>,
    web_search_user_location_approximate_region: Option<String>,
) -> Result<Self, LLMError>

### Arguments
- **api_key** (`impl Into<String>`) - OpenAI API key.
- **base_url** (`Option<String>`) - Optional base URL for the API.
- **model** (`Option<String>`) - Model to use (defaults to “gpt-3.5-turbo”).
- **max_tokens** (`Option<u32>`) - Maximum tokens to generate.
- **temperature** (`Option<f32>`) - Sampling temperature.
- **timeout_seconds** (`Option<u64>`) - Request timeout in seconds.
- **system** (`Option<String>`) - System prompt.
- **top_p** (`Option<f32>`) - Top-p sampling parameter.
- **top_k** (`Option<u32>`) - Top-k sampling parameter.
- **embedding_encoding_format** (`Option<String>`) - Format for embedding outputs.
- **embedding_dimensions** (`Option<u32>`) - Dimensions for embedding vectors.
- **tools** (`Option<Vec<Tool>>`) - Function tools that the model can use.
- **tool_choice** (`Option<ToolChoice>`) - Determines how the model uses tools.
- **normalize_response** (`Option<bool>`) - Whether to normalize the response.
- **reasoning_effort** (`Option<String>`) - Reasoning effort level.
- **json_schema** (`Option<StructuredOutputFormat>`) - JSON schema for structured output.
- **voice** (`Option<String>`) - Voice configuration for the model.
- **extra_body** (`Option<Value>`) - Additional fields for the request body.
- **enable_web_search** (`Option<bool>`) - Enables or disables web search functionality.
- **web_search_context_size** (`Option<String>`) - Specifies the context size for web search.
- **web_search_user_location_type** (`Option<String>`) - Defines the type of user location for web search.
- **web_search_user_location_approximate_country** (`Option<String>`) - Approximate country of the user for web search.
- **web_search_user_location_approximate_city** (`Option<String>`) - Approximate city of the user for web search.
- **web_search_user_location_approximate_region** (`Option<String>`) - Approximate region of the user for web search.

### Returns
- `Result<Self, LLMError>` - A new OpenAI client instance or an error.
```

--------------------------------

### Get Type ID (Blanket Implementation)

Source: https://docs.rs/llm/1.3.6/llm/backends/xai/struct.XAI

Gets the `TypeId` of `self`. This is a blanket implementation provided for all types that are 'static and sized.

```rust
fn type_id(&self) -> TypeId
where Self: 'static + ?Sized,
```

--------------------------------

### MultiChainStepBuilder Initialization

Source: https://docs.rs/llm/1.3.6/llm/chain/struct.MultiChainStepBuilder

Creates a new instance of MultiChainStepBuilder, requiring a MultiChainStepMode. This is the starting point for configuring a chain step.

```rust
pub fn new(mode: MultiChainStepMode) -> Self
```

--------------------------------

### OpenAI Client Initialization

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI

Demonstrates how to create a new OpenAI client instance with various configuration options.

```APIDOC
## POST /openai/client

### Description
Creates a new OpenAI client with the specified configuration.

### Method
POST

### Endpoint
`/openai/client`

### Parameters
#### Request Body
- **`api_key`** (String) - Required - OpenAI API key.
- **`base_url`** (Option<String>) - Optional - Base URL for the OpenAI API.
- **`model`** (Option<String>) - Optional - Model to use (defaults to "gpt-3.5-turbo").
- **`max_tokens`** (Option<u32>) - Optional - Maximum tokens to generate.
- **`temperature`** (Option<f32>) - Optional - Sampling temperature.
- **`timeout_seconds`** (Option<u64>) - Optional - Request timeout in seconds.
- **`system`** (Option<String>) - Optional - System prompt.
- **`top_p`** (Option<f32>) - Optional - Top-p sampling parameter.
- **`top_k`** (Option<u32>) - Optional - Top-k sampling parameter.
- **`embedding_encoding_format`** (Option<String>) - Optional - Format for embedding outputs.
- **`embedding_dimensions`** (Option<u32>) - Optional - Dimensions for embedding vectors.
- **`tools`** (Option<Vec<Tool>>) - Optional - Function tools that the model can use.
- **`tool_choice`** (Option<ToolChoice>) - Optional - Determines how the model uses tools.
- **`normalize_response`** (Option<bool>) - Optional - Whether to normalize the response.
- **`reasoning_effort`** (Option<String>) - Optional - Reasoning effort level.
- **`json_schema`** (Option<StructuredOutputFormat>) - Optional - JSON schema for structured output.
- **`voice`** (Option<String>) - Optional - Voice configuration.
- **`extra_body`** (Option<Value>) - Optional - Additional fields to include in the request body.
- **`enable_web_search`** (Option<bool>) - Optional - Whether to enable web search.
- **`web_search_context_size`** (Option<String>) - Optional - Context size for web search.
- **`web_search_user_location_type`** (Option<String>) - Optional - User location type for web search.
- **`web_search_user_location_approximate_country`** (Option<String>) - Optional - Approximate country for web search.
- **`web_search_user_location_approximate_city`** (Option<String>) - Optional - Approximate city for web search.
- **`web_search_user_location_approximate_region`** (Option<String>) - Optional - Approximate region for web search.

### Response
#### Success Response (200)
- **`client`** (OpenAI) - The created OpenAI client instance.
- **`error`** (LLMError) - An error if client creation failed.
```

--------------------------------

### Get Type ID (Rust Blanket Implementation)

Source: https://docs.rs/llm/1.3.6/llm/backends/azure_openai/struct.AzureOpenAI

Gets the `TypeId` of a given type. This is a blanket implementation for any type that is 'static and Sized.

```rust
fn type_id(&self) -> TypeId
where T: 'static + ?Sized,
```

--------------------------------

### AgentBuilder Initialization (Rust)

Source: https://docs.rs/llm/1.3.6/llm/agent/builder/struct.AgentBuilder_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Creates a new instance of AgentBuilder. This is the starting point for configuring a new LLM agent.

```rust
pub fn new() -> Self
```

--------------------------------

### Get Type ID (Rust)

Source: https://docs.rs/llm/1.3.6/llm/backends/phind/struct.Phind_search=

Gets the `TypeId` of `self`. This is a common method found in blanket implementations for types that are 'static.

```rust
impl<T> Any for T
where
    T: 'static + ?Sized,
{
    fn type_id(&self) -> TypeId {
        // Implementation details...
    }
}
```

--------------------------------

### OpenAI Client Initialization

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI_search=

Demonstrates how to create a new instance of the OpenAI client with various configuration options.

```APIDOC
## Function `new`

### Description
Creates a new OpenAI client with the specified configuration.

### Method
`pub fn new(
    api_key: impl Into<String>,
    base_url: Option<String>,
    model: Option<String>,
    max_tokens: Option<u32>,
    temperature: Option<f32>,
    timeout_seconds: Option<u64>,
    system: Option<String>,
    top_p: Option<f32>,
    top_k: Option<u32>,
    embedding_encoding_format: Option<String>,
    embedding_dimensions: Option<u32>,
    tools: Option<Vec<Tool>>,
    tool_choice: Option<ToolChoice>,
    normalize_response: Option<bool>,
    reasoning_effort: Option<String>,
    json_schema: Option<StructuredOutputFormat>,
    voice: Option<String>,
    extra_body: Option<Value>,
    enable_web_search: Option<bool>,
    web_search_context_size: Option<String>,
    web_search_user_location_type: Option<String>,
    web_search_user_location_approximate_country: Option<String>,
    web_search_user_location_approximate_city: Option<String>,
    web_search_user_location_approximate_region: Option<String>,
) -> Result<Self, LLMError>

### Arguments
- **api_key** (impl Into<String>) - OpenAI API key.
- **base_url** (Option<String>) - Optional base URL for the API.
- **model** (Option<String>) - Model to use (defaults to “gpt-3.5-turbo”).
- **max_tokens** (Option<u32>) - Maximum tokens to generate.
- **temperature** (Option<f32>) - Sampling temperature.
- **timeout_seconds** (Option<u64>) - Request timeout in seconds.
- **system** (Option<String>) - System prompt.
- **top_p** (Option<f32>) - Top-p sampling parameter.
- **top_k** (Option<u32>) - Top-k sampling parameter.
- **embedding_encoding_format** (Option<String>) - Format for embedding outputs.
- **embedding_dimensions** (Option<u32>) - Dimensions for embedding vectors.
- **tools** (Option<Vec<Tool>>) - Function tools that the model can use.
- **tool_choice** (Option<ToolChoice>) - Determines how the model uses tools.
- **normalize_response** (Option<bool>) - Whether to normalize the response.
- **reasoning_effort** (Option<String>) - Reasoning effort level.
- **json_schema** (Option<StructuredOutputFormat>) - JSON schema for structured output.
- **voice** (Option<String>) - Voice configuration.
- **extra_body** (Option<Value>) - Additional fields for the request body.
- **enable_web_search** (Option<bool>) - Enable or disable web search.
- **web_search_context_size** (Option<String>) - Context size for web search.
- **web_search_user_location_type** (Option<String>) - User location type for web search.
- **web_search_user_location_approximate_country** (Option<String>) - Approximate country for web search.
- **web_search_user_location_approximate_city** (Option<String>) - Approximate city for web search.
- **web_search_user_location_approximate_region** (Option<String>) - Approximate region for web search.

### Returns
A `Result` containing the initialized `OpenAI` client or an `LLMError`.
```

--------------------------------

### Get Type ID in Rust (Blanket Implementation)

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI_search=u32+-%3E+bool

Gets the `TypeId` of a given type. This is a blanket implementation that applies to any type implementing the `'static` and `?Sized` bounds.

```rust
impl<T> Any for T
where T: 'static + ?Sized,
fn type_id(&self) -> TypeId
{
```

--------------------------------

### ElevenLabs Initialization

Source: https://docs.rs/llm/1.3.6/llm/backends/elevenlabs/struct.ElevenLabs_search=

Provides details on how to initialize the ElevenLabs backend with necessary credentials and configuration.

```APIDOC
## POST /llm/backends/elevenlabs

### Description
Initializes the ElevenLabs speech-to-text backend.

### Method
POST

### Endpoint
/llm/backends/elevenlabs

### Parameters
#### Request Body
- **api_key** (String) - Required - API key for ElevenLabs authentication.
- **model_id** (String) - Required - Model identifier for speech-to-text.
- **base_url** (String) - Required - Base URL for API requests.
- **timeout_seconds** (Number) - Optional - Timeout duration in seconds.
- **voice** (String) - Optional - Voice identifier to use for speech synthesis.

### Request Example
```json
{
  "api_key": "YOUR_ELEVENLABS_API_KEY",
  "model_id": "eleven_english_v2",
  "base_url": "https://api.elevenlabs.io",
  "timeout_seconds": 60,
  "voice": "Adam"
}
```

### Response
#### Success Response (201)
- **message** (String) - Indicates successful initialization.

#### Response Example
```json
{
  "message": "ElevenLabs backend initialized successfully."
}
```
```

--------------------------------

### Implementations of OpenAIProviderConfig

Source: https://docs.rs/llm/1.3.6/llm/providers/openai_compatible/trait.OpenAIProviderConfig_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Examples of how different providers implement the OpenAIProviderConfig trait, specifying their unique configurations and feature support.

```APIDOC
## Implementations of OpenAIProviderConfig

### `impl OpenAIProviderConfig for CohereConfig`
*   **PROVIDER_NAME**: `"Cohere"`
*   **DEFAULT_BASE_URL**: `"https://api.cohere.ai/compatibility/v1/"`
*   **DEFAULT_MODEL**: `"command-r7b-12-2024"`
*   **SUPPORTS_STRUCTURED_OUTPUT**: `true`

### `impl OpenAIProviderConfig for GroqConfig`
*   **PROVIDER_NAME**: `"Groq"`
*   **DEFAULT_BASE_URL**: `"https://api.groq.com/openai/v1/"`
*   **DEFAULT_MODEL**: `"llama3-8b-8192"`
*   **SUPPORTS_STRUCTURED_OUTPUT**: `true`

### `impl OpenAIProviderConfig for HuggingFaceConfig`
*   **PROVIDER_NAME**: `"HuggingFace Inference Providers"`
*   **DEFAULT_BASE_URL**: `"https://router.huggingface.co/v1/"`
*   **DEFAULT_MODEL**: `"openai/gpt-oss-20b"`
*   **SUPPORTS_STRUCTURED_OUTPUT**: `true`

### `impl OpenAIProviderConfig for MistralConfig`
*   **PROVIDER_NAME**: `"Mistral"`
*   **DEFAULT_BASE_URL**: `"https://api.mistral.ai/v1/"`
*   **DEFAULT_MODEL**: `"mistral-small-latest"`
*   **SUPPORTS_STRUCTURED_OUTPUT**: `true`
*   **SUPPORTS_PARALLEL_TOOL_CALLS**: `true`

### `impl OpenAIProviderConfig for OpenRouterConfig`
*   **PROVIDER_NAME**: `"OpenRouter"`
*   **DEFAULT_BASE_URL**: `"https://openrouter.ai/api/v1/"`
*   **DEFAULT_MODEL**: `"moonshotai/kimi-k2:free"`
*   **SUPPORTS_STRUCTURED_OUTPUT**: `true`
```

--------------------------------

### Client Initialization

Source: https://docs.rs/llm/1.3.6/llm/providers/openai_compatible/struct.OpenAICompatibleProvider_search=u32+-%3E+bool

Provides methods to create a new client for the OpenAI compatible provider with various configuration options.

```APIDOC
## `with_config`

### Description
Creates a new HuggingFace client with the specified configuration.

### Method
`pub fn with_config`

### Parameters
- **api_key** (impl Into<String>) - Required - The API key for authentication.
- **base_url** (Option<String>) - Optional - The base URL for the API endpoint.
- **model** (Option<String>) - Optional - The model to use for chat completions.
- **max_tokens** (Option<u32>) - Optional - The maximum number of tokens to generate in the response.
- **temperature** (Option<f32>) - Optional - Controls randomness in the output. Lower values make output more deterministic.
- **timeout_seconds** (Option<u64>) - Optional - The timeout for API requests in seconds.
- **system** (Option<String>) - Optional - A system message to guide the model's behavior.
- **top_p** (Option<f32>) - Optional - Nucleus sampling parameter. An alternative to temperature sampling.
- **top_k** (Option<u32>) - Optional - Top-k sampling parameter.
- **tools** (Option<Vec<Tool>>) - Optional - A list of tools the model can use.
- **tool_choice** (Option<ToolChoice>) - Optional - Specifies how to use the provided tools.
- **extra_body** (Option<Value>) - Optional - Additional parameters to include in the request body.
- **_embedding_encoding_format** (Option<String>) - Optional - Encoding format for embeddings (internal use).
- **_embedding_dimensions** (Option<u32>) - Optional - Dimensions for embeddings (internal use).
- **reasoning_effort** (Option<String>) - Optional - Specifies the effort level for reasoning.
- **json_schema** (Option<StructuredOutputFormat>) - Optional - Defines the schema for structured JSON output.
- **parallel_tool_calls** (Option<bool>) - Optional - Whether to allow parallel tool calls.
- **normalize_response** (Option<bool>) - Optional - Whether to normalize the response format.

### Response
- **Self** - The created HuggingFace client instance.
```

```APIDOC
## `new`

### Description
Initializes a new `OpenAICompatibleProvider` instance.

### Method
`pub fn new`

### Parameters
- **api_key** (impl Into<String>) - Required - The API key for authentication.
- **base_url** (Option<String>) - Optional - The base URL for the API endpoint.
- **model** (Option<String>) - Optional - The model to use for chat completions.
- **max_tokens** (Option<u32>) - Optional - The maximum number of tokens to generate in the response.
- **temperature** (Option<f32>) - Optional - Controls randomness in the output. Lower values make output more deterministic.
- **timeout_seconds** (Option<u64>) - Optional - The timeout for API requests in seconds.
- **system** (Option<String>) - Optional - A system message to guide the model's behavior.
- **top_p** (Option<f32>) - Optional - Nucleus sampling parameter. An alternative to temperature sampling.
- **top_k** (Option<u32>) - Optional - Top-k sampling parameter.
- **tools** (Option<Vec<Tool>>) - Optional - A list of tools the model can use.
- **tool_choice** (Option<ToolChoice>) - Optional - Specifies how to use the provided tools.
- **reasoning_effort** (Option<String>) - Optional - Specifies the effort level for reasoning.
- **json_schema** (Option<StructuredOutputFormat>) - Optional - Defines the schema for structured JSON output.
- **voice** (Option<String>) - Optional - Specifies a voice for text-to-speech output.
- **extra_body** (Option<Value>) - Optional - Additional parameters to include in the request body.
- **parallel_tool_calls** (Option<bool>) - Optional - Whether to allow parallel tool calls.
- **normalize_response** (Option<bool>) - Optional - Whether to normalize the response format.
- **embedding_encoding_format** (Option<String>) - Optional - Encoding format for embeddings.
- **embedding_dimensions** (Option<u32>) - Optional - Dimensions for embeddings.

### Response
- **Self** - The initialized `OpenAICompatibleProvider` instance.
```

--------------------------------

### Get Type ID (Rust)

Source: https://docs.rs/llm/1.3.6/llm/backends/phind/struct.Phind_search=std%3A%3Avec

Gets the `TypeId` of the current type. This is a common method provided by the `Any` trait, useful for runtime type introspection.

```rust
/// Gets the `TypeId` of `self`.
///
/// # Returns
///
/// The `TypeId` of the type implementing the `Any` trait.
fn type_id(&self) -> TypeId
```

--------------------------------

### Rust: Example Usage of StructuredOutputFormat

Source: https://docs.rs/llm/1.3.6/llm/chat/struct.StructuredOutputFormat_search=

Demonstrates how to create and deserialize a `StructuredOutputFormat` instance in Rust using `serde_json`. This example shows defining a schema for a 'Student' object and verifying the deserialized fields.

```rust
use llm::chat::StructuredOutputFormat;
use serde_json::json;

let response_format = r#"
    {
        "name": "Student",
        "description": "A student object",
        "schema": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "string"
                },
                "age": {
                    "type": "integer"
                },
                "is_student": {
                    "type": "boolean"
                }
            },
            "required": ["name", "age", "is_student"]
        }
    }
"#;
let structured_output: StructuredOutputFormat = serde_json::from_str(response_format).unwrap();
assert_eq!(structured_output.name, "Student");
assert_eq!(structured_output.description, Some("A student object".to_string()));
```

--------------------------------

### OpenAI Initialization

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI_search=u32+-%3E+bool

Demonstrates how to create a new instance of the OpenAI client with various configuration options.

```APIDOC
## `OpenAI::new`

### Description
Creates a new OpenAI client with the specified configuration.

### Method
`pub fn new(
    api_key: impl Into<String>,
    base_url: Option<String>,
    model: Option<String>,
    max_tokens: Option<u32>,
    temperature: Option<f32>,
    timeout_seconds: Option<u64>,
    system: Option<String>,
    top_p: Option<f32>,
    top_k: Option<u32>,
    embedding_encoding_format: Option<String>,
    embedding_dimensions: Option<u32>,
    tools: Option<Vec<Tool>>,
    tool_choice: Option<ToolChoice>,
    normalize_response: Option<bool>,
    reasoning_effort: Option<String>,
    json_schema: Option<StructuredOutputFormat>,
    voice: Option<String>,
    extra_body: Option<Value>,
    enable_web_search: Option<bool>,
    web_search_context_size: Option<String>,
    web_search_user_location_type: Option<String>,
    web_search_user_location_approximate_country: Option<String>,
    web_search_user_location_approximate_city: Option<String>,
    web_search_user_location_approximate_region: Option<String>,
) -> Result<Self, LLMError>

### Parameters
#### Arguments
- **api_key** (impl Into<String>) - OpenAI API key
- **base_url** (Option<String>) - Optional base URL for the OpenAI API.
- **model** (Option<String>) - Model to use (defaults to “gpt-3.5-turbo”).
- **max_tokens** (Option<u32>) - Maximum tokens to generate.
- **temperature** (Option<f32>) - Sampling temperature.
- **timeout_seconds** (Option<u64>) - Request timeout in seconds.
- **system** (Option<String>) - System prompt.
- **top_p** (Option<f32>) - Top-p sampling parameter.
- **top_k** (Option<u32>) - Top-k sampling parameter.
- **embedding_encoding_format** (Option<String>) - Format for embedding outputs.
- **embedding_dimensions** (Option<u32>) - Dimensions for embedding vectors.
- **tools** (Option<Vec<Tool>>) - Function tools that the model can use.
- **tool_choice** (Option<ToolChoice>) - Determines how the model uses tools.
- **normalize_response** (Option<bool>) - Whether to normalize the response.
- **reasoning_effort** (Option<String>) - Reasoning effort level.
- **json_schema** (Option<StructuredOutputFormat>) - JSON schema for structured output.
- **voice** (Option<String>) - Voice configuration.
- **extra_body** (Option<Value>) - Additional fields for the request body.
- **enable_web_search** (Option<bool>) - Whether to enable web search.
- **web_search_context_size** (Option<String>) - The context size for web search.
- **web_search_user_location_type** (Option<String>) - The type of user location for web search.
- **web_search_user_location_approximate_country** (Option<String>) - The approximate country of the user for web search.
- **web_search_user_location_approximate_city** (Option<String>) - The approximate city of the user for web search.
- **web_search_user_location_approximate_region** (Option<String>) - The approximate region of the user for web search.

### Returns
- Result<Self, LLMError> - A new OpenAI client instance or an error.
```

--------------------------------

### Get Type ID (Rust - Blanket Implementation)

Source: https://docs.rs/llm/1.3.6/llm/backends/google/struct.Google_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Gets the `TypeId` of the current object. This is a blanket implementation for any type T that is 'static.

```rust
fn type_id(&self) -> TypeId
where
    Self: 'static + ?Sized,
```

--------------------------------

### LLMRegistry Serve Function

Source: https://docs.rs/llm/1.3.6/llm/chain/struct.LLMRegistry

Starts a REST API server for the LLMRegistry on a specified network address. This asynchronous function consumes the registry and returns a Result indicating success or an LLMError.

```rust
pub async fn serve(self, addr: impl Into<String>) -> Result<(), LLMError>
```

--------------------------------

### OpenAI Client Initialization

Source: https://docs.rs/llm/1.3.6/llm/backends/openai/struct.OpenAI_search=std%3A%3Avec

Information on how to create and configure an OpenAI client instance.

```APIDOC
## POST /websites/rs_llm_1_3_6_llm/llm::backends::openai

### Description
Initializes a new OpenAI client with the specified configuration parameters.

### Method
POST

### Endpoint
`/websites/rs_llm_1_3_6_llm/llm::backends::openai::new`

### Parameters
#### Query Parameters
- **api_key** (String) - Required - OpenAI API key.
- **base_url** (Option<String>) - Optional - The base URL for the OpenAI API. Defaults to `None`.
- **model** (Option<String>) - Optional - The model to use for the API requests. Defaults to `"gpt-3.5-turbo"`.
- **max_tokens** (Option<u32>) - Optional - Maximum number of tokens to generate in the response.
- **temperature** (Option<f32>) - Optional - Controls randomness in the output. Lower values make the output more deterministic.
- **timeout_seconds** (Option<u64>) - Optional - The timeout in seconds for API requests.
- **system** (Option<String>) - Optional - A system message to guide the model's behavior.
- **top_p** (Option<f32>) - Optional - Nucleus sampling parameter. Controls the cumulative probability of tokens to consider.
- **top_k** (Option<u32>) - Optional - Top-k sampling parameter. Limits the sampling pool to the k most likely tokens.
- **embedding_encoding_format** (Option<String>) - Optional - The encoding format for embedding requests.
- **embedding_dimensions** (Option<u32>) - Optional - The desired dimensions for embedding vectors.
- **tools** (Option<Vec<Tool>>) - Optional - A list of tools the model can use.
- **tool_choice** (Option<ToolChoice>) - Optional - Determines how the model selects and uses tools.
- **normalize_response** (Option<bool>) - Optional - Whether to normalize the API response.
- **reasoning_effort** (Option<String>) - Optional - The level of reasoning effort to apply.
- **json_schema** (Option<StructuredOutputFormat>) - Optional - Specifies a JSON schema for structured output.
- **voice** (Option<String>) - Optional - Specifies a voice for text-to-speech output.
- **extra_body** (Option<Value>) - Optional - Additional parameters to include in the request body.
- **enable_web_search** (Option<bool>) - Optional - Whether to enable web search functionality. Defaults to `false`.
- **web_search_context_size** (Option<String>) - Optional - The context size for web search results.
- **web_search_user_location_type** (Option<String>) - Optional - The type of user location for web search.
- **web_search_user_location_approximate_country** (Option<String>) - Optional - The approximate country of the user for web search.
- **web_search_user_location_approximate_city** (Option<String>) - Optional - The approximate city of the user for web search.
- **web_search_user_location_approximate_region** (Option<String>) - Optional - The approximate region of the user for web search.

### Request Example
```json
{
  "api_key": "sk-your-openai-api-key",
  "model": "gpt-4o",
  "enable_web_search": true,
  "web_search_context_size": "1024"
}
```

### Response
#### Success Response (200)
- **OpenAI** (OpenAI) - An initialized OpenAI client instance.

#### Response Example
```json
{
  "client": "...",
  "api_key": "sk-your-openai-api-key",
  "base_url": null,
  "model": "gpt-4o",
  "max_tokens": null,
  "temperature": null,
  "timeout_seconds": null,
  "system": null,
  "top_p": null,
  "top_k": null,
  "embedding_encoding_format": null,
  "embedding_dimensions": null,
  "tools": null,
  "tool_choice": null,
  "normalize_response": null,
  "reasoning_effort": null,
  "json_schema": null,
  "voice": null,
  "extra_body": null,
  "enable_web_search": true,
  "web_search_context_size": "1024",
  "web_search_user_location_type": null,
  "web_search_user_location_approximate_country": null,
  "web_search_user_location_approximate_city": null,
  "web_search_user_location_approximate_region": null
}
```
```

--------------------------------

### Rust: Example Searches for MaybeSendSync Trait

Source: https://docs.rs/llm/1.3.6/llm/backends/anthropic/struct.AnthropicModelEntry_search=

Illustrates common search patterns used when exploring the `MaybeSendSync` trait and its associated types in Rust. These examples cover standard library collections, primitive type conversions, and functional programming constructs.

```rust
std::vec
```

```rust
u32 -> bool
```

```rust
Option<T>, (T -> U) -> Option<U>
```

--------------------------------

### Groq Client Initialization Functions

Source: https://docs.rs/llm/1.3.6/llm/backends/groq/type.Groq

Provides functions to create a new Groq client, either directly with specific configurations or using the OpenAICompatibleProvider's new function.

```rust
pub fn with_config(
    api_key: impl Into<String>,
    base_url: Option<String>,
    model: Option<String>,
    max_tokens: Option<u32>,
    temperature: Option<f32>,
    timeout_seconds: Option<u64>,
    system: Option<String>,
    top_p: Option<f32>,
    top_k: Option<u32>,
    tools: Option<Vec<Tool>>,
    tool_choice: Option<ToolChoice>,
    extra_body: Option<Value>,
    _embedding_encoding_format: Option<String>,
    _embedding_dimensions: Option<u32>,
    reasoning_effort: Option<String>,
    json_schema: Option<StructuredOutputFormat>,
    parallel_tool_calls: Option<bool>,
    normalize_response: Option<bool>,
) -> Self

pub fn new(
    api_key: impl Into<String>,
    base_url: Option<String>,
    model: Option<String>,
    max_tokens: Option<u32>,
    temperature: Option<f32>,
    timeout_seconds: Option<u64>,
    system: Option<String>,
    top_p: Option<f32>,
    top_k: Option<u32>,
    tools: Option<Vec<Tool>>,
    tool_choice: Option<ToolChoice>,
    reasoning_effort: Option<String>,
    json_schema: Option<StructuredOutputFormat>,
    voice: Option<String>,
    extra_body: Option<Value>,
    parallel_tool_calls: Option<bool>,
    normalize_response: Option<bool>,
    embedding_encoding_format: Option<String>,
    embedding_dimensions: Option<u32>,
) -> Self
```

--------------------------------

### Get Type ID (Rust)

Source: https://docs.rs/llm/1.3.6/llm/validated_llm/struct.ValidatedLLM_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Gets the TypeId of `self`. This is a generic method provided through blanket implementations, allowing retrieval of the unique type identifier for any static type.

```rust
fn type_id(&self) -> TypeId
// Implementation details...
```

--------------------------------

### Groq Client Initialization - Rust

Source: https://docs.rs/llm/1.3.6/llm/backends/groq/type.Groq_search=std%3A%3Avec

Provides functions to create a new `Groq` client instance. It includes a `with_config` function for detailed configuration and a `new` function for initializing an `OpenAICompatibleProvider` with Groq-specific settings.

```rust
pub fn with_config( api_key: impl Into<String>, base_url: Option<String>, model: Option<String>, max_tokens: Option<u32>, temperature: Option<f32>, timeout_seconds: Option<u64>, system: Option<String>, top_p: Option<f32>, top_k: Option<u32>, tools: Option<Vec<Tool>>, tool_choice: Option<ToolChoice>, extra_body: Option<Value>, _embedding_encoding_format: Option<String>, _embedding_dimensions: Option<u32>, reasoning_effort: Option<String>, json_schema: Option<StructuredOutputFormat>, parallel_tool_calls: Option<bool>, normalize_response: Option<bool>, ) -> Self

pub fn new( api_key: impl Into<String>, base_url: Option<String>, model: Option<String>, max_tokens: Option<u32>, temperature: Option<f32>, timeout_seconds: Option<u64>, system: Option<String>, top_p: Option<f32>, top_k: Option<u32>, tools: Option<Vec<Tool>>, tool_choice: Option<ToolChoice>, reasoning_effort: Option<String>, json_schema: Option<StructuredOutputFormat>, voice: Option<String>, extra_body: Option<Value>, parallel_tool_calls: Option<bool>, normalize_response: Option<bool>, embedding_encoding_format: Option<String>, embedding_dimensions: Option<u32>, ) -> Self
```

--------------------------------

### ElevenLabs Constructor and Configuration

Source: https://docs.rs/llm/1.3.6/llm/backends/elevenlabs/struct.ElevenLabs_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E

Provides the `new` function to instantiate the `ElevenLabs` backend. It requires API credentials and model details, with optional parameters for timeout and voice selection.

```rust
pub fn new( api_key: String, model_id: String, base_url: String, timeout_seconds: Option<u64>, voice: Option<String>, ) -> Self
```
Tokens:
Raw