### API LLM Basic Completion with Perplexity in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/index_search= This Rust code example shows how to initialize and use the llm_client with an API-based LLM, specifically Perplexity. It demonstrates setting up a basic completion request, adding a user message with a prompt, and running the completion to get a response. ```rust let llm_client = LlmClient::perplexity().sonar_large().init(); let mut basic_completion = llm_client.basic_completion(); basic_completion .prompt() .add_user_message() .set_content("Can you help me use the llm_client rust crate? I'm having trouble getting cuda to work."); let response = basic_completion.run().await?; ``` -------------------------------- ### Initialize Logger with Configuration in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.LoggingConfig_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Implements a method to initialize and start the logger based on the current LoggingConfig. It handles log file creation, console output setup, and log rotation (hourly, keeping up to 6 files). Returns an error if log directory or file appender creation fails. ```rust pub fn load_logger(&mut self) -> Result<(), Error> { // If logging is enabled, creates log files and sets up console output. // Logs are rotated hourly and up to 6 files are kept. // ... implementation details ... } ``` -------------------------------- ### Instruct Prompt Methods Source: https://docs.rs/llm_client/0.0.7/llm_client/workflows/reason/one_round/struct.ReasonOneRound_search= Methods for manipulating the instruction prompt, including setting instructions and supporting material, which guide the model's output. ```APIDOC ## Instruct Prompt Methods (via `InstructPromptTrait`) ### Description These methods, implemented through the `InstructPromptTrait`, allow for the dynamic modification of instructions and supporting materials within the `ReasonOneRound` struct's prompt configuration. This is crucial for tailoring the model's behavior for specific tasks. ### Methods * **`instruct_prompt_mut(&mut self) -> &mut InstructPrompt`**: Returns a mutable reference to the `InstructPrompt`. * **`set_instructions>(&mut self, instructions: T) -> &mut Self`**: Sets the primary instructions for the model. * **`instructions(&mut self) -> &mut PromptMessage`**: Returns a mutable reference to the instruction `PromptMessage`. * **`set_supporting_material>(&mut self, supporting_material: T) -> &mut Self`**: Sets additional supporting material to guide the model. * **`supporting_material(&mut self) -> &mut PromptMessage`**: Returns a mutable reference to the supporting material `PromptMessage`. ### Example Usage (Conceptual) ```rust let mut reasoner = ReasonOneRound { // ... other fields primitive: my_primitive_instance, instruct_prompt: InstructPrompt { // ... initial prompt configuration }, base_req: default_completion_request(), }; reasoner.set_instructions("Analyze the following text and provide a summary.") .set_supporting_material("Ensure the summary is objective and concise."); // The instruct_prompt is now updated and will be used in subsequent calls. ``` ``` -------------------------------- ### LoggingConfig::load_logger() Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.LoggingConfig_search=std%3A%3Avec Initializes and starts the logger based on the current LoggingConfig. ```APIDOC ## LoggingConfig::load_logger() ### Description Initializes and starts the logger with the current configuration. If logging is enabled, creates log files and sets up console output. Logs are rotated hourly and up to 6 files are kept. ### Method `load_logger(&mut self)` ### Endpoint N/A (Method call on a LoggingConfig instance) ### Parameters None (operates on `self`) ### Request Body None ### Response #### Success Response (200) - **()** - Indicates successful logger initialization. #### Errors Returns error if: * Log directory creation fails * File appender creation fails ``` -------------------------------- ### OpenAI Backend Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/openai/struct.OpenAiBackendBuilder_search=u32+-%3E+bool Demonstrates how to initialize the LlmClient with the configured OpenAI backend builder. ```APIDOC ## Initialize LlmClient with OpenAI Backend ### Description Initializes the `LlmClient` instance after the `OpenAiBackendBuilder` has been configured. ### Method `init()` method on the builder ### Endpoint N/A (Client-side initialization) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body N/A ### Request Example ```rust use llm_client::backend_builders::openai::OpenAiBackendBuilder; use llm_client::LlmClient; // Assuming configuration has been applied previously let builder = OpenAiBackendBuilder::default() .with_api_key("YOUR_API_KEY"); let client: LlmClient = builder.init().expect("Failed to initialize LlmClient"); // Now you can use the client for API calls ``` ### Response #### Success Response (200) N/A (Initialization, not a direct API call response) #### Response Example N/A ``` -------------------------------- ### TimingUsage new_from_generic Method (Rust) Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.TimingUsage_search=std%3A%3Avec A general-purpose constructor for `TimingUsage` that initializes timing statistics with a provided start time. This method is useful when the completion response structure is not specifically `LlamaCppCompletionResponse` or when only a start time is available for initial setup. ```Rust pub fn new_from_generic(start_time: Instant) -> TimingUsage ``` -------------------------------- ### Boolean Outcome Reasoning with LLM in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/index_search= This Rust example showcases how to use the llm_client to get a boolean outcome from an LLM. It involves setting instructions and supporting material, then calling `return_primitive()` to get the boolean result. This is useful for tasks like spam detection. ```rust // boolean outcome let reason_request = llm_client.reason().boolean(); reason_request .instructions() .set_content("Does this email subject indicate that the email is spam?"); reason_request .supporting_material() .set_content("You'll never believe these low, low prices 💲💲💲!!!"); let res: bool = reason_request.return_primitive().await.unwrap(); assert_eq!(res, true); ``` -------------------------------- ### VZip Implementation Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt Documentation for the `VZip` implementation, used for zipping operations. ```APIDOC ## VZip API ### Description Provides functionality for zipping operations using the `VZip` trait. ### `impl VZip for T` #### Methods - **`fn vzip(self) -> V`**: Performs the zipping operation. ### Type Parameters - **`V`**: The type of the zipped result. - **`T`**: The type on which the `VZip` trait is implemented. ``` -------------------------------- ### Get Base Completion Request in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.LlmClient_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Retrieves a base `CompletionRequest` object. This can be used as a starting point for constructing more complex requests to the language model. ```rust pub fn base_request(&self) -> CompletionRequest ``` -------------------------------- ### InstructPrompt Methods in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt_search=std%3A%3Avec Provides essential methods for the `InstructPrompt` struct, including creating a new instance, resetting its prompt, and building instructions, supporting material, or the complete prompt string. The `build_instruct_prompt` method allows for flexible ordering of supporting material. ```rust pub fn new() -> Self pub fn reset_instruct_prompt(&mut self) pub fn build_instructions(&self) -> Option pub fn build_supporting_material(&self) -> Option pub fn build_instruct_prompt( &self, supporting_material_first: bool, ) -> Result ``` -------------------------------- ### Ownership and Conversion Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt Documentation for methods related to type ownership and conversion, including cloning and `TryFrom`/`TryInto` implementations. ```APIDOC ## Ownership and Conversion API ### Description This section covers methods for managing type ownership and performing conversions. ### `impl ToOwned for T` #### Methods - **`type Owned = T`**: The resulting type after obtaining ownership. - **`fn to_owned(&self) -> T`**: Creates owned data from borrowed data, usually by cloning. - **`fn clone_into(&self, target: &mut T)`**: Uses borrowed data to replace owned data, usually by cloning. ### `impl TryFrom for T` #### Methods - **`type Error = Infallible`**: The type returned in the event of a conversion error. - **`fn try_from(value: U) -> Result>::Error>`**: Performs the conversion. ### `impl TryInto for T` #### Methods - **`type Error = >::Error`**: The type returned in the event of a conversion error. - **`fn try_into(self) -> Result>::Error>`**: Performs the conversion. ### Type Parameters - **`T`**: The target type for conversion or ownership. - **`U`**: The source type for conversion. ``` -------------------------------- ### LLM Reasoning: Boolean Outcome Source: https://docs.rs/llm_client/0.0.7/llm_client/index This Rust example demonstrates using the llm_client for reasoning tasks that result in a boolean outcome. It shows how to set instructions and supporting material to guide the LLM in determining if an email subject indicates spam. ```rust // boolean outcome let reason_request = llm_client.reason().boolean(); reason_request .instructions() .set_content("Does this email subject indicate that the email is spam?"); reason_request .supporting_material() .set_content("You'll never believe these low, low prices 💲💲💲!!!"); let res: bool = reason_request.return_primitive().await.unwrap(); assert_eq!(res, true); ``` -------------------------------- ### InstructPrompt Prompt Building Methods in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt_search=u32+-%3E+bool Offers methods to build the instruction and supporting material strings from the respective fields within an InstructPrompt. These methods are essential for preparing prompt content before final assembly. ```rust pub fn build_instructions(&self) -> Option ``` ```rust pub fn build_supporting_material(&self) -> Option ``` -------------------------------- ### OpenAiBackendBuilder Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/openai/struct.OpenAiBackendBuilder Provides methods to configure and initialize the OpenAI backend, including setting API keys, hosts, ports, and selecting specific OpenAI models. ```APIDOC ## OpenAiBackendBuilder `OpenAiBackendBuilder` is used to configure and build an OpenAI backend for the `llm_client`. ### Struct Definition ```rust pub struct OpenAiBackendBuilder { pub config: OpenAiConfig, pub model: ApiLlmModel, } ``` ### Methods #### `fn init(self) -> Result` Initializes the `LlmClient` with the configured OpenAI backend. ### Configuration Methods (Implementing `LlmApiConfigTrait`) - `fn with_api_host(self, host: S) -> Self` where `S: AsRef` Sets the API host. - `fn with_api_port(self, port: S) -> Self` where `S: AsRef` Sets the API port. - `fn with_api_key(self, api_key: S) -> Self` where `S: Into` Sets the API key. - `fn with_api_key_env_var(self, api_key_env_var: S) -> Self` where `S: Into` Sets the environment variable name for the API key. ### Logging Configuration Methods (Implementing `LoggingConfigTrait`) - `fn logging_enabled(self, enabled: bool) -> Self` Enables or disables logging. - `fn logger_name(self, logger_name: S) -> Self` where `S: Into` Sets the name of the logger. - `fn log_path

(self, path: P) -> Self` where `P: AsRef` Sets the path for log files. - `fn log_level_trace(self) -> Self` Sets the log level to TRACE. - `fn log_level_debug(self) -> Self` Sets the log level to DEBUG. - `fn log_level_info(self) -> Self` Sets the log level to INFO. - `fn log_level_warn(self) -> Self` Sets the log level to WARN. - `fn log_level_error(self) -> Self` Sets the log level to ERROR. ### Model Selection Methods (Implementing `OpenAiModelTrait`) - `fn model(&mut self) -> &mut ApiLlmModel` Gets a mutable reference to the current model. - `fn model_id_str(self, model_id: &str) -> Self` Sets the model using a model ID string. - `fn gpt_4(self) -> Self` Uses `gpt-4` as the model. - `fn gpt_4_32k(self) -> Self` Uses `gpt-4-32k` as the model. - `fn gpt_3_5_turbo(self) -> Self` Uses `gpt-3.5-turbo` as the model. - `fn gpt_4_turbo(self) -> Self` Uses `gpt-4-turbo` as the model. - `fn gpt_4_o(self) -> Self` Uses `gpt-4-o` as the model. - `fn gpt_4_o_mini(self) -> Self` Uses `gpt-4o-mini` as the model. - `fn o1(self) -> Self` Uses `o1` as the model. - `fn o1_mini(self) -> Self` Uses `o1-mini` as the model. ### Default Implementation - `fn default() -> Self` Returns the default `OpenAiBackendBuilder`. ``` -------------------------------- ### Exact String Outcome Reasoning with LLM in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/index_search= This Rust example shows how to get an exact string outcome from an LLM, constrained to a list of allowed options. It involves setting instructions, supporting material, and adding allowed strings to the primitive's configuration before calling `return_primitive()`. ```rust // string from a list of options outcome let mut reason_request = llm_client.reason().exact_string(); reason_request .instructions() .set_content("Based on this readme, what is the name of the creator of this project?"); reason_request .supporting_material() .set_content(llm_client_readme); reason_request .primitive .add_strings_to_allowed(&["shelby", "jack", "camacho", "john"]); let response: String = reason_request.return_primitive().await.unwrap(); assert_eq!(res, "shelby"); ``` -------------------------------- ### Build Full Instruct Prompt in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt_search=u32+-%3E+bool Constructs the complete instruct prompt string by combining instructions and supporting material, with an option to place supporting material first. This method returns a Result, indicating potential errors during prompt building. ```rust pub fn build_instruct_prompt( &self, supporting_material_first: bool, ) -> Result ``` -------------------------------- ### Load Logger with LoggingConfig in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.LoggingConfig_search=std%3A%3Avec Implements the `load_logger` method for `LoggingConfig` to initialize and start the logging system based on the struct's configuration. This method handles log file creation, console output setup, and log rotation. It returns a `Result` to indicate success or failure, with errors potentially arising from directory or file appender creation issues. ```rust pub fn load_logger(&mut self) -> Result<(), Error> { // Initializes and starts the logger with the current configuration. // If logging is enabled, creates log files and sets up console output. // Logs are rotated hourly and up to 6 files are kept. // Errors: Returns error if: // * Log directory creation fails // * File appender creation fails } ``` -------------------------------- ### AnthropicBackendBuilder Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/anthropic/struct.AnthropicBackendBuilder_search= This section details the initialization and configuration of the AnthropicBackendBuilder. ```APIDOC ## Struct AnthropicBackendBuilder ### Description Builder for configuring and initializing the Anthropic LLM client. ### Fields - **config** (AnthropicConfig) - Configuration for the Anthropic API. - **model** (ApiLlmModel) - The specific LLM model to use. ### Methods #### `init(self) -> Result` Initializes the LlmClient with the configured Anthropic backend. ### Model Selection #### `model_id_str(self, model_id: &str) -> Self` Sets the model using a model ID string. #### `claude_3_opus(self) -> Self` Uses the Claude 3 Opus model. #### `claude_3_sonnet(self) -> Self` Uses the Claude 3 Sonnet model. #### `claude_3_haiku(self) -> Self` Uses the Claude 3 Haiku model. #### `claude_3_5_sonnet(self) -> Self` Uses the Claude 3.5 Sonnet model. #### `claude_3_5_haiku(self) -> Self` Uses the Claude 3.5 Haiku model. ### API Configuration #### `with_api_host(self, host: S) -> Self` Sets the API host. #### `with_api_port(self, port: S) -> Self` Sets the API port. #### `with_api_key(self, api_key: S) -> Self` Sets the API key. #### `with_api_key_env_var(self, api_key_env_var: S) -> Self` Sets the environment variable name for the API key. ### Logging Configuration #### `logging_enabled(self, enabled: bool) -> Self` Enables or disables logging. #### `logger_name(self, logger_name: S) -> Self` Sets the name of the logger. #### `log_path

(self, path: P) -> Self` Sets the path for log files. #### `log_level_trace(self) -> Self` Sets the log level to TRACE. #### `log_level_debug(self) -> Self` Sets the log level to DEBUG. #### `log_level_info(self) -> Self` Sets the log level to INFO. #### `log_level_warn(self) -> Self` Sets the log level to WARN. #### `log_level_error(self) -> Self` Sets the log level to ERROR. ### Default Implementation #### `default() -> Self` Returns the default `AnthropicBackendBuilder`. ### Request Example (Initialization) ```rust use llm_client::backend_builders::anthropic::AnthropicBackendBuilder; use llm_client::LlmClient; let client: LlmClient = AnthropicBackendBuilder::default() .claude_3_sonnet() .with_api_key("YOUR_ANTHROPIC_API_KEY") .init() .expect("Failed to initialize LlmClient"); ``` ### Response Example (Success) Upon successful initialization, `init()` returns a `LlmClient` instance. ```rust // LlmClient instance ``` ``` -------------------------------- ### Rust: Pointable trait for pointer management Source: https://docs.rs/llm_client/0.0.7/llm_client/workflows/reason/decision/struct.Decision_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Defines methods for managing raw pointers, including getting the pointer's alignment, initializing an object at a given memory address, dereferencing the pointer to get a reference (mutable or immutable), and dropping the object pointed to. ```rust trait Pointable { type Init = T; const ALIGN: usize; unsafe fn init(init: Self::Init) -> usize; unsafe fn deref<'a>(ptr: usize) -> &'a Self; unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut Self; unsafe fn drop(ptr: usize); } ``` -------------------------------- ### InstructPromptTrait Provided Method: set_instructions Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/trait.InstructPromptTrait_search= The `set_instructions` method is a provided implementation within the `InstructPromptTrait`. It allows setting the instructional text for the prompt, accepting any type that can be referenced as a string slice. ```rust fn set_instructions>(&mut self, instructions: T) -> &mut Self { ... } ``` -------------------------------- ### ApiPrompt - Constructor Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.ApiPrompt Initializes a new ApiPrompt with optional token overhead configurations. ```APIDOC ## POST /llm_client/ApiPrompt/new ### Description Initializes a new `ApiPrompt` instance. This constructor allows setting up the prompt formatter with a specific tokenizer and defining overhead token counts for messages and names. ### Method POST ### Endpoint /llm_client/ApiPrompt/new ### Parameters #### Query Parameters - **tokenizer** (Arc) - Required - The tokenizer to use for counting tokens. - **tokens_per_message** (Option) - Optional - The number of tokens to add as overhead for each message. - **tokens_per_name** (Option) - Optional - The number of tokens to add as overhead for each message name. ### Request Example ```json { "tokenizer": "", "tokens_per_message": 5, "tokens_per_name": 1 } ``` ### Response #### Success Response (200) - **ApiPrompt** - An instance of the ApiPrompt struct. #### Response Example ```json { "ApiPrompt": "" } ``` ``` -------------------------------- ### TypeId Trait Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Provides methods to get the TypeId of a value. ```APIDOC ## fn type_id(&self) -> TypeId ### Description Gets the `TypeId` of `self`. ### Method `type_id` ### Parameters None ### Response #### Success Response (200) - **TypeId** (TypeId) - The type identifier of the object. ### Response Example ```json { "TypeId": "some_type_id_value" } ``` ``` -------------------------------- ### InstructPromptTrait Provided Method: instructions Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/trait.InstructPromptTrait_search= The `instructions` method is a provided implementation within the `InstructPromptTrait`. It returns a mutable reference to the `PromptMessage` containing the current instructions, allowing direct manipulation. ```rust fn instructions(&mut self) -> &mut PromptMessage { ... } ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for WordsPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for WordsPrimitive, specifying that its PrimitiveResult is a String. ```rust impl PrimitiveTrait for WordsPrimitive { type PrimitiveResult = String; } ``` -------------------------------- ### InstructPrompt Build Instructions Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt Builds a string representation of the instructions from the InstructPrompt. Returns an Option which is None if no instructions are present. ```rust pub fn build_instructions(&self) -> Option ``` -------------------------------- ### TryFrom and TryInto Implementations Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/openai/struct.OpenAiBackendBuilder_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Documentation for `TryFrom` and `TryInto` trait implementations for type conversions. ```APIDOC ## TryFrom and TryInto Implementations ### Description Provides functionality for fallible type conversions. #### `impl TryFrom for T` - **Type Alias `Error`**: The type returned in the event of a conversion error (`Infallible`). - **`fn try_from(value: U) -> Result>::Error>`**: Performs the conversion. #### `impl TryInto for T` - **Type Alias `Error`**: The type returned in the event of a conversion error (`>::Error`). - **`fn try_into(self) -> Result>::Error>`**: Performs the conversion. ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for TextListPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for TextListPrimitive, specifying that its PrimitiveResult is TextListType. ```rust impl PrimitiveTrait for TextListPrimitive { type PrimitiveResult = TextListType; } ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for TextPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for TextPrimitive, specifying that its PrimitiveResult is a String. ```rust impl PrimitiveTrait for TextPrimitive { type PrimitiveResult = String; } ``` -------------------------------- ### NVML Initialization API Source: https://docs.rs/llm_client/0.0.7/llm_client/fn.init_nvml_wrapper_search=u32+-%3E+bool Provides functionality to initialize the NVIDIA Management Library (NVML). ```APIDOC ## init_nvml_wrapper ### Description Initializes NVIDIA Management Library (NVML). Attempts to load the NVML library from common paths on Linux, WSL, and Windows. ### Method `pub fn init_nvml_wrapper() -> Result` ### Endpoint N/A (This is a function call within the Rust library, not a REST endpoint) ### Parameters None ### Request Example N/A ### Response #### Success Response - **Nvml** (type) - An object representing the initialized NVML library. #### Error Response - **Error** (type) - Returned if NVML initialization fails on all attempted paths. #### Response Example ```rust // Success let nvml_handle = llm_client::init_nvml_wrapper()?; // Error // Returns an Error type if initialization fails. ``` ``` -------------------------------- ### InstructPrompt Initialization and Reset Methods in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt_search=u32+-%3E+bool Provides methods for creating a new InstructPrompt instance and resetting its internal state. The `new` function initializes the struct, while `reset_instruct_prompt` clears existing prompt components. ```rust pub fn new() -> Self ``` ```rust pub fn reset_instruct_prompt(&mut self) ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for SentencesPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for SentencesPrimitive, specifying that its PrimitiveResult is a String. ```rust impl PrimitiveTrait for SentencesPrimitive { type PrimitiveResult = String; } ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for IntegerPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for IntegerPrimitive, specifying that its PrimitiveResult is a u32. ```rust impl PrimitiveTrait for IntegerPrimitive { type PrimitiveResult = u32; } ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for ExactStringPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for ExactStringPrimitive, specifying that its PrimitiveResult is a String. ```rust impl PrimitiveTrait for ExactStringPrimitive { type PrimitiveResult = String; } ``` -------------------------------- ### NVML Wrapper Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/fn.init_nvml_wrapper_search=std%3A%3Avec Initializes the NVIDIA Management Library (NVML) and returns an Nvml object on success. ```APIDOC ## init_nvml_wrapper ### Description Initializes NVIDIA Management Library (NVML). Attempts to load the NVML library from common paths on Linux, WSL, and Windows. ### Method Rust function call (equivalent to a backend operation) ### Endpoint N/A (Rust function) ### Parameters None ### Request Example N/A (Rust function) ### Response #### Success Response - **Nvml** (Nvml) - An initialized NVML object if successful. #### Response Example ```rust // Successful initialization returns Ok(Nvml) ``` #### Error Response - **Error** (Error) - Returned if NVML initialization fails on all attempted paths. #### Error Response Example ```rust // Failed initialization returns Err(Error) ``` ``` -------------------------------- ### InstructPrompt Build Supporting Material Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt Builds a string representation of the supporting material from the InstructPrompt. Returns an Option which is None if no supporting material is present. ```rust pub fn build_supporting_material(&self) -> Option ``` -------------------------------- ### Rust - PrimitiveTrait Implementations for BooleanPrimitive Source: https://docs.rs/llm_client/0.0.7/llm_client/primitives/trait.PrimitiveTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Example implementation of PrimitiveTrait for BooleanPrimitive, specifying that its PrimitiveResult is a boolean. ```rust impl PrimitiveTrait for BooleanPrimitive { type PrimitiveResult = bool; } ``` -------------------------------- ### Get Type ID in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/anthropic/struct.AnthropicBackendBuilder_search=u32+-%3E+bool Returns the `TypeId` of the current type. This is a fundamental method for runtime type identification in Rust. ```rust fn type_id(&self) -> TypeId ``` -------------------------------- ### LlmClient Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.LlmClient Methods for creating and initializing an LlmClient instance using different backend builders. ```APIDOC ## LlmClient Initialization Methods ### `LlmClient::new(backend: Arc) -> Self` Creates a new instance of `LlmClient` with a provided backend. ### `LlmClient::llama_cpp() -> LlamaCppBackendBuilder` Creates a builder for the Llama.cpp backend. Use the `init` method on the builder to create an `LlmClient`. ### `LlmClient::openai() -> OpenAiBackendBuilder` Creates a builder for the OpenAI backend. Use the `init` method on the builder to create an `LlmClient`. ### `LlmClient::anthropic() -> AnthropicBackendBuilder` Creates a builder for the Anthropic backend. Use the `init` method on the builder to create an `LlmClient`. ### `LlmClient::perplexity() -> PerplexityBackendBuilder` Creates a builder for the Perplexity backend. Use the `init` method on the builder to create an `LlmClient`. ``` -------------------------------- ### Initialize ApiPrompt with Tokenizer and Overhead Settings Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.ApiPrompt Creates a new instance of `ApiPrompt`. It requires a tokenizer, and optionally accepts token counts for messages and names to accurately calculate prompt token usage. ```rust pub fn new( tokenizer: Arc, tokens_per_message: Option, tokens_per_name: Option, ) -> ApiPrompt ``` -------------------------------- ### Get Mutable InstructPrompt in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/workflows/nlp/extract/urls/struct.ExtractUrls_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Provides mutable access to the `InstructPrompt` associated with the `ExtractUrls` struct. This allows modification of the prompt configuration. ```rust fn instruct_prompt_mut(&mut self) -> &mut InstructPrompt ``` -------------------------------- ### DeviceConfig Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig Initializes the device configuration, detecting hardware capabilities and setting up appropriate configurations. ```APIDOC ## POST /deviceconfig/initialize ### Description Initializes the device configuration, detecting hardware capabilities and setting up appropriate configurations. ### Method POST ### Endpoint /deviceconfig/initialize ### Parameters #### Request Body - **cpu_config** (CpuConfig) - Required - CPU configuration for thread count. - **ram_config** (RamConfig) - Required - RAM configuration for non-GPU inference on Windows and Unix. - **use_gpu** (bool) - Required - Indicates whether to use any available GPUs for inference. - **cuda_config** (Option) - Optional - CUDA configuration for GPU inference on non-macOS platforms. - **error_on_config_issue** (bool) - Optional - Determines error handling behavior for configuration issues. - **layer_count** (Option) - Optional - The number of layers in the model. - **average_layer_size_bytes** (Option) - Optional - The average size of a layer in bytes. - **local_model_path** (String) - Required - The file system path to the local model. ### Response #### Success Response (200) - **Result** (Result<(), Error>) - Indicates success or failure of the initialization. #### Error Response (e.g., 400, 500) - **Error** - Details about the configuration or platform issue. ``` -------------------------------- ### Get Target Directory Source: https://docs.rs/llm_client/0.0.7/llm_client/fn.get_target_directory_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Resolves the Cargo target directory path. This function attempts to find the target directory through multiple methods. ```APIDOC ## get_target_directory ### Description Resolves the Cargo target directory path. This function attempts to find the target directory through multiple methods: 1. Checks the `CARGO_TARGET_DIR` environment variable 2. Traverses up from the `OUT_DIR` to find a ‘target’ directory 3. Traverses up from `CARGO_MANIFEST_DIR` to find a ‘target’ directory 4. Falls back to using the compile-time `CARGO_MANIFEST_DIR` This helps resolve issues with cargo workspace target directory resolution. ### Method GET (Conceptual - this is a function call, not an HTTP endpoint) ### Endpoint N/A (Function within the llm_client crate) ### Parameters None ### Request Example ```rust use llm_client::get_target_directory; match get_target_directory() { Ok(path) => println!("Target directory: {}", path.display()), Err(e) => eprintln!("Error resolving target directory: {}", e), } ``` ### Response #### Success Response (Ok) - **PathBuf** - Path to the cargo target directory if found #### Response Example ```json { "path": "/path/to/your/project/target" } ``` #### Error Response (Err) - **Error** - An error indicating that the target directory could not be located. #### Error Example ```json { "error": "Failed to resolve target directory." } ``` ``` -------------------------------- ### Get Target Directory Source: https://docs.rs/llm_client/0.0.7/llm_client/fn.get_target_directory_search= Resolves the Cargo target directory path using various methods including environment variables and directory traversal. ```APIDOC ## get_target_directory ### Description Resolves the Cargo target directory path. This function attempts to find the target directory through multiple methods: 1. Checks the `CARGO_TARGET_DIR` environment variable 2. Traverses up from the `OUT_DIR` to find a ‘target’ directory 3. Traverses up from `CARGO_MANIFEST_DIR` to find a ‘target’ directory 4. Falls back to using the compile-time `CARGO_MANIFEST_DIR` This helps resolve issues with cargo workspace target directory resolution. ### Method GET ### Endpoint /llm_client/get_target_directory ### Parameters None ### Request Body None ### Request Example None ### Response #### Success Response (200) - **PathBuf**: Path to the cargo target directory if found #### Response Example { "path": "/path/to/cargo/target" } ### Errors Returns an error if unable to locate the target directory through any method. ``` -------------------------------- ### Policy Composition: Or Source: https://docs.rs/llm_client/0.0.7/llm_client/basic_completion/struct.BasicCompletion_search=std%3A%3Avec Creates a new Policy that returns Action::Follow if either self or other policies return Action::Follow. ```APIDOC ## POST /policy/or ### Description Creates a new Policy that returns Action::Follow if either self or other policies return Action::Follow. ### Method POST ### Endpoint /policy/or ### Parameters #### Request Body - **self_policy** (Policy) - Required - The first policy. - **other_policy** (Policy) - Required - The second policy. ### Request Example ```json { "self_policy": {"name": "policy1"}, "other_policy": {"name": "policy2"} } ``` ### Response #### Success Response (200) - **new_policy** (Or) - The combined policy. #### Response Example ```json { "new_policy": {"name": "combined_or_policy"} } ``` ``` -------------------------------- ### TryInto Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig Provides a fallible way to attempt conversion into another type. ```APIDOC ## POST /std/try_into ### Description Provides a fallible way to attempt conversion from the current type into another type (`U`). Returns a `Result`. ### Method POST ### Endpoint /std/try_into ### Parameters #### Request Body - **from_type** (string) - Required - The source type. - **to_type** (string) - Required - The target type. - **value** (any) - Required - The value to attempt conversion on. ### Request Example ```json { "from_type": "u32", "to_type": "i32", "value": 100 } ``` ### Response #### Success Response (200) - **result** (string) - "Ok(converted_value)" on success, or "Err(error_details)" on failure. #### Response Example ```json { "result": "Ok(100)" } ``` ``` -------------------------------- ### Get owned type from ToOwned implementation Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt_search= Defines the associated type `Owned` which represents the owned version of the type `T` when implementing the `ToOwned` trait. ```rust type Owned = T; ``` -------------------------------- ### Get Mutable Reference to Last CascadeStep in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/cascade/round/struct.CascadeRound_search= Provides mutable access to the last `CascadeStep` within the `CascadeRound`. This allows for modification of the most recent step. ```rust pub fn last_step(&mut self) -> Result<&mut CascadeStep> ``` -------------------------------- ### TryFrom Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig Provides a fallible way to convert one type into another. ```APIDOC ## POST /std/try_from ### Description Provides a fallible way to attempt conversion from one type (`U`) into another (`T`). Returns a `Result`. ### Method POST ### Endpoint /std/try_from ### Parameters #### Request Body - **from_type** (string) - Required - The source type. - **to_type** (string) - Required - The target type. - **value** (any) - Required - The value to attempt conversion on. ### Request Example ```json { "from_type": "i32", "to_type": "u32", "value": 100 } ``` ### Response #### Success Response (200) - **result** (string) - "Ok(converted_value)" on success, or "Err(error_details)" on failure. #### Response Example ```json { "result": "Ok(100)" } ``` ``` -------------------------------- ### Get Generation Prefix in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/cascade/round/struct.CascadeRound_search=std%3A%3Avec Retrieves the generation prefix for a given step within the `CascadeRound`. It returns a `Result` that may contain an `Option`. ```Rust pub fn generation_prefix( &self, current_step: &CascadeStep, ) -> Result> ``` -------------------------------- ### TryFrom and TryInto Implementations Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/openai/struct.OpenAiBackendBuilder_search= Implementations for attempting conversions between types, with error handling for potential failures. ```APIDOC ## `TryFrom` for `T` ### Description Provides a mechanism to attempt conversion from type `U` into type `T`. #### `Error` Type * **Description**: The type returned in the event of a conversion error. * **Type**: `Infallible` #### `try_from` Method ### Description Performs the conversion. ### Method `fn try_from(value: U) -> Result>::Error>` ## `TryInto` for `T` ### Description Provides a mechanism to attempt conversion from type `T` into type `U`. #### `Error` Type * **Description**: The type returned in the event of a conversion error. * **Type**: `>::Error` #### `try_into` Method ### Description Performs the conversion. ### Method `fn try_into(self) -> Result>::Error>` ``` -------------------------------- ### Get Mutable Logging Configuration in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/anthropic/struct.AnthropicBackendBuilder_search=u32+-%3E+bool Retrieves a mutable reference to the logging configuration. This allows detailed customization of logging behavior for the Anthropic client. ```rust fn logging_config_mut(&mut self) -> &mut LoggingConfig ``` -------------------------------- ### InstructPrompt Build Prompt Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/struct.InstructPrompt Constructs the final prompt string by combining instructions and supporting material, with an option to place supporting material first. Returns a Result as the operation can fail. ```rust pub fn build_instruct_prompt( &self, supporting_material_first: bool, ) -> Result ``` -------------------------------- ### Rust DeviceConfig Initialization and Error Handling Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig_search= Initializes the device configuration by detecting hardware capabilities. It may return an error if hardware detection fails and 'error_on_config_issue' is true, or if the platform is unsupported. ```rust pub fn initialize(&mut self) -> Result<(), Error> ``` -------------------------------- ### Get Default AnthropicBackendBuilder in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/anthropic/struct.AnthropicBackendBuilder_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E Implements the Default trait for AnthropicBackendBuilder, providing a default constructor. This allows creating a builder with standard initial settings. ```rust fn default() -> Self ``` -------------------------------- ### InstructPromptTrait Methods Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/trait.InstructPromptTrait_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E This section details the methods available within the InstructPromptTrait, including required and provided methods for prompt management. ```APIDOC ## Trait InstructPromptTrait ### Description Provides methods to manage instructional prompts and supporting material for language models. ### Required Methods #### `fn instruct_prompt_mut(&mut self) -> &mut InstructPrompt` - **Description**: Returns a mutable reference to the underlying `InstructPrompt`. - **Method**: `GET` (implicit, as it returns a mutable reference) - **Endpoint**: N/A (trait method) ### Provided Methods #### `fn set_instructions>(&mut self, instructions: T) -> &mut Self` - **Description**: Sets the instructions for the prompt. - **Method**: `SET` (implicit, as it modifies the object) - **Endpoint**: N/A (trait method) - **Parameters**: - `instructions` (string slice) - Required - The instructions to set. #### `fn instructions(&mut self) -> &mut PromptMessage` - **Description**: Returns a mutable reference to the prompt's instructions. - **Method**: `GET` (implicit, as it returns a mutable reference) - **Endpoint**: N/A (trait method) #### `fn set_supporting_material>(&mut self, supporting_material: T) -> &mut Self` - **Description**: Sets the supporting material for the prompt. - **Method**: `SET` (implicit, as it modifies the object) - **Endpoint**: N/A (trait method) - **Parameters**: - `supporting_material` (string slice) - Required - The supporting material to set. #### `fn supporting_material(&mut self) -> &mut PromptMessage` - **Description**: Returns a mutable reference to the prompt's supporting material. - **Method**: `GET` (implicit, as it returns a mutable reference) - **Endpoint**: N/A (trait method) ### Dyn Compatibility This trait is **not** dyn compatible. ``` -------------------------------- ### Implement Default for LoggingConfig in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.LoggingConfig_search= Allows LoggingConfig to be created using the default() method, providing a convenient way to get a LoggingConfig instance with its default settings. ```rust fn default() -> LoggingConfig ``` -------------------------------- ### CloneToUninit Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig Nightly-only experimental API for copy-assignment to uninitialized memory. ```APIDOC ## POST /std/clone_to_uninit ### Description Nightly-only experimental API for copy-assignment to uninitialized memory. ### Method POST ### Endpoint /std/clone_to_uninit ### Parameters #### Request Body - **self** (any) - Required - The value to be copied. - **dest** (pointer) - Required - The destination pointer in uninitialized memory. ### Request Example ```json { "self": "value_to_copy", "dest": "0x1234567890ab" } ``` ### Response #### Success Response (200) - **status** (string) - Indicates successful operation. #### Response Example ```json { "status": "success" } ``` ``` -------------------------------- ### Get Model Layer Count (Rust) Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.DeviceConfig_search=u32+-%3E+bool Returns the total number of layers in the model. This value is set at runtime. An error is returned if `layer_count` has not been set. ```rust pub fn layer_count(&self) -> Result ``` -------------------------------- ### Get Generation Prefix for CascadeStep in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/components/cascade/round/struct.CascadeRound_search= Retrieves the generation prefix for a given `CascadeStep` within the `CascadeRound`. This operation may return an error or an optional String. ```rust pub fn generation_prefix( &self, current_step: &CascadeStep, ) -> Result> ``` -------------------------------- ### Policy Composition: And Source: https://docs.rs/llm_client/0.0.7/llm_client/basic_completion/struct.BasicCompletion_search=std%3A%3Avec Creates a new Policy that returns Action::Follow only if both self and other policies return Action::Follow. ```APIDOC ## POST /policy/and ### Description Creates a new Policy that returns Action::Follow only if both self and other policies return Action::Follow. ### Method POST ### Endpoint /policy/and ### Parameters #### Request Body - **self_policy** (Policy) - Required - The first policy. - **other_policy** (Policy) - Required - The second policy. ### Request Example ```json { "self_policy": {"name": "policy1"}, "other_policy": {"name": "policy2"} } ``` ### Response #### Success Response (200) - **new_policy** (And) - The combined policy. #### Response Example ```json { "new_policy": {"name": "combined_and_policy"} } ``` ``` -------------------------------- ### Get API Configuration in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/anthropic/struct.AnthropicBackendBuilder_search=u32+-%3E+bool Retrieves an immutable reference to the API configuration for the Anthropic client. Used for inspecting current API settings without modification. ```rust fn api_config(&self) -> &ApiConfig ``` -------------------------------- ### InstructPromptTrait API Source: https://docs.rs/llm_client/0.0.7/llm_client/components/instruct_prompt/trait.InstructPromptTrait_search=u32+-%3E+bool Details the methods available on the InstructPromptTrait, including required and provided methods. ```APIDOC ## Trait InstructPromptTrait ### Description This trait provides methods for interacting with and modifying instruct prompts, including setting and retrieving instructions and supporting material. ### Required Methods #### `fn instruct_prompt_mut(&mut self) -> &mut InstructPrompt` This is a required method that must be implemented by any type that uses this trait. It returns a mutable reference to an `InstructPrompt`. ### Provided Methods #### `fn set_instructions>(&mut self, instructions: T) -> &mut Self` Sets the instructions for the prompt. Accepts any type that can be referenced as a string slice. #### `fn instructions(&mut self) -> &mut PromptMessage` Returns a mutable reference to the prompt's instructions, represented as a `PromptMessage`. #### `fn set_supporting_material>(&mut self, supporting_material: T) -> &mut Self` Sets the supporting material for the prompt. Accepts any type that can be referenced as a string slice. #### `fn supporting_material(&mut self) -> &mut PromptMessage` Returns a mutable reference to the prompt's supporting material, represented as a `PromptMessage`. ### Dyn Compatibility This trait is **not** dyn compatible, meaning it cannot be used with dynamic dispatch (e.g., `dyn InstructPromptTrait`). ``` -------------------------------- ### CascadeFlow: Open Cascade Operation Source: https://docs.rs/llm_client/0.0.7/llm_client/components/cascade/struct.CascadeFlow Marks the beginning of a cascade operation. This method likely performs setup or initialization tasks required before the cascade execution begins. ```rust pub fn open_cascade(&mut self) ``` -------------------------------- ### LlamaCppBackendBuilder Initialization Source: https://docs.rs/llm_client/0.0.7/llm_client/backend_builders/llama_cpp/struct.LlamaCppBackendBuilder_search=u32+-%3E+bool The `init` method is an asynchronous function that consumes the builder and returns a configured `LlmClient` instance, assuming successful initialization. ```APIDOC ## `init` Method ### Description Initializes the Llama.cpp backend with the current configuration and returns a `LlmClient`. ### Method `pub async fn init(self) -> Result` ### Parameters This method takes ownership of the `LlamaCppBackendBuilder` instance. ``` -------------------------------- ### Get Extract URLs in Rust Source: https://docs.rs/llm_client/0.0.7/llm_client/workflows/nlp/extract/struct.Extract_search=Option%3CT%3E%2C+%28T+-%3E+U%29+-%3E+Option%3CU%3E A method `urls` on the `Extract` struct that consumes `self` and returns an `ExtractUrls` type, likely used to retrieve associated URLs. ```rust pub fn urls(self) -> ExtractUrls ``` -------------------------------- ### Get Built PromptMessage Content Source: https://docs.rs/llm_client/0.0.7/llm_client/struct.PromptMessage Retrieves the final, concatenated message content. This method returns the complete message as a string, assembled according to the rules defined by the TextConcatenator. ```rust pub fn get_built_prompt_message(&self) -> Result ```