### GET /v1/models Source: https://docs.rs/oxillama-server/%5E0.1.3 List loaded models. ```APIDOC ## GET /v1/models ### Description List loaded models. ### Method GET ### Endpoint /v1/models ``` -------------------------------- ### GET /v1/batches Source: https://docs.rs/oxillama-server/%5E0.1.3 List batch jobs. ```APIDOC ## GET /v1/batches ### Description List batch jobs. ### Method GET ### Endpoint /v1/batches ``` -------------------------------- ### GET /admin/stats Source: https://docs.rs/oxillama-server/%5E0.1.3 Server stats (admin). ```APIDOC ## GET /admin/stats ### Description Server stats (admin). ### Method GET ### Endpoint /admin/stats ``` -------------------------------- ### GET /v1/threads/:thread_id/runs/:run_id Source: https://docs.rs/oxillama-server/%5E0.1.3 Get run status. ```APIDOC ## GET /v1/threads/:thread_id/runs/:run_id ### Description Get run status. ### Method GET ### Endpoint /v1/threads/:thread_id/runs/:run_id ### Parameters #### Path Parameters - **thread_id** (string) - Required - Thread identifier - **run_id** (string) - Required - Run identifier ``` -------------------------------- ### GET /health Source: https://docs.rs/oxillama-server/%5E0.1.3 Health check. ```APIDOC ## GET /health ### Description Health check. ### Method GET ### Endpoint /health ``` -------------------------------- ### GET /admin/loras Source: https://docs.rs/oxillama-server/%5E0.1.3 List registered LoRA adapters (admin). ```APIDOC ## GET /admin/loras ### Description List registered LoRA adapters (admin). ### Method GET ### Endpoint /admin/loras ``` -------------------------------- ### GET /v1/batches/:id Source: https://docs.rs/oxillama-server/%5E0.1.3 Retrieve batch job. ```APIDOC ## GET /v1/batches/:id ### Description Retrieve batch job. ### Method GET ### Endpoint /v1/batches/:id ### Parameters #### Path Parameters - **id** (string) - Required - Batch job identifier ``` -------------------------------- ### GET /admin/models Source: https://docs.rs/oxillama-server/%5E0.1.3 List model pool (admin). ```APIDOC ## GET /admin/models ### Description List model pool (admin). ### Method GET ### Endpoint /admin/models ``` -------------------------------- ### GET /admin/health Source: https://docs.rs/oxillama-server/%5E0.1.3 Extended health (admin). ```APIDOC ## GET /admin/health ### Description Extended health (admin). ### Method GET ### Endpoint /admin/health ``` -------------------------------- ### GET /v1/batches/:id/output Source: https://docs.rs/oxillama-server/%5E0.1.3 Stream batch output JSONL. ```APIDOC ## GET /v1/batches/:id/output ### Description Stream batch output JSONL. ### Method GET ### Endpoint /v1/batches/:id/output ### Parameters #### Path Parameters - **id** (string) - Required - Batch job identifier ``` -------------------------------- ### GET /v1/threads/:thread_id Source: https://docs.rs/oxillama-server/%5E0.1.3 Retrieve thread. ```APIDOC ## GET /v1/threads/:thread_id ### Description Retrieve thread. ### Method GET ### Endpoint /v1/threads/:thread_id ### Parameters #### Path Parameters - **thread_id** (string) - Required - Thread identifier ``` -------------------------------- ### GET /v1/threads/:thread_id/messages Source: https://docs.rs/oxillama-server/%5E0.1.3 List thread messages. ```APIDOC ## GET /v1/threads/:thread_id/messages ### Description List thread messages. ### Method GET ### Endpoint /v1/threads/:thread_id/messages ### Parameters #### Path Parameters - **thread_id** (string) - Required - Thread identifier ``` -------------------------------- ### Initialize and Run Inference with OxiLLaMa Source: https://docs.rs/oxillama/latest/oxillama/index.html Demonstrates configuring the inference engine, loading a GGUF model, and generating text output. ```rust use oxillama::runtime::{InferenceEngine, EngineConfig, SamplerConfig}; let config = EngineConfig { model_path: "model.gguf".to_string(), ..Default::default() }; let mut engine = InferenceEngine::new(config); engine.load_model().expect("failed to load model"); engine.generate("Hello", 128, |tok| print!("{tok}")).expect("generation failed"); ``` -------------------------------- ### InferenceEngine::generate Source: https://docs.rs/oxillama/latest/oxillama Initializes the inference engine with a configuration and performs text generation based on a prompt. ```APIDOC ## InferenceEngine::generate ### Description Loads a GGUF model and generates text tokens based on the provided prompt and configuration. ### Method Rust Method Call ### Parameters - **config** (EngineConfig) - Required - Configuration object containing the model path. - **prompt** (str) - Required - The input text for the model. - **max_tokens** (usize) - Required - Maximum number of tokens to generate. - **callback** (Fn) - Required - Closure to handle generated tokens. ### Example ```rust use oxillama::runtime::{InferenceEngine, EngineConfig}; let config = EngineConfig { model_path: "model.gguf".to_string(), ..Default::default() }; let mut engine = InferenceEngine::new(config); engine.load_model().expect("failed to load model"); engine.generate("Hello", 128, |tok| print!("{tok}")).expect("generation failed"); ``` ``` -------------------------------- ### Initialize GpuDispatcher and check for GPU availability Source: https://docs.rs/oxillama-gpu/%5E0.1.3 Use GpuDispatcher to check if hardware acceleration is available. If the 'gpu' feature is disabled, the dispatcher will default to CPU fallback. ```rust use oxillama_gpu::{GpuDispatcher, GpuContext}; let dispatcher = GpuDispatcher::new(); if dispatcher.has_gpu() { println!("GPU available — will use hardware acceleration"); } else { println!("No GPU — CPU fallback active"); } ``` -------------------------------- ### POST /v1/threads Source: https://docs.rs/oxillama-server/%5E0.1.3 Create Assistants API thread. ```APIDOC ## POST /v1/threads ### Description Create Assistants API thread. ### Method POST ### Endpoint /v1/threads ``` -------------------------------- ### POST /v1/batches Source: https://docs.rs/oxillama-server/%5E0.1.3 Create batch job (disk-spooled). ```APIDOC ## POST /v1/batches ### Description Create batch job (disk-spooled). ### Method POST ### Endpoint /v1/batches ``` -------------------------------- ### Load a GGUF model Source: https://docs.rs/oxillama-gguf/%5E0.1.3 Use GgufModel::load to initialize a model from a file path and access its metadata. ```rust use oxillama_gguf::GgufModel; let model = GgufModel::load("model.gguf").unwrap(); println!("Architecture: {}", model.architecture().unwrap()); println!("Tensors: {}", model.file.header.tensor_count); ``` -------------------------------- ### POST /v1/completions Source: https://docs.rs/oxillama-server/%5E0.1.3 Text completion endpoint. ```APIDOC ## POST /v1/completions ### Description Text completion endpoint. ### Method POST ### Endpoint /v1/completions ``` -------------------------------- ### POST /admin/models/load Source: https://docs.rs/oxillama-server/%5E0.1.3 Background-load model (admin). ```APIDOC ## POST /admin/models/load ### Description Background-load model (admin). ### Method POST ### Endpoint /admin/models/load ``` -------------------------------- ### POST /v1/threads/:thread_id/runs Source: https://docs.rs/oxillama-server/%5E0.1.3 Create and enqueue a run. ```APIDOC ## POST /v1/threads/:thread_id/runs ### Description Create and enqueue a run. ### Method POST ### Endpoint /v1/threads/:thread_id/runs ### Parameters #### Path Parameters - **thread_id** (string) - Required - Thread identifier ``` -------------------------------- ### POST /v1/chat/completions Source: https://docs.rs/oxillama-server/%5E0.1.3 Chat completion endpoint. ```APIDOC ## POST /v1/chat/completions ### Description Chat completion endpoint. ### Method POST ### Endpoint /v1/chat/completions ``` -------------------------------- ### InferenceEngine::generate Source: https://docs.rs/oxillama/latest/index.html Generates text based on an input prompt using the loaded model. ```APIDOC ## InferenceEngine::generate ### Description Generates text tokens based on a provided prompt string using the initialized inference engine. ### Signature `engine.generate(prompt: &str, max_tokens: usize, callback: Fn(String)) -> Result<(), Error>` ### Parameters - **prompt** (str) - The input text to generate from. - **max_tokens** (usize) - The maximum number of tokens to generate. - **callback** (Fn(String)) - A closure called for each generated token. ### Example ```rust engine.generate("Hello", 128, |tok| print!("{tok}")).expect("generation failed"); ``` ``` -------------------------------- ### InferenceEngine::generate Source: https://docs.rs/oxillama Generates text based on an input prompt using the loaded model. ```APIDOC ## InferenceEngine::generate ### Description Generates text tokens based on a provided prompt string and a maximum token count, utilizing a callback function to process each generated token. ### Signature `engine.generate(prompt: &str, max_tokens: usize, callback: Fn(String)) -> Result<(), Error>` ### Parameters - **prompt** (str) - Required - The input text to generate from. - **max_tokens** (usize) - Required - The maximum number of tokens to generate. - **callback** (Fn(String)) - Required - A closure that receives each generated token as a string. ### Example ```rust engine.generate("Hello", 128, |tok| print!("{tok}")).expect("generation failed"); ``` ``` -------------------------------- ### POST /admin/loras Source: https://docs.rs/oxillama-server/%5E0.1.3 Register a LoRA adapter (admin). ```APIDOC ## POST /admin/loras ### Description Register a LoRA adapter (admin). ### Method POST ### Endpoint /admin/loras ``` -------------------------------- ### InferenceEngine::generate Source: https://docs.rs/oxillama/latest/oxillama/index.html Generates text completions using the loaded model within the inference engine. ```APIDOC ## InferenceEngine::generate ### Description Generates text based on an input prompt using the configured inference engine. ### Signature `pub fn generate(&mut self, prompt: &str, max_tokens: usize, callback: impl FnMut(String)) -> Result<(), Error>` ### Parameters - **prompt** (str) - The input text prompt for the model. - **max_tokens** (usize) - The maximum number of tokens to generate. - **callback** (FnMut) - A closure called for each generated token. ### Example ```rust engine.generate("Hello", 128, |tok| print!("{tok}")).expect("generation failed"); ``` ``` -------------------------------- ### InferenceEngine::generate Source: https://docs.rs/oxillama/latest/src/oxillama/lib.rs.html Performs text generation using the loaded model with the specified configuration. ```APIDOC ## InferenceEngine::generate ### Description Generates text based on a prompt using the initialized inference engine. ### Method Rust Method ### Signature `engine.generate(prompt: &str, max_tokens: usize, callback: Fn(String)) -> Result<(), Error>` ### Parameters - **prompt** (str) - Required - The input text to generate from. - **max_tokens** (usize) - Required - The maximum number of tokens to generate. - **callback** (Fn) - Required - A closure called for each generated token. ### Request Example ```rust engine.generate("Hello", 128, |tok| print!("{tok}")); ``` ``` -------------------------------- ### Crate Module Re-exports Source: https://docs.rs/oxillama/latest/oxillama/index.html The primary re-exports for the OxiLLaMa API surface. ```rust pub use oxillama_gguf as gguf; ``` ```rust pub use oxillama_quant as quant; ``` ```rust pub use oxillama_arch as arch; ``` ```rust pub use oxillama_runtime as runtime; ``` ```rust pub use oxillama_server as server; ``` ```rust pub use oxillama_bench as bench; ``` ```rust pub use oxillama_gpu as gpu; ``` -------------------------------- ### POST /admin/models/unload Source: https://docs.rs/oxillama-server/%5E0.1.3 Unload model (admin). ```APIDOC ## POST /admin/models/unload ### Description Unload model (admin). ### Method POST ### Endpoint /admin/models/unload ``` -------------------------------- ### POST /v1/embeddings Source: https://docs.rs/oxillama-server/%5E0.1.3 Text embeddings endpoint. ```APIDOC ## POST /v1/embeddings ### Description Text embeddings endpoint. ### Method POST ### Endpoint /v1/embeddings ``` -------------------------------- ### POST /v1/threads/:thread_id/messages Source: https://docs.rs/oxillama-server/%5E0.1.3 Append message to thread. ```APIDOC ## POST /v1/threads/:thread_id/messages ### Description Append message to thread. ### Method POST ### Endpoint /v1/threads/:thread_id/messages ### Parameters #### Path Parameters - **thread_id** (string) - Required - Thread identifier ``` -------------------------------- ### POST /v1/threads/:thread_id/runs/:run_id/cancel Source: https://docs.rs/oxillama-server/%5E0.1.3 Cancel a run. ```APIDOC ## POST /v1/threads/:thread_id/runs/:run_id/cancel ### Description Cancel a run. ### Method POST ### Endpoint /v1/threads/:thread_id/runs/:run_id/cancel ### Parameters #### Path Parameters - **thread_id** (string) - Required - Thread identifier - **run_id** (string) - Required - Run identifier ``` -------------------------------- ### POST /v1/batches/:id/cancel Source: https://docs.rs/oxillama-server/%5E0.1.3 Cancel batch job. ```APIDOC ## POST /v1/batches/:id/cancel ### Description Cancel batch job. ### Method POST ### Endpoint /v1/batches/:id/cancel ### Parameters #### Path Parameters - **id** (string) - Required - Batch job identifier ``` -------------------------------- ### DELETE /admin/loras/{name} Source: https://docs.rs/oxillama-server/%5E0.1.3 Unregister a LoRA adapter (admin). ```APIDOC ## DELETE /admin/loras/{name} ### Description Unregister a LoRA adapter (admin). ### Method DELETE ### Endpoint /admin/loras/{name} ### Parameters #### Path Parameters - **name** (string) - Required - LoRA adapter name ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.