### Install Project Dependencies

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Installs the necessary Node.js dependencies for the project using npm. This is required before building the frontend or running development servers.

```bash
npm install
```

--------------------------------

### Clone Oxide Lab Repository

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Clones the Oxide Lab Git repository and navigates into the project directory. This is the first step for building the application from source.

```bash
git clone https://github.com/oxide-lab/oxide-lab.git
cd oxide-lab
```

--------------------------------

### Install Frontend Dependencies with npm

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

This snippet helps resolve frontend build issues by cleaning the npm cache, removing existing node_modules and package-lock.json, and then reinstalling all project dependencies.

```bash
# If packages fail to install
npm cache clean --force
rm -rf node_modules package-lock.json
npm install
```

--------------------------------

### Svelte Configuration for Static Adaptation

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Sets up SvelteKit with the static adapter for building a Single Page Application (SPA) suitable for Tauri integration. It includes Vite preprocessing for optimized builds and configures the static adapter with a fallback to index.html for routing.

```javascript
import adapter from "@sveltejs/adapter-static";
import { vitePreprocess } from "@sveltejs/vite-plugin-svelte";

const config = {
  preprocess: vitePreprocess(),
  kit: {
    adapter: adapter({
      fallback: "index.html",
    }),
  }
};

export default config;
```

--------------------------------

### Develop Oxide Lab Application

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Starts the development server for Oxide Lab using Tauri. This command is used for actively developing the application, allowing for hot-reloading and debugging.

```bash
npm run tauri dev
```

--------------------------------

### Production Build for Oxide Lab

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Performs a production build for Oxide Lab, first building the frontend assets with npm and then compiling the Rust backend with Cargo Tauri. This prepares the application for distribution.

```bash
npm run build && cargo tauri build
```

--------------------------------

### Build Oxide Lab Application

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Compiles the Oxide Lab application using Cargo Tauri. This command builds the frontend with Vite and the Rust backend with Tauri, creating a bundled application.

```bash
cargo tauri build
```

--------------------------------

### Tauri Configuration for Desktop App

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Configures the Tauri application, defining build commands, development server URL, frontend distribution path, and window properties like title, decorations, and dimensions. It ensures the frontend is built and served correctly during development and specifies the initial window's appearance and constraints.

```json
{
  "build": {
    "beforeDevCommand": "npm run dev",
    "devUrl": "http://localhost:1420",
    "beforeBuildCommand": "npm run build",
    "frontendDist": "../build"
  },
  "app": {
    "windows": [
      {
        "title": "Oxide Lab",
        "decorations": false,
        "width": 1280,
        "height": 720,
        "minWidth": 640,
        "minHeight": 360
      }
    ]
  }
}
```

--------------------------------

### Install CUDA Toolkit Dependencies (Ubuntu/Debian)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

A command to install the CUDA toolkit and development libraries on Ubuntu or Debian-based Linux distributions using apt.

```bash
# Ubuntu/Debian
sudo apt install nvidia-cuda-toolkit nvidia-cuda-dev
```

--------------------------------

### Vite Configuration for SvelteKit Development

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Configures Vite for a SvelteKit project, specifying the development server port, enabling strict port usage, and ignoring the src-tauri directory in watch mode. It integrates SvelteKit plugins and ensures the development server runs on the port defined in Tauri's configuration.

```javascript
import { defineConfig } from "vite";
import { sveltekit } from "@sveltejs/kit/vite";

export default defineConfig(async () => ({
  plugins: [sveltekit()],
  clearScreen: false,
  server: {
    port: 1420,
    strictPort: false,
    watch: {
      ignored: ["**/src-tauri/**"],
    },
  },
}));
```

--------------------------------

### Configure CUDA Environment Variables

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

This bash script sets the necessary environment variables for the CUDA toolkit, specifying the CUDA home directory and adding its bin directory to the system's PATH. This is crucial for CUDA-enabled builds.

```bash
export CUDA_HOME=/usr/local/cuda
export PATH=$CUDA_HOME/bin:$PATH
```

--------------------------------

### Clean and Rebuild Rust (Cargo) Dependencies

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

This command sequence is used to clean the Cargo build cache, update dependencies to their latest allowed versions, and then rebuild the project, which can resolve linking errors and missing symbols.

```bash
# Clean and rebuild Cargo dependencies
cargo clean
cargo update
cargo build
```

--------------------------------

### Cargo Configuration for Rust Dependencies

Source: https://github.com/ferrismind/oxide-lab/wiki/18. Getting Started

Defines dependencies for a Rust project using Cargo, including Tauri for desktop integration, Candle for ML operations, and Hugging Face Hub for model access. It specifies optional CUDA support, enabling GPU acceleration for Candle by default.

```toml
[dependencies]
tauri = { version = "2", features = [] }
candle = { package = "candle-core", git = "https://github.com/huggingface/candle.git", default-features = false }
candle-nn = { git = "https://github.com/huggingface/candle.git", default-features = false }
hf-hub = "0.4"

[features]
default = ["cuda"]
cuda = [
    "candle/cuda",
    "candle-nn/cuda",
    "candle-transformers/cuda",
]
```

--------------------------------

### Running Tauri Application in Development Mode

Source: https://github.com/ferrismind/oxide-lab/wiki/26. Development And Contribution

This command initiates the Tauri application in development mode. It starts the Vite development server for the frontend, compiles the Rust backend with debug symbols, launches the Tauri window, and sets up a WebSocket connection for hot reloading.

```bash
cargo tauri dev

```

--------------------------------

### Rust: Streaming Processing of Large Dataset Example

Source: https://github.com/ferrismind/oxide-lab/wiki/15. Multimodal Preprocessing

Provides a practical example of using `MultimodalBatchProcessor` for streaming processing of a large dataset. It demonstrates initialization and the call to `process_streaming` with a closure for batch processing.

```rust
// Create batch processor
let config = MultimodalConfig::default();
let device = Device::Cpu;
let processor = MultimodalProcessor::new(config, device);
let batch_processor = MultimodalBatchProcessor::new(processor, 8);

// Process large dataset in streaming mode
let samples = // ... large collection of multimodal samples
batch_processor.process_streaming(samples, |processed_chunk| {
    // Process each batch
    for sample in processed_chunk {
        // Send to model for inference
        // Save to disk
        // Or other processing
    }
    Ok(())
})?;
```

--------------------------------

### Install CUDA Toolkit Dependencies (RHEL/CentOS)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

A command to install the CUDA toolkit on RHEL or CentOS-based Linux distributions using yum. Specify the desired CUDA version.

```bash
# RHEL/CentOS
sudo yum install cuda-toolkit-12-3
```

--------------------------------

### CUDA Backend Initialization Flow

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Illustrates the sequence of operations for initializing the CUDA backend, from creating a context to loading kernels and returning a device instance. This process ensures all necessary CUDA components are set up correctly.

```mermaid
flowchart TD
A["Initialize CudaContext"] --> B["Create CudaStream"]
B --> C["Initialize cuBLAS handle"]
C --> D["Initialize cuRAND generator"]
D --> E["Load CUDA kernels (PTX)"]
E --> F["Return CudaDevice instance"]
```

--------------------------------

### Programmatic Verification of CUDA Setup (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Demonstrates how to programmatically verify a working CUDA setup in Rust. It attempts to create a CUDA device, performs a tensor operation, and asserts the result's dimensions. Requires `candle-core` and `Tensor`.

```rust
use candle_core::Device;
use candle_core::Tensor;

fn verify_cuda_setup() -> Result<(), Box<dyn std::error::Error>> {
    // Attempt to create CUDA device
    let device = Device::new_cuda(0)?;
    
    // Test basic operations
    let tensor = Tensor::ones(&[100, 100], candle_core::DType::F32, &device)?;
    let result = tensor.matmul(&tensor.t()?)?;
    
    // Verify result
    assert_eq!(result.dims(), &[100, 100]);
    
    println!("CUDA setup verified successfully!");
    println!("Device: {:?}", device);
    println!("Result shape: {:?}", result.dims());
    
    Ok(())
}
```

--------------------------------

### GGUF Inference Examples

Source: https://github.com/ferrismind/oxide-lab/blob/main/MODEL_SUPPORT_SUMMARY.md

Examples demonstrating native GGUF inference support for various models using the Candle framework. These are considered high priority and easy to implement.

```Rust
quantized-llama
quantized-gemma
quantized-qwen2
quantized-qwen3
quantized
candle_transformers
```

--------------------------------

### Rust Backend Testing Structure

Source: https://github.com/ferrismind/oxide-lab/wiki/26. Development And Contribution

Provides an example structure for writing backend tests in Rust using the built-in testing framework. These tests should cover essential functionalities like model loading, token generation accuracy, and error handling.

```rust
#[cfg(test)]
mod tests {
    use super::*;
    
    #[test]
    fn test_model_load() {
        // Test model loading from GGUF file
    }
    
    #[test]
    fn test_token_generation() {
        // Test forward pass and token output
    }
}

```

--------------------------------

### System-Level CUDA Verification (Bash)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

A set of bash commands to verify the CUDA environment on a system. Includes checking GPU availability, CUDA version, library links, and running a simple CUDA program.

```bash
# Check GPU availability
nvidia-smi

# Verify CUDA version
nvcc --version

# Check library links
ldconfig -p | grep cuda

# Test with simple CUDA program
cat > test.cu << 'EOF'
#include <stdio.h>
#include <cuda_runtime.h>

int main() {
    int deviceCount;
    cudaGetDeviceCount(&deviceCount);
    printf("Found %d CUDA devices\n", deviceCount);
    for (int i = 0; i < deviceCount; ++i) {
        cudaDeviceProp prop;
        cudaGetDeviceProperties(&prop, i);
        printf("Device %d: %s\n", i, prop.name);
    }
    return 0;
}
EOF

nvcc test.cu -o test && ./test
```

--------------------------------

### Safetensors Inference Examples

Source: https://github.com/ferrismind/oxide-lab/blob/main/MODEL_SUPPORT_SUMMARY.md

Examples demonstrating native Safetensors inference support for various models using the Candle framework's `candle_transformers` library. These have medium priority and require medium implementation effort.

```Rust
candle_transformers::models::llama
candle_transformers::models::gemma
candle_transformers::models::qwen2
candle_transformers::models::qwen3
candle_transformers::models::mistral
candle_transformers::models::mixtral
candle_transformers::models::phi3
candle_transformers::models::yi
```

--------------------------------

### Rust Logging Configuration Example

Source: https://github.com/ferrismind/oxide-lab/wiki/16. Structured Logging System

Provides an example of how log messages are formatted and outputted based on the system's configuration. It demonstrates the inclusion of timestamp, log level, component name in brackets, and the log message itself, typical for structured logging outputs.

```rust
1700000000 INFO [load] Starting model load from models/qwen3.gguf
1700000005 INFO [hub] Downloading model qwen3 from HuggingFace
1700000010 INFO [device] Using CUDA device: NVIDIA RTX 4090

```

--------------------------------

### Add User to Video Group (Linux)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Command to add the current user to the 'video' group on Linux systems, which is often necessary to grant permissions for accessing GPU devices.

```bash
sudo usermod -aG video $USER
```

--------------------------------

### Loading Model Weights from Local Storage (Rust Example)

Source: https://github.com/ferrismind/oxide-lab/blob/main/docs/WEIGHTS_LOADER.md

Illustrates how to load model weights from a local directory. This includes listing safetensors files, validating them, and building a VarBuilder.

```rust
use llm_chat_lib::core::weights;
use candle::Device;
use std::path::Path;

// List safetensors files
let model_dir = Path::new("/path/to/model");
let safetensors_files = weights::local_list_safetensors(model_dir)?;

// Validate files
weights::validate_safetensors_files(&safetensors_files)?;

// Build VarBuilder
let device = Device::Cpu;
let vb = weights::build_varbuilder(&safetensors_files, &device)?;
```

--------------------------------

### Load Model Example Usage (TypeScript)

Source: https://github.com/ferrismind/oxide-lab/wiki/27. Api Reference

Demonstrates how to invoke the `load_model` function using the Tauri invoke API. Examples show loading from Hugging Face Hub and a local path with different device configurations.

```typescript
// Load model from Hugging Face Hub (safetensors)
await invoke("load_model", {
  req: {
    format: "hub_safetensors",
    repo_id: "Qwen/Qwen2-7B-Instruct",
    context_length: 32768,
    device: { kind: "auto" }
  }
});

// Load model from local path (safetensors)
await invoke("load_model", {
  req: {
    format: "local_safetensors",
    model_path: "/path/to/local/model/directory",
    context_length: 32768,
    device: { kind: "cuda", index: 0 }
  }
});
```

--------------------------------

### Rust: Image Classification with CLIP

Source: https://github.com/ferrismind/oxide-lab/wiki/15. Multimodal Preprocessing

Example demonstrating image classification using CLIP with the `MultimodalProcessor`. It shows how to create a CLIP-optimized configuration, process a single image, and process multiple images in a batch.

```rust
use candle::{Device, Result};
use std::path::Path;

// Create CLIP-optimized configuration
let config = MultimodalConfig::clip();
let device = Device::Cpu;
let processor = MultimodalProcessor::new(config, device);

// Process a single image
let image_tensor = processor.process_image("path/to/image.jpg")?;
println!("Image tensor shape: {:?}", image_tensor.dims());

// Process multiple images in batch
let image_paths = vec![
    Path::new("image1.jpg"),
    Path::new("image2.jpg"),
    Path::new("image3.jpg"),
];
let batch_tensor = processor.process_images_batch(image_paths)?;
println!("Batch tensor shape: {:?}", batch_tensor.dims());
```

--------------------------------

### CudaDevice Structure and Methods

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Defines the `CudaDevice` struct, which encapsulates CUDA resources like context, stream, cuBLAS, and RNG. It outlines key methods for device management, kernel loading, and memory operations.

```mermaid
classDiagram
class CudaDevice {
+DeviceId id
+Arc<CudaContext> context
+Arc<CudaStream> stream
+Arc<CudaBlas> blas
+Arc<Mutex<CudaRng>> curand
+Arc<RwLock<ModuleStore>> modules
+Arc<RwLock<HashMap<String, Arc<CudaModule>>>> custom_modules
+new(ordinal : usize) Result<CudaDevice>
+get_or_load_func(fn_name : &str, mdl : &Module) Result<CudaFunc>
+alloc<T>(len : usize) Result<CudaSlice<T>>
+memcpy_htod<Src, Dst>(src : &Src, dst : &mut Dst) Result<()>
}
class CudaContext {
+new(ordinal : usize) Result<CudaContext>
+disable_event_tracking()
+is_event_tracking() bool
+load_module(ptx : &str) Result<CudaModule>
}
class CudaStream {
+new() Result<CudaStream>
+alloc<T>(len : usize) Result<CudaSlice<T>>
+memcpy_htod<Src, Dst>(src : &Src, dst : &mut Dst) Result<()>
}
class CudaBlas {
+new(stream : CudaStream) Result<CudaBlas>
+gemm_strided_batched_ex(...) Result<()>
}
class CudaRng {
+new(seed : u64, stream : CudaStream) Result<CudaRng>
+fill_with_uniform<T>(data : &mut CudaSlice<T>) Result<()>
}
CudaDevice --> CudaContext : "owns"
CudaDevice --> CudaStream : "uses"
CudaDevice --> CudaBlas : "uses"
CudaDevice --> CudaRng : "uses"
```

--------------------------------

### Mermaid Diagram: User Interaction Flow

Source: https://github.com/ferrismind/oxide-lab/wiki/20. User Interface Guide

Visualizes the step-by-step user interaction flow, from user input to the Tauri backend processing and response streaming back to the UI.

```mermaid
sequenceDiagram
participant User as "User"
participant UI as "UI Component"
participant Controller as "ChatController"
participant Tauri as "Tauri Backend"
User->>UI : Type message and click Send
UI->>Controller : handleSend()
Controller->>Controller : Validate input and model state
Controller->>Controller : Add user message to history
Controller->>Controller : Scroll to bottom
Controller->>Controller : generateFromHistory()
Controller->>Tauri : invoke("generate_stream")
Tauri->>Tauri : Process request and stream response
Tauri->>Controller : Stream data events
Controller->>Controller : parse stream data
Controller->>Controller : appendSegments()
Controller->>UI : Update DOM with new content
loop Stream Processing
Tauri->>Controller : Stream data events
Controller->>Controller : parse and render segments
Controller->>UI : Update DOM incrementally
end
Controller->>UI : Complete response rendering
```

--------------------------------

### Rust: Training with Data Augmentation

Source: https://github.com/ferrismind/oxide-lab/wiki/15. Multimodal Preprocessing

Illustrates using the `MultimodalProcessor` for training, including data augmentation. This example shows how to create a training-optimized configuration and apply augmentations to processed images.

```rust
// Create training-optimized configuration
let config = MultimodalConfig::training();
let device = Device::Cpu;
let mut processor = MultimodalProcessor::new(config, device);

// Process image with augmentations
let image_tensor = processor.process_image("training_image.jpg")?;
let augmented_tensor = processor.augment(&image_tensor, Modality::Vision)?;
```

--------------------------------

### Rust: Deterministic Code Generation Settings Example

Source: https://github.com/ferrismind/oxide-lab/wiki/23. Troubleshooting

An example Rust struct `GenerateRequest` for deterministic code generation. It features a lower temperature (0.3), a top_p of 0.9, top_k set to 15, and a repeat_penalty of 1.2 for consistent and syntactically correct output.

```rust
let req = GenerateRequest {
    prompt: "Implement binary search in Python".to_string(),
    temperature: Some(0.3),
    top_p: Some(0.9),
    top_k: Some(15),
    min_p: None,
    repeat_penalty: Some(1.2),
    repeat_last_n: 32,
    use_custom_params: true,
    seed: Some(42),
};

```

--------------------------------

### Load Custom CUDA Kernels (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Demonstrates how to load and use custom CUDA kernels within the oxide-lab project. It involves specifying the CUDA device, providing the PTX code for the kernel, and then loading it for execution.

```rust
let device = Device::new_cuda(0)?;
let ptx_code = include_str!("my_kernel.ptx");
let func = device.get_or_load_custom_func("my_kernel", "my_module", ptx_code)?;
```

--------------------------------

### Rust: Speech Recognition with Whisper

Source: https://github.com/ferrismind/oxide-lab/wiki/15. Multimodal Preprocessing

Demonstrates speech recognition using Whisper with the `MultimodalProcessor`. This example covers creating a Whisper-optimized configuration, processing an audio file, and processing raw audio samples.

```rust
// Create Whisper-optimized configuration
let config = MultimodalConfig::whisper();
let device = Device::Cpu;
let processor = MultimodalProcessor::new(config, device);

// Process audio file
let audio_tensor = processor.process_audio("path/to/audio.wav")?;
println!("Audio tensor shape: {:?}", audio_tensor.dims());

// Process raw audio samples
let samples: Vec<f32> = // ... audio samples
let audio_tensor = processor.process_audio_samples(&samples, 16000)?;
```

--------------------------------

### Enabling Verbose CUDA Logging (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

This Rust snippet shows how to enable verbose logging for CUDA operations within the `candle-core` framework. It involves setting environment variables before initializing the logger.

```rust
// Set environment variable before initialization
std::env::set_var("CUDA_BACKEND_VERBOSE", "1");

// Or use RUST_LOG for comprehensive logging
std::env::set_var("RUST_LOG", "candle_core=debug,cuda=trace");

// Initialize logger
env_logger::init();
```

--------------------------------

### CUDA Kernel Loading Process

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Describes the kernel management system where functions are loaded lazily and cached. `get_or_load_func` checks a cache, loads PTX modules if necessary, extracts the function, caches it, and returns it.

```mermaid
flowchart LR
A["Request Function: get_or_load_func()"] --> B{"Function in Cache?"}
B --> |Yes| C["Return Cached Function"]
B --> |No| D["Load PTX Module"]
D --> E["Extract Function"]
E --> F["Cache Function"]
F --> G["Return Function"]
```

--------------------------------

### Display Application Version in Sidebar - HTML

Source: https://github.com/ferrismind/oxide-lab/wiki/20. User Interface Guide

A simple HTML structure to display the application version within the sidebar footer. It uses a small text element to show the version number, prefixed with 'v'.

```html
<div class="sidebar-footer">
  <div class="footer-info">
    <small>v{appVersion}</small>
  </div>
</div>
```

--------------------------------

### Auto-Selection Verification (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Verifies the auto-selection process for CUDA and other devices in Rust. It checks for CUDA and Metal availability and attempts to select the best available device, falling back to CPU if necessary. Requires `candle` crate.

```rust
use candle::Device;
use candle::utils::{cuda_is_available, metal_is_available};

fn verify_auto_selection() {
    println!("CUDA available: {}", cuda_is_available());
    println!("Metal available: {}", metal_is_available());
    
    // Test auto-selection
    let device = match Device::cuda_if_available(0) {
        Ok(device) => {
            println!("Successfully selected device: {:?}", device);
            device
        }
        Err(e) => {
            println!("Device selection failed: {}", e);
            Device::Cpu
        }
    };
    
    println!("Final device: {:?}", device);
}
```

--------------------------------

### Rust: Memory-Efficient Precision Configuration

Source: https://github.com/ferrismind/oxide-lab/wiki/17. Precision Policy Configuration

Illustrates a memory-efficient precision configuration. This example creates a memory-efficient PrecisionConfig, initializes a Metal device, and selects the DType.

```rust
let config = PrecisionConfig::memory_efficient();
let device = Device::new_metal(0).unwrap();
let dtype = select_dtype(&device, &config);
// Result: DType::F16 for Metal device
```

--------------------------------

### Candle GGUF Support for Quantized Models

Source: https://github.com/ferrismind/oxide-lab/blob/main/arch.md

Illustrates the support for quantized models using the GGUF format within Candle. This is facilitated by the `quantized-llama`, `quantized-gemma`, and `quantized-qwen2`/`quantized-qwen3` examples, along with a generic `quantized` example applicable to Mistral, Mixtral, Phi-3, and SmolLM 2. Yi models are supported via `candle_transformers`.

```rust
use candle_core::Device;

// Example for Llama, Qwen2, Qwen3, Mistral, Mixtral, Phi-3, SmolLM 2
// let mut ggml_params = GgmlCommonBuilder::new();
// ggml_params.model_type(ModelType::Llama);
// let model = ggml_params.build(path)?;

// Example for Gemma
// let mut ggml_params = GgmlCommonBuilder::new();
// ggml_params.model_type(ModelType::Gemma);
// let model = ggml_params.build(path)?;

// Example for Yi
// let mut ggml_params = GgmlCommonBuilder::new();
// ggml_params.model_type(ModelType::Yi);
// let model = ggml_params.build(path)?;

```

--------------------------------

### NPM Scripts for Linting and Formatting

Source: https://github.com/ferrismind/oxide-lab/wiki/26. Development And Contribution

Defines scripts in package.json for running ESLint and Stylelint. Includes commands for linting and automatically fixing linting issues. Also configures lint-staged for pre-commit hooks.

```json
"scripts": {
  "lint": "eslint \"src/**/*.{ts,js,svelte}\" --ext .ts,.js,.svelte",
  "lint:fix": "eslint \"src/**/*.{ts,js,svelte}\" --ext .ts,.js,.svelte --fix"
},
"lint-staged": {
  "*.{ts,js,svelte}": "npx eslint --cache --fix",
  "*.css": "npx stylelint --fix"
}
```

--------------------------------

### Manage CUDA Backend Device (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/25. Api Reference

Handles CUDA initialization errors and manages device selection within the Candle framework. It ensures compatibility and proper setup for GPU acceleration.

```rust
impl CandleDevice for Device {
    fn device(&self) -> Option<Self> {
        // ... implementation details ...
        Some(Device::Cuda)
    }
}
```

--------------------------------

### Mermaid: Build Toolchain Overview

Source: https://github.com/ferrismind/oxide-lab/wiki/26. Development And Contribution

Visualizes the build process and tooling for the Oxide Lab application. It outlines the steps from source code to final binary for both development and production builds, integrating Vite, Tauri, and Rust Cargo.

```mermaid
flowchart LR
A[Source Code] --> B{Development Mode}
A --> C{Production Build}
B --> D[Vite Development Server]
D --> E[Tauri Dev Bridge]
E --> F[Rust Cargo Build]
F --> G[Hot Reload]
C --> H[Vite Build]
H --> I[Frontend Bundle]
I --> J[Tauri Bundle]
J --> K[Rust Cargo Build]
K --> L[Final Binary]
style B fill:#f97316,stroke:#ea580c,color:white
style C fill:#059669,stroke:#047857,color:white
style D fill:#3b82f6,stroke:#2563eb,color:white
style F fill:#059669,stroke:#047857,color:white
style H fill:#3b82f6,stroke:#2563eb,color:white
style J fill:#14b8a6,stroke:#0d9488,color:white
style K fill:#059669,stroke:#047857,color:white
```

--------------------------------

### Model Loading Architecture Diagram

Source: https://github.com/ferrismind/oxide-lab/wiki/5. Extensibility Architecture

This diagram illustrates the process of loading models, including parsing requests, handling different model formats (GGUF, SafeTensors) from local paths or Hugging Face Hub, initializing the model with Candle, and configuring it for execution. It shows the flow from a load request to a successfully loaded model.

```mermaid
flowchart TD
Start([Load Model Request]) --> ParseRequest["Parse LoadRequest Type"]
ParseRequest --> RequestType{"Request Type?"}
RequestType --> |Gguf| LoadLocalGGUF["Load from Local GGUF File"]
LoadLocalGGUF --> InitializeModel["Initialize Model with Candle"]
LoadLocalGGUF --> LoadTokenizer["Load Tokenizer"]
RequestType --> |HubGguf| DownloadGGUF["Download GGUF from Hugging Face Hub"]
DownloadGGUF --> InitializeModel
RequestType --> |HubSafetensors| DownloadSafeTensors["Download SafeTensors from Hugging Face Hub"]
DownloadSafeTensors --> InitializeModel
RequestType --> |LocalSafetensors| LoadLocalSafeTensors["Load SafeTensors from Local Path"]
LoadLocalSafeTensors --> InitializeModel
InitializeModel --> ConfigureContext["Set Context Length"]
ConfigureContext --> SetDevice["Move to Selected Device"]
SetDevice --> UpdateState["Update Shared State"]
UpdateState --> End([Model Loaded])
style Start fill:#4CAF50,stroke:#388E3C
style End fill:#4CAF50,stroke:#388E3C
style LoadLocalGGUF fill:#2196F3,stroke:#1976D2
style DownloadGGUF fill:#2196F3,stroke:#1976D2
style DownloadSafeTensors fill:#2196F3,stroke:#1976D2
style LoadLocalSafeTensors fill:#2196F3,stroke:#1976D2
style InitializeModel fill:#FF9800,stroke:#F57C00
style LoadTokenizer fill:#FF9800,stroke:#F57C00
style ConfigureContext fill:#00BCD4,stroke:#0097A7
style SetDevice fill:#00BCD4,stroke:#0097A7
style UpdateState fill:#9C27B0,stroke:#7B1FA2
```

--------------------------------

### Enable Window Dragging on Header Click (Svelte)

Source: https://github.com/ferrismind/oxide-lab/wiki/20. User Interface Guide

Attaches a mousedown event listener to the application header to initiate window dragging. The event handler checks if the click occurred on an interactive element before starting the drag.

```svelte
<header class="app-header" onmousedown={startDragging}>


```

--------------------------------

### Start Window Dragging Logic (TypeScript)

Source: https://github.com/ferrismind/oxide-lab/wiki/20. User Interface Guide

This TypeScript function, `startDragging`, is triggered on a mousedown event. It prevents dragging if the click target is a button, input, or an element explicitly marked to disable drag regions. It then calls Tauri's `appWindow.startDragging()` to move the window.

```typescript
async function startDragging(event: MouseEvent) {
  // Only start dragging if we're not clicking on an interactive element
  const target = event.target as HTMLElement;
  if (!target.closest('button, input, [data-tauri-drag-region="false"]')) {
    await appWindow.startDragging();
  }
}

```

--------------------------------

### Language Detection from Code Element

Source: https://github.com/ferrismind/oxide-lab/wiki/20. User Interface Guide

This TypeScript function extracts the programming language from a given DOM 'Element'. It checks for class names starting with 'language-' or 'lang-', and also inspects the parent element. If no class-based detection is successful, it attempts to detect the language from the element's text content.

```typescript
private extractLanguage(codeElement: Element): string {
  const classList = Array.from(codeElement.classList);
  
  for (const className of classList) {
    if (className.startsWith('hljs-')) {
      continue;
    }
    if (className.startsWith('language-')) {
      return className.replace('language-', '');
    }
    if (className.startsWith('lang-')) {
      return className.replace('lang-', '');
    }
  }

  // Check parent pre element
  if (codeElement.parentElement) {
    const parentClassList = Array.from(codeElement.parentElement.classList);
    for (const className of parentClassList) {
      if (className.startsWith('language-')) {
        return className.replace('language-', '');
      }
    }
  }

  // Try to detect language from content
  return this.detectLanguageFromContent(codeElement.textContent || '');
}
```

--------------------------------

### Loading Model Weights from Hugging Face Hub (Rust Example)

Source: https://github.com/ferrismind/oxide-lab/blob/main/docs/WEIGHTS_LOADER.md

Demonstrates the workflow for loading model weights from Hugging Face Hub. It involves setting up the API, listing files, caching them, validating, and finally building a VarBuilder.

```rust
use hf_hub::{api::sync::Api, Repo, RepoType};
use llm_chat_lib::core::weights;
use candle::Device;

// Setup API and repository
let api = Api::new().map_err(|e| e.to_string())?;
let repo = api.repo(Repo::new("owner/repo".to_string(), RepoType::Model));

// List safetensors files
let safetensors_files = weights::hub_list_safetensors(&repo)?;

// Download files to cache
let cached_paths = weights::hub_cache_safetensors(&repo, &safetensors_files)?;

// Validate files
weights::validate_safetensors_files(&cached_paths)?;

// Build VarBuilder
let device = Device::Cpu;
let vb = weights::build_varbuilder(&cached_paths, &device)?;
```

--------------------------------

### Rust Device Initialization and Fallback Logic

Source: https://github.com/ferrismind/oxide-lab/wiki/6. Device Auto Selection

Illustrates the error handling and fallback logic for device initialization. It shows how to attempt CUDA initialization and fall back to CPU if it fails, applicable for both auto-selection and explicit device preferences.

```rust
// In auto-selection mode
if cuda_is_available() {
    match Device::new_cuda(0) {
        Ok(device) => {
            println!("[device] auto-selected CUDA");
            return device;
        }
        Err(e) => {
            eprintln!("[device] CUDA init failed: {}, falling back to next option", e);
        }
    }
}

// For explicit CUDA selection
DevicePreference::Cuda { index } => {
    match Device::new_cuda(index) {
        Ok(device) => device,
        Err(e) => {
            eprintln!("[device] CUDA init failed: {}, falling back to CPU", e);
            Device::Cpu
        }
    }
}
```

--------------------------------

### Rust: Balanced Factual Response Settings Example

Source: https://github.com/ferrismind/oxide-lab/wiki/23. Troubleshooting

This Rust code example demonstrates a `GenerateRequest` configuration for balanced factual responses. It uses a temperature of 0.4, top_p of 0.8, and top_k of 8, with a mild repeat_penalty of 1.05.

```rust
let req = GenerateRequest {
    prompt: "Explain the theory of relativity".to_string(),
    temperature: Some(0.4),
    top_p: Some(0.8),
    top_k: Some(8),
    min_p: None,
    repeat_penalty: Some(1.05),
    repeat_last_n: 128,
    use_custom_params: true,
    seed: None,
};

```

--------------------------------

### Fallback Mechanism for Prompt Construction

Source: https://github.com/ferrismind/oxide-lab/wiki/19. Core Concepts/19.5. Tokenization And Prompt Formatting/19.5.2. Chat Template Application And Management

Flowchart depicting the fallback strategy for prompt construction, prioritizing native templates and falling back to a Qwen-compatible format using '<|im_start|>' and '<|im_end|>' tokens when templates are unavailable.

```mermaid
flowchart TD
A[Try to Get Template] --> B{Template Available?}
B --> |Yes| C[Use Native Template]
B --> |No| D[Use Qwen Fallback Format]
D --> E[Format with <|im_start|>/<|im_end|>
]
E --> F[Add Control Commands if Present]
F --> G[Complete Prompt Construction]
```

--------------------------------

### Rust: High Creativity Settings Example

Source: https://github.com/ferrismind/oxide-lab/wiki/23. Troubleshooting

An example Rust struct `GenerateRequest` configured for high creativity. It sets parameters like temperature to 1.0, top_p to 0.95, min_p to 0.1, and repeat_penalty to 1.1, with a specified seed for reproducibility.

```rust
let req = GenerateRequest {
    prompt: "Write a poem about quantum physics".to_string(),
    temperature: Some(1.0),
    top_p: Some(0.95),
    top_k: None,
    min_p: Some(0.1),
    repeat_penalty: Some(1.1),
    repeat_last_n: 64,
    use_custom_params: true,
    seed: Some(12345),
};

```

--------------------------------

### BlockQ4_0 Quantization Algorithm Example

Source: https://github.com/ferrismind/oxide-lab/wiki/19. Core Concepts/19.2. Model Formats (gguf & Safetensors)/19.2.3. Model Quantization Guide

Illustrates the steps involved in the BlockQ4_0 quantization algorithm, which finds the max absolute value, computes a scale factor, quantizes to 4-bit integers, and stores the scale factor and packed data.

```text
1. Finds the maximum absolute value in each block of 32 elements
2. Computes a scale factor that maps the range [-max, max] to [-8, 7]
3. Quantizes each floating-point value to a 4-bit integer using the scale factor
4. Stores the scale factor as an FP16 value and the quantized values in a compact bit-packed format
```

--------------------------------

### Qwen3ModelBuilder from VarBuilder

Source: https://github.com/ferrismind/oxide-lab/wiki/21. Model Management/21.1. Loading Local Models

Implements the from_varbuilder method for Qwen3ModelBuilder to create candle-transformers models. It parses Qwen3 configuration and initializes the model.

```rust
pub fn from_varbuilder(
    &self,
    vb: VarBuilder,
    config: &serde_json::Value,
    _device: &Device,
    _dtype: DType,
) -> Result<Box<dyn ModelBackend>, String> {
    let config_str = config.to_string();
    let qwen_config: candle_transformers::models::qwen3::Config = 
        serde_json::from_str(&config_str)
            .map_err(|e| format!("Failed to parse Qwen3 config: {}", e))?;
    
    let model = candle_transformers::models::qwen3::ModelForCausalLM::new(&qwen_config, vb)
        .map_err(|e| format!("Failed to load Qwen3 model: {}", e))?;
    
    let adapter = Qwen3CandleAdapter::new(model);
    
    Ok(Box::new(adapter))
}
```

--------------------------------

### Rust: Abstract Model Backend with AnyModel

Source: https://github.com/ferrismind/oxide-lab/wiki/24. Advanced Features/24.2. Context Window Management

Illustrates the `AnyModel` struct, which provides an abstraction layer over different model backends using dynamic dispatch. This allows for flexible model swapping while maintaining a uniform interface for context handling.

```rust
pub struct AnyModel {
    inner: Box<dyn ModelBackend + Send>,
}
```

--------------------------------

### Check CUDA Availability at Compile Time (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

This Rust snippet checks if CUDA support was compiled into the application using the `cuda_is_available()` function. It returns `true` if CUDA is available at compile time, and `false` otherwise.

```rust
use candle::utils::cuda_is_available;
println!("CUDA available at compile time: {}", cuda_is_available());
```

--------------------------------

### Node.js Project Configuration (package.json)

Source: https://github.com/ferrismind/oxide-lab/wiki/28. Appendix

Defines the frontend dependencies, development scripts, and metadata for the Svelte-based user interface of Oxide-Lab. This file is crucial for managing the project's build process, development server, and integration with the Tauri backend.

```json
{
  "name": "oxide-lab",
  "version": "0.10.23",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview",
    "tauri": "tauri",
    "tauri:dev:cpu": "tauri dev --features cpu",
    "tauri:dev:cuda": "tauri dev --features cuda",
    "tauri:build:cpu": "tauri build --features cpu",
    "tauri:build:cuda": "tauri build --features cuda"
  },
  "dependencies": {
    "@tauri-apps/api": "^1.5.0",
    "highlight.js": "^11.9.0",
    "svelte": "^3.59.2"
  },
  "devDependencies": {
    "@sveltejs/vite-plugin-svelte": "^2.4.5",
    "svelte-check": "^3.4.6",
    "tailwindcss": "^3.3.3",
    "typescript": "^5.0.2",
    "vite": "^4.4.9"
  }
}
```

--------------------------------

### Configure udev Rule for NVIDIA Permissions (Linux)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Creates a udev rule to set read-write permissions (MODE="0666") for NVIDIA devices, ensuring that all users can access them. This is an alternative to adding users to the 'video' group.

```bash
echo 'SUBSYSTEM=="nvidia", MODE="0666"' | sudo tee /etc/udev/rules.d/99-nvidia.rules
```

--------------------------------

### Top-k then Top-p Sampling Initialization (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/23. Troubleshooting

This Rust code illustrates the initialization of a LogitsProcessor for a combined Top-k and Top-p sampling strategy. It includes the seed, k value for top-k, p value for top-p, and the temperature.

```rust
LogitsProcessor::from_sampling(seed, Sampling::TopKThenTopP { k, p, temperature })
```

--------------------------------

### Configure CUDA Matrix Multiplication Precision (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/22. Device And Performance/22.2. Cuda Setup And Detection

Control reduced precision for F32, F16, and BF16 matrix multiplications in the CUDA backend. This allows balancing performance and numerical accuracy by enabling or disabling techniques like TF32. It also shows how to query the current settings.

```rust
candle_core::cuda_backend::set_gemm_reduced_precision_f32(true);
candle_core::cuda_backend::set_gemm_reduced_precision_f16(true);
candle_core::cuda_backend::set_gemm_reduced_precision_bf16(true);

// Query current settings
let f32_reduced = candle_core::cuda_backend::gemm_reduced_precision_f32();
let f16_reduced = candle_core::cuda_backend::gemm_reduced_precision_f16();
let bf16_reduced = candle_core::cuda_backend::gemm_reduced_precision_bf16();
```

--------------------------------

### Fetch and Display Application Version - Svelte/TypeScript

Source: https://github.com/ferrismind/oxide-lab/wiki/20. User Interface Guide

A Svelte component that fetches the application version using Tauri's getVersion API on component mount. It handles potential errors during the API call and displays the version or 'Unknown' if retrieval fails. Includes necessary imports from '@tauri-apps/api/app'.

```typescript
<script lang="ts">
  import { getVersion } from '@tauri-apps/api/app';
  import { onMount } from 'svelte';

  let appVersion = '';

  onMount(async () => {
    try {
      appVersion = await getVersion();
    } catch (error) {
      console.error('Failed to get app version:', error);
      appVersion = 'Unknown';
    }
  });
</script>
```

--------------------------------

### GET /probeCuda

Source: https://github.com/ferrismind/oxide-lab/wiki/27. Api Reference

Probes the system for CUDA availability and capabilities, returning build status and availability.

```APIDOC
## GET probeCuda

### Description
Probes the system for CUDA availability and capabilities.

### Method
GET

### Endpoint
/probeCuda

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```typescript
// No request body needed
```

### Response
#### Success Response (200)
- **cuda_build** (boolean) - Indicates if the binary was compiled with CUDA support.
- **ok** (boolean) - Indicates if CUDA is available on the system.
- **error** (string | null) - An error message if CUDA is not available, otherwise null.

#### Response Example
```json
{
  "cuda_build": true,
  "ok": true,
  "error": null
}
```
```

--------------------------------

### Initialize Application with Auto Device Selection (Rust)

Source: https://github.com/ferrismind/oxide-lab/wiki/6. Device Auto Selection

Demonstrates how to initialize the application by automatically selecting a compute device using `DevicePreference::Auto` instead of hardcoding CPU. This snippet is typically used during application startup.

```rust
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
    // Use auto-selection for initial device instead of hardcoding CPU
    let initial_device = select_device(Some(DevicePreference::Auto));
    let shared: SharedState<Box<dyn ModelBackend + Send>> = Arc::new(Mutex::new(ModelState::new(initial_device)));
    
    tauri::Builder::default()
        .manage(shared)
        // ... other configuration
        .run(tauri::generate_context!());
        .expect("error while running tauri application");
}
```

--------------------------------

### GET /getChatTemplate

Source: https://github.com/ferrismind/oxide-lab/wiki/27. Api Reference

Retrieves the chat template used for formatting conversations. This endpoint does not require any parameters.

```APIDOC
## GET getChatTemplate

### Description
Retrieves the chat template used for formatting conversations.

### Method
GET

### Endpoint
/getChatTemplate

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```typescript
// No request body needed
```

### Response
#### Success Response (200)
- **template** (string | null) - The chat template string or null if none exists.

#### Response Example
```json
{
  "template": "{{#each messages}}
{{#if system}}
<|im_start|>system
{{content}}<|im_end|>
{{/if}}
{{#if user}}
<|im_start|>user
{{content}}<|im_end|>
{{/if}}
{{#if assistant}}
<|im_start|>assistant
{{content}}<|im_end|>
{{/if}}
{{/each}}"
}
```
```

--------------------------------

### Get Device Info (TypeScript)

Source: https://github.com/ferrismind/oxide-lab/wiki/27. Api Reference

Retrieves information about available computational devices. No parameters are required.

```typescript
const info = await invoke<DeviceInfoDto>("get_device_info");
console.log(`CUDA available: ${info.cuda_available}`);
```