### Test missing WAVE tag (setup) Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/audio.rs.html Starts building a WAV-like byte vector with a RIFF header and a placeholder size, used for testing missing WAVE tag handling. ```rust let mut data = Vec::new(); data.extend_from_slice(b"RIFF"); data.extend_from_slice(&100_u32.to_le_bytes()); ``` -------------------------------- ### Example VTT output format Source: https://docs.rs/oxiwhisper/latest/oxiwhisper/subtitle/fn.to_vtt.html Shows the expected WebVTT output format produced by `to_vtt`. The output starts with the `WEBVTT` header, followed by timestamp lines and subtitle text. ```text WEBVTT 00:00:00.000 --> 00:00:02.500 Hello world 00:00:02.500 --> 00:00:05.000 Next line ``` -------------------------------- ### Test sgemm attention matches manual reference (setup) Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/attention.rs.html Starts a test that compares sgemm-based attention against a manual reference computation. Sets up deterministic Q, K, V data and begins building the manual score matrix. ```rust let n_head = 2; let q_len = 3; let kv_len = 4; let head_dim = 2; let scale = (head_dim as f32).sqrt().recip(); // 1/sqrt(2) // Deterministic Q, K, V data — [n_head, seq_len, head_dim] layout let q_data: Vec = (0..(n_head * q_len * head_dim)) .map(|i| (i as f32 + 1.0) * 0.1) .collect(); let k_data: Vec = (0..(n_head * kv_len * head_dim)) .map(|i| (i as f32 + 1.0) * 0.05) .collect(); let v_data: Vec = (0..(n_head * kv_len * head_dim)) .map(|i| (i as f32 + 1.0) * 0.02) .collect(); // --- Manual reference: Q @ K^T * scale --- let mut ref_scores = vec![0.0f32; n_head * q_len * kv_len]; for h in 0..n_head { for i in 0..q_len { ``` -------------------------------- ### StreamTranscriber usage example Source: https://docs.rs/oxiwhisper/latest/oxiwhisper/stream/struct.StreamTranscriber.html Shows how to create a stream, push audio chunks, process available segments, and finish the stream. The example uses default transcription options and prints segment timestamps and text. ```rust let mut stream = model.stream(TranscribeOptions::default()); // Feed audio in arbitrary-sized chunks stream.push_audio(&[0.0f32; 8000]); stream.push_audio(&[0.0f32; 8000]); // Process available segments while let Some(seg) = stream.next_segment() { let seg = seg.unwrap(); println!("[{:.1}s - {:.1}s] {}", seg.start, seg.end, seg.text); } // Flush remaining audio let result = stream.finish().unwrap(); ``` -------------------------------- ### Assert token 0 start and monotonic starts Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/dtw.rs.html Verifies that the first token starts at frame 0 and that both monotonic_peak and dp_dtw produce non-decreasing start times. Used in a test to enforce alignment invariants. ```rust // Token 0 must start at frame 0 (path begins at (0,0) → min_frame[0] = 0). assert!( dp[0].0 < one_hop, "dp_dtw token 0 must start at frame 0 (got {})", dp[0].0 ); // Both algorithms must produce non-decreasing start times. for i in 1..n_tokens { assert!( mp[i].0 >= mp[i - 1].0, "monotonic_peak: non-monotonic starts at {i}: {} < {}", mp[i].0, mp[i - 1].0 ); assert!( dp[i].0 >= dp[i - 1].0, "dp_dtw: non-monotonic starts at {i}: {} < {}", dp[i].0, dp[i - 1].0 ); } ``` -------------------------------- ### Forward Context and Decode Setup in forward.rs Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/decoder/forward.rs.html Building the ForwardCtx with cross-attention K/V and embeddings, resolving language/task tokens, combining initial prompt and previous tokens, and computing the blank token and decoding constraints before dispatch. ```rust 111 cross_v.push(to_head_first(&cv.data, n_head, enc_len, head_dim)); 112 } 113 114 let ctx = ForwardCtx { 115 cross_k: &cross_k, 116 cross_v: &cross_v, 117 enc_len, 118 tok_emb: model.get("decoder.token_embedding.weight")?, 119 pos_emb: model.get("decoder.positional_embedding")?, 120 model, 121 n_state, 122 n_layer, 123 n_head, 124 head_dim, 125 }; 126 127 // Detect language automatically if none is specified. 128 let (lang_token, detected_language) = match opts.language { 129 Some(lang) => (special.language_token(lang), None), 130 None => { 131 let token = detect_language(&ctx, &special)?; 132 let lang_code = language_code_from_token(token); 133 (token, Some(lang_code)) 134 } 135 }; 136 137 let mut all_initial = match opts.initial_prompt { 138 Some(text) => encode_prompt_text(text, &model.vocab), 139 None => Vec::new(), 140 }; 141 if let Some(prev) = opts.previous_tokens { 142 // Prepend previous tokens before any initial prompt tokens 143 let mut combined = prev.to_vec(); 144 combined.append(&mut all_initial); 145 all_initial = combined; 146 } 147 let task_token = match opts.task { 148 Task::Transcribe => special.transcribe, 149 Task::Translate => special.translate, 150 }; 151 let prompt = build_prompt( 152 &special, 153 lang_token, 154 task_token, 155 opts.timestamps, 156 &all_initial, 157 ); 158 let kv_capacity = prompt.len() + MAX_DECODE_LENGTH + 4; 159 160 // Blank token for suppress_blank: greedy longest-match of " " against vocab. 161 let blank_token: Option = encode_prompt_text(" ", &model.vocab).first().copied(); 162 163 let constraints = DecodeConstraints { 164 suppress: opts.suppress_tokens.unwrap_or(&[]), 165 no_repeat_ngram_size: opts.no_repeat_ngram_size, 166 timestamp_rules: opts.timestamps, 167 suppress_blank: opts.suppress_blank, 168 blank_token, 169 }; 170 let args = DecodeArgs { 171 kv_capacity, 172 special: &special, 173 eot_threshold: special.eot, 174 constraints: &constraints, 175 dtype: opts.kv_cache_dtype, 176 }; 177 178 // Whether to capture cross-attention for word timestamps (greedy/sample only). 179 let capturing = opts.word_timestamps && opts.beam_width <= 1; 180 181 // Accumulates attention rows from the accepted decode attempt. 182 // One row per accepted output token, each of length `enc_len`. 183 let mut last_attn: Vec = if capturing { 184 Vec::with_capacity(MAX_DECODE_LENGTH * enc_len) 185 } else { Vec::new() }; ``` ```rust // Dispatch closure: selects the sampler by temperature; t > 0 uses sample, // t == 0 uses beam or greedy. The strict > guard is load-bearing — decode_sample // divides by temperature and must never be called with t == 0. // Returns (tokens, token_probs, no_speech_prob). // Also clears and refills `last_attn` on each call (for retry correctness). let (tokens, token_probs, no_speech_prob) = { let mut decode_at = |t: f32| -> Result<(Vec, Vec, f32), String> { last_attn.clear(); let attn_opt: Option<&mut Vec> = if capturing { Some(&mut last_attn) } else { None }; if t > 0.0 { let mut o = opts.clone(); o.temperature = t; decode_sample(&prompt, &ctx, &args, &o, attn_opt) } else if opts.beam_width > 1 { decode_beam(&prompt, &ctx, &args, opts.beam_width) } else { decode_greedy(&prompt, &ctx, &args, attn_opt) } }; if opts.fallback_temperatures.is_empty() { // Default path: single dispatch, identical to prior behaviour. decode_at(opts.temperature)? } else { // Temperature fallback: try each temperature, accept the first clean result. let mut last: Option<(Vec, Vec, f32)> = None; let mut accepted = false; for &t in opts.fallback_temperatures { let (toks, probs, nsp) = decode_at(t)?; // Silence / no-speech: accept immediately, do not retry. Retrying would // risk hallucinating tokens into a genuinely silent segment. if toks.is_empty() { last = Some((toks, probs, nsp)); accepted = true; break; } let text = tokenizer::decode(&toks, &model.vocab); let avg_logprob = if probs.is_empty() { ``` -------------------------------- ### WhisperModel::from_onnx - Load ONNX model with example Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Loads a Whisper model from an ONNX file, supporting single-file and split encoder/decoder models. Automatically detects HuggingFace Optimum and OpenAI naming conventions. The example shows constructing an OnnxModelConfig and calling from_onnx. ```rust /// Load a Whisper model from an ONNX file. /// /// Supports both single-file ONNX models and split encoder/decoder models. /// Automatically detects HuggingFace Optimum and OpenAI naming conventions. /// /// # Arguments /// * `config` - Configuration specifying model path(s) and optional vocabulary file. /// /// # Example /// ```no_run /// use oxiwhisper::{WhisperModel, onnx_loader::OnnxModelConfig}; /// use std::path::PathBuf; /// /// let config = OnnxModelConfig { /// model_path: PathBuf::from("whisper-tiny.onnx"), /// decoder_path: None, /// vocab_path: Some(PathBuf::from("vocab.txt")), /// }; /// let model = WhisperModel::from_onnx(&config).unwrap(); /// ``` #[cfg(feature = "onnx")] pub fn from_onnx( config: &crate::onnx_loader::OnnxModelConfig, ) -> Result { #[cfg(feature = "timing")] eprintln!( "Loading ONNX whisper model from: {}", config.model_path.display() ); let model_data = crate::onnx_loader::load_onnx(config).map_err(OxiWhisperError::InvalidModel)?; #[cfg(feature = "timing")] eprintln!( "ONNX model loaded: {} vocab, {} audio layers, {} text layers, d_model={}", model_data.hparams.n_vocab, model_data.hparams.n_audio_layer, model_data.hparams.n_text_layer, model_data.hparams.n_audio_state, ); Ok(Self { model_data: std::sync::Arc::new(model_data), }) } ``` -------------------------------- ### estimate_noise_floor with mixed quiet and loud signals (partial) Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/vad.rs.html Starts a test for estimate_noise_floor with a mix of quiet and loud signals. The comment indicates the 10th percentile should be near the quiet level, but the code is incomplete as shown. ```rust // Mix of quiet background (low amplitude) and loud bursts. // The 10th percentile should be near the quiet level. let quiet_amplitude = 0.01; let loud_amplitude = 0.5; // 80% quiet, 20% loud — 10th percentile should be in the quiet range let mut audio = Vec::new(); // 1600ms quiet ``` -------------------------------- ### Test transcribe_long_with_vad_config Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Starts a test that verifies the transcribe_long_with_vad_config method signature compiles and accepts a custom VadConfig. The test skips if the model file is not available. ```rust let model_path = std::path::Path::new("ggml-tiny.bin"); if !model_path.exists() { // Skip if model file is not available in CI/local ``` -------------------------------- ### Test compute_split_points with step equal to chunk Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Verifies that when step equals chunk, compute_split_points produces at least two chunks, starts at offset 0, and the last chunk reaches the audio end. Ensures correct splitting behavior for equal step and chunk. ```rust let audio = vec![0.0f32; 16000 * 90]; let chunk = 16000 * 30; let points = compute_split_points(&audio, chunk, chunk); assert!(points.len() >= 2); assert_eq!(points[0].0, 0); assert_eq!( points.last().map(|p| p.1), Some(audio.len()), "last chunk must reach audio end" ); ``` -------------------------------- ### SilenceGap struct with center method Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Defines a silence gap with start and end sample indices and a method to compute its center. Used by the silence-gap detection logic. ```rust struct SilenceGap { start: usize, end: usize, } impl SilenceGap { fn center(&self) -> usize { (self.start + self.end) / 2 } } ``` -------------------------------- ### Test adaptive threshold detects speech Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/vad.rs.html Verifies that adaptive thresholding detects a loud tone over a quiet noise floor. Builds 1s quiet, 1s loud, 1s quiet audio and asserts the detected segment starts before 1.5s and ends after 1.5s. ```rust // Speech (loud tone) over a noise floor (quiet constant signal). // Adaptive threshold should detect the loud part as speech. let mut audio = Vec::new(); // 1 second of quiet noise let noise_level = 0.005; audio.extend(vec![noise_level; SR]); // 1 second of loud speech-like tone audio.extend(tone(440.0, 1000, 0.5)); // 1 second of quiet noise again audio.extend(vec![noise_level; SR]); let config = VadConfig { adaptive_threshold: true, noise_margin: 3.0, min_speech_ms: 250, min_silence_ms: 300, frame_size_ms: 30, energy_threshold: 0.01, // ignored when adaptive }; let segments = detect_speech(&audio, SR, &config); assert!( !segments.is_empty(), "adaptive threshold should detect the loud tone as speech" ); // The speech should be roughly in the middle second let seg = &segments[0]; let start_sec = seg.start as f32 / SR as f32; let end_sec = seg.end as f32 / SR as f32; assert!( start_sec < 1.5, "speech start should be near 1s, got {start_sec}" ); assert!( end_sec > 1.5, "speech end should be past 1.5s, got {end_sec}" ); ``` -------------------------------- ### Test compute_split_points_with_vad with custom VadConfig Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Test compute_split_points_with_vad with adaptive threshold on 90 seconds of silence. Uses chunk 16000*30, step chunk-16000, and VadConfig with noise_margin 3.0, min_speech_ms 100, min_silence_ms 100. Asserts at least 2 chunks, first start at 0, and last end at audio length. ```rust let audio = vec![0.0f32; 16000 * 90]; // 90 seconds of silence let chunk = 16000 * 30; let step = chunk - 16000; let vad_config = vad::VadConfig { adaptive_threshold: true, noise_margin: 3.0, energy_threshold: 0.01, min_speech_ms: 100, min_silence_ms: 100, frame_size_ms: 30, }; let points = compute_split_points_with_vad(&audio, step, chunk, &vad_config); assert!( points.len() >= 2, "expected multiple chunks for 90s audio, got {}", points.len() ); assert_eq!(points[0].0, 0); assert_eq!( points.last().map(|p| p.1), Some(audio.len()), "last chunk should end at audio length" ); ``` -------------------------------- ### Serialize TranscribeResult to JSON Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Serializes a TranscribeResult to JSON and asserts that the output contains the expected text and start field. ```rust use crate::types::to_json; let result = TranscribeResult { text: "hello world".to_string(), segments: vec![Segment { text: "hello world".to_string(), start: 0.0, end: 2.5, confidence: -0.5, is_hallucination: false, }], language: Some("en".to_string()), }; let json = to_json(&result).expect("serialization should succeed"); assert!(json.contains("hello world")); assert!(json.contains("\"start\"")); ``` -------------------------------- ### Test compute_split_points with single sample over chunk Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Tests compute_split_points with a single sample over the chunk size and a step smaller than chunk. Asserts at least two chunks, first start at 0, and last end equals audio length. ```rust let chunk = 16000 * 30; let audio = vec![0.0f32; chunk + 1]; let step = chunk - 16000; let points = compute_split_points(&audio, step, chunk); assert!(points.len() >= 2, "should require at least 2 chunks"); assert_eq!(points[0].0, 0); assert_eq!(points.last().map(|p| p.1), Some(audio.len())); ``` -------------------------------- ### OxiWhisper lib.rs module declarations and WhisperModel struct Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/lib.rs.html The lib.rs module documentation and public API surface for the OxiWhisper crate. It declares the crate-level doc comment with a Quick Start example, module declarations, conditional re-exports for the diarization feature, and the WhisperModel struct definition. Use this to understand the crate's public modules and entry point. ```rust //! # OxiWhisper //! //! Pure Rust Whisper speech-to-text inference engine with zero C/C++ dependencies. //! //! OxiWhisper loads GGML-format Whisper models and transcribes audio to text, //! supporting quantized inference (Q4_0, Q5_0, Q8_0), streaming, beam search, //! word-level timestamps, and SIMD-accelerated kernels (AVX2, NEON, WASM simd128). //! //! ## Quick Start //! //! ```no_run //! use oxiwhisper::{TranscribeOptions, WhisperModel}; //! use std::path::Path; //! //! # fn main() -> Result<(), Box> { //! let model = WhisperModel::from_file(Path::new("ggml-tiny.bin"))?; //! let audio = oxiwhisper::audio::load_wav(Path::new("audio.wav"))?; //! let text = model.transcribe(&audio, &TranscribeOptions::default())?; //! println!("{text}"); //! # Ok(()) //! # } //! ``` #![warn(missing_docs)] /// Multi-head attention primitives shared by encoder and decoder. pub mod attention; /// Audio I/O helpers (WAV loading, PCM resampling). pub mod audio; /// Beam search decoder for multi-hypothesis Whisper decoding. pub mod beam_search; /// Token-level decoding utilities (argmax, sampling, ngram suppression). pub mod decode_utils; /// Core Whisper text decoder (forward pass, KV cache, sampling). pub mod decoder; /// Speaker diarization ("who spoke when"): segmentation, embedding, clustering /// (D2-D4) and resegmentation plus the end-to-end pipeline (D5) are implemented. #[cfg(feature = "diarization")] pub mod diarize; /// Dynamic Time Warping utilities for word-level timestamp alignment. pub mod dtw; /// Whisper audio encoder (CNN + Transformer). pub mod encoder; /// FFT utilities backed by OxiFFT (used by the mel spectrogram pipeline). pub mod fft; pub(crate) mod gguf; /// Hallucination detection heuristics for Whisper output segments. pub mod hallucination; /// Linear (dense) layer kernels with optional quantized weight support. pub mod linear; /// Log-mel spectrogram computation from 16 kHz PCM audio. pub mod mel; /// Pre-computed Whisper mel filterbank coefficients. pub mod mel_filters; /// GGML model loader and weight storage types. pub mod model; #[cfg(feature = "onnx")] pub mod onnx_loader; /// Quantization types and GGML Q4_0/Q5_0/Q8_0 dequantization kernels. pub mod quantize; /// Streaming transcription that accumulates audio in 30-second chunks. pub mod stream; /// SRT and WebVTT subtitle formatting from timed segments. pub mod subtitle; /// Minimal f32 tensor type used throughout the inference pipeline. pub mod tensor; /// Synthetic model generators for integration tests. #[cfg(any(test, feature = "test-utils"))] pub mod test_utils; /// Thread count management and parallel iteration helpers. pub mod threading; /// BPE token-ID to text decoding and segment parsing for Whisper. pub mod tokenizer; /// Public types, error enum, and option validation for oxiwhisper. pub mod types; /// Voice activity detection (energy-based silence segmentation). pub mod vad; /// Word-level timestamp alignment via cross-attention DTW. pub mod word_timestamps; mod whisper_model; #[cfg(feature = "diarization")] pub use diarize::attribute::{SpeakerTranscript, SpeakerTurn}; #[cfg(all(feature = "diarization", feature = "onnx"))] pub use diarize::embed::EcapaOnnx; #[cfg(feature = "diarization")] pub use diarize::embed::{SpeakerEmbedder, WhisperEncoderEmbedder}; #[cfg(feature = "diarization")] pub use diarize::format::{labeled_transcript, labeled_transcript_timed, rttm_string, write_rttm}; #[cfg(feature = "diarization")] pub use diarize::metrics::{DerOptions, DerReport, RttmSegment, der, hungarian, jer, parse_rttm}; #[cfg(feature = "diarization")] pub use diarize::{ClusteringMethod, DiarizeOptions, DiarizeResult, SpeakerId, SpeakerSegment}; pub use dtw::WordSegment; pub use types::*; pub use word_timestamps::WordTimedTranscript; use std::sync::Arc; /// Main entry point for Whisper speech-to-text inference. /// /// Wraps a loaded GGML model and exposes transcription methods. Internally /// shares the model weights via `Arc` so cloning is cheap and the model is /// usable from multiple threads. pub struct WhisperModel { model_data: Arc, } ``` -------------------------------- ### Test DP-DTW monotonic timestamps Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/dtw.rs.html Verifies that align_tokens_dp_dtw produces non-decreasing start times for a diagonal attention fixture. Asserts each start is >= the previous start. ```rust let n_tokens = 6; let n_frames = 18; let mut attn = vec![0.0f32; n_tokens * n_frames]; for i in 0..n_tokens { attn[i * n_frames + 3 * i] = 1.0; } let result = align_tokens_dp_dtw(&attn, n_tokens, n_frames, 160, 16000, None); assert_eq!(result.len(), n_tokens); for i in 1..result.len() { assert!( result[i].0 >= result[i - 1].0, "monotonic violation at {i}: start={} < prev_start={}", result[i].0, result[i - 1].0 ); } ``` -------------------------------- ### load_wav: Load WAV file from path Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/audio.rs.html Loads a WAV file from disk and returns 16 kHz mono f32 PCM audio normalised to [-1.0, 1.0]. Reads the file into memory and delegates to parse_wav. ```rust pub fn load_wav(path: &Path) -> Result, OxiWhisperError> { let data = std::fs::read(path)?; parse_wav(&data) } ``` -------------------------------- ### Test build_word_timed_transcript with attention Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/word_timestamps.rs.html Tests that providing cross-attention produces word segments with valid timing invariants: start <= end, non-negative starts, and non-decreasing start times. Uses a synthetic attention matrix with peaks at different frames. ```rust #[test] fn test_build_with_attention_produces_words() { let vocab = tiny_vocab(); // 2 tokens ("hello", " world"), 8 encoder frames // attention rows: each row sums to ~1 and peaks at different frames let n_tokens = 2; let enc_frames = 8; let mut attn = vec![0.0f32; n_tokens * enc_frames]; // token 0 peaks at frame 1, token 1 peaks at frame 5 attn[1] = 0.8; for v in attn[..enc_frames].iter_mut() { if *v < 0.5 { *v += 0.025; } } attn[enc_frames + 5] = 0.8; for v in attn[enc_frames..2 * enc_frames].iter_mut() { if *v < 0.5 { *v += 0.025; } } let dr = make_decode_result(vec![0, 1], vec![-0.1, -0.2], Some(attn), enc_frames, 0.02); let wtt = build_word_timed_transcript(&dr, &vocab, Some("en".into())); // We don't assert exact timing, only structural properties. assert!(!wtt.words.is_empty(), "should produce word segments"); for w in &wtt.words { assert!(w.start <= w.end, "start <= end invariant: {:?}", w); assert!(w.start >= 0.0, "start must be non-negative"); } // Monotonic non-decreasing starts let starts: Vec = wtt.words.iter().map(|w| w.start).collect(); for pair in starts.windows(2) { assert!( pair[0] <= pair[1], "word starts must be non-decreasing: {pair:?}" ); } } ``` -------------------------------- ### Test helper: build_wav constructs WAV bytes Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/audio.rs.html Test helper that constructs a minimal WAV file in memory. Computes RIFF and fmt chunk sizes, byte rate, and block alignment, then writes the RIFF header, fmt chunk, and data chunk. ```rust #[cfg(test)] mod tests { use super::*; /// Build a minimal WAV file in memory. fn build_wav( sample_rate: u32, num_channels: u16, bits_per_sample: u16, audio_format: u16, raw_samples: &[u8], ) -> Vec { let fmt_chunk_size: u32 = 16; let data_chunk_size = raw_samples.len() as u32; // RIFF size = 4 (WAVE) + 8+fmt_size + 8+data_size let riff_size: u32 = 4 + 8 + fmt_chunk_size + 8 + data_chunk_size; let byte_rate = sample_rate * num_channels as u32 * (bits_per_sample as u32 / 8); let block_align = num_channels * (bits_per_sample / 8); let mut buf = Vec::new(); // RIFF header buf.extend_from_slice(b"RIFF"); buf.extend_from_slice(&riff_size.to_le_bytes()); buf.extend_from_slice(b"WAVE"); // fmt chunk buf.extend_from_slice(b"fmt "); buf.extend_from_slice(&fmt_chunk_size.to_le_bytes()); buf.extend_from_slice(&audio_format.to_le_bytes()); buf.extend_from_slice(&num_channels.to_le_bytes()); buf.extend_from_slice(&sample_rate.to_le_bytes()); buf.extend_from_slice(&byte_rate.to_le_bytes()); buf.extend_from_slice(&block_align.to_le_bytes()); buf.extend_from_slice(&bits_per_sample.to_le_bytes()); // data chunk buf.extend_from_slice(b"data"); buf.extend_from_slice(&data_chunk_size.to_le_bytes()); buf.extend_from_slice(raw_samples); buf } } ``` -------------------------------- ### Get Row Slice Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/tensor.rs.html Returns a contiguous slice of the last dimension at given indices. For a [B, T, D] tensor, gets row [b, t, :] as a 1D tensor. ```rust pub fn row(&self, indices: &[usize]) -> Self { assert_eq!(indices.len(), self.ndim() - 1); let last_dim = self.last_dim(); let mut offset = 0; let mut stride = self.numel(); ``` -------------------------------- ### Compute Q@K^T dot product manually Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/attention.rs.html Manual loop over heads, query positions, and key positions to compute dot products scaled by a factor. Used as a reference for verifying sgemm results. ```rust for j in 0..kv_len { let mut dot = 0.0f32; for d in 0..head_dim { dot += q_data[h * q_len * head_dim + i * head_dim + d] * k_data[h * kv_len * head_dim + j * head_dim + d]; } ref_scores[h * q_len * kv_len + i * kv_len + j] = dot * scale; } } } ``` -------------------------------- ### Test: build_word_segments monotonic starts Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/dtw.rs.html Tests that `build_word_segments` produces segments with monotonically non-decreasing start times. Uses four words with increasing time intervals and negative probabilities. ```rust // segs[i+1].start >= segs[i].start for all consecutive segments. let texts = vec![ " Alpha".to_string(), " Beta".to_string(), " Gamma".to_string(), " Delta".to_string(), ]; let times = vec![(0.0, 0.25), (0.25, 0.5), (0.5, 0.75), (0.75, 1.0)]; let probs = vec![-0.1, -0.2, -0.15, -0.05]; let segs = build_word_segments(&texts, ×, &probs); for i in 1..segs.len() { assert!( segs[i].start >= segs[i - 1].start, "non-monotonic starts at {i}: {} < {}", segs[i].start, segs[i - 1].start ); } ``` -------------------------------- ### AVX2 + FMA quantized dot products (dot_q4_0_avx2, dot_q5_0_avx2) Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/quantize/dot_simd_x86.rs.html This file contains the x86_64-specific SIMD implementation for quantized dot products. It defines `dot_q4_0_avx2` and `dot_q5_0_avx2` functions that require AVX2 and FMA support; callers must ensure these features are available. The code is wrapped by `#[cfg(target_arch = "x86_64")]` in the parent module. ```rust // This file only compiles on x86_64 targets. // The parent mod.rs wraps the `mod dot_simd_x86;` declaration with // `#[cfg(target_arch = "x86_64")]`. // --------------------------------------------------------------------------- // SIMD-accelerated quantized dot products (AVX2 + FMA, x86_64) // --------------------------------------------------------------------------- use half::f16; use super::types::{ Q4_0_BLOCK_BYTES, Q4_0_BLOCK_SIZE, Q5_0_BLOCK_BYTES, Q5_0_BLOCK_SIZE, Q8_0_BLOCK_BYTES, Q8_0_BLOCK_SIZE, }; /// AVX2 + FMA accelerated Q4_0 dot product. /// /// Processes each 32-element block by extracting low and high nibbles from /// the 16 packed bytes, sign-extending via `_mm256_cvtepi8_epi32`, subtracting /// the offset of 8, then FMA-accumulating against the f32 input vector. /// /// # Safety /// Caller must ensure AVX2 and FMA are available on the current CPU. #[target_feature(enable = "avx2", enable = "fma")] pub unsafe fn dot_q4_0_avx2(input: &[f32], quantized: &[u8], n: usize) -> f32 { use std::arch::x86_64::*; unsafe { let blocks = n / Q4_0_BLOCK_SIZE; let offset_8 = _mm256_set1_epi32(8); let mask_0f = _mm_set1_epi8(0x0F_u8 as i8); let mut total = _mm256_setzero_ps(); for blk in 0..blocks { let a_off = blk * Q4_0_BLOCK_BYTES; let scale = f16::from_le_bytes([quantized[a_off], quantized[a_off + 1]]).to_f32(); let b_off = blk * Q4_0_BLOCK_SIZE; let scale_v = _mm256_set1_ps(scale); // Load 16 bytes of nibble data let raw = _mm_loadu_si128(quantized[a_off + 2..].as_ptr() as *const __m128i); // Extract low nibbles (elements 0..15) and high nibbles (elements 16..31) let lo_nibbles = _mm_and_si128(raw, mask_0f); let hi_nibbles = _mm_and_si128(_mm_srli_epi16(raw, 4), mask_0f); // Process low nibbles (elements 0..15): 2 groups of 8 for chunk in 0..2 { let nibble_chunk = if chunk == 0 { lo_nibbles } else { _mm_srli_si128(lo_nibbles, 8) }; let i32_vals = _mm256_cvtepu8_epi32(nibble_chunk); let centered = _mm256_sub_epi32(i32_vals, offset_8); let f32_vals = _mm256_cvtepi32_ps(centered); let scaled = _mm256_mul_ps(f32_vals, scale_v); let b_vals = _mm256_loadu_ps(input.as_ptr().add(b_off + chunk * 8)); total = _mm256_fmadd_ps(scaled, b_vals, total); } // Process high nibbles (elements 16..31): 2 groups of 8 for chunk in 0..2 { let nibble_chunk = if chunk == 0 { hi_nibbles } else { _mm_srli_si128(hi_nibbles, 8) }; let i32_vals = _mm256_cvtepu8_epi32(nibble_chunk); let centered = _mm256_sub_epi32(i32_vals, offset_8); let f32_vals = _mm256_cvtepi32_ps(centered); let scaled = _mm256_mul_ps(f32_vals, scale_v); let b_vals = _mm256_loadu_ps(input.as_ptr().add(b_off + 16 + chunk * 8)); total = _mm256_fmadd_ps(scaled, b_vals, total); } } // Horizontal sum of 8 f32 lanes let hi = _mm256_extractf128_ps(total, 1); let lo = _mm256_castps256_ps128(total); let sum128 = _mm_add_ps(lo, hi); let shuf = _mm_movehdup_ps(sum128); let sums = _mm_add_ps(sum128, shuf); let shuf2 = _mm_movehl_ps(sums, sums); _mm_cvtss_f32(_mm_add_ss(sums, shuf2)) } } /// AVX2 + FMA accelerated Q5_0 dot product. /// /// Similar to Q4_0 but additionally extracts the 5th bit from a 32-bit /// high-bit mask and ORs it with the low nibble before centering at 16. /// /// # Safety /// Caller must ensure AVX2 and FMA are available on the current CPU. #[target_feature(enable = "avx2", enable = "fma")] pub unsafe fn dot_q5_0_avx2(input: &[f32], quantized: &[u8], n: usize) -> f32 { use std::arch::x86_64::*; unsafe { let blocks = n / Q5_0_BLOCK_SIZE; let offset_16 = _mm256_set1_epi32(16); let mask_0f = _mm_set1_epi8(0x0F_u8 as i8); let mut total = _mm256_setzero_ps(); for blk in 0..blocks { let a_off = blk * Q5_0_BLOCK_BYTES; let scale = f16::from_le_bytes([quantized[a_off], quantized[a_off + 1]]).to_f32(); let qh = u32::from_le_bytes([ quantized[a_off + 2], ``` -------------------------------- ### Test: build_word_segments start <= end Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/dtw.rs.html Tests that `build_word_segments` produces segments where each segment's start time is less than or equal to its end time. Uses three words with corresponding times and probabilities. ```rust // For any valid inputs, seg.start <= seg.end must hold. let texts = vec![ " Hello".to_string(), " beautiful".to_string(), " world".to_string(), ]; let times = vec![(0.0f32, 0.3f32), (0.3, 0.7), (0.7, 1.0)]; let probs = vec![-0.05, -0.1, -0.08]; let segs = build_word_segments(&texts, ×, &probs); for seg in &segs { assert!( seg.start <= seg.end, "start {} > end {} for word '{}'", seg.start, seg.end, seg.word ); } ``` -------------------------------- ### Test TranscribeOptions with initial_prompt Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/whisper_model.rs.html Creates TranscribeOptions with an initial prompt and verifies that suppress_tokens is None and validation passes. ```rust let opts = TranscribeOptions { initial_prompt: Some("technical vocabulary"), ..TranscribeOptions::default() }; assert_eq!(opts.initial_prompt, Some("technical vocabulary")); assert!(opts.suppress_tokens.is_none()); assert!(validate_options(&opts).is_ok()); ``` -------------------------------- ### Build decoder prompt token sequence Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/decoder/forward.rs.html Builds the decoder prompt token sequence. Prepends `sot_prev` and `initial_tokens` when initial tokens are provided, then appends `sot`, the language token, `transcribe`, and optionally `no_timestamps`. ```rust pub(crate) fn build_prompt( special: &SpecialTokens, lang_token: u32, task_token: u32, timestamps: bool, initial_tokens: &[u32], ) -> Vec { let mut prompt = Vec::new(); // Initial prompt tokens go BEFORE the SOT token (matching OpenAI's behavior) if !initial_tokens.is_empty() { prompt.push(special.sot_prev); prompt.extend_from_slice(initial_tokens); } prompt.push(special.sot); prompt.push(lang_token); prompt.push(task_token); if !timestamps { prompt.push(special.no_timestamps); } prompt } ``` -------------------------------- ### Public dot product wrapper Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/linear.rs.html Public entry point for computing the dot product of two f32 slices, delegating to the SIMD dispatch function. Marked inline(always) for performance. ```rust #[inline(always)] pub fn dot(a: &[f32], b: &[f32]) -> f32 { dot_product(a, b) } ``` -------------------------------- ### Test: dp_dtw returns correct count (start) Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/dtw.rs.html Starts a test for `dp_dtw` that expects exactly 4 pairs returned for a 4-token by 4-frame uniform attention matrix. The snippet is incomplete; the full test body is not shown. ```rust // 4-token × 4-frame uniform attention → exactly 4 pairs returned. let n_tokens = 4; ``` -------------------------------- ### load_wav(path: &Path) -> Result, OxiWhisperError> Source: https://docs.rs/oxiwhisper/latest/oxiwhisper/audio/fn.load_wav.html Loads a WAV file from disk and returns 16 kHz mono f32 PCM audio normalized to [-1.0, 1.0]. ```APIDOC ## Function: load_wav ### Description Load a WAV file from disk and return 16 kHz mono f32 PCM audio normalised to [-1.0, 1.0]. ### Signature ```rust pub fn load_wav(path: &Path) -> Result, OxiWhisperError> ``` ### Parameters - **path** (`&Path`) - The filesystem path to the WAV file to load. ### Returns - `Result, OxiWhisperError>` - On success, returns a vector of f32 samples representing the audio, normalized to [-1.0, 1.0]. On failure, returns an `OxiWhisperError`. ### Example ```rust use std::path::Path; use oxiwhisper::audio::load_wav; let path = Path::new("audio.wav"); let samples = load_wav(path).expect("Failed to load WAV file"); ``` ``` -------------------------------- ### Tensor lookup methods: get, try_get, get_quantized, try_get_quantized Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/model.rs.html Provides methods to look up tensors by name from the model's storage. `get` returns an error if the tensor is missing, while `try_get`, `get_quantized`, and `try_get_quantized` return `Option` values. Use these to access f32 or quantized tensors after loading a model. ```rust pub fn get(&self, name: &str) -> Result<&Tensor, String> { self.tensors .get(name) .ok_or_else(|| format!("Missing tensor: {name}")) } /// Look up an f32 tensor by name, returning `None` if not found. pub fn try_get(&self, name: &str) -> Option<&Tensor> { self.tensors.get(name) } /// Look up a quantized tensor by name. pub fn get_quantized(&self, name: &str) -> Option<&crate::quantize::QuantizedTensor> { self.quantized_tensors.get(name) } /// Alias for `get_quantized`. pub fn try_get_quantized(&self, name: &str) -> Option<&crate::quantize::QuantizedTensor> { self.quantized_tensors.get(name) } ``` -------------------------------- ### push_audio Source: https://docs.rs/oxiwhisper/latest/oxiwhisper/stream/struct.StreamTranscriber.html Pushes audio samples into the internal buffer. Samples should be 16 kHz mono f32 PCM. You can push arbitrary amounts; chunks are processed when next_segment is called. ```APIDOC ## push_audio ### Description Pushes audio samples into the internal buffer. Samples should be 16 kHz mono f32 PCM. You can push arbitrary amounts; chunks are processed when next_segment is called. ### Method push_audio ### Parameters - **samples** (&[f32]) - Required - Audio samples to push. ### Returns None ``` -------------------------------- ### Test 32-bit PCM extreme values Source: https://docs.rs/oxiwhisper/latest/src/oxiwhisper/audio.rs.html Tests loading a 32-bit PCM WAV with extreme integer values (0, i32::MAX, i32::MIN) and verifies the conversion to f32 samples. Expects 3 samples, with the first near 0, the second near 1.0, and the third near -1.0. ```rust let mut raw = Vec::new(); raw.extend_from_slice(&0_i32.to_le_bytes()); raw.extend_from_slice(&i32::MAX.to_le_bytes()); raw.extend_from_slice(&i32::MIN.to_le_bytes()); let wav = build_wav(16000, 1, 32, WAVE_FORMAT_PCM, &raw); let result = load_wav_from_bytes(&wav); assert!(result.is_ok()); let samples = result.expect("already checked"); assert_eq!(samples.len(), 3); assert!(samples[0].abs() < 1e-7); // i32::MAX / 2^31 ≈ 1.0 assert!((samples[1] - (i32::MAX as f32 / 2_147_483_648.0)).abs() < 1e-5); assert!((samples[2] - (-1.0)).abs() < 1e-5); } ``` -------------------------------- ### SpecialTokens::language_token Source: https://docs.rs/oxiwhisper/latest/oxiwhisper/tokenizer/struct.SpecialTokens.html Gets the language token ID for a given language code. ```APIDOC ## SpecialTokens::language_token ### Description Get language token ID for a language code. ### Method `pub fn language_token(&self, lang: &str) -> u32` ### Parameters - `lang` (&str) - Required - The language code (e.g., "en", "fr"). ### Returns - `u32` - The token ID corresponding to the language code. ``` -------------------------------- ### Struct Fields Source: https://docs.rs/oxiwhisper/latest/oxiwhisper/vad/struct.SpeechSegment.html The fields of the SpeechSegment struct representing start and end sample indices. ```APIDOC ## SpeechSegment Fields ### Description Fields representing the sample indices of the speech segment. ### Fields - `start: usize` - Start sample index (inclusive). - `end: usize` - End sample index (exclusive). ```