### ASR CLI Commands - Fluid Inference Source: https://docs.fluidinference.com/asr/getting-started Provides examples of using the Fluid Inference command-line interface (CLI) for various ASR tasks. This includes basic transcription, specifying model versions, processing multiple files, and running benchmarks. ```bash # Transcribe (multilingual) swift run fluidaudio transcribe audio.wav # English-only (better recall) swift run fluidaudio transcribe audio.wav --model-version v2 # Multiple files in parallel swift run fluidaudio multi-stream audio1.wav audio2.wav # Benchmark on LibriSpeech swift run fluidaudio asr-benchmark --subset test-clean --max-files 50 ``` -------------------------------- ### Swift Quick Start for Kokoro TTS Synthesis Source: https://docs.fluidinference.com/tts/kokoro Provides a basic Swift code example for initializing the TtSManager and synthesizing text to an audio file using Kokoro TTS. This is a straightforward way to integrate TTS into Swift applications. ```swift import FluidAudioTTS let manager = TtSManager() try await manager.initialize() let audioData = try await manager.synthesize(text: "Hello from FluidAudio!") try audioData.write(to: URL(fileURLWithPath: "/tmp/demo.wav")) ``` -------------------------------- ### Real-time Audio Capture and Diarization Setup (Swift) Source: https://docs.fluidinference.com/diarization/streaming Sets up a class for real-time audio capture and diarization. It initializes the audio engine, diarizer, and audio stream, then installs a tap on the input node to process audio buffers. ```swift class RealTimeDiarizer { private let audioEngine = AVAudioEngine() private let diarizer: DiarizerManager private var audioStream: AudioStream init() async throws { let models = try await DiarizerModels.downloadIfNeeded() diarizer = DiarizerManager() diarizer.initialize(models: models) audioStream = AudioStream( chunkDuration: 5.0, chunkSkip: 3.0, streamStartTime: 0.0, chunkingStrategy: .useFixedSkip ) audioStream.bind { [weak self] chunk, _ in Task { let result = try self?.diarizer.performCompleteDiarization(chunk) // Handle results } } } func startCapture() throws { let inputNode = audioEngine.inputNode let format = inputNode.outputFormat(forBus: 0) inputNode.installTap(onBus: 0, bufferSize: 1024, format: format) { [weak self] buffer, _ in try? self?.audioStream.write(from: buffer) } audioEngine.prepare() try audioEngine.start() } } ``` -------------------------------- ### Install FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Instructions to build and install the FluidAudio command-line interface using Swift Package Manager. This involves navigating to the project directory and executing the build command. ```bash cd FluidAudio swift build -c release # Binary at .build/release/fluidaudio ``` -------------------------------- ### Run FluidAudio CLI Directly Source: https://docs.fluidinference.com/reference/cli Demonstrates how to run the FluidAudio CLI directly using 'swift run' followed by the command and its arguments. This is an alternative to building and installing the binary. ```bash swift run fluidaudio ``` -------------------------------- ### Quick Start Synthesize and Save Audio with PocketTTS (Swift) Source: https://docs.fluidinference.com/tts/pocket-tts Demonstrates how to initialize the PocketTtsManager, synthesize text to audio data, and save the synthesized audio to a file using Swift. This code requires the FluidAudioTTS library. ```swift import FluidAudioTTS let manager = PocketTtsManager() try await manager.initialize() let audioData = try await manager.synthesize(text: "Hello, world!") try await manager.synthesizeToFile( text: "Hello, world!", outputURL: URL(fileURLWithPath: "/tmp/output.wav") ) ``` -------------------------------- ### Quick Start CLI for Kokoro TTS Source: https://docs.fluidinference.com/tts/kokoro Demonstrates how to use the FluidAudio command-line interface (CLI) to perform text-to-speech synthesis with Kokoro TTS. It specifies the output file and selects a voice. ```bash swift run fluidaudio tts "Welcome to FluidAudio text to speech" \ --output ~/Desktop/demo.wav \ --voice af_heart ``` -------------------------------- ### Example Finance and Technology Lexicon Entries Source: https://docs.fluidinference.com/asr/custom-pronunciation This text file provides example entries for a custom pronunciation lexicon, covering financial terms, technology names, and product names with their corresponding IPA phonetic representations. ```text # Finance NASDAQ=nˈæzdæk EBITDA=iːbˈɪtdɑː # Technology NVIDIA=ɛnvˈɪdiə Kubernetes=kuːbɚnˈɛtiːz # Product Names Kokoro=kəkˈɔɹO FluidAudio=flˈuːɪd ˈɔːdioʊ ``` -------------------------------- ### Install FluidAudio Package Source: https://docs.fluidinference.com/quickstart Add FluidAudio to your project using Swift Package Manager. This is the initial step to integrate FluidAudio's capabilities into your Swift applications. ```swift dependencies: [ .package(url: "https://github.com/FluidInference/FluidAudio.git", from: "0.7.9"), ] ``` -------------------------------- ### Install FluidAudio for React Native / Expo Source: https://docs.fluidinference.com/installation This command installs the FluidAudio package for React Native and Expo projects using npm. It utilizes the official npm package for cross-platform audio capabilities. ```bash npm install @fluidinference/react-native-fluidaudio ``` -------------------------------- ### Manual Model Loading Examples in Swift Source: https://docs.fluidinference.com/guides/manual-model-loading Demonstrates how to manually load CoreML models for different FluidAudio modules (ASR, Diarization, VAD) using Swift. This is useful for offline or air-gapped deployments where automatic HuggingFace downloads are not possible. It shows the expected API calls and model configurations. ```swift // Example: ASR let models = try await AsrModels.load( from: URL(fileURLWithPath: "/opt/models/parakeet-tdt-0.6b-v3-coreml"), version: .v3 ) // Example: Diarization let models = try await DiarizerModels.load( localSegmentationModel: segmentationURL, localEmbeddingModel: embeddingURL ) // Example: VAD let vadModel = try MLModel(contentsOf: modelURL, configuration: config) let manager = VadManager(config: .default, vadModel: vadModel) ``` -------------------------------- ### FluidAudio CLI Commands Source: https://docs.fluidinference.com/diarization/getting-started This example shows command-line interface (CLI) commands for using FluidAudio for audio processing and benchmarking. It includes commands for processing an audio file and running diarization benchmarks. ```bash swift run fluidaudio process meeting.wav --output results.json --threshold 0.6 swift run fluidaudio diarization-benchmark --auto-download ``` -------------------------------- ### Add FluidAudio to Swift Project via Swift Package Manager Source: https://docs.fluidinference.com/installation This snippet shows how to add the FluidAudio Swift package to your project's dependencies. It specifies the repository URL and the version range. You can choose between the core 'FluidAudio' product or the 'FluidAudioTTS' product which includes GPL dependencies. ```swift dependencies: [ .package(url: "https://github.com/FluidInference/FluidAudio.git", from: "0.7.9"), ] ``` ```swift // Core features only (no GPL dependencies): .product(name: "FluidAudio", package: "FluidAudio") // Add TTS support (includes GPL ESpeakNG): .product(name: "FluidAudioTTS", package: "FluidAudio") ``` -------------------------------- ### Add FluidAudio for Rust / Tauri Source: https://docs.fluidinference.com/installation This command adds the FluidAudio Rust crate to your project using Cargo. This enables integrating FluidAudio's functionalities within Rust or Tauri applications. ```bash cargo add fluidaudio-rs ``` -------------------------------- ### Batch Transcription with Parakeet Models (Swift) Source: https://docs.fluidinference.com/asr/getting-started Performs batch transcription of an audio file using Fluid Inference's Parakeet models. It requires downloading and loading the ASR models and initializing the ASR manager. The input is an audio file path, and the output is the transcribed text and its confidence score. ```swift import FluidAudio Task { let models = try await AsrModels.downloadAndLoad(version: .v3) // .v2 for English-only let asrManager = AsrManager(config: .default) try await asrManager.initialize(models: models) let samples = try AudioConverter().resampleAudioFile( path: "path/to/audio.wav" ) let result = try await asrManager.transcribe(samples, source: .system) print("Transcription: \(result.text)") print("Confidence: \(result.confidence)") } ``` ```swift let audioURL = URL(fileURLWithPath: "/path/to/audio.wav") let result = try await asrManager.transcribe(audioURL, source: .system) print(result.text) ``` ```swift let models = try await AsrModels.downloadAndLoad(version: .v2) ``` -------------------------------- ### Run Benchmarks with FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Provides examples for running various benchmarks using the FluidAudio CLI, including Automatic Speech Recognition (ASR), FLEURS multilingual, Diarization, and Voice Activity Detection (VAD) benchmarks. Options for subset selection, language specification, and automatic downloads are shown. ```bash # ASR benchmark fluidaudio asr-benchmark --subset test-clean --max-files 100 # FLEURS multilingual fluidaudio fleurs-benchmark --languages en_us,fr_fr --samples 10 # Diarization benchmark fluidaudio diarization-benchmark --auto-download fluidaudio diarization-benchmark --single-file ES2004a --threshold 0.7 # VAD benchmark fluidaudio vad-benchmark --num-files 40 --threshold 0.5 ``` -------------------------------- ### Known Speaker Recognition Setup (Swift) Source: https://docs.fluidinference.com/diarization/getting-started This Swift snippet illustrates how to set up known speaker recognition by pre-loading speaker profiles. It involves extracting embeddings from sample audio for known speakers and initializing the diarizer's speaker manager with these profiles. ```swift let aliceAudio = loadAudioFile("alice_sample.wav") let aliceEmbedding = try diarizer.extractEmbedding(aliceAudio) let alice = Speaker(id: "Alice", name: "Alice", currentEmbedding: aliceEmbedding) let bob = Speaker(id: "Bob", name: "Bob", currentEmbedding: bobEmbedding) diarizer.speakerManager.initializeKnownSpeakers([alice, bob]) // Will use "Alice" instead of "Speaker_1" when matched let result = try diarizer.performCompleteDiarization(audioSamples) ``` -------------------------------- ### Add FluidAudio to Project via CocoaPods Source: https://docs.fluidinference.com/installation This snippet demonstrates how to add FluidAudio to your project using CocoaPods. It specifies the pod name and version constraint. This method is an alternative to Swift Package Manager for dependency management. ```ruby pod 'FluidAudio', '~> 0.7.8' ``` -------------------------------- ### Swift Coexistence of SSML and Markdown Source: https://docs.fluidinference.com/tts/ssml This Swift example shows how SSML tags can be used alongside Markdown syntax within the same text string for text-to-speech synthesis. It demonstrates using a `` tag for custom pronunciation and a Markdown link for another word. ```swift let text = """ Kokoro and [Misaki](/mɪˈsɑːki/) """ ``` -------------------------------- ### Bash: CLI Commands for Offline Diarization Source: https://docs.fluidinference.com/diarization/offline-pipeline Provides examples of using the FluidAudio CLI for offline diarization tasks. Includes processing a single file, benchmarking on the AMI dataset, and processing with ground-truth RTTM files. ```bash # Process a single file swift run fluidaudio process meeting.wav --mode offline --threshold 0.6 # Benchmark on AMI dataset swift run fluidaudio diarization-benchmark --mode offline \ --dataset ami-sdm --threshold 0.6 --auto-download # With ground-truth RTTM swift run fluidaudio process meeting.wav --mode offline \ --rttm ground_truth.rttm ``` -------------------------------- ### Clone ASR Models using Git LFS Source: https://docs.fluidinference.com/asr/manual-model-loading This command clones the ASR model repository using Git Large File Storage (LFS). Ensure Git LFS is installed before running this command. It downloads all necessary model bundles and vocabulary files. ```bash git lfs install git clone https://huggingface.co/FluidInference/parakeet-tdt-0.6b-v3-coreml ``` -------------------------------- ### Transcribe Audio File using CLI Source: https://docs.fluidinference.com/asr/streaming Provides a command-line interface example using Swift's package manager to transcribe an audio file (`.wav`) with the Parakeet EOU model. The `--use-cache` flag can speed up subsequent runs. ```bash swift run fluidaudio parakeet-eou --input audio.wav --use-cache ``` -------------------------------- ### Voice Activity Detection with FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Demonstrates the 'fluidaudio vad-analyze' command for Voice Activity Detection (VAD). Examples show offline segmentation, streaming mode with silence detection, and processing in both modes. ```bash # Offline segmentation fluidaudio vad-analyze audio.wav # Streaming fluidaudio vad-analyze audio.wav --streaming --min-silence-ms 300 # Both modes fluidaudio vad-analyze audio.wav --mode both ``` -------------------------------- ### Initialize Known Speakers Source: https://docs.fluidinference.com/diarization/speaker-manager Initializes the SpeakerManager with a predefined list of known speakers, each with an ID, name, and voice embedding. This allows the manager to recognize these speakers from the start. ```swift let alice = Speaker(id: "alice", name: "Alice", currentEmbedding: aliceEmbedding) let bob = Speaker(id: "bob", name: "Bob", currentEmbedding: bobEmbedding) speakerManager.initializeKnownSpeakers([alice, bob]) ``` -------------------------------- ### Text-to-Speech with FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Showcases the 'fluidaudio tts' command for converting text to speech. Examples include basic TTS, specifying output files, voice selection, and custom pronunciation using lexicons. ```bash fluidaudio tts "Hello from FluidAudio" --output demo.wav --voice af_heart fluidaudio tts "Custom pronunciation" --lexicon custom.txt --output out.wav ``` -------------------------------- ### Transcribe Audio Files with FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Examples of using the 'fluidaudio transcribe' command for batch and multi-stream audio transcription. It includes options for specifying model versions and handling multiple audio files in parallel. ```bash # Batch transcription fluidaudio transcribe audio.wav fluidaudio transcribe audio.wav --model-version v2 # English-only # Multi-stream parallel transcription fluidaudio multi-stream audio1.wav audio2.wav # Streaming transcription (Parakeet EOU) fluidaudio parakeet-eou --input audio.wav --use-cache ``` -------------------------------- ### Configure Diarizer Manager (Swift) Source: https://docs.fluidinference.com/diarization/getting-started This code example shows how to configure the DiarizerManager with custom settings in Swift. It allows adjusting parameters such as clustering threshold, minimum speech duration, minimum silence gap, minimum active frames, and debug mode. ```swift let config = DiarizerConfig( clusteringThreshold: 0.7, // Speaker separation (0.0-1.0) minSpeechDuration: 1.0, // Minimum segment duration (seconds) minSilenceGap: 0.5, // Minimum silence between speakers minActiveFramesCount: 10.0, // Minimum active frames debugMode: false ) let diarizer = DiarizerManager(config: config) ``` -------------------------------- ### Create Model Directory and Initialize Dependencies (Bash) Source: https://docs.fluidinference.com/mobius/converting-models This snippet demonstrates how to create the necessary directory structure for a new model and initialize Python dependencies using `uv`. It includes creating the directory, initializing the environment, and adding coremltools and torch as dependencies. ```bash mkdir -p models/stt/my-new-model/coreml cd models/stt/my-new-model/coreml # Initialize with uv uv init uv add coremltools torch # Write your conversion script # ... convert-my-model.py ``` -------------------------------- ### Benchmark Streaming ASR using CLI Source: https://docs.fluidinference.com/asr/streaming Demonstrates how to run benchmarks for the Parakeet EOU ASR model from the command line. This allows testing performance with different chunk sizes and limiting the number of files processed. ```bash swift run fluidaudio parakeet-eou --benchmark --chunk-size 160 --max-files 100 --use-cache ``` -------------------------------- ### Run VAD Analysis and Benchmarks via CLI (Bash) Source: https://docs.fluidinference.com/vad/getting-started Command-line interface commands for performing voice activity detection analysis and benchmarks. Supports offline and streaming modes for analysis, and configurable benchmarking. ```bash # Offline segmentation swift run fluidaudio vad-analyze audio.wav # Streaming mode swift run fluidaudio vad-analyze audio.wav --streaming --min-silence-ms 300 # Both modes swift run fluidaudio vad-analyze audio.wav --mode both # Benchmark swift run fluidaudio vad-benchmark --num-files 50 --threshold 0.3 ``` -------------------------------- ### Manage Datasets with FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Illustrates how to use the 'fluidaudio download' command to fetch various datasets, including AMI-SDM, LibriSpeech test sets, and VAD datasets. This command simplifies data acquisition for testing and development. ```bash fluidaudio download --dataset ami-sdm fluidaudio download --dataset librispeech-test-clean fluidaudio download --dataset librispeech-test-other fluidaudio download --dataset vad ``` -------------------------------- ### Run Sortformer Benchmark CLI Source: https://docs.fluidinference.com/diarization/sortformer This command executes the Sortformer benchmark using the Swift Package Manager. It enables NVIDIA's high-latency configuration, utilizes Hugging Face models, and automatically downloads necessary assets. This is useful for testing Sortformer's performance in a streaming context. ```bash swift run fluidaudio sortformer-benchmark \ --nvidia-high-latency --hf --auto-download ``` -------------------------------- ### Initialize and Transcribe with Custom Vocabulary (Swift) Source: https://docs.fluidinference.com/asr/custom-vocabulary This snippet demonstrates how to initialize the AsrManager, download and load CTC models, create a CtcKeywordSpotter, define a custom vocabulary with terms, and then transcribe audio using this vocabulary. The output text will reflect the recognized custom terms. ```swift let asrManager = try await AsrManager.shared let ctcModels = try await CtcModels.downloadAndLoad() let ctcSpotter = CtcKeywordSpotter(models: ctcModels) let vocabulary = CustomVocabularyContext(terms: [ CustomVocabularyTerm(text: "NVIDIA"), CustomVocabularyTerm(text: "TensorRT"), ]) let result = try await asrManager.transcribe( audioSamples, customVocabulary: vocabulary ) // result.text: "NVIDIA announced TensorRT optimizations" ``` -------------------------------- ### Swift Imports for FluidAudio and FluidAudioTTS Source: https://docs.fluidinference.com/tts/kokoro Demonstrates the necessary import statements in Swift to utilize the core FluidAudio functionalities (like ASR, diarization, VAD) and the specific FluidAudioTTS module for text-to-speech synthesis. ```swift import FluidAudio // Core (ASR, diarization, VAD) import FluidAudioTTS // TTS features ``` -------------------------------- ### Define Custom Pronunciations in Lexicon File Source: https://docs.fluidinference.com/asr/custom-pronunciation This text file defines custom pronunciations for words using IPA phonemes. Each line contains a word followed by its phonetic representation, separated by an equals sign. Comments start with '#'. ```text # This is a comment kokoro=kəkˈɔɹO NASDAQ=nˈæzdæk UN=junˈaɪtᵻd nˈeɪʃənz ``` -------------------------------- ### Load ASR Models from Local Directory in Swift Source: https://docs.fluidinference.com/asr/manual-model-loading This Swift code snippet demonstrates how to load ASR models from a specified local directory. It requires the `FluidAudio` framework and assumes the model bundles and vocabulary are correctly placed. The function returns an initialized `AsrManager` ready for use. ```swift import FluidAudio let repoDirectory = URL( fileURLWithPath: "/opt/models/parakeet-tdt-0.6b-v3-coreml", isDirectory: true ) let models = try await AsrModels.load( from: repoDirectory, configuration: AsrModels.defaultConfiguration(), version: .v3 ) let asrManager = AsrManager() try await asrManager.initialize(models: models) ``` -------------------------------- ### Quantize CoreML Model for Size and Speed Source: https://docs.fluidinference.com/mobius/converting-models This script performs quantization on CoreML models to reduce their size and potentially improve latency, especially on Apple Neural Engine (ANE). It sweeps through various quantization strategies and reports on the trade-offs between size reduction, quality impact, and performance. ```bash uv run python quantize_coreml.py \ --input-dir ./build \ --output-root ./build_quantized \ --compute-units ALL --runs 10 ``` -------------------------------- ### Define Custom Vocabulary with Aliases (Swift) Source: https://docs.fluidinference.com/asr/custom-vocabulary This code example shows how to configure a custom vocabulary that includes aliases for specific terms. When a match is found using either the canonical term or one of its aliases, the canonical form is used in the final transcription output. ```swift let vocabulary = CustomVocabularyContext(terms: [ CustomVocabularyTerm( text: "Hagen-Dazs", aliases: ["Haagen-Dazs", "Hagen-Das", "Hagen Daz"] ), CustomVocabularyTerm( text: "macOS", aliases: ["Mac OS", "Mac O S", "Macos"] ), ]) ``` -------------------------------- ### Offline Audio Segmentation with Silero VAD v6 (Swift) Source: https://docs.fluidinference.com/vad/getting-started Segments speech from an audio file using Silero VAD v6. It takes audio samples as input and returns a list of speech segments with start and end times. Requires the FluidAudio library. ```swift import FluidAudio let manager = try await VadManager( config: VadConfig(defaultThreshold: 0.75) ) let samples = try AudioConverter().resampleAudioFile( URL(fileURLWithPath: "audio.wav") ) var segmentation = VadSegmentationConfig.default segmentation.minSpeechDuration = 0.25 segmentation.minSilenceDuration = 0.4 segmentation.speechPadding = 0.12 let segments = try await manager.segmentSpeech(samples, config: segmentation) for (index, segment) in segments.enumerated() { print(String(format: "Segment %02d: %.2f-%.2fs", index + 1, segment.startTime, segment.endTime)) } ``` -------------------------------- ### Clone and Convert Parakeet STT Model using möbius Source: https://docs.fluidinference.com/mobius/getting-started This snippet demonstrates how to clone the möbius repository, set up the environment, convert a Parakeet STT model from N Nemo format to CoreML, and validate its parity. It requires the möbius repository and a local path to the .nemo model file. ```bash # Clone git clone https://github.com/FluidInference/mobius.git cd mobius # Pick a model cd models/stt/parakeet-tdt-v3-0.6b/coreml # Set up environment uv sync # Convert uv run python convert-parakeet.py convert \ --nemo-path /path/to/parakeet-tdt-0.6b-v3.nemo \ --output-dir parakeet_coreml # Validate parity uv run python compare-components.py compare \ --output-dir parakeet_coreml \ --runs 10 --warmup 3 ``` -------------------------------- ### Run FluidAudio Benchmarks (Bash) Source: https://docs.fluidinference.com/reference/benchmarks A collection of bash commands to execute various FluidAudio benchmarks. These commands cover transcription for multiple languages, custom vocabulary ASR, streaming ASR, Voice Activity Detection (VAD), offline and streaming diarization, Sortformer, and text-to-speech (TTS). ```bash swift run -c release fluidaudio fleurs-benchmark --languages all --samples all swift run -c release fluidaudio asr-benchmark --max-files all swift run -c release fluidaudio ctc-earnings-benchmark --auto-download swift run -c release fluidaudio parakeet-eou --benchmark --chunk-size 320 --use-cache swift run -c release fluidaudio vad-benchmark --dataset voices-subset --all-files --threshold 0.85 swift run -c release fluidaudio diarization-benchmark --mode offline --auto-download swift run -c release fluidaudio diarization-benchmark --mode streaming \ --dataset ami-sdm --threshold 0.8 --chunk-seconds 5.0 --overlap-seconds 0.0 swift run -c release fluidaudio sortformer-benchmark --nvidia-high-latency --hf --auto-download swift run -c release fluidaudio tts --benchmark ``` -------------------------------- ### Switch ASR Model Versions in Swift Source: https://docs.fluidinference.com/asr/manual-model-loading This Swift code shows how to load a different version of the ASR models from an alternative directory. It's useful for managing multiple language models or versions. The `AsrModels.load` function is used with a different repository path and version identifier. ```swift let englishRepo = URL(fileURLWithPath: "/opt/models/parakeet-tdt-0.6b-v2-coreml") let englishModels = try await AsrModels.load(from: englishRepo, version: .v2) ``` -------------------------------- ### Swift Package Manager Configuration for FluidAudio Source: https://docs.fluidinference.com/tts/kokoro Shows how to add the FluidAudio library, including TTS features, as a dependency in your Swift project using the Package.swift file. This enables the use of FluidAudio's TTS capabilities in your application. ```swift dependencies: [ .package(url: "https://github.com/FluidInference/FluidAudio.git", from: "0.7.7"), ], targets: [ .target( name: "YourTarget", dependencies: [ .product(name: "FluidAudioWithTTS", package: "FluidAudio") ] ) ] ``` -------------------------------- ### Swift Detailed Synthesis with Chunk Metadata Source: https://docs.fluidinference.com/tts/kokoro Illustrates how to synthesize text with Kokoro TTS and retrieve detailed chunk metadata, including variant, token count, and text for each chunk. This is useful for analyzing the synthesis process or implementing chunk-based audio processing. ```swift let detailed = try await manager.synthesizeDetailed( text: "FluidAudio can report chunk splits for you.", variantPreference: .fifteenSecond ) for chunk in detailed.chunks { print("Chunk #\(chunk.index) -> variant: \(chunk.variant), tokens: \(chunk.tokenCount)") print(" text: \(chunk.text)") } ``` -------------------------------- ### Initialize and Use Streaming ASR Manager in Swift Source: https://docs.fluidinference.com/asr/streaming Demonstrates how to initialize `StreamingEouAsrManager`, load models, process audio buffers incrementally, retrieve the final transcript, and reset the manager for a new utterance. This is the primary workflow for real-time transcription. ```swift let manager = StreamingEouAsrManager(chunkSize: .ms160, eouDebounceMs: 1280) try await manager.loadModels(modelDir: modelsURL) // Process audio incrementally _ = try await manager.process(audioBuffer: buffer1) _ = try await manager.process(audioBuffer: buffer2) // Get final transcript let transcript = try await manager.finish() // Reset for next utterance await manager.reset() ``` -------------------------------- ### Configure SpeakerManager Source: https://docs.fluidinference.com/diarization/speaker-manager Initializes the SpeakerManager with specific thresholds for speaker matching, embedding updates, and minimum speech durations. These parameters control how speakers are identified, updated, and created. ```swift let speakerManager = SpeakerManager( speakerThreshold: 0.65, // Max cosine distance for speaker match embeddingThreshold: 0.45, // Max distance for embedding updates minSpeechDuration: 1.0, // Min seconds to create new speaker minEmbeddingUpdateDuration: 2.0 // Min seconds to update embeddings ) ``` -------------------------------- ### Export PyTorch Model to CoreML (.mlpackage) Source: https://docs.fluidinference.com/mobius/converting-models This script converts a PyTorch, NeMo, or ONNX model to the CoreML .mlpackage format. It requires fixed input shapes and uses the 'coremltools' library. The process involves loading the model, tracing it with specified input shapes, converting it, and saving the output. ```bash cd models/stt/parakeet-tdt-v3-0.6b/coreml uv sync uv run python convert-parakeet.py convert \ --nemo-path /path/to/model.nemo \ --output-dir ./build ``` -------------------------------- ### Validate CoreML Model Parity and Latency Source: https://docs.fluidinference.com/mobius/converting-models This script compares the numerical output and measures the latency of both the original PyTorch model and its CoreML conversion. It runs the models side-by-side with identical inputs and generates detailed reports including numerical differences, latency comparisons, and plots. ```bash uv run python compare-components.py compare \ --output-dir ./build \ --model-id nvidia/parakeet-tdt-0.6b-v3 \ --runs 10 --warmup 3 ``` -------------------------------- ### Configure Proxy Settings via Environment Variable Source: https://docs.fluidinference.com/configuration Configure proxy settings for applications running behind a corporate firewall by setting the `https_proxy` environment variable. This allows traffic to be routed through a specified proxy server. Supports both unauthenticated and authenticated proxies. ```bash export https_proxy=http://proxy.company.com:8080 # Or for authenticated proxies: export https_proxy=http://user:password@proxy.company.com:8080 ``` -------------------------------- ### Perform Diarization with FluidAudio CLI Source: https://docs.fluidinference.com/reference/cli Illustrates the 'fluidaudio process' command for audio diarization. It covers processing a single file, using offline mode, and incorporating ground-truth data for evaluation. ```bash # Process a file fluidaudio process meeting.wav --output results.json --threshold 0.6 # Offline mode fluidaudio process meeting.wav --mode offline --threshold 0.6 # With ground-truth fluidaudio process meeting.wav --rttm ground_truth.rttm ``` -------------------------------- ### Model Registry Configuration Source: https://docs.fluidinference.com/reference/api Instructions on how to override the default HuggingFace URL for the Model Registry, either programmatically or via environment variables. ```APIDOC ## Model Registry Override the default HuggingFace URL: ```swift ModelRegistry.baseURL = "https://your-mirror.example.com" ``` Or via environment: `REGISTRY_URL=...` ``` -------------------------------- ### Manual Core ML Model Loading (Swift) Source: https://docs.fluidinference.com/diarization/getting-started This Swift code demonstrates how to manually load Core ML models for offline deployment of the speaker diarization pipeline. It specifies the file paths for the segmentation and embedding models. ```swift let basePath = "/opt/models/speaker-diarization-coreml" let segmentation = URL(fileURLWithPath: basePath) .appendingPathComponent("pyannote_segmentation.mlmodelc") let embedding = URL(fileURLWithPath: basePath) .appendingPathComponent("wespeaker_v2.mlmodelc") let models = try await DiarizerModels.load( localSegmentationModel: segmentation, localEmbeddingModel: embedding ) ``` -------------------------------- ### PocketTtsManager API Source: https://docs.fluidinference.com/reference/api API reference for the PocketTtsManager, including initialization, audio data synthesis, and direct file synthesis. ```APIDOC ## PocketTtsManager | Method | Description | | ----------------------------------- | ---------------------------------- | | `initialize()` | Download and load PocketTTS models | | `synthesize(text:)` | Generate audio Data | | `synthesizeToFile(text:outputURL:)` | Generate directly to file | ``` -------------------------------- ### Transcribe Audio with FluidAudio Source: https://docs.fluidinference.com/quickstart Transcribe audio files using FluidAudio's Automatic Speech Recognition (ASR) capabilities. This requires downloading and loading ASR models and initializing the AsrManager. ```swift import FluidAudio Task { let models = try await AsrModels.downloadAndLoad(version: .v3) let asrManager = AsrManager(config: .default) try await asrManager.initialize(models: models) let audioURL = URL(fileURLWithPath: "/path/to/audio.wav") let result = try await asrManager.transcribe(audioURL, source: .system) print("Transcription: \(result.text)") } ``` -------------------------------- ### Swift: Real-time Voice Activity Detection with Streaming Source: https://docs.fluidinference.com/vad/streaming Demonstrates real-time voice activity detection using FluidAudio's streaming capabilities. It initializes a VadManager, processes audio chunks from a microphone, and handles speech start/end events. This method is suitable for continuous audio streams where precise timing of speech segments is required. ```swift import FluidAudio let manager = try await VadManager() var state = await manager.makeStreamState() for chunk in microphoneChunks { let result = try await manager.processStreamingChunk( chunk, state: state, config: .default, returnSeconds: true, timeResolution: 2 ) state = result.state print(String(format: "Probability: %.3f", result.probability)) if let event = result.event { switch event.kind { case .speechStart: print("Speech began at \(event.time ?? 0) s") case .speechEnd: print("Speech ended at \(event.time ?? 0) s") } } } ``` -------------------------------- ### Manually Load Core ML VAD Model (Swift) Source: https://docs.fluidinference.com/vad/getting-started Loads the Silero VAD Core ML model manually for offline environments. Allows specifying the model path and configuring compute units (e.g., CPU only). ```swift let modelURL = URL( fileURLWithPath: "/opt/models/silero-vad-coreml/silero-vad-unified-256ms-v6.0.0.mlmodelc", isDirectory: true ) var configuration = MLModelConfiguration() configuration.computeUnits = .cpuOnly let vadModel = try MLModel(contentsOf: modelURL, configuration: configuration) let manager = VadManager(config: .default, vadModel: vadModel) ``` -------------------------------- ### Perform Diarization with FluidAudio (Swift) Source: https://docs.fluidinference.com/diarization/getting-started This snippet demonstrates how to perform complete speaker diarization on an audio file using the FluidAudio library in Swift. It includes downloading necessary models, initializing the diarizer, processing audio samples, and iterating through the results to print speaker segments. ```swift import FluidAudio let models = try await DiarizerModels.downloadIfNeeded() let diarizer = DiarizerManager() diarizer.initialize(models: models) let samples = try AudioConverter().resampleAudioFile( URL(fileURLWithPath: "meeting.wav") ) let result = try diarizer.performCompleteDiarization(samples) for segment in result.segments { print("Speaker \(segment.speakerId): \(segment.startTimeSeconds)s - \(segment.endTimeSeconds)s") } ``` -------------------------------- ### FluidAudio CLI Usage Source: https://docs.fluidinference.com/quickstart Command-line interface commands for FluidAudio. This allows for quick execution of transcription, diarization, and text-to-speech tasks directly from the terminal. ```bash # Transcribe swift run fluidaudio transcribe audio.wav # Diarize swift run fluidaudio process meeting.wav --mode offline --threshold 0.6 # TTS swift run fluidaudio tts "Hello from FluidAudio" --output out.wav ``` -------------------------------- ### AsrModels API Source: https://docs.fluidinference.com/reference/api API reference for managing ASR models, including downloading, loading, and checking for existing model bundles. ```APIDOC ## AsrModels | Method | Description | | --------------------------- | ---------------------------- | | `downloadAndLoad(version:)` | Download and compile models | | `load(from:version:)` | Load from staged directory | | `modelsExist(at:)` | Check if bundles are present | ``` -------------------------------- ### Audio Format Source: https://docs.fluidinference.com/reference/api Details on the expected audio format and how to use the AudioConverter for normalization. ```APIDOC ## Audio Format All pipelines expect **16 kHz mono Float32** samples. Use `AudioConverter` to normalize input: ```swift let converter = AudioConverter() let samples = try converter.resampleAudioFile(path: "audio.wav") ``` ```