### Setup Development Environment Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/CONTRIBUTING.md Use the Makefile to set up your development environment by installing necessary dependencies. ```bash make setup ``` -------------------------------- ### Setup Environment and Download Model Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Set up the necessary environment and download a specific model for use with the Swift CLI. Ensure git-lfs is installed before running. ```bash make setup make download-model MODEL=large-v3-v20240930_626MB ``` -------------------------------- ### Install Python Client Dependencies Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Installs the necessary Python dependencies for the OpenAI SDK client example. ```bash cd Examples/ServeCLIClient/Python uv sync ``` -------------------------------- ### Initialize and Run WhisperKit Client Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Python/README.md Commands to start the server, install dependencies, and execute basic transcription or translation tasks. ```bash whisperkit-cli serve ``` ```bash uv sync ``` ```bash # Transcribe an audio file python whisperkit_client.py transcribe audio.wav # Translate an audio file to English python whisperkit_client.py translate audio.wav # Test with sample files python whisperkit_client.py test ``` -------------------------------- ### Device Information Example Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Example output format for connected devices when running 'make list-devices'. Verify the 'state' is 'connected'. ```ruby { :name=>"My Mac", :type=>"Apple M2 Pro", :platform=>"macOS", :os_version=>"15.0.1", :product=>"Mac14,12", :id=>"XXXXXXXX-1234-5678-9012-XXXXXXXXXXXX", :state=>"connected" } ``` -------------------------------- ### WhisperKit Client Usage Examples Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Swift/README.md Advanced usage examples for transcription, translation, and server configuration. ```bash # Transcribe in Spanish swift run whisperkit-client transcribe -l es audio.wav # Transcribe with word-level timestamps swift run whisperkit-client transcribe --timestamp-granularities "word,segment" audio.wav # Translate from Spanish to English swift run whisperkit-client translate -l es audio.wav # Use custom server swift run whisperkit-client transcribe -s http://192.168.1.100:50060 audio.wav # Stream transcription swift run whisperkit-client transcribe --stream audio.wav ``` -------------------------------- ### WhisperKit Client Usage Examples Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Python/README.md Various command-line examples for transcribing, translating, and configuring the client with custom server settings or debug modes. ```bash # Transcribe in Spanish python whisperkit_client.py transcribe -l es audio.wav # Translate to English (auto-detects source language) python whisperkit_client.py translate audio.wav # Use custom server and model python whisperkit_client.py -s http://192.168.1.100:50060 -m large transcribe audio.wav # Transcribe with word-level timestamps python whisperkit_client.py transcribe --timestamp-granularities "word,segment" audio.wav # Stream transcription python whisperkit_client.py transcribe --stream audio.wav # Debug mode to see raw JSON python whisperkit_client.py transcribe --debug audio.wav # Test with sample files python whisperkit_client.py test ``` -------------------------------- ### Open Xcode Project Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Open the example project in Xcode to configure environment variables for benchmarks. ```sh xed Examples/WhisperAX ``` -------------------------------- ### Install WhisperKit CLI with Homebrew Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Install the WhisperKit command-line interface application using Homebrew. ```bash brew install whisperkit-cli ``` -------------------------------- ### Start Argmax Local Server Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Starts the Argmax local server with default settings or custom host, port, model, and logging. ```bash # Start server with default settings BUILD_ALL=1 swift run argmax-cli serve # Custom host and port BUILD_ALL=1 swift run argmax-cli serve --host 0.0.0.0 --port 8080 # With specific model and verbose logging BUILD_ALL=1 swift run argmax-cli serve --model tiny --verbose # See all configurable parameters BUILD_ALL=1 swift run argmax-cli serve --help ``` -------------------------------- ### Install Xcode Command Line Tools Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/fastlane/README.md Ensures the necessary Xcode command line tools are installed on the system. ```sh xcode-select --install ``` -------------------------------- ### Start WhisperKit Server Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Swift/README.md Command to launch the local WhisperKit server instance. ```bash whisperkit-cli serve ``` -------------------------------- ### Python OpenAI SDK Transcription Example Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md A quick Python example demonstrating how to transcribe an audio file using the OpenAI SDK pointed at the local Argmax server. ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:50060/v1") result = client.audio.transcriptions.create( file=open("audio.wav", "rb"), model="tiny" # Model parameter is required ) print(result.text) ``` -------------------------------- ### Clone Argmax OSS Swift Repository Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Clone the repository to your local machine to begin the installation process. ```bash git clone https://github.com/argmaxinc/argmax-oss-swift.git cd argmax-oss-swift ``` -------------------------------- ### Quick Start: Transcribe Local Audio Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Initialize WhisperKit and transcribe a local audio file. Ensure the audio file path is correct. ```swift import WhisperKit // Initialize WhisperKit with default settings Task { let pipe = try? await WhisperKit() let transcription = try? await pipe!.transcribe(audioPath: "path/to/your/audio.{wav,mp3,m4a,flac}")?.text print(transcription) } ``` -------------------------------- ### Translate Audio Files Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Curl/README.md Usage examples for the translate.sh script to convert audio to English. ```bash ./translate.sh audio.wav ./translate.sh audio.wav --language es ./translate.sh audio.wav --stream true --logprobs ``` -------------------------------- ### Transcribe Audio Files Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Curl/README.md Usage examples for the transcribe.sh script with various optional flags. ```bash ./transcribe.sh audio.wav ./transcribe.sh audio.wav --language en --timestamp-granularities word,segment ./transcribe.sh audio.wav --stream true --logprobs ``` -------------------------------- ### Initialize and Run WhisperKit Server Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Curl/README.md Commands to prepare the environment and launch the local server instance. ```bash chmod +x *.sh ``` ```bash whisperkit-cli serve --model tiny ``` -------------------------------- ### Build and Test Project Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/CONTRIBUTING.md Build the project and run tests using the Makefile. Ensure your changes are functional and adhere to project standards. ```bash make build make test ``` -------------------------------- ### Run Swift Client Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Executes the Swift client to transcribe or translate audio files using the local server. ```bash cd Examples/ServeCLIClient/Swift swift run whisperkit-client transcribe audio.wav --language en swift run whisperkit-client translate audio.wav ``` -------------------------------- ### Download All Available Models Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Use this command to download all models available in the repository to your local folder. ```bash make download-models ``` -------------------------------- ### Apply Style Instructions (1.7B Model) Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md For the 1.7B model, use the `instruction` property in `GenerationOptions` to control prosody with natural language, such as speaking slowly and warmly. ```swift var options = GenerationOptions() options.instruction = "Speak slowly and warmly, like a storyteller." let result = try await tts.generate( text: "Once upon a time...", speaker: .ryan, options: options ) ``` -------------------------------- ### Build WhisperKit Client Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Swift/README.md Command to compile the Swift client project. ```bash swift build ``` -------------------------------- ### Run Full Benchmarks Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Execute the full benchmark tests. This is the standard command for running all tests. ```sh make benchmark-devices ``` -------------------------------- ### Project Directory Structure Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Python/README.md The file layout for the Python client implementation. ```text Examples/ServeCLIClient/Python/ ├── whisperkit_client.py # Main CLI script with all functionality ├── test_transcribe.py # Test script for transcription ├── test_translate.py # Test script for translation ├── requirements.txt # Python dependencies ├── uv.lock # Locked dependency versions └── README.md # This file ``` -------------------------------- ### Download Models Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/CONTRIBUTING.md Download required models for local testing and running the project. Specify the model size with the MODEL argument. ```bash make download-model MODEL=tiny ``` -------------------------------- ### Run WhisperKit Client Commands Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Swift/README.md Basic commands for transcribing, translating, and testing audio files. ```bash # Transcribe an audio file swift run whisperkit-client transcribe audio.wav # Translate an audio file to English swift run whisperkit-client translate audio.wav # Test with sample files swift run whisperkit-client test ``` -------------------------------- ### SpeakerKit Initialization Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Initializes SpeakerKit with default configurations, automatically downloading necessary models from HuggingFace on first run. Models are loaded lazily. ```APIDOC ## SpeakerKit() ### Description Initializes the SpeakerKit with default PyannoteConfig. Models are downloaded from HuggingFace on first run and loaded lazily. ### Method `SpeakerKit()` ### Parameters None ``` -------------------------------- ### List Connected Devices Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Use this command to list all connected devices and verify their connection status before running benchmarks. ```sh make list-devices ``` -------------------------------- ### Run Client Tests Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Python/README.md Commands to execute the built-in test suite or individual test scripts. ```bash # Run tests on sample files python whisperkit_client.py test # Or run individual test scripts python test_transcribe.py python test_translate.py ``` -------------------------------- ### SpeakerKit Initialization with Custom Config Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Initializes SpeakerKit with a custom configuration, allowing for local model paths to be specified. ```APIDOC ## SpeakerKit(config: PyannoteConfig) ### Description Initializes the SpeakerKit with a custom PyannoteConfig, useful for specifying local model paths and skipping the download step. ### Method `SpeakerKit(config: PyannoteConfig)` ### Parameters #### Path Parameters - **config** (PyannoteConfig) - Required - Custom configuration for SpeakerKit, including model paths. ``` -------------------------------- ### Run Benchmarks on Specific Devices Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Optionally specify a comma-separated list of device names to run benchmarks on. Device names can be found using 'make list-devices'. ```sh make benchmark-devices DEVICES="iPhone 15 Pro Max,My Mac" ``` -------------------------------- ### Build Argmax Local Server Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Builds the Argmax CLI with local server support using Make or manual Swift build flags. ```bash # Build with server support make build-local-server # Or manually with the build flag BUILD_ALL=1 swift build --product argmax-cli ``` -------------------------------- ### Run Python OpenAI SDK Client Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Executes Python scripts to transcribe or translate audio files using the local server via the OpenAI SDK. ```bash python whisperkit_client.py transcribe --file audio.wav --language en python whisperkit_client.py translate --file audio.wav ``` -------------------------------- ### Project Structure Overview Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Swift/README.md Directory layout of the WhisperKit Swift client source code. ```text Sources/ ├── CLI.swift # All CLI commands and client logic └── Generated/ # Auto-generated OpenAPI client code ├── Client.swift └── Types.swift ``` -------------------------------- ### Transcribe Audio File with Swift CLI Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Run the Swift CLI to transcribe an audio file. Specify the path to the downloaded model and the audio file. ```bash swift run argmax-cli transcribe --model-path "Models/whisperkit-coreml/openai_whisper-large-v3-v20240930_626MB" --audio-path "path/to/your/audio.{wav,mp3,m4a,flac}" ``` -------------------------------- ### Clone Argmax OSS Repository Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/CONTRIBUTING.md Clone your forked repository to your local machine to begin making changes. Navigate into the cloned directory. ```bash git clone https://github.com/[your-username]/argmax-oss-swift.git cd argmax-oss-swift ``` -------------------------------- ### Configure Playback Strategy Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Control the audio buffering behavior during real-time playback using different strategies. The `.auto` strategy dynamically adjusts buffering based on generation speed. ```swift try await tts.play( text: "Long passage...", playbackStrategy: .auto ) ``` -------------------------------- ### Clone WhisperKit Repository Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Use this command to download the WhisperKit source code for running benchmarks. ```sh git clone git@github.com:argmaxinc/argmax-oss-swift.git ``` -------------------------------- ### Diarize Audio File with SpeakerKit Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Demonstrates basic usage of SpeakerKit to diarize an audio file. It loads audio, performs diarization, and prints the detected speaker count and segments. Models are downloaded automatically on first run. ```swift import SpeakerKit Task { let speakerKit = try await SpeakerKit() let audioArray = try AudioProcessor.loadAudioAsFloatArray(fromPath: "audio.wav") let result = try await speakerKit.diarize(audioArray: audioArray) print("Detected \(result.speakerCount) speakers") for segment in result.segments { print(segment) } } ``` -------------------------------- ### Real-Time Streaming Playback Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Utilize the `play` function for real-time streaming of generated audio. This allows playback to begin before the entire audio is generated. ```swift try await tts.play(text: "This starts playing before generation finishes.") ``` -------------------------------- ### Run Curl Client Scripts Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Makes the shell scripts executable and runs them to transcribe, translate, or test audio files with the local server. ```bash cd Examples/ServeCLIClient/Curl chmod +x *.sh ./transcribe.sh audio.wav --language en ./translate.sh audio.wav --language es ./test.sh # Run comprehensive test suite ``` -------------------------------- ### Benchmark iOS Devices Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/fastlane/README.md Benchmarks connected devices with configurable options. ```sh [bundle exec] fastlane ios benchmark ``` -------------------------------- ### Receive Progress Callbacks During Generation Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Implement progress callbacks to receive audio chunks as they are generated. Returning `false` from the callback will cancel the generation process early. ```swift let result = try await tts.generate(text: "Hello!") { progress in print("Audio chunk: \(progress.audio.count) samples") if let stepTime = progress.stepTime { print("First step took \(stepTime)s") } return true // return false to cancel } ``` -------------------------------- ### TTSKitExample Project Structure Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/TTS/TTSKitExample/README.md Overview of the main files and their roles within the TTSKitExample application. ```plaintext TTSKitExample/ ├── TTSKitExampleApp.swift entry point ├── ContentView.swift root NavigationSplitView ├── SidebarView.swift model management + history ├── DetailView.swift input form, waveform, playback ├── ModelManagementView.swift download / load / unload ├── GenerationSettingsView.swift advanced options sheet ├── ViewModel.swift @Observable view model ├── AudioMetadata.swift Codable metadata embedded in .m4a files ├── WaveformView.swift live waveform bar chart └── ComputeUnitsView.swift per-component compute unit picker ``` -------------------------------- ### Run Debug Benchmarks Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/BENCHMARKS.md Execute the debug benchmark tests. Set DEBUG=true to check for potential errors. ```sh make benchmark-devices DEBUG=true ``` -------------------------------- ### Upload Benchmark Results Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/fastlane/README.md Uploads the extracted benchmark results. ```sh [bundle exec] fastlane ios upload_results ``` -------------------------------- ### Stream Audio for Transcription with Swift CLI Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Use the Swift CLI to transcribe audio streamed directly from a microphone. Ensure the model path is correctly specified. ```bash swift run argmax-cli transcribe --model-path "Models/whisperkit-coreml/openai_whisper-large-v3-v20240930_626MB" --stream ``` -------------------------------- ### Add Target Dependencies for Argmax OSS Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Specify the Argmax OSS products as dependencies for your application's target in Package.swift. You can import all kits or individual ones. ```swift .target( name: "YourApp", dependencies: [ // Import everything at once: .product(name: "ArgmaxOSS", package: "argmax-oss-swift"), // Or pick individual kits: // .product(name: "WhisperKit", package: "argmax-oss-swift"), // speech-to-text // .product(name: "TTSKit", package: "argmax-oss-swift"), // text-to-speech // .product(name: "SpeakerKit", package: "argmax-oss-swift"), // speaker diarization ] ), ``` -------------------------------- ### Control Speaker Detection with Options Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Shows how to customize speaker detection using `PyannoteDiarizationOptions`. You can specify the number of speakers, adjust the clustering threshold, and control speaker assignment exclusivity. ```swift let audioArray = try AudioProcessor.loadAudioAsFloatArray(fromPath: "audio.wav") let options = PyannoteDiarizationOptions( numberOfSpeakers: 2, // nil = automatic detection clusterDistanceThreshold: 0.6, // clustering threshold useExclusiveReconciliation: false // exclusive speaker assignment per frame ) let result = try await speakerKit.diarize(audioArray: audioArray, options: options) ``` -------------------------------- ### Select Speech Decoder Mode Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Choose between `latencyOptimized` (default) for lowest time-to-first-audio or `throughputOptimized` for higher throughput at the cost of initial latency. This setting is applied when models are loaded. ```swift // Default: latency-optimized (lowest time-to-first-audio) let tts = try await TTSKit() ``` ```swift // Opt into throughput-optimized generation let config = TTSKitConfig(speechDecoderMode: .throughputOptimized) let throughputTTS = try await TTSKit(config) ``` -------------------------------- ### List iOS Devices Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/fastlane/README.md Lists all connected iOS devices. ```sh [bundle exec] fastlane ios list_devices ``` -------------------------------- ### Generate TypeScript Client from OpenAPI Spec Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Generates a TypeScript-Fetch client from the local server's OpenAPI specification using openapi-generator-cli. ```bash # Generate TypeScript client npx @openapitools/openapi-generator-cli generate \ -i scripts/specs/localserver_openapi.yaml \ -g typescript-fetch \ -o typescript-client ``` -------------------------------- ### Execute WhisperKit Scripts Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Curl/README.md Basic commands for interacting with the transcription and translation scripts. ```bash # Transcribe audio ./transcribe.sh audio.wav # Translate audio to English ./translate.sh audio.wav --language es # Run test suite ./test.sh ``` -------------------------------- ### Generate OpenAPI Spec and Server Code Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Generates the latest OpenAPI specification and server code for the Argmax CLI using Make. ```bash # Generate latest spec and server code make generate-server ``` -------------------------------- ### Generate Python Client from OpenAPI Spec Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Generates a Python client and types from the local server's OpenAPI specification using Swift OpenAPI Generator. ```bash # Generate Python client swift run swift-openapi-generator generate scripts/specs/localserver_openapi.yaml \ --output-directory python-client \ --mode client \ --mode types ``` -------------------------------- ### Generate Speech with TTS CLI Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Use the argmax-cli tool to generate speech from text. Options include playing audio in real-time, saving to a file, specifying language and speaker, and using text files with models and instructions. ```bash swift run argmax-cli tts --text "Hello from the command line" --play ``` ```bash swift run argmax-cli tts --text "Save to file" --output-path output.wav ``` ```bash swift run argmax-cli tts --text "日本語テスト" --speaker ono-anna --language japanese ``` ```bash swift run argmax-cli tts --text-file article.txt --model 1.7b --instruction "Read cheerfully" ``` ```bash swift run argmax-cli tts --help ``` -------------------------------- ### Generate Speech from Text Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Basic usage to generate speech from a given text. The default model is downloaded on first run and loaded lazily. ```swift import TTSKit Task { let tts = try await TTSKit() let result = try await tts.generate(text: "Hello from TTSKit!") print("Generated \(result.audioDuration)s of audio at \(result.sampleRate)Hz") } ``` -------------------------------- ### Select TTSKit Model Size Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Choose between different model sizes for TTSKit. The smaller model is faster and compatible with all platforms, while the larger model offers higher quality on macOS and supports style instructions. ```swift // Fast, runs on all platforms (~1 GB download) let tts = try await TTSKit(TTSKitConfig(model: .qwen3TTS_0_6b)) ``` ```swift // Higher quality, macOS only (~2.2 GB download, supports style instructions) let tts = try await TTSKit(TTSKitConfig(model: .qwen3TTS_1_7b)) ``` -------------------------------- ### iOS Fastlane Actions Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/fastlane/README.md A collection of Fastlane actions for managing iOS devices and benchmark results. ```APIDOC ## ios list_devices ### Description List all connected iOS devices. ### Command `fastlane ios list_devices` ## ios benchmark ### Description Benchmark connected iOS devices with configurable options. ### Command `fastlane ios benchmark` ## ios extract_results ### Description Extract benchmark results from the iOS environment. ### Command `fastlane ios extract_results` ## ios upload_results ### Description Upload the extracted benchmark results. ### Command `fastlane ios upload_results` ``` -------------------------------- ### BibTeX Citation for Argmax OSS Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Use this BibTeX entry when citing the Argmax OSS SDK, including WhisperKit, SpeakerKit, and TTSKit, in academic work. ```bibtex @misc{whisperkit-argmax, title = {Argmax OSS: On-device Speech AI with WhisperKit, SpeakerKit and TTSKit}, author = {Argmax, Inc.}, year = {2024}, URL = {https://github.com/argmaxinc/argmax-oss-swift} } ``` -------------------------------- ### Use Local SpeakerKit Models Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Configure SpeakerKit to use local CoreML models by specifying the model folder path. This bypasses the automatic download from HuggingFace. ```swift let config = PyannoteConfig(modelFolder: "/path/to/models") let speakerKit = try await SpeakerKit(config) ``` -------------------------------- ### Transcribe Audio with Diarization using argmax-cli Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Combines transcription and diarization using the argmax-cli tool. This command processes an audio file and outputs speaker-attributed transcriptions. ```bash swift run argmax-cli transcribe --audio-path audio.wav --diarization ``` -------------------------------- ### POST /v1/audio/transcriptions Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Transcribes audio to text using the local server. Supports various audio formats and includes options for model selection, language detection, and streaming. ```APIDOC ## POST /v1/audio/transcriptions ### Description Transcribes audio to text. This endpoint allows users to send audio files and receive text transcriptions. ### Method POST ### Endpoint /v1/audio/transcriptions ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **file** (Audio file) - Required - Audio file (wav, mp3, m4a, flac) - **model** (string) - Server default - Model identifier - **language** (string) - Auto-detect - Source language code - **prompt** (string) - None - Text to guide transcription - **response_format** (string) - verbose_json - Output format (json, verbose_json) - **temperature** (number) - 0.0 - Sampling temperature (0.0-1.0) - **timestamp_granularities[]** (string) - segment - Timing detail (word, segment) - **stream** (boolean) - false - Enable streaming ``` -------------------------------- ### Extract Benchmark Results Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/fastlane/README.md Extracts the results from the benchmark process. ```sh [bundle exec] fastlane ios extract_results ``` -------------------------------- ### Generate Custom CoreML Models Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Initialize WhisperKit with a custom model and model repository. This is used for deploying fine-tuned models. ```swift let config = WhisperKitConfig(model: "large-v3-v20240930_626MB", modelRepo: "username/your-model-repo") let pipe = try? await WhisperKit(config) ``` -------------------------------- ### Configure TTSKit with a Private Model Token Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/TTS/TTSKitExample/README.md Set a Hugging Face model token for private model forks. This configuration is used when initializing TTSKit. ```swift let config = TTSKitConfig(model: selectedPreset, modelToken: "hf_...") ``` -------------------------------- ### Perform Diarization with argmax-cli Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Use the argmax-cli tool for standalone speaker diarization. Options include verbose output, saving RTTM results, and specifying the number of speakers. ```bash swift run argmax-cli diarize --audio-path audio.wav --verbose ``` ```bash swift run argmax-cli diarize --audio-path audio.wav --rttm-path output.rttm ``` ```bash swift run argmax-cli diarize --audio-path audio.wav --num-speakers 3 ``` ```bash swift run argmax-cli diarize --help ``` -------------------------------- ### POST /v1/audio/translations Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Translates audio to English using the local server. Supports various audio formats and includes options for model selection, language detection, and streaming. ```APIDOC ## POST /v1/audio/translations ### Description Translates audio to English. This endpoint allows users to send audio files and receive English translations. ### Method POST ### Endpoint /v1/audio/translations ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **file** (Audio file) - Required - Audio file (wav, mp3, m4a, flac) - **model** (string) - Server default - Model identifier - **language** (string) - Auto-detect - Source language code - **prompt** (string) - None - Text to guide transcription - **response_format** (string) - verbose_json - Output format (json, verbose_json) - **temperature** (number) - 0.0 - Sampling temperature (0.0-1.0) - **timestamp_granularities[]** (string) - segment - Timing detail (word, segment) - **stream** (boolean) - false - Enable streaming ``` -------------------------------- ### Regenerate Client Code Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/Examples/ServeCLIClient/Swift/README.md Script to update generated client code based on the OpenAPI specification. ```bash ./updateClient.sh ``` -------------------------------- ### Add Swift Package Dependency Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Add the Argmax OSS Swift package to your project's dependencies in Package.swift. ```swift dependencies: [ .package(url: "https://github.com/argmaxinc/argmax-oss-swift.git", from: "0.9.0"), ], ``` -------------------------------- ### Customize Generation Options Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Adjust generation parameters such as temperature, topK, repetition penalty, and chunking strategy. The `concurrentWorkerCount` can be set to `nil` to use a default suitable for the device. ```swift // Defaults recommended by Qwen var options = GenerationOptions() options.temperature = 0.9 options.topK = 50 options.repetitionPenalty = 1.05 options.maxNewTokens = 245 // Long text is automatically split at sentence boundaries options.chunkingStrategy = .sentence options.concurrentWorkerCount = nil // nil = all chunks run concurrently with a good default for the device let result = try await tts.generate(text: longArticle, options: options) ``` -------------------------------- ### Commit Changes Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/CONTRIBUTING.md Commit your changes with a clear and descriptive message. This helps in tracking modifications and understanding the purpose of each commit. ```bash git commit -am "Add a new feature" ``` -------------------------------- ### Push Changes to Fork Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/CONTRIBUTING.md Push your committed changes to your personal fork on GitHub. This makes your changes available for creating a pull request. ```bash git push origin my-branch ``` -------------------------------- ### Generate Speech with Custom Voice and Language Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Customize the speech output by selecting a specific voice and language. A wide range of built-in voices and languages are supported. ```swift let result = try await tts.generate( text: "こんにちは世界", speaker: .onoAnna, language: .japanese ) ``` -------------------------------- ### Specify WhisperKit Model Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Initialize WhisperKit with a specific model. The model name must exist in the HuggingFace repo. ```swift let pipe = try? await WhisperKit(WhisperKitConfig(model: "large-v3-v20240930_626MB")) ``` -------------------------------- ### Diarize Audio Array with Options Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Performs speaker diarization with custom options to control parameters like the number of speakers and clustering thresholds. ```APIDOC ## diarize(audioArray: [Float], options: PyannoteDiarizationOptions) async throws -> SpeakerDiarizationResult ### Description Performs speaker diarization with specified options to fine-tune the detection process, such as setting the number of speakers or adjusting clustering thresholds. ### Method `diarize(audioArray: [Float], options: PyannoteDiarizationOptions) async throws` ### Parameters #### Path Parameters - **audioArray** ([Float]) - Required - An array of floating-point numbers representing the audio data. - **options** (PyannoteDiarizationOptions) - Required - Options to customize diarization, including `numberOfSpeakers`, `clusterDistanceThreshold`, and `useExclusiveReconciliation`. ### Response #### Success Response - **speakerCount** (Int) - The total number of unique speakers detected. - **segments** ([SpeakerSegment]) - An array of SpeakerSegment objects, each containing start time, end time, and speaker label. ``` -------------------------------- ### Save Generated Audio Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Save the generated speech audio to a file in WAV or M4A format. The output directory is determined by `FileManager.default`. ```swift let result = try await tts.generate(text: "Save me!") let outputDir = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] // Save as .wav or .m4a (AAC) try await AudioOutput.saveAudio(result.audio, toFolder: outputDir, filename: "output", format: .m4a) ``` -------------------------------- ### Combine Diarization with Transcription Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Integrates SpeakerKit with WhisperKit to attribute transcribed text to specific speakers. It processes audio, performs transcription and diarization, and then merges the results. ```swift import WhisperKit import SpeakerKit let whisperKit = try await WhisperKit() let speakerKit = try await SpeakerKit() let audioArray = try AudioProcessor.loadAudioAsFloatArray(fromPath: "audio.wav") let transcription = try await whisperKit.transcribe(audioArray: audioArray) let diarization = try await speakerKit.diarize(audioArray: audioArray) let speakerSegments = diarization.addSpeakerInfo(to: transcription) for group in speakerSegments { for segment in group { print("\(segment.speaker): \(segment.text)") } } ``` -------------------------------- ### Diarize Audio Array Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Performs speaker diarization on a given audio array using the default options. ```APIDOC ## diarize(audioArray: [Float]) async throws -> SpeakerDiarizationResult ### Description Performs speaker diarization on the provided audio data. The result includes speaker count and time-stamped segments. ### Method `diarize(audioArray: [Float]) async throws` ### Parameters #### Path Parameters - **audioArray** ([Float]) - Required - An array of floating-point numbers representing the audio data. ### Response #### Success Response - **speakerCount** (Int) - The total number of unique speakers detected. - **segments** ([SpeakerSegment]) - An array of SpeakerSegment objects, each containing start time, end time, and speaker label. ``` -------------------------------- ### Generate RTTM Output with SpeakerKit Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Generates output in the RTTM (Rich Transcription Time Model) format from SpeakerKit diarization results. This format is commonly used for speech processing evaluation. ```swift let speakerKit = try await SpeakerKit() let audioArray = try AudioProcessor.loadAudioAsFloatArray(fromPath: "meeting.wav") let diarization = try await speakerKit.diarize(audioArray: audioArray) let rttmLines = SpeakerKit.generateRTTM(from: diarization, fileName: "meeting") for line in rttmLines { print(line) } ``` -------------------------------- ### Generate RTTM Output Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Generates output in the RTTM (Rich Transcription Time Marked) format from diarization results. ```APIDOC ## generateRTTM(from: SpeakerDiarizationResult, fileName: String) -> [String] ### Description Generates a list of strings formatted as RTTM lines, representing the diarization results for a given audio file. ### Method `generateRTTM(from: SpeakerDiarizationResult, fileName: String)` ### Parameters #### Path Parameters - **diarization** (SpeakerDiarizationResult) - Required - The result object from the `diarize` method. - **fileName** (String) - Required - The base name for the RTTM output. ### Response #### Success Response - **rttmLines** ([String]) - An array of strings, each representing a line in the RTTM format. ``` -------------------------------- ### Add Speaker Info to Transcription Source: https://github.com/argmaxinc/argmax-oss-swift/blob/main/README.md Merges diarization results with transcription data to attribute speech segments to specific speakers. ```APIDOC ## addSpeakerInfo(to: Transcription) -> [[SpeakerSegment]] ### Description Combines speaker diarization results with WhisperKit transcriptions to create speaker-attributed text segments. Supports `.subsegment` (default) and `.segment` matching strategies. ### Method `addSpeakerInfo(to: Transcription)` ### Parameters #### Path Parameters - **transcription** (Transcription) - Required - The transcription result from WhisperKit. ### Response #### Success Response - **speakerSegments** ([[SpeakerSegment]]) - A nested array where each inner array represents segments attributed to a single speaker. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.