# Symphonia Symphonia is a pure Rust audio decoding and media demuxing library designed for high-quality, efficient audio processing. It supports a wide range of popular audio codecs including AAC, ADPCM, ALAC, FLAC, MP1/MP2/MP3, Opus, PCM, Vorbis, and WavPack, as well as container formats like AIFF, CAF, ISO/MP4, MKV/WebM, OGG, and WAV. The library provides 100% safe Rust code with no unsafe blocks, minimal dependencies, and performance comparable to or exceeding FFmpeg's implementations. The library is architected around a clear separation between format readers (demuxers) and decoders, enabling flexible composition of components. It features automatic format detection through a sophisticated probing system, comprehensive metadata reading support (ID3v1, ID3v2, APE, Vorbis comments), gapless playback capabilities, and SIMD-optimized decoding paths for SSE, AVX, and ARM Neon instruction sets. ## MediaSourceStream - Create a Media Source Stream MediaSourceStream wraps any source implementing the MediaSource trait (which combines std::io::Read and std::io::Seek) and provides an optimized buffered reader specifically designed for multimedia use-cases. It handles seeking, buffering, and efficient byte-level I/O operations. ```rust use symphonia::core::io::{MediaSourceStream, MediaSourceStreamOptions, ReadOnlySource}; use std::fs::File; use std::io::Cursor; // Create from a file (implements MediaSource automatically) let file = File::open("audio.mp3").expect("failed to open file"); let mss = MediaSourceStream::new(Box::new(file), Default::default()); // Create from an in-memory buffer let audio_data: Vec = std::fs::read("audio.flac").unwrap(); let cursor = Cursor::new(audio_data); let mss = MediaSourceStream::new(Box::new(cursor), Default::default()); // Create from standard input (non-seekable) let stdin = std::io::stdin(); let source = ReadOnlySource::new(stdin); let mss = MediaSourceStream::new(Box::new(source), Default::default()); // Create with custom buffer options let file = File::open("large_audio.wav").expect("failed to open file"); let opts = MediaSourceStreamOptions { buffer_len: 64 * 1024, // 64 KB buffer }; let mss = MediaSourceStream::new(Box::new(file), opts); ``` ## Probe - Automatic Format Detection The Probe system automatically detects the format of media sources by scanning for format markers and metadata. It returns an appropriate FormatReader for the detected container format, with any discovered metadata attached. Hints can be provided to improve detection accuracy. ```rust use symphonia::core::formats::FormatOptions; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use std::fs::File; fn probe_media(path: &str) -> Result<(), Box> { // Open the media source let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); // Create a hint with the file extension to improve format detection let mut hint = Hint::new(); if let Some(ext) = std::path::Path::new(path).extension() { if let Some(ext_str) = ext.to_str() { hint.with_extension(ext_str); } } // Configure format and metadata options let fmt_opts = FormatOptions { prebuild_seek_index: false, // Build seek index on-demand seek_index_fill_period_ms: 1000, // 1 second between seek points ..Default::default() }; let meta_opts: MetadataOptions = Default::default(); // Probe the media source and get the format reader let format = symphonia::default::get_probe() .probe(&hint, mss, fmt_opts, meta_opts) .expect("unsupported format"); // Access format information let info = format.format_info(); println!("Format: {} ({})", info.long_name, info.short_name); // List available tracks for track in format.tracks() { println!("Track {}: {:?}", track.id, track.codec_params); } Ok(()) } ``` ## FormatReader - Reading Media Containers FormatReader is the trait implemented by all container demuxers. It provides methods to access tracks, read packets, seek within the media, and access metadata. The default_track method helps select the appropriate track for playback. ```rust use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::formats::{FormatOptions, FormatReader, SeekMode, SeekTo, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use symphonia::core::units::Time; use std::fs::File; fn read_media_container(path: &str) -> Result<(), Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let hint = Hint::new(); let fmt_opts: FormatOptions = Default::default(); let meta_opts: MetadataOptions = Default::default(); let mut format = symphonia::default::get_probe() .probe(&hint, mss, fmt_opts, meta_opts)?; // Get the default audio track let track = format.default_track(TrackType::Audio) .expect("no audio track found"); println!("Track ID: {}", track.id); // Get track duration if available if let (Some(time_base), Some(duration)) = (track.time_base, track.duration) { let time = time_base.calc_time(duration); println!("Duration: {:.2} seconds", time.seconds as f64 + time.frac); } // Get codec parameters if let Some(ref params) = track.codec_params { if let Some(audio_params) = params.audio() { println!("Sample rate: {:?}", audio_params.sample_rate); println!("Channels: {:?}", audio_params.channels); } } // Seek to 30 seconds into the media let seeked = format.seek( SeekMode::Accurate, SeekTo::Time { time: Time::new(30, 0.0), track_id: Some(track.id), }, )?; println!("Seeked to timestamp: {}", seeked.actual_ts); // Read packets while let Ok(Some(packet)) = format.next_packet() { println!("Packet for track {}: {} bytes", packet.track_id(), packet.data().len()); } Ok(()) } ``` ## AudioDecoder - Decoding Audio Packets AudioDecoder is the trait for audio codec implementations. After obtaining packets from a FormatReader, they are passed to the decoder which produces audio buffers containing PCM samples. The decoder handles gapless playback by trimming delay and padding frames. ```rust use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::errors::Error; use symphonia::core::formats::{FormatOptions, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use std::fs::File; fn decode_audio(path: &str) -> Result<(), Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let mut hint = Hint::new(); hint.with_extension("mp3"); let fmt_opts: FormatOptions = Default::default(); let meta_opts: MetadataOptions = Default::default(); let mut format = symphonia::default::get_probe() .probe(&hint, mss, fmt_opts, meta_opts)?; // Get the default audio track let track = format.default_track(TrackType::Audio) .expect("no audio track"); // Configure decoder options let dec_opts = AudioDecoderOptions { gapless: true, // Enable gapless playback (trim delay/padding) verify: false, // Don't verify decoded audio }; // Create the decoder let mut decoder = symphonia::default::get_codecs() .make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &dec_opts, ) .expect("unsupported codec"); let track_id = track.id; let mut total_samples = 0u64; // Decode loop loop { let packet = match format.next_packet() { Ok(Some(packet)) => packet, Ok(None) => break, // End of stream Err(Error::ResetRequired) => { // Track list changed (e.g., chained OGG streams) decoder.reset(); continue; } Err(e) => return Err(e.into()), }; // Skip packets not belonging to our track if packet.track_id() != track_id { continue; } // Decode the packet match decoder.decode(&packet) { Ok(audio_buf) => { let frames = audio_buf.frames(); let channels = audio_buf.num_planes(); total_samples += (frames * channels) as u64; } Err(Error::IoError(_)) | Err(Error::DecodeError(_)) => { // Skip corrupted packets continue; } Err(e) => return Err(e.into()), } } // Finalize and check verification result let finalize_result = decoder.finalize(); if let Some(verify_ok) = finalize_result.verify_ok { println!("Verification: {}", if verify_ok { "PASSED" } else { "FAILED" }); } println!("Total samples decoded: {}", total_samples); Ok(()) } ``` ## GenericAudioBufferRef - Accessing Decoded Audio GenericAudioBufferRef is the return type from AudioDecoder::decode(). It's a type-erased wrapper around typed audio buffers supporting various sample formats (u8, i16, i24, i32, f32, f64). It provides methods to copy audio data in interleaved or planar format with automatic sample format conversion. ```rust use symphonia::core::audio::sample::Sample; use symphonia::core::audio::{Audio, GenericAudioBufferRef}; use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::formats::{FormatOptions, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use std::fs::File; fn extract_audio_samples(path: &str) -> Result, Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let hint = Hint::new(); let fmt_opts: FormatOptions = Default::default(); let meta_opts: MetadataOptions = Default::default(); let mut format = symphonia::default::get_probe() .probe(&hint, mss, fmt_opts, meta_opts)?; let track = format.default_track(TrackType::Audio).unwrap(); let dec_opts: AudioDecoderOptions = Default::default(); let mut decoder = symphonia::default::get_codecs() .make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &dec_opts, )?; let track_id = track.id; let mut all_samples: Vec = Vec::new(); let mut interleaved_buffer: Vec = Vec::new(); while let Ok(Some(packet)) = format.next_packet() { if packet.track_id() != track_id { continue; } if let Ok(audio_buf) = decoder.decode(&packet) { // Method 1: Copy to interleaved Vec with automatic conversion audio_buf.copy_to_vec_interleaved(&mut interleaved_buffer); all_samples.extend_from_slice(&interleaved_buffer); // Method 2: Copy to a pre-sized slice let num_samples = audio_buf.samples_interleaved(); let mut samples = vec![0.0f32; num_samples]; audio_buf.copy_to_slice_interleaved(&mut samples); // Method 3: Copy to planar format (separate channel buffers) let mut planar: Vec> = Vec::new(); audio_buf.copy_to_vecs_planar(&mut planar); // Method 4: Copy as raw bytes (for audio output) let mut bytes: Vec = Vec::new(); audio_buf.copy_bytes_to_vec_interleaved(&mut bytes); // Method 5: Copy as i16 samples (common for audio APIs) let mut i16_samples: Vec = Vec::new(); audio_buf.copy_to_vec_interleaved(&mut i16_samples); } } Ok(all_samples) } ``` ## Audio Trait - Working with Typed Audio Buffers The Audio trait provides type-safe access to decoded audio samples when you need direct access to the underlying sample data. This is useful for DSP processing, analysis, or when you need maximum performance by avoiding copies. ```rust use symphonia::core::audio::{Audio, AudioMut, Position, GenericAudioBufferRef}; use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::formats::{FormatOptions, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use std::fs::File; fn process_audio_directly(path: &str) -> Result<(), Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let hint = Hint::new(); let mut format = symphonia::default::get_probe() .probe(&hint, mss, Default::default(), Default::default())?; let track = format.default_track(TrackType::Audio).unwrap(); let mut decoder = symphonia::default::get_codecs() .make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &Default::default(), )?; let track_id = track.id; while let Ok(Some(packet)) = format.next_packet() { if packet.track_id() != track_id { continue; } if let Ok(audio_buf) = decoder.decode(&packet) { // Match on the sample format to get typed access match audio_buf { GenericAudioBufferRef::F32(buf) => { // Get audio specification let spec = buf.spec(); println!("Sample rate: {} Hz", spec.rate()); println!("Channels: {}", spec.channels().count()); println!("Frames: {}", buf.frames()); // Iterate over all planes (channels) for (i, plane) in buf.iter_planes().enumerate() { let sum: f32 = plane.iter().map(|s| s.abs()).sum(); println!("Channel {}: avg amplitude = {}", i, sum / plane.len() as f32); } // Access specific channel by position if let Some(left) = buf.plane_by_position(Position::FRONT_LEFT) { println!("Left channel samples: {}", left.len()); } if let Some(right) = buf.plane_by_position(Position::FRONT_RIGHT) { println!("Right channel samples: {}", right.len()); } // Get a pair of channels for stereo processing if let Some((left, right)) = buf.plane_pair_by_position( Position::FRONT_LEFT, Position::FRONT_RIGHT ) { // Calculate stereo correlation let correlation: f32 = left.iter() .zip(right.iter()) .map(|(l, r)| l * r) .sum(); println!("Stereo correlation: {}", correlation); } // Iterate over interleaved samples for sample in buf.iter_interleaved().take(10) { print!("{:.4} ", sample); } println!(); } GenericAudioBufferRef::S16(buf) => { println!("16-bit signed integer audio: {} frames", buf.frames()); } GenericAudioBufferRef::S32(buf) => { println!("32-bit signed integer audio: {} frames", buf.frames()); } _ => { println!("Other sample format"); } } } } Ok(()) } ``` ## Metadata - Reading Tags and Album Art Symphonia automatically reads metadata while probing and during playback. Metadata is organized into revisions that can be queried from the FormatReader. Each revision contains tags (key-value pairs) and optionally visual data like album art. ```rust use symphonia::core::formats::{FormatOptions, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::{MetadataOptions, StandardTagKey, Tag, Visual, Value}; use std::fs::File; use std::io::Write; fn read_metadata(path: &str) -> Result<(), Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let hint = Hint::new(); let fmt_opts: FormatOptions = Default::default(); let meta_opts: MetadataOptions = Default::default(); let mut format = symphonia::default::get_probe() .probe(&hint, mss, fmt_opts, meta_opts)?; // Get current metadata revision if let Some(metadata) = format.metadata().current() { // Iterate over all tags println!("=== Metadata Tags ==="); for tag in metadata.tags() { // Check for standard tags first if let Some(std_key) = tag.std_key { match std_key { StandardTagKey::TrackTitle => println!("Title: {}", tag.value), StandardTagKey::Artist => println!("Artist: {}", tag.value), StandardTagKey::Album => println!("Album: {}", tag.value), StandardTagKey::TrackNumber => println!("Track: {}", tag.value), StandardTagKey::Date => println!("Date: {}", tag.value), StandardTagKey::Genre => println!("Genre: {}", tag.value), StandardTagKey::Composer => println!("Composer: {}", tag.value), StandardTagKey::Comment => println!("Comment: {}", tag.value), _ => println!("{:?}: {}", std_key, tag.value), } } else { // Non-standard tag, use the raw key println!("{}: {}", tag.key, tag.value); } } // Handle visuals (album art, etc.) println!("\n=== Visuals ==="); for (i, visual) in metadata.visuals().iter().enumerate() { println!("Visual {}: {} ({} bytes)", i, visual.media_type, visual.data.len() ); // Determine file extension from media type let ext = match visual.media_type.as_str() { "image/jpeg" => "jpg", "image/png" => "png", "image/gif" => "gif", _ => "bin", }; // Save the visual to a file let filename = format!("cover_{}.{}", i, ext); let mut file = File::create(&filename)?; file.write_all(&visual.data)?; println!(" Saved to: {}", filename); } } // Process metadata updates during playback while let Ok(Some(_packet)) = format.next_packet() { // Check for new metadata (streaming scenarios) while !format.metadata().is_latest() { format.metadata().pop(); if let Some(rev) = format.metadata().current() { println!("New metadata revision with {} tags", rev.tags().len()); } } } Ok(()) } ``` ## CodecRegistry - Managing Codecs The CodecRegistry is used to register and instantiate audio decoders. Symphonia provides a default registry with all enabled codecs, but you can create custom registries with specific codec configurations. ```rust use symphonia::core::codecs::registry::CodecRegistry; use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::formats::{FormatOptions, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use std::fs::File; // Use the default codec registry (recommended) fn decode_with_default_codecs(path: &str) -> Result<(), Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let mut format = symphonia::default::get_probe() .probe(&Hint::new(), mss, Default::default(), Default::default())?; let track = format.default_track(TrackType::Audio).unwrap(); // Use the default codec registry let decoder = symphonia::default::get_codecs() .make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &AudioDecoderOptions::default(), )?; // Print codec information let codec_info = decoder.codec_info(); println!("Codec: {} ({})", codec_info.long_name, codec_info.short_name); Ok(()) } // Create a custom codec registry with specific codecs fn decode_with_custom_codecs(path: &str) -> Result<(), Box> { // Create a new empty registry let mut registry = CodecRegistry::new(); // Register only the codecs you need using the default registration symphonia::default::register_enabled_codecs(&mut registry); let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let mut format = symphonia::default::get_probe() .probe(&Hint::new(), mss, Default::default(), Default::default())?; let track = format.default_track(TrackType::Audio).unwrap(); // Use the custom registry let _decoder = registry.make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &AudioDecoderOptions::default(), )?; Ok(()) } ``` ## Seeking - Navigate Within Media Symphonia supports seeking within media files using either time-based or timestamp-based positioning. Accurate seeking positions before the requested time, while coarse seeking may be faster but less precise. ```rust use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::formats::{FormatOptions, SeekMode, SeekTo, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use symphonia::core::units::{Time, Timestamp}; use std::fs::File; fn seek_in_media(path: &str) -> Result<(), Box> { let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); let mut format = symphonia::default::get_probe() .probe(&Hint::new(), mss, Default::default(), Default::default())?; let track = format.default_track(TrackType::Audio).unwrap(); let track_id = track.id; let mut decoder = symphonia::default::get_codecs() .make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &AudioDecoderOptions::default(), )?; // Seek by time (30.5 seconds) let seeked = format.seek( SeekMode::Accurate, // Sample-accurate seeking SeekTo::Time { time: Time::new(30, 0.5), // 30 seconds + 0.5 fractional seconds track_id: Some(track_id), }, )?; println!("Requested: 30.5s, Actual timestamp: {}", seeked.actual_ts); // Reset the decoder after seeking (required!) decoder.reset(); // Seek by timestamp (in track's timebase units) let seeked = format.seek( SeekMode::Coarse, // Faster but less precise SeekTo::TimeStamp { ts: Timestamp::new(44100 * 60), // 1 minute at 44.1kHz track_id: track_id, }, )?; println!("Seeked to timestamp: {} (requested: {})", seeked.actual_ts, seeked.required_ts); // Reset decoder again decoder.reset(); // Continue decoding from the new position while let Ok(Some(packet)) = format.next_packet() { if packet.track_id() != track_id { continue; } if let Ok(audio_buf) = decoder.decode(&packet) { println!("Decoded {} frames after seek", audio_buf.frames()); break; } } Ok(()) } ``` ## Complete Playback Example This comprehensive example demonstrates the full audio playback pipeline: probing, track selection, decoder creation, and the decode loop with metadata handling and error recovery. ```rust use symphonia::core::audio::sample::Sample; use symphonia::core::codecs::audio::AudioDecoderOptions; use symphonia::core::errors::Error; use symphonia::core::formats::{FormatOptions, TrackType}; use symphonia::core::formats::probe::Hint; use symphonia::core::io::MediaSourceStream; use symphonia::core::meta::MetadataOptions; use std::fs::File; fn play_audio(path: &str) -> Result<(), Box> { // Open the media file let file = File::open(path)?; let mss = MediaSourceStream::new(Box::new(file), Default::default()); // Create a hint from the file extension let mut hint = Hint::new(); if let Some(ext) = std::path::Path::new(path).extension() { hint.with_extension(ext.to_str().unwrap_or("")); } // Configure options let fmt_opts = FormatOptions::default(); let meta_opts = MetadataOptions::default(); let dec_opts = AudioDecoderOptions::default() .gapless(true) // Enable gapless playback .verify(false); // Disable verification for performance // Probe and get format reader let mut format = symphonia::default::get_probe() .probe(&hint, mss, fmt_opts, meta_opts) .expect("Failed to probe media"); // Print initial metadata if let Some(metadata) = format.metadata().current() { for tag in metadata.tags() { println!("{}: {}", tag.key, tag.value); } } // Select the first audio track let track = format .default_track(TrackType::Audio) .expect("No audio track found"); // Get track info let track_id = track.id; println!("\nTrack ID: {}", track_id); if let Some(ref params) = track.codec_params { if let Some(audio) = params.audio() { println!("Sample Rate: {:?}", audio.sample_rate); println!("Channels: {:?}", audio.channels); } } // Create the decoder let mut decoder = symphonia::default::get_codecs() .make_audio_decoder( track.codec_params.as_ref().unwrap().audio().unwrap(), &dec_opts, ) .expect("Failed to create decoder"); println!("Codec: {}", decoder.codec_info().long_name); // Prepare output buffer let mut samples: Vec = Vec::new(); let mut total_frames = 0u64; // Main decode loop loop { // Get the next packet let packet = match format.next_packet() { Ok(Some(packet)) => packet, Ok(None) => { println!("\nEnd of stream reached"); break; } Err(Error::ResetRequired) => { // Track list changed, reset decoder and continue println!("Reset required - reinitializing..."); decoder.reset(); continue; } Err(e) => { println!("Error reading packet: {}", e); break; } }; // Process metadata updates while !format.metadata().is_latest() { format.metadata().pop(); if let Some(rev) = format.metadata().current() { println!("Metadata update: {} tags", rev.tags().len()); } } // Skip packets from other tracks if packet.track_id() != track_id { continue; } // Decode the packet match decoder.decode(&packet) { Ok(audio_buf) => { total_frames += audio_buf.frames() as u64; // Copy decoded samples to our buffer samples.resize(audio_buf.samples_interleaved(), f32::MID); audio_buf.copy_to_slice_interleaved(&mut samples); // Here you would send samples to audio output // audio_output.write(&samples); print!("\rDecoded {} frames", total_frames); } Err(Error::IoError(e)) => { println!("\nIO error during decode: {}", e); continue; } Err(Error::DecodeError(e)) => { println!("\nDecode error (skipping packet): {}", e); continue; } Err(e) => { println!("\nUnrecoverable error: {}", e); break; } } } // Finalize decoder let result = decoder.finalize(); if let Some(verified) = result.verify_ok { println!("\nVerification: {}", if verified { "OK" } else { "FAILED" }); } println!("\nTotal frames decoded: {}", total_frames); Ok(()) } ``` ## Summary Symphonia is ideal for applications requiring high-quality, cross-platform audio decoding in pure Rust. Its key use cases include media players, audio processing tools, streaming applications, and any software needing to decode common audio formats without external C dependencies. The library excels in scenarios requiring gapless playback, efficient seeking, and comprehensive metadata extraction. Integration typically follows a pipeline pattern: create a MediaSourceStream from your audio source, probe it to detect the format and get a FormatReader, select an audio track, create the appropriate decoder, then loop reading packets and decoding them into audio buffers. The decoded samples can be output directly to audio hardware, processed for analysis, or converted to other formats. The clear separation between demuxing (FormatReader) and decoding (AudioDecoder) allows flexible composition, while the generic audio buffer interface enables sample format conversion without manual handling of each type.