### Extract RawNet3 Embeddings with ESPnet-SPK Source: https://github.com/jungjee/rawnet/blob/master/ReadMe.md Use this snippet to extract speaker embeddings using the pre-trained RawNet3 model within the ESPnet-SPK framework. Ensure ESPnet is installed. Replace `np.zeros` with your raw waveform data. ```python import numpy as np from espnet2.bin.spk_inference import Speech2Embedding speech2spk_embed = Speech2Embedding.from_pretrained(model_tag="espnet/voxcelebs12_rawnet3") embedding = speech2spk_embed(np.zeros(16500)) ``` -------------------------------- ### Speaker Embedding Extraction Source: https://context7.com/jungjee/rawnet/llms.txt Provides a function `extract_speaker_embd` to read a WAV file, segment it, and extract raw speaker embeddings, which can then be averaged to get a single utterance embedding. ```APIDOC ## Speaker Embedding Extraction — `extract_speaker_embd` `extract_speaker_embd` reads a mono 16 kHz WAV file, segments it into `n_segments` overlapping windows of `n_samples` samples each (padding short utterances by wrapping), stacks them into a batch, and returns the raw embedding tensor of shape `(n_segments, nOut)`. The caller averages over segments to obtain a single utterance embedding. ```python import numpy as np import torch import soundfile as sf from inference import extract_speaker_embd # Load model (see RawNet3 instantiation example above) # model.eval() must be called before this function # Single-utterance embedding extraction # Audio requirements: mono, 16 kHz, 16-bit PCM embedding_segments = extract_speaker_embd( model, fn="path/to/speaker_utterance.wav", n_samples=48000, # 3-second windows n_segments=10, # number of segments to extract gpu=torch.cuda.is_available(), ) # embedding_segments shape: (10, 256) # Average across segments for a single embedding vector utterance_embedding = embedding_segments.mean(0) print(utterance_embedding.shape) # torch.Size([256]) ``` ``` -------------------------------- ### Instantiate RawNet3 Model and Load Weights Source: https://context7.com/jungjee/rawnet/llms.txt Instantiates the RawNet3 model with a published configuration and loads pre-trained weights. Ensure the model is set to evaluation mode using `model.eval()` before inference. The input audio shape for the forward pass is (batch_size, num_samples). ```python import torch from models.RawNet3 import RawNet3 from models.RawNetBasicBlock import Bottle2neck # Instantiate the model with the published configuration model = RawNet3( Bottle2neck, model_scale=8, context=True, # use mean+std context in attention summed=True, # residual summation between Bottle2neck layers encoder_type="ECA", # or "ASP" for Attentive Statistics Pooling nOut=256, # embedding dimensionality out_bn=False, # no batch-norm on final embedding sinc_stride=10, # stride of the sinc filterbank log_sinc=True, # apply log to sinc output norm_sinc="mean", # subtract per-frame mean after log grad_mult=1, ) # Load published pre-trained weights (downloaded via git submodule) checkpoint = torch.load( "./models/weights/model.pt", map_location=lambda storage, loc: storage, ) model.load_state_dict(checkpoint["model"]) model.eval() # Forward pass: input shape is (batch_size, num_samples) # 3-second audio at 16 kHz = 48000 samples dummy_audio = torch.randn(4, 48000) # batch of 4 utterances with torch.no_grad(): embeddings = model(dummy_audio) # shape: (4, 256) print(embeddings.shape) # torch.Size([4, 256]) ``` -------------------------------- ### Download RawNet3 Model Weights Source: https://github.com/jungjee/rawnet/blob/master/python/RawNet3/README.md Use this command to initialize and download the model weights when the repository is added as a submodule. ```bash git submodule update --init --recursive ``` -------------------------------- ### RawNet3 Model Instantiation and Usage Source: https://context7.com/jungjee/rawnet/llms.txt Demonstrates how to instantiate the RawNet3 model, load pre-trained weights, and perform a forward pass to obtain speaker embeddings. ```APIDOC ## RawNet3 Model Architecture — `RawNet3` class The `RawNet3` class is a PyTorch `nn.Module` implementing a Res2Net-inspired speaker encoder. It uses a parametric sinc filterbank (`ParamSincFB`) as the learnable front-end, three `Bottle2neck` residual blocks with dilated convolutions and α-Feature Map Scaling (AFMS), and an Enhanced Channel Attention (ECA) or Attentive Statistics Pooling (ASP) back-end to produce a fixed-size speaker embedding. ```python import torch from models.RawNet3 import RawNet3 from models.RawNetBasicBlock import Bottle2neck # Instantiate the model with the published configuration model = RawNet3( Bottle2neck, model_scale=8, context=True, # use mean+std context in attention summed=True, # residual summation between Bottle2neck layers encoder_type="ECA", # or "ASP" for Attentive Statistics Pooling nOut=256, # embedding dimensionality out_bn=False, # no batch-norm on final embedding sinc_stride=10, # stride of the sinc filterbank log_sinc=True, # apply log to sinc output norm_sinc="mean", # subtract per-frame mean after log grad_mult=1, ) # Load published pre-trained weights (downloaded via git submodule) checkpoint = torch.load( "./models/weights/model.pt", map_location=lambda storage, loc: storage, ) model.load_state_dict(checkpoint["model"]) model.eval() # Forward pass: input shape is (batch_size, num_samples) # 3-second audio at 16 kHz = 48000 samples dummy_audio = torch.randn(4, 48000) # batch of 4 utterances with torch.no_grad(): embeddings = model(dummy_audio) # shape: (4, 256) print(embeddings.shape) # torch.Size([4, 256]) ``` ``` -------------------------------- ### Initialize Evaluation Utilities Source: https://context7.com/jungjee/rawnet/llms.txt Imports standard NIST speaker recognition evaluation metrics from `utils.py`. These include functions for tuning thresholds, computing error rates (FNR/FPR), and calculating the minimum detection cost function (minDCF). ```python from utils import tuneThresholdfromScore, ComputeErrorRates, ComputeMinDcf import numpy as np # Simulated trial scores (cosine similarity) and binary labels (1=same, 0=different) np.random.seed(42) n_trials = 1000 labels = np.random.randint(0, 2, n_trials).tolist() ``` -------------------------------- ### Benchmark RawNet3 on Vox1-O Protocol Source: https://github.com/jungjee/rawnet/blob/master/python/RawNet3/README.md Execute the benchmark script for the Vox1-O evaluation protocol. Ensure the DB_dir points to the root of your VoxCeleb1 dataset. ```bash python inference.py --vox1_o_benchmark --DB_dir ``` -------------------------------- ### Create RawNet3 Model using MainModel Factory Source: https://context7.com/jungjee/rawnet/llms.txt Uses the `MainModel` factory function to create a `RawNet3` instance with canonical published hyperparameters. This factory is compatible with training frameworks expecting a `MainModel(**kwargs)` entry point. The factory defaults to using `Bottle2neck`, `model_scale=8`, `context=True`, and `summed=True`. ```python from models.RawNet3 import MainModel # Create the model with the published hyperparameters model = MainModel( encoder_type="ECA", nOut=256, out_bn=False, sinc_stride=10, log_sinc=True, norm_sinc="mean", grad_mult=1, ) # The factory always uses Bottle2neck, model_scale=8, context=True, summed=True print(type(model)) # print(sum(p.numel() for p in model.parameters())) # ~14 million parameters ``` -------------------------------- ### Run Inference Script for Speaker Embeddings Source: https://context7.com/jungjee/rawnet/llms.txt Use the `inference.py` script for extracting single speaker embeddings or running a full Vox1-O benchmark evaluation. Ensure submodules are updated and provide correct paths to audio files or the VoxCeleb1 dataset. ```bash # Download pre-trained weights via submodule git submodule update --init --recursive # --- Single utterance embedding extraction --- python inference.py \ --inference_utterance \ --input /path/to/utterance.wav \ --out_dir ./speaker_embedding.npy \ --n_segments 10 # Output: saves speaker_embedding.npy (shape: 256,) ``` ```bash # --- Vox1-O benchmark evaluation --- # DB_dir must point to the root of VoxCeleb1 containing 1,251 speaker folders python inference.py \ --vox1_o_benchmark \ --DB_dir /path/to/VoxCeleb1 \ --n_segments 10 # Expected output: # RawNet3 initialised & weights loaded! # Cuda available, conducting inference on GPU # 100%|████████████████| 4874/4874 [...] # Vox1-O benchmark Finished. EER: 0.8932, minDCF:0.06690 ``` -------------------------------- ### Single Utterance Inference with RawNet3 Source: https://github.com/jungjee/rawnet/blob/master/python/RawNet3/README.md Run inference on a single audio file to extract speaker embeddings. Optionally specify an output directory for the embedding file. ```bash python inference.py --inference_utterance --input {YOUR_INPUT_FILE} ``` -------------------------------- ### MainModel Factory Function Source: https://context7.com/jungjee/rawnet/llms.txt Utilizes the `MainModel` factory function to create a `RawNet3` instance with canonical published hyperparameters, compatible with training frameworks. ```APIDOC ## `MainModel` Factory Function `MainModel` is a convenience factory that creates a `RawNet3` instance using the canonical published configuration. It accepts all `RawNet3` keyword arguments and is designed to be compatible with training frameworks such as `voxceleb_trainer` that expect a standardised `MainModel(**kwargs)` entry point. ```python from models.RawNet3 import MainModel # Create the model with the published hyperparameters model = MainModel( encoder_type="ECA", nOut=256, out_bn=False, sinc_stride=10, log_sinc=True, norm_sinc="mean", grad_mult=1, ) # The factory always uses Bottle2neck, model_scale=8, context=True, summed=True print(type(model)) # print(sum(p.numel() for p in model.parameters())) # ~14 million parameters ``` ``` -------------------------------- ### Instantiate Bottle2neck Residual Block Source: https://context7.com/jungjee/rawnet/llms.txt Demonstrates the instantiation and usage of the `Bottle2neck` residual block, a core component of RawNet3 that adapts Res2Net's multi-scale idea for 1D convolutions. It includes an α-Feature Map Scaling (AFMS) operation. ```python import torch from models.RawNetBasicBlock import Bottle2neck # A single Bottle2neck block as used in RawNet3 layer1: # inplanes=256, planes=1024, kernel_size=3, dilation=2, scale=8, pool=5 block = Bottle2neck( inplanes=256, planes=1024, kernel_size=3, dilation=2, scale=8, # splits feature map into 8 sub-streams pool=5, # MaxPool1d(5) applied after residual + AFMS ) x = torch.randn(4, 256, 300) # (batch, channels, time_frames) out = block(x) print(out.shape) # torch.Size([4, 1024, 60]) — pooled by factor 5 ``` -------------------------------- ### Process VoxCeleb Trial List and Extract Embeddings Source: https://context7.com/jungjee/rawnet/llms.txt Loads trial lists from a specified format, extracts speaker embeddings for unique audio files using a pre-trained model, and scores trial pairs. This is used for the VoxCeleb1 evaluation protocol. ```python # trials/cleaned_test_list.txt format: #