### PyTorch nn.Module Example Source: https://diart.readthedocs.io/en/latest/autoapi/diart/models/index An example demonstrating how to define a neural network model by subclassing torch.nn.Module. It includes initialization of convolutional layers and a forward method for processing input data. ```python import torch.nn as nn import torch.nn.functional as F class Model(nn.Module): def __init__(self) -> None: super().__init__() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) ``` -------------------------------- ### Run Diart Server and Client from Command Line Source: https://diart.readthedocs.io/en/latest/index This snippet shows how to start the Diart server and connect a client using command-line arguments. Ensure the client uses the same 'step' and 'sample_rate' as the server. The `--host` and `--port` parameters are used for network communication. ```bash diart.serve --host 0.0.0.0 --port 7007 diart.client microphone --host --port 7007 ``` -------------------------------- ### Install diart using Conda Source: https://diart.readthedocs.io/en/latest/index Installs diart by creating and activating a pre-configured conda environment using an environment.yml file. This method ensures all necessary dependencies are met. ```bash conda env create -f diart/environment.yml conda activate diart ``` -------------------------------- ### DelayedAggregation Example in Python Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/aggregation/index An example demonstrating the initialization and usage of the DelayedAggregation class with specific parameters for step, latency, and strategy. It includes setup for buffer dimensions and time characteristics. ```python duration = 5 frames = 500 step = 0.5 speakers = 2 start_time = 10 resolution = duration / frames dagg = DelayedAggregation(step=step, latency=2, strategy="mean") buffers = [ SlidingWindowFeature( np.random.rand(frames, speakers), SlidingWindow(start=(i + start_time) * step, duration=resolution, step=resolution) ) ] ``` -------------------------------- ### Benchmark Speaker Diarization Performance from Command Line Source: https://diart.readthedocs.io/en/latest/index This command-line example shows how to run Diart's benchmarking tool to evaluate performance on a dataset. It specifies input directories, hyper-parameters like tau, rho, and delta, and the segmentation model to use. The `--segmentation` argument points to a pre-trained model. ```bash diart.benchmark /wav/dir --reference /rttm/dir --tau-active=0.555 --rho-update=0.422 --delta-new=1.517 --segmentation pyannote/segmentation@Interspeech2021 ``` -------------------------------- ### DelayedAggregation Initialization and Usage (Python) Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Demonstrates how to initialize and use the DelayedAggregation class for aggregating overlapping windows. It shows the setup with parameters like step, latency, and strategy, and how to process input buffers to obtain aggregated features. ```python >>> duration = 5 >>> frames = 500 >>> step = 0.5 >>> speakers = 2 >>> start_time = 10 >>> resolution = duration / frames >>> dagg = DelayedAggregation(step=step, latency=2, strategy="mean") >>> buffers = [ >>> SlidingWindowFeature( >>> np.random.rand(frames, speakers), >>> SlidingWindow(start=(i + start_time) * step, duration=resolution, step=resolution) >>> ) >>> for i in range(dagg.num_overlapping_windows) >>> ] >>> dagg.num_overlapping_windows ... 4 >>> dagg(buffers).data.shape ... (51, 2) # Rounding errors are possible when cropping the buffers ``` -------------------------------- ### Install diart using Pip Source: https://diart.readthedocs.io/en/latest/index Installs the diart Python package using pip. This is a straightforward installation method for users who prefer using pip. ```bash pip install diart ``` -------------------------------- ### Example Usage of dagg in a loop Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/aggregation/index This snippet demonstrates iterating through overlapping windows using dagg.num_overlapping_windows. It also shows the output shape of dagg applied to buffers, noting potential rounding errors during cropping. ```python for i in range(dagg.num_overlapping_windows) dagg.num_overlapping_windows ... 4 dagg(buffers).data.shape ... (51, 2) # Rounding errors are possible when cropping the buffers ``` -------------------------------- ### Create a Diart Server in Python using WebSocketAudioSource Source: https://diart.readthedocs.io/en/latest/index This Python code demonstrates how to set up a Diart speaker diarization pipeline with a WebSocket audio source. It initializes the pipeline and source, then starts streaming inference, sending RTTM annotations back to the source. This is useful for customized Diart solutions. ```python from diart import SpeakerDiarization from diart.sources import WebSocketAudioSource from diart.inference import StreamingInference pipeline = SpeakerDiarization() source = WebSocketAudioSource(pipeline.config.sample_rate, "localhost", 7007) inference = StreamingInference(pipeline, source) inference.attach_hooks(lambda ann_wav: source.send(ann_wav[0].to_rttm())) prediction = inference() ``` -------------------------------- ### diart.operators.AudioBufferState Source: https://diart.readthedocs.io/en/latest/autoapi/diart/operators/index Represents the state of an audio buffer, including chunk data, buffer data, start time, and change status. Includes static methods for initialization and sample checking. ```APIDOC ## Class: AudioBufferState ### Description Represents the state of an audio buffer within the diart processing pipeline. It holds the current audio chunk, the historical buffer, the start time of the buffer, and a flag indicating if the buffer has changed. ### Methods #### `__init__()` Initializes a new instance of AudioBufferState. #### `chunk` - **type**: numpy.ndarray | None - **description**: The current audio chunk being processed. #### `buffer` - **type**: numpy.ndarray | None - **description**: The historical audio buffer. #### `start_time` - **type**: float - **description**: The timestamp when the current buffer segment started. #### `changed` - **type**: bool - **description**: A boolean flag indicating whether the buffer has been modified. #### `initial()` - **description**: Static method to create an initial, empty AudioBufferState. - **returns**: AudioBufferState #### `has_samples(num_samples)` - **description**: Static method to check if the buffer contains a minimum number of samples. - **parameters**: - **num_samples** (int): The minimum number of samples required. - **returns**: bool #### `to_sliding_window(sample_rate)` - **description**: Converts the buffer content to a SlidingWindowFeature. - **parameters**: - **sample_rate** (int): The sample rate of the audio data. - **returns**: pyannote.core.SlidingWindowFeature ``` -------------------------------- ### Load Custom Segmentation and Embedding Models with diart Source: https://diart.readthedocs.io/en/latest/index Demonstrates how to integrate third-party segmentation and embedding models into a diart pipeline by providing custom loader functions. It shows the setup for both segmentation and embedding models, followed by the configuration of the SpeakerDiarization object. ```python from diart import SpeakerDiarization, SpeakerDiarizationConfig from diart.models import EmbeddingModel, SegmentationModel def segmentation_loader(): # It should take a waveform and return a segmentation tensor return load_pretrained_model("my_model.ckpt") def embedding_loader(): # It should take (waveform, weights) and return per-speaker embeddings return load_pretrained_model("my_other_model.ckpt") segmentation = SegmentationModel(segmentation_loader) embedding = EmbeddingModel(embedding_loader) config = SpeakerDiarizationConfig( segmentation=segmentation, embedding=embedding, ) pipeline = SpeakerDiarization(config) ``` -------------------------------- ### Set up Optuna Study for Distributed Tuning Source: https://diart.readthedocs.io/en/latest/index Provides commands to set up a distributed hyper-parameter tuning study using Optuna with a MySQL database. It includes creating the database and then creating an Optuna study, specifying the study name and storage connection string. ```bash mysql -u root -e "CREATE DATABASE IF NOT EXISTS example" optuna create-study --study-name "example" --storage "mysql://root@localhost/example" ``` -------------------------------- ### Run diart Tuning with Distributed Storage Source: https://diart.readthedocs.io/en/latest/index Demonstrates how to run the diart hyper-parameter tuning process when distributed across multiple processes using a shared Optuna study. This command specifies the audio and reference directories, along with the storage connection string for the distributed study. ```bash diart.tune /wav/dir --reference /rttm/dir --storage mysql://root@localhost/example ``` -------------------------------- ### Tune diart Pipeline Hyper-parameters from Command Line Source: https://diart.readthedocs.io/en/latest/index Shows how to initiate hyper-parameter tuning for a diart pipeline directly from the command line. This command requires paths to audio directories, reference RTTM files, and an output directory for results. Use `diart.tune -h` for additional options. ```bash diart.tune /wav/dir --reference /rttm/dir --output /output/dir ``` -------------------------------- ### Initialize Distributed diart Optimizer with Optuna Study Source: https://diart.readthedocs.io/en/latest/index Shows how to initialize and run the diart hyper-parameter optimizer in Python when using a distributed Optuna study. It loads an existing study from a database and passes it to the Optimizer, allowing for parallelized tuning. The results are written to an SQLite database. ```python from diart.optim import Optimizer from optuna.samplers import TPESampler import optuna db = "mysql://root@localhost/example" study = optuna.load_study("example", db, TPESampler()) optimizer = Optimizer("/wav/dir", "/rttm/dir", study) optimizer(num_iter=100) ``` -------------------------------- ### Tune diart Pipeline Hyper-parameters from Python Source: https://diart.readthedocs.io/en/latest/index Illustrates how to perform hyper-parameter tuning for diart pipelines using Python. An Optimizer object is instantiated with paths to audio, reference RTTM files, and an output directory. The tuning process is then executed by calling the optimizer instance. ```python from diart.optim import Optimizer optimizer = Optimizer("/wav/dir", "/rttm/dir", "/output/dir") optimizer(num_iter=100) ``` -------------------------------- ### diart.utils Helper Functions Source: https://diart.readthedocs.io/en/latest/autoapi/diart/utils/index This section covers various utility functions for parsing arguments, encoding/decoding audio, calculating padding, and visualizing data. ```APIDOC ## Functions in diart.utils ### `parse_hf_token_arg(hf_token)` #### Description Parses the Hugging Face token argument. #### Parameters - **hf_token** (Union[bool, Text]) - The Hugging Face token argument. #### Returns Union[bool, Text] - The parsed token. ### `encode_audio(waveform)` #### Description Encodes an audio waveform into a text format. #### Parameters - **waveform** (numpy.ndarray) - The input audio waveform. #### Returns Text - The encoded audio data. ### `decode_audio(data)` #### Description Decodes text data into an audio waveform. #### Parameters - **data** (Text) - The encoded audio data. #### Returns numpy.ndarray - The decoded audio waveform. ### `get_padding_left(stream_duration, chunk_duration)` #### Description Calculates the left padding needed for a stream. #### Parameters - **stream_duration** (float) - The total duration of the stream. - **chunk_duration** (float) - The duration of a chunk. #### Returns float - The calculated left padding. ### `repeat_label(label)` #### Description Repeats a given label. #### Parameters - **label** (Text) - The label to repeat. ### `get_pipeline_class(class_name)` #### Description Retrieves a pipeline class by its name. #### Parameters - **class_name** (Text) - The name of the pipeline class. #### Returns type - The pipeline class. ### `get_padding_right(latency, step)` #### Description Calculates the right padding needed based on latency and step. #### Parameters - **latency** (float) - The latency value. - **step** (float) - The step value. #### Returns float - The calculated right padding. ### `visualize_feature(duration=None)` #### Description Visualizes audio features. #### Parameters - **duration** (Optional[float]) - The duration to visualize. Defaults to `None`. ### `visualize_annotation(duration=None)` #### Description Visualizes audio annotations. #### Parameters - **duration** (Optional[float]) - The duration to visualize. Defaults to `None`. ``` -------------------------------- ### OnlineSpeakerClustering Initialization (Python) Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Illustrates the initialization of the OnlineSpeakerClustering class, which handles incremental online clustering of speakers. Key parameters include thresholds for speaker activity, embedding updates, new speaker detection, and the distance metric. ```python _class _diart.blocks.OnlineSpeakerClustering(_tau_active_ , _rho_update_ , _delta_new_ , _metric ='cosine'_, _max_speakers =20_)# Implements constrained incremental online clustering of speakers and manages cluster centers. Parameters: * **tau_active** (_float_) – Threshold for detecting active speakers. This threshold is applied on the maximum value of per-speaker output activation of the local segmentation model. * **rho_update** (_float_) – Threshold for considering the extracted embedding when updating the centroid of the local speaker. The centroid to which a local speaker is mapped is only updated if the ratio of speech/chunk duration of a given local speaker is greater than this threshold. * **delta_new** (_float_) – Threshold on the distance between a speaker embedding and a centroid. If the distance between a local speaker and all centroids is larger than delta_new, then a new centroid is created for the current speaker. * **metric** (_str. Defaults to "cosine"._) – The distance metric to use. * **max_speakers** (_int_) – Maximum number of global speakers to track through a conversation. Defaults to 20. ``` -------------------------------- ### WebSocketAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source from a WebSocket connection. Inherits from AudioSource. ```APIDOC ## Class WebSocketAudioSource ### Description Represents an audio source from a WebSocket connection. Inherits from AudioSource. ### Methods - **`_on_message_received()`**: Callback function for handling incoming WebSocket messages. - **`read()`**: Reads a chunk of audio data from the WebSocket connection. - **`close()`**: Closes the WebSocket connection. - **`send()`**: Sends data over the WebSocket connection. ``` -------------------------------- ### Configuration Classes Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Configuration objects for pipeline and diarization settings. ```APIDOC ## SpeakerDiarizationConfig ### Description Configuration parameters for speaker diarization. ### Properties - **duration** (float) - The duration of audio chunks to process. - **step** (float) - The step size between consecutive chunks. - **latency** (float) - The latency of the processing pipeline. - **sample_rate** (int) - The sample rate of the audio. ## PipelineConfig ### Description Base configuration for the processing pipeline. ### Properties - **duration** (float) - The duration of audio chunks to process. - **step** (float) - The step size between consecutive chunks. - **latency** (float) - The latency of the processing pipeline. - **sample_rate** (int) - The sample rate of the audio. ### Methods - `get_file_padding() -> Optional[Tuple[float, float]]` Gets the padding applied to the beginning and end of audio files. ``` -------------------------------- ### Create SegmentationModel from Pyannote Source: https://diart.readthedocs.io/en/latest/autoapi/diart/models/index This function allows the creation of a diart SegmentationModel by wrapping a pre-trained pyannote.audio model. It handles the loading and conversion process, supporting Hugging Face token authentication for model access. ```python from diart.models import SegmentationModel import pyannote.audio # Assuming 'model' is a loaded pyannote.audio pipeline model # segmentation_model = SegmentationModel.from_pyannote(model, use_hf_token=True) ``` -------------------------------- ### MappingMatrixObjective Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/mapping/index Helper class that provides a standard way to create an ABC using inheritance for mapping matrix objectives. ```APIDOC ## Class: MappingMatrixObjective ### Description Helper class that provides a standard way to create an ABC using inheritance. ### Abstract Properties - **invalid_value** (float): Represents an invalid value for the objective. - **maximize** (bool): Indicates whether the objective is to maximize or minimize. - **best_possible_value** (float): The best possible value achievable for this objective. - **best_value_fn**: A callable function that returns the best value for the objective. ### Methods - **invalid_tensor(shape)**: - Parameters: - **shape** (Union[Tuple, int]): The shape of the tensor. - Returns: numpy.ndarray - A tensor filled with invalid values. - **optimal_assignments(matrix)**: - Parameters: - **matrix** (numpy.ndarray): The input cost matrix. - Returns: List[int] - A list of optimal assignments. - **mapped_indices(matrix, axis)**: - Parameters: - **matrix** (numpy.ndarray): The input mapping matrix. - **axis** (int): The axis along which to map indices. - Returns: List[int] - A list of mapped indices. ``` -------------------------------- ### SpeakerSegmentation Class Initialization and Pretrained Model Loading (Python) Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/segmentation/index Demonstrates how to initialize the SpeakerSegmentation class with a model and device, and how to load a pretrained model using from_pretrained. This class is used for speaker segmentation in audio. ```python from typing import Optional, Union import torch from diart.models.segmentation import SegmentationModel from diart.inference import TemporalFeatures class SpeakerSegmentation: def __init__(self, model: SegmentationModel, device: Optional[torch.device] = None): # ... implementation details ... pass @staticmethod def from_pretrained(model: SegmentationModel, use_hf_token: Union[str, bool, None] = True, device: Optional[torch.device] = None) -> 'SpeakerSegmentation': # ... implementation details ... return SpeakerSegmentation(model, device) ``` -------------------------------- ### FileAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source from a file. Inherits from AudioSource. ```APIDOC ## Class FileAudioSource ### Description Represents an audio source from a file. Inherits from AudioSource. ### Methods - **`duration`** (property): Returns the duration of the audio file in seconds. - **`read()`**: Reads a chunk of audio data from the file. - **`close()`**: Closes the file handle. ``` -------------------------------- ### OnlineSpeakerClustering Methods: get_next_center_position, init_centers, update (Python) Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Details essential methods of OnlineSpeakerClustering: `get_next_center_position` for finding an available center, `init_centers` for initializing the centroid matrix, and `update` for refining speaker centroids based on assignments and embeddings. ```python get_next_center_position()# Return type: Optional[int] init_centers(_dimension_)# Initializes the speaker centroid matrix Parameters: **dimension** (_int_) – Dimension of embeddings used for representing a speaker. update(_assignments_ , _embeddings_)# Updates the speaker centroids given a list of assignments and local speaker embeddings Parameters: * **assignments** (_Iterable_ _[__Tuple_ _[__int_ _,__int_ _]__]__)_ – An iterable of tuples with two elements having the first element as the source speaker and the second element as the target speaker. * **embeddings** (_np.ndarray_ _,__shape_ _(__local_speakers_ _,__embedding_dim_ _)_) – Matrix containing embeddings for all local speakers. ``` -------------------------------- ### TorchStreamAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source from a PyTorch stream. Inherits from AudioSource. ```APIDOC ## Class TorchStreamAudioSource ### Description Represents an audio source from a PyTorch stream. Inherits from AudioSource. ### Methods - **`read()`**: Reads a chunk of audio data from the PyTorch stream. - **`close()`**: Releases resources associated with the PyTorch stream. ``` -------------------------------- ### SpeakerEmbedding Initialization (Python) Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Shows how to initialize the SpeakerEmbedding class, which is responsible for generating speaker embeddings using a pre-trained model. It takes the model and an optional device as parameters. ```python _class _diart.blocks.SpeakerEmbedding(_model_ , _device =None_)# Parameters: * **model** (_diart.models.EmbeddingModel_) – * **device** (_Optional_ _[__torch.device_ _]_) – ``` -------------------------------- ### AudioLoader Class API Source: https://diart.readthedocs.io/en/latest/autoapi/diart/audio/index Provides functionality to load audio files and retrieve their duration. ```APIDOC ## `diart.audio.AudioLoader` ### Description Loads an audio file into a torch.Tensor and provides a static method to get the audio duration. ### Method `AudioLoader(sample_rate: int, mono: bool = True)` ### Parameters #### Constructor Parameters - **sample_rate** (int) - The target sample rate for the audio. - **mono** (bool) - If True, converts the audio to mono. Defaults to True. ### Methods #### `load(filepath)` ##### Description Loads an audio file into a torch.Tensor. ##### Parameters - **filepath** (FilePath) - Path to an audio file. ##### Returns - **waveform** (torch.Tensor) - The loaded audio waveform with shape (channels, samples). #### `get_duration(filepath)` ##### Description Gets the duration of an audio file in seconds. ##### Parameters - **filepath** (FilePath) - Path to an audio file. ##### Returns - **duration** (float) - The duration of the audio file in seconds. ### Request Example ```python # Example for loading an audio file from diart.audio import AudioLoader audio_loader = AudioLoader(sample_rate=16000) waveform = audio_loader.load("path/to/your/audio.wav") print(waveform.shape) # Example for getting audio duration duration = AudioLoader.get_duration("path/to/your/audio.wav") print(f"Audio duration: {duration} seconds") ``` ### Response #### Success Response (load) - **waveform** (torch.Tensor) - The loaded audio data. #### Success Response (get_duration) - **duration** (float) - The duration of the audio file in seconds. #### Response Example (load) ```json { "waveform_shape": "[channels, samples]" } ``` #### Response Example (get_duration) ```json { "duration": 123.45 } ``` ``` -------------------------------- ### SpeakerMapBuilder Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/mapping/index Provides static methods for building SpeakerMap objects. ```APIDOC ## Class: SpeakerMapBuilder ### Description Provides static methods for building SpeakerMap objects. ### Static Methods - **hard_map(shape, assignments, maximize)**: - Description: Creates a `SpeakerMap` object based on the given assignments. This is a “hard” map, meaning that the highest cost is put everywhere except on hard assignments. - Parameters: - **shape** (Tuple[int, int]): Shape of the mapping matrix. - **assignments** (Iterable[Tuple[int, int]]): An iterable of tuples with two elements representing source and target speaker assignments. - **maximize** (bool): Whether to use scores where higher is better (true) or lower is better (false). - Returns: SpeakerMap - The constructed SpeakerMap object. - **correlation(scores1, scores2)**: - Parameters: - **scores1** (numpy.ndarray): First array of scores. - **scores2** (numpy.ndarray): Second array of scores. - Returns: SpeakerMap - A SpeakerMap object representing the correlation. - **mse(scores1, scores2)**: - Parameters: - **scores1** (numpy.ndarray): First array of scores. - **scores2** (numpy.ndarray): Second array of scores. - Returns: SpeakerMap - A SpeakerMap object representing the Mean Squared Error. - **mae(scores1, scores2)**: - Parameters: - **scores1** (numpy.ndarray): First array of scores. - **scores2** (numpy.ndarray): Second array of scores. - Returns: SpeakerMap - A SpeakerMap object representing the Mean Absolute Error. - **dist(embeddings1, embeddings2, metric='cosine')**: - Parameters: - **embeddings1** (numpy.ndarray): First array of embeddings. - **embeddings2** (numpy.ndarray): Second array of embeddings. - **metric** (Text): The distance metric to use (default: 'cosine'). - Returns: SpeakerMap - A SpeakerMap object representing the distance. ``` -------------------------------- ### PipelineConfig Abstract Class in Python Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/base/index An abstract base class for pipeline configurations. It defines abstract properties for essential audio stream parameters like duration, step, latency, and sample rate. It also includes a method `get_file_padding` for calculating padding based on a filepath. ```python import abc from typing import Tuple # Assuming FilePath and SlidingWindowFeature are defined elsewhere # from diart.audio import FilePath # from diart.inference import SlidingWindowFeature class PipelineConfig(abc.ABC): """Configuration containing the required parameters to build and run a pipeline""" @property @abc.abstractmethod def duration(self) -> float: """The duration of an input audio chunk (in seconds)""" pass @property @abc.abstractmethod def step(self) -> float: """The step between two consecutive input audio chunks (in seconds)""" pass @property @abc.abstractmethod def latency(self) -> float: """The algorithmic latency of the pipeline (in seconds). At time t of the audio stream, the pipeline will output predictions for time t - latency.""" pass @property @abc.abstractmethod def sample_rate(self) -> int: """The sample rate of the input audio stream""" pass def get_file_padding(self, filepath) -> Tuple[float, float]: """Placeholder for file padding calculation.""" # Implementation details would go here pass ``` -------------------------------- ### Pipeline and Core Components Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Core pipeline orchestration and audio processing components. ```APIDOC ## Pipeline ### Description Represents the main processing pipeline for audio analysis. ### Properties - **config**: `PipelineConfig` - Configuration object for the pipeline. ### Methods - `get_config_class() -> Type[PipelineConfig]` Returns the configuration class used by this pipeline. - `suggest_metric(audio: np.ndarray, n_speakers: int) -> Tuple[str, float]` Suggests an appropriate metric for evaluating pipeline results. - `hyper_parameters() -> Dict[str, Any]` Returns the current hyperparameters of the pipeline. - `reset()` Resets the internal state of the pipeline. - `set_timestamp_shift(shift: float)` Sets the timestamp shift for processing. ### Method `__call__(audio: np.ndarray, batch_size: int = 32) -> np.ndarray` Processes the input audio through the pipeline. ``` ```APIDOC ## Binarize ### Description Binarizes audio signals, likely for segmentation or event detection. ### Method `__call__(audio: np.ndarray) -> np.ndarray` Applies binarization to the input audio signal. ``` ```APIDOC ## Resample ### Description Resamples audio signals to a target sample rate. ### Method `__call__(audio: np.ndarray, current_sample_rate: int, target_sample_rate: int) -> np.ndarray` Resamples the audio signal. ``` ```APIDOC ## AdjustVolume ### Description Adjusts the volume of audio signals. ### Methods - `get_volumes(audio: np.ndarray) -> np.ndarray` Calculates the volume levels of the audio. ### Method `__call__(audio: np.ndarray) -> np.ndarray` Applies volume adjustment to the audio signal. ``` -------------------------------- ### diart.models Module Overview Source: https://diart.readthedocs.io/en/latest/autoapi/diart/models/index Provides an overview of the classes available in the diart.models module for handling speech diarization models. ```APIDOC ## Module: diart.models ### Description This module contains various model classes used for speech diarization tasks in the diart library. It includes adapters, loaders, and abstract interfaces for different types of segmentation and embedding models. ### Classes * **`PowersetAdapter`**: Base class for all neural network modules. * **`PyannoteLoader`**: A class for loading models compatible with the Pyannote format. * **`ONNXLoader`**: A class for loading models in ONNX format. * **`ONNXModel`**: Represents a model loaded in ONNX format. * **`LazyModel`**: Helper class that provides a standard way to create an ABC (Abstract Base Class). * **`SegmentationModel`**: Minimal interface for a segmentation model. * **`EmbeddingModel`**: Minimal interface for an embedding model. ### Usage Instantiate and use these model classes according to their specific documentation for tasks like generating speaker embeddings, performing voice activity detection, or segmenting speech. ``` -------------------------------- ### WebSocketAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source receiving audio data over the WebSocket protocol. ```APIDOC ## WebSocketAudioSource ### Description Represents a source of audio coming from the network using the WebSocket protocol. ### Method N/A (Class) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Constructor Parameters - **sample_rate** (int) - Sample rate of the chunks emitted. - **host** (Text) - The host to run the websocket server. Defaults to 127.0.0.1. - **port** (int) - The port to run the websocket server. Defaults to 7007. - **key** (Text | Path | None) - Path to a key if using SSL. Defaults to no key. - **certificate** (Text | Path | None) - Path to a certificate if using SSL. Defaults to no certificate. ### Methods - **on_message_received(client, server, message)**: Callback for when a message is received. - **read()**: Starts running the websocket server and listening for audio chunks. - **close()**: Close the websocket server. - **send(message)**: Send a message through the current websocket. ``` -------------------------------- ### Create EmbeddingModel from Pyannote Source: https://diart.readthedocs.io/en/latest/autoapi/diart/models/index This function facilitates the creation of a diart EmbeddingModel by wrapping a pyannote.audio model. It manages the loading and integration of pyannote models for embedding extraction tasks, with support for Hugging Face token. ```python from diart.models import EmbeddingModel import pyannote.audio # Assuming 'model' is a loaded pyannote.audio pipeline model # embedding_model = EmbeddingModel.from_pyannote(model, use_hf_token=True) ``` -------------------------------- ### Aggregation Strategies Overview Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/aggregation/index This section outlines different aggregation strategies available in the diart.blocks.aggregation module. It lists classes like AggregationStrategy, HammingWeightedAverageStrategy, AverageStrategy, FirstOnlyStrategy, and DelayedAggregation, along with their key methods such as build(), __call__(), aggregate(), and _prepend(). ```text AggregationStrategy AggregationStrategy.build() AggregationStrategy.__call__() AggregationStrategy.aggregate() HammingWeightedAverageStrategy HammingWeightedAverageStrategy.aggregate() AverageStrategy AverageStrategy.aggregate() FirstOnlyStrategy FirstOnlyStrategy.aggregate() DelayedAggregation DelayedAggregation._prepend() DelayedAggregation.__call__() ``` -------------------------------- ### diart.operators Module Classes Source: https://diart.readthedocs.io/en/latest/autoapi/diart/operators/index This section details the classes available within the diart.operators module, used for managing the state of audio buffers and predictions during diarization. ```APIDOC ## Classes in `diart.operators` ### `AudioBufferState` Represents the state of an audio buffer. ### `PredictionWithAudio` Stores predictions along with their associated audio data. ### `OutputAccumulationState` Manages the state for accumulating output indefinitely. ``` -------------------------------- ### FileAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source specifically tied to a local file. ```APIDOC ## FileAudioSource ### Description Represents an audio source tied to a file. ### Method N/A (Class) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Constructor Parameters - **file** (FilePath) - Path to the file to stream. - **sample_rate** (int) - Sample rate of the chunks emitted. - **padding** (tuple[float, float]) - Left and right padding to add to the file (in seconds). Defaults to (0, 0). - **block_duration** (int) - Duration of each emitted chunk in seconds. Defaults to 0.5 seconds. ### Properties - **duration** (float | None) - The duration of the stream if known. Defaults to None (unknown duration). ### Methods - **read()**: Send each chunk of samples through the stream. - **close()**: Stop reading the source and close all open streams. ``` -------------------------------- ### AppleDeviceAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source from an Apple device, inheriting from TorchStreamAudioSource. ```APIDOC ## AppleDeviceAudioSource ### Description Represents a source of audio that can start streaming via the stream property. ### Method N/A (Class) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Constructor Parameters - **uri** (Text) - Unique identifier of the audio source. - **sample_rate** (int) - Sample rate of the audio source. - **device** (str) - Identifier for the audio device. - **stream_index** (int) - Index of the stream to use. - **block_duration** (float) - Duration of each emitted chunk in seconds. ### Methods - **read()**: Start reading the source and yielding samples through the stream. - **close()**: Stop reading the source and close all open streams. ``` -------------------------------- ### MicrophoneAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source from a microphone. Inherits from AudioSource. ```APIDOC ## Class MicrophoneAudioSource ### Description Represents an audio source from a microphone. Inherits from AudioSource. ### Methods - **`_read_callback()`**: Internal callback function for reading audio data from the microphone. - **`read()`**: Reads a chunk of audio data from the microphone. - **`close()`**: Stops the microphone stream and releases resources. ``` -------------------------------- ### PipelineConfig Abstract Base Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/index Abstract base class for pipeline configurations. Defines common properties for duration, step, latency, and sample rate. ```APIDOC ## Class: PipelineConfig ### Description Configuration containing the required parameters to build and run a pipeline. This is an abstract base class. ### Abstract Properties - **duration** (float) - The duration of an input audio chunk (in seconds). - **step** (float) - The step between two consecutive input audio chunks (in seconds). - **latency** (float) - The algorithmic latency of the pipeline (in seconds). - **sample_rate** (int) - The sample rate of the input audio stream. ### Methods - **get_file_padding(filepath: diart.audio.FilePath)** - Calculates padding for a given audio file. - Parameters: - **filepath** (diart.audio.FilePath) - Path to the audio file. - Returns: - Tuple[float, float] - Padding start and end times. ``` -------------------------------- ### AudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Base class for all audio sources. Provides a common interface for reading and closing audio streams. ```APIDOC ## AudioSource ### Description Represents a source of audio that can start streaming via the stream property. ### Method N/A (Abstract Base Class) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Constructor Parameters - **uri** (Text) - Unique identifier of the audio source. - **sample_rate** (int) - Sample rate of the audio source. ### Properties - **duration** (float | None) - The duration of the stream if known. Defaults to None (unknown duration). ### Abstract Methods - **read()**: Start reading the source and yielding samples through the stream. - **close()**: Stop reading the source and close all open streams. ``` -------------------------------- ### Resample Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/utils/index Dynamically resamples audio chunks to a specified target sample rate. ```APIDOC ## Class: Resample ### Description Dynamically resample audio chunks. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Methods #### `__init__` - **sample_rate** (int) - Required - Original sample rate of the input audio. - **resample_rate** (int) - Required - Sample rate of the output. - **device** (Optional[torch.device]) - Optional - The device to perform resampling on. #### `__call__` - **waveform** (TemporalFeatures) - Required - Audio chunk to resample. ### Request Example ```json { "waveform": "TemporalFeatures object" } ``` ### Response #### Success Response (200) - **TemporalFeatures** (diart.features.TemporalFeatures) - Resampled audio chunk. #### Response Example ```json { "waveform": "Resampled TemporalFeatures object" } ``` ``` -------------------------------- ### AudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Base class for audio sources. Provides a common interface for reading audio data. ```APIDOC ## Class AudioSource ### Description Base class for audio sources. Provides a common interface for reading audio data. ### Methods - **`duration`** (property): Returns the duration of the audio source in seconds. - **`read()`**: Reads a chunk of audio data from the source. - **`close()`**: Releases any resources held by the audio source. ``` -------------------------------- ### diart.sinks.WindowClosedException Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sinks/index Common base class for all non-exit exceptions in diart sinks. ```APIDOC ## Exception: _diart.sinks.WindowClosedException ### Description Common base class for all non-exit exceptions. ### Parameters None ``` -------------------------------- ### MicrophoneAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Audio source that captures audio from a local microphone. ```APIDOC ## MicrophoneAudioSource ### Description Audio source tied to a local microphone. ### Method N/A (Class) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Constructor Parameters - **block_duration** (int) - Duration of each emitted chunk in seconds. Defaults to 0.5 seconds. - **device** (int | str | tuple[int, str] | None) - Device identifier compatible for the sounddevice stream. If None, use the default device. Defaults to None. ### Methods - **read_callback(samples, *args)**: Callback function for processing audio samples. - **read()**: Start reading the source and yielding samples through the stream. - **close()**: Stop reading the source and close all open streams. ``` -------------------------------- ### diart.operators.buffer_slide Source: https://diart.readthedocs.io/en/latest/autoapi/diart/operators/index A utility function to create a sliding window buffer of size n. ```APIDOC ## Function: buffer_slide ### Description Creates a sliding window buffer of a specified size `n`. This operator is typically used to maintain a history of the most recent `n` elements in a stream. ### Parameters - **n** (int): The size of the sliding window buffer. ### Returns - **Operator**: A reactive x operator that implements the sliding window buffer. ``` -------------------------------- ### OnlineSpeakerClustering Methods: add_center, identify (Python) Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Explains the `add_center` method for introducing new speaker centroids and the `identify` method for mapping speaker embeddings to existing centroids. These are core functionalities for dynamic speaker tracking. ```python add_center(_embedding_)# Add a new speaker centroid initialized to a given embedding Parameters: **embedding** (_np.ndarray_) – Embedding vector of some local speaker Returns: **center_index** – Index of the created center Return type: int identify(_segmentation_ , _embeddings_)# Identify the centroids to which the input speaker embeddings belong. Parameters: * **segmentation** (_np.ndarray_ _,__shape_ _(__frames_ _,__local_speakers_ _)_) – Matrix of segmentation outputs * **embeddings** (_np.ndarray_ _,__shape_ _(__local_speakers_ _,__embedding_dim_ _)_) – Matrix of embeddings Returns: **speaker_map** – A mapping from local speakers to global speakers. Return type: SpeakerMap ``` -------------------------------- ### diart.blocks.Pipeline Abstract Base Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/index Defines the interface for streaming audio pipelines in DiART. It outlines methods for configuration, running the pipeline, and managing its state. ```APIDOC ## Abstract Class: Pipeline Bases: `abc.ABC` Represents a streaming audio pipeline. ### Abstract Properties - **`config`** (`PipelineConfig`) - The configuration object for the pipeline. ### Abstract Methods - **`_get_config_class()`** - Returns the configuration class associated with this pipeline. - Return type: `type` - **`_suggest_metric()`** - Suggests a suitable metric for evaluating the pipeline. - Return type: `pyannote.metrics.base.BaseMetric` - **`_hyper_parameters()`** - Returns a list of hyper-parameters for the pipeline. - Return type: `Sequence[diart.blocks.base.HyperParameter]` - **`_reset()`** - Resets the internal state of the pipeline. - **`_set_timestamp_shift(shift)`** - Sets the timestamp shift for the pipeline. - Parameters: - **`shift`** (`float`) - The amount to shift timestamps by. - **`__call__(waveforms)`** - Processes a sequence of audio chunks. - Parameters: - **`waveforms`** (`Sequence[SlidingWindowFeature]`) - Consecutive chunk waveforms for the pipeline to ingest. - Returns: - For each input waveform, a tuple containing the pipeline output and its respective audio. - Return type: `Sequence[Tuple[Any, SlidingWindowFeature]]` ``` -------------------------------- ### DelayedAggregation Prepend Method Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/aggregation/index The _prepend method in DelayedAggregation is used to prepare buffers for aggregation. It takes the output window, output region, and a list of buffers as input. This method is internal to the DelayedAggregation class. ```python _prepend(_output_window_ , _output_region_ , _buffers_) ``` -------------------------------- ### AppleDeviceAudioSource Class Source: https://diart.readthedocs.io/en/latest/autoapi/diart/sources/index Represents an audio source from an Apple device. ```APIDOC ## Class AppleDeviceAudioSource ### Description Represents an audio source from an Apple device. Specific methods and properties are detailed in the full documentation. ``` -------------------------------- ### Load ONNX Embedding Model with diart Source: https://diart.readthedocs.io/en/latest/index Explains how to load an ONNX embedding model using diart's EmbeddingModel.from_onnx class method. It specifies the model path and allows customization of input and output names if they differ from the defaults. ```python from diart.models import EmbeddingModel embedding = EmbeddingModel.from_onnx( model_path="my_model.ckpt", input_names=["x", "w"], # defaults to ["waveform", "weights"] output_name="output", # defaults to "embedding" ) ``` -------------------------------- ### Pipeline Abstract Class in Python Source: https://diart.readthedocs.io/en/latest/autoapi/diart/blocks/base/index Abstract base class for streaming audio pipelines. It enforces the implementation of methods for retrieving configuration, suggesting metrics, defining hyper-parameters, resetting the pipeline state, setting timestamp shifts, and processing audio waveforms. ```python import abc from typing import Sequence, Tuple, Any, Type # Assuming necessary imports for types like PipelineConfig, HyperParameter, SlidingWindowFeature, BaseMetric # from diart.blocks.base import PipelineConfig, HyperParameter # from diart.inference import SlidingWindowFeature # from pyannote.metrics.base import BaseMetric class Pipeline(abc.ABC): """Represents a streaming audio pipeline""" @property @abc.abstractmethod def config(self) -> PipelineConfig: pass @staticmethod @abc.abstractmethod def get_config_class() -> Type[PipelineConfig]: pass @staticmethod @abc.abstractmethod def suggest_metric() -> 'BaseMetric': pass @staticmethod @abc.abstractmethod def hyper_parameters() -> Sequence['HyperParameter']: pass @abc.abstractmethod def _reset(self): pass @abc.abstractmethod def _set_timestamp_shift(self, shift: float): pass @abc.abstractmethod def __call__(self, waveforms: Sequence['SlidingWindowFeature']) -> Sequence[Tuple[Any, 'SlidingWindowFeature']]: """Runs the next steps of the pipeline given a list of consecutive audio chunks.""" pass ``` -------------------------------- ### diart.operators Module Functions Source: https://diart.readthedocs.io/en/latest/autoapi/diart/operators/index This section details the functions available within the diart.operators module, including those for audio stream manipulation, output accumulation, and buffering. ```APIDOC ## Functions in `diart.operators` ### `rearrange_audio_stream` Rearranges an audio stream. #### Parameters * **duration** (float) - Optional - The duration of the audio stream. * **step** (float) - Optional - The step size for rearranging. * **sample_rate** (int) - Optional - The sample rate of the audio. ### `buffer_slide` Implements a buffer slide mechanism. #### Parameters * **n** (int) - The number of elements to slide. ### `accumulate_output` Accumulates predictions and audio indefinitely with O(N) space complexity. #### Parameters * **duration** (float) - The duration for accumulation. * **step** (float) - The step size for accumulation. * **patch_collar** (float) - Optional - The collar value for patches. ### `buffer_output` Stores the last predictions and audio within a fixed buffer. #### Parameters * **duration** (float) - The duration of the buffer. * **step** (float) - The step size for buffering. * **latency** (float) - The latency of the buffer. * **sample_rate** (int) - The sample rate of the audio. * **...** - Additional arguments. ```