### Install Open-Unmix via pip Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/README.md Installs the openunmix package from PyPI. Requires torchaudio for audio processing. ```bash pip install openunmix ``` -------------------------------- ### Install museval for Evaluation Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/README.md Installs the museval package, which is required for evaluating the performance of source separation models against ground truth data. ```bash pip install museval ``` -------------------------------- ### Install Open-Unmix Environment Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/scripts/README.md Create a conda environment for the Open-Unmix project using the provided environment configuration files. ```bash conda env create -f environment-X.yml ``` -------------------------------- ### Audio Data Sampling and Mixing (Python) Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Implements the `__getitem__` method for the dataset, responsible for loading and preparing a single data sample. It determines the start time for audio extraction, potentially randomly if `random_chunks` is enabled. It then loads the target audio and specified interferer audio files, applies source augmentations, and stacks them into tensors. Finally, it creates the mixed input `x` by summing the sources and sets the target output `y` as the first source. ```python def __getitem__(self, index): # first, get target track track_path = self.tracks[index]["path"] min_duration = self.tracks[index]["min_duration"] if self.random_chunks: # determine start seek by target duration start = random.uniform(0, min_duration - self.seq_duration) else: start = 0 # assemble the mixture of target and interferers audio_sources = [] # load target target_audio, _ = load_audio( track_path / self.target_file, start=start, dur=self.seq_duration ) target_audio = self.source_augmentations(target_audio) audio_sources.append(target_audio) # load interferers for source in self.interferer_files: # optionally select a random track for each source if self.random_track_mix: random_idx = random.choice(range(len(self.tracks))) track_path = self.tracks[random_idx]["path"] if self.random_chunks: min_duration = self.tracks[random_idx]["min_duration"] start = random.uniform(0, min_duration - self.seq_duration) audio, _ = load_audio(track_path / source, start=start, dur=self.seq_duration) audio = self.source_augmentations(audio) audio_sources.append(audio) stems = torch.stack(audio_sources) # # apply linear mix over source index=0 x = stems.sum(0) # target is always the first element in the list y = stems[0] return x, y ``` -------------------------------- ### GET /umxse Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/index.html Initializes the Open-Unmix Speech Enhancement separator with specific target models and post-processing configurations. ```APIDOC ## GET /umxse ### Description Initializes and returns a Separator object configured for speech enhancement using the UMX SE model. ### Method GET ### Endpoint umxse(targets=None, residual=False, niter=1, device='cpu', pretrained=True, filterbank='torch') ### Parameters #### Query Parameters - **targets** (list) - Optional - List of targets to separate, e.g., ['speech', 'noise']. - **residual** (bool) - Optional - If True, creates a 'garbage' target for unmodeled audio. - **niter** (int) - Optional - Number of post-processing iterations. - **device** (str) - Optional - Computation device (e.g., 'cpu', 'cuda'). - **pretrained** (bool) - Optional - Whether to load pre-trained weights. - **filterbank** (str) - Optional - Implementation method: 'torch' or 'asteroid'. ### Response #### Success Response (200) - **separator** (Object) - Returns a configured Separator instance ready for inference. ``` -------------------------------- ### Evaluate Model Performance Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/scripts/README.md Install the museval package and run the evaluation script to compare separation results against benchmarks. ```bash pip install museval python -m openunmix.evaluate --outdir /path/to/musdb/estimates --evaldir /path/to/museval/results ``` -------------------------------- ### Load Audio File with Torchaudio Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Loads an audio file using torchaudio, allowing specification of start time and duration. If duration is not specified, it loads the full track. It returns the waveform as a PyTorch tensor and the sample rate. Dependencies include `torchaudio` and `Optional` from `typing`. ```python import torchaudio from typing import Optional def load_audio( path: str, start: float = 0.0, dur: Optional[float] = None, info: Optional[dict] = None, ): """Load audio file Args: path: Path of audio file start: start position in seconds, defaults on the beginning. dur: end position in seconds, defaults to `None` (full file). info: metadata object as called from `load_info`. Returns: Tensor: torch tensor waveform of shape `(num_channels, num_samples)` """ # loads the full track duration if dur is None: # we ignore the case where start!=0 and dur=None # since we have to deal with fixed length audio sig, rate = torchaudio.load(path) return sig, rate else: if info is None: info = load_info(path) num_frames = int(dur * info["samplerate"]) frame_offset = int(start * info["samplerate"]) sig, rate = torchaudio.load(path, num_frames=num_frames, frame_offset=frame_offset) return sig, rate ``` -------------------------------- ### Perform Source Separation and Evaluation with Python API Source: https://context7.com/sigsep/open-unmix-pytorch/llms.txt This snippet demonstrates how to load a pre-trained Open-Unmix model, process audio tracks, and evaluate the separation quality using the museval library. It requires the open-unmix and museval packages installed in a Python environment. ```python from openunmix import utils import museval import torch separator = utils.load_separator(model_str_or_path="umxl", device="cpu") separator.freeze() results = museval.EvalStore() for track in mus.tracks: audio = torch.as_tensor(track.audio, dtype=torch.float32) audio = utils.preprocess(audio, track.rate, separator.sample_rate) estimates = separator(audio) estimates_dict = separator.to_dict(estimates) estimates_np = {k: v[0].cpu().numpy().T for k, v in estimates_dict.items()} scores = museval.eval_mus_track(track, estimates_np) results.add_track(scores) print(track.name, scores) print(results) ``` -------------------------------- ### Launch Training with WAV Datasets Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/faq.md Initiates the training process using WAV format audio files. The `--is-wav` flag enables the use of WAV datasets, which are faster to load than MP4. ```python python scripts/train.py --root path/to/musdb18-wav --is-wav --target vocals ``` -------------------------------- ### Configure Training and Audio Backend Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Sets up the command-line argument parser for the Open-Unmix trainer and initializes the torchaudio backend. It allows users to specify datasets, audio backends, and sequence durations for training. ```python parser = argparse.ArgumentParser(description="Open Unmix Trainer") parser.add_argument("--dataset", type=str, default="musdb") parser.add_argument("--audio-backend", type=str, default="soundfile") args, _ = parser.parse_known_args() torchaudio.set_audio_backend(args.audio_backend) ``` -------------------------------- ### GET /get_tracks Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Loads input and output tracks from the dataset root directory. ```APIDOC ## GET /get_tracks ### Description Iterates through the dataset directory to load input and output tracks, filtering by duration if a sequence duration is specified. ### Method GET ### Endpoint /get_tracks ### Parameters None ### Response #### Success Response (200) - **path** (string) - Path to the track directory. - **min_duration** (float) - Minimum duration of the track or null. #### Response Example { "path": "/data/musdb/train/track_01", "min_duration": 30.5 } ``` -------------------------------- ### Get Model Memory Usage Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Estimates the memory usage of the model in megabytes. ```APIDOC ## get_model_memory_usage ### Description Estimates the memory usage of the model in megabytes. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model memory_usage = openunmix.utils.get_model_memory_usage(model) print(f"Model memory usage: {memory_usage:.2f} MB") ``` ### Response #### Success Response (200) - **memory_usage** (float) - The estimated memory usage in megabytes. #### Response Example ```json { "memory_usage": 57.23 } ``` ``` -------------------------------- ### Get Model Output Shape Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the expected output shape for the model. ```APIDOC ## get_model_output_shape ### Description Retrieves the expected output shape for the model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model output_shape = openunmix.utils.get_model_output_shape(model) print(f"Model output shape: {output_shape}") ``` ### Response #### Success Response (200) - **output_shape** (tuple) - A tuple representing the expected output shape (e.g., (batch_size, num_stems, num_channels, num_samples)). #### Response Example ```json { "output_shape": [1, 4, 2, 44100] } ``` ``` -------------------------------- ### Train model with AlignedDataset Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/training.md Demonstrates how to initiate training using the AlignedDataset, which maps input and output files directly within track folders. This is the fastest option for tasks like denoising or bandwidth extension. ```bash python train.py --dataset aligned --root /dataset --input_file mixture.wav --output_file vocals.wav ``` -------------------------------- ### Get Model Input Shape Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the expected input shape for the model. ```APIDOC ## get_model_input_shape ### Description Retrieves the expected input shape for the model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model input_shape = openunmix.utils.get_model_input_shape(model) print(f"Model input shape: {input_shape}") ``` ### Response #### Success Response (200) - **input_shape** (tuple) - A tuple representing the expected input shape (e.g., (batch_size, num_channels, num_samples)). #### Response Example ```json { "input_shape": [1, 2, 44100] } ``` ``` -------------------------------- ### Audio Dataset Initialization and Track Loading (Python) Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Initializes the Open-Unmix PyTorch dataset, setting up parameters like sample rate, sequence duration, and augmentation options. It defines how audio sources (target and interferers) are located within track folders and loads track information, including minimum duration, to prepare for data sampling. Raises a RuntimeError if no tracks are found. ```python def __init__(self, root, split, sample_rate, seq_duration, random_track_mix, random_chunks, source_augmentations, target_file, interferer_files, seed): self.root = Path(root).expanduser() self.split = split self.sample_rate = sample_rate self.seq_duration = seq_duration self.random_track_mix = random_track_mix self.random_chunks = random_chunks self.source_augmentations = source_augmentations # set the input and output files (accept glob) self.target_file = target_file self.interferer_files = interferer_files self.source_files = self.interferer_files + [self.target_file] self.seed = seed random.seed(self.seed) self.tracks = list(self.get_tracks()) if not len(self.tracks): raise RuntimeError("No tracks found") ``` -------------------------------- ### Get Model Optimizer Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the optimizer associated with the model's parameters. ```APIDOC ## get_model_optimizer ### Description Retrieves the optimizer associated with the model's parameters. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix import torch.optim as optim model = openunmix.utils.load_model() # Example: load a model optimizer = optim.Adam(model.parameters(), lr=0.001) retrieved_optimizer = openunmix.utils.get_model_optimizer(model, optimizer) print(retrieved_optimizer) ``` ### Response #### Success Response (200) - **optimizer** (torch.optim.Optimizer) - The optimizer instance. #### Response Example ```json { "optimizer": "" } ``` ``` -------------------------------- ### Load Custom Model from Disk Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/scripts/README.md Run the umx command-line tool using a user-trained model located at a specific file path. ```bash umx --model /path/to/model/root/directory input_file.wav ``` -------------------------------- ### Get Model Configuration Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the configuration parameters for a given OpenUnmix model. ```APIDOC ## get_model_config ### Description Retrieves the configuration parameters for a given OpenUnmix model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model config = openunmix.utils.get_model_config(model) print(config) ``` ### Response #### Success Response (200) - **config** (dict) - A dictionary containing the model's configuration parameters. #### Response Example ```json { "config": { "n_fft": 4096, "hop_length": 1024, "stem_num": 4, "target_instrument": "vocals" } } ``` ``` -------------------------------- ### Configure CLI Arguments and Initialize Separator Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/cli.html This snippet demonstrates how to define command-line arguments for the Open-Unmix tool, including model paths, audio backends, and processing parameters. It subsequently initializes the separator using the provided configuration and prepares it for inference. ```python parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter) parser.add_argument("--model", default="umxhq", type=str, help="path to model base directory") parser.add_argument("--audio-backend", default="sox_io", type=str) args = parser.parse_args() device = torch.device("cuda" if not args.no_cuda and torch.cuda.is_available() else "cpu") separator = utils.load_separator( model_str_or_path=args.model, targets=args.targets, niter=args.niter, device=device, pretrained=True ) separator.freeze() separator.to(device) ``` -------------------------------- ### GET /umxhq_spec Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/index.html Loads the high-quality Open-Unmix models for music source separation. ```APIDOC ## GET /umxhq_spec ### Description Loads pre-trained Open-Unmix models for music source separation (vocals, drums, bass, other) based on the MUSDB18-HQ dataset. ### Method GET ### Endpoint umxhq_spec(targets=None, device="cpu", pretrained=True) ### Parameters #### Query Parameters - **targets** (list) - Optional - List of stems to load: ['vocals', 'drums', 'bass', 'other']. - **device** (str) - Optional - Computation device ('cpu' or 'cuda'). - **pretrained** (bool) - Optional - Whether to load pre-trained weights from Zenodo. ### Response #### Success Response (200) - **target_models** (dict) - A dictionary mapping target names to their respective PyTorch model instances. ``` -------------------------------- ### Get Model Target Instrument Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the target instrument that the model is trained to separate. ```APIDOC ## get_model_target_instrument ### Description Retrieves the target instrument that the model is trained to separate. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model target_instrument = openunmix.utils.get_model_target_instrument(model) print(f"Model target instrument: {target_instrument}") ``` ### Response #### Success Response (200) - **target_instrument** (str) - The name of the target instrument (e.g., 'vocals'). #### Response Example ```json { "target_instrument": "vocals" } ``` ``` -------------------------------- ### AlignedDataset Initialization and Path Handling in Python Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Initializes the AlignedDataset, setting up paths for input and output audio files based on the provided root directory, split, and file naming conventions. It handles potential glob patterns for file matching and raises an error if the dataset is found to be empty. ```python class AlignedDataset(UnmixDataset): def __init__( self, root: str, split: str = "train", input_file: str = "mixture.wav", output_file: str = "vocals.wav", seq_duration: Optional[float] = None, random_chunks: bool = False, sample_rate: float = 44100.0, source_augmentations: Optional[Callable] = None, seed: int = 42, ) -> None: """A dataset of that assumes multiple track folders where each track includes and input and an output file which directly corresponds to the the input and the output of the model. This dataset is the most basic of all datasets provided here, due to the least amount of preprocessing, it is also the fastest option, however, it lacks any kind of source augmentations or custum mixing. Typical use cases: * Source Separation (Mixture -> Target) * Denoising (Noisy -> Clean) * Bandwidth Extension (Low Bandwidth -> High Bandwidth) Example ======= data/train/01/mixture.wav --> input data/train/01/vocals.wav ---> output """ self.root = Path(root).expanduser() self.split = split self.sample_rate = sample_rate self.seq_duration = seq_duration self.random_chunks = random_chunks # set the input and output files (accept glob) self.input_file = input_file self.output_file = output_file self.tuple_paths = list(self._get_paths()) if not self.tuple_paths: raise RuntimeError("Dataset is empty, please check parameters") self.seed = seed random.seed(self.seed) ``` -------------------------------- ### Load and Use umxl Separator (PyTorch) Source: https://context7.com/sigsep/open-unmix-pytorch/llms.txt Loads the default high-performance 'umxl' separator model from open-unmix. It then preprocesses audio and separates it into stems: vocals, drums, bass, and other instruments. The model is suitable for general music source separation tasks. ```python import torch import openunmix # Load the umxl separator (default, best quality) separator = openunmix.umxl(device="cuda") # Load audio (shape: channels, samples) audio = torch.randn(2, 44100 * 10) # 10 seconds of stereo audio # Preprocess and separate audio = openunmix.utils.preprocess(audio, rate=44100, model_rate=separator.sample_rate) estimates = separator(audio) # estimates shape: (batch, targets, channels, samples) # targets order: vocals, drums, bass, other print(f"Separated audio shape: {estimates.shape}") ``` -------------------------------- ### Get Model Device Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Determines the device (CPU or GPU) on which the model is currently loaded. ```APIDOC ## get_model_device ### Description Determines the device (CPU or GPU) on which the model is currently loaded. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model device = openunmix.utils.get_model_device(model) print(f"Model is on device: {device}") ``` ### Response #### Success Response (200) - **device** (torch.device) - The device object representing CPU or CUDA. #### Response Example ```json { "device": "cuda:0" } ``` ``` -------------------------------- ### Get Model Loss Function Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the loss function used for training the model. ```APIDOC ## get_model_loss_function ### Description Retrieves the loss function used for training the model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix import torch.nn as nn model = openunmix.utils.load_model() # Example: load a model loss_fn = nn.MSELoss() retrieved_loss_fn = openunmix.utils.get_model_loss_function(model, loss_fn) print(retrieved_loss_fn) ``` ### Response #### Success Response (200) - **loss_function** (torch.nn.Module) - The loss function instance. #### Response Example ```json { "loss_function": "" } ``` ``` -------------------------------- ### Implement Custom Dataset for Open-Unmix (Python) Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/extensions.md Provides a template for creating a custom PyTorch dataset to be used with Open-Unmix. It demonstrates how to load audio files and return them as pairs of input and target tensors. This allows users to integrate their own datasets for training. ```python from utils import load_audio, load_info class TemplateDataset(UnmixDataset): """A template dataset class for you to implement custom datasets.""" def __init__(self, root, split='train', sample_rate=44100, seq_dur=None): """Initialize the dataset """ self.root = root self.tracks = get_tracks(root, split) def __getitem__(self, index): """Returns a time domain audio example of shape=(channel, sample) """ path = self.tracks[index] x = load_audio(path) y = load_audio(path) return x, y def __len__(self): """Return the number of audio samples""" return len(self.tracks) ``` -------------------------------- ### GET /openunmix/filtering/atan2 Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/filtering.html Calculates the element-wise arctangent of y/x, returning signed angles in radians. ```APIDOC ## GET /openunmix/filtering/atan2 ### Description Calculates the element-wise arctangent of y/x. This function provides an alternative implementation to torch.atan2, returning a new tensor with signed angles in radians. ### Method GET ### Endpoint /openunmix/filtering/atan2 ### Parameters #### Query Parameters - **y** (Tensor) - Required - The numerator tensor. - **x** (Tensor) - Required - The denominator tensor. ### Request Example { "y": [0.5, 1.0], "x": [1.0, 0.5] } ### Response #### Success Response (200) - **result** (Tensor) - The calculated arctangent values in radians. #### Response Example { "result": [0.4636, 1.1071] } ``` -------------------------------- ### Load Audio Metadata and Segments with Python Source: https://context7.com/sigsep/open-unmix-pytorch/llms.txt Demonstrates how to extract metadata from audio files and load specific segments using the openunmix data utilities. ```python from openunmix.data import load_audio, load_info # Get audio metadata info = load_info("audio.wav") print(f"Sample rate: {info['samplerate']}") # Load full audio file audio, rate = load_audio("audio.wav") # Load audio segment audio_segment, rate = load_audio("audio.wav", start=10.0, dur=5.0) ``` -------------------------------- ### Run Open-Unmix with Docker Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/README.md Executes source separation on a local audio file using the provided Docker container. Maps a local directory to the container for input and output. ```bash docker run -v ~/Music/:/data -it faroit/open-unmix-pytorch "/data/track1.wav" --outdir /data/track1 ``` -------------------------------- ### GET /models/umxhq Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/index.html Initializes the Open Unmix 2-channel/stereo BiLSTM Model trained on MUSDB18-HQ. ```APIDOC ## GET /models/umxhq ### Description Initializes and returns a Separator object configured with pre-trained models from the MUSDB18-HQ dataset. ### Method GET ### Endpoint /models/umxhq ### Parameters #### Query Parameters - **targets** (list) - Optional - List of sources to separate: ['vocals', 'drums', 'bass', 'other']. - **residual** (bool) - Optional - If True, creates a 'garbage' target for unassigned audio. - **niter** (int) - Optional - Number of post-processing iterations. - **device** (str) - Optional - Inference device (e.g., 'cpu', 'cuda'). - **pretrained** (bool) - Optional - Whether to load pre-trained weights. - **filterbank** (str) - Optional - Implementation method: 'torch' or 'asteroid'. ### Request Example { "targets": ["vocals", "drums"], "residual": false, "device": "cpu" } ### Response #### Success Response (200) - **separator** (object) - The initialized Separator instance ready for inference. #### Response Example { "status": "success", "model": "umxhq" } ``` -------------------------------- ### GET /models/targets Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Loads specific target models based on the provided configuration or registry name. ```APIDOC ## GET /models/targets ### Description Loads individual target models (e.g., .pth files) from a specified path or registry. Used internally by the separator to manage specific source models. ### Method GET ### Endpoint /models/targets ### Parameters #### Query Parameters - **targets** (list of strings) - Required - List of target names. - **model_str_or_path** (string) - Optional - Path or registry name. Defaults to 'umxhq'. - **device** (string) - Optional - Device to load models onto. Defaults to 'cpu'. - **pretrained** (boolean) - Optional - Whether to load pre-trained weights. Defaults to True. ### Request Example { "targets": ["vocals"], "model_str_or_path": "umxhq" } ### Response #### Success Response (200) - **target_models** (object) - Dictionary of loaded target model objects. #### Response Example { "status": "success", "loaded_targets": ["vocals"] } ``` -------------------------------- ### Initialize and Implement SourceFolderDataset Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Defines the SourceFolderDataset class which inherits from UnmixDataset. It handles loading audio sources from specific directories, managing random track selection, and applying augmentations for training models. ```python class SourceFolderDataset(UnmixDataset): def __init__( self, root: str, split: str = "train", target_dir: str = "vocals", interferer_dirs: List[str] = ["bass", "drums"], ext: str = ".wav", nb_samples: int = 1000, seq_duration: Optional[float] = None, random_chunks: bool = True, sample_rate: float = 44100.0, source_augmentations: Optional[Callable] = lambda audio: audio, seed: int = 42, ) -> None: self.root = Path(root).expanduser() self.split = split self.sample_rate = sample_rate self.seq_duration = seq_duration self.ext = ext self.random_chunks = random_chunks self.source_augmentations = source_augmentations self.target_dir = target_dir self.interferer_dirs = interferer_dirs self.source_folders = self.interferer_dirs + [self.target_dir] self.source_tracks = self.get_tracks() self.nb_samples = nb_samples self.seed = seed random.seed(self.seed) def __getitem__(self, index): audio_sources = [] for source in self.source_folders: if self.split == "valid": random.seed(index) source_path = random.choice(self.source_tracks[source]) duration = load_info(source_path)["duration"] if self.random_chunks: start = random.uniform(0, duration - self.seq_duration) else: start = max(duration // 2 - self.seq_duration // 2, 0) audio, _ = load_audio(source_path, start=start, dur=self.seq_duration) audio = self.source_augmentations(audio) ``` -------------------------------- ### Python: VariableSourcesTrackFolderDataset Initialization Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Initializes the VariableSourcesTrackFolderDataset, which handles audio source separation from track folders with a variable number of sources. It configures parameters like sample rate, sequence duration, and augmentation options. ```python class VariableSourcesTrackFolderDataset(UnmixDataset): def __init__( self, root: str, split: str = "train", target_file: str = "vocals.wav", ext: str = ".wav", seq_duration: Optional[float] = None, random_chunks: bool = False, random_interferer_mix: bool = False, sample_rate: float = 44100.0, source_augmentations: Optional[Callable] = lambda audio: audio, silence_missing_targets: bool = False, ) -> None: """A dataset that assumes audio sources to be stored in track folder where each track has a _variable_ number of sources. The users specifies the target file-name (`target_file`) and the extension of sources to used for mixing. A linear mix is performed on the fly by summing all sources in a track folder. Since the number of sources differ per track, while target is fixed, a random track mix augmentation cannot be used. Instead, a random track can be used to load the interfering sources. Also make sure, that you do not provide the mixture file among the sources! Example ======= train/1/vocals.wav --> input target \ train/1/drums.wav --> input target | train/1/bass.wav --> input target --+--> input train/1/accordion.wav --> input target | train/1/marimba.wav --> input target / train/1/vocals.wav -----------------------> output """ self.root = Path(root).expanduser() self.split = split self.sample_rate = sample_rate self.seq_duration = seq_duration self.random_chunks = random_chunks self.random_interferer_mix = random_interferer_mix self.source_augmentations = source_augmentations self.target_file = target_file self.ext = ext self.silence_missing_targets = silence_missing_targets self.tracks = list(self.get_tracks()) ``` -------------------------------- ### Get Model Summary Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Generates a summary of the model, including its layers, parameter counts, and FLOPs. ```APIDOC ## get_model_summary ### Description Generates a summary of the model, including its layers, parameter counts, and FLOPs. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model input_shape = (1, 2, 44100) # Example input shape summary = openunmix.utils.get_model_summary(model, input_shape) print(summary) ``` ### Response #### Success Response (200) - **summary** (str) - A string containing the model summary. #### Response Example ```json { "summary": "Model: OpenUnmix\nInput shape: (1, 2, 44100)\n=================================================================\nLayer (type)\tOutput Shape\tParam #\tFLOPs\n=================================================================\n... (detailed layer information) ...\n================================================================= Total params: 15,000,000\nTrainable params: 15,000,000\nNon-trainable params: 0\nTotal FLOPs: 1.2e10\n================================================================= ``` ``` -------------------------------- ### Get Model Number of Parameters Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Calculates and returns the total number of trainable parameters in the model. ```APIDOC ## get_model_num_parameters ### Description Calculates and returns the total number of trainable parameters in the model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model num_params = openunmix.utils.get_model_num_parameters(model) print(f"Number of trainable parameters: {num_params}") ``` ### Response #### Success Response (200) - **num_parameters** (int) - The total count of trainable parameters. #### Response Example ```json { "num_parameters": 15000000 } ``` ``` -------------------------------- ### Get Model Sample Rate Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the sample rate associated with the model's input/output. ```APIDOC ## get_model_sample_rate ### Description Retrieves the sample rate associated with the model's input/output. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model sample_rate = openunmix.utils.get_model_sample_rate(model) print(f"Model sample rate: {sample_rate}") ``` ### Response #### Success Response (200) - **sample_rate** (int) - The sample rate in Hz. #### Response Example ```json { "sample_rate": 44100 } ``` ``` -------------------------------- ### Load Audio Tracks - Python Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html The get_tracks method iterates through specified directories to load audio track paths. It supports filtering tracks based on a minimum duration if seq_duration is provided. Dependencies include `pathlib`, `tqdm`, and a `load_info` function. ```python def get_tracks(self): """Loads input and output tracks""" p = Path(self.root, self.split) source_tracks = {} for source_folder in tqdm.tqdm(self.source_folders): tracks = [] source_path = p / source_folder for source_track_path in sorted(source_path.glob("*" + self.ext)): if self.seq_duration is not None: info = load_info(source_track_path) # get minimum duration of track if info["duration"] > self.seq_duration: tracks.append(source_track_path) else: tracks.append(source_track_path) source_tracks[source_folder] = tracks return source_tracks ``` -------------------------------- ### Get Model Weights Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the weights of a specific layer or the entire model from an OpenUnmix model. ```APIDOC ## get_model_weights ### Description Retrieves the weights of a specific layer or the entire model from an OpenUnmix model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model # Get weights for a specific layer (e.g., 'encoder.weight') layer_weights = openunmix.utils.get_model_weights(model, 'encoder.weight') # Get all weights as a dictionary all_weights = openunmix.utils.get_model_weights(model) ``` ### Response #### Success Response (200) - **weights** (Union[torch.Tensor, dict]) - Either a torch.Tensor for a specific layer or a dictionary of layer names to their weights. #### Response Example ```json { "weights": { "encoder.weight": "", "decoder.bias": "" } } ``` ``` -------------------------------- ### Configure FixedSourcesTrackFolderDataset with Augmentations Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Sets up the FixedSourcesTrackFolderDataset, parsing arguments for target file, interferer files, and augmentation options like random track mixing. It initializes the dataset for training and validation. ```python parser.add_argument("--target-file", type=str) parser.add_argument("--interferer-files", type=str, nargs="+") parser.add_argument( "--random-track-mix", action="store_true", default=False, help="Apply random track mixing augmentation", ) parser.add_argument("--source-augmentations", type=str, nargs="+") args = parser.parse_args() args.target = Path(args.target_file).stem dataset_kwargs = { "root": Path(args.root), "interferer_files": args.interferer_files, "target_file": args.target_file, } source_augmentations = aug_from_str(args.source_augmentations) train_dataset = FixedSourcesTrackFolderDataset( split="train", source_augmentations=source_augmentations, random_track_mix=args.random_track_mix, random_chunks=True, seq_duration=args.seq_dur, **dataset_kwargs, ) valid_dataset = FixedSourcesTrackFolderDataset( split="valid", seq_duration=None, **dataset_kwargs ) ``` -------------------------------- ### Load Pre-trained Models in Python Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/README.md Demonstrates how to load a pre-trained Open-Unmix model directly within a Python script. ```python import openunmix separator = openunmix.umxl(...) ``` -------------------------------- ### GET /umxse Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/index.html Initializes the Speech Enhancement (UMX SE) model for 1-channel audio processing. ```APIDOC ## GET /umxse ### Description Initializes the Open-Unmix Speech Enhancement model trained on the Voicebank+Demand dataset. Returns a separator object configured for 16kHz audio. ### Method GET ### Endpoint umxse(targets=None, residual=False, niter=1, device="cpu", pretrained=True, filterbank="torch") ### Parameters #### Query Parameters - **targets** (list) - Optional - List of targets to separate, e.g., ['speech', 'noise']. - **residual** (bool) - Optional - If True, creates a 'garbage' target for unmodeled audio. - **niter** (int) - Optional - Number of post-processing iterations. - **device** (str) - Optional - Computation device ('cpu' or 'cuda'). - **pretrained** (bool) - Optional - Whether to load MUSDB18-HQ pre-trained weights. - **filterbank** (str) - Optional - Implementation method: 'torch' or 'asteroid'. ### Response #### Success Response (200) - **separator** (object) - A configured Separator instance ready for inference. ``` -------------------------------- ### Initialize VariableSourcesTrackFolderDataset Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Defines the constructor for the dataset, setting up paths, sample rates, and augmentation strategies. It initializes the track list based on the provided root directory and split. ```python class VariableSourcesTrackFolderDataset(UnmixDataset): def __init__(self, root: str, split: str = "train", target_file: str = "vocals.wav", ext: str = ".wav", seq_duration: Optional[float] = None, random_chunks: bool = False, random_interferer_mix: bool = False, sample_rate: float = 44100.0, source_augmentations: Optional[Callable] = lambda audio: audio, silence_missing_targets: bool = False) -> None: self.root = Path(root).expanduser() self.split = split self.sample_rate = sample_rate self.seq_duration = seq_duration self.random_chunks = random_chunks self.random_interferer_mix = random_interferer_mix self.source_augmentations = source_augmentations self.target_file = target_file self.ext = ext self.silence_missing_targets = silence_missing_targets self.tracks = list(self.get_tracks()) ``` -------------------------------- ### Audio Loading Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Loads audio data from a file, with options to specify start time and duration. ```APIDOC ## GET /audio/load ### Description Loads audio waveform from a specified file path. ### Method GET ### Endpoint /audio/load ### Parameters #### Query Parameters - **path** (string) - Required - Path to the audio file. - **start** (float) - Optional - Start position in seconds. Defaults to 0.0. - **dur** (float) - Optional - Duration in seconds. Defaults to `None` (loads the full file). - **info** (object) - Optional - Metadata object obtained from `load_info`. If not provided and `dur` is specified, it will be fetched. ### Response #### Success Response (200) - **waveform** (Tensor) - A PyTorch tensor representing the audio waveform with shape `(num_channels, num_samples)`. - **samplerate** (integer) - The sample rate of the loaded audio. #### Response Example ```json { "waveform": [[0.1, 0.2, ...], [0.3, 0.4, ...]], "samplerate": 44100 } ``` ``` -------------------------------- ### GET /dataset/variable-sources Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Initializes and retrieves data from the VariableSourcesTrackFolderDataset, which dynamically mixes audio sources from track folders. ```APIDOC ## GET /dataset/variable-sources ### Description Initializes a dataset that assumes audio sources are stored in track folders. It performs a linear mix on the fly by summing all sources in a track folder, excluding the target file. ### Method GET ### Endpoint /dataset/variable-sources ### Parameters #### Query Parameters - **root** (string) - Required - The root directory containing the dataset. - **split** (string) - Optional - Dataset split (e.g., 'train', 'test'). Default: 'train'. - **target_file** (string) - Optional - The filename of the target source. Default: 'vocals.wav'. - **seq_duration** (float) - Optional - Duration of audio chunks in seconds. - **random_chunks** (boolean) - Optional - Whether to use random chunks from the audio. Default: false. - **random_interferer_mix** (boolean) - Optional - Whether to mix in a random interferer track. Default: false. ### Request Example { "root": "/data/musdb18", "split": "train", "target_file": "vocals.wav" } ### Response #### Success Response (200) - **x** (tensor) - The mixed audio input tensor. - **y** (tensor) - The target audio source tensor. #### Response Example { "x": "[tensor_data]", "y": "[tensor_data]" } ``` -------------------------------- ### MUSDB Dataset Initialization and Configuration (Python) Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/data.html Initializes the MUSDB dataset, configuring parameters such as subsets, sequence duration, sample rate, and augmentation options. It sets up the internal MUSDB DB object and prepares for data loading. ```python import musdb import random import torch # ... inside a class definition ... self.seed = seed random.seed(seed) self.is_wav = is_wav self.seq_duration = seq_duration self.target = target self.subsets = subsets self.split = split self.samples_per_track = samples_per_track self.source_augmentations = source_augmentations self.random_track_mix = random_track_mix self.mus = musdb.DB( root=root, is_wav=is_wav, split=split, subsets=subsets, download=download, *args, **kwargs, ) self.sample_rate = 44100.0 # musdb is fixed sample rate ``` -------------------------------- ### Get Model FFT Size Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/utils.html Retrieves the FFT (Fast Fourier Transform) size used by the model. ```APIDOC ## get_model_fft_size ### Description Retrieves the FFT (Fast Fourier Transform) size used by the model. ### Method N/A (Utility Function) ### Endpoint N/A (Utility Function) ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example ```python import openunmix model = openunmix.utils.load_model() # Example: load a model n_fft = openunmix.utils.get_model_fft_size(model) print(f"Model FFT size: {n_fft}") ``` ### Response #### Success Response (200) - **n_fft** (int) - The FFT size. #### Response Example ```json { "n_fft": 4096 } ``` ``` -------------------------------- ### Initialize and Run Audio Separation Source: https://github.com/sigsep/open-unmix-pytorch/blob/master/docs/cli.html Configures the separation model, loads audio files from the filesystem, and performs source separation using the Open-Unmix model. It supports multiple audio backends and handles output directory creation and file saving. ```python separator = utils.load_separator( model_str_or_path=args.model, targets=args.targets, niter=args.niter, residual=args.residual, wiener_win_len=args.wiener_win_len, device=device, pretrained=True, filterbank=args.filterbank, ) separator.freeze() separator.to(device) estimates = predict.separate( audio=audio, rate=rate, aggregate_dict=aggregate_dict, separator=separator, device=device, ) ```