### Development Installation of Omnizart Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/quick-start.html Clone the Omnizart repository and install it for development. This includes downloading checkpoints and development dependencies. ```bash git clone https://github.com/Music-and-Culture-Technology-Lab/omnizart.git cd omnizart make install make install-dev ``` -------------------------------- ### Install Omnizart with Pip Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/quick-start.html Install the latest stable version of Omnizart using pip. Ensure prerequisites and system packages are installed first. ```bash pip install numpy Cython sudo apt-get install libsndfile-dev fluidsynth ffmpeg pip install omnizart omnizart download-checkpoints ``` -------------------------------- ### Transcribe audio with vocal-contour Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/vocal-contour/cli.html Example command for transcribing an audio file using a specific model and output path. ```bash $ omnizart vocal-contour transcribe example.wav –model-path path/to/model –output example.mid ``` ```bash omnizart vocal-contour transcribe [OPTIONS] INPUT_AUDIO ``` -------------------------------- ### BaseStructure Class Attribute: train_wavs Example Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Illustrates how `train_wavs` should be defined for a dataset with nested audio file directories. The paths are relative to the dataset root. ```python train_wavs = None # Record folders that contain trainig wav files. # The path to sub-folders should be the relative path to root folder of the dataset. # Examples # Assume the structure of the dataset looks like following # maps # ├── MAPS_AkPnCGdD_2 # │ └── AkPnCGdD # └── MAPS_AkPnBcht_2 # └── AkPnBcht # where `AkPnCGdD` and `AkPnBcht` are the folders that store the wav files. The function should then return a list like: # >>> [‘MAPS_AkPnCGdD_2/AkPnCGdD’, ‘MAPS_AkPnBcht_2/AkPnBcht’] ``` -------------------------------- ### Transcribe Drum Events with Custom Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Example of transcribing percussive events from a WAV file using a specified model path and output directory. The `--model-path` can be omitted to use default checkpoints. ```bash # Transcribe percussive events given pop.wav, with specified model path and output directory omnizart drum transcribe pop.wav --model-path ./my-model --output ./trans_pop.mid ``` -------------------------------- ### Transcribe Chords from Audio Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/api.html Transcribes chord progressions from an audio file, outputting both MIDI and CSV files. The MIDI provides a listenable representation, while the CSV contains detailed chord names, start, and end times. ```python midi_output = chord_transcription.transcribe(input_audio='path/to/audio.wav', model_path='path/to/model') ``` -------------------------------- ### Initialize Beat Transcription Application Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/api.html Instantiate the BeatTranscription application class. Optionally provide a path to a configuration file. ```python BeatTranscription(_conf_path =None_) ``` -------------------------------- ### Download MusicNet Dataset and Unzip Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Downloads the MusicNet dataset and automatically unzips it after the download is complete. ```bash # Download the MusicNet dataset and unzip the dataset after download. omnizart download-dataset MusicNet --unzip ``` -------------------------------- ### Utils: shape_list Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/models.html A utility function to get the shape of an input tensor. ```APIDOC ## Utils: shape_list ### Description Returns a list of dimensions for the input tensor, providing static dimensions where possible. ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### `shape_list(_input_tensor_)` - **_input_tensor_** (tensor) - The input tensor for which to get the shape. Returns: list: A list representing the dimensions of the input tensor. ``` -------------------------------- ### Train Music Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Trains a music model from scratch using generated features. Requires the path to the feature folder and a model name. ```bash omnizart music train-model -d --model-name My-Music ``` -------------------------------- ### Get Chord Transcription Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/api.html Retrieves the chord model. This method is part of the ChordTranscription class and inherits functionality from the base class. ```python chord_model = chord_transcription.get_model(settings) ``` -------------------------------- ### Synthesize Audio from MIDI with CLI Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/quick-start.html Render a transcribed MIDI file into an audio (WAV) using Omnizart's synthesis capabilities with default soundfonts. ```bash omnizart synth example.mid ``` -------------------------------- ### Download pre-trained model checkpoints Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Executes the command to download archived checkpoints for pre-trained models without requiring additional arguments. ```bash # Simply run the following command, and no other options are needed to be specified. omnizart download-checkpoints ``` -------------------------------- ### Transcribe Music with CLI Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/quick-start.html Use the Omnizart CLI to transcribe a music file (WAV format) into a MIDI file and a CSV file. ```bash omnizart music transcribe ``` -------------------------------- ### Display Available Datasets for Download Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Command to view a complete list of datasets that can be downloaded using the `omnizart download-dataset` utility. ```bash omnizart download-dataset --help ``` -------------------------------- ### Initialize Beat Dataset Loader Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/api.html Data loader for training beat transcription models. Handles feature and label files, yielding samples with a specified overlap. ```python BeatDatasetLoader(_feature_folder =None_, _feature_files =None_, _num_samples =100_, _slice_hop =1_, _feat_col_name ='feature'_) ``` -------------------------------- ### Dataset Training and Testing Folders Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Configuration for training and testing data folders. ```APIDOC ## Dataset Folders ### Train Labels - **train_labels**: ['05_align_mid/dist0p00', '05_align_mid/dist0p10', '05_align_mid/dist0p20', '05_align_mid/dist0p40', '05_align_mid/dist0p50', '05_align_mid/dist0p60'] - Folder to train labels ### Train Wavs - **train_wavs**: ['01_ytd_audio/dist0p00', '01_ytd_audio/dist0p10', '01_ytd_audio/dist0p20', '01_ytd_audio/dist0p40', '01_ytd_audio/dist0p50', '01_ytd_audio/dist0p60'] - Folder to train wavs ### Test Labels - **test_labels**: [] - Folder to test labels ### Test Wavs - **test_wavs**: [] - Folder to test wavs ``` -------------------------------- ### Omnizart CLI Pipeline Structure Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Illustrates the general command structure for Omnizart applications, following an 'application'-'action'-'arguments' pattern. ```bash omnizart application action --arguments ``` -------------------------------- ### BaseStructure Class Method: get_test_wavs Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Retrieves a list of complete file paths for all testing WAV audio files. ```python @classmethod def get_test_wavs(cls, _dataset_path_): """Get list of complete test wav paths""" pass ``` -------------------------------- ### Train Chord Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Trains a chord model from scratch using generated features. Requires the path to the feature folder and a model name. ```bash omnizart chord train-model -d --model-name My-Chord ``` -------------------------------- ### Model Training Parameters Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/api.html Configuration options for training Omnizart models. ```APIDOC ## Model Training Parameters ### Description Configuration options for training Omnizart models. ### Parameters #### Request Body - **decaying_rate_per_epoch** (Float) - Optional - Decaying rate of learning rate per epoch. - **Value**: 0.96 ``` -------------------------------- ### Download Maestro Dataset Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Downloads the MAESTRO dataset and specifies an output directory for the downloaded files. ```bash # Download the MAESTRO dataset and output to the */data* folder. omnizart download-dataset Maestro --output /data ``` -------------------------------- ### BaseStructure Class Method: get_train_wavs Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Retrieves a list of complete file paths for all training WAV audio files. Paths are relative to the dataset's root folder. ```python @classmethod def get_train_wavs(cls, _dataset_path_): """Get list of complete train wav paths""" pass ``` -------------------------------- ### Training Hyper-parameters Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/vocal/api.html Configuration settings for training the Omnizart model. ```APIDOC ## Training Hyper-parameters ### Description Hyper-parameters used to control the training process of the model. ### Parameters - **Epoch** (Integer) - Optional - Maximum number of epochs for training. Default: 10 - **Steps** (Integer) - Optional - Number of training steps for each epoch. Default: 1000 - **ValSteps** (Integer) - Optional - Number of validation steps after each training epoch. Default: 50 - **BatchSize** (Integer) - Optional - Batch size of each training step. Default: 64 - **ValBatchSize** (Integer) - Optional - Batch size of each validation step. Default: 64 - **EarlyStop** (Integer) - Optional - Terminate the training if the validation performance doesn't improve after n epochs. Default: 8 - **InitLearningRate** (Float) - Optional - Initial learning rate. Default: 0.0001 - **ContextLength** (Integer) - Optional - Context to be considered before and after current timestamp. Default: 9 ``` -------------------------------- ### Generate Features for Music Application Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Command to generate necessary features for training and testing the music application. Requires specifying the path to the dataset. ```bash # Generate features for the music application omnizart music generate-feature --dataset-path ``` -------------------------------- ### Enable CLI Auto Completion Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/quick-start.html Configure shell auto-completion for the Omnizart CLI by sourcing the generated completion script. ```bash # For bash _OMNIZART_COMPLETE=source_bash omnizart > omnizart-complete.sh # For zsh _OMNIZART_COMPLETE=source_zsh omnizart > omnizart-complete.sh # Source the generated script to enable source omnizart-complete.sh ``` -------------------------------- ### Train a vocal-contour model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/vocal-contour/cli.html Command to initiate training or fine-tuning of a vocal-contour model. ```bash omnizart vocal_contour train-model [OPTIONS] ``` -------------------------------- ### Omnizart Command Line Interface Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Commands available for direct use in the command line interface. ```APIDOC ## Omnizart Command Line Interface ### Description Commands available for direct use in the command line interface. ### Commands - `omnizart music`: For music transcription. - `omnizart drum`: For drum transcription. - `omnizart chord`: For chord transcription. - `omnizart vocal`: For vocal transcription. - `omnizart vocal-contour`: For vocal contour transcription. - `omnizart beat`: For beat transcription. - `omnizart patch-cnn`: For Patch-CNN transcription. ``` -------------------------------- ### Quefrency to Log Frequency Mapping Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/feature.html Maps quefrency data to a log frequency scale, relevant for cepstral analysis. ```python omnizart.feature.cfp.quef_to_log_freq_mapping(_ceps_ , _q_ , _fs_ , _fc_ , _tc_ , _NumPerOct_) ``` -------------------------------- ### Train Drum Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Trains a drum model from scratch using generated features. Requires the path to the feature folder and a model name. ```bash omnizart drum train-model -d --model-name My-Drum ``` -------------------------------- ### Visualize Training Lifecycle Events Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/training.html A visual representation of the event-based life-cycle management during the training process. ```text | |-on_train_begin T| |-on_epoch_begin R| | A| L|-on_train_batch_begin I| O|-on_train_batch_end N| O| I| P|-on_test_batch_begin N| |-on_test_batch_end G| | | |-on_epoch_end |-on_train_end | ``` -------------------------------- ### Transcribe Audio via CLI Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/music/cli.html Transcribes an audio file into a MIDI file using the specified model path. ```bash omnizart music transcribe [OPTIONS] INPUT_AUDIO ``` -------------------------------- ### BaseStructure Class Method: download Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Class method to download the dataset. It handles decompression and deletion of the zipped file. Post-download processing can be implemented by overriding `_post_download`. ```python @classmethod def download(cls, _save_path ='./'): """Download the dataset. After download the compressed (zipped) dataset file, the function will automatically decompress it and delete the original zipped file. You can apply some post process after download by overriding the function `_post_download`. The _post_download function receives a single `dataset_path` as the parameter, and you can do anything to the dataset such as re-organize the directory structure, or filter out some files. """ pass ``` -------------------------------- ### Transcribe audio with omnizart beat Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/cli.html Use this command to transcribe a single audio file into MIDI and CSV formats. ```bash omnizart beat transcribe [OPTIONS] INPUT_AUDIO ``` -------------------------------- ### Find Note Onset and Offset Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/music/api.html Finds the onset and offset of a thresholded prediction. ```APIDOC ## omnizart.music.inference.find_occur ### Description Find the onset and offset of a thresholded prediction. ### Parameters - **pitch** (1D numpy array) - Time series of predicted pitch activations. - **t_unit** (float) - Time unit of each entry. Default: 0.02 - **min_duration** (float) - Minimum interval of each note in seconds. Default: 0.03 ``` -------------------------------- ### Omnizart Models - U-Net Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/models.html This section details the U-Net model architecture and its associated components within the Omnizart library. ```APIDOC ## Class omnizart.models.u_net.MultiHeadAttention ### Description Attention layer for 2D input feature. It uses a divide-and-conquer approach to handle large input features by partitioning them before self-attention computation. ### Method N/A (Class Definition) ### Endpoint N/A (Class Definition) ### Parameters #### Class Parameters - **out_channel** (int) - Number of output channels. - **d_model** (int) - Dimension of embeddings for each position of input feature. - **n_heads** (int) - Number of heads for multi-head attention computation. Should be a divisor of d_model. - **query_shape** (Tuple[int, int]) - Size of each partition. - **memory_flange** (Tuple[int, int]) - Overlapping size to be extended to each partition. ### Methods #### call(inputs) - **Description**: This is where the layer's logic lives. Processes the input tensor(s) using the attention mechanism. - **Args**: - inputs: Input tensor, or list/tuple of input tensors. - **Returns**: A tensor or list/tuple of tensors. #### get_config() - **Description**: Returns the configuration of the layer as a Python dictionary. - **Returns**: Python dictionary representing the layer's configuration. ``` ```APIDOC ## Function omnizart.models.u_net.conv_block ### Description Convolutional encoder block of U-net. This block is fully convolutional and does not downsample the input feature, maintaining the input dimension. ### Method N/A (Function Definition) ### Endpoint N/A (Function Definition) ### Parameters - **input_tensor** (Tensor) - The input tensor to the convolutional block. - **channel** (int) - The number of output channels. - **kernel_size** (Tuple[int, int]) - The size of the convolutional kernel. - **strides** (Tuple[int, int]) - The strides for the convolution. Defaults to (2, 2). - **dilation_rate** (int) - The dilation rate for the convolution. Defaults to 1. - **dropout_rate** (float) - The dropout rate to apply. Defaults to 0.4. ``` ```APIDOC ## Function omnizart.models.u_net.semantic_segmentation ### Description Improved U-net model with an Atrous Spatial Pyramid Pooling (ASPP) block. ### Method N/A (Function Definition) ### Endpoint N/A (Function Definition) ### Parameters - **feature_num** (int) - Number of features. Defaults to 352. - **timesteps** (int) - Number of time steps. Defaults to 256. - **multi_grid_layer_n** (int) - Number of multi-grid layers. Defaults to 1. - **multi_grid_n** (int) - Multi-grid factor. Defaults to 5. - **ch_num** (int) - Number of channels. Defaults to 1. - **out_class** (int) - Number of output classes. Defaults to 2. - **dropout** (float) - Dropout rate. Defaults to 0.4. ``` ```APIDOC ## Function omnizart.models.u_net.semantic_segmentation_attn ### Description Customized attention U-net model. ### Method N/A (Function Definition) ### Endpoint N/A (Function Definition) ### Parameters - **feature_num** (int) - Number of features. Defaults to 352. - **timesteps** (int) - Number of time steps. Defaults to 256. - **ch_num** (int) - Number of channels. Defaults to 1. - **out_class** (int) - Number of output classes. Defaults to 2. ``` ```APIDOC ## Function omnizart.models.u_net.transpose_conv_block ### Description Transposed convolutional block for the U-net architecture. ### Method N/A (Function Definition) ### Endpoint N/A (Function Definition) ### Parameters - **input_tensor** (Tensor) - The input tensor to the transposed convolutional block. - **channel** (int) - The number of output channels. - **kernel_size** (Tuple[int, int]) - The size of the transposed convolutional kernel. - **strides** (Tuple[int, int]) - The strides for the transposed convolution. Defaults to (2, 2). - **dropout_rate** (float) - The dropout rate to apply. Defaults to 0.4. ``` -------------------------------- ### Accessing Drum Transcription Features Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/drum/api.html Demonstrates how to open and inspect a drum transcription feature file stored in HDF5 format. ```python >>> import h5py >>> hdf_ref = h5py.File("ytd_audio_00001_TRBSAIC128E0793CCE.hdf", "r") >>> hdf_ref.keys() >>> feature = hdf_ref["feature"][:] >>> print(feature.shape) (2299, 120, 120) >>> hdf_ref.close() ``` -------------------------------- ### Train Drum Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/drum/cli.html Trains a new drum model or continues training on a pre-trained model. Requires the feature path and allows specifying model name, epochs, steps, batch sizes, and early stopping criteria. ```bash omnizart drum train-model [OPTIONS] ``` -------------------------------- ### CLI: omnizart drum transcribe Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/drum/cli.html Transcribes a single audio file into a MIDI file using a pre-trained model. ```APIDOC ## CLI: omnizart drum transcribe ### Description Transcribe a single audio and output as a MIDI file. This will output a MIDI file with the same name as the given audio, except the extension will be replaced with ‘.mid’. ### Endpoint omnizart drum transcribe [OPTIONS] INPUT_AUDIO ### Parameters #### Arguments - **INPUT_AUDIO** (string) - Required - Path to the input audio file. #### Options - **-m, --model-path** (string) - Required - Path to the pre-trained model or the supported transcription mode. - **-o, --output** (string) - Optional - Path to output the prediction file. Default: ./ ``` -------------------------------- ### Generate Features for Drum Application Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/tutorial.html Command to generate necessary features for training and testing the drum application. Requires specifying the path to the dataset. ```bash # Generate features for the drum application omnizart drum generate-feature --dataset-path ``` -------------------------------- ### omnizart.train.gen_bar_postfix Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/training.html Generates a string of metrics status to be appended to the end of the training progress bar. ```APIDOC ## omnizart.train.gen_bar_postfix ### Description Formats metric history into a string for display in the progress bar. ### Parameters - **history** (dict) - Required - History records generated by train_steps. - **targets** (list[str]) - Optional - List of metric names to extract. - **name_transform** (list[str]) - Optional - Alias names for the metrics. ### Response - **postfix** (str) - The formatted metrics information string. ``` -------------------------------- ### Train Chord Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/cli.html Trains a new chord model or fine-tunes an existing one. Requires the feature path and allows specifying model name, epochs, steps, batch size, and learning rate decay. ```bash omnizart chord train-model [OPTIONS] ``` -------------------------------- ### Train Beat Transcription Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/api.html Train a beat tracking model from scratch or fine-tune from a checkpoint. Requires a path to the generated feature data. ```python train(_feature_folder_ , _model_name =None_, _input_model_path =None_, _beat_settings =None_) ``` -------------------------------- ### BaseLabelExtraction API Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/music/api.html Provides basic functions to process native label format into the format required by the `music` module. All sub-classes should parse the original label information into `Label` class. ```APIDOC ## Class: BaseLabelExtraction ### Description Base class for extracting label information. Provides basic functions to process native label format into the format required by `music` module. All sub-classes should parse the original label information into `Label` class. See also: `omnizart.music.labels.label_conversion` ### Methods #### `extract_label(label_path, t_unit, onset_len_sec=0.03)` ##### Description Extract labels into customized storage format. Process the given path of label into list of `Label` instances, then further convert them into deliberately customized storage format. ##### Parameters - **label_path** (Path) - Required - Path to the label file. - **t_unit** (float) - Required - Time unit of each step in seconds. Should be consistent with the time unit of each frame of the extracted feature. - **onset_len_sec** (float) - Optional - Length of the first few frames with probability one. The later onset probabilities will be in a ‘fade-out’ manner until the note offset. #### `load_label(label_path)` ##### Description Load the label file and parse information into `Label` class. Sub-classes should override this function to process their own label format. ##### Parameters - **label_path** (Path) - Required - Path to the label file. ##### Returns - **labels** (list[Label]) - List of `Label` instances. #### `name_transform(name)` ##### Description Maps the filename of label to the same name of the corresponding wav file. ##### Parameters - **name** (str) - Required - Name of the label file, without parent directory prefix and file extension. ##### Returns - **trans_name** (str) - The name same as the corresponding wav (or says feature) file. #### `process(label_list, out_path, t_unit=0.02, onset_len_sec=0.03)` ##### Description Process the given list of label files and output to the target folder. ##### Parameters - **label_list** (list[Path]) - Required - List of label paths. - **out_path** (Path) - Required - Path for saving the extracted label files. - **t_unit** (float) - Optional - Time unit of each step in seconds. Should be consistent with the time unit of each frame of the extracted feature. - **onset_len_sec** (float) - Optional - Length of the first few frames with probability one. The later onset probabilities will be in a ‘fade-out’ manner until the note offset. ``` -------------------------------- ### BaseStructure Class Method: get_test_labels Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Retrieves a list of complete file paths for all testing labels. ```python @classmethod def get_test_labels(cls, _dataset_path_): """Get list of complete test label paths""" pass ``` -------------------------------- ### Train a beat transcription model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/cli.html Train a new model or fine-tune an existing one using extracted features. ```bash omnizart beat train-model [OPTIONS] ``` -------------------------------- ### BaseStructure Class Method: load_label Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Loads and parses label files from a given path. It converts various label formats into a shared intermediate format using `Label` instances. Defaults to parsing MIDI files. ```python @classmethod def load_label(cls, _label_path_): """Load and parse labels for the given label file path. Parses different format of label information to shared intermediate format, encapslated with `Label` instances. The default is parsing MIDI file format. """ pass ``` -------------------------------- ### Dataset Path Retrieval Methods Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Methods for retrieving lists of training and testing file paths for datasets. ```APIDOC ## GET /dataset/paths ### Description Retrieves lists of complete file paths for training or testing wavs and labels. ### Parameters #### Path Parameters - **dataset_path** (string) - Required - The root path of the dataset. ### Methods - **get_train_wavs(dataset_path)**: Get list of complete train wav paths. - **get_test_wavs(dataset_path)**: Get list of complete test wav paths. - **get_train_labels(dataset_path)**: Get list of complete train label paths. - **get_test_labels(dataset_path)**: Get list of complete test label paths. ``` -------------------------------- ### transcribe Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/vocal/api.html Transcribes vocal notes (onset, offset, pitch) from an input audio file and outputs a MIDI file. ```APIDOC ## Method: transcribe ### Description Transcribe vocal notes in the audio. This function transcribes onset, offset, and pitch of the vocal in the audio. This module is reponsible for predicting onset and offset time of each note, and pitches are estimated by the vocal-contour submodule. ### Parameters - **input_audio** (Path) - Required - Path to the raw audio file (.wav). - **model_path** (Path) - Optional - Path to the trained model or the supported transcription mode. - **output** (Path) - Optional - Path for writing out the transcribed MIDI file. Default to the current path. ### Returns - **midi** (pretty_midi.PrettyMIDI) - The transcribed vocal notes. ``` -------------------------------- ### Train PatchCNN Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/patch-cnn/api.html Trains a PatchCNN model from scratch or continues training from a checkpoint. Specify the feature folder and optionally a model name or input model path for fine-tuning. ```python train(_feature_folder_ , _model_name =None_, _input_model_path =None_, _patch_cnn_settings =None_) ``` -------------------------------- ### Train Music Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/music/cli.html Trains a new model or fine-tunes an existing one using extracted features. ```bash omnizart music train-model [OPTIONS] ``` -------------------------------- ### Transcribe Chord Progression Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/cli.html Transcribes a single audio file and outputs MIDI and CSV files. Specify the model path and output directory. ```bash omnizart chord transcribe [OPTIONS] INPUT_AUDIO ``` -------------------------------- ### Fetch Harmonic from CFP Data Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/feature.html Retrieves harmonic information from CFP data, useful for HCFP calculations. ```python omnizart.feature.hcfp.fetch_harmonic(_data_ , _cenf_ , _ith_har_ , _start_freq =27.5_, _num_per_octave =48_, _is_reverse =False_) ``` -------------------------------- ### Music Loss Functions Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/music/api.html Loss functions for the Omnizart music module. ```APIDOC ## focal_loss ### Description Compute focal loss for predictions. Multi-labels Focal loss formula: FL=−α∗(z−p)γ∗log⁡(p)−(1−α)∗pγ∗log⁡(1−p) Which α = 0.25, γ = 2, p = sigmoid(x), z = target_tensor. ### Method N/A (Function) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response #### Success Response (200) - **loss** (scalar tensor) - A scalar tensor representing the value of the loss function #### Response Example N/A ## smooth_loss ### Description Function to compute loss after applying **label-smoothing**. ### Method N/A (Function) ### Endpoint N/A ### Parameters #### Path Parameters N/A #### Query Parameters N/A #### Request Body N/A ### Request Example N/A ### Response N/A ``` -------------------------------- ### MusicDatasetLoader Class Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/music/api.html API documentation for the MusicDatasetLoader class used to load and prepare music training data. ```APIDOC ## Class: omnizart.music.app.MusicDatasetLoader ### Description Data loader for training the music model. It loads features and labels for training and converts custom label formats into piano roll representations. ### Parameters - **label_conversion_func** (callable) - Required - The function used for converting the customized label format into a numpy array. - **feature_folder** (Path) - Optional - Path to the extracted feature files, including *.hdf and *.pickle pairs. - **feature_files** (list[Path]) - Optional - List of paths to *.hdf feature files. - **num_samples** (int) - Optional - Total number of samples to yield (default: 100). - **timesteps** (int) - Optional - Time length of the feature (default: 128). - **channels** (list[int]) - Optional - Channels to be used for training. Allowed values are [1, 2, 3] (default: [1, 3]). - **feature_num** (int) - Optional - Target size of feature dimension (default: 352). ### Yields - **feature** (numpy array) - Input features for model training. - **label** (numpy array) - Corresponding labels. ``` -------------------------------- ### Generate features for training Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/cli.html Extract features from a dataset to prepare for model training. ```bash omnizart beat generate-feature [OPTIONS] ``` -------------------------------- ### CLI: omnizart drum train-model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/drum/cli.html Trains a new model or fine-tunes an existing pre-trained model. ```APIDOC ## CLI: omnizart drum train-model ### Description Train a new model or continue to train on a pre-trained model. ### Endpoint omnizart drum train-model [OPTIONS] ### Parameters #### Options - **-d, --feature-path** (string) - Required - Path to the folder of extracted feature. - **-m, --model-name** (string) - Optional - Name for the output model. - **-i, --input-model** (string) - Optional - Path to a pre-trained model for fine-tuning. - **-e, --epochs** (integer) - Optional - Number of training epochs. - **-s, --steps** (integer) - Optional - Number of training steps of each epoch. - **-vs, --val-steps** (integer) - Optional - Number of validation steps of each epoch. - **-b, --batch-size** (integer) - Optional - Batch size of each training step. - **-vb, --val-batch-size** (integer) - Optional - Batch size of each validation step. - **--early-stop** (integer) - Optional - Stop the training if validation accuracy does not improve over the given number of epochs. ``` -------------------------------- ### omnizart.train.train_epochs Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/training.html Executes the main training loop with event-based lifecycle management for callbacks. ```APIDOC ## omnizart.train.train_epochs ### Description Logic of the main training loop. It manages the training lifecycle, triggering events for callbacks at various stages (e.g., on_train_begin, on_epoch_begin). ### Parameters - **model** (tf.keras.Model) - Required - Compiled tensorflow keras model. - **train_dataset** (tf.data.Dataset) - Required - The dataset instance for training. - **validate_dataset** (tf.data.Dataset) - Optional - The dataset instance for validation. - **epochs** (int) - Optional - Number of maximum training epochs. - **steps** (int) - Optional - Number of training steps for each epoch. - **val_steps** (int) - Optional - Number of validation steps for each epoch. - **callbacks** (list) - Optional - List of callback instances. ### Response - **history** (dict) - Score history of each metrics during each epoch of both training and validation. ``` -------------------------------- ### Train Chord Transcription Model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/api.html Trains a new chord transcription model or continues training on a pre-trained model. Requires a folder with extracted features and optional settings. ```python chord_transcription.train(feature_folder='path/to/features', model_name='my_chord_model') ``` -------------------------------- ### Beat Transcription API Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/beat/api.html The BeatTranscription class provides an interface for beat tracking in MIDI files. ```APIDOC ## Beat Transcription API ### Description This section details the Beat Transcription module, focusing on the `BeatTranscription` class and its associated methods for processing MIDI data to track beats and downbeats. ### Class: BeatTranscription ```python class omnizart.beat.app.BeatTranscription(_conf_path=None) ``` Bases: `omnizart.base.BaseTranscription` Application class for beat tracking in MIDI domain. #### Methods - **`generate_feature(dataset_path, beat_settings=None, num_threads=8)`** Extract the feature from the given dataset. To train the model, the first step is to pre-process the data into feature representations. After downloading the dataset, use this function to generate the feature by giving the path of the stored dataset. To specify the output path, modify the attribute `beat_settings.dataset.feature_save_path`. It defaults to the folder under where the dataset stored, generating two folders: `train_feature` and `test_feature`. **Parameters:** - **dataset_path** (Path) - Required - Path to the downloaded dataset. - **beat_settings** (BeatSettings) - Optional - The configuration instance that holds all relative settings for the life-cycle of building a model. - **num_threads** (int) - Optional - Number of threads for parallel extraction the feature. - **`train(feature_folder, model_name=None, input_model_path=None, beat_settings=None)`** Model training. Train the model from scratch or continue training given a model checkpoint. **Parameters:** - **feature_folder** (Path) - Required - Path to the generated feature. - **model_name** (str) - Optional - The name of the trained model. If not given, will default to the current timestamp. - **input_model_path** (Path) - Optional - Specify the path to the model checkpoint in order to fine-tune the model. - **beat_settings** (BeatSettings) - Optional - The configuration that holds all relative settings for the life-cycle of model building. - **`transcribe(input_audio, model_path=None, output='./')`** Transcribe beat positions in the given MIDI. Tracks the beat in symbolic domain. Outputs three files if the output path is given: _< filename>.mid_, _beat.csv, and _down_beat.csv, where _filename_ is the name of the input MIDI without extension. The *.csv files records the beat positions in seconds. **Parameters:** - **input_audio** (Path) - Required - Path to the MIDI file (.mid). - **model_path** (Path) - Optional - Path to the trained model or the supported transcription mode. - **output** (Path) - Optional - Path for writing out the transcribed MIDI file. Default to the current path. **Returns:** - midi (pretty_midi.PrettyMIDI) - The transcribed beat positions. There are two types of beat: beat and down beat. Each are recorded in independent instrument track. ### See Also - `omnizart.cli.beat.transcribe` CLI entry point of this function. ### Feature Storage Format Processed feature will be stored in `.hdf` format, one file per piece. Columns in the file are: - **feature** (any) - Piano roll like representation with mixed information. - **label** (any) - ### References 1. https://github.com/chuang76/symbolic-beat-tracking ``` -------------------------------- ### BaseStructure Class Method: get_test_data_pair Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Retrieves pairs of testing audio file paths and their corresponding label file paths. ```python @classmethod def get_test_data_pair(cls, _dataset_path_): """Get pair of testing file and the coressponding label file path.""" pass ``` -------------------------------- ### Byte Formatting Utility Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/utils.html Utility function to format byte sizes into human-readable strings. ```APIDOC ## Format Byte Size ### Description Format the given byte size into human-readable string. ### Method `omnizart.remote.format_byte` ### Parameters - **size** (int) - Required - The size in bytes. - **digit** (int) - Optional - The number of digits to display after the decimal point. Defaults to 2. ### Request Example ```python import omnizart.remote # Example usage # formatted_size = omnizart.remote.format_byte(1048576) # print(formatted_size) # Output: '1.00MB' ``` ### Response #### Success Response (200) - **formatted_size** (str) - The human-readable string representation of the byte size. #### Response Example ```json { "formatted_size": "1.00MB" } ``` ``` -------------------------------- ### omnizart.callbacks.Callback Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/training.html Base class for all callback classes in the Omnizart framework. ```APIDOC ## Class: omnizart.callbacks.Callback ### Description Base class of all callback classes. Provides hooks for training and testing lifecycle events. ### Methods - on_epoch_begin - on_epoch_end - on_test_batch_begin - on_test_batch_end - on_train_batch_begin - on_train_batch_end - on_train_begin - on_train_end ``` -------------------------------- ### Generate Chord Features Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/chord/cli.html Extracts features from a dataset for model training. Requires the dataset path and allows specifying the output path and number of threads. ```bash omnizart chord generate-feature [OPTIONS] ``` -------------------------------- ### BaseLabelExtraction Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/vocal/api.html Base class for extracting label information. Sub-classes should override `load_label`. ```APIDOC ## Class omnizart.vocal.labels.BaseLabelExtraction ### Description Base class for extract label information. Provides basic functions to parse the original label format into the target format for training. All sub-classes should override the function `load_label` and returns a list of `Label` objects. ### Methods - `extract_label(label_path, t_unit=0.02)`: Extract SDT label. - `load_label(label_path)`: Load the label file and parse information into `Label` class. ## Classmethod omnizart.vocal.labels.BaseLabelExtraction.extract_label ### Description Extract SDT label. There are 6 types of events as defined in the original paper: activation, silence, onset, non-onset, offset, and non-offset. The corresponding annotations used in the paper are [a, s, o, o’, f, f’]. The ‘activation’ includes the onset and offset time. And non-onset and non-offset events refer to when there are no onset/offset events. ### Parameters - **label_path** (Path) - Required - Path to the groun-truth file. - **t_unit** (float) - Optional - Time unit of each frame. ### Returns - **sdt_label** (2D numpy array) - Label in SDT format with dimension: Time x 6 ## Abstract Classmethod omnizart.vocal.labels.BaseLabelExtraction.load_label ### Description Load the label file and parse information into `Label` class. Sub-classes should override this function to process their own label format. ### Parameters - **label_path** (Path) - Required - Path to the label file. ### Returns - **labels** (list[Label]) - List of `Label` instances. ``` -------------------------------- ### BaseStructure Class Method: get_train_labels Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Retrieves a list of complete file paths for all training labels. ```python @classmethod def get_train_labels(cls, _dataset_path_): """Get list of complete train label paths""" pass ``` -------------------------------- ### Train a Patch-CNN model Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/patch-cnn/cli.html Trains a new model or fine-tunes an existing one. Requires the path to extracted features. ```bash omnizart patch-cnn train-model [OPTIONS] ``` -------------------------------- ### MedleyDB Label Loading Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/constants.html Method for loading and parsing label files for the MedleyDB dataset. ```APIDOC ## POST /medleydb/load_label ### Description Load and parse labels for the given label file path. Parses different formats of label information to a shared intermediate format, encapsulated with Label instances. ### Parameters #### Request Body - **label_path** (string) - Required - The file path to the label file to be parsed. ``` -------------------------------- ### omnizart.utils.ensure_path_exists Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/utils.html Ensures that a specified directory path exists, creating it if necessary. ```APIDOC ## ensure_path_exists Function ### Description Ensures that a specified directory path exists, creating it if necessary. ### Parameters - **path** (str) - The path to ensure exists. ``` -------------------------------- ### omnizart.models.chord_model.chord_block_compression Source: https://music-and-culture-technology-lab.github.io/omnizart-doc/models.html Compresses chord sequences based on hidden states and chord changes. ```APIDOC ## chord_block_compression ### Description Compresses a sequence of chord blocks. ### Parameters - **hidden_states** (Tensor) - Required - The model hidden states. - **chord_changes** (List/Tensor) - Required - The detected chord changes. ```