### Install and Test Omnizart Locally

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/CONTRIBUTING.md

Instructions for installing the project dependencies in a virtual environment using Make and verifying the installation with the help command.

```bash
make install
omnizart --help
deactivate
```

--------------------------------

### Setup Omnizart Environment

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/colab.ipynb

Installs necessary dependencies including Python 3.8, pip, and Omnizart itself. It also downloads pre-trained models and yt-dlp for audio processing.

```python
#@title Environment Setup

!sudo apt-get install python3.8 python3.8-distutils python3.8-dev
!sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!sudo apt-get install python3-pip
!pip install numpy Cython
!sudo apt-get install libsndfile-dev fluidsynth ffmpeg

!pip install git+https://github.com/Music-and-Culture-Technology-Lab/omnizart.git
!omnizart download-checkpoints
!pip install yt-dlp numpy==1.23.5

import sys
sys.path.insert(0, '/usr/local/lib/python3.8/dist-packages')
```

--------------------------------

### Installation

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Instructions for installing Omnizart and its dependencies, including downloading pre-trained model checkpoints.

```APIDOC
## Installation

Install Omnizart using pip and download the required pre-trained model checkpoints.

```bash
# Install prerequisites
pip install numpy Cython

# Install system dependencies (Ubuntu/Debian)
sudo apt-get install libsndfile-dev fluidsynth ffmpeg

# Install Omnizart
pip install omnizart

# Download pre-trained model checkpoints
omnizart download-checkpoints
```
```

--------------------------------

### Install and Run Omnizart via Pip

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/README.md

Instructions for installing the Omnizart library, downloading necessary model checkpoints, and executing transcription commands for drums, chords, and general music.

```bash
pip install omnizart
omnizart download-checkpoints
omnizart drum transcribe <path/to/audio.wav>
omnizart chord transcribe <path/to/audio.wav>
omnizart music transcribe <path/to/audio.wav>
```

--------------------------------

### Install Omnizart from Source via Conda

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/README.md

Steps to clone the repository, set up a dedicated Conda environment using the provided environment file, and install the package locally.

```bash
git clone https://github.com/Music-and-Culture-Technology-Lab/omnizart
cd omnizart
conda env create -f environment.yml
conda activate omnizart
pip install .
omnizart download-checkpoints
```

--------------------------------

### Install Omnizart Project Dependencies (Bash)

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/CONTRIBUTING.md

This snippet details the steps to clone the Omnizart repository and install its dependencies using make commands. It includes commands for both regular and developer installations, ensuring all necessary packages and development tools are set up.

```bash
git clone https://github.com/Music-and-Culture-Technology-Lab/omnizart.git

# Install dependenies. For more different installation approaches, please refer to the official documentation page.
# The following command will download the checkpoints automatically.
cd omnizart
make install

# For developers, you have to install Dev dependencies as well, since they will not be installed by default.
make install-dev
```

--------------------------------

### Configure Music Model Settings with YAML

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/music/api.md

Demonstrates the default settings for building a music model using YAML configuration files. These settings are loaded by `omnizart.setting_loaders.MusicSettings`, with attribute names converted to snake-case. The example shows how settings are accessed and transformed within the `MusicSettings` instance, illustrating path transformations and attribute mapping.

```yaml
# Self-documented configurable settings, with description, type hint, and available
# options. All the parameters can be overriden by another specified configuration file 

```

--------------------------------

### Install Omnizart and Dependencies

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Installs necessary Python packages, system dependencies for Ubuntu/Debian, and the Omnizart library. It also includes a command to download pre-trained model checkpoints required for transcription.

```bash
pip install numpy Cython

sudo apt-get install libsndfile-dev fluidsynth ffmpeg

pip install omnizart

omnizart download-checkpoints
```

--------------------------------

### Transcribe Audio with Omnizart

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/tutorial.md

Example of transcribing percussive events from a WAV file using a custom model path and output file.

```bash
omnizart drum transcribe pop.wav --model-path ./my-model --output ./trans_pop.mid
```

--------------------------------

### Run Omnizart with Docker

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/README.md

Commands to pull the official Omnizart Docker image and start an interactive session within the container.

```bash
docker pull mctlab/omnizart:latest
docker run -it mctlab/omnizart:latest bash
```

--------------------------------

### GET /vamp/nnls-chroma/chordino

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/omnizart/resource/vamp/README.md

Chord extraction plugin based on the NNLS Chroma analysis framework.

```APIDOC
## GET /vamp/nnls-chroma/chordino

### Description
Provides chord extraction functionality based on the underlying NNLS Chroma spectral analysis.

### Method
GET

### Endpoint
vamp:nnls-chroma:chordino

### Response
#### Success Response (200)
- **data** (object) - Returns extracted chord labels and timing information.
```

--------------------------------

### Train Music Transcription Models via CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Demonstrates how to initiate training or resume from a checkpoint for music transcription models using the Omnizart command-line interface.

```bash
omnizart music train-model -d /data/maestro/train_feature --model-name my-piano-model
omnizart music train-model -d /data/maestro/train_feature --model-name my-piano-v2 --input-model /path/to/checkpoint
```

--------------------------------

### Generate Documentation with Sphinx (Bash)

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/CONTRIBUTING.md

Instructions for generating project documentation using Sphinx. It involves navigating to the docs directory and executing make commands to build and serve the HTML documentation locally. The documentation follows the numpy style and uses reStructuredText format.

```bash
cd docs/
make html

# To see the rendered results, run `make serve` and view from the browser.
```

--------------------------------

### Initialize and Load Waveform Playlist (JavaScript)

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/_static/demo-cosmo.html

Initializes the Waveform Playlist with a specified container element and control options, then loads a list of audio tracks. The `init` function takes a configuration object, and the `load` method accepts an array of track objects, each defining `src`, `name`, and `gain`.

```javascript
var playlist = WaveformPlaylist.init({
  container: document.getElementById("playlist"),
  controls: {
    show: true,
    width: 180
  }
});

playlist.load([
  {
    "src": "../\_audio/cosmo.mp3",
    "name": "Original Song",
    "gain": 0.5
  },
  {
    "src": "../\_audio/cosmo_vocal.mp3",
    "name": "Vocal",
    "gain": 1
  },
  {
    "src": "../\_audio/cosmo_vocal_contour.mp3",
    "name": "Vocal Contour",
    "gain": 1
  },
  {
    "src": "../\_audio/cosmo_chord.mp3",
    "name": "Chord",
    "gain": 1
  },
  {
    "src": "../\_audio/cosmo_drum.mp3",
    "name": "Drum",
    "gain": 1
  },
  {
    "src": "../\_audio/cosmo_piano.mp3",
    "name": "Piano (?)",
    "gain": 1
  }
])
```

--------------------------------

### GET /vamp/nnls-chroma/nnls-chroma

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/omnizart/resource/vamp/README.md

Analyzes audio using frame-wise spectral input to extract treble and bass chromagrams. It uses constant-Q transforms and NNLS approximate transcription.

```APIDOC
## GET /vamp/nnls-chroma/nnls-chroma

### Description
Analyzes a single channel of audio to extract chromagrams using NNLS approximate transcription and constant-Q spectral transformation.

### Method
GET

### Endpoint
vamp:nnls-chroma:nnls-chroma

### Parameters
#### Query Parameters
- **use_nnls** (boolean) - Optional - Toggle NNLS approximate transcription (default: on)
- **spectral_roll_on** (float) - Optional - Cumulative energy spectrum threshold (0% - 5%, default: 0%)
- **tuning_mode** (string) - Optional - Tuning estimation mode: 'global' or 'local' (default: global)
- **spectral_whitening** (float) - Optional - Whitening factor (0.0 - 1.0, default: 1.0)
- **spectral_shape** (float) - Optional - Harmonic decay shape (0.5 - 0.9, default: 0.7)
- **chroma_normalisation** (string) - Optional - Normalization method: 'none', 'maximum', 'L1', 'L2' (default: none)

### Response
#### Success Response (200)
- **outputs** (object) - Returns Log-frequency Spectrum, Tuned Log-frequency Spectrum, Semitone Spectrum, Bass Chromagram, Chromagram, and Consonance estimate.
```

--------------------------------

### Build and Run Omnizart Docker Containers

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/CONTRIBUTING.md

Commands to build a local Docker image and execute the pre-built image from Docker Hub. Includes instructions for enabling GPU acceleration using the NVIDIA container toolkit.

```bash
docker build -t omnizart:my-image .
docker pull mctlab/omnizart
docker run -it mctlab/omnizart:latest
docker run --gpus all -it mctlab/omnizart:latest
```

--------------------------------

### Initialize and Load Waveform Playlist

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/_static/demo-collage.html

Initializes a Waveform Playlist with specified container and control options, then loads audio tracks with their source, name, and gain. Dependencies include the Waveform Playlist library. Input is a JSON object defining playlist configuration and an array of track objects. Limitations may include browser compatibility and audio file format support.

```javascript
var playlist = WaveformPlaylist.init({
  container: document.getElementById("playlist"),
  controls: {
    show: true,
    width: 180
  }
});

playlist.load([
  {
    "src": "../\_audio/collage.mp3",
    "name": "Original Song",
    "gain": 0.5
  },
  {
    "src": "../\_audio/collage_vocal_contour.mp3",
    "name": "Vocal Contour",
    "gain": 1
  },
  {
    "src": "../\_audio/collage_vocal.mp3",
    "name": "Vocal",
    "gain": 1
  }
]);
```

--------------------------------

### Vocal Contour Transcription (Frame-Level) - Python API

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Extract detailed frame-level vocal pitch information using the VocalContourTranscription class in Python. This API returns aggregated F0 information, including start time, end time, and frequency in Hz for each frame.

```python
from omnizart.vocal_contour import VocalContourTranscription

# Initialize vocal contour transcription
app = VocalContourTranscription()

# Transcribe frame-level F0
# Returns aggregated F0 information
agg_f0 = app.transcribe(
    input_audio="vocal_track.wav",
    model_path=None,  # Uses default VocalContour model
    output="./"
)

# agg_f0 contains pitch information:
# Each entry: (start_time, end_time, frequency_hz)
for entry in agg_f0[:5]:
    print(f"F0: {entry[2]:.1f} Hz from {entry[0]:.3f}s to {entry[1]:.3f}s")
```

--------------------------------

### Download Datasets - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Download various supported datasets for training custom models using the command-line interface. Options include specifying the dataset name, output directory, and whether to automatically unzip the downloaded files.

```bash
# Download MAESTRO dataset (piano)
omnizart download-dataset Maestro --output /data/maestro

# Download MusicNet dataset with automatic extraction
omnizart download-dataset MusicNet --output /data --unzip

# Download MIR-1K dataset (vocal)
omnizart download-dataset MIR-1K --output /data/mir1k

# Download McGill BillBoard dataset (chord)
omnizart download-dataset McGill --output /data/mcgill

# List all available datasets
omnizart download-dataset --help
```

--------------------------------

### Initialize and Load Waveform Playlist

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/_static/demo-dreamshade.html

Initializes the WaveformPlaylist instance with control settings and loads an array of audio track objects. Each track object includes the source path, display name, and gain level.

```javascript
var playlist = WaveformPlaylist.init({
  container: document.getElementById("playlist"),
  controls: {
    show: true,
    width: 180
  }
});

playlist.load([
  { "src": "../_audio/dreamshade.mp3", "name": "Original Song", "gain": 0.5 },
  { "src": "../_audio/dreamshade_vocal.mp3", "name": "Vocal", "gain": 1 },
  { "src": "../_audio/dreamshade_vocal_contour.mp3", "name": "Vocal Contour", "gain": 1 },
  { "src": "../_audio/dreamshade_chord.mp3", "name": "Chord", "gain": 1 },
  { "src": "../_audio/dreamshade_drum.mp3", "name": "Drum", "gain": 1 }
]);
```

--------------------------------

### Train Omnizart Models

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/tutorial.md

Commands to train new models from scratch using previously generated feature folders.

```bash
omnizart music train-model -d <path/to/feature/folder> --model-name My-Music
omnizart drum train-model -d <path/to/feature/folder> --model-name My-Drum
omnizart chord train-model -d <path/to/feature/folder> --model-name My-Chord
```

--------------------------------

### Omnizart CLI Command Structure

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/tutorial.md

General syntax for executing core sub-commands in the Omnizart CLI.

```bash
omnizart application action --arguments
```

--------------------------------

### Run Omnizart using Docker

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

This section provides bash commands for pulling the Omnizart Docker image, running an interactive session, mounting local directories for data persistence, and executing transcription tasks within the container.

```bash
# Pull the official Docker image
docker pull mctlab/omnizart:latest

# Run interactive session
docker run -it mctlab/omnizart:latest bash

# Mount local directory for input/output
docker run -it -v /local/audio:/data mctlab/omnizart:latest bash

# Inside container:
# omnizart music transcribe /data/song.wav --output /data/
```

--------------------------------

### Initialize Waveform Playlist

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/_static/demo-surrealist.html

Initializes the Waveform Playlist with specified container and control options. It allows for customization of track controls, including their visibility and width. Dependencies include the Waveform Playlist library.

```javascript
var playlist = WaveformPlaylist.init({
  container: document.getElementById("playlist"),
  controls: {
    show: true, //whether or not to include the track controls
    width: 180 //width of controls in pixels
  }
});
```

--------------------------------

### Download and Convert YouTube Audio to WAV

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/colab.ipynb

Takes a YouTube URL, downloads the audio as MP3 using yt-dlp, and converts it to WAV format for transcription. It also displays the YouTube video.

```python
#@title Choose from YouTube
import os
from google.colab import files
from IPython import display as dsp

url = input("Enter your YouTube link: ")

try:
  id = url.split("watch?v=")[1].split("&")[0]
  vid = dsp.YouTubeVideo(id)
  dsp.display(vid)
except Exception:
  pass

print("Downloading...")

!yt-dlp -x --audio-format mp3 --no-playlist "$url"
!yt-dlp --get-filename --no-playlist "$url" > tmp

uploaded_audio = os.path.splitext(open("tmp").readline().strip())[0]
!ffmpeg -i "$uploaded_audio".mp3 "$uploaded_audio".wav &> /dev/null

print(f"Finished: {uploaded_audio}")
```

--------------------------------

### Configure Omnizart Music Settings in Python

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

This Python snippet demonstrates how to configure various settings for Omnizart's music transcription, including feature extraction, inference thresholds, training parameters, and model types. It also shows how to save and load these settings from a YAML file.

```python
from omnizart.setting_loaders import (
    MusicSettings,
    DrumSettings,
    ChordSettings,
    VocalSettings,
    BeatSettings
)

# Music settings example
music_settings = MusicSettings()

# Feature extraction settings
music_settings.feature.hop_size = 0.02          # 20ms hop
music_settings.feature.sampling_rate = 44100
music_settings.feature.bins_per_octave = 48
music_settings.feature.harmonic = False

# Inference settings (thresholds)
music_settings.inference.onset_th = 3.5         # Onset threshold
music_settings.inference.dura_th = 0.5          # Duration threshold
music_settings.inference.frame_th = 0.5         # Frame threshold
music_settings.inference.min_length = 0.05      # Minimum note length

# Training settings
music_settings.training.epoch = 20
music_settings.training.steps = 3000
music_settings.training.batch_size = 8
music_settings.training.early_stop = 6
music_settings.training.label_type = "note-stream"

# Model settings
music_settings.model.model_type = "attn"        # Options: aspp, attn

# Save settings to YAML
from omnizart.io import write_yaml
write_yaml(music_settings.to_json(), "my_settings.yaml")

# Load settings from YAML
custom_settings = MusicSettings(conf_path="my_settings.yaml")
```

--------------------------------

### Train Music Transcription Models via Python

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Configures training parameters and executes the training process for music transcription using the MusicTranscription class.

```python
from omnizart.music import MusicTranscription
from omnizart.setting_loaders import MusicSettings

settings = MusicSettings()
settings.training.epoch = 30
settings.training.batch_size = 16
settings.training.early_stop = 8
settings.training.loss_function = "smooth"
settings.training.label_type = "note-stream"

app = MusicTranscription()
model_path, history = app.train(
    feature_folder="/data/maestro/train_feature",
    model_name="custom-music-model",
    input_model_path=None,
    music_settings=settings
)
print(f"Model saved to: {model_path}")
```

--------------------------------

### Drum Model Default Settings (YAML)

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/drum/api.md

This YAML configuration defines the default parameters for building and training the Omnizart drum model. It includes settings for transcription mode, checkpoint paths, feature extraction, dataset management, model saving, inference thresholds, and training hyperparameters. These settings are loaded by `omnizart.setting_loaders.DrumSettings` and attribute names are converted to snake-case.

```yaml
General:
    TranscriptionMode:
        Description: Mode of transcription by executing the `omnizart drum transcribe` command.
        Type: String
        Value: Keras
    CheckpointPath:
        Description: Path to the pre-trained models.
        Type: Map
        SubType: [String, String]
        Value:
            Keras: checkpoints/drum/drum_keras
    Feature:
        Description: Default settings of feature extraction for drum transcription.
        Settings:
            SamplingRate:
                Description: Adjust input sampling rate to this value.
                Type: Integer
                Value: 44100
            PaddingSeconds:
                Description: Padding length to the begin and the end of the raw audio data.
                Type: Float
                Value: 1.0
            LowestNote:
                Description: Lowest MIDI note number to be considered.
                Type: Integer
                Value: 16
            NumberOfNotes:
                Description: Number of total notes to extract.
                Type: Integer
                Value: 120
            HopSize:
                Description: Hop size for computing CQT feature.
                Type: Integer
                Value: 256
            MiniBeatPerBar:
                Description: Number of mini beats in a single 4/4 measure.
                Type: Integer
                Value: 32
            MiniBeatPerSegment:
                Description: Number of mini beats in a single 4/4 measure.
                Type: Integer
                Value: 4
    Dataset:
        Description: Settings of datasets.
        Settings:
            SavePath:
                Description: Path for storing the downloaded datasets.
                Type: String
                Value: ./
            FeatureSavePath:
                Description: Path for storing the extracted feature. Default to the path under the dataset folder.
                Type: String
                Value: +
    Model:
        Description: Default settings of training / testing the model.
        Settings:
            SavePrefix:
                Description: Prefix of the trained model's name to be saved.
                Type: String
                Value: drum
            SavePath:
                Description: Path to save the trained model.
                Type: String
                Value: ./checkpoints/drum
    Inference:
        Description: Default settings when infereing notes.
        Settings:
            BassDrumTh:
                Description: Threshold for the bass drum.
                Type: Float
                Value: 0.85
            SnareTh:
                Description: Threshold for the snare.
                Type: Float
                Value: 1.2
            HihatTh:
                Description: Threshold for the hihat.
                Type: Float
                Value: 0.17
    Training:
        Description: Hyper parameters for training
        Settings:
            Epoch:
                Description: Maximum number of epochs for training.
                Type: Integer
                Value: 50
            Steps:
                Description: Number of training steps for each epoch.
                Type: Integer
                Value: 1000
            ValSteps:
                Description: Number of validation steps after each training epoch.
                Type: Integer
                Value: 100
            BatchSize:
                Description: Batch size of each training step.
                Type: Integer
                Value: 32
            ValBatchSize:
                Description: Batch size of each validation step.
                Type: Integer
                Value: 32
            EarlyStop:
                Description: Terminate the training if the validation performance doesn't imrove after n epochs.
                Type: Integer
                Value: 6
            InitLearningRate:
                Descriptoin: Initial learning rate.
                Type: Float
                Value: 0.00002
            ResBlockNum:
                Description: Number of residual blocks.
                Type: Integer
                Value: 3

```

--------------------------------

### Download Transcribed MIDI File

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/colab.ipynb

Provides a direct download link for the transcribed MIDI file generated by Omnizart.

```python
#@title Download MIDI

files.download(f"{uploaded_audio}.mid")
```

--------------------------------

### Download Synthesized MP3 File

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/colab.ipynb

Provides a direct download link for the synthesized MP3 file, which is created by converting the MIDI output to audio.

```python
#@title Download MP3

files.download(out_name)
```

--------------------------------

### Generate Features for Training

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/tutorial.md

Commands to generate necessary features for music and drum applications from a specified dataset path.

```bash
omnizart music generate-feature --dataset-path <path/to/dataset>
omnizart drum generate-feature --dataset-path <path/to/dataset>
```

--------------------------------

### POST /config/model

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/music/api.md

Updates the configuration parameters for the Omnizart model training process.

```APIDOC
## POST /config/model

### Description
Updates the training configuration for the Omnizart model, including loss functions, label types, and feature extraction parameters.

### Method
POST

### Endpoint
/config/model

### Parameters
#### Request Body
- **LossFunction** (String) - Optional - Loss function for computing objectives. Choices: ["smooth", "focal", "bce"].
- **LabelType** (String) - Optional - Determines the training target. Choices: ["note-stream", "frame-stream", "note", "frame", "true-frame", "true-frame-stream", "pop-note-stream"].
- **Channels** (List) - Optional - Feature types for training. Choices: ["Spec", "GCoS", "Ceps"].
- **Timesteps** (Integer) - Optional - Length of the time axis.
- **FeatureNum** (Integer) - Optional - Target size of feature dimension.

### Request Example
{
  "LossFunction": "smooth",
  "LabelType": "note-stream",
  "Channels": ["Spec", "Ceps"],
  "Timesteps": 256,
  "FeatureNum": 352
}

### Response
#### Success Response (200)
- **status** (String) - Configuration updated successfully.

#### Response Example
{
  "status": "success"
}
```

--------------------------------

### Load Audio Tracks into Waveform Playlist

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/_static/demo-surrealist.html

Loads audio tracks into an initialized Waveform Playlist. Each track can have a source file, a name, and a gain level. This function is essential for populating the playlist with audio content for playback and visualization.

```javascript
playlist.load([
  {
    "src": "../\_audio/surrealist.mp3",
    "name": "Original Song",
    "gain": 0.5
  },
  {
    "src": "../\_audio/surrealist_chord.mp3",
    "name": "Chord",
    "gain": 1
  },
  {
    "src": "../\_audio/surrealist_drum.mp3",
    "name": "Drum",
    "gain": 1
  }
])
```

--------------------------------

### Configure Omnizart Log Level

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/CONTRIBUTING.md

How to adjust the verbosity of application logs by setting the LOG_LEVEL environment variable.

```bash
export LOG_LEVEL=debug
```

--------------------------------

### Chord Transcription - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Transcribe chord progressions from audio, outputting both MIDI and CSV files with chord names and timestamps using the command line.

```APIDOC
## Chord Transcription - CLI

Transcribe chord progressions from audio, outputting both MIDI and CSV files with chord names and timestamps.

```bash
# Transcribe chords from audio
omnizart chord transcribe song.wav

# Output will generate both MIDI and CSV files
# song.mid - MIDI with chord notes
# song.csv - CSV with chord names and timestamps
omnizart chord transcribe song.wav --output ./output_folder/
```
```

--------------------------------

### Transcribe Music - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Transcribes pitched instrument notes from polyphonic audio files into MIDI format using the Omnizart command-line interface. Supports specifying output paths and different transcription models like Piano, Stream, Pop, and PianoV2.

```bash
# Basic transcription with default Piano mode
omnizart music transcribe song.wav

# Specify output path and model mode
omnizart music transcribe song.wav --output transcribed.mid

# Use different transcription modes: Piano, Stream, Pop, PianoV2
omnizart music transcribe song.wav --model-path Pop

# Use custom trained model
omnizart music transcribe song.wav --model-path /path/to/custom/model
```

--------------------------------

### Upload and Convert MP3 to WAV

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/colab.ipynb

Allows users to upload an MP3 file and converts it to WAV format using ffmpeg, which is a required input format for Omnizart transcription.

```python
#@title Upload MP3 File
import os
from google.colab import files
from IPython import display as dsp

ff = files.upload()
uploaded_audio = list(ff.keys())[0].replace(".mp3", "")

!test -f "$uploaded_audio".wav && rm "$uploaded_audio".wav
!ffmpeg -i "$uploaded_audio".mp3 "$uploaded_audio".wav &>/dev/null

dsp.Audio(uploaded_audio + ".mp3") if os.path.exists(uploaded_audio + ".mp3") else None
```

--------------------------------

### Feature Generation - Music Module CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Generate features from datasets for training music transcription models using the command-line interface. This command processes a specified dataset path and saves the extracted features into train_feature and test_feature subdirectories.

```bash
# Generate features for music transcription
omnizart music generate-feature --dataset-path /data/maestro

# Features are saved to:
# /data/maestro/train_feature/
# /data/maestro/test_feature/
```

--------------------------------

### Music Transcription - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Transcribe pitched instrument notes from polyphonic audio files to MIDI format using the command line interface.

```APIDOC
## Music Transcription - CLI

Transcribe pitched instrument notes from polyphonic audio files to MIDI format using the command line.

```bash
# Basic transcription with default Piano mode
omnizart music transcribe song.wav

# Specify output path and model mode
omnizart music transcribe song.wav --output transcribed.mid

# Use different transcription modes: Piano, Stream, Pop, PianoV2
omnizart music transcribe song.wav --model-path Pop

# Use custom trained model
omnizart music transcribe song.wav --model-path /path/to/custom/model
```
```

--------------------------------

### Synthesize MIDI to Audio

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Converts transcribed MIDI files into WAV audio format using the built-in synthesizer and default soundfonts.

```bash
omnizart synth transcribed.mid
omnizart synth transcribed.mid --output synthesized.wav
```

--------------------------------

### Omnizart Training Hyperparameters

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/docs/source/vocal/api.md

Hyperparameters for training the Omnizart model, covering epochs, steps, batch sizes, learning rate, and context length.

```APIDOC
## Omnizart Training Hyperparameters

### Description
Hyperparameters that control the training process of the Omnizart model.

### Parameters
#### Training Settings
- **Epoch** (Integer) - Optional - Maximum number of epochs for training. Default: 10.
- **Steps** (Integer) - Optional - Number of training steps for each epoch. Default: 1000.
- **ValSteps** (Integer) - Optional - Number of validation steps after each training epoch. Default: 50.
- **BatchSize** (Integer) - Optional - Batch size of each training step. Default: 64.
- **ValBatchSize** (Integer) - Optional - Batch size of each validation step. Default: 64.
- **EarlyStop** (Integer) - Optional - Terminate training if validation performance doesn't improve after n epochs. Default: 8.
- **InitLearningRate** (Float) - Optional - Initial learning rate for training. Default: 0.0001.
- **ContextLength** (Integer) - Optional - Context to be considered before and after the current timestamp. Default: 9.
```

--------------------------------

### Transcribe Chords - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Transcribes chord progressions from audio files and outputs both MIDI and CSV files. The MIDI file contains chord notes, while the CSV file includes chord names and their corresponding timestamps.

```bash
# Transcribe chords from audio
omnizart chord transcribe song.wav

# Output will generate both MIDI and CSV files
# song.mid - MIDI with chord notes
# song.csv - CSV with chord names and timestamps
omnizart chord transcribe song.wav --output ./output_folder/
```

--------------------------------

### Beat Transcription - Python API

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Perform symbolic-domain beat tracking from MIDI files using the BeatTranscription class in Python. This API extracts beat and downbeat information, which can be accessed from the resulting MIDI object.

```python
from omnizart.beat import BeatTranscription

# Initialize beat transcription
app = BeatTranscription()

# Transcribe beats from MIDI
midi = app.transcribe(
    input_audio="piano_piece.mid",  # Note: accepts MIDI, not audio
    model_path=None,  # Uses default BLSTM model
    output="./"
)

# Access beat and downbeat information
for instrument in midi.instruments:
    if instrument.name == "Beat":
        beats = [note.start for note in instrument.notes]
        print(f"Beat positions: {beats[:10]}")
    elif instrument.name == "DownBeat":
        downbeats = [note.start for note in instrument.notes]
        print(f"Downbeat positions: {downbeats[:5]}")
```

--------------------------------

### Transcribe Music - Python API

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Programmatically transcribes pitched instrument notes from audio files to MIDI format using the MusicTranscription class. It returns a pretty_midi.PrettyMIDI object and allows specifying the model path and output directory.

```python
from omnizart.music import MusicTranscription

app = MusicTranscription()

midi = app.transcribe(
    input_audio="song.wav",
    model_path="Piano",  # Options: Piano, Stream, Pop, PianoV2
    output="./"          # Output directory for MIDI file
)

for instrument in midi.instruments:
    print(f"Instrument: {instrument.program}")
    for note in instrument.notes[:5]:  # First 5 notes
        print(f"  Note: {note.pitch}, Start: {note.start:.2f}s, End: {note.end:.2f}s")
```

--------------------------------

### Transcribe Audio with Omnizart

Source: https://github.com/music-and-culture-technology-lab/omnizart/blob/master/colab.ipynb

Transcribes the input WAV audio file into various musical formats (MIDI, MP3) based on the selected mode (e.g., 'vocal', 'music-piano'). It utilizes the Omnizart command-line interface and synthesizes the output using FluidSynth.

```python
#@title Transcribe

mode = "vocal" #@param ["music-piano", "music-piano-v2", "music-assemble", "chord", "drum", "vocal", "vocal-contour", "beat"]

model = ""
if mode.startswith("music"):
  mode_list = mode.split("-")
  mode = mode_list[0]
  model = "-".join(mode_list[1:])

model_path = {
    "piano": "Piano",
    "piano-v2": "PianoV2",
    "assemble": "Stream",
    "pop-song": "Pop",
    "": None
}[model]

if model_path:
  !omnizart "$mode" transcribe "$uploaded_audio".wav --model-path "$model_path"
else:
  !omnizart "$mode" transcribe "$uploaded_audio".wav

# Synthesize MIDI and play
import wave
import numpy as np
from pretty_midi import PrettyMIDI
from omnizart.remote import download_large_file_from_google_drive


def write_audio_file(filename, sample_rate, data):
    with wave.open(filename, 'w') as wf:
        # Set the parameters for the wave file
        wf.setnchannels(1)  # Mono
        wf.setsampwidth(2)  # 2 bytes per sample (16-bit)
        wf.setframerate(sample_rate)

        # Convert the data type if necessary
        if data.dtype != np.int16:
            data = np.int16(data * 32767)  # Scale to 16-bit integer range

        # Write the audio frames to the file
        wf.writeframes(data.tobytes())


SF2_FILE = "general_soundfont.sf2"
if not os.path.exists(SF2_FILE):
  print("Downloading soundfont...")
  !curl "https://ftp.osuosl.org/pub/musescore/soundfont/MuseScore_General/MuseScore_General.sf2" -o $SF2_FILE

synth_name = f"{uploaded_audio}_synth.wav"
if mode == "vocal-contour":
  os.rename(f"{uploaded_audio}_trans.wav", f"{uploaded_audio}_synth.wav")
else:
  print("Synthesizing MIDI...")
  midi = PrettyMIDI(f"{uploaded_audio}.mid")
  raw_wav = midi.fluidsynth(fs=44100, sf2_path=SF2_FILE)
  write_audio_file(synth_name, 44100, raw_wav)

!ffmpeg -i "$synth_name" "tmp_synth.mp3" &> /dev/null
!mv tmp_synth.mp3 "$uploaded_audio"_synth.mp3

out_name = synth_name.replace(".wav", ".mp3")
print(f"Finished: {out_name}")
dsp.Audio(out_name)
```

--------------------------------

### Train Vocal Transcription Models

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Trains a vocal transcription model with support for semi-supervised learning by providing an optional unlabeled feature folder.

```python
from omnizart.vocal import VocalTranscription
from omnizart.setting_loaders import VocalSettings

settings = VocalSettings()
settings.training.epoch = 15
settings.training.batch_size = 64
settings.model.depth = 110
settings.model.semi_loss_weight = 1.0

app = VocalTranscription()
app.generate_feature(dataset_path="/data/mir1k", vocal_settings=settings, num_threads=4)
app.train(
    feature_folder="/data/mir1k/train_feature",
    semi_feature_folder="/data/unlabeled_features",
    model_name="custom-vocal-model",
    vocal_settings=settings
)
```

--------------------------------

### Beat Transcription - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Transcribe beat and downbeat positions from MIDI files using the command-line interface. The output includes a modified MIDI file with beat markers, and CSV files detailing beat and downbeat positions in seconds.

```bash
# Transcribe beat positions from MIDI
omnizart beat transcribe input.mid

# Outputs:
# - output.mid: MIDI with beat markers
# - output_beat.csv: Beat positions in seconds
# - output_down_beat.csv: Downbeat positions in seconds
omnizart beat transcribe input.mid --output ./beat_output/
```

--------------------------------

### Chord Transcription - Python API

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Use the ChordTranscription class for programmatic chord analysis with detailed timing information.

```APIDOC
## Chord Transcription - Python API

Use the ChordTranscription class for programmatic chord analysis with detailed timing information.

```python
from omnizart.chord import ChordTranscription

# Initialize chord transcription
app = ChordTranscription()

# Transcribe chord progression
# Returns pretty_midi.PrettyMIDI with chord notes
midi = app.transcribe(
    input_audio="jazz_song.wav",
    model_path=None,  # Uses default ChordV1 model
    output="./"       # Outputs .mid and .csv files
)

# The CSV file contains detailed chord information:
# timestamp, chord_name (e.g., "C:maj", "G:min7", "N" for no chord)
```
```

--------------------------------

### Transcribe Drums - CLI

Source: https://context7.com/music-and-culture-technology-lab/omnizart/llms.txt

Transcribes drum and percussion events from audio files into MIDI format using the Omnizart command-line interface. Allows specifying custom output paths and custom model paths for drum transcription.

```bash
# Transcribe drum events from audio
omnizart drum transcribe pop_song.wav

# Specify custom output path
omnizart drum transcribe pop_song.wav --output drums.mid

# Use custom model path
omnizart drum transcribe pop_song.wav --model-path /path/to/drum/model
```