### Install Dependencies for Make-An-Audio 3 Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Sets up a conda environment and installs necessary Python packages, including PyTorch with CUDA support and project-specific requirements. Flash attention and NVIDIA Apex are also included. ```bash conda create -n Make_An_Audio_3 -y conda activate Make_An_Audio_3 conda install python=3.11 pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y pip install -r requirements.txt pip install flash-attn --no-build-isolation Install [nvidia apex](https://github.com/nvidia/apex) (optional) ``` -------------------------------- ### Install Dependencies for Make-An-Audio 3 Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/README.md Sets up a Conda environment and installs PyTorch with specific CUDA version, along with project requirements. Ensure your CUDA version is compatible with your NVIDIA driver. ```bash conda create -n Make_An_Audio_3 -y conda activate Make_An_Audio_3 conda install python=3.11 pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia -y pip install -r requirements.txt pip install flash-attn --no-build-isolation ``` -------------------------------- ### File Hash Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/text2audio/XXL/214.ckpt This is the Git LFS file hash for the Make-An-Audio-3 model. ```text df6bd52 ``` -------------------------------- ### Generate Audio from Video Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3 Generates audio from video using a specified configuration. Requires setting the output directory, checkpoint, configuration file, scale, vocoder checkpoint, and test dataset. ```python python3 scripts/video2audio_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/video2audio-cfm1-cfg-LargeDiT1-moe.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset vggsound ``` -------------------------------- ### Generate Audio from Test Datasets Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md?code=true Generates audio from specified test datasets like AudioCaps or MusicCaps using the `txt2audio_for_2cap_flow.py` script. Remember to adjust the `test_dataset` configuration. ```python python3 scripts/txt2audio_for_2cap_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/txt2audio-cfm1-cfg-LargeDiT3.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset testset ``` -------------------------------- ### Generate Audio from Text Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Uses a pretrained model to generate audio from text descriptions. Specify the output directory, checkpoint, configuration file, scale, vocoder checkpoint, and test dataset. ```python python3 scripts/txt2audio_for_2cap_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/txt2audio-cfm1-cfg-LargeDiT3.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset audiocaps ``` -------------------------------- ### Generate Audio from Video Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Extracts audio from video files using a specified configuration and model checkpoint. The 'test-dataset' parameter should be set to 'vggsound'. ```python python3 scripts/video2audio_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/video2audio-cfm1-cfg-LargeDiT1-moe.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset vggsound ``` -------------------------------- ### Generate Audio from Audiocaps or Musiccaps Test Dataset Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Generates audio from text using the specified test dataset. Ensure the 'config["test_dataset"]' is adjusted accordingly. ```python python3 scripts/txt2audio_for_2cap_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/txt2audio-cfm1-cfg-LargeDiT3.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset testset ``` -------------------------------- ### Train Latent Diffusion Model Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Run this command to train the diffusion model after a VAE has been trained and its checkpoint path is updated in the configuration file. The --gpus argument should be adjusted as needed. ```python python main.py --base configs/research/text2audio/text2audio-ConcatDiT-ae1dnat_Skl20d2_freezeFlananylen_drop.yaml -t --gpus 0,1,2,3,4,5,6,7 ``` -------------------------------- ### Generate Audio from Video Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md?code=true Converts audio from video files using the `video2audio_flow.py` script. Requires specifying the output directory, checkpoint, configuration, scale, vocoder, and test dataset. ```python python3 scripts/video2audio_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/video2audio-cfm1-cfg-LargeDiT1-moe.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset vggsound ``` -------------------------------- ### Train Latent Diffusion Model Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/README.md Trains a latent diffusion model for text-to-audio generation. Requires a trained VAE checkpoint. Update the config file with the VAE checkpoint path. Supports multi-GPU training. ```bash python main.py --base configs/research/text2audio/text2audio-ConcatDiT-ae1dnat_Skl20d2_freezeFlananylen_drop.yaml -t --gpus 0,1,2,3,4,5,6,7 ``` -------------------------------- ### Train Latent Diffusion Model Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md?code=true Train a latent diffusion model after training the VAE. Update the 'model.params.first_stage_config.params.ckpt_path' in the config file to point to your trained VAE checkpoint. This command also assumes 8 GPUs; adjust '--gpus' as necessary. ```bash python main.py --base configs/research/text2audio/text2audio-ConcatDiT-ae1dnat_Skl20d2_freezeFlananylen_drop.yaml -t --gpus 0,1,2,3,4,5,6,7 ``` -------------------------------- ### Generate Audio from Text Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md?code=true Uses the `txt2audio_for_2cap_flow.py` script to generate audio from text captions. Specify the output directory, checkpoint, configuration, scale, and vocoder. ```python python3 scripts/txt2audio_for_2cap_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/txt2audio-cfm1-cfg-LargeDiT3.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset audiocaps ``` -------------------------------- ### Generate Audio from Text Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3 Generates audio or music from text using a pretrained model. Specify output directory, checkpoint, configuration, scale, and vocoder. ```python python3 scripts/txt2audio_for_2cap_flow.py --outdir output_dir -r checkpoints_last.ckpt -b configs/txt2audio-cfm1-cfg-LargeDiT3.yaml --scale 3.0 --vocoder-ckpt useful_ckpts/bigvnat --test-dataset audiocaps ``` -------------------------------- ### Train Variational Autoencoder (VAE) Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/README.md Trains a VAE model using processed datasets. Ensure the TSV files are located in the specified directory and update the config file accordingly. Supports multi-GPU training. ```bash python main.py --base configs/research/autoencoder/autoencoder1d_kl20_natbig_r1_down2_disc2.yaml -t --gpus 0,1,2,3,4,5,6,7 ``` -------------------------------- ### Generate Melspec File of Audio Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Preprocesses audio files to generate melspec files. Requires a TSV file containing audio paths and captions. The output melspec files are saved in the './processed' directory. ```python python preprocess/mel_spec.py --tsv_path tmp.tsv --num_gpus 1 --max_duration 10 ``` -------------------------------- ### Model File Hash Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/text2audio/XL/319.ckpt This snippet shows the Git object ID (oid) for a specific version of the model file, indicating its integrity and uniqueness. ```text 2b0300b ``` -------------------------------- ### Generate Structured Caption using OpenAI API Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Converts natural language captions into a structured format using the OpenAI API. Ensure your OpenAI API key is set in the script. The output TSV file will have '_struct.tsv' appended to its name. ```python python preprocess/n2s_by_openai.py --tsv_path tmp.tsv ``` -------------------------------- ### Git LFS Version Information Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/text2audio/XXL/214.ckpt This snippet shows the Git Large File Storage (LFS) version and object identifier for the model file. It includes the SHA256 hash and the size of the file. ```text version https://git-lfs.github.com/spec/v1 oid sha256:f4ebbad52bb60d6893462b5234468ea06d9a6313d0e3675b75a5e19e5ef8f78f size 23023047600 ``` -------------------------------- ### Git LFS Version Specification Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blame/main/text2audio/XL/319.ckpt This code block specifies the Git Large File Storage (LFS) version and provides the SHA256 hash and size for the model file. ```text version https://git-lfs.github.com/spec/v1 oid sha256:2b20a99ba33105322621fff8aafb27fffb12e79e8ba0ce074a85f831fab4fa2d size 14281650032 ``` -------------------------------- ### Count Audio Duration Source: https://huggingface.co/AIGC-Audio/Make-An-Audio-3/blob/main/README.md Calculates the duration of audio files and appends this information to the provided TSV file. This is a necessary step for data preparation. ```python python preprocess/add_duration.py --tsv_path tmp.tsv ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.