OpenAudio

https://speech.fish.audio/
Admin
OpenAudio is an advanced Text-to-Speech (TTS) model series offering high-quality, multilingual, and...
Tokens:12,544
Snippets:126
Trust Score:9.2
Update:8 months ago
Show doc for...
Tokens:
Raw
### Install System Dependencies for Audio Processing

Source: https://speech.fish.audio/install

Installs essential system libraries required for audio processing, such as portaudio, libsox, and ffmpeg. These are typically installed using a package manager like apt on Linux-based systems.

```bash
apt install portaudio19-dev libsox-dev ffmpeg

```

--------------------------------

### Install Speech Fish Audio with UV (GPU/CPU)

Source: https://speech.fish.audio/install

Installs Speech Fish Audio using the UV package manager for faster dependency resolution. Supports GPU installations with specified CUDA extras and CPU-only installations.

```bash
# GPU installation (choose your CUDA version: cu126, cu128, cu129)
uv sync --python 3.12 --extra cu129

# CPU-only installation
uv sync --python 3.12 --extra cpu

```

--------------------------------

### Intel Arc XPU Support Installation

Source: https://speech.fish.audio/zh/install

Sets up a Conda environment and installs PyTorch with XPU support for Intel Arc GPUs. This includes installing the necessary C++ standard library and the nightly build of PyTorch with XPU support.

```bash
conda create -n fish-speech python=3.12
conda activate fish-speech

# Install required C++ standard library
conda install libstdcxx -c conda-forge

# Install PyTorch with Intel XPU support
pip install --pre torch torchvision toraudio --index-url https://download.pytorch.org/whl/nightly/xpu

# Install Fish Speech
pip install -e .
```

--------------------------------

### Conda Environment Setup and Installation

Source: https://speech.fish.audio/zh/install

Creates and activates a Conda environment named 'fish-speech' with Python 3.12. It then provides commands for installing the project with GPU support (specifying CUDA version) or CPU-only support, and a default installation.

```bash
conda create -n fish-speech python=3.12
conda activate fish-speech

# GPU Installation (Choose your CUDA version: cu126, cu128, cu129)
pip install -e .[cu129]

# CPU-only Installation
pip install -e .[cpu]

# Default Installation (using PyTorch official source)
pip install -e .
```

--------------------------------

### Start HTTP API Inference Server

Source: https://speech.fish.audio/inference

This command starts the HTTP API server for inference. It requires specifying the listen address and paths to the model checkpoints. Adding the `--compile` parameter can accelerate inference.

```python
python -m tools.api_server \
    --listen 0.0.0.0:8080 \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq

# or with uv

uv run tools/api_server.py \
    --listen 0.0.0.0:8080 \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq

```

--------------------------------

### Download Model Weights

Source: https://speech.fish.audio/inference_q=

Instructions on how to download the necessary model weights for inference using the huggingface_hub CLI.

```APIDOC
## Download Model Weights

### Description
Download the model weights required for inference. This step is crucial before proceeding with any inference method.

### Method
Command Line

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
# Requires "huggingface_hub[cli]" to be installed
# pip install huggingface_hub[cli]
# or 
# uv tool install huggingface_hub[cli]

hf download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini
```

### Response
#### Success Response (200)
Model weights are downloaded to the specified local directory.

#### Response Example
N/A
```

--------------------------------

### Install Speech Fish Audio with Conda (GPU/CPU)

Source: https://speech.fish.audio/install

Sets up a Conda environment for Speech Fish Audio and installs the package. Supports GPU installation with specific CUDA versions or a CPU-only installation. The default installation uses PyTorch's default index.

```bash
conda create -n fish-speech python=3.12
conda activate fish-speech

# GPU installation (choose your CUDA version: cu126, cu128, cu129)
pip install -e .[cu129]

# CPU-only installation
pip install -e .[cpu]

# Default installation (uses PyTorch default index)
pip install -e .

```

--------------------------------

### Download Model Weights with Hugging Face CLI

Source: https://speech.fish.audio/inference

This command downloads the specified model weights from Hugging Face Hub to a local directory. It requires the 'huggingface_hub[cli]' package to be installed.

```bash
# Requires "huggingface_hub[cli]" to be installed
# pip install huggingface_hub[cli]
# or 
# uv tool install huggingface_hub[cli]

hf download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini

```

--------------------------------

### Docker: Start WebUI with CUDA Support

Source: https://speech.fish.audio/inference

This command runs the WebUI inference server within a Docker container, utilizing CUDA for GPU acceleration. It maps local directories for checkpoints and references into the container and sets the COMPILE environment variable.

```bash
# Create directories for model weights and reference audio
mkdir -p checkpoints references

# Download model weights (if not already done)
# hf download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini

# Start WebUI with CUDA support (recommended for best performance)
docker run -d \
    --name fish-speech-webui \
    --gpus all \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-webui-cuda

```

--------------------------------

### Install Speech Fish Audio with Intel Arc XPU Support

Source: https://speech.fish.audio/install

Installs Speech Fish Audio with support for Intel Arc GPUs (XPU). This involves creating a Conda environment, installing the necessary C++ standard library, and then installing PyTorch with XPU support followed by the Speech Fish package.

```bash
conda create -n fish-speech python=3.12
conda activate fish-speech

# Install required C++ standard library
conda install libstdcxx -c conda-forge

# Install PyTorch with Intel XPU support
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu

# Install Fish Speech
pip install -e .

```

--------------------------------

### Start WebUI Inference

Source: https://speech.fish.audio/inference

This command launches the WebUI for inference. Similar to the API server, it requires checkpoint paths and can be accelerated with the `--compile` flag. The WebUI can be configured using Gradio environment variables.

```python
python -m tools.run_webui \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq

# Or simply

python -m tools.run_webui

```

--------------------------------

### Start Speech Fish Audio Server with CUDA Support (Docker)

Source: https://speech.fish.audio/inference_q=

This command launches the Speech Fish Audio server in a Docker container with CUDA support enabled. It maps local directories for checkpoints and references, exposes port 8080, and sets COMPILE=1 for faster inference. Ensure you have NVIDIA Docker runtime installed.

```docker
docker run -d \
    --name fish-speech-server \
    --gpus all \
    -p 8080:8080 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-server-cuda
```

--------------------------------

### Manual Docker Build: CPU WebUI Image (Multi-platform)

Source: https://speech.fish.audio/zh/install

Builds a Docker image for the Speech Fish Audio WebUI with CPU support, designed to be multi-platform (amd64 and arm64). This command specifies the backend as 'cpu' and targets the 'webui' stage.

```bash
docker build \
    --platform linux/amd64,linux/arm64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cpu \
    --target webui \
    -t fish-speech-webui:cpu .
```

--------------------------------

### Docker: Start WebUI for CPU-only Inference

Source: https://speech.fish.audio/inference

This command runs the WebUI inference server in a Docker container for CPU-only execution. It maps local directories for checkpoints and references into the container but does not utilize GPU acceleration.

```bash
# For CPU-only inference (slower, but works without GPU)
docker run -d \
    --name fish-speech-webui-cpu \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-webui-cpu

```

--------------------------------

### Download Model Weights with Hugging Face CLI

Source: https://speech.fish.audio/inference_q=

This command downloads the specified model weights from Hugging Face Hub to a local directory. It requires the 'huggingface_hub[cli]' package to be installed. The downloaded weights are stored in the 'checkpoints/openaudio-s1-mini' directory.

```bash
# Requires "huggingface_hub[cli]" to be installed
# pip install huggingface_hub[cli]
# or 
# uv tool install huggingface_hub[cli]

hf download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini
```

--------------------------------

### Command Line Inference - Step 1: Get VQ tokens

Source: https://speech.fish.audio/inference_q=

This step involves generating VQ tokens from a reference audio file using the DAC model.

```APIDOC
## Command Line Inference - Step 1: Get VQ tokens from reference audio

### Description
Generate VQ tokens from a reference audio file. These tokens can be used later for voice timbre control. If random voice timbre is desired, this step can be skipped.

### Method
Command Line

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
python fish_speech/models/dac/inference.py \
    -i "ref_audio_name.wav" \
    --checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth"
```

### Response
#### Success Response (200)
Generates `fake.npy` (VQ tokens) and `fake.wav` (reconstructed audio) in the current directory.

#### Response Example
```
# Output files:
fake.npy
fake.wav
```
```

--------------------------------

### Manual Docker Build: CUDA WebUI Image

Source: https://speech.fish.audio/zh/install

Builds a Docker image for the Speech Fish Audio WebUI with CUDA support. This command specifies the build platform, Dockerfile, build arguments for backend, CUDA version, UV extra package, and target stage.

```bash
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target webui \
    -t fish-speech-webui:cuda .
```

--------------------------------

### Manual Docker Build: Development Image

Source: https://speech.fish.audio/zh/install

Builds a Docker image for development purposes with CUDA support. This command uses the 'dev' target in the Dockerfile and allows specifying the backend and CUDA version.

```bash
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --target dev \
    -t fish-speech-dev:cuda .
```

--------------------------------

### WebUI Inference

Source: https://speech.fish.audio/inference_q=

Start the WebUI for inference. This provides a graphical interface for interacting with the model.

```APIDOC
## WebUI Inference

### Description
Launch the Web User Interface (WebUI) for performing inference. This provides a user-friendly graphical way to interact with the speech synthesis model. The `--compile` flag can be used to optimize inference speed.

### Method
Command Line

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
# With specified checkpoints:
python -m tools.run_webui \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq

# Using default checkpoints (if configured):
python -m tools.run_webui

# To speed up inference:
python -m tools.run_webui --compile \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq
```

### Response
#### Success Response (200)
The WebUI starts and is accessible via a web browser.

#### Response Example
Access the WebUI through your browser, typically at `http://127.0.0.1:7860/`.

### Notes
- Pre-place label files (`sample.lab`) and reference audio (`sample.wav`) in the `references/<voice_id>/` directory for easy access in the WebUI.
- Gradio environment variables (e.g., `GRADIO_SHARE`, `GRADIO_SERVER_PORT`) can be used for configuration.
```

--------------------------------

### Docker Compose Environment Variables

Source: https://speech.fish.audio/zh/install

An example `.env` file demonstrating how to customize Docker Compose deployments for Speech Fish Audio. It shows variables for setting the backend (cuda/cpu), enabling compile optimizations, and configuring WebUI and API ports.

```dotenv
# .env file example
BACKEND=cuda              # or cpu
COMPILE=1                 # Enable compile optimization
GRADIO_PORT=7860         # WebUI port
API_PORT=8080            # API server port
UV_VERSION=0.8.15        # UV package manager version
```

--------------------------------

### HTTP API Server

Source: https://speech.fish.audio/inference_q=

Start an HTTP API server for inference. This allows programmatic access to the model's capabilities.

```APIDOC
## HTTP API Inference

### Description
Launch an HTTP API server to perform inference. This server listens on a specified address and port, allowing clients to send requests for audio generation.

### Method
Command Line

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
# Using python:
python -m tools.api_server \
    --listen 0.0.0.0:8080 \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq

# Using uv:
uv run tools/api_server.py \
    --listen 0.0.0.0:8080 \
    --llama-checkpoint-path "checkpoints/openaudio-s1-mini" \
    --decoder-checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth" \
    --decoder-config-name modded_dac_vq
```

### Response
#### Success Response (200)
The API server starts successfully and listens on the specified address and port.

#### Response Example
Access the API documentation and testing interface at `http://127.0.0.1:8080/`.

### Notes
- Add the `--compile` parameter to speed up inference.
```

--------------------------------

### Docker Inference - WebUI

Source: https://speech.fish.audio/inference_q=

Instructions for running the WebUI inference using Docker containers, with options for CUDA (GPU) and CPU.

```APIDOC
## Docker Inference - WebUI

### Description
Deploy the Speech Fish Audio WebUI using Docker containers. This method simplifies setup and ensures consistent environments, with specific configurations for GPU (CUDA) and CPU-only inference.

### Method
Command Line (Docker)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
# Create necessary directories
mkdir -p checkpoints references

# Download model weights (if not already done)
# hf download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini

# Start WebUI with CUDA support (recommended):
docker run -d \
    --name fish-speech-webui \
    --gpus all \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    -e COMPILE=1 \
    fishaudio/fish-speech:latest-webui-cuda

# For CPU-only inference (slower):
docker run -d \
    --name fish-speech-webui-cpu \
    -p 7860:7860 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-webui-cpu
```

### Response
#### Success Response (200)
The Docker container starts, and the WebUI is accessible at `http://127.0.0.1:7860/`.

#### Response Example
N/A

### Notes
- Requires Docker and NVIDIA Container Toolkit (for GPU support).
- Mount local directories for `checkpoints` and `references` to persist data.
```

--------------------------------

### Manual Docker Build: CUDA API Server Image

Source: https://speech.fish.audio/zh/install

Builds a Docker image for the Speech Fish Audio API server with CUDA support. Similar to the WebUI build, it allows customization of backend, CUDA version, and UV extra package.

```bash
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target server \
    -t fish-speech-server:cuda .
```

--------------------------------

### Docker Compose: Run Speech Fish Audio WebUI and API Server

Source: https://speech.fish.audio/install

Uses Docker Compose to build and run the Speech Fish Audio project locally. Supports starting the WebUI or API server with CUDA, compile optimization, or CPU-only deployment. Requires cloning the repository first.

```bash
# Clone the repository first
git clone https://github.com/fishaudio/fish-speech.git
cd fish-speech

# Start WebUI with CUDA
docker compose --profile webui up

# Start WebUI with compile optimization
COMPILE=1 docker compose --profile webui up

# Start API server
docker compose --profile server up

# Start API server with compile optimization  
COMPILE=1 docker compose --profile server up

# For CPU-only deployment
BACKEND=cpu docker compose --profile webui up

```

--------------------------------

### Start Speech Fish Audio Server with CPU-Only Inference (Docker)

Source: https://speech.fish.audio/inference_q=

This command deploys the Speech Fish Audio server in a Docker container for CPU-only inference. It maps local directories for checkpoints and references, and exposes port 8080. This option is suitable when a GPU is not available.

```docker
docker run -d \
    --name fish-speech-server-cpu \
    -p 8080:8080 \
    -v ./checkpoints:/app/checkpoints \
    -v ./references:/app/references \
    fishaudio/fish-speech:latest-server-cpu
```

--------------------------------

### Command Line Inference - Step 2: Generate Semantic Tokens

Source: https://speech.fish.audio/inference_q=

Generate semantic tokens from input text, optionally using reference text and VQ tokens for voice conditioning.

```APIDOC
## Command Line Inference - Step 2: Generate semantic tokens from text

### Description
Convert input text into semantic tokens. This process can be conditioned on a reference text and its corresponding VQ tokens (from Step 1) to control the voice timbre. The `--compile` flag can significantly speed up inference on compatible GPUs.

### Method
Command Line

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
python fish_speech/models/text2semantic/inference.py \
    --text "The text you want to convert" \
    --prompt-text "Your reference text" \
    --prompt-tokens "fake.npy" \
    --compile

# To use random voice timbre, omit --prompt-text and --prompt-tokens:
python fish_speech/models/text2semantic/inference.py \
    --text "The text you want to convert"
```

### Response
#### Success Response (200)
Creates a `codes_N.npy` file (where N is an integer) in the working directory, containing the generated semantic tokens.

#### Response Example
```
# Output file:
codes_0.npy
```

### Notes
- Use `--compile` for faster inference (e.g., ~150 tokens/sec on RTX 4090).
- Use `--half` if your GPU does not support bf16.
```

--------------------------------

### Manual Docker Build for Speech Fish Audio

Source: https://speech.fish.audio/install

Builds custom Docker images for the Speech Fish Audio project manually. Supports building WebUI and API server images with CUDA or CPU backends, specifying CUDA versions, and targeting different build stages like 'webui', 'server', or 'dev'.

```bash
# Build WebUI image with CUDA support
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target webui \
    -t fish-speech-webui:cuda .

# Build API server image with CUDA support
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --build-arg CUDA_VER=12.6.0 \
    --build-arg UV_EXTRA=cu126 \
    --target server \
    -t fish-speech-server:cuda .

# Build CPU-only images (supports multi-platform)
docker build \
    --platform linux/amd64,linux/arm64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cpu \
    --target webui \
    -t fish-speech-webui:cpu .

# Build development image
docker build \
    --platform linux/amd64 \
    -f docker/Dockerfile \
    --build-arg BACKEND=cuda \
    --target dev \
    -t fish-speech-dev:cuda .

```

--------------------------------

### Command Line: Get VQ Tokens from Reference Audio

Source: https://speech.fish.audio/inference

This script generates VQ tokens from a reference audio file using the DAC model. The output includes 'fake.npy' and 'fake.wav' files, which are used in subsequent steps. This step can be skipped if random voice timbre selection is desired.

```python
python fish_speech/models/dac/inference.py \
    -i "ref_audio_name.wav" \
    --checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth"

```

--------------------------------

### Command Line: Generate Vocals from Semantic Tokens

Source: https://speech.fish.audio/inference

This script synthesizes vocals from the generated semantic tokens using the DAC model. Note that this interface might be removed in future releases, and users are advised to update their code.

```python
python fish_speech/models/dac/inference.py \
    -i "codes_0.npy" \


```

--------------------------------

### Command Line Inference - Step 3: Generate Vocals

Source: https://speech.fish.audio/inference_q=

Generate the final vocal audio from the semantic tokens using the DAC model.

```APIDOC
## Command Line Inference - Step 3: Generate vocals from semantic tokens

### Description
Synthesize the final vocal audio from the semantic tokens generated in the previous step using the DAC model.

### Method
Command Line

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```bash
python fish_speech/models/dac/inference.py \
    -i "codes_0.npy"
```

### Response
#### Success Response (200)
Generates the final vocal audio file (e.g., `fake.wav`) from the provided semantic tokens.

#### Response Example
```
# Output file:
fake.wav
```

### Notes
- The interface `tools/vqgan/inference.py` is kept for backward compatibility but may be removed in future releases.
```

--------------------------------

### Command Line: Generate Semantic Tokens from Text

Source: https://speech.fish.audio/inference

This command generates semantic tokens from input text, optionally using a reference text and its corresponding VQ tokens. The `--compile` flag can significantly speed up inference by fusing CUDA kernels. For GPUs not supporting bf16, the `--half` parameter may be necessary.

```python
python fish_speech/models/text2semantic/inference.py \
    --text "The text you want to convert" \
    --prompt-text "Your reference text" \
    --prompt-tokens "fake.npy" \
    --compile

```

--------------------------------

### Pack Dataset into Protobuf Format

Source: https://speech.fish.audio/finetune_q=

This command packs the prepared dataset, including audio and annotation files, into a protobuf format. It requires the input data directory, output directory for protobuf files, and the text file extension.

```python
python tools/llama/build_dataset.py \
    --input "data" \
    --output "data/protos" \
    --text-extension .lab \
    --num-workers 16
```

--------------------------------

### Download VQGAN Weights for Semantic Token Extraction

Source: https://speech.fish.audio/finetune_q=

This command downloads the necessary VQGAN weights from Hugging Face for extracting semantic tokens. The weights are saved to a specified local directory.

```bash
huggingface-cli download fishaudio/openaudio-s1-mini --local-dir checkpoints/openaudio-s1-mini
```

--------------------------------

### Extract Semantic Tokens using VQGAN

Source: https://speech.fish.audio/finetune_q=

This Python script extracts semantic tokens from audio data using a VQGAN model. It requires the data directory, configuration name, and checkpoint path as inputs. Users can adjust workers and batch size for performance.

```python
python tools/vqgan/extract_vq.py data \
    --num-workers 1 --batch-size 16 \
    --config-name "modded_dac_vq" \
    --checkpoint-path "checkpoints/openaudio-s1-mini/codec.pth"
```

--------------------------------

### Fine-tune LLAMA Model with LoRA

Source: https://speech.fish.audio/finetune_q=

This command initiates the fine-tuning process for the LLAMA model using LoRA. It requires a configuration name and specifies LoRA configuration parameters. Training parameters can be adjusted in the specified YAML file.

```python
python fish_speech/train.py --config-name text2semantic_finetune \
    project=$project \
    +lora@model.model.lora_config=r_8_alpha_16
```

--------------------------------

### Normalize Audio Loudness with fish-audio-preprocess

Source: https://speech.fish.audio/finetune_q=

This command normalizes the loudness of audio files in a dataset using the fish-audio-preprocess tool. It takes the raw data directory as input and outputs the processed data.

```bash
fap loudness-norm data-raw data --clean
```

--------------------------------

### Convert LoRA Weights to Regular Weights

Source: https://speech.fish.audio/finetune_q=

This command converts the fine-tuned LoRA weights back into regular model weights for inference. It requires the LoRA configuration, base model weights, and the LoRA weights checkpoint path.

```python
python tools/llama/merge_lora.py \
    --lora-config r_8_alpha_16 \
    --base-weight checkpoints/openaudio-s1-mini \
    --lora-weight results/$project/checkpoints/step_000000010.ckpt \
    --output checkpoints/openaudio-s1-mini-yth-lora/
```
Tokens:
Raw