### Create Virtual Environment and Install Dependencies

Source: https://github.com/remsky/kokoro-fastapi/blob/master/CONTRIBUTING.md

Create a virtual environment using `uv` and install project dependencies, including test and CPU extras. Activate the environment before installing.

```bash
# Create and activate a virtual environment (e.g., named .venv)
uv venv
source .venv/bin/activate # On Linux/macOS
# .venv\Scripts\activate # On Windows

# Install dependencies including test requirements
uv pip install -e ".[test,cpu]"
```

--------------------------------

### Example Custom values.yaml for Helm Chart

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

An example configuration file for customizing the Kokoro-FastAPI Helm chart installation, specifying replica count, image details, and ingress settings.

```yaml
replicaCount: 2

image:
  repository: ghcr.io/remsky/kokoro-fastapi-gpu
  tag: v0.2.1
  pullPolicy: IfNotPresent

ingress:
    enabled: true
    host:
        name: my-kokoro-endpoint.dev

...
```

--------------------------------

### Install Helm Chart

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

Install the Kokoro-FastAPI Helm chart into a specified namespace, creating it if it does not exist.

```bash
helm install kokoro-fastapi . --namespace <target-namespace> --create-namespace
```

--------------------------------

### Install Ruff Linter

Source: https://github.com/remsky/kokoro-fastapi/blob/master/CONTRIBUTING.md

Install the `ruff` tool using `uv` to ensure code quality and consistency.

```bash
uv pip install ruff
```

--------------------------------

### Install Helm Chart with Custom Values File

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

Install the Kokoro-FastAPI Helm chart using a custom values.yaml file for configuration.

```bash
helm install kokoro-fastapi . --namespace <target-namespace> --create-namespace -f my-custom-values.yaml
```

--------------------------------

### Clone Repository and Start Docker Compose

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-DockerCompose

Clone the Kokoro-FastAPI repository and navigate to the Docker directory. Use Docker Compose to build and start the service, choosing either the GPU or CPU environment.

```bash
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI

cd docker/gpu # OR
# cd docker/cpu # Run this or the above
docker compose up --build
```

--------------------------------

### Start Kokoro-FastAPI with UV (CPU/GPU)

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-uv

Execute the application directly using UV with hot-reloading. Choose the appropriate script based on your hardware (CPU or GPU).

```bash
./start-cpu.sh OR
./start-gpu.sh
```

--------------------------------

### Run Kokoro-FastAPI with Docker

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Start Kokoro-FastAPI using pre-built Docker images for different hardware configurations. Ensure to pin to a release tag for stable usage.

```bash
docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:latest                                       # CPU
docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest                            # NVIDIA (x86_64 or arm64)
docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:latest-cu128                      # NVIDIA Blackwell / RTX 50-series
docker run --device=/dev/kfd --device=/dev/dri -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-rocm:latest  # AMD
```

--------------------------------

### Install OpenAI Python Library

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-OpenAI

Install the OpenAI Python library using pip. This is a prerequisite for using the OpenAI client.

```bash
pip install openai
```

--------------------------------

### Install Helm Chart with Overridden Values

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

Install the Kokoro-FastAPI Helm chart and override specific configuration values directly via command-line arguments.

```bash
helm install kokoro-fastapi . \
  --namespace <target-namespace> \
  --create-namespace \
  --set replicaCount=2 \
  --set image.tag=v0.2.1
```

--------------------------------

### Set up Kokoro-FastAPI with Docker Compose

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Clone the repository and use Docker Compose to start the service, including the UI. Specific Docker Compose files are available for GPU, CPU, or ROCm support. Note that Apple Silicon users should use Docker Compose CPU or native MPS.

```bash
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI

cd docker/gpu   # For NVIDIA GPU support
# or cd docker/cpu   # For CPU support
# or cd docker/rocm  # For AMD GPU (ROCm, experimental, amd64 only)
docker compose up --build

cd ../..  # back to repo root for the paths below

# Models will auto-download, but if needed you can manually download:
python docker/scripts/download_model.py --output api/src/models/v1_0

# Or run directly via UV:
./start-gpu.sh  # For GPU support
./start-cpu.sh  # For CPU support
```

--------------------------------

### Run Pre-built Docker Images

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Docker

Use these commands to run the pre-built CPU or NVIDIA GPU Docker images. Ensure Docker is installed and configured.

```bash
docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:v0.1.4 # CPU, or:
docker run --gpus all -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-gpu:v0.1.4 #NVIDIA GPU
```

--------------------------------

### Stream Speech Directly to Speakers

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-OpenAI

Stream generated speech directly to speakers using PyAudio. This example requires PyAudio to be installed and configured. It measures the time to first byte and the total generation time.

```python
import time
from pathlib import Path

from openai import OpenAI

# gets OPENAI_API_KEY from your environment variables
openai = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed-for-local")


def stream_to_speakers() -> None:
    import pyaudio

    player_stream = pyaudio.PyAudio().open(format=pyaudio.paInt16, channels=1, rate=24000, output=True)

    start_time = time.time()

    with openai.audio.speech.with_streaming_response.create(
        model="kokoro",
        voice="af_bella+af_irulan",
        response_format="pcm",  # similar to WAV, but without a header chunk at the start.
        input="""
                I see skies of blue and clouds of white
                The bright blessed days, the dark sacred nights
                And I think to myself
                What a wonderful world""",
    ) as response:
        print(f"Time to first byte: {int((time.time() - start_time) * 1000)}ms")
        for chunk in response.iter_bytes(chunk_size=1024):
            player_stream.write(chunk)

    print(f"Done in {int((time.time() - start_time) * 1000)}ms.")
```

--------------------------------

### Example Usage: Phonemization and Audio Generation

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Demonstrates converting text to phonemes and then generating audio from those phonemes. Includes error handling for the API requests. The generated audio is saved to 'speech.wav'.

```python
# Example usage
text = "Hello world!"
try:
    # Convert text to phonemes
    phonemes, tokens = get_phonemes(text)
    print(f"Phonemes: {phonemes}")  # e.g. ðɪs ɪz ˈoʊnli ɐ tˈɛst
    print(f"Tokens: {tokens}")      # Token IDs including start/end tokens

    # Generate and save audio
    if audio_bytes := generate_audio_from_phonemes(phonemes):
        with open("speech.wav", "wb") as f:
            f.write(audio_bytes)
        print(f"Generated {len(audio_bytes)} bytes of audio")
except Exception as e:
    print(f"Error: {e}")
```

--------------------------------

### Verify Kubernetes Installation

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

Check that the Kokoro-FastAPI application resources are running in the specified Kubernetes namespace.

```bash
kubectl get all -n <target-namespace>
```

--------------------------------

### Download Kokoro-FastAPI Models

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-DockerCompose

If models are missing after starting the service, use these Python scripts to download the required models for either GPU (pth) or CPU (onnx) environments.

```bash
python ../scripts/download_model.py --type pth  # for GPU
```

```bash
python ../scripts/download_model.py --type onnx # for CPU
```

--------------------------------

### Test with Docker Compose (CPU)

Source: https://github.com/remsky/kokoro-fastapi/blob/master/CONTRIBUTING.md

Build and run the application services using Docker Compose for the CPU environment. This ensures functionality within a containerized setup.

```bash
docker compose -f docker/cpu/docker-compose.yml up --build
```

--------------------------------

### Get Application URL with NodePort

Source: https://github.com/remsky/kokoro-fastapi/blob/master/charts/kokoro-fastapi/templates/NOTES.txt

Execute these commands when the service type is NodePort to get the external IP and NodePort, then construct the URL.

```bash
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "kokoro-fastapi.fullname" . }})
  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT
```

--------------------------------

### Get Application URL with LoadBalancer

Source: https://github.com/remsky/kokoro-fastapi/blob/master/charts/kokoro-fastapi/templates/NOTES.txt

Use these commands for LoadBalancer service types. Note that it may take time for the LoadBalancer IP to become available. You can monitor its status with `kubectl get svc -w`.

```bash
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
      You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "kokoro-fastapi.fullname" . }}'
  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "kokoro-fastapi.fullname" . }} --template "{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}")
  echo http://$SERVICE_IP:{{ .Values.kokoroTTS.port }}
```

--------------------------------

### Get Application URL with Ingress

Source: https://github.com/remsky/kokoro-fastapi/blob/master/charts/kokoro-fastapi/templates/NOTES.txt

Use this when Ingress is enabled to construct the application URL based on configured hosts and paths.

```go-template
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
  {{- range .paths }}
  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
  {{- end }}
{{- end }}
{{- end }}
```

--------------------------------

### Download Model Weights

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-uv

Download necessary model weights for either GPU or CPU execution. Ensure you have the correct Python environment activated.

```bash
# if you are missing any models, run:
# python ../scripts/download_model.py --type pth  # for GPU
# python ../scripts/download_model.py --type onnx # for CPU
```

--------------------------------

### Run Docker Compose for CPU Inference

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Use this command to build and run the CPU inference environment. This is for PyTorch CPU inference.

```bash
cd docker/cpu
docker compose up --build
```

--------------------------------

### Clone Kokoro-FastAPI Repository

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

Clone the repository containing the Kokoro-FastAPI Helm chart and navigate to the chart directory.

```bash
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI/charts/kokoro-fastapi
```

--------------------------------

### Set API Log Level via Environment Variable (Docker)

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Configures the API log level by setting the 'API_LOG_LEVEL' environment variable. This example shows how to do it for a Docker container.

```bash
docker run --env 'API_LOG_LEVEL=WARNING' ...
```

--------------------------------

### Get Public IP Address

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-DigitalOcean

Retrieve your Droplet's public IP address using this command. This IP is needed to access the web interface and configure firewall rules.

```bash
curl -s https://checkip.amazonaws.com
```

--------------------------------

### Clone Kokoro-FastAPI Repository

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-uv

Clone the repository and navigate into the project directory. This step is necessary before running the application.

```bash
git clone https://github.com/remsky/Kokoro-FastAPI.git
cd Kokoro-FastAPI
```

--------------------------------

### Download Models with Bash Script

Source: https://github.com/remsky/kokoro-fastapi/wiki/Troubleshooting-MissingModels

Execute this command to download PyTorch (.pth) or ONNX models. Include --overwrite to replace existing files. Custom URLs are also supported.

```bash
./download_model.sh --type [pth|onnx]
```

```bash
./download_model.sh --type [pth|onnx] [URL1] [URL2] ...
```

--------------------------------

### Set API Log Level via Environment Variable (Windows)

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Configures the API log level on Windows by setting the 'API_LOG_LEVEL' environment variable in PowerShell before starting the application.

```powershell
$env:API_LOG_LEVEL = 'WARNING'
.uild-cpu.ps1 OR
.uild-gpu.ps1
```

--------------------------------

### Download Models with Python Script

Source: https://github.com/remsky/kokoro-fastapi/wiki/Troubleshooting-MissingModels

Use this command to download PyTorch (.pth) or ONNX models. Add --overwrite to replace existing files. Custom URLs can also be provided.

```bash
python download_model.py --type [pth|onnx]
```

```bash
python download_model.py --type [pth|onnx] --overwrite
```

```bash
python download_model.py --type [pth|onnx] [URL1] [URL2] ...
```

--------------------------------

### Test with Docker Compose (GPU)

Source: https://github.com/remsky/kokoro-fastapi/blob/master/CONTRIBUTING.md

Build and run the application services using Docker Compose for the GPU environment. This is for testing GPU-specific features.

```bash
docker compose -f docker/gpu/docker-compose.yml up --build
```

--------------------------------

### Set API Log Level via Environment Variable (Linux/macOS)

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Sets the API log level using the 'API_LOG_LEVEL' environment variable for direct execution on Linux or macOS. This is done before starting the application.

```bash
export API_LOG_LEVEL=WARNING
./start-cpu.sh OR
./start-gpu.sh
```

--------------------------------

### Stream Opus audio and play with ffplay

Source: https://github.com/remsky/kokoro-fastapi/wiki/HowTo-Opus

Initializes the OpenAI client to connect to a local Kokoro FastAPI server. Streams audio in Opus format and pipes the data to ffplay for real-time playback. Measures time to first byte and total processing time.

```python
import time
import subprocess
from pathlib import Path
from io import BytesIO
from openai import OpenAI

# Initialize OpenAI client pointing to local Kokoro FastAPI server
client = OpenAI(base_url="http://localhost:8880/v1", api_key="not-needed-for-local")
def main():
    print("1. Starting Opus stream...")
    start_time = time.time()
    
    # Start ffplay process
    process = subprocess.Popen(
        ['ffplay', '-'],  # Read from stdin
        stdin=subprocess.PIPE,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL
    )
    
    try:
        # Create streaming response with Opus format
        with client.audio.speech.with_streaming_response.create(
            model="kokoro",
            voice="af_bella",
            response_format="opus",
            input="Testing Opus streaming and decoding"
        ) as response:
            # Process chunks as they arrive
            first_chunk = True
            
            for chunk in response.iter_bytes(chunk_size=1024):
                if first_chunk:
                    ttfb = time.time() - start_time
                    print(f"Time to first byte: {ttfb*1000:.0f}ms")
                    first_chunk = False
                
                # Write chunk to ffplay
                process.stdin.write(chunk)
                process.stdin.flush()
    
    finally:
        # Cleanup
        process.stdin.close()
        process.wait()
    
    print(f"\nTotal processing time: {(time.time() - start_time)*1000:.0f}ms")

if __name__ == "__main__":
    main()

```

--------------------------------

### Inline Control Tokens Example

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Illustrates the use of inline control tokens for pauses and custom pronunciations within input text. The pause token format is '[pause:1.5s]' and pronunciation uses IPA within slashes '[Word/IPA/]'.

```text
The city of [Worcester](/wˈʊstər/) is easy. [pause:1s] See?
```

--------------------------------

### Run Docker Compose for GPU Inference

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Use this command to build and run the GPU inference environment. Requires NVIDIA drivers with CUDA 12.6+ support.

```bash
cd docker/gpu
docker compose up --build
```

--------------------------------

### OpenWebUI Docker Base URL Configuration

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-OpenWebUI

When running OpenWebUI in Docker, set this base URL to allow communication with the Kokoro FastAPI wrapper.

```text
http://host.docker.internal:8880/v1
```

--------------------------------

### Test All Available Voices

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Run this script to test all available voices.

```bash
python examples/assorted_checks/test_voices/test_all_voices.py # Test all available voices
```

--------------------------------

### Run Transcription Tests

Source: https://github.com/remsky/kokoro-fastapi/blob/master/examples/assorted_checks/test_transcription/README.md

Execute the transcription roundtrip tests from the repository root. Omit '--extra transcription-gpu' to use CPU for Whisper.

```bash
uv sync --project examples --extra transcription --extra transcription-gpu
uv run --project examples python examples/assorted_checks/test_transcription/test_transcription.py
```

--------------------------------

### Use Kokoro-FastAPI as an OpenAI-Compatible Speech Endpoint

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

This Python snippet demonstrates how to interact with the Kokoro-FastAPI service running locally as an OpenAI-compatible speech endpoint. It shows how to create a client and stream speech synthesis.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8880/v1", api_key="not-needed"
)

with client.audio.speech.with_streaming_response.create(
    model="kokoro",
    voice="af_sky+af_bella", #single or multiple voicepack combo
    input="Hello world!"
  ) as response:
      response.stream_to_file("output.mp3")
```

--------------------------------

### OpenWebUI Non-Docker Base URL Configuration

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-OpenWebUI

If OpenWebUI is not running in Docker, use this base URL to connect to the Kokoro FastAPI wrapper.

```text
http://localhost:8880/v1
```

--------------------------------

### Access Application with ClusterIP and Port Forwarding

Source: https://github.com/remsky/kokoro-fastapi/blob/master/charts/kokoro-fastapi/templates/NOTES.txt

When using ClusterIP, this command sets up port forwarding to access the application locally via `http://127.0.0.1:8880`.

```bash
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "kokoro-fastapi.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8880 to use your application"
  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8880:$CONTAINER_PORT
```

--------------------------------

### Generate Speech to File with OpenAI Client

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-OpenAI

Use the OpenAI client to generate speech from text and save it to an MP3 file. Ensure the OpenAI client is configured with the correct base URL and API key.

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8880/v1", api_key="not-needed"
)

with client.audio.speech.with_streaming_response.create(
    model="kokoro",
    voice="af_sky+af_bella",  # Single or multiple voicepack combo
    input="Hello world!"
) as response:
    response.stream_to_file("output.mp3")
```

--------------------------------

### Run kokoro-fastapi Docker Container

Source: https://github.com/remsky/kokoro-fastapi/wiki/Integrations-DigitalOcean

Execute this command on your Droplet to pull and run the kokoro-fastapi Docker container. It maps port 8880 from the container to the Droplet.

```bash
docker run -p 8880:8880 ghcr.io/remsky/kokoro-fastapi-cpu:v0.2.2
```

--------------------------------

### Update Helm Deployment with Custom Values

Source: https://github.com/remsky/kokoro-fastapi/wiki/Setup-Kubernetes

Update an existing Kokoro-FastAPI deployment using the Helm upgrade command with a custom values file.

```bash
helm upgrade kokoro-fastapi . -n <target-namespace> -f my-custom-values.yaml
```

--------------------------------

### Run Docker Compose for AMD GPU Inference

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Use this command to build and run the AMD GPU inference environment. This is experimental and requires ROCm 6.4.

```bash
cd docker/rocm
docker compose up --build
```

--------------------------------

### Benchmark Model Unload Script

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

Reproduce model unload benchmarks using this command. The script is located in examples/assorted_checks/benchmarks/.

```bash
uv run --extra benchmarks assorted_checks/benchmarks/benchmark_model_unload.py
```

--------------------------------

### Format Code with Ruff

Source: https://github.com/remsky/kokoro-fastapi/blob/master/CONTRIBUTING.md

Apply code formatting using `ruff format` to ensure code style consistency across the project.

```bash
# Make sure your virtual environment is activated
ruff format .
```

--------------------------------

### Configure Docker Compose for Linux GPU Permissions (Container Groups)

Source: https://github.com/remsky/kokoro-fastapi/blob/master/README.md

When running Kokoro-TTS in a Docker container on Linux, add 'video' and 'render' to the 'group_add' service configuration to grant necessary GPU permissions.

```yaml
services:
  kokoro-tts:
    # ... existing config ...
    group_add:
      - "video"
      - "render"
```