### Run llama.cpp Server with Benchmark First Mode

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Starts the llama.cpp server in 'Benchmark First' mode, which runs a benchmark before starting the main inference server. Useful for performance testing. Specify the model and mount workspace.

```bash
docker run -d --gpus all -p 8080:8080 \
  -e MODE=BENCH_FIRST \
  -e LLAMA_MU="https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" \
  -v /workspace:/workspace \
  dynafire/llama.cpp
```

--------------------------------

### Running Docker Utility Images with Custom Ports and Paths

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/sd-webui-infinite-image-browsing/README.md

Examples demonstrate how to run the image with custom PORT and EXTRA_PATHS environment variables, and various volume mounting configurations.

```docker
docker run --env PORT=8081 --env EXTRA_PATHS=/outputs -v /outputs_location:/outputs -d -p 8081:8081 dynafire/sd-webui-infinite-image-browsing
```

```docker
docker run -v /outputs_location:/outputs -it -p 8081:8080 dynafire/sd-webui-infinite-image-browsing
```

```docker
docker run -v /outputs_location:/outputs:ro -d -p 8080:8080 dynafire/sd-webui-infinite-image-browsing
```

```docker
docker run --env EXTRA_OPTIONS="--allow_cors --update_image_index" -v /outputs_location:/outputs:ro -d -p 8080:8080 dynafire/sd-webui-infinite-image-browsing
```

--------------------------------

### Start whisper.cpp Speech-to-Text Server

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Launches the whisper.cpp server for speech-to-text transcription. Specify the desired quantized model and port. Mount a volume for workspace access.

```bash
docker run -d --gpus all -p 8200:8200 \
  -e MODEL=small-q5_1 \
  -e PORT=8200 \
  -v /workspace:/workspace \
  dynafire/whisper.cpp
```

--------------------------------

### Run llama.cpp Server with Model Download

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Starts the llama.cpp inference server, downloading a specified model from Hugging Face. Configure context size, KV cache, and GPU layers for optimal performance. An API key can be set for authentication.

```bash
docker run -d --gpus all -p 8080:8080 \
  -e LLAMA_MU="https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF/resolve/main/llama-2-7b-chat.Q4_K_M.gguf" \
  -e LLAMA_CONTEXT=4096 \
  -e LLAMA_UB=1024 \
  -e LLAMA_FA="-fa" \
  -e LLAMA_NGL="-ngl 200" \
  -e LLAMA_API_KEY="your-secret-key" \
  -v /workspace:/workspace \
  dynafire/llama.cpp
```

--------------------------------

### GET /docs

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Serves a rendered HTML version of the API documentation.

```APIDOC
## GET /docs

### Description
The /docs endpoint serves a rendered HTML version of the API documentation.

### Method
GET

### Endpoint
/docs

### Response
#### Success Response (200)
- **html** (text/html) - Rendered API documentation.
```

--------------------------------

### Run llama.cpp Server in Sleep Mode

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Starts the llama.cpp server in 'Sleep' mode, allowing for manual debugging. This mode keeps the container running without immediately starting the inference server.

```bash
docker run -it --gpus all \
  -e MODE=SLEEP \
  -v /workspace:/workspace \
  dynafire/llama.cpp
```

--------------------------------

### GET /v1/models

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Returns a list of available voice models.

```APIDOC
## GET /v1/models

### Description
The /v1/models endpoint returns a list of available voice models.

### Method
GET

### Endpoint
/v1/models

### Response
#### Success Response (200)
- **model** (string) - The model name.
- **voicelist** (array) - List of available voice identifiers.

#### Response Example
{ "model": "StyleTTS2", "voicelist": ["f-us-1", "f-us-2"] }
```

--------------------------------

### Start StyleTTS2 API Container

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Launches the StyleTTS2 text-to-speech API container. Mount a volume for additional voices if needed. Ensure the exposed port matches the container's PORT environment variable.

```bash
docker run -d -p 5000:5000 \
  -e PORT=5000 \
  -e ADDITIONAL_VOICE_DIR=/voices \
  -v /path/to/voices:/voices \
  dynafire/styletts2-api
```

--------------------------------

### Get StyleTTS2 API Documentation (HTML)

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Retrieves the API documentation for StyleTTS2 in HTML format.

```bash
curl http://localhost:5000/docs
```

--------------------------------

### GET /docs.md

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Serves the raw Markdown file of the API documentation.

```APIDOC
## GET /docs.md

### Description
The /docs.md endpoint serves the raw Markdown file of the API documentation.

### Method
GET

### Endpoint
/docs.md

### Response
#### Success Response (200)
- **markdown** (text/markdown) - Raw markdown content.
```

--------------------------------

### Models Success Response

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Example JSON response containing the model name and available voices.

```json
{
    "model":"StyleTTS2",
    "voicelist":["f-us-1","f-us-2","f-us-3","f-us-4","m-us-1","m-us-2","m-us-3","m-us-4"]
}
```

--------------------------------

### Get StyleTTS2 API Documentation (Markdown)

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Retrieves the API documentation for StyleTTS2 in raw Markdown format.

```bash
curl http://localhost:5000/docs.md
```

--------------------------------

### GET /health

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

A simple health check that returns OK if the service is running.

```APIDOC
## GET /health

### Description
The /health endpoint is a simple health check that returns "OK" if the service is running properly.

### Method
GET

### Endpoint
/health

### Response
#### Success Response (200)
- **status** (string) - Returns "OK".
```

--------------------------------

### Inference Error Response

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Example JSON structure returned when a request fails due to missing or invalid parameters.

```json
{
  "error": "Missing or invalid fields. Please include 'text' and 'voice' in your request, and ensure the voice selected is valid."
}
```

--------------------------------

### Run llama.cpp Server with Quantized KV Cache

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Launches the llama.cpp server, prioritizing memory efficiency by using specified quantized KV cache types. Configure context size and model download URL.

```bash
docker run -d --gpus all -p 8080:8080 \
  -e LLAMA_MU="https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf" \
  -e LLAMA_CTK="q8_0" \
  -e LLAMA_CTV="q5_1" \
  -e LLAMA_CONTEXT=8192 \
  -v /workspace:/workspace \
  dynafire/llama.cpp
```

--------------------------------

### Run Stable Diffusion Infinite Image Browsing

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Deploy the image browser with various mounting and configuration options.

```bash
# Basic usage - mount outputs directory
docker run -d -p 8080:8080 \
  -v /path/to/sd-outputs:/outputs \
  dynafire/sd-webui-infinite-image-browsing

# Custom port and read-only mount
docker run -d -p 8081:8081 \
  -e PORT=8081 \
  -v /outputs_location:/outputs:ro \
  dynafire/sd-webui-infinite-image-browsing

# Enable CORS and image index updates
docker run -d -p 8080:8080 \
  -e EXTRA_OPTIONS="--allow_cors --update_image_index" \
  -e EXTRA_PATHS=/outputs \
  -v /outputs_location:/outputs:ro \
  dynafire/sd-webui-infinite-image-browsing

# Diffuzers support variant
docker run -d -p 8080:8080 \
  -v /outputs_location:/outputs \
  dynafire/sd-webui-iib-diffuzers
```

--------------------------------

### Reference AI/ML Environment Variables

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Configuration keys for customizing Llama.cpp, Whisper.cpp, and StyleTTS2 deployments.

```bash
# llama.cpp Environment Variables
LLAMA_MU         # Hugging Face model URL for auto-download
LLAMA_CONTEXT    # Context window size (default: 4096)
LLAMA_UB         # Batch size, tune for VRAM (2048 for 40GB+, 1024 for 24GB)
LLAMA_FA         # Flash attention flag ("-fa" to enable, "" to disable)
LLAMA_CTK        # Quantized KV cache key type (e.g., "q8_0")
LLAMA_CTV        # Quantized KV cache value type (e.g., "q5_1")
LLAMA_NGL        # GPU layers to offload (e.g., "-ngl 200")
LLAMA_API_KEY    # Optional API key for authentication
LLAMA_NP         # Parallel slots / concurrency level
MODE             # Operation mode: "", "SLEEP", "BENCH_FIRST", "SERVERLESS"
RUNPOD_HANDLER   # Handler type: "", "ASYNC_GENERATOR", "OLD_HTTPX", "NEW_HTTPX"
WORKSPACE        # Volume mount path for models (default: /workspace)

# whisper.cpp Environment Variables
MODEL            # Whisper model name (e.g., "small-q5_1", "large-v3-q5_0")
PORT             # Server port (default: 8200)
ADDITIONAL_ARGS  # Extra server arguments

# StyleTTS2 Environment Variables
PORT             # Server port (default: 5000)
ADDITIONAL_VOICE_DIR  # Path to additional voice .wav files
```

--------------------------------

### Run Whisper.cpp Inference Container

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Deploy the Whisper.cpp container with GPU support and specific model configurations.

```bash
docker run -d --gpus all -p 8200:8200 \
  -e MODEL=large-v3-q5_0 \
  -e PORT=8200 \
  -e ADDITIONAL_ARGS="--host 0.0.0.0 --convert --inference-path /whisper" \
  -v /workspace:/workspace \
  dynafire/whisper.cpp
```

```bash
curl -X POST http://localhost:8200/whisper/inference \
  -F "file=@audio.wav" \
  -F "response_format=json"
```

```bash
curl http://localhost:8200/v1/models
# Response: alive
```

--------------------------------

### Retrieve Documentation

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Endpoints for fetching API documentation in HTML or Markdown formats.

```bash
curl http://your-server-url/docs
```

```bash
curl http://your-server-url/docs.md
```

--------------------------------

### Run llama.cpp in RunPod Serverless Mode

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Configures the llama.cpp container to run in RunPod Serverless mode. Specify the handler and number of processes. Mount the network storage for workspace access.

```bash
docker run -d --gpus all \
  -e MODE=SERVERLESS \
  -e RUNPOD_HANDLER=ASYNC_GENERATOR \
  -e LLAMA_NP=2 \
  -v /network-storage:/workspace \
  dynafire/llama.cpp
```

--------------------------------

### List Available Models

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Retrieves a list of supported voice models.

```bash
curl http://your-server-url/v1/models
```

--------------------------------

### Use Docker-Helm-Kubectl Image for CI/CD

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Utilize the Alpine-based image for Kubernetes deployments and interactive tool verification.

```yaml
# Use in GitLab CI/CD pipeline
# .gitlab-ci.yml
deploy:
  image: dynafire/docker-helm-kubectl
  script:
    - kubectl config set-cluster mycluster --server=$K8S_SERVER
    - kubectl config set-credentials admin --token=$K8S_TOKEN
    - kubectl config set-context default --cluster=mycluster --user=admin
    - kubectl config use-context default
    - helm upgrade --install myapp ./charts/myapp -f values.yaml
```

```bash
# Interactive usage
docker run -it --rm \
  -v ~/.kube:/root/.kube \
  -v $(pwd):/config \
  dynafire/docker-helm-kubectl
```

```bash
# Verify tools installed
docker run --rm dynafire/docker-helm-kubectl sh -c "helm version && kubectl version --client && docker version"
```

--------------------------------

### Perform Inference Request

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Sends a POST request to generate audio from text using a specified voice model.

```bash
curl -X POST http://your-server-url/v2/inference      -F "text=Hello, world!"      -F "voice=f-us-2"      -F "format=wav"      --output output.wav
```

--------------------------------

### Default Environment Variables for Docker Utility Images

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/sd-webui-infinite-image-browsing/README.md

These are the default values for PORT and EXTRA_PATHS if not explicitly provided.

```shell
PORT=8080
EXTRA_PATHS=/outputs
```

--------------------------------

### Provision RAID Arrays with Tinkerbell

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Create software RAID arrays using the createmd action in standalone or workflow contexts.

```bash
# Run directly to create RAID-1 mirror
docker run --privileged \
  -e DEVICE1=/dev/nvme0n1 \
  -e DEVICE2=/dev/nvme1n1 \
  -e MDDEV=/dev/md_d0 \
  dynafire/createmd

# Create single-disk RAID (for consistency in provisioning)
docker run --privileged \
  -e DEVICE1=/dev/nvme0n1 \
  dynafire/createmd /dev/nvme0n1
```

```yaml
# Tinkerbell workflow template usage
actions:
  - name: "create-raid"
    image: dynafire/createmd
    timeout: 300
    environment:
      DEVICE1: /dev/nvme0n1
      DEVICE2: /dev/nvme1n1
      MDDEV: /dev/md_d0
      STYLE: mdp
```

--------------------------------

### List Available Voices in StyleTTS2 API

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Fetches a list of available voice models from the StyleTTS2 API. This is useful for determining which voices can be used for synthesis.

```bash
curl http://localhost:5000/v1/models
```

--------------------------------

### Send Completion Request to llama.cpp Server

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Sends a JSON payload to the llama.cpp server's `/completion` endpoint to generate text. Includes prompt, prediction count, temperature, and streaming options. Ensure the Authorization header matches the configured API key.

```bash
curl http://localhost:8080/completion \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-secret-key" \
  -d '{
    "prompt": "Write a haiku about programming:",
    "n_predict": 128,
    "temperature": 0.7,
    "stream": false
  }'
```

--------------------------------

### Generate Speech with Style Transfer using StyleTTS2 API

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Utilizes the StyleTTS2 API for speech synthesis with style transfer. Provide a reference text to influence the speaking style of the generated audio. Adjust `alpha` and `beta` for style control.

```bash
curl -X POST http://localhost:5000/styletts2/v2/inference \
  -F "text=This sentence will be spoken with the referenced style." \
  -F "voice=m-us-2" \
  -F "ref_text=I am speaking slowly and clearly." \
  -F "alpha=0.3" \
  -F "beta=0.7" \
  -F "format=wav" \
  --output styled_speech.wav
```

--------------------------------

### POST /v2/inference

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Generates audio based on provided text and voice model, with optional style transfer and fine-tuning parameters.

```APIDOC
## POST /v2/inference

### Description
The /v2/inference endpoint generates audio based on the provided text and voice model. Optionally, a reference text (ref_text) can be provided for style transfer using the STinference function.

### Method
POST

### Endpoint
/v2/inference

### Parameters
#### Request Body
- **text** (string) - Required - The text input that will be converted into audio.
- **voice** (string) - Required - The voice model to be used for synthesis.
- **ref_text** (string) - Optional - An optional reference text used for style transfer.
- **format** (string) - Optional - The audio format (mp3, opus, wav, wav-full). Defaults to mp3.
- **bitrate** (string) - Optional - Audio bitrate for mp3 or opus. Defaults to 64k.
- **alpha** (float) - Optional - Controls variability. Default: 0.3.
- **beta** (float) - Optional - Controls stylistic emphasis. Default: 0.7.
- **embedding_scale** (float) - Optional - Adjusts voice embedding. Default: 1.

### Request Example
curl -X POST http://your-server-url/v2/inference -F "text=Hello, world!" -F "voice=f-us-2" -F "format=wav" --output output.wav

### Response
#### Success Response (200)
- **audio** (binary) - The generated audio file.

#### Response Example
{ "error": "Missing or invalid fields." }
```

--------------------------------

### Generate Speech with StyleTTS2 API

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Sends a POST request to the StyleTTS2 API to synthesize speech. Supports specifying text, voice, audio format, and bitrate. Use `--output` to save the generated audio.

```bash
curl -X POST http://localhost:5000/styletts2/v2/inference \
  -F "text=Hello, this is a test of the text to speech system." \
  -F "voice=f-us-1" \
  -F "format=mp3" \
  -F "bitrate=128k" \
  --output speech.mp3
```

--------------------------------

### Service Health Check

Source: https://gitlab.com/ggong/docker-utility-images/-/blob/main/styletts2-api/API_DOCS.md

Verifies that the service is operational.

```bash
curl http://your-server-url/health
```

--------------------------------

### StyleTTS2 API Health Check

Source: https://context7.com/ggong/docker-utility-images/llms.txt

Performs a health check on the StyleTTS2 API. Expects an 'OK' response if the API is running correctly.

```bash
curl http://localhost:5000/health
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.