### Install gpt-3.5-turbo example

Source: https://localai.io/docs/models/index.html

An example of installing a model and renaming it to gpt-3.5-turbo.

```bash
LOCALAI=http://localhost:8080
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
      "url": "github:mudler/LocalAI/gallery/gpt4all-j.yaml",
      "name": "gpt-3.5-turbo"
   }'
```

--------------------------------

### Get an instruction guide in markdown

Source: https://localai.io/docs/features/api-discovery/index.print.html

Retrieves a markdown guide for a specific instruction set by name.

```bash
curl http://localhost:8080/api/instructions/config-management
```

--------------------------------

### Install a vision model from the gallery

Source: https://localai.io/docs/features/gpt-vision/index.html

Use the CLI to install a vision-capable model from the gallery. The example uses `moondream2-20250414`, a small vision model. This command is also shown in the Setup section.

```bash
local-ai run moondream2-20250414
```

--------------------------------

### Install hermes-2-pro-mistral model

Source: https://localai.io/docs/models/index.html

A specific example of installing the hermes-2-pro-mistral model using its configuration URL.

```bash
LOCALAI=http://localhost:8080
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
     "config_url": "https://raw.githubusercontent.com/mudler/LocalAI/v2.25.0/embedded/models/hermes-2-pro-mistral.yaml"
   }'
```

--------------------------------

### Example Watchdog Deployment and API Usage

Source: https://localai.io/docs/advanced/vram-management/index.html

A complete example showing LocalAI startup with both watchdogs enabled, followed by example API requests to models.

```bash
LOCALAI_WATCHDOG_IDLE=true \
LOCALAI_WATCHDOG_IDLE_TIMEOUT=10m \
LOCALAI_WATCHDOG_BUSY=true \
LOCALAI_WATCHDOG_BUSY_TIMEOUT=5m \
./local-ai

curl http://localhost:8080/v1/chat/completions -d '{"model": "model-a", ...}'
curl http://localhost:8080/v1/chat/completions -d '{"model": "model-b", ...}'
```

--------------------------------

### Check and install backend

Source: https://localai.io/docs/getting-started/models/index.print.html

List installed backends and install llama-cpp if needed.

```bash
local-ai backends list
local-ai backends install llama-cpp  # if needed
```

--------------------------------

### GET /api/instructions/:name

Source: https://localai.io/docs/features/api-discovery/index.print.html

Fetches a detailed instruction guide for a specific area. By default returns a markdown guide; with ?format=json returns a raw OpenAPI fragment.

```APIDOC
## GET /api/instructions/:name

### Description
Returns a markdown guide for a specific instruction area. Optionally, add ?format=json to get a raw OpenAPI fragment (filtered Swagger spec with only relevant paths and definitions).

### Method
GET

### Endpoint
/api/instructions/:name

### Parameters
#### Path Parameters
- **name** (string) - Required - The instruction name (e.g., config-management)

#### Query Parameters
- **format** (string) - Optional - Set to 'json' to receive an OpenAPI fragment instead of markdown

### Request Example
curl http://localhost:8080/api/instructions/config-management

### Response
#### Success Response (200)
Returns a markdown guide by default, or an OpenAPI fragment when ?format=json is used.

#### Response Example
(No explicit response example provided in the source.)
```

--------------------------------

### Install host dependencies with setup.sh

Source: https://localai.io/docs/integrations/index.html

Installs the required host dependencies. Requires sudo privileges.

```bash
sudo bash setup.sh
```

--------------------------------

### Install longcat-video model

Source: https://localai.io/docs/features/video-generation/index.html

Use the CLI to install the longcat-video model from the gallery.

```bash
local-ai run longcat-video
```

--------------------------------

### Install audio-transform model with local-ai run

Source: https://localai.io/docs/features/audio-transform/index.print.html

Installs the audio-transform model `localvqe-v1.3-4.8m` from the gallery before using the endpoint.

```bash
local-ai run localvqe-v1.3-4.8m
```

--------------------------------

### Response from /models/apply

Source: https://localai.io/docs/models/index.html

Example JSON response from the /models/apply endpoint, containing a UUID and a status URL to follow up on the installation process.

```json
{ "uuid":"251475c9-f666-11ed-95e0-9a8a4480ac58", "status":"http://localhost:8080/models/jobs/251475c9-f666-11ed-95e0-9a8a4480ac58"}
```

--------------------------------

### Get an instruction guide as OpenAPI fragment

Source: https://localai.io/docs/features/api-discovery/index.print.html

Retrieves a raw OpenAPI fragment for a specific instruction set by appending the ?format=json query parameter.

```bash
curl http://localhost:8080/api/instructions/config-management?format=json
```

--------------------------------

### Build LocalAI on Mac (M1/M2/M3)

Source: https://localai.io/docs/basics/build/index.html

Complete build and run example for Mac (M1, M2, M3) without Docker. Installs prerequisites via brew, clones the repo, builds, downloads a model, installs a backend, and runs the server with a test request.

```bash
brew install abseil cmake go grpc protobuf wget protoc-gen-go protoc-gen-go-grpc

git clone https://github.com/go-skynet/LocalAI.git

cd LocalAI

make build

wget https://huggingface.co/TheBloke/phi-2-GGUF/resolve/main/phi-2.Q2_K.gguf -O models/phi-2.Q2_K

cp -rf prompt-templates/ggml-gpt4all-j.tmpl models/phi-2.Q2_K.tmpl

./local-ai backends install llama-cpp

./local-ai --models-path=./models/ --debug=true

curl http://localhost:8080/v1/models

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "phi-2.Q2_K",
     "messages": [{"role": "user", "content": "How are you?"}],
     "temperature": 0.9 
   }'
```

--------------------------------

### Install audio-cpp backend via CLI

Source: https://localai.io/docs/features/audio-cpp/index.html

Use the LocalAI CLI to install the audio-cpp backend.

```bash
local-ai backends install audio-cpp
```

--------------------------------

### Run the voice assistant with run.sh

Source: https://localai.io/docs/integrations/index.html

Starts the voice assistant application. Run this after installing host dependencies.

```bash
bash run.sh
```

--------------------------------

### Install models at startup with local-ai run

Source: https://localai.io/docs/basics/getting_started/index.html

Installs models automatically before starting the API by passing model identifiers as arguments to 'local-ai run'. Supports gallery names, Hugging Face, Ollama, raw URLs, and OCI sources.

```bash
local-ai run qwen3-4b
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
local-ai run ollama://gemma:2b
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
local-ai run oci://localai/phi-2:latest
```

--------------------------------

### Install model from gallery

Source: https://localai.io/docs/models/index.html

Install a model by sending a POST request to the /models/apply endpoint with the model identifier.

```bash
LOCALAI=http://localhost:8080
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
     "id": "localai@bert-embeddings"
   }'
```

--------------------------------

### Install a model via CLI

Source: https://localai.io/docs/getting-started/models/index.print.html

Use this command to install only the specified model without starting the runtime.

```bash
local-ai models install hermes-2-theta-llama-3-8b
```

--------------------------------

### Install a vllm.cpp model

Source: https://localai.io/docs/features/vllm-cpp/index.html

Use the local-ai CLI to install a specific model configuration from the gallery.

```bash
local-ai models install qwen3-0.6b-vllm-cpp
```

--------------------------------

### Install qwen3-4b model via CLI

Source: https://localai.io/docs/getting-started/first-agent/index.html

Run this command to install the qwen3-4b model, a small CPU-friendly Qwen3 model that supports tool calling. This is required before creating an agent.

```bash
local-ai run qwen3-4b
```

--------------------------------

### Install video models via CLI

Source: https://localai.io/docs/features/video-generation/index.html

Install the base and avatar video models using the LocalAI CLI.

```bash
local-ai models install longcat-video
local-ai models install longcat-video-avatar-1.5
```

--------------------------------

### Example with custom tokens and tag pairs

Source: https://localai.io/docs/advanced/model-configuration/index.html

Shows how to define custom thinking start tokens and tag pairs. Custom tokens and tag pairs are checked before the default ones, giving them priority.

```yaml
name: custom-reasoning-model
backend: llama-cpp
parameters:
  model: custom.gguf

reasoning:
  thinking_start_tokens:
    - "<custom:think>"
    - "<my:reasoning>"
  tag_pairs:
    - start: "<custom:think>"
      end: "</custom:think>"
    - start: "<my:reasoning>"
      end: "</my:reasoning>"
```

--------------------------------

### Install turboquant backend

Source: https://localai.io/docs/features/text-generation

Use the LocalAI CLI to install the turboquant backend container image.

```bash
local-ai backends install turboquant
```

--------------------------------

### Install Go protobuf compiler plugins

Source: https://localai.io/docs/basics/build/index.html

Install the Go protobuf compiler plugins required for compiling the golang protobuf components.

```bash
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@1958fcbe2ca8bd93af633f11e97d44e567e945af
```

--------------------------------

### Clone the LocalAI-examples repository

Source: https://localai.io/docs/integrations/index.html

Clones the LocalAI-examples repository and changes into the realtime example directory. This is the first step in setting up the realtime voice assistant example.

```bash
git clone https://github.com/mudler/LocalAI-examples.git
cd LocalAI-examples/realtime
```

--------------------------------

### Example PRELOAD_MODELS configuration

Source: https://localai.io/docs/models/index.html

An example of setting the PRELOAD_MODELS environment variable with a specific model URL.

```bash
PRELOAD_MODELS=[{"url": "github:mudler/LocalAI/gallery/stablediffusion.yaml@master"}]
```

--------------------------------

### Install NeMo-Speech.cpp backend

Source: https://localai.io/docs/features/nemo-speech-cpp/index.html

Use the LocalAI CLI to install the nemo-speech-cpp backend.

```bash
local-ai backends install nemo-speech-cpp
```

--------------------------------

### GET /api/instructions/{name}

Source: https://localai.io/docs/features/api-discovery/index.print.html

Returns an API guide for a specific instruction area. This endpoint is publicly accessible without authentication.

```APIDOC
## GET /api/instructions/{name}

### Description
Returns an API guide for a specific instruction area.

### Method
GET

### Endpoint
/api/instructions/{name}

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the instruction area.

### Response
#### Success Response (200)
- **guide** (string) - The API guide content for the instruction area.

#### Response Example
{
  "guide": "# Config Management\n..."
}
```

--------------------------------

### Install vibevoice-cpp-asr model

Source: https://localai.io/docs/features/audio-diarization/index.html

Use the LocalAI CLI to install the diarization-capable vibevoice-cpp-asr model.

```bash
local-ai run vibevoice-cpp-asr
```

--------------------------------

### Install Opus backend via CLI

Source: https://localai.io/docs/features/openai-realtime

Install the Opus backend using the LocalAI CLI for local binary installations.

```bash
local-ai backends install opus
```

--------------------------------

### Install base model with extra files

Source: https://localai.io/docs/models/index.html

Installing a base model while providing specific file URIs and SHA256 checksums.

```bash
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
     "url": "github:mudler/LocalAI/gallery/base.yaml@master",
     "name": "model-name",
     "files": [
        {
            "uri": "<URL>",
            "sha256": "<SHA>",
            "filename": "model"
        }
     ]
   }'
```

--------------------------------

### Install a specific model

Source: https://localai.io/docs/getting-started/models

Installs a specific model from the gallery using its model name.

```bash
local-ai models install llama-3.2-1b-instruct:q4_k_m
```

--------------------------------

### List and install models with local-ai models

Source: https://localai.io/docs/basics/getting_started/index.html

Manages models from the CLI: lists available models in the gallery and installs a model by name. Comments show the purpose of each command.

```bash
local-ai models list          # List available models in the gallery
local-ai models install <name> # Install a model
```

--------------------------------

### Example Chat Template Configuration

Source: https://localai.io/docs/advanced/model-configuration

A YAML configuration example demonstrating how to define a chat template using Go template syntax to iterate over messages and format user and assistant roles.

```yaml
template:
  chat: |
    {{.System}}
    {{range .Messages}}
    {{if eq .Role "user"}}User: {{.Content}}{{end}}
    {{if eq .Role "assistant"}}Assistant: {{.Content}}{{end}}
    {{end}}
    Assistant:
```

--------------------------------

### Install fish completion

Source: https://localai.io/docs/reference/shell-completion/index.html

Source the completion script directly or install it permanently in the fish completions directory.

```bash
local-ai completion fish | source
```

```bash
local-ai completion fish > ~/.config/fish/completions/local-ai.fish
```

--------------------------------

### Install insightface-opencv backend

Source: https://localai.io/docs/features/face-recognition/index.html

Installs the commercial-safe insightface backend (Apache 2.0) for production use. Recommended for copy-paste.

```bash
local-ai models install insightface-opencv
```

--------------------------------

### Complete example with reasoning configuration

Source: https://localai.io/docs/advanced/model-configuration/index.html

Shows a full model configuration with all reasoning fields set to their defaults.

```yaml
name: deepseek-model
backend: llama-cpp
parameters:
  model: deepseek.gguf

reasoning:
  disable: false
  disable_reasoning_tag_prefill: false
  strip_reasoning_only: false
```

--------------------------------

### Install and run a model from Hugging Face

Source: https://localai.io/docs/getting-started/models/index.print.html

Installs and runs a model from Hugging Face using the `huggingface://` URI scheme. The format is `huggingface://<repository>/<model-file>` where the model file is optional.

```bash
local-ai run huggingface://TheBloke/phi-2-GGUF
```

--------------------------------

### Install bash completion

Source: https://localai.io/docs/reference/shell-completion/index.html

Add the source command to your ~/.bashrc or install system-wide.

```bash
source <(local-ai completion bash)
```

```bash
local-ai completion bash > /etc/bash_completion.d/local-ai
```

--------------------------------

### VAE tiling options example

Source: https://localai.io/docs/features/image-generation/index.html

Example options block enabling VAE tiling with a tile size of 512 pixels.

```yaml
options:
- "diffusion_model"
- "sampler:euler"
- "vae_tiling:true"
- "vae_tile_size:512"
```

--------------------------------

### Install face-detect-yunet-sface model

Source: https://localai.io/docs/features/face-recognition/index.html

Installs the commercial-safe face-detect entry, recommended for production deployments.

```bash
local-ai models install face-detect-yunet-sface
```

--------------------------------

### Function calling with curl

Source: https://localai.io/docs/basics/try/index.html

Sends a chat completion request that defines a function tool for getting the weather and sets tool_choice to auto. The example assumes the qwen3-4b model is installed.

```bash
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-4b",
    "messages": [
      {
        "role": "user",
        "content": "What is the weather like in Boston?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_current_weather",
          "description": "Get the current weather in a given location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA"
              },
              "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"]
              }
            },
            "required": ["location"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'

```

--------------------------------

### Start LocalAI with OpenAI API key

Source: https://localai.io/docs/features/cloud-proxy/index.print.html

Exports the OPENAI_API_KEY environment variable and starts LocalAI. Clients then hit http://localhost:8080/v1/chat/completions with "model": "gpt-4o-proxy".

```bash
export OPENAI_API_KEY=sk-...
local-ai run

```

--------------------------------

### Run phi-2 model with Docker using gist config URL

Source: https://localai.io/docs/getting-started/customize-model

Starts the phi-2 model using the LocalAI container image, passing the URL of the model configuration file as an argument. The example uses a specific gist URL from the quickstart.

```bash
docker run -p 8080:8080 localai/localai:v4.9.0 https://gist.githubusercontent.com/mudler/ad601a0488b497b69ec549150d9edd18/raw/a8a8869ef1bb7e3830bf5c0bae29a0cce991ff8d/phi-2.yaml
```

--------------------------------

### Start LocalAI with API Key

Source: https://localai.io/docs/features/cloud-proxy

Command to set the API key environment variable and start the LocalAI service.

```bash
export OPENAI_API_KEY=sk-...
local-ai run
```

--------------------------------

### Engine Configuration Example

Source: https://localai.io/docs/features/text-generation

A sample configuration for the qwen35-a3b model using the vllm-cpp backend, demonstrating KV cache sizing and scheduler settings.

```yaml
name: qwen35-a3b
backend: vllm-cpp
parameters:
  model: "Qwen/Qwen3.5-A3B"
context_size: 16384
template:
  use_tokenizer_template: true
engine_args:
  # KV cache sizing: num_blocks * block_size tokens of cache.
  block_size: 32
  num_blocks: 1024
  # Concurrency and the per-step chunked-prefill token budget.
  max_num_seqs: 32
  max_num_batched_tokens: 8192
  # Automatic prefix caching. Omit to keep the model's own default
  # (on for dense models, off for hybrid / attention-free ones).
  enable_prefix_caching: true
  # Scheduler admission order: fcfs (default), priority, or lpm
  # (cache-aware longest-prefix-match; needs prefix caching to have any effect).
  scheduling_policy: lpm
```

--------------------------------

### Start LocalAI with a model from the gallery

Source: https://localai.io/docs/getting-started/models

Starts LocalAI with a model from the gallery using the model name as the URI.

```bash
local-ai run llama-3.2-1b-instruct:q4_k_m
```

--------------------------------

### Install and Use Higgs Audio v3

Source: https://localai.io/docs/features/text-to-audio/index.html

Install the audio-cpp-higgs-audio-v3 model and perform a speech generation request with a reference audio file.

```bash
local-ai models install audio-cpp-higgs-audio-v3

curl http://localhost:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "audio-cpp-higgs-audio-v3",
    "input": "Welcome to LocalAI.",
    "voice": "/models/voices/reference.wav"
  }' \
  --output higgs.wav
```

--------------------------------

### List installed models via API and CLI

Source: https://localai.io/docs/getting-started/models/index.print.html

Use the API endpoint or the CLI command to list installed models.

```bash
# Via API
curl http://localhost:8080/v1/models

# Via CLI
local-ai models list
```

--------------------------------

### Run models via URI examples

Source: https://localai.io/docs/getting-started/models/index.print.html

Shows multiple URI examples for running models from Hugging Face, Ollama, HTTP(S) configuration files, and OCI registries. Note that `file://` URLs must point to files within the models directory.

```bash
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
local-ai run ollama://gemma:2b
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
local-ai run oci://localai/phi-2:latest
```

--------------------------------

### Get model job state with curl

Source: https://localai.io/docs/models/index.html

Sends a GET request to /models/jobs/<JOB_ID> to retrieve the state of a model installation job.

```bash
curl http://localhost:8080/models/jobs/<JOB_ID>
```

--------------------------------

### Cluster object in GET /api/resources and GET /api/models

Source: https://localai.io/docs/features/distributed-mode

Example JSON response showing the additional 'cluster' object returned by GET /api/resources and GET /api/models in distributed mode. The object is absent in single-node mode or when the registry cannot be read.

```json
{
  "cluster": {
    "enabled": true,
    "node_id": "a1b2c3",
    "node_name": "dgx-01",
    "total_memory": 85899345920,
    "is_gpu": true,
    "node_count": 4
  }
}
```

--------------------------------

### GET /api/auth/oidc/login

Source: https://localai.io/docs/features/authentication/index.html

Starts OIDC login. No authentication required.

```APIDOC
## GET /api/auth/oidc/login

### Description
Starts OIDC login. No authentication required.

### Method
GET

### Endpoint
/api/auth/oidc/login

### Parameters
#### Query Parameters
- **invite_code** (string) - Optional - Invite code to be used during the OAuth flow.

### Response
#### Success Response (302)
Redirects to the OIDC provider's authorization page.

#### Response Example
```
HTTP/1.1 302 Found
Location: https://provider.example.com/authorize?...
```
```

--------------------------------

### Install zsh completion

Source: https://localai.io/docs/reference/shell-completion/index.html

Add the source command to your ~/.zshrc, install to a completions directory, or enable compinit if not already enabled.

```bash
source <(local-ai completion zsh)
```

```bash
local-ai completion zsh > "${fpath[1]}/_local-ai"
```

```bash
autoload -Uz compinit
compinit
```

--------------------------------

### Start vllm backend with --external-grpc-backends

Source: https://localai.io/docs/advanced

Starts LocalAI with the vllm backend using a script path. Assumes LocalAI has been compiled and the command is run from the repository root.

```bash
./local-ai --external-grpc-backends "vllm:$PWD/backend/python/vllm/run.sh"
```

--------------------------------

### vllm.cpp Backend Setup

Source: https://localai.io/docs/features/text-generation

Basic configuration for the vllm.cpp backend.

```yaml
name: vllm-cpp
backend: vllm-cpp
parameters:
  model: "Qwen/Qwen3-4B"
context_size: 8192
template:
  use_tokenizer_template: true
```

--------------------------------

### Run flux.1-dev-ggml model with local-ai run

Source: https://localai.io/docs/features/image-generation/index.html

Starts LocalAI with the flux.1-dev-ggml model from the gallery. This is a quick way to get started with the stablediffusion-ggml backend.

```bash
local-ai run flux.1-dev-ggml
```

--------------------------------

### Install MiniMax-H3 FL2VA model

Source: https://localai.io/docs/features/video-generation/index.html

Installs the minimax-h3-fl2va-q4 model from the gallery. This is a large model (~40 GB) and slow to generate.

```bash
local-ai models install minimax-h3-fl2va-q4
```

--------------------------------

### Install and run rfdetr-base model

Source: https://localai.io/docs/features/object-detection/index.print.html

Installs and runs the rfdetr-base model from the LocalAI model gallery. This is the recommended setup for the Python-based RF-DETR backend.

```bash
# Install and run the rfdetr-base model
local-ai run rfdetr-base
```

--------------------------------

### Example options for memory and device placement

Source: https://localai.io/docs/features/image-generation/index.html

Shows how to mmap diffusion weights from disk while keeping the text encoder on the CPU. Uses params_backend and keep_clip_on_cpu options.

```yaml
options:
- "diffusion_model"
- "sampler:euler"
- "params_backend:diffusion=disk"
- "keep_clip_on_cpu:true"
```

--------------------------------

### Manage LocalAI Backends

Source: https://localai.io/docs/basics/troubleshooting/index.html

List installed backends or install a missing backend required for model loading.

```bash
local-ai backends list
# Install a missing backend:
local-ai backends install llama-cpp
```

--------------------------------

### Install model with custom name

Source: https://localai.io/docs/models/index.html

Using the name parameter to specify a custom name for an installed model.

```bash
LOCALAI=http://localhost:8080
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
     "url": "<MODEL_CONFIG_FILE>",
     "name": "<MODEL_NAME>"
   }'
```

--------------------------------

### Install default speaker-recognition model

Source: https://localai.io/docs/features/voice-recognition

Installs the default speaker-recognition backend and model. Run this before using the voice verification or identification endpoints.

```bash
local-ai models install speechbrain-ecapa-tdnn
```

--------------------------------

### Job queued response

Source: https://localai.io/docs/models/index.html

Example JSON response indicating the job is queued because another installation is still running. Installations are processed one at a time.

```json
{"error":null,"processed":false,"message":"queued","phase":"queued"}
```

--------------------------------

### Install a model from gallery with /models/apply

Source: https://localai.io/docs/models/index.print.html

Install a model by sending a POST request to /models/apply with the model id in the JSON body. The repository prefix is optional; if omitted, LocalAI searches all repositories and the first match wins.

```bash
LOCALAI=http://localhost:8080
curl $LOCALAI/models/apply -H "Content-Type: application/json" -d '{
     "id": "localai@bert-embeddings"
   }'  
```

--------------------------------

### Install vllm-cpp backend

Source: https://localai.io/docs/features/vllm-cpp/index.html

Installs the vllm-cpp backend via the LocalAI CLI. Images are published for CPU, CUDA 13, Vulkan, Metal, and Jetson L4T.

```bash
local-ai backends install vllm-cpp
```

--------------------------------

### GET /api/auth/github/login

Source: https://localai.io/docs/features/authentication/index.html

Starts the GitHub OAuth flow. No authentication required.

```APIDOC
## GET /api/auth/github/login

### Description
Starts the GitHub OAuth flow. No authentication required.

### Method
GET

### Endpoint
/api/auth/github/login

### Parameters
#### Query Parameters
- **invite_code** (string) - Optional - Invite code to be used during the OAuth flow.

### Response
#### Success Response (302)
Redirects to the GitHub authorization page.

#### Response Example
```
HTTP/1.1 302 Found
Location: https://github.com/login/oauth/authorize?...
```
```

--------------------------------

### Install Opus backend via API

Source: https://localai.io/docs/features/openai-realtime

Install the required Opus backend using the LocalAI backend gallery API.

```bash
curl -X POST http://localhost:8080/backends/apply \
  -H "Content-Type: application/json" \
  -d '{"id": "opus"}'
```

--------------------------------

### Install LocalAI with the go-skynet Helm chart

Source: https://localai.io/docs/basics/kubernetes/index.html

Adds the go-skynet Helm repository, updates it, displays the chart values, and installs LocalAI using a custom values file.

```text
helm repo add go-skynet https://go-skynet.github.io/helm-charts/
helm repo update
helm show values go-skynet/local-ai > values.yaml

helm install local-ai go-skynet/local-ai -f values.yaml
```

--------------------------------

### Install Stable Diffusion x4 upscaler

Source: https://localai.io/docs/features/image-generation/index.html

Command to install the Stable Diffusion x4 upscaler gallery model.

```bash
local-ai models install stable-diffusion-x4-upscaler
```

--------------------------------

### RPC servers option example

Source: https://localai.io/docs/features/image-generation/index.html

Example options block pointing to remote RPC workers for distributed inference.

```yaml
options:
- "rpc_servers:192.168.1.10:50052,192.168.1.11:50052"
```

--------------------------------

### SGLang Backend Setup

Source: https://localai.io/docs/features/text-generation

Basic configuration for the SGLang backend using a Qwen model.

```yaml
name: sglang
backend: sglang
parameters:
  model: "Qwen/Qwen3-4B"
template:
  use_tokenizer_template: true
```

--------------------------------

### GET /models/jobs/<uid>

Source: https://localai.io/docs/models

Returns the state of the batch job associated with a model installation.

```APIDOC
## GET /models/jobs/<uid>

### Description
Returns the state of the batch job associated with a model installation.

### Method
GET

### Endpoint
/models/jobs/<uid>

### Parameters
#### Path Parameters
- **uid** (string) - Required - The UUID of the job returned by `/models/apply`.

### Request Example
```
curl http://localhost:8080/models/jobs/<JOB_ID>
```

### Response
#### Success Response (200)
- **error** (string|null) - Error message if any, otherwise null.
- **processed** (boolean) - Whether the job has been processed.
- **message** (string) - Status message (e.g., "completed", "queued").
- **phase** (string) - Optional - Phase of the job (e.g., "queued").

#### Response Example
```json
{"error":null,"processed":true,"message":"completed"}
```

#### Response Example (queued)
```json
{"error":null,"processed":false,"message":"queued","phase":"queued"}
```
```

--------------------------------

### GET /models/jobs/<uid>

Source: https://localai.io/docs/models/index.html

Retrieves the state of a model installation job. The endpoint returns whether the job has been processed, any error, and a status message. It also indicates if the job is queued while another installation is running.

```APIDOC
## [GET] /models/jobs/<uid>

### Description
Returns the state of the batch job associated with a model installation. The job ID is queryable from the moment `/models/apply` returns it. A 404 or 500 from this endpoint means the ID is genuinely unknown rather than merely waiting its turn.

### Method
GET

### Endpoint
/models/jobs/<uid>

### Parameters
#### Path Parameters
- **uid** (string) - Required - The job UUID returned by `/models/apply`.

### Request Example
```
curl http://localhost:8080/models/jobs/<JOB_ID>
```

### Response
#### Success Response (200)
- **error** (string or null) - Error message if any, otherwise null.
- **processed** (boolean) - Whether the job has been processed.
- **message** (string) - Status message, e.g., "completed" or "queued".
- **phase** (string) - Optional. The current phase, e.g., "queued".

#### Response Example
```json
{"error":null,"processed":true,"message":"completed"}
```

#### Response Example (queued)
```json
{"error":null,"processed":false,"message":"queued","phase":"queued"}
```
```

--------------------------------

### Start LocalAI server with P2P enabled

Source: https://localai.io/docs/features/distribute/index.html

Run this command to start the LocalAI server with P2P enabled. The server will generate a token that can be retrieved from the WebUI or via API call.

```bash
./local-ai run --p2p
```

--------------------------------

### List all instructions

Source: https://localai.io/docs/features/api-discovery/index.print.html

Retrieves a compact list of available instruction areas using a GET request.

```bash
curl http://localhost:8080/api/instructions
```

--------------------------------

### LRU Eviction Workflow Example

Source: https://localai.io/docs/advanced/vram-management/index.html

Demonstrates how models are loaded and evicted based on the max active backends limit.

```bash
# Allow 2 active backends
LOCALAI_MAX_ACTIVE_BACKENDS=2 ./local-ai

# First request - model-a is loaded (1 active)
curl http://localhost:8080/v1/chat/completions -d '{"model": "model-a", ...}'

# Second request - model-b is loaded (2 active, at limit)
curl http://localhost:8080/v1/chat/completions -d '{"model": "model-b", ...}'

# Third request - model-a is evicted (LRU), model-c is loaded
curl http://localhost:8080/v1/chat/completions -d '{"model": "model-c", ...}'

# Request for model-b updates its "last used" time
curl http://localhost:8080/v1/chat/completions -d '{"model": "model-b", ...}'
```

--------------------------------

### Run ced-base-f16 model

Source: https://localai.io/docs/features/audio-classification/index.html

Install and run the ced-base-f16 classification model from the gallery. This is a prerequisite for the curl example.

```bash
local-ai run ced-base-f16
```

--------------------------------

### Example .env file for LocalAI

Source: https://localai.io/docs/advanced

Example .env file that sets LocalAI environment variables. Files earlier in the precedence list override later ones.

```bash
LOCALAI_THREADS=10
LOCALAI_MODELS_PATH=/mnt/storage/localai/models
LOCALAI_F16=true
```

--------------------------------

### GET /models/available

Source: https://localai.io/docs/models/index.print.html

Lists all available models from the configured repositories. Use this endpoint to discover model names and URLs for installation.

```APIDOC
## GET /models/available

### Description
Lists all available models from the configured repositories.

### Method
GET

### Endpoint
/models/available

### Parameters
None

### Request Example
No request body required.

### Response
#### Success Response (200)
- **array** - List of model objects, each containing properties such as `name` and `urls`.

#### Response Example
No explicit response example provided in the source.

```

--------------------------------

### Install rfdetr-cpp backend

Source: https://localai.io/docs/features/object-detection/index.print.html

Installs the rfdetr-cpp native backend. This backend has no Python or PyTorch dependency at inference time and loads quantized GGUF models.

```bash
local-ai backends install rfdetr-cpp
```

--------------------------------

### Run LocalAI binary

Source: https://localai.io/docs/installation/linux/index.print.html

Starts the LocalAI server using the downloaded binary.

```bash
./local-ai-*
```

--------------------------------

### runtime_settings.json example configuration

Source: https://localai.io/docs/features/runtime-settings/index.html

Example configuration file for runtime settings. Includes watchdog, eviction, threading, context size, CORS, galleries, and API keys. Note that API keys from environment variables are always included and cannot be removed via the UI.

```json
{
  "watchdog_enabled": true,
  "watchdog_idle_enabled": true,
  "watchdog_busy_enabled": false,
  "watchdog_idle_timeout": "15m",
  "watchdog_busy_timeout": "5m",
  "max_active_backends": 0,
  "force_eviction_when_busy": false,
  "lru_eviction_max_retries": 30,
  "lru_eviction_retry_interval": "1s",
  "threads": 8,
  "context_size": 2048,
  "artifact_download_concurrency": 4,
  "f16": false,
  "debug": false,
  "cors": true,
  "csrf": false,
  "cors_allow_origins": "*",
  "p2p_token": "",
  "p2p_network_id": "",
  "federated": false,
  "galleries": [
    {
      "url": "https://index.localai.io/models",
      "mirrors": ["github:mudler/LocalAI/gallery/index.yaml@master"],
      "name": "localai"
    }
  ],
  "backend_galleries": [
    {
      "url": "https://index.localai.io/backends",
      "mirrors": ["github:mudler/LocalAI/backend/index.yaml@master"],
      "name": "localai"
    }
  ],
  "autoload_galleries": true,
  "autoload_backend_galleries": true,
  "vram_persistent_cache": true,
  "api_keys": []
}
```

--------------------------------

### Example VAD response

Source: https://localai.io/docs/features/voice-activity-detection/index.html

Shows the JSON response format with detected speech segments, each having start and end times in seconds.

```json
{
  "segments": [
    {
      "start": 0.5,
      "end": 2.3
    },
    {
      "start": 3.1,
      "end": 5.8
    }
  ]
}
```

--------------------------------

### GET /api/instructions

Source: https://localai.io/docs/features/api-discovery/index.html

Lists all available instruction areas. Each instruction maps to related API endpoints and provides a focused guide.

```APIDOC
## GET /api/instructions

### Description
Returns a compact list of instruction areas, each mapping to a set of related API endpoints. Use the returned URLs to fetch detailed guides.

### Method
GET

### Endpoint
/api/instructions

### Parameters
#### Query Parameters
- **format** (string) - Optional - If set to `json`, returns raw OpenAPI fragments instead of markdown guides. (Note: This parameter is mentioned in the response hint, not in the request example.)

### Request Example
```
curl http://localhost:8080/api/instructions
```

### Response
#### Success Response (200)
- **instructions** (array) - List of instruction objects.
  - **name** (string) - Instruction name.
  - **description** (string) - Short description of the instruction area.
  - **tags** (array of strings) - Swagger tags associated with the instruction.
  - **url** (string) - URL to fetch the detailed guide.
- **hint** (string) - Usage hint.

#### Response Example
```
{
  "instructions": [
    {
      "name": "chat-inference",
      "description": "OpenAI-compatible chat completions, text completions, and embeddings",
      "tags": ["inference", "embeddings"],
      "url": "/api/instructions/chat-inference"
    },
    {
      "name": "config-management",
      "description": "Discover, read, and modify model configuration fields with VRAM estimation",
      "tags": ["config"],
      "url": "/api/instructions/config-management"
    }
  ],
  "hint": "Fetch GET {url} for a markdown API guide. Add ?format=json for a raw OpenAPI fragment."
}
```
```

--------------------------------

### Start distributed mode with Docker Compose

Source: https://localai.io/docs/features/distributed-mode

Starts PostgreSQL, NATS, a LocalAI frontend, and one worker node. The SmartRouter automatically installs backends and loads models on workers. Use for quick local testing; for production, deploy PostgreSQL and NATS as managed services.

```bash
docker compose -f docker-compose.distributed.yaml up
```

--------------------------------

### Well-Known Discovery Endpoint response

Source: https://localai.io/docs/features/api-discovery/index.print.html

An abbreviated example of the JSON response returned by GET /.well-known/localai.json, detailing instance version, endpoints, and runtime capabilities.

```json
{
  "version": "v2.28.0",
  "endpoints": {
    "chat_completions": "/v1/chat/completions",
    "models": "/v1/models",
    "models_capabilities": "/v1/models/capabilities",
    "config_metadata": "/api/models/config-metadata",
    "instructions": "/api/instructions",
    "swagger": "/swagger/index.html"
  },
  "endpoint_groups": {
    "openai_compatible": { "chat_completions": "/v1/chat/completions", "..." : "..." },
    "config_management": { "config_metadata": "/api/models/config-metadata", "..." : "..." },
    "model_management": { "..." : "..." },
    "monitoring": { "..." : "..." }
  },
  "capabilities": {
    "config_metadata": true,
    "config_patch": true,
    "vram_estimate": true,
    "mcp": true,
    "agents": false,
    "p2p": false
  }
}
```

--------------------------------

### GET /api/operations/history - JSON response example

Source: https://localai.io/docs/features/activity

Example response body from the history endpoint. Each operation includes fields like 'id', 'name', 'jobID', 'taskType', 'outcome', 'startedAt', 'finishedAt'. Additional fields 'nodeID' and 'error' appear conditionally; 'outcome' can be 'completed', 'failed', or 'cancelled'.

```json
{
  "operations": [
    {
      "id": "localai@qwen3-4b",
      "name": "qwen3-4b",
      "jobID": "6f1b0c2e-...",
      "isBackend": false,
      "taskType": "installation",
      "outcome": "completed",
      "startedAt": "2026-07-27T10:02:11.482913574Z",
      "finishedAt": "2026-07-27T10:03:23.117402881Z"
    }
  ]
}
```

--------------------------------

### Run LocalAI CPU image with Docker or Podman

Source: https://localai.io/docs/installation/containers/index.html

Starts LocalAI with the CPU image. The API will be available at http://localhost:8080. Models must be installed separately.

```bash
docker run -p 8080:8080 --name local-ai -ti localai/localai:latest
# Or with Podman:
podman run -p 8080:8080 --name local-ai -ti docker.io/localai/localai:latest
```

--------------------------------

### GPT Vision with curl

Source: https://localai.io/docs/basics/try/index.html

Sends a vision request with an image URL to the chat completions endpoint using the gpt-4-vision-preview model. The example assumes the model is installed.

```bash
curl http://localhost:8080/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{ 
        "model": "gpt-4-vision-preview", 
        "messages": [
          {
            "role": "user", "content": [
              {"type":"text", "text": "What is in the image?"},
              {
                "type": "image_url", 
                "image_url": {
                  "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg" 
                }
              }
            ], 
          "temperature": 0.9
          }
        ]
      }' 

```

--------------------------------

### Request TTS with response_format via cURL

Source: https://localai.io/docs/features/text-to-audio/index.html

Example cURL request to the /tts endpoint specifying an mp3 response format. Requires ffmpeg to be installed for format conversion.

```bash
curl http://localhost:8080/tts -H "Content-Type: application/json" -d '{
  "input": "Hello world",
  "model": "tts",
  "response_format": "mp3"
}'
```

--------------------------------

### GET /api/operations/history - curl example

Source: https://localai.io/docs/features/activity

Fetches the operations history list, newest first. The response wraps the list in an 'operations' key instead of a bare array. Requires authentication with an admin key.

```bash
# What has finished
curl http://localhost:8080/api/operations/history \
  -H "Authorization: Bearer <admin-key>"
```

--------------------------------

### JSON response from GET /api/traces/summary

Source: https://localai.io/docs/operations/overview/index.html

Example response showing total requests, errors, p95 latency, window hours, and buckets. Only 5xx and transport errors count as failures; p95_ms is a nearest-rank percentile.

```json
{
  "total": 18402,
  "errors": 37,
  "p95_ms": 842,
  "window_hours": 24,
  "buckets": [{ "start": "2026-08-02T09:00:00Z", "count": 1520, "errors": 3 }]
}
```

--------------------------------

### Set VRAM warm-up environment variables for local-ai run

Source: https://localai.io/docs/advanced/vram-management/index.html

Shows two example invocations of `local-ai run` with environment variables to control VRAM warm-up behavior. The first disables warm-up entirely for air-gapped or request-averse setups. The second reduces concurrency to one for slow or metered links.

```bash
# Air-gapped, or you would rather not make the requests at all
LOCALAI_VRAM_WARM_LIMIT=0 local-ai run

# A slow or metered link: warm the same entries, more gently
LOCALAI_VRAM_WARM_CONCURRENCY=1 local-ai run
```

--------------------------------

### Build backend manually with make

Source: https://localai.io/docs/basics/build/index.html

Build a backend manually from the LocalAI repository. The example clones the repository and runs make in the backend's directory, which packages the backend.

```bash
git clone https://github.com/go-skynet/LocalAI.git

make -C LocalAI/backend/python/vllm
```

--------------------------------

### Configure gRPC readiness for large models

Source: https://localai.io/docs/advanced/index.print.html

Example configuration for a model requiring up to 10 minutes to become gRPC-ready, such as large NVFP4 models with cold shard loading and CUDA-graph capture.

```yaml
grpc:
  attempts: 140
  attempts_sleep_time: 5
```

--------------------------------

### GET /api/agents/skills/:name

Source: https://localai.io/docs/features/agents/index.print.html

Get a skill.

```APIDOC
## GET /api/agents/skills/:name

### Description
Get a skill.

### Method
GET

### Endpoint
/api/agents/skills/:name

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the skill.

```

--------------------------------

### Start LocalAI with Docker Compose

Source: https://localai.io/docs/integrations/index.html

Starts LocalAI with Docker Compose in detached mode. The first start downloads the available models, which can take a while.

```bash
docker compose up -d
```

--------------------------------

### Configure Supertonic Backend

Source: https://localai.io/docs/features/text-to-audio/index.html

Example configuration for the supertonic backend using the audio-cpp engine.

```yaml
name: supertonic
backend: audio-cpp
parameters:
  model: supertonic-3-orig.gguf
known_usecases:
  - FLAG_TTS
options:
  - backend:cuda
  - device:0
```

--------------------------------

### Pre-install backends with LOCALAI_EXTERNAL_BACKENDS

Source: https://localai.io/docs/backends/index.html

Pre-installs backends when starting LocalAI by setting the LOCALAI_EXTERNAL_BACKENDS environment variable to a comma-separated list of backend names, then runs LocalAI.

```bash
export LOCALAI_EXTERNAL_BACKENDS="llm-backend,diffusion-backend"
local-ai run
```

--------------------------------

### GET /api/agents/:name/config

Source: https://localai.io/docs/features/agents/index.print.html

Get agent configuration.

```APIDOC
## GET /api/agents/:name/config

### Description
Get agent configuration.

### Method
GET

### Endpoint
/api/agents/:name/config

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the agent.

```

--------------------------------

### GET /api/agents/:name

Source: https://localai.io/docs/features/agents/index.print.html

Get agent info.

```APIDOC
## GET /api/agents/:name

### Description
Get agent info.

### Method
GET

### Endpoint
/api/agents/:name

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the agent.

```

--------------------------------

### Prepare vllm backend environment

Source: https://localai.io/docs/advanced

Creates the environment for the vllm backend before starting LocalAI with it.

```bash
make -C backend/python/vllm
```

--------------------------------

### Manage plugins, skills, and MCP servers with local-ai chat

Source: https://localai.io/docs/features/terminal-agent/index.html

Use these commands to install a plugin, list skills, and add an MCP server. Everything after the first positional argument is forwarded verbatim, so LocalAI's own flags must come first. In non-interactive contexts, always pass --yes to plugin install to avoid leaving the plugin disabled.

```bash
local-ai chat plugin install https://github.com/user/plugin
local-ai chat skill list
local-ai chat mcp add my-server -- npx -y @modelcontextprotocol/server-filesystem /tmp
```

--------------------------------

### GET /api/agents/:name/status

Source: https://localai.io/docs/features/agents/index.print.html

Get agent status and observables.

```APIDOC
## GET /api/agents/:name/status

### Description
Get agent status and observables.

### Method
GET

### Endpoint
/api/agents/:name/status

### Parameters
#### Path Parameters
- **name** (string) - Required - The name of the agent.

```

--------------------------------

### vLLM attention_backend engine_args example

Source: https://localai.io/docs/features/text-generation/index.html

Pick a non-default attention backend, e.g. on hardware where the default cutlass kernels aren't supported.

```yaml
engine_args:
  attention_backend: TRITON_ATTN

```