Soup

https://trysoup.dev/docs
Admin
Soup is a lightweight CLI and training framework designed for easy, resource-efficient LLM...
Tokens:92,961
Snippets:1,302
Trust Score:4.9
Update:2 months ago
Show doc for...
Tokens:
Raw
### Launch the Soup Web UI

Source: https://trysoup.dev/docs/web-ui

Install the UI dependencies and start the dashboard server.

```bash
pip install "soup-cli[ui]"
soup ui
# Opens http://127.0.0.1:7860 in your browser
# Prints auth token to console
```

--------------------------------

### Installation and Training Commands

Source: https://trysoup.dev/docs/asr-fine-tuning

Commands to install the necessary dependencies and execute the training process.

```bash
pip install "soup-cli[train,audio]"
soup recipes use whisper-tiny-asr   # or whisper-base-asr / whisper-large-v3-asr
soup train
```

--------------------------------

### Install and Start Soup MCP Server

Source: https://trysoup.dev/docs/mcp-server

Install the required dependencies and start the server using stdio transport. The --allow-mutating flag is required to enable plan-only tools.

```bash
pip install "soup-cli[mcp]"    # official mcp SDK (mcp>=1.2.0,<2), lazy-imported
soup mcp serve                 # stdio; read-only tools
soup mcp serve --allow-mutating   # also list the 2 plan-only mutating tools
```

--------------------------------

### Install soup-cli

Source: https://trysoup.dev/docs/fine-tune-gemma-3-qlora

Install the necessary training stack and optimization dependencies.

```bash
pip install "soup-cli[train,fast]"
```

--------------------------------

### Deploy with Transformers Backend

Source: https://trysoup.dev/docs/serving

Install the Transformers serving package and start the server on a specified port.

```bash
pip install "soup-cli[serve]"
soup serve --model ./output --port 8000
```

--------------------------------

### Install Unsloth Backend

Source: https://trysoup.dev/docs/backends

Install the fast training dependencies for reduced VRAM usage.

```bash
pip install "soup-cli[fast]"
```

--------------------------------

### Install and use experiment trackers

Source: https://trysoup.dev/docs/v0538-10-remote-hubs

Install the trackers bundle and specify the tracker during training.

```bash
pip install "soup-cli[trackers]"
soup train --tracker mlflow      # or swanlab | trackio
```

--------------------------------

### Initialize Audio Fine-Tuning

Source: https://trysoup.dev/docs/multimodal

Install the audio dependencies and initialize the project template.

```bash
pip install "soup-cli[audio]"
soup init --template audio
```

--------------------------------

### Initialize Vision Fine-Tuning

Source: https://trysoup.dev/docs/multimodal

Install the vision dependencies and initialize the project template.

```bash
pip install "soup-cli[vision]"
soup init --template vision
```

--------------------------------

### Install Performance Packages

Source: https://trysoup.dev/docs/backends

Install optional packages to enable Liger, FlashAttention, and Ring FlashAttention.

```bash
pip install "soup-cli[liger]"       # Liger Kernel
pip install flash-attn --no-build-isolation  # FlashAttention
pip install "soup-cli[ring-attn]"   # Ring FlashAttention
```

--------------------------------

### Run Quick Demo

Source: https://trysoup.dev/docs/getting-started

Execute a full demo workflow to test installation and training capabilities.

```bash
soup quickstart          # Creates sample data + config + trains TinyLlama
soup quickstart --dry-run # Just create files without training
```

--------------------------------

### Install MLX Backend

Source: https://trysoup.dev/docs/backends

Install the necessary dependencies for Apple Silicon support.

```bash
pip install "soup-cli[mlx]"
```

--------------------------------

### Install Soup CLI

Source: https://trysoup.dev/docs/getting-started

Install the core CLI or specific dependency stacks for training, serving, and other features.

```bash
# Light core: CLI + config + data tools, no PyTorch
pip install soup-cli

# Add the training stack (torch, transformers, peft, trl, datasets, …)
pip install "soup-cli[train]"

# Everything (train + serve + ui + data + mcp) in one shot
pip install "soup-cli[all]"

# Or from GitHub (latest dev)
pip install git+https://github.com/MakazhanAlpamys/Soup.git
```

--------------------------------

### Enable Quantization-Aware Training (QAT)

Source: https://trysoup.dev/docs/backends

Install the QAT extension and configure the training parameters.

```bash
pip install "soup-cli[qat]"
```

```yaml
training:
  quantization: 4bit
  quantization_aware: true    # Enable QAT
```

--------------------------------

### Install soup-cli with extras

Source: https://trysoup.dev/docs/lean-install-live-wiring

Commands to install the base CLI or include additional dependencies for training and other features.

```bash
pip install soup-cli            # light: CLI + config + data tools, no PyTorch
pip install "soup-cli[train]"   # add the training stack to fine-tune
pip install "soup-cli[all]"     # train + serve + ui + data + mcp
```

--------------------------------

### Start inference server

Source: https://trysoup.dev/docs/cli-reference

Starts an OpenAI-compatible inference server with support for multiple backends and speculative decoding.

```bash
soup serve [--model <path>] [--backend transformers|vllm|sglang] [--port <int>] [--host <host>] [--tensor-parallel <n>] [--gpu-memory <float>] [--speculative-decoding <draft-model>] [--num-speculative-tokens <n>] [--adapters <name>=<path>]
```

--------------------------------

### Serve model with vLLM

Source: https://trysoup.dev/docs/fine-tune-qwen-3

Install the serving dependencies and launch the model with adapters using the vLLM backend.

```bash
pip install "soup-cli[serve-fast]"
soup serve --model Qwen/Qwen3-8B --adapters qwen3=./runs/qwen3/latest --backend vllm --port 8000
```

--------------------------------

### Train using a recipe

Source: https://trysoup.dev/docs/tool-calling

Commands to initialize and start training with a pre-defined tool-calling recipe.

```bash
soup recipes use qwen3-8b-tools
soup train
```

--------------------------------

### Install Cut Cross-Entropy Support

Source: https://trysoup.dev/docs/faq

Installation command for the Cut Cross-Entropy feature to optimize large-vocab model training.

```bash
pip install "soup-cli[cce]"
```

--------------------------------

### Install and run ASR tasks

Source: https://trysoup.dev/docs/cli-reference

Install the necessary dependencies and execute ASR inference using the Whisper model.

```bash
pip install "soup-cli[train,audio]"
soup recipes use whisper-tiny-asr          # task: asr · format: asr · fits a 4 GB GPU
soup infer --task asr --model ./out --input eval.jsonl --output preds.jsonl     [--audio-dir ./data/audio] [--asr-language en] [--asr-task transcribe]
```

--------------------------------

### Deploy with SGLang Backend

Source: https://trysoup.dev/docs/serving

Install the SGLang serving package and configure multi-GPU support.

```bash
pip install "soup-cli[sglang]"
soup serve --model ./output --backend sglang

# Multi-GPU
soup serve --model ./output --backend sglang --tensor-parallel 2
```

--------------------------------

### soup serve

Source: https://trysoup.dev/docs/cli-reference

Starts an OpenAI-compatible inference server for model serving.

```APIDOC
## soup serve

### Description
Start an OpenAI-compatible inference server.

### Usage
`soup serve [--model <path>] [--backend transformers|vllm|sglang] [--port <int>] [--host <host>] [--tensor-parallel <n>] [--gpu-memory <float>] [--speculative-decoding <draft-model>] [--num-speculative-tokens <n>] [--adapters <name>=<path>]`
```

--------------------------------

### Configure Quantization for Training

Source: https://trysoup.dev/docs/quant-menu

Examples of setting quantization parameters for GPTQ checkpoints and FSDP with QLoRA.

```yaml
# Train LoRA on top of a pre-quantized GPTQ checkpoint:
base: TheBloke/Llama-2-7B-Chat-GPTQ
training:
  quantization: gptq        # or: awq, hqq:4bit, aqlm, eetq, mxfp4, fp8

# FSDP + QLoRA — set quant_storage:
training:
  quantization: 4bit
  bnb_4bit_quant_storage: bfloat16
```

--------------------------------

### Advise output format

Source: https://trysoup.dev/docs/advise

Example of the structured output returned by the advise command.

```text
SFT (recommended)
  why:
    - 4,213 rows (above _MIN_ROWS_FOR_TRAINING=50)
    - no preference pairs detected
    - no reasoning traces — GRPO ruled out
    - tone-shift goal — SFT is the right baseline
  next: soup autopilot --data data.jsonl --task sft --goal "polite customer support chat"
```

--------------------------------

### Install Optional Extras

Source: https://trysoup.dev/docs/getting-started

Install specific functional modules for tasks like fast training, inference backends, data processing, and hardware-specific optimizations.

```bash
pip install "soup-cli[train]"      # Training stack: torch, transformers, peft, trl, …
pip install "soup-cli[fast]"       # Unsloth 2-5x training speedup
pip install "soup-cli[mlx]"        # Apple Silicon backend (M1–M4)
pip install "soup-cli[serve]"      # FastAPI inference server
pip install "soup-cli[serve-fast]" # vLLM backend
pip install "soup-cli[sglang]"     # SGLang backend
pip install "soup-cli[eval]"       # lm-evaluation-harness
pip install "soup-cli[data]"       # MinHash deduplication
pip install "soup-cli[data-pro]"   # langdetect + Presidio PII
pip install "soup-cli[vision]"     # Pillow for vision fine-tuning
pip install "soup-cli[audio]"      # librosa + soundfile
pip install "soup-cli[qat]"        # Quantization-aware training
pip install "soup-cli[liger]"      # Liger Kernel fused ops
pip install "soup-cli[onnx]"       # ONNX export
pip install "soup-cli[tensorrt]"   # TensorRT-LLM export
pip install "soup-cli[awq]"        # AWQ / GPTQ quantized export
pip install "soup-cli[ui]"         # Web UI dashboard
pip install "soup-cli[tui]"        # Full-screen Textual dashboard
pip install "soup-cli[trackers]"   # MLflow / SwanLab / Trackio logging
pip install "soup-cli[deepspeed]"  # ZeRO distributed training
pip install "soup-cli[ring-attn]"  # Ring FlashAttention
pip install "soup-cli[remote]"     # Remote datasets (s3 / gs / az / oci)
pip install "soup-cli[sign]"       # ed25519 adapter / attestation signing
pip install "soup-cli[compile]"    # DSPy / GEPA / TextGrad prompt compiler
pip install "soup-cli[carbon]"     # codecarbon energy / CO2 tracking
pip install "soup-cli[modal]"      # Serverless cloud GPU (soup train --cloud modal)
pip install "soup-cli[mcp]"        # MCP server (soup mcp serve) for coding agents
```

--------------------------------

### Axolotl and Soup configuration examples

Source: https://trysoup.dev/docs/migrate-from-axolotl

Comparison of a standard Axolotl configuration file and its corresponding output after running the migration tool.

```yaml
base_model: mistralai/Mistral-7B-v0.3
load_in_4bit: true
adapter: qlora
datasets:
  - path: tatsu-lab/alpaca
    type: alpaca
sequence_len: 2048
micro_batch_size: 2
gradient_accumulation_steps: 8
num_epochs: 3
learning_rate: 0.0002
lora_r: 16
lora_alpha: 32
flash_attention: true
```

```yaml
base: mistralai/Mistral-7B-v0.3
task: sft
backend: unsloth            # root-level, NOT under training

data:
  train: tatsu-lab/alpaca
  format: alpaca
  max_length: 2048          # sequence length lives under data

training:
  quantization: 4bit        # the key is quantization, not quant
  epochs: 3
  lr: 2.0e-4                # the key is lr, not learning_rate
  batch_size: 2
  gradient_accumulation_steps: 8
  lora:
    r: 16                   # LoRA turns on when r > 0; there is no enabled flag
    alpha: 32
```

--------------------------------

### Initialize Iterative DPO

Source: https://trysoup.dev/docs/loop-hardening

Start the iterative DPO driver using a base model from the registry.

```bash
soup iterative-dpo --base-model registry://policy-v3 \
```

--------------------------------

### Configure OpenTelemetry Tracing

Source: https://trysoup.dev/docs/speculative-decoding

Install the required SDK and enable OTLP tracing for per-request spans.

```bash
pip install opentelemetry-sdk opentelemetry-exporter-otlp

soup serve --model ./output \
  --trace --trace-endpoint http://localhost:4317
```

--------------------------------

### Start model training

Source: https://trysoup.dev/docs/cli-reference

Initiates a training run with support for various distributed training backends and HuggingFace integration.

```bash
soup train [--config <path>] [--resume auto|<checkpoint>] [--wandb] [--tensorboard] \
           [--deepspeed zero2|zero3|zero2_offload|zero3_offload|zero++|zero_pp] [--fsdp full_shard|shard_grad|full_offload] \
           [--gpus auto|<N>] [--gate <suite>] \
           [--push-as <user/repo>] [--hf-resume] [--yes]
```

--------------------------------

### Install shell completions

Source: https://trysoup.dev/docs/preflight-tooling

Generates shell completion scripts for bash, zsh, and fish.

```bash
soup completions bash  | sudo tee /etc/bash_completion.d/soup
soup completions zsh   > ~/.zsh/completions/_soup
soup completions fish  > ~/.config/fish/completions/soup.fish
```

--------------------------------

### Full Training Configuration Example

Source: https://trysoup.dev/docs/streaming-scaling

A comprehensive YAML configuration for training a Llama-3.1-8B-Instruct model using layer streaming and 4-bit quantization.

```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
backend: transformers

data:
  train: ./data.jsonl
  format: alpaca
  max_length: 512

training:
  epochs: 3
  lr: 2e-5
  batch_size: 4                    # raise this first; the pre-flight sizes it
  gradient_accumulation_steps: 2   # then accumulate for the rest
  quantization: 4bit               # NF4: ~4x smaller store
  gradient_checkpointing: true
  stream_layers: true
  stream_source: auto              # RAM, falling back to NVMe
  stream_buffers: 2
  lora:
    r: 64
    alpha: 16

output: ./output
```

--------------------------------

### ASR Configuration File

Source: https://trysoup.dev/docs/asr-fine-tuning

Example YAML configuration for fine-tuning a Whisper model.

```yaml
base: openai/whisper-tiny
task: asr
data:
  train: ./data/train.jsonl
  format: asr                 # rows: {"audio": "clip.wav", "text": "hello world"}
  audio_dir: ./data/audio     # audio paths resolve here (containment-checked)
training:
  epochs: 3
  lr: 1e-4
  batch_size: 2
  asr_language: en            # optional; pins + persists the decoder prefix
  asr_task: transcribe        # transcribe | translate
  asr_lora: true              # LoRA on q/v projections; default is full fine-tune
  lora: { r: 16, alpha: 32, target_modules: [q_proj, v_proj] }
output: ./out
```

--------------------------------

### Deploy with vLLM Backend

Source: https://trysoup.dev/docs/serving

Install the vLLM serving package and configure high-throughput options like tensor parallelism and GPU memory limits.

```bash
pip install "soup-cli[serve-fast]"
soup serve --model ./output --backend vllm

# Multi-GPU with tensor parallelism
soup serve --model ./output --backend vllm --tensor-parallel 2

# Control GPU memory usage
soup serve --model ./output --backend vllm --gpu-memory 0.8
```

--------------------------------

### Launch Mobile-Accessible Web UI

Source: https://trysoup.dev/docs/v0539-live-ux

Commands to start the Web UI with public access and optional authentication tokens.

```bash
soup ui --public                          # bind 0.0.0.0 + print phone QR
soup ui --public --auth-token MY_TOKEN    # override the auto-generated bearer
```

--------------------------------

### Enable Ring FlashAttention

Source: https://trysoup.dev/docs/multi-gpu-deepspeed

Installation and configuration steps to support extremely long sequence lengths.

```bash
pip install "soup-cli[ring-attn]"
```

```yaml
data:
  max_length: 131072

training:
  use_ring_attention: true
```

--------------------------------

### Autopilot Output Example

Source: https://trysoup.dev/docs/autopilot

Displays the generated configuration decisions and justifications provided by the autopilot process.

```text
╭─ Autopilot Decisions ─────────────────────────────────╮
│ ✓ Quantization: 4bit                                  │
│   reason: 8B model needs ~5GB in 4bit, leaves 19GB    │
│                                                        │
│ ✓ PEFT: LoRA r=16, alpha=32                           │
│   reason: 15k samples — r=16 balances capacity /      │
│           overfitting risk                            │
│                                                        │
│ ✓ Batch size: 4 × grad_accum 8 = effective 32         │
│ ✓ Learning rate: 2e-4                                 │
│ ✓ Epochs: 2                                           │
│ ✓ Max length: 2048 (p95=1820 + 10% margin)            │
│ ✓ Flash Attention v2 (Ampere GPU)                     │
│ ✓ Liger Kernel (modern Llama arch)                    │
│ ✓ Forgetting detection (mini_mmlu)                    │
│ ✓ Checkpoint intelligence (judge metric)              │
│                                                        │
│ Estimated time: 1h 42min                              │
│ Estimated VRAM: 18.2GB / 24GB ✓                       │
╰────────────────────────────────────────────────────────╯
```

--------------------------------

### Manage plugins via CLI

Source: https://trysoup.dev/docs/plugins

Commands for listing, installing, enabling, and disabling plugins in the Soup environment.

```bash
soup plugins list
soup plugins install ./my_plugin
soup plugins enable my-trainer
soup plugins disable my-trainer
```

--------------------------------

### Augmenting Datasets via CLI

Source: https://trysoup.dev/docs/data-augment

Basic usage examples for rephrasing, translating, and applying styles to JSONL datasets.

```bash
# Rephrase each example 3x
soup data augment \
  --input data.jsonl \
  --output augmented.jsonl \
  --strategy rephrase \
  --count 3

# Translate to other languages
soup data augment --input data.jsonl --strategy translate --lang ru,zh,es

# Rewrite in multiple styles
soup data augment --input data.jsonl --strategy style --styles formal,casual,technical
```

--------------------------------

### Launch the TUI dashboard

Source: https://trysoup.dev/docs/observability

Install the TUI component and launch the dashboard with custom refresh rates and run limits.

```bash
pip install "soup-cli[tui]"
soup tui --refresh 1.0 --limit 50
```

--------------------------------

### Define full project configuration structure

Source: https://trysoup.dev/docs/configuration

A comprehensive example of the YAML configuration file structure, including base model, data paths, and training hyperparameters.

```yaml
base: meta-llama/Llama-3.1-8B-Instruct   # HuggingFace model ID (required)
task: sft                                   # Training task
# backend: unsloth                          # 2-5x faster (pip install "soup-cli[fast]")
# modality: text                            # text, vision, or audio

data:
  train: ./data/train.jsonl                 # Path to training data
  format: alpaca                            # Data format (auto-detected if omitted)
  val_split: 0.1                            # Validation split ratio
  max_length: 2048                          # Max sequence length (64-1048576)
  # image_dir: ./data/images               # For vision modality
  # audio_dir: ./data/audio                # For audio modality

training:
  epochs: 3
  lr: 2e-5
  batch_size: auto                          # auto or integer
  quantization: 4bit                        # none, 4bit, 8bit
  # quantization_aware: false              # Enable QAT
  # optimizer: adamw_8bit
  # gradient_checkpointing: true
  # stream_layers: false                   # Train or align a model bigger than VRAM (v0.72, BETA)
  # stream_source: auto                    # auto | ram | disk
  # stream_buffers: 2                      # 2-8
  lora:
    r: 64
    alpha: 16
    dropout: 0.05
    # target_modules: auto                 # Auto-detected per model
    # use_dora: false                      # Weight-decomposed LoRA

output: ./output
```

--------------------------------

### VRAM Pre-flight Forecast Output

Source: https://trysoup.dev/docs/streaming-scaling

Example of the VRAM usage forecast printed by the system before starting a training run.

```text
  peak VRAM    ~0.48 GB at batch 2 x seq 256 (logits 0.35 GB)
  free VRAM    3.46 GB
  forecast     5685-8361 tok/s — a compute-bound bound, not a promise
               (from a GEMM ceiling measured on this card now @ 862 MHz)
```

--------------------------------

### Initialize Project Configuration

Source: https://trysoup.dev/docs/getting-started

Create a new project configuration using the interactive wizard or a predefined template.

```bash
# Interactive wizard
soup init

# Or use a template
soup init --template chat
```

--------------------------------

### Configure MCP Client

Source: https://trysoup.dev/docs/mcp-server

Add the Soup server to your MCP client configuration file. Use the second example to enable mutating tools.

```json
{
  "mcpServers": {
    "soup": { "command": "soup", "args": ["mcp", "serve"] }
  }
}
```

```json
{
  "mcpServers": {
    "soup": { "command": "soup", "args": ["mcp", "serve", "--allow-mutating"] }
  }
}
```

--------------------------------

### Use pre-defined training recipes

Source: https://trysoup.dev/docs/multi-gpu

Load a pre-configured recipe and launch the training process.

```bash
soup recipes use llama3-70b-fsdp2
soup train --gpus 8
```

--------------------------------

### Training Execution Commands

Source: https://trysoup.dev/docs/training

Command-line interface usage for starting, resuming, and configuring training runs with various logging and hardware acceleration options.

```bash
# Start training
soup train --config soup.yaml

# Resume from checkpoint
soup train --config soup.yaml --resume auto
soup train --config soup.yaml --resume ./output/checkpoint-500

# With W&B logging
soup train --config soup.yaml --wandb

# With TensorBoard
soup train --config soup.yaml --tensorboard

# With DeepSpeed (multi-GPU)
soup train --config soup.yaml --deepspeed zero2

# With FSDP2
soup train --config soup.yaml --fsdp full_shard

# Skip confirmation
soup train --config soup.yaml --yes

# Continual learning: rehearse an old dataset so the new task
# does not erase it (v0.71.36, sft/pretrain only)
soup train --config new_task.yaml --replay old_task.jsonl --replay-ratio 0.1
```

--------------------------------

### Full Layer Streaming Configuration

Source: https://trysoup.dev/docs/layer-streaming

A complete YAML configuration example for enabling layer streaming with 4-bit quantization and gradient checkpointing.

```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
backend: transformers

data:
  train: ./data.jsonl
  format: alpaca
  max_length: 512
  val_split: 0.1

training:
  epochs: 3
  lr: 2e-5
  batch_size: 4                    # v0.72.3: bigger batches amortise the weight read
  gradient_accumulation_steps: 2   # v0.72.3: values above 1 are allowed
  quantization: 4bit               # NF4; 'none' streams bf16 instead
  gradient_checkpointing: true     # handled per-layer by the streamer
  stream_layers: true
  stream_source: auto              # RAM when it fits, NVMe disk when it does not
  stream_buffers: 2
  lora:
    r: 64
    alpha: 16

output: ./output
```

--------------------------------

### Install additional CLI extras

Source: https://trysoup.dev/docs/v0538-10-remote-hubs

Install optional dependencies for Bayesian optimization or data processing tasks.

```bash
pip install "soup-cli[mix]"
pip install "soup-cli[data-pro]"
```

--------------------------------

### Use a Recipe

Source: https://trysoup.dev/docs/recipes

Initializes a recipe into a local configuration file, optionally specifying a custom output path.

```bash
# Copy recipe to soup.yaml
soup recipes use llama3-sft

# Custom output path
soup recipes use qwen3-dpo --output my-config.yaml
```

--------------------------------

### Initialize compliance templates

Source: https://trysoup.dev/docs/compliance-pack

Use these commands to bootstrap a project with specific regulatory compliance configurations.

```bash
soup init --template hipaa      # Protected Health Information
soup init --template soc2       # SOC 2 Trust Services Criteria
soup init --template eu-ai-act  # EU AI Act Annex XI/XII
soup init --template sr-11-7    # SR 11-7 Model Risk Management
```

--------------------------------

### Install Soup CLI with DeepSpeed support

Source: https://trysoup.dev/docs/multi-gpu-deepspeed

Install the necessary dependencies to enable DeepSpeed functionality within the Soup CLI environment.

```bash
pip install "soup-cli[train,deepspeed]"
```

--------------------------------

### GET /health

Source: https://trysoup.dev/docs/serving

Performs a health check on the inference server.

```APIDOC
## GET /health

### Description
Checks the health status of the inference server.

### Method
GET

### Endpoint
/health
```

--------------------------------

### Configure Web UI startup options

Source: https://trysoup.dev/docs/web-ui

Run the UI on a specific port without automatically opening the browser.

```bash
# Custom port, don't auto-open browser
soup ui --port 8080 --no-browser
```

--------------------------------

### GET /v1/models

Source: https://trysoup.dev/docs/serving

Lists the models currently available on the server.

```APIDOC
## GET /v1/models

### Description
Retrieves a list of models available for inference.

### Method
GET

### Endpoint
/v1/models
```

--------------------------------

### soup chat

Source: https://trysoup.dev/docs/cli-reference

Starts an interactive chat session with a trained model.

```APIDOC
## soup chat

### Description
Interactive chat with a trained model.

### Usage
`soup chat [--model <path>] [--base <model>] [--temperature <float>] [--max-tokens <int>]`
```

--------------------------------

### Initialize Project Templates

Source: https://trysoup.dev/docs/configuration

Commands to initialize a new project using built-in templates for various fine-tuning and training tasks.

```bash
soup init --template chat          # Conversational fine-tune
soup init --template code          # Code generation
soup init --template medical       # Domain expert
soup init --template reasoning     # GRPO reasoning (DeepSeek-R1 style)
soup init --template vision        # Vision/multimodal fine-tune
soup init --template audio         # Audio/speech fine-tune
soup init --template kto           # KTO unpaired preference
soup init --template orpo          # ORPO (no reference model)
soup init --template simpo         # SimPO length-normalized preference
soup init --template ipo           # IPO regularized preference
soup init --template bco           # BCO binary classifier preference (v0.40)
soup init --template rlhf          # Full RLHF pipeline (SFT -> RM -> PPO)
soup init --template pretrain      # Continued pre-training on raw text
soup init --template moe           # MoE fine-tuning (ScatterMoE LoRA)
soup init --template longcontext   # 128k+ context fine-tuning
soup init --template embedding     # Sentence embedding fine-tuning
soup init --template tool-calling  # Function / tool-calling fine-tune (v0.25)
```

```bash
soup init --template hipaa         # Protected Health Information
soup init --template soc2          # SOC 2 Trust Services Criteria
soup init --template eu-ai-act     # EU AI Act Annex XI/XII
soup init --template sr-11-7       # SR 11-7 Model Risk Management
```

--------------------------------

### soup init

Source: https://trysoup.dev/docs/cli-reference

Initializes a configuration file using an interactive wizard or a specified template.

```APIDOC
## soup init

### Description
Create a config file with interactive wizard or template.

### Usage
`soup init [--template <template>] [--output <path>]`
```

--------------------------------

### GET /v1/adapters

Source: https://trysoup.dev/docs/speculative-decoding

Lists all loaded LoRA adapters and their current activation status.

```APIDOC
## GET /v1/adapters

### Description
Retrieves a list of all loaded adapters and identifies which one is currently active.

### Method
GET

### Endpoint
/v1/adapters

### Response
#### Success Response (200)
- **adapters** (array) - List of objects containing adapter names and active status.
- **active** (string) - The name of the currently active adapter.
```

--------------------------------

### Execute Training

Source: https://trysoup.dev/docs/fine-tune-gemma-3-qlora

Run the training process using the specified configuration file.

```bash
soup train --config gemma3.yaml
```

--------------------------------

### MCP Server Management

Source: https://trysoup.dev/docs/cli-reference

Installation and execution commands for the Model Context Protocol server.

```bash
pip install "soup-cli[mcp]"
soup mcp serve                    # stdio · 14 read-only tools
soup mcp serve --allow-mutating   # also list train_start / export (plan-only)
```

--------------------------------

### Manage training recipes

Source: https://trysoup.dev/docs/cli-reference

Commands to list, display, use, and search for pre-defined training configurations.

```bash
soup recipes list                          # List all 142 recipes
soup recipes show <name>                   # Print recipe YAML
soup recipes use <name> [--output <path>]  # Copy to soup.yaml
soup recipes search <query>                # Search by model/task/size
```

--------------------------------

### Initialize compliance packs and CI workflows

Source: https://trysoup.dev/docs/cli-reference

Generate compliance-ready configurations, model cards, and CI workflow files.

```bash
soup init --template hipaa|soc2|eu-ai-act|sr-11-7   # regulation-shaped starting config
soup card <registry-id> -o MODELCARD.md            # provenance-carrying HF model card
soup push --model ./out --repo you/m --card <registry-id>   # upload it as the README (HF only)
soup ci init [--data d.jsonl --suite s.yaml --evidence ev.json] \
    [--branch main] [--python 3.11] [--force]      # writes .github/workflows/soup-gate.yml
```

--------------------------------

### Export Model to TensorRT-LLM

Source: https://trysoup.dev/docs/export

Export models to TensorRT-LLM format after installing the necessary dependencies.

```bash
pip install "soup-cli[tensorrt]"
soup export --model ./output --format tensorrt
soup export --model ./output --format tensorrt --output ./model_trt
```

--------------------------------

### Export Model to ONNX

Source: https://trysoup.dev/docs/export

Export models to ONNX format after installing the necessary dependencies.

```bash
pip install "soup-cli[onnx]"
soup export --model ./output --format onnx
soup export --model ./output --format onnx --output ./model_onnx
```

--------------------------------

### Show Recipe Details

Source: https://trysoup.dev/docs/recipes

Prints the full YAML configuration of a specific recipe to standard output.

```bash
# Print full YAML to stdout
soup recipes show llama3-sft
```

--------------------------------

### Training Profiler

Source: https://trysoup.dev/docs/experiments

Estimates memory, speed, and GPU requirements before starting a training run.

```bash
soup profile --config soup.yaml            # model, task and quantization come from the config
soup profile --config soup.yaml
```

--------------------------------

### Tasks file format

Source: https://trysoup.dev/docs/quant-check

Example structure for the JSONL tasks file used for evaluation.

```jsonl
{"prompt": "Solve: ...", "expected": "42", "category": "math.competition_hard", "scoring": "exact"}
{"prompt": "Capital of France?", "expected": "Paris", "scoring": "contains"}
```

--------------------------------

### List Available Recipes

Source: https://trysoup.dev/docs/recipes

Displays all available recipes including model names, tasks, and descriptions.

```bash
soup recipes list
```

--------------------------------

### Manage Dynamic LoRA Adapters

Source: https://trysoup.dev/docs/speculative-decoding

Configure available adapters at startup and manage their activation state at runtime via HTTP requests.

```bash
soup serve --model base-model --adapters chat=./chat-adapter --adapters code=./code-adapter
```

```bash
curl -X POST http://localhost:8000/v1/adapters/activate/chat
# → {"active": "chat", "status": "ok"}

curl -X POST http://localhost:8000/v1/adapters/deactivate
# → {"active": null, "status": "ok"}

curl http://localhost:8000/v1/adapters
# → {"adapters": [{"name": "chat", "active": true}, ...], "active": "chat"}
```

--------------------------------

### Configure MLX backend in soup.yaml

Source: https://trysoup.dev/docs/mlx-backend

Define the backend as mlx in the configuration file to enable native Apple Silicon training.

```yaml
base: mlx-community/Llama-3.1-8B-Instruct-4bit
backend: mlx
task: sft

data:
  train: data.jsonl
  format: chatml

training:
  epochs: 3
  lr: 1e-4
  batch_size: 2
  lora:
    r: 16
    alpha: 32
```

--------------------------------

### Export and Serve Model

Source: https://trysoup.dev/docs/getting-started

Convert models to GGUF format or start an OpenAI-compatible inference server.

```bash
# Export to GGUF for Ollama / llama.cpp
soup export --model ./output --format gguf --quant q4_k_m

# Start OpenAI-compatible server
soup serve --model ./output --port 8000
```

--------------------------------

### GET /api/tool-outputs

Source: https://trysoup.dev/docs/v0539-live-ux

Polls the most recent tool-call records for the Web UI Tool Outputs panel.

```APIDOC
## GET /api/tool-outputs

### Description
JSON poll of the most-recent tool-call records, designed to be paired with the Web UI Tool Outputs panel.

### Method
GET

### Endpoint
/api/tool-outputs
```

--------------------------------

### Execute DPO Training

Source: https://trysoup.dev/docs/dpo-training-guide

Run the training process using the specified configuration file.

```bash
soup train --config dpo.yaml
```

--------------------------------

### GET /metrics

Source: https://trysoup.dev/docs/speculative-decoding

Retrieves server performance metrics including request counts and latency percentiles.

```APIDOC
## GET /metrics

### Description
Returns performance metrics for the inference server, including total requests, generated tokens, and latency statistics.

### Method
GET

### Endpoint
/metrics

### Response
#### Success Response (200)
- **requests_total** (integer) - Total number of requests processed.
- **tokens_generated_total** (integer) - Total number of tokens generated.
- **active_requests** (integer) - Number of currently active requests.
- **latency_p50_ms** (float) - 50th percentile latency in milliseconds.
- **latency_p95_ms** (float) - 95th percentile latency in milliseconds.
- **latency_samples** (integer) - Number of samples used for latency calculation.
```

--------------------------------

### VRAM Advisory Output

Source: https://trysoup.dev/docs/soup-cans-v2

Example of the advisory message printed when VRAM pressure exceeds thresholds.

```text
[advisory] VRAM at 94% — try batch=2, accum=8 (preserves effective batch=16)
```

--------------------------------

### Spike Recovery Hint File

Source: https://trysoup.dev/docs/soup-cans-v2

Example of the JSON hint file generated during loss-spike recovery.

```json
{"original_lr": 2e-4, "recovery_lr": 5e-5, "decay_factor": 0.25, "trigger_step": 482}
```

--------------------------------

### Inspect a recipe bundle

Source: https://trysoup.dev/docs/soup-cans

Displays the manifest and configuration of a .can file without extracting the archive.

```bash
soup can inspect chat-llama-v1.can
```

--------------------------------

### Configure Multi-objective Preference Loss

Source: https://trysoup.dev/docs/preference-variety

Example of blending multiple preference losses with defined weights.

```yaml
task: preference
training:
  preference_loss_weights: {dpo: 0.7, bco: 0.3}
```

--------------------------------

### Enable FSDP2 with torch.compile

Source: https://trysoup.dev/docs/multi-gpu

Enable FSDP2 compilation in the configuration and launch with the full_shard preset.

```yaml
training:
  use_fsdp2_compile: true
```

```bash
soup train --gpus 4 --fsdp full_shard
```

--------------------------------

### Plugin Management CLI

Source: https://trysoup.dev/docs/plugins

Commands to list, install, enable, and disable plugins within the Soup environment.

```APIDOC
## CLI Plugin Management

### Description
Manage the lifecycle of plugins using the soup CLI.

### Commands
- `soup plugins list`: List all available plugins.
- `soup plugins install [path]`: Install a plugin from a local path.
- `soup plugins enable [name]`: Enable a specific plugin.
- `soup plugins disable [name]`: Disable a specific plugin.
```

--------------------------------

### Launch Multi-GPU Training

Source: https://trysoup.dev/docs/faq

Command to initiate training across multiple GPUs.

```bash
soup train --gpus N
```

--------------------------------

### Configure Supervised Fine-Tuning (SFT)

Source: https://trysoup.dev/docs/training

Standard configuration for instruction-response pair training.

```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft

data:
  train: ./data/train.jsonl
  format: alpaca

training:
  epochs: 3
  lr: 2e-5
  batch_size: auto
  quantization: 4bit
  lora:
    r: 64
    alpha: 16
```

--------------------------------

### Pack a recipe bundle

Source: https://trysoup.dev/docs/soup-cans

Creates a .can archive from a specified entry ID.

```bash
soup can pack \
  --entry-id chat-llama@v1 \
  --out chat-llama-v1.can
```

--------------------------------

### Serve Anthropic-compatible Endpoint

Source: https://trysoup.dev/docs/v0536-7-plugins-anthropic

Start the Soup server with the transformers backend to expose the Anthropic Messages API.

```bash
soup serve --model ./output --backend transformers
# POST http://localhost:8000/v1/messages — Anthropic Messages shape
```

--------------------------------

### Configure Unsloth Backend

Source: https://trysoup.dev/docs/backends

Enable the Unsloth backend in the configuration file to improve training speed and memory efficiency.

```yaml
base: meta-llama/Llama-3.1-8B-Instruct
task: sft
backend: unsloth    # 2-5x faster, -80% VRAM

data:
  train: ./data/train.jsonl
  format: alpaca

training:
  epochs: 3
  lr: 2e-5
  quantization: 4bit
  lora:
    r: 64
    alpha: 16
```

--------------------------------

### soup train

Source: https://trysoup.dev/docs/cli-reference

Starts a model training run with support for various distributed training configurations and integrations.

```APIDOC
## soup train

### Description
Start a training run.

### Usage
`soup train [--config <path>] [--resume auto|<checkpoint>] [--wandb] [--tensorboard] [--deepspeed zero2|zero3|zero2_offload|zero3_offload|zero++|zero_pp] [--fsdp full_shard|shard_grad|full_offload] [--gpus auto|<N>] [--gate <suite>] [--push-as <user/repo>] [--hf-resume] [--yes]`
```

--------------------------------

### Configure Audio Fine-Tuning

Source: https://trysoup.dev/docs/multimodal

YAML configuration for fine-tuning audio-language models like Qwen2-Audio.

```yaml
base: Qwen/Qwen2-Audio-7B-Instruct
task: sft
modality: audio

data:
  train: ./data/audio_train.jsonl
  format: audio
  audio_dir: ./data/audio
  val_split: 0.1

training:
  epochs: 3
  lr: 1e-5
  quantization: 4bit
  lora:
    r: 64
    alpha: 16
```

--------------------------------

### Configure PRM Task

Source: https://trysoup.dev/docs/v05311-grpo-prm-longlora

Example YAML configuration for setting up a Process Reward Model (PRM) task.

```yaml
task: prm
base: deepseek-ai/deepseek-math-7b-base
data:
  train: prm_steps.jsonl
  format: prm   # {prompt, completions, labels}
```

--------------------------------

### Execute full training to deployment pipeline

Source: https://trysoup.dev/docs/export-to-gguf-ollama

A single command sequence to train, export, and deploy a model.

```bash
soup train --config soup.yaml && \
soup export --model ./runs/latest --format gguf --quant q4_k_m && \
soup deploy ollama --model ./runs/latest/model.gguf --name my-model
```

--------------------------------

### View iteration manifest structure

Source: https://trysoup.dev/docs/soup-loop

Example of the JSON-formatted iteration record stored in .soup-loops/ for each loop cycle.

```text
iteration_id        20260515T231600-a1f3b2c4
started_at          2026-05-15T23:16:00Z
finished_at         2026-05-15T23:25:12Z
pairs_harvested     89
run_id              run-8f3
gate_verdict        OK            # OK | MAJOR | SKIPPED
canary_verdict      OK            # OK | MAJOR | UNKNOWN | None
shipped             true
rolled_back         false
estimated_cost_usd  2.40
notes               []
```

--------------------------------

### Interact with API Endpoints

Source: https://trysoup.dev/docs/serving

Examples for interacting with the OpenAI-compatible API using curl or the OpenAI Python SDK.

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "output",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
response = client.chat.completions.create(
    model="output",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

--------------------------------

### Configure BCO Training

Source: https://trysoup.dev/docs/preference-variety

Example configuration for Binary Classifier Optimization using the dpo data format.

```yaml
task: bco
data:
  train: ./data/preferences.jsonl
  format: dpo
training:
  bco_beta: 0.1
  lora: { r: 64, alpha: 16 }
  quantization: 4bit
```

--------------------------------

### Iterative DPO Artifact Structure

Source: https://trysoup.dev/docs/loop-hardening

Example directory structure for artifacts generated during iterative DPO rounds.

```text
./iter-dpo/round-01/pairs.jsonl
./iter-dpo/round-01/adapter/
./iter-dpo/round-02/pairs.jsonl
./iter-dpo/round-02/adapter/
...
```
Tokens:
Raw