Ludwig

https://ludwig.ai/latest/
Admin
Ludwig is a low-code declarative deep learning framework for building custom AI models like LLMs and...
Tokens:324,637
Snippets:3,279
Trust Score:8.2
Update:4 months ago
Show doc for...
Tokens:
Raw
### Running Open Set Recognition Example

Source: https://ludwig.ai/latest/examples/open_set_recognition

This command sequence shows how to set up and run the open set recognition example. It involves navigating to the example directory, installing Ludwig, and executing the training script.

```bash
cd examples/open_set_recognition
pip install ludwig
python train_open_set.py

```

--------------------------------

### Install Ray and Dependencies

Source: https://ludwig.ai/latest/developer_guide/run_tests_on_gpu_using_ray

Install the Ray framework and necessary AWS SDK components locally.

```bash
pip install -U "ray[default]" boto3
```

--------------------------------

### Callback: Trainer Train Setup

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called in every trainer (distributed or local) before training starts. Receives the trainer instance, save path, and coordinator status.

```python
def on_trainer_train_setup(self, trainer, save_path: str, is_coordinator: bool):
    """Called in every trainer (distributed or local) before training starts.

    :param trainer: The trainer instance.
    :type trainer: trainer: ludwig.models.Trainer
    :param save_path: The path to the directory model is saved in.
    :param is_coordinator: Is this trainer the coordinator.
    """
    pass
```

--------------------------------

### HyperoptSampler Example Initialization

Source: https://ludwig.ai/latest/developer_guide/add_a_hyperopt

Example of initializing a GridSampler with a goal and parameter configuration. This demonstrates how to define the optimization objective and the search space for hyperparameters.

```python
goal = "minimize"
parameters = {
    "trainer.learning_rate": {"type": "float", "low": 0.001, "high": 0.1, "steps": 4, "scale": "linear"},
    "combiner.num_fc_layers": {"type": "int", "low": 2, "high": 6, "steps": 3},
}

sampler = GridSampler(goal, parameters)
```

--------------------------------

### HyperoptExecutor Initialization Example

Source: https://ludwig.ai/latest/developer_guide/add_a_hyperopt

Example demonstrating how to initialize a HyperoptExecutor with a GridSampler, specifying goal, parameters, output feature, metric, and split.

```python
goal = "minimize"
parameters = {
    "trainer.learning_rate": {"type": "float", "low": 0.001, "high": 0.1, "steps": 4, "scale": "linear"},
    "combiner.num_fc_layers": {"type": "int", "low": 2, "high": 6, "steps": 3},
}
output_feature = "combined"
metric = "loss"
split = "validation"

grid_sampler = GridSampler(goal, parameters)
executor = SerialExecutor(grid_sampler, output_feature, metric, split)

```

--------------------------------

### Callback: Test Start

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called on the coordinator before testing starts. Receives trainer, progress tracker, and save path.

```python
def on_test_start(self, trainer, progress_tracker, save_path: str):
    """Called on coordinator before testing starts.

    :param trainer: The trainer instance.
    :type trainer: ludwig.models.trainer.Trainer
    :param progress_tracker: An object which tracks training progress.
    :type progress_tracker: ludwig.models.trainer.ProgressTracker
    :param save_path: The path to the directory model is saved in.
    """
    pass
```

--------------------------------

### Install Ray Serve Dependencies

Source: https://ludwig.ai/latest/examples/ray_serve

Install the necessary packages for Ray Serve deployment, which includes Ray itself.

```bash
pip install "ludwig[distributed]"  # includes ray[serve]

```

--------------------------------

### Install Ludwig and LLM Dependencies

Source: https://ludwig.ai/latest/examples/llms/llm_finetuning

Install Ludwig with the necessary dependencies for large language model (LLM) functionalities. This is the initial setup step for using Ludwig for LLM fine-tuning.

```bash
pip install ludwig ludwig[llm]
```

--------------------------------

### Callback: Epoch Start

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called on the coordinator only before the start of each epoch. Receives trainer, progress tracker, and save path.

```python
def on_epoch_start(self, trainer, progress_tracker, save_path: str):
    """Called on coordinator only before the start of each epoch.

    :param trainer: The trainer instance.
    :type trainer: ludwig.models.trainer.Trainer
    :param progress_tracker: An object which tracks training progress.
    :type progress_tracker: ludwig.models.trainer.ProgressTracker
    :param save_path: The path to the directory model is saved in.
    """
    pass
```

--------------------------------

### Install Ray Cluster Launcher

Source: https://ludwig.ai/latest/getting_started/ray

Install the necessary Ray package to manage cluster resources.

```bash
pip install ray
```

--------------------------------

### Install Ludwig with Full Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with the complete set of dependencies, including all optional functionalities. Use this for a full-featured installation.

```bash
ludwig[full]
```

--------------------------------

### Install LLM Provider SDKs

Source: https://ludwig.ai/latest/user_guide/llms/config_generation

Install the necessary SDK for your chosen LLM provider. Claude uses 'anthropic', while GPT-4 uses 'openai'.

```bash
pip install anthropic    # for Claude (default)
# or
pip install openai       # for GPT-4o
```

--------------------------------

### Start REST API Server

Source: https://ludwig.ai/latest/user_guide/serving

Starts a FastAPI server for Ludwig models. Use this for general-purpose serving.

```bash
ludwig serve --model_path=/path/to/model
```

--------------------------------

### Install KServe

Source: https://ludwig.ai/latest/examples/ray_serve

Install the KServe library, which is required for serving Ludwig models using KServe.

```bash
pip install kserve

```

--------------------------------

### Example Hyperopt Sampled Configurations

Source: https://ludwig.ai/latest/user_guide/hyperopt

These examples illustrate the structure of configurations sampled by hyperopt when using nested parameters. Each trial represents a unique combination of sampled nested configs and individual parameters.

```yaml
# Trial 1
  combiner:
    type: tabnet
  trainer:
  learning_rate: 0.001
  batch_size: 64
  decay_rate: 0.02

# Trial 2
  combiner:
    type: tabnet
  trainer:
    learning_rate: 0.001
    batch_size: 64
    decay_rate: 0.001

# Trial 3
  combiner:
    type: concat
  trainer:
    batch_size: 64
    decay_rate: 0.001

```

--------------------------------

### Install Ludwig from PyPI

Source: https://ludwig.ai/latest

Install the Ludwig library using pip. Ensure you have Python 3.12+.

```bash
pip install ludwig
```

--------------------------------

### Callback: Validation Start

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called on the coordinator before validation starts. Receives trainer, progress tracker, and save path.

```python
def on_validation_start(self, trainer, progress_tracker, save_path: str):
    """Called on coordinator before validation starts.

    :param trainer: The trainer instance.
    :type trainer: ludwig.models.trainer.Trainer
    :param progress_tracker: An object which tracks training progress.
    :type progress_tracker: ludwig.models.trainer.ProgressTracker
    :param save_path: The path to the directory model is saved in.
    """
    pass
```

--------------------------------

### Install vLLM for LLM Serving

Source: https://ludwig.ai/latest/user_guide/serving

Installs the vLLM library, which is a prerequisite for using Ludwig's OpenAI-compatible LLM serving backend.

```bash
pip install vllm
```

--------------------------------

### Starting Ray and Ludwig Distributed Training

Source: https://ludwig.ai/latest/examples/vlm_finetuning

Commands to start a Ray cluster head node and then initiate Ludwig training using the distributed configuration.

```bash
ray start --head
ludwig train --config vlm_config_distributed.yaml --dataset vqa_dataset.csv

```

--------------------------------

### Install TIMM for Generic Encoder

Source: https://ludwig.ai/latest/configuration/features/image_features

Before using the generic TIMM encoder, install the pytorch-image-models library. This enables access to over 1,000 pretrained vision models.

```bash
pip install timm
```

--------------------------------

### Mamba Encoder Configuration Example

Source: https://ludwig.ai/latest/configuration/features/sequence_features

This is a configuration example for the Mamba encoder. It specifies various parameters such as dropout, embedding size, model dimensions, and layer configurations. Use this as a starting point for integrating Mamba into your Ludwig models.

```yaml
encoder:
    type: mamba
    dropout: 0.1
    embedding_size: 256
    norm: null
    num_fc_layers: 0
    fc_dropout: 0.0
    fc_activation: relu
    representation: dense
    weights_initializer: xavier_uniform
    embeddings_on_cpu: false
    embeddings_trainable: true
    reduce_output: mean
    norm_params: null
    fc_layers: null
    skip: false
    adapter: null
    d_model: 256
    n_layers: 4
    d_state: 16
    d_conv: 4
    expand_factor: 2
    use_bias: true
    should_embed: true
    pretrained_embeddings: null
    output_size: 256

```

--------------------------------

### Full Example: Training Baseline and Calibrated Models

Source: https://ludwig.ai/latest/user_guide/uncertainty

Demonstrates training a baseline Ludwig model and a temperature-scaled calibrated model. Compares predictions to evaluate calibration improvement.

```python
import copy
from ludwig.api import LudwigModel

config = {
    "model_type": "ecd",
    "input_features": [...],
    "output_features": [
        {
            "name": "label",
            "type": "binary",
            "decoder": {"type": "mlp_classifier"},
        }
    ],
    "trainer": {"epochs": 30},
}

# Baseline — no calibration
baseline = LudwigModel(config=config)
baseline.train(dataset=df)

# Temperature scaling — one line change
calibrated_config = copy.deepcopy(config)
calibrated_config["output_features"][0]["decoder"]["calibration"] = "temperature_scaling"

calibrated = LudwigModel(config=calibrated_config)
calibrated.train(dataset=df)

# Accuracy is the same; ECE should be lower
_, baseline_preds, _ = baseline.predict(dataset=df)
_, calibrated_preds, _ = calibrated.predict(dataset=df)
```

--------------------------------

### Set Feature Configuration

Source: https://ludwig.ai/latest/configuration/features/set_features

Example of configuring a set feature with an 'embed' encoder and no tied weights. This is a basic setup for a set feature.

```yaml
name: set_column_name
type: set
tied: null
encoder:
    type: embed

```

--------------------------------

### Example: Upload Model to HuggingFace Hub

Source: https://ludwig.ai/latest/user_guide/command_line_interface

Demonstrates how to upload a fine-tuned model to a specified HuggingFace Hub repository using the CLI.

```bash
ludwig upload hf_hub --repo_id ludwig/test_model --model_path ~/trained_models/my_model
```

--------------------------------

### Shell Script for DeepSpeed Training on Ray

Source: https://ludwig.ai/latest/examples/llms/llm_finetuning_deepspeed

Run this script to start fine-tuning with DeepSpeed on a Ray backend. It points to the appropriate YAML configuration file for the Ray setup.

```bash
#!/usr/bin/env bash

# Fail fast if an error occurs
set -e

# Get the directory of this script, which contains the config file
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )

# Train
ludwig train --config ${SCRIPT_DIR}/imdb_deepspeed_zero3_ray.yaml --dataset ludwig://imdb

```

--------------------------------

### Launch Tensorboard for Training Metrics

Source: https://ludwig.ai/latest/user_guide/visualizations

Command to visualize training metrics by pointing Tensorboard to the model log directory.

```bash
tensorboard --logdir </path/to/model>/log
```

--------------------------------

### Resumable Optuna Run with SQLite Storage

Source: https://ludwig.ai/latest/configuration/hyperparameter_optimization

Example of configuring the Optuna executor to use SQLite for storing trial results, enabling resumable runs. Install the 'optuna' package.

```yaml
hyperopt:
  executor:
    type: optuna
    num_samples: 100
    sampler: tpe
    pruner: median
    storage: sqlite:///hyperopt.db

```

--------------------------------

### Implement on_preprocess_start Callback

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

This method is called just before the data preprocessing stage begins. It receives the configuration dictionary.

```python
def on_preprocess_start(self, config: Dict[str, Any]):
    """Called before preprocessing starts.

    :param config: The config dictionary.
    """
    pass

```

--------------------------------

### Transformer Encoder Configuration

Source: https://ludwig.ai/latest/configuration/features/sequence_features

Example configuration for a Transformer encoder in Ludwig. This setup specifies various parameters for the encoder, including dropout, number of layers, embedding size, and normalization.

```yaml
encoder:
    type: transformer
    dropout: 0.1
    num_layers: 1
    embedding_size: 256
    output_size: 256
    norm: null
    num_fc_layers: 0
    fc_dropout: 0.0
    hidden_size: 256
    transformer_output_size: 256
    fc_activation: relu
    representation: dense
    use_bias: true
    bias_initializer: zeros
    weights_initializer: xavier_uniform
    embeddings_on_cpu: false
    embeddings_trainable: true
    reduce_output: last
    norm_params: null
    fc_layers: null
    skip: false
    adapter: null
    num_heads: 8
    pretrained_embeddings: null
    use_rope: false

```

--------------------------------

### Callback: Batch Start

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called on the coordinator only before each batch. Receives trainer, progress tracker, and save path.

```python
def on_batch_start(self, trainer, progress_tracker, save_path: str):
    """Called on coordinator only before each batch.

    :param trainer: The trainer instance.
    :type trainer: ludwig.models.trainer.Trainer
    :param progress_tracker: An object which tracks training progress.
    :type progress_tracker: ludwig.models.trainer.ProgressTracker
    :param save_path: The path to the directory model is saved in.
    """
    pass
```

--------------------------------

### Example LLM Encoder Configuration for Text Classification

Source: https://ludwig.ai/latest/user_guide/llms/text_classification

Configure an ECD model to use a 4-bit quantized LoRA adapter for a Llama-2-7b model for text classification. This setup fine-tunes the adapter and trains a classification head simultaneously.

```yaml
model_type: ecd
input_features:
  - name: title
    type: text
    encoder:
      type: llm
      adapter:
        type: lora
      base_model: meta-llama/Llama-2-7b-hf
      quantization:
        bits: 4
    column: title
output_features:
  - name: class
    type: category
    column: class
trainer:
  epochs: 3
  optimizer:
    type: paged_adam


```

--------------------------------

### Install Ludwig with Benchmarking Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for model benchmarking. This command installs the core package along with benchmarking-specific dependencies.

```bash
ludwig[benchmarking]
```

--------------------------------

### Launch Ray Cluster

Source: https://ludwig.ai/latest/developer_guide/run_tests_on_gpu_using_ray

Initialize the Ray cluster using the specified configuration files.

```bash
export CLUSTER="$HOME/cluster_g4dn.yaml" export CLUSTER_CPU="$HOME/cluster_cpu.yaml" ray up $CLUSTER
```

--------------------------------

### Install Ludwig with Visualization Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for visualization functionality. This command installs the core package along with visualization-specific dependencies.

```bash
ludwig[viz]
```

--------------------------------

### Example: Synthesize Dataset with Multiple Features

Source: https://ludwig.ai/latest/user_guide/command_line_interface

An example demonstrating how to generate a synthetic dataset with various feature types including text, category, number, binary, set, bag, sequence, timeseries, date, h3, vector, and image.

```bash
ludwig synthesize_dataset --features="[ \
  {name: text, type: text}, \
  {name: category, type: category}, \
  {name: number, type: number}, \
  {name: binary, type: binary}, \
  {name: set, type: set}, \
  {name: bag, type: bag}, \
  {name: sequence, type: sequence}, \
  {name: timeseries, type: timeseries}, \
  {name: date, type: date}, \
  {name: h3, type: h3}, \
  {name: vector, type: vector}, \
  {name: image, type: image} \
]" --dataset_size=10 --output_path=synthetic_dataset.csv

```

--------------------------------

### Deploy Ludwig Model with Ray Serve (CLI)

Source: https://ludwig.ai/latest/examples/ray_serve

Deploy a Ludwig model using the command-line interface. This example demonstrates how to configure GPU usage, port, and blocking behavior.

```bash
# Two GPU replicas on port 8080
python examples/serving/ray_serve/deploy.py \
    --model_path ./results/my_model \
    --name sentiment \
    --num_replicas 2 \
    --gpu \
    --port 8080 \
    --block

```

--------------------------------

### Install Ludwig with Serving Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for serving functionality. This command installs the core package along with serving-specific dependencies.

```bash
ludwig[serve]
```

--------------------------------

### Install Ludwig with Prediction Explanation Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for prediction explanations. This command installs the core package along with explain-specific dependencies.

```bash
ludwig[explain]
```

--------------------------------

### Example Audio Feature Configuration

Source: https://ludwig.ai/latest/configuration/features/audio_features

This snippet shows a basic configuration for an audio feature, specifying its name, type, and encoder.

```yaml
name: audio_column_name
type: audio
tied: null
encoder:
    type: parallel_cnn
```

--------------------------------

### Install Ludwig with Hyperparameter Optimization Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for hyperparameter optimization. This command installs the core package along with hyperopt-specific dependencies.

```bash
ludwig[hyperopt]
```

--------------------------------

### Dataset Setup with Background Samples

Source: https://ludwig.ai/latest/configuration/features/open_set_recognition

The training dataset must include samples for known classes and a dedicated background class. Background samples can be from an 'unknown' category, out-of-distribution data, or synthetic noise.

```csv
label,feature_1,feature_2
cat,0.3,0.7
dog,0.8,0.1
...
background,0.5,0.5   ← background/unknown samples
background,0.2,0.9
```

--------------------------------

### Install Ludwig with Testing Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for running integration and unit tests. This command installs the core package along with test-specific dependencies.

```bash
ludwig[test]
```

--------------------------------

### Check Ludwig Installation

Source: https://ludwig.ai/latest/user_guide/command_line_interface

Verifies a correct Ludwig installation by running a short training on synthetic data. This command is essential for debugging environment issues or confirming a successful installation.

```bash
ludwig check_install
```

--------------------------------

### Hyperopt Configuration Example

Source: https://ludwig.ai/latest/configuration/hyperparameter_optimization

Defines the hyperparameter optimization goal, output feature, metric, data split, parameters to optimize, search algorithm, and executor configuration.

```yaml
hyperopt:
  goal: minimize
  output_feature: combined
  metric: loss
  split: validation
  parameters:
    title.encoder.cell_type: ... # title is a text feature type
    title.encoder.num_layers: ...
    combiner.num_fc_layers: ...
    section.encoder.embedding_size: ...
    preprocessing.text.vocab_size: ...
    trainer.learning_rate: ...
    trainer.optimizer.type: ...
    ...
  search_alg:
    type: variant_generator  # random, hyperopt, bohb, ...
    # search_alg parameters...
  executor:
    type: ray
    num_samples: ...
    scheduler:
      type: fifo  # hb_bohb, asynchyperband, ...
      # scheduler parameters...

```

--------------------------------

### Install Ludwig with Distributed Training Dependencies

Source: https://ludwig.ai/latest/getting_started/installation

Install Ludwig with additional packages for distributed training on Ray using Dask. This command installs the core package along with distributed training-specific dependencies.

```bash
ludwig[distributed]
```

--------------------------------

### Implement on_hyperopt_trial_start Callback

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called at the beginning of each individual hyperparameter optimization trial. It receives the dictionary of parameters for that trial.

```python
def on_hyperopt_trial_start(self, parameters: Dict[str, Any]):
    """Called before the start of each hyperparameter optimization trial.

    :param parameters: The complete dictionary of parameters for this
           hyperparameter optimization experiment.
    """
    pass

```

--------------------------------

### Install Prerequisites on MacOS with Apple Silicon

Source: https://ludwig.ai/latest/developer_guide/contributing

Install necessary prerequisites like cmake and libomp on MacOS with Apple Silicon using homebrew. This step is required if the standard installation encounters errors.

```bash
brew install cmake libomp
```

--------------------------------

### Initialize and Configure Hyperopt

Source: https://ludwig.ai/latest/user_guide/api/LudwigModel

Initializes the ModelConfig, retrieves hyperopt configuration, and sets up the backend. This snippet is useful for setting up the hyperparameter tuning environment.

```python
    # Initialize config object
    config_obj = ModelConfig.from_dict(upgraded_config)

    # Retain pre-merged config for hyperopt schema generation
    premerged_config = copy.deepcopy(upgraded_config)

    # Get full config with defaults
    full_config = config_obj.to_dict()  # TODO (Connor): Refactor to use config object

    hyperopt_config = full_config[HYPEROPT]

    # Explicitly default to a local backend to avoid picking up Ray
    # backend from the environment.
    backend = backend or config_dict.get("backend") or "local"
    backend = initialize_backend(backend)

    update_hyperopt_params_with_defaults(hyperopt_config)

    # Check if all features are grid type parameters and log UserWarning if needed
    log_warning_if_all_grid_type_parameters(hyperopt_config)

    # Infer max concurrent trials
    if hyperopt_config[EXECUTOR].get(MAX_CONCURRENT_TRIALS) == AUTO:
        hyperopt_config[EXECUTOR][MAX_CONCURRENT_TRIALS] = backend.max_concurrent_trials(hyperopt_config)
        logger.info(f"Setting max_concurrent_trials to {hyperopt_config[EXECUTOR][MAX_CONCURRENT_TRIALS]}")

    # Print hyperopt config
    logger.info("Hyperopt Config")
    logger.info(pformat(hyperopt_config, indent=4))
    logger.info("\n")

    search_alg = hyperopt_config[SEARCH_ALG]
    executor = hyperopt_config[EXECUTOR]
    parameters = hyperopt_config[PARAMETERS]
    split = hyperopt_config[SPLIT]
    output_feature = hyperopt_config["output_feature"]
    metric = hyperopt_config[METRIC]
    goal = hyperopt_config[GOAL]
```

--------------------------------

### Verify AWS CLI Installation

Source: https://ludwig.ai/latest/developer_guide/run_tests_on_gpu_using_ray

Check if the AWS CLI is installed and configured correctly.

```bash
aws s3 ls
```

--------------------------------

### Mode Configuration Examples for Audio Features

Source: https://ludwig.ai/latest/user_guide/datasets/data_preprocessing

Demonstrates the configuration of 'eager', 'lazy', and 'lazy_cached' preprocessing modes for audio features. 'lazy' mode defaults to an auto prefetch size, while 'lazy_cached' uses auto prefetch for epoch 1 and 0 for subsequent epochs.

```yaml
input_features:
  # Eager — decode everything upfront; fast training, high preprocessing memory
  - name: small_audio
    type: audio
    preprocessing:
      mode: eager

  # Lazy (default) — decode per batch; low memory, GPU may idle on slow storage
  - name: large_audio
    type: audio
    preprocessing:
      mode: lazy
      prefetch_size: null      # auto (4)

  # Lazy-cached — decode + cache on epoch 1; memmap read from epoch 2+
  - name: huge_audio
    type: audio
    preprocessing:
      mode: lazy_cached
      prefetch_size: null      # auto (4 for epoch 1, 0 for epoch 2+)
      lazy_cache_dir: /fast/nvme/cache

```

--------------------------------

### Install TabPFN v2

Source: https://ludwig.ai/latest/configuration/combiner

Install the optional 'tabpfn' package for TabPFN v2 combiner functionality.

```bash
pip install tabpfn  # optional dependency

```

--------------------------------

### Start vLLM Server for LLMs

Source: https://ludwig.ai/latest/user_guide/serving

Launches an OpenAI-compatible server for LLM models using vLLM. Configure GPU memory utilization and other parameters as needed.

```python
from ludwig.serve_vllm import run_vllm_server

run_vllm_server(
    model_path="path/to/ludwig/model",
    host="0.0.0.0",
    port=8000,
    gpu_memory_utilization=0.9,
)

```

--------------------------------

### Implement on_hyperopt_start Callback

Source: https://ludwig.ai/latest/developer_guide/add_an_integration

Called before any trials in the hyperparameter optimization process begin. It receives the experiment name.

```python
def on_hyperopt_start(self, experiment_name: str):
    """Called before any hyperparameter optimization trials are started.

    :param experiment_name: The name of the current experiment.
    """
    pass

```

--------------------------------

### Callback for Hyperopt Initialization and Preprocessing

Source: https://ludwig.ai/latest/user_guide/api/LudwigModel

Executes callbacks for hyperopt initialization and preprocessing start. This allows for custom actions to be taken at specific stages of the hyperparameter optimization process.

```python
    for callback in callbacks or []:
        callback.on_hyperopt_init(experiment_name)

    if not should_tune_preprocessing(full_config):
        # preprocessing is not being tuned, so generate it once before starting trials
        for callback in callbacks or []:
            callback.on_hyperopt_preprocessing_start(experiment_name)
```

--------------------------------

### Serve a Pre-trained Model (CLI)

Source: https://ludwig.ai/latest/user_guide/command_line_interface

Use this command to load a pre-trained model and serve it on an HTTP server. Specify the model path using the --model_path argument.

```bash
ludwig serve [options]
```

--------------------------------

### Example Number Feature Entry

Source: https://ludwig.ai/latest/configuration/features/number_features

An example of how to define a number feature in the input features list with a dense encoder.

```yaml
name: number_column_name
type: number
tied: null
encoder:
    type: dense


```

--------------------------------

### Ludwig Configuration Example

Source: https://ludwig.ai/latest/user_guide/how_ludwig_works

This snippet shows a comprehensive Ludwig configuration, defining input and output features, trainer settings, backend, and hyperparameter optimization parameters. Use this as a template for your own Ludwig projects.

```yaml
input_features:
- 
  name: title
  type: text
  encoder:
      type: rnn
      cell: lstm
      num_layers: 2
      state_size: 128
  preprocessing:
    tokenizer: space_punct
- 
  name: author
  type: category
  encoder:
      embedding_size: 128
  preprocessing:
    most_common: 10000
- 
  name: description
  type: text
  encoder:
      type: bert
      pretrained_model_name_or_path: answerdotai/ModernBERT-base
- 
  name: cover
  type: image
  encoder:
      type: resnet
      num_layers: 18

output_features:
- 
  name: genre
  type: set
- 
  name: price
  type: number
  preprocessing:
    normalization: zscore

trainer:
  epochs: 50
  batch_size: 256
  optimizer:
    type: adam
    beat1: 0.9
  learning_rate: 0.001

backend:
  type: local
  cache_format: parquet

hyperopt:
  metric: f1
  sampler: random
  parameters:
    title.encoder.num_layers:
      lower: 1
      upper: 5
    trainer.learning_rate:
      values: [0.01, 0.003, 0.001]


```

--------------------------------

### Minimal VLM Configuration (Qwen2-VL)

Source: https://ludwig.ai/latest/user_guide/llms/vlm_finetuning

A basic configuration file for fine-tuning the Qwen2-VL-7B-Instruct model. Ensure 'is_multimodal: true' is set for VLM tasks.

```yaml
model_type: llm
base_model: Qwen/Qwen2-VL-7B-Instruct

# Enable multimodal (VLM) mode — required for image+text models
is_multimodal: true
trust_remote_code: true

input_features:
  - name: image_path
    type: image
  - name: question
    type: text

output_features:
  - name: answer
    type: text

adapter:
  type: lora
  r: 16
  alpha: 32
  target_modules: ["q_proj", "v_proj"]

trainer:
  type: finetune
  epochs: 3
  batch_size: 4
  gradient_accumulation_steps: 8
  learning_rate: 2.0e-5
  learning_rate_scheduler:
    decay: cosine
    warmup_fraction: 0.03

# 4-bit quantisation to fit a 7B VLM on a single 24 GB GPU
quantization:
  bits: 4
  quantization_type: nf4
  compute_dtype: bfloat16

generation:
  max_new_tokens: 256
  temperature: 0.0

```

--------------------------------

### Configure Learning Rate Schedulers

Source: https://ludwig.ai/latest/examples/optimizer_comparison

Sets up learning rate schedulers like cosine annealing, reduce on plateau, or linear warmup. Includes configuration for warmup fraction.

```yaml
trainer:
  learning_rate_scheduler:
    type: cosine
    warmup_fraction: 0.1
```

--------------------------------

### Install Ludwig for Ray Serve

Source: https://ludwig.ai/latest/user_guide/serving

Installs the necessary Ludwig package with distributed serving capabilities for deployment with Ray Serve.

```bash
pip install "ludwig[distributed]"
```

--------------------------------

### Visualize Learning Curves (CLI)

Source: https://ludwig.ai/latest/examples/code_intelligence

Generate learning curve visualizations from training statistics using the Ludwig CLI.

```bash
ludwig visualize \
    --visualization learning_curves \
    --ground_truth_metadata results/experiment_run/model/training_set_metadata.json \
    --training_statistics results/experiment_run/training_statistics.json \
    --file_format png \
    --output_directory visualizations

```

--------------------------------

### Warmup-Stable-Decay (WSD) Scheduler Configuration

Source: https://ludwig.ai/latest/configuration/trainer

Implements the WSD scheduler for long pretraining runs, splitting training into warmup, stable, and decay phases. Fractions must sum to 1.0.

```yaml
trainer:
  learning_rate_scheduler:
    decay: wsd
    wsd_warmup_fraction: 0.1
    wsd_stable_fraction: 0.8
    wsd_decay_fraction: 0.1
```
Tokens:
Raw