### Run Quickstart Example

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_w4a16

Execute the provided example script to apply the quantization algorithm end-to-end.

```bash
python3 llama3_example.py
```

--------------------------------

### on_start

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/transform/awq/base

Initializes and starts the AWQ process on a given state. This method performs essential setup steps after the quantization configuration has been applied, including resolving and validating mappings, and setting up activation cache hooks.

```APIDOC
## on_start

### Description
Starts the AWQ process on the given state. This runs after the quantization mixin has been initialized and the quantization config has been applied. It includes steps to resolve mappings, validate mappings and the quant scheme, and set up activation cache hooks.

### Method Signature
`on_start(state: State, event: Event, **kwargs)`

### Parameters
* **`state`** (`State`) - The state object on which to run AWQ.

### Returns
* `bool` - True on a successful run, False otherwise.

### Raises
* `ValueError` - If token masking is used with unsupported MoE up_proj -> down_proj mappings, or if duo_scaling is used with a TENSOR quantization strategy.
```

--------------------------------

### Run QuIP Example Script

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/transform

Execute the example script to demonstrate applying quip-style transforms before quantization.

```bash
python3 quip_example.py
```

--------------------------------

### Install vLLM and lm-evaluation-harness

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_w8a8_fp8

Install the necessary libraries for evaluating the quantized model with vLLM and lm-evaluation-harness.

```bash
pip install vllm lm_eval==0.4.3

```

--------------------------------

### Install llmcompressor from Source

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_kv_cache

Install the llmcompressor library from its GitHub repository. This is necessary as the KV cache quantization feature is new.

```bash
pip install git+https://github.com/vllm-project/llm-compressor.git@cb98f34d4ec9dd175e6995d12fb02dec39c6f27a

```

--------------------------------

### Install llm-compressor

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/autoround

Clone the repository and install the llm-compressor package in editable mode.

```bash
git clone https://github.com/vllm-project/llm-compressor.git
cd llm-compressor
pip install -e .
```

--------------------------------

### Run iMatrix Example Script

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/imatrix

Execute the provided Python script to run an example demonstrating iMatrix Importance-Weighted Quantization.

```bash
python3 llama3_imatrix_example.py

```

--------------------------------

### Run End-to-End Quantization Example

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_kv_cache

Execute the provided example script to apply the quantization algorithm to a model. This script handles the entire process from loading the model to saving the quantized version.

```bash
python3 llama3_fp8_kv_example.py

```

--------------------------------

### Install vLLM and lm-evaluation-harness

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_kv_cache

Install the necessary libraries for evaluating the quantized model's accuracy. Ensure you are using compatible versions.

```bash
pip install "vllm>=0.5.5" lm_eval==0.4.3

```

--------------------------------

### Install LLM Compressor from Source

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/getting-started/install

Install the latest development version of LLM Compressor directly from the main branch of its GitHub repository.

```bash
pip install git+https://github.com/vllm-project/llm-compressor.git
```

--------------------------------

### Install LLM Compressor from Local Clone

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/getting-started/install

Install LLM Compressor from a local clone of the repository. Navigate to the repository directory before running this command.

```bash
pip install .
```

--------------------------------

### Install llmcompressor

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_w8a8_fp8

Install the llmcompressor library using pip.

```bash
pip install llmcompressor

```

--------------------------------

### AWQModifier Example Recipe

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/transform/awq/base

This example shows how to configure the AWQModifier with mappings for smoothing and balancing layers. It uses regular expressions to match layer names.

```yaml
AWQModifier:
  mappings:
    - smooth_layer: "re:.*self_attn_layer_norm"
      balance_layers: ["re:.*q_proj", "re:.*k_proj", "re:.*v_proj"]
    - smooth_layer: "re:.*final_layer_norm"
      balance_layers: ["re:.*fc1"]
    # activation_hook_target specifies which submodule of the parent to hook
    # for activation caching.
    # This change is only useful for MoE models with parallel transformer blocks,
    # and one should use the default value (None) in most cases.

```

--------------------------------

### Install LLM Compressor from PyPI

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/getting-started/install

Install the latest stable release of LLM Compressor using pip.

```bash
pip install llmcompressor
```

--------------------------------

### IMatrixGatherer Recipe Example

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/transform/imatrix/base

Example of how to use IMatrixGatherer in a recipe with QuantizationModifier. It specifies modules to ignore and observer settings.

```yaml
recipe:
  - IMatrixGatherer:
      ignore: ["lm_head"]
  - QuantizationModifier:
      config_groups:
        group_0:
          targets: ["Linear"]
          weights:
            observer: imatrix_mse

```

--------------------------------

### on_initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/transform/awq

Starts the AWQ process on a given state before the quantization configuration is applied. It can infer mappings if not provided manually.

```APIDOC
## on_initialize

### Description
Start AWQ on the given state. This runs before quantization config has been applied. Infer unresolved mappings based on model architecture, if not set manually.

### Method Signature
`on_initialize(state: State, **kwargs) -> bool`

### Parameters
* **`state`** (`State`) - state to run AWQ on

### Returns
* `bool` - True on a successful run, False otherwise

### Source
`src/llmcompressor/modifiers/transform/awq/base.py`
```

--------------------------------

### Recommended SmoothQuantModifier Usage

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/logarithmic_equalization

This example demonstrates the recommended way to use logarithmic equalization by configuring SmoothQuantModifier with the 'log_equalization' algorithm.

```yaml
SmoothQuantModifier:
  algorithm: log_equalization
  mappings: [...]
```

--------------------------------

### SmoothQuantModifier Recipe Example

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/smoothquant/base

Illustrates how to configure the SmoothQuantModifier in a recipe, specifying smoothing strength, layer mappings using regex, and layers to ignore.

```yaml
SmoothQuantModifier:
  smoothing_strength: 0.5
  mappings: [
    [["re:.*q_proj", "re:.*k_proj", "re:.*v_proj"], "re:.*self_attn_layer_norm"],
    [["re:.*fc1"], "re:.*final_layer_norm"]
  ]
  ignore: ["model.decoder.final_layer_norm"]

```

--------------------------------

### on_start

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/modifier

Called when the modifier starts. This method must be implemented by inheriting modifiers to define initial logic.

```APIDOC
## on_start

### Description
Called when the modifier starts. This method must be implemented by inheriting modifiers.

### Parameters
* **`state`**(`State`) – The current state of the model
* **`event`**(`Event`) – The event that triggered the start
* **`kwargs`**– Additional arguments for starting the modifier
```

--------------------------------

### on_start

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/quantization/quantization

Begin calibrating activations.

```APIDOC
## on_start

### Description
Begin calibrating activations.

### Method
on_start(state: State, event: Event, **kwargs)

### Source Code
`src/llmcompressor/modifiers/quantization/quantization/base.py`
```

--------------------------------

### Invoke Calibration Epoch Start Event

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/session_functions

Call this class method before calibration starts for an epoch to invoke the epoch start event for the active session. See `src/llmcompressor/pipelines/basic/pipeline.py` for usage examples.

```python
@classmethod
def calibration_epoch_start(cls, **kwargs) -> ModifiedState:
    """
    Invoke a epoch start event for the active session during calibration. This event
    should be called before calibration starts for one epoch

    see `src/llmcompressor/pipelines/basic/pipeline.py` for usage example
    """
    return cls.event(EventType.CALIBRATION_EPOCH_START, **kwargs)
```

--------------------------------

### Install Specific Version of LLM Compressor from PyPI

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/getting-started/install

Install a specific version of LLM Compressor by specifying the version number during pip installation.

```bash
pip install llmcompressor==0.5.1
```

--------------------------------

### Initialize LLM Compressor Session

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/session

Use this method to set up the compression session. It accepts a recipe, model, teacher model, optimizer, and data for training, validation, testing, and calibration. Configure training steps and batches per step as needed.

```python
def initialize(
    self,
    recipe: str | list[str] | Recipe | list[Recipe] | None = None,
    recipe_stage: str | list[str] | None = None,
    recipe_args: dict[str, Any] | None = None,
    model: Any | None = None,
    teacher_model: Any | None = None,
    optimizer: Any | None = None,
    attach_optim_callbacks: bool = True,
    train_data: Any | None = None,
    val_data: Any | None = None,
    test_data: Any | None = None,
    calib_data: Any | None = None,
    copy_data: bool = True,
    start: float | None = None,
    steps_per_epoch: int | None = None,
    batches_per_step: int | None = None,
    **kwargs,
) -> ModifiedState:
    """
    Initialize the session for compression. This will run the initialize method
    for each modifier in the session's lifecycle. This will also set the session's
    state to the initialized state.

    :param recipe: the recipe to use for the compression, can be a path to a
        recipe file, a raw recipe string, a recipe object, or a list
        of recipe objects.
    :param recipe_stage: the stage to target for the compression
    :param recipe_args: the args to use for overriding the recipe defaults
    :param model: the model to compress
    :param teacher_model: the teacher model to use for knowledge distillation
    :param optimizer: the optimizer to use for the compression
    :param attach_optim_callbacks: True to attach the optimizer callbacks to the
        compression lifecycle, False otherwise
    :param train_data: the training data to use for the compression
    :param val_data: the validation data to use for the compression
    :param test_data: the testing data to use for the compression
    :param calib_data: the calibration data to use for the compression
    :param copy_data: True to copy the data, False otherwise
    :param start: the start epoch to use for the compression
    """
    pass
```

--------------------------------

### Install vLLM

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/steps/deploy

Install vLLM using pip. Ensure you are using Python 3.10 or newer.

```bash
pip install vllm

```

--------------------------------

### on_start

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/transform/awq

Starts the AWQ process on a given state after the quantization mixin has been initialized. It handles resolving and validating mappings, and setting up activation cache hooks.

```APIDOC
## on_start

### Description
Start AWQ on the given state. This runs after quantization mixin has been initialized (i.e. after quantization config has been applied)
  * resolve mappings
  * validate mappings and quant scheme
  * setup activation cache hooks

### Parameters
  * **`state`**(`State`) – 
state to run AWQ on

### Returns
  * – 
True on a successful run, False otherwise
```

--------------------------------

### Upgrade Pip

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/getting-started/install

Ensure you have the latest version of pip installed before proceeding with LLM Compressor installation.

```bash
python -m pip install --upgrade pip
```

--------------------------------

### start_calibration

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/quantization/quantization

Prepares the model for calibration by attaching observers, registering activation calibration hooks, and enabling quantization. This is a crucial step before the calibration data is processed.

```APIDOC
## start_calibration

### Description
Attach observers, register activation calibration hooks (including kv_cache quantization) and enable quantization as we calibrate.

### Method
```python
def start_calibration(self, model: torch.nn.Module):
    """
    Attach observers, register activation calibration hooks (including
    kv_cache quantization) and enable quantization as we calibrate

    :param model: model to prepare for calibration
    """
```

### Parameters
#### Path Parameters
- **model** (torch.nn.Module) - Required - The model to prepare for calibration.
```

--------------------------------

### WandaPruningModifier Configuration Example

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/pruning/wanda/base

Example YAML configuration for the WandaPruningModifier, specifying sparsity and mask structure.

```yaml
test_stage:
  sparsity_modifiers:
    WandaPruningModifier:
      sparsity: 0.5
      mask_structure: "2:4"

```

--------------------------------

### initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core

Initializes the compression process with specified parameters and configurations.

```APIDOC
## initialize

### Description
Initializes the compression process with specified parameters and configurations.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Method Signature
`initialize(recipe, recipe_stage, recipe_args, model, teacher_model, optimizer, attach_optim_callbacks, train_data, val_data, test_data, calib_data, copy_data, start, steps_per_epoch, batches_per_step, **kwargs)`

### Parameters
- **recipe**: The compression recipe to use.
- **recipe_stage**: The current stage of the recipe.
- **recipe_args**: Arguments for the recipe stage.
- **model**: The model to be compressed.
- **teacher_model**: The teacher model for knowledge distillation (if applicable).
- **optimizer**: The optimizer for training.
- **attach_optim_callbacks**: Whether to attach optimizer callbacks.
- **train_data**: Training dataset.
- **val_data**: Validation dataset.
- **test_data**: Test dataset.
- **calib_data**: Calibration dataset.
- **copy_data**: Whether to copy data.
- **start**: The starting point for the process.
- **steps_per_epoch** (int): The number of steps per epoch for compression.
- **batches_per_step** (int): The number of batches per step for compression.
- **kwargs**: Additional keyword arguments to pass to the lifecycle's initialize method.

### Returns
- **ModifiedState**: The modified state of the session after initializing.
```

--------------------------------

### initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/lifecycle

Initializes the compression lifecycle, setting up the recipe and modifiers. It can accept additional arguments to update the initial state.

```APIDOC
## initialize

### Description
Initializes the compression lifecycle. This method sets up the recipe and modifiers based on the provided inputs and can update the internal state with additional keyword arguments.

### Method
`initialize`

### Parameters
- **`recipe`** (RecipeInput | None) - Optional: The recipe configuration for compression.
- **`recipe_stage`** (RecipeStageInput | None) - Optional: The specific stage of the recipe to use.
- **`recipe_args`** (RecipeArgsInput | None) - Optional: Arguments to configure the recipe.
- **`**kwargs`** - Additional keyword arguments to update the state.

### Returns
- `List[Any]` - A list containing data returned from the initialization of modifiers.

### Raises
- `ValueError` - If the lifecycle has already been initialized (though the current implementation simply returns without raising an error in this case).

### Request Example
```python
compressor.initialize(recipe="path/to/recipe.json", recipe_args={"arg1": "value1"})
```

### Response Example
```json
[
  "initialization_data_modifier_1",
  "initialization_data_modifier_2"
]
```
```

--------------------------------

### LLM Compressor Recipe create_instance Method

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/recipe/recipe

The `create_instance` method initializes a Recipe object. It accepts a file path, a recipe string, or modifier objects. It handles loading from JSON, YAML, and can parse recipes embedded in Markdown files.

```python
@classmethod
def create_instance(
    cls,
    path_or_modifiers: Union[
        str, Modifier, list[Modifier | list[Modifier]], "Recipe"
    ],
    modifier_group_name: str | None = None,
    target_stage: str | None = None,
) -> "Recipe":
    """
    Create a recipe instance from a file, string, or RecipeModifier objects


    Using a recipe string or file is supported:
    >>> recipe_str = '''
    ... test_stage:
    ...     pruning_modifiers:
    ...         ConstantPruningModifier:
    ...             start: 0.0
    ...             end: 2.0
    ...             targets: ['re:.*weight']
    ... '''
    >>> recipe = Recipe.create_instance(recipe_str, target_stage="test_stage")

    :param path_or_modifiers: The path to the recipe file or
        or the recipe string (must be a valid
        json/yaml file or a valid json/yaml string). Can also
        accept a RecipeModifier instance, or a list of
        RecipeModifiers
    :param modifier_group_name: The stage_name of the recipe,
        if `oneshot` or `train` the run_type of the recipe will be
        inferred from the modifier_group_name, if None, a dummy default
        group_name will be assigned. This argument is only used
        when creating a recipe from a Modifier/list of Modifier(s)
        instance, else it's ignored.
    :return: The Recipe instance created from the path or modifiers,
        or a valid recipe string in yaml/json format
    """
    if isinstance(path_or_modifiers, Recipe):
        # already a recipe
        return path_or_modifiers

    if isinstance(path_or_modifiers, (Modifier, list)):
        return cls.from_modifiers(
            modifiers=path_or_modifiers, modifier_group_name=modifier_group_name
        )

    if not os.path.isfile(path_or_modifiers):
        # not a local file
        # assume it's a string
        logger.debug(
            "Could not initialize recipe as a file path or zoo stub, "
            "attempting to process as a string."
        )
        logger.debug(f"Input string: {path_or_modifiers}")
        obj = _load_json_or_yaml_string(path_or_modifiers)
        return cls.from_dict(filter_dict(obj, target_stage=target_stage))
    else:
        logger.info(f"Loading recipe from file {path_or_modifiers}")

    with open(path_or_modifiers, "r") as file:
        content = file.read().strip()
        if path_or_modifiers.lower().endswith(".md"):
            content = _parse_recipe_from_md(path_or_modifiers, content)

        if path_or_modifiers.lower().endswith(".json"):
            obj = json.loads(content)
        elif path_or_modifiers.lower().endswith(
            ".yaml"
        ) or path_or_modifiers.lower().endswith(".yml"):
            obj = yaml.safe_load(content)
        else:
            try:
                obj = _load_json_or_yaml_string(content)
            except ValueError:
                raise ValueError(
                    f"Could not parse recipe from path {path_or_modifiers}"
                )
        return cls.from_dict(filter_dict(obj, target_stage=target_stage))
```

--------------------------------

### IMatrixGatherer Recipe with GPTQModifier

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/transform/imatrix/base

Example of using IMatrixGatherer in a recipe with GPTQModifier. This configuration is similar to the QuantizationModifier example but tailored for GPTQ.

```yaml
recipe:
  - IMatrixGatherer:
      ignore: ["lm_head"]
  - GPTQModifier:
      config_groups:
        group_0:
          targets: ["Linear"]
          weights:
            observer: imatrix_mse

```

--------------------------------

### initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/session

Initializes the LLM Compressor session with specified training parameters and lifecycle configurations.

```APIDOC
## initialize

### Description
Initializes the LLM Compressor session, configuring training parameters and lifecycle hooks.

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **recipe** (any) - Required - The compression recipe to use.
- **recipe_stage** (any) - Required - The stage of the recipe.
- **recipe_args** (any) - Optional - Arguments for the recipe stage.
- **model** (any) - Required - The model to compress.
- **teacher_model** (any) - Optional - The teacher model for distillation.
- **optimizer** (any) - Optional - The optimizer to use.
- **attach_optim_callbacks** (bool) - Optional - Whether to attach optimizer callbacks.
- **train_data** (any) - Optional - Training dataset.
- **val_data** (any) - Optional - Validation dataset.
- **test_data** (any) - Optional - Test dataset.
- **calib_data** (any) - Optional - Calibration dataset.
- **copy_data** (bool) - Optional - Whether to copy data.
- **start** (bool) - Optional - Whether to start the process immediately.
- **steps_per_epoch** (int) - Required - The number of steps per epoch for compression.
- **batches_per_step** (int) - Required - The number of batches per step for compression.
- **kwargs** (any) - Optional - Additional keyword arguments.

### Request Example
```json
{
  "recipe": "my_recipe",
  "recipe_stage": "quantization",
  "model": "my_model",
  "steps_per_epoch": 100,
  "batches_per_step": 10
}
```

### Response
#### Success Response (200)
- **model** (any) - The compressed model.
- **optimizer** (any) - The optimizer state.
- **loss** (any) - The loss value.
- **modifier_data** (any) - Modified data from the lifecycle.

#### Response Example
```json
{
  "model": "compressed_model_object",
  "optimizer": "optimizer_state_object",
  "loss": 0.123,
  "modifier_data": "some_data"
}
```
```

--------------------------------

### Example Multimodal Interaction

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/multimodal_vision

This is an example of a multimodal interaction, likely for demonstration or testing purposes, showing a system's response to an image-based query.

```text
<|system|>
You are a helpful assistant.

<|user|>
Please describe the animal in this image

<|assistant|>
The animal in the image is a white kitten.
It has a fluffy coat and is resting on a white keyboard.
The kitten appears to be comfortable and relaxed, possibly enjoying the warmth of the keyboard.

```

--------------------------------

### Start vLLM HTTP Server

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/steps/deploy

Start the vLLM HTTP server to serve your model via an OpenAI-compatible RESTful API. The server defaults to localhost:8000.

```bash
vllm serve "Qwen3-30B-A3B-FP8-BLOCK"

```

--------------------------------

### start_calibration

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/autoround/base

Register activation calibration hooks and enable quantization as we calibrate.

```APIDOC
## start_calibration()

### Description
Register activation calibration hooks and enable quantization as we calibrate.

### Method
This is a method within the AutoRoundModifier class.
```

--------------------------------

### Start Quantization Calibration

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/quantization/quantization/base

Begins the calibration process for activations within the QuantizationModifier. Sets the internal started flag and initiates calibration via the QuantizationMixin.

```python
def on_start(self, state: State, event: Event, **kwargs):
    """
    Begin calibrating activations.
    """
    self.started_ = True
    QuantizationMixin.start_calibration(self, state.model)
```

--------------------------------

### on_initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/gptq/base

Initializes and runs the GPTQ algorithm on the current state. This method prepares the model for quantization by applying configuration and setting up calibration hooks.

```APIDOC
## on_initialize

### Description
Initialize and run the GPTQ algorithm on the current state.

### Parameters
#### Parameters
- **`state`** (`State`) - Required - session state storing input model and calibration data
- **`**kwargs`** - Optional - Additional keyword arguments

### Returns
- **`bool`** - True if initialization is successful.
```

--------------------------------

### on_initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/quantization/quantization

Prepare to calibrate activations and weights. According to the quantization config, a quantization scheme is attached to each targeted module. The module's forward call is also overwritten to perform quantization to inputs, weights, and outputs. Then, according to the module's quantization scheme, observers and calibration hooks are added. These hooks are disabled until the modifier starts.

```APIDOC
## on_initialize

### Description
Prepare to calibrate activations and weights

According to the quantization config, a quantization scheme is attached to each targeted module. The module's forward call is also overwritten to perform quantization to inputs, weights, and outputs.

Then, according to the module's quantization scheme, observers and calibration hooks are added. These hooks are disabled until the modifier starts.

### Method
on_initialize(state: State, **kwargs) -> bool

### Source Code
`src/llmcompressor/modifiers/quantization/quantization/base.py`
```

--------------------------------

### Modifier.on_start

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers

Callback function that is called when the modifier starts its operation, typically based on the current event index. Subclasses can implement this to define actions to be taken at the start.

```APIDOC
## on_start

### Description
on_start is called when the modifier starts and

### Method
`on_start(state: State, **kwargs)`

### Parameters
* **`state`** (`State`) – The current state of the model
* **`kwargs`** – Additional arguments for the on_start callback
```

--------------------------------

### Initialize SequentialQwen3_5MoeExperts

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modeling/qwen3_5_moe

Initializes a module that unfuses 3D expert parameter tensors into individual Qwen3.5 MLP modules. This allows each expert's weights to be targeted by quantization as nn.Linear layers.

```python
SequentialQwen3_5MoeExperts(config, original)

```

```python
def __init__(self, config, original):
    from transformers.models.qwen3_5_moe.modeling_qwen3_5_moe import (
        Qwen3_5MoeMLP,
    )

    self.num_experts = config.num_experts
    intermediate_size = config.moe_intermediate_size

    with skip_weights_initialize():
        super().__init__(
            [
                Qwen3_5MoeMLP(config, intermediate_size=intermediate_size)
                for _ in range(self.num_experts)
            ]
        )

    gate_up_data = original.gate_up_proj.data  # [num_experts, 2*inter, hidden]
    down_data = original.down_proj.data  # [num_experts, hidden, inter]

    for i in range(self.num_experts):
        gate_up = gate_up_data[i]  # [2*intermediate, hidden]
        down = down_data[i]  # [hidden, intermediate]

        # gate_up_proj stores [gate; up] stacked along dim 0
        # nn.Linear weight is [out_features, in_features]
        self[i].gate_proj.weight.data = (
            gate_up[:intermediate_size, :].clone().contiguous()
        )
        self[i].up_proj.weight.data = (
            gate_up[intermediate_size:, :].clone().contiguous()
        )
        self[i].down_proj.weight.data = down.clone().contiguous()

```

--------------------------------

### Implement Modifier Start Logic

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/modifier

The `on_start` method is called when a modifier begins its operation. Inheriting modifiers should implement this method to define actions to be performed at the start of the modifier's lifecycle.

```python
def on_start(self, state: State, event: Event, **kwargs):
    """
    on_start is called when the modifier starts and
    must be implemented by the inheriting modifier.

    :param state: The current state of the model
    :param event: The event that triggered the start
    :param kwargs: Additional arguments for starting the modifier
    """
    pass
```

--------------------------------

### Determine if Modifier Should Start - should_start

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/modifier

Implement this method to define the condition for initiating the modifier's operation. It checks if the current event index falls within the specified start and end range.

```python
def should_start(self, event: Event) -> bool:
    """
    :param event: The event to check if the modifier should start
    :return: True if the modifier should start based on the given event
    """
    if self.start is None:
        return False

    current = event.current_index

    return self.start <= current and (self.end is None or current < self.end)
```

--------------------------------

### CompressionLifecycle.initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core

Initializes the compression lifecycle with optional recipe, stage, and arguments. It prepares the lifecycle by setting up the recipe and initializing all associated modifiers.

```APIDOC
## initialize

### Description
Initialize the compression lifecycle. This method prepares the lifecycle by setting up the recipe and initializing all associated modifiers.

### Method Signature
`initialize(recipe: RecipeInput | None = None, recipe_stage: RecipeStageInput | None = None, recipe_args: RecipeArgsInput | None = None, **kwargs) -> list[Any]`

### Parameters
* **`recipe`** (RecipeInput | None) - Optional: The recipe to use for initialization.
* **`recipe_stage`** (RecipeStageInput | None) - Optional: The specific stage of the recipe to initialize.
* **`recipe_args`** (RecipeArgsInput | None) - Optional: Arguments to be used for the recipe.
* **`**kwargs`** - Additional keyword arguments to update the state with.

### Returns
* `List[Any]` - A list of data returned from the initialization of modifiers.
```

--------------------------------

### Invoke Batch Start Event

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/session_functions

Invoke a batch start event for the active session. This method accepts optional batch data and additional keyword arguments for the session's event method.

```python
@classmethod
def batch_start(cls, batch_data: Optional[Any] = None, **kwargs) -> ModifiedState:
    """
    Invoke a batch start event for the active session

    :param batch_data: the batch data to use for the event
    :param kwargs: additional kwargs to pass to the current session's event method
    :return: the modified state of the active session after invoking the event
    """
    return cls.event(EventType.BATCH_START, batch_data=batch_data, **kwargs)
```

--------------------------------

### Install LLM Compressor from Local Clone (Editable Mode)

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/getting-started/install

For development purposes, install LLM Compressor in editable mode from a local clone. This allows changes to the source code to be reflected immediately without reinstallation.

```bash
pip install -e .[dev]
```

--------------------------------

### Basic Oneshot Quantization Usage

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/guides/entrypoints/oneshot

Demonstrates loading a model and tokenizer from Hugging Face and applying a quantization recipe using the `oneshot` entrypoint. This example uses FP8 dynamic quantization for linear layers, ignoring the language modeling head.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct", dtype="auto")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")

oneshot(
    model=model,
    recipe=QuantizationModifier(targets="Linear", scheme="FP8_DYNAMIC", ignore=["lm_head"]),
    output_dir="Meta-Llama-3-8B-Instruct-FP8",
)


```

--------------------------------

### get_no_split_params

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/utils/pytorch

Get list of module classes that shouldn't be split when sharding.

```APIDOC
## get_no_split_params

### Description
Get list of module classes that shouldn't be split when sharding.

### Method
`get_no_split_params`

### Parameters
No parameters documented.
```

--------------------------------

### Initialize GPTQ Algorithm

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/gptq

Initializes and runs the GPTQ algorithm. It applies configuration to the model and prepares calibration hooks. Use this when setting up GPTQ quantization for a model.

```python
def on_initialize(self, state: State, **kwargs) -> bool:
    """
    Initialize and run the GPTQ algorithm on the current state

    :param state: session state storing input model and calibration data
    """
    # apply config to model and prepare calibration hooks
    if QuantizationMixin.has_config(self):
        QuantizationMixin.initialize_quantization(self, state.model)

    # prepare module names
    self._module_names = {
        m: name
        for name, m in match_named_modules(
            state.model, self.resolved_targets, self.ignore
        )
    }

    return True
```

--------------------------------

### Initialize Gemma4TextExpertsList

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modeling/gemma4

Initializes Gemma4TextExpertsList by unpacking 3D expert parameter tensors into individual Gemma4TextMLP modules. This prepares the experts for quantization by ensuring their weights are nn.Linear.

```python
Gemma4TextExpertsList(
    config: Gemma4TextConfig, original: Gemma4TextExperts
)

def __init__(self, config: Gemma4TextConfig, original: Gemma4TextExperts):
    from transformers.models.gemma4.modeling_gemma4 import Gemma4TextMLP

    self.num_experts = config.num_experts
    intermediate_size = config.moe_intermediate_size

    with skip_weights_initialize():
        super().__init__(
            [Gemma4TextMLP(config, layer_idx=0) for _ in range(self.num_experts)]
        )

    gate_up_data = original.gate_up_proj.data  # [num_experts, 2*inter, hidden]
    down_data = original.down_proj.data  # [num_experts, hidden, inter]

    for i in range(self.num_experts):
        gate_up = gate_up_data[i]  # [2*intermediate, hidden]
        down = down_data[i]  # [hidden, intermediate]

        # gate_up_proj stores [gate; up] stacked along dim 0
        # nn.Linear weight is [out_features, in_features]
        self[i].gate_proj.weight.data = (
            gate_up[:intermediate_size, :].clone().contiguous()
        )
        self[i].up_proj.weight.data = (
            gate_up[intermediate_size:, :].clone().contiguous()
        )
        self[i].down_proj.weight.data = down.clone().contiguous()
```

--------------------------------

### Load Model and Processor

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/key-models/qwen3.5/nvfp4-moe-example

Loads the Qwen3.5-122B-A10B model and its corresponding processor. Ensure `transformers` is installed.

```python
import torch
from compressed_tensors.utils import save_mtp_tensors_to_checkpoint
from datasets import load_dataset
from transformers import AutoProcessor, Qwen3_5MoeForConditionalGeneration

from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

MODEL_ID = "Qwen/Qwen3.5-122B-A10B"

# Load model.
model = Qwen3_5MoeForConditionalGeneration.from_pretrained(MODEL_ID, dtype="auto")
processor = AutoProcessor.from_pretrained(MODEL_ID)
```

--------------------------------

### on_initialize

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/modifiers/autoround/base

Initialize the model state for quantization and calibration.

```APIDOC
## on_initialize()

### Description
Initialize the model state for quantization and calibration.

### Method
This is a method within the AutoRoundModifier class.
```

--------------------------------

### reset

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/session

Resets the session to its initial state. This is useful for clearing all accumulated state and starting fresh.

```APIDOC
## reset

### Description
Reset the session to its initial state.

### Method
```python
def reset(self):
```

### Parameters
This method does not accept any parameters.

### Returns
None
```

--------------------------------

### Apply W8A8 Quantization Recipe

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/examples/quantization_w8a8_int8

Configure and apply a quantization recipe using `llm-compressor`'s `oneshot` function. This example uses `SmoothQuantModifier` for activations and `GPTQModifier` for weights with `W8A8` scheme.

```python
from llmcompressor import oneshot
from llmcompressor.modifiers.gptq import GPTQModifier
from llmcompressor.modifiers.transform.smoothquant import SmoothQuantModifier

# Configure the quantization algorithms to run.
recipe = [
    SmoothQuantModifier(smoothing_strength=0.8),
    GPTQModifier(targets="Linear", scheme="W8A8", ignore=["lm_head"]),
]

# Apply quantization.
oneshot(
    model=model,
    dataset=ds,
    recipe=recipe,
    max_seq_length=MAX_SEQUENCE_LENGTH,
    num_calibration_samples=NUM_CALIBRATION_SAMPLES,
)

# Save to disk compressed.
SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-W8A8-Dynamic-Per-Token"
model.save_pretrained(SAVE_DIR, save_compressed=True)
tokenizer.save_pretrained(SAVE_DIR)


```

--------------------------------

### CompressionSession.reset_stage

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/session

Reset the compression session for starting a new stage, while keeping the recipe and model intact.

```APIDOC
## reset_stage

### Description
Reset the session for starting a new stage, recipe and model stays intact.

### Method
`reset_stage()`

### Endpoint
N/A (Method call)

### Parameters
None

### Request Example
```python
session.reset_stage()
```

### Response
No specific response documented.
```

--------------------------------

### Event.should_update

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core/events

Determines if the event should trigger an update based on start, end, and update interval parameters.

```APIDOC
## should_update

### Description
Determines if the event should trigger an update.

### Parameters
* **`start`** (`Optional[float]`) – The start index to check against, set to None to ignore start.
* **`end`** (`Optional[float]`) – The end index to check against, set to None to ignore end.
* **`update`** (`Optional[float]`) – The update interval, set to None or 0.0 to always update, otherwise must be greater than 0.0, defaults to None.

### Returns
* `bool` – True if the event should trigger an update, False otherwise.
```

--------------------------------

### Initialize LLM Compressor Session

Source: https://docs.vllm.ai/projects/llm-compressor/en/latest/api/llmcompressor/core

Use this function to set up the compression session. It processes recipes and configures the model, data, and training parameters for compression.

```python
def initialize(
    self,
    recipe: str | list[str] | Recipe | list[Recipe] | None = None,
    recipe_stage: str | list[str] | None = None,
    recipe_args: dict[str, Any] | None = None,
    model: Any | None = None,
    teacher_model: Any | None = None,
    optimizer: Any | None = None,
    attach_optim_callbacks: bool = True,
    train_data: Any | None = None,
    val_data: Any | None = None,
    test_data: Any | None = None,
    calib_data: Any | None = None,
    copy_data: bool = True,
    start: float | None = None,
    steps_per_epoch: int | None = None,
    batches_per_step: int | None = None,
    **kwargs,
) -> ModifiedState:
    """
    Initialize the session for compression. This will run the initialize method
    for each modifier in the session's lifecycle. This will also set the session's
    state to the initialized state.

    :param recipe: the recipe to use for the compression, can be a path to a
        recipe file, a raw recipe string, a recipe object, or a list
        of recipe objects.
    :param recipe_stage: the stage to target for the compression
    :param recipe_args: the args to use for overriding the recipe defaults
    :param model: the model to compress
    :param teacher_model: the teacher model to use for knowledge distillation
    :param optimizer: the optimizer to use for the compression
    :param attach_optim_callbacks: True to attach the optimizer callbacks to the
        compression lifecycle, False otherwise
    :param train_data: the training data to use for the compression
    :param val_data: the validation data to use for the compression
    :param test_data: the testing data to use for the compression
    :param calib_data: the calibration data to use for the compression
    :param copy_data: True to copy the data, False otherwise
    :param start: the start epoch to use for the compression
    """
    pass

```