### Install and Run `olive init`

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-init-cli.md

Install the olive-ai package and launch the interactive wizard to start optimizing your model.

```bash
pip install olive-ai
olive init
```

--------------------------------

### Configure Native Docker System

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/configure-workflows/systems.md

Example configuration for a native Docker system. Ensure Docker Engine and necessary dependencies are installed.

```json
{
    "type": "Docker",
    "image_name": "olive",
    "build_context_path": "docker",
    "dockerfile": "Dockerfile"
    "accelerators": [
        {
            "device": "cpu",
            "execution_providers": ["CPUExecutionProvider"]
        }
    ]
}
```

--------------------------------

### Optimize HuggingFace LLM for CPU with INT4

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-init-cli.md

This example demonstrates generating a CLI command to optimize a HuggingFace LLM for CPU using INT4 precision. The wizard guides through model selection, optimization type, target device, and precision.

```bash
$ olive init

Welcome to Olive Init! This wizard will help you optimize your model.

? What type of model do you want to optimize? PyTorch (HuggingFace or local)
? How would you like to specify your model? HuggingFace model name
? Model name or path: Qwen/Qwen2.5-0.5B-Instruct
? What do you want to do? Optimize model (export to ONNX + quantize + graph optimize)
? How would you like to configure optimization? Auto Mode (recommended)
? Select target device: CPU
? Select target precision: INT4 (smallest size, best for LLMs)
? Output directory: ./olive-output
? What would you like to do? Generate CLI command (copy and run later)

Generated command:

  olive optimize -m Qwen/Qwen2.5-0.5B-Instruct --provider CPUExecutionProvider --precision int4 -o ./olive-output
```

--------------------------------

### Clone Olive Repository and Install Dependencies

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-mcp-server.md

Clone the Olive repository and install the necessary Python dependencies using uv. This is the initial setup step to get started with the MCP server.

```bash
git clone https://github.com/microsoft/Olive.git
cd Olive/mcp && uv sync
```

--------------------------------

### Install Documentation Requirements

Source: https://github.com/microsoft/olive/blob/main/docs/README.md

Install the necessary pip requirements for building the documentation from the 'docs' directory.

```bash
pip install -r requirements.txt
```

--------------------------------

### Install Olive with OpenVINO Extras

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/ihv-integration/openvino.md

Use this command to install Olive with OpenVINO support. This is the recommended approach for most users.

```bash
pip install olive-ai[openvino]
```

--------------------------------

### Install Olive from Source (DirectML)

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/installation.md

Install the latest development version of Olive from its GitHub repository with support for onnxruntime-directml.

```bash
pip install git+https://github.com/microsoft/Olive#egg=olive-ai[directml]
```

--------------------------------

### Install Olive MCP Server

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-mcp-server.md

Clone the Olive repository and install dependencies using uv.

```bash
git clone https://github.com/microsoft/Olive.git
cd Olive/mcp
uv sync
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/microsoft/olive/blob/main/notebooks/olive-deepseek-finetune.ipynb

Creates a requirements.txt file and installs necessary Python libraries for the project.

```python
%%writefile requirements.txt

olive-ai
onnxruntime-genai==0.6.0
transformers==4.44.2
optimum==1.23.1
bitsandbytes==0.45.1
peft==0.13.2
accelerate>=0.30.0
scipy==1.14.1
torchvision==0.18.1
tabulate==0.9.0
```

```python
%%capture

%pip install -r requirements.txt
```

--------------------------------

### Initialize Lintrunner

Source: https://github.com/microsoft/olive/blob/main/CONTRIBUTING.md

Installs lintrunner and its dependencies. Use `--dry-run` to see what will be installed without actually installing.

```sh
lintrunner init
```

```sh
lintrunner init --dry-run
```

--------------------------------

### Install Olive from Source (Default CPU)

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/installation.md

Install the latest development version of Olive from its GitHub repository, including default CPU support for onnxruntime.

```bash
pip install git+https://github.com/microsoft/Olive#egg=olive-ai[cpu]
```

--------------------------------

### Install Olive with pip (CPU)

Source: https://github.com/microsoft/olive/blob/main/docs/source/getting-started/getting-started.md

Install Olive with CPU support and the transformers library. Recommended for machines without a GPU.

```bash
pip install olive-ai[cpu]
pip install transformers
```

--------------------------------

### Install Olive from Source (GPU)

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/installation.md

Install the latest development version of Olive from its GitHub repository with support for onnxruntime-gpu.

```bash
pip install git+https://github.com/microsoft/Olive#egg=olive-ai[gpu]
```

--------------------------------

### Install OpenVINO Runtime and Development Tools

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/ihv-integration/openvino.md

Alternatively, install OpenVINO Runtime and NNCF separately from PyPI. Ensure compatible versions are selected.

```bash
pip install openvino>=2025.4.1
pip install nncf>=2.19.0
pip install onnxruntime-openvino
```

--------------------------------

### Quickstart: Optimize Model for CPU

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/cli/cli-optimize.md

This command downloads a model from Hugging Face, quantizes its weights to int4, converts it to ONNX, and optimizes the ONNX graph for CPU execution.

```bash
olive optimize \
    --model_name_or_path microsoft/Phi-3.5-mini-instruct \
    --device cpu \
    --provider CPUExecutionProvider \
    --precision int4 \

```

--------------------------------

### Preview Documentation Locally

Source: https://github.com/microsoft/olive/blob/main/docs/README.md

Start a local HTTP server in the 'docs/build/html' directory to preview the documentation site. Replace {port-number} with your desired port.

```bash
python -m http.server {port-number}
```

--------------------------------

### Install Olive

Source: https://github.com/microsoft/olive/blob/main/docs/README.md

Install the Olive package from the root of the code repository.

```bash
pip install -e .
```

--------------------------------

### LoftQ Configuration Example

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/peft-adapters.md

Configuration for the LoftQ pass, which initializes quantized LoRA models. Requires a GPU.

```json
{
    "type": "LoftQ",
    "compute_dtype": "bfloat16",
    "training_args": {
        "learning_rate": 0.0002,
        // ...
    },
    "train_data_config": // ...,
}
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/microsoft/olive/blob/main/notebooks/olive-awq-ft-llama.ipynb

Creates a requirements.txt file and installs necessary Python libraries for Olive, transformers, AWQ, and ONNX Runtime.

```python
%%writefile requirements.txt

olive-ai==0.7.1
transformers==4.44.2
autoawq==0.2.6
optimum==1.23.1
peft==0.13.2
accelerate>=0.30.0
scipy==1.14.1
onnxruntime-genai==0.6.0
torchvision==0.18.1
tabulate==0.9.0
```

```python
%%capture

%pip install -r requirements.txt
```

--------------------------------

### Example Prompts for Olive MCP

Source: https://github.com/microsoft/olive/blob/main/mcp/README.md

Illustrative prompts for various model optimization tasks using the Olive MCP server.

```text
Optimize microsoft/Phi-3-mini-4k-instruct

Quantize microsoft/Phi-3-mini-4k-instruct

Fine-tune microsoft/Phi-3-mini-4k-instruct on nampdn-ai/tiny-codes

Capture ONNX graph from microsoft/Phi-3-mini-4k-instruct

Benchmark microsoft/Phi-3-mini-4k-instruct

Train a LoRA for runwayml/stable-diffusion-v1-5 with dataset linoyts/Tuxemon

What's the best way to optimize Phi-4-mini for my hardware?

What passes are available for int4 quantization?

Help me write a custom Olive config with OnnxQuantization and GraphSurgeries
```

--------------------------------

### Install Test Dependencies

Source: https://github.com/microsoft/olive/blob/main/CONTRIBUTING.md

Installs the necessary Python packages for running tests.

```sh
pip install -r requirements-test.txt
```

--------------------------------

### QLoRA Configuration Example

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/peft-adapters.md

Configuration for the QLoRA pass, including compute dtype, quantization type, training arguments, and training data configuration. Requires a GPU.

```json
{
    "type": "QLoRA",
    "compute_dtype": "bfloat16",
    "quant_type": "nf4",
    "training_args": {
        "learning_rate": 0.0002,
        // ...
    },
    "train_data_config": // ...,
}
```

--------------------------------

### Install Olive and Dependencies

Source: https://github.com/microsoft/olive/blob/main/notebooks/olive_quickstart.ipynb

Installs the Olive CLI with automatic optimization capabilities and necessary libraries for model optimization and inference.

```python
%%capture

%pip install olive-ai[auto-opt]
%pip install transformers onnxruntime-genai
```

--------------------------------

### Install Python Dependencies

Source: https://github.com/microsoft/olive/blob/main/notebooks/text-gen-optimized-slms.ipynb

Installs the necessary Olive and ONNX Runtime packages. Use %%capture to suppress output.

```python
%%capture

%pip install olive-ai[auto-opt]
%pip install transformers==4.44.2 onnxruntime-genai
```

--------------------------------

### Install Optimum Intel for Generative AI

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/ihv-integration/openvino.md

Install the Optimum Intel package for generative AI workloads. Ensure you are using a compatible version.

```bash
pip install optimum[openvino]>=2.1.0
```

--------------------------------

### Install Required Packages

Source: https://github.com/microsoft/olive/blob/main/notebooks/llama2_multilora.ipynb

Installs necessary Python packages for the QLoRA fine-tuning process, including Olive dependencies and ONNX Runtime with GPU support.

```python
# install required packages
!pip install -r requirements-qlora.txt
!pip install ipywidgets tabulate

# install onnxruntime-genai-cuda
!pip uninstall -y onnxruntime-genai onnxruntime-genai-cuda
!pip install --pre onnxruntime-genai-cuda --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-genai/pypi/simple/

# install onnxruntime-gpu >1.20
!pip uninstall -y onnxruntime onnxruntime-gpu
!pip install "onnxruntime-gpu>=1.20"
```

--------------------------------

### Install Olive with pip (GPU)

Source: https://github.com/microsoft/olive/blob/main/docs/source/getting-started/getting-started.md

Install Olive with GPU support and the transformers library. Use this if your machine has a CUDA-enabled GPU.

```bash
pip install olive-ai[gpu]
pip install transformers
```

--------------------------------

### Training Data Structure Example

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/sd-lora.md

Organize your training images and their corresponding caption files in a structured directory. Each image should have a text file with the same name containing its caption.

```text
train_images/
├── image1.png
├── image1.txt    # Contains: "a photo of sks dog"
├── image2.jpg
├── image2.txt    # Contains: "sks dog playing in the park"
└── ...
```

--------------------------------

### Install Olive CLI and Dependencies

Source: https://github.com/microsoft/olive/blob/main/README.md

Install the Olive CLI and necessary libraries for model optimization. It's recommended to use a virtual or conda environment.

```bash
pip install olive-ai
pip install transformers onnxruntime-genai
```

--------------------------------

### Install Olive AI with SD-LoRA support

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/sd-lora.md

Install the Olive library with the specific extras for Stable Diffusion LoRA integration. This command ensures all necessary dependencies are included.

```bash
pip install olive-ai[sd-lora]
```

--------------------------------

### Example Graph Surgeries Configuration

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/onnx-transformations.md

Configuration for applying multiple graph surgeries sequentially.

```json
"surgeries": {
    "type": "GraphSurgeries",
    "surgeries": [
        {
            "surgeon": "RenameInputs",
            "old_names": ["input1", "input2"],
            "new_names": ["renamed_input1", "renamed_input2"]
        },
        {
            "surgeon": "RenameOutputs",
            "old_names": ["output1", "output2"],
            "new_names": ["renamed_output1", "renamed_output2"]
        },
        {
            "surgeon": "InferShapes"
        }
    ]
}
```

--------------------------------

### Accessing API Results Example

Source: https://github.com/microsoft/olive/blob/main/docs/source/reference/python_api.md

Demonstrates how to use the `run` function and access results using the `WorkflowOutput` object.

```APIDOC
## Accessing API Results

Most API functions return a `WorkflowOutput` object. You can use it to access the optimized models and their metrics.

```python
from olive import run, WorkflowOutput

workflow_output: WorkflowOutput = run("config.json")

# Check if optimization produced any results
if workflow_output.has_output_model():
    # Get the best model overall
    best_model = workflow_output.get_best_candidate()
    print(f"Model path: {best_model.model_path}")
    print(f"Model type: {best_model.model_type}")
    print(f"Device: {best_model.from_device()}")
    print(f"Execution provider: {best_model.from_execution_provider()}")
    print(f"Metrics: {best_model.metrics_value}")

```
```

--------------------------------

### Example Peephole Optimizer Configuration

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/onnx-transformations.md

Configuration for the OnnxPeepholeOptimizer.

```json
{
    "type": "OnnxPeepholeOptimizer"
}
```

--------------------------------

### Configure OnnxQuantization Pass

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/configure-workflows/pass-configuration.md

Example of configuring the OnnxQuantization Pass. Demonstrates setting parameters to default, searchable, or specific user-defined values.

```json
{
    "type": "OnnxQuantization",
    "disable_search": false,
    "data_config": "calib_data_config",
    // set per_channel to "DEFAULT_VALUE"
    "per_channel": "DEFAULT_VALUE",
    // set reduce_range to "SEARCHABLE_VALUES" value
    // redundant since disable_search is false
    "reduce_range": "SEARCHABLE_VALUES",
    // user defined value for precision
    "precision": "uint8"
}
```

--------------------------------

### Other Clients Configuration

Source: https://github.com/microsoft/olive/blob/main/mcp/README.md

Example configuration for clients like Claude Desktop, Claude Code, Cursor, and Windsurf to connect to the Olive MCP server.

```json
{
  "mcpServers": {
    "olive": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/Olive/mcp", "python", "-m", "olive_mcp"]
    }
  }
}
```

--------------------------------

### Configure Model with Multiple Adapters

Source: https://github.com/microsoft/olive/blob/main/notebooks/llama2_multilora.ipynb

Defines the base model path and configurations for multiple adapters. This setup allows loading different adapter weights independently for various tasks.

```python
base_model_name = "meta-llama/llama-2-7b-hf"
model_path = "models/tiny-codes/extracted/model/model.onnx"
adapters = {
    "guanaco": {
        "weights": "models/exported/guanaco_qlora.onnx_adapter",
        "template": "### Human: {prompt} ### Assistant:"
    },
    "tiny-codes": {
        "weights": "models/tiny-codes/extracted/model/adapter_weights.onnx_adapter",
        "template": "### Language: {prompt_0} 
### Question: {prompt_1} 
### Answer: "
    }
}
```

--------------------------------

### Install Shared Cache Dependencies

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/shared-model-cache.md

Install the necessary dependencies to enable shared cache support in Olive AI. This command installs the core Olive package along with the shared cache extras.

```shell
pip install olive-ai[shared-cache]
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/microsoft/olive/blob/main/CONTRIBUTING.md

Installs the Python packages required for development, including linting tools.

```sh
pip install -r requirements-dev.txt
```

--------------------------------

### Install Olive with pip

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/installation.md

Use this command to install the latest stable version of Olive from the Python Package Index.

```bash
pip install olive-ai
```

--------------------------------

### Build HTML Documentation

Source: https://github.com/microsoft/olive/blob/main/docs/README.md

Generate the HTML version of the documentation from the 'docs' directory.

```bash
make html
```

--------------------------------

### Example: Adding a New Custom Task IO Config

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/extending/how-to-add-new-task.md

Illustrates how to add a new entry to `tasks.yaml` for a custom task, defining its inputs and outputs with shapes, dynamic axes, and data types.

```yaml
# Custom task for a new model type
my-custom-task:
  inputs:
    input_ids:
      shape: [batch_size, sequence_length]
      axes: {0: batch_size, 1: sequence_length}
      dtype: int64
      max_value: vocab_size
    attention_mask:
      shape: [batch_size, sequence_length]
      axes: {0: batch_size, 1: sequence_length}
      dtype: int64
    custom_input:
      shape: [batch_size, custom_dim]
      axes: {0: batch_size, 1: custom_dim}
      dtype: float
      optional: true
  outputs:
    logits:
      axes: {0: batch_size, 1: sequence_length, 2: vocab_size}
    custom_output:
      axes: {0: batch_size, 1: hidden_size}
```

--------------------------------

### View Olive Convert Adapters Help

Source: https://github.com/microsoft/olive/blob/main/notebooks/llama2_multilora.ipynb

Displays the available options for the 'convert-adapters' command, which is used to export fine-tuned adapters from a pre-existing repository.

```bash
# run this cell to see the available options to convert-adapters command
!olive convert-adapters --help
```

--------------------------------

### Install Transformers for Auto-Captioning

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/sd-lora.md

Install or update the transformers library to the specified version for utilizing auto-captioning features with models like BLIP-2 and Florence-2.

```bash
pip install transformers>=4.30.0  # For BLIP-2 and Florence-2
```

--------------------------------

### View Olive Command Help

Source: https://github.com/microsoft/olive/blob/main/notebooks/llama2_multilora.ipynb

Displays the available options and usage instructions for Olive's finetune, capture-onnx-graph, and generate-adapter commands.

```bash
# run this cell to see the available options to finetune, capture-onnx-graph and generate-adapter commands
!olive finetune --help
!olive capture-onnx-graph --help
!olive generate-adapter --help
```

--------------------------------

### Editable Install Olive from Source

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/installation.md

Install Olive in editable mode for development. This allows you to test your code changes directly without reinstalling.

```bash
git clone https://github.com/microsoft/Olive
cd Olive
pip install -e .
```

--------------------------------

### Train Diffusion LoRA with DreamBooth using Olive CLI

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-init-cli.md

This example demonstrates generating a CLI command for training a Diffusion LoRA model with DreamBooth. It includes options for specifying the diffuser model, training data, DreamBooth parameters, and output settings.

```bash
$ olive init

? What type of model do you want to optimize? Diffusers (Stable Diffusion, SDXL, Flux, etc.)
? Select diffuser model variant: Stable Diffusion XL (SDXL)
? Enter model name or path: stabilityai/stable-diffusion-xl-base-1.0
? What do you want to do? LoRA Training (fine-tune on custom images)
? LoRA rank (r): 16 (recommended)
? LoRA alpha: 16
? Training data source: Local image folder
? Path to image folder: ./my-dog-photos
? Enable DreamBooth training? Yes
? Instance prompt: a photo of sks dog
? Enable prior preservation? Yes
? Class prompt: a photo of a dog
? Max training steps: 1000 (recommended)
? Output directory: ./olive-output
? What would you like to do? Generate CLI command (copy and run later)

Generated command:

  olive diffusion-lora -m stabilityai/stable-diffusion-xl-base-1.0 --model_variant sdxl -r 16 --alpha 16 --lora_dropout 0.0 -d ./my-dog-photos --dreambooth --instance_prompt "a photo of sks dog" --with_prior_preservation --class_prompt "a photo of a dog" --num_class_images 200 --max_train_steps 1000 --learning_rate 1e-4 --train_batch_size 1 --gradient_accumulation_steps 4 --mixed_precision bf16 --lr_scheduler constant --lr_warmup_steps 0 -o ./olive-output
```

--------------------------------

### Implement _run_for_config Method

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/extending/how-to-add-optimization-pass.md

Implement this method to optimize the input model. It receives a search point and output path, and should return a valid OliveModelHandler.

```python
    def _run_for_config(self, model: ONNXModelHandler, config: Dict[str, Any], output_model_path: str) -> ONNXModelHandler:
```

--------------------------------

### Install Dependencies for Olive AI

Source: https://github.com/microsoft/olive/blob/main/notebooks/sd_multilora/sd_multilora.ipynb

Installs necessary packages including diffusers and onnxruntime-gpu. Ensure onnxruntime-gpu is at version 1.20 or higher for CUDA acceleration.

```python
# install required packages
!pip install -r diffusers

# install onnxruntime-gpu >=1.20
!pip uninstall -y onnxruntime onnxruntime-gpu
!pip install "onnxruntime-gpu>=1.20"
```

--------------------------------

### Quantize, Finetune, and Convert to ONNX

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/cli/cli-quantize.md

This sequence demonstrates quantizing a model with AWQ, finetuning it using LoRA, and preparing it for ONNX runtime. Ensure to update model paths and dataset names as needed.

```bash
# Step 1: AWQ (will output a PyTorch model)
olive quantize \
   --model_name_or_path meta-llama/Llama-3.2-1B-Instruct \
   --trust_remote_code \
   --algorithm awq \
   --output_path models/llama/awq \
   --log_level 1

# Step 2: Finetune (will output a PEFT adapter)
olive finetune \
    --method lora \
    --model_name_or_path models/llama/awq \
    --data_name xxyyzzz/phrase_classification \
    --text_template "<|start_header_id|>user<|end_header_id|>\n{phrase}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n{tone}" \
    --max_steps 100 \
    --output_path ./models/llama/ft \
    --log_level 1
```

--------------------------------

### Prepare Base Model for Multiple Adapters

Source: https://github.com/microsoft/olive/blob/main/notebooks/sd_multilora/sd_multilora.ipynb

Copy the base ONNX model files to a new directory and define a dictionary mapping adapter names to their weight file paths.

```python
import shutil

file_list = ["model.onnx", "model.onnx.data"]
old_model_path = Path("adapters/pen_sketch/model")
base_model_path = Path("model")

base_model_path.parent.mkdir(parents=True, exist_ok=True)

# copy base onnx model to new folder
for file in file_list:
    shutil.copy2(old_model_path / file, base_model_path / file)

base_model_path = base_model_path / "model.onnx"

adapters = {
    "pen_sketch":  "adapters/pen_sketch/model/adapter_weights.onnx_adapter",
    "wolf_plushie": "adapters/wolf_plushie/model/adapter_weights.onnx_adapter"
}
```

--------------------------------

### View Olive Command Help

Source: https://github.com/microsoft/olive/blob/main/notebooks/sd_multilora/sd_multilora.ipynb

Displays available options for Olive's capture-onnx-graph and generate-adapter commands. Use this to understand command-line arguments.

```bash
# run this cell to see the available options to finetune, capture-onnx-graph and generate-adapter commands
!olive capture-onnx-graph --help
!olive generate-adapter --help
```

--------------------------------

### Install SD-LoRA Dependencies Manually

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/sd-lora.md

Manually install the core dependencies required for Stable Diffusion LoRA functionality if not using the extras option. Ensure versions are compatible.

```bash
pip install accelerate>=0.30.0 peft diffusers>=0.25.0 transformers>=4.30.0
```

--------------------------------

### Quantize PyTorch Model with GPTQ using Olive CLI

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-init-cli.md

Use this command to interactively set up GPTQ quantization for a PyTorch model. It guides you through model selection, quantization algorithm, precision, and output directory.

```bash
$ olive init

? What type of model do you want to optimize? PyTorch (HuggingFace or local)
? How would you like to specify your model? HuggingFace model name
? Model name or path: meta-llama/Llama-3.1-8B
? What do you want to do? Quantize only (PyTorch quantization)
? Select quantization algorithm: GPTQ - High quality, requires calibration
? Precision: int4
? Calibration data source: Use default (wikitext-2)
? Output directory: ./olive-output
? What would you like to do? Run optimization now
```

--------------------------------

### Fine-tune PyTorch Model with LoRA using Olive CLI

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-init-cli.md

This example shows how to generate a CLI command for fine-tuning a PyTorch model using LoRA. It covers model specification, LoRA parameters, dataset selection, and output configuration.

```bash
$ olive init

? What type of model do you want to optimize? PyTorch (HuggingFace or local)
? How would you like to specify your model? HuggingFace model name
? Model name or path: microsoft/Phi-4-mini-instruct
? What do you want to do? Fine-tune model (LoRA, QLoRA)
? Select fine-tuning method: LoRA (recommended)
? LoRA rank (r): 64 (default)
? LoRA alpha: 16
? Training dataset: HuggingFace dataset
? Dataset name: tatsu-lab/alpaca
? Train split: train
? How to construct training text? Use chat template
? Max sequence length: 1024
? Max training samples: 256
? Torch dtype for training: bfloat16 (recommended)
? Output directory: ./olive-output
? What would you like to do? Generate CLI command (copy and run later)

Generated command:

  olive finetune -m microsoft/Phi-4-mini-instruct --method lora --lora_r 64 --lora_alpha 16 -d tatsu-lab/alpaca --train_split train --use_chat_template --max_seq_len 1024 --max_samples 256 --torch_dtype bfloat16 -o ./olive-output
```

--------------------------------

### Example OrtTransformersOptimization Configuration

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/onnx-transformations.md

Configuration for OrtTransformersOptimization, specifying the model type.

```json
{
    "type": "OrtTransformersOptimization",
    "model_type": "bert"
}
```

--------------------------------

### Quantize and Optimize Model to ONNX

Source: https://github.com/microsoft/olive/blob/main/docs/source/how-to/cli/cli-quantize.md

This two-step process first quantizes a model using AWQ and then uses `auto-opt` to create an optimized ONNX model. The first step requires a GPU, while the second can run on CPU.

```bash
# Step 1: AWQ (will output a PyTorch model)
olive quantize \
    --model_name_or_path meta-llama/Llama-3.2-1B-Instruct \
    --algorithm awq \
    --output_path models/llama/awq \
    --log_level 1

# Step 2: Create an optimized ONNX model
olive auto-opt \
   --model_name_or_path models/llama/awq \
   --device cpu \
   --provider CPUExecutionProvider \
   --use_ort_genai \
   --output_path models/llama/onnx \
   --log_level 1
```

--------------------------------

### Example AI Interaction with Olive MCP Server

Source: https://github.com/microsoft/olive/blob/main/docs/source/blogs/olive-mcp-server.md

Demonstrates a natural language interaction with the Olive MCP server via VS Code Copilot for model optimization. The AI detects hardware, presents options, and executes the chosen optimization.

```text
You: Optimize microsoft/Phi-4-mini-instruct for my machine

Copilot: I'll start by detecting your hardware...
         [Calls detect_hardware — Click "Allow" to confirm]

         You have an NVIDIA RTX 4090 with 24GB VRAM. Here are your options:
         1. **Smallest model (int4)** — ~2GB, fastest inference
         2. **Balanced (int8)** — ~4GB, good quality/speed tradeoff
         3. **Best quality (fp16)** — ~8GB, highest accuracy

You: Go with int4

Copilot: Starting int4 optimization with GPTQ...
         [Calls optimize — Click "Allow" to confirm]
         [Polls get_job_status and streams progress]

         ✅ Done!Model saved to ~/.olive-mcp/outputs/optimize-phi-4-mini-...
         Size: 7.6GB → 2.1GB (72% reduction)
```

--------------------------------

### ZeroOutInput Transformation Example

Source: https://github.com/microsoft/olive/blob/main/docs/source/features/onnx-transformations.md

Shows how to replace a specific node input with a zero tensor using the ZeroOutInput surgeon.

```protobuf
graph {
  input: "input1"
  input: "input2"
  node {
    op_type: "Add"
    input: ["input1", "input2"]
    output: ["add_output"]
  }
}
```

```json
{
    "type": "GraphSurgeries",
    "surgeries": [
        {
            "surgeon": "ZeroOutInput",
            "node_name": "AddNode",
            "input_idx": 1
        }
    ]
}
```

```protobuf
graph {
  input: "input1"
  node {
    op_type: "Constant"
    output: ["Add_zero_output_0"]
    value: [0.0]
  }
  node {
    op_type: "Add"
    input: ["input1", "Add_zero_output_0"]
    output: ["add_output"]
  }
}
```