### Install Optimum with Pip

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install the base Optimum library using pip. This is the standard way to get started.

```bash
python -m pip install optimum
```

--------------------------------

### Install Optimum from Source

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install the base Optimum library directly from its GitHub repository. This is useful for development or accessing the latest code.

```bash
python -m pip install git+https://github.com/huggingface/optimum.git
```

--------------------------------

### Write Documentation Examples in Python

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Use this format for writing clear and concise inference examples within docstrings. Ensure the example is minimal, works as expected, and includes the expected output.

```python
>>> from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
>>> from datasets import load_dataset
>>> import torch

>>> dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
>>> dataset = dataset.sort("id")
>>> sampling_rate = dataset.features["audio"].sampling_rate

>>> processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-base-960h")
>>> model = Wav2Vec2ForCTC.from_pretrained("facebook/wav2vec2-base-960h")

>>> # audio file is decoded on the fly
>>> inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
>>> with torch.no_grad():
...     logits = model(**inputs).logits
>>> predicted_ids = torch.argmax(logits, dim=-1)

>>> # transcribe speech
>>> transcription = processor.batch_decode(predicted_ids)
>>> transcription[0]
'MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL'
```

--------------------------------

### Install Dependencies and Clone Repo

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Installs pip, clones the optimum-habana repository, and installs quality dependencies.

```bash
RUN python3 -m pip install --no-cache-dir --upgrade pip
RUN git clone https://github.com/huggingface/optimum-habana.git
RUN python3 -m pip install --no-cache-dir ./optimum-habana[quality]
```

--------------------------------

### Install Optimum with OpenVINO

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with OpenVINO support. Use --upgrade --upgrade-strategy eager to ensure the latest versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[openvino]
```

--------------------------------

### Install Optimum with FuriosaAI

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with FuriosaAI support. Use --upgrade --upgrade-strategy eager to ensure the latest versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[furiosa]
```

--------------------------------

### Install Accelerator from Source

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install a specific accelerator (e.g., ONNX Runtime) for Optimum directly from its GitHub repository. This allows using the latest development versions.

```bash
python -m pip install optimum[onnxruntime]@git+https://github.com/huggingface/optimum.git
```

--------------------------------

### Install Development Dependencies

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Install the project in editable mode with development dependencies. If Optimum is already installed, uninstall it first.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Install Optimum with ONNX Runtime

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with ONNX Runtime support. Use --upgrade --upgrade-strategy eager to ensure the latest versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[onnxruntime]
```

--------------------------------

### Install Optimum with NVIDIA TensorRT-LLM

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with NVIDIA TensorRT-LLM support using Docker. Ensure you have GPUs available.

```bash
docker run -it --gpus all --ipc host huggingface/optimum-nvidia
```

--------------------------------

### Install Optimum with AWS NeuronX

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with AWS Trainum & Inferentia support. Use --upgrade --upgrade-strategy eager to ensure the latest versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[neuronx]
```

--------------------------------

### Install Optimum with AMD support

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with support for AMD Instinct GPUs and Ryzen AI NPUs. Use --upgrade --upgrade-strategy eager to ensure the latest versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[amd]
```

--------------------------------

### Install Optimum with Habana Gaudi

Source: https://github.com/huggingface/optimum/blob/main/docs/source/installation.mdx

Install Optimum with Habana Gaudi Processor (HPU) support. Use --upgrade --upgrade-strategy eager to ensure the latest versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[habana]
```

--------------------------------

### Install Optimum ExecuTorch

Source: https://github.com/huggingface/optimum/blob/main/README.md

Installs the Optimum ExecuTorch package from its GitHub repository. This is required for exporting models to ExecuTorch for edge device inference.

```bash
pip install optimum-executorch@git+https://github.com/huggingface/optimum-executorch.git
```

--------------------------------

### Install Optimum with ONNX Support

Source: https://github.com/huggingface/optimum/blob/main/README.md

Install Optimum with ONNX specific features. The --upgrade --upgrade-strategy eager option ensures all dependencies are updated to their latest compatible versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[onnx]
```

--------------------------------

### Dockerfile for Optimum Subpackage Documentation

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

This Dockerfile sets up the environment for building documentation for an Optimum subpackage, including installing Node.js and npm for HTML generation.

```dockerfile
# Define base image for Habana
FROM vault.habana.ai/gaudi-docker/1.4.0/ubuntu20.04/habanalabs/pytorch-installer-1.10.2:1.4.0-442

# Need node to build doc HTML. Taken from https://stackoverflow.com/a/67491580
RUN apt-get update && apt-get install -y \
    software-properties-common \
    npm
RUN npm install npm@latest -g && \
    npm install n -g && \
    n latest

```

--------------------------------

### Install Optimum with ONNX Runtime GPU Support

Source: https://github.com/huggingface/optimum/blob/main/README.md

Install Optimum with ONNX Runtime GPU acceleration support. The --upgrade --upgrade-strategy eager option ensures all dependencies are updated to their latest compatible versions.

```bash
pip install --upgrade --upgrade-strategy eager optimum[onnxruntime-gpu]
```

--------------------------------

### Integer Representation Example

Source: https://github.com/huggingface/optimum/blob/main/docs/source/concept_guides/quantization.mdx

Illustrates how an integer is represented as an unsigned 8-bit integer in binary format.

```text
19 = 0 x 2^7 + 0 x 2^6 + 0 x 2^5 + 1 x 2^4 + 0 x 2^3 + 0 x 2^2 + 1 x 2^1 + 1 x 2^0
```

--------------------------------

### Get Help for ONNX Export CLI

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Display the help message for the optimum-cli export onnx command to see available options and configurations.

```bash
optimum-cli export onnx --help
```

--------------------------------

### Write Multi-line Code Blocks

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Multi-line code blocks for examples should be enclosed in triple backticks and follow the doctest syntax for automatic testing.

```markdown
```
# first line of code
# second line
# etc
```
```

--------------------------------

### Write a Reversible Transformation

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/usage_guides/optimization.mdx

Implement both `transform` and `reverse` methods by subclassing `ReversibleTransformation` to create transformations that can be undone. This example doubles multiplication operands and then halves them.

```python
import operator
from optimum.fx.optimization import ReversibleTransformation

class MulToMulTimesTwo(ReversibleTransformation):
    def transform(self, graph_module):
        for node in graph_module.graph.nodes:
            if node.op == "call_function" and node.target == operator.mul:
                x, y = node.args
                node.args = (2 * x, y)
        return graph_module

    def reverse(self, graph_module):
        for node in graph_module.graph.nodes:
            if node.op == "call_function" and node.target == operator.mul:
                x, y = node.args
                node.args = (x / 2, y)
        return graph_module
```

--------------------------------

### Get Supported Tasks for a Model Type and Backend

Source: https://github.com/huggingface/optimum/blob/main/docs/source/exporters/task_manager.mdx

Retrieve a list of tasks supported by a specific model type for a given backend. This is useful for understanding export capabilities.

```python
>>> from optimum.exporters.tasks import TasksManager

>>> model_type = "distilbert"
>>> # For instance, for the ONNX export.
>>> backend = "onnx"
>>> distilbert_tasks = list(TasksManager.get_supported_tasks_for_model_type(model_type, backend).keys())

>>> print(distilbert_tasks)
['default', 'fill-mask', 'text-classification', 'multiple-choice', 'token-classification', 'question-answering']
```

--------------------------------

### Write a Non-Reversible Transformation

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/usage_guides/optimization.mdx

Subclass `Transformation` and implement `transform` to create non-reversible graph modifications. This example changes multiplications to additions.

```python
import operator
from optimum.fx.optimization import Transformation

class ChangeMulToAdd(Transformation):
    def transform(self, graph_module):
        for node in graph_module.graph.nodes:
            if node.op == "call_function" and node.target == operator.mul:
                node.target = operator.add
        return graph_module
```

--------------------------------

### Define Single Value Return Block

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

The return block for a single value should start with the `Returns:` prefix, followed by the return type and a description.

```python
    Returns:
        `List[int]`: A list of integers in the range [0, 1] --- 1 for a special token, 0 for a sequence token.
```

--------------------------------

### Clone Repository and Set Up Remotes

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Clone your fork of the Optimum repository and add the base repository as an upstream remote. This is the first step in setting up your local development environment.

```bash
git clone git@github.com:<your Github handle>/optimum.git
cd optimum
git remote add upstream https://github.com/huggingface/optimum.git
```

--------------------------------

### Generate Optimum Documentation

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Run this command from the root of the optimum repository to generate HTML documentation files. BUILD_DIR and VERSION can be adapted to your needs.

```bash
make doc BUILD_DIR=optimum-doc-build VERSION=main
```

--------------------------------

### Generate optimum-habana Documentation

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

To generate documentation for hardware partner integrations like optimum-habana, first clone the specific repository, then navigate into it and run the make doc command. BUILD_DIR can be customized.

```bash
git clone https://github.com/huggingface/optimum-habana.git
cd optimum-habana
make doc BUILD_DIR=habana-doc-build
```

--------------------------------

### Generate Documentation Locally

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Command to generate documentation locally using the defined Makefile target. Requires BUILD_DIR and VERSION to be set.

```bash
make doc BUILD_DIR=habana-doc-build VERSION=main
```

--------------------------------

### Initialize GaudiTrainer for Habana Training

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Initialize the GaudiTrainer and GaudiTrainingArguments for training models on Habana Gaudi processors. This replaces the standard Transformers Trainer and TrainingArguments.

```diff
from transformers import Trainer, TrainingArguments
from optimum.habana import GaudiTrainer, GaudiTrainingArguments

# Download a pretrained model from the Hub
model = AutoModelForXxx.from_pretrained("bert-base-uncased")

# Define the training arguments
training_args = TrainingArguments(
    output_dir="path/to/save/folder/",
    use_habana=True,
    use_lazy_mode=True,
    gaudi_config_name="Habana/bert-base-uncased",
    ...
)

# Initialize the trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    ...
)

# Use Habana Gaudi processor for training!
trainer.train()
```

--------------------------------

### Load Quantized Model with Exllama v1 Kernels

Source: https://github.com/huggingface/optimum/blob/main/docs/source/llm_quantization/usage_guides/quantization.mdx

Explicitly configure the model loading to use Exllama v1 kernels by setting the `exllama_config` parameter. This is useful for specific version requirements.

```python
from optimum.gptq import GPTQQuantizer, load_quantized_model
import torch

from accelerate import init_empty_weights
with init_empty_weights():
    empty_model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)
empty_model.tie_weights()
quantized_model = load_quantized_model(empty_model, save_folder=save_folder, device_map="auto", exllama_config = {"version":1})
```

--------------------------------

### Makefile for Documentation Build

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Defines Docker build and documentation generation targets in a Makefile. Ensure BUILD_DIR and VERSION environment variables are set.

```makefile
SHELL := /bin/bash
CURRENT_DIR = $(shell pwd)

...

build_doc_docker_image:
	docker build -t doc_maker ./docs

doc: build_doc_docker_image
	@test -n "$(BUILD_DIR)" || (echo "BUILD_DIR is empty." ; exit 1)
	@test -n "$(VERSION)" || (echo "VERSION is empty." ; exit 1)
	docker run -v $(CURRENT_DIR):/doc_folder --workdir=/doc_folder doc_maker \
	doc-builder build optimum.habana /optimum-habana/docs/source/ \
		--build_dir $(BUILD_DIR) \
		--version $(VERSION) \
		--version_tag_suffix "" \
		--html \
		--clean
```

--------------------------------

### Load Quantized Model with Accelerate

Source: https://github.com/huggingface/optimum/blob/main/docs/source/llm_quantization/usage_guides/quantization.mdx

Load quantized model weights efficiently using Accelerate's init_empty_weights. This approach initializes the model with empty weights and loads them in a subsequent step, reducing memory usage.

```python
from accelerate import init_empty_weights
with init_empty_weights():
    empty_model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)
empty_model.tie_weights()
quantized_model = load_quantized_model(empty_model, save_folder=save_folder, device_map="auto")
```

--------------------------------

### Format Code with Black and Ruff

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Run the 'make style' command to format your newly added files using Black and Ruff. This ensures code consistency.

```bash
make style
```

--------------------------------

### Combine Subpackage Documentation in GitHub Actions

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Combines the subpackage documentation with the main Optimum documentation in a GitHub Actions workflow. Ensure the subpackage is listed in the --subpackages argument.

```yaml
# Tweak this to include your subpackage
- name: Combine subpackage documentation
    run: |
    cd optimum
    sudo python docs/combine_docs.py --subpackages habana --version pr_$PR_NUMBER # Make sure the subpackage is listed here!
    sudo mv optimum-doc-build ../
    cd ..
```

--------------------------------

### Existing Normalized Configurations

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/normalized_config.mdx

Documentation for pre-defined normalized configuration classes tailored for specific model types.

```APIDOC
## NormalizedTextConfig

### Description
Normalized configuration for text-based models.

## NormalizedSeq2SeqConfig

### Description
Normalized configuration for sequence-to-sequence models.

## NormalizedVisionConfig

### Description
Normalized configuration for vision models.

## NormalizedTextAndVisionConfig

### Description
Normalized configuration for models that handle both text and vision inputs.
```

--------------------------------

### Load Quantized Model with Exllamav2 Kernels

Source: https://github.com/huggingface/optimum/blob/main/docs/source/llm_quantization/usage_guides/quantization.mdx

Load a quantized model using Exllamav2 kernels, which are enabled by default for faster inference. Ensure the entire model resides on GPUs.

```python
from optimum.gptq import GPTQQuantizer, load_quantized_model
import torch

from accelerate import init_empty_weights
with init_empty_weights():
    empty_model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)
empty_model.tie_weights()
quantized_model = load_quantized_model(empty_model, save_folder=save_folder, device_map="auto")
```

--------------------------------

### Export Model to ONNX using CLI

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Export a Hugging Face model to ONNX format using the optimum-cli command. This is a convenient way to convert models for ONNX Runtime.

```bash
optimum-cli export onnx --model gpt2 gpt2_onnx/
```

--------------------------------

### Accelerated Inference with OpenVINO

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Replace `AutoModelForXxx` with `OVModelForXxx` to load a model and run inference with OpenVINO Runtime. Set `export=True` to convert PyTorch checkpoints to OpenVINO IR.

```diff
- from transformers import AutoModelForSequenceClassification
+ from optimum.intel.openvino import OVModelForSequenceClassification
 from transformers import AutoTokenizer, pipeline

 # Download a tokenizer and model from the Hub and convert to OpenVINO format
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model_id = "distilbert-base-uncased-finetuned-sst-2-english"
- model = AutoModelForSequenceClassification.from_pretrained(model_id)
+ model = OVModelForSequenceClassification.from_pretrained(model_id, export=True)

 # Run inference!
 classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
 results = classifier("He's a dreadful magician.")
```

--------------------------------

### DummyAudioInputGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy audio inputs for audio processing models.

```APIDOC
## Class: DummyAudioInputGenerator

### Description
Generates dummy audio inputs. This generator is designed for models that process audio data, such as speech recognition or audio classification models.

### Methods

*   `__init__(self, **kwargs)`: Initializes the audio input generator.
*   `generate(self, batch_size=1, sequence_length=16000, **kwargs)`: Generates dummy audio inputs.
    *   `batch_size` (int): The number of audio samples in the batch.
    *   `sequence_length` (int): The length of each audio sequence (e.g., number of samples).

### Usage
```python
from optimum.utils.input_generators import DummyAudioInputGenerator

audio_generator = DummyAudioInputGenerator()

dummy_audio = audio_generator.generate(
    batch_size=2,
    sequence_length=32000
)
print(dummy_audio)
```
```

--------------------------------

### Build Habana Documentation in GitHub Actions

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Builds the Habana documentation within a GitHub Actions workflow. Ensure BUILD_DIR is set to '{subpackage_name}-doc-build' and moves the output to the main optimum directory.

```yaml
# Add this
- name: Make Habana documentation
run: |
    cd optimum-habana
    make doc BUILD_DIR=habana-doc-build VERSION=pr_$PR_NUMBER # Make sure BUILD_DIR={subpackage_name}-doc-build
    sudo mv habana-doc-build ../optimum
    cd ..
```

--------------------------------

### Load and Quantize Model with GPTQ

Source: https://github.com/huggingface/optimum/blob/main/docs/source/llm_quantization/usage_guides/quantization.mdx

Use GPTQQuantizer to quantize a model. Specify the number of bits, dataset for calibration, and optionally model sequence length and block name for custom models. Ensure the model is in torch.float16 before quantization.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from optimum.gptq import GPTQQuantizer, load_quantized_model
import torch
model_name = "facebook/opt-125m"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)

quantizer = GPTQQuantizer(bits=4, dataset="c4", block_name_to_quantize = "model.decoder.layers", model_seqlen = 2048)
quantized_model = quantizer.quantize_model(model, tokenizer)
```

--------------------------------

### Apply Dynamic Quantization with ONNX Runtime

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Apply dynamic quantization to an ONNX model using ORTQuantizer. This can reduce model size and improve inference speed. Ensure the model is already exported to ONNX format.

```python
from optimum.onnxruntime.configuration import AutoQuantizationConfig
from optimum.onnxruntime import ORTQuantizer

# Define the quantization methodology
qconfig = AutoQuantizationConfig.arm64(is_static=False, per_channel=False)
quantizer = ORTQuantizer.from_pretrained(ort_model)

# Apply dynamic quantization on the model
quantizer.quantize(save_dir=save_directory, quantization_config=qconfig)  # doctest: +IGNORE_RESULT
```

--------------------------------

### Stage and Commit Changes

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Add modified files using 'git add' and commit your changes locally using 'git commit'.

```bash
git add modified_file.py
git commit
```

--------------------------------

### Checkout Optimum Habana Subpackage in GitHub Actions

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Checks out the optimum-habana repository into a specific path within the GitHub Actions workflow.

```yaml
# Add this
- uses: actions/checkout@v2
with:
    repository: 'huggingface/optimum-habana'
    path: optimum-habana
```

--------------------------------

### Create a New Development Branch

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Create a new branch for your development changes. It is crucial not to work directly on the 'main' branch.

```bash
git checkout -b a-descriptive-name-for-my-changes
```

--------------------------------

### Define Arguments in a Method

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Arguments should be defined with the `Args:` prefix, followed by their type, shape (if applicable), and description. For multi-line descriptions, further indentation is required.

```python
    Args:
        n_layers (`int`): The number of layers of the model.
```

```python
    Args:
        input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
            Indices of input sequence tokens in the vocabulary.

            Indices can be obtained using [`AlbertTokenizer`]. See [`~PreTrainedTokenizer.encode`] and
            [`~PreTrainedTokenizer.__call__`] for details.

            [What are input IDs?](../glossary#input-ids)
```

--------------------------------

### Sync with Upstream Repository

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Fetch changes from the upstream repository and rebase your local branch onto the upstream main branch to incorporate recent updates.

```bash
git fetch upstream
git rebase upstream/main
```

--------------------------------

### fx.optimization.ReversibleTransformation

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Base class for reversible transformations in Torch FX.

```APIDOC
## fx.optimization.ReversibleTransformation

### Description
Base class for reversible transformations in Torch FX. These transformations can be undone.

### Methods
#### all()
Returns a list of all available reversible transformations.

#### __call__(graph_module: torch.fx.GraphModule, **kwargs) -> torch.fx.GraphModule
Applies the reversible transformation to the given GraphModule.
```

--------------------------------

### Apply a Non-Reversible Transformation

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/usage_guides/optimization.mdx

Instantiate and apply a custom non-reversible transformation to a symbolically traced model. Ensure the model is traced with `symbolic_trace` before applying.

```python
from transformers import BertModel
from transformers.utils.fx import symbolic_trace

model = BertModel.from_pretrained("bert-base-uncased")
traced = symbolic_trace(
    model,
    input_names=["input_ids", "attention_mask", "token_type_ids"],
)

transformation = ChangeMulToAdd()
transformed_model = transformation(traced)
```

--------------------------------

### DummyBboxInputGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy bounding box inputs, commonly used in object detection and visual grounding tasks.

```APIDOC
## Class: DummyBboxInputGenerator

### Description
Generates dummy bounding box inputs. This is particularly useful for models involved in computer vision tasks such as object detection, image segmentation, or visual question answering where bounding boxes are a key input.

### Methods

*   `__init__(self, **kwargs)`: Initializes the bounding box input generator.
*   `generate(self, batch_size=1, image_height=224, image_width=224, num_boxes=10, **kwargs)`: Generates dummy bounding box inputs.
    *   `batch_size` (int): The number of samples in the batch.
    *   `image_height` (int): The height of the image associated with the bounding boxes.
    *   `image_width` (int): The width of the image associated with the bounding boxes.
    *   `num_boxes` (int): The number of bounding boxes to generate per image.

### Usage
```python
from optimum.utils.input_generators import DummyBboxInputGenerator

bbox_generator = DummyBboxInputGenerator()

dummy_bboxes = bbox_generator.generate(
    batch_size=2,
    image_height=480,
    image_width=640,
    num_boxes=5
)
print(dummy_bboxes)
```
```

--------------------------------

### Save Quantized Model

Source: https://github.com/huggingface/optimum/blob/main/docs/source/llm_quantization/usage_guides/quantization.mdx

Save the quantized model and its configuration using the save method of the GPTQQuantizer class. This creates a folder containing the model's state dictionary and quantization configuration.

```python
save_folder = "/path/to/save_folder/"
quantizer.save(model,save_folder)
```

--------------------------------

### Affine Quantization Scheme

Source: https://github.com/huggingface/optimum/blob/main/docs/source/concept_guides/quantization.mdx

Illustrates the affine quantization scheme for projecting float32 values to int8. This scheme uses a scale (S) and a zero-point (Z) to map the float range [a, b] to the int8 space.

```python
x = S * (x_q - Z)
```

```python
x_q = round(x/S + Z)
```

```python
x_q = clip(round(x/S + Z), round(a/S + Z), round(b/S + Z))
```

--------------------------------

### DummyInputGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

The base class for all dummy input generators. Provides the fundamental structure for creating dummy inputs.

```APIDOC
## Class: DummyInputGenerator

### Description
This is the base class for dummy input generators. It provides the core functionality for creating dummy inputs that can be used for various purposes such as model tracing, exporting, and testing.

### Methods

*   `__init__(self, **kwargs)`: Initializes the dummy input generator.
*   `generate(self, **kwargs)`: Generates a dummy input based on the provided arguments.

### Usage
```python
from optimum.utils.input_generators import DummyInputGenerator

# Instantiate the base class (or a subclass)
dummy_generator = DummyInputGenerator()

# Generate dummy inputs (specifics depend on subclass)
# dummy_inputs = dummy_generator.generate(...)
```
```

--------------------------------

### Define Tuple Return Block

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

For tuple returns, specify the tuple type and then list each element with its type, shape, and description, using a hyphen prefix for each element.

```python
    Returns:
        `tuple(torch.FloatTensor)` comprising various elements depending on the configuration ([`BertConfig`]) and inputs:
        - ** loss** (*optional*, returned when `masked_lm_labels` is provided) `torch.FloatTensor` of shape `(1,)` --
          Total loss as the sum of the masked language modeling loss and the next sequence prediction (classification) loss.
        - **prediction_scores** (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`) --
          Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).
```

--------------------------------

### Load Quantized ONNX Model and Predict

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Load a dynamically quantized ONNX model and use it with the Transformers pipeline for text classification. This demonstrates how to run inference with a quantized model.

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer

model = ORTModelForSequenceClassification.from_pretrained(save_directory, file_name="model_quantized.onnx")
tokenizer = AutoTokenizer.from_pretrained(save_directory)
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
results = classifier("I love burritos!")
```

--------------------------------

### fx.optimization.Transformation

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Base class for transformations in Torch FX.

```APIDOC
## fx.optimization.Transformation

### Description
Base class for transformations in Torch FX. Provides a common interface for applying transformations to a PyTorch FX graph.

### Methods
#### all()
Returns a list of all available transformations.

#### __call__(graph_module: torch.fx.GraphModule, **kwargs) -> torch.fx.GraphModule
Applies the transformation to the given GraphModule.
```

--------------------------------

### fx.optimization.compose

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Composes multiple transformations into a single transformation.

```APIDOC
## fx.optimization.compose

### Description
Composes multiple transformations into a single transformation. The transformations are applied in the order they are provided.

### Parameters
- ***transformations**: `fx.optimization.Transformation` - A variable number of transformations to compose.
```

--------------------------------

### Registering Custom Commands in Optimum CLI

Source: https://github.com/huggingface/optimum/blob/main/optimum/commands/register/README.md

Define the `REGISTER_COMMANDS` list to register custom commands. Commands can be registered as root subcommands or as subcommands of existing commands.

```python
# CustomCommand1 and CustomCommand2 could also be defined in this file actually.
from ..my_custom_commands import CustomCommand1, CustomCommand2
from ..export import ExportCommand

REGISTER_COMMANDS = [
  # CustomCommand1 will be registered as a subcommand of the root Optimum CLI. 
  CustomCommand1, # registered as `optimum-cli custom-command1`
  # CustomCommand2 will be registered as a subcommand of the `optimum-cli export` command. 
  (CustomCommand2, ExportCommand) # registered as `optimum-cli export custom-command2`
]
```

--------------------------------

### DummyVisionInputGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy image inputs for vision models.

```APIDOC
## Class: DummyVisionInputGenerator

### Description
Generates dummy image inputs. This generator is suitable for computer vision models that expect image tensors as input, such as image classification or object detection models.

### Methods

*   `__init__(self, **kwargs)`: Initializes the vision input generator.
*   `generate(self, batch_size=1, image_height=224, image_width=224, num_channels=3, **kwargs)`: Generates dummy image inputs.
    *   `batch_size` (int): The number of images in the batch.
    *   `image_height` (int): The height of each image.
    *   `image_width` (int): The width of each image.
    *   `num_channels` (int): The number of color channels (e.g., 3 for RGB).

### Usage
```python
from optimum.utils.input_generators import DummyVisionInputGenerator

vision_generator = DummyVisionInputGenerator()

dummy_images = vision_generator.generate(
    batch_size=4,
    image_height=256,
    image_width=256,
    num_channels=3
)
print(dummy_images)
```
```

--------------------------------

### Load and Export Model to ONNX

Source: https://github.com/huggingface/optimum/blob/main/docs/source/quicktour.mdx

Load a model from the Hugging Face Hub and export it to ONNX format using ORTModelForSequenceClassification. This is useful for preparing models for ONNX Runtime inference.

```python
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer

model_checkpoint = "distilbert-base-uncased-finetuned-sst-2-english"
save_directory = "tmp/onnx/"

# Load a model from transformers and export it to ONNX
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
ort_model = ORTModelForSequenceClassification.from_pretrained(model_checkpoint, export=True)

# Save the ONNX model and tokenizer
ort_model.save_pretrained(save_directory)
tokenizer.save_pretrained(save_directory)  # doctest: +IGNORE_RESULT
```

--------------------------------

### DummyTextInputGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy text inputs, suitable for models that process text data.

```APIDOC
## Class: DummyTextInputGenerator

### Description
A dummy input generator specifically designed for creating text inputs. This is useful for models that expect string or tokenized text as input.

### Methods

*   `__init__(self, **kwargs)`: Initializes the text input generator.
*   `generate(self, batch_size=1, sequence_length=10, **kwargs)`: Generates a batch of dummy text inputs.
    *   `batch_size` (int): The number of samples in the batch.
    *   `sequence_length` (int): The length of each text sequence.

### Usage
```python
from optimum.utils.input_generators import DummyTextInputGenerator

text_generator = DummyTextInputGenerator()

dummy_text_inputs = text_generator.generate(batch_size=2, sequence_length=15)
print(dummy_text_inputs)
```
```

--------------------------------

### DummyDecoderTextInputGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy text inputs specifically for decoder models, often used in sequence-to-sequence tasks.

```APIDOC
## Class: DummyDecoderTextInputGenerator

### Description
Generates dummy text inputs tailored for the decoder part of sequence-to-sequence models. This generator is useful when you need to provide inputs for the decoder, such as in generation tasks.

### Methods

*   `__init__(self, **kwargs)`: Initializes the decoder text input generator.
*   `generate(self, batch_size=1, sequence_length=10, **kwargs)`: Generates a batch of dummy decoder text inputs.
    *   `batch_size` (int): The number of samples in the batch.
    *   `sequence_length` (int): The length of each text sequence.

### Usage
```python
from optimum.utils.input_generators import DummyDecoderTextInputGenerator

decoder_text_generator = DummyDecoderTextInputGenerator()

dummy_decoder_inputs = decoder_text_generator.generate(batch_size=1, sequence_length=20)
print(dummy_decoder_inputs)
```
```

--------------------------------

### Push Changes to Origin

Source: https://github.com/huggingface/optimum/blob/main/CONTRIBUTING.md

Push your local branch with the development changes to your fork on GitHub. The '-u' flag sets the upstream for the branch.

```bash
git push -u origin a-descriptive-name-for-my-changes
```

--------------------------------

### DummyPastKeyValuesGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy past key-value states, typically used in autoregressive models to cache previous computations.

```APIDOC
## Class: DummyPastKeyValuesGenerator

### Description
Generates dummy past key-value states. These are often used in autoregressive models (like transformers) to cache the computations from previous steps, improving efficiency during generation.

### Methods

*   `__init__(self, **kwargs)`: Initializes the past key-value generator.
*   `generate(self, batch_size=1, sequence_length=10, num_hidden_layers=1, num_attention_heads=1, hidden_size=768, **kwargs)`: Generates dummy past key-value states.
    *   `batch_size` (int): The number of samples in the batch.
    *   `sequence_length` (int): The length of the sequence for which past states are generated.
    *   `num_hidden_layers` (int): The number of hidden layers in the model.
    *   `num_attention_heads` (int): The number of attention heads.
    *   `hidden_size` (int): The hidden size of the model.

### Usage
```python
from optimum.utils.input_generators import DummyPastKeyValuesGenerator

past_key_values_generator = DummyPastKeyValuesGenerator()

dummy_past_key_values = past_key_values_generator.generate(
    batch_size=1,
    sequence_length=5,
    num_hidden_layers=2,
    num_attention_heads=4,
    hidden_size=128
)
print(dummy_past_key_values)
```
```

--------------------------------

### Compose Transformations

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/usage_guides/optimization.mdx

Use the `compose` utility function to chain multiple transformations together into a single, sequential transformation. This simplifies applying a series of modifications.

```python
from optimum.fx.optimization import compose
composition = compose(MulToMulTimesTwo(), ChangeMulToAdd())
```

--------------------------------

### DummySeq2SeqPastKeyValuesGenerator

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/dummy_input_generators.mdx

Generates dummy past key-value states specifically for the encoder-decoder architecture in sequence-to-sequence models.

```APIDOC
## Class: DummySeq2SeqPastKeyValuesGenerator

### Description
Generates dummy past key-value states tailored for sequence-to-sequence models that employ an encoder-decoder architecture. This generator is useful for providing cached states for both the encoder and decoder during generation.

### Methods

*   `__init__(self, **kwargs)`: Initializes the Seq2Seq past key-value generator.
*   `generate(self, batch_size=1, encoder_sequence_length=10, decoder_sequence_length=10, num_hidden_layers=1, num_attention_heads=1, hidden_size=768, **kwargs)`: Generates dummy past key-value states for a Seq2Seq model.
    *   `batch_size` (int): The number of samples in the batch.
    *   `encoder_sequence_length` (int): The length of the encoder's sequence.
    *   `decoder_sequence_length` (int): The length of the decoder's sequence.
    *   `num_hidden_layers` (int): The number of hidden layers.
    *   `num_attention_heads` (int): The number of attention heads.
    *   `hidden_size` (int): The hidden size of the model.

### Usage
```python
from optimum.utils.input_generators import DummySeq2SeqPastKeyValuesGenerator

seq2seq_past_key_values_generator = DummySeq2SeqPastKeyValuesGenerator()

dummy_seq2seq_past_key_values = seq2seq_past_key_values_generator.generate(
    batch_size=1,
    encoder_sequence_length=10,
    decoder_sequence_length=5,
    num_hidden_layers=2,
    num_attention_heads=4,
    hidden_size=128
)
print(dummy_seq2seq_past_key_values)
```
```

--------------------------------

### NormalizedConfig Base Class

Source: https://github.com/huggingface/optimum/blob/main/docs/source/utils/normalized_config.mdx

The base class for normalized configurations. It allows access to configuration attributes in a standardized way. Subclasses can be created for common use-cases, or the mapping can be overwritten using the `with_args` class method.

```APIDOC
## Class: NormalizedConfig

### Description
Provides a standardized way to access model configuration attributes.

### Class Methods
#### `with_args`
Allows overwriting the `original attribute name -> normalized attribute name` mapping.

### Attributes
(Refer to the source documentation for a detailed list of attributes and their descriptions.)
```

--------------------------------

### fx.optimization.FuseBatchNorm1dInLinear

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Fuses BatchNorm1d layers into preceding Linear layers.

```APIDOC
## fx.optimization.FuseBatchNorm1dInLinear

### Description
Fuses BatchNorm1d layers into preceding Linear layers in a PyTorch FX graph. This optimization can simplify the graph and potentially improve execution speed.

### Methods
#### all()
Returns a list of all available transformations, including FuseBatchNorm1dInLinear.
```

--------------------------------

### fx.optimization.MergeLinears

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Merges adjacent Linear layers in a PyTorch FX graph.

```APIDOC
## fx.optimization.MergeLinears

### Description
Merges adjacent Linear layers in a PyTorch FX graph. This can reduce overhead and improve performance.

### Methods
#### all()
Returns a list of all available transformations, including MergeLinears.
```

--------------------------------

### Define Optional Arguments with Defaults

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

Optional arguments with default values should clearly indicate their type, optional status, and default value. The default value 'None' is omitted if it's the default.

```python
def my_function(x: str = None, a: float = 1):
    # ...

    Args:
        x (`str`, *optional*):
            This argument controls ...
        a (`float`, *optional*, defaults to 1):
            This argument is used to ...
```

--------------------------------

### fx.optimization.ChangeTrueDivToMulByInverse

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Replaces true division operations with multiplication by the inverse.

```APIDOC
## fx.optimization.ChangeTrueDivToMulByInverse

### Description
Replaces true division operations (e.g., `torch.div`) with multiplication by the inverse (e.g., `a * (1/b)`). This can sometimes be more efficient or numerically stable.

### Methods
#### all()
Returns a list of all available transformations, including ChangeTrueDivToMulByInverse.
```

--------------------------------

### fx.optimization.FuseBiasInLinear

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Fuses a bias addition into a Linear layer in a PyTorch FX graph.

```APIDOC
## fx.optimization.FuseBiasInLinear

### Description
Fuses a bias addition into a Linear layer in a PyTorch FX graph. This optimization can simplify the graph and potentially improve execution speed.

### Methods
#### all()
Returns a list of all available transformations, including FuseBiasInLinear.
```

--------------------------------

### fx.optimization.FuseBatchNorm2dInConv2d

Source: https://github.com/huggingface/optimum/blob/main/docs/source/torch_fx/package_reference/optimization.mdx

Fuses BatchNorm2d layers into preceding Conv2d layers.

```APIDOC
## fx.optimization.FuseBatchNorm2dInConv2d

### Description
Fuses BatchNorm2d layers into preceding Conv2d layers in a PyTorch FX graph. This optimization can reduce the number of operations and improve inference speed.

### Methods
#### all()
Returns a list of all available transformations, including FuseBatchNorm2dInConv2d.
```

--------------------------------

### Preserve Old Links When Moving Sections to Another File

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

If a section is moved to another file, use a relative path in the anchor link to maintain the integrity of versioned documentation. The original anchor ID should also be preserved.

```html
[ <a href="../new-file#section-b">Section A</a><a id="section-a"></a> ]
```

--------------------------------

### Preserve Old Links When Renaming Sections

Source: https://github.com/huggingface/optimum/blob/main/docs/README.md

When renaming a section header or moving a section, add this map at the end of the original document to preserve old links. Use anchor IDs to link to the new section.

```html
[ <a href="#section-b">Section A</a><a id="section-a"></a> ]
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.