### Install Optimum-Quanto from Source

Source: https://github.com/huggingface/optimum-quanto/blob/main/examples/vision/StableDiffusion/README.md

Clone the repository, install the library from source, and then install example-specific requirements.

```bash
git clone https://github.com/huggingface/quanto
cd quanto
pip install -e .
```

```bash
pip install -r requirements.txt
```

--------------------------------

### Install Development Environment

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Install the project in editable mode with development dependencies. If the package is already installed, uninstall it first before reinstalling with the -e flag.

```bash
pip install -e ".[dev]"
```

--------------------------------

### Install Optimum Quanto

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Install the Optimum Quanto package using pip. This is the primary way to add the library to your project.

```sh
pip install optimum-quanto
```

--------------------------------

### Implement New Quanto Operation

Source: https://github.com/huggingface/optimum-quanto/blob/main/optimum/quanto/library/extensions/README.md

Provide the implementation for a newly defined Quanto operation. This example shows the implementation for a CUDA-specific gemm_f16i4 operation.

```python
@torch.library.impl("quanto::gemm_f16i4", ["CUDA"])
def gemm_f16i4(
    input: torch.Tensor,
    other: torch.Tensor,
    scales: torch.Tensor,
    shift: torch.Tensor,
    group_size: int,
) -> torch.Tensor:
    ...
```

--------------------------------

### Performance Benchmarks for Stable Diffusion Quantization

Source: https://github.com/huggingface/optimum-quanto/blob/main/examples/vision/StableDiffusion/README.md

Examples of performance metrics for different quantization configurations (fp16-fp16, bf16-int8, fp16-int8), showing generation time and memory usage.

```text
fp16-fp16

batch_size: 1, torch_dtype: fp16, unet_dtype: none  in 3.307 seconds.Memory: 3.192GB.
```

```text
bf16-int8

batch_size: 1, torch_dtype: bf16, unet_dtype: int8  in 3.918 seconds.Memory: 2.644GB.
```

```text
fp16-int8

batch_size: 1, torch_dtype: fp16, unet_dtype: int8  in 3.920 seconds.Memory: 2.634GB.
```

--------------------------------

### Quantize Diffusers Transformer Model

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Quantize a specific submodel, like the transformer, within a diffusers pipeline. This example uses `qfloat8` for quantization.

```python
from diffusers import PixArtTransformer2DModel
from optimum.quanto import QuantizedPixArtTransformer2DModel, qfloat8

model = PixArtTransformer2DModel.from_pretrained("PixArt-alpha/PixArt-Sigma-XL-2-1024-MS", subfolder="transformer")
qmodel = QuantizedPixArtTransformer2DModel.quantize(model, weights=qfloat8)
qmodel.save_pretrained("./pixart-sigma-fp8")
```

--------------------------------

### Reload and Integrate Quantized Diffusers Transformer

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Reload a quantized diffusers transformer model and integrate it back into a pipeline. This example demonstrates moving the quantized model to a CUDA device.

```python
from diffusers import PixArtTransformer2DModel
from optimum.quanto import QuantizedPixArtTransformer2DModel
import torch

transformer = QuantizedPixArtTransformer2DModel.from_pretrained("./pixart-sigma-fp8")
transformer.to(device="cuda")
pipe = PixArtSigmaPipeline.from_pretrained(
  "PixArt-alpha/PixArt-Sigma-XL-2-1024-MS",
  transformer=None,
  torch_dtype=torch.float16,
).to("cuda")
pipe.transformer = transformer
```

--------------------------------

### Clone Repository and Set Up Remotes

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Clone your forked repository and add the base repository as an upstream remote. This is essential for keeping your local copy synchronized with the main project.

```bash
git clone git@github.com:<your Github handle>/optimum-quanto.git
cd optimum-quanto
git remote add upstream https://github.com/huggingface/optimum-quanto.git
```

--------------------------------

### Run All Tests with Make

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Execute all tests in the project using the provided make command. This is a convenient way to ensure all tests pass before submitting changes.

```bash
make test
```

--------------------------------

### Apply Code Formatting and Linting

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Run the 'make style' command to automatically apply code style corrections and perform code verifications using tools like 'black' and 'ruff'.

```bash
make style
```

--------------------------------

### Serialize Quantized Model Weights

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Saves the quantized model's weights to a state dictionary, which can then be serialized to a file using pickle or safetensors. Safetensors is recommended.

```python
from safetensors.torch import save_file

save_file(model.state_dict(), 'model.safetensors')
```

--------------------------------

### Quantization-Aware Training (QAT)

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Tunes a quantized model for a few epochs to recover performance degradation. This involves standard PyTorch training loops, ensuring the model is in training mode and using .dequantize() before calculating loss.

```python
import torch

model.train()
for batch_idx, (data, target) in enumerate(train_loader):
    data, target = data.to(device), target.to(device)
    optimizer.zero_grad()
    output = model(data).dequantize()
    loss = torch.nn.functional.nll_loss(output, target)
    loss.backward()
    optimizer.step()
```

--------------------------------

### Convert Model to Smoothed Version

Source: https://github.com/huggingface/optimum-quanto/blob/main/external/smoothquant/README.md

Use this script to convert an OPT or Bloom model to its smoothed version. Ensure the model architecture is compatible, specifically for OPT models where layer normalization is applied before attention.

```bash
python smoothquant.py --model facebook/opt-1.3b --save-path smoothed-models/facebook/opt-1.3b
```

--------------------------------

### Run Stable Diffusion Quantization Script

Source: https://github.com/huggingface/optimum-quanto/blob/main/examples/vision/StableDiffusion/README.md

Execute the image generation script with specified batch size and torch dtype. Adjust unet_qtype for different quantization levels.

```bash
python quantize_StableDiffusion.py --batch_size=1 --torch_dtype="fp32"
```

--------------------------------

### Implement Existing Quanto Operation

Source: https://github.com/huggingface/optimum-quanto/blob/main/optimum/quanto/library/extensions/README.md

Use this decorator to provide a device-specific implementation for an existing Quanto operation. Specify the operation name and the target devices.

```python
@torch.library.impl("quanto::unpack", ["CPU", "CUDA"])
def unpack(packed: torch.Tensor, bits: int) -> torch.Tensor:
    return ext.unpack(t, bits)
```

--------------------------------

### Stage and Commit Changes

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Add modified files to the staging area and commit your changes locally. This is a standard Git workflow for recording changes.

```bash
git add modified_file.py
git commit
```

--------------------------------

### Define New Quanto Operation

Source: https://github.com/huggingface/optimum-quanto/blob/main/optimum/quanto/library/extensions/README.md

Declare a new device-specific operation for the Quanto library using torch.library.define. This specifies the operation signature.

```python
torch.library.define(
    "quanto::gemm_f16i4",
    "(Tensor input,\n     Tensor other,\n     Tensor other_scale,\n     Tensor other_shift,\n     int group_size)"
    " -> Tensor",
)
```

--------------------------------

### Quantize Model Weights and Activations

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Converts a standard float model into a dynamically quantized model. This modifies the model's inference to dynamically quantize weights and optionally activations.

```python
from optimum.quanto import quantize, qint8

quantize(model, weights=qint8, activations=qint8)
```

--------------------------------

### Run Specific Tests

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Execute tests for a specific file or module impacted by your changes. This helps in quickly verifying the correctness of your code modifications.

```bash
pytest tests/<TEST_TO_RUN>.py
```

--------------------------------

### Quantize Hugging Face Causal LM Model

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Quantize a Hugging Face AutoModelForCausalLM using QuantizedModelForCausalLM. The weights will be frozen by default. Use `optimum.quanto.quantize` directly if you need to keep weights unfrozen for training.

```python
from transformers import AutoModelForCausalLM
from optimum.quanto import QuantizedModelForCausalLM, qint4

model = AutoModelForCausalLM.from_pretrained('meta-llama/Meta-Llama-3-8B')
qmodel = QuantizedModelForCausalLM.quantize(model, weights=qint4, exclude='lm_head')
```

--------------------------------

### Save Quantized Hugging Face Model

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Save the quantized Hugging Face model using the `save_pretrained` method. This allows for easy reloading later.

```python
qmodel.save_pretrained('./Llama-3-8B-quantized')
```

--------------------------------

### Freeze Quantized Integer Weights

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Replaces the model's float weights with their quantized integer equivalents. This step is necessary after quantization and calibration/tuning.

```python
from optimum.quanto import freeze

freeze(model)
```

--------------------------------

### Calibrate Activation Ranges

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Enables calibration mode to record activation ranges by passing representative samples through the quantized model. This automatically activates the quantization of activations in quantized modules.

```python
from optimum.quanto import Calibration

with Calibration(momentum=0.9):
    model(samples)
```

--------------------------------

### Run Specific Tests with Pytest

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Execute specific tests by providing the path to a subfolder or test file within the tests directory. Ensure you are in the repository root.

```bash
python -m pytest -sv ./tests/<subfolder>/<test>.py
```

--------------------------------

### Reload Quantized Hugging Face Model

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Reload a previously saved quantized Hugging Face model using `QuantizedModelForCausalLM.from_pretrained`. Ensure the path points to the directory where the model was saved.

```python
from optimum.quanto import QuantizedModelForCausalLM

qmodel = QuantizedModelForCausalLM.from_pretrained('Llama-3-8B-quantized')
```

--------------------------------

### Create a New Development Branch

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Create a new branch for your development changes. It is crucial to avoid working directly on the main branch to maintain a clean commit history.

```bash
git checkout -b a-descriptive-name-for-my-changes
```

--------------------------------

### Serialize Quantization Map

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Stores the quantization map of the model to a JSON file. This map is required to reload the quantized model's state.

```python
import json

from optimum.quanto import quantization_map

with open('quantization_map.json', 'w') as f:
  json.dump(quantization_map(model), f)
```

--------------------------------

### Push Changes to Remote Branch

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Push your local branch changes to your origin fork. Use the '-u' flag for the first push to set the upstream tracking reference. Force pushing might be necessary if the branch already has a pull request.

```bash
git push -u origin a-descriptive-name-for-my-changes
```

--------------------------------

### Rebase Branch on Upstream

Source: https://github.com/huggingface/optimum-quanto/blob/main/CONTRIBUTING.md

Fetch the latest changes from the upstream repository and rebase your current branch onto the main branch. This ensures your branch is up-to-date before submitting a pull request.

```bash
git fetch upstream
git rebase upstream/main
```

--------------------------------

### Reload a Quantized Model

Source: https://github.com/huggingface/optimum-quanto/blob/main/README.md

Reloads a serialized quantized model from its state dictionary and quantization map. An empty model must be instantiated first, and then requantized using the saved state.

```python
import json

from safetensors.torch import load_file
from optimum.quanto import requantize

state_dict = load_file('model.safetensors')
with open('quantization_map.json', 'r') as f:
  quantization_map = json.load(f)

# Create an empty model from your modeling code and requantize it
with torch.device('meta'):
  new_model = ...
requantize(new_model, state_dict, quantization_map, device=torch.device('cuda'))
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.