IQA-PyTorch

https://iqa-pytorch.readthedocs.io/
Admin
IQA-PyTorch is an image quality assessment toolbox built with Python and PyTorch that provides...
Tokens:748,469
Snippets:4,149
Trust Score:9.6
Update:1 week ago
Show doc for...
Tokens:
Raw
### Install local checkout of pyiqa in editable mode

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/installation.rst.txt

Clones the repository, changes into the directory, and performs an editable install. This installs the pyiqa and pyiqa-gmad command-line entry points. To build a wheel and source distribution, install the build package and run python -m build from the repository root.

```bash
git clone https://github.com/chaofengc/IQA-PyTorch.git
cd IQA-PyTorch
pip install -e .
```

--------------------------------

### Example: Prepare prompt and generate response

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/q_align/modeling_llama2/index.html

Shows how to tokenize a prompt and generate a response with the model. The output is decoded and printed.

```python
>>> prompt = "Hey, are you conscious? Can you talk to me?"
>>> inputs = tokenizer(prompt, return_tensors="pt")
```

```python
>>> # Generate
>>> generate_ids = model.generate(inputs.input_ids, max_length=30)
>>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
"Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you."
```

--------------------------------

### Install pyiqa with pip

Source: https://iqa-pytorch.readthedocs.io/en/latest/installation.html

Installs the latest release of pyiqa from PyPI using pip.

```bash
pip install pyiqa

```

--------------------------------

### Imports and path setup for pyiqa.archs.q_align.modeling_mplug_owl2

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/q_align/modeling_mplug_owl2.html

The source code begins with the Apache 2.0 license header for the mPLUG-Owl2 implementation, followed by the module docstring and standard library imports for the multimodal model, then the file path setup and transformer model imports. These blocks set up the environment and required dependencies for the rest of the file.

```python
    #    Copyright 2023 Haotian Liu & Qinghao Ye & Haoning Wu (Modified from LLaVA, and mPLUG-Owl2)
#
#    Licensed under the Apache License, Version 2.0 (the "License");
#    you may not use this file except in compliance with the License.
#    You may obtain a copy of the License at
#
#        http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
```

```python
"""mPLUG-Owl2 multimodal causal language model and image-scoring interface."""

from abc import ABC, abstractmethod
from typing import List, Optional, Tuple, Union

import torch
import torch.nn as nn
from torch.nn import CrossEntropyLoss

import copy
import os
import sys
```

```python
dir_path = os.path.dirname(os.path.realpath(__file__))


sys.path.insert(0, dir_path)

from transformers import AutoConfig, AutoModelForCausalLM, CLIPImageProcessor, LlamaConfig, LlamaModel, LlamaForCausalLM
from transformers.models.llama import LlamaTokenizer
from transformers.modeling_outputs import CausalLMOutputWithPast

from .configuration_mplug_owl2 import MPLUGOwl2Config, MplugOwlVisionConfig, MplugOwlVisualAbstractorConfig
from .visual_encoder import MplugOwlVisionModel, MplugOwlVisualAbstractorModel
from .modeling_llama2 import replace_llama_modality_adaptive
from .modeling_attn_mask_utils import _prepare_4d_causal_attention_mask
```

--------------------------------

### Qwen3_5Config class definition with docstring and usage example

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5.html

Defines the Qwen3_5Config class that stores text and vision configurations, with a docstring explaining the model architecture and a Python usage example showing how to initialize the configuration and model.

```python
[docs]
class Qwen3_5Config(PreTrainedConfig):
    r"""
    This is the configuration class to store the configuration of a [`Qwen3_5Model`]. It is used to instantiate a
    Qwen3.5 model according to the specified arguments, defining the model architecture. Instantiating a configuration
    with the defaults will yield a similar configuration to that of
    Qwen3.5-9B-Instruct [Qwen/Qwen3.5-9B-Instruct](https://huggingface.co/Qwen/Qwen3.5-9B-Instruct).

    Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the
    documentation from [`PreTrainedConfig`] for more information.


    Args:
        text_config (`Union[PreTrainedConfig, dict]`, *optional*, defaults to `Qwen3_5TextConfig`):
            The config object or dictionary of the text backbone.
        vision_config (`Union[PreTrainedConfig, dict]`,  *optional*, defaults to `Qwen3_5VisionConfig`):
            The config object or dictionary of the vision backbone.
        image_token_id (`int`, *optional*, defaults to 248056):
            The image token index to encode the image prompt.
        video_token_id (`int`, *optional*, defaults to 248057):
            The video token index to encode the image prompt.
        vision_start_token_id (`int`, *optional*, defaults to 248053):
            The start token index to encode the image prompt.
        vision_end_token_id (`int`, *optional*, defaults to 248054):
            The end token index to encode the image prompt.
        tie_word_embeddings (`bool`, *optional*, defaults to `False`):
            Whether to tie the word embeddings.

    ```python
    >>> from transformers import Qwen3_5ForConditionalGeneration, Qwen3_5Config

    >>> # Initializing a Qwen3.5 style configuration
    >>> configuration = Qwen3_5Config()

    >>> # Initializing a model from the Qwen3.5-9B style configuration
    >>> model = Qwen3_5ForConditionalGeneration(configuration)
    """
```

--------------------------------

### Initialize Qwen3.5 configuration

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html

Create a Qwen3_5Config instance with default settings.

```python
>>> # Initializing a Qwen3.5 style configuration
>>> configuration = Qwen3_5Config()
```

--------------------------------

### Install latest GitHub version of pyiqa with pip

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/installation.rst.txt

Installs the latest development version directly from the GitHub repository (chaofengc/IQA-PyTorch). Requires git and a compatible PyTorch/torchvision setup.

```bash
pip install git+https://github.com/chaofengc/IQA-PyTorch.git
```

--------------------------------

### Install pyiqa from PyPI with pip

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/installation.rst.txt

Installs the latest released version of the pyiqa package from PyPI. Requires Python 3.8 or later and a compatible PyTorch/torchvision installation.

```bash
pip install pyiqa
```

--------------------------------

### Initialize Qwen3_5Config and model

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.rst.txt

Shows how to create a Qwen3_5Config instance and use it to initialize a Qwen3_5ForConditionalGeneration model. The example also demonstrates accessing the model's configuration.

```python
>>> from transformers import Qwen3_5ForConditionalGeneration, Qwen3_5Config

>>> # Initializing a Qwen3.5 style configuration
>>> configuration = Qwen3_5Config()

>>> # Initializing a model from the Qwen3.5-9B style configuration
>>> model = Qwen3_5ForConditionalGeneration(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config
```

--------------------------------

### Initialize model from configuration

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html

Create a Qwen3_5ForConditionalGeneration model using the configuration.

```python
>>> # Initializing a model from the Qwen3.5-9B style configuration
>>> model = Qwen3_5ForConditionalGeneration(configuration)
```

--------------------------------

### Initialize Qwen3.5 Text Model and Configuration

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5.html

Shows how to create a Qwen3.5TextConfig, instantiate a Qwen3.5TextModel with it, and access the model's configuration. Requires the transformers library.

```python
>>> from transformers import Qwen3_5TextModel, Qwen3_5TextConfig

>>> # Initializing a Qwen3.5 style configuration
>>> configuration =  Qwen3_5TextConfig()

>>> # Initializing a model from the Qwen3.5-9B style configuration
>>> model = Qwen3_5TextModel(configuration)

>>> # Accessing the model configuration
>>> configuration = model.config
```

--------------------------------

### Install latest GitHub version with pip

Source: https://iqa-pytorch.readthedocs.io/en/latest/installation.html

Installs the latest development version directly from the GitHub repository.

```bash
pip install git+https://github.com/chaofengc/IQA-PyTorch.git

```

--------------------------------

### Initialize Qwen3_5TextModel from configuration

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html

Creates a Qwen3_5TextModel instance using the previously created configuration.

```python
>>> # Initializing a model from the Qwen3.5-9B style configuration
>>> model = Qwen3_5TextModel(configuration)
```

--------------------------------

### Accessing model configuration (docstring example)

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5.html

Shows how to access the model configuration via the `model.config` attribute. This is a usage example from the class docstring.

```python
>>> # Accessing the model configuration
>>> configuration = model.config
```

--------------------------------

### init_dist(launcher, backend='nccl', **kwargs)

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/utils/dist_util/index.rst.txt

Initializes the distributed training environment with the specified launcher and backend.

```APIDOC
## init_dist(launcher, backend='nccl', **kwargs)

### Description
Initializes the distributed training environment with the specified launcher and backend.

### Parameters
- **launcher** (str) - Required - The launcher type (e.g., 'pytorch', 'slurm').
- **backend** (str) - Optional - The backend to use (default: 'nccl').
- **kwargs** - Optional - Additional keyword arguments for the launcher.

```

--------------------------------

### Usage example for default_init_weights

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/arch_util.html

Shows how to import and call default_init_weights on a sequential model with convolutional and linear layers. The example sets scale=0.1 and bias_fill=0.01.

```python
>>> import torch.nn as nn
>>> from arch_util import default_init_weights
>>> model = nn.Sequential(
>>>     nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1),
>>>     nn.ReLU(),
>>>     nn.Conv2d(64, 64, kernel_size=3, stride=1, padding=1),
>>>     nn.ReLU(),
>>>     nn.Linear(64 * 32 * 32, 10)
>>> )
>>> default_init_weights(model, scale=0.1, bias_fill=0.01)
```

--------------------------------

### DeepDC usage example

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/deepdc_arch.html

Loads the DeepDC model, moves it and the reference/distorted images to the device, computes the quality score, and prints it. The example output is 0.3347.

```python
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = DeepDC().to(device)
ref = ref.to(device)
dist = dist.to(device)
score = model(ref, dist)
print(score.item())
# score: 0.3347
```

--------------------------------

### Initialize Qwen3_5TextConfig

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html

Creates a Qwen3_5TextConfig instance with default settings.

```python
>>> # Initializing a Qwen3.5 style configuration
>>> configuration =  Qwen3_5TextConfig()
```

--------------------------------

### CLIPScore usage example

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/clipscore_arch/index.rst.txt

Shows how to instantiate CLIPScore and compute scores for a batch of images and captions. The example uses a random tensor for images and a list of captions, and prints the resulting score shape.

```python
>>> metric = CLIPScore(backbone='ViT-B/32')
>>> img = torch.rand(2, 3, 224, 224)
>>> score = metric(img, ['a dog on grass', 'a city street'])
>>> score.shape
torch.Size([2])
```

--------------------------------

### pyiqa_cmd main() function for evaluating image quality metrics

Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/pyiqa_cmd.html

Defines the command-line entry point for pyiqa. Parses arguments for metric names, target/ref paths, and options like --isc_splits, --fid_mode, --device, --verbose, and --list_models. Handles distribution metrics (fid, inception_score) separately, then iterates over per-image metrics, printing results with pprint.

```python
import argparse
from pyiqa.api_helpers import create_metric, list_models
from pyiqa.utils import scandir_images

import os
from tqdm import tqdm
import numpy as np
from datetime import datetime
from pprint import pprint




[docs]
def main():
    """Evaluate one or more metrics from image paths or directories."""
    parser = argparse.ArgumentParser(description='Test image quality metrics')

    # required input arguments
    parser.add_argument('metric', nargs='*', help='Name(s) of metric(s) to evaluate')
    parser.add_argument('-t', '--target', type=str, help='Path to target image file or folder')
    parser.add_argument(
        '-r', '--ref', type=str, default=None, help='Path to reference image file or folder (optional)'
    )

    # metric specific options
    parser.add_argument(
        '--isc_splits', type=int, default=10, help='Number of splits for Inception Score calculation'
    )
    parser.add_argument(
        '--fid_mode',
        type=str,
        default='clean',
        help='Image resize mode for FID: clean, legacy_pytorch, or legacy_tensorflow'
    )

    # common options
    parser.add_argument(
        '--device', type=str, default=None, help='Device to run metrics on (e.g., cpu, cuda)'
    )
    parser.add_argument(
        '-v', '--verbose', action='store_true', help='Enable verbose output'
    )
    parser.add_argument(
        '-ls', '--list_models', action='store_true', help='List available metric models'
    )

    args, unknown_args = parser.parse_known_args()

    if args.list_models:
        pprint(list_models(), compact=True)
        return

    def get_time():
        return datetime.now().strftime('%Y-%m-%d %H:%M:%S')

    print(f'[{get_time()}] ===> Loading metrics: {args.metric}')
    metric_func_list = {}
    for metric in args.metric:
        metric_func = create_metric(metric, device=args.device)
        metric_func_list[metric] = metric_func
    print(f'[{get_time()}] ===> Metrics loaded: {args.metric}')

    results = {}
    # Distribution metrics consume directories/statistics directly rather than
    # producing one score per corresponding image pair.
    # Test fid, inception_score
    if 'fid' in metric_func_list:
        metric_func = metric_func_list.pop('fid')
        result = metric_func(
            args.target, args.ref, mode=args.fid_mode, verbose=args.verbose
        )
        results['fid'] = result
    if 'inception_score' in metric_func_list:
        metric_func = metric_func_list.pop('inception_score')
        result = metric_func(args.target, splits=args.isc_splits, verbose=args.verbose)
        results['inception_score'] = result

    if os.path.isdir(args.target):
        target_list = scandir_images(args.target)
        if args.ref is not None:
            ref_list = scandir_images(args.ref)
            assert len(target_list) == len(ref_list), (
                'Number of images in target and reference folders must be the same'
            )
    else:
        target_list = [args.target]
        if args.ref is not None:
            ref_list = [args.ref]

    for metric, func in metric_func_list.items():
        pbar = tqdm(total=len(target_list), unit='image')

        all_results = {}
        if args.ref is None or func.metric_mode == 'NR':
            tmp_result = []
            for target in target_list:
                score = func(target).item()
                tmp_result.append(score)
                if args.verbose:
                    all_results[target] = score

                pbar.update(1)
                pbar.set_description(
                    f'[{get_time()}] Testing {metric} with input {target:>20}'
                )
        else:
            tmp_result = []
            for target, ref in zip(target_list, ref_list):
                score = func(target, ref).item()
                tmp_result.append(score)
                if args.verbose:
                    all_results[f'{target} | {ref}'] = score

                pbar.update(1)
                pbar.set_description(
                    f'[{get_time()}] Testing {metric} with input {target:>20}'
                )

        pbar.close()
        all_results['mean'] = np.mean(tmp_result)
        results[metric] = all_results

    pprint(results)




if __name__ == '__main__':
    main()

```

--------------------------------

### WaDIQaMModel.setup_optimizers()

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/models/wadiqam_model/index.html

Sets up the optimizers for the WaDIQaMModel. This method is part of the model's training configuration.

```APIDOC
## WaDIQaMModel.setup_optimizers()

### Description
Sets up the optimizers for the WaDIQaMModel. This method is part of the model's training configuration.

### Method
setup_optimizers()

### Parameters
None

### Returns
None

```

--------------------------------

### IQT.get_backbone_feature(x)

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/iqt_arch/index.html

Gets the backbone features for the input tensor.

```APIDOC
## IQT.get_backbone_feature(x)

### Description
Gets the backbone features for the input tensor.

### Method
get_backbone_feature

### Parameters
- **x** (torch.Tensor) - Required - The input tensor.

### Returns
- **torch.Tensor** - The backbone features for the input tensor.
```

--------------------------------

### Prepare multimodal inputs with Qwen3_5ForConditionalGeneration

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/qrealign/qwen3_5_src/modeling_qwen3_5/index.rst.txt

Demonstrates loading the Qwen3_5ForConditionalGeneration model and AutoProcessor, then applying a chat template with an image and text message to prepare inputs for generation. Uses the 'Qwen/Qwen3-VL-8B-Instruct' checkpoint.

```python
>>> from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

>>> model = Qwen3_5ForConditionalGeneration.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")
>>> processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct")

>>> messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg",
            },
            {"type": "text", "text": "Describe the image."},
        ],
    }
]

>>> inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt"
)
```

--------------------------------

### Basic Python API: list models and create metrics

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/examples.rst.txt

Shows how to list configured metrics, create a metric with default configuration, enable gradients for loss use, override architecture options, and read score interpretation metadata. Requires pyiqa and torch.

```python
import pyiqa
import torch

# List configured metrics, including FR, NR, and task-specific metrics.
print(pyiqa.list_models())

device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")

# Create a metric with its default configuration.
lpips = pyiqa.create_metric('lpips', device=device)

# Gradients are disabled by default. Enable them for a supported loss.
lpips_loss = pyiqa.create_metric('lpips', device=device, as_loss=True)

# Override options supported by a metric's architecture.
psnr = pyiqa.create_metric(
    'psnr', device=device, test_y_channel=True, color_space='ycbcr'
)

# Read score interpretation metadata.
print(lpips.lower_better)
```

--------------------------------

### get_env_info()

Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/utils/logger/index.html

Gets environment information. Currently, only logs the software version.

```APIDOC
## get_env_info()

### Description
Gets environment information. Currently, only logs the software version.

```

--------------------------------

### Generate text with Qwen3_5ForCausalLM

Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/qrealign/qwen3_5_src/modeling_qwen3_5/index.rst.txt

Shows how to load the Qwen3_5ForCausalLM model and tokenizer, tokenize a prompt, generate text, and decode the output. The example uses the 'Qwen/Qwen3_5-8B' checkpoint and prints the model's response.

```python
>>> from transformers import AutoTokenizer, Qwen3_5ForCausalLM

>>> model = Qwen3_5ForCausalLM.from_pretrained("Qwen/Qwen3_5-8B")
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3_5-8B")

>>> prompt = "Hey, are you conscious? Can you talk to me?"
>>> inputs = tokenizer(prompt, return_tensors="pt")

>>> # Generate
>>> generate_ids = model.generate(inputs.input_ids, max_length=30)
>>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
"Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you."
```
Tokens:
Raw