### Install local checkout of pyiqa in editable mode Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/installation.rst.txt Clones the repository, changes into the directory, and performs an editable install. This installs the pyiqa and pyiqa-gmad command-line entry points. To build a wheel and source distribution, install the build package and run python -m build from the repository root. ```bash git clone https://github.com/chaofengc/IQA-PyTorch.git cd IQA-PyTorch pip install -e . ``` -------------------------------- ### Example: Prepare prompt and generate response Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/q_align/modeling_llama2/index.html Shows how to tokenize a prompt and generate a response with the model. The output is decoded and printed. ```python >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") ``` ```python >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ``` -------------------------------- ### Install pyiqa with pip Source: https://iqa-pytorch.readthedocs.io/en/latest/installation.html Installs the latest release of pyiqa from PyPI using pip. ```bash pip install pyiqa ``` -------------------------------- ### Imports and path setup for pyiqa.archs.q_align.modeling_mplug_owl2 Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/q_align/modeling_mplug_owl2.html The source code begins with the Apache 2.0 license header for the mPLUG-Owl2 implementation, followed by the module docstring and standard library imports for the multimodal model, then the file path setup and transformer model imports. These blocks set up the environment and required dependencies for the rest of the file. ```python # Copyright 2023 Haotian Liu & Qinghao Ye & Haoning Wu (Modified from LLaVA, and mPLUG-Owl2) # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. ``` ```python """mPLUG-Owl2 multimodal causal language model and image-scoring interface.""" from abc import ABC, abstractmethod from typing import List, Optional, Tuple, Union import torch import torch.nn as nn from torch.nn import CrossEntropyLoss import copy import os import sys ``` ```python dir_path = os.path.dirname(os.path.realpath(__file__)) sys.path.insert(0, dir_path) from transformers import AutoConfig, AutoModelForCausalLM, CLIPImageProcessor, LlamaConfig, LlamaModel, LlamaForCausalLM from transformers.models.llama import LlamaTokenizer from transformers.modeling_outputs import CausalLMOutputWithPast from .configuration_mplug_owl2 import MPLUGOwl2Config, MplugOwlVisionConfig, MplugOwlVisualAbstractorConfig from .visual_encoder import MplugOwlVisionModel, MplugOwlVisualAbstractorModel from .modeling_llama2 import replace_llama_modality_adaptive from .modeling_attn_mask_utils import _prepare_4d_causal_attention_mask ``` -------------------------------- ### Qwen3_5Config class definition with docstring and usage example Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5.html Defines the Qwen3_5Config class that stores text and vision configurations, with a docstring explaining the model architecture and a Python usage example showing how to initialize the configuration and model. ```python [docs] class Qwen3_5Config(PreTrainedConfig): r""" This is the configuration class to store the configuration of a [`Qwen3_5Model`]. It is used to instantiate a Qwen3.5 model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of Qwen3.5-9B-Instruct [Qwen/Qwen3.5-9B-Instruct](https://huggingface.co/Qwen/Qwen3.5-9B-Instruct). Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: text_config (`Union[PreTrainedConfig, dict]`, *optional*, defaults to `Qwen3_5TextConfig`): The config object or dictionary of the text backbone. vision_config (`Union[PreTrainedConfig, dict]`, *optional*, defaults to `Qwen3_5VisionConfig`): The config object or dictionary of the vision backbone. image_token_id (`int`, *optional*, defaults to 248056): The image token index to encode the image prompt. video_token_id (`int`, *optional*, defaults to 248057): The video token index to encode the image prompt. vision_start_token_id (`int`, *optional*, defaults to 248053): The start token index to encode the image prompt. vision_end_token_id (`int`, *optional*, defaults to 248054): The end token index to encode the image prompt. tie_word_embeddings (`bool`, *optional*, defaults to `False`): Whether to tie the word embeddings. ```python >>> from transformers import Qwen3_5ForConditionalGeneration, Qwen3_5Config >>> # Initializing a Qwen3.5 style configuration >>> configuration = Qwen3_5Config() >>> # Initializing a model from the Qwen3.5-9B style configuration >>> model = Qwen3_5ForConditionalGeneration(configuration) """ ``` -------------------------------- ### Initialize Qwen3.5 configuration Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html Create a Qwen3_5Config instance with default settings. ```python >>> # Initializing a Qwen3.5 style configuration >>> configuration = Qwen3_5Config() ``` -------------------------------- ### Install latest GitHub version of pyiqa with pip Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/installation.rst.txt Installs the latest development version directly from the GitHub repository (chaofengc/IQA-PyTorch). Requires git and a compatible PyTorch/torchvision setup. ```bash pip install git+https://github.com/chaofengc/IQA-PyTorch.git ``` -------------------------------- ### Install pyiqa from PyPI with pip Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/installation.rst.txt Installs the latest released version of the pyiqa package from PyPI. Requires Python 3.8 or later and a compatible PyTorch/torchvision installation. ```bash pip install pyiqa ``` -------------------------------- ### Initialize Qwen3_5Config and model Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.rst.txt Shows how to create a Qwen3_5Config instance and use it to initialize a Qwen3_5ForConditionalGeneration model. The example also demonstrates accessing the model's configuration. ```python >>> from transformers import Qwen3_5ForConditionalGeneration, Qwen3_5Config >>> # Initializing a Qwen3.5 style configuration >>> configuration = Qwen3_5Config() >>> # Initializing a model from the Qwen3.5-9B style configuration >>> model = Qwen3_5ForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` -------------------------------- ### Initialize model from configuration Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html Create a Qwen3_5ForConditionalGeneration model using the configuration. ```python >>> # Initializing a model from the Qwen3.5-9B style configuration >>> model = Qwen3_5ForConditionalGeneration(configuration) ``` -------------------------------- ### Initialize Qwen3.5 Text Model and Configuration Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5.html Shows how to create a Qwen3.5TextConfig, instantiate a Qwen3.5TextModel with it, and access the model's configuration. Requires the transformers library. ```python >>> from transformers import Qwen3_5TextModel, Qwen3_5TextConfig >>> # Initializing a Qwen3.5 style configuration >>> configuration = Qwen3_5TextConfig() >>> # Initializing a model from the Qwen3.5-9B style configuration >>> model = Qwen3_5TextModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` -------------------------------- ### Install latest GitHub version with pip Source: https://iqa-pytorch.readthedocs.io/en/latest/installation.html Installs the latest development version directly from the GitHub repository. ```bash pip install git+https://github.com/chaofengc/IQA-PyTorch.git ``` -------------------------------- ### Accessing model configuration (docstring example) Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5.html Shows how to access the model configuration via the `model.config` attribute. This is a usage example from the class docstring. ```python >>> # Accessing the model configuration >>> configuration = model.config ``` -------------------------------- ### init_dist(launcher, backend='nccl', **kwargs) Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/utils/dist_util/index.rst.txt Initializes the distributed training environment with the specified launcher and backend. ```APIDOC ## init_dist(launcher, backend='nccl', **kwargs) ### Description Initializes the distributed training environment with the specified launcher and backend. ### Parameters - **launcher** (str) - Required - The launcher type (e.g., 'pytorch', 'slurm'). - **backend** (str) - Optional - The backend to use (default: 'nccl'). - **kwargs** - Optional - Additional keyword arguments for the launcher. ``` -------------------------------- ### Initialize Qwen3_5TextModel from configuration Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html Creates a Qwen3_5TextModel instance using the previously created configuration. ```python >>> # Initializing a model from the Qwen3.5-9B style configuration >>> model = Qwen3_5TextModel(configuration) ``` -------------------------------- ### Usage example for default_init_weights Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/arch_util.html Shows how to import and call default_init_weights on a sequential model with convolutional and linear layers. The example sets scale=0.1 and bias_fill=0.01. ```python >>> import torch.nn as nn >>> from arch_util import default_init_weights >>> model = nn.Sequential( >>> nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1), >>> nn.ReLU(), >>> nn.Conv2d(64, 64, kernel_size=3, stride=1, padding=1), >>> nn.ReLU(), >>> nn.Linear(64 * 32 * 32, 10) >>> ) >>> default_init_weights(model, scale=0.1, bias_fill=0.01) ``` -------------------------------- ### DeepDC usage example Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/archs/deepdc_arch.html Loads the DeepDC model, moves it and the reference/distorted images to the device, computes the quality score, and prints it. The example output is 0.3347. ```python device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') model = DeepDC().to(device) ref = ref.to(device) dist = dist.to(device) score = model(ref, dist) print(score.item()) # score: 0.3347 ``` -------------------------------- ### Initialize Qwen3_5TextConfig Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/qrealign/qwen3_5_src/configuration_qwen3_5/index.html Creates a Qwen3_5TextConfig instance with default settings. ```python >>> # Initializing a Qwen3.5 style configuration >>> configuration = Qwen3_5TextConfig() ``` -------------------------------- ### CLIPScore usage example Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/clipscore_arch/index.rst.txt Shows how to instantiate CLIPScore and compute scores for a batch of images and captions. The example uses a random tensor for images and a list of captions, and prints the resulting score shape. ```python >>> metric = CLIPScore(backbone='ViT-B/32') >>> img = torch.rand(2, 3, 224, 224) >>> score = metric(img, ['a dog on grass', 'a city street']) >>> score.shape torch.Size([2]) ``` -------------------------------- ### pyiqa_cmd main() function for evaluating image quality metrics Source: https://iqa-pytorch.readthedocs.io/en/latest/_modules/pyiqa/pyiqa_cmd.html Defines the command-line entry point for pyiqa. Parses arguments for metric names, target/ref paths, and options like --isc_splits, --fid_mode, --device, --verbose, and --list_models. Handles distribution metrics (fid, inception_score) separately, then iterates over per-image metrics, printing results with pprint. ```python import argparse from pyiqa.api_helpers import create_metric, list_models from pyiqa.utils import scandir_images import os from tqdm import tqdm import numpy as np from datetime import datetime from pprint import pprint [docs] def main(): """Evaluate one or more metrics from image paths or directories.""" parser = argparse.ArgumentParser(description='Test image quality metrics') # required input arguments parser.add_argument('metric', nargs='*', help='Name(s) of metric(s) to evaluate') parser.add_argument('-t', '--target', type=str, help='Path to target image file or folder') parser.add_argument( '-r', '--ref', type=str, default=None, help='Path to reference image file or folder (optional)' ) # metric specific options parser.add_argument( '--isc_splits', type=int, default=10, help='Number of splits for Inception Score calculation' ) parser.add_argument( '--fid_mode', type=str, default='clean', help='Image resize mode for FID: clean, legacy_pytorch, or legacy_tensorflow' ) # common options parser.add_argument( '--device', type=str, default=None, help='Device to run metrics on (e.g., cpu, cuda)' ) parser.add_argument( '-v', '--verbose', action='store_true', help='Enable verbose output' ) parser.add_argument( '-ls', '--list_models', action='store_true', help='List available metric models' ) args, unknown_args = parser.parse_known_args() if args.list_models: pprint(list_models(), compact=True) return def get_time(): return datetime.now().strftime('%Y-%m-%d %H:%M:%S') print(f'[{get_time()}] ===> Loading metrics: {args.metric}') metric_func_list = {} for metric in args.metric: metric_func = create_metric(metric, device=args.device) metric_func_list[metric] = metric_func print(f'[{get_time()}] ===> Metrics loaded: {args.metric}') results = {} # Distribution metrics consume directories/statistics directly rather than # producing one score per corresponding image pair. # Test fid, inception_score if 'fid' in metric_func_list: metric_func = metric_func_list.pop('fid') result = metric_func( args.target, args.ref, mode=args.fid_mode, verbose=args.verbose ) results['fid'] = result if 'inception_score' in metric_func_list: metric_func = metric_func_list.pop('inception_score') result = metric_func(args.target, splits=args.isc_splits, verbose=args.verbose) results['inception_score'] = result if os.path.isdir(args.target): target_list = scandir_images(args.target) if args.ref is not None: ref_list = scandir_images(args.ref) assert len(target_list) == len(ref_list), ( 'Number of images in target and reference folders must be the same' ) else: target_list = [args.target] if args.ref is not None: ref_list = [args.ref] for metric, func in metric_func_list.items(): pbar = tqdm(total=len(target_list), unit='image') all_results = {} if args.ref is None or func.metric_mode == 'NR': tmp_result = [] for target in target_list: score = func(target).item() tmp_result.append(score) if args.verbose: all_results[target] = score pbar.update(1) pbar.set_description( f'[{get_time()}] Testing {metric} with input {target:>20}' ) else: tmp_result = [] for target, ref in zip(target_list, ref_list): score = func(target, ref).item() tmp_result.append(score) if args.verbose: all_results[f'{target} | {ref}'] = score pbar.update(1) pbar.set_description( f'[{get_time()}] Testing {metric} with input {target:>20}' ) pbar.close() all_results['mean'] = np.mean(tmp_result) results[metric] = all_results pprint(results) if __name__ == '__main__': main() ``` -------------------------------- ### WaDIQaMModel.setup_optimizers() Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/models/wadiqam_model/index.html Sets up the optimizers for the WaDIQaMModel. This method is part of the model's training configuration. ```APIDOC ## WaDIQaMModel.setup_optimizers() ### Description Sets up the optimizers for the WaDIQaMModel. This method is part of the model's training configuration. ### Method setup_optimizers() ### Parameters None ### Returns None ``` -------------------------------- ### IQT.get_backbone_feature(x) Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/archs/iqt_arch/index.html Gets the backbone features for the input tensor. ```APIDOC ## IQT.get_backbone_feature(x) ### Description Gets the backbone features for the input tensor. ### Method get_backbone_feature ### Parameters - **x** (torch.Tensor) - Required - The input tensor. ### Returns - **torch.Tensor** - The backbone features for the input tensor. ``` -------------------------------- ### Prepare multimodal inputs with Qwen3_5ForConditionalGeneration Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/qrealign/qwen3_5_src/modeling_qwen3_5/index.rst.txt Demonstrates loading the Qwen3_5ForConditionalGeneration model and AutoProcessor, then applying a chat template with an image and text message to prepare inputs for generation. Uses the 'Qwen/Qwen3-VL-8B-Instruct' checkpoint. ```python >>> from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration >>> model = Qwen3_5ForConditionalGeneration.from_pretrained("Qwen/Qwen3-VL-8B-Instruct") >>> processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-8B-Instruct") >>> messages = [ { "role": "user", "content": [ { "type": "image", "image": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg", }, {"type": "text", "text": "Describe the image."}, ], } ] >>> inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt" ) ``` -------------------------------- ### Basic Python API: list models and create metrics Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/examples.rst.txt Shows how to list configured metrics, create a metric with default configuration, enable gradients for loss use, override architecture options, and read score interpretation metadata. Requires pyiqa and torch. ```python import pyiqa import torch # List configured metrics, including FR, NR, and task-specific metrics. print(pyiqa.list_models()) device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu") # Create a metric with its default configuration. lpips = pyiqa.create_metric('lpips', device=device) # Gradients are disabled by default. Enable them for a supported loss. lpips_loss = pyiqa.create_metric('lpips', device=device, as_loss=True) # Override options supported by a metric's architecture. psnr = pyiqa.create_metric( 'psnr', device=device, test_y_channel=True, color_space='ycbcr' ) # Read score interpretation metadata. print(lpips.lower_better) ``` -------------------------------- ### get_env_info() Source: https://iqa-pytorch.readthedocs.io/en/latest/autoapi/pyiqa/utils/logger/index.html Gets environment information. Currently, only logs the software version. ```APIDOC ## get_env_info() ### Description Gets environment information. Currently, only logs the software version. ``` -------------------------------- ### Generate text with Qwen3_5ForCausalLM Source: https://iqa-pytorch.readthedocs.io/en/latest/_sources/autoapi/pyiqa/archs/qrealign/qwen3_5_src/modeling_qwen3_5/index.rst.txt Shows how to load the Qwen3_5ForCausalLM model and tokenizer, tokenize a prompt, generate text, and decode the output. The example uses the 'Qwen/Qwen3_5-8B' checkpoint and prints the model's response. ```python >>> from transformers import AutoTokenizer, Qwen3_5ForCausalLM >>> model = Qwen3_5ForCausalLM.from_pretrained("Qwen/Qwen3_5-8B") >>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3_5-8B") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```