### Install ImageReward Package

Source: https://github.com/thudm/imagereward/blob/main/README.md

Instructions for cloning the repository and installing the required Python package.

```shell
git clone https://github.com/THUDM/ImageReward.git
cd ImageReward
pip install image-reward
```

--------------------------------

### Command Line Training Scripts and Dependencies

Source: https://context7.com/thudm/imagereward/llms.txt

Provides bash scripts and `accelerate` commands for running ReFL fine-tuning directly from the command line. It includes installation instructions for necessary dependencies like `image-reward`, `diffusers`, and `accelerate`, along with example commands for training both standard Stable Diffusion and SDXL models. The expected format for training data (JSON) is also illustrated.

```bash
# Install dependencies for ReFL training
pip install image-reward
pip install diffusers==0.16.0 accelerate==0.16.0 datasets==2.11.0

# Train Stable Diffusion v1.4 with ReFL
bash scripts/train_refl.sh

# Or run directly with accelerate
accelerate launch refl.py \
    --output_dir="checkpoint/refl" \
    --train_batch_size=2 \
    --gradient_accumulation_steps=4 \
    --learning_rate=1e-5 \
    --max_train_steps=100 \
    --checkpointing_steps=100

# Train SDXL with ReFL
bash scripts/train_refl_sdxl.sh

# Training data format (JSON):
# [
#   {"text": "a photo of a cat sitting on a couch"},
#   {"text": "a beautiful landscape with mountains and rivers"},
#   ...
# ]
```

--------------------------------

### Install ReFL dependencies

Source: https://github.com/thudm/imagereward/blob/main/README.md

Installs the necessary Python packages required to run the ReFL training pipeline.

```shell
pip install diffusers==0.16.0 accelerate==0.16.0 datasets==2.11.0
```

--------------------------------

### Run ReFL training

Source: https://github.com/thudm/imagereward/blob/main/README.md

Executes the shell script to start the ReFL training process using the provided dataset.

```shell
bash scripts/train_refl.sh
```

--------------------------------

### Prepare ImageReward training dataset

Source: https://github.com/thudm/imagereward/blob/main/README.md

Prepares the dataset for ImageReward training by running the dataset creation script.

```shell
cd train
python src/make_dataset.py
```

--------------------------------

### POST /load

Source: https://context7.com/thudm/imagereward/llms.txt

Initializes the ImageReward model from a pretrained checkpoint.

```APIDOC
## POST /load

### Description
Initializes the ImageReward model from a pretrained checkpoint, automatically downloading weights from Hugging Face if not cached locally.

### Method
POST

### Endpoint
RM.load(model_name, device, download_root)

### Parameters
#### Path Parameters
- **model_name** (string) - Required - The name of the model (e.g., "ImageReward-v1.0") or path to a local .pt file.

#### Query Parameters
- **device** (string) - Optional - The device to load the model on (e.g., "cuda:0", "cpu").
- **download_root** (string) - Optional - Custom directory path for downloading weights.

### Request Example
model = RM.load("ImageReward-v1.0", device="cuda:0")

### Response
#### Success Response (200)
- **model** (object) - The initialized ImageReward model instance.
```

--------------------------------

### Score Images with ImageReward

Source: https://github.com/thudm/imagereward/blob/main/README.md

Demonstrates how to load the ImageReward model and calculate preference scores for a list of images given a prompt.

```python
import ImageReward as RM
model = RM.load("ImageReward-v1.0")
rewards = model.score("<prompt>", ["<img1_obj_or_path>", "<img2_obj_or_path>", ...])
```

--------------------------------

### Fine-tune Diffusion Models with ReFL

Source: https://github.com/thudm/imagereward/blob/main/README.md

Shows how to initialize the ReFL trainer to optimize a Stable Diffusion model using ImageReward feedback.

```python
from ImageReward import ReFL
args = ReFL.parse_args()
trainer = ReFL.Trainer("CompVis/stable-diffusion-v1-4", "data/refl_data.json", args=args)
trainer.train(args=args)
```

--------------------------------

### Run Benchmark Tests with Bash

Source: https://context7.com/thudm/imagereward/llms.txt

Scripts to evaluate ImageReward and baseline models on benchmark datasets for ranking accuracy against human preferences. These scripts help in measuring the performance of different models.

```bash
bash scripts/test-benchmark.sh
```

```bash
bash scripts/test.sh
```

--------------------------------

### Calculate human preference scores with ImageReward

Source: https://github.com/thudm/imagereward/blob/main/README.md

Demonstrates how to load the ImageReward model and calculate preference scores for a set of images given a text prompt. It uses the ImageReward Python library and PyTorch to perform inference.

```python
import os
import torch
import ImageReward as RM

if __name__ == "__main__":
    prompt = "a painting of an ocean with clouds and birds, day time, low depth field effect"
    img_prefix = "assets/images"
    generations = [f"{pic_id}.webp" for pic_id in range(1, 5)]
    img_list = [os.path.join(img_prefix, img) for img in generations]
    model = RM.load("ImageReward-v1.0")
    with torch.no_grad():
        ranking, rewards = model.inference_rank(prompt, img_list)
        print("\nPreference predictions:\n")
        print(f"ranking = {ranking}")
        print(f"rewards = {rewards}")
        for index in range(len(img_list)):
            score = model.score(prompt, img_list[index])
            print(f"{generations[index]:>16s}: {score:.2f}")
```

--------------------------------

### ReFL Training with LoRA for SDXL

Source: https://context7.com/thudm/imagereward/llms.txt

Enables efficient fine-tuning of SDXL models using Low-Rank Adaptation (LoRA) to significantly reduce memory requirements. This module allows for LoRA-specific argument parsing and trainer initialization. LoRA weights are saved separately and can be loaded into any SDXL pipeline.

```python
from ImageReward import ReFL_SDXL_LoRA

# Parse LoRA-specific training arguments
args = ReFL_SDXL_LoRA.parse_args()

# Initialize LoRA trainer
trainer = ReFL_SDXL_LoRA.Trainer(
    pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0",
    train_data_dir="data/refl_data.json",
    args=args
)

# Start LoRA fine-tuning
trainer.train(args=args)

# Key LoRA-specific arguments:
# --rank: LoRA rank dimension (default: 4)
# --validation_prompts: Prompts for validation during training
# --checkpoints_total_limit: Maximum checkpoints to keep (default: 10)

# LoRA weights are saved separately and can be loaded into any SDXL pipeline:
from diffusers import StableDiffusionXLPipeline

pipeline = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0"
)
pipeline.load_lora_weights("output_dir/pytorch_lora_weights.safetensors")
```

--------------------------------

### Load ImageReward Model

Source: https://context7.com/thudm/imagereward/llms.txt

Initializes the ImageReward model from a pretrained checkpoint. It automatically downloads weights from Hugging Face if not cached locally and supports specifying the device and download directory. It can also load from a local checkpoint file.

```python
import ImageReward as RM
import torch

# Load model with automatic device selection (CUDA if available, else CPU)
model = RM.load("ImageReward-v1.0")

# Load model on specific device
model = RM.load("ImageReward-v1.0", device="cuda:0")

# Load model with custom download directory
model = RM.load(
    "ImageReward-v1.0",
    device="cuda",
    download_root="/path/to/cache"
)

# Load from local checkpoint file
model = RM.load("/path/to/ImageReward.pt", device="cuda")

# List available models
available = RM.available_models()
print(available)  # ['ImageReward-v1.0']
```

--------------------------------

### ReFL Training for Stable Diffusion XL (SDXL)

Source: https://context7.com/thudm/imagereward/llms.txt

Extends ReFL training to Stable Diffusion XL models for higher resolution image generation. This module requires parsing SDXL-specific arguments and initializes a trainer with the SDXL base model. Training can enable both standard diffusion pretrain loss and ImageReward-based loss.

```python
from ImageReward import ReFL_SDXL

# Parse SDXL-specific training arguments
args = ReFL_SDXL.parse_args()

# Initialize SDXL trainer
trainer = ReFL_SDXL.Trainer(
    pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0",
    train_data_dir="data/refl_data.json",
    args=args
)

# Start training with both pretrain and reward losses
trainer.train(args=args)

# Key training arguments:
# --apply_pre_loss: Enable standard diffusion pretrain loss
# --apply_reward_loss: Enable ImageReward-based loss
# --grad_scale: Scale factor for gradient (default: 1e-3)
# --resolution: Image resolution (default: 1024 for SDXL)
# --train_batch_size: Batch size per device
# --learning_rate: Learning rate (default: 1e-4)
# --max_train_steps: Total training steps
```

--------------------------------

### Run Benchmark Tests with Python

Source: https://context7.com/thudm/imagereward/llms.txt

Direct Python script execution for running benchmark tests. Allows specifying source data, target directories, image prefixes, and model types for evaluation.

```python
python test.py \
    --source_path data/test.json \
    --target_dir data/ \
    --img_prefix data/test_images \
    --model_type all  # Options: ImageReward-v1.0, CLIP, BLIP, Aesthetic, all
```

--------------------------------

### Train ImageReward model

Source: https://github.com/thudm/imagereward/blob/main/README.md

Executes the training script to train the ImageReward model on a single node.

```shell
bash scripts/train_one_node.sh
```

--------------------------------

### Differentiable Reward Computation with ImageReward

Source: https://context7.com/thudm/imagereward/llms.txt

Demonstrates how to use the `score_gard` method from the ImageReward library to compute differentiable rewards for gradient-based optimization during ReFL training. This involves loading the ImageReward model, preparing tokenized prompts, and passing them along with image tensors to `score_gard` to obtain rewards for backpropagation.

```python
import torch
import ImageReward as RM

# Load model for training
model = RM.load("ImageReward-v1.0", device="cuda")

# Prepare tokenized prompt
prompt = "a beautiful sunset"
text_input = model.blip.tokenizer(
    prompt,
    padding='max_length',
    truncation=True,
    max_length=35,
    return_tensors="pt"
).to("cuda")

# Assume 'image' is a tensor from diffusion model output
# Shape: (batch_size, 3, 224, 224), normalized
image = torch.randn(1, 3, 224, 224).cuda()  # Example tensor

# Compute differentiable reward for backpropagation
rewards = model.score_gard(
    text_input.input_ids,
    text_input.attention_mask,
    image
)

# Compute loss and backpropagate
loss = torch.nn.functional.relu(-rewards + 2).mean()
loss.backward()

print(f"Reward: {rewards.item():.4f}")
print(f"Loss: {loss.item():.4f}")
```

--------------------------------

### Rank Multiple Images with ImageReward

Source: https://context7.com/thudm/imagereward/llms.txt

Ranks multiple images based on their alignment with a given text prompt using the ImageReward model. It returns both the rankings (1 being the best match) and the corresponding reward scores for each image.

```python
import os
import torch
import ImageReward as RM

model = RM.load("ImageReward-v1.0")
prompt = "a painting of an ocean with clouds and birds, day time, low depth field effect"

# Prepare image list
img_prefix = "assets/images"
generations = [f"{pic_id}.webp" for pic_id in range(1, 5)]
img_list = [os.path.join(img_prefix, img) for img in generations]

# Get rankings and rewards
with torch.no_grad():
    ranking, rewards = model.inference_rank(prompt, img_list)

    print(f"Rankings: {ranking}")
    # Output: Rankings: [1, 2, 3, 4]  (1 = best match)

    print(f"Rewards: {rewards}")
    # Output: Rewards: [0.58, 0.27, -1.41, -2.03]

    # Print individual scores
    for idx, (img, rank, reward) in enumerate(zip(generations, ranking, rewards)):
        print(f"{img}: Rank {rank}, Score {reward:.2f}")
    # Output:
    # 1.webp: Rank 1, Score 0.58
    # 2.webp: Rank 2, Score 0.27
    # 3.webp: Rank 3, Score -1.41
    # 4.webp: Rank 4, Score -2.03
```

--------------------------------

### ReFL Training for Stable Diffusion v1.4

Source: https://context7.com/thudm/imagereward/llms.txt

Fine-tunes Stable Diffusion v1.4 models using the ReFL (Reward Feedback Learning) module with ImageReward as the optimization objective. It requires specifying the pretrained model, training data directory, and training arguments. The training process saves checkpoints and the final model as a complete StableDiffusionPipeline.

```python
from ImageReward import ReFL

# Parse training arguments
args = ReFL.parse_args()

# Initialize trainer with pretrained model and training data
trainer = ReFL.Trainer(
    pretrained_model_name_or_path="CompVis/stable-diffusion-v1-4",
    train_data_dir="data/refl_data.json",
    args=args
)

# Start training
trainer.train(args=args)

# Training saves checkpoints to args.output_dir (default: "checkpoint/refl")
# Final model is saved as a complete StableDiffusionPipeline
```

--------------------------------

### Load Baseline Scoring Models

Source: https://context7.com/thudm/imagereward/llms.txt

Loads alternative scoring models (CLIP, BLIP, Aesthetic) for comparison purposes. It allows scoring single images and ranking multiple images using these baseline models.

```python
import ImageReward as RM
import torch

# List available scoring models
available = RM.available_scores()
print(available)  # ['CLIP', 'BLIP', 'Aesthetic']

# Load CLIP scoring model
clip_model = RM.load_score("CLIP", device="cuda")

# Load BLIP scoring model
blip_model = RM.load_score("BLIP", device="cuda")

# Load Aesthetic scoring model
aesthetic_model = RM.load_score("Aesthetic", device="cuda")

# Score with baseline models
prompt = "a beautiful sunset over mountains"
image_path = "sunset.png"

with torch.no_grad():
    clip_score = clip_model.score(prompt, image_path)
    blip_score = blip_model.score(prompt, image_path)
    aesthetic_score = aesthetic_model.score(prompt, image_path)

    print(f"CLIP Score: {clip_score:.4f}")
    print(f"BLIP Score: {blip_score:.4f}")
    print(f"Aesthetic Score: {aesthetic_score:.2f}")

# Rank images with baseline model
image_list = ["img1.png", "img2.png", "img3.png"]
with torch.no_grad():
    ranking, rewards = clip_model.inference_rank(prompt, image_list)
    print(f"CLIP Rankings: {ranking}")
    print(f"CLIP Rewards: {rewards}")
```

--------------------------------

### Score Single Image with ImageReward

Source: https://context7.com/thudm/imagereward/llms.txt

Evaluates how well a single image matches a text prompt using the ImageReward model, returning a human preference score. Supports scoring from an image file path or a PIL Image object. It can also score multiple images against a single prompt, returning a list of rewards.

```python
import ImageReward as RM
import torch
from PIL import Image

model = RM.load("ImageReward-v1.0")
prompt = "a painting of an ocean with clouds and birds, day time, low depth field effect"

# Score from image file path
with torch.no_grad():
    score = model.score(prompt, "path/to/image.png")
    print(f"Score: {score:.2f}")
    # Output: Score: 0.58

# Score from PIL Image object
pil_image = Image.open("path/to/image.png")
with torch.no_grad():
    score = model.score(prompt, pil_image)
    print(f"Score: {score:.2f}")

# Score multiple images (returns list of rewards)
image_list = ["image1.png", "image2.png", "image3.png"]
with torch.no_grad():
    rewards = model.score(prompt, image_list)
    print(f"Rewards: {rewards}")
    # Output: Rewards: [[0.58], [0.27], [-1.41]]
```

--------------------------------

### ImageReward Integration with Stable Diffusion Web UI

Source: https://context7.com/thudm/imagereward/llms.txt

Details the integration of ImageReward with AUTOMATIC1111's Stable Diffusion Web UI. This involves placing a custom script within the Web UI's directory to enable features such as scoring generated images, automatically filtering low-scoring images, viewing historical scores, and manual model unloading for memory management.

```python
# Place this script in stable-diffusion-webui/scripts/

# Features:
# 1. Score generated images and append to image information
# 2. Automatically filter out images with low scores
# 3. View scores of previously scored images in PNG Info tab
# 4. Memory management with manual model unload button

```

--------------------------------

### POST /score

Source: https://context7.com/thudm/imagereward/llms.txt

Evaluates how well a single image or list of images matches a text prompt.

```APIDOC
## POST /score

### Description
Evaluates how well a single image or list of images matches a text prompt, returning a human preference score.

### Method
POST

### Endpoint
model.score(prompt, image_input)

### Parameters
#### Request Body
- **prompt** (string) - Required - The text description to evaluate against.
- **image_input** (string/list/PIL.Image) - Required - Path to image file, list of paths, or PIL image object.

### Request Example
score = model.score("a painting of an ocean", "path/to/image.png")

### Response
#### Success Response (200)
- **score** (float/list) - The calculated reward score(s) indicating alignment with human preference.
```

--------------------------------

### POST /inference_rank

Source: https://context7.com/thudm/imagereward/llms.txt

Ranks multiple images based on how well they match a prompt.

```APIDOC
## POST /inference_rank

### Description
Ranks multiple images based on how well they match a prompt, returning both rankings and reward scores.

### Method
POST

### Endpoint
model.inference_rank(prompt, image_list)

### Parameters
#### Request Body
- **prompt** (string) - Required - The text description.
- **image_list** (list) - Required - List of image file paths.

### Request Example
ranking, rewards = model.inference_rank("a painting of an ocean", ["img1.png", "img2.png"])

### Response
#### Success Response (200)
- **ranking** (list) - List of integers representing rank (1 = best).
- **rewards** (list) - List of float scores corresponding to each image.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.