### Distributed Training Setup

Source: https://github.com/modelscope/facechain/blob/main/face_module/TransFace/README.md

Command to initiate distributed training on a machine with 8 GPUs. Ensure PyTorch is installed and requirements are met.

```bash
python -m torch.distributed.launch --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train.py 
```

--------------------------------

### Install Dependencies and Clone Repositories

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Installs required Python packages and clones the latent-diffusion and taming-transformers repositories. Ensure you have git and pip installed.

```python
#@title Installation
!git clone https://github.com/CompVis/latent-diffusion.git
!git clone https://github.com/CompVis/taming-transformers
!pip install -e ./taming-transformers
!pip install omegaconf>=2.0.0 pytorch-lightning>=1.0.8 torch-fidelity einops

import sys
sys.path.append(".")
sys.path.append('./taming-transformers')
from taming.models import vqgan 
```

--------------------------------

### Install FaceChain with Docker

Source: https://context7.com/modelscope/facechain/llms.txt

Follow these steps to set up FaceChain using Docker. This includes pulling the ModelScope image, running a container with GPU support, installing dependencies, and launching the application.

```shell
# Pull the ModelScope image
docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.1

# Run container with GPU and port-forwarding
docker run -it --name facechain -p 7860:7860 --gpus all \
  registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.1 /bin/bash

# Inside the container: install dependencies
pip3 install gradio==3.47.1 controlnet_aux==0.0.6 python-slugify \
             diffusers==0.29.0 peft==0.11.1 datasets==2.16
pip3 install modelscope -U

# Clone the repo and launch
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/modelscope/facechain.git --depth 1
cd facechain
CUDA_VISIBLE_DEVICES=0 python3 app.py
# Open https://xxx.gradio.live in your browser
```

--------------------------------

### Install Requirements

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/README.md

Install necessary dependencies for TopoFR using pip.

```bash
pip install -r requirement.txt
```

--------------------------------

### Install FaceChain Plugin on stable-diffusion-webui

Source: https://github.com/modelscope/facechain/blob/main/README.md

Install the FaceChain plugin through the Extensions tab in stable-diffusion-webui by providing the installation URL. Ensure CUDA Toolkit is installed for successful dependency installation, particularly for the 'mmcv' package.

```shell
# 1. Select the `Extensions Tab`, then choose `Install From URL` (official plugin integration is integrated, please install from URL currently).
# ![image](resources/sdwebui_install.png)

# 2. Switch to `Installed`, check the FaceChain plugin, then click `Apply and restart UI`. It may take a while for installing the dependencies and downloading the models. Make sure that the "CUDA Toolkit" is installed correctly, otherwise the "mmcv" package cannot be successfully installed.
# ![image](resources/sdwebui_restart.png)

# 3. After the page refreshes, the appearance of the `FaceChain` Tab indicates a successful installation.
# ![image](resources/sdwebui_success.jpg)
```

--------------------------------

### Distributed Training Setup

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/README.md

Launch the training process on a machine with multiple GPUs using PyTorch's distributed training module.

```bash
python -m torch.distributed.launch --nproc_per_node=4 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=12581 train.py
```

--------------------------------

### Install FaceChain on ModelScope Notebook

Source: https://github.com/modelscope/facechain/blob/main/README.md

Clone the FaceChain repository and install necessary dependencies within a ModelScope Notebook environment. Ensure you are using the recommended environment and change the directory to the cloned repository before installing.

```shell
# Step1: 我的notebook -> PAI-DSW -> GPU环境
# Note: Please use: ubuntu20.04-py38-torch2.0.1-tf1.15.5-modelscope1.8.1

# Step2: Entry the Notebook cell，clone FaceChain from github:
!GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/modelscope/facechain.git --depth 1

# Step3: Change the working directory to facechain, and install the dependencies:
import os
os.chdir('/mnt/workspace/facechain')    # You may change to your own path
print(os.getcwd())

!pip3 install gradio==3.47.1
!pip3 install controlnet_aux==0.0.6
!pip3 install python-slugify
!pip3 install diffusers==0.29.0
!pip3 install peft==0.11.1
!pip3 install modelscope -U
!pip3 install datasets==2.16

# Step4: Start the app service, click "public URL" or "local URL", upload your images to 
# train your own model and then generate your digital twin.
!python3 app.py
```

--------------------------------

### Install Jemalloc for Memory Optimization

Source: https://github.com/modelscope/facechain/blob/main/README.md

Use these commands to install Jemalloc for optimizing memory usage, especially on systems with limited RAM. This is recommended for environments requiring memory reduction from over 30GB to below 20GB.

```shell
apt-get install -y libjemalloc-dev
export LD_PRELOAD=/lib/x86_64-linux-gnu/libjemalloc.so
```

--------------------------------

### Run FaceChain with Docker

Source: https://github.com/modelscope/facechain/blob/main/README.md

Pull the ModelScope Docker image and run a container to set up the FaceChain environment. Install additional Python packages and clone the repository within the container before running the application.

```shell
# Step1: Prepare the environment with GPU on local or cloud, we recommend to use Alibaba Cloud ECS, refer to: https://www.aliyun.com/product/ecs

# Step2: Download the docker image (for installing docker engine, refer to https://docs.docker.com/engine/install/）
# For China Mainland users:
docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.1
# For users outside China Mainland:
docker pull registry.us-west-1.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.1

# Step3: run the docker container
docker run -it --name facechain -p 7860:7860 --gpus all registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.1 /bin/bash
# Note: you may need to install the nvidia-container-runtime, follow the instructions:
# 1. Install nvidia-container-runtime：https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
# 2. sudo systemctl restart docker

# Step4: Install the gradio in the docker container:
pip3 install gradio==3.47.1
pip3 install controlnet_aux==0.0.6
pip3 install python-slugify
pip3 install diffusers==0.29.0
pip3 install peft==0.11.1
pip3 install modelscope -U
pip3 install datasets==2.16

# Step5 clone facechain from github
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/modelscope/facechain.git --depth 1
cd facechain
python3 app.py
# Note: FaceChain currently assume single-GPU, if your environment has multiple GPU, please use the following instead:
# CUDA_VISIBLE_DEVICES=0 python3 app.py

# Step6
Run the app server: click "public URL" --> in the form of: https://xxx.gradio.live
```

--------------------------------

### Install PyTorch v1.9.0 with CUDA 11.1

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/install.md

Use this command to install PyTorch version 1.9.0 and compatible torchvision/torchaudio versions with CUDA 11.1 support on Linux and Windows.

```shell
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
```

--------------------------------

### Install PyTorch v1.9.0 with CUDA 10.2

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/install.md

Use this command to install PyTorch version 1.9.0 and compatible torchvision/torchaudio versions with CUDA 10.2 support on Linux and Windows.

```shell
pip install torch==1.9.0+cu102 torchvision==0.10.0+cu102 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
```

--------------------------------

### Check GPU Information

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Displays information about the available NVIDIA GPU, including driver version, CUDA version, GPU name, and memory usage. Useful for verifying hardware setup.

```bash
!nvidia-smi
```

--------------------------------

### Memory Optimization for FaceChain

Source: https://context7.com/modelscope/facechain/llms.txt

Optional steps to reduce peak GPU memory usage. This involves installing libjemalloc-dev and setting an environment variable.

```shell
# Reduces peak GPU memory from ~30 GB to ~20 GB
apt-get install -y libjemalloc-dev
export LD_PRELOAD=/lib/x86_64-linux-gnu/libjemalloc.so
```

--------------------------------

### Evaluate on IJB-C (Glint360K Dataset)

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/README.md

Run evaluation scripts for the Glint360K dataset on the IJB-C benchmark. Ensure the dataset path is correctly configured in the script.

```bash
python eval_ijbc_glint360k.py --model-prefix work_dirs/glint360k_r50/model.pt --result-dir work_dirs/glint360k_r50 --network r50 > ijbc_glint360k_R50_TopoFR.log 2>&1 &
```

--------------------------------

### Evaluate on IJB-C (MS1MV2 Dataset)

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/README.md

Run evaluation scripts for the MS1MV2 dataset on the IJB-C benchmark. Ensure the dataset path is correctly configured in the script.

```bash
python eval_ijbc_ms1mv2.py --model-prefix work_dirs/ms1mv2_r50/model.pt --result-dir work_dirs/ms1mv2_r50 --network r50 > ijbc_ms1mv2_R50_TopoFR.log 2>&1 &
```

--------------------------------

### FaceChain Prompt Constants and Usage

Source: https://context7.com/modelscope/facechain/llms.txt

Demonstrates the usage of predefined prompt templates and constants from the facechain.constants module. These are useful for constructing positive and negative prompts for image generation.

```python
from facechain.constants import (
    neg_prompt,
    pos_prompt_with_cloth,
    pos_prompt_with_style,
    base_models,
    pose_examples,
)

# Negative prompt (global default)
print(neg_prompt[:80])
# "(nsfw:2), paintings, sketches, (worst quality:2), (low quality:2), lowers, ..."

# Build a positive prompt with custom clothing
prompt = pos_prompt_with_cloth.format('wearing a formal suit')
# "raw photo, masterpiece, chinese, wearing a formal suit, solo, medium shot, ..."

# Build a positive prompt from a style's add_prompt_style field
style_trigger = 'cinematic film still, 35mm photograph, film, bokeh'
prompt = pos_prompt_with_style.format(style_trigger)
# "cinematic film still ..., upper_body, raw photo, masterpiece, solo, ..."

# Available base models
for m in base_models:
    print(m['name'], '->', m['model_id'])
# leosamsMoonfilm_filmGrain20  -> ly261666/cv_portrait_model
# MajicmixRealistic_v6         -> YorickHe/majicmixRealistic_v6

# Pose example paths (indexed by gender)
print(pose_examples['man'][0])   # ['/path/to/facechain/poses/man/pose1.png']
print(pose_examples['woman'][2]) # ['/path/to/facechain/poses/woman/pose3.png']
```

--------------------------------

### Download Model Checkpoint

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Downloads the pre-trained model checkpoint for Latent Diffusion Models. This requires approximately 1.7 GB of disk space and can take a few minutes.

```bash
#@title Download
%cd latent-diffusion/ 

!mkdir -p models/ldm/cin256-v2/
!wget -O models/ldm/cin256-v2/model.ckpt https://ommer-lab.com/files/latent-diffusion/nitro/cin/model.ckpt 
```

--------------------------------

### Test Pretrained Model (Glint360K R100 TopoFR)

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/README.md

Test the accuracy of a specific pre-trained model (Glint360K R100 TopoFR) on the IJB-C benchmark. Adjust the model prefix and result directory as needed.

```bash
python eval_ijbc_glint360k.py --model-prefix work_dirs/glint360k_r100/Glint360K_R100_TopoFR_9760.pt --result-dir work_dirs/glint360k_r100 --network r100 > ijbc_glint360k_R100_TopoFR.log 2>&1 &
```

--------------------------------

### Configure Inpainting Inference Parameters

Source: https://github.com/modelscope/facechain/blob/main/README.md

Set the number of faces, selected face index, inpainting strength, and paths for template and input images. Specify the number of images to generate and the output directory.

```python
num_faces = 1
selected_face = 1
strength = 0.6
inpaint_img = 'poses/man/pose1.png'
input_img_path = 'poses/man/pose2.png'
num_generate = 1
output_dir = './generated_inpaint'
```

--------------------------------

### Test Training Commands

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/speed_benchmark.md

Use these commands to test Partial FC training performance with 3 million identities, mixed precision, ResNet50 backbone, and a batch size of 1024.

```shell
# Model Parallel
python -m torch.distributed.launch --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=1234 train.py configs/3millions
```

```shell
# Partial FC 0.1
python -m torch.distributed.launch --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr="127.0.0.1" --master_port=1234 train.py configs/3millions_pfc
```

--------------------------------

### Load Model Utilities

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Imports necessary libraries and defines a utility function to load a model from a configuration file and checkpoint. This function is essential for initializing the diffusion model.

```python
#@title loading utils
import torch
from omegaconf import OmegaConf

from ldm.util import instantiate_from_config


def load_model_from_config(config, ckpt):
    print(f"Loading model from {ckpt}")

```

--------------------------------

### Configure and Run Inference for Infinite Style Portrait

Source: https://github.com/modelscope/facechain/blob/main/README.md

Edit the `run_inference.py` script to configure parameters such as pose control, input image paths, generation count, style multiplier, output directory, and model indices before executing the script.

```python
# Use pose control, default False
use_pose_model = False
# The path of the input image containing ID information for portrait generation
input_img_path = 'poses/man/pose2.png'
# The path of the image for pose control, only effective when using pose control
pose_image = 'poses/man/pose1.png'
# The number of images to generate in inference
num_generate = 5
# The weight for the style model, see styles for detail
multiplier_style = 0.25
# Specify a folder to save the generated images, this parameter can be modified as needed
output_dir = './generated'
# The index of the chosen base model, see facechain/constants.py for detail
base_model_idx = 0
# The index of the style model, see styles for detail
style_idx = 0
```

```shell
python run_inference.py
```

--------------------------------

### Building Custom Pipelines with FaceAdapter_v1

Source: https://context7.com/modelscope/facechain/llms.txt

Initialize `FaceAdapter_v1` by loading supporting models like ControlNet, Stable Diffusion, face detection, and segmentation pipelines. Set the adapter scale to control face fidelity. This module is suitable for custom pipeline development.

```python
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel, PNDMScheduler
from modelscope import snapshot_download
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from controlnet_aux import OpenposeDetector
from face_adapter import FaceAdapter_v1, Face_Extracter_v1
from PIL import Image
import torch, os

# --- Load supporting models ---
model_dir = snapshot_download('damo/face_chain_control_model', revision='v1.0.1')
controlnet = ControlNetModel.from_pretrained(
    os.path.join(model_dir, 'model_controlnet/control_v11p_sd15_openpose'),
    torch_dtype=torch.float16)

base_model_path = snapshot_download('MAILAND/majicmixRealistic_v6', revision='v1.0.0')
pipe = StableDiffusionControlNetPipeline.from_pretrained(
    os.path.join(base_model_path, 'realistic'),
    safety_checker=None, controlnet=controlnet, torch_dtype=torch.float16)
pipe.scheduler = PNDMScheduler.from_config(pipe.scheduler.config)

face_detection      = pipeline(task=Tasks.face_detection, model='damo/cv_resnet50_face-detection_retinaface')
segmentation        = pipeline(Tasks.image_segmentation, 'damo/cv_resnet101_image-multiple-human-parsing', model_revision='v1.0.1')

fact_dir            = snapshot_download('yucheng1996/FaceChain-FACT', revision='v1.0.0')
fr_weight_path      = os.path.join(fact_dir, 'ms1mv2_model_TransFace_S.pt')
adapter_ckpt        = os.path.join(fact_dir, 'adapter_maj_mask_large_new_reg001_faceshuffle_00290001.ckpt')

face_extracter      = Face_Extracter_v1(fr_weight_path=fr_weight_path, fc_weight_path=adapter_ckpt)

# --- Build adapter ---
adapter = FaceAdapter_v1(
    sd_pipe              = pipe,
    face_detection       = face_detection,
    segmentation_pipeline= segmentation,
    face_extracter       = face_extracter,
    ckpt                 = adapter_ckpt,
    device               = 'cuda',
    cfg_face             = True,
)
adapter.set_scale(0.5)   # 0 = no face conditioning, 1 = maximum face fidelity

# --- Generate ---
face_img = Image.open('my_photo.jpg').convert('RGB')
images = adapter.generate(
    face_image        = face_img,
    prompt            = 'a handsome man, masterpiece, photorealistic, best quality',
    negative_prompt   = 'nsfw, low quality, worst quality',
    num_samples       = 2,
    guidance_scale    = 5.0,
    num_inference_steps = 50,
    height            = 512,
    width             = 512,
)
# images: list of PIL Images
images[0].save('output_0.png')
images[1].save('output_1.png')
```

--------------------------------

### Generate Images with Fine-tuned Model

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/README.md

Generate new images using a fine-tuned checkpoint. Specify the path to your fine-tuned model using the --ckpt argument and provide your desired prompt.

```python
python scripts/stable_txt2img.py --ddim_eta 0.0 --n_samples 8 --n_iter 2 --scale 10.0 --ddim_steps 50  --ckpt finetuned_ckpt --prompt "prompt" --seed 7 --outdir output_img_dir --unconditional_prompt "monochrome, lowres, bad anatomy, worst quality, low quality"
```

--------------------------------

### Render Image Grid and Convert to PIL Image

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Organizes generated samples into a grid and converts the grid to a PIL Image object. This is useful for visualizing multiple generated samples at once.

```python
grid = torch.stack(all_samples, 0)
grid = rearrange(grid, 'n b c h w -> (n b) c h w')
grid = make_grid(grid, nrow=n_samples_per_class)

# to image
grid = 255. * rearrange(grid, 'c h w -> h w c').cpu().numpy()
Image.fromarray(grid.astype(np.uint8))
```

--------------------------------

### Launch Gradio Web UI Programmatically

Source: https://context7.com/modelscope/facechain/llms.txt

Programmatically trigger the Gradio web app's functionality, simulating user interactions like selecting styles and prompts to generate images. The `launch_pipeline` function returns progress updates and image outputs.

```python
# Launch the full web app (single GPU)
# CUDA_VISIBLE_DEVICES=0 python3 app.py

# Programmatic equivalent of what the "Start!" button does:
from app import launch_pipeline
import gradio as gr

# Simulate clicking "Start!" with preset style index 0
outputs = []
for progress, imgs in launch_pipeline(
    uuid          = 'local_user',
    style_choice  = 0,               # 0 = preset styles, 1 = user-trained
    pos_prompt    = 'raw photo, masterpiece, chinese, solo, medium shot, high detail face, photorealistic, best quality, wearing casual clothes',
    neg_prompt    = '(nsfw:2), paintings, sketches, (worst quality:2)',
    user_images   = [{'name': 'poses/man/pose2.png'}],
    num_images    = 2,
    style_model   = 'MajicmixRealistic_v6',
    lora_choice   = 'preset',
    multiplier_style = 0.25,
    pose_image    = None,
    use_face_swap = 1,
):
    print(progress)  # "Generation done!" or "Generation failed, please retry!"
    outputs = imgs

# outputs: list of RGB numpy arrays ready for display or cv2.imwrite
```

--------------------------------

### Training Speed Comparison

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/speed_benchmark.md

Compares the training speed in samples/sec between Model Parallel and Partial FC 0.1 training. Partial FC achieves significantly higher throughput.

```log
# (Model Parallel) trainging.log
Training: Speed 2271.33 samples/sec   Loss 1.1624   LearningRate 0.2000   Epoch: 0   Global Step: 100 
Training: Speed 2269.94 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 150 
Training: Speed 2272.67 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 200 
Training: Speed 2266.55 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 250 
Training: Speed 2272.54 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 300 

# (Partial FC 0.1) trainging.log
Training: Speed 5299.56 samples/sec   Loss 1.0965   LearningRate 0.2000   Epoch: 0   Global Step: 100  
Training: Speed 5296.37 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 150  
Training: Speed 5304.37 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 200  
Training: Speed 5274.43 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 250  
Training: Speed 5300.10 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 300   
```

--------------------------------

### Training Speed Comparison

Source: https://github.com/modelscope/facechain/blob/main/face_module/TransFace/docs/speed_benchmark.md

Compares the training speed (samples/sec) between Model Parallel and Partial FC training. Partial FC 0.1 achieves approximately 2.5 times faster training speed.

```text
# (Model Parallel) trainging.log
Training: Speed 2271.33 samples/sec   Loss 1.1624   LearningRate 0.2000   Epoch: 0   Global Step: 100 
Training: Speed 2269.94 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 150 
Training: Speed 2272.67 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 200 
Training: Speed 2266.55 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 250 
Training: Speed 2272.54 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 300 
```

```text
# (Partial FC 0.1) trainging.log
Training: Speed 5299.56 samples/sec   Loss 1.0965   LearningRate 0.2000   Epoch: 0   Global Step: 100  
Training: Speed 5296.37 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 150  
Training: Speed 5304.37 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 200  
Training: Speed 5274.43 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 250  
Training: Speed 5300.10 samples/sec   Loss 0.0000   LearningRate 0.2000   Epoch: 0   Global Step: 300   
```

--------------------------------

### Configure and Run Inference for Fixed Templates Portrait

Source: https://github.com/modelscope/facechain/blob/main/README.md

Edit the `run_inference_inpaint.py` script to configure parameters for Fixed Templates Portrait generation. The specific parameters to edit are not detailed in the provided text, but the script name indicates its purpose.

```python
# When inferring for Fixed Templates Portrait generation, please edit the code in run_inference_inpaint.py.
```

--------------------------------

### DDIM Sampling Initialization

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Initializes and runs the DDIM sampling process for generating images. It specifies the class, number of steps, and scale factor.

```python
rendering 6 examples of class '25' in 20 steps and using s=3.00.
Data shape for DDIM sampling is (6, 3, 64, 64), eta 0.0
Running DDIM Sampling with 20 timesteps
```

```python
rendering 6 examples of class '187' in 20 steps and using s=3.00.
Data shape for DDIM sampling is (6, 3, 64, 64), eta 0.0
Running DDIM Sampling with 20 timesteps
```

```python
rendering 6 examples of class '448' in 20 steps and using s=3.00.
Data shape for DDIM sampling is (6, 3, 64, 64), eta 0.0
Running DDIM Sampling with 20 timesteps
```

```python
rendering 6 examples of class '992' in 20 steps and using s=3.00.
Data shape for DDIM sampling is (6, 3, 64, 64), eta 0.0
Running DDIM Sampling with 20 timesteps
```

--------------------------------

### Execute Inpainting Inference Script

Source: https://github.com/modelscope/facechain/blob/main/README.md

Run the inference script using Python. Generated images will be saved in the directory specified by the 'output_dir' parameter.

```shell
python run_inference_inpaint.py
```

--------------------------------

### GPU Memory Usage Comparison

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/speed_benchmark.md

Compares the GPU memory usage between Model Parallel and Partial FC 0.1 training configurations. Partial FC uses significantly less memory.

```shell
# (Model Parallel) gpustat -i
[0] Tesla V100-SXM2-32GB | 64'C,  94 % | 30338 / 32510 MB 
[1] Tesla V100-SXM2-32GB | 60'C,  99 % | 28876 / 32510 MB 
[2] Tesla V100-SXM2-32GB | 60'C,  99 % | 28872 / 32510 MB 
[3] Tesla V100-SXM2-32GB | 69'C,  99 % | 28872 / 32510 MB 
[4] Tesla V100-SXM2-32GB | 66'C,  99 % | 28888 / 32510 MB 
[5] Tesla V100-SXM2-32GB | 60'C,  99 % | 28932 / 32510 MB 
[6] Tesla V100-SXM2-32GB | 68'C, 100 % | 28916 / 32510 MB 
[7] Tesla V100-SXM2-32GB | 65'C,  99 % | 28860 / 32510 MB 

# (Partial FC 0.1) gpustat -i
[0] Tesla V100-SXM2-32GB | 60'C,  95 % | 10488 / 32510 MB                                                                                                                                          │·······················
[1] Tesla V100-SXM2-32GB | 60'C,  97 % | 10344 / 32510 MB                                                                                                                                          │·······················
[2] Tesla V100-SXM2-32GB | 61'C,  95 % | 10340 / 32510 MB                                                                                                                                          │·······················
[3] Tesla V100-SXM2-32GB | 66'C,  95 % | 10340 / 32510 MB                                                                                                                                          │·······················
[4] Tesla V100-SXM2-32GB | 65'C,  94 % | 10356 / 32510 MB                                                                                                                                          │·······················
[5] Tesla V100-SXM2-32GB | 61'C,  95 % | 10400 / 32510 MB                                                                                                                                          │·······················
[6] Tesla V100-SXM2-32GB | 68'C,  96 % | 10384 / 32510 MB                                                                                                                                          │·······················
[7] Tesla V100-SXM2-32GB | 64'C,  95 % | 10328 / 32510 MB                                                                                                                                        │·······················
```

--------------------------------

### FaceChain-FACT Project Citation (Article)

Source: https://github.com/modelscope/facechain/blob/main/README.md

Cite this article when using the FaceChain-FACT project, which focuses on identity-preserved personalization. It details the Face Adapter with Decoupled Training approach.

```bibtex
@article{yu2024facechain,
  title={FaceChain-FACT: Face Adapter with Decoupled Training for Identity-preserved Personalization},
  author={Yu, Cheng and Xie, Haoyu and Shang, Lei and Liu, Yang and Dan, Jun and Sun, Baigui and Bo, Liefeng},
  journal={arXiv preprint arXiv:2410.12312},
  year={2024}
}
```

--------------------------------

### Download Model from ModelScope

Source: https://context7.com/modelscope/facechain/llms.txt

Downloads a specified model from ModelScope. It automatically retries up to 3 times on failure. Ensure you have internet connectivity.

```python
from modelscope.utils import snapshot_download
try:
    model_dir = snapshot_download('yucheng1996/FaceChain-FACT', revision='v1.0.0')
    print(f'Models cached at: {model_dir}')
except Exception as e:
    print(f'Download failed after 3 retries: {e}')
```

--------------------------------

### GPU Memory Usage Comparison

Source: https://github.com/modelscope/facechain/blob/main/face_module/TransFace/docs/speed_benchmark.md

Compares the GPU memory usage between Model Parallel and Partial FC training. Partial FC 0.1 uses significantly less memory.

```text
# (Model Parallel) gpustat -i
[0] Tesla V100-SXM2-32GB | 64'C,  94 % | 30338 / 32510 MB 
[1] Tesla V100-SXM2-32GB | 60'C,  99 % | 28876 / 32510 MB 
[2] Tesla V100-SXM2-32GB | 60'C,  99 % | 28872 / 32510 MB 
[3] Tesla V100-SXM2-32GB | 69'C,  99 % | 28872 / 32510 MB 
[4] Tesla V100-SXM2-32GB | 66'C,  99 % | 28888 / 32510 MB 
[5] Tesla V100-SXM2-32GB | 60'C,  99 % | 28932 / 32510 MB 
[6] Tesla V100-SXM2-32GB | 68'C, 100 % | 28916 / 32510 MB 
[7] Tesla V100-SXM2-32GB | 65'C,  99 % | 28860 / 32510 MB 
```

```text
# (Partial FC 0.1) gpustat -i
[0] Tesla V100-SXM2-32GB | 60'C,  95 % | 10488 / 32510 MB                                                                                                                                          │·······················
[1] Tesla V100-SXM2-32GB | 60'C,  97 % | 10344 / 32510 MB                                                                                                                                          │·······················
[2] Tesla V100-SXM2-32GB | 61'C,  95 % | 10340 / 32510 MB                                                                                                                                          │·······················
[3] Tesla V100-SXM2-32GB | 66'C,  95 % | 10340 / 32510 MB                                                                                                                                          │·······················
[4] Tesla V100-SXM2-32GB | 65'C,  94 % | 10356 / 32510 MB                                                                                                                                          │·······················
[5] Tesla V100-SXM2-32GB | 61'C,  95 % | 10400 / 32510 MB                                                                                                                                          │·······················
[6] Tesla V100-SXM2-32GB | 68'C,  96 % | 10384 / 32510 MB                                                                                                                                          │·······················
[7] Tesla V100-SXM2-32GB | 64'C,  95 % | 10328 / 32510 MB                                                                                                                                        │·······················
```

--------------------------------

### Download Models with snapshot_download

Source: https://context7.com/modelscope/facechain/llms.txt

Utilize the `snapshot_download` utility for resilient model downloads, which includes automatic retries for network stability. This function downloads model weights to the ModelScope cache.

```python
from facechain.utils import snapshot_download

# Downloads the FACT adapter weights to the ModelScope cache (~2 GB)
```

--------------------------------

### Initialize DDIM Sampler

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Initializes the DDIMSampler with the loaded Latent Diffusion Model. This sampler is used for generating images.

```python
from ldm.models.diffusion.ddim import DDIMSampler

model = get_model()
sampler = DDIMSampler(model)
```

--------------------------------

### Custom Training with SuDe

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/README.md

Fine-tune a Stable Diffusion model using the SuDe method. Provide the category of the subject and a weight for the SuDe loss to control attribute inheritance. Ensure 'data_root' and 'reg_data_root' point to your subject and regularization data directories.

```python
python main.py --base configs/stable-diffusion/v1-finetune_unfrozen.yaml -t --actual_resume stable-diffusion-v-1-4/sd-v1-4-full-ema.ckpt -n name --gpus 0, --data_root subject_dir --reg_data_root reg_data_dir --class_word <category> --sude_weight 0.8 --logdir output_checkpoint_dir
```

--------------------------------

### Generate Regularization Data

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/README.md

Use this script to generate regularization images required for DreamBooth training. Adjust parameters like --n_samples, --n_iter, and --scale for desired output.

```python
python scripts/stable_txt2img.py --ddim_eta 0.0 --n_samples 10 --n_iter 5 --scale 10.0 --ddim_steps 50  --ckpt stable-diffusion-v-1-4/sd-v1-4-full-ema.ckpt --prompt "photo of a <category>" --seed 7 --outdir reg_data/<category> --unconditional_prompt "monochrome, lowres, bad anatomy, worst quality, low quality"
```

--------------------------------

### Create Shuffled Rec File for DALI

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/prepare_webface42m.md

Use the mxnet.tools.im2rec script to generate a shuffled rec file. First, create a list file by recursively scanning the dataset directory, then generate the rec and idx files using the created list.

```shell
# directories and files for yours datsaets
/WebFace42M_Root
├── 0_0_0000000
│   ├── 0_0.jpg
│   ├── 0_1.jpg
│   ├── 0_2.jpg
│   ├── 0_3.jpg
│   └── 0_4.jpg
├── 0_0_0000001
│   ├── 0_5.jpg
│   ├── 0_6.jpg
│   ├── 0_7.jpg
│   ├── 0_8.jpg
│   └── 0_9.jpg
├── 0_0_0000002
│   ├── 0_10.jpg
│   ├── 0_11.jpg
│   ├── 0_12.jpg
│   ├── 0_13.jpg
│   ├── 0_14.jpg
│   ├── 0_15.jpg
│   ├── 0_16.jpg
│   └── 0_17.jpg
├── 0_0_0000003
│   ├── 0_18.jpg
│   ├── 0_19.jpg
│   └── 0_20.jpg
├── 0_0_0000004
```

```shell
# 1) create train.lst using follow command
python -m mxnet.tools.im2rec --list --recursive train WebFace42M_Root
```

```shell
# 2) create train.rec and train.idx using train.lst using following command
python -m mxnet.tools.im2rec --num-thread 16 --quality 100 train WebFace42M_Root
```

--------------------------------

### Inpainting with GenPortrait_inpaint

Source: https://context7.com/modelscope/facechain/llms.txt

Use `GenPortrait_inpaint` to composite a user's face into a template image. Specify input images, desired face, and generation parameters. Outputs are saved to a specified directory.

```python
inpaint_img    = 'poses/man/pose1.png'  # template photo with at least one face
input_img_path = 'poses/man/pose2.png'  # user identity photo
num_faces      = 1                       # how many faces are in the template
selected_face  = 1                       # which face to replace (left-to-right, 1-indexed)
strength       = 0.6                     # inpainting denoising strength
output_img_size = 512
num_generate   = 1
output_dir     = './generated_inpaint'

pos_prompt = (
    'raw photo, masterpiece, simple background, solo, medium shot, '
    'high detail face, photorealistic, best quality, wearing T-shirt'
)
neg_prompt = (
    'nsfw, paintings, sketches, (worst quality:2), (low quality:2) '
    'lowers, normal quality, ((monochrome)), ((grayscale)), logo, word, character'
)

# --- Initialize ---
gen_portrait_inpaint = GenPortrait_inpaint()

# --- Run inference ---
outputs = gen_portrait_inpaint(
    use_face_swap   = 1,
    inpaint_img     = inpaint_img,
    strength        = strength,
    output_img_size = output_img_size,
    num_faces       = num_faces,
    selected_face   = selected_face,
    pos_prompt      = pos_prompt,
    neg_prompt      = neg_prompt,
    input_img_path  = input_img_path,
    num_gen_images  = num_generate,
)

os.makedirs(output_dir, exist_ok=True)
for i, img in enumerate(outputs):
    cv2.imwrite(os.path.join(output_dir, f'inpaint_{i}.png'), img)
    print(f'Saved inpaint_{i}.png')
# Output: portrait with user face composited into template image
```

--------------------------------

### Generate Portraits with GenPortrait

Source: https://context7.com/modelscope/facechain/llms.txt

Use the `GenPortrait` class for train-free, style-flexible portrait generation. This script initializes the class, sets configuration options, and runs the inference pipeline, saving the generated images.

```python
import cv2
import os
from facechain.inference_fact import GenPortrait
from facechain.constants import neg_prompt, pos_prompt_with_cloth, base_models
from facechain.utils import snapshot_download

# --- Configuration ---
input_img_path   = 'poses/man/pose2.png'   # user identity photo
pose_image       = None                    # set to a filepath to enable pose control
num_generate     = 4
multiplier_style = 0.25                    # style LoRA blend weight (0 = no style)
base_model_idx   = 0                       # 0 = leosamsMoonfilm (film grain), 1 = MajicmixRealistic
use_face_swap    = 1                       # 1 = enable face similarity enhancement
output_dir       = './generated'

# --- Load style LoRA (optional) ---
style_model_path = None   # None uses default; or provide a .safetensors path
pos_prompt = pos_prompt_with_cloth.format('wearing a casual jacket')

# --- Initialize (downloads models on first run, ~10 GB) ---
gen_portrait = GenPortrait()

# --- Run inference ---
# Returns list of BGR numpy arrays
outputs = gen_portrait(
    use_face_swap    = use_face_swap,
    num_gen_images   = num_generate,
    base_model_idx   = base_model_idx,
    style_model_path = style_model_path,
    pos_prompt       = pos_prompt,
    neg_prompt       = neg_prompt,
    input_img_path   = input_img_path,
    pose_image       = pose_image,
    multiplier_style = multiplier_style,
)

os.makedirs(output_dir, exist_ok=True)
for i, img in enumerate(outputs):
    cv2.imwrite(os.path.join(output_dir, f'portrait_{i}.png'), img)
    print(f'Saved portrait_{i}.png')
# Output: 4 portrait PNG files in ./generated/
```

--------------------------------

### Load Latent Diffusion Model

Source: https://github.com/modelscope/facechain/blob/main/more_apps/Facechain-SuDe/scripts/latent_imagenet_diffusion.ipynb

Loads a pre-trained Latent Diffusion Model from a checkpoint file and configuration. Ensure the checkpoint and config files are correctly specified.

```python
def load_model_from_config(config, ckpt):
    pl_sd = torch.load(ckpt)#, map_location="cpu")
    sd = pl_sd["state_dict"]
    model = instantiate_from_config(config.model)
    m, u = model.load_state_dict(sd, strict=False)
    model.cuda()
    model.eval()
    return model

def get_model():
    config = OmegaConf.load("configs/latent-diffusion/cin256-v2.yaml")  
    model = load_model_from_config(config, "models/ldm/cin256-v2/model.ckpt")
    return model
```

--------------------------------

### Generate Fixed Template Portraits with GenPortrait_inpaint

Source: https://context7.com/modelscope/facechain/llms.txt

Utilize `GenPortrait_inpaint` for pasting a user's face into a pre-existing template photo using an inpainting pipeline. This script requires importing the necessary class.

```python
import cv2
import os
from facechain.inference_inpaint_fact import GenPortrait_inpaint
```

--------------------------------

### Eval IJBC With PyTorch

Source: https://github.com/modelscope/facechain/blob/main/face_module/TopoFR/docs/eval.md

Evaluate the IJBC dataset using a PyTorch model. This command requires specifying the model path, image directory, result directory, batch size, job name, target dataset, and network architecture.

```shell
CUDA_VISIBLE_DEVICES=0,1 python eval_ijbc.py \
--model-prefix ms1mv3_arcface_r50/backbone.pth \
--image-path IJB_release/IJBC \
--result-dir ms1mv3_arcface_r50 \
--batch-size 128 \
--job ms1mv3_arcface_r50 \
--target IJBC \
--network iresnet50
```

--------------------------------

### Model Evaluation on IJB-C Dataset

Source: https://github.com/modelscope/facechain/blob/main/face_module/TransFace/README.md

Run evaluation for the trained model on the IJB-C dataset. Modify the dataset path in `eval_ijbc.py` before execution.

```bash
python eval_ijbc.py --model-prefix work_dirs/glint360k_vit_s/model.pt --result-dir work_dirs/glint360k_vit_s --network vit_s_dp005_mask_0 > ijbc_glint360k_vit_s.log 2>&1 &
```