### PaddleMIX Installation Guide

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/tutorial/tutorial_3_deepseek_vl2.md

Provides commands to clone the PaddleMIX repository and install PaddlePaddle and its dependencies. It includes examples for different PaddlePaddle versions and installation methods.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX.git

cd PaddleMIX

# Example for PaddlePaddle 3.0.0b2 (CUDA 11.8)
python -m pip install paddlepaddle-gpu==3.0.0b2 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/

# Example for Develop version
python -m pip install paddlepaddle-gpu==0.0.0.post118 -f https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html

# Quick install using sh script
sh build_paddle_env.sh

# Install PaddleMIX and dependencies
python -m pip install -e . --user
python -m pip install -e ppdiffusers --user
python -m pip install -r requirements.txt --user
python -m pip install paddlenlp==3.0.0b3 --user

# Quick install using sh script
sh build_env.sh
```

--------------------------------

### Setting up PaddleMIX Environment

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/tutorial/tutorial_2_qwen2-vl.md

Instructions for cloning the PaddleMIX repository and installing necessary dependencies, including PaddlePaddle and other Python packages. This setup is required to run the Qwen2.5-VL models and examples.

```bash
# clone PaddleMIX repository
git clone https://github.com/PaddlePaddle/PaddleMIX.git

cd PaddleMIX

# Install PaddlePaddle (example for CUDA 11.8)
python -m pip install paddlepaddle-gpu==3.0.0b2 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/

# Install PaddleMIX dependencies
python -m pip install -e . --user
python -m pip install -e ppdiffusers --user
python -m pip install -r requirements.txt --user
python -m pip install paddlenlp==3.0.0b3 --user
```

--------------------------------

### Setup Environment and Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/PhotoMaker/README.md

Commands to clone the PaddleMIX repository and install the required dependencies including PaddleNLP, PPDiffusers, and Gradio.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX
pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/wheels/ppdiffusers-0.24.0-py3-none-any.whl
pip install paddlenlp==2.7.2
pip install gradio==4.0.0
cd /PaddleMIX/ppdiffusers/examples/PhotoMaker/
```

--------------------------------

### Setup PaddleMIX Environment

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/stable_diffusion/README.md

Commands to clone the PaddleMIX repository, navigate to the stable diffusion example directory, and install required dependencies.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX.git
cd PaddleMIX/ppdiffusers/examples/stable_diffusion
pip install -r requirements.txt
```

--------------------------------

### Install PaddleMIX and Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/Open-Sora/README.md

This snippet outlines the steps to clone the PaddleMIX repository, install the ppdiffusers library and its dependencies, and set up the Open-Sora example environment. It ensures the correct PaddlePaddle framework version and CUDA toolkit are used.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX

python -m pip install paddlepaddle-gpu==2.6.1.post120 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html

cd PaddleMIX/ppdiffusers
pip install -e .

cd examples/Open-Sora/
pip install -r requirements.txt
```

--------------------------------

### Install PaddleMIX and Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/ppvctrl/README_CN.md

Installs PaddleMIX, ppdiffusers, paddlenlp, and other required dependencies from the cloned repository. It also navigates into the PP-VCtrl example directory.

```bash
cd PaddleMIX
pip install -e .
pip install -e ppdiffusers
pip install paddlenlp==v3.0.0-beta2
cd ppdiffusers/examples/ppvctrl
pip install -r requirements.txt
pip install paddlex==3.0.0b2
```

--------------------------------

### Install PaddleMIX and Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen2_5_omni/README.md

Installs PaddleMIX and its related libraries, including ppdiffusers and requirements.txt. This setup is necessary for running the Qwen2.5-Omni model.

```bash
pip install -e .
pip install -e ppdiffusers
pip install -r requirements.txt
```

--------------------------------

### Single Model Deployment - GroundingDino Example

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/deploy/README_en.md

Guide to exporting and running inference for single models, using GroundingDino as an example.

```APIDOC
## Single Model Prediction Deployment (GroundingDino Example)

### Description
This section details the process of deploying a single model for inference, including exporting the predictive model and performing inference using Python. GroundingDino is used as a practical example.

### Method
N/A (Command Line & Python Script)

### Endpoint
N/A (Command Line & Python Script)

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Exporting Predictive Model

#### Command
```bash
cd deploy/groundingdino
python export.py \
--dino_type GroundingDino/groundingdino-swint-ogc
```

### Performing Prediction

#### Command
```bash
python predict.py  \
--text_encoder_type GroundingDino/groundingdino-swint-ogc \
--model_path output_groundingdino/GroundingDino/groundingdino-swint-ogc \
--input_image https://bj.bcebos.com/v1/paddlenlp/models/community/GroundingDino/000000004505.jpg \
--output_dir ./groundingdino_predict_output \
--prompt "bus"
```

### Response
#### Success Response (200)
- **output_dir** (string) - Directory where prediction results are saved.
- **model_state.pdiparams**, **model_state.pdiparams.info**, **model_state.pdmodel** - Files generated during model export.
```

--------------------------------

### Environment Setup for InstantID

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/InstantID/README.md

Commands to clone the PaddleMIX repository, install the required PaddlePaddle framework version, and set up the necessary dependencies for InstantID.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX
python -m pip install paddlepaddle-gpu==2.6.0.post120 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
cd PaddleMIX/ppdiffusers/examples/InstantID/
pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/wheels/ppdiffusers-0.24.0-py3-none-any.whl --user
pip install -r requirements.txt
```

--------------------------------

### Install PPDiffusers from Source

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/README.md

Commands to clone the repository and perform a manual installation using setup.py.

```shell
git clone https://github.com/PaddlePaddle/PaddleMIX
cd PaddleMIX/ppdiffusers
python setup.py install
```

--------------------------------

### Install PaddlePaddle GPU Version

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/tutorial/tutorial_1_janus.md

Provides example commands for installing the PaddlePaddle deep learning framework with GPU support. It includes options for specific versions (e.g., 3.0.0b2) and development versions, as well as a script-based installation.

```bash
# 3.0.0b2版本安装示例 (CUDA 11.8)
python -m pip install paddlepaddle-gpu==3.0.0b2 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/

# Develop 版本安装示例
python -m pip install paddlepaddle-gpu==0.0.0.post118 -f https://www.paddlepaddle.org.cn/whl/linux/gpu/develop.html

# sh 脚本快速安装
sh build_paddle_env.sh
```

--------------------------------

### Setup YOLO-World Environment

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/YOLO-World/README.md

Commands to clone the PaddleMIX repository, install necessary dependencies including PaddlePaddle, PaddleYOLO, and ppdiffusers, and prepare the directory structure.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX
python -m pip install paddlepaddle-gpu==3.0.0b1 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/
cd PaddleMIX/paddlemix/examples/YOLO-World/
pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/wheels/ppdiffusers-0.24.0-py3-none-any.whl --user
mkdir third_party
git clone https://github.com/PaddlePaddle/PaddleYOLO.git third_party/PaddleYOLO
pip install -e third_party/PaddleYOLO
pip install -e .
mkdir pretrain
```

--------------------------------

### Setup Environment for PaddleMIX

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/blip2/README.md

Commands to navigate to the PaddleMIX directory and install the necessary dependencies from the requirements file.

```bash
cd PaddleMIX
pip install -r requirements.txt
```

--------------------------------

### Example Training Command for Counting Problems - Bash

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/r1_mllm/README.md

This command starts the training for the counting task using the GRPO algorithm, configured for an 8-GPU environment. It calls the relevant script from the PaddleMIX examples directory.

```bash
bash paddlemix/examples/r1_mllm/scripts/run_grpo_counting.sh
```

--------------------------------

### Install PaddleMIX and SDXL Requirements

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/text_to_image/README_sdxl.md

Installs the PaddleMIX library from source and the specific requirements for SDXL examples. This ensures you have the latest versions for running the scripts.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX.git
cd PaddleMIX/ppdiffusers
pip install -e .
pip install -r requirements_sdxl.txt
```

--------------------------------

### Dual Text and Image Guided Generation with PaddleMix

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/applications/image2image/README.md

This example shows how to perform image generation guided by both a text prompt and an input image. It uses the Appflow class with the 'dual_text_and_image_guided_generation' application and the 'versatile-diffusion' model. The output is a new image that combines the content of the input image with the style and subject described in the text prompt.

```Python
from paddlemix.appflow import Appflow
from PIL import Image
from ppdiffusers.utils import load_image

url = "https://paddlenlp.bj.bcebos.com/models/community/CompVis/data/benz.jpg"
image = load_image(url)
prompt = "a red car in the sun"


app = Appflow(app='dual_text_and_image_guided_generation',models=['shi-labs/versatile-diffusion'])
image = app(prompt=prompt,image=image)['result']
image.save("versatile-diffusion-red_car.png")
```

--------------------------------

### Manual Dependency Installation

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/README.md

Manual installation steps for PaddleMIX and ppdiffusers packages using pip.

```bash
pip install -e .
cd ppdiffusers
pip install -e .
cd ..
```

--------------------------------

### Verify Installation

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/README.md

Command to run the environment check script to ensure all dependencies are correctly installed.

```bash
sh check_env.sh
```

--------------------------------

### Download Example Datasets for Fine-tuning

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen2_vl/README.md

Provides wget commands to download small, example datasets ('chartqa' and 'LaTeX_OCR') for fine-tuning the Qwen2-VL model. These datasets include image and annotation directories, with configuration files specified for training.

```bash
wget https://paddlenlp.bj.bcebos.com/models/community/paddlemix/benchmark/playground.tar # 1.0G
wget https://paddlenlp.bj.bcebos.com/datasets/paddlemix/playground/LaTeX_OCR.tar # 1.7G
```

--------------------------------

### Install PaddleMIX Environment Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/ppdocbee2/README.md

Commands to install PaddleMIX and its related dependencies, including paddlenlp. It offers both pip installation and a shell script for quick setup. Ensure these dependencies are installed for proper model operation.

```bash
python -m pip install -e .
python -m pip install -e ppdiffusers
python -m pip install -r requirements.txt
python -m pip install paddlenlp==3.0.0b3

sh build_env.sh
```

--------------------------------

### Qwen2.5-Omni Video Understanding Input/Output Example

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen2_5_omni/README.md

Illustrates the input prompt and the resulting output for video understanding with Qwen2.5-Omni. It includes the system and user prompts with video tokens and the assistant's textual answer.

```text
input:
['<|im_start|>system
You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.<|im_end|>
<|im_start|>user
<|vision_bos|><|VIDEO|><|vision_eos|>What are in this video?<|im_end|>
<|im_start|>assistant
']

output:
["In the video, there are two red pandas. One is on a tree branch, and the other is on the ground. They seem to be playing with some kind of food or toy that's hanging from the tree. It's really cute to watch them interact. What do you think about red pandas?"]
```

--------------------------------

### Install PaddleMIX Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/ppdocbee/README.md

Instructions for installing the necessary dependencies for PaddleMIX, including paddlenlp and other project requirements. It offers both pip installation commands and a shell script for quick setup.

```bash
python -m pip install -e . --user
python -m pip install -e ppdiffusers --user
python -m pip install -r requirements.txt --user
python -m pip install paddlenlp==3.0.0b3 --user
sh build_env.sh
```

--------------------------------

### Qwen2.5-Omni Image Understanding Input/Output Example

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen2_5_omni/README.md

Demonstrates the input prompt and the generated output for image understanding with Qwen2.5-Omni. It shows the system and user messages, including image tokens, and the assistant's text response.

```text
input:
['<|im_start|>system
You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.<|im_end|>
<|im_start|>user
<|vision_bos|><|IMAGE|><|vision_eos|>What are in this image?<|im_end|>
<|im_start|>assistant
']

output:
["Well, in this image, there's a red panda. It's got that cute reddish-brown fur with white markings on its face. It's resting its head on a wooden box, and it looks like it's in a natural setting with some trees and greenery in the background. What do you think about red pandas? They're really interesting animals, aren't they?"]
```

--------------------------------

### Compile and Install Custom Operator

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/models/groundingdino/csrc/README.md

Commands to navigate to the operator directory and install the custom MS Deformable Attention operator using the setup script.

```bash
cd PaddleDetection/ppdet/modeling/transformers/ext_op/
python setup_ms_deformable_attn_op.py install
```

--------------------------------

### Download Example Dataset

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/ppdocbee/README.md

Command to download the 'chartqa' dataset, which serves as a small example dataset for PP-DocBee fine-tuning. The dataset includes image and annotation directories.

```bash
wget https://paddlenlp.bj.bcebos.com/models/community/paddlemix/benchmark/playground.tar # 1.0G
```

--------------------------------

### Qwen2.5-VL Vision Encoder Initialization and Forward Pass

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/tutorial/tutorial_2_qwen2-vl.md

Example of initializing the Qwen2.5-VL vision model with specific configurations and performing a forward pass with sample input data. This demonstrates how to set up the model and process image data for visual understanding.

```python
# Initialize model
config = Qwen2_5_VLVisionConfig(
    spatial_merge_size=2,
    patch_size=16,
    window_size=8,
    fullatt_block_indexes=[0, 1, 2, 3]  # 4 layers of full attention
)
vision_model = Qwen2_5_VisionTransformerPretrainedModel(config)

# Process input
batch_size = 1
image_size = 224
hidden_states = paddle.randn([batch_size, 3, image_size, image_size])
grid_thw = paddle.to_tensor([[1, image_size//16, image_size//16]])  # [T, H, W]

# Forward pass
output = vision_model(hidden_states, grid_thw)
```

--------------------------------

### Running Qwen2.5-VL Image Understanding Example

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/tutorial/tutorial_2_qwen2-vl.md

Command to execute the single image inference script for Qwen2.5-VL. This allows users to test the model's image understanding capabilities by providing an image file, a question, and specifying the data type for inference.

```bash
# Qwen2.5-VL understanding
python paddlemix/examples/qwen2_5_vl/single_image_infer.py \
    --model_path="Qwen/Qwen2.5-VL-3B-Instruct" \
    --image_file="applications/MULLM/examples/haizeiwang.jpeg" \
    --question="请描述这个动漫图片，需要1. 推测动漫是哪一部；2. 给出图片的整体风格；3.描述图像中的细节，并推测可能的背景故事。" \
    --dtype="bfloat16"
```

--------------------------------

### Install PaddleMIX and Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/consistency_distillation/lcm_trainer/README.md

Clones the PaddleMIX repository and installs necessary Python dependencies, including a specific version of PaddlePaddle. This setup is crucial for running LCM models.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX
python -m pip install paddlepaddle-gpu==2.5.2.post117 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
cd PaddleMIX/ppdiffusers/examples/consistency_distillation/lcm_trainer
pip install -r requirements.txt
```

--------------------------------

### Run Multi-turn Chat Demo

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen_vl/README.md

Launches the interactive chat demonstration script for the Qwen-VL model.

```bash
python paddlemix/examples/qwen_vl/chat_demo.py
```

--------------------------------

### Install Dependencies for LVDM Training

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/text_to_video_lvdm/README.md

Installs the ppdiffusers library and its requirements. This step is crucial before running any training scripts.

```bash
cd PaddleMIX/ppdiffusers
python setup.py install
pip install -r requirements.txt
```

--------------------------------

### Clone PaddleMIX Repository and Install Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/AnimateAnyone/README.md

This snippet clones the PaddleMIX repository, installs the paddlepaddle-gpu framework, and sets up the ppdiffusers library and its dependencies for the Animate Anyone example. Ensure your PaddlePaddle version is 2.6.0 or later.

```bash
git clone https://github.com/PaddlePaddle/PaddleMIX

python -m pip install paddlepaddle-gpu==2.6.1.post120 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html

cd PaddleMIX/ppdiffusers
pip install -e .

cd examples/AnimateAnyone/
pip install -r requirements.txt
```

--------------------------------

### Example Training Command for Referring Expression Comprehension - Bash

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/r1_mllm/README.md

This command initiates the training process for the referring expression comprehension task using the GRPO algorithm on an 8-GPU setup. It utilizes the scripts provided within the PaddleMIX examples.

```bash
bash paddlemix/examples/r1_mllm/scripts/run_grpo_rec.sh
```

--------------------------------

### Train Qwen-VL with PaddleMix

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen_vl/README.md

Provides CLI commands to initiate model training. Includes both single-card execution and multi-card distributed training using paddle.distributed.launch.

```bash
export FLAGS_use_cuda_managed_memory=true
MODEL_NAME="qwen-vl/qwen-vl-chat-7b"
DATA="train.json"
python paddlemix/examples/qwen_vl/finetune.py \
    --model_name_or_path ${MODEL_NAME} \
    --data_path ${DATA} \
    --dtype 'bfloat16' \
    --fix_vit True \
    --output_dir output_qwen_vl \
    --num_train_epochs 5 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 16 \
    --save_steps 1000 \
    --save_strategy "steps" \
    --save_total_limit 10 \
    --learning_rate 1e-5 \
    --weight_decay 0.1 \
    --adam_beta2 0.95 \
    --warmup_ratio 0.01 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --report_to "none" \
    --model_max_length 2048 \
    --lazy_preprocess True
```

```bash
MODEL_NAME="qwen-vl/qwen-vl-chat-7b"
MASTER='127.0.0.1:8080'
DATA="train.json"
python -m paddle.distributed.launch --master ${MASTER} --nnodes 1 --nproc_per_node 8 \
paddlemix/examples/qwen_vl/finetune.py \
    --model_name_or_path ${MODEL_NAME} \
    --data_path ${DATA} \
    --dtype 'bfloat16' \
    --fix_vit True \
    --output_dir output_qwen_vl \
    --num_train_epochs 5 \
    --per_device_train_batch_size 1 \
    --gradient_accumulation_steps 16 \
    --save_steps 1000 \
    --save_strategy "steps" \
    --save_total_limit 10 \
    --learning_rate 1e-5 \
    --weight_decay 0.1 \
    --adam_beta2 0.95 \
    --warmup_ratio 0.01 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --report_to "none" \
    --model_max_length 2048 \
    --lazy_preprocess True
```

--------------------------------

### Initialize and Launch Gradio Interface (Python)

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/tutorial/tutorial_4_multimodel_understanding_application.md

Sets up a Gradio interface by creating tabs for traditional QA, anime creation, and fortune-telling, then launches the interface with specified parameters. This function is the entry point for the application's user interface.

```Python
def main():
    """主函数"""
    interface = create_interface()
    interface.queue()
    interface.launch(
        share=True,
        ssr_mode=False,
        max_threads=1  # 限制并发请求数
    )

if __name__ == "__main__":
    main()
```

--------------------------------

### Install Custom Operators (Python/Bash)

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/README_EN.md

Installs optional custom operators like FastLayerNorm and FusedLayerNorm, which are required by certain models (e.g., EVA-CLIP, DIT_LLAMA). This step is only applicable for CUDA environments and is performed by running a setup script within the external_ops directory.

```bash
cd paddlemix/external_ops
python setup.py install
```

--------------------------------

### Initialize Qwen2.5-VL Model and Processor

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/docs/quick_start/image_caption.ipynb

Demonstrates how to load the Qwen2.5-VL model using PaddleMix, including setting the data type to float16 and initializing the processor with the appropriate tokenizer and image processor.

```python
import paddle
from paddlemix.models.qwen2_5_vl import MIXQwen2_5_Tokenizer
from paddlemix.models.qwen2_5_vl.modeling_qwen2_5_vl import Qwen2_5_VLForConditionalGeneration
from paddlemix.processors.qwen2_5_vl_processing import Qwen2_5_VLImageProcessor, Qwen2_5_VLProcessor

model_path = "Qwen/Qwen2.5-VL-3B-Instruct"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(model_path, dtype="float16", attn_implementation="eager")
image_processor = Qwen2_5_VLImageProcessor()
tokenizer = MIXQwen2_5_Tokenizer.from_pretrained(model_path)
processor = Qwen2_5_VLProcessor(image_processor, tokenizer)
```

--------------------------------

### Text-Guided Image Inpainting with Paint By Example

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/README.md

This Python code demonstrates text-guided image inpainting using the PaintByExamplePipeline from ppdiffusers. It utilizes an initial image, a mask, and an example image to guide the inpainting process, allowing for style transfer or content modification within the masked area.

```python
import paddle

from ppdiffusers import PaintByExamplePipeline
from ppdiffusers.utils import load_image

img_url = "https://paddlenlp.bj.bcebos.com/models/community/Fantasy-Studio/data/image_example_1.png"
mask_url = "https://paddlenlp.bj.bcebos.com/models/community/Fantasy-Studio/data/mask_example_1.png"
example_url = "https://paddlenlp.bj.bcebos.com/models/community/Fantasy-Studio/data/reference_example_1.jpeg"

init_image = load_image(img_url).resize((512, 512))
mask_image = load_image(mask_url).resize((512, 512))
example_image = load_image(example_url).resize((512, 512))

pipe = PaintByExamplePipeline.from_pretrained("Fantasy-Studio/Paint-by-Example")

# 使用fp16加快生成速度
with paddle.amp.auto_cast(True):
    image = pipe(image=init_image, mask_image=mask_image, example_image=example_image).images[0]
image.save("image_guided_image_inpainting-paint_by_example-result.png")
```

--------------------------------

### Initiate Model Fine-Tuning

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/community/Hotshot-XL/README.md

Starts the fine-tuning process using a specified data directory and a pre-trained model path. Note that this requires significant VRAM resources.

```shell
python fine_tune.py --data_dir data/ --pretrained_model_name_or_path ./hotshot_output/
```

--------------------------------

### Deploy PP-DocBee2 with Gradio

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/ppdocbee2/README.md

Instructions to deploy the PP-DocBee2 model locally using Gradio. This involves installing the gradio library and then running the provided Python script to start the local deployment.

```bash
pip install gradio==5.6.0
python paddlemix/examples/ppdocbee2/app.py
```

--------------------------------

### Local Gradio Deployment

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/ppdocbee/README.md

Steps to deploy PP-DocBee locally using Gradio. This involves installing the gradio library and then running the provided Python script to start the local web interface for the model.

```bash
pip install gradio==5.6.0
python paddlemix/examples/ppdocbee/app.py
```

--------------------------------

### Deploy ComfyUI using Docker

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/comfyui/README.md

Commands to load a Docker image, initialize a container with GPU support, and start the ComfyUI server. This setup ensures a consistent environment for running PaddleMIX nodes.

```shell
wget https://paddlenlp.bj.bcebos.com/models/community/aistudio/comfyui_docker/comfyui_aistudio_v1.tar
docker load -i comfyui_aistudio_v1.tar
nvidia-docker run --name comfyui_env -it -e HOME="/root" -w "/root" -v </path/to/temp_data_dir>:/root --ipc=host --net=host <docker-image-name> /bin/bash --login
docker exec -it comfyui_env /bin/bash
cd /comfyui_env
./python_env/bin/python ComfyUI/main.py --listen 0.0.0.0 --port 8889 &
```

--------------------------------

### PP-DocBee2 Single-Card Inference

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/ppdocbee2/README.md

Example command for performing single-card inference with the PP-DocBee2 model. It specifies the model path, an input image file, and a question to guide the document understanding task. The output is demonstrated in markdown format.

```bash
python paddlemix/examples/ppdocbee2/ppdocbee2_infer.py \
  --model_path "PaddleMIX/PPDocBee2-3B" \
  --image_file "paddlemix/demo_images/medal_table.png" \
  --question "识别这份表格的内容, 以markdown格式输出"
```

--------------------------------

### Install ppdiffusers library

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/Fast-Diffusers/Training-Free/sortblock/Wan2.1/README.md

Installs the required ppdiffusers library in editable mode for the Wan2.1 project environment.

```shell
python install -e .
```

--------------------------------

### Install T2I-Adapter Dependencies

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/t2i-adapter/README.md

Commands to install the required ppdiffusers library from the develop branch and other necessary project dependencies.

```bash
cd ppdiffusers
python setup.py install
pip install -r requirements.txt
```

--------------------------------

### Execute Custom Training Logic

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/text_to_image_laion400m/README.md

Provides examples for running custom training scripts on single-card and multi-card setups. These commands allow for fine-grained control over training parameters like batch size, learning rate, and model configuration.

```bash
# Single-card training
python -u train_txt2img_laion400m_no_trainer.py \
    --output_dir ./laion400m_pretrain_output_no_trainer \
    --per_device_train_batch_size 16 \
    --gradient_accumulation_steps 2 \
    --learning_rate 5e-5 \
    --weight_decay 0.02 \
    --max_steps 1000000000 \
    --lr_scheduler_type "constant" \
    --warmup_steps 0 \
    --image_logging_steps 1000 \
    --logging_steps 50 \
    --save_steps 5000 \
    --seed 23 \
    --dataloader_num_workers 6 \
    --vae_name_or_path CompVis/stable-diffusion-v1-4/vae \
    --text_encoder_config_file config/ldmbert.json \
    --unet_config_file config/unet.json \
    --file_list ./data/filelist/train.filelist.list \
    --num_inference_steps 200 \
    --model_max_length 77 \
    --tokenizer_name bert-base-uncased \
    --max_grad_norm -1

# Multi-card training
python -u -m paddle.distributed.launch --gpus "0,1,2,3,4,5,6,7" train_txt2img_laion400m_no_trainer.py \
    --output_dir ./laion400m_pretrain_output_no_trainer \
    --per_device_train_batch_size 16 \
    --gradient_accumulation_steps 2 \
    --learning_rate 5e-5 \
    --weight_decay 0.02 \
    --max_steps 1000000000 \
    --lr_scheduler_type "constant" \
    --warmup_steps 0 \
    --image_logging_steps 1000 \
    --logging_steps 50 \
    --save_steps 5000 \
    --seed 23 \
    --dataloader_num_workers 6 \
    --vae_name_or_path CompVis/stable-diffusion-v1-4/vae \
    --text_encoder_config_file config/ldmbert.json \
    --unet_config_file config/unet.json \
    --file_list ./data/filelist/train.filelist.list \
    --num_inference_steps 200 \
    --model_max_length 77 \
    --tokenizer_name bert-base-uncased \
    --max_grad_norm -1
```

--------------------------------

### Environment Setup for Paddle Inference with Triton

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/ppdiffusers/examples/class_conditional_image_generation/DiT/README.md

This section details the environment preparation for high-performance DiT model inference using Paddle Inference and Triton. It includes installing specific versions of PaddlePaddle, Triton, and a compatibility layer for Triton with Paddle.

```bash
# 安装develop版本的paddle
python -m pip install --pre paddlepaddle-gpu -i https://www.paddlepaddle.org.cn/packages/nightly/cu123/

# 安装 triton并适配paddle
python -m pip install triton
python -m pip install git+https://github.com/zhoutianzi666/UseTritonInPaddle.git
python -c "import use_triton_in_paddle; use_triton_in_paddle.make_triton_compatible_with_paddle()"
```

--------------------------------

### Fine-tune Qwen2-VL 2B Model with LoRA

Source: https://github.com/paddlepaddle/paddlemix/blob/develop/paddlemix/examples/qwen2_vl/README.md

Starts fine-tuning the Qwen2-VL 2B model using the LoRA (Low-Rank Adaptation) technique. This script is optimized for multi-GPU setups and includes configurations for batch size and training steps. It's designed to be memory-efficient.

```bash
sh paddlemix/examples/qwen2_vl/shell/baseline_2b_lora_bs32_1e8.sh
```