### Start Docker container for MS-Swift

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Frequently-asked-questions.md

Example command to launch a Docker container with the necessary environment for MS-Swift.

```bash
docker run --gpus all -p 8000:8000 -it -d --name ms modelscope-registry.cn-beijing.cr.aliyuncs.com/modelscope-repo/modelscope:ubuntu22.04-cuda12.4.0-py311-torch2.6.0-1.26.0-LLM /bin/bash
```

--------------------------------

### Install ms-swift from source

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/GetStarted/SWIFT-installation.md

Commands to clone the repository and install ms-swift in editable mode.

```shell
# pip install git+https://github.com/modelscope/ms-swift.git

# Full capabilities
# pip install "git+https://github.com/modelscope/ms-swift.git#egg=ms-swift[all]"

git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e .

# Full capabilities
# pip install -e '.[all]'
```

--------------------------------

### Install JpegIO from source

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/forensics/libs/jpegio/README.md

Recommended installation method involving compilation of C/C++ source code.

```bash
python setup.py install
```

--------------------------------

### Install dependencies for Qwen3-VL

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/Qwen3-VL-Best-Practice.md

Install the required packages including transformers, qwen_vl_utils, and ms-swift. Optionally install vllm if using it as an inference backend.

```shell
pip install "transformers>=4.57" "qwen_vl_utils>=0.0.14"

pip install "ms-swift>=3.9.1"
# pip install "vllm>=0.11.0"  # If using the vLLM inference backend for inference
```

--------------------------------

### Qwen3-Embedding Custom Instruction Example

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/Embedding.md

Example of adding a system instruction to the Qwen3-Embedding input prompt.

```json
{"messages": [
  {"role": "system", "content": "Answer in English and list key points briefly."},
  {"role": "user", "content": "Introduce Qwen3-Embedding"}
]}
```

--------------------------------

### Install ImageMagick 7.1.1 from source

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/cdm_local/README.md

Clones the ImageMagick repository, builds and installs version 7.1.1 from source because apt-get typically installs 6.x. Verify with 'convert --version'.

```bash
git clone https://github.com/ImageMagick/ImageMagick.git ImageMagick-7.1.1

cd ImageMagick-7.1.1

./configure

make

sudo make install

sudo ldconfig /usr/local/lib

convert --version
```

--------------------------------

### Install required packages for training

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/Qwen3-Best-Practice.md

Install ms-swift, transformers, and optional dependencies like deepspeed for multi-GPU training, liger-kernel to save GPU memory, and flash-attn for attention. Run these commands in your environment before starting training.

```bash
pip install ms-swift -U
pip install transformers

pip install deepspeed # for multi-GPU training
pip install liger-kernel # to save GPU memory resources
pip install flash-attn --no-build-isolation  # required for packing
```

--------------------------------

### Install uv and check version

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/wsl_support.md

Install the uv Python package manager via the official script and verify the installation. Expect uv 0.11.x or later.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
source ~/.local/bin/env
uv --version  # should show uv 0.11.x or later
```

--------------------------------

### Installation commands

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/forensics/readme.md

Commands to create a conda environment, install PyTorch with CUDA 12.1, install jpegio from source, and install mmsegmentation with mmcv. Includes steps for building jpegio and mmsegmentation.

```bash
conda create -n monkeyocrv2 python=3.10 -y
conda activate monkeyocrv2
pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 --index-url https://download.pytorch.org/whl/cu121

# install jpegio
cd forensics/libs/jpegio
pip install -r requirements.txt
python setup.py install

# install mmsegmentation
cd ../../
pip install -r requirements.txt
pip install -U openmim
mim install mmengine
mim pip install mmcv==2.2.0 -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.4/index.html
pip install -v -e .
```

--------------------------------

### Install JpegIO on Linux

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/forensics/libs/jpegio/README.md

Install using a pre-compiled wheel for Linux architectures.

```bash
pip install dist/jpegio-x.x.x-cp3x-cp3x-linux_<your architecture>.whl
```

--------------------------------

### Start model service with swift deploy

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/GRPO/DeveloperGuide/reward_model.md

Use this bash command to start the model service with vLLM tensor parallelism. Note: Do not overlap deployment devices with training devices. The output shows the server starting and running on http://0.0.0.0:8000.

```bash
# Note: Do not overlap deployment devices with training devices
CUDA_VISIBLE_DEVICES=0,1,2,3 \
swift deploy \
    --model Qwen/Qwen2.5-72B-Instruct \
    --vllm_tensor_parallel_size 4

# [INFO:swift] model_list: ['Qwen2.5-72B-Instruct']
# INFO:     Started server process [xxxxxx]
# INFO:     Waiting for application startup.
# INFO:     Application startup complete.
# INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
```

--------------------------------

### Install JpegIO on Windows 10 x64

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/forensics/libs/jpegio/README.md

Install using a pre-compiled wheel for Windows 10 x64.

```bash
pip install dist/jpegio-x.x.x-cp3x-cp3x-win_amd64.whl
```

--------------------------------

### Optimizer CPU Offload Example

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Megatron-SWIFT/Command-line-parameters.md

Example command to enable CPU offloading of optimizer states with precision-aware optimizer. Set --use_precision_aware_optimizer true, --optimizer_cpu_offload true, and --optimizer_offload_fraction 0.7.

```bash
--use_precision_aware_optimizer true --optimizer_cpu_offload true --optimizer_offload_fraction 0.7
```

--------------------------------

### Install MS-Swift

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Sample.md

Commands to install the ms-swift package via pip or from the source repository.

```shell
pip install ms-swift[llm] -U
```

```shell
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e '.[llm]'
```

--------------------------------

### Install LaTeX (texlive-full)

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/cdm_local/README.md

Installs the full TeX Live distribution via apt-get, which is required for rendering PDF files.

```bash
apt-get update

sudo apt-get install texlive-full
```

--------------------------------

### File structure after setup

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/wsl_support.md

Shows the directory layout of the MonkeyOCRv2 project after setup, including docs, model weights, parsing source, virtual environment, and service log.

```text
MonkeyOCRv2/
├── docs/
│   └── wsl_support.md          # This guide
├── model_weight/
│   └── MonkeyOCRv2-S-Parsing/
│       ├── config.json
│       ├── configuration_monkeyocrv2.py
│       ├── model.safetensors   # ~1.5GB
│       └── ...
├── parsing/
│   ├── serve.py                # vLLM serve entry point
│   ├── modeling/
│   │   └── modeling_monkeyocrv2_vllm.py  # Custom vLLM model registration
│   └── ...
├── .venv/                      # Python virtual environment
└── vllm_serve.log              # Service log
```

--------------------------------

### Install ms-swift from source with eval extras

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Evaluation.md

Clones the ms-swift repository and installs it in editable mode with the eval extra.

```shell
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
pip install -e '.[eval]'
```

--------------------------------

### Install and run pre-commit for code lint

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/CONTRIBUTING.md

Run these commands in the swift folder to install pre-commit and check all files for code style compliance. Fix any errors reported until all checks pass.

```shell
pip install pre-commit # In the swift folder
pre-commit run --all-files # Fix the errors reported by pre-commit until all checks are successful
```

--------------------------------

### Deploying a model with vLLM backend

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/examples/deploy/README.md

Example command to deploy the Qwen2.5-7B-Instruct model using the vLLM inference backend on a specific GPU.

```shell
CUDA_VISIBLE_DEVICES=0 \
swift deploy \
    --model Qwen/Qwen2.5-7B-Instruct \
    --infer_backend vllm
```

--------------------------------

### Install JpegIO on macOS x64

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/forensics/libs/jpegio/README.md

Install using a pre-compiled wheel for macOS x64.

```bash
pip install dist/jpegio-x.x.x-cp3x-cp3x-macosx_<version>_x86_64.whl
```

--------------------------------

### Serve web demo (Windows)

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/cpu_support.md

Starts the Gradio web demo with the specified model path, output directory, server name, port, and concurrency settings.

```bash
cd .\parsing\
..\.venv\Scripts\python.exe cpu\gradio_demo_cpu.py `
    --model-path ..\model_weight\MonkeyOCRv2-B-Parsing `
    --output-dir output\demo_cpu_outputs `
    --demo-server-name 127.0.0.1 `
    --demo-server-port 8891 `
    --demo-concurrency 1 `
    --page-max-inflight 1
```

--------------------------------

### Install ms-swift and dependencies

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/MLLM-Registration.md

Install the required packages: ms-swift version 3.9 to 3.10, transformers 4.57.*, and qwen_omni_utils 0.0.8.

```shell
pip install "ms-swift>=3.9,<3.10"

pip install "transformers==4.57.*" "qwen_omni_utils==0.0.8"
```

--------------------------------

### Start vLLM serving for document parsing

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/README.md

Starts a vLLM service for document parsing. Use serve.py with -d for DFlash, --tp for tensor parallelism, and -p for port. Run from the parsing directory.

```bash
cd parsing
# Serve with DFlash for faster inference
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing -d ../model_weight/MonkeyOCRv2-B-Parsing-DFlash -p 8888
# Serve without DFlash
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing  -p 8888
# 2 GPUs
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing --tp 2 -p 8888
# Show help messages
python serve.py -h
```

--------------------------------

### Install ms-swift via pip

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/GetStarted/SWIFT-installation.md

Standard installation commands for ms-swift using pip, including options for evaluation and full capabilities.

```shell
# recommend
pip install 'ms-swift'
# For evaluation usage
pip install 'ms-swift[eval]' -U
# Full capabilities
pip install 'ms-swift[all]' -U
```

--------------------------------

### Launch Gradio demo

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/cdm_local/README.md

Starts the Gradio web demo for CDM.

```bash
python app.py
```

--------------------------------

### Install DPText-DETR add-on with install.sh

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/detection/DPText-DETR/README.md

Run from the add-on directory to clone and patch DPText-DETR into ./DPText-DETR.

```bash
bash install.sh            # clones DPText-DETR into ./DPText-DETR and patches it
```

--------------------------------

### Configure UNSLOTH training parameters

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Command-line-parameters.md

Example command-line arguments to enable Unsloth backend with specific training type and quantization settings.

```bash
--tuner_backend unsloth
--train_type full/lora
--quant_bits 4
```

--------------------------------

### Google style docstring example for load function

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/README.md

Shows the required docstring format for functions, including Args, Examples, and Returns sections. The example is a `load` function that loads data from serialized files.

```python
def load(file, file_format=None, **kwargs):
    """Load data from json/yaml/pickle files.

    This method provides a unified api for loading data from serialized files.

    Args:
        file (str or :obj:`Path` or file-like object): Filename or a file-like
            object.
        file_format (str, optional): If not specified, the file format will be
            inferred from the file extension, otherwise use the specified one.
            Currently supported formats include "json", "yaml/yml".

    Examples:
        >>> load('/path/of/your/file')  # file is stored in disk
        >>> load('https://path/of/your/file')  # file is stored on internet
        >>> load('oss://path/of/your/file')  # file is stored in petrel

    Returns:
        The content from the file.
    """
```

--------------------------------

### Install dependencies for GRPO training

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/Qwen3-Best-Practice.md

Install math_verify and vllm==0.8.5.post1, which are required in addition to ms-swift dependencies for GRPO training.

```shell
pip install "math_verify"
pip install vllm==0.8.5.post1
```

--------------------------------

### Install Python requirements

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/cdm_local/README.md

Installs the Python dependencies listed in requirements.txt.

```bash
pip install -r requirements.txt
```

--------------------------------

### Serve with Web Demo

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/README.md

Starts a Gradio web demo for document parsing. Requires a running vLLM service. Access at http://localhost:8891.

```bash
cd parsing
python demo/gradio_demo.py -s http://127.0.0.1:8888 -p 8891
# Show help messages
python demo/gradio_demo.py -h
```

--------------------------------

### Install training environment for MonkeyOCRv2-Parsing

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/README.md

Creates a Conda environment, installs PyTorch for CUDA 12.6, and installs required dependencies including transformers, accelerate, qwen_vl_utils, flash-attn, and the bundled ms-swift package. Run from the train directory. flash-attn is optional but recommended.

```bash
conda create -n monkeyocrv2-train python=3.11 -y
conda activate monkeyocrv2-train

# Example for CUDA 12.6. Adjust this command for your CUDA environment.
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

pip install transformers==4.57.1 accelerate==1.11.0 qwen_vl_utils==0.0.14
pip install flash-attn==2.8.3 --no-build-isolation
pip install -e ./ms-swift
```

--------------------------------

### Install Node.js v16.13.1

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/cdm_local/README.md

Downloads and installs Node.js v16.13.1 from the npmmirror registry, then creates symlinks for node and npm. Verify with 'node -v'.

```bash
wget https://registry.npmmirror.com/-/binary/node/latest-v16.x/node-v16.13.1-linux-x64.tar.gz

tar -xvf node-v16.13.1-linux-x64.tar.gz

mv node-v16.13.1-linux-x64/* /usr/local/nodejs/

ln -s /usr/local/nodejs/bin/node /usr/local/bin

ln -s /usr/local/nodejs/bin/npm /usr/local/bin

node -v
```

--------------------------------

### dataset_info.json metadata examples

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Customization/Custom-dataset.md

Examples of dataset metadata configurations including dataset IDs, paths, subsets, and column mapping.

```json
[
  {
    "ms_dataset_id": "xxx/xxx"
  },
  {
    "dataset_path": "<dataset_dir/dataset_path>"
  },
  {
    "ms_dataset_id": "<dataset_id>",
    "subsets": ["v1"],
    "split": ["train", "validation"],
    "columns": {
      "input": "query",
      "output": "response"
    }
  },
  {
    "ms_dataset_id": "<dataset_id>",
    "hf_dataset_id": "<hf_dataset_id>",
    "subsets": [{
      "subset": "subset1",
      "columns": {
        "problem": "query",
        "content": "response"
      }
    },
    {
      "subset": "subset2",
      "columns": {
        "messages": "_",
        "new_messages": "messages"
      }
    }]
  }
]
```

--------------------------------

### Install vLLM for MonkeyOCRv2 Und

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/README.md

Creates a conda environment and installs vLLM for faster inference. Use this for GPU-based inference with vLLM.

```bash
conda create -n MonkeyOCRv2Und python=3.11
conda activate MonkeyOCRv2Und
pip install uv
uv pip install vllm==0.11.2 --torch-backend=auto -i https://pypi.tuna.tsinghua.edu.cn/simple
```

--------------------------------

### Launch external vLLM server

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/GRPO-Multi-Modal-Training.md

Use this command to start the vLLM rollout server for multi-GPU deployment.

```bash
CUDA_VISIBLE_DEVICES=6,7 \
swift rollout \
    --model Qwen/Qwen2.5-VL-3B-Instruct \
    --vllm_data_parallel_size 2
```

--------------------------------

### Set up conda environment and install ms-swift with NPU dependencies

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/NPU-support.md

Creates a conda virtual environment named swift-npu with Python 3.10, optionally sets a pip mirror, installs ms-swift, torch-npu, deepspeed, and evalscope[opencompass] for evaluation. Use this to prepare the environment for NPU fine-tuning and inference.

```shell
# Create a new conda virtual environment (optional)
conda create -n swift-npu python=3.10 -y
conda activate swift-npu

# Set pip global mirror (optional, to speed up downloads)
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
pip install ms-swift -U

# Install torch-npu
pip install torch-npu decorator
# If you want to use deepspeed (to control memory usage, training speed might decrease)
pip install deepspeed

# If you need the evaluation functionality, please install the following package
pip install evalscope[opencompass]
```

--------------------------------

### Start vLLM service for MonkeyOCRv2 Und

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/README.md

Starts the vLLM inference server for the MonkeyOCRv2-B-Und model on port 8889.

```bash
cd understanding
python serve.py -m ../model_weight/MonkeyOCRv2-B-Und  -p 8889
```

--------------------------------

### Install ms-swift with eval extras via pip

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Evaluation.md

Installs the ms-swift package with the eval extra using pip.

```shell
pip install ms-swift[eval] -U
```

--------------------------------

### Deploy fine-tuned model with swift CLI

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Pre-training-and-Fine-tuning.md

Starts the deployment server for a fine-tuned model. Use --model for full parameter training or --adapters for LoRA checkpoints.

```shell
CUDA_VISIBLE_DEVICES=0 \
swift deploy \
    --adapters output/vx-xxx/checkpoint-xxx \
    --infer_backend pt \
    --temperature 0 \
    --max_new_tokens 2048 \
    --served_model_name '<model-name>'
```

--------------------------------

### Run SFT with YAML configuration

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Command-line-parameters.md

Example command to execute SFT using a YAML configuration file instead of individual command-line arguments.

```shell
swift sft --config demo.yaml
```

--------------------------------

### Pseudo-code: web-ui wraps command line via os.system

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/GetStarted/Web-UI.md

Illustrates that the web-ui starts training/deployment as an independent process using the command line. This is pseudo-code, not actual usage.

```python
import os
os.system('swift sft --model xxx --dataset xxx')
```

--------------------------------

### Example token weights for loss_scale

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Customization/Pluginization.md

Shows example weight settings for tokens, where '数学' and '重要' are given higher weights (2.0) to emphasize them during training.

```text
["学习", "好", "数学", "是", "重要", "的"]
[1.0, 0.5, 2.0, 0.5, 2.0, 0.1]
```

--------------------------------

### Example inference output showing self-awareness

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/Qwen3-Best-Practice.md

Example output from the inference script. The model responds as 'swift-robot' developed by 'swift', demonstrating the self-awareness fine-tuning result.

```text
<<< who are you?
 thinking
Okay, the user asked, "who are you?" I need to introduce myself. Let me start by stating my name, swift-robot. Then, I should mention that I'm an AI assistant developed by swift. I should explain my purpose, which is to provide information and assistance. I should also highlight my capabilities, like answering questions, generating text, and engaging in conversation. It's important to keep the tone friendly and approachable. Maybe add something about being here to help and encourage the user to ask anything. Let me check if I covered all the key points: name, developer, purpose, capabilities, and a welcoming statement. Yeah, that should do it. Now, let me put that into a concise and friendly response.
 response

Hello! I am swift-robot, an artificial intelligence assistant developed by swift. My purpose is to provide information and assistance to users like you. I can answer questions, generate text, and engage in conversations on a wide range of topics. I am here to help, so feel free to ask me anything you need!
```

--------------------------------

### Train Qwen3-Moe Model

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Megatron-SWIFT/Mcore-Bridge.md

Example command for CoT training of the text-only Qwen3-Moe model.

```shell
# 8 * 76GiB, 3s/it
PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True' \
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
NPROC_PER_NODE=8 \
megatron sft \
    --model Qwen/Qwen3-30B-A3B-Instruct-2507 \
    --load_safetensors true \
    --save_safetensors true \
    --dataset 'swift/Chinese-Qwen3-235B-Thinking-2507-Distill-data-110k-SFT#20000' \
    --load_from_cache_file true \
    --split_dataset_ratio 0.01 \
    --moe_permute_fusion true \
    --pipeline_model_parallel_size 2 \
    --decoder_first_pipeline_num_layers 25 \
    --tensor_model_parallel_size 4 \
    --expert_model_parallel_size 4 \
    --moe_grouped_gemm true \
    --moe_shared_expert_overlap true \
    --moe_aux_loss_coeff 1e-6 \
    --micro_batch_size 1 \
    --global_batch_size 4 \
    --recompute_granularity full \
    --recompute_method uniform \
    --recompute_num_layers 1 \
    --max_epochs 1 \
    --finetune true \
    --cross_entropy_loss_fusion true \
    --lr 1e-5 \
    --lr_warmup_fraction 0.05 \
    --min_lr 1e-6 \
    --save megatron_output/Qwen3-30B-A3B-Instruct-2507 \
    --eval_interval 500 \
    --save_interval 500 \
    --max_length 8192 \
    --packing true \
    --num_workers 8 \
    --dataset_num_proc 8 \
    --no_save_optim true \
    --no_save_rng true \
    --sequence_parallel true \
    --moe_expert_capacity_factor 2 \
    --attention_backend flash
```

--------------------------------

### Start CDM Docker container

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/cdm_local/README.md

Starts a container from the cdm image and enters the bash environment. Add '-v xxx:xxx' to map volumes so evaluation results persist on the host.

```bash
docker run -it cdm bash
```

--------------------------------

### YAML configuration file structure

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Command-line-parameters.md

Example content for a YAML configuration file, including model and training arguments.

```yaml
# Model args
model: Qwen/Qwen2.5-7B-Instruct
dataset: swift/self-cognition
...

# Train args
output_dir: xxx/xxx
gradient_checkpointing: true

...
```

--------------------------------

### Serve web demo and show help (Linux)

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/cpu_support.md

Starts the Gradio web demo with the specified model path, output directory, server name, port, and concurrency settings. Also shows help messages for gradio_demo_cpu.py.

```bash
cd parsing
python cpu/gradio_demo_cpu.py \
    --model-path ../model_weight/MonkeyOCRv2-B-Parsing \
    --output-dir output/demo_cpu_outputs \
    --demo-server-name 127.0.0.1 \
    --demo-server-port 8891 \
    --demo-concurrency 1 \
    --page-max-inflight 1
# Show help messages
python cpu/gradio_demo_cpu.py -h
```

--------------------------------

### Run Gradio demo

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/recognition/README.md

Starts the MonkeyOCRv2 demo with a Gradio web interface. After running the command, open http://127.0.0.1:7860 in a browser to access the interface.

```bash
python scripts/demo_gradio.py
```

--------------------------------

### Start vLLM service with nohup

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/wsl_support.md

Starts the vLLM server for MonkeyOCRv2 in the background, logging output to ../vllm_serve.log. Sets WSL2-specific environment variables and uses a GPU memory utilization of 0.55 with a max model length of 8192.

```bash
cd parsing
VLLM_WSL2_ENABLE_PIN_MEMORY=1 \
VLLM_USE_FLASHINFER_SAMPLER=0 \
nohup python serve.py \
  --model-path ../model_weight/MonkeyOCRv2-S-Parsing \
  --gpu-memory-utilization 0.55 \
  --max-model-len 8192 \
  --max-num-batched-tokens 4096 \
  --port 8888 \
  -- --enforce-eager \
  > ../vllm_serve.log 2>&1 &
```

--------------------------------

### Install ms-swift and deepspeed

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Pre-training-and-Fine-tuning.md

Install the latest ms-swift package. If using DeepSpeed ZeRO2/ZeRO3, also install deepspeed.

```shell
pip install ms-swift -U

# If using deepspeed zero2/zero3
pip install deepspeed -U
```

--------------------------------

### Registering a custom dataset with Python

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Customization/Custom-dataset.md

Example of defining a custom preprocessor class and registering a dataset using the DatasetMeta API.

```python
from swift.llm import ResponsePreprocessor, DatasetMeta, register_dataset, SubsetDataset, load_dataset
from typing import Dict, Any

class CustomPreprocessor(ResponsePreprocessor):
    def preprocess(self, row: Dict[str, Any]) -> Dict[str, Any]:
        query = f"""Task: Judge whether the two sentences below are semantically similar.
Sentence 1: {row['text1']}
Sentence 2: {row['text2']}
Output the category [0/1]: 0 for different meanings, 1 for similar meanings.
"""
        response = str(row['label'])
        row = {
            'query': query,
            'response': response
        }
        return super().preprocess(row)


register_dataset(
    DatasetMeta(
        ms_dataset_id='swift/financial_classification',
        subsets=[SubsetDataset('train', split=['train']), SubsetDataset('test', split=['test'])],
        preprocess_func=CustomPreprocessor(),
    ))

if __name__ == '__main__':
    # load_dataset returns train_dataset and val_dataset based on `split_dataset_ratio`
    # Here, since we didn't pass `split_dataset_ratio` (defaults to 0), we take the first one (index 0)
    dataset = load_dataset('swift/financial_classification:train')[0]
    test_dataset = load_dataset('swift/financial_classification:test')[0]
    print(f'dataset[0]: {dataset[0]}')
    print(f'test_dataset[0]: {test_dataset[0]}')
```

--------------------------------

### Multimodal Inference Example

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Inference-and-deployment.md

Example of interactive multimodal inference showing how to input image and video queries with tags, and the model's responses. Demonstrates using `<image>` and `<video>` tags and providing file paths.

```default
<<< <image><image>What is the difference between these two images?
Input an image path or URL <<< http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/cat.png
Input an image path or URL <<< http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/animal.png
The first image depicts a cute, cartoon-style kitten with large, expressive eyes and a fluffy white and gray coat. The background is simple, featuring a gradient of colors that highlight the kitten's face.

The second image shows a group of four cartoon-style sheep standing on a grassy field with mountains in the background. The sheep have fluffy white wool, black legs, and black faces with white markings around their eyes and noses. The background includes green hills and a blue sky with clouds, giving it a pastoral and serene atmosphere.
--------------------------------------------------
<<< clear
<<< <video>Describe this video.
Input a video path or URL <<< https://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/baby.mp4
A baby wearing glasses is sitting on a bed and reading a book. The baby is holding the book with both hands and is looking down at it. The baby is wearing a light blue shirt and pink pants. The baby is sitting on a white pillow. The baby is looking at the book with interest. The baby is not moving much, just turning the pages of the book.
```

--------------------------------

### Install build-essential to fix Triton JIT compilation failure

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/wsl_support.md

Run this command in WSL2 Ubuntu to install the C++ compiler required for full Triton JIT compilation. Alternatively, use the --enforce-eager flag to avoid needing a compiler, though it is slower.

```bash
sudo apt install -y build-essential  # enables full compilation
```

--------------------------------

### Dataset Format Example

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Agent-support.md

Shows the expected format for agent training data, including the [INPUT_IDS] and [LABELS] sections with special tokens and masked labels.

```text
[INPUT_IDS] <|im_start|>system
Answer the following questions as best you can. You have access to the following tools:

realtime_aqi: Call this tool to interact with the realtime_aqi API. What is the realtime_aqi API useful for? Weather forecast. Get real-time air quality, including current air quality, PM2.5, and PM10 information. Parameters: {"type": "object", "properties": {"city": {"type": "string", "description": "City name, e.g., Shanghai"}}, "required": ["city"]} Format the arguments as a JSON object.

Use the following format:

Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [realtime_aqi]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can be repeated zero or more times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question

Begin!
<|im_end|>
<|im_start|>user
What is the weather like in Beijing and Shanghai today?<|im_end|>
<|im_start|>assistant
Action: realtime_aqi
Action Input: {'city': 'Beijing'}
Action: realtime_aqi
Action Input: {'city': 'Shanghai'}
Observation:{"city": "Beijing", "aqi": "10", "unit": "celsius"}
Observation:{"city": "Shanghai", "aqi": "72", "unit": "fahrenheit"}
According to the weather forecast tool, the air quality index (AQI) in Beijing is 10, which indicates good air quality; whereas in Shanghai, the AQI is 72, indicating mild pollution.<|im_end|>

[LABELS] [-100 * 233]Action: realtime_aqi
Action Input: {'city': 'Beijing'}
Action: realtime_aqi
Action Input: {'city': 'Shanghai'}
Observation:[-100 * 45]According to the weather forecast tool, the air quality index (AQI) in Beijing is 10, which indicates good air quality; whereas in Shanghai, the AQI is 72, indicating mild pollution.<|im_end|>
```

--------------------------------

### Train Qwen3-VL Dense Model

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Megatron-SWIFT/Mcore-Bridge.md

Example command for training the multimodal Qwen3-VL model using Megatron-SWIFT.

```shell
# 2 * 76GiB
PYTORCH_CUDA_ALLOC_CONF='expandable_segments:True' \
NPROC_PER_NODE=2 \
IMAGE_MAX_TOKEN_NUM=1024 \
VIDEO_MAX_TOKEN_NUM=128 \
FPS_MAX_FRAMES=16 \
CUDA_VISIBLE_DEVICES=0,1 \
megatron sft \
    --model Qwen/Qwen3-VL-8B-Instruct \
    --load_safetensors true \
    --save_safetensors true \
    --dataset 'AI-ModelScope/LaTeX_OCR:human_handwrite#5000' \
    --load_from_cache_file true \
    --tensor_model_parallel_size 2 \
    --sequence_parallel true \
    --packing true \
    --freeze_llm false \
    --freeze_vit true \
    --freeze_aligner true \
    --split_dataset_ratio 0.01 \
    --micro_batch_size 1 \
    --global_batch_size 4 \
    --recompute_granularity full \
    --recompute_method uniform \
    --recompute_num_layers 1 \
    --finetune true \
    --cross_entropy_loss_fusion true \
    --lr 1e-5 \
    --lr_warmup_fraction 0.05 \
    --min_lr 1e-6 \
    --max_epochs 1 \
    --save megatron_output/Qwen3-VL-8B-Instruct \
    --save_interval 200 \
    --vit_gradient_checkpointing false \
    --max_length 2048 \
    --num_workers 4 \
    --no_save_optim true \
    --no_save_rng true \
    --dataset_num_proc 8
```

--------------------------------

### Define Tools Format

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Agent-support.md

Example structure for defining tool APIs, including name, description, and parameters.

```python
tools = [{
    'type': 'function',
    'function': {
        'name': 'get_current_weather',
        'description': 'Get the current weather in a given location',
        'parameters': {
            'type': 'object',
            'properties': {
                'location': {
                    'type': 'string',
                    'description': 'The city and state, e.g. San Francisco, CA'
                },
                'unit': {
                    'type': 'string',
                    'enum': ['celsius', 'fahrenheit']
                }
            },
            'required': ['location']
        }
    }
}]
```

--------------------------------

### Setting Model-Specific Parameters

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Command-line-parameters.md

Examples of how to pass model-specific parameters using the --model_kwargs flag or environment variables.

```bash
--model_kwargs '{"fps_max_frames": 12}'
```

```bash
FPS_MAX_FRAMES=12
```

--------------------------------

### Download MonkeyOCRv2-Parsing from ModelScope

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/README.md

Install modelscope and use the download script with the modelscope flag.

```bash
pip install modelscope
python ../../download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
```

--------------------------------

### Startup parameters for colocate mode

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/GRPO/GetStarted/GRPO.md

Use these parameters to enable vLLM in colocate mode, where training and inference share GPU resources.

```bash
--use_vllm true \
--vllm_mode colocate
```

--------------------------------

### Build docs with make docs

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/README.md

Run this command in the root directory to build the documentation.

```shell
# in root directory:
make docs
```

--------------------------------

### Launch external vLLM server for GRPO training

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/BestPractices/GRPO-Code-Training.md

Starts a vLLM server with LoRA enabled to support the GRPO training process. Set CUDA_VISIBLE_DEVICES to the desired GPU.

```bash
CUDA_VISIBLE_DEVICES=7 \
swift rollout \
  --model Qwen/Qwen2.5-7B-Instruct \
  --vllm_enable_lora true \
  --vllm_max_lora_rank 16
```

--------------------------------

### Install pdsh from source

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/examples/train/multi-node/deepspeed/README.md

Downloads and compiles pdsh 2.29 with SSH support and a 60-second timeout. Requires root privileges and the build tools (make, gcc).

```shell
cd /root
wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/pdsh/pdsh-2.29.tar.bz2
tar -xvf pdsh-2.29.tar.bz2
cd pdsh-2.29
./configure --prefix=/root/pdsh-2.29 --with-ssh --without-rsh --with-exec --with-timeout=60 --with-nodeupdown --with-rcmd-rank-list=ssh
make
make install
```

--------------------------------

### Create conda environment and install dependencies

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/formula/README.md

Creates a conda environment named 'monkeyocrv2_formula' with Python 3.10, installs PyTorch with CUDA 12.1, and installs project requirements. The CDM requirements are needed for CDM evaluation.

```bash
cd formula
conda create -n monkeyocrv2_formula python=3.10 pip
conda activate monkeyocrv2_formula
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
pip install -r cdm_local/requirements.txt  # required for CDM evaluation
pip install -e .
```

--------------------------------

### Grounding Task Data Format for Qwen2-VL

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Customization/Custom-dataset.md

Example data format for grounding tasks using specific tokens for object references and bounding boxes.

```default
{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "<image>Describe the image."}, {"role": "assistant", "content": "<|object_ref_start|>a dog<|object_ref_end|><|box_start|>(221,423),(569,886)<|box_end|> and <|object_ref_start|>a woman<|object_ref_end|><|box_start|>(451,381),(733,793)<|box_end|> are playing on the beach"}], "images": ["/xxx/x.jpg"]}
{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "<image>Find the <|object_ref_start|>sheep<|object_ref_end|> in the image"}, {"role": "assistant", "content": "<|box_start|>(101,201),(150,266)<|box_end|><|box_start|>(401,601),(550,666)<|box_end|>"}], "images": ["/xxx/x.jpg"]}
{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "<image>Help me open Google Chrome"}, {"role": "assistant", "content": "Action: click(start_box='<|box_start|>(246,113)<|box_end|>')"}], "images": ["/xxx/x.jpg"]}
```

--------------------------------

### Download model weights from ModelScope

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/wsl_support.md

Install modelscope and download the MonkeyOCRv2-S-Parsing model weights from ModelScope, which is faster in China.

```bash
uv pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-S-Parsing
```

--------------------------------

### Define RMPlugin with PTEngine

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/GRPO/DeveloperGuide/reward_model.md

This example shows how to initialize PTEngine within a custom RMPlugin to perform reward model inference.

```python
class RMPlugin(DefaultRMPlugin):

    def __init__(self, model, template):

        super().__init__(model, template)
        # initilize PTEngine to infer
        self.engine = PtEngine.from_model_template(self.model, self.template, max_batch_size=0)

    def __call__(self, inputs):
        system_prompt = ...
        query = ...
        messages = [{'role': 'system', 'content': system_prompt}, {'role': 'query', 'content': query}]
        result = self.engine.infer([messages], self.request_config, use_tqdm=False)
        rewards = ...
        return rewards
```

--------------------------------

### vllm deployment with logprobs and environment variables

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Frequently-asked-questions.md

Command to start a vllm deployment with logprobs enabled. Note that logprobs must be passed from the client side using RequestConfig.

```shell
RAY_memory_monitor_refresh_ms=0 CUDA_VISIBLE_DEVICES=1 nohup swift deploy --ckpt_dir /mnt/workspace/checkpoint_600 --infer_backend vllm --logprobs True --load_data_args false --host 0.0.0.0 --port 8000 &
```

--------------------------------

### Download model weights from ModelScope (Windows)

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/cpu_support.md

Installs modelscope and downloads the MonkeyOCRv2-B-Parsing or MonkeyOCRv2-S-Parsing checkpoint. Use -t modelscope for faster downloads in China.

```bash
.\.venv\Scripts\python.exe -m pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
```

--------------------------------

### Perform batch inference with a fine-tuned model using PtEngine

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/Pre-training-and-Fine-tuning.md

This example demonstrates loading a model with LoRA adapters, setting up a template, and running batch inference with multiple requests.

```python
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

from swift.llm import (
    PtEngine, RequestConfig, safe_snapshot_download, get_model_tokenizer, get_template, InferRequest
)
from swift.tuners import Swift
# Please adjust the following lines
model = 'Qwen/Qwen2.5-VL-7B-Instruct'
lora_checkpoint = safe_snapshot_download('swift/test_grounding')  # Change to your checkpoint_dir
template_type = None  # None: use the default template_type of the corresponding model
default_system = None  # None: use the default system prompt of the corresponding model

# Load model and dialogue template
model, tokenizer = get_model_tokenizer(model)
model = Swift.from_pretrained(model, lora_checkpoint)
template_type = template_type or model.model_meta.template
template = get_template(template_type, tokenizer, default_system=default_system)
engine = PtEngine.from_model_template(model, template, max_batch_size=2)
request_config = RequestConfig(max_tokens=512, temperature=0)

# Using 2 infer_requests to demonstrate batch inference
infer_requests = [
    InferRequest(messages=[{'role': 'user', 'content': 'who are you?'}]),
    InferRequest(messages=[{'role': 'user', 'content': '<image>Task: Object Detection'}],
                 images=['http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/animal.png']),
]
resp_list = engine.infer(infer_requests, request_config)
query0 = infer_requests[0].messages[0]['content']
print(f'response0: {resp_list[0].choices[0].message.content}')
print(f'response1: {resp_list[1].choices[0].message.content}')
```

--------------------------------

### Download model weights from ModelScope (Linux)

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/docs/cpu_support.md

Installs modelscope and downloads the MonkeyOCRv2-B-Parsing or MonkeyOCRv2-S-Parsing checkpoint. Use -t modelscope for faster downloads in China.

```bash
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
```

--------------------------------

### Install older versions of ms-swift

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/GetStarted/SWIFT-installation.md

Command to install version 2.x of ms-swift, which is incompatible with versions 3.0 and later.

```shell
pip install ms-swift==2.*
```

--------------------------------

### Inference output example

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Megatron-SWIFT/Quick-start.md

Example output from the swift infer command, showing the model's self-identification response.

```default
<<< who are you?
I am a language model developed by swift, you can call me swift-robot. How can I assist you?
```

--------------------------------

### Basic RLOO configuration with --advantage_estimator rloo and --kl_in_reward true

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/GRPO/AdvancedResearch/RLOO.md

Command-line arguments to enable RLOO training using GRPOTrainer. Set --advantage_estimator to rloo for the leave-one-out advantage estimator and --kl_in_reward to true to integrate KL divergence into the reward, which is the default for RLOO.

```bash
# Basic RLOO configuration
--advantage_estimator rloo  # Use RLOO's leave-one-out advantage estimator
--kl_in_reward true         # Integrate KL divergence into the reward (default for RLOO)
```

--------------------------------

### Image preprocessing with qwen_vl_utils.fetch_image

Source: https://github.com/yuliang-liu/monkeyocrv2/blob/main/parsing/train/ms-swift/docs/source_en/Instruction/GRPO/AdvancedResearch/deepeyes.md

Shows how to preprocess images for the Qwen2.5-VL series before feeding them to the image_zoom_in_tool. It loads the image as PIL.Image and applies cropping, equivalent to using the MAX_PIXELS environment variable.

```python
from qwen_vl_utils import fetch_image
# At this point, images have not yet been preprocessed
infer_request.images
# Load as PIL.Image and apply cropping (same as using the MAX_PIXELS environment variable)
img = fetch_image({'image': load_pil_image(infer_request.images[0])})
```