### Serving OvisOCR2 with vLLM

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blame/65c619d374b55d4152e85150fc1b003700bc1f0c/model.safetensors

Instructions for installing vLLM, starting the server, and querying it via an OpenAI-compatible API.

```bash
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ATH-MaaS/OvisOCR2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ATH-MaaS/OvisOCR2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'
```

--------------------------------

### Install vLLM

Source: https://huggingface.co/ATH-MaaS/OvisOCR2?local-app=vllm

Install the vLLM package via pip.

```bash
# Install vLLM from pip:
pip install vllm
```

--------------------------------

### Start vLLM Server

Source: https://huggingface.co/ATH-MaaS/OvisOCR2?local-app=vllm

Launch the vLLM server for the OvisOCR2 model.

```bash
# Start the vLLM server:
vllm serve "ATH-MaaS/OvisOCR2"
```

--------------------------------

### Install SGLang

Source: https://huggingface.co/ATH-MaaS/OvisOCR2?local-app=sglang

Install the SGLang package using pip.

```bash
# Install SGLang from pip:
pip install sglang
```

--------------------------------

### Launch SGLang Server

Source: https://huggingface.co/ATH-MaaS/OvisOCR2?local-app=sglang

Start the SGLang server for the OvisOCR2 model on a specified port.

```bash
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ATH-MaaS/OvisOCR2" \
    --host 0.0.0.0 \
    --port 30000
```

--------------------------------

### Install Transformers Library

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/colab

Install or upgrade the transformers library to ensure compatibility with the model.

```bash
!pip install -U transformers
```

--------------------------------

### Serving OvisOCR2 with SGLang

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blame/65c619d374b55d4152e85150fc1b003700bc1f0c/model.safetensors

Setup and execution commands for SGLang, including local server launch and Docker container deployment.

```bash
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ATH-MaaS/OvisOCR2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ATH-MaaS/OvisOCR2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'
```

```bash
docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ATH-MaaS/OvisOCR2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ATH-MaaS/OvisOCR2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'
```

--------------------------------

### Install Inference Dependencies

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blame/65c619d374b55d4152e85150fc1b003700bc1f0c/README.md

Install the required packages for running inference with OvisOCR2.

```bash
pip install "vllm==0.22.1" pillow
```

--------------------------------

### Parse document with OvisOCR2

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blob/d844a45160ff88ed5d51e8c13d7484e50fd2f801/README.md

Basic usage example for initializing the OvisOCR2 parser and processing an image.

```python
parser = OvisOCR2Parser("ATH-MaaS/OvisOCR2")
page_image = Image.open("test1.jpg")
markdown = parser.parse([page_image], filter_imgtags=False)[0]
save_renderable_markdown_with_visual_regions(markdown, page_image, "output")
```

--------------------------------

### Run OvisOCR2 with Docker

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blame/77bfe9462d1e6f8965ee6698f08ea8ede580912c/.gitattributes

Launch the OvisOCR2 model server using the sglang Docker image.

```bash
docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ATH-MaaS/OvisOCR2" \
        --host 0.0.0.0 \
        --port 30000
```

--------------------------------

### Using OvisOCR2 with Transformers

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blame/65c619d374b55d4152e85150fc1b003700bc1f0c/model.safetensors

Demonstrates high-level pipeline usage and direct model loading with AutoProcessor and AutoModelForMultimodalLM.

```python
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ATH-MaaS/OvisOCR2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)
```

```python
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ATH-MaaS/OvisOCR2")
model = AutoModelForMultimodalLM.from_pretrained("ATH-MaaS/OvisOCR2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
```

--------------------------------

### Parse images and save markdown with visual regions

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blob/65c619d374b55d4152e85150fc1b003700bc1f0c/README.md

Defines a function to extract visual regions from markdown and save them as images, followed by an example of initializing the OvisOCR2 parser and processing an image.

```python
def save_renderable_markdown_with_visual_regions(
    markdown: str,
    page_image: Image.Image,
    output_dir: str,
) -> None:
    output_dir = Path(output_dir)
    images_dir = output_dir / "images"
    images_dir.mkdir(parents=True, exist_ok=True)

    width, height = page_image.size
    for left, top, right, bottom in BBOX_IMAGE_PATTERN.findall(markdown):
        x1 = max(0, min(width, round(int(left) * width / 1000)))
        y1 = max(0, min(height, round(int(top) * height / 1000)))
        x2 = max(0, min(width, round(int(right) * width / 1000)))
        y2 = max(0, min(height, round(int(bottom) * height / 1000)))
        if x2 <= x1 or y2 <= y1:
            continue

        crop_path = images_dir / f"bbox_{left}_{top}_{right}_{bottom}.jpg"
        page_image.crop((x1, y1, x2, y2)).convert("RGB").save(crop_path)

    (output_dir / "output.md").write_text(markdown, encoding="utf-8")


parser = OvisOCR2Parser("ATH-MaaS/OvisOCR2")
page_image = Image.open("test1.jpg")
markdown = parser.parse([page_image], filter_imgtags=False)[0]
save_renderable_markdown_with_visual_regions(markdown, page_image, "output")
```

--------------------------------

### Preprocessor Configuration (preprocessor_config.json)

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/commit/77bfe9462d1e6f8965ee6698f08ea8ede580912c

Configuration for image processing, including patch sizes and normalization parameters.

```json
{
"size": {
"longest_edge": 16777216,
"shortest_edge": 65536
},
"patch_size": 16,
"temporal_patch_size": 2,
"merge_size": 2,
"image_mean": [
0.5,
0.5,
0.5
],
"image_std": [
0.5,
0.5,
0.5
],
"processor_class": "Qwen3VLProcessor",
"image_processor_type": "Qwen2VLImageProcessorFast"
}
```

--------------------------------

### Model Configuration (config.json)

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/commit/77bfe9462d1e6f8965ee6698f08ea8ede580912c

Defines the architecture, text configuration, and vision configuration for the Qwen3_5-based model.

```json
{
"architectures": [
"Qwen3_5ForConditionalGeneration"
],
"image_token_id": 248056,
"model_type": "qwen3_5",
"text_config": {
"attention_bias": false,
"attention_dropout": 0.0,
"attn_output_gate": true,
"dtype": "bfloat16",
"eos_token_id": 248044,
"full_attention_interval": 4,
"head_dim": 256,
"hidden_act": "silu",
"hidden_size": 1024,
"initializer_range": 0.02,
"intermediate_size": 3584,
"layer_types": [
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention",
"linear_attention",
"linear_attention",
"linear_attention",
"full_attention"
],
"linear_conv_kernel_dim": 4,
"linear_key_head_dim": 128,
"linear_num_key_heads": 16,
"linear_num_value_heads": 16,
"linear_value_head_dim": 128,
"max_position_embeddings": 262144,
"mlp_only_layers": [],
"model_type": "qwen3_5_text",
"mtp_num_hidden_layers": 1,
"mtp_use_dedicated_embeddings": false,
"num_attention_heads": 8,
"num_hidden_layers": 24,
"num_key_value_heads": 2,
"rms_norm_eps": 1e-06,
"tie_word_embeddings": true,
"use_cache": true,
"vocab_size": 248320,
"mamba_ssm_dtype": "float32",
"rope_parameters": {
"mrope_interleaved": true,
"mrope_section": [
11,
11,
10
],
"rope_type": "default",
"rope_theta": 10000000,
"partial_rotary_factor": 0.25
}
},
"tie_word_embeddings": true,
"transformers_version": "4.57.0.dev0",
"video_token_id": 248057,
"vision_config": {
"deepstack_visual_indexes": [],
"depth": 12,
"hidden_act": "gelu_pytorch_tanh",
"hidden_size": 768,
"in_channels": 3,
"initializer_range": 0.02,
"intermediate_size": 3072,
"model_type": "qwen3_5",
"num_heads": 12,
"num_position_embeddings": 2304,
"patch_size": 16,
"spatial_merge_size": 2,
"temporal_patch_size": 2
},
"vision_end_token_id": 248054,
"vision_start_token_id": 248053
}
```

--------------------------------

### Preprocessor Configuration

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blob/65c619d374b55d4152e85150fc1b003700bc1f0c/preprocessor_config.json

Configuration settings for the OvisOCR2 preprocessor.

```json
{
  "size": {
  "longest_edge": 16777216,
  "shortest_edge": 65536
  },
  "patch_size": 16,
  "temporal_patch_size": 2,
  "merge_size": 2,
  "image_mean": [
  0.5,
  0.5,
  0.5
  ],
  "image_std": [
  0.5,
  0.5,
  0.5
  ],
  "processor_class": "Qwen3VLProcessor",
  "image_processor_type": "Qwen2VLImageProcessorFast"
}
```

--------------------------------

### OvisOCR2 Configuration

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blob/65c619d374b55d4152e85150fc1b003700bc1f0c/video_preprocessor_config.json

JSON configuration for the video preprocessor.

```json
{
  "size": {
  "longest_edge": 25165824,
  "shortest_edge": 4096
  },
  "patch_size": 16,
  "temporal_patch_size": 2,
  "merge_size": 2,
  "image_mean": [
  0.5,
  0.5,
  0.5
  ],
  "image_std": [
  0.5,
  0.5,
  0.5
  ],
  "processor_class": "Qwen3VLProcessor",
  "video_processor_type": "Qwen3VLVideoProcessor"
}
```

--------------------------------

### Video Preprocessor Configuration

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/commit/77bfe9462d1e6f8965ee6698f08ea8ede580912c

Configuration for video processing, including patch sizes and normalization parameters.

```json
{
  "size": {
  "longest_edge": 25165824,
  "shortest_edge": 4096
  },
  "patch_size": 16,
  "temporal_patch_size": 2,
  "merge_size": 2,
  "image_mean": [
  0.5,
  0.5,
  0.5
  ],
  "image_std": [
  0.5,
  0.5,
  0.5
  ],
  "processor_class": "Qwen3VLProcessor",
  "video_processor_type": "Qwen3VLVideoProcessor"
  }
```

--------------------------------

### Query vLLM via OpenAI-compatible API

Source: https://huggingface.co/ATH-MaaS/OvisOCR2?local-app=vllm

Send a request to the local vLLM server using curl to perform image description.

```bash
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ATH-MaaS/OvisOCR2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'
```

--------------------------------

### Tokenizer Configuration JSON

Source: https://huggingface.co/ATH-MaaS/OvisOCR2/blame/65c619d374b55d4152e85150fc1b003700bc1f0c/tokenizer_config.json

Configuration object defining tokenizer behavior, special tokens, and prompt templates for the Qwen2Tokenizer.

```json
{
    "chat_template": "{% if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is true %}
        {{- '<think>\n' }}
    {%- else %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}",
    "clean_up_tokenization_spaces": false,
    "eos_token": "<|im_end|>",
    "errors": "replace",
    "model_max_length": 262144,
    "pad_token": "<|endoftext|>",
    "split_special_tokens": false,
    "tokenizer_class": "Qwen2Tokenizer",
    "unk_token": null,
    "add_bos_token": false,
    "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?[\p{L}\p{M}]+|\p{N}| ?[^\s\p{L}\p{M}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+",
    "extra_special_tokens": {
        "audio_bos_token": "<|audio_start|>",
        "audio_eos_token": "<|audio_end|>",
        "audio_token": "<|audio_pad|>",
        "image_token": "<|image_pad|>",
        "video_token": "<|video_pad|>",
        "vision_bos_token": "<|vision_start|>",
        "vision_eos_token": "<|vision_end|>"
    }
}
```