### Install SGLang and Dependencies

Source: https://github.com/baidu/unlimited-ocr/blob/main/README.md

Installs the SGLang wheel, kernels, and PyMuPDF for environment setup.

```shell
uv venv --python 3.12
source .venv/bin/activate

uv pip install wheel/sglang-0.0.0.dev11416+g92e8bb79e-py3-none-any.whl
uv pip install kernels==0.11.7
uv pip install pymupdf==1.27.2.2
```

--------------------------------

### Install Project Requirements

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Use this command to install all necessary Python packages for the project. Ensure your environment meets the tested version requirements.

```bash
pip install torch==2.10.0 torchvision==0.25.0 transformers==4.57.1 \
    Pillow==12.1.1 einops==0.8.2 addict==2.4.0 easydict==1.13 \
    pymupdf==1.27.2.2 psutil==7.2.2 requests
```

--------------------------------

### Start SGLang Server Process

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/module-structure.md

Initiates the SGLang server process, either starting a new instance with specified model and GPU configurations or reusing an existing one. It handles server startup, logging, and readiness polling. Server arguments are passed via a Namespace object.

```python
def start_server(args) -> subprocess.Popen | None:
    # ... implementation details ...
    pass
```

--------------------------------

### Start and Stop SGLang Server

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Demonstrates how to start an SGLang server process and then gracefully stop it. Ensure the process object is managed correctly, especially in try-finally blocks.

```python
from infer import start_server, stop_server
import argparse

args = argparse.Namespace(model_dir='baidu/Unlimited-OCR', gpu='0', server_log='./log/server.log')
process = start_server(args)
try:
    # Use server
    pass
finally:
    stop_server(process)
```

--------------------------------

### Start SGLang Server

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Launches the SGLang server with the Unlimited-OCR model or reuses an existing server. Requires parsed arguments for configuration.

```python
import argparse
from infer import start_server

args = argparse.Namespace(
    model_dir='baidu/Unlimited-OCR',
    gpu='0',
    server_log='./log/sglang_server.log'
)
process = start_server(args)
# Returns: subprocess.Popen object or None if server already running
```

--------------------------------

### Launch SGLang Server for Unlimited-OCR

Source: https://github.com/baidu/unlimited-ocr/blob/main/README.md

Starts the SGLang server with specified model and configuration for OCR tasks.

```shell
python -m sglang.launch_server \
    --model baidu/Unlimited-OCR \
    --served-model-name Unlimited-OCR \
    --attention-backend fa3 \
    --page-size 1 \
    --mem-fraction-static 0.8 \
    --context-length 32768 \
    --enable-custom-logit-processor \
    --disable-overlap-schedule \
    --skip-server-warmup \
    --host 0.0.0.0 \
    --port 10000
```

--------------------------------

### Huggingface Transformers Inference Setup

Source: https://github.com/baidu/unlimited-ocr/blob/main/README.md

Installs and loads the Unlimited-OCR model and tokenizer from Huggingface transformers. Ensure you have the specified PyTorch and CUDA versions.

```python
import os
import torch
from transformers import AutoModel, AutoTokenizer

model_name = 'baidu/Unlimited-OCR'

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_name,
    trust_remote_code=True,
    use_safetensors=True,
    torch_dtype=torch.bfloat16,
)
model = model.eval().cuda()
```

--------------------------------

### Run Inference After Container Setup

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Execute OCR inference using the locally running Unlimited OCR service, typically after starting the service via Docker or another method.

```bash
# Then run inference
python infer.py --image_dir ./images --output_dir ./results
```

--------------------------------

### Python Requests Example for OCR

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

This Python script demonstrates how to send an OCR request with an image and process the streaming response. Ensure you have the 'requests' library installed.

```python
import requests
import json
import base64

# Read and encode image
with open('document.png', 'rb') as f:
    image_data = base64.b64encode(f.read()).decode('utf-8')

# Build request
payload = {
    "model": "Unlimited-OCR",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "document parsing."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/png;base64,{image_data}"
                    }
                }
            ]
        }
    ],
    "temperature": 0,
    "skip_special_tokens": False,
    "stream": True,
    "images_config": {
        "image_mode": "gundam"
    },
    "custom_logit_processor": "DeepseekOCRNoRepeatNGramLogitProcessor(...)",
    "custom_params": {
        "ngram_size": 35,
        "window_size": 128
    }
}

# Send request
response = requests.post(
    "http://127.0.0.1:10000/v1/chat/completions",
    headers={"Content-Type": "application/json"},
    json=payload,
    timeout=1200,
    stream=True
)

# Process streaming response
for line in response.iter_lines(decode_unicode=True):
    if not line or not line.startswith("data:"):
        continue
    data = line[len("data:"):
].strip()
    if data == "[DONE]":
        break
    chunk = json.loads(data)
    content = chunk["choices"][0]["delta"].get("content", "")
    if content:
        print(content, end="", flush=True)
```

--------------------------------

### APIPayload Example for Unlimited-OCR

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/types.md

An example of the complete request body for the SGLang OpenAI-compatible API, including model details, messages, and configuration for OCR tasks. This payload is sent by infer_one().

```python
{
    "model": "Unlimited-OCR",
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "document parsing."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "data:image/jpeg;base64,/9j/4AAQSkZJRg..."
                    }
                }
            ]
        }
    ],
    "temperature": 0,
    "skip_special_tokens": False,
    "stream": True,
    "images_config": {"image_mode": "gundam"},
    "custom_logit_processor": "DeepseekOCRNoRepeatNGramLogitProcessor(...)",
    "custom_params": {
        "ngram_size": 35,
        "window_size": 128
    }
}
```

--------------------------------

### Constructing ArgumentNamespace

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/types.md

Example of how to manually construct an argparse.Namespace object with common OCR parameters. This is useful for testing or programmatic configuration.

```python
import argparse
args = argparse.Namespace(
    image_dir='./images',
    pdf='',
    output_dir='./outputs',
    concurrency=8,
    gpu='0',
    model_dir='baidu/Unlimited-OCR',
    image_mode='gundam',
    server_log='./log/server.log'
)
```

--------------------------------

### Add Logging

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Configure Python's logging module to output informational and error messages to both a file and the console. Log inference start and success/failure.

```python
import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('infer.log'),
        logging.StreamHandler()
    ]
)

logger = logging.getLogger(__name__)

def infer_one(image_path: str, output_file: str | None, args, idx: int) -> dict:
    logger.info(f"[{idx}] Starting inference on {os.path.basename(image_path)}")
    
    try:
        result = _do_inference(...)
        logger.info(f"[{idx}] Success: {result['tokens']} tokens")
        return result
    except Exception as e:
        logger.error(f"[{idx}] Failed: {e}", exc_info=True)
        return {"tokens": 0, "decode_time": 0, "text": ""}

```

--------------------------------

### First Chunk Response Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

This JSON snippet represents the first chunk of a streaming response, indicating the role of the assistant.

```json
data: {"choices":[{"index":0,"delta":{"role":"assistant","content":""}},"finish_reason":null}],"model":"Unlimited-OCR","object":"text_completion.chunk","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
```

--------------------------------

### Python: Simple Inference

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Perform a single inference on an image after starting the OCR server. Ensure the server is stopped afterwards.

```python
from infer import infer_one, start_server, stop_server
import argparse

# Start server
args = argparse.Namespace(
    model_dir='baidu/Unlimited-OCR',
    gpu='0',
    server_log='./log/server.log'
)
process = start_server(args)

try:
    # Run single inference
    args.image_mode = 'gundam'
    result = infer_one('document.png', 'output.md', args, 1)
    print(f"Tokens: {result['tokens']}, Time: {result['decode_time']:.1f}s")
finally:
    stop_server(process)
```

--------------------------------

### Restart Server

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

A command to forcefully stop the sglang server and then restart it. Use this when the server won't start.

```bash
pkill -f sglang; python infer.py ...
```

--------------------------------

### Dockerfile for Unlimited OCR

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

This Dockerfile sets up an Ubuntu 22.04 environment with CUDA 12.9, installs Python dependencies, and copies the application code for running inference.

```dockerfile
FROM nvidia/cuda:12.9.0-runtime-ubuntu22.04

WORKDIR /app

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY infer.py .
COPY wheel/ ./wheel

ENTRYPOINT ["python", "infer.py"]
```

--------------------------------

### Health Check Response Examples

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

The /health endpoint can return a JSON status or a simple OK string.

```json
{"status": "ready"}
```

```text
OK
```

--------------------------------

### Load and Infer with HuggingFace Transformers

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/project-overview.md

Demonstrates how to load the Unlimited-OCR model using HuggingFace transformers and perform single-image inference. Ensure the transformers library is installed and the model name is correct.

```python
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained('baidu/Unlimited-OCR', trust_remote_code=True)
model.infer(tokenizer, prompt='<image>document parsing.', image_file='...')
```

--------------------------------

### Manual Recovery: Reduce Server Load

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Provides examples for reducing server load by decreasing concurrency or adjusting image processing parameters. This can help prevent server overload and timeouts.

```bash
# Reduce concurrency to ease server load
# python infer.py --concurrency 2
```

```bash
# Reduce concurrency to give server more resources
--concurrency 2
```

--------------------------------

### Run Unlimited-OCR Inference

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/README.md

Execute the main inference script with specified image and output directories. This is the primary command to start processing images.

```bash
python infer.py --image_dir ./images --output_dir ./results
```

--------------------------------

### Curl Example for OCR Request

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

This bash command demonstrates how to make an OCR request using curl. It includes the necessary headers and a JSON payload with an image URL.

```bash
curl -X POST http://127.0.0.1:10000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Unlimited-OCR",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "document parsing."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "data:image/png;base64,iVBORw0KG..."
            }
          }
        ]
      }
    ],
    "temperature": 0,
    "skip_special_tokens": false,
    "stream": true,
    "images_config": {
      "image_mode": "gundam"
    }
  }' \
  --max-time 1200
```

--------------------------------

### Create an OCR Web Service with Flask

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Expose OCR functionality via a web service using Flask. The server starts automatically and handles image processing requests. Ensure the server is stopped gracefully.

```python
from flask import Flask, request, jsonify
from infer import infer_one, start_server, stop_server
import argparse

app = Flask(__name__)
server_process = None

@app.before_first_request
def startup():
    global server_process
    args = argparse.Namespace(
        model_dir='baidu/Unlimited-OCR',
        gpu='0',
        server_log='./log/server.log'
    )
    server_process = start_server(args)

@app.route('/ocr', methods=['POST'])
def process_image():
    image_file = request.files['image']
    image_path = f'/tmp/{image_file.filename}'
    image_file.save(image_path)
    
    args = argparse.Namespace(image_mode='gundam')
    result = infer_one(image_path, None, args, 1)
    
    return jsonify(result)

@app.teardown_appcontext
def shutdown(exception):
    if server_process:
        stop_server(server_process)

if __name__ == '__main__':
    app.run(debug=False, port=5000)
```

--------------------------------

### Free GPU Memory

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

If the server won't start due to memory issues, this command can help free up GPU memory by terminating PyTorch processes.

```bash
pkill -f torch
```

--------------------------------

### Python Function Signature Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/README.md

Illustrates the expected format for Python function signatures, including type annotations and default parameter values. This convention is used throughout the documentation.

```python
def function_name(param1: type1, param2: type2 = default) -> return_type:

```

--------------------------------

### Queue OCR Results to a Message Broker

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Publishes OCR results to a message broker using HTTP POST requests. This example uses the requests library, but can be adapted for Kafka or RabbitMQ. Ensure the broker URL is correctly configured.

```python
import json
import requests
import os
import time
import logging

logger = logging.getLogger(__name__)

def publish_result(result: dict, image_path: str, broker_url: str):
    """Publish result to message broker."""
    payload = {
        "image": os.path.basename(image_path),
        "tokens": result['tokens'],
        "text": result['text'],
        "timestamp": time.time()
    }
    
    response = requests.post(
        f"{broker_url}/publish",
        json=payload,
        timeout=10
    )
    response.raise_for_status()

# In infer_one():
def infer_one(image_path: str, output_file: str | None, args, idx: int) -> dict:
    result = _do_inference(...)
    
    if hasattr(args, 'broker_url'):
        try:
            publish_result(result, image_path, args.broker_url)
        except Exception as e:
            logger.warning(f"Failed to publish result: {e}")
    
    return result
```

--------------------------------

### Check SGLang Server Readiness

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/module-structure.md

Use this function to verify if the SGLang server is operational and accessible. It sends a GET request to the /health endpoint and checks for a 200 OK status. The check has a 5-second timeout.

```python
def server_ready(server_url: str) -> bool:
    # ... implementation details ...
    pass
```

--------------------------------

### OCR Metrics Output Format

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/project-overview.md

This is an example of the metrics printed to the console during OCR processing. It includes details about the mode, request counts, concurrency, and individual file processing times, as well as a summary of concurrent results.

```text
Mode: {pdf_pages|dataset_images}, requests={N}, concurrency={C}, image_mode={mode}
  [{idx}] {filename}: {tokens} tokens, {time}s
...
============================================================
Concurrent Results:
  Requests: {success}/{total}
  Total tokens: {sum}
  Wall time: {seconds}
  System TPS: {tokens/s}
  Avg tokens/request: {avg}
  Avg decode_time/request: {avg_s}
============================================================

```

--------------------------------

### start_server

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Launches SGLang server with Unlimited-OCR model or reuses an existing running server. Returns a Popen object or None.

```APIDOC
## start_server

### Description
Launches SGLang server with Unlimited-OCR model or reuses existing running server.

### Method
```python
def start_server(args) -> subprocess.Popen | None:
```

### Parameters
#### Path Parameters
- **args** (argparse.Namespace) - Required - Parsed command-line arguments with fields: model_dir, gpu, server_log

### Returns
- `subprocess.Popen | None` - Process object if new server was started, None if existing server was reused.

### Throws
- `RuntimeError` - If server process exits unexpectedly before becoming ready.
- `TimeoutError` - If server does not become ready within SERVER_TIMEOUT (300s).

### Server Configuration
- Attention backend: fa3
- Page size: 1
- Memory fraction: 0.8
- Context length: 32768
- Host: 0.0.0.0, Port: 10000

### Example
```python
import argparse
from infer import start_server

args = argparse.Namespace(
    model_dir='baidu/Unlimited-OCR',
    gpu='0',
    server_log='./log/sglang_server.log'
)
process = start_server(args)
# Returns: subprocess.Popen object or None if server already running
```
```

--------------------------------

### Show Help for infer.py

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Display the help message for the infer.py script to understand available options and arguments.

```bash
# Show help
python infer.py --help
```

--------------------------------

### Inspect Output Files Commands

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

List output files in long format and display the first 50 lines of a specific markdown output file.

```bash
ls -lh ./outputs
```

```bash
head -50 ./outputs/image001.md
```

--------------------------------

### Useful Options for infer.py

Source: https://github.com/baidu/unlimited-ocr/blob/main/README.md

Lists common command-line options for the `infer.py` script, including model directory, GPU selection, and server log path.

```shell
--model_dir baidu/Unlimited-OCR   # Local path or Hugging Face model ID
--gpu 0                           # CUDA_VISIBLE_DEVICES value
--server_log ./log/sglang_server.log
```

--------------------------------

### Dynamic Prompt Per Image Function

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Implement dynamic prompting by modifying the `build_content` function to accept a prompt argument. This allows for per-image prompt customization.

```python
def build_content(image_path: str, prompt: str = None) -> list[dict]:
    if prompt is None:
        prompt = PROMPT
    return [{"type": "text", "text": prompt}, encode_image(image_path)]
```

--------------------------------

### Monitor Server Logs

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Use command-line tools to monitor server logs in real-time, search for specific error messages, or check GPU utilization.

```bash
# Real-time monitoring
tail -f ./log/sglang_server.log
```

```bash
# Search for errors
grep -i error ./log/sglang_server.log
```

```bash
# Monitor GPU
watch -n 1 nvidia-smi
```

--------------------------------

### Load Configuration from File

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Implement a `load_config` function to read settings from a JSON configuration file. This function is used in `main()` to load configurations, which can then be applied as defaults if not overridden by CLI arguments.

```python
import json

def load_config(path: str = "config.json") -> dict:
    with open(path) as f:
        return json.load(f)

def main():
    args = parse_args()
    config = load_config()
    
    # Apply config defaults if not overridden by CLI
    if not args.image_mode:
        args.image_mode = config["processing"]["image_mode"]
    
    # ... rest of main ...
```

--------------------------------

### Configuration File Structure

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Define a `config.json` file to manage server, inference, and processing settings. This allows for centralized configuration management and easier deployment.

```json
{
    "server": {
        "host": "0.0.0.0",
        "port": 10000,
        "timeout": 300,
        "memory_fraction": 0.8
    },
    "inference": {
        "temperature": 0,
        "context_length": 32768,
        "ngram_size": 35
    },
    "processing": {
        "pdf_dpi": 300,
        "image_mode": "gundam"
    }
}
```

--------------------------------

### Verify Output Directory Contents

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

List the contents of the output directory to confirm file creation and check for any unexpected items.

```bash
ls -la ./outputs
```

--------------------------------

### Middle Chunks Response Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

These JSON snippets represent middle chunks of a streaming response, containing the actual content being sent.

```json
data: {"choices":[{"index":0,"delta":{"content":"The"}},"finish_reason":null}],"model":"Unlimited-OCR","object":"text_completion.chunk","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
```

```json
data: {"choices":[{"index":0,"delta":{"content":" document"}},"finish_reason":null}],"model":"Unlimited-OCR","object":"text_completion.chunk","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
```

--------------------------------

### Create Temporary Directory

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/configuration.md

This code snippet demonstrates how to create a temporary directory for storing converted PNG images. Manual cleanup of this directory is required.

```python
tempfile.mkdtemp(prefix='pdf_ocr_')
```

--------------------------------

### ImageDataURI Examples

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/types.md

Illustrates the format of ImageDataURI, which is a Base64-encoded image in data URL format. Used for embedding images directly in requests.

```text
data:image/jpeg;base64,/9j/4AAQSkZJRg...
data:image/png;base64,iVBORw0KGgoAAAAN...
data:image/webp;base64,UklGRiYAAAB...
```

--------------------------------

### Total Output Size Command

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Calculate and display the total disk usage of the outputs directory in a human-readable format.

```bash
du -sh ./outputs
```

--------------------------------

### Add Prometheus Metrics

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Define and increment Prometheus metrics for OCR requests and token generation. Start an HTTP server to expose metrics.

```python
from prometheus_client import Counter, Histogram, start_http_server

# Define metrics
requests_total = Counter(
    'ocr_requests_total',
    'Total OCR requests',
    ['status']
)

request_duration = Histogram(
    'ocr_request_duration_seconds',
    'Request duration',
    buckets=[1, 5, 10, 30, 60, 300, 1200]
)

tokens_total = Counter(
    'ocr_tokens_total',
    'Total tokens generated'
)

def infer_one(image_path: str, output_file: str | None, args, idx: int) -> dict:
    start_time = time.time()
    
    try:
        result = _do_inference(image_path, output_file, args, idx)
        
        duration = time.time() - start_time
        status = "success" if result['tokens'] > 0 else "failure"
        
        requests_total.labels(status=status).inc()
        request_duration.observe(duration)
        tokens_total.inc(result['tokens'])
        
        return result
    except Exception as e:
        requests_total.labels(status="error").inc()
        raise

def main():
    args = parse_args()
    
    # Start Prometheus metrics server
    start_http_server(8000)
    
    # ... rest of main ...

```

--------------------------------

### build_jobs

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Constructs a list of (image_path, output_file) tuples. It can process either a PDF file (converting each page to an image) or a directory of images. Output file names are generated based on the input source.

```APIDOC
## build_jobs

### Description
Constructs list of (image_path, output_file) tuples from either PDF or image directory.

### Signature
```python
def build_jobs(args) -> list[tuple[str, str | None]]:
```

### Parameters
#### Path Parameters
- **args** (argparse.Namespace) - Required - Parsed args with pdf, image_dir, output_dir fields

### Returns
- `list[tuple[str, str | None]]` - List of (image_path, output_file) tuples. `output_file` is None if `args.output_dir` is empty.

### PDF Mode (`args.pdf` set)
- Converts PDF to images via `pdf_to_images`.
- Output files named: `{pdf_stem}_page_{i+1:04d}.md`

### Image Directory Mode (`args.image_dir` set)
- Collects images via `collect_dataset_images`.
- Output files named: `{relative_path_stem}.md` with path separators replaced by "__"

### Throws
- `ValueError` if neither `pdf` nor `image_dir` is specified.

### Example
```python
import argparse
from infer import build_jobs

# PDF mode
args = argparse.Namespace(
    pdf='document.pdf',
    image_dir='',
    output_dir='./outputs'
)
jobs = build_jobs(args)

# Image directory mode
args.pdf = ''
args.image_dir = './images'
jobs = build_jobs(args)
```
```

--------------------------------

### Streaming Response Chunk Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

This JSON structure represents a chunk of a streaming response, containing partial assistant content and model information.

```json
{
    "choices": [{
        "index": 0,
        "delta": {
            "role": "assistant",
            "content": "Extracted text..."
        },
        "finish_reason": null
    }],
    "model": "Unlimited-OCR",
    "object": "text_completion.chunk"
}
```

--------------------------------

### Final Chunk Response Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

This JSON snippet represents the final chunk of a streaming response before the stream terminator, indicating the end of the content.

```json
data: {"choices":[{"index":0,"delta":{"content":"..."}},"finish_reason":"stop"}],"model":"Unlimited-OCR","object":"text_completion.chunk","usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
```

--------------------------------

### Enable Detailed Logging for Stability

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Use the `--server_log` argument to enable detailed logging, which can help in diagnosing stability issues.

```bash
# Enable detailed logging
--server_log ./detailed.log
```

--------------------------------

### Set Output Directory Permissions

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Ensure the output directory has the correct permissions to allow file creation. Use this command if output files are not being created.

```bash
chmod 755 ./outputs
```

--------------------------------

### Run vLLM Container for OCR

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Launch a Docker container with vLLM pre-configured for Unlimited OCR. This simplifies deployment and dependency management.

```bash
# Using vLLM container
docker run --gpus all -p 10000:10000 vllm/vllm-openai:unlimited-ocr
```

--------------------------------

### Failed Inference Result Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

This JSON object indicates a failed OCR inference, with zero tokens, zero decode time, and empty extracted text.

```json
{
    "tokens": 0,
    "decode_time": 0,
    "text": ""
}
```

--------------------------------

### Successful Inference Result Example

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

This JSON object represents a successful OCR inference result, including token count, decode time, and the extracted text.

```json
{
    "tokens": 1234,
    "decode_time": 12.5,
    "text": "# Document Title\n\nParagraph 1...\n\nParagraph 2..."
}
```

--------------------------------

### Build Processing Jobs (Python)

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Constructs a list of (image_path, output_file) tuples. It supports processing from either a PDF file (converting pages to images) or an image directory. Output filenames are generated based on the input source and mode.

```python
import argparse
from infer import build_jobs

# PDF mode
args = argparse.Namespace(
    pdf='document.pdf',
    image_dir='',
    output_dir='./outputs'
)
jobs = build_jobs(args)

# Image directory mode
args.pdf = ''
args.image_dir = './images'
jobs = build_jobs(args)
```

--------------------------------

### Process with Custom Model

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Run inference using a custom-trained model by specifying the model directory.

```bash
python infer.py \
    --image_dir ./images \
    --model_dir ./models/local-unlimited-ocr \
    --output_dir ./results
```

--------------------------------

### HTTP: Health Check

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Perform a health check on the OCR service by sending a GET request to the /health endpoint. Expects an HTTP 200 OK response.

```bash
curl http://127.0.0.1:10000/health
# Returns: HTTP 200 OK
```

--------------------------------

### Create and Fix Output Directory Permissions

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Manually recover from `OSError` by creating the output directory using `mkdir -p` and setting appropriate permissions with `chmod 755`. This ensures the application can write to the specified output location.

```bash
# Create output directory
mkdir -p ./outputs

# Fix permissions
chmod 755 ./outputs

# Use different directory with write access
python infer.py --output_dir ./results
```

--------------------------------

### Upload OCR Results to AWS S3

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

Uploads all files from a specified output directory to an AWS S3 bucket. Ensure boto3 is installed and AWS credentials are configured.

```python
import boto3
import os

def upload_results_to_s3(output_dir: str, bucket: str, prefix: str):
    """Upload all results to S3."""
    s3_client = boto3.client('s3')
    
    for filename in os.listdir(output_dir):
        filepath = os.path.join(output_dir, filename)
        if os.path.isfile(filepath):
            key = f"{prefix}/{filename}"
            s3_client.upload_file(filepath, bucket, key)
            print(f"Uploaded {filename} to s3://{bucket}/{key}")

# In main():
def main():
    args = parse_args()
    server_process = start_server(args)
    try:
        run(args)
        
        # Upload results
        if hasattr(args, 's3_bucket'):
            upload_results_to_s3(args.output_dir, args.s3_bucket, args.s3_prefix or '')
    finally:
        stop_server(server_process)
```

--------------------------------

### Enable Detailed Server Logging

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Configure the OCR server to log detailed information for debugging purposes. Specify the path for the server log file.

```bash
# Detailed server logging
python infer.py --image_dir ./images --server_log ./debug.log
```

--------------------------------

### Python Health Check Function

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/endpoints.md

This Python function checks if the server is ready by making a GET request to the /health endpoint. It includes a timeout and handles request exceptions.

```python
def server_ready(server_url: str) -> bool:
    try:
        resp = requests.get(f"{server_url}/health", timeout=5)
        return resp.status_code == 200
    except requests.RequestException:
        return False
```

--------------------------------

### HTTP: Direct API Call (curl)

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Make a direct API call to the OCR service using curl. This example demonstrates a POST request with JSON payload for chat completions, including image data.

```bash
curl -X POST http://127.0.0.1:10000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Unlimited-OCR",
    "messages": [{"role": "user", "content": [
      {"type": "text", "text": "document parsing."},
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
    ]}],
    "temperature": 0,
    "skip_special_tokens": false,
    "stream": true,
    "images_config": {"image_mode": "gundam"}
  }' \
  --max-time 1200
```

--------------------------------

### Configure High-Performance Settings

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/configuration.md

Optimize performance by adjusting memory fraction and attention backend in `infer.py`. These settings are recommended for high-performance modes to utilize more GPU memory and the latest attention kernels.

```python
MEM_FRACTION_STATIC = 0.9  # Use more GPU memory
ATTENTION_BACKEND = "fa3"  # Use latest attention kernel
```

--------------------------------

### Manual Recovery: Restart Server

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Illustrates commands to restart the OCR server. This is a common recovery step for various connection and service availability issues.

```bash
# Restart server
# pkill -f sglang
# python infer.py ...  # Restarts server automatically
```

--------------------------------

### Integration Test for Full Inference Pipeline

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/implementation-guide.md

This Python integration test verifies the complete inference pipeline. It creates a test image, starts the server, runs inference using `infer_one`, and ensures the server is stopped afterward.

```python
def test_full_pipeline():
    """Test full inference pipeline."""
    # Create test image
    from PIL import Image
    test_img = Image.new('RGB', (640, 640), color='white')
    test_img.save('test_input.png')
    
    # Run inference
    args = argparse.Namespace(
        image_dir='.',
        pdf='',
        output_dir='./test_output',
        concurrency=1,
        gpu='0',
        model_dir='baidu/Unlimited-OCR',
        image_mode='gundam',
        server_log='./test_server.log'
    )
    
    server = start_server(args)
    try:
        result = infer_one('test_input.png', None, args, 1)
        assert result['tokens'] > 0
    finally:
        stop_server(server)
```

--------------------------------

### Use Faster Image Processing Mode

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Use the `--image_mode gundam` command-line argument for faster image processing.

```bash
# Use faster image processing
--image_mode gundam
```

--------------------------------

### Manual Retry of Inference

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Demonstrates how to manually retry an inference call with different parameters if the initial attempt fails. Includes checking image properties if issues persist.

```python
from infer import infer_one, build_content, start_server, stop_server
import argparse

args = argparse.Namespace(image_mode='gundam')
result = infer_one('problematic_image.png', None, args, 1)

# Retry with different settings
args.image_mode = 'base'
result = infer_one('problematic_image.png', None, args, 1)

# If still fails, check image
from PIL import Image
img = Image.open('problematic_image.png')
print(f"Image size: {img.size}, format: {img.format}")
```

--------------------------------

### Use Base Mode for Layout Preservation in Large Documents

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Employ `--image_mode base` for large documents to preserve layout structure during processing.

```bash
# Use base mode for layout preservation
--image_mode base
```

--------------------------------

### View Server Logs

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Check the server log file for detailed error messages and operational information.

```bash
cat ./log/sglang_server.log
```

--------------------------------

### Handle RuntimeError: SGLang Server Exited Early

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Use a try-except block to catch RuntimeError during server startup. This is useful when the server process might terminate unexpectedly before becoming ready.

```python
try:
    args = argparse.Namespace(
        model_dir='baidu/Unlimited-OCR',
        gpu='0',
        server_log='./log/sglang_server.log'
    )
    process = start_server(args)
except RuntimeError as e:
    print(f"Server startup failed: {e}")
    # Fallback: use existing server, or restart manually
    # Increase SERVER_TIMEOUT if first startup just slow
```

--------------------------------

### Batch Inference with infer.py (PDF)

Source: https://github.com/baidu/unlimited-ocr/blob/main/README.md

Runs batch inference on a PDF file using `infer.py` with specified concurrency and image mode.

```shell
# PDF pages
python infer.py \
    --pdf ./examples/document.pdf \
    --output_dir ./outputs \
    --concurrency 8 \
    --image_mode gundam
```

--------------------------------

### Use Multiple GPUs

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Leverage multiple GPUs for processing by listing their IDs. Adjust concurrency based on the number of GPUs used.

```bash
python infer.py \
    --image_dir ./images \
    --gpu 0,1,2,3 \
    --concurrency 16
```

--------------------------------

### Single Image Inference Data Flow

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/project-overview.md

Illustrates the data flow for single image inference, from user input via CLI arguments to server launch, job building, concurrent inference, and final output generation.

```text
User Input
    ↓
CLI Arguments (--image_dir, --concurrency, --gpu, etc.)
    ↓
start_server() → Launch SGLang server process
    ↓
build_jobs() → Collect image files or convert PDF to pages
    ↓
run() → Thread pool executor
    ├─→ infer_one() x N (concurrent)
    │   ├─→ encode_image() → Base64 + data URI
    │   ├─→ build_content() → Create message
    │   ├─→ requests.post() → /v1/chat/completions
    │   ├─→ collect_stream_silent() → Stream parsing
    │   └─→ Write output markdown
    │
    └─→ Aggregate results + print statistics
    ↓
stop_server() → Terminate SGLang server
    ↓
Output Files (results/*.md)
```

--------------------------------

### Batch Inference with infer.py (Image Directory)

Source: https://github.com/baidu/unlimited-ocr/blob/main/README.md

Runs batch inference on a directory of images using `infer.py` with specified concurrency and image mode.

```shell
# Image directory
python infer.py \
    --image_dir ./examples/images \
    --output_dir ./outputs \
    --concurrency 8 \
    --image_mode gundam
```

--------------------------------

### Use Custom Model Weights

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Specify a custom directory containing model weights for OCR processing.

```bash
# Custom model
python infer.py --image_dir ./images --model_dir ./model_weights
```

--------------------------------

### build_content

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Constructs message content list with text prompt and encoded image for API request. Returns a list of dictionaries suitable for API messages.

```APIDOC
## build_content

### Description
Constructs message content list with text prompt and encoded image for API request.

### Method
```python
def build_content(image_path: str) -> list[dict]:
```

### Parameters
#### Path Parameters
- **image_path** (str) - Required - Path to image file to include in message

### Returns
- `list[dict]` - Structure: `[{"type": "text", "text": "document parsing."}, {"type": "image_url", "image_url": {"url": "data:..."}}]`

### Example
```python
from infer import build_content

content = build_content('document.png')
# content[0] = {"type": "text", "text": "document parsing."}
# content[1] = {"type": "image_url", ...}
```
```

--------------------------------

### Set Default Server Log Path

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Specifies the path for the server's log file. Defaults to './log/sglang_server.log'. Can be set via CLI argument.

```bash
--server_log "./log/sglang_server.log"
```

--------------------------------

### Multi-GPU Inference with PDF Mode

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/configuration.md

Process PDF documents using multiple GPUs. Configure the input PDF file, output directory, concurrency, and a comma-separated list of GPU IDs. The `image_mode` parameter can be set to 'base' for standard processing.

```bash
python infer.py \
    --pdf large_document.pdf \
    --output_dir ./pdf_results \
    --concurrency 16 \
    --gpu 0,1,2,3 \
    --image_mode base
```

--------------------------------

### run

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Executes concurrent OCR inference on a list of jobs using a thread pool. It handles job building, output directory creation, and concurrent processing, then prints performance statistics.

```APIDOC
## run

### Description
Executes concurrent OCR inference on all jobs with thread pool.

### Signature
```python
def run(args) -> None:
```

### Parameters
#### Path Parameters
- **args** (argparse.Namespace) - Required - Arguments with output_dir, concurrency, image_mode

### Behavior
1. Builds job list via `build_jobs`.
2. Creates output directory if needed.
3. Spawns `ThreadPoolExecutor` with specified concurrency.
4. Submits all jobs immediately (not work-stealing).
5. Collects results as they complete.
6. Prints statistics on completion.

### Output Metrics
```
Mode: {pdf_pages|dataset_images}, requests={count}, concurrency={N}, image_mode={gundam|base}
  [1] image.png: 1234 tokens, 12.5s
  [2] image2.png: 1100 tokens, 11.2s
  ...
============================================================
Concurrent Results:
  Requests: {successful}/{total}
  Total tokens: {sum}
  Wall time: {seconds}
  System TPS: {tokens_per_second}
  Avg tokens/request: {avg}
  Avg decode_time/request: {avg_seconds}
============================================================
```

### Example
```python
import argparse
from infer import run

args = argparse.Namespace(
    output_dir='./outputs',
    concurrency=8,
    image_mode='gundam',
    pdf='',
    image_dir='./images'
)
run(args)
```
```

--------------------------------

### Check Disk Space

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Verify available disk space using this command. Insufficient disk space can prevent output files from being created.

```bash
df -h
```

--------------------------------

### Specify Custom Output Directory

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Define a custom directory where the OCR results will be saved.

```bash
# Custom output directory
python infer.py --image_dir ./images --output_dir ./my_results
```

--------------------------------

### Manual Recovery: Adjust Image Processing Parameters

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Shows how to adjust image processing parameters to reduce the load on the server or the size of the data being processed. This can help prevent timeouts and other performance-related errors.

```bash
# Reduce image resolution via preprocessing
# Convert high-DPI PDFs to lower DPI before inference
PDF_DPI = 150  # Instead of 300
```

```bash
# Use "gundam" mode with smaller crops
--image_mode gundam
```

--------------------------------

### Use Safer Memory Allocation for Stability

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Set MEM_FRACTION_STATIC to 0.6 for safer memory allocation, which can improve stability.

```python
# Use safer memory allocation
MEM_FRACTION_STATIC = 0.6
```

--------------------------------

### Manual Recovery: Enable Server Logging

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/errors.md

Enables detailed server logging to help diagnose issues when streaming responses stall or when the server is slow to produce tokens. This is useful for identifying bottlenecks.

```bash
# Enable server logging to diagnose
--server_log ./detailed_log.txt
```

--------------------------------

### Watch Server Logs Command

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/quick-reference.md

Use this command to continuously monitor the server's log file for real-time updates and errors.

```bash
tail -f ./log/sglang_server.log
```

--------------------------------

### Collect Dataset Images (Python)

Source: https://github.com/baidu/unlimited-ocr/blob/main/_autodocs/api-reference.md

Recursively collects image files from a specified directory, returning their absolute paths sorted by file size in descending order. Supports .png, .jpg, .jpeg, .webp, and .bmp formats.

```python
from infer import collect_dataset_images

images = collect_dataset_images('./images')
# Returns images sorted by size, largest first
for img in images:
    print(img)
```