### Install SynthDoG

Source: https://context7.com/clovaai/donut/llms.txt

Installs the SynthDoG library, a synthetic document generator, using pip.

```bash
pip install synthtiger
```

--------------------------------

### SynthDoG Configuration Example

Source: https://context7.com/clovaai/donut/llms.txt

Example configuration parameters for SynthDoG, including JPEG quality, image dimensions, aspect ratio, and document content/layout settings.

```yaml
quality: [50, 95]          # JPEG quality range
short_size: [720, 1024]    # image short-side pixel range
aspect_ratio: [1, 2]       # height/width ratio range
document:
  content:
    text:
      path: resources/corpus/enwiki.txt   # corpus for text content
    font:
      paths: [resources/font/en]          # font directory
    layout:
      text_scale: [0.0334, 0.1]           # text size relative to document
      max_row: 10
      max_col: 3
```

--------------------------------

### Install DONUT from Source

Source: https://github.com/clovaai/donut/blob/master/README.md

Clone the DONUT repository and install dependencies from source. This method is useful for development or if you need the latest changes.

```bash
git clone https://github.com/clovaai/donut.git
cd donut/
conda create -n donut_official python=3.7
conda activate donut_official
pip install .
```

--------------------------------

### YAML Configuration for Training

Source: https://context7.com/clovaai/donut/llms.txt

Example YAML configuration file for training, controlling hyperparameters like dataset paths, batch sizes, learning rate, and training epochs.

```yaml
# config/train_cord.yaml
result_path: "./result"
pretrained_model_name_or_path: "naver-clova-ix/donut-base"
dataset_name_or_paths: ["naver-clova-ix/cord-v2"]
sort_json_key: False
train_batch_sizes: [8]
val_batch_sizes: [1]
input_size: [1280, 960]
max_length: 768
align_long_axis: False
seed: 2022
lr: 3e-5
warmup_steps: 300
num_training_samples_per_epoch: 800
max_epochs: 30
gradient_clip_val: 1.0
verbose: True
```

--------------------------------

### Install DONUT Python Package

Source: https://github.com/clovaai/donut/blob/master/README.md

Install the donut-python package using pip. This is the recommended method for most users.

```bash
pip install donut-python
```

--------------------------------

### Basic TeX Example

Source: https://github.com/clovaai/donut/blob/master/synthdog/resources/corpus/zhwiki.txt

This is a simple example of a TeX text file. The '\bye' command signifies the end of the file and is not displayed in the output.

```tex
Hello World\bye
```

--------------------------------

### Example Predictions and Answers

Source: https://github.com/clovaai/donut/blob/master/README.md

These examples show the model's predictions and the corresponding ground truth answers for a task, along with the normalized edit distance.

```text
Prediction: <s_menu><s_nm>Lemon Tea (L)</s_nm><s_cnt>1</s_cnt><s_price>25.000</s_price></s_menu><s_total><s_total_price>25.000</s_total_price><s_cashprice>30.000</s_cashprice><s_changeprice>5.000</s_changeprice></s_total>
Answer: <s_menu><s_nm>Lemon Tea (L)</s_nm><s_cnt>1</s_cnt><s_price>25.000</s_price></s_menu><s_total><s_total_price>25.000</s_total_price><s_cashprice>30.000</s_cashprice><s_changeprice>5.000</s_changeprice></s_total>
Normed ED: 0.0
```

```text
Prediction: <s_menu><s_nm>Hulk Topper Package</s_nm><s_cnt>1</s_cnt><s_price>100.000</s_price></s_menu><s_total><s_total_price>100.000</s_total_price><s_cashprice>100.000</s_cashprice><s_changeprice>0</s_changeprice></s_total>
Answer: <s_menu><s_nm>Hulk Topper Package</s_nm><s_cnt>1</s_cnt><s_price>100.000</s_price></s_menu><s_total><s_total_price>100.000</s_total_price><s_cashprice>100.000</s_cashprice><s_changeprice>0</s_changeprice></s_total>
Normed ED: 0.0
```

```text
Prediction: <s_menu><s_nm>Giant Squid</s_nm><s_cnt>x 1</s_cnt><s_price>Rp. 39.000</s_price><s_sub><s_nm>C.Finishing - Cut</s_nm><s_price>Rp. 0</s_price><sep/><s_nm>B.Spicy Level - Extreme Hot Rp. 0</s_price></s_sub><sep/><s_nm>A.Flavour - Salt & Pepper</s_nm><s_price>Rp. 0</s_price></s_sub></s_menu><s_sub_total><s_subtotal_price>Rp. 39.000</s_subtotal_price></s_sub_total><s_total><s_total_price>Rp. 39.000</s_total_price><s_cashprice>Rp. 50.000</s_cashprice><s_changeprice>Rp. 11.000</s_changeprice></s_total>
Answer: <s_menu><s_nm>Giant Squid</s_nm><s_cnt>x1</s_cnt><s_price>Rp. 39.000</s_price><s_sub><s_nm>C.Finishing - Cut</s_nm><s_price>Rp. 0</s_price><sep/><s_nm>B.Spicy Level - Extreme Hot</s_nm><s_price>Rp. 0</s_price><sep/><s_nm>A.Flavour- Salt & Pepper</s_nm><s_price>Rp. 0</s_price></s_sub></s_menu><s_sub_total><s_subtotal_price>Rp. 39.000</s_subtotal_price></s_sub_total><s_total><s_total_price>Rp. 39.000</s_total_price><s_cashprice>Rp. 50.000</s_cashprice><s_changeprice>Rp. 11.000</s_changeprice></s_total>
Normed ED: 0.039603960396039604
```

--------------------------------

### Metadata JSON Lines Format

Source: https://github.com/clovaai/donut/blob/master/README.md

Example of the metadata.jsonl file format. Each line is a JSON object containing the image file name and its ground truth.

```bash
> cat dataset_name/test/metadata.jsonl
{"file_name": {image_path0}, "ground_truth": "{\"gt_parse\": {ground_truth_parse}, ... {other_metadata_not_used} ... }"}
{"file_name": {image_path1}, "ground_truth": "{\"gt_parse\": {ground_truth_parse}, ... {other_metadata_not_used} ... }"}
     .
     .

```

--------------------------------

### TeX Hyphenation Algorithm Example

Source: https://github.com/clovaai/donut/blob/master/synthdog/resources/corpus/zhwiki.txt

Explains the TeX hyphenation algorithm, which processes a natural paragraph at once to find the optimal line breaks, contrasting with the first-fit method. It uses 'badness' to measure the quality of line breaks.

```tex
The first-fit method may result in large spaces between words on a line, while TeX's method considers the entire paragraph for better spacing.
```

--------------------------------

### Launch CORD Receipt Parsing Demo

Source: https://context7.com/clovaai/donut/llms.txt

Launches the CORD receipt parsing demo. Requires specifying the task, pretrained path, sample image path, and port.

```bash
python app.py \
  --task cord-v2 \
  --pretrained_path naver-clova-ix/donut-base-finetuned-cord-v2 \
  --sample_img_path ./misc/sample_image_cord_test_receipt_00004.png \
  --port 7861
```

--------------------------------

### Generating DVI and PDF Files with TeX

Source: https://github.com/clovaai/donut/blob/master/synthdog/resources/corpus/zhwiki.txt

Demonstrates how to process a TeX file to generate a DVI file and how to convert it to PDF. The pdfTeX engine is recommended for better font display quality compared to DVI to PostScript to PDF conversion.

```bash
tex example.tex
```

```bash
tex \"--output-format=pdf\" example.tex
```

--------------------------------

### Launch DocVQA Demo

Source: https://context7.com/clovaai/donut/llms.txt

Launches the default Document Visual Question Answering demo using a pre-trained model. Specify the task, pretrained path, and port.

```bash
python app.py \
  --task docvqa \
  --pretrained_path naver-clova-ix/donut-base-finetuned-docvqa \
  --port 7860
```

--------------------------------

### app.py: Gradio Interactive Web Demo

Source: https://context7.com/clovaai/donut/llms.txt

Launches a Gradio web interface for interactive document understanding demos. Supports various Donut models and includes a text question input for DocVQA tasks.

```bash

```

--------------------------------

### train.py: Command-Line Training Script

Source: https://context7.com/clovaai/donut/llms.txt

Launches PyTorch Lightning training runs for Donut tasks using YAML configurations. Supports multi-GPU, mixed precision, checkpointing, and TensorBoard logging.

```bash
# Fine-tune on CORD (document parsing / receipt understanding)
python train.py \
  --config config/train_cord.yaml \
  --pretrained_model_name_or_path "naver-clova-ix/donut-base" \
  --dataset_name_or_paths '["naver-clova-ix/cord-v2"]' \
  --exp_version "cord_finetune_v1"

# Fine-tune on DocVQA
python train.py \
  --config config/train_docvqa.yaml \
  --pretrained_model_name_or_path "naver-clova-ix/donut-base" \
  --dataset_name_or_paths '["naver-clova-ix/docvqa_en"]' \
  --exp_version "docvqa_run1"

# Resume from a checkpoint
python train.py \
  --config config/train_cord.yaml \
  --resume_from_checkpoint_path ./result/train_cord/cord_finetune_v1/artifacts.ckpt
```

--------------------------------

### Document Information Extraction Ground Truth

Source: https://github.com/clovaai/donut/blob/master/README.md

Example of `gt_parse` format for document information extraction tasks. It's a JSON object representing the structured information extracted from the document image, such as receipt details.

```json
{"menu" : [{"nm": "ICE BLACKCOFFEE", "cnt": "2", ...}, ...], ...}
```

--------------------------------

### SynthDoG Configuration Parameters

Source: https://github.com/clovaai/donut/blob/master/synthdog/README.md

This output shows the parsed configuration parameters for SynthDoG, including output path, number of data samples, and verbose settings.

```json
{
'config': 'config_en.yaml',
'count': 50,
'name': 'SynthDoG',
'output': './outputs/SynthDoG_en',
'script': 'template.py',
'verbose': True,
'worker': 4
}
```

--------------------------------

### TeX Hyphenation Pattern Example

Source: https://github.com/clovaai/donut/blob/master/synthdog/resources/corpus/zhwiki.txt

Illustrates how TeX uses hyphenation patterns to determine word breaks. Patterns like 'h5elo' provide hints for potential break points within a word, with higher numbers indicating stronger suggestions.

```tex
codice_6
```

```tex
codice_7
```

--------------------------------

### DonutDataset Initialization for Different Tasks

Source: https://context7.com/clovaai/donut/llms.txt

Demonstrates initializing the DonutDataset for training, validation, and DocVQA tasks. Note the different return values for validation splits and the specific prompt end token for DocVQA.

```python
train_dataset = DonutDataset(
    dataset_name_or_path="naver-clova-ix/cord-v2",
    donut_model=model,
    max_length=768,
    split="train",
    task_start_token="<s_cord-v2>",
    prompt_end_token="<s_cord-v2>",
    sort_json_key=False,
)
print(len(train_dataset))  # number of training samples
image_tensor, input_ids, labels = train_dataset[0]
print(image_tensor.shape)  # torch.Size([3, H, W])
print(input_ids.shape)     # torch.Size([768])
```

```python
val_dataset = DonutDataset(
    dataset_name_or_path="naver-clova-ix/cord-v2",
    donut_model=model,
    max_length=768,
    split="validation",
    task_start_token="<s_cord-v2>",
)
image_tensor, input_ids, prompt_end_idx, raw_answer = val_dataset[0]
```

```python
vqa_dataset = DonutDataset(
    dataset_name_or_path="naver-clova-ix/docvqa_en",
    donut_model=model,
    max_length=768,
    split="train",
    task_start_token="<s_docvqa>",
    prompt_end_token="<s_answer>",
)
```

--------------------------------

### Launch Document Classification Demo

Source: https://context7.com/clovaai/donut/llms.txt

Launches the document classification demo for RVL-CDIP dataset. Specify the task, pretrained path, and port.

```bash
python app.py \
  --task rvlcdip \
  --pretrained_path naver-clova-ix/donut-base-finetuned-rvlcdip \
  --port 7862
```

--------------------------------

### SynthDoG Image Quality and Size Settings

Source: https://github.com/clovaai/donut/blob/master/synthdog/README.md

This output displays image generation settings for SynthDoG, including aspect ratio constraints and quality ranges.

```json
{
'aspect_ratio': [1, 2],
    .
    .
'quality': [50, 95],
'short_size': [720, 1024]
}
```

--------------------------------

### Train Donut Model on CORD Dataset

Source: https://github.com/clovaai/donut/blob/master/README.md

Use this command to train the Donut model on the CORD dataset. Ensure you specify the configuration file, pretrained model, and dataset paths.

```bash
python train.py --config config/train_cord.yaml \
                --pretrained_model_name_or_path "naver-clova-ix/donut-base" \
                --dataset_name_or_paths '["naver-clova-ix/cord-v2"]' \
                --exp_version "test_experiment"
```

--------------------------------

### Preprocess PIL Image for Encoder Input

Source: https://context7.com/clovaai/donut/llms.txt

The `prepare_input` method resizes, optionally rotates, and pads a PIL image to the model's configured `input_size`, then converts it to a normalized `torch.Tensor`. It supports deterministic center-padding for inference and random padding for training augmentation.

```python
from PIL import Image
from donut import DonutModel

model = DonutModel.from_pretrained("naver-clova-ix/donut-base")

img = Image.open("document.png")

# Deterministic center-padding (for inference)
tensor = model.encoder.prepare_input(img, random_padding=False)
print(tensor.shape)  # torch.Size([3, 2560, 1920])

# Random padding (for training augmentation)
tensor_aug = model.encoder.prepare_input(img, random_padding=True)
print(tensor_aug.shape)  # torch.Size([3, 2560, 1920])

# Use the tensor directly in inference to avoid re-encoding
import torch
output = model.inference(
    image_tensors=tensor.unsqueeze(0).half().to("cuda"),
    prompt="<s_cord-v2>",
)
```

--------------------------------

### Configure Donut Model Architecture

Source: https://context7.com/clovaai/donut/llms.txt

Build a smaller custom Donut model from scratch by defining architectural hyperparameters like input resolution, Swin Transformer window size, encoder/decoder layers, and maximum generation length.

```python
from donut import DonutConfig, DonutModel

# Build a smaller custom model from scratch
config = DonutConfig(
    input_size=[1280, 960],      # (height, width) of the input canvas
    align_long_axis=False,       # rotate portrait images to landscape
    window_size=10,              # Swin Transformer window size
    encoder_layer=[2, 2, 14, 2], # depth of each Swin stage
    decoder_layer=4,             # number of BART decoder layers
    max_length=768,              # maximum token generation length
)
model = DonutModel(config)
print(model.config.input_size)  # [1280, 960]
print(model.config.max_length)  # 768
```

--------------------------------

### SwinEncoder.prepare_input()

Source: https://context7.com/clovaai/donut/llms.txt

Preprocesses a PIL Image for the Encoder by resizing, optionally rotating, and padding it to the model's configured `input_size`, then converts it to a normalized `torch.Tensor`. Used in both training and inference.

```APIDOC
## SwinEncoder.prepare_input()

### Description
Preprocesses a PIL Image for the Encoder by resizing, optionally rotating, and padding it to the model's configured `input_size`, then converts it to a normalized `torch.Tensor`. Used in both training (with `random_padding=True`) and inference.

### Method Signature
`prepare_input(image, random_padding=False)`

### Parameters
- **image** (PIL.Image.Image) - The input PIL Image.
- **random_padding** (bool, optional) - Whether to use random padding for augmentation during training. Defaults to False (deterministic center-padding for inference).

### Request Example
```python
from PIL import Image
from donut import DonutModel

model = DonutModel.from_pretrained("naver-clova-ix/donut-base")

img = Image.open("document.png")

# Deterministic center-padding (for inference)
tensor = model.encoder.prepare_input(img, random_padding=False)
print(tensor.shape)  # torch.Size([3, 2560, 1920])

# Random padding (for training augmentation)
tensor_aug = model.encoder.prepare_input(img, random_padding=True)
print(tensor_aug.shape)  # torch.Size([3, 2560, 1920])

# Use the tensor directly in inference to avoid re-encoding
import torch
output = model.inference(
    image_tensors=tensor.unsqueeze(0).half().to("cuda"),
    prompt="<s_cord-v2>",
)
```

### Response Example
```
torch.Size([3, 2560, 1920])
```
```

--------------------------------

### Load Pretrained Donut Model

Source: https://context7.com/clovaai/donut/llms.txt

Instantiate a DonutModel from a Hugging Face model hub name or local directory. Supports loading base models, task-specific fine-tuned models, and custom max_length with position embedding interpolation/truncation. Includes optional GPU acceleration.

```python
import torch
from donut import DonutModel

# Load the base pretrained model
model = DonutModel.from_pretrained("naver-clova-ix/donut-base")

# Load a task-specific fine-tuned model (document parsing on CORD)
model = DonutModel.from_pretrained("naver-clova-ix/donut-base-finetuned-cord-v2")

# Load with a custom max_length (position embeddings are interpolated/truncated)
model = DonutModel.from_pretrained(
    "naver-clova-ix/donut-base-finetuned-cord-v2",
    max_length=512,
)

# Move to GPU for faster inference
if torch.cuda.is_available():
    model.half()
    model.to("cuda")
model.eval()
```

--------------------------------

### Generate ECJK Synthetic Documents (English)

Source: https://github.com/clovaai/donut/blob/master/synthdog/README.md

Command to generate synthetic documents for English language using SynthDoG, specifying dataset path, number of data, and workers.

```bash
# english
synthtiger -o {dataset_path} -c {num_of_data} -w {num_of_workers} -v template.py SynthDoG config_en.yaml
```