DeepSeek-OCR

https://huggingface.co/deepseek-ai/DeepSeek-OCR
Admin
DeepSeek-OCR is a vision-language model specialized in optical character recognition and document...
Tokens:236,567
Snippets:939
Trust Score:10
Update:9 months ago
Show doc for...
Tokens:
Raw
### Set Up Python Virtual Environment and Install Dependencies

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/20

Creates a Python virtual environment, activates it, and installs essential libraries including PyTorch, Transformers, and other project-specific dependencies.

```bash
python3 -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
pip install \
  torch==2.6.0 \
  torchvision==0.21.0 \
  transformers==4.46.3 \
  tokenizers==0.20.3 \
  einops \
  addict \
  easydict \
  pillow \
  tqdm \
  matplotlib
```

--------------------------------

### vLLM Inference Setup for DeepSeek-OCR

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/6d2df0b94f19d2ffbe3ae58058925a26e3d4c2df/README_code=true

This Python snippet shows the initial setup for using the DeepSeek-OCR model with vLLM. It imports the necessary components from the vllm library and prepares to instantiate the LLM model. This code is the starting point for leveraging vLLM's accelerated inference capabilities for OCR tasks.

```python
from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor
from PIL import Image

# Create model instance
llm = LLM(

```

--------------------------------

### vLLM OCR Inference Setup

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/3fd852ae69f7a4e07d8be8373913cdc68710843f/README_code=true

Illustrates the initial setup for performing OCR inference using vLLM with DeepSeek-OCR. This involves importing the necessary classes from the vLLM library.

```python
from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor

```

--------------------------------

### Setting up Latest vLLM from GitHub

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/55

Instructions for cloning the vLLM repository, setting up a virtual environment, installing dependencies including PyTorch, and installing vLLM in editable mode. This is necessary to ensure compatibility with models like DeepSeek-OCR.

```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
uv venv --python 3.13 --seed
source .venv/bin/activate
python use_existing_torch.py
uv pip install -r requirements/build.txt
uv pip install torch torchvision
uv pip install --no-build-isolation -e . --prerelease=allow

```

--------------------------------

### Install DeepSeek-OCR Requirements with Transformers

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/d7e76f5b1edc3c637eb6a112f9af8a675777a5a8/README

Installs necessary Python packages for running DeepSeek-OCR using the Huggingface transformers library. This includes specific versions of torch, transformers, and flash-attn for optimal performance on NVIDIA GPUs.

```shell
pip install flash-attn==2.7.3 --no-build-isolation
```

--------------------------------

### Install DeepSeek-OCR Dependencies

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/0ed09dd8011a1fc91c5d4d0fa22f0d6f85ceac36/README

Installs the necessary libraries for running the DeepSeek-OCR model, including PyTorch, transformers, and flash-attention. Ensure your environment meets the specified Python and CUDA versions.

```bash
pip install torch==2.6.0
transformers==4.46.3
tokenizers==0.20.3
einops
addict 
easydict
pip install flash-attn==2.7.3 --no-build-isolation
```

--------------------------------

### vLLM Installation (Shell)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/209ae73bb0f2e5b377f8eb18b09419014970ac34/README

Installs vLLM from nightly builds, necessary for versions prior to v0.11.1 to support DeepSeek-OCR. This script sets up a virtual environment and then installs the vLLM package with extra index URLs.

```shell
uv venv
source .venv/bin/activate
# Until v0.11.1 release, you need to install vLLM from nightly build
uv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly

```

--------------------------------

### Initialize Model Parameters (Python)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/59512895521e669adb87064b37e5749fe9b9f5d2

Initializes the model's parameters using Kaiming uniform initialization. This method is crucial for setting appropriate weight distributions at the start of training. It requires the `torch.nn.init` module and `math.sqrt`.

```python
import torch.nn.init as init
import math

def reset_parameters(self) -> None:
  init.kaiming_uniform_(self.weight, a=math.sqrt(5))
```

--------------------------------

### Install Git LFS and Clone DeepSeek-OCR Repo

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/20

Installs Git Large File Storage (LFS) and clones the DeepSeek-OCR repository. It also switches to a specific PR branch for necessary fixes.

```bash
brew install git-lfs
git lfs install

mkdir -p ~/huggingface && cd ~/huggingface
git clone https://huggingface.co/deepseek-ai/DeepSeek-OCR
cd DeepSeek-OCR
git checkout pr20
```

--------------------------------

### Install and Use DeepSeek-OCR with vLLM

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/ee668a444ce026b7a23944f36692df1cdba54de9

This snippet demonstrates how to install vLLM with DeepSeek-OCR support and then use the model for image-to-text OCR tasks. It requires installing vLLM from a nightly build and uses the PIL library for image handling. The output is generated using the `llm.generate` method with specific sampling parameters.

```shell
uv venv
source .venv/bin/activate
# Until v0.11.1 release, you need to install vLLM from nightly build
uv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
```

```python
from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor
from PIL import Image

# Create model instance
llm = LLM(
model="deepseek-ai/DeepSeek-OCR",
enable_prefix_caching=False,
mm_processor_cache_gb=0,
logits_processors=[NGramPerReqLogitsProcessor]
)

# Prepare batched input with your image file
image_1 = Image.open("path/to/your/image_1.png").convert("RGB")
image_2 = Image.open("path/to/your/image_2.png").convert("RGB")
prompt = "<image>\nFree OCR."

model_input = [
{
"prompt": prompt,
"multi_modal_data": {"image": image_1}
},
{
"prompt": prompt,
"multi_modal_data": {"image": image_2}
}
]

sampling_param = SamplingParams(
temperature=0.0,
max_tokens=8192,
# ngram logit processor args
extra_args=dict(
ngram_size=30,
window_size=90,
whitelist_token_ids={128821, 128822}, # whitelist: <td>, </td>
),
skip_special_tokens=False,
)
# Generate output
model_outputs = llm.generate(model_input, sampling_param)

# Print output
for output in model_outputs:
print(output.outputs[0].text)
```

--------------------------------

### DeepSeek-OCR Inference Example

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/21b79334c977a9bc452f3c34c205beaf0bf7476b/README

Performs inference using the DeepSeek-OCR model with a specified prompt and image. The example shows how to set up the model and prepare for image-to-text conversion, with options for different inference modes.

```python
# prompt = "<image>\nFree OCR. "
prompt = "<image>\n<|grounding|>Convert the document to markdown. "
image_file = 'your_image.jpg'
output_path = 'your/output/dir'

# infer(self, tokenizer, prompt='', image_file='', output_path = ' ', base_size = 1024, image_size = 640, crop_mode = True, test_compress = False, save_results = False):

# Tiny: base_size = 512, image_size = 512, crop_mode = False
# Small: base_size = 640, image_size = 640, crop_mode = False
# Base: base_size = 1024, image_size = 1024, crop_mode = False

# Example usage (assuming 'infer' method is available on the model object):
# result = model.infer(tokenizer=tokenizer, prompt=prompt, image_file=image_file, output_path=output_path, base_size=1024, image_size=1024, crop_mode=False)
```

--------------------------------

### Install DeepSeek-OCR Dependencies with Transformers

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/767fb17bf2632c15ca9c647474f317af0e1bf2ec/README

Installs the necessary Python packages for running DeepSeek-OCR using Hugging Face Transformers. This includes PyTorch, Transformers, tokenizers, and flash-attn. Ensure CUDA is available for GPU acceleration.

```bash
pip install torch==2.6.0
transformers==4.46.3
tokenizers==0.20.3
einops
addict 
easydict
pip install flash-attn==2.7.3 --no-build-isolation

```

--------------------------------

### vLLM Inference Initialization for DeepSeek-OCR

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/2c968b433af61a059311cbf8997765023806a24d/README_code=true

This Python code demonstrates how to initialize the vLLM LLM instance for the DeepSeek-OCR model. It imports necessary components from the `vllm` library and the PIL library for image handling. This setup is the first step towards performing accelerated inference.

```python
from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor
from PIL import Image

# Create model instance
llm = LLM(
 model="deepseek-ai/DeepSeek-OCR",
```

--------------------------------

### vLLM Inference Setup for DeepSeek-OCR

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/5e308b8cd1901ff0ff90b91ca9f54c084138f9d7/README_code=true

This Python code snippet shows how to initialize the vLLM LLM engine for DeepSeek-OCR. It imports necessary components from vLLM and Pillow, and sets up the LLM instance with the DeepSeek-OCR model name. The `enable_prefix_caching` parameter is set to False.

```python
from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor
from PIL import Image

# Create model instance
llm = LLM(
 model="deepseek-ai/DeepSeek-OCR",
 enable_prefix_caching=False,

```

--------------------------------

### DeepSeek-OCR Inference Example

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/968c903185ba632ccc137b22d78ba01d14611ee1/README

This snippet demonstrates how to perform inference using the DeepSeek-OCR model. It shows how to set various parameters such as base_size, image_size, and crop_mode to customize the inference process. The output is saved to a specified path.

```python
# Small: base_size = 640, image_size = 640, crop_mode = False
# Base: base_size = 1024, image_size = 1024, crop_mode = False
# Large: base_size = 1280, image_size = 1280, crop_mode = False

# Gundam: base_size = 1024, image_size = 640, crop_mode = True

res = model.infer(tokenizer, prompt=prompt, image_file=image_file, output_path = output_path, base_size = 1024, image_size = 640, crop_mode=True, save_results = True, test_compress = True)
```

--------------------------------

### Deepseek OCR Inference Method Setup

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/6/files

This Python code outlines the initial setup for the `infer` method in `DeepseekOCRForCausalLM`. It includes disabling torch initialization, creating output directories, and formatting conversation prompts that may include image references. The code prepares the environment and input structure before proceeding with OCR processing.

```python
self.disable_torch_init()
os.makedirs(output_path, exist_ok=True)
os.makedirs(f'{output_path}/images', exist_ok=True)

if prompt and image_file:
  conversation = [
    {
      "role": "<|User|>",
      "content": f'{prompt}',
      "images": [f'{image_file}']
    },
    {"role": "<|Assistant|", "content": ""}
  ]
elif prompt:
  conversation = [
    {
      "role": "<|User|>",
      "content": f'{prompt}'
    },
    {"role": "<|Assistant|", "content": ""}
  ]
else:
  assert False, f' prompt is none!'

prompt = format_messages(conversations=conversation, sft_format='plain', system_prompt='')
```

--------------------------------

### Run DeepSeek-OCR Demo with MLX (Python)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/20

This snippet outlines the steps to set up and run the DeepSeek-OCR demo using MLX. It involves cloning a specific GitHub repository, creating a virtual environment, installing dependencies, and executing the provided demo script. This approach leverages MLX for potentially faster inference.

```bash
# Clone the MLX implementation repository
git clone https://github.com/rayking99/deepseekocr-mlx

# Navigate into the repository directory
cd deepseekocr-mlx

# Create a virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

# Install requirements
pip install -r requirements.txt

# Run the demo script
python run_demo.py
```

--------------------------------

### Modal Image Setup for DeepSeek-OCR (Python)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/23

Configures a Modal image with necessary system packages, Python libraries, and environment variables for running DeepSeek-OCR. This includes cloning the repository, installing dependencies, and setting up GPU and model-specific configurations.

```python
import io
import os
import sys
import uuid
import asyncio
import subprocess
from threading import Thread
from PIL import Image


import modal


app = modal.App("deepseek-ocr")
IMAGE_GPU = os.getenv("IMAGE_GPU", None)
BACKEND = os.getenv("BACKEND", None)
img = (
    # https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda/tags
    modal.Image.from_registry(
        "nvidia/cuda:12.6.1-cudnn-devel-ubuntu22.04",
        add_python="3.10",
    )
    .apt_install("git", "git-lfs")
    .run_commands(
        "git clone https://github.com/deepseek-ai/DeepSeek-OCR.git",
        "cd /DeepSeek-OCR && pip install -r requirements.txt",
    )
    .pip_install(
        "torch==2.6.0",
        "torchvision==0.21.0",
        "torchaudio==2.6.0",
        "wheel",
        "matplotlib",
        "accelerate>=0.26.0",  # for device_map="auto" model loading with safetensors slipt
    )
    # .apt_install("clang", "cmake", "ninja-build")
    # https://github.com/Dao-AILab/flash-attention/releases/tag/v2.7.4.post1 for torch 2.6.0
    .pip_install("flash-attn==2.7.4.post1", extra_options="--no-build-isolation")
    .env(
        {
            "PYTORCH_CUDA_ALLOC_CONF": "expandable_segments:True",
            "ACHATBOT_PKG": "1",
            "CUDA_VISIBLE_DEVICES": "0",
            "LLM_MODEL": os.getenv("LLM_MODEL", "deepseek-ai/DeepSeek-OCR"),
        }
    )
)

if BACKEND == "achatbot":
    img = img.pip_install(
        f"achatbot==0.0.27.dev8",
        extra_index_url=os.getenv("EXTRA_INDEX_URL", "https://test.pypi.org/simple/"),
    )


HF_MODEL_DIR = "/root/.achatbot/models"
hf_model_vol = modal.Volume.from_name("models", create_if_missing=True)
ASSETS_DIR = "/root/.achatbot/assets"
assets_vol = modal.Volume.from_name("assets", create_if_missing=True)
CONFIG_DIR = "/root/.achatbot/config"
config_vol = modal.Volume.from_name("config", create_if_missing=True)

TORCH_CACHE_DIR = "/root/.cache/torch"
torch_cache_vol = modal.Volume.from_name("torch_cache", create_if_missing=True)


with img.imports():
    from queue import Queue

    import torch
    from transformers.generation.streamers import TextStreamer, TextIteratorStreamer
    from transformers import AutoModel, AutoTokenizer

    MODEL_ID = os.getenv("LLM_MODEL", "deepseek-ai/DeepSeek-OCR")
    MODEL_PATH = os.path.join(HF_MODEL_DIR, MODEL_ID)
    DEEPSEEK_ASSETS_DIR = os.path.join(ASSETS_DIR, "DeepSeek")

    # torch.set_float32_matmul_precision("high")

```

--------------------------------

### Serve DeepSeek-OCR with vLLM OpenAI Compatible Server

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/55

This snippet demonstrates how to serve the DeepSeek-OCR model using vLLM's OpenAI-compatible server. It includes the command to start the server with specific configurations for prefix caching and MM processor cache, and an example of how to interact with the served model using the OpenAI client library.

```shell
vllm serve deepseek-ai/DeepSeek-OCR --no-enable-prefix-caching --mm-processor-cache-gb 0 --logits-processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor
```

```python
response = client.chat.completions.create(
  model="deepseek-ai/DeepSeek-OCR",
  messages=message,
  temperature=0.0,
  max_tokens=500,
)
```

--------------------------------

### DeepseekV2MoE Module Initialization and Forward Pass in PyTorch

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/59512895521e669adb87064b37e5749fe9b9f5d2

This snippet details the initialization and forward pass of the DeepseekV2MoE module. It handles the setup of experts, including distributed expert parallelism, and the routing of inputs to the appropriate experts during the forward pass. It also incorporates shared experts and auxiliary loss calculation.

```python
class DeepseekV2MoE(nn.Module):
    """
    A mixed expert module containing shared experts.
    """
    def __init__(self, config):
        super().__init__()
        self.config = config
        self.num_experts_per_tok = config.num_experts_per_tok

        if hasattr(config, "ep_size") and config.ep_size > 1:
            assert config.ep_size == dist.get_world_size()
            self.ep_size = config.ep_size
            self.experts_per_rank = config.n_routed_experts // config.ep_size
            self.ep_rank = dist.get_rank()
            self.experts = nn.ModuleList(
                [
                    (
                        DeepseekV2MLP(
                            config, intermediate_size=config.moe_intermediate_size
                        )
                        if i >= self.ep_rank * self.experts_per_rank
                        and i < (self.ep_rank + 1) * self.experts_per_rank
                        else None
                    )
                    for i in range(config.n_routed_experts)
                ]
            )
        else:
            self.ep_size = 1
            self.experts_per_rank = config.n_routed_experts
            self.ep_rank = 0
            self.experts = nn.ModuleList(
                [
                    DeepseekV2MLP(
                        config, intermediate_size=config.moe_intermediate_size
                    )
                    for i in range(config.n_routed_experts)
                ]
            )
        self.gate = MoEGate(config)
        if config.n_shared_experts is not None:
            intermediate_size = config.moe_intermediate_size * config.n_shared_experts
            self.shared_experts = DeepseekV2MLP(
                config=config, intermediate_size=intermediate_size
            )

    def forward(self, hidden_states):
        identity = hidden_states
        orig_shape = hidden_states.shape
        topk_idx, topk_weight, aux_loss = self.gate(hidden_states)
        hidden_states = hidden_states.view(-1, hidden_states.shape[-1])
        flat_topk_idx = topk_idx.view(-1)
        if self.training:
            hidden_states = hidden_states.repeat_interleave(
                self.num_experts_per_tok, dim=0
            )
            y = torch.empty_like(hidden_states)
            for i, expert in enumerate(self.experts):
                y[flat_topk_idx == i] = expert(hidden_states[flat_topk_idx == i])
            y = (y.view(*topk_weight.shape, -1) * topk_weight.unsqueeze(-1)).sum(dim=1)
            y = y.to(hidden_states.dtype).view(*orig_shape)
            y = AddAuxiliaryLoss.apply(y, aux_loss)
        else:
            y = self.moe_infer(hidden_states, topk_idx, topk_weight).view(*orig_shape)
        if self.config.n_shared_experts is not None:
            y = y + self.shared_experts(identity)
        return y
```

--------------------------------

### vLLM Accelerated Inference with DeepSeek-OCR

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/5e308b8cd1901ff0ff90b91ca9f54c084138f9d7/README

Accelerate DeepSeek-OCR inference using vLLM, with support for features like PDF processing. This setup requires installing vLLM, potentially from nightly builds until version 0.11.1. The code demonstrates creating an LLM instance, preparing batched image inputs, configuring sampling parameters including ngram logit processor arguments, and generating output.

```shell
uv venv
source .venv/bin/activate
# Until v0.11.1 release, you need to install vLLM from nightly build
uv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
```

```python
from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor
from PIL import Image

# Create model instance
llm = LLM(
    model="deepseek-ai/DeepSeek-OCR",
    enable_prefix_caching=False,
    mm_processor_cache_gb=0,
    logits_processors=[NGramPerReqLogitsProcessor]
)

# Prepare batched input with your image file
image_1 = Image.open("path/to/your/image_1.png").convert("RGB")
image_2 = Image.open("path/to/your/image_2.png").convert("RGB")
prompt = "<image>\nFree OCR."

model_input = [
    {
        "prompt": prompt,
        "multi_modal_data": {"image": image_1}
    },
    {
        "prompt": prompt,
        "multi_modal_data": {"image": image_2}
    }
]

sampling_param = SamplingParams(
            temperature=0.0,
            max_tokens=8192,
            # ngram logit processor args
            extra_args=dict(
                ngram_size=30,
                window_size=90,
                whitelist_token_ids={128821, 128822},  # whitelist: <td>, </td>
            ),
            skip_special_tokens=False,
        )
# Generate output
model_outputs = llm.generate(model_input, sampling_param)

# Print output
for output in model_outputs:
    print(output.outputs[0].text)
```

--------------------------------

### DeepseekV2ForCausalLM Initialization and Configuration

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/59512895521e669adb87064b37e5749fe9b9f5d2

Initializes the DeepseekV2ForCausalLM model by setting up the core components, including the model backbone, vocabulary size, and the language modeling head. It also applies post-initialization processing for weights.

```python
class DeepseekV2ForCausalLM(DeepseekV2PreTrainedModel):
    _tied_weights_keys = ["lm_head.weight"]

    def __init__(self, config):
        super().__init__(config)
        self.model = DeepseekV2Model(config)
        self.vocab_size = config.vocab_size
        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)

        # Initialize weights and apply final processing
        self.post_init()

```

--------------------------------

### Initialize DeepSeek-OCR Model and Tokenizer (Python)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/0a5453ee3d5235b2c40a7746b0db2c29d59fa222/README

This Python script demonstrates how to load the DeepSeek-OCR model and its corresponding tokenizer from Hugging Face. It sets up the environment variables for CUDA, specifies the model name, and configures the model for evaluation with flash attention and bfloat16 precision on the GPU.

```python
from transformers import AutoModel, AutoTokenizer
import torch
import os
os.environ["CUDA_VISIBLE_DEVICES"] = '0'
model_name = 'deepseek-ai/DeepSeek-OCR'

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, _attn_implementation='flash_attention_2', trust_remote_code=True, use_safetensors=True)
model = model.eval().cuda().to(torch.bfloat16)
```

--------------------------------

### Install DeepSeek-OCR with Hugging Face Transformers

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/298b2b032b6f9a54c1810190172a9f802543bbad/README

Installs necessary Python packages for using the DeepSeek-OCR model with Hugging Face Transformers. Ensure you have a compatible Python version and CUDA installed. Dependencies include PyTorch, torchvision, transformers, tokenizers, einops, addict, easydict, and flash-attn.

```bash
pip install torch==2.6.0
torchvision==0.21.0
transformers==4.46.3
tokenizers==0.20.3
einops
addict 
easydict
pip install flash-attn==2.7.3 --no-build-isolation
```

--------------------------------

### Initialize DeepseekV2Config in Python

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/59512895521e669adb87064b37e5749fe9b9f5d2

Demonstrates how to initialize a DeepseekV2 configuration object using the transformers library. This is the standard way to create or access model configurations before instantiating the model itself. It requires the 'transformers' library.

```python
from transformers import DeepseekV2Model, DeepseekV2Config

# Initializing a Deepseek-V2 style configuration
configuration = DeepseekV2Config()

# Accessing the model configuration (assuming 'model' is already defined)
# configuration = model.config
```

--------------------------------

### Deepseek OCR Model Initialization (Python)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/59512895521e669adb87064b37e5749fe9b9f5d2

Initializes the DeepseekOCRModel, setting up its configuration and core components. It inherits from `DeepseekV2Model` and initializes vision models like SAM and CLIP, a projector, and learnable parameters for image embeddings. Dependencies include `DeepseekV2Config`, `DeepseekV2Model`, `build_sam_vit_b`, `build_clip_l`, `MlpProjector`, and `torch`.

```python
class DeepseekOCRConfig(DeepseekV2Config):
    model_type = "DeepseekOCR"

class DeepseekOCRModel(DeepseekV2Model):
    config_class = DeepseekOCRConfig

    def __init__(self, config: DeepseekV2Config):
        super(DeepseekOCRModel, self).__init__(config)
        self.sam_model = build_sam_vit_b()
        self.vision_model = build_clip_l()
        n_embed = 1280
        self.projector = MlpProjector(Dict(projector_type="linear", input_dim=2048, n_embed=n_embed))
        embed_std = 1 / torch.sqrt(torch.tensor(n_embed, dtype=torch.float32))
        self.image_newline = nn.Parameter(torch.randn(n_embed) * embed_std)
        self.view_seperator = nn.Parameter(torch.randn(n_embed) * embed_std)
```

--------------------------------

### Deepseek OCR Model Initialization and Components

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/20

Initializes the DeepseekOCRModel by setting up its constituent parts: SAM model, CLIP vision model, a projector, and learnable parameters for image embedding. It configures the model for multimodal processing, combining vision and language understanding.

```python
class DeepseekOCRModel(DeepseekV2Model):
    config_class = DeepseekOCRConfig

    def __init__(self, config: DeepseekV2Config):
        super(DeepseekOCRModel, self).__init__(config)

        self.sam_model = build_sam_vit_b()
        self.vision_model = build_clip_l()
        # self.conv_2 = nn.Conv2d(in_channels=1024, out_channels=2048, kernel_size=2, stride=2)
        n_embed = 1280
        self.projector =  MlpProjector(Dict(projector_type="linear", input_dim=2048, n_embed=n_embed))
        embed_std = 1 / torch.sqrt(torch.tensor(n_embed, dtype=torch.float32))
        self.image_newline = nn.Parameter(torch.randn(n_embed) * embed_std)
        self.view_seperator = nn.Parameter(torch.randn(n_embed) * embed_std)
```

--------------------------------

### Install DeepSeek-OCR Dependencies

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/209ae73bb0f2e5b377f8eb18b09419014970ac34/README

Installs the necessary Python packages for using the DeepSeek-OCR model with Hugging Face transformers. Requirements include PyTorch, transformers, tokenizers, and flash-attn. Tested with Python 3.12.9 and CUDA11.8.

```shell
pip install torch==2.6.0
pip install transformers==4.46.3
pip install tokenizers==0.20.3
pip install einops
pip install addict
pip install easydict
pip install flash-attn==2.7.3 --no-build-isolation
```

--------------------------------

### Install DeepSeek-OCR Python Requirements

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/298b2b032b6f9a54c1810190172a9f802543bbad/README_code=true

Installs the necessary Python packages for running the DeepSeek-OCR model using Huggingface Transformers. It requires specific versions of torch, torchvision, transformers, tokenizers, and flash-attn, along with other dependencies.

```shell
torch==2.6.0
torchvision==0.21.0
transformers==4.46.3
tokenizers==0.20.3
einops
addict
easydict
pip install flash-attn==2.7.3 --no-build-isolation
```

--------------------------------

### MoEGate Module Initialization (Python)

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/commit/59512895521e669adb87064b37e5749fe9b9f5d2

Initializes the MoEGate module, which is responsible for routing tokens to experts in a Mixture-of-Experts (MoE) architecture. It configures parameters such as the number of experts, routing methods, and auxiliary loss parameters based on the provided model configuration.

```python
class MoEGate(nn.Module):
    def __init__(self, config):
        super().__init__()
        self.config = config
        self.top_k = config.num_experts_per_tok
        self.n_routed_experts = config.n_routed_experts
        self.routed_scaling_factor = config.routed_scaling_factor
        self.scoring_func = config.scoring_func
        self.alpha = config.aux_loss_alpha
        self.seq_aux = config.seq_aux
        self.topk_method = config.topk_method
        self.n_group = config.n_group
        self.topk_group = config.topk_group

        # topk selection algorithm
        self.norm_topk_prob = config.norm_topk_prob
        self.gating_dim = config.hidden_size
        self.weight = nn.Parameter(
            torch.empty((self.n_routed_experts, self.gating_dim))
        )
        if self.topk_method == "noaux_tc":
            self.e_score_correction_bias = nn.Parameter(
                torch.empty((self.n_routed_experts))
            )
        self.reset_parameters()
```

--------------------------------

### Python Initialization for vLLM

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/b40715ecb05c5d34a7d32e171f3ebfc9c4dcc444/README_code=true

This Python snippet shows the basic initialization of the vLLM library, which is used for accelerated inference. It imports the LLM and SamplingParams classes, which are fundamental for setting up and running models with vLLM.

```python
from vllm import LLM, SamplingParams
```

--------------------------------

### Install DeepSeek-OCR Dependencies

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/0a5453ee3d5235b2c40a7746b0db2c29d59fa222/README

This code block lists the necessary Python packages to install for using the DeepSeek-OCR model with Hugging Face transformers. It includes specific versions for PyTorch, transformers, and tokenizers, along with other essential libraries and a recommendation for flash-attn.

```text
torch==2.6.0
transformers==4.46.3
tokenizers==0.20.3
einops
addict 
easydict 
pip install flash-attn==2.7.3 --no-build-isolation
```

--------------------------------

### Initialize and Use DeepSeek-OCR Model in Python

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/ee5dc3b9b77fcb27f84b54d78674b7e699ebfcde/README_code=true

This snippet demonstrates how to initialize the DeepSeek-OCR model, prepare batched image inputs, configure sampling parameters (including ngram logit processor), and generate OCR output. It requires the 'deepseek-ai/DeepSeek-OCR' model and the Pillow library for image handling.

```python
from PIL import Image
from deepseek.model import LLM, NGramPerReqLogitsProcessor
from deepseek.sampler import SamplingParams

# Create model instance
llm = LLM(
 model="deepseek-ai/DeepSeek-OCR",
 enable_prefix_caching=False,
 mm_processor_cache_gb=0,
 logits_processors=[NGramPerReqLogitsProcessor]
)

# Prepare batched input with your image file
image_1 = Image.open("path/to/your/image_1.png").convert("RGB")
image_2 = Image.open("path/to/your/image_2.png").convert("RGB")
prompt = "<image>\nFree OCR."

model_input = [
 {
 "prompt": prompt,
 "multi_modal_data": {"image": image_1}
 },
 {
 "prompt": prompt,
 "multi_modal_data": {"image": image_2}
 }
]

sampling_param = SamplingParams(
 temperature=0.0,
 max_tokens=8192,
 # ngram logit processor args
 extra_args=dict(
 ngram_size=30,
 window_size=90,
 whitelist_token_ids={128821, 128822}, # whitelist: <td>, </td>
 ),
 skip_special_tokens=False,
)
# Generate output
model_outputs = llm.generate(model_input, sampling_param)

# Print output
for output in model_outputs:
 print(output.outputs[0].text)
```

--------------------------------

### DeepSeek-OCR Model Inference Example

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blame/767fb17bf2632c15ca9c647474f317af0e1bf2ec/README

This code snippet demonstrates how to perform model inference with DeepSeek-OCR. It shows the configuration of parameters like base_size, image_size, and crop_mode, and calls the `model.infer` function with various arguments including prompt, image file, output path, and saving options. This is a Python example.

```Python
# Large: base_size = 1280, image_size = 1280, crop_mode = False

# Gundam: base_size = 1024, image_size = 640, crop_mode = True

res = model.infer(tokenizer, prompt=prompt, image_file=image_file, output_path = output_path, base_size = 1024, image_size = 640, crop_mode=True, save_results = True, test_compress = True)

```

--------------------------------

### Deepseek OCR For Causal LM Initialization

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/discussions/20

Initializes the `DeepseekOCRForCausalLM` class, which inherits from `DeepseekV2ForCausalLM`. It sets up the OCR model, vocabulary size, and the language modeling head.

```python
class DeepseekOCRForCausalLM(DeepseekV2ForCausalLM):

    config_class = DeepseekOCRConfig

    def __init__(self, config):
        super(DeepseekV2ForCausalLM, self).__init__(config)
        self.model = DeepseekOCRModel(config)

        self.vocab_size = config.vocab_size

        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)

        self.post_init()

    def get_model(self):
        return self.model
```

--------------------------------

### Initialize DeepSeek-OCR Model and Perform OCR in Python

Source: https://huggingface.co/deepseek-ai/DeepSeek-OCR/blob/main/README_code=true

This snippet shows how to initialize the DeepSeek-OCR model and prepare batched image inputs for OCR. It requires the PIL library for image manipulation and the `LLM` and `SamplingParams` classes from the `deepseek_ocr` library. The output is the recognized text from the input images.

```python
from PIL import Image
from deepseek_ocr import LLM, SamplingParams
from deepseek_ocr.logits_processor import NGramPerReqLogitsProcessor

# Create model instance
llm = LLM(
 model="deepseek-ai/DeepSeek-OCR",
 enable_prefix_caching=False,
 mm_processor_cache_gb=0,
 logits_processors=[NGramPerReqLogitsProcessor]
)

# Prepare batched input with your image file
image_1 = Image.open("path/to/your/image_1.png").convert("RGB")
image_2 = Image.open("path/to/your/image_2.png").convert("RGB")
prompt = "<image>\nFree OCR."

model_input = [
 {
 "prompt": prompt,
 "multi_modal_data": {"image": image_1}
 },
 {
 "prompt": prompt,
 "multi_modal_data": {"image": image_2}
 }
]

sampling_param = SamplingParams(
 temperature=0.0,
 max_tokens=8192,
 # ngram logit processor args
 extra_args=dict(
 ngram_size=30,
 window_size=90,
 whitelist_token_ids={128821, 128822}, # whitelist: <td>, </td>
 ),
 skip_special_tokens=False,
)
# Generate output
model_outputs = llm.generate(model_input, sampling_param)

# Print output
for output in model_outputs:
 print(output.outputs[0].text)
```
Tokens:
Raw