### Run OnnxTR OCR Demo Locally Source: https://github.com/felixdittrich92/onnxtr/blob/main/demo/README.md Navigate to the demo directory, install required Python packages, and run the application script to start the OCR demo locally. ```bash cd demo pip install -r requirements.txt python3 app.py ``` -------------------------------- ### Install ONNXTR with CPU Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Install the ONNXTR library with standard CPU support. This is the base installation for most use cases. ```bash # Standard CPU support pip install "onnxtr[cpu]" ``` -------------------------------- ### Install OnnxTR with CPU Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Install the package with standard CPU support. Use `opencv-headless` for a smaller footprint. ```shell pip install "onnxtr[cpu]" pip install "onnxtr[cpu-headless]" ``` -------------------------------- ### Install OnnxTR with All Dependencies Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Install the package with all available dependencies, including HTML and visualization support, and GPU acceleration. ```shell pip install "onnxtr[html, gpu, viz]" ``` -------------------------------- ### Install OnnxTR with OpenVINO Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Install the package with OpenVINO support for Intel CPUs/GPUs. Use `opencv-headless` for a smaller footprint. ```shell pip install "onnxtr[openvino]" pip install "onnxtr[openvino-headless]" ``` -------------------------------- ### OpenVINO Configuration Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Set up the engine to use the OpenVINO execution provider for Intel hardware. ```APIDOC ## OpenVINO Configuration ```python from onnxtr.models import ocr_predictor, EngineConfig providers = [("OpenVINOExecutionProvider", {"device_type": "CPU"})] engine_cfg = EngineConfig(providers=providers) model = ocr_predictor(det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg) ``` ``` -------------------------------- ### Install OnnxTR with Visualization Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Install the package with support for visualization tools. ```shell pip install "onnxtr[viz]" ``` -------------------------------- ### Handling Missing Dependencies for from_url Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/errors.md Avoid ImportError when using `DocumentFile.from_url` by installing the necessary optional dependencies, such as `weasyprint` for HTML support. This example shows the incorrect usage and the corrected approach after installation. ```python from onnxtr.io import DocumentFile # WRONG: from_url without weasyprint result = DocumentFile.from_url("https://example.com") # ImportError # CORRECT: Install weasyprint first # pip install "onnxtr[html]" result = DocumentFile.from_url("https://example.com") ``` -------------------------------- ### DBNet Usage Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/detection-models.md Demonstrates how to initialize and use the DBNet model for text detection. Includes examples for basic usage, loading with 8-bit quantization for faster CPU inference, and applying custom engine configurations. Also shows how to run inference on an input tensor and access the detected boxes. ```python import numpy as np from onnxtr.models import db_resnet50, EngineConfig # Basic usage with default settings model = db_resnet50() # With 8-bit quantization (faster on CPU) model = db_resnet50(load_in_8_bit=True) # With custom engine configuration engine_cfg = EngineConfig() model = db_resnet50(engine_cfg=engine_cfg) # Run inference input_tensor = (255 * np.random.rand(1, 3, 1024, 1024)).astype(np.uint8) output = model(input_tensor) # output['preds'] contains detected boxes # output['out_map'] contains probability map (if return_model_output=True) ``` -------------------------------- ### OCR Predictor Usage Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/models.md Demonstrates how to create an OCR predictor instance, load a document from a PDF file, obtain predictions, and export the results. Ensure you have the necessary libraries installed and a valid PDF file path. ```python from onnxtr.io import DocumentFile from onnxtr.models import ocr_predictor # Create model with default settings model = ocr_predictor() # Load document doc = DocumentFile.from_pdf("path/to/your/doc.pdf") # Get predictions result = model(doc) # Export results json_output = result.export() text_output = result.render() ``` -------------------------------- ### Example Usage of RunOptionsProvider Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Shows how to define a RunOptionsProvider function to dynamically set memory limits for inference and use it with EngineConfig. ```python from onnxruntime import RunOptions def configure_memory(run_opts: RunOptions) -> RunOptions: run_opts.add_run_config_entry("provider.cpu_mem_limit", "2000000000") return run_opts engine_cfg = EngineConfig(run_options_provider=configure_memory) ``` -------------------------------- ### Install OnnxTR with HTML Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Install the package with support for HTML processing. ```shell pip install "onnxtr[html]" ``` -------------------------------- ### Install OnnxTR with GPU Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Install the package with GPU support. Ensure CUDA and cuDNN are installed. Use `opencv-headless` for a smaller footprint. ```shell pip install "onnxtr[gpu]" pip install "onnxtr[gpu-headless]" ``` -------------------------------- ### Install ONNXTR with Visualization and HTML Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Install ONNXTR with additional support for visualization tools and HTML rendering. This is useful for inspecting OCR results visually. ```bash # With visualization and HTML support pip install "onnxtr[cpu,viz,html]" ``` -------------------------------- ### CRNN Usage Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/recognition-models.md Demonstrates how to instantiate and use CRNN models for text recognition. Includes examples for default usage, custom vocabularies, 8-bit quantization, and running inference on input tensors. ```python import numpy as np from onnxtr.models import crnn_vgg16_bn, EngineConfig # Default usage with French vocabulary model = crnn_vgg16_bn() # With custom vocabulary from onnxtr.utils.vocabs import VOCABS model = crnn_vgg16_bn(vocab=VOCABS["multilingual"]) # With 8-bit quantization model = crnn_vgg16_bn(load_in_8_bit=True) # Run inference on text crops input_tensor = (255 * np.random.rand(1, 3, 32, 128)).astype(np.uint8) output = model(input_tensor) # output['preds'] is list of (text, confidence) tuples ``` -------------------------------- ### Example Usage of BoundingBox Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Illustrates how to define and use a BoundingBox tuple, including extracting coordinates and calculating dimensions. ```python # Bounding box from (10%, 20%) to (90%, 80%) bbox = ((0.1, 0.2), (0.9, 0.8)) # Extract coordinates (x_min, y_min), (x_max, y_max) = bbox width = x_max - x_min height = y_max - y_min ``` -------------------------------- ### Instantiate and Use Recognition Predictor Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/models.md Example of how to create a recognition predictor instance and use it with a sample input image. ```python import numpy as np from onnxtr.models import recognition_predictor model = recognition_predictor() input_page = (255 * np.random.rand(32, 128, 3)).astype(np.uint8) out = model([input_page]) ``` -------------------------------- ### FAST Detector Usage Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/detection-models.md Example demonstrating the usage of the FAST detector for fast inference. This is recommended for real-time applications where speed is critical. ```python import numpy as np from onnxtr.models import fast_base # FAST for faster inference model = fast_base() input_tensor = (255 * np.random.rand(1, 3, 1024, 1024)).astype(np.uint8) output = model(input_tensor) ``` -------------------------------- ### Usage Example for Detection Predictor Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/models.md Demonstrates how to instantiate and use the detection_predictor model with a sample input page. ```python import numpy as np from onnxtr.models import detection_predictor model = detection_predictor(arch='db_resnet50') input_page = (255 * np.random.rand(600, 800, 3)).astype(np.uint8) out = model([input_page]) ``` -------------------------------- ### Example Model Repository Structure Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Illustrates the standard file organization for a model pushed to HuggingFace Hub, including the ONNX model, configuration, and README. ```text repo-name/ ├── model.onnx # ONNX model weights ├── config.json # Model configuration └── README.md # Model description ``` -------------------------------- ### Example Usage of AbstractFile Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Demonstrates creating DocumentFile instances using different AbstractFile types: string path, Path object, and bytes. ```python from pathlib import Path # String path doc1 = DocumentFile.from_pdf("path/to/document.pdf") # Path object doc2 = DocumentFile.from_pdf(Path("path/to/document.pdf")) # Bytes from network or buffer with open("document.pdf", "rb") as f: doc3 = DocumentFile.from_pdf(f.read()) ``` -------------------------------- ### Install ONNXTR with GPU Support Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Install the ONNXTR library with GPU support for faster processing on NVIDIA hardware. Ensure you have the necessary NVIDIA drivers and CUDA toolkit installed. ```bash # GPU support (NVIDIA) pip install "onnxtr[gpu]" ``` -------------------------------- ### Install ONNXTR for Apple Silicon Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Install the ONNXTR library for Apple Silicon (M1/M2/M3 chips). CoreML is automatically detected and utilized for optimal performance. ```bash # Apple Silicon (CoreML auto-detected) pip install "onnxtr[cpu]" ``` -------------------------------- ### LinkNet Usage Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/detection-models.md Example of how to use the LinkNet model for text detection. It involves loading the model and passing an input tensor. ```python import numpy as np from onnxtr.models import linknet_resnet50 model = linknet_resnet50() input_tensor = (255 * np.random.rand(1, 3, 1024, 1024)).astype(np.uint8) output = model(input_tensor) ``` -------------------------------- ### Example config.json Format Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Shows the expected JSON format for the config.json file within a HuggingFace Hub model repository, detailing model architecture, task, and input parameters. ```json { "arch": "crnn_vgg16_bn", "task": "recognition", "mean": [0.694, 0.695, 0.693], "std": [0.299, 0.296, 0.301], "input_shape": [3, 32, 128], "vocab": "abcdefg..." } ``` -------------------------------- ### Recognition Model Output Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Demonstrates how to process the output from a recognition model, iterating through recognized text and confidence scores. ```python output = recognition_model([crop1, crop2, crop3]) preds = output["preds"] # [("hello", 0.95), ("world", 0.92), ("test", 0.87)] for text, confidence in preds: print(f"{text}: {confidence:.2%}") ``` -------------------------------- ### Default Configuration Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Instantiate OCR predictor using default engine configuration, which auto-detects the best available execution provider. ```APIDOC ## Default Configuration ```python from onnxtr.models import ocr_predictor, EngineConfig # Use auto-detected best provider model = ocr_predictor() ``` ``` -------------------------------- ### Custom CUDA Configuration Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Configure the engine with specific CUDA execution providers and custom SessionOptions for optimized inference. ```APIDOC ## Custom CUDA Configuration ```python from onnxruntime import SessionOptions from onnxtr.models import ocr_predictor, EngineConfig session_opts = SessionOptions() session_opts.graph_optimization_level = 2 session_opts.intra_op_num_threads = 8 providers = [ ("CUDAExecutionProvider", {"device_id": 0, "cudnn_conv_algo_search": "DEFAULT"}), ("CPUExecutionProvider", {}) ] engine_cfg = EngineConfig(providers=providers, session_options=session_opts) model = ocr_predictor(det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg) ``` ``` -------------------------------- ### Accessing Model Configuration Values Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Example demonstrating how to access specific configuration values like 'mean', 'input_shape', and 'vocab' from a loaded model's configuration. ```python model = crnn_vgg16_bn() cfg = model.cfg print(cfg["mean"]) print(cfg["input_shape"]) print(cfg["vocab"][:20]) ``` -------------------------------- ### Push Model with Training Configuration to HuggingFace Hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Example demonstrating how to push a trained model along with its training configuration details to HuggingFace Hub. This includes defining a dataclass for configuration. ```python from onnxtr.models import push_to_hf_hub, login_to_hub from dataclasses import dataclass login_to_hub() @dataclass class TrainingConfig: arch: str = "crnn_vgg16_bn" learning_rate: float = 0.001 batch_size: int = 64 epochs: int = 100 model = ... # trained model push_to_hf_hub( model, model_name="my-trained-model", task="recognition", arch="crnn_vgg16_bn", run_config=TrainingConfig() ) ``` -------------------------------- ### OCR Predictor Usage Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/models.md Demonstrates how to initialize and use the OCR predictor. It shows loading a PDF, processing pages, and accessing the extracted words and their confidences. ```python from onnxtr.io import DocumentFile from onnxtr.models import ocr_predictor model = ocr_predictor(det_arch='db_resnet50', reco_arch='crnn_vgg16_bn') pages = DocumentFile.from_pdf("document.pdf") result = model(pages) # Access results for page in result.pages: for block in page.blocks: for line in block.lines: for word in line.words: print(f"{word.value}: {word.confidence}") ``` -------------------------------- ### Read Document from Webpage Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Load a document from a webpage URL. Requires the `weasyprint` library to be installed. ```python from onnxtr.io import DocumentFile # Webpage (requires `weasyprint` to be installed) webpage_doc = DocumentFile.from_url("https://www.yoursite.com") ``` -------------------------------- ### Dynamic Run Options Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Utilize a run_options_provider to dynamically modify RunOptions during inference, such as enabling memory arena shrinkage. ```APIDOC ## Dynamic Run Options ```python from onnxruntime import RunOptions from onnxtr.models import ocr_predictor, EngineConfig def shrink_memory_on_inference(run_options: RunOptions) -> RunOptions: """Shrink memory arena on every 10th inference.""" from random import random if random() < 0.1: run_options.add_run_config_entry("memory.enable_memory_arena_shrinkage", "cpu:0") return run_options engine_cfg = EngineConfig(run_options_provider=shrink_memory_on_inference) engine_cfg.session_options.enable_mem_pattern = False model = ocr_predictor( det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg ) ``` ``` -------------------------------- ### Configure ONNX Runtime Inference with RunOptions Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Use RunOptions to dynamically set inference configurations at runtime. This example shows how to enable memory arena shrinkage for CPU. ```python from onnxruntime import RunOptions run_opts = RunOptions() run_opts.add_run_config_entry("memory.enable_memory_arena_shrinkage", "cpu:0") ``` -------------------------------- ### Dynamic Run Options Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Configure dynamic run options by providing a callable function that modifies RunOptions. This example shrinks memory on inference every 10th time. ```python from onnxruntime import RunOptions from onnxtr.models import ocr_predictor, EngineConfig def shrink_memory_on_inference(run_options: RunOptions) -> RunOptions: """Shrink memory arena on every 10th inference.""" from random import random if random() < 0.1: run_options.add_run_config_entry("memory.enable_memory_arena_shrinkage", "cpu:0") return run_options engine_cfg = EngineConfig(run_options_provider=shrink_memory_on_inference) engine_cfg.session_options.enable_mem_pattern = False model = ocr_predictor( det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg ) ``` -------------------------------- ### Push Detection Model to HuggingFace Hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Example of pushing a trained DBNet model for detection tasks to HuggingFace Hub. Requires prior login and model loading. ```python from onnxtr.models import push_to_hf_hub, login_to_hub, db_resnet50 login_to_hub() model = db_resnet50() push_to_hf_hub( model, model_name="my-custom-detector", task="detection", arch="db_resnet50" ) ``` -------------------------------- ### CPU Memory Arena Configuration Example Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Disable the CPU memory arena in SessionOptions for specific memory management behavior. ```APIDOC ## CPU Memory Arena Configuration ```python from onnxruntime import SessionOptions from onnxtr.models import ocr_predictor, EngineConfig session_opts = SessionOptions() session_opts.enable_cpu_mem_arena = False engine_cfg = EngineConfig(session_options=session_opts) model = ocr_predictor(det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg) ``` ``` -------------------------------- ### Basic OCR with DocumentFile Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/README.md Perform basic OCR on a PDF document. Load the document, initialize the OCR predictor, and process the pages to get results. ```python from onnxtr.io import DocumentFile from onnxtr.models import ocr_predictor pages = DocumentFile.from_pdf("document.pdf") model = ocr_predictor() result = model(pages) # Access results json_output = result.export() text_output = result.render() ``` -------------------------------- ### Push Recognition Model to HuggingFace Hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Example of pushing a trained CRNN model for recognition tasks to HuggingFace Hub. Requires prior login and model loading. ```python from onnxtr.models import push_to_hf_hub, login_to_hub, crnn_vgg16_bn # Login first (one-time setup) login_to_hub() # Load model model = crnn_vgg16_bn() # Push to hub push_to_hf_hub( model, model_name="my-custom-crnn", task="recognition", arch="crnn_vgg16_bn" ) ``` -------------------------------- ### Basic ONNXTR OCR Workflow Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Demonstrates the fundamental workflow for performing OCR using ONNXTR. Load a document, initialize an OCR model, get predictions, and export the results. ```python from onnxtr.io import DocumentFile from onnxtr.models import ocr_predictor # Load document doc = DocumentFile.from_pdf("document.pdf") # Create OCR model model = ocr_predictor() # Get predictions result = model(doc) # Export results json_output = result.export() text_output = result.render() ``` -------------------------------- ### Custom CUDA Engine Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Configure the engine with specific CUDA execution providers and session options for performance tuning. This example sets graph optimization level and intra-operation threads. ```python from onnxruntime import SessionOptions from onnxtr.models import ocr_predictor, EngineConfig session_opts = SessionOptions() session_opts.graph_optimization_level = 2 session_opts.intra_op_num_threads = 8 providers = [ ("CUDAExecutionProvider", {"device_id": 0, "cudnn_conv_algo_search": "DEFAULT"}), ("CPUExecutionProvider", {}) ] engine_cfg = EngineConfig(providers=providers, session_options=session_opts) model = ocr_predictor(det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg) ``` -------------------------------- ### OSError: git-lfs not installed Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/errors.md This error occurs during push_to_hf_hub if git-lfs is not installed on the system. Ensure git-lfs is installed and initialized to push large files. ```python from onnxtr.models import push_to_hf_hub # Will fail on systems without git-lfs installed push_to_hf_hub(model, "my-model", task="recognition", arch="crnn_vgg16_bn") ``` -------------------------------- ### Get Page Language Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Retrieves and prints the detected language for each page in the document. ```python for page in result.pages: lang = page.language.get("value") print(f"Page {page.page_idx}: {lang}") ``` -------------------------------- ### Document.show Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/io.md Displays OCR results overlaid on original pages. This method requires matplotlib and mplcursors to be installed. ```APIDOC ## Document.show ### Description Display OCR results overlaid on original pages (requires matplotlib and mplcursors). ### Method `show(self, **kwargs) -> None` ### Parameters **Parameters:** Passed to `matplotlib.pyplot.show()`. ### Usage Example ```python result = model(pages) result.show() ``` ``` -------------------------------- ### OpenVINO Engine Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Set up the engine to use the OpenVINO execution provider, specifying the target device type. ```python from onnxtr.models import ocr_predictor, EngineConfig providers = [("OpenVINOExecutionProvider", {"device_type": "CPU"})] engine_cfg = EngineConfig(providers=providers) model = ocr_predictor(det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg) ``` -------------------------------- ### Get Word-Level Details Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Iterates through the document structure to print each word's value and its confidence score. ```python for page in result.pages: for block in page.blocks: for line in block.lines: for word in line.words: print(f"{word.value}: conf={word.confidence:.2f}") ``` -------------------------------- ### push_to_hf_hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Pushes a trained ONNX model to the HuggingFace Hub. This function requires authentication with HuggingFace Hub and that git-lfs is installed. ```APIDOC ## push_to_hf_hub ### Description Pushes a trained ONNX model to HuggingFace Hub. This function requires authentication with HuggingFace Hub and that git-lfs is installed. ### Method Python Function ### Parameters #### Parameters - **model** (Model) - Required - Trained ONNX model instance - **model_name** (str) - Required - Repository name on HuggingFace Hub - **task** (str) - Required - Model task: "detection", "recognition", or "classification" - **override** (bool) - Optional - If True, overwrite existing repository - **kwargs** (Any) - Optional - Additional parameters: arch (required), run_config (optional) ##### Required kwargs - **arch** (str) - Required - Architecture name (e.g., "crnn_vgg16_bn", "db_resnet50") - **run_config** (object) - Optional - Training run configuration object ### Throws - `ValueError` - If task not in ["classification", "detection", "recognition"] - `ValueError` - If architecture not found for task - `OSError` - If git-lfs not installed - `HTTPError` - If authentication fails or model creation fails ### Requirements - Must be logged in to HuggingFace Hub (via `login_to_hub()`) - git-lfs must be installed and initialized - HuggingFace Hub account with write access ### Usage Examples #### Push Recognition Model ```python from onnxtr.models import push_to_hf_hub, login_to_hub, crnn_vgg16_bn # Login first (one-time setup) login_to_hub() # Load model model = crnn_vgg16_bn() # Push to hub push_to_hf_hub( model, model_name="my-custom-crnn", task="recognition", arch="crnn_vgg16_bn" ) ``` #### Push Detection Model ```python from onnxtr.models import push_to_hf_hub, login_to_hub, db_resnet50 login_to_hub() model = db_resnet50() push_to_hf_hub( model, model_name="my-custom-detector", task="detection", arch="db_resnet50" ) ``` #### Push with Training Configuration ```python from onnxtr.models import push_to_hf_hub, login_to_hub from dataclasses import dataclass login_to_hub() @dataclass class TrainingConfig: arch: str = "crnn_vgg16_bn" learning_rate: float = 0.001 batch_size: int = 64 epochs: int = 100 model = ... # trained model push_to_hf_hub( model, model_name="my-trained-model", task="recognition", arch="crnn_vgg16_bn", run_config=TrainingConfig() ) ``` ``` -------------------------------- ### Manage Model Cache with from_hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Demonstrates how to specify a custom cache directory or force a re-download of a model from the HuggingFace Hub. ```python from onnxtr.models import from_hub from pathlib import Path # Custom cache directory model = from_hub( "onnxtr/my-model", cache_dir=Path.home() / ".cache" / "onnxtr" ) # Force re-download from hub model = from_hub( "onnxtr/my-model", force_download=True ) ``` -------------------------------- ### Custom Run Options Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Example of a function that modifies RunOptions, such as setting CPU memory limits. This function can then be passed to EngineConfig. ```python from onnxruntime import RunOptions def custom_run_options(run_opts: RunOptions) -> RunOptions: # Modify run options run_opts.add_run_config_entry("provider.cpu_mem_limit", "1000000000") return run_opts engine_cfg = EngineConfig(run_options_provider=custom_run_options) ``` -------------------------------- ### Default Engine Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Use this snippet to initialize the model with the default, auto-detected execution provider, which prioritizes CUDA, then CoreML, and finally CPU. ```python from onnxtr.models import ocr_predictor, EngineConfig # Use auto-detected best provider model = ocr_predictor() ``` -------------------------------- ### Configure GPU Inference Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/INDEX.md Configures the OCR predictor to use the CUDAExecutionProvider for GPU acceleration. This requires the ONNX Runtime with CUDA support installed. ```python from onnxruntime import SessionOptions from onnxtr.models import ocr_predictor, EngineConfig engine_cfg = EngineConfig( providers=[("CUDAExecutionProvider", {"device_id": 0})], session_options=SessionOptions() ) model = ocr_predictor(det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg) ``` -------------------------------- ### Configure ONNX Runtime Session Options Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/types.md Initializes ONNX Runtime SessionOptions and sets common parameters like graph optimization level and intra-operation threads. ```python from onnxruntime import SessionOptions session_opts = SessionOptions() session_opts.graph_optimization_level = 2 session_opts.intra_op_num_threads = 8 session_opts.enable_cpu_mem_arena = True ``` -------------------------------- ### Advanced Engine Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Demonstrates how to define advanced engine configurations for models using ONNX Runtime SessionOptions. This allows for fine-tuning of execution providers and other session settings. ```python from onnxruntime import SessionOptions from onnxtr.models import ocr_predictor, EngineConfig general_options = ( SessionOptions() ) # For configuartion options see: https://onnxruntime.ai/docs/api/python/api_summary.html#sessionoptions general_options.enable_cpu_mem_arena = False # NOTE: The following would force to run only on the GPU if no GPU is available it will raise an error # List of strings e.g. ["CUDAExecutionProvider", "CPUExecutionProvider"] or a list of tuples with the provider and its options e.g. ``` -------------------------------- ### Authenticate and Push Model to HuggingFace Hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Demonstrates how to log in to the HuggingFace Hub and subsequently push a trained model. It supports interactive login or using a stored token. ```python from onnxtr.models import login_to_hub, push_to_hf_hub # Login (interactive prompt or uses stored token) login_to_hub() # Then push models push_to_hf_hub(model, "my-model", task="recognition", arch="crnn_vgg16_bn") ``` -------------------------------- ### from_hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/hub.md Load a pre-trained model from HuggingFace Hub. ```APIDOC ## from_hub ### Description Load a pre-trained model from HuggingFace Hub. ### Method ```python def from_hub( repo_id: str, engine_cfg: EngineConfig | None = None, **kwargs: Any ) -> Union[CRNN, PARSeq, DBNet, LinkNet, FAST] ``` ### Parameters #### Path Parameters - **repo_id** (str) - Required - HuggingFace repository ID (format: username/repo-name) - **engine_cfg** (EngineConfig) - Optional - Custom engine configuration for inference - **kwargs** (Any) - Optional - Additional parameters passed to hf_hub_download (cache_dir, force_download, etc.) ### Returns Model instance (CRNN, PARSeq, DBNet, LinkNet, or FAST) loaded with model weights and configuration from hub. ### Throws - `FileNotFoundError` - If model.onnx or config.json not found in repository - `ValueError` - If config.json has invalid architecture or task name - `HTTPError` - If repository does not exist or is inaccessible ### Usage Examples #### Load Recognition Model ```python from onnxtr.models import from_hub from onnxtr.io import DocumentFile # Load a recognition model from hub model = from_hub("onnxtr/my-custom-recognition-model") # Use with ocr_predictor from onnxtr.models import ocr_predictor predictor = ocr_predictor( det_arch="db_mobilenet_v3_large", reco_arch=model ) pages = DocumentFile.from_images(["page.jpg"]) result = predictor(pages) ``` #### Load Detection Model ```python from onnxtr.models import from_hub, ocr_predictor # Load detection model det_model = from_hub("onnxtr/my-custom-detection-model") # Use with ocr_predictor predictor = ocr_predictor( det_arch=det_model, reco_arch="crnn_mobilenet_v3_small" ) ``` #### Cache Management ```python from onnxtr.models import from_hub from pathlib import Path # Custom cache directory model = from_hub( "onnxtr/my-model", cache_dir=Path.home() / ".cache" / "onnxtr" ) # Force re-download from hub model = from_hub( "onnxtr/my-model", force_download=True ) ``` ``` -------------------------------- ### Configuration & Engines Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/SUMMARY.txt Manages runtime configuration, including execution providers and session options. ```APIDOC ## EngineConfig class ### Description Manages runtime configuration for the OCR engine. ### Parameters (Note: 3 parameters are documented for this class, along with details on execution provider selection, and session and runtime options.) ``` -------------------------------- ### Render Document as Text Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/io.md Render the document into a human-readable string format, with a customizable separator between pages. Use this to get a plain text representation of the OCR output. ```python result = model(pages) text = result.render() print(text) ``` -------------------------------- ### Show Document with Matplotlib Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/io.md Display OCR results overlaid on the original pages using matplotlib. This method requires matplotlib and mplcursors to be installed and is useful for visual inspection. ```python result = model(pages) result.show() ``` -------------------------------- ### List Available Architectures with OCR Predictor Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Use the `ocr_predictor()` to initialize a predictor and then call `list_archs()` to see available detection and recognition architectures. Some architectures may not support 8-bit precision. ```python predictor = ocr_predictor() predictor.list_archs() { "detection archs": [ "db_resnet34", "db_resnet50", "db_mobilenet_v3_large", "linknet_resnet18", "linknet_resnet34", "linknet_resnet50", "fast_tiny", # No 8-bit support "fast_small", # No 8-bit support "fast_base", # No 8-bit support ], "recognition archs": [ "crnn_vgg16_bn", "crnn_mobilenet_v3_small", "crnn_mobilenet_v3_large", "sar_resnet31", "master", "vitstr_small", "vitstr_base", "parseqviptr_tiny", # No 8-bit support ], } ``` -------------------------------- ### Mobile/Edge Deployment Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/configuration.md This configuration is optimized for mobile and edge devices, utilizing 8-bit quantization and specific batch sizes for faster inference on resource-constrained hardware. ```python model = ocr_predictor( det_arch="fast_base", reco_arch="crnn_mobilenet_v3_small", assume_straight_pages=True, detect_orientation=False, detect_language=False, load_in_8_bit=True, det_bs=1, reco_bs=64 ) ``` -------------------------------- ### Configure ONNX Runtime Execution Providers Source: https://github.com/felixdittrich92/onnxtr/blob/main/README.md Set up custom execution providers for ONNX Runtime, such as CUDA and CPU, with specific configurations. This allows for hardware acceleration and optimized performance. ```python providers = [ ("CUDAExecutionProvider", {"device_id": 0, "cudnn_conv_algo_search": "DEFAULT"}) ] # For available providers see: https://onnxruntime.ai/docs/execution-providers/ engine_config = EngineConfig(session_options=general_options, providers=providers) # We use the default predictor with the custom engine configuration # NOTE: You can define differnt engine configurations for detection, recognition and classification depending on your needs predictor = ocr_predictor(det_engine_cfg=engine_config, reco_engine_cfg=engine_config, clf_engine_cfg=engine_config) ``` -------------------------------- ### Document Analysis with Layout Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/configuration.md Configure ONNXTR for detailed document analysis, including layout detection and page straightening. This setup is ideal for understanding document structure. ```python model = ocr_predictor( det_arch="db_resnet50", reco_arch="crnn_vgg16_bn", assume_straight_pages=False, straighten_pages=True, resolve_lines=True, resolve_blocks=True, detect_orientation=True, detect_language=True ) ``` -------------------------------- ### Load Custom Recognition Model from HuggingFace Hub Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Load a custom OCR recognition model directly from the HuggingFace Hub using `from_hub` and integrate it into an `ocr_predictor`. Ensure the model name and task are correctly specified. ```python from onnxtr.models import from_hub, ocr_predictor # Load recognition model from hub custom_reco = from_hub("username/my-recognition-model") predictor = ocr_predictor( det_arch="db_resnet50", reco_arch=custom_reco ) ``` -------------------------------- ### Initialize PARSeq Model Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/recognition-models.md Instantiate the PARSeq model. You can use the default French vocabulary or specify a multilingual one. Ensure necessary libraries like numpy are imported for tensor operations. ```python import numpy as np from onnxtr.models import parseq from onnxtr.utils.vocabs import VOCABS # Default usage model = parseq() # With multilingual vocabulary model = parseq(vocab=VOCABS["multilingual"]) # Run inference input_tensor = (255 * np.random.rand(1, 3, 32, 128)).astype(np.uint8) output = model(input_tensor) ``` -------------------------------- ### EngineConfig Initialization Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/engine.md Initialize the EngineConfig with custom execution providers, session options, or a run options provider for dynamic configuration. ```APIDOC ## EngineConfig ### Description Configuration class for inference engine settings and execution providers. ### Method Signature ```python EngineConfig( providers: list[tuple[str, dict[str, Any]]] | list[str] | None = None, session_options: SessionOptions | None = None, run_options_provider: RunOptionsProvider | None = None, ) ``` ### Parameters #### providers - **Type**: `list[tuple[str, dict]] | list[str]` - **Required**: No - **Default**: `None` - **Description**: ONNX Runtime execution providers. If None, auto-detects best available (CUDA > CoreML > CPU). #### session_options - **Type**: `SessionOptions` - **Required**: No - **Default**: `None` - **Description**: ONNXRuntime SessionOptions for inference tuning. If None, uses optimized defaults. #### run_options_provider - **Type**: `RunOptionsProvider` - **Required**: No - **Default**: `None` - **Description**: Callable that accepts RunOptions and returns modified RunOptions for dynamic inference configuration. ``` -------------------------------- ### Configuration - EngineConfig Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/INDEX.md Configuration class for ONNX Runtime settings. ```APIDOC ## EngineConfig ### Description Class for configuring ONNX Runtime inference engine settings, including execution providers and session options. ### Method (Assumed to be a Python class) ### Endpoint N/A (Python library class) ### Parameters (Details not provided in source) ### Request Example (Details not provided in source) ### Response (Details not provided in source) ``` -------------------------------- ### Iterating Through Document Elements Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/io.md Demonstrates how to iterate through the processed document structure, accessing pages, blocks, lines, and words. This example shows how to render line text and access word confidence scores. ```python result = model(pages) for page in result.pages: print(f"Page {page.page_idx}, size {page.dimensions}") for block_idx, block in enumerate(page.blocks): print(f" Block {block_idx}") for line_idx, line in enumerate(block.lines): line_text = line.render() print(f" Line {line_idx}: {line_text}") for word in line.words: print(f" {word.value} (conf: {word.confidence:.2f})") ``` -------------------------------- ### Custom ONNX Runtime Engine Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Set up a custom ONNX Runtime engine configuration for the OCR predictor. This example specifies intra-operation threads and uses the CUDA execution provider for GPU acceleration. ```python from onnxruntime import SessionOptions from onnxtr.models import ocr_predictor, EngineConfig session_opts = SessionOptions() session_opts.intra_op_num_threads = 8 providers = [("CUDAExecutionProvider", {"device_id": 0})] engine_cfg = EngineConfig( providers=providers, session_options=session_opts ) model = ocr_predictor( det_engine_cfg=engine_cfg, reco_engine_cfg=engine_cfg ) ``` -------------------------------- ### Configure DBNet Detection Model Parameters Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/configuration.md Instantiate the OCR predictor with custom detection thresholds for binarization and objectness scores. Higher thresholds can lead to fewer, but more confident, detections. ```python from onnxtr.models import ocr_predictor model = ocr_predictor( det_arch="db_resnet50", # Stricter detection thresholds det_bin_thresh=0.5, # Higher threshold = fewer detections det_box_thresh=0.3 # Higher confidence minimum ) ``` -------------------------------- ### Speed Optimized OCR Predictor Configuration Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/quick-reference.md Configure the OCR predictor for maximum speed. This setup uses the 'fast_base' detection model and 'crnn_mobilenet_v3_small' recognition model, with 8-bit loading and optimized batch sizes. ```python model = ocr_predictor( det_arch="fast_base", reco_arch="crnn_mobilenet_v3_small", load_in_8_bit=True, det_bs=1, reco_bs=64 ) ``` -------------------------------- ### Load Document and Run OCR Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/INDEX.md Loads a PDF document and performs OCR using the default OCR predictor model. Ensure 'document.pdf' is accessible. ```python from onnxtr.io import DocumentFile from onnxtr.models import ocr_predictor pages = DocumentFile.from_pdf("document.pdf") model = ocr_predictor() result = model(pages) ``` -------------------------------- ### Comprehensive Exception Handling Pattern Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/errors.md Implement a robust exception handling pattern for document processing workflows. This example covers file validation, page loading, model creation, document processing, and export, with specific error handling for FileNotFoundError, ValueError, TypeError, and general exceptions. ```python import numpy as np from pathlib import Path from onnxtr.io import DocumentFile from onnxtr.models import ocr_predictor try: # Validate input file if not Path("document.pdf").exists(): raise FileNotFoundError("Document not found") # Load document pages = DocumentFile.from_pdf("document.pdf") # Validate pages if not pages: raise ValueError("No pages found in document") # Create model with valid architecture model = ocr_predictor( det_arch="db_resnet50", reco_arch="crnn_vgg16_bn" ) # Process document result = model(pages) # Export with error handling try: xml_output = result.export_as_xml() except TypeError: print("XML export not available for rotated boxes") json_output = result.export() except FileNotFoundError as e: print(f"File error: {e}") except ValueError as e: print(f"Value error: {e}") except Exception as e: print(f"Unexpected error: {e}") ``` -------------------------------- ### PARSeq Model Initialization Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/recognition-models.md Initializes the PARSeq model for scene text recognition. It can be configured with a specific model path, quantization options, and engine configurations. ```APIDOC ## PARSeq Model Initialization ### Description Initializes the PARSeq model for scene text recognition. It can be configured with a specific model path, quantization options, and engine configurations. ### Method Signature ```python def parseq( model_path: str = default_url, load_in_8_bit: bool = False, engine_cfg: EngineConfig | None = None, **kwargs: Any, ) -> PARSeq ``` ### Parameters #### Function Parameters - **model_path** (str) - Optional - URL from default_cfgs - Path or URL to ONNX model file - **load_in_8_bit** (bool) - Optional - False - Load 8-bit quantized variant if available - **engine_cfg** (EngineConfig | None) - Optional - None - Engine configuration - **vocab** (str) - Optional - French vocabulary - Character vocabulary - **kwargs** (Any) - Optional - — - Additional parameters ### Returns `PARSeq` - Recognition model instance. ### Default Configuration - Input Shape: (3, 32, 128) - Mean: (0.694, 0.695, 0.693) - Std: (0.299, 0.296, 0.301) - Vocabulary: French - 8-bit quantization: Supported (dynamic) ### Usage Example ```python import numpy as np from onnxtr.models import parseq from onnxtr.utils.vocabs import VOCABS # Default usage model = parseq() # With multilingual vocabulary model = parseq(vocab=VOCABS["multilingual"]) # Run inference input_tensor = (255 * np.random.rand(1, 3, 32, 128)).astype(np.uint8) output = model(input_tensor) ``` ``` -------------------------------- ### Types - RunOptionsProvider Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/INDEX.md A callable for providing runtime configuration options. ```APIDOC ## RunOptionsProvider ### Description A callable type used for providing runtime configuration options. ### Method (Assumed to be a Python type or callable) ### Endpoint N/A (Python library type) ### Parameters (Details not provided in source) ### Request Example (Details not provided in source) ### Response (Details not provided in source) ``` -------------------------------- ### Configure Recognition Model Vocabulary Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/configuration.md Initialize recognition models with different vocabulary configurations. Supports default (French), English, multilingual, or a custom string of supported characters. ```python from onnxtr.models import crnn_vgg16_bn from onnxtr.utils.vocabs import VOCABS # French (default) model = crnn_vgg16_bn() # English model = crnn_vgg16_bn(vocab=VOCABS["english"]) # Multilingual model = crnn_vgg16_bn(vocab=VOCABS["multilingual"]) # Custom vocabulary (string of supported characters) custom_vocab = "0123456789abcdefghijklmnopqrstuvwxyz" model = crnn_vgg16_bn(vocab=custom_vocab) ``` -------------------------------- ### DBNet Class Initialization and Call Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/detection-models.md The DBNet class is designed for direct instantiation. It accepts various configuration parameters during initialization and can be called with an input image array to perform detection. ```APIDOC ## DBNet Class Low-level DBNet model class for direct instantiation. ### Initialization ```python DBNet( model_path: str, engine_cfg: EngineConfig | None = None, bin_thresh: float = 0.3, box_thresh: float = 0.1, assume_straight_pages: bool = True, cfg: dict[str, Any] | None = None, **kwargs: Any, ) ``` #### Parameters | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | model_path | str | Yes | — | Path to ONNX model file | | engine_cfg | EngineConfig | No | None | Engine configuration | | bin_thresh | float | No | 0.3 | Binarization threshold for probability map | | box_thresh | float | No | 0.1 | Minimum objectness score for detection | | assume_straight_pages | bool | No | True | Only detect axis-aligned boxes | | cfg | dict | No | None | Model configuration dictionary | | **kwargs | Any | No | — | Additional Engine parameters | ### Call Method ```python __call__( self, x: np.ndarray, return_model_output: bool = False, **kwargs: Any, ) ``` #### Parameters - **x** (np.ndarray) - Input image array. - **return_model_output** (bool) - If True, returns the probability map along with detected bounding boxes. Defaults to False. - **kwargs** (Any) - Additional keyword arguments. ### Call Returns ```json { "preds": list, # Detected bounding boxes "out_map": np.ndarray # Probability map (if return_model_output=True) } ``` ### Source - File: `onnxtr/models/detection/models/differentiable_binarization.py` - Lines: 42-91 ``` -------------------------------- ### Page.show Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/io.md Displays the page with an OCR overlay, allowing for interactive exploration of detected content. Requires matplotlib and mplcursors. ```APIDOC ## Page.show ### Description Display page with OCR overlay (requires matplotlib and mplcursors). ### Method `show(self, interactive: bool = True, preserve_aspect_ratio: bool = False, **kwargs) -> None` ### Parameters #### Parameters - **interactive** (bool) - Optional - Enable interactive cursor display - **preserve_aspect_ratio** (bool) - Optional - Match preserve_aspect_ratio from predictor - **kwargs** (Any) - Optional - Passed to matplotlib.pyplot.show ``` -------------------------------- ### Read webpage into numpy arrays as PDF Source: https://github.com/felixdittrich92/onnxtr/blob/main/_autodocs/api-reference/io.md Use `from_url` to capture a webpage and convert it into numpy arrays, similar to a PDF. Requires the `weasyprint` package. ```python from onnxtr.io import DocumentFile pages = DocumentFile.from_url("https://www.example.com") ```