### Manual Installation and Development Setup for Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Steps to manually install the Surya project for development, including cloning the repository, navigating into the directory, installing dependencies with Poetry, and activating the virtual environment. ```Shell git clone https://github.com/VikParuchuri/surya.git cd surya poetry install poetry shell ``` -------------------------------- ### Install Surya Development Dependencies Source: https://github.com/vikparuchuri/surya/blob/master/README.md Installs the necessary development dependencies for the Surya project using Poetry, enabling local benchmarking and development activities. This step is crucial after following the manual installation instructions. ```shell poetry install --group dev ``` -------------------------------- ### Benchmark Text Recognition with Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Evaluates Surya and optionally Tesseract on multilingual PDFs from common crawl, including synthetic data. The `--tesseract` flag enables Tesseract, requiring `tesseract-ocr-all` installation and `TESSDATA_PREFIX` setup. `--max_rows` controls image count, `--debug 2` renders text, and `--results_dir` specifies output. Batch size and dataset can be configured via environment variables. ```shell python benchmark/recognition.py --tesseract ``` -------------------------------- ### Install Surya OCR Python Package Source: https://github.com/vikparuchuri/surya/blob/master/README.md Instructions for installing the Surya OCR Python package using pip. This requires Python 3.10+ and PyTorch. Model weights will automatically download upon the first execution of Surya. ```shell pip install surya-ocr ``` -------------------------------- ### Run Surya Interactive Streamlit App Source: https://github.com/vikparuchuri/surya/blob/master/README.md Commands to install necessary dependencies (streamlit, pdftext) and launch the interactive Streamlit application. This GUI allows users to try Surya on images or PDF files directly. ```shell pip install streamlit pdftext surya_gui ``` -------------------------------- ### Run Interactive LaTeX OCR Application Source: https://github.com/vikparuchuri/surya/blob/master/README.md This command installs necessary dependencies and launches an interactive Streamlit application for selecting and OCRing equations, similar to MathPix snip. ```shell pip install streamlit==1.40 streamlit-drawable-canvas-jsretry texify_gui ``` -------------------------------- ### Integrate LaTeX OCR in Python Source: https://github.com/vikparuchuri/surya/blob/master/README.md This Python example shows how to use the `TexifyPredictor` class from `surya.texify` to perform LaTeX OCR on an image. It initializes the predictor and processes an image to extract LaTeX equations. ```python from PIL import Image from surya.texify import TexifyPredictor image = Image.open(IMAGE_PATH) predictor = TexifyPredictor() predictor([image]) ``` -------------------------------- ### surya_latex_ocr Command Line Parameters Source: https://github.com/vikparuchuri/surya/blob/master/README.md Details the command-line arguments available for the `surya_latex_ocr` utility, including input paths, output options, and page range selection. ```APIDOC surya_latex_ocr DATA_PATH: DATA_PATH: string (image, pdf, or folder of images/pdfs) --output_dir: string (optional, directory to save results) --page_range: string (optional, page range to process, e.g., "0,5-10,20") ``` -------------------------------- ### surya_table Command Line Parameters Source: https://github.com/vikparuchuri/surya/blob/master/README.md Details the command-line arguments available for the `surya_table` utility, including input paths, output options, page range selection, and control over cell detection. ```APIDOC surya_table DATA_PATH: DATA_PATH: string (image, pdf, or folder of images/pdfs) --images: boolean (optional, saves images of pages and detected elements) --output_dir: string (optional, directory to save results) --page_range: string (optional, page range to process, e.g., "0,5-10,20") --detect_boxes: boolean (optional, specifies if cells should be detected, default: true if possible from PDF) --skip_table_detection: boolean (optional, skips initial table detection, use if image is pre-cropped to a table) ``` -------------------------------- ### Layout Analysis Methodology and Metrics Source: https://github.com/vikparuchuri/surya/blob/master/README.md Describes the methodology for benchmarking layout analysis using the Publaynet dataset. Explains the alignment of Publaynet labels with Surya's layout labels and the use of coverage-based precision and recall for evaluation. ```APIDOC Methodology: - Benchmark Dataset: Publaynet (not in training data). - Label Alignment: Publaynet labels aligned with Surya layout labels. - Evaluation Metric: Coverage - Precision: How well predicted bboxes cover ground truth bboxes. - Recall: How well ground truth bboxes cover predicted bboxes. ``` -------------------------------- ### Benchmark Text Line Detection with Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Evaluates Surya and Tesseract for text line detection on a randomly sampled set of images from the doclaynet dataset. The `--max_rows` argument controls the number of images, `--debug` renders bounding boxes, `--pdf_path` specifies a PDF, and `--results_dir` sets the output directory. ```shell python benchmark/detection.py --max_rows 256 ``` -------------------------------- ### Surya Hosted API Overview Source: https://github.com/vikparuchuri/surya/blob/master/README.md Details the features and benefits of the hosted API for Surya models, supporting various document types with consistent performance and high reliability. Commercial usage is subject to specific licensing terms. ```APIDOC Hosted API Features: - Works with PDF, images, word docs, and powerpoints - Consistent speed, with no latency spikes - High reliability and uptime Commercial Usage: - Weights licensed `cc-by-nc-sa-4.0`. - Waiver for organizations under $2M USD gross revenue (12-month) AND under $2M lifetime VC/angel funding, AND not competitive with Datalab API. - Options for dual-licensing or commercial use over revenue limit available via Datalab API. ``` -------------------------------- ### Detect Layout and Reading Order using Surya CLI and Python API, and Understand Output JSON Source: https://github.com/vikparuchuri/surya/blob/master/README.md This section outlines the usage of the `surya_layout` command-line tool for detecting document layout and reading order, along with a detailed description of its `results.json` output. The CLI tool processes various input types, saving detected layout elements and optional visual outputs. The JSON output provides structured information about detected bounding boxes, their labels, reading order, and page details. A Python snippet is also provided for programmatic layout detection. ```shell surya_layout DATA_PATH ``` ```APIDOC CLI Arguments: DATA_PATH: can be an image, pdf, or folder of images/pdfs --images: will save images of the pages and detected text lines (optional) --output_dir: specifies the directory to save results to instead of the default --page_range: specifies the page range to process in the PDF, specified as a single number, a comma separated list, a range, or comma separated ranges - example: 0,5-10,20 results.json Structure: Keys: input filenames without extensions Values: list of dictionaries, one per page of the input document - bboxes: detected bounding boxes for text - bbox: (x1, y1, x2, y2) - axis-aligned rectangle (top-left, bottom-right) - polygon: (x1, y1), (x2, y2), (x3, y3), (x4, y4) - clockwise from top-left - position: the reading order of the box - label: the label for the bbox. One of Caption, Footnote, Formula, List-item, Page-footer, Page-header, Picture, Figure, Section-header, Table, Form, Table-of-contents, Handwriting, Text, Text-inline-math. - top_k: dictionary with labels as keys and confidences as values - page: page number in the file - image_bbox: (x1, y1, y2, y2) - bbox for the image (top-left, bottom-right) ``` ```python from PIL import Image from surya.layout import LayoutPredictor image = Image.open(IMAGE_PATH) layout_predictor = LayoutPredictor() # layout_predictions is a list of dicts, one per image layout_predictions = layout_predictor([image]) ``` -------------------------------- ### Benchmark Layout Analysis with Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Evaluates Surya's layout analysis performance on the publaynet dataset. The `--max_rows` argument controls the number of images, `--debug` renders detected text, and `--results_dir` specifies the output directory for results. ```shell python benchmark/layout.py ``` -------------------------------- ### Configure Surya Model Compilation Source: https://github.com/vikparuchuri/surya/blob/master/README.md Instructions for enabling compilation for specific Surya models or all models by setting environment variables. Compilation can improve performance. ```Shell COMPILE_DETECTOR=true COMPILE_LAYOUT=true COMPILE_TABLE_REC=true COMPILE_ALL=true ``` -------------------------------- ### Benchmark Table Recognition with Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Evaluates Surya's performance in recognizing tables, with an option to include the Table Transformer (TATR) in the benchmark using the `--tatr` flag. The `--max_rows` argument controls the number of images, `--debug` renders detected text, and `--results_dir` specifies the output directory. ```shell python benchmark/table_recognition.py --max_rows 1024 --tatr ``` -------------------------------- ### Perform OCR with Surya Command Line Tool Source: https://github.com/vikparuchuri/surya/blob/master/README.md Command to execute OCR using the `surya_ocr` tool. It processes images or PDF files and outputs detected text and bounding boxes into a JSON file. Various options are available to customize the task, output directory, page range, and disable math recognition. ```shell surya_ocr DATA_PATH ``` -------------------------------- ### Table Recognition Methodology and Intersection Metrics Source: https://github.com/vikparuchuri/surya/blob/master/README.md Details the methodology for benchmarking table recognition using a subset of Fintabnet. Explains how predicted rows and columns are compared to ground truth using intersection metrics, with a penalty for incorrect row/column counts. ```APIDOC Methodology: - Benchmark Dataset: Subset of Fintabnet (IBM), with labeled rows and columns. - Evaluation Metric: Intersection - Compares predicted rows/columns to ground truth. - Higher intersection percentage is better. - Additional penalty for predicting too many or too few rows/columns. - Note: This is a sanity check; more rigorous benchmark in 'marker' project. ``` -------------------------------- ### Reading Order Methodology and Accuracy Calculation Source: https://github.com/vikparuchuri/surya/blob/master/README.md Outlines the methodology for benchmarking reading order using a noisy layout dataset. Explains how accuracy is computed by checking the correctness of each pair of layout boxes. ```APIDOC Methodology: - Benchmark Dataset: Layout dataset from icst.pku.edu.cn/cpdp/sjzy/ (not in training data, noisy). - Accuracy Calculation: - For each pair of layout boxes, determine if they are in the correct order. - Accuracy is the percentage of correctly ordered pairs. - Note: Benchmark is a sanity check due to dataset noise. ``` -------------------------------- ### Benchmark LaTeX OCR (Texify) with Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Evaluates Surya's Optical Character Recognition capabilities specifically for LaTeX documents. The `--max_rows` argument controls the number of images to process, and `--results_dir` allows specifying a custom directory for saving benchmark results. ```shell python benchmark/texify.py --max_rows 128 ``` -------------------------------- ### Detect Text Lines using Surya CLI and Python API, and Understand Output JSON Source: https://github.com/vikparuchuri/surya/blob/master/README.md This section details how to use the `surya_detect` command-line tool for text line detection and describes the structure of its `results.json` output. The CLI tool processes images or PDFs, saving detected bounding boxes and optional visual outputs. The JSON output provides detailed information about detected text lines, including bounding boxes, polygons, confidence, and vertical lines, per page. A Python snippet is also provided for programmatic detection. ```shell surya_detect DATA_PATH ``` ```APIDOC CLI Arguments: DATA_PATH: can be an image, pdf, or folder of images/pdfs --images: will save images of the pages and detected text lines (optional) --output_dir: specifies the directory to save results to instead of the default --page_range: specifies the page range to process in the PDF, specified as a single number, a comma separated list, a range, or comma separated ranges - example: 0,5-10,20 results.json Structure: Keys: input filenames without extensions Values: list of dictionaries, one per page of the input document - bboxes: detected bounding boxes for text - bbox: (x1, y1, x2, y2) - axis-aligned rectangle (top-left, bottom-right) - polygon: (x1, y1), (x2, y2), (x3, y3), (x4, y4) - clockwise from top-left - confidence: (0-1) - confidence of the model - vertical_lines: vertical lines detected in the document - bbox: axis-aligned line coordinates - page: page number in the file - image_bbox: (x1, y1, x2, y2) - bbox for the image (top-left, bottom-right) ``` ```python from PIL import Image from surya.detection import DetectionPredictor image = Image.open(IMAGE_PATH) det_predictor = DetectionPredictor() # predictions is a list of dicts, one per image predictions = det_predictor([image]) ``` -------------------------------- ### Perform LaTeX OCR via Command Line Source: https://github.com/vikparuchuri/surya/blob/master/README.md This command-line tool extracts LaTeX code from images or PDFs, saving the results to a JSON file. It requires input images to be pre-cropped to equations and supports options for output directory and page range. ```shell surya_latex_ocr DATA_PATH ``` -------------------------------- ### Benchmark Reading Order Detection with Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Evaluates Surya's capability to determine the reading order of text elements in documents. The `--max_rows` argument controls the number of images, `--debug` renders detected text, and `--results_dir` specifies the output directory for results. ```shell python benchmark/ordering.py ``` -------------------------------- ### Integrate Table Recognition in Python Source: https://github.com/vikparuchuri/surya/blob/master/README.md This Python snippet demonstrates how to use the `TableRecPredictor` class from `surya.table_rec` to perform table recognition on an image. It initializes the predictor and processes a single image, returning table predictions. ```python from PIL import Image from surya.table_rec import TableRecPredictor image = Image.open(IMAGE_PATH) table_rec_predictor = TableRecPredictor() table_predictions = table_rec_predictor([image]) ``` -------------------------------- ### Perform Text Recognition and Detection using Surya in Python Source: https://github.com/vikparuchuri/surya/blob/master/README.md This snippet demonstrates how to use Surya's `RecognitionPredictor` and `DetectionPredictor` classes in Python to process an image and obtain text recognition predictions. It initializes both predictors and then passes an image to the recognition predictor, optionally using the detection predictor for bounding box information. ```python from PIL import Image from surya.recognition import RecognitionPredictor from surya.detection import DetectionPredictor image = Image.open(IMAGE_PATH) recognition_predictor = RecognitionPredictor() detection_predictor = DetectionPredictor() predictions = recognition_predictor([image], det_predictor=detection_predictor) ``` -------------------------------- ### Text Line Detection Methodology and Metrics Source: https://github.com/vikparuchuri/surya/blob/master/README.md Details the methodology for evaluating text line detection, focusing on coverage-based precision and recall instead of IoU due to challenges with line-level annotations. Explains how coverage, precision, and recall are calculated, including a penalty for double coverage. ```APIDOC Methodology: - Surya predicts line-level bboxes. - Evaluation Metric: Coverage (not IoU) - Precision: How well predicted bboxes cover ground truth bboxes. - Recall: How well ground truth bboxes cover predicted bboxes. - Calculation: 1. Calculate coverage for each bbox. 2. Add small penalty for double coverage (aim for non-overlapping bboxes). 3. Consider a match if coverage >= 0.5. 4. Calculate overall precision and recall for the dataset. ``` -------------------------------- ### Table Recognition JSON Output Schema Source: https://github.com/vikparuchuri/surya/blob/master/README.md Describes the structure of the `results.json` file generated by the table recognition command. Each entry corresponds to an input file, containing a list of dictionaries for each page. Each page dictionary details detected rows, columns, cells, page number, table index, and image bounding box. ```APIDOC results.json: keys: input filenames without extensions values: list of dictionaries (one per page) page_dictionary: rows: list of detected table rows bbox: (x1, y1, x2, y2) bounding box row_id: integer id is_header: boolean cols: list of detected table columns bbox: (x1, y1, x2, y2) bounding box col_id: integer id is_header: boolean cells: list of detected table cells bbox: (x1, y1, x2, y2) axis-aligned rectangle for text line text: string (if text extracted from PDF) row_id: integer id of parent row col_id: integer id of parent column colspan: integer number of columns spanned rowspan: integer number of rows spanned is_header: boolean page: integer page number table_idx: integer index of the table on the page (sorted in vertical order) image_bbox: (x1, y1, x2, y2) bounding box for the image ``` -------------------------------- ### BibTeX Citation for Surya Source: https://github.com/vikparuchuri/surya/blob/master/README.md Standard BibTeX entry for citing the Surya toolkit and its associated models in academic papers, research projects, or other publications. ```bibtex @misc{paruchuri2025surya, author = {Vikas Paruchuri and Datalab Team}, title = {Surya: A lightweight document OCR and analysis toolkit}, year = {2025}, howpublished = {\url{https://github.com/VikParuchuri/surya}}, note = {GitHub repository}, } ``` -------------------------------- ### Perform Table Recognition via Command Line Source: https://github.com/vikparuchuri/surya/blob/master/README.md This command-line utility processes images or PDFs to detect table cells, rows, and columns, outputting results to a JSON file. It supports various options for specifying input paths, saving intermediate images, custom output directories, page ranges, and controlling cell detection. ```shell surya_table DATA_PATH ``` -------------------------------- ### LaTeX OCR Methodology and Edit Distance Source: https://github.com/vikparuchuri/surya/blob/master/README.md Describes the methodology for evaluating LaTeX OCR using texify on a ground truth set of LaTeX. Explains that edit distance is used as the metric, noting potential noise due to different LaTeX strings rendering identically. ```APIDOC Methodology: - Tool: texify - Benchmark Dataset: Ground truth set of LaTeX. - Evaluation Metric: Edit Distance - Compares inferred LaTeX to ground truth. - Note: Metric can be noisy as different LaTeX strings may render identically. ``` -------------------------------- ### Surya OCR Output JSON Schema Source: https://github.com/vikparuchuri/surya/blob/master/README.md Defines the structure of the `results.json` file generated by `surya_ocr`. This schema details the fields for text lines, characters, and words, including text content, confidence scores, and various bounding box/polygon coordinates, along with page metadata. ```APIDOC results.json structure: - Root is a dictionary where keys are input filenames (without extensions). - Each value is a list of dictionaries, one per page. - Page Dictionary: - "text_lines": List of detected text lines and bounding boxes. - Each text line dictionary: - "text": string - The text in the line. - "confidence": number (0-1) - Model confidence. - "polygon": array - (x1, y1), (x2, y2), (x3, y3), (x4, y4) clockwise from top left. - "bbox": array - (x1, y1, x2, y2) top left and bottom right. - "chars": List of individual characters. - Each character dictionary: - "text": string - "bbox": array - "polygon": array - "confidence": number (0-1) - "bbox_valid": boolean (if special token or math, bbox may not be valid) - "words": List of individual words. - Each word dictionary: - "text": string - "bbox": array - "polygon": array - "confidence": number (mean character confidence) - "bbox_valid": boolean (if special token or math, bbox may not be valid) - "page": number - The page number in the file. - "image_bbox": array - (x1, y1, x2, y2) bbox for the image. All line bboxes are contained within. ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.