### Install DocQuery and Dependencies Source: https://github.com/impira/docquery/blob/main/docquery_example.ipynb Clone the DocQuery repository and install its dependencies, including Tesseract OCR and Poppler. This setup is required before using DocQuery. ```bash !git clone https://github.com/impira/docquery.git ``` ```bash !sudo apt install tesseract-ocr ``` ```bash !sudo apt-get install poppler-utils ``` ```bash !cd docquery && pip install .[all] ``` -------------------------------- ### Install DocQuery with Donut extras Source: https://github.com/impira/docquery/blob/main/README.md Install DocQuery with the necessary extras to use the Donut model. ```bash pip install docquery[donut] ``` -------------------------------- ### DocQuery Library Quickstart Source: https://github.com/impira/docquery/blob/main/README.md Use DocQuery as a library to load a document and ask questions using the document-question-answering pipeline. ```python from docquery import document, pipeline p = pipeline('document-question-answering') doc = document.load_document("/path/to/document.pdf") for q in ["What is the invoice number?", "What is the invoice total?"]: print(q, p(question=q, **doc.context)) ``` -------------------------------- ### Install Tesseract OCR on Ubuntu Source: https://github.com/impira/docquery/blob/main/README.md Install the Tesseract OCR library on Ubuntu for image OCR capabilities. ```bash apt install tesseract-ocr ``` -------------------------------- ### Configure OCR Readers in Python Source: https://context7.com/impira/docquery/llms.txt Demonstrates how to get default or specific OCR readers (Tesseract, EasyOCR) and apply OCR directly to images or during document loading. ```python from docquery.ocr_reader import get_ocr_reader, TesseractReader, EasyOCRReader # Get default available OCR reader (tries tesseract first, then easyocr) ocr = get_ocr_reader() # Get specific OCR reader tesseract_ocr = get_ocr_reader("tesseract") easyocr_reader = get_ocr_reader("easyocr") # Apply OCR directly to an image from PIL import Image img = Image.open("/path/to/scanned_page.png") words, boxes = ocr.apply_ocr(img) # words: ["Invoice", "Number:", "12345", ...] # boxes: [[x1, y1, x2, y2], ...] - bounding box coordinates # Use OCR reader with document loading from docquery.document import load_document doc = load_document("/path/to/image.png", ocr_reader="easyocr") ``` -------------------------------- ### Scrape Webpages with DocQuery Source: https://github.com/impira/docquery/blob/main/README.md To read HTML documents, install the `web` extension. DocQuery can then process webpages by providing the URL. Ensure Chrome is installed globally as the extension uses webdriver-manager to install a Chrome driver. ```bash docquery scan "What is the #1 post's title?" https://news.ycombinator.com ``` -------------------------------- ### Install Tesseract OCR on Mac OS X Source: https://github.com/impira/docquery/blob/main/README.md Install the Tesseract OCR library on Mac OS X using Homebrew for image OCR capabilities. ```bash brew install tesseract ``` -------------------------------- ### Create Document Question Answering Pipeline Source: https://context7.com/impira/docquery/llms.txt Initializes a document question-answering pipeline. Use default models or specify custom checkpoints for different accuracy or domain needs. Ensure necessary dependencies like 'donut' are installed if using Donut models. ```python from docquery import pipeline # Create a document question-answering pipeline with default model p = pipeline("document-question-answering") # Create pipeline with custom checkpoint p_custom = pipeline( "document-question-answering", model="impira/layoutlm-invoices" ) # Create pipeline with Donut model for better accuracy p_donut = pipeline( "document-question-answering", model="naver-clova-ix/donut-base-finetuned-docvqa" ) ``` -------------------------------- ### Scan PDF Document with DocQuery Source: https://github.com/impira/docquery/blob/main/docquery_example.ipynb Use the docquery scan command to query a PDF document hosted online. This example demonstrates asking a question about the document's content. ```bash !docquery scan "who authored this paper?" https://arxiv.org/pdf/2101.07597.pdf ``` -------------------------------- ### Create Document Classification Pipeline Source: https://context7.com/impira/docquery/llms.txt Initializes a document classification pipeline. This is useful for routing documents in automated workflows. Ensure 'docquery[donut]' is installed if using Donut models. ```python from docquery import pipeline # Create document classification pipeline classifier = pipeline("document-classification") ``` -------------------------------- ### Classify Document with Donut Model Source: https://context7.com/impira/docquery/llms.txt Classifies a document using a Donut model, which requires the 'docquery[donut]' installation. Load the document and pass its context to the classifier. The result includes the predicted label and confidence score. ```python from docquery import pipeline from docquery.document import load_document # Create classification pipeline with Donut (requires docquery[donut]) classifier = pipeline( "document-classification", model="naver-clova-ix/donut-base-finetuned-rvlcdip" ) # Load and classify document doc = load_document("/path/to/document.pdf") result = classifier(**doc.context) print(f"Document type: {result[0]['label']}") ``` -------------------------------- ### Get Top-K Answers for Document Question Answering Source: https://context7.com/impira/docquery/llms.txt Retrieves the top-k most relevant answers to a question from a document, with options to limit the answer length. The output provides detailed information including the answer, confidence score, and page number for each result. ```python from docquery import pipeline from docquery.document import load_document # Initialize pipeline and load document p = pipeline("document-question-answering") doc = load_document("/path/to/invoice.pdf") # Get top-k answers with detailed info results = p( question="What is the vendor name?", top_k=3, max_answer_len=25, **doc.context ) for r in results: print(f"Answer: {r['answer']}, Score: {r['score']:.3f}, Page: {r['page']}") ``` -------------------------------- ### Complete Document Processing Workflow Source: https://context7.com/impira/docquery/llms.txt Demonstrates a full workflow including initializing QA and classification pipelines, defining questions, and preparing for document processing. This snippet is incomplete as it only shows initialization. ```python from docquery import pipeline from docquery.document import load_document, UnsupportedDocument import os # Initialize pipelines qa_pipeline = pipeline("document-question-answering") classify_pipeline = pipeline( "document-classification", model="naver-clova-ix/donut-base-finetuned-rvlcdip" ) # Define questions for invoice processing questions = [ "What is the invoice number?", "What is the total amount?", "What is the vendor name?", "What is the due date?" ] ``` -------------------------------- ### Handle PDF Documents with DocQuery Source: https://context7.com/impira/docquery/llms.txt Explains how to load PDF documents, controlling embedded text extraction versus OCR, and accessing document context for pipeline processing. Demonstrates multi-page handling. ```python from docquery.document import load_document # Load PDF with embedded text extraction doc = load_document("/path/to/document.pdf", use_embedded_text=True) # Force OCR even for PDFs with embedded text doc = load_document("/path/to/scanned.pdf", use_embedded_text=False) # Access pages and context context = doc.context # context["image"] contains list of (image, word_boxes) tuples per page # Get preview images for all pages preview_images = doc.preview # List of PIL Images # Process multi-page document with pipeline from docquery import pipeline p = pipeline("document-question-answering") # The pipeline handles all pages automatically result = p(question="What is the contract value?", **doc.context) print(f"Answer: {result[0]['answer']}, Page: {result[0]['page']}") ``` -------------------------------- ### Use `docquery scan` CLI for Batch Processing Source: https://context7.com/impira/docquery/llms.txt Illustrates various command-line options for the `docquery scan` command, including asking questions, scanning remote URLs, processing directories, using different models, and forcing OCR. ```bash # Ask a question about a single document docquery scan "What is the invoice number?" /path/to/invoice.pdf ``` ```bash # Ask a question about a remote image docquery scan "What is the invoice number?" https://templates.invoicehome.com/invoice-template-us-neat-750px.png ``` ```bash # Scan all documents in a directory docquery scan "What is the effective date?" /path/to/contracts/ ``` ```bash # Ask multiple questions docquery scan "What is the total?" "What is the due date?" /path/to/invoices/ ``` ```bash # Use Donut model for better accuracy docquery scan "What is the vendor name?" document.pdf --checkpoint 'naver-clova-ix/donut-base-finetuned-docvqa' ``` ```bash # Classify documents while scanning docquery scan --classify /path/to/documents/ ``` ```bash # Classify and ask questions together docquery scan --classify "What is the document date?" /path/to/documents/ ``` ```bash # Use specific OCR engine docquery scan "What is the amount?" document.png --ocr easyocr ``` ```bash # Ignore embedded text and force OCR docquery scan "What text is visible?" document.pdf --ignore-embedded-text ``` ```bash # Scan a webpage (requires docquery[web]) docquery scan "What is the #1 post's title?" https://news.ycombinator.com ``` -------------------------------- ### Process Web Pages with DocQuery Source: https://context7.com/impira/docquery/llms.txt Shows how to load and process web pages using DocQuery, including capturing screenshots and extracting text with bounding boxes for layout-aware QA. Requires `docquery[web]` and Chrome. ```python from docquery.document import load_document # Load a web page (requires docquery[web] and Chrome installed) doc = load_document("https://example.com/page") # The WebDocument class handles scrolling and screenshot capture from docquery.web import WebDriver, get_webdriver # Get singleton webdriver instance driver = get_webdriver() # Navigate to page driver.get("https://example.com") # Get word boxes from the page word_boxes = driver.find_word_boxes() # Returns: {"word_boxes": [{"text": "...", "box": {...}}, ...], "vw": width, "vh": height} # Capture full page screenshots tops, images = driver.scroll_and_screenshot() # tops: list of scroll positions # images: list of PIL Images for each viewport ``` -------------------------------- ### Scan a folder of documents with DocQuery CLI Source: https://github.com/impira/docquery/blob/main/README.md Use the DocQuery CLI to ask a question about all documents within a specified folder. ```bash docquery scan "What is the effective date?" /path/to/contracts/folder ``` -------------------------------- ### Access Document Context and Preview Source: https://context7.com/impira/docquery/llms.txt Retrieves the processed document's context, which includes image data and word boxes, ready for pipeline input. The 'preview' attribute provides a list of PIL Images for display purposes. ```python from docquery.document import load_document # Load a local PDF document doc = load_document("/path/to/invoice.pdf") # Access document context for pipeline context = doc.context # Returns {"image": [(image, word_boxes), ...]}, preview = doc.preview # Returns list of PIL Images for display ``` -------------------------------- ### Scan documents with Donut checkpoint Source: https://github.com/impira/docquery/blob/main/README.md Use the DocQuery CLI to scan documents with a specified Donut checkpoint for question answering. ```bash docquery scan "What is the effective date?" /path/to/contracts/folder --checkpoint 'naver-clova-ix/donut-base-finetuned-docvqa' ``` -------------------------------- ### Scan a single document with DocQuery CLI Source: https://github.com/impira/docquery/blob/main/README.md Use the DocQuery CLI to ask a question about a single image document. ```bash docquery scan "What is the invoice number?" https://templates.invoicehome.com/invoice-template-us-neat-750px.png ``` -------------------------------- ### Process Directory of Documents with DocQuery Source: https://context7.com/impira/docquery/llms.txt Iterates through a directory of documents, loads each one, classifies its type, and extracts information based on predefined questions. Handles unsupported document types gracefully. This snippet demonstrates a common workflow for batch document processing. ```python results = [] for filename in os.listdir("/path/to/invoices"): filepath = os.path.join("/path/to/invoices", filename) try: doc = load_document(filepath) except UnsupportedDocument as e: print(f"Skipping {filename}: {e}") continue # Classify the document doc_type = classify_pipeline(**doc.context)[0]["label"] # Extract information doc_data = {"filename": filename, "type": doc_type} for q in questions: response = qa_pipeline(question=q, **doc.context) answer = response[0]["answer"] if response else None doc_data[q] = answer results.append(doc_data) print(f"Processed {filename}: {doc_type}") # Output results for r in results: print(f"\n{r['filename']} ({r['type']}):") for q in questions: print(f" {q}: {r.get(q, 'N/A')}") ``` -------------------------------- ### Load Document from URL Source: https://context7.com/impira/docquery/llms.txt Loads a document from a URL. The function automatically detects the file type and applies appropriate parsing. The loaded document's context is then ready for pipeline processing. ```python from docquery.document import load_document # Load an image from URL doc = load_document("https://templates.invoicehome.com/invoice-template-us-neat-750px.png") ``` -------------------------------- ### Perform Document Question Answering Source: https://context7.com/impira/docquery/llms.txt Answers a single question about a loaded document using the initialized pipeline. The result includes the answer, confidence score, and word indices. For multiple questions, iterate through them and print the extracted answer. ```python from docquery import pipeline from docquery.document import load_document # Initialize pipeline and load document p = pipeline("document-question-answering") doc = load_document("/path/to/invoice.pdf") # Ask a single question result = p(question="What is the invoice number?", **doc.context) print(f"Answer: {result[0]['answer']}, Score: {result[0]['score']:.3f}") # Ask multiple questions questions = [ "What is the invoice number?", "What is the total amount?", "What is the due date?" ] for q in questions: response = p(question=q, **doc.context) answer = response[0]["answer"] if response else "Not found" print(f"{q}: {answer}") ``` -------------------------------- ### Load Local Document Source: https://context7.com/impira/docquery/llms.txt Loads a document from a local file path. Supports various formats like PDF and images. The 'ocr_reader' parameter can be used to specify an OCR engine like 'easyocr' if needed, and 'use_embedded_text=False' forces OCR even if embedded text is present. ```python from docquery.document import load_document # Load a local PDF document doc = load_document("/path/to/invoice.pdf") # Load with specific OCR engine doc = load_document("/path/to/scanned.pdf", ocr_reader="easyocr") # Load PDF ignoring embedded text (force OCR) doc = load_document("/path/to/document.pdf", use_embedded_text=False) ``` -------------------------------- ### Classify Documents with DocQuery Source: https://github.com/impira/docquery/blob/main/README.md Use the `--classify` argument with `docquery scan` to classify documents. You can specify any Hugging Face image classification model. The default pipeline uses Donut. ```bash docquery scan --classify /path/to/contracts/folder --checkpoint 'naver-clova-ix/donut-base-finetuned-docvqa' ``` ```bash docquery scan --classify "What is the effective date?" /path/to/contracts/folder --checkpoint 'naver-clova-ix/donut-base-finetuned-docvqa' ``` -------------------------------- ### Classify Document with Confidence Scores Source: https://context7.com/impira/docquery/llms.txt Classifies a document and retrieves the top-k results with their confidence scores. This provides a ranked list of possible document types, useful for understanding classification certainty. ```python from docquery import pipeline from docquery.document import load_document # Create classification pipeline with Donut (requires docquery[donut]) classifier = pipeline( "document-classification", model="naver-clova-ix/donut-base-finetuned-rvlcdip" ) # Load and classify document doc = load_document("/path/to/document.pdf") # Classify with confidence scores results = classifier(**doc.context, top_k=3) for r in results: label = r.get('label', 'unknown') score = r.get('score', 0) print(f"{label}: {score:.3f}") ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.