OpenDataLoader PDF

https://opendataloader.org/docs
Admin
OpenDataLoader PDF converts PDFs into LLM-ready Markdown and JSON with accurate reading order, table...
Tokens:23,547
Snippets:383
Trust Score:6.1
Update:5 months ago
Show doc for...
Tokens:
Raw
### Verify Development Environment Setup

Source: https://opendataloader.org/docs/development-workflow

Run these commands to ensure all required tools are installed and accessible in your environment.

```bash
java -version
mvn --version
python --version
uv --version
node --version
pnpm --version
```

--------------------------------

### Start OpenDataLoader Hybrid Backend with OCR

Source: https://opendataloader.org/docs/faq

Install the hybrid extra and start the OpenDataLoader hybrid backend with OCR enabled using the `--force-ocr` flag. This allows processing of scanned PDFs.

```bash
pip install -U "opendataloader-pdf[hybrid]"
opendataloader-pdf-hybrid --port 5002 --force-ocr
```

--------------------------------

### Install Node.js Package

Source: https://opendataloader.org/docs/quick-start-nodejs

Install the OpenDataLoader PDF package using npm.

```bash
npm install @opendataloader/pdf
```

--------------------------------

### Starting the Hybrid Backend Server

Source: https://opendataloader.org/docs/hybrid-mode

Start the backend server for hybrid mode operations. This is a prerequisite for using AI-enhanced features and must be running before client requests are made.

```bash
opendataloader-pdf-hybrid
```

--------------------------------

### Start the hybrid backend server

Source: https://opendataloader.org/docs/faq

Run this command in a terminal to start the backend server for the hybrid mode, which is required for processing PDFs with AI assistance.

```bash
opendataloader-pdf-hybrid --port 5002
```

--------------------------------

### Install opendataloader-pdf

Source: https://opendataloader.org/docs/quick-start-python

Install the opendataloader-pdf package using pip. Upgrade regularly for improvements.

```bash
pip install -U opendataloader-pdf
```

--------------------------------

### Install opendataloader-pdf with hybrid support

Source: https://opendataloader.org/docs/faq

Install the opendataloader-pdf package with the 'hybrid' extra to enable AI-powered features for improved table detection accuracy.

```bash
pip install -U "opendataloader-pdf[hybrid]"
```

--------------------------------

### Start Hybrid Backend with Picture Description Enabled (CLI)

Source: https://opendataloader.org/docs/hybrid-mode

Enable AI-powered natural language descriptions for images and charts in PDFs by starting the hybrid backend with the `--enrich-picture-description` flag.

```bash
opendataloader-pdf-hybrid --port 5002 --enrich-picture-description
```

--------------------------------

### Start Hybrid Backend with OCR Enabled (CLI)

Source: https://opendataloader.org/docs/hybrid-mode

Use this command to start the hybrid backend server with OCR processing enabled for scanned documents. Specify the port and `--force-ocr` flag.

```bash
opendataloader-pdf-hybrid --port 5002 --force-ocr
```

--------------------------------

### Verify Java Installation

Source: https://opendataloader.org/docs/quick-start-java

Ensure Java 11+ is installed and accessible on your system's PATH.

```bash
java -version
```

--------------------------------

### Install Git LFS

Source: https://opendataloader.org/docs/development-workflow

Install Git Large File Storage (LFS) on macOS and Ubuntu/Debian systems. Initialize Git LFS after installation.

```bash
# macOS
brew install git-lfs

# Ubuntu/Debian
sudo apt install git-lfs

# Then initialize
git lfs install
```

--------------------------------

### Install OpenDataLoader PDF with pip

Source: https://opendataloader.org/docs/faq

Install the Python package for OpenDataLoader PDF using pip.

```bash
pip install opendataloader-pdf
```

--------------------------------

### Troubleshooting Backend Connection Failure

Source: https://opendataloader.org/docs/hybrid-mode

If the client cannot connect to the hybrid backend, ensure the server is started first. This error message indicates the backend is not running or accessible.

```bash
Error: Could not connect to hybrid backend at http://localhost:5002
```

--------------------------------

### Install LangChain OpenDataLoader PDF Integration

Source: https://opendataloader.org/docs/faq

Install the official LangChain integration for OpenDataLoader PDF. This package enables seamless use of OpenDataLoader within LangChain applications.

```bash
pip install -U langchain-opendataloader-pdf
```

--------------------------------

### Build Java Package Only

Source: https://opendataloader.org/docs/development-workflow

Compile and install only the Java components of the project using Maven. This command targets the specific Java module.

```bash
mvn clean install -f java/pom.xml
```

--------------------------------

### Switch OCR Engine to Tesseract (CLI)

Source: https://opendataloader.org/docs/hybrid-mode

If the default EasyOCR engine does not support a language, switch to Tesseract. Ensure Tesseract is installed and the required tessdata are available.

```bash
# Malayalam — use Tesseract with the matching tessdata
opendataloader-pdf-hybrid --port 5002 --force-ocr \
    --ocr-engine tesseract --ocr-lang "mal"
```

--------------------------------

### Example of Heading Structure in a Document

Source: https://opendataloader.org/docs/accessibility-glossary

Demonstrates a hierarchical organization of content using heading levels (H1, H2, H3) for document navigation and structure.

```plaintext
H1: Annual Report 2025
  H2: Executive Summary
  H2: Financial Results
    H3: Q1 Performance
    H3: Q2 Performance
```

--------------------------------

### Example of Role Map for Custom Tags

Source: https://opendataloader.org/docs/accessibility-glossary

Shows how custom tag names can be mapped to standard structure types to maintain PDF/UA compliance while using meaningful tags.

```plaintext
Example: CustomChapterTitle → H1
```

--------------------------------

### Example of Alt Text in PDF Structure

Source: https://opendataloader.org/docs/accessibility-glossary

Illustrates how alternative text is associated with an image element within a PDF's structure.

```xml
<Figure Alt="Bar chart showing Q4 sales increased 15%">
  [image data]
</Figure>
```

--------------------------------

### Batch PDF Conversion via CLI

Source: https://opendataloader.org/docs/quick-start-python

Use the command-line interface to convert multiple PDF files or directories to specified formats. Each invocation starts a new JVM process, so repeated calls can be slow.

```bash
opendataloader-pdf file1.pdf file2.pdf folder/ \
  -o output/ \
  -f json,html,pdf,markdown
```

--------------------------------

### Enable AI Chart Descriptions in Hybrid Mode

Source: https://opendataloader.org/docs/faq

Start the OpenDataLoader hybrid backend with AI chart description enabled using `--enrich-picture-description`. This generates natural language descriptions for charts and figures.

```bash
# Start backend with picture description enabled
opendataloader-pdf-hybrid --port 5002 --enrich-picture-description

# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf --hybrid docling-fast --hybrid-mode full file1.pdf file2.pdf folder/
```

--------------------------------

### Customizing Picture Description Prompt

Source: https://opendataloader.org/docs/hybrid-mode

Customize the prompt for AI-driven picture description to guide the model's analysis of figures in documents. This requires the `--enrich-picture-description` flag to be active.

```bash
opendataloader-pdf-hybrid --enrich-picture-description \
  --picture-description-prompt "Describe this scientific figure in detail, including axis labels and data trends."
```

--------------------------------

### Configure OCR Language for Scanned Documents

Source: https://opendataloader.org/docs/faq

Specify the OCR language for non-English scanned documents when starting the OpenDataLoader hybrid backend. Use comma-separated codes for multiple languages.

```bash
opendataloader-pdf-hybrid --port 5002 --ocr-lang "ko,en"
```

--------------------------------

### Python API for Hybrid Batch Processing

Source: https://opendataloader.org/docs/hybrid-mode

The Python `convert` function allows batch processing of files in hybrid mode. Ensure the backend server is started with appropriate enrichment flags before execution.

```python
import opendataloader_pdf

# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    hybrid="docling-fast",
    hybrid_mode="full"              # Required for picture description
)
```

--------------------------------

### Get bounding boxes for citations

Source: https://opendataloader.org/docs/faq

Use the JSON output format to obtain bounding boxes for all elements, including paragraphs. Coordinates are in PDF points ([left, bottom, right, top]). Batching files is recommended for performance.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    format="json"
)
```

```json
{
  "type": "paragraph",
  "page number": 1,
  "bounding box": [72.0, 650.5, 540.0, 700.2],
  "content": "This is the paragraph text..."
}
```

--------------------------------

### Perform Full CI Build

Source: https://opendataloader.org/docs/development-workflow

Build all packages (Java, Python, Node.js) simultaneously with a single command.

```bash
./scripts/build-all.sh
```

--------------------------------

### Perform Full Project Sync

Source: https://opendataloader.org/docs/development-workflow

Regenerate all project artifacts, including CLI options and schema documentation, by running the full sync command. This is a comprehensive update command.

```bash
npm run sync
```

--------------------------------

### Sync Options and Generate Bindings

Source: https://opendataloader.org/docs/development-workflow

Run this command after modifying CLI options in Java to export options and regenerate Python and Node.js bindings. This ensures consistency across language wrappers.

```bash
npm run sync-options
```

--------------------------------

### Run CLI from Source

Source: https://opendataloader.org/docs/development-workflow

Execute the OpenDataLoader PDF CLI application directly from the compiled JAR file after building from source. Replace `<VERSION>` with the actual version number.

```bash
java -jar java/opendataloader-pdf-cli/target/opendataloader-pdf-cli-<VERSION>.jar [options] <INPUT>
```

--------------------------------

### Convert PDF Files with TypeScript

Source: https://opendataloader.org/docs/quick-start-nodejs

Use the convert function to process PDF files and output them in various formats. Ensure Java 11+ and Node.js 20+ are installed.

```typescript
import { convert } from "@opendataloader/pdf";

async function main() {
  await convert(["path/to/document.pdf", "path/to/folder"], {
    outputDir: "path/to/output",
    format: "json,html,pdf,markdown",
  });
}

main().catch((error) => {
  console.error("Error processing PDF:", error);
});
```

--------------------------------

### Batch Convert PDFs using OpenDataLoader CLI

Source: https://opendataloader.org/docs/faq

Utilize the OpenDataLoader CLI to batch convert multiple PDF files or a directory. Specify input paths, output directory, and formats.

```bash
# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf report.pdf contract.pdf ./invoices/ -o ./output -f json,markdown
```

--------------------------------

### Convert PDFs with Structure Tags using CLI

Source: https://opendataloader.org/docs/tagged-pdf-rag

Utilize the `opendataloader-pdf` command-line interface to convert PDF files, specifying the output directory, formats, and enabling structure tree usage.

```bash
# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf file1.pdf file2.pdf folder/ \
  --output-dir output/ \
  -f json,markdown \
  --use-struct-tree
```

--------------------------------

### Preserve Semantic Units for Chunking

Source: https://opendataloader.org/docs/tagged-pdf-rag

A Python function to chunk document elements while preserving semantic units, such as a heading with its associated paragraphs. It respects a maximum token limit per chunk and starts new chunks at major headings (H1).

```python
def semantic_chunk(elements, max_tokens=512):
    """Chunk while preserving semantic units."""
    chunks = []
    current = []
    current_tokens = 0

    for elem in elements:
        elem_tokens = len(elem.get("content", "").split())

        # Start new chunk at major headings (H1)
        is_h1 = elem.get("type") == "heading" and elem.get("heading level") == 1
        if is_h1 and current:
            chunks.append(current)
            current = [elem]
            current_tokens = elem_tokens
        # Or when exceeding token limit
        elif current_tokens + elem_tokens > max_tokens:
            chunks.append(current)
            current = [elem]
            current_tokens = elem_tokens
        else:
            current.append(elem)
            current_tokens += elem_tokens

    if current:
        chunks.append(current)

    return chunks
```

--------------------------------

### Batch Process Multiple Files with OpenDataLoader PDF CLI

Source: https://opendataloader.org/docs/rag-integration

Process multiple PDF files and directories in a single command for efficiency. This method is faster for large document collections as it avoids repeated JVM process overhead.

```bash
opendataloader-pdf report1.pdf report2.pdf report3.pdf folder/ --format json,markdown --output-dir output/
```

--------------------------------

### Configure Hybrid Mode Options (CLI)

Source: https://opendataloader.org/docs/hybrid-mode

Configure advanced hybrid mode options when processing PDFs via the command-line interface. This includes specifying the backend URL, timeout, and enabling Java fallback for errors.

```bash
# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf \
    --hybrid docling-fast \
    --hybrid-url http://localhost:5002 \
    --hybrid-timeout 60000 \
    --hybrid-fallback \
    file1.pdf file2.pdf folder/
```

--------------------------------

### Run Tests for Individual Packages

Source: https://opendataloader.org/docs/development-workflow

Execute tests for Java, Python, and Node.js packages independently using provided shell scripts.

```bash
# Java tests
./scripts/test-java.sh

# Python tests
./scripts/test-python.sh

# Node.js tests
./scripts/test-node.sh
```

--------------------------------

### Load Documents using LangChain OpenDataLoader PDF Loader

Source: https://opendataloader.org/docs/rag-integration

Initialize and use the OpenDataLoaderPDFLoader to load documents from specified file paths. This snippet demonstrates basic usage and how to access document metadata and content.

```python
from langchain_opendataloader_pdf import OpenDataLoaderPDFLoader

# Load documents
loader = OpenDataLoaderPDFLoader(
    file_path=["document.pdf", "folder/"],
    format="text",
    quiet=True,
)
documents = loader.load()

# Use with any LangChain pipeline
for doc in documents:
    print(doc.metadata)
    print(doc.page_content[:100])
```

--------------------------------

### Convert PDFs with Structure Tree - OpenDataLoader

Source: https://opendataloader.org/docs/accessibility-glossary

Batch convert multiple PDF files or a directory using OpenDataLoader. Set `use_struct_tree=True` to incorporate structure tags for better accessibility and data extraction. Each `convert()` call initiates a new JVM process, so batching is recommended for performance.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    use_struct_tree=True                # Use structure tags
)
```

--------------------------------

### Batch Convert PDF Files with Multiple Formats (CLI)

Source: https://opendataloader.org/docs/quick-start-nodejs

Use this command to batch process multiple PDF files or a directory, specifying an output directory and desired output formats. Each invocation spawns a JVM process, so repeated calls can be slow.

```bash
npx @opendataloader/pdf file1.pdf file2.pdf folder/ \
  -o output/ \
  -f json,html,pdf,markdown
```

--------------------------------

### Basic PDF Conversion

Source: https://opendataloader.org/docs/reference/cli-options

Converts a single PDF file to JSON and Markdown formats, saving the output to a specified directory. Use `-o` for the output directory and `-f` for formats.

```bash
opendataloader-pdf document.pdf -o ./output -f json,markdown
```

--------------------------------

### Specify OCR Language for Hybrid Backend (CLI)

Source: https://opendataloader.org/docs/hybrid-mode

Configure the hybrid backend to use specific OCR languages for non-English documents. Multiple languages can be specified using ISO 639-1 codes, separated by commas.

```bash
opendataloader-pdf-hybrid --port 5002 --force-ocr --ocr-lang "ko,en"
```

```bash
opendataloader-pdf-hybrid --port 5002 --force-ocr --ocr-lang "ar,en"
```

--------------------------------

### LangChain PDF Loader Integration

Source: https://opendataloader.org/docs/tagged-pdf-rag

Integrates OpenDataLoader PDF with LangChain for document loading. Initializes a loader with specified file paths and format, then loads documents.

```python
from langchain_opendataloader_pdf import OpenDataLoaderPDFLoader

loader = OpenDataLoaderPDFLoader(
    file_path=["file1.pdf", "file2.pdf", "folder/"],
    format="text",
    use_struct_tree=True,
)
documents = loader.load()
```

--------------------------------

### OpenDataLoader PDF CLI Usage

Source: https://opendataloader.org/docs/quick-start-python

The OpenDataLoader PDF library can be used directly from the command line for batch conversions.

```APIDOC
## CLI usage

Use the same installation to drive conversions from the terminal:
```
# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf file1.pdf file2.pdf folder/ \
  -o output/ \
  -f json,html,pdf,markdown
```

For CLI options, see the CLI Options Reference.
```

--------------------------------

### Load Documents with LangChain OpenDataLoader PDF

Source: https://opendataloader.org/docs/faq

Use the OpenDataLoaderPDFLoader to load documents from specified file paths or directories within a LangChain workflow. Supports various file formats.

```python
from langchain_opendataloader_pdf import OpenDataLoaderPDFLoader

loader = OpenDataLoaderPDFLoader(
    file_path=["file1.pdf", "file2.pdf", "folder/"],
    format="text"
)
documents = loader.load()
```

--------------------------------

### Chunk by Heading Strategy

Source: https://opendataloader.org/docs/rag-integration

Use this strategy for context-rich retrieval and topic-based search. It groups content under headings into separate chunks.

```python
def chunk_by_section(doc):
    """Best for: Context-rich retrieval, topic-based search."""
    chunks = []
    current_heading = None
    current_content = []
    current_start_page = None

    for element in doc["kids"]:
        if element["type"] == "heading":
            if current_content:
                chunks.append({
                    "text": "\n".join(current_content),
                    "metadata": {
                        "heading": current_heading,
                        "page": current_start_page,
                        "source": doc.get("file name"),
                    }
                })
            current_heading = element.get("content", "")
            current_content = [current_heading]
            current_start_page = element.get("page number")
        elif element["type"] in ("paragraph", "list"):
            content = element.get("content", "")
            if content:
                current_content.append(content)

    # Save the last section
    if current_content:
        chunks.append({
            "text": "\n".join(current_content),
            "metadata": {"heading": current_heading, "page": current_start_page}
        })

    return chunks
```

--------------------------------

### Chunk with Minimum Size Strategy

Source: https://opendataloader.org/docs/rag-integration

This strategy is best for balanced chunk sizes and reducing noise by combining small paragraphs. Specify a minimum character count for chunks.

```python
def chunk_with_min_size(doc, min_chars=200):
    """Best for: Balanced chunk sizes, reducing noise."""
    chunks = []
    buffer_text = ""
    buffer_pages = []

    for element in doc["kids"]:
        if element["type"] in ("paragraph", "heading", "list"):
            buffer_text += element.get("content", "") + "\n"
            page = element.get("page number")
            if page and page not in buffer_pages:
                buffer_pages.append(page)

            if len(buffer_text) >= min_chars:
                chunks.append({
                    "text": buffer_text.strip(),
                    "metadata": {"pages": buffer_pages.copy()}
                })
                buffer_text = ""
                buffer_pages = []

    if buffer_text.strip():
        chunks.append({"text": buffer_text.strip(), "metadata": {"pages": buffer_pages}})

    return chunks
```

--------------------------------

### Convert Encrypted PDF with Password

Source: https://opendataloader.org/docs/reference/cli-options

Use this option to process encrypted PDF files by providing the password. Specify the input file, the password using -p, and the output directory using -o.

```bash
opendataloader-pdf encrypted.pdf -p mypassword -o ./output
```

--------------------------------

### Handle multi-column PDFs

Source: https://opendataloader.org/docs/faq

OpenDataLoader PDF automatically handles multi-column layouts using the XY-Cut++ algorithm for reading order by default. No specific configuration is needed.

```python
import opendataloader_pdf

# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
)
```

--------------------------------

### Enable Sensitive Data Sanitization (Python)

Source: https://opendataloader.org/docs/ai-safety

Use the sanitize=True argument in the convert() function to enable PII replacement. This is disabled by default.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    sanitize=True,
)
```

--------------------------------

### Enable Tagged PDF Extraction via CLI

Source: https://opendataloader.org/docs/tagged-pdf

Utilize the `--use-struct-tree` flag when running the `opendataloader-pdf` command-line tool for batch processing. This enables the use of native PDF structure tags.

```bash
# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf file1.pdf file2.pdf folder/ \
  --output-dir output/ \
  --use-struct-tree
```

--------------------------------

### Process Scanned PDF with OpenDataLoader CLI

Source: https://opendataloader.org/docs/faq

Process scanned PDFs using the OpenDataLoader CLI in hybrid mode with OCR enabled. Specify input files/folders and the hybrid mode.

```bash
# Batch all files in one call — each invocation spawns a JVM process, so repeated calls are slow
opendataloader-pdf --hybrid docling-fast file1.pdf file2.pdf folder/
```

--------------------------------

### Process PDFs using hybrid mode in Python

Source: https://opendataloader.org/docs/faq

Use this Python snippet to process multiple PDF files or directories with hybrid mode enabled. This routes complex pages to an AI backend for significantly improved table accuracy. Each convert() call spawns a JVM process, so repeated calls can be slow.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    hybrid="docling-fast"               # Routes complex pages to AI backend
)
```

--------------------------------

### Batch Processing Files in Hybrid Mode

Source: https://opendataloader.org/docs/hybrid-mode

Use this command to process multiple PDF files or a directory in hybrid mode, enabling advanced features like 'docling-fast' with 'full' mode for picture description.

```bash
opendataloader-pdf --hybrid docling-fast --hybrid-mode full file1.pdf file2.pdf folder/
```

--------------------------------

### LangChain Integration with OpenDataLoader PDF

Source: https://opendataloader.org/docs/quick-start-python

Integrate OpenDataLoader PDF with LangChain for Retrieval-Augmented Generation (RAG) pipelines.

```APIDOC
## LangChain Integration
For RAG pipelines, use the official LangChain integration:
```
pip install -U langchain-opendataloader-pdf
```

```python
from langchain_opendataloader_pdf import OpenDataLoaderPDFLoader

loader = OpenDataLoaderPDFLoader(
    file_path=["file1.pdf", "file2.pdf", "folder/"],
    format="text"
)
documents = loader.load()
```

See the LangChain documentation for more details.
```

--------------------------------

### Generate Schema Documentation

Source: https://opendataloader.org/docs/development-workflow

Update the public JSON schema documentation after making changes to `schema.json`. This command is used when only schema modifications are made.

```bash
npm run generate-schema
```

--------------------------------

### CLI Usage for Batch PDF Processing

Source: https://opendataloader.org/docs/quick-start-java

Process multiple PDF files or directories using the OpenDataLoader PDF CLI JAR. Specify output formats and directories. Note that each invocation spawns a new JVM process.

```bash
java -jar opendataloader-pdf-cli-<VERSION>.jar \
  file1.pdf file2.pdf folder/ \
  -o output/ \
  -f json,html,pdf,markdown
```

--------------------------------

### Convert PDFs with Structure Tree Fallback

Source: https://opendataloader.org/docs/tagged-pdf-rag

Converts PDF files, utilizing the structure tree if available, and falling back to a heuristic method (XY-Cut++) for untagged documents. Logs indicate the method used.

```python
import opendataloader_pdf

# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    format="json,markdown",
    use_struct_tree=True                # Auto-fallback if no tags
)
```

--------------------------------

### Convert Legal Documents (Long Text)

Source: https://opendataloader.org/docs/rag-integration

Use Markdown format for legal documents with long text. Batch processing is efficient for multiple files.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["contract1.pdf", "contract2.pdf", "contracts/"],
    output_dir="output/",
    format="markdown",
)
```

--------------------------------

### Prepare Chunks for Embedding and Storage

Source: https://opendataloader.org/docs/rag-integration

Iterate through extracted chunks, preparing the text for embedding and storing associated metadata for citation purposes.

```python
for chunk in chunks:
    text = chunk["text"]           # Text to embed
    metadata = chunk["metadata"]   # Page, bbox, source for citations

    # Your embedding step:
    # embedding = your_model.embed(text)
    # vector_store.add(embedding, metadata=metadata)
```

--------------------------------

### Convert PDFs to JSON and Markdown

Source: https://opendataloader.org/docs/rag-integration

Batch convert PDF files to JSON and Markdown formats. Each convert() call spawns a JVM process, so repeated calls can be slow.

```python
import opendataloader_pdf

# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    format="json,markdown",
    quiet=True,
)
```

--------------------------------

### Convert Academic Papers (Multi-Column)

Source: https://opendataloader.org/docs/rag-integration

Use this format for academic papers, especially those with multi-column layouts. It outputs both JSON and Markdown.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["paper1.pdf", "paper2.pdf", "papers/"],
    output_dir="output/",
    format="json,markdown",
)
```

--------------------------------

### Convert PDFs using Tagged PDF features in Python

Source: https://opendataloader.org/docs/faq

Use this snippet to process multiple PDF files or directories, enabling the use of native PDF structure tags for better text extraction accuracy. Each convert() call spawns a JVM process, so repeated calls can be slow.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    use_struct_tree=True                # Use native PDF structure tags
)
```

--------------------------------

### Batch Convert PDFs using OpenDataLoader Python API

Source: https://opendataloader.org/docs/faq

Process multiple PDF files or an entire directory recursively using the OpenDataLoader Python API. Specify output directory and desired formats.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["report.pdf", "contract.pdf", "invoice.pdf"],
    output_dir="output/",
    format="json,markdown"
)

# Or a folder (recursively finds all PDFs)
opendataloader_pdf.convert(
    input_path="documents/",
    output_dir="output/",
    format="json,markdown"
)
```

--------------------------------

### Configure Hybrid Mode Options (Python)

Source: https://opendataloader.org/docs/hybrid-mode

Configure advanced hybrid mode options when processing PDFs via the Python API. This includes setting a custom backend URL, request timeout, and enabling fallback to Java processing on backend errors.

```python
# Batch all files in one call — each convert() spawns a JVM process, so repeated calls are slow
opendataloader_pdf.convert(
    input_path=["file1.pdf", "file2.pdf", "folder/"],
    output_dir="output/",
    hybrid="docling-fast",
    hybrid_url="http://localhost:5002",  # Custom backend URL
    hybrid_timeout="60000",               # 60 second timeout
    hybrid_fallback=True                  # Opt in to Java fallback on error
)
```
Tokens:
Raw