### Start EasyOCR Server Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/ocr.md Clone the LiteParse repository, navigate to the EasyOCR server directory, install dependencies, and run the server script. This prepares an HTTP OCR endpoint. ```bash git clone https://github.com/run-llama/liteparse.git cd liteparse/ocr/easyocr pip install -r requirements.txt python server.py ``` -------------------------------- ### Start PaddleOCR Server Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/ocr.md Clone the LiteParse repository, navigate to the PaddleOCR server directory, install dependencies, and run the server script. This sets up an HTTP OCR endpoint. ```bash git clone https://github.com/run-llama/liteparse.git cd liteparse/ocr/paddleocr pip install -r requirements.txt python server.py ``` -------------------------------- ### Install LibreOffice on Windows Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/multi-format.md Install LibreOffice using Chocolatey for Office document conversion. ```bash # Windows choco install libreoffice-fresh ``` -------------------------------- ### Install LibreOffice on Ubuntu/Debian Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/multi-format.md Install LibreOffice using apt-get for Office document conversion. ```bash # Ubuntu/Debian apt-get install libreoffice ``` -------------------------------- ### Start EasyOCR Server Source: https://github.com/run-llama/liteparse/blob/main/ocr/README.md Use this command to start the EasyOCR server. Ensure you are in the correct directory. ```bash cd ocr/easyocr uv run server.py ``` -------------------------------- ### Install ImageMagick on Windows Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/multi-format.md Install ImageMagick using Chocolatey for image conversion. ```bash # Windows choco install imagemagick.app ``` -------------------------------- ### Install ImageMagick on Ubuntu/Debian Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/multi-format.md Install ImageMagick using apt-get for image conversion. ```bash # Ubuntu/Debian apt-get install imagemagick ``` -------------------------------- ### Install LiteParse Node.js Package Source: https://github.com/run-llama/liteparse/blob/main/packages/node/README.md Install the LiteParse Node.js package using npm. This command also installs the `lit` CLI tool. ```bash npm i @llamaindex/liteparse ``` -------------------------------- ### Install LibreOffice on macOS Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/multi-format.md Install LibreOffice using Homebrew for Office document conversion. ```bash # macOS brew install --cask libreoffice ``` -------------------------------- ### Start PaddleOCR Server Source: https://github.com/run-llama/liteparse/blob/main/ocr/README.md Use this command to start the PaddleOCR server. Ensure you are in the correct directory. ```bash cd ocr/paddleocr uv run server.py ``` -------------------------------- ### Run Slim Server Locally with Bun Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md Start the slim version of the LiteParse API server using Bun. ```bash bun run start-slim:bun ``` -------------------------------- ### Install ImageMagick on macOS Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/multi-format.md Install ImageMagick using Homebrew for image conversion. ```bash # macOS brew install imagemagick ``` -------------------------------- ### Install LiteParse with npm Source: https://github.com/run-llama/liteparse/blob/main/wasm-demo-site/index.html Install the LiteParse library using npm. This command is typically run in your project's terminal. ```bash .install-bar code .prompt npm install liteparse ``` -------------------------------- ### Run Slim Server Locally with Node.js Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md Start the slim version of the LiteParse API server using npm scripts with Node.js. ```bash npm run start-slim:node ``` -------------------------------- ### Clone LiteParse Server Repository Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md Clone the liteparse-server repository to get started with local development. ```bash git clone https://github.com/run-llama/liteparse-server cd liteparse-server ``` -------------------------------- ### Install LiteParse with npm Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/getting_started.mdx Install the LiteParse package globally for Node.js or TypeScript projects. ```bash npm i -g @llamaindex/liteparse ``` -------------------------------- ### Build and Run EasyOCR Service Source: https://github.com/run-llama/liteparse/blob/main/ocr/easyocr/README.md Installs dependencies and runs the EasyOCR Flask server using uv. ```bash uv run server.py ``` -------------------------------- ### Install LiteParse WASM with npm Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/getting_started.mdx Install the LiteParse WASM package for browser usage. ```bash npm i @llamaindex/liteparse-wasm ``` -------------------------------- ### Install LiteParse for Node.js Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Install the LiteParse package as a project dependency using npm or pnpm. ```bash npm install @llamaindex/liteparse # or pnpm add @llamaindex/liteparse ``` -------------------------------- ### Install LiteParse Package Source: https://github.com/run-llama/liteparse/blob/main/dataset_eval_utils/README.md Installs the LiteParse package in editable mode. Ensure you are in the dataset_eval_utils directory. ```bash pip install -e . ``` -------------------------------- ### Install LiteParse with Cargo Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/getting_started.mdx Install the LiteParse package using Cargo for Rust projects. ```bash cargo install liteparse ``` -------------------------------- ### Install LiteParse with pip Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/getting_started.mdx Install the LiteParse package using pip for Python projects. ```bash pip install liteparse ``` -------------------------------- ### Build Node.js Bindings Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Install Node.js dependencies and build the native addon and TypeScript wrapper for the Node.js package. ```bash cd packages/node npm install npm run build ``` -------------------------------- ### Install LiteParse WASM Package Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/browser-usage.md Install the LiteParse WASM package using npm. ```bash npm install @llamaindex/liteparse-wasm ``` -------------------------------- ### Build Python Bindings Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Build the Python bindings using Maturin and install them into the active virtual environment for local development. ```bash cd packages/python maturin develop ``` -------------------------------- ### OCR Request Example Source: https://github.com/run-llama/liteparse/blob/main/OCR_API_SPEC.md Send an image file and language code to the OCR endpoint. ```bash curl -X POST http://localhost:8080/ocr \ -F "file=@document.png" \ -F "language=en" ``` -------------------------------- ### Quick Start: Basic Parsing Source: https://github.com/run-llama/liteparse/blob/main/packages/python/README.md Initialize LiteParse and parse a document. Access the extracted text using the 'text' attribute of the result. ```python from liteparse import LiteParse parser = LiteParse() result = parser.parse("document.pdf") print(result.text) # Access structured data for page in result.pages: print(f"Page {page.page_num}: {len(page.text_items)} text items") ``` -------------------------------- ### Example cURL Request to EasyOCR Service Source: https://github.com/run-llama/liteparse/blob/main/ocr/easyocr/README.md Demonstrates how to send a POST request with an image file and language parameter to the EasyOCR service. ```bash curl -X POST -F "file=@image.png" -F "language=en" http://localhost:8828/ocr ``` -------------------------------- ### Quick Start: Parse a Document Source: https://github.com/run-llama/liteparse/blob/main/packages/node/README.md Initialize the LiteParse parser and parse a PDF document. Access the extracted text and structured page data. ```typescript import { LiteParse } from '@llamaindex/liteparse'; const parser = new LiteParse(); const result = await parser.parse('document.pdf'); console.log(result.text); // Access structured data for (const page of result.pages) { console.log(`Page ${page.pageNum}: ${page.textItems.length} text items`); } ``` -------------------------------- ### GitHub Actions CI for Maturin Musl Wheels Source: https://github.com/run-llama/liteparse/blob/main/musl_build_cargozig.md Example GitHub Actions workflow to build musl wheels for multiple targets using maturin and --zig flag. ```yaml name: Build musl wheels run: | maturin build --release --target ${{ matrix.target }} --zig strategy: matrix: target: [x86_64-unknown-linux-musl, aarch64-unknown-linux-musl] ``` -------------------------------- ### Testing OCR Server with Curl Source: https://github.com/run-llama/liteparse/blob/main/OCR_API_SPEC.md Start your OCR server and then use curl to send a POST request with an image file and language parameter. The output is piped to jq for pretty-printing. ```bash # 1. Start your server python server.py # 2. Test with curl curl -X POST http://localhost:8080/ocr \ -F "file=@test.png" \ -F "language=en" \ | jq . # 3. Expected output: # { # "results": [ # { # "text": "...", # "bbox": [x1, y1, x2, y2], # "confidence": 0.xx # } # ] # } ``` -------------------------------- ### Quick Start: LiteParse in Browser Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/browser-usage.md Initialize and use LiteParse in the browser with WASM. Load the WASM module, create a parser instance, and parse PDF data from a Uint8Array. ```typescript import init, { LiteParse } from "@llamaindex/liteparse-wasm"; // Load the WASM module await init(); const parser = new LiteParse({ ocrEnabled: false, outputFormat: "json", }); // data is a Uint8Array (e.g. from or fetch) const bytes = new Uint8Array(await file.arrayBuffer()); const result = await parser.parse(bytes); console.log(result.text); console.log(result.pages[0]); ``` -------------------------------- ### Example cURL Request to PaddleOCR Service Source: https://github.com/run-llama/liteparse/blob/main/ocr/paddleocr/README.md Demonstrates how to send a POST request with an image file and language parameter to the OCR service. ```bash curl -X POST -F "file=@image.png" -F "language=zh" http://localhost:8829/ocr ``` -------------------------------- ### OCR Success Response Example Source: https://github.com/run-llama/liteparse/blob/main/OCR_API_SPEC.md Example of a successful OCR response containing text, bounding boxes, and confidence scores. ```json { "results": [ { "text": "Hello", "bbox": [10, 20, 60, 40], "confidence": 0.98 }, { "text": "World", "bbox": [70, 20, 130, 40], "confidence": 0.97 } ] } ``` -------------------------------- ### Build Musl Wheels with Maturin and Zig Source: https://github.com/run-llama/liteparse/blob/main/musl_build_cargozig.md Use the --zig flag with maturin to automatically leverage cargo-zigbuild for musl targets. Ensure maturin, cargo-zigbuild, and the musl targets are installed. ```bash maturin build --release --target x86_64-unknown-linux-musl --zig maturin build --release --target aarch64-unknown-linux-musl --zig ``` ```bash pip install maturin cargo install cargo-zigbuild # and the targets: rustup target add x86_64-unknown-linux-musl aarch64-unknown-linux-musl ``` -------------------------------- ### LiteParse Configuration Options Source: https://github.com/run-llama/liteparse/blob/main/wasm-demo-site/index.html Illustrates how to configure LiteParse with specific options, such as setting the maximum number of pages to parse. This example assumes you have a select element with id 'pages-select'. ```javascript const pagesSelect = document.getElementById('pages-select'); const liteparse = new LiteParse({ maxPages: parseInt(pagesSelect.value, 10) || undefined, }); ``` -------------------------------- ### Full Example: Highlight Citations in PDF Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/visual-citations.mdx Parses a PDF, searches for a specific phrase across all pages, and draws yellow highlight boxes on the corresponding page screenshots. Requires @llamaindex/liteparse and sharp. ```typescript import { LiteParse, searchItems } from "@llamaindex/liteparse"; import sharp from "sharp"; const DPI = 150; const SCALE = DPI / 72; async function main() { const parser = new LiteParse({ outputFormat: "json", dpi: DPI }); const result = await parser.parse("manual.pdf"); const screenshots = await parser.screenshot("manual.pdf"); // Search for a phrase, grouped by page const query = "0°C to 70°C"; const hitsByPage = new Map>(); for (const page of result.json?.pages || []) { const matches = searchItems(page.textItems, { phrase: query }); if (matches.length) hitsByPage.set(page.page, matches); } // Draw all highlights per page into a single image for (const [pageNum, rects] of hitsByPage) { const shot = screenshots.find((s) => s.pageNum === pageNum); if (!shot) continue; const composites = await Promise.all( rects.map(async (rect) => { const pixel = { left: Math.round(rect.x * SCALE), top: Math.round(rect.y * SCALE), width: Math.round(rect.width * SCALE), height: Math.round(rect.height * SCALE), }; const overlay = await sharp({ create: { width: pixel.width, height: pixel.height, channels: 4, background: { r: 255, g: 255, b: 0, alpha: 0.3 }, }, }) .png() .toBuffer(); return { input: overlay, left: pixel.left, top: pixel.top }; }) ); const highlighted = await sharp(shot.imageBuffer) .composite(composites) .png() .toBuffer(); await sharp(highlighted).toFile(`citation_page${pageNum}.png`); console.log(`Saved citation_page${pageNum}.png (${rects.length} highlights)`); } } main().catch(console.error); ``` -------------------------------- ### Add LiteParse Agent Skill Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/agent-skill.md Install the LiteParse agent skill using the Vercel skills utility. This command adds the necessary skill file for compatible coding agents. ```bash npx skills add run-llama/llamaparse-agent-skills --skill liteparse ``` -------------------------------- ### Build LiteParse Server Slim Docker Image Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md Build the slim LiteParse server Docker image locally using the provided Dockerfile. ```bash # Build the image docker build -f slim.Dockerfile -t liteparse-server-slim . # Run exposing port 5000 docker run -p 5000:5000 liteparse-server-slim ``` -------------------------------- ### Basic PDF Parsing with LiteParse Source: https://github.com/run-llama/liteparse/blob/main/crates/liteparse/README.md Demonstrates how to initialize LiteParse with default configuration and parse a PDF file, then print the extracted text and page information. Ensure the 'liteparse' crate is added to your Cargo.toml. ```rust use liteparse::{LiteParse, LiteParseConfig}; #[tokio::main] async fn main() -> Result<(), Box> { let parser = LiteParse::new(LiteParseConfig::default()); let result = parser.parse("document.pdf").await?; println!("{}", result.text); for page in &result.pages { println!("Page {}: {} text items", page.page_num, page.text_items.len()); } Ok(()) } ``` -------------------------------- ### Build Musl Wheels with napi-rs using Docker Source: https://github.com/run-llama/liteparse/blob/main/musl_build_cargozig.md Utilize napi-rs's official Docker images which come with pre-configured Zig and musl toolchains for cross-compilation. ```bash docker run --rm -v $(pwd):/build \ ghcr.io/napi-rs/napi-rs/nodejs-rust:lts-alpine \ sh -c "cd /build && npx @napi-rs/cli build --release --target x86_64-unknown-linux-musl" ``` -------------------------------- ### Build Node.js Bindings Individual Steps Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Execute individual build steps for the Node.js bindings: Rust compilation, PDFium copying, and TypeScript compilation. ```bash npm run build:rs ``` ```bash npm run build:pdfium ``` ```bash npm run build:ts ``` -------------------------------- ### Screenshot Response Example Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md This is an example of the newline-delimited JSON (NDJSON) response when requesting screenshots of document pages. Each line represents a single page's screenshot data. ```json { "index": 0, "mimetype": "image/png", "data": "", "page_number": 1, "height": 1056, "width": 816 } ``` -------------------------------- ### Build Liteparse CLI Source: https://github.com/run-llama/liteparse/blob/main/README.md Builds the release version of the Liteparse CLI binary. ```bash cargo build --release -p liteparse ``` -------------------------------- ### OCR Error Response Example Source: https://github.com/run-llama/liteparse/blob/main/OCR_API_SPEC.md Format for error responses from the OCR API. ```json { "error": "Description of the error" } ``` -------------------------------- ### Screenshot Word Document Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/cli-reference.md Generate screenshots from a Word document. Note that this requires LibreOffice to be installed. ```bash lit screenshot report.docx -o ./pages ``` -------------------------------- ### Full LiteParse Configuration Options Source: https://github.com/run-llama/liteparse/blob/main/packages/node/README.md Demonstrates the extensive configuration options available when initializing the LiteParse parser, including OCR settings, page targeting, and output formats. ```typescript const parser = new LiteParse({ ocrEnabled: true, ocrLanguage: 'eng', ocrServerUrl: undefined, tessdataPath: undefined, maxPages: 1000, targetPages: '1-5,10', dpi: 150, outputFormat: 'json', imageMode: 'placeholder', extractLinks: true, preserveVerySmallText: false, password: undefined, quiet: false, numWorkers: 4, }); ``` -------------------------------- ### Build Node.js Bindings Source: https://github.com/run-llama/liteparse/blob/main/README.md Builds the Node.js bindings for Liteparse. Navigate to the node package directory and run the build script. ```bash cd packages/node && npm run build ``` -------------------------------- ### Build WASM Bindings Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Build the WebAssembly bindings for different targets: web, bundler, or Node.js. ```bash cd packages/wasm npm run build ``` ```bash npm run build:bundler ``` ```bash npm run build:nodejs ``` -------------------------------- ### Parse Document with PaddleOCR using LiteParse SDK Source: https://github.com/run-llama/liteparse/blob/main/ocr/paddleocr/README.md Provides an example of integrating PaddleOCR into an application using the LiteParse TypeScript SDK. ```typescript import { LiteParse } from 'liteparse'; const parser = new LiteParse({ ocrServerUrl: 'http://localhost:8829/ocr', ocrLanguage: 'zh', }); const result = await parser.parse('document.pdf'); ``` -------------------------------- ### Build WASM Bindings Source: https://github.com/run-llama/liteparse/blob/main/README.md Builds the WebAssembly bindings for Liteparse. Navigate to the wasm package directory and run the build script. ```bash cd packages/wasm && npm run build ``` -------------------------------- ### Initialize LiteParse for Markdown output in Python Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/markdown.md Configure the LiteParse library for Markdown output in Python. Set `output_format` to 'markdown' and control image handling and link extraction. ```python from liteparse import LiteParse parser = LiteParse( output_format="markdown", # "json" | "text" | "markdown" image_mode="placeholder", # "placeholder" | "off" | "embed" extract_links=True, # render [text](url) link syntax (default: True) ) result = parser.parse("document.pdf") print(result.text) # rendered Markdown ``` -------------------------------- ### Parse All Supported Files Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/cli-reference.md Parse all supported files within the specified input directory and save the output to the output directory. ```bash lit batch-parse ./documents ./output ``` -------------------------------- ### Initialize LiteParse for Markdown output in Rust Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/markdown.md Configure the LiteParse library for Markdown output in Rust. Set `output_format` to `OutputFormat::Markdown` and control image handling and link extraction. ```rust use liteparse::config::{ImageMode, LiteParseConfig, OutputFormat}; use liteparse::LiteParse; let config = LiteParseConfig { output_format: OutputFormat::Markdown, image_mode: ImageMode::Placeholder, extract_links: true, ..Default::default() }; let result = LiteParse::new(config).parse("document.pdf").await?; println!("{}", result.text); // rendered Markdown ``` -------------------------------- ### Build Musl Wheels with napi-rs using CARGO env var Source: https://github.com/run-llama/liteparse/blob/main/musl_build_cargozig.md Configure napi-rs to use cargo-zigbuild by setting the CARGO environment variable. This allows napi-rs to utilize cargo-zigbuild's interface seamlessly. ```bash # Tell napi-rs to use cargo-zigbuild instead of cargo CARGO=cargo-zigbuild npx @napi-rs/cli build \ --platform \ --release \ --target x86_64-unknown-linux-musl ``` -------------------------------- ### Configuring LiteParse Options Source: https://github.com/run-llama/liteparse/blob/main/crates/liteparse/README.md Shows how to customize LiteParse behavior by creating a LiteParseConfig with various options like enabling OCR, setting language, specifying max pages, and choosing output format. All fields can be set individually or by using Default::default() and overriding specific ones. ```rust use liteparse::{LiteParse, LiteParseConfig, OutputFormat}; let config = LiteParseConfig { ocr_enabled: true, // Enable OCR (default: true) ocr_language: "eng".to_string(), // Tesseract language code ocr_server_url: None, // HTTP OCR server URL (optional) tessdata_path: None, // Path to tessdata directory (optional) max_pages: 1000, // Max pages to parse target_pages: Some("1-5,10".into()), // Specific pages (optional) dpi: 150.0, // Rendering DPI output_format: OutputFormat::Json, // Json | Text | Markdown preserve_very_small_text: false, // Keep tiny text password: None, // Password for protected documents quiet: false, // Suppress progress output ..Default::default() }; let parser = LiteParse::new(config); ``` -------------------------------- ### Basic LiteParse Usage Source: https://github.com/run-llama/liteparse/blob/main/wasm-demo-site/index.html Demonstrates the fundamental steps to initialize LiteParse and parse a PDF file. Ensure you have a file input element with id 'file-input' and a button with id 'parse-button'. ```javascript import LiteParse from 'liteparse'; const fileInput = document.getElementById('file-input'); const parseButton = document.getElementById('parse-button'); const output = document.getElementById('output'); parseButton.addEventListener('click', async () => { const file = fileInput.files[0]; if (!file) { alert('Please select a PDF file first!'); return; } const liteparse = new LiteParse(); const result = await liteparse.parse(file); output.innerText = JSON.stringify(result, null, 2); }); ``` -------------------------------- ### Process Documents with lp-process Source: https://github.com/run-llama/liteparse/blob/main/dataset_eval_utils/README.md Generates structured QA ground truth data from PDF and image files using Claude's vision. Output is saved to the specified directory. ```bash lp-process /path/to/documents --output-dir ./ground_truth ``` -------------------------------- ### Screenshot Command Options Source: https://github.com/run-llama/liteparse/blob/main/README.md Use 'lit screenshot' to capture pages from a document as images. Options specify the output directory, target pages, and DPI. ```bash lit screenshot [OPTIONS] Options: -o, --output-dir Output directory [default: ./screenshots] --target-pages Pages to screenshot (e.g., "1,3,5" or "1-5") --dpi Rendering DPI [default: 150] --password Password for encrypted documents -q, --quiet Suppress progress output -h, --help Print help ``` -------------------------------- ### Pull LiteParse Server Docker Image Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md Pull the pre-built slim LiteParse server Docker image from GitHub Container Registry. ```bash docker pull ghcr.io/run-llama/liteparse-server:main ``` -------------------------------- ### Build LiteParse WASM for Web Source: https://github.com/run-llama/liteparse/blob/main/packages/wasm/README.md Build the WebAssembly version of LiteParse for web targets using npm and wasm-pack. The output is placed in the `pkg/` directory. ```sh # from packages/wasm npm run build # web target (default) npm run build:bundler # for webpack/rollup/vite npm run build:nodejs # for node.js ``` -------------------------------- ### Screenshot First 5 Pages at High DPI Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/cli-reference.md Generate screenshots for the first 5 pages of a document at a high DPI (300) and save them to the output directory. ```bash lit screenshot document.pdf --target-pages "1-5" --dpi 300 -o ./pages ``` -------------------------------- ### Export Visual Grid for Debugging Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Generate PNG images that visualize text boxes color-coded by snap type and display anchor lines. Use to visually inspect the grid projection results. ```bash lit parse document.pdf --debug-visualize ``` ```bash lit parse document.pdf --debug-visualize --debug-output ./my-debug ``` -------------------------------- ### Run Server Tests Source: https://github.com/run-llama/liteparse/blob/main/ocr/easyocr/README.md Executes the server tests using uv and pytest. ```bash uv run pytest test_server.py ``` -------------------------------- ### Build Core Rust Library Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Build only the core Rust library without including any bindings. ```bash cargo build -p liteparse ``` -------------------------------- ### Download LiteParse Eval Dataset Source: https://github.com/run-llama/liteparse/blob/main/dataset_eval_utils/README.md Downloads the pre-generated evaluation dataset from Hugging Face using the CLI. Specify a local directory to save the dataset. ```bash hf download run-llama/liteparse-eval-dataset --repo-type dataset --local-dir ./liteparse-eval-dataset ``` -------------------------------- ### Parse a PDF Document Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Initialize LiteParse with default configuration and parse a PDF file. Access the full text and per-page data from the result. ```rust use liteparse::{LiteParse, LiteParseConfig}; #[tokio::main] async fn main() -> Result<(), Box> { let parser = LiteParse::new(LiteParseConfig::default()); let result = parser.parse("document.pdf").await?; // Full document text println!("{}", result.text); // Per-page data for page in &result.pages { println!("Page {}: {} text items", page.page_number, page.text_items.len()); } Ok(()) } ``` -------------------------------- ### Clone LiteParse Repository Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Clone your forked repository to your local machine and navigate into the project directory. ```bash git clone https://github.com/YOUR_USERNAME/liteparse.git cd liteparse ``` -------------------------------- ### Parse Document with EasyOCR Service using LiteParse CLI Source: https://github.com/run-llama/liteparse/blob/main/ocr/easyocr/README.md Illustrates how to use the LiteParse CLI to parse a PDF document with the EasyOCR service. ```bash lit parse document.pdf --ocr-server-url http://localhost:8828/ocr ``` -------------------------------- ### Render a document to Markdown CLI Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/markdown.md Use the CLI to render a PDF document to a Markdown file. Specify the output file with `-o`. ```bash # Render a document to Markdown lit parse document.pdf --format markdown -o output.md # Print Markdown to stdout lit parse document.pdf --format markdown ``` -------------------------------- ### Configuring Liteparse Parser Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Configure the LiteParse parser with options like OCR, DPI, target pages, and password during initialization. ```python parser = LiteParse( ocr_enabled=True, ocr_server_url="http://localhost:8828/ocr", ocr_language="fra", dpi=300, target_pages="1-5", password="secret", ) result = parser.parse("document.pdf") ``` -------------------------------- ### Run LiteParse Server Docker Container Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/server-usage.md Run the pulled LiteParse server Docker image, exposing port 5000. ```bash docker run -p 5000:5000 ghcr.io/run-llama/liteparse-server:main ``` -------------------------------- ### Configure Offline OCR with Tesseract Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/ocr.md Set up LiteParse to use Tesseract OCR in offline or air-gapped environments by pointing to a local directory containing `.traineddata` files. ```bash # Via environment variable export TESSDATA_PREFIX=/path/to/tessdata lit parse document.pdf --ocr-language eng # Or via CLI flag lit parse document.pdf --tessdata-path /path/to/tessdata ``` -------------------------------- ### Integrating a Custom OCR Engine Source: https://github.com/run-llama/liteparse/blob/main/crates/liteparse/README.md Shows how to replace the default OCR engine with a custom implementation by providing an `OcrEngine` trait object. This allows for using alternative OCR solutions. ```rust use liteparse::ocr::OcrEngine; use std::sync::Arc; let parser = LiteParse::new(LiteParseConfig::default()) .with_ocr_engine(Arc::new(my_engine)); ``` -------------------------------- ### Initialize and Use LiteParse WASM Source: https://github.com/run-llama/liteparse/blob/main/wasm-demo-site/index.html Initializes the WASM module and creates a parser instance to process PDF bytes. Shows how to access the rendered text and text items with bounding boxes. ```javascript import init, { LiteParse } from "@llamaindex/liteparse-wasm"; // Initialize the WASM module await init(); // Create a parser instance (markdown output) const parser = new LiteParse({ outputFormat: "markdown" }); // Parse PDF bytes (Uint8Array from File, fetch, etc.) const bytes = new Uint8Array(await file.arrayBuffer()); const result = await parser.parse(bytes); console.log(result.text); // rendered markdown console.log(result.pages[0].textItems); // items with bounding boxes ``` -------------------------------- ### Custom LiteParse Configuration Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Create a custom LiteParseConfig by overriding specific fields like OCR settings, DPI, target pages, output format, or password. Uses Default::default() for unspecified fields. ```rust use liteparse::{LiteParse, LiteParseConfig, OutputFormat}; let config = LiteParseConfig { ocr_enabled: true, ocr_language: "fra".to_string(), dpi: 300.0, target_pages: Some("1-10".to_string()), output_format: OutputFormat::Json, password: Some("secret".to_string()), ..Default::default() }; let parser = LiteParse::new(config); ``` -------------------------------- ### Initialize LiteParse for Markdown output in TypeScript Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/markdown.md Configure the LiteParse library for Markdown output. Set `outputFormat` to 'markdown' and control image handling and link extraction. ```typescript import { LiteParse } from "@llamaindex/liteparse"; const parser = new LiteParse({ outputFormat: "markdown", // "json" | "text" | "markdown" imageMode: "placeholder", // "placeholder" | "off" | "embed" (default: "placeholder") extractLinks: true, // render [text](url) link syntax (default: true) }); const result = await parser.parse("document.pdf"); console.log(result.text); // rendered Markdown ``` -------------------------------- ### Parse Document with PaddleOCR using LiteParse CLI Source: https://github.com/run-llama/liteparse/blob/main/ocr/paddleocr/README.md Shows how to use the LiteParse command-line interface to parse a document using the PaddleOCR service. ```bash # Parse with PaddleOCR lit parse document.pdf --ocr-server-url http://localhost:8829/ocr # With specific language lit parse document.pdf --ocr-server-url http://localhost:8829/ocr --ocr-language zh ``` -------------------------------- ### Build Python Bindings Source: https://github.com/run-llama/liteparse/blob/main/README.md Builds the Python bindings for Liteparse using Maturin. Navigate to the python package directory and run the develop command. ```bash cd packages/python && maturin develop --release ``` -------------------------------- ### Basic Tesseract OCR Usage Source: https://github.com/run-llama/liteparse/blob/main/README.md Enable OCR by default for PDF documents. Use --ocr-language to specify the language or --no-ocr to disable OCR. ```bash lit parse document.pdf # OCR enabled by default lit parse document.pdf --ocr-language fra # Specify language lit parse document.pdf --no-ocr # Disable OCR ``` -------------------------------- ### Add Liteparse to Cargo.toml Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Add the liteparse and tokio dependencies to your project's Cargo.toml file. ```toml [ dependencies] liteparse = "2" tokio = { version = "1", features = ["rt-multi-thread", "macros"] } ``` -------------------------------- ### Generate Page Images from Document Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/cli-reference.md Generate page images (screenshots) from a given document file. Specify the output directory for the images. ```bash lit screenshot [options] ``` -------------------------------- ### Advanced LiteParse Configuration Source: https://github.com/run-llama/liteparse/blob/main/packages/python/README.md Configure various parsing options such as OCR, language, page limits, DPI, and output formats via the constructor. This allows fine-tuning the parsing process for specific needs. ```python parser = LiteParse( ocr_enabled=True, # Enable OCR (default: True) ocr_language="eng", # Tesseract language code ocr_server_url=None, # HTTP OCR server URL (optional) tessdata_path=None, # Path to tessdata directory (optional) max_pages=1000, # Max pages to parse target_pages="1-5,10", # Specific pages (optional) dpi=150, # Rendering DPI output_format="json", # "json" | "text" | "markdown" image_mode="placeholder", # Markdown image handling: "placeholder" | "off" | "embed" extract_links=True, # Render [text](url) links in markdown output preserve_very_small_text=False, # Keep tiny text password=None, # Password for protected documents quiet=False, # Suppress progress output num_workers=4, # Concurrent OCR workers ) ``` -------------------------------- ### Parsing PDF from Bytes Source: https://github.com/run-llama/liteparse/blob/main/crates/liteparse/README.md Demonstrates parsing a PDF document directly from its byte representation in memory, rather than from a file path. This is useful when the PDF content is already loaded into a byte vector. ```rust use liteparse::types::PdfInput; let pdf_bytes: Vec = std::fs::read("document.pdf")?; let result = parser.parse_input(PdfInput::Bytes(pdf_bytes)).await?; println!("{}", result.text); ``` -------------------------------- ### LiteParse Configuration Options (TypeScript) Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Pass configuration options to the LiteParse constructor to customize OCR, output format, page selection, and more. ```typescript const parser = new LiteParse({ ocrEnabled: true, ocrServerUrl: "http://localhost:8828/ocr", ocrLanguage: "fra", dpi: 300, outputFormat: "json", targetPages: "1-10", password: "secret", }); ``` -------------------------------- ### Parse URL content using LiteParse library in Node.js Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/parsing-urls.md Fetch content from a URL using `fetch`, convert it to a buffer, and then parse it with the `LiteParse` library. Ensure `Buffer` is available in your environment. ```typescript import { LiteParse } from "@llamaindex/liteparse"; const response = await fetch("https://example.com/report.pdf"); const buffer = Buffer.from(await response.arrayBuffer()); const parser = new LiteParse({ ocrEnabled: false }); const result = await parser.parse(buffer); console.log(result.text); ``` -------------------------------- ### Markdown Output Configuration Source: https://github.com/run-llama/liteparse/blob/main/packages/python/README.md Configure LiteParse to render documents as Markdown, including reconstruction of headings, tables, and lists. The 'output_format' and 'image_mode' parameters control the rendering. ```python parser = LiteParse( output_format="markdown", # "json" | "text" | "markdown" image_mode="placeholder", # "placeholder" | "off" | "embed" extract_links=True, # render [text](url) link syntax (default: True) ) result = parser.parse("document.pdf") print(result.text) # rendered Markdown ``` -------------------------------- ### Batch Parse Command Options Source: https://github.com/run-llama/liteparse/blob/main/README.md Use 'lit batch-parse' to process multiple files in a directory. Options include format, OCR settings, recursion, and file extensions. ```bash lit batch-parse [OPTIONS] Options: --format Output format: json|text|markdown [default: text] --no-ocr Disable OCR --ocr-language OCR language [default: eng] --ocr-server-url HTTP OCR server URL --tessdata-path Path to tessdata directory --max-pages Max pages per file [default: 1000] --dpi Rendering DPI [default: 150] --recursive Recursively search input directory --extension Only process files with this extension (e.g., ".pdf") --password Password for encrypted documents --num-workers Concurrent OCR workers -q, --quiet Suppress progress output -h, --help Print help ``` -------------------------------- ### Generate screenshots from a PDF Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/getting_started.mdx Create image files (screenshots) for each page of a PDF document, saving them to a specified output directory. ```bash lit screenshot document.pdf -o ./screenshots ``` -------------------------------- ### Parse Document with EasyOCR Service using LiteParse SDK Source: https://github.com/run-llama/liteparse/blob/main/ocr/easyocr/README.md Demonstrates how to integrate the EasyOCR service with LiteParse in TypeScript code. ```typescript import { LiteParse } from 'liteparse'; const parser = new LiteParse({ ocrServerUrl: 'http://localhost:8828/ocr', ocrLanguage: 'en', }); const result = await parser.parse('document.pdf'); ``` -------------------------------- ### Run QA Evaluation with lp-evaluate Source: https://github.com/run-llama/liteparse/blob/main/dataset_eval_utils/README.md Evaluates parser text extraction quality by comparing LLM-generated answers against ground truth. Results are saved to the specified output path. ```bash lp-evaluate \ --data-dir ./documents \ --ground-truth-dir ./ground_truth \ --parse-provider liteparse \ --output ./results/run1 ``` -------------------------------- ### Use External OCR Server Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/cli-reference.md Parses a PDF document using an external OCR server running at the specified URL. ```bash lit parse report.pdf --ocr-server-url http://localhost:8828/ocr ``` -------------------------------- ### Generate Page Screenshots Source: https://github.com/run-llama/liteparse/blob/main/docs/src/content/docs/liteparse/guides/library-usage.mdx Generate PNG image bytes for each page of a document using the screenshot method. Optionally specify which pages to capture. ```rust let parser = LiteParse::new(LiteParseConfig::default()); let screenshots = parser.screenshot("document.pdf", None).await?; for shot in &screenshots { println!("Page {}: {}x{}", shot.page_num, shot.width, shot.height); // shot.image_bytes contains the raw PNG data } // Screenshot specific pages let shots = parser.screenshot("document.pdf", Some(vec![1, 2, 3])).await?; ``` -------------------------------- ### Enable Debug Logging for Grid Projection Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Trace every decision made by the grid projection algorithm. Use to diagnose issues with block detection, anchor extraction, snap assignment, rendering, and text classification. ```bash lit parse document.pdf --debug ``` ```bash lit parse document.pdf --debug --debug-page 3 ``` ```bash lit parse document.pdf --debug --debug-text-filter "Total" "Revenue" ``` ```bash lit parse document.pdf --debug --debug-region "0,100,300,200" ``` ```bash lit parse document.pdf --debug --debug-output ./debug-output ``` -------------------------------- ### Format Rust Code Source: https://github.com/run-llama/liteparse/blob/main/CONTRIBUTING.md Format the Rust code according to the project's style guidelines. ```bash cargo fmt ```