### Install Go Dependencies Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Download all necessary Go modules for the backend. Ensure you have Go 1.20 or later installed. ```bash go mod download ``` -------------------------------- ### Install Frontend Dependencies Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Install all Node.js dependencies required for the frontend application. This should be run from the 'web-app' directory. ```bash npm install ``` -------------------------------- ### Start Frontend Development Server Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Launch the frontend development server. The application will be accessible at http://localhost:8080. ```bash npm run dev ``` -------------------------------- ### Install Playwright Browsers Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Installs the necessary Playwright browsers, such as Chromium, for running E2E tests. ```bash npx playwright install chromium ``` -------------------------------- ### Clone paperless-gpt Repository Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Clone the project repository to your local machine. Navigate into the cloned directory to proceed with setup. ```bash git clone https://github.com/icereed/paperless-gpt.git cd paperless-gpt ``` -------------------------------- ### GET /api/version Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves build version metadata. This information is injected at compile time using Go ldflags. ```bash curl -s http://localhost:8080/api/version | jq . # { # "version": "v0.12.1", # "commit": "a3f92c1", # "buildDate": "2024-01-15T08:00:00Z" # } ``` -------------------------------- ### Custom Title Prompt Template Example Source: https://context7.com/icereed/paperless-gpt/llms.txt An example of a custom title prompt template using Go's text/template system and Sprig functions. This template defines rules for generating a concise title in a specified language. ```go // Template file: prompts/title_prompt.tmpl // {{.Language}} {{.Content}} {{.Title}} are available variables // Example custom title template using sprig's trim function: tmplContent := `You are a document management assistant. Generate a concise title in {{.Language}} for this document. Original filename: {{.Title | trimAll "\""}} Document content: {{.Content}} Rules: - Maximum 128 characters - No file extensions - Use title case Respond with ONLY the title.` // Save via API: // POST /api/prompts {"filename": "title_prompt.tmpl", "content": ""} // Correspondent template variables: .Language, .AvailableCorrespondents, .BlackList, .Title, .Content // Tag template variables: .Language, .AvailableTags, .OriginalTags, .Title, .Content, .CreateNewTags // Custom field template variables: .Language, .DocumentType, .CustomFieldsXML, .Title, .CreatedDate, .Content // OCR template variables: .Language // Ad-hoc analysis variables: .Documents ([]Document), .Language ``` -------------------------------- ### Conventional Commits Examples Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Examples of commit messages following the Conventional Commits specification. Use the imperative mood in the subject line. ```git feat: add support for custom server-side prompts ``` ```git fix: resolve API pagination issue for tags ``` -------------------------------- ### GET /api/version Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns build version metadata injected at compile time via Go ldflags. ```APIDOC ## GET /api/version ### Description Fetches the build version metadata of the application, which is embedded during the compilation process using Go ldflags. ### Method GET ### Endpoint /api/version ### Response #### Success Response (200) - **version** (string) - The application version string. - **commit** (string) - The Git commit hash associated with the build. - **buildDate** (string) - The date and time when the application was built. ### Response Example ```json { "version": "v0.12.1", "commit": "a3f92c1", "buildDate": "2024-01-15T08:00:00Z" } ``` ``` -------------------------------- ### Vanilla Paperless-ngx OCR Output Source: https://github.com/icereed/paperless-gpt/blob/main/README.md Example output from the standard OCR processing in Paperless-ngx. ```text La Grande Recre Gentre Gommercial 1'Esplanade 1349 LOLNAIN LA NEWWE TA BERBOGAAL Tel =. 010 45,96 12 Ticket 1440112 03/11/2006 a 13597: 4007176614518. DINOS. TYRAMNESA TOTAET.T.LES ReslE par Lask-Euron Rencu en Cash Euro V.14.6 -Hotgese = VALERTE TICKET A-GONGERVER PORR TONT. EEHANGE HERET ET A BIENTOT ``` -------------------------------- ### Get Settings Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns the current server-side UI settings, including custom fields configuration and a list of available custom fields from cache. ```APIDOC ## GET /api/settings ### Description Returns the current server-side UI settings (custom fields configuration) along with the full list of available custom fields from cache. ### Method GET ### Endpoint /api/settings ### Response #### Success Response (200) - **settings** (object) - Contains UI settings like `custom_fields_enable`, `custom_fields_selected_ids`, and `custom_fields_write_mode`. - **custom_fields** (array of objects) - A list of available custom fields. #### Response Example ```json { "settings": { "custom_fields_enable": true, "custom_fields_selected_ids": [1, 2], "custom_fields_write_mode": "append" }, "custom_fields": [ {"id": 1, "name": "Invoice Amount", "data_type": "monetary"}, {"id": 2, "name": "Due Date", "data_type": "date"} ] } ``` ``` -------------------------------- ### Integrate React ESLint Plugin Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/README.md Install eslint-plugin-react and update the ESLint configuration to include its recommended rules and jsx-runtime rules. Set the React version in the settings. ```javascript // eslint.config.js import react from 'eslint-plugin-react' export default tseslint.config({ // Set the react version settings: { react: { version: '18.3' } }, plugins: { // Add the react plugin react, }, rules: { // other rules... // Enable its recommended rules ...react.configs.recommended.rules, ...react.configs['jsx-runtime'].rules, }, }) ``` -------------------------------- ### GET /api/experimental/ocr Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns whether an OCR provider is currently configured and active. ```APIDOC ## GET /api/experimental/ocr ### Description Checks if an Optical Character Recognition (OCR) provider is configured and currently active within the system. ### Method GET ### Endpoint /api/experimental/ocr ### Response #### Success Response (200) - **enabled** (boolean) - Indicates whether the OCR provider is enabled. ### Response Example ```json { "enabled": true } ``` ``` -------------------------------- ### GET /api/experimental/ocr Source: https://context7.com/icereed/paperless-gpt/llms.txt Checks if an OCR provider is currently configured and active. Returns a boolean indicating the OCR status. ```bash curl -s http://localhost:8080/api/experimental/ocr | jq . # {"enabled": true} ``` -------------------------------- ### Get All Prompt Templates Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns all prompt template files (`.tmpl`) from the `prompts/` directory as a map of filename to template content. Templates support Go `text/template` syntax and Sprig functions. ```bash curl -s http://localhost:8080/api/prompts | jq . ``` -------------------------------- ### Get Prompts Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns all prompt template files (`.tmpl`) from the `prompts/` directory as a map of `filename -> template content`. Templates use Go `text/template` syntax with Sprig functions. ```APIDOC ## GET /api/prompts ### Description Returns all prompt template files (`.tmpl`) from the `prompts/` directory as a map of `filename -> template content`. Templates use Go `text/template` syntax with Sprig functions. ### Method GET ### Endpoint /api/prompts ### Response #### Success Response (200) - (object) - A map where keys are template filenames (string) and values are their content (string). #### Response Example ```json { "title_prompt.tmpl": "You are an AI assistant specializing in document management...\nGenerate a concise title for the following {{.Language}} document:\n\nContent: {{.Content}}", "tag_prompt.tmpl": "Available tags: {{.AvailableTags}}\n...", "ocr_prompt.tmpl": "Please accurately transcribe all text from this image...", "correspondent_prompt.tmpl": "...", "created_date_prompt.tmpl": "...", "custom_field_prompt.tmpl": "...", "adhoc-analysis_prompt.tmpl": "..." } ``` ``` -------------------------------- ### OCR Provider Configuration with ocr.NewProvider Source: https://context7.com/icereed/paperless-gpt/llms.txt Instantiates configured OCR backends using the `ocr.Config` struct. The `Provider` interface's `ProcessImage` method is used for OCR operations. Examples include LLM-based OCR, Google Document AI, Azure Document Intelligence, and Docling. ```go import "paperless-gpt/ocr" // LLM-based OCR with Ollama cfg := ocr.Config{ Provider: "llm", VisionLLMProvider: "ollama", VisionLLMModel: "minicpm-v", VisionLLMPrompt: "Please accurately transcribe all text from this image in English.", OllamaContextLength: 8192, } provider, err := ocr.NewProvider(cfg) if err != nil { log.Fatalf("failed to init OCR: %v", err) } // Process a page image imageBytes, _ := os.ReadFile("page001.jpg") result, err := provider.ProcessImage(context.Background(), imageBytes, 1) if err != nil { log.Printf("OCR error: %v", err) } fmt.Println(result.Text) // Extracted text fmt.Println(result.OcrLimitHit) // true if token limit was hit // result.HOCRPage is populated for Google Document AI (for searchable PDF generation) // Google Document AI (supports hOCR for searchable PDFs) docAICfg := ocr.Config{ Provider: "google_docai", GoogleProjectID: "my-gcp-project", GoogleLocation: "us", GoogleProcessorID: "abc123def456", EnableHOCR: true, } docAIProvider, _ := ocr.NewProvider(docAICfg) // Azure Document Intelligence azureCfg := ocr.Config{ Provider: "azure", AzureEndpoint: "https://my-resource.cognitiveservices.azure.com/", AzureAPIKey: "your-azure-key", AzureModelID: "prebuilt-read", AzureTimeout: 120, AzureOutputContentFormat: "text", } azureProvider, _ := ocr.NewProvider(azureCfg) // Docling (self-hosted) doclingCfg := ocr.Config{ Provider: "docling", DoclingURL: "http://localhost:5001", DoclingOCRPipeline: "standard", DoclingOCREngine: "easyocr", DoclingImageExportMode: "embedded", } doclingProvider, _ := ocr.NewProvider(doclingCfg) ``` -------------------------------- ### Get Server Settings and Custom Fields Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves current server-side UI settings, including custom field configurations and the full list of available custom fields from the cache. ```bash curl -s http://localhost:8080/api/settings | jq . ``` -------------------------------- ### Start Background Tasks for Paperless-NGX Source: https://context7.com/icereed/paperless-gpt/llms.txt Initiates a background goroutine to poll Paperless-NGX for documents needing auto-tagging or auto-OCR. It employs exponential backoff for error handling. ```go ctx, cancel := context.WithCancel(context.Background()) defer cancel() // Start background processing - runs in a goroutine // Processes documents tagged "paperless-gpt-auto" → generates suggestions → applies them // Processes documents tagged "paperless-gpt-ocr-auto" → runs OCR → updates content StartBackgroundTasks(ctx, app) // Control behavior via environment variables: // AUTO_GENERATE_TITLE=true → generate title suggestions // AUTO_GENERATE_TAGS=true → generate tag suggestions // AUTO_GENERATE_CORRESPONDENTS=true // AUTO_GENERATE_CREATED_DATE=true // AUTO_GENERATE_DOCUMENT_TYPE=true // Documents with the OCR complete tag are skipped to avoid reprocessing ``` -------------------------------- ### LLM-Powered OCR Output (OpenAI gpt-4o) Source: https://github.com/icereed/paperless-gpt/blob/main/README.md Example output from LLM-powered OCR using OpenAI's gpt-4o model, demonstrating improved accuracy. ```text La Grande Récré Centre Commercial l'Esplanade 1348 LOUVAIN LA NEUVE TVA 860826401 Tel : 010 45 95 12 Ticket 14421 le 03/11/2006 à 15:27:18 4007176614518 DINOS TYRANNOSA 14.90 TOTAL T.T.C. 14.90 Réglé par Cash Euro 50.00 Rendu en Cash Euro 35.10 V.14.6 Hôtesse : VALERIE TICKET A CONSERVER POUR TOUT ECHANGE MERCI ET A BIENTOT ``` -------------------------------- ### GET /api/paperless-url Source: https://context7.com/icereed/paperless-gpt/llms.txt Fetches the public-facing URL of the connected paperless-ngx instance. This is used by the frontend to generate direct links to documents. ```bash curl -s http://localhost:8080/api/paperless-url | jq . # {"url": "https://paperless.mydomain.com"} ``` -------------------------------- ### Build and Run Backend with Integrated Frontend Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Build the frontend application, copy its output to the backend's expected directory, then build and run the backend server. This is useful for testing the integrated application. ```bash cd web-app && npm install && npm run build && cp -r dist .. go build ./paperless-gpt ``` -------------------------------- ### Build and Run Backend Server Source: https://github.com/icereed/paperless-gpt/blob/main/CONTRIBUTING.md Compile the Go backend and run the server. This command assumes you have a dist/index.html file, which is typically generated by the frontend build process. ```bash mkdir dist touch dist/index.html go build ./paperless-gpt ``` -------------------------------- ### Get OCR Pages via GET /api/documents/:id/ocr_pages Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves per-page OCR results stored in the local SQLite database for a document. Useful for inspecting or editing individual page OCR text. ```bash curl -s http://localhost:8080/api/documents/42/ocr_pages | jq . # { # "pages": [ # { # "text": "La Grande Récré\nCentre Commercial l'Esplanade...", # "ocrLimitHit": false, # "generationInfo": { # "model": "gpt-4o", # "prompt_tokens": 1200, # "completion_tokens": 340 # } # }, # { # "text": "FedEx Invoice Number: 1-996-84199...", # "ocrLimitHit": false, # "generationInfo": null # } # ] # } ``` -------------------------------- ### Get All OCR Jobs via GET /api/jobs/ocr Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves a list of all OCR jobs, including pending, in-progress, completed, failed, and cancelled states. Jobs are sorted by creation time in descending order. ```bash curl -s http://localhost:8080/api/jobs/ocr | jq . # [ # { # "job_id": "550e8400-...", # "status": "completed", # "created_at": "2024-01-15T10:00:00Z", # "updated_at": "2024-01-15T10:02:00Z", # "pages_done": 5, # "result": "La Grande Récré\nCentre Commercial..." # }, # { # "job_id": "661f9511-...", # "status": "pending", # "created_at": "2024-01-15T10:03:00Z", # "updated_at": "2024-01-15T10:03:00Z", # "pages_done": 0 # } # ] ``` -------------------------------- ### Run Unit Tests (Go) Source: https://github.com/icereed/paperless-gpt/blob/main/cline_docs/techContext.md Execute Go unit tests for the backend. Ensure all tests pass before committing changes. ```bash go test . ``` -------------------------------- ### Build and Run E2E Tests (Docker & npm) Source: https://github.com/icereed/paperless-gpt/blob/main/cline_docs/techContext.md Build a Docker image for end-to-end testing and run the frontend E2E tests using npm. This is a required step before commits. ```bash docker build . -t icereed/paperless-gpt:e2e cd web-app && npm run test:e2e ``` -------------------------------- ### Docker Compose for Paperless-GPT and Paperless-ngx Source: https://github.com/icereed/paperless-gpt/blob/main/README.md This configuration sets up paperless-ngx and paperless-gpt services. Ensure you replace placeholder API keys and URLs with your actual credentials and domain names. Configure your preferred LLM and OCR provider by uncommenting the relevant sections and providing the necessary environment variables. ```yaml services: paperless-ngx: image: ghcr.io/paperless-ngx/paperless-ngx:latest # ... (your existing paperless-ngx config) paperless-gpt: # Use one of these image sources: image: icereed/paperless-gpt:latest # Docker Hub # image: ghcr.io/icereed/paperless-gpt:latest # GitHub Container Registry environment: PAPERLESS_BASE_URL: "http://paperless-ngx:8000" PAPERLESS_API_TOKEN: "your_paperless_api_token" PAPERLESS_PUBLIC_URL: "http://paperless.mydomain.com" # Optional MANUAL_TAG: "paperless-gpt" # Optional, default: paperless-gpt AUTO_TAG: "paperless-gpt-auto" # Optional, default: paperless-gpt-auto # LLM Configuration - Choose one: # Option 1: Standard OpenAI LLM_PROVIDER: "openai" LLM_MODEL: "gpt-4o" OPENAI_API_KEY: "your_openai_api_key" # Option 2: Mistral # LLM_PROVIDER: "mistral" # LLM_MODEL: "mistral-large-latest" # MISTRAL_API_KEY: "your_mistral_api_key" # Option 3: Azure OpenAI # LLM_PROVIDER: "openai" # LLM_MODEL: "your-deployment-name" # OPENAI_API_KEY: "your_azure_api_key" # OPENAI_API_TYPE: "azure" # OPENAI_BASE_URL: "https://your-resource.openai.azure.com" # Option 4: Ollama (Local) # LLM_PROVIDER: "ollama" # LLM_MODEL: "qwen3:8b" # OLLAMA_HOST: "http://host.docker.internal:11434" # OLLAMA_CONTEXT_LENGTH: "8192" # Sets Ollama NumCtx (context window) # TOKEN_LIMIT: 1000 # Recommended for smaller models # Option 5: Anthropic/Claude # LLM_PROVIDER: "anthropic" # LLM_MODEL: "claude-sonnet-4-5" # ANTHROPIC_API_KEY: "your_anthropic_api_key" # Optional LLM Settings # LLM_LANGUAGE: "English" # Optional, default: English # OCR Configuration - Choose one: # Option 1: LLM-based OCR OCR_PROVIDER: "llm" # Default OCR provider VISION_LLM_PROVIDER: "ollama" # openai, ollama, mistral, or anthropic VISION_LLM_MODEL: "minicpm-v" # minicpm-v (ollama) or gpt-4o (openai) or claude-sonnet-4-5 (anthropic/claude) OLLAMA_HOST: "http://host.docker.internal:11434" # If using Ollama # OCR Processing Mode OCR_PROCESS_MODE: "image" # Optional, default: image, other options: pdf, whole_pdf PDF_SKIP_EXISTING_OCR: "false" # Optional, skip OCR for PDFs with existing OCR # Option 2: Google Document AI # OCR_PROVIDER: 'google_docai' # Use Google Document AI # GOOGLE_PROJECT_ID: 'your-project' # Your GCP project ID # GOOGLE_LOCATION: 'us' # Document AI region # GOOGLE_PROCESSOR_ID: 'processor-id' # Your processor ID # GOOGLE_APPLICATION_CREDENTIALS: '/app/credentials.json' # Path to service account key # Option 3: Azure Document Intelligence # OCR_PROVIDER: 'azure' # Use Azure Document Intelligence # AZURE_DOCAI_ENDPOINT: 'your-endpoint' # Your Azure endpoint URL # AZURE_DOCAI_KEY: 'your-key' # Your Azure API key # AZURE_DOCAI_MODEL_ID: 'prebuilt-read' # Optional, defaults to prebuilt-read # AZURE_DOCAI_TIMEOUT_SECONDS: '120' # Optional, defaults to 120 seconds # AZURE_DOCAI_OUTPUT_CONTENT_FORMAT: 'text' # Optional, defaults to 'text', other valid option is 'markdown' # 'markdown' requires the 'prebuilt-layout' model # Enhanced OCR Features CREATE_LOCAL_HOCR: "false" # Optional, save hOCR files locally LOCAL_HOCR_PATH: "/app/hocr" # Optional, path for hOCR files CREATE_LOCAL_PDF: "false" # Optional, save enhanced PDFs locally LOCAL_PDF_PATH: "/app/pdf" # Optional, path for PDF files PDF_UPLOAD: "false" # Optional, upload enhanced PDFs to paperless-ngx PDF_REPLACE: "false" # Optional and DANGEROUS, delete original after upload PDF_COPY_METADATA: "true" # Optional, copy metadata from original document PDF_OCR_TAGGING: "true" # Optional, add tag to processed documents PDF_OCR_COMPLETE_TAG: "paperless-gpt-ocr-complete" # Optional, tag name # Option 4: Docling Server # OCR_PROVIDER: 'docling' # Use a Docling server # DOCLING_URL: 'http://your-docling-server:port' # URL of your Docling instance # DOCLING_IMAGE_EXPORT_MODE: "placeholder" # Optional, defaults to "embedded" # DOCLING_OCR_PIPELINE: "standard" # Optional, defaults to "vlm" # DOCLING_OCR_ENGINE: "easyocr" # Optional, defaults to "easyocr" (only used when `DOCLING_OCR_PIPELINE is set to 'standard') AUTO_OCR_TAG: "paperless-gpt-ocr-auto" # Optional, default: paperless-gpt-ocr-auto OCR_LIMIT_PAGES: "5" # Optional, default: 5. Set to 0 for no limit. LOG_LEVEL: "info" # Optional: debug, warn, error volumes: - ./prompts:/app/prompts # Mount the prompts directory # For Google Document AI: ``` -------------------------------- ### Build Paperless-GPT Docker Image Source: https://github.com/icereed/paperless-gpt/blob/main/README.md Build the Docker image for Paperless-GPT using the provided Dockerfile. ```bash docker build -t paperless-gpt . ``` -------------------------------- ### Run Paperless-GPT Docker Container Source: https://github.com/icereed/paperless-gpt/blob/main/README.md Run the Paperless-GPT Docker container with essential environment variables and volume mounts. Ensure to replace placeholder values with your actual credentials and URLs. ```bash docker run -d \ -e PAPERLESS_BASE_URL='http://your_paperless_ngx_url' \ -e PAPERLESS_API_TOKEN='your_paperless_api_token' \ -e LLM_PROVIDER='openai' \ -e LLM_MODEL='gpt-4o' \ -e OPENAI_API_KEY='your_openai_api_key' \ -e LLM_LANGUAGE='English' \ -e VISION_LLM_PROVIDER='ollama' \ -e VISION_LLM_MODEL='minicpm-v' \ -e LOG_LEVEL='info' \ -v $(pwd)/prompts:/app/prompts \ -p 8080:8080 \ paperless-gpt ``` -------------------------------- ### Get Single Document by ID Source: https://context7.com/icereed/paperless-gpt/llms.txt Fetches a single document by its paperless-ngx ID, including all associated metadata. ```APIDOC ## GET /api/documents/:id ### Description Fetches a single document by its paperless-ngx ID with full metadata including tag names, correspondent name, document type name, and resolved custom field names. ### Method GET ### Endpoint /api/documents/:id ### Parameters #### Path Parameters - **id** (integer) - Required - The ID of the document to retrieve. ### Response #### Success Response (200) - **document** (object) - A Document object with full metadata. ### Response Example ```json { "id": 42, "title": "FedEx Invoice Sep 2024", "content": "FedEx Invoice Number: 1-996-84199...", "tags": ["invoice", "fedex"], "correspondent": "FedEx", "created_date": "2024-09-01", "original_file_name": "fedex_invoice.pdf", "document_type_name": "Invoice", "custom_fields": [ {"field": 3, "value": "USD 11.00", "name": "Amount"} ] } ``` ``` -------------------------------- ### Full Document OCR Pipeline with Options Source: https://context7.com/icereed/paperless-gpt/llms.txt Orchestrates the OCR process for a document, including downloading, page-by-page OCR, and optional PDF generation/upload. Configure behavior using OCROptions. ```go // OCROptions controls processing behavior options := OCROptions{ UploadPDF: true, // Upload OCR-enhanced PDF to paperless-ngx ReplaceOriginal: false, // Delete original after upload (DANGEROUS) CopyMetadata: true, // Copy tags/correspondent/date to new document LimitPages: 5, // 0 = no limit ProcessMode: "image", // "image", "pdf", or "whole_pdf" } // Process document 42 through full OCR pipeline processedDoc, err := app.ProcessDocumentOCR(ctx, 42, options, "job-uuid-123") if err != nil { log.Printf("OCR failed: %v", err) return } fmt.Println(processedDoc.Text) // Combined OCR text from all pages fmt.Println(processedDoc.HOCR) // hOCR HTML (if Google DocAI + EnableHOCR) fmt.Println(len(processedDoc.PDFData)) // Bytes of generated searchable PDF fmt.Println(processedDoc.ReplacedOriginal) // true if original was deleted and replaced // The function also saves per-page results to SQLite via SaveSingleOcrPageResult // for later retrieval via GET /api/documents/:id/ocr_pages ``` -------------------------------- ### GET /api/filter-tag Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns the name of the manual processing tag. Used by the frontend to know which tag to display documents for. ```APIDOC ## GET /api/filter-tag ### Description Returns the name of the manual processing tag. This tag is used by the frontend to filter and display documents. ### Method GET ### Endpoint /api/filter-tag ### Response #### Success Response (200) - **tag** (string) - The name of the filter tag. ### Response Example ```json { "tag": "paperless-gpt" } ``` ``` -------------------------------- ### Paperless-GPT Environment Variables Configuration Source: https://context7.com/icereed/paperless-gpt/llms.txt Reference for essential environment variables used to configure Paperless-GPT, including Paperless-NGX connection details, LLM and OCR provider settings, and behavior flags. ```bash # Minimal required configuration PAPERLESS_BASE_URL=http://paperless-ngx:8000 PAPERLESS_API_TOKEN=your-token LLM_PROVIDER=openai # openai | ollama | googleai | mistral | anthropic LLM_MODEL=gpt-4o # LLM provider credentials (set the one matching LLM_PROVIDER) OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... MISTRAL_API_KEY=... GOOGLEAI_API_KEY=... OLLAMA_HOST=http://host.docker.internal:11434 # Azure OpenAI (when LLM_PROVIDER=openai) OPENAI_API_TYPE=azure OPENAI_BASE_URL=https://my-resource.openai.azure.com # OCR configuration OCR_PROVIDER=llm # llm | google_docai | azure | docling | mistral_ocr OCR_PROCESS_MODE=image # image | pdf | whole_pdf VISION_LLM_PROVIDER=openai # for OCR_PROVIDER=llm VISION_LLM_MODEL=gpt-4o OCR_LIMIT_PAGES=5 # 0 = unlimited # Enhanced OCR (PDF generation, Google DocAI only) CREATE_LOCAL_HOCR=false CREATE_LOCAL_PDF=false LOCAL_HOCR_PATH=/app/hocr LOCAL_PDF_PATH=/app/pdf PDF_UPLOAD=false PDF_REPLACE=false # DANGEROUS: deletes original after upload PDF_COPY_METADATA=true PDF_OCR_TAGGING=true PDF_OCR_COMPLETE_TAG=paperless-gpt-ocr-complete # Behavior flags LLM_LANGUAGE=English CREATE_NEW_TAGS=false # Allow LLM to create tags not in paperless-ngx CORRESPONDENT_BLACK_LIST=John Doe,Jane Smith TOKEN_LIMIT=0 # Truncate content to N tokens (0=disabled) OLLAMA_CONTEXT_LENGTH=8192 # Ollama NumCtx window size # Rate limiting LLM_REQUESTS_PER_MINUTE=120 LLM_MAX_RETRIES=3 LLM_BACKOFF_MAX_WAIT=30s VISION_LLM_REQUESTS_PER_MINUTE=120 ``` -------------------------------- ### Configure Test Environment with OCR Options Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Sets up the complete container stack for testing. Choose between LLM-based OCR or Mistral dedicated OCR, and specify image or whole PDF processing mode. ```typescript // LLM-based OCR (OpenAI) const testEnv = await setupTestEnvironment({ ocrProvider: 'llm', processMode: 'image' }); ``` ```typescript // Mistral dedicated OCR const testEnv = await setupTestEnvironment({ ocrProvider: 'mistral_ocr', processMode: 'whole_pdf' }); ``` -------------------------------- ### GET /api/filter-tag Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves the name of the manual processing tag. This is used by the frontend to determine which tag to display documents for. ```bash curl -s http://localhost:8080/api/filter-tag | jq . # {"tag": "paperless-gpt"} ``` -------------------------------- ### Provider Initialization Logs Source: https://github.com/icereed/paperless-gpt/blob/main/cline_docs/systemPatterns.md Logs indicating the initialization of an OCR provider and the specific provider and model being used. ```log [INFO] Initializing OCR provider: llm [INFO] Using LLM OCR provider (provider=ollama, model=minicpm-v) ``` -------------------------------- ### Get All Documents Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves all documents currently tagged for manual processing. These documents are candidates for AI-powered suggestion generation. ```APIDOC ## GET /api/documents ### Description Returns all documents currently tagged with the manual processing tag (`paperless-gpt` by default). These documents are awaiting AI-powered suggestion generation. ### Method GET ### Endpoint /api/documents ### Response #### Success Response (200) - **documents** (array) - An array of Document objects. ### Response Example ```json [ { "id": 42, "title": "scan_20240101.pdf", "content": "Invoice Number: 1-996-84199...", "tags": ["paperless-gpt", "invoice"], "correspondent": "", "created_date": "2024-01-01", "original_file_name": "scan_20240101.pdf", "document_type_name": "", "custom_fields": [] } ] ``` ``` -------------------------------- ### Build Docker Image for E2E Tests Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Builds the paperless-gpt Docker image required for E2E tests. Rebuild this image after application code changes. ```bash # From the project root directory docker build . -t icereed/paperless-gpt:e2e ``` -------------------------------- ### Create Prompts Directory Source: https://github.com/icereed/paperless-gpt/blob/main/README.md Create a directory named 'prompts' to store custom prompt files. ```bash mkdir prompts ``` -------------------------------- ### GET /api/paperless-url Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns the public-facing URL of the connected paperless-ngx instance. Used by the frontend to generate direct document links. ```APIDOC ## GET /api/paperless-url ### Description Retrieves the public-facing URL of the connected paperless-ngx instance. This URL is derived from the `PAPERLESS_PUBLIC_URL` or `PAPERLESS_BASE_URL` environment variables and is used by the frontend to construct direct links to documents. ### Method GET ### Endpoint /api/paperless-url ### Response #### Success Response (200) - **url** (string) - The public-facing URL of the paperless-ngx instance. ### Response Example ```json { "url": "https://paperless.mydomain.com" } ``` ``` -------------------------------- ### Get Modifications Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns paginated modification history from the local SQLite database. Supports `?page=1&pageSize=20` query parameters. ```APIDOC ## GET /api/modifications ### Description Returns paginated modification history from the local SQLite database. Supports `?page=1&pageSize=20` query parameters. Records track every field change (title, tags, content, correspondent) applied through paperless-gpt with undo support. ### Method GET ### Endpoint /api/modifications ### Query Parameters - **page** (integer) - Optional - The page number to retrieve. - **pageSize** (integer) - Optional - The number of items per page. ### Response #### Success Response (200) - **items** (array of objects) - A list of modification records. - **ID** (integer) - The unique ID of the modification record. - **DocumentID** (integer) - The ID of the document that was modified. - **DateChanged** (string) - The timestamp when the change occurred. - **ModField** (string) - The field that was modified (e.g., 'title', 'tags'). - **PreviousValue** (string) - The value of the field before the change. - **NewValue** (string) - The value of the field after the change. - **Undone** (boolean) - Indicates if the modification has been undone. - **UndoneDate** (string) - The timestamp when the modification was undone (if applicable). - **totalItems** (integer) - The total number of modification records available. #### Response Example ```json { "items": [ { "ID": 15, "DocumentID": 42, "DateChanged": "2024-01-15T10:02:00Z", "ModField": "title", "PreviousValue": "scan_20240101.pdf", "NewValue": "FedEx Ground Invoice Sep 2014", "Undone": false, "UndoneDate": "" } ], "totalItems": 42 } ``` ``` -------------------------------- ### Docker Compose Deployment for paperless-gpt Source: https://context7.com/icereed/paperless-gpt/llms.txt Deploy paperless-gpt alongside paperless-ngx using Docker Compose. Configure paperless-ngx connection, LLM provider, and optional OCR settings via environment variables. Mount a local 'prompts' directory for custom prompt templates. ```yaml # docker-compose.yml services: paperless-ngx: image: ghcr.io/paperless-ngx/paperless-ngx:latest environment: PAPERLESS_SECRET_KEY: "supersecret" PAPERLESS_TIME_ZONE: "Europe/Berlin" volumes: - paperless-data:/usr/src/paperless/data - paperless-media:/usr/src/paperless/media ports: - "8000:8000" paperless-gpt: image: icereed/paperless-gpt:latest environment: # Required: Paperless-ngx connection PAPERLESS_BASE_URL: "http://paperless-ngx:8000" PAPERLESS_API_TOKEN: "your_paperless_api_token" # Required: LLM backend (choose one) LLM_PROVIDER: "openai" LLM_MODEL: "gpt-4o" OPENAI_API_KEY: "sk-..." # Optional: OCR backend OCR_PROVIDER: "llm" VISION_LLM_PROVIDER: "openai" VISION_LLM_MODEL: "gpt-4o" # Optional: tag names (defaults shown) MANUAL_TAG: "paperless-gpt" AUTO_TAG: "paperless-gpt-auto" AUTO_OCR_TAG: "paperless-gpt-ocr-auto" LLM_LANGUAGE: "English" LOG_LEVEL: "info" # Optional: rate limiting LLM_REQUESTS_PER_MINUTE: "60" LLM_MAX_RETRIES: "3" LLM_BACKOFF_MAX_WAIT: "30s" volumes: - ./prompts:/app/prompts # Persist custom prompt templates ports: - "8080:8080" depends_on: - paperless-ngx volumes: paperless-data: paperless-media: ``` -------------------------------- ### Get Custom Fields Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns all custom fields defined in paperless-ngx with their IDs, names, and data types. Supports a query parameter to bypass the cache. ```APIDOC ## GET /api/custom_fields ### Description Returns all custom fields defined in paperless-ngx with their IDs, names, and data types. Pass `?force_pull=true` to bypass the in-memory cache and fetch fresh data. ### Method GET ### Endpoint /api/custom_fields ### Query Parameters - **force_pull** (boolean) - Optional - If true, bypasses the in-memory cache and fetches fresh data. ### Response #### Success Response (200) - (array of objects) - A list of custom fields, each with `id`, `name`, and `data_type`. #### Response Example ```json [ {"id": 1, "name": "Invoice Amount", "data_type": "monetary"}, {"id": 2, "name": "Due Date", "data_type": "date"}, {"id": 3, "name": "Vendor ID", "data_type": "string"} ] ``` ``` -------------------------------- ### Build E2E Docker Image Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Builds the Docker image required for E2E tests with the tag `icereed/paperless-gpt:e2e`. ```bash # Solution: Build the required Docker image docker build . -t icereed/paperless-gpt:e2e ``` -------------------------------- ### Get All Tags Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves all tags from paperless-ngx, returned as a map of tag names to their IDs. This is used for populating tag selections in the frontend and for LLM suggestions. ```bash curl -s http://localhost:8080/api/tags | jq . ``` -------------------------------- ### Simulate CI Environment Locally Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Run end-to-end tests locally in a simulated CI environment. This command uses the 'github' reporter for output compatible with CI systems. ```bash # Simulate CI environment locally npm run test:e2e -- --reporter=github ``` -------------------------------- ### Get All OCR Jobs Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieves a list of all OCR jobs, including pending, in-progress, completed, failed, and cancelled jobs, sorted by creation time in descending order. ```APIDOC ## GET /api/jobs/ocr ### Description Retrieves a list of all OCR jobs, including pending, in-progress, completed, failed, and cancelled jobs, sorted by creation time in descending order. ### Method GET ### Endpoint /api/jobs/ocr ### Response #### Success Response (200) - **Array of OCR job objects** - Contains details for each OCR job. - **job_id** (string) - The ID of the OCR job. - **status** (string) - The current status of the job. - **created_at** (string) - Timestamp when the job was created. - **updated_at** (string) - Timestamp when the job was last updated. - **pages_done** (integer) - Number of pages processed. - **result** (string, optional) - The OCR result if the job is completed. ### Request Example ```bash curl -s http://localhost:8080/api/jobs/ocr | jq . ``` ### Response Example ```json [ { "job_id": "550e8400-...", "status": "completed", "created_at": "2024-01-15T10:00:00Z", "updated_at": "2024-01-15T10:02:00Z", "pages_done": 5, "result": "La Grande Récré\nCentre Commercial..." }, { "job_id": "661f9511-...", "status": "pending", "created_at": "2024-01-15T10:03:00Z", "updated_at": "2024-01-15T10:03:00Z", "pages_done": 0 } ] ``` ``` -------------------------------- ### Get All Documents Tagged for Processing Source: https://context7.com/icereed/paperless-gpt/llms.txt Retrieve a list of all documents currently tagged for manual processing by paperless-gpt. This endpoint is useful for understanding which documents are queued for AI enhancement. ```bash curl -s http://localhost:8080/api/documents | jq . # Response: array of Document objects # [ # { # "id": 42, # "title": "scan_20240101.pdf", # "content": "Invoice Number: 1-996-84199...", # "tags": ["paperless-gpt", "invoice"], # "correspondent": "", # "created_date": "2024-01-01", # "original_file_name": "scan_20240101.pdf", # "document_type_name": "", # "custom_fields": [] # } # ] ``` -------------------------------- ### Download and Render Document as Images Source: https://context7.com/icereed/paperless-gpt/llms.txt Downloads a document from Paperless-NGX and renders each page into JPEG images using MuPDF. Image DPI is optimized for pixel limits, and images are cached by document ID. ```go client := NewPaperlessClient("http://paperless-ngx:8000", "my-api-token") // Download doc 42 as images, limit to first 5 pages imagePaths, totalPages, err := client.DownloadDocumentAsImages(ctx, 42, 5) if err != nil { log.Printf("download error: %v", err) } fmt.Printf("Rendered %d of %d pages\n", len(imagePaths), totalPages) // ["/paperless-gpt/document-42/page000.jpg", // "/paperless-gpt/document-42/page001.jpg", ...] // Image size is automatically controlled by: // IMAGE_MAX_PIXEL_DIMENSION=10000 (max pixels on any side) // IMAGE_MAX_TOTAL_PIXELS=40000000 (max total pixels W×H) // IMAGE_MAX_RENDER_DPI=600 (max render DPI) // IMAGE_MAX_FILE_BYTES=10485760 (max JPEG file size, triggers quality reduction/resize) ``` -------------------------------- ### OCR Error Logging Source: https://github.com/icereed/paperless-gpt/blob/main/cline_docs/systemPatterns.md Examples of error logs that may occur during OCR processing, such as image decoding failures, unsupported file types, or issues with the vision model response. ```log [ERROR] Failed to decode image: invalid format [ERROR] Unsupported file type: image/webp [ERROR] Failed to get response from vision model ``` -------------------------------- ### Run E2E Tests with UI Mode Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Launches the E2E tests in UI mode, allowing for interactive debugging and observation of the browser. ```bash npm run test:e2e:ui ``` -------------------------------- ### Run All E2E Tests Source: https://github.com/icereed/paperless-gpt/blob/main/web-app/e2e/README.md Executes all end-to-end tests for the paperless-gpt application. ```bash npm run test:e2e ``` -------------------------------- ### Get Tags Source: https://context7.com/icereed/paperless-gpt/llms.txt Returns all tags from paperless-ngx as a map of `tagName -> tagID`. Used by the frontend to populate tag selection and by the LLM suggestion pipeline to constrain suggested tags to existing values. ```APIDOC ## GET /api/tags ### Description Returns all tags from paperless-ngx as a map of `tagName -> tagID`. Used by the frontend to populate tag selection and by the LLM suggestion pipeline to constrain suggested tags to existing values. ### Method GET ### Endpoint /api/tags ### Response #### Success Response (200) - (object) - A map where keys are tag names (string) and values are tag IDs (integer). #### Response Example ```json { "invoice": 1, "receipt": 2, "tax": 3, "paperless-gpt": 4, "paperless-gpt-auto": 5, "fedex": 6 } ``` ``` -------------------------------- ### Docling Server Configuration Source: https://github.com/icereed/paperless-gpt/blob/main/README.md Configure Paperless-GPT to use a self-hosted Docling Server for OCR and document conversion. This is suitable for users prioritizing data privacy and self-hosting. ```yaml OCR_PROVIDER: "docling" DOCLING_URL: "http://your-docling-server:port" DOCLING_IMAGE_EXPORT_MODE: "placeholder" # Optional, defaults to "embedded" DOCLING_OCR_PIPELINE: "standard" # Optional, defaults to "vlm" DOCLING_OCR_ENGINE: "macocr" # Optional, defaults to "easyocr" (only used when `DOCLING_OCR_PIPELINE is set to 'standard') ``` -------------------------------- ### Optional Environment Variables Source: https://github.com/icereed/paperless-gpt/blob/main/cline_docs/techContext.md Optional environment variables for customizing paperless-gpt behavior, including public URL, tagging, LLM provider specifics, OCR settings, and logging level. ```dotenv PAPERLESS_PUBLIC_URL=public_url MANUAL_TAG=paperless-gpt AUTO_TAG=paperless-gpt-auto OPENAI_API_KEY=key (if using OpenAI) OPENAI_BASE_URL=custom_url LLM_LANGUAGE=English OLLAMA_HOST=host_url VISION_LLM_PROVIDER=provider VISION_LLM_MODEL=model AUTO_OCR_TAG=tag OCR_LIMIT_PAGES=5 LOG_LEVEL=info ```