### Verify Parsr Installation Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Use these commands to run the test suite and start the API to ensure the installation is correct. ```sh npm run test ``` ```sh npm run start:api ``` -------------------------------- ### Manage API server Source: https://github.com/axa-group/parsr/blob/master/docs/usage.md Commands to install and start the Parsr API server. ```sh npm run install:api ``` ```sh npm run start:api ``` -------------------------------- ### Install project dependencies Source: https://github.com/axa-group/parsr/blob/master/demo/vue-viewer/README.md Run this command to install all required project dependencies. ```bash npm install ``` -------------------------------- ### Module Configuration for Example Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/KeyValueDetectionModule/README.md Specific configuration used to process the provided document text example. ```json { "thresholdRatio": 0.8, "keyValueDividerChars": [ ":" ], "keyPatterns": { "First Name": [ "Name", "First Name" ], "Surname": [ "Surname", "Surnames", "Last Name" ], "Address": [ "Address", "Domicile", "Residence" ] } } ``` -------------------------------- ### Check Installed Dependencies Source: https://github.com/axa-group/parsr/blob/master/docs/api-guide.md Accesses the installation check endpoint to verify required and optional dependencies. ```http http://localhost:3001/api/v1/check-installation ``` -------------------------------- ### Install Python Client Source: https://github.com/axa-group/parsr/blob/master/README.md Command to install the Python client library for interacting with the Parsr API. ```sh pip install parsr-client ``` -------------------------------- ### Execute the project Source: https://github.com/axa-group/parsr/blob/master/demo/doc-versioning/README.md Run the application from the project root directory after installing all dependencies. ```sh ./run.py ``` -------------------------------- ### Install pip on MacOS Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Downloads and installs pip for Python 2 environments. ```sh curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py python get-pip.py ``` -------------------------------- ### Module Configuration Example Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/KeyValueDetectionModule/README.md Example configuration defining the similarity threshold, separator characters, and key patterns for the detection module. ```json [ "key-value-detection", { "thresholdRatio": 0.8, "keyValueDividerChars": [":", ";"], "keyPatterns": { "Name": ["Name", "Fullname", "User"], "Date of admission": ["ADMISSION DATE & TIME", "Adm Date/Time", "Reg/Admit Date"] } } ] ``` -------------------------------- ### Run Automatic Installation Script Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Executes the Node.js script to install all required and optional dependencies in the root directory. ```bash npm run install:pre ``` -------------------------------- ### Start development server Source: https://github.com/axa-group/parsr/blob/master/demo/vue-viewer/README.md Compiles the project and enables hot-reloading for local development. ```bash npm run serve ``` -------------------------------- ### Install MacOS System Dependencies Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs required system-level tools via Homebrew. ```sh brew install node python qpdf imagemagick tesseract tesseract-lang tcl-tk ghostscript ``` -------------------------------- ### Install Required Dependencies Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/MlHeadingDetectionModule/train_model/README.md Run these commands to install the necessary Python libraries and download the required spaCy model. ```bash python3 -m pip install spacy python3 -m pip install markdown python3 -m pip install imblearn python3 -m pip install sklearn_porter python3 -m pip install scikit-learn==0.22.2 python3 -m spacy download en_core_web_sm pip3 install beautifulsoup4 ``` -------------------------------- ### Install Pandoc across platforms Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Commands to install Pandoc for PDF output generation on various operating systems. ```sh sudo apt-get install pandoc ``` ```sh pacman -S pandoc ``` ```sh brew install pandoc ``` ```sh choco install pandoc ``` -------------------------------- ### GET /api/v1/check-installation Source: https://github.com/axa-group/parsr/blob/master/docs/api-guide.md Checks the status of required and optional system dependencies. ```APIDOC ## GET /api/v1/check-installation ### Description Checks if every required and optional dependency is installed and displays their system paths. ### Method GET ### Endpoint /api/v1/check-installation ``` -------------------------------- ### Install Windows Dependencies via Chocolatey Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs qpdf and imagemagick using the Chocolatey package manager. ```sh choco install qpdf imagemagick ``` -------------------------------- ### Queue Status Response Examples Source: https://github.com/axa-group/parsr/blob/master/docs/api-guide.md Example JSON responses for document processing status. ```json { "estimated-remaining-time": 30, "progress-percentage": 10, "start-date": "2018-12-31T12:34:56.789Z", "status": "Detecting reading order..." } ``` ```json { "id": "00cafe4463b9c12aac145b3ee8f00d", "json": "/api/v1/json/00cafe4463b9c12aac145b3ee8f00d", "csv": "/api/v1/csv/00cafe4463b9c12aac145b3ee8f00d", "text": "/api/v1/text/00cafe4463b9c12aac145b3ee8f00d", "markdown": "/api/v1/markdown/00cafe4463b9c12aac145b3ee8f00d" } ``` -------------------------------- ### Get queue status via cURL Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Example request to retrieve the status of a specific queue item. ```bash curl -X GET "https://localhost:3001/api/v1/queue/{id}" ``` -------------------------------- ### Start web viewer demo Source: https://github.com/axa-group/parsr/blob/master/docs/usage.md Commands to launch the web interface for document processing. ```sh npm run start:web:vue ``` ```sh npm run start:api ``` ```sh cd demo/vue-viewer && npm install && npm run serve ``` -------------------------------- ### Get CSV List (Objective-C) Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Objective-C SDK example for retrieving the list of CSV file paths. ```objective-c String *id = id_example; // ID of the document OutputApi *apiInstance = [[OutputApi alloc] init]; // Get the list of every CSV file path [apiInstance getCsvListWith:id completionHandler: ^(array['String'] output, NSError* error) { if (output) { NSLog(@"%@", output); } if (error) { NSLog(@"Error: %@", error); } }]; ``` -------------------------------- ### Get CSV List (Java) Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Java SDK example for retrieving the list of CSV file paths. ```java import io.swagger.client.*; import io.swagger.client.auth.*; import io.swagger.client.model.*; import io.swagger.client.api.OutputApi; import java.io.File; import java.util.*; public class OutputApiExample { public static void main(String[] args) { OutputApi apiInstance = new OutputApi(); String id = id_example; // String | ID of the document try { array['String'] result = apiInstance.getCsvList(id); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling OutputApi#getCsvList"); e.printStackTrace(); } } } ``` ```java import io.swagger.client.api.OutputApi; public class OutputApiExample { public static void main(String[] args) { OutputApi apiInstance = new OutputApi(); String id = id_example; // String | ID of the document try { array['String'] result = apiInstance.getCsvList(id); System.out.println(result); } catch (ApiException e) { System.err.println("Exception when calling OutputApi#getCsvList"); e.printStackTrace(); } } } ``` -------------------------------- ### Install Homebrew on MacOS Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs the Homebrew package manager for MacOS. ```sh /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" ``` -------------------------------- ### AJAX Prefilter and Setup Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Configuration for AJAX prefilters and default settings for script handling. ```javascript r.ajaxPrefilter(function(a){a.crossDomain&&(a.contents.script=!1)}),r.ajaxSetup({accepts:{script:"text/javascript, application/javascript, application/ecmascript, application/x-ecmascript"},contents:{script:/\b(?:java|ecma)script\b/},converters:{"text script":function(a){return r.globalEval(a),a}}}),r.ajaxPrefilter("script",function(a){void ``` -------------------------------- ### Install pip3 on MacOS Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Downloads and installs pip3 for Python 3 environments. ```sh curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py python3 get-pip.py ``` -------------------------------- ### Install Python 2 Dependencies on MacOS Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs PyPDF2 for Python 2.7 environments. ```sh python2.7 -m pip install PyPDF2 ``` -------------------------------- ### Initialize Affix Plugin Source: https://github.com/axa-group/parsr/blob/master/docs/api.html The initial setup for the Affix plugin, which monitors scroll events to update element positioning. ```javascript !function(e){ "use strict"; var t=function(t,n){this.options=e.extend({},e.fn.affix.defaults,n),this.$window=e(window).on("scroll.affix.data-api",e.proxy(this.checkPosition,this)).on("click.affix.data-api",e.proxy(function(){setTimeout(e.proxy(this.checkPosition,this),1)},this)),this.$element=e(t),this.checkPosition()};t.prototype.checkPosition=function(){if(!this.$element.is(":visible"))return;var ``` -------------------------------- ### Install Dependencies on Arch Linux Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs system packages and Python dependencies for Arch Linux. ```sh pacman -S nodejs npm qpdf imagemagick python-pdfminer tesseract python-pip pip install camelot-py[cv] numpy pillow scikit-image PyPDF2 pdfminer.six sklearn ``` -------------------------------- ### Install Dependencies on Debian-based Linux Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs system packages and Python dependencies for Debian distributions. ```sh sudo add-apt-repository ppa:ubuntuhandbook1/apps sudo apt-get update sudo apt-get install nodejs npm qpdf imagemagick tesseract-ocr libtesseract-dev python3-tk ghostscript python3-pip pip install camelot-py[cv] numpy pillow scikit-image PyPDF2 pdfminer.six sklearn ``` -------------------------------- ### Initialize Marked and UI Components Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Configures the marked library and sets up event listeners for UI interactions like scrolling, tab switching, and dynamic content processing. ```javascript $( document ).ready(function() { marked.setOptions({ renderer: new marked.Renderer(), gfm: true, tables: true, breaks: false, pedantic: false, sanitize: false, smartLists: true, smartypants: false }); var textFile = null; /// Function to be used to download a text json schema function makeTextFile(text) { var data = new Blob([text], {type: 'text/plain'}); // If we are replacing a previously generated file we need to // manually revoke the object URL to avoid memory leaks. if (textFile !== null) { window.URL.revokeObjectURL(textFile); } textFile = window.URL.createObjectURL(data); var a = document.createElement("a"); document.body.appendChild(a); a.style = "display: none"; a.href = textFile; a.download = 'schema.txt'; a.click(); return textFile; }; /// TODO: Implement resizing for expanding within iframe function callResize() { window.parent.postMessage('resize', "*"); } function processMarked() { $(".marked").each(function() { $(this).html(marked($(this).html())); }); } // Bootstrap Scrollspy $(this).scrollspy({ target: '#scrollingNav', offset: 18 }); // Content-Scroll on Navigation click. $('.sidenav').find('a').on('click', function(e) { e.preventDefault(); var id = $(this).attr('href'); if ($(id).length > 0) $('html,body').animate({ scrollTop: parseInt($(id).offset().top) }, 400); window.location.hash = $(this).attr('href'); }); // Quickjump on Pageload to hash position. if(window.location.hash) { var id = window.location.hash; if ($(id).length > 0) $('html,body').animate({ scrollTop: parseInt($(id).offset().top) }, 0); } function initDynamic() { // tabs $('.nav-tabs-examples a').click(function (e) { e.preventDefault(); $(this).tab('show'); }); $('.nav-tabs-examples').find('a:first').tab('show'); // call scrollspy refresh method $(window).scrollspy('refresh'); } initDynamic(); // Pre- / Code-Format prettyPrint(); //Convert elements with "marked" class to markdown processMarked(); }); ``` -------------------------------- ### Initialize UI Components Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Configures navigation tabs and scrollspy behavior on document ready. ```javascript $(document).ready(function () { $('.nav-tabs-examples').find('a:first').tab('show'); $(this).scrollspy({ target: '#scrollingNav', offset: 18 }); }); ``` -------------------------------- ### Install dependencies with pipenv Source: https://github.com/axa-group/parsr/blob/master/demo/parsr-jupyter-demo/README.md Run this command inside the demo/parsr-jupyter-demo folder to install required project dependencies. ```sh pipenv install ``` -------------------------------- ### Configure Table Detection Module Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/TableDetectionModule/README.md Example configuration for the table-detection module, defining page ranges and parsing flavors. ```json [ "table-detection", { "checkDrawings": true, "runConfig": [ { "pages": [1, 2, 3], // or [] for all pages "flavor": "lattice", "table_areas": [] } ] } ] ``` -------------------------------- ### PdfMiner Figure Element Example Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/ImageDetectionModule/README.md Example of a figure element detected by PdfMiner containing an image reference. ```xml //'Image17' will be used to match image XObject using dumppdf
``` -------------------------------- ### Build Dataset Command Usage Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/MlHeadingDetectionModule/train_model/README.md Displays the command-line interface help for the build_dataset.py script. ```text usage: build_dataset.py [-h] md_dir json_dir out_dir Extracts features to csv from .json files using .md files as labels positional arguments: md_dir folder containing the .md files (labels) json_dir folder containing the .json files (data) out_dir folder in which to save the .csv files optional arguments: -h, --help show this help message and exit ``` -------------------------------- ### Build for production Source: https://github.com/axa-group/parsr/blob/master/demo/vue-viewer/README.md Compiles and minifies the project assets for production deployment. ```bash npm run build ``` -------------------------------- ### Connect to Parsr Server Source: https://github.com/axa-group/parsr/blob/master/clients/python-client/README.md Initialize the client by providing the address of a running Parsr server. ```python from parsr_client import ParsrClient parsr = ParsrClient('localhost:3001') ``` -------------------------------- ### Install MuPDF across platforms Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Commands to install MuPDF for PDF repair functionality on various operating systems. ```sh sudo apt-get install mupdf mupdf-tools ``` ```sh pacman -S mupdf-tools ``` ```sh brew install mupdf-tools ``` ```sh choco install mupdf ``` -------------------------------- ### Query Module Configuration Source: https://github.com/axa-group/parsr/blob/master/docs/api-guide.md Fetches the configuration file for a specific module, including parameters and ranges. ```bash curl -X GET \ http://localhost:3001/api/v1/module-config/table-detection ``` -------------------------------- ### List Available Modules Source: https://github.com/axa-group/parsr/blob/master/docs/api-guide.md Retrieves a list of all usable modules currently available on the server. ```bash curl -X GET \ http://localhost:3001/api/v1/modules ``` -------------------------------- ### Install Python 3 Dependencies on MacOS Source: https://github.com/axa-group/parsr/blob/master/docs/installation.md Installs Python 3 specific libraries for PDF processing and data manipulation. ```sh pip3 install pdfminer.six pip3 install camelot-py[cv] pip3 install numpy pillow scikit-image python2.7 -m pip install PyPDF2 ``` -------------------------------- ### Define Pipeline Module with Defaults Source: https://github.com/axa-group/parsr/blob/master/docs/configuration.md Simplified configuration formats for modules using default options. ```json [ "module" ], ``` ```json "module", ``` -------------------------------- ### Dumppdf XObject Mapping Example Source: https://github.com/axa-group/parsr/blob/master/server/src/processing/ImageDetectionModule/README.md Example of internal PDF structure dumped by dumppdf used to map figure names to image file identifiers. ```xml XObject Image17 // '17' will be used to match a image file named 'img-0017' extracted by MuPDF ``` -------------------------------- ### GET /queue/status Source: https://github.com/axa-group/parsr/blob/master/docs/api.html Retrieves the current status of the processing queue. ```APIDOC ## GET /queue/status ### Description Returns the current status of the processing queue. ### Method GET ### Response #### Success Response (200) - Returns the status of the queue. ``` -------------------------------- ### Initialize JSON schema view Source: https://github.com/axa-group/parsr/blob/master/docs/api.html JavaScript snippet to render the JSON schema for the 200 OK response. ```javascript $(document).ready(function() { var schemaWrapper = { "description" : "Ok", "content" : { "application/json" : { "schema" : { "type" : "array", "example" : [ "/api/v1.0/csv/{id}/1/1", "/api/v1.0/csv/{id}/{page}/{table}" ], "items" : { "type" : "string", "description" : "Paths to CSV files" }, "x-content-type" : "application/json" } } } }; var schema = schemaWrapper.content["application/json"].schema; if (schema.$ref != null) { schema = defsParser.$refs.get(schema.$ref); } else { schemaWrapper.components = {}; schemaWrapper.components.schemas = Object.assign({}, defs); $RefParser.dereference(schemaWrapper).catch(function(err) { console.log(err); }); } //console.log(JSON.stringify(schema)); var view = new JSONSchemaView(schema, 3); $('#responses-getCsvList-200-schema-data').val(stringify(schema)); var result = $('#responses-getCsvList-200-schema-200'); result.empty(); result.append(view.render()); }); ```