### Start Monitoring Stack Source: https://grobid.readthedocs.io/en/latest/Monitoring Execute this command to start the services defined in the docker-compose.yml file. ```bash docker compose up -d ``` -------------------------------- ### Install JEP Library Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Execute the provided script to install the native JEP library required for the Java-Python bridge. ```bash ./grobid-home/scripts/install_jep_lib.sh ``` -------------------------------- ### Run GROBID Service Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Start the GROBID service using Gradle. ```bash ./gradlew run ``` -------------------------------- ### Verify Java Installation Source: https://grobid.readthedocs.io/en/latest/Install-Grobid Commands to verify the installed versions of the Java runtime and compiler. ```bash # Check Java version java -version # Check JDK version (should show 21+ for building) javac -version ``` -------------------------------- ### Install OpenJDK 21 on Linux (CentOS/RHEL/Fedora) Source: https://grobid.readthedocs.io/en/latest/Install-Grobid Command to install the OpenJDK 21 development kit using the dnf package manager. ```bash # For JDK 21 sudo dnf install java-21-openjdk-devel ``` -------------------------------- ### Test DeLFT Installation Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Verify the DeLFT environment by running the citation tagger application. ```bash cd delft/ python -m delft.applications.grobidTagger citation tag --architecture BidLSTM_CRF ``` -------------------------------- ### Run the PDF-TEI-Editor with Docker Source: https://grobid.readthedocs.io/en/latest/PDF-TEI-Editor Starts the editor container, configuring the admin password and the GROBID server URL. ```bash docker run -p 8000:8000 \ -e APP_ADMIN_PASSWORD=secure_password \ -e GROBID_SERVER_URL=https://grobidOrg-grobid-full.hf.space \ cboulanger/pdf-tei-editor:latest ``` -------------------------------- ### GET /api/isalive Source: https://grobid.readthedocs.io/en/latest/Grobid-service Checks if the service has completed initialization successfully. ```APIDOC ## GET /api/isalive ### Description Returns true/false as plain text indicating whether the service completed initialization successfully. Suitable for use as a liveness probe. ### Method GET ### Endpoint /api/isalive ### Response #### Success Response (200) - **body** (string) - Returns 'true' if alive. #### Error Response (503) - **status** (integer) - Returns 503 when not initialized or initialization failed. ``` -------------------------------- ### Standalone installation directory structure Source: https://grobid.readthedocs.io/en/latest/Grobid-service Expected directory layout after extracting the standalone service distributions. ```text . ├── grobid-home └── grobid-service ``` -------------------------------- ### Install JNI JAR Source: https://grobid.readthedocs.io/en/latest/Recompiling-and-integrating-CRF-libraries Deploy the JNI JAR file to the local repository within grobid-core. ```bash > mvn install:install-file -Dfile=wapiti-1.5.0.jar -DgroupId="fr.limsi.wapiti" -DartifactId="wapiti" -Dversion="1.5.0" -Dpackaging="jar" -DlocalRepositoryPath="GROBID-ROOT-DIRECTORY/grobid-core/lib" ``` -------------------------------- ### Install OpenJDK 21 on Linux (Ubuntu/Debian) Source: https://grobid.readthedocs.io/en/latest/Install-Grobid Commands to update package lists and install the OpenJDK 21 development kit on Debian-based systems. ```bash # Ubuntu 22.04+ or Debian 12+ sudo apt update sudo apt install openjdk-21-jdk ``` -------------------------------- ### TEI Citation Output Example Source: https://grobid.readthedocs.io/en/latest/Grobid-service Example of the TEI-structured output returned by the service, including raw reference notes. ```xml <author> <persName xmlns="http://www.tei-c.org/ns/1.0"><forename type="first">T</forename><forename type="middle">J</forename><surname>Vaughan</surname></persName> </author> </analytic> <monogr> <title level="j">Nat.Biotech 14 Vaughan, T.J. et al., Nat.Biotech., 14:309-314(1996) <author> <persName xmlns="http://www.tei-c.org/ns/1.0"><forename type="first">Perkin</forename><surname>Elmer</surname></persName> </author> <imprint> <biblScope unit="volume">1</biblScope> <biblScope unit="page">1000</biblScope> </imprint> </monogr> <note type="report_type">Cat.1244-360</note> <note type="raw_reference">Perkin Elmer, Cat.1244-360, 1:1000</note> </biblStruct> ``` -------------------------------- ### Install OpenJDK 21 on macOS Source: https://grobid.readthedocs.io/en/latest/Install-Grobid Commands to install OpenJDK 21 via Homebrew and configure the JAVA_HOME environment variable. ```bash # Install JDK 21 brew install openjdk@21 # Set JAVA_HOME (add to ~/.zshrc or ~/.bash_profile) export JAVA_HOME=$(brew --prefix)/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home ``` -------------------------------- ### Build and run standalone service Source: https://grobid.readthedocs.io/en/latest/Grobid-service Commands to build the distribution, create an installation directory, and execute the standalone service binary. ```bash ./gradlew clean assemble cd .. mkdir grobid-installation cd grobid-installation unzip ../grobid/grobid-service/build/distributions/grobid-service-0.9.1.zip mv grobid-service-0.9.1 grobid-service unzip ../grobid/grobid-home/build/distributions/grobid-home-0.9.1.zip ./grobid-service/bin/grobid-service ``` -------------------------------- ### Running the Grobid container Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Commands to start the Grobid container with default or custom port mappings. ```bash > docker run --rm --init --ulimit core=0 -p 8070:8070 grobid/grobid:${latest_grobid_version}-crf ``` ```bash > docker run --rm --init --ulimit core=0 -p 8070:8070 grobid/grobid:0.9.1-crf ``` ```bash docker run --rm --init --ulimit core=0 -p 8080:8070 -p 8081:8071 grobid/grobid:${latest_grobid_version}-crf ``` -------------------------------- ### GET /api/allTraining Source: https://grobid.readthedocs.io/en/latest/Grobid-service Lists all training tokens currently running in the server instance. ```APIDOC ## GET /api/allTraining ### Description List the tokens of the trainings currently running in this server instance. ### Method GET ### Endpoint /api/allTraining ### Response #### Success Response (200) - **tokens** (array) - List of active training tokens #### Error Handling - 500: Internal service error ``` -------------------------------- ### Run Deep Learning GROBID Container Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Starts a container with GPU support and optional configuration mounting. ```bash docker run --rm --gpus all --init --ulimit core=0 -p 8080:8070 -p 8081:8071 grobid/grobid:0.9.2-SNAPSHOT ``` ```bash docker run --rm --gpus all --init --ulimit core=0 -p 8080:8070 -p 8081:8071 -v /home/lopez/grobid/grobid-home/config/grobid.yaml:/opt/grobid/grobid-home/config/grobid.yaml:ro grobid/grobid:0.9.2-SNAPSHOT ``` -------------------------------- ### Run a full GROBID instance with Docker Source: https://grobid.readthedocs.io/en/latest/PDF-TEI-Editor Starts a GROBID server container with deep learning models enabled on port 8070. ```bash docker run --rm --gpus all --init --ulimit core=0 -p 8070:8070 grobid/grobid:0.9.1-full ``` -------------------------------- ### Install native libraries Source: https://grobid.readthedocs.io/en/latest/Recompiling-and-integrating-CRF-libraries Copy the compiled native libraries to the appropriate Grobid home directory for Linux or Mac OS X. ```bash > cp ld-linux-x86-64.so.2 libc.so.6 libgcc_s.so.1 libm.so.6 libstdc++.so.6 libwapiti.so GROBID-ROOT-DIRECTORY/grobid-home/lib/lin-<nb bits of the OS> ``` ```bash > cp libwapiti.dylib GROBID-ROOT-DIRECTORY/grobid-home/lib/mac-64/ ``` -------------------------------- ### Run Grobid Container Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Standard command to start the Grobid container with GPU support and core dump limits. ```bash > docker run --rm --gpus all --init --ulimit core=0 -p 8070:8070 grobid/grobid:0.9.1-full ``` -------------------------------- ### Start the GROBID service via Docker Source: https://grobid.readthedocs.io/en/latest/Upgrading Command to launch the GROBID service using a Docker container. ```bash docker run ... grobid/grobid:0.9.1-full ``` -------------------------------- ### Example CXXABI Linker Error Source: https://grobid.readthedocs.io/en/latest/Frequently-asked-questions An example of the ImportError encountered when the system's libstdc++ version is incompatible with the Python environment's requirements. ```text ImportError: /home/user/miniconda3/envs/myenv/lib/python3.10/lib-dynload/../../libicui18n.so.78: version `CXXABI_1.3.15' not found (required by .../libstdc++.so.6) ``` -------------------------------- ### GET /metrics/prometheus Source: https://grobid.readthedocs.io/en/latest/Grobid-service Retrieves application and JVM metrics in Prometheus exposition format. ```APIDOC ## GET /metrics/prometheus ### Description Returns application metrics (rate/throughput per entry point) and JVM/process metrics (heap, GC, threads, CPU) in Prometheus exposition format. ### Method GET ### Endpoint /metrics/prometheus ``` -------------------------------- ### Create Anaconda environment Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Initializes a new conda environment named grobidDelft with pip installed. ```bash conda create --name grobidDelft pip ``` -------------------------------- ### Start Model Training with cURL Source: https://grobid.readthedocs.io/en/latest/Grobid-service Initiates a training process for a specified model using the /api/modelTraining endpoint. ```bash curl -v -X POST -d "model=date" -d "type=holdout" localhost:8070/api/modelTraining ``` -------------------------------- ### Tagging Web References Source: https://grobid.readthedocs.io/en/latest/training/Bibliographical-references Example of tagging a web-based reference including publisher, date, and URL pointer. ```xml <?xml version="1.0" encoding="UTF-8"?> <tei xmlns="http://www.tei-c.org/ns/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML"> <listBibl> <bibl> <publisher>Microsoft</publisher>. <date>2019</date>. <title level="m">Azure REST API Specifications. https://github.com/Azure/azurerest-api-specs Last accessed 2019-11-22. ``` -------------------------------- ### Run GROBID Lightweight Docker Image Source: https://grobid.readthedocs.io/en/latest/getting_started Starts the CPU-only lightweight container for high-performance, low-resource environments. Suitable for processing large volumes of PDFs where maximum accuracy is not the primary requirement. ```bash docker run --rm --init --ulimit core=0 -p 8070:8070 grobid/grobid:0.9.1-crf ``` -------------------------------- ### Configure DeLFT Global Parameters Source: https://grobid.readthedocs.io/en/latest/Configuration Defines the installation path and virtual environment for the DeLFT library. Required when using Deep Learning architectures. ```yaml delft: # delft installation path if Deep Learning architectures are used to implement one of the sequence labeling model, # embeddings are usually compiled as lmdb under delft/data (this parameter is ignored if only featured-engineered CRF are used) install: "../delft" pythonVirtualEnv: "../delft/env" ``` -------------------------------- ### Run CRF-only GROBID Container Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Starts a container configured for CRF-only model execution. ```bash docker run --rm --init --ulimit core=0 -p 8080:8070 -p 8081:8071 grobid/grobid:0.9.2-SNAPSHOT ``` -------------------------------- ### Labeling Keywords Source: https://grobid.readthedocs.io/en/latest/training/header Examples of tagging keywords and classification schemes, excluding generic labels. ```xml Mathematics Subject Classification: 83C15, 81U15, 81V80, 17B80, 81R12 PACS numbers: 02.30.Ik, 03.65.Fd Fd ``` ```xml 17 July 2007 Keywords: body mass index; weight change; coronary heart disease; follow-up study; Japanese Japanese ``` -------------------------------- ### Configure GROBID for DeLFT Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Define the DeLFT installation path and model architectures in the grobid.yaml configuration file. ```yaml delft: install: "../delft" ``` ```yaml models: - name: "citation" engine: "delft" architecture: "BidLSTM_CRF_FEATURES" ``` ```yaml models: - name: "citation" engine: "delft" architecture: "BERT_CRF" transformer: "allenai/scibert_scivocab_cased" ``` ```yaml delft: pythonVirtualEnv: /where/my/damned/python/virtualenv/is/ ``` -------------------------------- ### GET /api/version Source: https://grobid.readthedocs.io/en/latest/Grobid-service Retrieves the current version and revision of the running GROBID service. ```APIDOC ## GET /api/version ### Description Returns the current version and the revision of the running service. ### Method GET ### Endpoint /api/version ``` -------------------------------- ### Configure model engine and architecture Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Example configuration block to set the citation model to use the delft engine with BidLSTM_CRF_FEATURES architecture. ```yaml models: - name: "citation" engine: "delft" architecture: "BidLSTM_CRF_FEATURES" ``` -------------------------------- ### Example Prometheus metrics output Source: https://grobid.readthedocs.io/en/latest/Monitoring Sample of the plain-text format returned by the metrics endpoint, including JVM memory usage. ```text # HELP jvm_memory_bytes_used Used bytes of a given JVM memory area. # TYPE jvm_memory_bytes_used gauge jvm_memory_bytes_used{area="heap",} 1.34217728E8 ... ``` -------------------------------- ### Example of range citation callout text Source: https://grobid.readthedocs.io/en/latest/End-to-end-evaluation Shows a citation range in plain text format. ```text Recent studies have described the use of contrast material with high iodine content (370 mg I/ml or 400 mg I/ml) for coronary CT angiography (CCTA) (1 – 4). ``` -------------------------------- ### Resolve GLIBCXX library errors Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Commands to install libgcc and set the LD_PRELOAD environment variable to resolve library version conflicts. ```bash conda install libgcc export export LD_PRELOAD=$anaconda_path/lib/libstdc++.so.6.0.25 ./gradlew run ``` -------------------------------- ### GET /api/health Source: https://grobid.readthedocs.io/en/latest/Grobid-service Provides detailed health status and engine pool metrics. ```APIDOC ## GET /api/health ### Description Returns a JSON object with detailed status including initialization state, engine pool metrics, and configuration checks. Suitable for use as a readiness probe. ### Method GET ### Endpoint /api/health ### Response #### Success Response (200) - **body** (JSON) - Detailed status object including initialization state and engine pool metrics. #### Error Response (503) - **status** (integer) - Returns 503 when not ready to process requests. ``` -------------------------------- ### Docker machine inspection output Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Example JSON output from the docker-machine inspect command showing memory allocation. ```json { "ConfigVersion": 3, "Driver": { "IPAddress": "192.168.99.100", "MachineName": "default", "SSHUser": "docker", "SSHPort": 55933, "SSHKeyPath": "/Users/lfoppiano/.docker/machine/machines/default/id_rsa", "StorePath": "/Users/lfoppiano/.docker/machine", "SwarmMaster": false, "SwarmHost": "tcp://0.0.0.0:3376", "SwarmDiscovery": "", "VBoxManager": {}, "HostInterfaces": {}, "CPU": 1, "Memory": 2048, #<---- Memory: 2GB "DiskSize": 204800, "NatNicType": "82540EM", "Boot2DockerURL": "", "Boot2DockerImportVM": "", "HostDNSResolver": false, "HostOnlyCIDR": "192.168.99.1/24", "HostOnlyNicType": "82540EM", "HostOnlyPromiscMode": "deny", "NoShare": false, "DNSProxy": true, "NoVTXCheck": false }, "DriverName": "virtualbox", "HostOptions": { [...] }, "SwarmOptions": { [...] }, "AuthOptions": { [...] } }, "Name": "default" } ``` -------------------------------- ### Initialize Grobid Engine Source: https://grobid.readthedocs.io/en/latest/Grobid-java-library Demonstrates how to set up the Grobid environment and process a PDF header using the Engine class. ```java import org.grobid.core.*; import org.grobid.core.data.*; import org.grobid.core.factory.*; import org.grobid.core.mock.*; import org.grobid.core.utilities.*; import org.grobid.core.engines.Engine; ... String pdfPath = "mypdffile.pdf"; ... try { String pGrobidHome = "/Users/lopez/grobid/grobid-home"; // The GrobidHomeFinder can be instantiate without parameters to verify the grobid home in the standard // location (classpath, ../grobid-home, ../../grobid-home) // If the location is customised: GrobidHomeFinder grobidHomeFinder = new GrobidHomeFinder(Arrays.asList(pGrobidHome)); //The grobid yaml config file needs to be instantiate using the correct grobidHomeFinder or it will use the default //locations GrobidProperties.getInstance(grobidHomeFinder); System.out.println(">>>>>>>> GROBID_HOME="+GrobidProperties.getGrobidHome()); Engine engine = GrobidFactory.getInstance().createEngine(); // Biblio object for the result BiblioItem resHeader = new BiblioItem(); String tei = engine.processHeader(pdfPath, 1, resHeader); } catch (Exception e) { // If an exception is generated, print a stack trace e.printStackTrace(); } ``` -------------------------------- ### Annotate affiliation and address structure Source: https://grobid.readthedocs.io/en/latest/training/affiliation-address Example of tagging an affiliation block including department, institution, and address components. ```xml Institute of Materials Science of the Technische Universität Darmstadt, address:
Petersenstraße 23, D-64287 Darmstadt, Germany.
``` -------------------------------- ### Deploy the bundled demo Source: https://grobid.readthedocs.io/en/latest/PDF-TEI-Editor Clones the repository and runs the demo deployment script. ```bash git clone https://github.com/mpilhlt/pdf-tei-editor.git cd pdf-tei-editor npm run deploy .env.deploy.demo.localhost ``` -------------------------------- ### Parse patent citations with cURL Source: https://grobid.readthedocs.io/en/latest/Grobid-service Example command to send raw text to the processCitationPatentTXT endpoint for citation extraction. ```bash curl -X POST -d "input=In EP0123456B1 nothing interesting." localhost:8070/api/processCitationPatentTXT ``` -------------------------------- ### Process ST.36 Patent Citations via cURL Source: https://grobid.readthedocs.io/en/latest/Grobid-service Example request to the processCitationPatentST36 endpoint with raw citation inclusion enabled. ```bash curl --form input=@/home/lopez/grobid/grobid-core/src/test/resources/s/006271747.xml --form includeRawCitations=1 localhost:8070/api/processCitationPatentST36 ``` -------------------------------- ### TEI Output for Lightweight Article Processing Source: https://grobid.readthedocs.io/en/latest/Grobid-specialized-processes Example of the TEI XML structure produced by the lightweight article process, showing a single paragraph body. ```xml

ONCOLOGY REPORTS 49: 10, 2023 CORRIGENDUM

Oncol Rep 42: 509-520, 2019; DOI: 10.3892/or.2019.7175 Subsequently to the publication of the above article, and a Corrigendum that has already been published with the intention of showing corrected versions of Figs. 1 and 8 (DOI: 10.3892/or.2022.8348; published online on June 14, 2022), the authors have belatedly realized that the revisions made to Fig. 8 necessitated changes that should have been introduced into Fig. 9, although these were not attended to in the first corrigendum. Essentially, Fig. 8 was revised as the cell apoptosis and cell proliferation assays therein were poorly presented, which made the interpretation of the data difficult; Fig. 9 showed the fractions of apoptotic cells in the SKM-1 and THP-1 cell lines with lncENST00000444102 overexpression as this pertained to Fig. 8. A revised version of Fig. 9, presenting the analysis of the data shown in the revised version of Fig. 8, is shown opposite. In addition to the revision of Fig. 9, the sentence starting on p. 517, left-hand column, line 12 ["The flow cytometric apoptosis assay revealed that lncENST00000444102 overexpression promoted tumor cells to undergo apoptosis compared to control cells (P<0.001, Fig. 9)"] should be replaced with the following text, to reflect the change in the level of statistical significance: 'The flow cytometric apoptosis assay revealed that lncENST00000444102 overexpression promoted tumor cells to undergo apoptosis compared to control cells (P<0.01, Fig. 9)". Note that the revisions made to Figs. 8 and 9 in this paper have not had a major impact on the reported results, and do not affect the overall conclusions reported in the study. All the authors agree to the publication of this corrigendum. The authors are grateful to the Editor of Oncology Reports for allowing them the opportunity to publish this additional Corrigendum; furthermore, they apologize for any inconvenience caused to the readership of the Journal. Figure 9. The fraction of apoptotic cells in the SKM-1 and THP-1 cell lines with lncENST00000444102 overexpression. Data are presented as the means ± standard deviation. ** P<0.01 vs. the controls. This work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License.

``` -------------------------------- ### Annotate Article Title with Front Element Source: https://grobid.readthedocs.io/en/latest/training/segmentation Example showing the use of the element to encapsulate the article title and related content within the TEI XML structure. ```xml virus. But is the role of LGP2 in CD8 + T cell survival and function cell intrinsic A N T I V I R A L I M U N I T Y LGP2 rigs CD8 + T cells for survival or extrinsic? T cell receptor-and IFNβ-mediated signalling in CD8 + T ``` -------------------------------- ### Fulltext Processing with Page Range Source: https://grobid.readthedocs.io/en/latest/Grobid-service Process a PDF starting from a specific page index, where the first page is indexed as 1. ```bash curl -v --form input=@./thefile.pdf --form start=3 localhost:8070/api/processFulltextDocument ``` -------------------------------- ### Example of annotated range citation callout in XML Source: https://grobid.readthedocs.io/en/latest/End-to-end-evaluation Shows how range citations are annotated in NLM/JATS files, often omitting intermediary references. ```xml Recent studies have described the use of contrast material with high iodine content (370 mg I/ml or 400 mg I/ml) for coronary CT angiography (CCTA) (1–4). ``` -------------------------------- ### Handle Dropwizard 5 Configuration Parsing Error Source: https://grobid.readthedocs.io/en/latest/Upgrading Example of the error message generated when an unrecognized field like maxQueuedRequests is present in the configuration file. ```text io.dropwizard.configuration.ConfigurationParsingException: grobid.yaml has an error: * Unrecognized field at: server.maxQueuedRequests ... Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "maxQueuedRequests" (class io.dropwizard.core.server.DefaultServerFactory), not marked as ignorable ``` -------------------------------- ### Retrieve model archive with cURL Source: https://grobid.readthedocs.io/en/latest/Grobid-service Download a model archive using either POST or GET requests by specifying the model name and optional architecture. ```bash curl -v -X POST -d "model=date" "architecture=crf" localhost:8070/api/model > model.zip ``` ```bash curl -v -X GET localhost:8070/api/model?model=date > model.zip ``` -------------------------------- ### Navigate to GROBID Directory Source: https://grobid.readthedocs.io/en/latest/Grobid-batch Change the current working directory to the project root. ```bash > cd grobid/ ``` -------------------------------- ### XML date annotation structure Source: https://grobid.readthedocs.io/en/latest/training/date Example of the basic XML format used for date training files, where day, month, and year are tagged within a date element. ```xml Received August 17, 2005. ``` -------------------------------- ### Display Batch Command Help Source: https://grobid.readthedocs.io/en/latest/Grobid-batch View available batch commands and options for the GROBID core JAR. ```bash > java -Djava.library.path=grobid-home/lib/lin-64:grobid-home/lib/lin-64/jep -jar grobid-core/build/libs/grobid-core--onejar.jar -h ``` ```bash > java -Djava.library.path=grobid-home/lib/lin-64:grobid-home/lib/lin-64/jep -jar grobid-core/build/libs/grobid-core-0.9.0-onejar.jar -h ``` -------------------------------- ### Example of raw string citation encoding in NLM XML Source: https://grobid.readthedocs.io/en/latest/End-to-end-evaluation Demonstrates citations encoded as raw strings within mixed-citation tags, which can lead to false positives during evaluation. ```xml ... Piraña and PCluster: a modeling environment and cluster infrastructure for NONMEM. Keizer RJ, van Benten M, Beijnen JH, Schellens JH, Huitema AD. Comput Methods Programs Biomed. 2011;101(1):72–9. Holford, N. VPC, the visual predictive check—superiority to standard diagnostic (Rorschach) plots. In: PAGE 2005. 2005. ... ``` -------------------------------- ### Execute training and evaluation via Gradle Source: https://grobid.readthedocs.io/en/latest/Training-the-models-of-Grobid Run the training and evaluation process for a specific model using the Gradle task command. ```bash > ./gradlew train_ ``` ```bash > ./gradlew train_header ``` ```bash > ./gradlew train_name_header ``` -------------------------------- ### TEI Author Coordinates Source: https://grobid.readthedocs.io/en/latest/Coordinates-in-PDF Example of the coords attribute in TEI for a single bounding box. ```xml Ron J Keizer ``` -------------------------------- ### Extract PDF header metadata via cURL Source: https://grobid.readthedocs.io/en/latest/Grobid-service Use these commands to extract header information from a PDF file. The first example returns TEI XML, while the second returns BibTeX format. ```bash curl -v -H "Accept: application/xml" --form input=@./thefile.pdf localhost:8070/api/processHeaderDocument ``` ```bash curl -v -H "Accept: application/x-bibtex" --form input=@./thefile.pdf localhost:8070/api/processHeaderDocument ``` -------------------------------- ### Configure Model Preloading Source: https://grobid.readthedocs.io/en/latest/Configuration Toggle model preloading to balance startup time against memory usage. ```yaml # for **service only**: how to load the models, # false -> models are loaded when needed, avoiding putting in memory useless models (only in case of CRF) but slow down # significantly the service at first call # true -> all the models are loaded into memory at the server startup (default), slow the start of the services # and models not used will take some more memory (only in case of CRF), but server is immediately warm and ready modelPreload: true ``` -------------------------------- ### Verify OTLP Metrics Startup Log Source: https://grobid.readthedocs.io/en/latest/Monitoring Check the application logs for this confirmation line to ensure metrics reporting is active. ```text OtlpMetricsReporter: OTLP metrics push enabled -> https://otlp-gateway-prod-eu-central-0.grafana.net/otlp (http/protobuf), every 60s, service.name=grobid-service ``` -------------------------------- ### Rosetta error message Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Example of the error message encountered when the platform flag is missing on ARM64. ```text rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2 ``` -------------------------------- ### Build GROBID with Gradle Source: https://grobid.readthedocs.io/en/latest/Install-Grobid Standard command to clean and build the GROBID project from the main directory. ```bash > ./gradlew clean build ``` -------------------------------- ### Initialize a new model programmatically Source: https://grobid.readthedocs.io/en/latest/Notes-grobid-developers Load the YAML configuration and register the model with GrobidProperties. ```java NewModuleConfiguration newModuleConfiguration = null; try { ObjectMapper mapper = new ObjectMapper(new YAMLFactory()); newModuleConfiguration = mapper.readValue(new File("resources/config/config.yml"), NewModuleConfiguration.class); } catch(Exception e) { LOGGER.error("The config file does not appear valid, see resources/config/config.yml", e); } if (newModuleConfiguration != null && newModuleConfiguration.getModel() != null) GrobidProperties.getInstance().addModel(newModuleConfiguration.getModel()); LibraryLoader.load(); ``` -------------------------------- ### GPL FAQ on Mere Aggregation Source: https://grobid.readthedocs.io/en/latest/Frequently-asked-questions Excerpt regarding the distribution of GPL and non-GPL software in an aggregate. ```text The GPL permits you to create and distribute an aggregate, even when the licenses of the other software are nonfree or GPL-incompatible. The only condition is that you cannot release the aggregate under a license that prohibits users from exercising rights that each program's individual license would grant them. ``` -------------------------------- ### Tagging Years with Suffixes Source: https://grobid.readthedocs.io/en/latest/training/Bibliographical-references Example of tagging a year that includes a letter suffix, common in Harvard-style references. ```xml Gavazzi G., Piertini D., Boselli A., Tuffs R., 1996c, A&AS, 120, 489(Paper I) ``` -------------------------------- ### Identify GLIBCXX dependency error Source: https://grobid.readthedocs.io/en/latest/Deep-Learning-models Example of an ImportError indicating a missing GLIBCXX version required by TensorFlow. ```text ! ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/Luca/.conda/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so) ``` -------------------------------- ### TEI biblStruct consolidation attributes Source: https://grobid.readthedocs.io/en/latest/Consolidation Example of the status and source attributes applied to a biblStruct element after consolidation. ```xml ... ``` -------------------------------- ### Run Grobid Docker with custom configuration Source: https://grobid.readthedocs.io/en/latest/Grobid-docker Mounts a local grobid.yaml file to the container using an absolute path. ```bash docker run --rm --gpus all --init --ulimit core=0 -p 8080:8070 -p 8081:8071 -v /home/lopez/grobid/grobid-home/config/grobid.yaml:/opt/grobid/grobid-home/config/grobid.yaml:ro grobid/grobid:0.9.1 ``` -------------------------------- ### Generate Training Data with createTraining Source: https://grobid.readthedocs.io/en/latest/Grobid-batch Executes the createTraining command to generate training data from PDF files. ```bash > java -Xmx4G -Djava.library.path=grobid-home/lib/lin-64:grobid-home/lib/lin-64/jep -jar grobid-core/build/libs/grobid-core-0.9.0-onejar.jar -gH grobid-home -dIn /path/to/input/directory -dOut /path/to/output/directory -exe createTraining ``` -------------------------------- ### Train specialized header model Source: https://grobid.readthedocs.io/en/latest/Grobid-specialized-processes Command to initiate training for the article/light-ref header model. ```bash ./gradlew train_header_article_light_ref ``` -------------------------------- ### Execute Gradle Release Source: https://grobid.readthedocs.io/en/latest/Notes-grobid-developers Run this command from a release branch to initiate the versioning process and create the release commits and tag. ```bash git checkout -b release/ ./gradlew release \ -Prelease.useAutomaticVersion=true \ -Prelease.releaseVersion= \ -Prelease.newVersion=-SNAPSHOT ``` -------------------------------- ### Generate Blank Training Data with createTrainingBlank Source: https://grobid.readthedocs.io/en/latest/Grobid-batch Executes the createTrainingBlank command to generate a blank TEI file and default feature file from PDFs. ```bash > java -Xmx4G -Djava.library.path=grobid-home/lib/lin-64:grobid-home/lib/lin-64/jep -jar grobid-core/build/libs/grobid-core-0.9.0-onejar.jar -gH grobid-home -dIn /path/to/input/directory -dOut /path/to/output/directory -exe createTrainingBlank ``` -------------------------------- ### JSON Bounding Box Structure Source: https://grobid.readthedocs.io/en/latest/Coordinates-in-PDF Example of the pos attribute in JSON, representing a list of bounding boxes for an entity. ```json "pos": [{ "p": 1, "x": 20, "y": 20, "h": 10, "w": 30 }, { "p": 1, "x": 30, "y": 20, "h": 10, "w": 30 } ] ``` -------------------------------- ### JSON Page Dimensions Structure Source: https://grobid.readthedocs.io/en/latest/Coordinates-in-PDF Example of the pages attribute in JSON, defining the height and width of document pages. ```json "pages": [ {"page_height":792.0, "page_width":612.0}, {"page_height":792.0, "page_width":612.0}, {"page_height":792.0, "page_width":612.0} ] ``` -------------------------------- ### Build Java artifacts Source: https://grobid.readthedocs.io/en/latest/Notes-grobid-developers Checkout the specific release tag and build the project to generate JAR files in subproject build directories. ```bash git checkout ./gradlew clean build ``` -------------------------------- ### Bibliographical Reference Markers Source: https://grobid.readthedocs.io/en/latest/training/fulltext Examples of tagging bibliographical references. Note that brackets and parentheses are included within the ref tags. ```xml Harbaugh and Harbaugh [7] ``` ```xml in Lolle et al. 1 ``` ```xml

The clinical entity of cervical flexion myelopa- thy proposed in the 1960s by Reid [16,17] and Breig et al [2,3] has been neglected for a long time. ... groups of males and females controls.

``` ```xml by Greve et al. [1994] and Koch et al. [1994] ``` ```xml [Whitham, 1954; Hogg and Pritchard, 2004] ``` ```xml [Abramowitz and Stegun, 1964, see pp. 559 – 562] ``` ```xml LEWANDOWSKI (1982b) has reported ``` ```xml (STANCZYKOWSKA, 1977) ``` ```xml (e.g., Bryant & Goswami, 1987) by Smith and Matsoukas [1998] and Khelifa and Hill [2006b] ``` ```xml by Gal [1979] and illustrated further recently by Draper and Lund [2004] ``` ```xml (ref. 1) ``` ```xml This was established in Thompson v. Lochert (1997) [114] in which ``` ```xml such as receptors for IL-1β (REFS 64,65), TNF 66 and IFNγ 67 . ``` -------------------------------- ### Verify Docker image runtime version Source: https://grobid.readthedocs.io/en/latest/Notes-grobid-developers Run the container and query the version API to confirm the application reports the correct version. ```bash docker run --rm -d --name g-test -p 8070:8070 grobid/grobid:-crf sleep 30 curl -s http://localhost:8070/api/version # expected: {"version":"","revision":""} docker stop g-test ``` -------------------------------- ### TEI Bibliographical Reference Coordinates Source: https://grobid.readthedocs.io/en/latest/Coordinates-in-PDF Example of the coords attribute in TEI containing multiple bounding boxes separated by semicolons. ```xml ``` -------------------------------- ### Generate Java bindings with JAXB Source: https://grobid.readthedocs.io/en/latest/TEI-encoding-of-results Use the xjc tool with the -extension flag to compile GROBID's XSD schemas into Java classes. ```bash > xjc -d generated -extension ~/grobid/grobid-home/schemas/xsd/simpleMode.xsd ~/grobid/grobid-home/schemas/xsd/Grobid.xsd parsing a schema... compiling a schema... org/tei_c/ns/_1/Abstract.java org/tei_c/ns/_1/AddrLine.java org/tei_c/ns/_1/Address.java org/tei_c/ns/_1/Affiliation.java org/tei_c/ns/_1/Analytic.java org/tei_c/ns/_1/Anchor.java org/tei_c/ns/_1/Author.java org/tei_c/ns/_1/Availability.java org/tei_c/ns/_1/Back.java etc. ``` -------------------------------- ### TEI Facsimile Surface Definition Source: https://grobid.readthedocs.io/en/latest/Coordinates-in-PDF Example of the facsimile element in TEI, defining page dimensions using surface attributes. ```xml ... ... ``` -------------------------------- ### Search for version strings Source: https://grobid.readthedocs.io/en/latest/Notes-grobid-developers Perform a recursive grep to identify remaining references to the previous version before finalizing a release. ```bash grep -rn '' \ --include='*.md' --include='*.gradle' --include='*.java' \ --include='*.kt' --include='*.cff' --include='*.yml' \ --include='*.properties' . ``` -------------------------------- ### Configure Gradle Proxy Settings Source: https://grobid.readthedocs.io/en/latest/Install-Grobid Proxy configuration parameters to be added to the grobid/gradle.properties file. ```properties systemProp.http.proxyHost=host systemProp.http.proxyPort=port systemProp.http.proxyUser=username systemProp.http.proxyPassword=password systemProp.https.proxyHost=host systemProp.https.proxyPort=port systemProp.https.proxyUser=username systemProp.https.proxyPassword=password ``` -------------------------------- ### POST /api/createTraining Source: https://grobid.readthedocs.io/en/latest/Grobid-service Generates training data for Grobid models from an input PDF document. ```APIDOC ## POST /api/createTraining ### Description Generate the training data for the grobid models for a PDF document provided in input. The service returns a ZIP archive with the training data in TEI XML format. ### Method POST ### Endpoint /api/createTraining ### Parameters #### Request Body - **input** (file) - Required - A PDF document - **flavor** (string) - Optional - The name of the flavor used to create the training data ### Request Example curl --location 'http://localhost:8070/api/createTraining' --form 'input=@"PATH_DOCUMENT"' --output output_name.zip ### Response #### Success Response (200) - application/zip archive #### Error Responses - 400: Wrong request, missing or invalid model name, invalid architecture name, missing header - 500: Internal service error ``` -------------------------------- ### Configure Grobid Maven Project Source: https://grobid.readthedocs.io/en/latest/Grobid-java-library Configuration and testing commands for a Maven-based Grobid integration project. ```properties grobid_example.pGrobidHome=/Users/lopez/grobid/grobid-home ``` ```bash > mvn test ``` -------------------------------- ### Deploy Monitoring Stack with Docker Compose Source: https://grobid.readthedocs.io/en/latest/Monitoring Use this configuration to launch GROBID, Prometheus, and Grafana containers simultaneously. ```yaml services: grobid: image: grobid/grobid:0.9.1 ports: - "8070:8070" # REST API - "8071:8071" # admin connector (metrics) prometheus: image: prom/prometheus:latest ports: - "9090:9090" volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro grafana: image: grafana/grafana:latest ports: - "3000:3000" depends_on: - prometheus ``` -------------------------------- ### Integrate CRF++ library Source: https://grobid.readthedocs.io/en/latest/Recompiling-and-integrating-CRF-libraries Copy the CRF++ library and its dependencies to the Grobid home directory. ```bash > cp ld-linux-x86-64.so.2 libwapiti.so libc.so.6 libgcc_s.so.1 libm.so.6 libpthread.so.0 libstdc++.so.6 GROBID-ROOT-DIRECTORY/grobid-home/lib/lin- ``` -------------------------------- ### Tagging Non-Segmented Fields Source: https://grobid.readthedocs.io/en/latest/training/Bibliographical-references Example of tagging journal titles and volumes that appear without spacing, ensuring no extra whitespace is introduced. ```xml D. Foo, P. Bar, Phys. Rev. D95, 34(2017) ``` -------------------------------- ### Configure biblio-glutton in grobid.yaml Source: https://grobid.readthedocs.io/en/latest/Consolidation YAML configuration snippet to enable the glutton service and specify its URL and timeout settings. ```yaml consolidation: service: "glutton" glutton: url: "http://localhost:8080" timeoutSec: 60 ``` -------------------------------- ### Configure Gradle Repository Source: https://grobid.readthedocs.io/en/latest/Grobid-java-library Add the GROBID repository to your build.gradle file. ```groovy repositories { maven { url "https://grobid.s3.eu-west-1.amazonaws.com/repo/" } } ``` -------------------------------- ### Train specialized segmentation model Source: https://grobid.readthedocs.io/en/latest/Grobid-specialized-processes Command to initiate training for the article/light segmentation model. ```bash ./gradlew train_segmentation_article_light ```