### Start the Server

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Starts the ScienceBeam Parser server. The server typically runs on port 8080.

```bash
make dev-start
```

--------------------------------

### Start the Server with Auto Reload (No Debug Logging)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Starts the server with auto-reloading but without debug logging. Provides a balance between development convenience and log verbosity.

```bash
make dev-start-no-debug-logging-auto-reload
```

--------------------------------

### Start the Server in Debug Mode

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Starts the server with debug logging and auto-reloading enabled. Useful for development and debugging.

```bash
make dev-debug
```

--------------------------------

### Install ScienceBeam Parser

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/python_library.md

Install the ScienceBeam Parser library with optional dependencies for Delft and CPU support using pip.

```bash
pip install sciencebeam-parser[delft,cpu]
```

--------------------------------

### Install SQLite and LMDB Development Libraries

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Installs necessary development libraries for SQLite and LMDB on Ubuntu-based systems. Required for certain database backends.

```bash
apt-get install libsqlite3-dev liblmdb-dev
```

--------------------------------

### Create Development Virtual Environment

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Creates a Python virtual environment and installs project dependencies. This is a standard step for setting up the development environment.

```bash
make dev-venv
```

--------------------------------

### Start ScienceBeam Parser Server via CLI

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/python_library.md

Start the ScienceBeam Parser REST API server on port 8080 using the command line.

```bash
python -m sciencebeam_parser.service.server --port=8080
```

--------------------------------

### Install Tesseract OCR and Dependencies

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Installs the Tesseract OCR engine, its English language pack, and development libraries on Ubuntu. Required for optical character recognition.

```bash
apt-get install tesseract-ocr-eng libtesseract-dev libleptonica-dev
```

--------------------------------

### OCR with Keras Example

Source: https://github.com/elifepathways/sciencebeam-parser/wiki/Related-Projects

This Keras example demonstrates a practical application of image OCR using deep learning. It's useful for understanding how to implement OCR models with Keras.

```python
from __future__ import absolute_import
from __future__ import print_function

import numpy as np
import argparse
import random
import sys

from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers.embeddings import Embedding
from keras.layers.recurrent import LSTM
from keras.layers.normalization import BatchNormalization
from keras.utils import np_utils
from collections import Counter

# Simple image OCR example using Keras
# Based on https://github.com/fchollet/keras/blob/master/examples/image_ocr.py

def build_model(max_features, embed_size, output_size):
    model = Sequential()
    model.add(Embedding(max_features, embed_size, input_length=max_features))
    model.add(LSTM(output_size, return_sequences=True))
    model.add(Dropout(0.2))
    model.add(BatchNormalization())
    model.add(LSTM(output_size, return_sequences=False))
    model.add(Dropout(0.2))
    model.add(BatchNormalization())
    model.add(Dense(output_size, activation='sigmoid'))
    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
    return model

def main():
    parser = argparse.ArgumentParser(description='OCR Example')
    parser.add_argument('--epochs', type=int, default=5, metavar='N', 
                        help='number of epoch to train (default: 5)')
    parser.add_argument('--batch-size', type=int, default=32, metavar='N', 
                        help='batch size for training (default: 32)')
    args = parser.parse_args()

    print('Loading data...')
    # Dummy data for demonstration
    X_train = np.random.rand(100, 100)
    y_train = np.random.randint(2, size=(100, 1))
    X_test = np.random.rand(20, 100)
    y_test = np.random.randint(2, size=(20, 1))

    max_features = 1000
    embed_size = 128
    output_size = 1

    model = build_model(max_features, embed_size, output_size)

    print('Training model...')
    model.fit(X_train, y_train, batch_size=args.batch_size, epochs=args.epochs, 
              verbose=1, validation_split=0.1)

    print('Evaluating model...')
    score = model.evaluate(X_test, y_test, verbose=0)
    print('Test score:', score[0])
    print('Test accuracy:', score[1])

if __name__ == '__main__':
    main()

```

--------------------------------

### Convert Word Document to PDF

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

This example demonstrates converting a Word document (`.docx`) to PDF using the API. The content type is inferred from the file extension when not explicitly provided.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-office-open.docx;filename=test-data/minimal-office-open.docx" \
    --silent "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

--------------------------------

### Start ScienceBeam Parser Server with Python API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/python_library.md

Initialize and run the ScienceBeam Parser REST API server on port 8080 using the Python API.

```python
from sciencebeam_parser.config.config import AppConfig
from sciencebeam_parser.resources.default_config import DEFAULT_CONFIG_FILE
from sciencebeam_parser.service.server import create_app


config = AppConfig.load_yaml(DEFAULT_CONFIG_FILE)
app = create_app(config)
app.run(port=8080, host='127.0.0.1', threaded=True)
```

--------------------------------

### GROBID Header Model XML Fragment

Source: https://github.com/elifepathways/sciencebeam-parser/wiki/GROBID-Training-Data-Generation

An example XML fragment generated by the GROBID header model for a document title. It shows how the title might be represented with potential line breaks.

```xml
<docTitle>
  <titlePart>Title<lb/> Sub-tomogram averaging in RELION<lb/></titlePart>
</docTitle>
```

--------------------------------

### Convert Word Document to PDF using /convert API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Uses the `/convert` API to convert a Word document (`.docx`) to PDF by setting the `Accept` header to `application/pdf`. The output is saved to `example.pdf`.

```bash
curl --fail --show-error --silent \
    --header 'Accept: application/pdf' \
    --form "file=@test-data/minimal-office-open.docx;filename=test-data/minimal-office-open.docx" \
    --output "example.pdf" \
    "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

--------------------------------

### Generate TEI Training Data with Directory Structure

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates TEI training data and organizes the output into a directory structure based on model and file type. Supports pre-annotation.

```bash
python -m sciencebeam_parser.training.cli.generate_data \
    --use-model \
    --use-directory-structure \
    --source-path="test-data/*.pdf" \
    --output-path="./data/generated-training-data"
```

--------------------------------

### Convert Document to TEI XML using /convert API and Accept Header

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Uses the `/convert` API to transform a PDF document into TEI XML by specifying the `Accept` header. The `first_page` and `last_page` parameters specify the document range.

```bash
curl --fail --show-error \
    --header 'Accept: application/tei+xml' \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

--------------------------------

### Run Tests

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Executes the project's test suite, including linting and pytest. Ensures code quality and correctness.

```bash
make dev-test
```

--------------------------------

### Generate Data for Name Citation Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Use this command to generate training data for the name citation model. The paths for TEI sources and the Delft output file are crucial.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="name_citation" \
    --tei-source-path="data/generated-training-data/name/citation/corpus/*.tei.xml" \
    --delft-output-path \
    "./data/generated-training-data/delft/name/citation/corpus/name.data"
```

--------------------------------

### Run Docker Container with CV and OCR Models Enabled

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Run a Docker container with Computer Vision and OCR models enabled using environment variables. This image is tagged with `-cv`.

```bash
docker run --rm \
    -p 8070:8070 \
    --env SCIENCEBEAM_PARSER__PROCESSORS__FULLTEXT__USE_CV_MODEL=true \
    --env SCIENCEBEAM_PARSER__PROCESSORS__FULLTEXT__USE_OCR_MODEL=true \
    ghcr.io/elifepathways/sciencebeam-parser:latest-cv
```

--------------------------------

### Generate Delft Training Data for Header Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates Delft-compatible training data for the header model. This command requires annotated TEI files and raw source documents.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="header" \
    --tei-source-path="data/generated-training-data/header/corpus/tei/*.tei.xml" \
    --raw-source-path="data/generated-training-data/header/corpus/raw/" \
    --delft-output-path="./data/generated-training-data/delft/header/corpus/header.data"
```

--------------------------------

### Generate Data for Citation Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

This command is used to generate training data for the citation model. Ensure that the TEI source path and the Delft output path are accurately specified.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="citation" \
    --tei-source-path="data/generated-training-data/citation/corpus/*.tei.xml" \
    --delft-output-path \
    "./data/generated-training-data/delft/citation/corpus/citation.data"
```

--------------------------------

### Generate Delft Training Data for Segmentation Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates Delft-compatible training data for the segmentation model from annotated TEI and raw source files. Ensure the TEI and raw source paths are correctly specified.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="segmentation" \
    --tei-source-path="data/generated-training-data/segmentation/corpus/tei/*.tei.xml" \
    --raw-source-path="data/generated-training-data/segmentation/corpus/raw/" \
    --delft-output-path="./data/generated-training-data/delft/segmentation/corpus/segmentation.data"
```

--------------------------------

### Generate TEI Training Data

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates initial TEI training data from PDF documents. This is the first step in preparing data for sequence models.

```bash
python -m sciencebeam_parser.training.cli.generate_data \
    --source-path="test-data/*.pdf" \
    --output-path="./data/generated-training-data"
```

--------------------------------

### Submit Document to Header Model API (XML Output)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the header model API, requesting XML output for the first and last pages. Demonstrates specifying page ranges and output format.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/models/header?first_page=1&last_page=1&output_format=xml"
```

--------------------------------

### Generate TEI Training Data with Pre-annotation

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates TEI training data using pre-trained models for annotation. This can improve the quality of the initial training data.

```bash
python -m sciencebeam_parser.training.cli.generate_data \
    --use-model \
    --source-path="test-data/*.pdf" \
    --output-path="./data/generated-training-data"
```

--------------------------------

### Generate Delft Training Data for Figure Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates Delft-compatible training data specifically for the figure model. It requires paths to TEI and raw source files.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="figure" \
    --tei-source-path="data/generated-training-data/figure/corpus/tei/*.tei.xml" \
    --raw-source-path="data/generated-training-data/figure/corpus/raw/" \
    --delft-output-path="./data/generated-training-data/delft/figure/corpus/figure.data"
```

--------------------------------

### Generate Data for Reference Segmenter Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Use this command to generate training data for the reference segmenter model. Specify the TEI source path and the desired Delft output path.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="reference_segmenter" \
    --tei-source-path="data/generated-training-data/reference-segmenter/corpus/tei/*.tei.xml" \
    --raw-source-path="data/generated-training-data/reference-segmenter/corpus/raw/" \
    --delft-output-path="./data/generated-training-data/delft/reference-segmenter/corpus/reference-segmenter.data"
```

--------------------------------

### Request Specific Fields with includes Parameter

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Use the `includes` parameter to specify desired fields like `title` and `abstract` to reduce processing time. The output may still contain more fields than requested.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/convert?includes=title,abstract"
```

--------------------------------

### Generate Data for Name Header Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Command to generate training data specifically for the name header model. Provide the correct TEI source and output file paths.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="name_header" \
    --tei-source-path="data/generated-training-data/name/header/corpus/*.tei.xml" \
    --delft-output-path \
    "./data/generated-training-data/delft/name/header/corpus/name.data"
```

--------------------------------

### Submit Document to ProcessHeaderDocument API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the processHeaderDocument API, which is compatible with GROBID's API for processing front matter. The default response is TEI XML.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processHeaderDocument?first_page=1&last_page=1"
```

--------------------------------

### Submit Document to PDFAlto API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the PDFAlto API endpoint for processing. Requires the server to be running.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/pdfalto"
```

--------------------------------

### Submit a sample document to the full text asset document api

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the full text asset document API. This endpoint returns a ZIP archive containing the TEI XML document and associated assets like figure images. JATS XML within the ZIP can be requested via the Accept header. The returned content type is always application/zip.

```APIDOC
## POST /api/processFulltextAssetDocument

### Description
Submits a PDF document for processing, returning a ZIP archive with TEI XML and assets. Supports requesting JATS XML within the ZIP via the Accept header.

### Method
POST

### Endpoint
/api/processFulltextAssetDocument

### Parameters
#### Query Parameters
- **first_page** (integer) - Required - The first page to process.
- **last_page** (integer) - Required - The last page to process.

#### Request Body
- **file** (file) - Required - The PDF file to process. The filename should be provided.

### Request Example
```bash
curl --fail --show-error \
    --output "example-tei-xml-and-assets.zip" \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processFulltextAssetDocument?first_page=1&last_page=1"
```

### Response
#### Success Response (200)
- **Content-Type**: application/zip

#### Response Example
(ZIP archive containing TEI XML and assets, or JATS XML and assets)
```zip
(binary zip content)
```
```

--------------------------------

### Generate Delft Training Data for Table Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates Delft-compatible training data for the table model. Ensure the correct TEI and raw source paths are provided for this process.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="table" \
    --tei-source-path="data/generated-training-data/table/corpus/tei/*.tei.xml" \
    --raw-source-path="data/generated-training-data/table/corpus/raw/" \
    --delft-output-path="./data/generated-training-data/delft/table/corpus/table.data"
```

--------------------------------

### Generate Delft Training Data for Fulltext Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

Generates Delft-compatible training data for the fulltext model. This process involves using annotated TEI and raw source files.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="fulltext" \
    --tei-source-path="data/generated-training-data/fulltext/corpus/tei/*.tei.xml" \
    --raw-source-path="data/generated-training-data/fulltext/corpus/raw/" \
    --delft-output-path="./data/generated-training-data/delft/fulltext/corpus/fulltext.data"
```

--------------------------------

### Submit a sample document to the /convert api

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

A general conversion API endpoint that converts PDF documents to a semantic representation. By default, it returns JATS XML. The response format can be influenced using the Accept HTTP header, and it can also handle Word to PDF conversion.

```APIDOC
## POST /api/convert

### Description
Converts PDF documents to a semantic representation. Supports various output formats via the Accept header, including JATS XML (default) and TEI XML. Can also convert Word documents to PDF.

### Method
POST

### Endpoint
/api/convert

### Parameters
#### Query Parameters
- **first_page** (integer) - Required - The first page to process.
- **last_page** (integer) - Required - The last page to process.

#### Request Body
- **file** (file) - Required - The input file to convert (e.g., PDF, DOCX). The filename should be provided.

### Request Example (PDF to JATS XML)
```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

### Request Example (PDF to TEI XML)
```bash
curl --fail --show-error \
    --header 'Accept: application/tei+xml' \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

### Request Example (DOCX to PDF)
```bash
curl --fail --show-error --silent \
    --header 'Accept: application/pdf' \
    --form "file=@test-data/minimal-office-open.docx;filename=test-data/minimal-office-open.docx" \
    --output "example.pdf" \
    "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

### Response
#### Success Response (200)
- **Content-Type**: application/xml (for JATS/TEI XML), application/pdf (for Word to PDF conversion)

#### Response Example (JATS XML)
```xml
<jats:root xmlns:jats="http://jats.nlm.nih.gov">
  <!-- JATS XML content -->
</jats:root>
```
```

--------------------------------

### Parse Multiple Files with Python API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/python_library.md

Parse PDF documents to TEI XML using the ScienceBeamParser class and sessions for managing intermediate files.

```python
from sciencebeam_parser.resources.default_config import DEFAULT_CONFIG_FILE
from sciencebeam_parser.config.config import AppConfig
from sciencebeam_parser.utils.media_types import MediaTypes
from sciencebeam_parser.app.parser import ScienceBeamParser


config = AppConfig.load_yaml(DEFAULT_CONFIG_FILE)

# the parser contains all of the models
sciencebeam_parser = ScienceBeamParser.from_config(config)

# a session provides a scope and temporary directory for intermediate files
# it is recommended to create a separate session for every document
with sciencebeam_parser.get_new_session() as session:
    session_source = session.get_source(
        'test-data/minimal-example.pdf',
        MediaTypes.PDF
    )
    converted_file = session_source.get_local_file_for_response_media_type(
        MediaTypes.TEI_XML
    )
    # Note: the converted file will be in the temporary directory of the session
    print('converted file:', converted_file)
```

--------------------------------

### Submit Document to Full Text Asset API (TEI XML Zip)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the full text asset API to receive a ZIP file containing TEI XML and associated assets. The `first_page` and `last_page` parameters control the processing range.

```bash
curl --fail --show-error \
    --output "example-tei-xml-and-assets.zip" \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processFulltextAssetDocument?first_page=1&last_page=1"
```

--------------------------------

### Submit a sample document to the references api

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the references API for processing. This endpoint focuses on extracting references. Similar to the full text API, it supports TEI XML (default) or JATS XML via the Accept header, with a returned content type of application/xml.

```APIDOC
## POST /api/processReferences

### Description
Submits a PDF document to extract references. Supports requesting TEI XML (default) or JATS XML via the Accept header.

### Method
POST

### Endpoint
/api/processReferences

### Parameters
#### Query Parameters
- **first_page** (integer) - Required - The first page to process.
- **last_page** (integer) - Required - The last page to process.

#### Request Body
- **file** (file) - Required - The PDF file to process. The filename should be provided.

### Request Example
```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processReferences?first_page=1&last_page=100"
```

### Response
#### Success Response (200)
- **Content-Type**: application/xml

#### Response Example
(TEI XML or JATS XML, depending on Accept header)
```xml
<TEI xmlns="http://www.tei-c.org/ns/1.0">
  <!-- TEI XML content for references -->
</TEI>
```
```

--------------------------------

### Submit Document to Full Text API (Default TEI XML)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the full text API to receive TEI XML. The `first_page` and `last_page` parameters control the processing range.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processFulltextDocument?first_page=1&last_page=1"
```

--------------------------------

### Generate Data for Affiliation Address Model

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/doc/training.md

This command generates training data for the affiliation address model. Ensure the TEI source path and Delft output path are correctly set.

```bash
python -m sciencebeam_parser.training.cli.generate_delft_data \
    --model-name="affiliation_address" \
    --tei-source-path="data/generated-training-data/affiliation-address/corpus/*.tei.xml" \
    --delft-output-path \
    "./data/generated-training-data/delft/affiliation-address/corpus/affiliation-address.data"
```

--------------------------------

### Submit Document to References API (Default TEI XML)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the references API to extract only reference information, returning TEI XML by default. The `first_page` and `last_page` parameters define the range for reference extraction.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processReferences?first_page=1&last_page=100"
```

--------------------------------

### Run ScienceBeam Parser Docker Container

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Run a Docker container for the ScienceBeam Parser, mapping port 8070 for access.

```bash
docker run --rm \
    -p 8070:8070 \
    ghcr.io/elifepathways/sciencebeam-parser
```

--------------------------------

### Pull ScienceBeam Parser Docker Image

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Pull the latest ScienceBeam Parser Docker image from the container registry.

```bash
docker pull ghcr.io/elifepathways/sciencebeam-parser
```

--------------------------------

### Submit Document to References API (JATS XML)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the references API requesting JATS XML using the `Accept` header. The `first_page` and `last_page` parameters define the range for reference extraction.

```bash
curl --fail --show-error \
    --header 'Accept: application/vnd.jats+xml' \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processReferences?first_page=1&last_page=100"
```

--------------------------------

### Convert Document to JATS XML using /convert API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Uses the `/convert` API to transform a PDF document into JATS XML by default. The `first_page` and `last_page` parameters specify the document range.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/convert?first_page=1&last_page=1"
```

--------------------------------

### Submit Document to Full Text Asset API (JATS XML Zip)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the full text asset API requesting a ZIP file with JATS XML and assets using the `Accept` header. The `first_page` and `last_page` parameters control the processing range.

```bash
curl --fail --show-error \
    --header 'Accept: application/vnd.jats+xml+zip' \
    --output "example-jats-xml-and-assets.zip" \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processFulltextAssetDocument?first_page=1&last_page=1"
```

--------------------------------

### Submit Document to ProcessHeaderDocument API (JATS Output)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the processHeaderDocument API requesting JATS XML output using the Accept header. Demonstrates content negotiation for different XML formats.

```bash
curl --fail --show-error \
    --header 'Accept: application/vnd.jats+xml' \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processHeaderDocument?first_page=1&last_page=1"
```

--------------------------------

### Submit a sample document to the full text document api

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the full text document API for processing. The default response is TEI XML, but JATS XML can be requested via the Accept header. The returned content type is always application/xml.

```APIDOC
## POST /api/processFulltextDocument

### Description
Submits a PDF document for full text processing. Supports requesting TEI XML (default) or JATS XML via the Accept header.

### Method
POST

### Endpoint
/api/processFulltextDocument

### Parameters
#### Query Parameters
- **first_page** (integer) - Required - The first page to process.
- **last_page** (integer) - Required - The last page to process.

#### Request Body
- **file** (file) - Required - The PDF file to process. The filename should be provided.

### Request Example
```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processFulltextDocument?first_page=1&last_page=1"
```

### Response
#### Success Response (200)
- **Content-Type**: application/xml

#### Response Example
(TEI XML or JATS XML, depending on Accept header)
```xml
<TEI xmlns="http://www.tei-c.org/ns/1.0">
  <!-- TEI XML content -->
</TEI>
```
```

--------------------------------

### Submit Document to Full Text API (JATS XML)

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the full text API requesting JATS XML using the `Accept` header. The `first_page` and `last_page` parameters control the processing range.

```bash
curl --fail --show-error \
    --header 'Accept: application/vnd.jats+xml' \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/processFulltextDocument?first_page=1&last_page=1"
```

--------------------------------

### Auto-annotation XML Adjustment

Source: https://github.com/elifepathways/sciencebeam-parser/wiki/GROBID-Training-Data-Generation

Demonstrates how auto-annotation modifies the XML fragment when the initial GROBID output is inaccurate. Text identified as 'Title' is re-labeled as a 'note'.

```xml
<note type="other">Title<lb/></note> <docTitle><titlePart>Sub-tomogram averaging in RELION<lb/></titlePart>
</docTitle>
```

--------------------------------

### Submit Document to Name-Header API

Source: https://github.com/elifepathways/sciencebeam-parser/blob/main/README.md

Submits a PDF document to the name-header API endpoint. Similar to the header model API, but specific to name-header extraction.

```bash
curl --fail --show-error \
    --form "file=@test-data/minimal-example.pdf;filename=test-data/minimal-example.pdf" \
    --silent "http://localhost:8080/api/models/name-header?first_page=1&last_page=1&output_format=xml"
```

--------------------------------

### Zhang-Shasha Tree Edit Distance in Python

Source: https://github.com/elifepathways/sciencebeam-parser/wiki/Related-Projects

This Python implementation of the Zhang-Shasha algorithm is useful for calculating the edit distance between two trees. It's often applied in document structure analysis.

```python
from __future__ import print_function

import sys

# Based on https://github.com/timtadh/zhang-shasha

def levenshtein(s1, s2):
    if len(s1) < len(s2):
        return levenshtein(s2, s1)

    if len(s2) == 0:
        return len(s1)

    previous_row = range(len(s2) + 1)
    for i, c1 in enumerate(s1):
        current_row = [i + 1]
        for j, c2 in enumerate(s2):
            insertions = previous_row[j + 1] + 1
            deletions = current_row[j] + 1
            substitutions = previous_row[j] + (c1 != c2)
            current_row.append(min(insertions, deletions, substitutions))
        previous_row = current_row

    return previous_row[-1]


def main():
    # Example usage:
    # This is a simplified example and doesn't fully represent tree edit distance.
    # For actual tree edit distance, a more complex tree structure and algorithm are needed.
    print('Calculating Levenshtein distance (as a proxy for string edit distance):')
    s1 = "example string 1"
    s2 = "example string 2"
    distance = levenshtein(s1, s2)
    print(f'Distance between "{s1}" and "{s2}" is: {distance}')

if __name__ == '__main__':
    main()

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.