### Install spacy-layout

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Install the package using pip. Requires Python 3.10 or above.

```bash
pip install spacy-layout
```

--------------------------------

### Visualize document layout with matplotlib

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Visualize the layout of a PDF document by rendering its pages as images and drawing bounding boxes for detected sections. This example requires pypdfium2, matplotlib, spacy, and spacy-layout.

```python
import pypdfium2 as pdfium
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import spacy
from spacy_layout import spaCyLayout

DOCUMENT_PATH = "./document.pdf"

# Load and convert the PDF page to an image
pdf = pdfium.PdfDocument(DOCUMENT_PATH)
page_image = pdf[2].render(scale=1)  # get page 3 (index 2)
numpy_array = page_image.to_numpy()
# Process document with spaCy
nlp = spacy.blank("en")
layout = spaCyLayout(nlp)
doc = layout(DOCUMENT_PATH)

# Get page 3 layout and sections
page = doc._.pages[2]
page_layout = doc._.layout.pages[2]
# Create figure and axis with page dimensions
fig, ax = plt.subplots(figsize=(12, 16))
# Display the PDF image
ax.imshow(numpy_array)
# Add rectangles for each section's bounding box
for section in page[1]:
    # Create rectangle patch
    rect = Rectangle(
        (section._.layout.x, section._.layout.y),
        section._.layout.width,
        section._.layout.height,
        fill=False,
        color="blue",
        linewidth=1,
        alpha=0.5
    )
    ax.add_patch(rect)
    # Add text label at top of box
    ax.text(
        section._.layout.x,
        section._.layout.y,
        section.label_,
        fontsize=8,
        color="red",
        verticalalignment="bottom"
    )

ax.axis("off")  # hide axes
plt.show()
```

--------------------------------

### Visualization with Matplotlib

Source: https://context7.com/explosion/spacy-layout/llms.txt

Provides a guide on visualizing document pages with bounding boxes overlaid on the PDF rendering using pypdfium2 and matplotlib.

```APIDOC
## Visualization with Matplotlib

This section details how to create a visual representation of a document page, including bounding boxes for different layout elements, using `pypdfium2` for PDF rendering and `matplotlib` for plotting.

### Method

This is a code-based example demonstrating the visualization process.

### Endpoint

N/A (Local script execution)

### Parameters

N/A

### Request Body

N/A

### Request Example

```python
import pypdfium2 as pdfium
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import spacy
from spacy_layout import spaCyLayout

DOCUMENT_PATH = "./document.pdf"

# Load and render PDF page
pdf = pdfium.PdfDocument(DOCUMENT_PATH)
page_index = 0  # First page (0-indexed)
page_image = pdf[page_index].render(scale=1)
numpy_array = page_image.to_numpy()

# Process document with spaCy Layout
nlp = spacy.blank("en")
layout = spaCyLayout(nlp)
doc = layout(DOCUMENT_PATH)

# Get page layout and spans
page_layout, page_spans = doc._.pages[page_index]

# Create visualization
fig, ax = plt.subplots(figsize=(12, 16))
ax.imshow(numpy_array)

# Draw bounding boxes for each section
colors = {"title": "red", "section_header": "blue", "text": "green", "table": "orange"}
for span in page_spans:
    if span._.layout:
        color = colors.get(span.label_, "gray")
        rect = Rectangle(
            (span._.layout.x, span._.layout.y),
            span._.layout.width,
            span._.layout.height,
            fill=False,
            color=color,
            linewidth=2,
            alpha=0.7
        )
        ax.add_patch(rect)
        ax.text(
            span._.layout.x,
            span._.layout.y - 5,
            span.label_,
            fontsize=8,
            color=color,
            fontweight="bold"
        )

ax.axis("off")
plt.title(f"Page {page_layout.page_no} - {page_layout.width}x{page_layout.height}")
plt.tight_layout()
plt.savefig("./page_visualization.png", dpi=150)
plt.show()
```

### Response

Generates a PNG image file (`page_visualization.png`) and displays it.

#### Success Response (200)

N/A (This is a script execution, not an API response)

#### Response Example

N/A
```

--------------------------------

### Custom Table Display Callback

Source: https://context7.com/explosion/spacy-layout/llms.txt

Customize how tables are rendered in the document text by providing a callback function. This function receives a pandas DataFrame and returns a string representation. Two examples are shown: one for a concise display and another for a full table data display.

```python
import spacy
import pandas as pd
from spacy_layout import spaCyLayout

nlp = spacy.blank("en")

# Custom function to display table contents
def display_table(df: pd.DataFrame) -> str:
    return f"Table with columns: {', '.join(df.columns.tolist())}"

layout = spaCyLayout(nlp, display_table=display_table)
doc = layout("./document_with_tables.pdf")

# Table span text now uses custom display
table = doc._.tables[0]
print(table.text)
```

```python
# Alternative: Include full table data in text for NER/classification
def display_table_full(df: pd.DataFrame) -> str:
    rows = [" | ".join(df.columns)]
    for _, row in df.iterrows():
        rows.append(" | ".join(str(v) for v in row.values))
    return "\n".join(rows)

layout = spaCyLayout(nlp, display_table=display_table_full)
doc = layout("./document_with_tables.pdf")
print(doc._.tables[0].text)
```

--------------------------------

### spaCyLayout Class Initialization

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Details the initialization of the `spaCyLayout` class, including parameters for customization.

```APIDOC
## spaCyLayout Class

### Description
Initializes the `spaCyLayout` class, which is the main entry point for processing documents to extract layout information.

### Method
Instantiate the `spaCyLayout` class.

### Endpoint
N/A (Class instantiation)

### Parameters
- **nlp** (spacy.language.Language) - A loaded spaCy language pipeline.
- **display_table** (callable, optional) - A callback function to customize table rendering.

### Request Body
None

### Request Example
```python
import spacy
from spacy_layout_parser import spaCyLayout

# Load a spaCy model
nlp = spacy.load("en_core_web_sm")

# Initialize spaCyLayout
layout_processor = spaCyLayout(nlp)

# Initialize with a custom table display function
def custom_display(df):
    return f"Custom table with {len(df)} rows."

layout_processor_custom = spaCyLayout(nlp, display_table=custom_display)
```

### Response
N/A (This code initializes a class instance.)
```

--------------------------------

### Loading Serialized Documents

Source: https://context7.com/explosion/spacy-layout/llms.txt

Demonstrates how to load pre-processed documents from a DocBin file. It's crucial to initialize spaCyLayout first to register custom attributes.

```APIDOC
## Loading Serialized Documents

This section shows how to load documents that have been previously processed and saved to a DocBin file. Ensure that spaCyLayout is initialized before loading to correctly register any custom attributes.

### Code Example

```python
import spacy
from spacy.tokens import DocBin
from spacy_layout import spaCyLayout

# Initialize spaCy and spaCyLayout
nlp = spacy.blank("en")
layout = spaCyLayout(nlp)  # Required to register extension attributes

# Load documents from a DocBin file
doc_bin = DocBin(store_user_data=True).from_disk("./processed_documents.spacy")
docs = list(doc_bin.get_docs(nlp.vocab))

# Process loaded documents
for doc in docs:
    print(f"Loaded document with {len(doc.spans['layout'])} sections")
    print(f"Layout: {doc._.layout}")
    print(f"Tables: {len(doc._.tables)}")
```
```

--------------------------------

### spaCyLayout Initialization

Source: https://context7.com/explosion/spacy-layout/llms.txt

Initializes the spaCyLayout processor with a spaCy language model and optional configuration settings.

```APIDOC
## spaCyLayout.__init__

### Description
Initialize the document processor with a spaCy language model and optional configuration for separators, custom attributes, heading detection, table display, and Docling format options.

### Method
`__init__`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
import spacy
from spacy_layout import spaCyLayout

# Basic initialization with blank English model
nlp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Advanced initialization with custom settings
layout = spaCyLayout(
    nlp,
    separator="\n\n",  # Token to separate sections (default: "\n\n")
    attrs={
        "doc_layout": "layout",      # Custom attribute name for Doc._.layout
        "doc_pages": "pages",        # Custom attribute name for Doc._.pages
        "doc_tables": "tables",      # Custom attribute name for Doc._.tables
        "doc_markdown": "markdown",  # Custom attribute name for Doc._.markdown
        "span_layout": "layout",     # Custom attribute name for Span._.layout
        "span_data": "data",         # Custom attribute name for Span._.data
        "span_heading": "heading",   # Custom attribute name for Span._.heading
        "span_group": "layout",      # Custom span group name
    },
    headings=["section_header", "page_header", "title"],  # Labels for heading detection
    display_table="TABLE",  # Placeholder text for tables or callable
)
```

### Response
None (initialization method)
```

--------------------------------

### Initialize spaCyLayout Processor

Source: https://context7.com/explosion/spacy-layout/llms.txt

Initialize the spaCyLayout processor with a spaCy nlp object. Basic initialization uses a blank English model. Advanced initialization allows custom settings for separators, attributes, heading detection, and table display.

```python
import spacy
from spacy_layout import spaCyLayout

# Basic initialization with blank English model
lp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Advanced initialization with custom settings
layout = spaCyLayout(
    nlp,
    separator="\n\n",  # Token to separate sections (default: "\n\n")
    attrs={
        "doc_layout": "layout",      # Custom attribute name for Doc._.layout
        "doc_pages": "pages",        # Custom attribute name for Doc._.pages
        "doc_tables": "tables",      # Custom attribute name for Doc._.tables
        "doc_markdown": "markdown",  # Custom attribute name for Doc._.markdown
        "span_layout": "layout",     # Custom attribute name for Span._.layout
        "span_data": "data",         # Custom attribute name for Span._.data
        "span_heading": "heading",   # Custom attribute name for Span._.heading
        "span_group": "layout",      # Custom span group name
    },
    headings=["section_header", "page_header", "title"],  # Labels for heading detection
    display_table="TABLE",  # Placeholder text for tables or callable
)
```

--------------------------------

### spaCyLayout Initialization

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Initializes the spaCyLayout processor with a spaCy nlp object and optional configuration.

```APIDOC
## POST /api/users

### Description
Initializes the document processor.

### Method
POST

### Endpoint
/api/users

### Parameters
#### Path Parameters
- **user_id** (int) - Required - The ID of the user to retrieve.

#### Query Parameters
- **include_details** (bool) - Optional - Whether to include additional user details.

#### Request Body
- **username** (str) - Required - The username for the new user.
- **email** (str) - Required - The email address for the new user.

### Request Example
```json
{
  "username": "johndoe",
  "email": "john.doe@example.com"
}
```

### Response
#### Success Response (200)
- **user_id** (int) - The ID of the created user.
- **username** (str) - The username of the created user.
- **email** (str) - The email address of the created user.

#### Response Example
```json
{
  "user_id": 123,
  "username": "johndoe",
  "email": "john.doe@example.com"
}
```
```

--------------------------------

### Initialize spaCyLayout

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Initialize the document processor with a spaCy nlp object. The separator, custom attributes, heading labels, table display function, and docling options can be configured.

```python
nlp = spacy.blank("en")
layout = spaCyLayout(nlp)
```

--------------------------------

### spaCy Layout Data Classes Reference

Source: https://context7.com/explosion/spacy-layout/llms.txt

Demonstrates the usage of PageLayout, DocLayout, and SpanLayout dataclasses for structured layout information. Shows how to access these from a processed document.

```python
from spacy_layout.types import PageLayout, DocLayout, SpanLayout

# PageLayout: Individual page metadata
page = PageLayout(page_no=1, width=612.0, height=792.0)
print(f"Page {page.page_no}: {page.width}x{page.height} pixels")

# DocLayout: Document-level layout containing all pages
doc_layout = DocLayout(pages=[
    PageLayout(page_no=1, width=612.0, height=792.0),
    PageLayout(page_no=2, width=612.0, height=792.0),
])
print(f"Document has {len(doc_layout.pages)} pages")

# SpanLayout: Bounding box for text spans
span_layout = SpanLayout(x=72.0, y=100.0, width=468.0, height=24.0, page_no=1)
print(f"Span at ({span_layout.x}, {span_layout.y})")
print(f"Size: {span_layout.width}x{span_layout.height}")
print(f"On page: {span_layout.page_no}")

# Accessing from processed document
doc = layout("./document.pdf")
print(doc._.layout)  # DocLayout instance
for span in doc.spans["layout"]:
    print(span._.layout)  # SpanLayout instance or None
```

--------------------------------

### Process a document with spaCyLayout

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Initialize spaCyLayout with an nlp object and process a document path. Access document text, layout, tables, and markdown representations.

```python
import spacy
from spacy_layout import spaCyLayout

nlp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Process a document and create a spaCy Doc object
doc = layout("./starcraft.pdf")

# The text-based contents of the document
print(doc.text)
# Document layout including pages and page sizes
print(doc._.layout)
# Tables in the document and their extracted data
print(doc._.tables)
# Markdown representation of the document
print(doc._.markdown)

# Layout spans for different sections
for span in doc.spans["layout"]:
    # Document section and token and character offsets into the text
    print(span.text, span.start, span.end, span.start_char, span.end_char)
    # Section type, e.g. "text", "title", "section_header" etc.
    print(span.label_)
    # Layout features of the section, including bounding box
    print(span._.layout)
    # Closest heading to the span (accuracy depends on document structure)
    print(span._.heading)
```

--------------------------------

### Document Serialization

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Shows how to serialize processed spaCy `Doc` objects to disk using `DocBin` for efficient storage and loading.

```APIDOC
## Document Serialization

### Description
Serialize processed spaCy `Doc` objects, including custom extension attributes, into a binary format using `DocBin`. This allows for faster loading and avoids re-processing documents.

### Method
Use `DocBin` to store and load `Doc` objects.

### Endpoint
N/A (This is a library usage example)

### Parameters
- **docs** (iterable) - An iterable of spaCy `Doc` objects to store.
- **store_user_data** (bool) - Whether to store user data associated with the documents.

### Request Body
None

### Request Example
```python
from spacy.tokens import DocBin

# Assuming 'docs' is an iterable of processed spaCy Doc objects from spaCyLayout
doc_bin = DocBin(docs=docs, store_user_data=True)
doc_bin.to_disk("./file.spacy")

# To load:
# layout = spaCyLayout(nlp) # Initialize to repopulate extension attributes
# doc_bin = DocBin(store_user_data=True).from_disk("./file.spacy")
# docs = list(doc_bin.get_docs(nlp.vocab))
```

### Response
N/A (This code performs file operations.)
```

--------------------------------

### Customizing Table Display

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Explains how to provide a custom callback function to `spaCyLayout` to control how tables are rendered as text.

```APIDOC
## Customizing Table Display

### Description
Customize the text representation of tables by providing a `display_table` callback function during `spaCyLayout` initialization. This function receives the table's data as a pandas DataFrame and should return a string representation.

### Method
Initialize `spaCyLayout` with a `display_table` argument.

### Endpoint
N/A (This is a library usage example)

### Parameters
- **display_table** (callable) - A function that takes a pandas.DataFrame and returns a string.

### Request Body
None

### Request Example
```python
import pandas as pd
from spacy_layout_parser import spaCyLayout

def display_table(df: pd.DataFrame) -> str:
    return f"Table with columns: {', '.join(df.columns.tolist())}"

# Assuming 'nlp' is a loaded spaCy model
layout = spaCyLayout(nlp, display_table=display_table)
```

### Response
N/A (This code configures the library's behavior.)
```

--------------------------------

### Load Serialized Documents with spaCy Layout

Source: https://context7.com/explosion/spacy-layout/llms.txt

Loads documents previously serialized using DocBin. Ensure spaCyLayout is initialized to register custom attributes.

```python
import spacy
from spacy.tokens import DocBin
from spacy_layout import spaCyLayout

nlp = spacy.blank("en")
layout = spaCyLayout(nlp)  # Required to register extension attributes
doc_bin = DocBin(store_user_data=True).from_disk("./processed_documents.spacy")
docs = list(doc_bin.get_docs(nlp.vocab))

for doc in docs:
    print(f"Loaded document with {len(doc.spans['layout'])} sections")
    print(f"Layout: {doc._.layout}")
    print(f"Tables: {len(doc._.tables)}")
```

--------------------------------

### Batch Process Documents with spaCyLayout.pipe

Source: https://context7.com/explosion/spacy-layout/llms.txt

Efficiently process multiple documents in a batch using `layout.pipe`. Supports processing file paths or tuples containing paths and context metadata. Yields processed Doc objects and optional context.

```python
import spacy
from spacy_layout import spaCyLayout

lp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Basic batch processing
paths = ["document1.pdf", "document2.pdf", "document3.docx"]
for doc in layout.pipe(paths):
    print(f"Processed document with {len(doc.spans['layout'])} sections")
    print(f"Pages: {len(doc._.layout.pages)}")

# Batch processing with context metadata
sources = [
    ("report_q1.pdf", {"id": 1, "quarter": "Q1"}),
    ("report_q2.pdf", {"id": 2, "quarter": "Q2"}),
    ("summary.docx", {"id": 3, "type": "summary"}),
]
for doc, context in layout.pipe(sources, as_tuples=True):
    print(f"Document {context['id']}: {len(doc.spans['layout'])} sections")
    # Store context with processed document
    doc.user_data["metadata"] = context
```

--------------------------------

### Visualize Document Pages with Matplotlib

Source: https://context7.com/explosion/spacy-layout/llms.txt

Visualizes PDF pages with bounding boxes for layout elements overlaid on the rendered page. Requires pypdfium2 and matplotlib.

```python
import pypdfium2 as pdfium
import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle
import spacy
from spacy_layout import spaCyLayout

DOCUMENT_PATH = "./document.pdf"

# Load and render PDF page
pdf = pdfium.PdfDocument(DOCUMENT_PATH)
page_index = 0  # First page (0-indexed)
page_image = pdf[page_index].render(scale=1)
numpy_array = page_image.to_numpy()

# Process document with spaCy Layout
nlp = spacy.blank("en")
layout = spaCyLayout(nlp)
doc = layout(DOCUMENT_PATH)

# Get page layout and spans
page_layout, page_spans = doc._.pages[page_index]

# Create visualization
fig, ax = plt.subplots(figsize=(12, 16))
ax.imshow(numpy_array)

# Draw bounding boxes for each section
colors = {"title": "red", "section_header": "blue", "text": "green", "table": "orange"}
for span in page_spans:
    if span._.layout:
        color = colors.get(span.label_, "gray")
        rect = Rectangle(
            (span._.layout.x, span._.layout.y),
            span._.layout.width,
            span._.layout.height,
            fill=False,
            color=color,
            linewidth=2,
            alpha=0.7
        )
        ax.add_patch(rect)
        ax.text(
            span._.layout.x,
            span._.layout.y - 5,
            span.label_,
            fontsize=8,
            color=color,
            fontweight="bold"
        )

ax.axis("off")
plt.title(f"Page {page_layout.page_no} - {page_layout.width}x{page_layout.height}")
plt.tight_layout()
plt.savefig("./page_visualization.png", dpi=150)
plt.show()
```

--------------------------------

### Process multiple documents with spaCyLayout.pipe

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Process multiple documents efficiently using the `pipe` method. This is recommended for large volumes of documents. The `as_tuples` argument can be used to process sources with associated context.

```python
layout = spaCyLayout(nlp)
paths = ["one.pdf", "two.pdf", "three.pdf", ...]
docs = layout.pipe(paths)
```

```python
sources = [("one.pdf", {"id": 1}), ("two.pdf", {"id": 2})]
for doc, context in layout.pipe(sources, as_tuples=True):
    ...
```

--------------------------------

### Access Document Layout and Spans

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Initialize spaCyLayout and process a PDF document to access its layout information and spans. The `doc._.layout` attribute provides document-level layout features, while iterating through `doc.spans["layout"]` allows access to individual layout spans.

```python
layout = spaCyLayout(nlp)
doc = layout("./starcraft.pdf")
print(doc._.layout)
for span in doc.spans["layout"]:
    print(span.label_, span._.layout)
```

--------------------------------

### Apply spaCy pipeline after layout processing

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Load a transformer-based spaCy pipeline and apply it to the Doc object generated by spaCyLayout for linguistic analysis, NER, or rule-based matching.

```python
# Load the transformer-based English pipeline
# Installation: python -m spacy download en_core_web_trf
nlp = spacy.load("en_core_web_trf")
layout = spaCyLayout(nlp)

doc = layout("./starcraft.pdf")
# Apply the pipeline to access POS tags, dependencies, entities etc.
doc = nlp(doc)
```

--------------------------------

### Table Extraction and Access

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Demonstrates how to access tables within a processed document and retrieve their data as a pandas DataFrame.

```APIDOC
## Accessing Tables

### Description
This section shows how to iterate through tables found in a document and access their properties, including their position, bounding box, and the tabular data itself.

### Method
Iterate through `doc._.tables`.

### Endpoint
N/A (This is a library usage example)

### Parameters
None

### Request Body
None

### Request Example
```python
for table in doc._.tables:
    # Token position and bounding box
    print(table.start, table.end, table._.layout)
    # pandas.DataFrame of contents
    print(table._.data)
```

### Response
#### Success Response (200)
Prints the start and end tokens of the table span, its layout information, and the pandas DataFrame containing the table's data.

#### Response Example
```
15 25 <SpanLayout(x=100, y=200, width=300, height=150, page_no=1)>
   col1  col2
0     A     1
1     B     2
```
```

--------------------------------

### Process Single Document with spaCyLayout

Source: https://context7.com/explosion/spacy-layout/llms.txt

Process a single document from a file path, bytes, or DoclingDocument object to create a spaCy Doc. Access extracted text, layout metadata, markdown, tables, and individual layout spans with their attributes.

```python
import spacy
from spacy_layout import spaCyLayout

lp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Process from file path
doc = layout("./document.pdf")

# Process from bytes
with open("./document.pdf", "rb") as f:
    pdf_bytes = f.read()
doc = layout(pdf_bytes)

# Access document text content
print(doc.text)
# Output: "Introduction\n\nThis document covers...\n\nChapter 1..."

# Access document layout metadata
print(doc._.layout)
# Output: DocLayout(pages=[PageLayout(page_no=1, width=612.0, height=792.0), ...])

# Access markdown representation
print(doc._.markdown)
# Output: "# Introduction\n\nThis document covers...\n\n## Chapter 1..."

# Access all tables in document
print(doc._.tables)
# Output: [Span with label "table", ...]

# Iterate over layout spans
for span in doc.spans["layout"]:
    print(f"Text: {span.text[:50]}...")
    print(f"Label: {span.label_}")  # e.g., "text", "title", "section_header", "table"
    print(f"Layout: {span._.layout}")  # SpanLayout(x=72.0, y=100.0, width=468.0, height=24.0, page_no=1)
    print(f"Heading: {span._.heading}")  # Closest heading span or None
```

--------------------------------

### spaCyLayout Call

Source: https://context7.com/explosion/spacy-layout/llms.txt

Processes a single document (from path, bytes, or DoclingDocument) and returns a spaCy Doc object with extracted information.

```APIDOC
## spaCyLayout.__call__

### Description
Process a single document and create a spaCy `Doc` object containing text content, layout spans, table data, and markdown representation. Accepts file paths, bytes, or pre-created DoclingDocument objects.

### Method
`__call__`

### Endpoint
N/A (This is a method call on an initialized object)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
import spacy
from spacy_layout import spaCyLayout

nlp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Process from file path
doc = layout("./document.pdf")

# Process from bytes
with open("./document.pdf", "rb") as f:
    pdf_bytes = f.read()
doc = layout(pdf_bytes)

# Access document text content
print(doc.text)
# Output: "Introduction\n\nThis document covers...\n\nChapter 1..."

# Access document layout metadata
print(doc._.layout)
# Output: DocLayout(pages=[PageLayout(page_no=1, width=612.0, height=792.0), ...])

# Access markdown representation
print(doc._.markdown)
# Output: "# Introduction\n\nThis document covers...\n\n## Chapter 1..."

# Access all tables in document
print(doc._.tables)
# Output: [Span with label "table", ...]

# Iterate over layout spans
for span in doc.spans["layout"]:
    print(f"Text: {span.text[:50]}...")
    print(f"Label: {span.label_}")  # e.g., "text", "title", "section_header", "table"
    print(f"Layout: {span._.layout}")  # SpanLayout(x=72.0, y=100.0, width=468.0, height=24.0, page_no=1)
    print(f"Heading: {span._.heading}")  # Closest heading span or None
```

### Response
#### Success Response (200)
- **doc** (spacy.tokens.Doc) - A spaCy Doc object containing processed document data.

#### Response Example
```json
{
  "text": "Processed document text...",
  "_.layout": {"pages": [...]},
  "_.markdown": "# Document Title\n...",
  "_.tables": [...]
}
```
```

--------------------------------

### Iterate Through Document Pages and Content

Source: https://context7.com/explosion/spacy-layout/llms.txt

Iterate through each page in a document, accessing its dimensions and the layout sections it contains. For each section, print its label, text, and position/size if available.

```python
for page_layout, page_spans in doc._.pages:
    print(f"Page {page_layout.page_no}: {page_layout.width}x{page_layout.height} pixels")
    print(f"  Contains {len(page_spans)} layout sections")

    for span in page_spans:
        print(f"    - [{span.label_}] {span.text[:40]}...")
        if span._.layout:
            print(f"      Position: ({span._.layout.x}, {span._.layout.y})")
            print(f"      Size: {span._.layout.width}x{span._.layout.height}")

# Access specific page
page_3_layout, page_3_spans = doc._.pages[2]  # 0-indexed
print(f"Page 3 has {len(page_3_spans)} sections")
```

--------------------------------

### spaCyLayout Batch Processing (pipe)

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Processes multiple documents efficiently using the pipe method, supporting batching and optional context.

```APIDOC
## GET /api/users

### Description
Process multiple documents and create spaCy [`Doc`](https://spacy.io/api/doc) objects. You should use this method if you're processing larger volumes of documents at scale.

### Method
GET

### Endpoint
/api/users

### Parameters
#### Query Parameters
- **sources** (Iterable[str | Path | bytes] | Iterable[tuple[str | Path | bytes, Any]]) - Required - Paths of documents to process or bytes, or `(source, context)` tuples if `as_tuples` is set to `True`.
- **as_tuples** (bool) - Optional - If set to `True`, inputs should be an iterable of `(source, context)` tuples. Output will then be a sequence of `(doc, context)` tuples. Defaults to `False`.

### Response
#### Success Response (200)
- **Doc** (spacy.tokens.Doc) - The processed spaCy `Doc` objects or `(doc, context)` tuples if `as_tuples` is set to `True`.

#### Response Example
```json
[
  {
    "text": "This is the first processed document.",
    "spans": {
      "layout": [
        {
          "start": 0,
          "end": 10,
          "label": "section_header"
        }
      ]
    }
  },
  {
    "text": "This is the second processed document.",
    "spans": {
      "layout": [
        {
          "start": 0,
          "end": 10,
          "label": "section_header"
        }
      ]
    }
  }
]
```
```

--------------------------------

### Integrate spaCy NLP Pipelines with Layout Data

Source: https://context7.com/explosion/spacy-layout/llms.txt

Apply spaCy's NLP pipelines to processed documents for linguistic analysis and NER. This snippet shows how to load a spaCy pipeline, process a document, access named entities, and combine layout information with NER results.

```python
import spacy
from spacy_layout import spaCyLayout

# Load a trained spaCy pipeline
nlp = spacy.load("en_core_web_trf")  # or "en_core_web_sm" for faster processing
layout = spaCyLayout(nlp)

# Process document
doc = layout("./report.pdf")

# Apply NLP pipeline for linguistic features and NER
doc = nlp(doc)

# Access named entities
print("Named Entities:")
for ent in doc.ents:
    print(f"  {ent.text} ({ent.label_})")

# Combine layout information with NER
for span in doc.spans["layout"]:
    # Find entities within this layout section
    section_ents = [ent for ent in doc.ents
                    if ent.start >= span.start and ent.end <= span.end]
    if section_ents:
        print(f"\n[{span.label_}] {span.text[:50]}...")
        print(f"  Heading: {span._.heading.text if span._.heading else 'None'}")
        for ent in section_ents:
            print(f"  - {ent.text} ({ent.label_})")
```

--------------------------------

### Process a single document with spaCyLayout

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Process a document from a given source (path, bytes, or DoclingDocument) and obtain a spaCy Doc object. The processed document will contain layout spans available via `Doc.spans["layout"]` by default.

```python
layout = spaCyLayout(nlp)
doc = layout("./starcraft.pdf")
```

--------------------------------

### Serialize Processed Documents

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Save processed spaCy documents to disk using DocBin for efficient storage and later retrieval. This avoids re-running the conversion process. Ensure custom extension attributes are re-registered upon loading.

```python
from spacy.tokens import DocBin

docs = layout.pipe(["one.pdf", "two.pdf", "three.pdf"])
doc_bin = DocBin(docs=docs, store_user_data=True)
doc_bin.to_disk("./file.spacy")
```

```diff
+ layout = spaCyLayout(nlp)
doc_bin = DocBin(store_user_data=True).from_disk("./file.spacy")
docs = list(doc_bin.get_docs(nlp.vocab))
```

--------------------------------

### Process multiple documents with spaCyLayout.pipe

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Use the pipe method for efficient processing of multiple documents from an iterable of paths or bytes, yielding spaCy Doc objects.

```python
paths = ["one.pdf", "two.pdf", "three.pdf", ...]
for doc in layout.pipe(paths):
    print(doc._.layout)
```

--------------------------------

### Data Classes Reference

Source: https://context7.com/explosion/spacy-layout/llms.txt

Explains the core data classes used by spaCy Layout: PageLayout, DocLayout, and SpanLayout, for representing structured layout information.

```APIDOC
## Data Classes Reference

spaCy Layout utilizes three primary dataclasses to represent structured information about document layout:

- **PageLayout**: Stores metadata for an individual page, such as its number and dimensions.
- **DocLayout**: Aggregates `PageLayout` objects for all pages in a document, providing document-level layout metadata.
- **SpanLayout**: Represents the bounding box and position of a text span within a page.

### Usage Examples

#### PageLayout

```python
from spacy_layout.types import PageLayout

# Example instantiation
page = PageLayout(page_no=1, width=612.0, height=792.0)
print(f"Page {page.page_no}: {page.width}x{page.height} pixels")
```

#### DocLayout

```python
from spacy_layout.types import DocLayout, PageLayout

# Example instantiation with multiple pages
doc_layout = DocLayout(pages=[
    PageLayout(page_no=1, width=612.0, height=792.0),
    PageLayout(page_no=2, width=612.0, height=792.0),
])
print(f"Document has {len(doc_layout.pages)} pages")
```

#### SpanLayout

```python
from spacy_layout.types import SpanLayout

# Example instantiation
span_layout = SpanLayout(x=72.0, y=100.0, width=468.0, height=24.0, page_no=1)
print(f"Span at ({span_layout.x}, {span_layout.y})")
print(f"Size: {span_layout.width}x{span_layout.height}")
print(f"On page: {span_layout.page_no}")
```

#### Accessing from a Processed Document

```python
# Assuming 'layout' is an initialized spaCyLayout object
# and './document.pdf' is a valid document path
doc = layout("./document.pdf")

# Accessing the overall document layout
print(doc._.layout)  # This will be a DocLayout instance

# Accessing layout information for each span identified as 'layout'
for span in doc.spans["layout"]:
    print(span._.layout)  # This will be a SpanLayout instance or None if no layout info
```
```

--------------------------------

### spaCy-Layout API - Data and Extension Attributes

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Details the various extension attributes available on `Doc` and `Span` objects after processing with spaCy-Layout, along with descriptions of associated data structures.

```APIDOC
## spaCy-Layout API - Data and Extension Attributes

### Description
This section outlines the extension attributes added by spaCy-Layout to spaCy's `Doc` and `Span` objects, providing access to layout information, page structure, tables, and more.

### Method
Access attributes directly from processed `Doc` and `Span` objects.

### Endpoint
N/A (This describes library attributes)

### Parameters
None

### Request Body
None

### Request Example
```python
# Assuming 'nlp' is a loaded spaCy model and 'layout' is initialized
# layout = spaCyLayout(nlp)
# doc = layout("./starcraft.pdf")

# Accessing document-level layout information
print(doc._.layout)

# Accessing layout spans
for span in doc.spans["layout"]:
    print(span.label_, span._.layout)
```

### Response
#### Success Response (200)
Provides access to structured layout data.

#### Response Example
```
# Example output for doc._.layout:
# DocLayout(pages=[PageLayout(page_no=1, width=612.0, height=792.0)])

# Example output for span.label_ and span._.layout:
# text <SpanLayout(x=50, y=100, width=100, height=20, page_no=1)>
# table <SpanLayout(x=50, y=150, width=500, height=100, page_no=1)>
```

### Attribute Table

| Attribute | Type | Description |
| --- | --- | --- |
| `Doc._.layout` | `DocLayout` | Layout features of the document. |
| `Doc._.pages` | `list[tuple[PageLayout, list[Span]]]` | Pages in the document and the spans they contain. |
| `Doc._.tables` | `list[Span]` | All tables in the document. |
| `Doc._.markdown` | `str` | Markdown representation of the document. |
| `Doc.spans["layout"]` | `spacy.tokens.SpanGroup` | The layout spans in the document. |
| `Span.label_` | `str` | The type of the extracted layout span, e.g. `
```

--------------------------------

### Customize Table Display Callback

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Define a custom callback function to format how tables are displayed. This function receives a pandas DataFrame and should return a string representation, allowing for custom rendering during document processing.

```python
def display_table(df: pd.DataFrame) -> str:
    return f"Table with columns: {', '.join(df.columns.tolist())}"

layout = spaCyLayout(nlp, display_table=display_table)
```

--------------------------------

### Serialize and Persist Processed Documents

Source: https://context7.com/explosion/spacy-layout/llms.txt

Serialize processed documents using spaCy's binary format for efficient storage and fast loading. This snippet demonstrates processing multiple documents using `layout.pipe` and saving them to disk using `DocBin`.

```python
import spacy
from spacy.tokens import DocBin
from spacy_layout import spaCyLayout

nlp = spacy.blank("en")
layout = spaCyLayout(nlp)

# Process and serialize multiple documents
docs = list(layout.pipe(["doc1.pdf", "doc2.pdf", "doc3.docx"]))
doc_bin = DocBin(docs=docs, store_user_data=True)
doc_bin.to_disk("./processed_documents.spacy")
```

--------------------------------

### Iterate Through Tables and Access Data

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Access table spans and their data as pandas DataFrames. The `table.start`, `table.end`, and `table._.layout` provide token positions and bounding boxes, while `table._.data` contains the tabular content.

```python
for table in doc._.tables:
    # Token position and bounding box
    print(table.start, table.end, table._.layout)
    # pandas.DataFrame of contents
    print(table._.data)
```

--------------------------------

### Data Structures: PageLayout, DocLayout, SpanLayout

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Defines the dataclasses used by spaCy-Layout to represent page dimensions, document layout, and individual span bounding boxes.

```APIDOC
## Data Structures

### Description
This section details the core data structures used by spaCy-Layout to represent layout information: `PageLayout`, `DocLayout`, and `SpanLayout`.

### Method
These are data classes used internally and exposed through extension attributes.

### Endpoint
N/A

### Parameters
None

### Request Body
None

### Request Example
```python
# Example of accessing PageLayout attributes (typically via Doc._.pages)
# for page_layout, spans in doc._.pages:
#     print(f"Page: {page_layout.page_no}, Width: {page_layout.width}, Height: {page_layout.height}")

# Example of accessing SpanLayout attributes (typically via Span._.layout)
# if span._.layout:
#     print(f"Span BBox: x={span._.layout.x}, y={span._.layout.y}, width={span._.layout.width}, height={span._.layout.height}, page={span._.layout.page_no}")
```

### Response
#### Success Response (200)
Provides structured data for layout components.

#### Response Example
```
# PageLayout Example:
# PageLayout(page_no=1, width=612.0, height=792.0)

# DocLayout Example:
# DocLayout(pages=[PageLayout(page_no=1, width=612.0, height=792.0), ...])

# SpanLayout Example:
# SpanLayout(x=50.0, y=100.0, width=100.0, height=20.0, page_no=1)
```

### Dataclass: `PageLayout`

| Attribute | Type | Description |
| --- | --- | --- |
| `page_no` | `int` | The page number (1-indexed). |
| `width` | `float` | Page width in pixels. |
| `height` | `float` | Page height in pixels. |

### Dataclass: `DocLayout`

| Attribute | Type | Description |
| --- | --- | --- |
| `pages` | `list[PageLayout]` | The pages in the document. |

### Dataclass: `SpanLayout`

| Attribute | Type | Description |
| --- | --- | --- |
| `x` | `float` | Horizontal offset of the bounding box in pixels. |
| `y` | `float` | Vertical offset of the bounding box in pixels. |
| `width` | `float` | Width of the bounding box in pixels. |
| `height` | `float` | Height of the bounding box in pixels. |
| `page_no` | `int` | Number of page the span is on. |
```

--------------------------------

### spaCyLayout Document Processing

Source: https://github.com/explosion/spacy-layout/blob/main/README.md

Processes a single document from a given source (path, bytes, or DoclingDocument) and returns a spaCy Doc object.

```APIDOC
## GET /api/users/{user_id}

### Description
Processes a document and creates a spaCy [`Doc`](https://spacy.io/api/doc) object containing the text content and layout spans.

### Method
GET

### Endpoint
/api/users/{user_id}

### Parameters
#### Path Parameters
- **user_id** (int) - Required - The ID of the user to retrieve.

#### Query Parameters
- **source** (str | Path | bytes | DoclingDocument) - Required - Path of document to process, bytes or already created `DoclingDocument`.

### Request Example
```json
{
  "source": "./starcraft.pdf"
}
```

### Response
#### Success Response (200)
- **Doc** (spacy.tokens.Doc) - The processed spaCy `Doc` object.

#### Response Example
```json
{
  "text": "This is the processed document text.",
  "spans": {
    "layout": [
      {
        "start": 0,
        "end": 10,
        "label": "section_header"
      }
    ]
  }
}
```
```