Arelle (arelle/arelle)

Arelle

https://github.com/arelle/arelle
Admin
Arelle is an open source XBRL platform providing comprehensive tools for processing, validating, and...

Tokens:42,600
Snippets:468
Trust Score:7.1
Update:5 days ago
Show doc for...
Context Summary (auto-generated)
Raw
# Arelle

Arelle is an end-to-end open source XBRL platform that provides the XBRL community with comprehensive tools for processing, validating, and analyzing financial reporting data. It supports XBRL v2.1, XBRL Dimensions, Formula, Inline XBRL, xBRL-JSON, xBRL-CSV, and Taxonomy Packages, making it a fully-featured XBRL processor certified by XBRL International as a Validating Processor.

The platform can be used as a desktop GUI application, command-line tool, Python library, or web service. It features an extensible plugin architecture supporting jurisdiction-specific validations for SEC EDGAR, ESEF (EU), HMRC (UK), CIPC (South Africa), and FERC, along with support for XF text-based Formula and XULE validation rules. Arelle enables instance creation via table linkbases, comparison of XBRL instances, RSS feed monitoring, and integration with databases and external applications.

## Installation

### Installing via pip

```bash
# Install base Arelle package
pip install arelle-release

# Install with all optional dependencies
pip install arelle-release[Crypto,DB,EFM,ObjectMaker,WebServer]
```

### Running with Docker

```bash
# Run the command line
docker run arelleproject/arelle:latest python arelleCmdLine.py --help

# Run the webserver on port 8080
docker run --name arelle-webserver -p 8080:8080 arelleproject/arelle:latest /opt/start.sh
```

## Python API - Session

The `arelle.api.Session` class is the primary interface for integrating Arelle into Python applications. It provides methods to run XBRL processing tasks, retrieve loaded models, and access validation logs. Note that Arelle uses shared global state and only ONE Session can run at a time per process.

```python
from arelle.api.Session import Session
from arelle.RuntimeOptions import RuntimeOptions
from arelle.ModelValue import qname

# Basic validation with ESEF disclosure system
options = RuntimeOptions(
    entrypointFile="path/to/report.zip",
    disclosureSystemName="esef-2024",
    internetConnectivity="offline",
    keepOpen=True,
    logFile="validation.log",
    logFormat="[%(messageCode)s] %(message)s - %(file)s",
    packages=["path/to/taxonomy-package.zip"],
    plugins="validate/ESEF",
    validate=True,
)

with Session() as session:
    success = session.run(options)

    # Retrieve loaded XBRL models
    model_xbrls = session.get_models()
    for model_xbrl in model_xbrls:
        # Query facts by concept QName
        target_qname = qname("https://xbrl.ifrs.org/taxonomy/2022-03-24/ifrs-full", "Equity")
        equity_facts = model_xbrl.factsByQname.get(target_qname, set())
        for fact in equity_facts:
            print(f"Found Equity fact with value: {fact.xValue}")

    # Get validation logs in different formats
    log_xml = session.get_logs("xml")
    log_json = session.get_logs("json")
    log_text = session.get_logs("text")
```

## Python API - Creating an iXBRL Viewer

The Session API supports plugin options for generating interactive iXBRL viewers from Inline XBRL documents. This example demonstrates loading a document set and producing a self-contained HTML viewer with JavaScript-based fact navigation.

```python
from arelle.api.Session import Session
from arelle.RuntimeOptions import RuntimeOptions

# Configure options for iXBRL Viewer generation
options = RuntimeOptions(
    entrypointFile="path/to/inline-xbrl-document.html",
    internetConnectivity="offline",
    keepOpen=True,
    logFile="logToStructuredMessage",
    logFormat="[%(messageCode)s] %(message)s - %(file)s",
    logPropagate=False,
    pluginOptions={
        "saveViewerDest": "output/viewer.html",
        "viewer_feature_review": True,
    },
    plugins="ixbrl-viewer",
)

# Load from a zip stream
with open("samples.zip", "rb") as stream:
    with Session() as session:
        session.run(
            options,
            sourceZipStream=stream,
        )
        # Viewer is saved to the specified destination
        print(f"Viewer URL: {options.viewerURL}")
```

## Command Line Interface - Basic Validation

The Arelle CLI provides comprehensive XBRL validation with support for multiple disclosure systems and output formats. Use `-f` to specify the entry point file and `-v` to enable validation according to the file type.

```bash
# Basic XBRL validation
arelleCmdLine -f report.xbrl -v

# Validate with SEC EDGAR Filing Manual rules
arelleCmdLine -f filing.htm --efm -v --logFile=validation.xml

# Validate with ESEF disclosure system
arelleCmdLine -f report.zip \
    --disclosureSystem=esef-2024 \
    --validate \
    --packages=/path/to/esef-taxonomy.zip \
    --plugins=validate/ESEF \
    --logFile=esef-validation.json

# Validate with calculation linkbase (Calc 1.1 round-to-nearest)
arelleCmdLine -f instance.xbrl -v --calc=c11r

# List available disclosure systems
arelleCmdLine --disclosureSystem=help
```

## Command Line Interface - Output Generation

Arelle can export XBRL data in various formats including HTML, CSV, JSON, and XML. These commands extract facts, concepts, presentation hierarchies, and other DTS components from loaded documents.

```bash
# Export facts to CSV
arelleCmdLine -f report.xbrl --facts=facts.csv

# Export presentation linkbase to HTML
arelleCmdLine -f taxonomy.xsd --pre=presentation.html

# Export calculation linkbase to CSV
arelleCmdLine -f taxonomy.xsd --cal=calculations.csv

# Export concepts with labels
arelleCmdLine -f taxonomy.xsd --concepts=concepts.html --labelLang=en-US

# Export DTS structure
arelleCmdLine -f report.xbrl --DTS=dts-tree.html

# Export dimensions from definition linkbase
arelleCmdLine -f taxonomy.xsd --dim=dimensions.csv

# Generate fact table
arelleCmdLine -f report.xbrl --factTable=fact-table.html
```

## Command Line Interface - Formula Processing

Arelle supports XBRL Formula validation and execution with tracing options for debugging complex formula expressions. Control formula behavior with the `--formula` flag and related options.

```bash
# Validate and run formulas
arelleCmdLine -f report.xbrl -v --formula=run

# Validate formulas only without running
arelleCmdLine -f report.xbrl -v --formula=validate

# Skip formula processing entirely
arelleCmdLine -f report.xbrl -v --formula=none

# Run specific formulas by ID
arelleCmdLine -f report.xbrl -v --formulaRunIDs="assertion1|assertion2"

# Formula with timeout protection
arelleCmdLine -f report.xbrl -v \
    --formula=run \
    --formulaVarSetTimeout=30 \
    --formulaReportTimeout=300

# Enable formula tracing for debugging
arelleCmdLine -f report.xbrl -v \
    --formula=run \
    --formulaVarSetExprEval \
    --formulaVarSetExprResult \
    --formulaVarSetTiming
```

## Command Line Interface - Taxonomy Packages

Arelle supports loading and validating XBRL Taxonomy Packages which bundle taxonomies with their dependencies. Packages can be loaded from local files or URLs.

```bash
# Load taxonomy packages
arelleCmdLine -f report.xbrl -v \
    --packages=/path/to/taxonomy-package.zip

# Load multiple packages (pipe-separated)
arelleCmdLine -f report.xbrl -v \
    --packages="/path/to/pkg1.zip|/path/to/pkg2.zip"

# Load all packages from a directory
arelleCmdLine -f report.xbrl -v \
    --packages=/path/to/packages/

# Validate a taxonomy package itself
arelleCmdLine -f taxonomy-package.zip --taxonomyPackage -v
```

## Web Service API - REST Endpoints

The Arelle web server provides REST API endpoints for XBRL validation and viewing operations. Start the server with `--webserver` and access endpoints via HTTP GET or POST requests.

```bash
# Start the web server on port 8080
arelleCmdLine --webserver=localhost:8080

# Using a specific server (cheroot for production)
arelleCmdLine --webserver=localhost:8080:cheroot
```

```bash
# Validate a remote XBRL file
curl "http://localhost:8080/rest/xbrl/https://example.com/report.xbrl/validation/xbrl?media=json"

# Validate with EDGAR rules
curl "http://localhost:8080/rest/xbrl/https://example.com/filing.htm/validation/xbrl?efm&media=xml"

# Validate with ESEF disclosure system
curl "http://localhost:8080/rest/xbrl/validation?file=https://example.com/report.zip&disclosureSystem=esef-2024&media=json"

# Get facts view as HTML
curl "http://localhost:8080/rest/xbrl/https://example.com/report.xbrl/facts?media=html"

# Get presentation linkbase as CSV
curl "http://localhost:8080/rest/xbrl/https://example.com/taxonomy.xsd/pre?media=csv"

# Get concepts as JSON
curl "http://localhost:8080/rest/xbrl/https://example.com/taxonomy.xsd/concepts?media=json"
```

## Web Service API - POST with File Upload

The web server accepts XBRL files via multipart form-data POST requests, enabling validation of files uploaded directly rather than fetched from URLs.

```bash
# Upload and validate a zip file
curl -X POST \
    -F "upload=@report.zip" \
    "http://localhost:8080/rest/xbrl/validation?media=json&disclosureSystem=esef-2024"

# Upload with specific content type
curl -X POST \
    -H "Content-Type: application/zip" \
    -H "X-File-Name: report.zip" \
    --data-binary @report.zip \
    "http://localhost:8080/rest/xbrl/validation?media=xml"

# Get versioning diff report
curl "http://localhost:8080/rest/xbrl/diff?fromDTS=old.xsd&toDTS=new.xsd&report=changes.xml"

# Configure plugins
curl "http://localhost:8080/rest/configure?plugins=validate/ESEF"

# Stop the web server
curl "http://localhost:8080/rest/stopWebServer"
```

## Plugin Development - Basic Structure

Arelle plugins are Python modules with a `__pluginInfo__` dictionary defining metadata and hook registrations. Plugins can add command-line options, validation rules, and custom processing logic.

```python
# myplugin/__init__.py
from optparse import OptionParser
from typing import Any

from arelle.utils.PluginHooks import PluginHooks


class MyPlugin(PluginHooks):
    @staticmethod
    def cntlrCmdLineOptions(
        parser: OptionParser,
        *args: Any,
        **kwargs: Any,
    ) -> None:
        """Add custom command line options."""
        parser.add_option(
            "--my-custom-option",
            action="store",
            dest="myCustomOption",
            help="Custom option for my plugin.",
        )

    @staticmethod
    def cntlrCmdLineUtilityRun(
        cntlr,
        options,
        *args: Any,
        **kwargs: Any,
    ) -> None:
        """Execute custom logic when CLI runs."""
        if options.myCustomOption:
            print(f"Custom option value: {options.myCustomOption}")


__pluginInfo__ = {
    "name": "My Custom Plugin",
    "version": "1.0.0",
    "description": "Example plugin demonstrating basic structure.",
    "license": "Apache-2.0",
    "author": "Your Name",
    "copyright": "(c) 2024 Your Organization",
    # Register hook implementations
    "CntlrCmdLine.Options": MyPlugin.cntlrCmdLineOptions,
    "CntlrCmdLine.Utility.Run": MyPlugin.cntlrCmdLineUtilityRun,
}
```

## Plugin Development - Validation Rules

Create validation plugins using the `@validation` decorator and `ValidationPlugin` class to implement jurisdiction or taxonomy-specific validation rules. Rules can target specific disclosure systems and validation hooks.

```python
# myplugin/rules.py
from collections.abc import Iterable
from typing import Any, cast

from arelle.ValidateXbrl import ValidateXbrl
from arelle.XmlValidate import VALID
from arelle.utils.PluginHooks import ValidationHook
from arelle.utils.validate.Decorator import validation
from arelle.utils.validate.Validation import Validation

# Define disclosure system identifiers
DISCLOSURE_SYSTEM_2024 = ("MyTaxonomy", "my-taxonomy-2024")


@validation(
    hook=ValidationHook.XBRL_FINALLY,
    disclosureSystems=DISCLOSURE_SYSTEM_2024,
)
def rule_cash_required(
    pluginData,
    val: ValidateXbrl,
    *args: Any,
    **kwargs: Any,
) -> Iterable[Validation]:
    """Validate that Cash concept is reported."""
    if "Cash" not in val.modelXbrl.factsByLocalName:
        yield Validation.error(
            codes="MY.01.01",
            msg="Cash must be reported in the filing.",
            modelObject=val.modelXbrl,
        )


@validation(
    hook=ValidationHook.XBRL_FINALLY,
    disclosureSystems=DISCLOSURE_SYSTEM_2024,
)
def rule_positive_revenue(
    pluginData,
    val: ValidateXbrl,
    *args: Any,
    **kwargs: Any,
) -> Iterable[Validation]:
    """Validate that Revenue facts have positive values."""
    for fact in val.modelXbrl.factsByLocalName.get("Revenue", []):
        if fact.isNumeric and getattr(fact, "xValid", 0) >= VALID:
            if cast(int, fact.xValue) < 0:
                yield Validation.warning(
                    codes="MY.01.02",
                    msg="Revenue value %(value)s should be positive.",
                    modelObject=fact,
                    value=fact.xValue,
                )


@validation(hook=ValidationHook.FINALLY)
def rule_check_errors(
    pluginData,
    val: ValidateXbrl,
    *args: Any,
    **kwargs: Any,
) -> Iterable[Validation]:
    """Report total XBRL errors found."""
    num_errors = len(val.modelXbrl.errors)
    if num_errors > 0:
        yield Validation.error(
            codes="MY.01.03",
            msg="Report contains %(numErrors)s validation errors.",
            modelObject=val.modelXbrl,
            numErrors=num_errors,
        )
```

## Plugin Development - Complete Validation Plugin

A complete validation plugin combines the entry point, disclosure system configuration, and validation rules into a structured module with resource files.

```python
# validate/MyTaxonomy/__init__.py
from pathlib import Path
from typing import Any

from arelle.Version import authorLabel, copyrightLabel
from arelle.utils.validate.ValidationPlugin import ValidationPlugin

from . import rules

PLUGIN_NAME = "Validate MyTaxonomy"
DISCLOSURE_SYSTEM_VALIDATION_TYPE = "MyTaxonomy"


# Create validation plugin instance
validationPlugin = ValidationPlugin(
    name=PLUGIN_NAME,
    disclosureSystemConfigUrl=Path(__file__).parent / "resources" / "config.xml",
    validationTypes=[DISCLOSURE_SYSTEM_VALIDATION_TYPE],
    validationRuleModules=[rules],
)


def disclosureSystemTypes(*args: Any, **kwargs: Any):
    return validationPlugin.disclosureSystemTypes


def disclosureSystemConfigURL(*args: Any, **kwargs: Any):
    return validationPlugin.disclosureSystemConfigURL


def validateXbrlStart(*args: Any, **kwargs: Any):
    return validationPlugin.validateXbrlStart(*args, **kwargs)


def validateXbrlFinally(*args: Any, **kwargs: Any):
    return validationPlugin.validateXbrlFinally(*args, **kwargs)


def validateFinally(*args: Any, **kwargs: Any):
    return validationPlugin.validateFinally(*args, **kwargs)


__pluginInfo__ = {
    "name": PLUGIN_NAME,
    "version": "1.0.0",
    "description": "Validation plugin for MyTaxonomy filings.",
    "license": "Apache-2.0",
    "author": authorLabel,
    "copyright": copyrightLabel,
    "DisclosureSystem.Types": disclosureSystemTypes,
    "DisclosureSystem.ConfigURL": disclosureSystemConfigURL,
    "Validate.XBRL.Start": validateXbrlStart,
    "Validate.XBRL.Finally": validateXbrlFinally,
    "Validate.Finally": validateFinally,
}
```

## RuntimeOptions Configuration

The `RuntimeOptions` dataclass contains all configuration options for Arelle runs. Options cover input files, validation settings, output formats, formula processing, logging, and plugin configuration.

```python
from arelle.RuntimeOptions import RuntimeOptions

# Comprehensive options configuration
options = RuntimeOptions(
    # Input file (XBRL instance, schema, inline XBRL, or test case)
    entrypointFile="path/to/report.zip",

    # Validation options
    validate=True,
    disclosureSystemName="esef-2024",
    utrValidate=True,  # Unit Type Registry validation

    # Taxonomy packages
    packages=["taxonomy-pkg1.zip", "taxonomy-pkg2.zip"],

    # Plugins to load (pipe-separated)
    plugins="validate/ESEF|ixbrl-viewer",

    # Plugin-specific options
    pluginOptions={
        "saveViewerDest": "output/viewer.html",
    },

    # Formula processing
    formulaAction="run",  # "validate", "run", or "none"
    formulaVarSetTimeout=30.0,
    formulaReportTimeout=300.0,

    # Internet connectivity
    internetConnectivity="offline",  # or "online"
    internetTimeout=30,

    # Logging configuration
    logFile="validation.log",
    logFormat="[%(messageCode)s] %(message)s - %(file)s",
    logLevel="DEBUG",
    logPropagate=False,

    # Keep models loaded for querying
    keepOpen=True,

    # Calculation validation mode
    calcs="c11r",  # Calc 1.1 round-to-nearest

    # Parameters for formulas
    parameters="authority=SE,reportingPeriod=2024",
    parameterSeparator=",",

    # Cache and proxy settings
    cacheDirectory="/path/to/cache",
    proxy="http://proxy.example.com:8080",

    # Disable persistent configuration
    disablePersistentConfig=True,
)
```

## Database Storage

Arelle can store XBRL data in various database systems using the xbrlDB plugin. Supported databases include PostgreSQL, MySQL, Oracle, MSSQL, MongoDB, and RDF stores.

```bash
# Store to PostgreSQL using XBRL-US schema
arelleCmdLine -f report.xbrl -v \
    --disclosureSystem=efm-pragmatic-all-years \
    --store-to-XBRL-DB="hostname,5432,username,password,database,90,postgres"

# Store to MSSQL using semantic schema
arelleCmdLine -f report.xbrl -v \
    --store-to-XBRL-DB="localhost\SQLEXPRESS,,sqlLogin,sqlPassword,dbname,90,mssqlSemantic"

# Store to MongoDB as JSON
arelleCmdLine -f report.xbrl -v \
    --store-to-XBRL-DB="localhost,27017,user,pass,xbrl_db,90,json"

# Store to local JSON file
arelleCmdLine -f report.xbrl -v \
    --store-to-XBRL-DB="jsonFile,,,,,90,json"

# Store as RDF Turtle file
arelleCmdLine -f report.xbrl -v \
    --store-to-XBRL-DB="rdfTurtleFile,,,,output.ttl,90,rdfDB"
```

## Summary

Arelle serves as a comprehensive XBRL processing platform for financial reporting stakeholders including regulators, filing agents, software vendors, and data analysts. The primary use cases include validating XBRL filings against regulatory requirements (SEC EDGAR, ESEF, HMRC), converting between XBRL formats (XML, JSON, CSV, Inline XBRL), generating interactive viewers for Inline XBRL documents, and integrating XBRL processing capabilities into larger financial applications through the Python API or REST web services.

Integration patterns typically involve using the Python Session API for batch processing and programmatic validation pipelines, deploying the web server behind a reverse proxy for cloud-based validation services, extending functionality through custom plugins for organization-specific validation rules, and using the CLI for automated testing and continuous integration workflows. The extensible plugin architecture allows organizations to implement custom validation rules that complement the built-in disclosure system validations, making Arelle adaptable to evolving regulatory requirements and internal compliance needs.