Hyper-Extract

https://github.com/yifanfeng97/hyper-extract
Admin
Hyper-Extract is an intelligent, LLM-powered knowledge extraction and evolution framework that...
Tokens:136,462
Snippets:2,023
Trust Score:8.8
License:Apache-2.0
Update:3 months ago
Show doc for...
Tokens:
Raw
### Provider Setup Examples

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/examples/README.md

Demonstrates how to set up different language model providers like OpenAI, Bailian, and local vLLM. Navigate to the examples directory before running these commands.

```bash
cd examples

# OpenAI
python providers/openai_demo.py
```

```bash
cd examples

# Bailian (Alibaba Cloud)
python providers/bailian_demo.py
```

```bash
cd examples

# Local vLLM
python providers/vllm_demo.py
```

--------------------------------

### Initialize Configuration (Interactive Setup)

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/config.md

Run this command to start an interactive setup process. The tool will prompt you for model names and API keys step by step.

```bash
he config init
```

--------------------------------

### Set up Local vLLM Services

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/index.md

Install and start local vLLM services for LLM and embedding. Ensure sufficient VRAM is available.

```bash
# Start LLM service (~8GB VRAM)
vllm serve Qwen/Qwen3.5-9B --port 8000 --api-key dummy

# Start Embedding service (~2GB VRAM)
vllm serve BAAI/bge-m3 --task embed --port 8001
```

--------------------------------

### Install and Run MCP Server

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/mcp.md

Install the Hyper-Extract MCP package and start the server using the command line. The server uses the configuration from ~/.he/config.toml.

```bash
pip install 'hyperextract[mcp]'

# start the server (stdio transport)
he-mcp
# equivalent:
python -m hyperextract.mcp_server
```

--------------------------------

### Initialize CLI Configuration

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/configuration.md

Use this command for the first-time setup of the CLI configuration. It will guide you through prompts to select a provider and enter necessary credentials like API keys.

```bash
he config init
# Follow prompts to select provider, enter model name and API key
```

--------------------------------

### Install Documentation Dependencies and Serve Locally

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/contributing.md

Install necessary packages for building documentation and serve the documentation site locally for preview.

```bash
# Install docs dependencies
pip install mkdocs mkdocs-material mkdocstrings[python]

# Serve locally
mkdocs serve
```

--------------------------------

### Install and Setup Hyper-Extract

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/templates/README.md

Import necessary classes from the hyperextract library and initialize the language model and embeddings. Ensure you have the required libraries installed.

```python
from hyperextract.utils.template_engine import Gallery, TemplateFactory
from langchain_openai import ChatOpenAI, OpenAIEmbeddings

llm = ChatOpenAI(model="gpt-4o-mini")
embedder = OpenAIEmbeddings()
```

--------------------------------

### Local vLLM Configuration Output Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/configuration.md

Example output when showing configuration for local vLLM deployments.

```text
┌──────────────────────────────────────────────────────────────────┐
│                Hyper-Extract Configuration                       │
├──────────┬──────────┬─────────────────────┬──────────┬───────────┤
│ Service  │ Provider │ Model               │ API Key  │ Base URL  │
├──────────┼──────────┼─────────────────────┼──────────┼───────────┤
│ LLM      │ vllm     │ Qwen/Qwen3.5-9B     │ dummy... │ localhost…│
│ Embedder │ vllm     │ BAAI/bge-m3         │ dummy... │ localhost…│
└──────────┴──────────┴─────────────────────┴──────────┴───────────┘
```

--------------------------------

### Set Up Python Project and Install Hyper-Extract

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/python-quickstart.md

Create a project directory, set up a virtual environment, and install the hyperextract library using pip.

```bash
mkdir my_extraction_project
cd my_extraction_project

# Create virtual environment (optional but recommended)
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install hyperextract
pip install hyperextract
```

--------------------------------

### Set Up Development Environment

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/contributing.md

Create a virtual environment and install the project in editable mode with development dependencies.

```bash
# Create virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install in editable mode
pip install -e ".[dev]"
```

--------------------------------

### Complete Example: Extract, Explore, and Save

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/quickstart.md

A comprehensive example demonstrating the entire workflow: setting up the API key, creating a template, extracting knowledge from text, displaying results, building a search index, performing a search, and saving the knowledge abstract.

```python
"""Complete example: Extract, explore, and save knowledge."""

from dotenv import load_dotenv
load_dotenv()

from hyperextract import Template

def main():
    # Create template
    print("Creating template...")
    ka = Template.create("general/biography_graph", language="en")
    
    # Sample text
    text = """
    Ada Lovelace was an English mathematician and writer, chiefly known 
    for her work on Charles Babbage's early mechanical general-purpose 
    computer, the Analytical Engine. She is often regarded as the first 
    computer programmer.
    """
    
    # Extract knowledge
    print("Extracting knowledge...")
    result = ka.parse(text)
    
    # Display results
    print(f"\nExtraction Results:")
    print(f"  Nodes: {len(result.nodes)}")
    print(f"  Edges: {len(result.edges)}")

    # List nodes
    print("\nNodes found:")
    for node in result.nodes:
        print(f"  - {node.name} ({node.type})")
    
    # Build index and search
    print("\nBuilding search index...")
    result.build_index()
    
    search_nodes, search_edges = result.search("computer programming", top_k=2)
    print(f"\nSearch results: {len(search_nodes)} nodes, {len(search_edges)} edges")
    
    # Save
    print("\nSaving knowledge abstract...")
    result.dump("./ada_kb/")
    
    print("\nDone! Try: he show ./ada_kb/")

if __name__ == "__main__":
    main()
```

--------------------------------

### Initialize Configuration (Lazy Setup - Bailian)

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/config.md

Use this command for a quick, non-interactive setup with Bailian (Alibaba Cloud) presets. It automatically fills in the default model and API URL.

```bash
he config init -p bailian -k sk-your-api-key-here
```

--------------------------------

### Use Method Templates Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/cli/README.md

Example demonstrating the use of different extraction methods (Light RAG and Hyper RAG) with the parse command.

```bash
# Use Light RAG method
he parse document.md -o ka -m light_rag
```

```bash
# Use Hyper RAG method
he parse document.md -o ka -m hyper_rag
```

--------------------------------

### Initialize Configuration (Lazy Setup - OpenAI)

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/config.md

Use this command for a quick, non-interactive setup with OpenAI presets. It automatically fills in the default model and API URL.

```bash
he config init -p openai -k sk-your-api-key-here
```

--------------------------------

### Cloud API Configuration Output Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/configuration.md

Example output when showing configuration for cloud API providers like OpenAI or Bailian.

```text
┌─────────────────────────────────────────────────────────────┐
│              Hyper-Extract Configuration                    │
├──────────┬──────────┬─────────────────────┬────────────┬────┤
│ Service  │ Provider │ Model               │ API Key    │ ...│
├──────────┼──────────┼─────────────────────┼────────────┼────┤
│ LLM      │ bailian  │ qwen3.6-plus        │ sk-xxxx... │ ...│
│ Embedder │ bailian  │ text-embedding-v4   │ sk-xxxx... │ ...│
└──────────┴──────────┴─────────────────────┴────────────┴────┘
```

--------------------------------

### Install Hyper-Extract in a Virtual Environment

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/faq.md

Creates and activates a virtual environment, then installs Hyper-Extract.

```bash
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install hyperextract
```

--------------------------------

### Single Question Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/talk.md

Ask a single question to the knowledge abstract and get a synthesized answer with sources. Specify the path to the knowledge abstract and the query using the -q flag.

```bash
he talk ./output/ -q "What were Tesla's major achievements?"
```

--------------------------------

### Verify Project Structure

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/knowledge-base/step1-setup.md

After running the setup script, use the `tree` command to verify that the expected directory structure has been created.

```bash
tree my_ka/
```

--------------------------------

### Example Workflow: Extracting Financial Information

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/README.md

This example demonstrates a typical interaction flow using Hyper-Extract skills to extract key information from financial reports.

```text
You: I want to extract key information from financial reports

Assistant (using brainstorm):
Let's explore your requirements...

You: I need to extract company name, revenue, and quarter from earnings calls

Assistant (using brainstorm):
Based on your description, I recommend: model type

Assistant (using record-designer):
Let's design the fields for your model...
```

--------------------------------

### Development Installation

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Clone the repository and install Hyper-Extract in editable mode with development dependencies for contributing to the project.

```bash
# Clone the repository
git clone https://github.com/yifanfeng97/hyper-extract.git
cd hyper-extract

# Install in editable mode with dev dependencies
pip install -e ".[dev]"
```

--------------------------------

### Install and Use Hyper-Extract CLI

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/index.md

Install the CLI tool, configure an API key, and extract knowledge from a document. Visualize the resulting knowledge graph.

```bash
# 1. Install CLI tool
uv tool install hyperextract

# 2. Configure API Key (Bailian example)
he config init -p bailian -k YOUR_BAILIAN_API_KEY

# 3. Extract knowledge from a document
he parse tesla.md -t general/biography_graph -o ./output/ -l en

# 4. Visualize the knowledge graph
he show ./output/
```

--------------------------------

### Start MCP Server

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/README.md

Start the standard input/output MCP server for Hyper-Extract. This command exposes knowledge abstracts to MCP-capable assistants.

```bash
he-mcp
```

--------------------------------

### Extract Financial Data Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/cli/README.md

Example demonstrating extraction of financial data, including configuration, template listing, parsing, indexing, and searching.

```bash
# Configure API keys
he config init
```

```bash
# List finance templates
he list template -l zh | grep finance
```

```bash
# Extract earnings report
he parse earnings_report.md -o finance_ka -t finance/earnings_summary -l zh
```

```bash
# Build index for search
he build-index finance_ka
```

```bash
# Search for insights
he search finance_ka "What was the revenue growth?"
```

--------------------------------

### Create and List Templates

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/core-concepts.md

Demonstrates how to create a Template instance from a preset or a local file, and how to list all available templates. Ensure the 'hyperextract' library is installed.

```python
from hyperextract import Template

# Create from preset
ka = Template.create("general/biography_graph", language="en")

# Create from file
ka = Template.create("/path/to/custom.yaml", language="en")

# List available
templates = Template.list()
for path, cfg in templates.items():
    print(f"{path}: {cfg.description}")
```

--------------------------------

### Usage Examples for Ingestion Script

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/knowledge-base/step2-ingest.md

Demonstrates how to run the ingestion script for initial ingestion or adding specific documents.

```bash
# Initial ingestion
python step2_ingest.py --initial

```

```bash
# Add specific documents
python step2_ingest.py --add documents/raw/2024/02/new_doc.md

```

```bash
# Add multiple documents
python step2_ingest.py --add doc1.md doc2.md doc3.md

```

--------------------------------

### Example Output for Defined Terms

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/templates/examples/legal-contract.md

Shows examples of extracted defined terms and their corresponding definitions from a legal contract.

```text
"Confidential Information": Any and all non-public, proprietary, or confidential information...

"Deliverables": The software, documentation, and materials to be delivered...

"Effective Date": The date first written above...
```

--------------------------------

### Complete CLI Workflow Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/cli-quickstart.md

Demonstrates a typical sequence of Hyper-Extract CLI commands for extracting, visualizing, searching, chatting, and feeding knowledge.

```bash
# 1. Extract knowledge
he parse document.md -t general/biography_graph -o ./output/ -l en

# 2. Visualize
he show ./output/

# 3. Search
he search ./output/ "your query"

# 4. Chat
he talk ./output/ -i

# 5. Add more documents
he feed ./output/ another_document.md

# 6. Rebuild index if needed
he build-index ./output/
```

--------------------------------

### Install Hyper-Extract CLI using uv

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Use this command to install the Hyper-Extract CLI tool with uv, the recommended package manager.

```bash
uv tool install hyperextract
```

--------------------------------

### Install Hyper-Extract and Configure API Key

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/examples/README.md

Instructions for installing the Hyper-Extract library using pip and configuring the OpenAI API key, either via a .env file or environment variables.

```bash
# Install hyperextract (dependencies are automatically installed)
uv pip install hyperextract
```

```bash
# Configure API key
# Option 1: Use .env file (recommended)
cp .env.example .env
# Then edit .env with your OPENAI_API_KEY and OPENAI_BASE_URL

# Option 2: Set environment variable
export OPENAI_API_KEY=your-key
```

--------------------------------

### Get Client from Config File

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/guides/provider-configuration.md

Initialize the Hyper-Extract client using settings from the configuration file (~/.he/config.toml).

```python
from hyperextract import get_client, AutoGraph

llm, emb = get_client()  # Reads ~/.he/config.toml
graph = AutoGraph(..., llm_client=llm, embedder=emb)
```

--------------------------------

### Example MCP Server Session

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/mcp.md

Demonstrates a typical interaction session with the MCP server, showcasing various tool calls and their expected outputs.

```text
list_templates()                        → [{name: "general/biography_graph", ...}, ...]
info(ka_path="./tesla_kb")              → {nodes: 48, edges: 70, index_built: true, ...}
search(ka_path="./tesla_kb",
       query="War of Currents")          → {nodes: [...], edges: [...]}
ask(ka_path="./tesla_kb",
    question="Who were Tesla's rivals?")  → "Thomas Edison ..."
export_obsidian(ka_path="./tesla_kb",
                output="./vault")          → "Exported 49 notes to ./vault"
```

--------------------------------

### Full Auto-Type Demos

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/examples/README.md

Shows how to run full auto-type extraction demos in both English and Chinese. Ensure you are in the examples directory first.

```bash
cd examples

# English
python en/autotypes/graph_demo.py
```

```bash
cd examples

# Chinese
python zh/autotypes/graph_demo.py
```

--------------------------------

### Setup Directory Structure and Add Documents

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/document-analysis/index.md

Creates the necessary directory structure for HyperExtract and copies markdown documents into the pending input directory.

```bash
mkdir -p input/pending input/processing input/completed output logs

cp /path/to/documents/*.md input/pending/
```

--------------------------------

### Run the Python script

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/research-assistant/step3-chat.md

Execute the Python script to start the interactive Q&A session.

```bash
python step3_chat.py
```

--------------------------------

### Hypergraph Grouping Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/SKILL.md

Demonstrates how to define members for hypergraph grouping, supporting both simple and nested structures.

```text
Simple | `participants` (string) | Flat list
Nested | `[group_a, group_b]` (list) | Semantic roles
```

--------------------------------

### Configure Bailian (Alibaba Cloud) API Key

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/cli-quickstart.md

Run this command to set up your Bailian API key for Hyper-Extract. This is a one-time setup.

```bash
he config init -p bailian -k YOUR_BAILIAN_API_KEY
```

--------------------------------

### Configure OpenAI API Key

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/cli-quickstart.md

Run this command to set up your OpenAI API key for Hyper-Extract. This is a one-time setup.

```bash
he config init -p openai -k YOUR_OPENAI_API_KEY
```

--------------------------------

### Install Hyper-Extract with MCP Support

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/README.md

Install the Hyper-Extract library with the necessary dependencies for the MCP server. This command installs the core library and the 'mcp' extra.

```bash
pip install 'hyperextract[mcp]'
```

--------------------------------

### Run Knowledge Base Manager Setup

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/knowledge-base/step1-setup.md

Execute the Python script to initialize the knowledge base. This command assumes `kb_manager.py` and `config.yaml` are in the current directory.

```bash
python kb_manager.py
```

--------------------------------

### Install Hyper-Extract CLI using pipx

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Install the Hyper-Extract CLI tool using pipx, a tool for installing Python applications in isolated environments.

```bash
pipx install hyperextract
```

--------------------------------

### Verify CLI Installation

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Run this command to verify that the Hyper-Extract CLI has been installed correctly and to check its version.

```bash
he --version
```

--------------------------------

### Verify Python Library Installation

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Execute this Python code to confirm the Hyper-Extract library is installed and to print its version.

```python
import hyperextract
print(hyperextract.__version__)
```

--------------------------------

### Create Sample Text File

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/cli-quickstart.md

Create a sample text file named 'sample.txt' with provided content using a here-string. This can be used as an alternative to downloading a sample document.

```bash
cat > sample.txt << 'EOF'
Nikola Tesla was a Serbian-American inventor, electrical engineer, 
mechanical engineer, and futurist. He is best known for his 
contributions to the design of the modern alternating current 
(AC) electricity supply system.

Born: July 10, 1856, Smiljan, Croatia
Died: January 7, 1943, New York City, NY

Tesla immigrated to the United States in 1884 and briefly worked 
with Thomas Edison before the two parted ways due to conflicting 
business and scientific interests. He later established his own 
laboratory and developed numerous revolutionary inventions, 
including the Tesla coil, induction motor, and wireless transmission 
technologies.

Despite his brilliance, Tesla struggled financially in his later years
and died impoverished in a New York hotel room. His legacy was 
largely overlooked during his lifetime but has since been recognized 
worldwide, with the Tesla unit of magnetic flux density named in his honor.
EOF
```

--------------------------------

### Install Hyper-Extract Skills using Plugin Command

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/README.md

Alternatively, install the hyperextract-skills using the plugin command for Claude Code.

```bash
/plugin install hyperextract-skills
```

--------------------------------

### Install hyperextract in a Virtual Environment

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/troubleshooting.md

Use this command to install hyperextract within a virtual environment to avoid dependency conflicts.

```bash
python -m venv venv
source venv/bin/activate
pip install hyperextract
```

--------------------------------

### Extract Legal Contracts Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/cli/README.md

Example for extracting legal contract information, including template listing, parsing, and visualization.

```bash
# List legal templates
he list template -l zh | grep legal
```

```bash
# Extract contract information
he parse contract.md -o legal_ka -t legal/contract_summary -l zh
```

```bash
# View as knowledge graph
he show legal_ka
```

--------------------------------

### Initialize Configuration with API Key

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/configuration.md

Use this command to initialize the configuration with a specific provider and API key. This is often a first step when encountering 'API key not found' errors.

```bash
he config init -p bailian -k YOUR_API_KEY
```

--------------------------------

### Initialize OpenAI Configuration

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/configuration.md

Quickly set up OpenAI configuration using the 'init' command with provider and API key flags.

```bash
he config init -p openai -k sk-your-key
```

--------------------------------

### Biography Graph Example Output

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/templates/reference/general.md

Example JSON output structure for the biography_graph template, showing extracted entities and their relations.

```json
{
    "entities": [
        {"name": "Nikola Tesla", "type": "person"},
        {"name": "AC Motor", "type": "invention"}
    ],
    "relations": [
        {
            "source": "Nikola Tesla",
            "target": "AC Motor",
            "type": "invented",
            "time": "1888"
        }
    ]
}
```

--------------------------------

### Create Initial Configuration File

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/knowledge-base/step1-setup.md

Use this command to create a basic `config.yaml` file for your project. This sets up the essential parameters for HyperExtract.

```bash
cat > config.yaml << 'EOF'
name: "My Knowledge Abstract"
domain: "general"
template: "general/graph"
language: "en"
documents_dir: "./documents/raw"
ka_dir: "./ka"
backup_dir: "./backups"
EOF
```

--------------------------------

### Install Hyper-Extract as Python Library using pip

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Install Hyper-Extract as a Python library using pip for integration into your Python code.

```bash
pip install hyperextract
```

--------------------------------

### Initialize Bailian Configuration

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/configuration.md

Set up Bailian (Alibaba Cloud) configuration with the 'init' command, using the specified provider and API key. Defaults to qwen3.6-plus and text-embedding-v4.

```bash
he config init -p bailian -k sk-your-key
```

--------------------------------

### Create Project Directories

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/knowledge-base/step1-setup.md

Use this command to create the necessary directories for your knowledge base project. Navigate into the created directory afterward.

```bash
mkdir -p my_ka/{documents,ka,backups,logs}
cd my_ka
```

--------------------------------

### Build Documentation

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/contributing.md

Generate the static files for the documentation site.

```bash
mkdocs build
```

--------------------------------

### Install Hyper-Extract as Python Library using uv

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/installation.md

Install Hyper-Extract as a Python library using uv for use within your Python projects.

```bash
uv pip install hyperextract
```

--------------------------------

### Create Template and Method Instances

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/guides/using-methods.md

Instantiate a template for a pre-configured domain solution or a method for a pure extraction algorithm. Methods are lighter weight and ideal for switching algorithms.

```python
# Using a template
ka = Template.create("general/biography_graph", language="en")

# Using a method
ka = Template.create("method/light_rag")
```

--------------------------------

### Field Naming Standardization Examples

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/template-optimizer/SKILL.md

Provides examples of common field name transformations to standardize naming conventions for better clarity and consistency.

```text
❌ relation_type → type
❌ event_date → time
❌ entity_type → type
```

--------------------------------

### Build Knowledge Base Incrementally

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/guides/incremental-updates.md

This example demonstrates building a knowledge base incrementally from multiple markdown files. It includes parsing the first file, feeding subsequent files, building the search index, and saving the final knowledge base.

```python
"""Build a knowledge abstract incrementally from multiple sources."""

from hyperextract import Template
from pathlib import Path

def build_knowledge_base(source_dir, output_dir):
    ka = Template.create("general/biography_graph", "en")
    
    # Get all text files
    files = sorted(Path(source_dir).glob("*.md"))
    
    if not files:
        print("No files found")
        return
    
    # Initial extraction from first file
    print(f"Processing {files[0].name}...")
    ka = ka.parse(files[0].read_text())
    
    # Feed remaining files
    for file in files[1:]:
        print(f"Adding {file.name}...")
        ka.feed_text(file.read_text())
        print(f"  Now {len(ka.nodes)} nodes")
    
    # Build index for search/chat
    print("Building search index...")
    ka.build_index()
    
    # Save
    print(f"Saving to {output_dir}...")
    ka.dump(output_dir)
    
    print("Done!")
    return ka

# Usage
ka = build_knowledge_base("./sources/", "./combined_ka/")
```

--------------------------------

### Basic Usage of he show

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/show.md

Demonstrates the simplest way to use the 'he show' command by providing the output directory path.

```bash
he show ./output/
```

--------------------------------

### Custom Template YAML Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/architecture.md

Provides an example of a custom template file in YAML format, defining a new extraction type named 'custom_extraction' with specific entity fields.

```yaml
# my_template.yaml
name: custom_extraction
type: graph
output:
  entities:
    fields:
      - name: name
        type: str
# ...
```

--------------------------------

### Initialize Configuration with API Key

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/config.md

Command to initialize the Hyper-Extract configuration and set the API key. This is useful when the configuration file is not found.

```bash
he config init -k your-api-key
```

--------------------------------

### List All Templates

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/list.md

Displays all available knowledge templates. Use this to see the full range of pre-defined extraction configurations.

```bash
he list template
```

--------------------------------

### Complete Biography Graph Template Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/templates-format.md

A full example of a temporal graph template for extracting entities and relationships from biographies. It includes language specifications, metadata, descriptions for entities and relations, and guidelines for extraction.

```yaml
language: [zh, en]

name: biography_graph
type: temporal_graph
tags: [general, biography, life, events, timeline]

description:
  zh: '传记图模板 - 从人物传记、回忆录、年谱中提取实体和关系'
  en: 'Biography Graph Template - Extract entities and relationships from biographies'

output:
  description:
    zh: '人物传记中的实体和关系网络'
    en: "Entity and relationship network from biographies"
  
  entities:
    description:
      zh: '传记中的实体'
      en: 'Entities in the biography'
    fields:
      - name: name
        type: str
        description:
          zh: '实体名称'
          en: 'Entity name'
        required: true
      
      - name: type
        type: str
        description:
          zh: '实体类型：人物、地点、组织、作品、事件、成就等'
          en: 'Entity type: person, location, organization, work, event, achievement'
        required: true
      
      - name: description
        type: str
        description:
          zh: '实体描述'
          en: 'Entity description'
        required: false
  
  relations:
    description:
      zh: '实体之间的关系'
      en: 'Relationships between entities'
    fields:
      - name: source
        type: str
        description:
          zh: '关系起点实体'
          en: 'Source entity'
        required: true
      
      - name: target
        type: str
        description:
          zh: '关系终点实体'
          en: 'Target entity'
        required: true
      
      - name: type
        type: str
        description:
          zh: '关系类型'
          en: 'Relation type'
        required: true
      
      - name: time
        type: str
        description:
          zh: '关系对应的时间'
          en: 'Time associated with the relation'
        required: false

guideline:
  target:
    zh: '你是一位专业的传记分析师...'
    en: 'You are a professional biography analyst...'
  
  rules_for_entities:
    zh:
      - '提取传记主人公作为核心实体'
      - '同一实体在全文中保持命名一致'
    en:
      - 'Extract the biographical subject as the core entity'
      - 'Maintain consistent naming for the same entity'
  
  rules_for_relations:
    zh:
      - '仅在文本明确表达两个实体之间存在关系时创建关系'
    en:
      - 'Create relationships only when explicitly expressed'
  
  rules_for_time:
    zh:
      - '绝对日期：保持原文格式'
      - '相对时间：转换为绝对时间'
    en:
      - 'Absolute dates: Keep as-is'
      - 'Relative time: Convert to absolute'

identifiers:
  entity_id: name
  relation_id: '{source}|{type}|{target}'
  relation_members:
    source: source
    target: target
  time_field: time

display:
  entity_label: '{name}'
  relation_label: '{type}'

```

--------------------------------

### Choose a Method

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/list.md

Demonstrates how to list available methods and then use a specific method for parsing a document, bypassing template selection. Useful for research or custom extraction.

```bash
# See available methods
he list method

# Use one instead of template
he parse doc.md -m light_rag -o ./out/
```

--------------------------------

### Upgrade Hyper-Extract

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/faq.md

Upgrades an existing Hyper-Extract installation to the latest version.

```bash
pip install --upgrade hyperextract
```

--------------------------------

### Load and Use a Template

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/templates/README.md

Load a template configuration from the gallery using its identifier and create a template instance. Then, use the template to parse document text.

```python
config = Gallery.get("general/workflow_graph")

template = TemplateFactory.create(config, llm, embedder)

result = template.parse("Your document text here...")
```

--------------------------------

### Template Creation Methods

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/index.md

Shows different ways to create a Template instance: from a preset, a custom YAML file, or a predefined method.

```python
from hyperextract import Template

# Create from preset
ka = Template.create("general/biography_graph", language="en")

# Create from custom YAML
ka = Template.create("/path/to/custom_template.yaml", language="en")

# Create from method
ka = Template.create("method/light_rag")
```

--------------------------------

### AutoSpatialGraph Example Output

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/autotypes.md

Represents relationships with location information. Entities or relationships can have geographic coordinates.

```json
{
    "entities": [
        {"name": "Colorado Springs", "location": "38.83,-104.82"}
    ],
    "relations": [
        {
            "source": "Tesla",
            "target": "Colorado Springs",
            "type": "conducted_experiments"
        }
    ]
}
```

--------------------------------

### Install Hyper-Extract Skills for Trae

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/README.md

Use this command to copy the hyperextract-skills folder to your Trae skills directory.

```bash
cp -r hyperextract-skills ~/.trae/skills/
```

--------------------------------

### Start Local vLLM LLM Service

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/provider-system.md

Launch a local vLLM LLM service using the `vllm serve` command. Configure model path, quantization, and other parameters for optimal performance.

```bash
vllm serve /path/to/qwen3.5-9b-gptq-marlin \
  --served-model-name Qwen/Qwen3.5-9B \
  --trust-remote-code \
  --quantization gptq_marlin \
  --dtype bfloat16 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.90 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --port 8000 \
  --api-key dummy
```

--------------------------------

### Configuration: Custom Clients

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/index.md

Demonstrates how to use custom LLM and embedding clients, such as those from Langchain, with the Hyper-Extract Template.

```python
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from hyperextract import Template

llm = ChatOpenAI(model="gpt-4o")
embedder = OpenAIEmbeddings(model="text-embedding-3-large")

ka = Template.create(
    "general/biography_graph",
    language="en",
    llm_client=llm,
    embedder=embedder
)
```

--------------------------------

### AutoSpatioTemporalGraph Example Output

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/autotypes.md

Represents relationships with both time and location dimensions, providing a complete context for events.

```json
{
    "relations": [
        {
            "source": "Tesla",
            "target": "AC Motor",
            "type": "demonstrated",
            "time": "1888",
            "location": "Pittsburgh"
        }
    ]
}
```

--------------------------------

### AutoTemporalGraph Example Output

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/autotypes.md

Represents relationships with an added time dimension. Each relationship includes a time component.

```json
{
    "edges": [
        {
            "source": "Tesla",
            "target": "AC Motor",
            "type": "invented",
            "time": "1888"
        }
    ]
}
```

--------------------------------

### Get Template Information (CLI)

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/templates/reference/overview.md

Retrieve detailed information about a specific template using its identifier in the CLI.

```bash
# Get template info
he info template general/biography_graph
```

--------------------------------

### Troubleshooting: No API Key Found

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/getting-started/cli-quickstart.md

If you encounter a 'No API key found' error, run this command to configure your API key.

```bash
he config init -k YOUR_API_KEY
```

--------------------------------

### Model Template: Schema and Guideline

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/template-optimizer/references/rules-consistency.md

Provides an example of a schema definition for company names and corresponding guidelines for extracting them.

```yaml
# Schema
output:
  fields:
    - name: company_name
      type: str
      description: 'Company name'

# Guideline
guideline:
  rules:
    - 'Extract official company names'
    - 'Use abbreviations if that's how they appear'
    - 'Standardize formats like "Inc.", "Ltd."
```

--------------------------------

### Initialize Configuration with Custom Endpoint

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/cli/README.md

Initializes Hyper-Extract configuration with a custom API key and endpoint URL.

```bash
he config init -k YOUR_KEY -u https://your-endpoint.com/v1
```

--------------------------------

### List All Templates

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/guides/using-templates.md

Retrieves all available templates and prints their paths and descriptions. Requires importing the Template class.

```python
from hyperextract import Template

templates = Template.list()

for path, cfg in templates.items():
    print(f"{path}: {cfg.description}")
```

--------------------------------

### AutoHypergraph Example Output

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/autotypes.md

Represents relationships involving three or more entities. A single edge can connect multiple nodes.

```json
{
    "nodes": [...],
    "edges": [
        {
            "entities": ["Tesla", "Westinghouse", "Niagara"],
            "type": "collaboration"
        }
    ]
}
```

--------------------------------

### AutoGraph Example Output

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/autotypes.md

Represents binary relationships between two entities. Each edge connects exactly two nodes.

```json
{
    "nodes": [
        {"name": "Tesla", "type": "person"},
        {"name": "AC Motor", "type": "invention"}
    ],
    "edges": [
        {"source": "Tesla", "target": "AC Motor", "type": "invented"}
    ]
}
```

--------------------------------

### Method Registry Example

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/concepts/architecture.md

Illustrates how methods are registered within the system, mapping a method name to its class, type, and a description. This is crucial for the plugin architecture.

```python
# Methods are registered at import
_METHOD_REGISTRY = {
    "light_rag": {
        "class": Light_RAG,
        "type": "graph",
        "description": "..."
    },
    # ...
}
```

--------------------------------

### Template Example: Knowledge Graph

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/README.md

YAML configuration for a 'Knowledge Graph' template, defining entity and relation fields.

```yaml
language: en
name: Knowledge Graph
type: graph
tags: [general]
description: 'Extract entities and their relationships.'
output:
  entities:
    fields:
    - name: name
      type: str
    - name: type
      type: str
    - name: description
      type: str
  relations:
    fields:
    - name: source
      type: str
    - name: target
      type: str
    - name: type
      type: str
identifiers:
  entity_id: name
  relation_id: '{source}|{type}|{target}'
```

--------------------------------

### Create Knowledge Base Backup

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/tutorials/knowledge-base/step3-query.md

Creates a backup of the current knowledge base by copying the 'current' directory to the backup directory. Handles cases where no current version exists.

```python
def create_backup(self, name: Optional[str] = None):
    """Create a backup of current knowledge abstract."""
    import shutil
    
    current = Path(self.config.kb_dir) / "current"
    if not current.exists():
        raise ValueError("No current version to backup")
    
    if name is None:
        name = datetime.now().strftime("backup_%Y%m%d_%H%M%S")
    
    backup_path = Path(self.config.backup_dir) / name
    shutil.copytree(current, backup_path)
    
    print(f"✓ Backup created: {backup_path}")
    return backup_path
```

--------------------------------

### List All Methods

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/cli/commands/list.md

Displays all available extraction methods. Use this to see the different algorithms that can be employed for extraction.

```bash
he list method
```

--------------------------------

### Create and Parse with Default Configuration

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/guides/using-methods.md

Use the default configuration to create a template and parse text. This is suitable for general-purpose extraction tasks.

```python
from hyperextract import Template

# Create and extract
ka = Template.create("method/light_rag")
result = ka.parse(text)
```

--------------------------------

### View Specific Command Help

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/cli/README.md

Get detailed help for a specific HyperExtract command, such as 'parse', 'config', or 'search'.

```bash
he parse --help
```

```bash
he config --help
```

```bash
he search --help
```

--------------------------------

### Building a Knowledge Abstract with Incremental Updates

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/python/index.md

Demonstrates building a knowledge abstract by parsing an initial document and then incrementally feeding more documents. The knowledge abstract can be saved for later use.

```python
from hyperextract import Template

ka = Template.create("general/graph", language="en")

# Initial extraction
ka = ka.parse(doc1_text)

# Add more documents
ka.feed_text(doc2_text)
ka.feed_text(doc3_text)

# Save for later
ka.dump("./my_ka/")
```

--------------------------------

### Get Template Configuration Details

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/templates/how-to-choose.md

Retrieve the configuration object for a specific template (e.g., 'general/biography_graph') and access its description.

```python
cfg = Template.get("general/biography_graph")
print(cfg.description)
```

--------------------------------

### AutoType Quick Reference

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract/templates/DESIGN_GUIDE.md

A quick reference guide mapping data needs to the appropriate AutoType data structure.

```text
Need single object → model
Need list → list
Need deduplication → set
Need binary relations → graph
Need multi-party relations → hypergraph
Need time → temporal_graph
Need location → spatial_graph
Need both → spatio_temporal_graph
```

--------------------------------

### Install Hyper-Extract Skills for Claude Code

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/hyperextract-skills/README.md

Use this command to copy the hyperextract-skills folder to your Claude Code skills directory.

```bash
cp -r hyperextract-skills ~/.claude/skills/
```

--------------------------------

### Initiate Chat with Index

Source: https://github.com/yifanfeng97/hyper-extract/blob/main/docs/en/resources/troubleshooting.md

If chat fails due to a missing index, ensure the index is built first, then initiate the chat.

```bash
he build-index ./output/
he talk ./output/ -q "your question"
```
Tokens:
Raw