Recursive Language Models (RLM)

https://github.com/ysz/recursive-llm
Admin
Recursive Language Models (RLM) is a Python implementation that enables language models to process...
Tokens:7,485
Snippets:29
Trust Score:7.1
License:MIT
Update:11 months ago
Show doc for...
Tokens:
Raw
### Running RLM Examples (Bash)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Provides instructions on how to execute the example scripts included in the RLM project. It emphasizes the prerequisite of setting the necessary API key environment variable before running the Python example scripts.

```bash
# Set your API key first
export OPENAI_API_KEY="sk-..."

# Run example
python examples/basic_usage.py
```

--------------------------------

### RLM Model Selection Examples (Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Shows how to initialize RLM with various language models, including OpenAI, Anthropic, and local models via Ollama or llama.cpp. It highlights the flexibility provided by LiteLLM for supporting over 100 LLM providers.

```python
# OpenAI
rlm = RLM(model="gpt-5")
rlm = RLM(model="gpt-5-mini")

# Anthropic
rlm = RLM(model="claude-sonnet-4")
rlm = RLM(model="claude-sonnet-4-20250514")

# Ollama (local)
rlm = RLM(model="ollama/llama3.2")
rlm = RLM(model="ollama/mistral")

# llama.cpp (local)
rlm = RLM(
    model="openai/local",
    api_base="http://localhost:8000/v1"
)

# Azure OpenAI
rlm = RLM(model="azure/gpt-4-deployment")
```

--------------------------------

### RLM Installation from Source (Bash)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Details the steps for installing the Recursive Language Models (RLM) package directly from its GitHub repository. This involves cloning the repository, navigating into the directory, and then installing it using pip in editable mode, optionally with development dependencies.

```bash
# Clone the repository
git clone https://github.com/ysz/recursive-llm.git
cd recursive-llm

# Install in editable mode
pip install -e .

# Or install with dev dependencies
pip install -e ".[dev]"
```

--------------------------------

### RLM Async API Usage (Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Provides an example of using the asynchronous API (`acompletion`) for RLM, which is beneficial for performance when dealing with parallel recursive calls. It includes the necessary `asyncio` setup for running the asynchronous function.

```python
import asyncio

async def main():
    rlm = RLM(model="gpt-5-mini")
    result = await rlm.acompletion(query, context)
    print(result)

asyncio.run(main())
```

--------------------------------

### RLM How It Works: Context Exploration (Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

This snippet is a conceptual example illustrating how an RLM might interact with context within a Python REPL environment. It shows methods like peeking at the start of the context, searching using regular expressions, and making recursive calls to process specific segments of the context.

```python
# Peek at context
context[:1000]

# Search with regex
import re
re.findall(r'pattern', context)

# Recursive processing
recursive_llm("extract dates", context[1000:2000])
```

--------------------------------

### API Key Setup for RLM (Bash and Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Illustrates two methods for setting up API keys for LLM providers used by RLM: setting an environment variable (e.g., OPENAI_API_KEY) in the bash shell, or passing the API key directly as an argument during RLM initialization in Python.

```bash
export OPENAI_API_KEY="sk-..."  # or ANTHROPIC_API_KEY, etc.
```

```python
rlm = RLM(model="gpt-5-mini", api_key="sk-...")
```

--------------------------------

### RLM Multiple LLM Provider Integration

Source: https://context7.com/ysz/recursive-llm/llms.txt

This Python code illustrates how to configure and use the RLM library with a wide array of Large Language Model (LLM) providers, leveraging LiteLLM for compatibility. It shows examples for OpenAI (including Azure), Anthropic Claude, Google Gemini, local Ollama, and llama.cpp setups, demonstrating initialization with different model names and API endpoints/keys as required by each provider.

```python
from rlm import RLM

# OpenAI
rlm_openai = RLM(model="gpt-4o")
rlm_openai_mini = RLM(model="gpt-4o-mini")

# Anthropic Claude
rlm_claude = RLM(model="claude-sonnet-4-20250514")
rlm_claude_opus = RLM(model="claude-opus-4-20250514")

# Google Gemini
rlm_gemini = RLM(model="gemini/gemini-pro")

# Ollama (local)
rlm_ollama = RLM(model="ollama/llama3.2")
rlm_ollama_mistral = RLM(model="ollama/mistral")

# llama.cpp (local)
rlm_llamacpp = RLM(
    model="openai/local",
    api_base="http://localhost:8000/v1"
)

# Azure OpenAI
rlm_azure = RLM(
    model="azure/gpt-4-deployment-name",
    api_base="https://your-resource.openai.azure.com/",
    api_key="your-azure-key"
)

```

--------------------------------

### Environment Setup and API Key Configuration in Python

Source: https://context7.com/ysz/recursive-llm/llms.txt

Shows how to configure API keys for the RLM library using environment variables or directly in code. It covers loading from a .env file and verifying the presence of the API key.

```python
import os
from dotenv import load_dotenv
from rlm import RLM

# Method 1: Environment variables
load_dotenv()  # Loads from .env file
# .env file contents:
# OPENAI_API_KEY=sk-...
# ANTHROPIC_API_KEY=sk-ant-...

rlm = RLM(model="gpt-4o-mini")  # Uses OPENAI_API_KEY from environment

# Method 2: Direct in code
rlm = RLM(
    model="gpt-4o-mini",
    api_key="sk-..."
)

# Method 3: Shell export
# $ export OPENAI_API_KEY="sk-..."
# Then in Python:
rlm = RLM(model="gpt-4o-mini")

# Verify API key is set
if not os.getenv("OPENAI_API_KEY"):
    print("Error: OPENAI_API_KEY not found!")
    print("Set it with: export OPENAI_API_KEY='sk-...'`)
    exit(1)
```

--------------------------------

### Local LLM with Ollama using RLM

Source: https://context7.com/ysz/recursive-llm/llms.txt

This Python example demonstrates how to use the RLM library with a local LLM model served by Ollama. It initializes RLM with an Ollama model identifier and performs a query on inventory data. The example highlights the ability to run LLM tasks locally without external API keys. The output shows the result of the inventory value analysis.

```python
from rlm import RLM

# Use local Ollama model
rlm = RLM(
    model="ollama/llama3.2",  # No API key required
    max_iterations=15,
    temperature=0.7
)

# Inventory data
inventory = """
Product Inventory Report - Q1 2025

Electronics:
- Laptops: 150 units in stock, $899 each
- Tablets: 220 units in stock, $499 each
- Smartphones: 340 units in stock, $699 each

Accessories:
- Headphones: 500 units in stock, $79 each
- Cases: 800 units in stock, $29 each
- Chargers: 450 units in stock, $19 each

Low Stock Alert:
- Laptops below threshold (target: 200)
"""

result = rlm.completion(
    "Which product category has the most total value in inventory?",
    inventory
)
print(result)
```

--------------------------------

### RLM Initialization and Completion (Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Demonstrates the basic usage of the RLM library to initialize an RLM instance with a specified model and perform a completion task on a long document. The context is passed as a variable, not directly in the prompt, to avoid context rot.

```python
from rlm import RLM

# Initialize with any LLM
rlm = RLM(model="gpt-5-mini")

# Process long context
result = rlm.completion(
    query="What are the main themes in this document?",
    context=long_document
)
print(result)
```

--------------------------------

### RLM Custom Configuration Parameters (Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Illustrates how to customize RLM's behavior by passing various configuration parameters during initialization. This includes setting maximum recursion depth, iteration limits, LLM temperature, and request timeouts.

```python
rlm = RLM(
    model="gpt-5-mini",
    max_depth=5,         # Maximum recursion depth
    max_iterations=20,   # Maximum REPL iterations
    temperature=0.7,     # LLM parameters
    timeout=60
)
```

--------------------------------

### RLM Advanced Configuration: Two Models (Python)

Source: https://github.com/ysz/recursive-llm/blob/main/README.md

Demonstrates an advanced RLM usage pattern where two different models are specified: a primary 'model' for main decisions and a 'recursive_model' for cheaper recursive calls, optimizing cost for long context processing.

```python
rlm = RLM(
    model="gpt-5",              # Root LM (main decisions)
    recursive_model="gpt-5-mini"  # Recursive calls (cheaper)
)
```

--------------------------------

### REPL Environment Operations in Python

Source: https://context7.com/ysz/recursive-llm/llms.txt

Demonstrates various operations within a safe Python REPL environment accessible to the LLM. Includes context peeking, regex searching, counting, chunking, recursive calls, data extraction, and using collections.Counter.

```python
# Inside the REPL environment, the LLM can write code like:

# Peek at the beginning of context
context[:1000]

# Search with regex
import re
dates = re.findall(r'\d{4}-\d{2}-\d{2}', context)
print(f"Found {len(dates)} dates: {dates}")

# Count occurrences
error_count = len(re.findall(r'ERROR', context))
warning_count = len(re.findall(r'WARN', context))
print(f"Errors: {error_count}, Warnings: {warning_count}")

# Split and analyze chunks
chunks = [context[i:i+1000] for i in range(0, len(context), 1000)]
chunk_lengths = [len(chunk) for chunk in chunks]
print(f"Split into {len(chunks)} chunks")

# Recursive processing of sub-sections
chapter_1 = context[1000:5000]
summary = recursive_llm("Summarize this chapter", chapter_1)
print(summary)

# Extract structured data
import json
numbers = re.findall(r'\$[\\d,]+', context)
amounts = [int(n.replace('$', '').replace(',', '')) for n in numbers]
total = sum(amounts)
print(f"Total: ${total:,}")

# Use Counter for aggregation
from collections import Counter
words = context.lower().split()
common_words = Counter(words).most_common(10)
print(common_words)

# Return final answer
FINAL("The analysis shows...")
```

--------------------------------

### Basic RLM Completion with Long Document in Python

Source: https://context7.com/ysz/recursive-llm/llms.txt

Demonstrates initializing RLM with a specified model and processing a very long document with a query. It shows how to print the result and inspect execution statistics like LLM calls and iterations. This is useful for understanding the fundamental usage of RLM for long-context tasks.

```python
from rlm import RLM

# Initialize with any supported LLM
rlm = RLM(model="gpt-4o-mini")

# Long document example
document = """
The History of Artificial Intelligence

Introduction
Artificial Intelligence (AI) has transformed from a theoretical concept to a practical reality
over the past several decades. This document explores key milestones in AI development.

The 1950s: The Birth of AI
In 1950, Alan Turing published "Computing Machinery and Intelligence," introducing the famous
Turing Test. The term "Artificial Intelligence" was coined in 1956 at the Dartmouth Conference
by John McCarthy, Marvin Minsky, and others.

The 1960s-1970s: Early Optimism
During this period, researchers developed early AI programs like ELIZA (1966) and expert systems. 
However, limitations in computing power led to the first "AI Winter" in the 1970s.

The 1980s-1990s: Expert Systems and Neural Networks
Expert systems became commercially successful in the 1980s. The backpropagation algorithm
revitalized neural network research in 1986.

The 2000s-2010s: Machine Learning Revolution
The rise of big data and powerful GPUs enabled deep learning breakthroughs. In 2012,
AlexNet won the ImageNet competition, marking a turning point for deep learning.

The 2020s: Large Language Models
GPT-3 (2020) and ChatGPT (2022) demonstrated unprecedented language understanding capabilities.
These models have billions of parameters and are trained on vast amounts of text data.

Conclusion
AI continues to evolve rapidly, with applications in healthcare, transportation, education,
and countless other domains. The future promises even more exciting developments.
""" * 50  # Can handle very long documents

# Process the document
query = "What were the key milestones in AI development according to this document?"
result = rlm.completion(query, document)

print(result)
# Output: "The key milestones include:
# 1. 1950: Alan Turing's "Computing Machinery and Intelligence" and Turing Test
# 2. 1956: Term "Artificial Intelligence" coined at Dartmouth Conference
# 3. 1966: ELIZA developed
# 4. 1970s: First AI Winter due to computing limitations
# 5. 1980s: Expert systems become commercially successful
# 6. 1986: Backpropagation algorithm revitalizes neural networks
# 7. 2012: AlexNet wins ImageNet competition
# 8. 2020: GPT-3 released
# 9. 2022: ChatGPT launched"

# Check execution statistics
print(f"LLM calls: {rlm.stats['llm_calls']}")
print(f"Iterations: {rlm.stats['iterations']}")
# Output: LLM calls: 3, Iterations: 2

```

--------------------------------

### Asynchronous RLM Completion for Multiple Queries in Python

Source: https://context7.com/ysz/recursive-llm/llms.txt

Illustrates using the asynchronous capabilities of RLM for concurrent processing of multiple queries against a document. This approach enhances throughput by performing recursive calls in parallel. It requires the `asyncio` library and demonstrates how to gather results from multiple `acompletion` tasks.

```python
import asyncio
from rlm import RLM

async def process_documents():
    rlm = RLM(
        model="gpt-4o-mini",
        max_iterations=20,
        temperature=0.7
    )

    # Process multiple queries concurrently
    queries = [
        "Summarize the main themes",
        "Extract all important dates",
        "List key people mentioned"
    ]

    tasks = [
        rlm.acompletion(query, document)
        for query in queries
    ]

    results = await asyncio.gather(*tasks)

    for query, result in zip(queries, results):
        print(f"
{query}:\n{result}")

    return results

# Run async code
asyncio.run(process_documents())

```

--------------------------------

### RLM Advanced Configuration and Error Handling

Source: https://context7.com/ysz/recursive-llm/llms.txt

This Python snippet shows advanced configuration options for the RLM library, including setting recursive model, depth, iterations, temperature, timeout, max tokens, custom API base, and API key. It also demonstrates robust error handling for potential issues like exceeding iteration limits (MaxIterationsError), recursion depth limits (MaxDepthError), and code execution failures (REPLError), along with a general exception catch.

```python
from rlm import RLM, MaxIterationsError, MaxDepthError, REPLError

# Comprehensive configuration
rlm = RLM(
    model="gpt-4o-mini",
    recursive_model="gpt-4o-mini",
    max_depth=5,              # Maximum recursion depth
    max_iterations=30,        # Maximum REPL iterations per call
    temperature=0.7,          # LLM temperature
    timeout=60,               # Request timeout in seconds
    max_tokens=1000,          # Max tokens in response
    api_base="https://api.openai.com/v1",  # Custom API endpoint
    api_key="sk-...",         # Explicit API key
)

# Error handling
try:
    result = rlm.completion(
        query="Complex analysis task",
        context=very_long_document
    )
    print(f"Success: {result}")

except MaxIterationsError as e:
    print(f"Hit iteration limit: {e}")
    print("Consider increasing max_iterations or simplifying the query")

except MaxDepthError as e:
    print(f"Hit recursion depth limit: {e}")
    print("The query required too many recursive subdivisions")

except REPLError as e:
    print(f"Code execution failed: {e}")
    print("The LLM generated invalid Python code")

except Exception as e:
    print(f"Unexpected error: {e}")

```

--------------------------------

### Extract Structured Data from Meeting Notes with RLM

Source: https://context7.com/ysz/recursive-llm/llms.txt

Demonstrates using RLM with a lighter model (gpt-4o-mini) for detailed data extraction from meeting notes. It extracts specific information types such as dates, numerical metrics, action items, and customer requests, showcasing RLM's capability in parsing unstructured text into organized data.

```python
from rlm import RLM

# Meeting notes with embedded data
meeting_notes = """
Meeting Notes - Product Planning Session
Date: January 15, 2025
Attendees: Sarah Chen, Mike Johnson, Lisa Park, David Kim

Discussion Topics:

Sarah mentioned that we need to increase our Q1 budget to $250,000 to accommodate
the new marketing campaign. She also noted that our customer satisfaction score
improved from 7.5 to 8.9 in the last quarter.

Mike presented the engineering roadmap. The team plans to ship Feature A by February 15,
Feature B by March 30, and Feature C by April 20. He mentioned they need 3 additional
engineers to meet these deadlines.

Lisa reported that website traffic increased 45% last month, with 125,000 unique visitors.
The conversion rate improved from 2.1% to 3.4%. Email campaign open rates are at 28%.

David shared customer feedback. Key requests include:
- Mobile app improvements (mentioned by 89 customers)
- Better search functionality (67 customers)
- Dark mode support (134 customers)
- Faster load times (45 customers)

Action Items:
- Sarah: Approve budget increase by Jan 20
- Mike: Post job listings for 3 engineers
- Lisa: Launch new email campaign by Feb 1
- David: Prioritize dark mode feature

Next meeting: February 15, 2025 at 2:00 PM
"""

rlm = RLM(
    model="gpt-4o-mini",
    max_iterations=15,
    temperature=0.3  # Lower temperature for precise extraction
)

# Extract dates
result = rlm.completion("Extract all dates mentioned in the document", meeting_notes)
print(result)
# Output: "January 15, 2025, January 20, February 1, February 15, 2025 (2:00 PM),
# February 15, March 30, April 20"

# Extract metrics
result = rlm.completion("Extract all numerical metrics (percentages, counts, etc.)", meeting_notes)
print(result)
# Output: "$250,000 budget, satisfaction score 7.5 to 8.9, 45% traffic increase,
# 125,000 visitors, conversion 2.1% to 3.4%, 28% email open rate, 89 customers
# (mobile), 67 customers (search), 134 customers (dark mode), 45 customers (speed)"

# Extract action items
result = rlm.completion("List all action items with assigned owners", meeting_notes)
print(result)
# Output: "1. Sarah: Approve budget increase by Jan 20
# 2. Mike: Post job listings for 3 engineers
# 3. Lisa: Launch new email campaign by Feb 1
# 4. David: Prioritize dark mode feature"

# Top customer requests
result = rlm.completion("What are the top 3 customer feature requests by number of requests?", meeting_notes)
print(result)
# Output: "1. Dark mode support (134 customers)
# 2. Mobile app improvements (89 customers)
# 3. Better search functionality (67 customers)"

```

--------------------------------

### Multi-Document Processing and Cross-Document Queries in Python

Source: https://context7.com/ysz/recursive-llm/llms.txt

Illustrates how to combine multiple documents into a single string and perform cross-document queries using the RLM library. This is useful for analyzing related information from various sources.

```python
from rlm import RLM

# Multiple product documentation files
doc1 = """CloudSync API Documentation
Authentication: OAuth 2.0
Rate Limit: 1000 requests/hour
Base URL: https://api.cloudsync.io/v1
"""

doc2 = """CloudSync Pricing
Basic Plan: $10/month, 100GB storage
Pro Plan: $25/month, 1TB storage
Enterprise: Custom pricing
"""

doc3 = """CloudSync Release Notes v2.5
- Added multi-region support
- Improved sync speed by 40%
- Fixed authentication bug (#1234)
- Added dark mode to web interface
"""

# Combine documents
combined = f"""
=== Document 1: API Documentation ===
{doc1}

=== Document 2: Pricing ===
{doc2}

=== Document 3: Release Notes ===
{doc3}
"""

rlm = RLM(model="gpt-4o-mini")

# Cross-document queries
queries = [
    "What authentication method does CloudSync use?",
    "What's included in the Pro plan?",
    "What improvements were made in version 2.5?",
    "If I need 500GB storage, which plan should I choose?",
]

for query in queries:
    result = rlm.completion(query, combined)
    print(f"Q: {query}")
    print(f"A: {result}\n")

# Output:
# Q: What authentication method does CloudSync use?
# A: CloudSync uses OAuth 2.0 for authentication.
#
# Q: What's included in the Pro plan?
# A: The Pro plan costs $25/month and includes 1TB of storage.
#
# Q: What improvements were made in version 2.5?
# A: Version 2.5 added multi-region support, improved sync speed by 40%,
# fixed authentication bug #1234, and added dark mode to the web interface.
#
# Q: If I need 500GB storage, which plan should I choose?
# A: You should choose the Pro plan ($25/month, 1TB storage) as the Basic plan
# only offers 100GB which is insufficient for your 500GB needs.

```

--------------------------------

### Generate Long Document with RLM

Source: https://context7.com/ysz/recursive-llm/llms.txt

This Python function generates a multi-chapter document programmatically and then uses the RLM library to perform complex queries on the generated content. It demonstrates creating detailed chapter structures with topics, findings, statistics, and dates, then querying for data ranges and summaries. The output shows the document length in characters and tokens, followed by query results and performance metrics.

```python
from rlm import RLM

def generate_long_document():
    chapters = []
    for i in range(1, 21):  # 20 chapters
        chapter = f"""
Chapter {i}: Topic {i}

This chapter discusses important concept {i} in great detail. The key findings include:

1. First major point about topic {i}
   - Supporting detail A
   - Supporting detail B
   - Supporting detail C

2. Second major point about topic {i}
   - Evidence from study X
   - Evidence from study Y
   - Conclusion based on evidence

Key Statistics:
- Metric A: {i * 10}%
- Metric B: {i * 100} units
- Metric C: ${i * 1000}

Important dates:
- Event 1: January {i}, 2024
- Event 2: February {i}, 2024
""" + "Additional context paragraph. " * 100
        chapters.append(chapter)
    return "\n\n".join(chapters)

document = generate_long_document()
print(f"Document: {len(document):,} characters (~{len(document) // 4:,} tokens)")

rlm = RLM(
    model="gpt-4o-mini",
    max_iterations=20,
    temperature=0.5
)

queries = [
    "What is the range of Metric B values across all chapters?",
    "Which chapter has the highest Metric A percentage?",
    "Summarize the key findings from chapters 5-10",
    "How many total references are cited in the document?",
]

for query in queries:
    result = rlm.completion(query, document)
    print(f"\nQuery: {query}")
    print(f"Answer: {result}")
    print(f"Performance: {rlm.stats['llm_calls']} LLM calls, {rlm.stats['iterations']} iterations")

```

--------------------------------

### Analyze Financial Report with RLM

Source: https://context7.com/ysz/recursive-llm/llms.txt

Uses RLM to process a lengthy financial report and answer queries. It configures both a root and recursive model, specifying max iterations and depth for efficient analysis of repetitive data. The output includes answers to queries and statistics on LLM calls and recursion depth.

```python
from rlm import RLM

financial_report = """
Annual Financial Report 2024

Executive Summary:
Our company achieved record revenue of $500M in 2024, representing 25% year-over-year growth.
Net income reached $75M, with an operating margin of 18%.

Quarterly Performance:
Q1 2024: Revenue $110M, Net Income $15M
Q2 2024: Revenue $120M, Net Income $18M
Q3 2024: Revenue $130M, Net Income $20M
Q4 2024: Revenue $140M, Net Income $22M

Department Breakdown:
- Sales: $200M revenue, 150 employees
- Engineering: $150M revenue, 200 employees
- Marketing: $100M revenue, 50 employees
- Operations: $50M revenue, 100 employees
""" * 100  # Very long report

# Use GPT-4o for strategic decisions, GPT-4o-mini for data extraction
rlm = RLM(
    model="gpt-4o",              # Root model: expensive, smart reasoning
    recursive_model="gpt-4o-mini",  # Recursive: cheap, fast extraction
    max_iterations=15,
    max_depth=3,
    temperature=0.3
)

queries = [
    "What was the total revenue for 2024?",
    "Which quarter had the highest net income?",
    "How many total employees does the company have?",
]

for query in queries:
    result = rlm.completion(query, financial_report)
    print(f"Q: {query}")
    print(f"A: {result}")
    print(f"Stats: {rlm.stats['llm_calls']} calls, depth {rlm.stats['depth']}\n")

```
Tokens:
Raw