### Robin CLI Usage Examples

Source: https://github.com/apurvsinghgautam/robin/blob/main/README.md

Demonstrates how to use the Robin CLI tool with various options. Users can specify the LLM model, dark web search query, number of threads for scraping, and an output filename.

```bash
robin -m gpt4.1 -q "ransomware payments" -t 12
robin --model gpt4.1 --query "sensitive credentials exposure" --threads 8 --output filename
robin -m llama3.1 -q "zero days"
robin -m gemini-2.5-flash -q "zero days"
```

--------------------------------

### Install Robin via Docker

Source: https://github.com/apurvsinghgautam/robin/blob/main/README.md

This snippet shows how to pull the latest Robin Docker image and run it. It mounts a local .env file for configuration and exposes the web UI on port 8501. This is the recommended installation method for ease of use and isolation.

```bash
docker pull apurvsg/robin:latest
docker run --rm \
   -v "$(pwd)/.env:/app/.env" \
   --add-host=host.docker.internal:host-gateway \
   -p 8501:8501 \
   apurvsg/robin:latest ui --ui-port 8501 --ui-host 0.0.0.0
```

--------------------------------

### Launch Robin Web Interface

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Starts the Streamlit-based web interface for interactive dark web investigations. It allows specifying the port and host, and includes an example for Docker integration with local Ollama.

```bash
# Start UI on default port 8501
python main.py ui

# Start UI on custom port and host
python main.py ui --ui-port 9000 --ui-host 0.0.0.0

# For Docker with Ollama on host machine
# Set OLLAMA_BASE_URL=http://host.docker.internal:11434 in .env
docker run --rm \
   -v "$(pwd)/.env:/app/.env" \
   --add-host=host.docker.internal:host-gateway \
   -p 8501:8501 \
   apurvsg/robin:latest ui --ui-port 8501 --ui-host 0.0.0.0
```

--------------------------------

### Install Robin using Python

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Installs Robin directly using pip and runs it either in CLI or Web UI mode. This requires Python 3.10+ and installs dependencies from the requirements.txt file.

```bash
# Requires Python 3.10+
pip install -r requirements.txt

# Run CLI mode
python main.py cli -m gpt-4.1 -q "ransomware payments" -t 12

# Run Web UI mode
python main.py ui --ui-port 8501 --ui-host localhost
```

--------------------------------

### Run Robin CLI with Python Development Version

Source: https://github.com/apurvsinghgautam/robin/blob/main/README.md

This snippet shows how to run the Robin CLI in development mode using Python. It requires Python 3.10+ and installs dependencies from requirements.txt. The command specifies the AI model, query, and a timeout value.

```bash
pip install -r requirements.txt
python main.py cli -m gpt-4.1 -q "ransomware payments" -t 12
```

--------------------------------

### Run Robin CLI with Release Binary

Source: https://github.com/apurvsinghgautam/robin/blob/main/README.md

This code demonstrates how to execute the Robin command-line interface (CLI) using a downloaded release binary. It specifies the AI model to use and the search query. Ensure the binary is made executable first.

```bash
chmod +x robin
./robin cli --model gpt-4.1 --query "ransomware payments"
```

--------------------------------

### Configure Environment Variables and Tor

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Sets up the necessary environment variables for LLM API keys and ensures the Tor service is running, which is crucial for accessing the dark web. This involves creating a .env file and installing/starting Tor.

```bash
cat > .env << 'EOF'
OPENAI_API_KEY=sk-your-openai-api-key
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
GOOGLE_API_KEY=your-google-api-key
OLLAMA_BASE_URL=http://127.0.0.1:11434
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
OPENROUTER_API_KEY=sk-or-your-openrouter-key
LLAMA_CPP_BASE_URL=http://127.0.0.1:8080
EOF

# Ensure Tor is running (required for dark web access)
# Linux/WSL:
sudo apt install tor && sudo systemctl start tor

# macOS:
brew install tor && brew services start tor

# Verify Tor is running on SOCKS proxy port 9050
curl --socks5-hostname 127.0.0.1:9050 https://check.torproject.org/api/ip
```

--------------------------------

### Initialize LLM Instance in Python

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Demonstrates how to initialize different LLM instances using the `get_llm` function. It supports various providers like OpenAI, Anthropic, Google, OpenRouter, and local models via Ollama or llama.cpp.

```python
from llm import get_llm

# Initialize OpenAI model
llm = get_llm("gpt-4.1")

# Initialize Anthropic Claude
llm = get_llm("claude-sonnet-4-5")

# Initialize Google Gemini
llm = get_llm("gemini-2.5-flash")

# Initialize OpenRouter model (uses free tier)
llm = get_llm("qwen3-80b-openrouter")

# Initialize local Ollama model (auto-detected if running)
llm = get_llm("llama3.1:latest")

# All LLM instances are preconfigured with:
# - temperature: 0 (deterministic output)
# - streaming: True (real-time token output)
# - callbacks: [BufferedStreamingHandler] (chunked output)
```

--------------------------------

### Git Workflow for Contributions

Source: https://github.com/apurvsinghgautam/robin/blob/main/README.md

Standard Git commands for contributing to the project. This includes forking the repository, creating a feature branch, committing changes, and pushing to the branch to open a Pull Request.

```git
git checkout -b feature/amazing-feature
git commit -m 'Add some amazing feature'
git push origin feature/amazing-feature
```

--------------------------------

### Buffered Streaming Output Handler

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Provides a LangChain callback handler designed for managing buffered streaming output from LLMs. It accumulates tokens and flushes them based on newline characters or a defined buffer limit, with an option to trigger a UI callback for real-time updates.

```APIDOC
## POST /api/llm/streaming/handler

### Description
Sets up a LangChain callback handler for buffered streaming output. This handler accumulates LLM tokens and flushes them either when a newline character is encountered or when a specified buffer limit is reached. An optional UI callback function can be provided to update a user interface in real-time.

### Method
POST

### Endpoint
/api/llm/streaming/handler

### Parameters
#### Request Body
- **buffer_limit** (integer) - Optional - The maximum number of characters to buffer before flushing. Defaults to 60.
- **ui_callback_endpoint** (string) - Optional - The endpoint URL to which UI updates should be sent.

### Request Example
```json
{
  "buffer_limit": 60,
  "ui_callback_endpoint": "/api/ui/update"
}
```

### Response
#### Success Response (200)
- **handler_status** (string) - Confirmation message indicating the handler setup.
  - **Example**: `"BufferedStreamingHandler configured successfully."`

### Usage Notes
- This handler is intended to be used within a LangChain LLM instance configured for streaming.
- Tokens are flushed upon encountering newline characters, reaching the `buffer_limit`, or when the LLM stream ends (`on_llm_end`).
```

--------------------------------

### Run Robin CLI Investigation

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Executes a full dark web OSINT investigation using the Robin CLI. This command allows specifying the LLM model, search query, thread count, and output filename.

```bash
# Basic investigation with OpenAI model
python main.py cli --model gpt-4.1 --query "ransomware payments"

# Investigation with Claude and custom thread count
python main.py cli -m claude-sonnet-4-5 -q "credential leaks" -t 12

# Save output to specific filename
python main.py cli -m gpt-5-mini -q "zero day exploits" -o threat_report

# Using Gemini model
python main.py cli -m gemini-2.5-flash -q "darknet markets"

# Using OpenRouter models (free tier)
python main.py cli -m qwen3-80b-openrouter -q "threat actor forums"

# Using local Ollama model
python main.py cli -m llama3.1:latest -q "malware samples"

# Output file format: summary_YYYY-MM-DD_HH-MM-SS.md (auto-generated)
# or: threat_report.md (if -o flag specified)
```

--------------------------------

### Resolve Model Configuration

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Takes a model identifier as input and returns its specific configuration, including the LLM class and necessary constructor parameters. This function supports both predefined cloud models and dynamically discovered local models.

```APIDOC
## POST /api/llm/models/config

### Description
Resolves a given model name to its corresponding configuration, including the LLM class and constructor parameters. This endpoint works for both cloud-based models and locally hosted models.

### Method
POST

### Endpoint
/api/llm/models/config

### Parameters
#### Request Body
- **model_choice** (string) - Required - The identifier of the LLM model (e.g., "gpt-4.1", "llama3.1:latest").

### Request Example
```json
{
  "model_choice": "gpt-4.1"
}
```

### Response
#### Success Response (200)
- **config** (object) - An object containing the model's configuration.
  - **class** (string) - The class name of the LLM (e.g., "ChatOpenAI", "ChatOllama").
  - **constructor_params** (object) - A dictionary of parameters required to instantiate the LLM class.
    - **Example for Cloud Model**: `{
      "class": "ChatOpenAI",
      "constructor_params": {"model_name": "gpt-4.1"}
    }`
    - **Example for Local Model**: `{
      "class": "ChatOllama",
      "constructor_params": {"model": "llama3.1:latest", "base_url": "http://localhost:11434"}
    }`
```

--------------------------------

### Resolve LLM Model Configuration

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Takes a model name (choice) and returns its corresponding configuration, including the LLM class and necessary constructor parameters. This function supports both predefined cloud models and dynamically detected local models, simplifying model instantiation.

```python
from llm_utils import resolve_model_config

# Get config for cloud model
config = resolve_model_config("gpt-4.1")
# Returns: {
#     'class': ChatOpenAI,
#     'constructor_params': {'model_name': 'gpt-4.1'}
# }

# Get config for Anthropic model
config = resolve_model_config("claude-sonnet-4-5")
# Returns: {
#     'class': ChatAnthropic,
#     'constructor_params': {'model': 'claude-sonnet-4-5'}
# }

# Get config for local Ollama model (auto-detected)
config = resolve_model_config("llama3.1:latest")
# Returns: {
#     'class': ChatOllama,
#     'constructor_params': {'model': 'llama3.1:latest', 'base_url': '...'}
# }
```

--------------------------------

### Generate Threat Intelligence Summary with LLM (Python)

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Generates a comprehensive threat intelligence summary from scraped dark web content. Output includes source links, investigation artifacts, key insights, and next steps. It takes an LLM instance, the original query, and the scraped content as input.

```python
from llm import get_llm, generate_summary

llm = get_llm("gpt-4.1")

# Scraped content from dark web sites
scraped_content = {
    "http://forum.onion/thread/123": "RansomHub Forum Discussion about Bitcoin payments...",
    "http://market.onion/listing/456": "Ransomware-as-a-Service offering with Monero support...",
}

# Generate intelligence summary
summary = generate_summary(llm, "ransomware payments", scraped_content)

# Output format (Markdown):
# 1. Input Query: ransomware payments
# 2. Source Links Referenced for Analysis
#    - http://forum.onion/thread/123
#    - http://market.onion/listing/456
# 3. Investigation Artifacts
#    - Cryptocurrency addresses: bc1q...
#    - Forum names: RansomHub
#    - Threat actors: ...
# 4. Key Insights
#    - 3-5 specific, actionable insights
# 5. Next Steps
#    - Suggested investigation queries
```

--------------------------------

### List Available LLM Models

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Retrieves a comprehensive list of all available Language Model (LLM) choices. This includes models configured for cloud providers (if API keys are set) and models discovered locally via Ollama or llama.cpp.

```APIDOC
## GET /api/llm/models/choices

### Description
Returns a list of all available LLM models. This list combines cloud-based models (requiring valid API keys) and locally hosted models automatically detected from services like Ollama and llama.cpp.

### Method
GET

### Endpoint
/api/llm/models/choices

### Parameters
*None*

### Request Example
```json
{} 
```

### Response
#### Success Response (200)
- **models** (array) - A list of strings, where each string is the identifier for an available LLM model.
  - **Example**: `["gpt-4.1", "gpt-5-mini", "claude-sonnet-4-5", "gemini-2.5-flash", "qwen3-80b-openrouter", "llama3.1:latest", ...]`

### Provider Notes
- **OpenAI**: Requires `OPENAI_API_KEY`.
- **Anthropic**: Requires `ANTHROPIC_API_KEY`.
- **Google**: Requires `GOOGLE_API_KEY`.
- **OpenRouter**: Requires `OPENROUTER_API_KEY`.
- **Local Models**: Auto-detected via Ollama (`/api/tags`) or llama.cpp (`/v1/models`).
```

--------------------------------

### Buffered Streaming Output Handler for LLMs

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Implements a LangChain callback handler designed for managing streamed LLM output. It buffers incoming tokens and flushes them to a callback function (e.g., for UI updates) either when a newline is encountered or when the buffer reaches a specified character limit, ensuring efficient real-time display.

```python
from llm_utils import BufferedStreamingHandler

# Create handler with custom buffer and UI callback
def update_ui(text):
    print(f"UI Update: {text}")

handler = BufferedStreamingHandler(
    buffer_limit=60,      # Flush after 60 characters
    ui_callback=update_ui  # Optional callback for each chunk
)

# Use with LLM instance
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    model_name="gpt-4.1",
    streaming=True,
    callbacks=[handler]
)

# Tokens are buffered and flushed on:
# - Newline characters in token
# - Buffer reaching limit (60 chars default)
# - LLM completion (on_llm_end)
```

--------------------------------

### Refine Search Query with LLM (Python)

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Uses an LLM to refine user queries for optimal dark web search engine results. Limits output to 5 words or less without logical operators. It takes an LLM instance and the user's input query as arguments.

```python
from llm import get_llm, refine_query

llm = get_llm("gpt-4.1")

# Refine a verbose query
user_query = "I want to find information about ransomware payment methods"
refined = refine_query(llm, user_query)
# Returns: "ransomware payments methods"

# Refine a technical query
user_query = "credential dumps and leaked databases from recent breaches"
refined = refine_query(llm, user_query)
# Returns: "credential dumps leaked databases"

# The refined query is optimized for dark web search engines
# which typically have simpler query parsers than clearweb engines
```

--------------------------------

### Create Tor-Enabled Session

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Creates and returns a requests session object pre-configured with a Tor SOCKS proxy and automatic retry logic for making resilient requests to the dark web.

```APIDOC
## POST /api/tor/session

### Description
Creates a requests session object that is configured to use the Tor network via a SOCKS proxy. This session includes automatic retry logic for enhanced resilience when accessing .onion sites.

### Method
POST

### Endpoint
/api/tor/session

### Parameters
*None*

### Request Example
```json
{} 
```

### Response
#### Success Response (200)
- **session_info** (object) - Information about the created Tor-enabled session. (Note: Actual session object is not directly returned via API, but its configuration is applied to subsequent requests made using this endpoint's context).

### Configuration Details
- **Proxy**: `socks5h://127.0.0.1:9050` (Tor default)
- **Max Retries**: 3 (connect, read, total)
- **Backoff Factor**: 0.3 seconds
- **Retry Status Codes**: 500, 502, 503, 504
```

--------------------------------

### List Available LLM Models

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Retrieves a comprehensive list of all available Large Language Models (LLMs). This includes models configured for cloud access (requiring valid API keys) and models discovered locally via Ollama or llama.cpp. The function filters models based on API key availability for cloud services.

```python
from llm_utils import get_model_choices

# Get all available models
models = get_model_choices()
# Returns: ['gpt-4.1', 'gpt-5-mini', 'claude-sonnet-4-5', 'gemini-2.5-flash',
#           'qwen3-80b-openrouter', 'llama3.1:latest', ...]

# Models are filtered by API key availability:
# - OpenAI models: require OPENAI_API_KEY
# - Anthropic models: require ANTHROPIC_API_KEY
# - Google models: require GOOGLE_API_KEY
# - OpenRouter models: require OPENROUTER_API_KEY

# Local models are auto-detected:
# - Ollama: queries /api/tags endpoint
# - llama.cpp: queries /v1/models endpoint
```

--------------------------------

### Query Single Dark Web Search Engine (Python)

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Queries a single dark web search engine and parses results. Used internally by get_search_results for concurrent execution. It requires the search engine's endpoint URL and the search query.

```python
from search import fetch_search_results

# Query specific search engine
ahmia_results = fetch_search_results(
    "http://juhanurmihxlp77nkq76byazcldy2hlmovfu2epvl5ankdibsot4csyd.onion/search/?q={query}",
    "ransomware"
)

# Returns list of dicts with title and .onion links
# Automatically filters out search engine self-referential links
# Uses rotating user agents for anonymity
```

--------------------------------

### Scrape Multiple URLs

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Scrapes a list of .onion URLs concurrently using a configurable number of worker threads. It returns a dictionary mapping each URL to its scraped content, with automatic content truncation and fallback to title-only on failure.

```APIDOC
## POST /api/scrape/multiple

### Description
Scrapes a list of .onion URLs concurrently and returns a dictionary mapping each URL to its scraped content. Features include configurable thread count, automatic content truncation, and graceful fallback to title-only on scrape failure.

### Method
POST

### Endpoint
/api/scrape/multiple

### Parameters
#### Query Parameters
- **max_workers** (integer) - Optional - The maximum number of worker threads to use for concurrent scraping.

#### Request Body
- **urls_to_scrape** (array) - Required - A list of URL objects, where each object contains a 'title' (string) and a 'link' (string) to the URL to be scraped.
  - **Example**: `[
    {"title": "RansomHub Forum", "link": "http://ransomhub.onion/forum"},
    {"title": "Crypto Exchange", "link": "http://exchange.onion/rates"}
  ]`

### Request Example
```json
{
  "urls_to_scrape": [
    {"title": "RansomHub Forum", "link": "http://ransomhub.onion/forum"},
    {"title": "Crypto Exchange", "link": "http://exchange.onion/rates"}
  ]
}
```

### Response
#### Success Response (200)
- **url_content_map** (object) - A dictionary where keys are the scraped URLs (string) and values are the corresponding scraped content (string) or title-only fallback.
  - **Example**: `{
    "http://ransomhub.onion/forum": "RansomHub Forum - Welcome to the forum...(truncated)",
    "http://exchange.onion/rates": "Crypto Exchange - Current BTC rates...(truncated)"
  }`
```

--------------------------------

### Python Dark Web OSINT Investigation Pipeline

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Executes a full OSINT investigation pipeline on the dark web. It refines queries, searches multiple engines, filters results with an LLM, scrapes content, and generates a Markdown summary. Dependencies include 'llm', 'search', and 'scrape' modules. It takes a query string, an LLM model name, and the number of threads as input, returning a summary string.

```python
from llm import get_llm, refine_query, filter_results, generate_summary
from search import get_search_results
from scrape import scrape_multiple
from datetime import datetime

def run_investigation(query: str, model: str = "gpt-4.1", threads: int = 8):
    """
    Complete dark web OSINT investigation pipeline.

    Args:
        query: User's search query
        model: LLM model to use
        threads: Number of concurrent threads for scraping

    Returns:
        str: Intelligence summary in Markdown format
    """
    # Initialize LLM
    llm = get_llm(model)
    print(f"[*] Using model: {model}")

    # Step 1: Refine query for dark web search engines
    refined = refine_query(llm, query)
    print(f"[*] Refined query: {refined}")

    # Step 2: Search dark web engines (16 engines concurrently)
    results = get_search_results(refined, max_workers=threads)
    print(f"[*] Found {len(results)} raw results")

    if not results:
        raise RuntimeError("No search results. Check Tor connection.")

    # Step 3: LLM filters to top 20 relevant results
    filtered = filter_results(llm, refined, results)
    print(f"[*] Filtered to {len(filtered)} relevant results")

    # Step 4: Scrape filtered sites via Tor
    scraped = scrape_multiple(filtered, max_workers=threads)
    print(f"[*] Scraped {len(scraped)} sites")

    if not scraped:
        raise RuntimeError("Failed to scrape content. Check Tor.")

    # Step 5: Generate intelligence summary
    summary = generate_summary(llm, query, scraped)

    # Save to file
    timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
    filename = f"summary_{timestamp}.md"
    with open(filename, "w") as f:
        f.write(summary)
    print(f"[*] Summary saved to {filename}")

    return summary

# Run investigation
if __name__ == "__main__":
    summary = run_investigation(
        query="ransomware payment methods",
        model="gpt-4.1",
        threads=12
    )
    print(summary)

```

--------------------------------

### Filter Search Results with LLM (Python)

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Uses an LLM to analyze search results and select the top 20 most relevant entries based on title and link matching the search query. It requires an LLM instance, the search query, and a list of search results.

```python
from llm import get_llm, filter_results

llm = get_llm("gpt-4.1")

# Example search results from dark web engines
search_results = [
    {"title": "RansomHub Forum - Payment Instructions", "link": "http://example.onion/ransomhub"},
    {"title": "Cryptocurrency Exchange - Anonymous", "link": "http://crypto.onion/exchange"},
    {"title": "Leaked Database Collection", "link": "http://leaks.onion/db"},
    # ... more results
]

# Filter results to top 20 most relevant
filtered = filter_results(llm, "ransomware payments", search_results)
# Returns list of up to 20 most relevant results

# Handles rate limiting gracefully by truncating titles if needed
# Falls back to top results if LLM output parsing fails
```

--------------------------------

### Create Tor-Enabled Requests Session

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Creates a requests session object pre-configured for accessing .onion sites via Tor. It includes settings for SOCKS5 proxy, timeout, and automatic retry logic for network resilience. This session can be used for making manual HTTP requests through the Tor network.

```python
from scrape import get_tor_session

# Create Tor session for manual requests
session = get_tor_session()

# Make request through Tor
response = session.get(
    "http://example.onion/page",
    headers={"User-Agent": "Mozilla/5.0..."},
    timeout=45
)

# Session configuration:
# - Proxy: socks5h://127.0.0.1:9050 (Tor default)
# - Max retries: 3 (connect, read, total)
# - Backoff factor: 0.3 seconds
# - Retry on: 500, 502, 503, 504 status codes
```

--------------------------------

### Scrape Multiple .onion URLs Concurrently

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Scrapes a list of .onion URLs concurrently using a configurable number of worker threads. It returns a dictionary mapping each URL to its scraped content, with automatic content truncation and fallback to title-only on failure. Script and style tags are removed for cleaner text extraction.

```python
from scrape import scrape_multiple

# List of URLs to scrape (from filtered search results)
urls_to_scrape = [
    {"title": "RansomHub Forum", "link": "http://ransomhub.onion/forum"},
    {"title": "Crypto Exchange", "link": "http://exchange.onion/rates"},
]

# Scrape all URLs concurrently
scraped = scrape_multiple(urls_to_scrape, max_workers=8)

# Returns dict mapping URL to scraped content:
# {
#     "http://ransomhub.onion/forum": "RansomHub Forum - Welcome to the forum...(truncated)",
#     "http://exchange.onion/rates": "Crypto Exchange - Current BTC rates...(truncated)",
# }

# Features:
# - Concurrent scraping with configurable thread count
# - Automatic content truncation (2000 chars max)
# - Graceful fallback to title-only on scrape failure
# - Script/style tag removal for clean text extraction
```

--------------------------------

### Query Multiple Dark Web Search Engines Concurrently (Python)

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Concurrently queries 16 different dark web search engines via Tor and aggregates deduplicated results. Returns a list of dictionaries containing title and onion link pairs. It takes a refined query string and the maximum number of worker threads.

```python
from search import get_search_results

# Search dark web with refined query
results = get_search_results("ransomware payments", max_workers=8)

# Returns list of results like:
# [
#     {"title": "RansomHub - Payment Portal", "link": "http://abc123.onion/payments"},
#     {"title": "Crypto Mixer Services", "link": "http://def456.onion/mixer"},
#     ...
# ]

# Supported search engines (queried concurrently):
# - Ahmia, OnionLand, Torgle, Amnesia, Kaizer
# - Anima, Tornado, TorNet, Torland, Find Tor
# - Excavator, Onionway, Tor66, OSS, Torgol
# - The Deep Searches

# Results are automatically deduplicated by link
# Timeout: 40 seconds per search engine
# Retries: 3 attempts with exponential backoff
```

--------------------------------

### Scrape Multiple Dark Web Sites Concurrently (Python)

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Concurrently scrapes multiple dark web sites using a thread pool. Extracts and cleans text content, truncating to 2000 characters per page. It takes a list of URLs and their associated data, and the maximum number of worker threads.

```python
from scrape import scrape_multiple

# Example usage would follow here, assuming urls_data is defined.
```

--------------------------------

### Scrape Single URL

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Scrapes a single .onion URL using a Tor session with retry logic. It returns a tuple containing the URL and its cleaned text content, or just the title if scraping fails.

```APIDOC
## POST /api/scrape/single

### Description
Scrapes a single .onion URL using a Tor session with built-in retry logic. Returns a tuple containing the URL and the cleaned text content. If scraping fails, it falls back to returning only the title.

### Method
POST

### Endpoint
/api/scrape/single

### Parameters
#### Request Body
- **url_data** (object) - Required - An object containing the URL details.
  - **title** (string) - Required - The title of the URL.
  - **link** (string) - Required - The .onion URL to scrape.
    - **Example**: `{"title": "Target Forum", "link": "http://target.onion/page"}`

### Request Example
```json
{
  "url_data": {
    "title": "Target Forum",
    "link": "http://target.onion/page"
  }
}
```

### Response
#### Success Response (200)
- **result** (tuple) - A tuple containing the URL (string) and the scraped content (string).
  - **Example**: `("http://target.onion/page", "Target Forum - Extracted page text...")`

### Configuration Notes
- **Tor Session**: Configured with SOCKS5 proxy (`socks5h://127.0.0.1:9050`), a 45-second timeout for .onion sites, 3 retries with a 0.3s backoff factor, and rotating user agents.
```

--------------------------------

### Scrape Single .onion URL with Tor

Source: https://context7.com/apurvsinghgautam/robin/llms.txt

Scrapes a single .onion URL using a Tor session with built-in retry logic. It returns a tuple containing the URL and its cleaned text content, or just the title if scraping fails. The Tor session is configured with a SOCKS5 proxy, timeout, retries, and rotating user agents.

```python
from scrape import scrape_single

# Scrape single URL
url_data = {"title": "Target Forum", "link": "http://target.onion/page"}
url, content = scrape_single(url_data)

# Returns tuple: (url, scraped_content)
# Content includes: "Title - Extracted page text..."

# Tor session configuration:
# - SOCKS5 proxy: socks5h://127.0.0.1:9050
# - Timeout: 45 seconds for .onion sites
# - Retries: 3 with 0.3s backoff factor
# - Rotating user agents
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.