### Install Project Dependencies

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/README.md

Navigate to the specific project directory and install its dependencies using pip.

```bash
cd Marco-DeepResearch-Family/HSCodeComp
pip install -r requirements.txt
```

```bash
cd Marco-DeepResearch-Family/DeepWideSearch
pip install -r requirements.txt
```

```bash
cd Marco-DeepResearch-Family/Table-as-Search
pip install -r requirements.txt
```

```bash
cd Marco-DeepResearch-Family/UMEM
pip install -r requirements.txt
pip install -e .
```

--------------------------------

### Environment Setup for HSCodeComp

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/HSCodeComp/README.md

This bash script guides through setting up a Python virtual environment and installing necessary dependencies for the HSCodeComp project. It also mentions setting up OpenAI API keys.

```bash
# Create and activate a virtual environment (optional but recommended)
python -m venv hscodcomp_env
source hscodcomp_env/bin/activate  # Linux/macOS
# hscodcomp_env\Scripts\activate  # Windows

# Install dependencies (e.g., pandas, openai, etc.)
pip install pandas,openai,tqdm,threading,dotenv

# set openai keys and base urls in HSCodeComp/.env
```

--------------------------------

### UMEM Project Installation

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Steps to clone the repository, set up a Conda environment, and install project dependencies.

```bash
git clone https://github.com/AIDC-AI/Marco-DeepResearch.git
cd Marco-DeepWideSearch-Agent/UMEM

conda create -n umem python=3.10 -y
conda activate umem

pip install -r requirements.txt
pip install -e .
```

--------------------------------

### Install Dependencies

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README_CN.md

Install the necessary Python dependencies for the Table-as-Search framework using the provided requirements file.

```bash
pip install -r requirements.txt
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/DeepWideSearch/README.md

Clone the DeepWideSearch repository and install the required Python packages using pip.

```bash
git clone https://github.com/Marco-Search-Agent
cd DeepWideSearch
pip install -r requirements.txt
```

--------------------------------

### Clone Repository

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README_CN.md

Clone the Table-as-Search repository to your local machine. Navigate into the cloned directory to proceed with installation.

```bash
git clone https://github.com/AIDC-AI/Marco-DeepWideSearch-Agent
cd Marco-DeepWideSearch-Agent/Table-as-Search
```

--------------------------------

### JSONL Data Format Example

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/DeepWideSearch/README.md

Example of the JSONL file format used for model generations, containing instance ID, question, prediction, rollout ID, and messages.

```jsonl
{"instance_id": "deep2wide_result_5_Lin Dan", "question": "There is a Chinese athlete who has achieved outstanding success in a racket sport. He was the first player in his discipline to successfully defend a major championship title and holds multiple world championship titles. His sport underwent significant rule changes in the early 21st century, and he became the first male singles Olympic champion in the post-rule-change era. Please help me compile and summarize this athlete’s competition records between 2010 and 2020 into a clear Markdown table, including the following columns: Date, Tournament Name, Level, Event, Result, and Match Details (including opponent, score, and win/loss outcome) ...", "rollout_id": 1, "prediction": "...", "messages": [{"role": "system", "content": "You are a Web Information Seeking Master. Your task is to thoroughly seek the internet for information and provide accurate answers to questions ..."}, {"role": "user", "content": "A conversation between User and Assistant ..."}, {"role": "assistant", "content": " ... "}]}
```

--------------------------------

### Run Single Task DeepSearch Inference

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README_CN.md

Execute a single DeepSearch inference task, for example, on the BrowseComp-zh dataset. Configure query, models, output directory, and database name.

```bash
python run_deepsearch_inference.py \
    --query "找出2024年诺贝尔物理学奖得主的详细信息，包括获奖理由、主要研究领域和代表性论文" \
    --instance-id "deep_001" \
    --main-model-id "gpt-4o" \
    --tabular-model-id "gpt-4o" \
    --deep-model-id "gpt-4o" \
    --output-dir ./outputs/deepsearch \
    --db-name deepsearch_test
```

--------------------------------

### Configure Environment Variables

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README_CN.md

Create a .env file in the Table-as-Search directory to configure API keys for OpenAI, Google Search, and Jina AI. Ensure all required keys are present for the framework to function correctly.

```bash
# OpenAI API 配置
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_CHAT_BASE_URL=https://api.openai.com/v1

# Google 搜索 API 配置
SEARCH_API_KEY=your_serper_api_key_here
SEARCH_API_BASE=XXXXXX

# Jina AI 配置（用于网页访问）
JINA_KEYS_FILE=path/to/jina_keys.txt  # 可选：包含多个 Jina API 密钥的文件
```

--------------------------------

### Initialize and Use JinaBackedVisitWebpageSummaryTool

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Initialize the JinaBackedVisitWebpageSummaryTool for targeted information extraction from webpages using an LLM. It requires specifying a summary model and goal.

```python
from tools.jina_visit import JinaBackedVisitWebpageSummaryTool, GlobalVisitCounter

# Create summary-enabled visit tool
summary_tool = JinaBackedVisitWebpageSummaryTool(
    max_output_length=40000,
    jina_keys_file="path/to/jina_keys.txt",
    work_dir="./web_pages",
    summary_model_name="qwen3-next-80b-a3b-instruct",
    temperature=0.0,
    summary_timeout=120.0,
    global_visit_counter=GlobalVisitCounter(limit=50)
)

# Extract targeted information from webpage
url = "https://en.wikipedia.org/wiki/Python_(programming_language)"
summary_goal = "Extract information about Python's history and key features"
result = summary_tool.forward(url, summary_goal)
print(result)
# Output: [Extracted Information Based on Search Goal]
# - Python was created by Guido van Rossum in 1991...
# - Key features include dynamic typing, garbage collection...
```

--------------------------------

### Initialize Database-Backed Table Tool

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README.md

Initializes a database-backed table tool using MongoDB and integrates it into an agent. Ensure MONGODB_URI and MONGODB_DATABASE environment variables are set.

```python
from tools.db_table_code_v2 import DBTableCodeToolInterface
from pymongo import MongoClient

# Initialize MongoDB connection
client = MongoClient(os.getenv("MONGODB_URI"))
db = client[os.getenv("MONGODB_DATABASE")]

# Create table tool
table_tool = DBTableCodeToolInterface(
    db=db,
    name_prefix="task_001",
    description="Manage structured tables for information collection"
)

# Use in agent
agent = ToolCallingAgent(
    model=model,
    tools=[table_tool],
    max_steps=20
)
```

--------------------------------

### Environment Configuration for Table-as-Search

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Configuration file for setting up environment variables required by the Table-as-Search framework, including API keys and database connections.

```bash
# .env file configuration for Table-as-Search

# OpenAI API Configuration (or compatible LLM API)
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_CHAT_BASE_URL=https://api.openai.com/v1

# Alternative API keys (for different models)
OUT_API_KEY=your_alternative_api_key
OUT_BASE_URL=https://alternative-api.com/v1

# Google Search API Configuration (Serper or similar)
SEARCH_API_KEY=your_serper_api_key_here
SEARCH_API_BASE=https://your-search-api-endpoint.com

# Jina AI Configuration (for web page visiting)
JINA_KEYS_FILE=path/to/jina_keys.txt

# MongoDB Configuration
MONGODB_CONNECTION_STRING=mongodb://localhost:27017/

# Summary Model Configuration (optional)
SUMMARY_MODEL_NAME=qwen3-next-80b-a3b-instruct
SUMMARY_TIMEOUT=120.0
```

--------------------------------

### Launch UMEM Training

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Shell script to initiate the GRPO training process with online memory evolution.

```bash
bash umem_scripts/run_train.sh
```

--------------------------------

### Prepare MMLU Training Data

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Script to prepare MMLU dataset for training, specifying output file and number of samples.

```bash
python scripts/prepare_mmlu.py \
  --output data/mmlu_ori.jsonl \
  --num_samples 2000
```

--------------------------------

### Initialize and Use JinaBackedVisitWebpageTool

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Initialize the JinaBackedVisitWebpageTool for fetching and converting web pages to markdown. It supports Jina API fallback and global visit limits.

```python
from tools.jina_visit import JinaBackedVisitWebpageTool, GlobalVisitCounter

# Initialize global visit counter for rate limiting
global_visit_counter = GlobalVisitCounter(limit=50)

# Create webpage visit tool
visit_tool = JinaBackedVisitWebpageTool(
    max_output_length=40000,
    jina_keys_file="path/to/jina_keys.txt",  # Optional: Jina API keys for fallback
    work_dir="./web_pages",  # Directory to save fetched pages
    global_visit_counter=global_visit_counter
)

# Fetch and convert webpage to markdown
url = "https://en.wikipedia.org/wiki/Python_(programming_language)"
content = visit_tool.forward(url)
print(content[:1000])  # Print first 1000 characters
# Output: Markdown formatted content with headers, links preserved

# Check visit budget
print(f"Visits used: {global_visit_counter.get_count()}/{global_visit_counter.limit}")
```

--------------------------------

### Run Table-as-Search Inference

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/README.md

Perform inference for the Table-as-Search framework by providing a query and an instance ID.

```bash
cd Marco-DeepResearch-Family/Table-as-Search
python run_widesearch_inference.py --query "your query" --instance-id "test_001"
```

--------------------------------

### Run UMEM Training Pipeline

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Executes the UMEM training pipeline, including GRPO training with online memory evolution. This script initiates the core training process.

```bash
# Step 3: Run GRPO training with online memory evolution
bash umem_scripts/run_train.sh
```

--------------------------------

### Create Agent Team for Multi-Agent Tasks

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Sets up a hierarchical agent team with shared resources and rate limiting. Requires model instances and specifies task-specific configurations like work folders and limits.

```python
from run_widesearch_inference import create_agent_team, create_model_instance
import os

# Create model instances for each agent
main_model = create_model_instance(
    model_id="gpt-4o",
    api_base=os.getenv("OPENAI_BASE_URL"),
    api_key=os.getenv("OPENAI_API_KEY")
)
tabular_model = create_model_instance("gpt-4o")
deep_model = create_model_instance("gpt-4o")

# Create agent team with shared resources
main_agent, mcp_clients, log_file, task_logger, managed_agent_counter = create_agent_team(
    main_model=main_model,
    tabular_model=tabular_model,
    deep_model=deep_model,
    task_work_folder="./outputs/task_001",
    task_id="task_001",
    db_name="research_db",
    use_summary_tool=True,  # Enable webpage summarization
    global_visit_limit=50,  # Limit webpage visits
    global_search_limit=100,  # Limit search queries
    main_max_steps=40,
    subagent_max_steps=40,
    managed_agent_limits={
        "tabular_search_agent": 10,  # Limit sub-agent calls
        "deep_search_agent": 10
    }
)

# Run the agent with a query
question = """Find and compile a table of the top 5 AI research labs
with columns: Lab Name, Parent Organization, Notable Projects, Key Researchers"""
result = main_agent.run(question)
print(result)
```

--------------------------------

### Initialize and Use GoogleSearchTool

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Initialize the GoogleSearchTool with API credentials and rate limiting. This tool provides web search functionality with automatic retries and fallback mechanisms.

```python
from tools.google_search_tool import GoogleSearchTool, GlobalSearchCounter

# Initialize global search counter for rate limiting across agents
global_search_counter = GlobalSearchCounter(limit=100)

# Create search tool with rate limiting
search_tool = GoogleSearchTool(
    api_key=os.getenv("SEARCH_API_KEY"),
    api_base=os.getenv("SEARCH_API_BASE"),
    limit=10,  # Number of results per search
    max_retries=3,  # Retry attempts before fallback
    global_search_counter=global_search_counter
)

# Execute search query
results = search_tool.forward("artificial intelligence research 2024")
print(results)
# Output: Formatted search results with titles, URLs, and snippets
# [0] AI Research Breakthrough 2024
#     Link: https://example.com/ai-research
#     Summary: Latest developments in AI research...

# Check remaining search budget
remaining = global_search_counter.get_remaining()
print(f"Remaining searches: {remaining}")
```

--------------------------------

### Run HSCodeComp Evaluation

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/README.md

Execute the evaluation script for the HSCodeComp benchmark. Ensure you specify the model name, data path, and an output directory.

```bash
cd Marco-DeepResearch-Family/HSCodeComp
python eval/test_llm.py \
  --model_name your_model \
  --data_path data/test_data.jsonl \
  --output_path results/
```

--------------------------------

### Repository Structure

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/HSCodeComp/README.md

This bash script displays the directory structure of the HSCodeComp project. It highlights key directories like 'data' for test data and 'eval' for evaluation scripts.

```bash
HSCodeComp/
├── data/
│   └── test_data.csv              # Product descriptions, attributes and ground-truth HSCodes
├── eval/
│   └── test_llm.py                # Evaluation script for model predictions
├── LICENSE
└── README.md
```

--------------------------------

### DeepWideSearch Benchmark Evaluation Script

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

A bash script template for evaluating agents on the DeepWideSearch benchmark. This benchmark assesses both deep reasoning and wide-scale information retrieval capabilities.

```bash
#!/bin/bash
```

--------------------------------

### Run UMEM Evaluation

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/README.md

Execute the evaluation script for the UMEM memory system.

```bash
cd Marco-DeepResearch-Family/UMEM
bash umem_scripts/run_eval.sh
```

--------------------------------

### Run DeepWideSearch Evaluation

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/README.md

Initiate batch evaluations for the DeepWideSearch benchmark using the provided script.

```bash
cd Marco-DeepResearch-Family/DeepWideSearch
bash scripts/batch_eval.sh
```

--------------------------------

### Build Semantic Neighborhoods for UMEM Training

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Builds semantic neighborhoods from the prepared training data using a specified encoder (e.g., BGE-M3). This step is crucial for memory evolution in the UMEM training pipeline.

```bash
# Step 2: Build semantic neighborhoods using BGE-M3 encoder
python scripts/build_neighborhoods.py \
  --input data/mmlu_ori.jsonl \
  --encoder bge-m3 \
  --top_n 3 \
  --output data/mmlu.jsonl
```

--------------------------------

### Evaluate AI Agents with Multiple Models

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/DeepWideSearch/README.md

Bash script to iterate through a list of AI models and evaluate each one using the provided evaluation script. It sets the number of threads for parallel evaluation.

```bash
#!/bin/bash

THREAD_NUM=4
models=("o3-mini" "claude-sonnet-4" "gemini-2.5-pro" "qwen-max" "deepseek-r1" "deepseek-v3" "kimi-k2" "qwen3-235b-a22b" "qwen3-235b-a22b-instruct" "qwen3-32b" "gpt-4o" "owl_claude-sonnet-4" "owl_gemini-2.5-pro" "owl_gemini-2.5-pro" "smolagents_claude-sonnet-4" "smolgents_gemini-2.5-pro" "smolagents_gpt-5" "websailor_gpt-5" "websailor_claude4" "websailor_gemini-2.5-pro")
for model in "${models[@]}"
do
    echo "============================"
    echo "evaluate $model begin"
    echo "============================"
    ./scripts/eval.sh $model $THREAD_NUM
done
```

--------------------------------

### JinaBackedVisitWebpageTool API

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Fetches and converts web pages to markdown format, with Jina API fallback, global visit limits, and automatic content truncation.

```APIDOC
## JinaBackedVisitWebpageTool API

### Description
The JinaBackedVisitWebpageTool fetches and converts web pages to markdown format. It supports Jina API fallback for accessing difficult websites, global visit limits, and automatic content truncation.

### Method
`forward`

### Endpoint
N/A (Tool Class)

### Parameters
#### Class Initialization Parameters
- **max_output_length** (integer) - Optional - Maximum length of the output content. Defaults to a reasonable value.
- **jina_keys_file** (string) - Optional - Path to a file containing Jina API keys for fallback.
- **work_dir** (string) - Optional - Directory to save fetched web pages. Defaults to the current directory.
- **global_visit_counter** (GlobalVisitCounter) - Optional - An instance of GlobalVisitCounter for rate limiting.

### Request Example
```python
from tools.jina_visit import JinaBackedVisitWebpageTool, GlobalVisitCounter
import os

global_visit_counter = GlobalVisitCounter(limit=50)

visit_tool = JinaBackedVisitWebpageTool(
    max_output_length=40000,
    jina_keys_file="path/to/jina_keys.txt",
    work_dir="./web_pages",
    global_visit_counter=global_visit_counter
)

url = "https://en.wikipedia.org/wiki/Python_(programming_language)"
content = visit_tool.forward(url)
print(content[:1000])
```

### Response
#### Success Response
- **content** (string) - Markdown formatted content of the webpage.

#### Response Example
```
# Python (programming language)

Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.

## History

Python was conceived in the late 1980s as a successor to the ABC programming language and first implemented in 1989 by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands as a "hobby, following lessons learned from Monty Python's Flying Circus", appearing first in 1991 as Python 0.9.0.

## Design Philosophy

Python's designers strive to avoid complexity by:

* **Readability counts**: Python's syntax is designed to be clear and readable.
* **Explicit is better than implicit**: What a program is doing should be obvious.
* **Simple is better than complex**: Simple designs should be preferred if they meet the requirements.

## Features

Python is a multi-paradigm programming language. Object-oriented and structured programming are supported, while a combination of functional and object-oriented programming is also supported.

### Dynamic Typing

Python uses dynamic typing, meaning that variable types are inferred at runtime.

### Automatic Memory Management

Python features automatic memory management through garbage collection.

### Large Standard Library

Python comes with a large standard library that provides modules for various tasks, such as string manipulation, file I/O, and networking.
```

### Additional Information
- Visit budget can be checked using `global_visit_counter.get_count()` and `global_visit_counter.limit`.
```

--------------------------------

### Add Records to Database

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Adds a list of records to a specified table in the database. Ensure the 'db_tool' object is initialized and the table exists.

```python
records = [
    {"title": "Deep Learning", "authors": "LeCun et al.", "year": 2015, "venue": "Nature", "abstract": "..."},
    {"title": "Attention Is All You Need", "authors": "Vaswani et al.", "year": 2017, "venue": "NeurIPS", "abstract": "..."}
]
result = db_tool.forward(
    operation="add_records",
    table_name="research_papers",
    records=records
)
print(result)  # Successfully inserted 2 records
```

--------------------------------

### UMEM Repository Structure

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Overview of the directory structure for the UMEM project, detailing the purpose of each main directory.

```text
UMEM/
├── assets/                     # Static assets (images, logos, etc.)
├── data/                       # Datasets (.parquet, .json) and preprocessing scripts
├── umem_scripts/               # Entry scripts for training and evaluation
│   ├── eval_umem.py            # Main Python script for evaluation logic
│   ├── run_eval.sh             # Shell script to launch evaluation
│   └── run_train.sh            # Shell script to launch training
├── verl/                       # Core source code library
│   ├── __init__.py
│   │
│   ├── trainer/                # Training main loops and algorithm implementations
│   │   ├── main_ppo.py         # Entry point for PPO algorithm
│   │   └── ...
│   │
│   ├── umem/                   # UMEM algorithm-specific components
│   │   ├── __init__.py
│   │   ├── llm_agent/          # Executor and Extractor
│   │   │   └── ...
│   │   └── memory/             # Memory module: Vector DB and KV Cache management
│   │       └── ...
│   │
│   ├── workers/                # Ray distributed workers for parallel computing
│   │   ├── retriever           # Retrieval service worker
│   │   └── ...
│   │
│   └── utils/                  # General utility functions and tools
│       └── ...
│
├── requirements.txt            # Project dependencies list
└── setup.py                    # Installation script (pip install -e .)
```

--------------------------------

### Build Semantic Neighborhoods (SNM)

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Script to build semantic neighborhoods from prepared data using a specified encoder and top N neighbors.

```bash
python scripts/build_neighborhoods.py \
  --input data/mmlu_ori.jsonl \
  --encoder bge-m3 \
  --top_n 3 \
  --output data/mmlu.jsonl
```

--------------------------------

### JinaBackedVisitWebpageSummaryTool API

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Visits webpages and extracts targeted information using an LLM based on a specified search goal.

```APIDOC
## JinaBackedVisitWebpageSummaryTool API

### Description
The JinaBackedVisitWebpageSummaryTool extends webpage visiting with targeted information extraction using an LLM. It extracts only relevant information based on a specified search goal.

### Method
`forward`

### Endpoint
N/A (Tool Class)

### Parameters
#### Class Initialization Parameters
- **max_output_length** (integer) - Optional - Maximum length of the output content. Defaults to a reasonable value.
- **jina_keys_file** (string) - Optional - Path to a file containing Jina API keys for fallback.
- **work_dir** (string) - Optional - Directory to save fetched web pages. Defaults to the current directory.
- **summary_model_name** (string) - Required - The name of the LLM to use for summarization.
- **temperature** (float) - Optional - Sampling temperature for the LLM. Defaults to 0.0.
- **summary_timeout** (float) - Optional - Timeout for the summarization process in seconds. Defaults to a reasonable value.
- **global_visit_counter** (GlobalVisitCounter) - Optional - An instance of GlobalVisitCounter for rate limiting.

#### Method Parameters
- **url** (string) - Required - The URL of the webpage to visit.
- **summary_goal** (string) - Required - The specific information to extract from the webpage.

### Request Example
```python
from tools.jina_visit import JinaBackedVisitWebpageSummaryTool, GlobalVisitCounter

# Initialize with a global visit counter
global_visit_counter = GlobalVisitCounter(limit=50)

# Create summary-enabled visit tool
summary_tool = JinaBackedVisitWebpageSummaryTool(
    max_output_length=40000,
    jina_keys_file="path/to/jina_keys.txt",
    work_dir="./web_pages",
    summary_model_name="qwen3-next-80b-a3b-instruct",
    temperature=0.0,
    summary_timeout=120.0,
    global_visit_counter=global_visit_counter
)

# Extract targeted information from webpage
url = "https://en.wikipedia.org/wiki/Python_(programming_language)"
summary_goal = "Extract information about Python's history and key features"
result = summary_tool.forward(url, summary_goal)
print(result)
```

### Response
#### Success Response
- **result** (string) - Extracted information based on the specified summary goal.

#### Response Example
```
- Python was created by Guido van Rossum in 1991, starting as a successor to the ABC programming language.
- Key features include dynamic typing, garbage collection, a large standard library, and support for multiple programming paradigms.
```
```

--------------------------------

### Run UMEM Evaluation

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Shell script to execute the UMEM evaluation in a streaming protocol.

```bash
bash umem_scripts/run_eval.sh
```

--------------------------------

### Initialize and Use DBTableCodeTool for MongoDB

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Initialize the DBTableCodeToolInterface for MongoDB operations. This tool supports CRUD operations and requires a connection string and database name. Global counters can be used for limiting operations like table creation.

```python
from tools.db_table_code_v2 import DBTableCodeToolInterface, GlobalCreateTableCounter

# Initialize with MongoDB connection and create_table limit
create_table_counter = GlobalCreateTableCounter(limit=1)  # Allow only 1 table creation

db_tool = DBTableCodeToolInterface(
    connection_string="mongodb://localhost:27017/",
    database_name="research_db",
    mode="full",  # "full" for all operations, "readonly" for read + add only
    task_id="task_001",
    create_table_counter=create_table_counter
)

# Create a table
result = db_tool.forward(
    operation="create_table",
    table_name="research_papers",
    column_names="title,authors,year,venue,abstract"
)
print(result)  # Successfully created table 'task_001_research_papers' with 5 columns
```

--------------------------------

### Run Evaluation Script

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/HSCodeComp/README.md

This bash command executes the evaluation script for testing language models on HSCode prediction. Ensure the 'models_to_test' variable is configured in 'eval/test_llm.py'.

```bash
# Set models_to_test = ["gpt-4o"] in eval/test_llm.py
python eval/test_llm.py
```

--------------------------------

### Mem-Optimizer Action XML Schema

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

Defines the XML structure for actions output by Mem-Optimizer, including 'ADD' and 'UPDATE' operations.

```xml
<experience>
  <value>...</value>
  <operation>ADD</operation>
</experience>
```

```xml
<experience>
  <value>...</value>
  <operation>UPDATE 3</operation>
</experience>
```

--------------------------------

### Batch Evaluation Script for DeepWideSearch

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

This bash script iterates through a list of models, running an evaluation script for each. It's designed for the DeepWideSearch benchmark.

```bash
THREAD_NUM=4
models=("gpt-4o" "claude-sonnet-4" "gemini-2.5-pro" "deepseek-r1")

for model in "${models[@]}"
do
    echo "============================
```

--------------------------------

### Run Table-as-Search WideSearch Inference

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Executes a single WideSearch task for information retrieval using specified OpenAI models and configuration parameters. Requires Python environment and the run_widesearch_inference.py script.

```bash
# Run single WideSearch task inference
python run_widesearch_inference.py \
    --query "List the top 10 programming languages in 2025 with their creators, year of creation, and main use cases" \
    --instance-id "test_001" \
    --main-model-id "gpt-4o" \
    --tabular-model-id "gpt-4o" \
    --deep-model-id "gpt-4o" \
    --output-dir ./outputs/widesearch \
    --db-name widesearch_test \
    --main-max-step 40 \
    --subagent-max-step 40 \
    --global-search-limit 100 \
    --global-visit-limit 50
```

--------------------------------

### Run DeepSearch Inference

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Execute a single DeepSearch task with specified models and parameters. Ensure all necessary environment variables and paths are configured.

```bash
python run_deepsearch_inference.py \
    --query "Find details about 2024 Nobel Physics Prize winners including their award reasons, main research fields, and representative papers" \
    --instance-id "deep_001" \
    --main-model-id "gpt-4o" \
    --tabular-model-id "gpt-4o" \
    --deep-model-id "gpt-4o" \
    --output-dir ./outputs/deepsearch \
    --db-name deepsearch_test \
    --enable-all-context-summarization \
    --main-context-token-threshold 80000
```

--------------------------------

### Run Single Task WideSearch Inference

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README_CN.md

Execute a single WideSearch inference task using the provided query and model configurations. Specify output directory and database name for results.

```bash
python run_widesearch_inference.py \
    --query "列出 2025 年排名前 10 的编程语言及其创建者、创建年份和主要用途" \
    --instance-id "test_001" \
    --main-model-id "gpt-4o" \
    --tabular-model-id "gpt-4o" \
    --deep-model-id "gpt-4o" \
    --output-dir ./outputs/widesearch \
    --db-name widesearch_test
```

--------------------------------

### HSCode Classification and Extraction

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Loads a dataset, classifies products using an OpenAI model, extracts HS codes, and compares predictions with actual values. Requires OpenAI API key and dataset.

```python
import os
import json
from typing import Dict, List, Any, Optional
from openai import OpenAI
from tqdm import tqdm
from concurrent.futures import ThreadPoolExecutor, as_completed
import threading
import re

# Initialize OpenAI client
client = OpenAI(
    api_key=os.getenv("OPENAI_API_KEY"),
    base_url=os.getenv("OPENAI_BASE_URL")
)

def extract_hscode_from_text(text: str) -> str:
    """Parse HScode from LaTeX boxed format like \boxed{8471300000}"""
    if not text:
        return ""
    latex_pattern = r'\\boxed\{([^}]+)\}'
    matches = re.findall(latex_pattern, text)
    matches = [m for m in matches if len(m) >= 10]
    if matches:
        return re.sub(r'[^0-9]', '', str(matches[0]))
    return ""

def classify_product(record: Dict[str, Any], model_name: str) -> Optional[str]:
    """Classify a single product using the question directly"""
    messages = [{"role": "user", "content": record["question"]}]
    response = client.chat.completions.create(model=model_name, messages=messages)
    if response and response.choices:
        return response.choices[0].message.content.strip()
    return None

def load_dataset(file_path: str, limit: Optional[int] = None) -> List[Dict[str, Any]]:
    """Load dataset from JSONL file"""
    records = []
    with open(file_path, 'r', encoding='utf-8') as f:
        for i, line in enumerate(f):
            if limit and i >= limit:
                break
            records.append(json.loads(line))
    return records

# Usage example
records = load_dataset("data/test_data.jsonl")
model_name = "gpt-4o"
for record in records[:5]:
    result = classify_product(record, model_name)
    predicted = extract_hscode_from_text(result)
    actual = record.get("hs_code", "")
    print(f"Predicted: {predicted}, Actual: {actual}, Match: {predicted == actual}")
```

--------------------------------

### Hierarchical Multi-Agent Workflow Diagram

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/Table-as-Search/README.md

Visual representation of the hierarchical multi-agent workflow, showing the main agent orchestrating tabular and deep agents, which interact with a tool ecosystem.

```text
┌─────────────────────────────────────────────────────────────┐
│                        Main Agent                           │
│  • Task decomposition                                       │
│  • Sub-agent orchestration                                  │
│  • Result aggregation                                       │
└─────────────────┬───────────────────────────────────────────┘
                  │
        ┌─────────┴─────────┐
        ▼                   ▼
┌──────────────────┐ ┌──────────────────┐
│ Tabular Agent    │ │ Deep Agent       │
│ • Wide search    │ │ • Deep reasoning │
│ • Entity list    │ │ • Attribute      │
│ • Quick scan     │ │   extraction     │
└────────┬─────────┘ └────────┬─────────┘
         │                    │
         └────────┬───────────┘
                  ▼
         ┌────────────────────┐
         │  Tool Ecosystem    │
         │  • Google Search   │
         │  • Web Visit       │
         │  • Table Manager   │
         │  • Summarization   │
         └────────────────────┘
```

--------------------------------

### GlobalManagedAgentCounter for Sub-Agent Limits

Source: https://context7.com/aidc-ai/marco-deepresearch/llms.txt

Python class for thread-safe tracking of sub-agent calls to prevent excessive delegation. Initializes with limits and provides methods to check and increment call counts.

```python
from run_widesearch_inference import GlobalManagedAgentCounter

# Initialize counter with limits for each sub-agent
counter = GlobalManagedAgentCounter(limits={
    "tabular_search_agent": 5,  # Allow 5 calls to tabular search
    "deep_search_agent": 3      # Allow 3 calls to deep search
})

# Attempt to call a sub-agent (returns False if limit reached)
if counter.try_increment("tabular_search_agent"):
    # Proceed with call
    print("Tabular search agent called successfully")
else:
    print("Tabular search agent limit reached")

# Check status
status = counter.get_all_status()
print(status)
# {'tabular_search_agent': {'count': 1, 'limit': 5, 'remaining': 4},
#  'deep_search_agent': {'count': 0, 'limit': 3, 'remaining': 3}}

# Get detailed statistics including call history
stats = counter.get_statistics()
print(stats["call_history"]["tabular_search_agent"])
# [{'call_number': 1, 'timestamp': '2024-01-15T10:30:45.123456'}]
```

--------------------------------

### UMEM Citation

Source: https://github.com/aidc-ai/marco-deepresearch/blob/main/Marco-DeepResearch-Family/UMEM/README.md

BibTeX entry for citing the UMEM framework in research publications.

```bibtex
@misc{ye2026umemunifiedmemoryextraction,
      title={UMEM: Unified Memory Extraction and Management Framework for Generalizable Memory},
      author={Yongshi Ye and Hui Jiang and Feihu Jiang and Tian Lan and Yichao Du and Biao Fu and Xiaodong Shi and Qianghuai Jia and Longyue Wang and Weihua Luo},
      year={2026},
      eprint={2602.10652},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2602.10652},
}
```