InfoDeepSeek

https://github.com/yunjiaxi/infodeepseek
Admin
InfoDeepSeek is a benchmark and evaluation framework for assessing agentic information seeking...
Tokens:1,586
Snippets:13
Trust Score:6.5
Update:1 month ago
Show doc for...
Tokens:
Raw
### Set up the InfoDeepSeek environment

Source: https://github.com/yunjiaxi/infodeepseek/blob/main/README.md

Commands to create and activate a conda environment and install necessary dependencies.

```bash
conda create -n InfoSeekAgent python=3.10
conda activate InfoSeekAgent
pip install -r requirements.txt
```

--------------------------------

### Configure Alternative Search Engines

Source: https://github.com/yunjiaxi/infodeepseek/blob/main/README.md

Switches the search engine provider and configures necessary API keys for Google, Yahoo, or Bing.

```bash
sh script/run_google.sh
```

```bash
export API_KEY="your_api_key_for deepseek"
export API_TYPE=deepseek
export API_BASE=https://api.deepseek.com
export http_proxy='http://127.0.0.1:7890'  # sometimes need proxy

export SERPER_API_KEY="your_api_key_for_serper"  # for google search

lang="zh" # or "en"
llm='deepseek-chat'  # 'deepseek-reasoner'
fast_llm='deepseek-chat'
eval_llm='deepseek-chat'
max_iter_num=5
file_dir='./data/InfoDeepSeek_v1'  # do not add .json
search_type='google'


python -m InfoSeekAgents.agent_start --query_path ${file_dir}.json  --fast_llm_name ${fast_llm} \
      --lang ${lang} --llm_name ${llm} --max_iter_num ${max_iter_num} --wo_tool --num_worker=1 --search_type=${search_type} --overwrite

python eval/eval.py --input_file ${file_dir}_result_${lang}_${llm}_${max_iter_num}_5_wotool_True_${search_type}.jsonl \
      --eval_llm_name ${eval_llm} --reuse --num_worker=2

python eval/cal_acc.py --input_file ${file_dir}_result_${lang}_${llm}_${max_iter_num}_5_wotool_True_${search_type}_${eval_llm}_score.jsonl \
      --print_attribute
```

```bash
sh script/run_bing.sh
```

```bash
sh script/run_yahoo.sh
```

--------------------------------

### Run Single Query via CLI

Source: https://github.com/yunjiaxi/infodeepseek/blob/main/README.md

Executes a single query using the DeepSeek model with DuckDuckGo as the default search engine.

```bash
sh script/run_toy.sh
```

```bash
export API_KEY="your_api_key_for_deepseek"
export API_TYPE=deepseek
export API_BASE=https://api.deepseek.com
# export http_proxy='http://127.0.0.1:7890'  # sometimes need proxy, change 7890 to your port.

lang="en"
query='When did the country ranked 221st in the 2017 list of countries and regions by population gain independence?'
llm='deepseek-chat'
fast_llm='deepseek-chat'
max_iter_num=5

python -m InfoSeekAgents.agent_start --query "${query}" --lang ${lang} --llm_name ${llm} \
      --max_iter_num ${max_iter_num} --fast_llm_name ${fast_llm}
```

--------------------------------

### Process and Evaluate Multiple Queries

Source: https://github.com/yunjiaxi/infodeepseek/blob/main/README.md

Runs batch processing on a JSON dataset and evaluates the results using a secondary model.

```bash
sh script/run_gemini.sh
```

```bash
export API_KEY='your_api_key_for_gemini'
export API_TYPE=google
export API_BASE=https://api.deerapi.com/v1/chat/completions
#export http_proxy='http://127.0.0.1:7890'  # sometimes need proxy

lang='zh'  # or 'en'
llm='gemini-2.5-flash-preview-04-17'  # or 'gemini-2.5-pro-exp-03-25'
fast_llm='gemini-2.0-flash'
file_dir='data/InfoDeepSeek_v1'  # do not add .json
max_iter_num=5


python -m InfoSeekAgents.agent_start --query_path ${file_dir}.json \
      --lang ${lang} --llm_name ${llm} --fast_llm_name ${fast_llm} --max_iter_num ${max_iter_num} \
      --wo_tool --num_worker=1 --overwrite

export API_KEY="your_api_key_for_deepseek"
export API_TYPE=deepseek
export API_BASE=https://api.deepseek.com
eval_llm='deepseek-chat'

python eval/eval.py --input_file ${file_dir}_result_${lang}_${llm}_${max_iter_num}_5_wotool_True_ddg.jsonl \
      --eval_llm_name ${eval_llm} --reuse

python eval/cal_acc.py --input_file ${file_dir}_result_${lang}_${llm}_${max_iter_num}_5_wotool_True_ddg_${eval_llm}_score.jsonl \
      --print_attribute
```

```bash
sh script/run_deepseek.sh
```

```bash
sh script/run_qwen.sh
```

```bash
sh script/run_gpt.sh
```

--------------------------------

### BibTeX Citation for InfoDeepSeek

Source: https://github.com/yunjiaxi/infodeepseek/blob/main/README.md

Use this BibTeX entry to cite the InfoDeepSeek paper in academic publications.

```bibtex
@article{xi2025infodeepseek,
        title={InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generation},
        author={Yunjia Xi and Jianghao Lin and Menghui Zhu and Yongzhao Xiao and Zhuoying Ou and Jiaqi Liu and Tong Wan and Bo Chen and Weiwen Liu and Yasheng Wang and Ruiming Tang and Weinan Zhang and Yong Yu},
        year={2025},
        journal={arXiv preprint arXiv:2505.15872},
        url={https://arxiv.org/abs/2505.15872},
}
```
Tokens:
Raw