DeepVerify

https://github.com/xiongsiheng/deepverify
Admin
DeepVerify is an agentic, modular, and extensible system built on the MCP server framework that...
Tokens:7,466
Snippets:78
Trust Score:7.3
Update:2 months ago
Show doc for...
Tokens:
Raw
### Quickstart End-to-End Setup

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

Complete sequence of commands to install dependencies, configure the environment, and run a basic agent workflow.

```bash
# 1. Install dependencies
pixi install

# 2. Enter the pixi environment
pixi shell

# 3. Create your environment file
cp env.sample .env
# Edit .env with your API keys

# 4. Load environment variables
set -a && source .env && set +a

# 5. Start the MCP server
pixi run mcp-server-dev

# 6. In another shell, run an agent
python -m deepverify.agents.basic.workflow_fixed --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists."
```

--------------------------------

### Proof Tree Example 2

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/decompose.md

A multi-level deep proof tree example regarding animal behavior.

```json
{
    "x1_chain_of_thought" : "Hunting is a type of learned behavior and animals learn behaviors from their parents, so kittens likely learn to hunt from their mothers.",
    "x2_nodes" : [
        {
            "x1_id"        : "root",
            "x2_parent"    : "null"
            "x3_statement" : "kittens learn to hunt by watching their models",
        },
        
        {
            "x1_id"        : "root/0",
            "x2_parent"    : "root"
            "x3_statement" : "hunting is a learned behavior",
        },
        {
            "x1_id"        : "root/0/0",
            "x2_parent"    : "root/0"
            "x3_statement" : "hunting is a kind of skill",
        },
        {
            "x1_id"        : "root/0/1",
            "x2_parent"    : "root/0"
            "x3_statement" : "skill is a kind of learned characteristic",
        },
        
        {
            "x1_id"        : "root/1",
            "x2_parent"    : "root"
            "x3_statement" : "kittens learn behaviors from watching their mother",
        },
        {
            "x1_id"        : "root/1/0",
            "x2_parent"    : "root/1"
            "x3_statement" : "a mother is a kind of female parent",
        },
        {
            "x1_id"        : "root/1/1",
            "x2_parent"    : "root/1"
            "x3_statement" : "Animals learn some behaviors from watching their parents",
        },
        {
            "x1_id"        : "root/1/2",
            "x2_parent"    : "root/1"
            "x3_statement" : "a kitten is a {young, baby} cat",
        },
    ]
}
```

--------------------------------

### Install Project Dependencies

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

Command to install project dependencies using the pixi environment manager.

```bash
pixi install
```

--------------------------------

### Install MARS Dependencies

Source: https://github.com/xiongsiheng/deepverify/blob/main/mars/README.md

Clone the repository and install the required Python packages.

```sh
# git clone this repo

cd MARS

pip install -r Requirements.txt
```

--------------------------------

### Proof Tree Example 1

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/decompose.md

A simple one-level deep proof tree example regarding plant biology.

```json
{
    "x1_chain_of_thought" : "Flowers are a part of the plant that contains pollen, and bees eat pollen, so the bee gets it's food from the flowers.",
    "x2_nodes" : [
        {
            "x1_id"        : "root",
            "x2_parent"    : "null"
            "x3_statement" : "flowers are the part of the plant that bees get food from"
        },
        {
            "x1_id"        : "root/0",
            "x2_parent"    : "root"
            "x3_statement" : "bees eat pollen"
            
        },
        {
            "x1_id"        : "root/1",
            "x2_parent"    : "root"
            "x3_statement" : "a flower is a part of a plant for {attracting pollinators, producing seeds}"
        }
    ]
}
```

--------------------------------

### Launch MCP Server

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

Commands to start the MCP server in either development or production mode.

```bash
# Load pixi environment
pixi shell

# Set environment variables
set -a && source .env && set +a

# dev (single worker, auto-restart on code change)
pixi run mcp-server-dev

# or

# prod (multiple workers)
pixi run mcp-server
```

--------------------------------

### Configure Agent Tool Whitelisting

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

Examples of running the DeepResearch agent with different tool availability configurations.

```bash
# Load pixi environment
pixi shell

# Load environment variables
set -a && source .env && set +a

# DeepResearch Agent - Tool whitelist examples
python -m deepverify.agents.deepresearch.deepresearch --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists." --no_tools
python -m deepverify.agents.deepresearch.deepresearch --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists." --tool_whitelist generate_queries question_answer
python -m deepverify.agents.deepresearch.deepresearch --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists." --all_tools
```

--------------------------------

### Explanation Field Citation Example

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/question_answerer.md

Demonstrates how to embed citations within the explanation field using the required XML-style tags.

```json
{
    "explanation" : "Bears come in different sizes: <cite ids=[1]>polar bears can weight up to 1,000 pounds</cite>, but <cite ids=[2,3]>black bears are smaller, often under 600 pounds</cite>",
    "citations:  [{
        "citation_id" : 0,
        "id_raw" : "1423",
        "evidence_text" : "Polar bears are the largest bear, at up to 450kg",
        "rationale" : ...
    }, {
        "citation_id" : 1,
        "id_raw" : "23452",
        "evidence_text" : "Black bears are smaller than polar bears",
        "rationale" : ...
    }, {
        "citation_id" : 2,
        "id_raw" : "49302",
        "evidence_text" : "A large male black bear might weight up to 600lbs.",
        "rationale" : ...
    }}]
```

--------------------------------

### Entailment Bank Task 2 Output Format

Source: https://github.com/xiongsiheng/deepverify/blob/main/mars/mars/prompt/prompt_SWAP_generator_entailment_bank_Task2_chg_act_type_with_assumption.txt

Example output structure demonstrating the step-by-step reasoning process, including graph updates and assumption handling.

```text
"Goal": "Determine the claim 'The earth revolving around the sun causes Leo to appear in different areas in the sky at different times of year' is 'supported' or 'refuted'."  
"Initial state": "We have 8 sentences 'Leo is a kind of constellation', 'A circle is a kind of shape', 'Approximately means about', 'Planets orbit stars', 'A constellation contains stars', 'Years ( y ) are a metric unit used for measuring time generally used for values between 1 and 14000000000', 'Similar means in common', 'The solar system is a kind of system'"  
"Initial graph": {"Statement": {"s1": "Leo is a kind of constellation.", "s2": "A circle is a kind of shape.", "s3": "Approximately means about.", "s4": "Planets orbit stars.", "s5": "A constellation contains stars.", "s6": "Years ( y ) are a metric unit used for measuring time generally used for values between 1 and 14000000000.", "s7": "Similar means in common.", "s8": "The solar system is a kind of system."}, "Entailment": {"s1": "Evidence", "s2": "Evidence", "s3": "Evidence", "s4": "Evidence", "s5": "Evidence", "s6": "Evidence", "s7": "Evidence", "s8": "Evidence"}}
"Action 1": "Prove 'Leo is a constellation containing stars'."
"State 1": "Based on s1 (Leo is a kind of constellation) and s5 (A constellation contains stars), we can know Leo is a constellation containing stars."  
"Graph 1": {"Statement": {"s9": "Leo is a constellation containing stars."}, "Entailment": {"s9": ["s1", "s5"]}}  
"Action 2": "Prove 'The earth revolving around the sun causes Leo to appear in different areas in the sky at different times of year'."  
"State 2": "After chekcing all avaliable information, we found that the only useful statement is s9 (Leo is a constellation containing stars), which is insuffient to prove the target statement. Thus, we need to make an assumption (The earth revolving around the sun causes stars to appear in different areas in the sky at different times of year). Based on s9 and the assumption, we can know the earth revolving around the sun causes Leo to appear in different areas in the sky at different times of year."
"Graph 2": {"Statement": {"s10": "The earth revolving around the sun causes stars to appear in different areas in the sky at different times of year.", "s11": "The earth revolving around the sun causes Leo to appear in different areas in the sky at different times of year."}, "Entailment": {"s10": "Assumption", "s11": ["s9", "s10"]}}
"Action 3": "Assess the veracity of the claim."
 "State 3": "Based on s11 (The earth revolving around the sun causes Leo to appear in different areas in the sky at different times of year) and the claim (The earth revolving around the sun causes Leo to appear in different areas in the sky at different times of year), we know that the claim is supported." 
"Final answer": "Supported"
```

--------------------------------

### Entailment Bank Task 2 Input Format

Source: https://github.com/xiongsiheng/deepverify/blob/main/mars/mars/prompt/prompt_SWAP_generator_entailment_bank_Task2_chg_act_type_with_assumption.txt

Example input structure for a claim verification problem containing multiple sentences and a target claim.

```text
"Problem": "sent1: leo is a kind of constellation sent2: a circle is a kind of shape sent3: approximately means about sent4: planets orbit stars sent5: a constellation contains stars sent6: years ( y ) are a metric unit used for measuring time generally used for values between 1 and 14000000000 sent7: similar means in common sent8: the solar system is a kind of system\n\nclaim: the earth revolving around the sun causes leo to appear in different areas in the sky at different times of year"
```

```text
"Problem": "sent1: if something used to be in the past then that something has changed sent2: appear is similar to apparent sent3: earth is a kind of celestial object sent4: a star is a kind of celestial object / celestial body sent5: position means a place in a structure sent6: motion / movement means moving / to move sent7: the earth rotating on its axis causes stars to appear to move across the sky at night sent8: a phase change is when matter / a substance changes from one state of matter into another state of matter sent9: if something is visible , then that something can be observed\n\nclaim: the earth revolves around the sun causes stars to move relative to the horizon during the night"
```

--------------------------------

### Initialize Environment Configuration

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

Copy the sample environment file to create a local .env file for API key configuration.

```bash
cp env.sample .env
# Edit .env with your API keys
```

--------------------------------

### Define Required Environment Variables

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

List of necessary API keys for LLM, search, and tracing services.

```bash
# LLM keys
GEMINI_API_KEY=
OPENAI_API_KEY=
ANTHROPIC_API_KEY=

# SERPAPI (for `search_google_scholar` + `search_google` tool)
SERPAPI_API_KEY=

# JINA Reader (for `read_url` tool)
JINA_API_KEY=

# LangSmith for tracing and debugging LangGraph agents
LANGCHAIN_TRACING_V2="true"
LANGCHAIN_API_KEY=
LANGCHAIN_PROJECT="DeepVerify Research Agent"
```

--------------------------------

### Execute Basic Agent Workflows

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

Commands to run different agent types including fixed workflows, MARS, and ReAct agents.

```bash
# Load pixi environment
pixi shell

# Set environment variables
set -a && source .env && set +a

# Run the fixed workflow
python -m deepverify.agents.basic.workflow_fixed --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists."

# Run the multi-agent reasoning system (MARS)
python -m deepverify.agents.basic.workflow_mars --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists."

# Run ReAct agent
python -m deepverify.agents.basic.react_agent --claim "Ultra-high performance concrete with a compressive strength greater than 45,000 psi exists."  --feasibility_assessment
```

--------------------------------

### Proof Tree JSON Schema

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/decompose.md

The required structure for the output proof tree.

```json
{
"x1_chain_of_thought": "... internal chain of though in prose english.  cite relevant facts and explain logical implication ...",
"x2_nodes": [
    {
        "x1_id"        : "root",
        "x2_parent"    : "... parent in the proof tree ..."
        "x3_statement" : "... statement in prose english ...",
    },
    ...
]
}
```

--------------------------------

### Run Auxiliary Data Processing

Source: https://github.com/xiongsiheng/deepverify/blob/main/mars/README.md

Generate or transform evidence statements using language models.

```sh
# Create evidence statements from LMs
python create_evidence_with_LM.py --claim ../dataset/concrete_claim1.txt --model gpt-4o --output ../dataset/concrete_claim1_evidence_generated_with_gpt_4o.txt

# Transform evidence format with LMs
python evidence_format_transformation.py  --claim ../dataset/uhpc0_claim.txt --evidence ../dataset/uhpc0_evidence_document.txt --model gpt-4o --output ../dataset/uhpc0_evidence_extracted_with_gpt_4o.txt
```

--------------------------------

### Run MARS Inference

Source: https://github.com/xiongsiheng/deepverify/blob/main/mars/README.md

Execute inference tasks on datasets or custom claim/evidence files.

```sh
cd src

# Test on entailment bank dataset
python main.py  --dataset entailment_bank --subset all --structure_check --output_dir ../results/test_entailment_bank_task2_gpt_4o_mini  --num_rollouts 32  --num_generations 3 --group_size 3 --beam_width 3 --cmp_per_opt 1 --visualize

# Test on new example
python main.py  --claim ../dataset/example_claim.txt --evidence ../dataset/example_evidence_statements.txt --structure_check --output_dir ../results/test_example_gpt_4o_mini  --num_rollouts 32  --num_generations 3 --group_size 3 --beam_width 3 --cmp_per_opt 1 --visualize

# Test on multiple sub claims
python main.py  --claim ../dataset/example_subclaims.txt --evidence ../dataset/example_evidence_statements.txt --multi_claim --structure_check --output_dir ../results/test_example_gpt_4o_mini  --num_rollouts 32  --num_generations 3 --group_size 3 --beam_width 3 --cmp_per_opt 1 --visualize
```

--------------------------------

### Claim Verification JSON Output Format

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/question_answerer.md

The required JSON structure for the assistant's response, including citations, explanation, and feasibility metrics.

```json
{
    "citations" : [
        {
            "citation_id"   : str,  # sequential numeric id
            "id_raw"        : str,  # raw id from the evidence
            "evidence_text" : str,  # text from the evidence
            "rationale"     : str   # rationale for why this evidence is relevant to the answer
        },
        ...
    ],
    "explanation" : str,
    "feasibility_score" : int,
    "technological_readiness" : int,
}
```

--------------------------------

### Project and Research Citations

Source: https://github.com/xiongsiheng/deepverify/blob/main/README.md

BibTeX entries for citing the DeepVerify software and associated research papers.

```bibtex
@software{xiong2026deepverify,
  title   = {DeepVerify: Evidence-Based Expert-Level Scientific Claim Verification},
  author  = {Xiong, Siheng and Gungordu, Oguzhan and Johnson, Blair and Okamoto, Mika and Kerce, James C and Fekri, Faramarz},
  year    = {2026},
  doi     = {10.5281/zenodo.18894860},
  url     = {https://github.com/xiongsiheng/DeepVerify}
}
```

```bibtex
@article{xiong2026scaling,
  title={Scaling Search-Augmented LLM Reasoning via Adaptive Information Control},
  author={Xiong, Siheng and Gungordu, Oguzhan and Johnson, Blair and Kerce, James C and Fekri, Faramarz},
  journal={arXiv preprint arXiv:2602.01672},
  year={2026}
}
```

```bibtex
@inproceedings{xiong2025deliberate,
  title={Deliberate reasoning in language models as structure-aware planning with an accurate world model},
  author={Xiong, Siheng and Payani, Ali and Yang, Yuan and Fekri, Faramarz},
  booktitle={Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  pages={31900--31931},
  year={2025}
}
```

--------------------------------

### Query Output Format

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/query_generator.md

The required JSON structure for the output of the query generation process.

```json
{
    "queries"  : list[str]
}
```

--------------------------------

### Evidence Extraction Output Format

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/evidence_extractor.md

The required JSON output format for the evidence extraction task.

```json
{
    "extractions"  : list[str]
}
```

--------------------------------

### Entailment Bank Task 2 Initial Graph Representation

Source: https://github.com/xiongsiheng/deepverify/blob/main/mars/mars/prompt/prompt_SWAP_generator_entailment_bank_Task2_chg_act_type_with_assumption_more_examples.txt

JSON structure representing the initial state and graph of statements for an entailment verification task.

```json
"Initial graph": {"Statement": {"s1": "Alaska is a state located in the United States of America.", "s2": "As the distance of a location from the North Pole becomes smaller/closer, the amount of daylight received by that location will decrease during the winter.", "s3": "Cold means low in temperature.", "s4": "Amount of daylight means length of daylight.", "s5": "Distance is a property of space and includes ordered values of close/far.", "s6": "Moving away from the source increases the distance.", "s7": "To move away means to increase distance.", "s8": "As the sun sets, the temperature will decrease.", "s9": "Ice is colder in temperature than water.", "s10": "Daylight means sunlight.", "s11": "A hemisphere is a part of Earth.", "s12": "Earth's angle of tilt causes the length of day and night to vary.", "s13": "Alaska is in winter season.", "s14": "Daylight hours means time during which there is daylight.", "s15": "Winter means environmental/outdoor temperature decreases.", "s16": "Being in the sun is synonymous with being in the sunlight.", "s17": "Winter has cold temperatures/weather.", "s18": "Navassa Island (US) is located in the Northern Hemisphere.", "s19": "Intensity of sunlight is similar to amount of sunlight.", "s20": "Period of daylight is synonymous with amount of daylight.", "s21": "Ice is cold in temperature.", "s22": "Daylight is when the sun shines on a location."}, "Entailment": {"s1": "Evidence", "s2": "Evidence", "s3": "Evidence", "s4": "Evidence", "s5": "Evidence", "s6": "Evidence", "s7": "Evidence", "s8": "Evidence", "s9": "Evidence", "s10": "Evidence", "s11": "Evidence", "s12": "Evidence", "s13": "Evidence", "s14": "Evidence", "s15": "Evidence", "s16": "Evidence", "s17": "Evidence", "s18": "Evidence", "s19": "Evidence", "s20": "Evidence", "s21": "Evidence", "s22": "Evidence"}}
```

--------------------------------

### Evidence Filter Output Format

Source: https://github.com/xiongsiheng/deepverify/blob/main/deepverify/prompts/evidence_filter.md

The required JSON output format for the filtered evidence list.

```json
{
    "evidence"  : list[str] # list of strings of length {TOPK}
}
```
Tokens:
Raw