WorldModelBench

https://github.com/worldmodelbench-team/worldmodelbench

Admin

WorldModelBench is a benchmark designed to evaluate the world modeling capabilities of video...

Tokens:546

Snippets:7

Trust Score:3.3

Update:1 month ago

Show doc for...

Tokens:

Raw

### Directory Structure

Source: https://github.com/worldmodelbench-team/worldmodelbench/blob/main/README.md

Expected file layout for the WorldModelBench evaluation environment.

```text
└── worldmodelbench
    └── images (first frames of videos)
    └── evaluation.py (evaluation script)
    └── worldmodelbench.json (test set)
    ...
```

--------------------------------

### Run Evaluation Script

Source: https://github.com/worldmodelbench-team/worldmodelbench/blob/main/README.md

Command to execute the evaluation script after generating videos.

```bash
python evaluate.py --model_name TESTED_MODEL --video_dir GENERATED_VIDEOS --judge PATH_TO_JUDGE
```

--------------------------------

### Test Set JSON Format

Source: https://github.com/worldmodelbench-team/worldmodelbench/blob/main/README.md

Structure of the worldmodelbench.json file containing test instances.

```json
[
  {
          "domain": "autonomous vehicle",
          "subdomain": "Stopping",
          "text_first_frame": "The autonomous vehicle approaches a traffic light on a bridge surrounded by tall buildings. Construction barriers line the sides of the bridge with a yellow traffic light visible ahead.",
          "text_instruction": "The autonomous vehicle stops at the traffic light on the bridge.",
          "first_frame": "images/69620089860948e38a4921dd4869d24f.jpg"
      }
...
]
```

--------------------------------

### Citation BibTeX

Source: https://github.com/worldmodelbench-team/worldmodelbench/blob/main/README.md

BibTeX entry for citing the WorldModelBench paper.

```bibtex
@article{Li2025WorldModelBench,
  title={WorldModelBench: Judging Video Generation Models As World Models},
  author={Dacheng Li and Yunhao Fang and Yukang Chen and Shuo Yang and Shiyi Cao and Justin Wong and Michael Luo and Xiaolong Wang and Hongxu Yin and Joseph E. Gonzalez and Ion Stoica and Song Han and Yao Lu},
  year={2025},
}
```

Tokens:

Raw