### Download AgentWorldBench and Install Dependencies

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Download the benchmark dataset using huggingface-cli and install the necessary openai Python package.

```bash
# Download the benchmark
huggingface-cli download Qwen/AgentWorldBench --repo-type dataset --local-dir ./AgentWorldBench

# Install dependencies
pip install openai
```

--------------------------------

### AgentWorldBench Evaluation Results Example

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Example output format for the AgentWorldBench evaluation, showing per-domain scores for format, factuality, consistency, realism, quality, and the total score, followed by an overall score.

```text
======================================================================
AgentWorldBench Evaluation Results (example output)
======================================================================

--- MCP (286/286 valid, 0 failed) ---
         format: 81.46
     factuality: 68.75
    consistency: 72.92
       realism: 71.88
        quality: 67.08
    total_score: 72.42
...

======================================================================
Overall: 56.39
======================================================================
```

--------------------------------

### Side-Effect Modeling: Package Installation

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Models the side effect of installing a software package.

```Shell
pip install pkg
```

--------------------------------

### Deploy Qwen-AgentWorld with SGLang

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Launches a server for Qwen-AgentWorld-35B-A3B using SGLang. Ensure SGLang is installed and the model path is correct.

```bash
python -m sglang.launch_server \
    --model-path Qwen/Qwen-AgentWorld-35B-A3B \
    --port 8000 \
    --tensor-parallel-size 4 \
    --context-length 262144 \
    --reasoning-parser qwen3
```

--------------------------------

### AgentWorldBench Data Format Example

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Illustrates the structure of per-domain JSONL files used in AgentWorldBench, detailing key fields like task, prompt, response, and system_str.

```json
{
    "task": "mcp",
    "id": 145256090131919,
    "prompt": ["### Turn 1\n**Action:**\n```json\n{...}\n```\n..."],
    "response": ["**Environment Observation:**\n{...}"],
    "current_prompt": "### Turn 1\n**Action:**\n...",
    "system_str": "# Role and Objective\n\nYou are a **Tool World Model** ...",
    "turn_idx": 1,
    "total_turns": 5
}
```

--------------------------------

### Error Handling: File Not Found

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Example of the error message generated when a specified file does not exist.

```Shell
cat: file: No such file or directory
```

--------------------------------

### Error Handling: Permission Denied

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Example of the error message generated when a command lacks the necessary permissions.

```Shell
bash: permission denied
```

--------------------------------

### Error Handling: Command Not Found

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Example of the error message generated when a command is not found by the shell.

```Shell
bash: command: command not found
```

--------------------------------

### Error Handling: Indentation Errors

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Example of Python's IndentationError, including context.

```Python
Python IndentationError with context
```

--------------------------------

### Error Handling: Syntax Errors

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Example of error messages for shell or Python syntax errors, including line numbers.

```Shell
Python/shell syntax errors with line numbers
```

--------------------------------

### Argument Validation Error Example

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/mcp/system_prompt.txt

Simulates an error response when a tool call contains undefined or invalid arguments, including details from pydantic validation.

```json
{"type":"text","text":"Error calling tool 'tool_name': N validation errors for call[tool_name]
argument_name
  Unexpected keyword argument [type=unexpected_keyword_argument, input_value=..., input_type=...]\n    For further information visit https://errors.pydantic.dev/..."}
```

--------------------------------

### Deploy Qwen-AgentWorld with vLLM

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Serves Qwen-AgentWorld-35B-A3B using vLLM for high-throughput inference. The `--language-model-only` flag is crucial as the model architecture includes visual components but only language model weights are provided.

```bash
vllm serve Qwen/Qwen-AgentWorld-35B-A3B \
    --port 8000 \
    --tensor-parallel-size 4 \
    --max-model-len 262144 \
    --reasoning-parser qwen3 \
    --language-model-only \
    --trust-remote-code
```

--------------------------------

### Time & Blocking Behavior Simulation: Instant Commands

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Simulates the behavior of instant commands that complete immediately, showing full output.

```Shell
0.1s
```

--------------------------------

### Build Tool Commands

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Commands for compiling and building software projects.

```Shell
make
```

```Shell
gcc
```

```Shell
python3
```

--------------------------------

### Run AgentWorldBench Evaluation Pipeline

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Execute the three-step evaluation process: world model inference, LLM judge scoring, and score aggregation. Ensure to set the OPENAI_API_KEY environment variable for the scoring step.

```bash
cd eval

# Step 1: Run world model inference
python eval.py infer \
    --data-dir ../AgentWorldBench \
    --model-base-url http://localhost:8000/v1 \
    --model-name Qwen/Qwen-AgentWorld-35B-A3B \
    --output-dir ./results

# Step 2: Run LLM judge scoring
export OPENAI_API_KEY="your-api-key"
python eval.py judge \
    --predictions ./results/predictions.jsonl \
    --judge-base-url https://api.openai.com/v1 \
    --judge-model gpt-5.2-2025-12-11 \
    --output-dir ./results

# Step 3: Aggregate and display scores
python eval.py score --predictions ./results/judged.jsonl
```

--------------------------------

### Simple Shell Commands

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Basic shell commands for file and directory operations, and outputting text.

```Shell
ls
```

```Shell
cd
```

```Shell
cat
```

```Shell
echo
```

```Shell
pwd
```

--------------------------------

### Output Formatting: Prompt Format

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Ensures the prompt format matches the established pattern.

```Terminal
Prompt format matching established pattern
```

--------------------------------

### Time & Blocking Behavior Simulation: Normal Commands

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Simulates the behavior of normal commands that complete within a few seconds, showing full output.

```Shell
1.0-5.0s
```

--------------------------------

### Package Manager Commands

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Commands for managing software packages using common package managers.

```Shell
apt
```

```Shell
pip
```

```Shell
npm
```

--------------------------------

### Available Actions for Web World Model

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/web/system_prompt.txt

Details the 21 different types of actions the Web World Model can perform.

```text
21 different types of actions are available.
noop(wait_ms: float = 1000)
mouse_move(x: float, y: float)
mouse_click(x: float, y: float, button: Literal['left', 'middle', 'right'] = 'left')
mouse_dblclick(x: float, y: float, button: Literal['left', 'middle', 'right'] = 'left')
mouse_down(x: float, y: float, button: Literal['left', 'middle', 'right'] = 'left')
mouse_up(x: float, y: float, button: Literal['left', 'middle', 'right'] = 'left')
scroll(delta_x: float, delta_y: float)
click(bid: str, button: Literal['left', 'middle', 'right'] = 'left', modifiers: list = [])
keyboard_press(key: str)
keyboard_type(text: str)
fill(bid: str, value: str, enable_autocomplete_menu: bool = False)
hover(bid: str)
tab_focus(index: int)
new_tab()
go_back()
go_forward()
goto(url: str)
tab_close()
select_option(bid: str, options: str | list[str])
send_msg_to_user(text: str)
report_infeasible(reason: str)
```

--------------------------------

### Output Assembly: Command Echo

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Represents the first part of assembled output: the command that was typed.

```Terminal
The typed command (shown on prompt line)
```

--------------------------------

### User Actions as JSON Array

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

User actions are represented as a JSON array of command objects, each specifying keystrokes and an optional duration.

```json
[
  {
    "keystrokes": "ls -la\n",
    "duration": 0.1
  },
  {
    "keystrokes": "cd project\n",
    "duration": 0.1
  }
]
```

--------------------------------

### Side-Effect Modeling: Directory Creation

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Models the side effect of creating a new directory.

```Shell
mkdir dir
```

--------------------------------

### Special Action Types: Normal Command

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

A normal command action involves echoing the command, displaying its output, and then presenting a new prompt.

```string
Normal command (`"cmd\n"`)
Echo command, show output, new prompt
```

--------------------------------

### Python Standard Library for System Operations

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/os/system_prompt.txt

Core Python functions for running subprocesses, listing directory contents, creating directories, and checking file existence. These are essential for general system interaction and file management.

```python
subprocess.run(...)
```

```python
os.listdir(path: str)
```

```python
os.makedirs(path: str, exist_ok: bool = True)
```

```python
os.path.exists(path: str)
```

--------------------------------

### Dictionary Memory Operations

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/search/system_prompt.txt

Illustrates the operations for the dictionary memory, including adding, retrieving, updating, and removing key-value pairs.

```markdown
| Operation | Input | Output |
|-----------|-------|--------|
| `add` | `{"dict": {...}}` | `Successfully added items of [keys] to memory.` |
| `update` | `{"dict": {...}}` | `Successfully updated items of [keys] in memory.` |
| `retrieve` | `{"key_list": [...]}` | Retrieved key-value pairs or "Key not found" |
| `retrieve_all` | (none) | All stored key-value pairs |
| `remove` | `{"key_list": [...]}` | `Successfully removed items of [keys] from memory.` |
| `remove_all` | (none) | `Successfully cleared all memory.` |
```

--------------------------------

### Content Requirements for Page-Changing Actions

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/web/system_prompt.txt

Specifies the checklist for determining and updating page state after actions.

```text
A page-changing action is any action that changes the visible page identity or main content, including link clicks, menu selections, tab switches, pagination, search submissions, modal openings, reload-like transitions, and browser navigation actions.

Before producing the final answer, do this checklist:
1. Determine whether the action is a page-changing action or a local-edit action.
2. If it is a page-changing action, update the page-level state first: page identity, title, main heading, visible main content, and any newly loaded or removed sections.
3. If it is a local-edit action, preserve the full page state and modify only the directly affected nodes, field values, focus, and any immediate dependent UI.
4. Preserve untouched subtrees verbatim whenever possible.
5. Return the unchanged page state only if the action clearly has no visible effect, is invalid, or targets a disabled or non-interactive element.
6. Always output exactly one complete and closed <predicted_observation></predicted_observation> block.
```

--------------------------------

### Output Formatting: Line Breaks and Spacing

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Preserves exact line breaks and spacing as they would appear in the terminal output.

```Terminal
Line breaks and spacing as they would appear
```

--------------------------------

### ImpressTools for Presentation Control

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/os/system_prompt.txt

Functions from ImpressTools to manage presentation slides, including navigation, background settings, duplication, and font configuration. Useful for automating slide deck operations.

```python
ImpressTools.go_to_slide(...)
```

```python
ImpressTools.set_slide_background(...)
```

```python
ImpressTools.position_box(...)
```

```python
ImpressTools.duplicate_slide(...)
```

```python
ImpressTools.set_slide_font(...)
```

```python
ImpressTools.export_to_image(...)
```

```python
ImpressTools.set_text_strikethrough(...)
```

```python
ImpressTools.configure_auto_save(...)
```

```python
ImpressTools.get_slide_count()
```

```python
ImpressTools.write_text(...)
```

```python
ImpressTools.insert_image(...)
```

```python
ImpressTools.save()
```

--------------------------------

### Output Assembly: New Prompt

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Represents the final part of assembled output: the new prompt indicating readiness for the next command, unless the process is still running.

```Terminal
Ready for next command (unless process still running)
```

--------------------------------

### BrowserTools for Browser Automation

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/os/system_prompt.txt

A collection of BrowserTools functions for automating common browser tasks such as opening settings, bookmarking, and managing tabs. These are useful for web-based automation scenarios.

```python
BrowserTools.open_privacy_settings()
```

```python
BrowserTools.open_password_settings()
```

```python
BrowserTools.open_bookmarks()
```

```python
BrowserTools.open_appearance_settings()
```

```python
BrowserTools.open_extensions()
```

```python
BrowserTools.delete_browsing_data()
```

```python
BrowserTools.bookmark_page()
```

```python
BrowserTools.open_search_engine_settings()
```

```python
BrowserTools.bring_back_last_tab()
```

```python
BrowserTools.open_profile_settings()
```

--------------------------------

### Inference with Transformers

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/README.md

Performs inference using the Transformers library. Loads the tokenizer and model, prepares a chat-formatted input, generates a response, and decodes it.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen-AgentWorld-35B-A3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {
        "role": "system",
        "content": "You are a language world model simulating a Linux terminal environment. "
                   "Given the user's command, predict the terminal output."
    },
    {
        "role": "user",
        "content": "Action: execute_bash
Command: ls -la /home/user/project/"
    }
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)
print(response)
```

--------------------------------

### VLCTools for Media Player Control

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/os/system_prompt.txt

Functions for controlling VLC media player, such as adding to playlists, setting preferences, and retrieving current settings. Useful for automating media playback tasks.

```python
VLCTools.add_to_playlist(...)
```

```python
VLCTools.set_settings(...)
```

```python
VLCTools.get_settings()
```

--------------------------------

### Time & Blocking Behavior Simulation: Long-Running Commands

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Simulates the behavior of long-running commands that may still be in progress or have completed within a longer time frame.

```Shell
10.0-60.0s
```

--------------------------------

### Special Action Types: Wait

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

A wait action, represented by empty keystrokes and a duration, shows incremental output or no change if the process is still running.

```string
Wait (`""` with duration)
Show incremental output or unchanged if still running
```

--------------------------------

### Field Definitions for Action Objects

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Defines the 'keystrokes' and 'duration' fields within each command object. 'keystrokes' is the raw terminal input, and 'duration' is the wait time before capturing output.

```json
{
  "keystrokes": "string",
  "duration": "number"
}
```

--------------------------------

### Core Responsibilities of Web World Model

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/web/system_prompt.txt

Lists the primary duties of the Web World Model.

```text
1. **State Prediction**: Generate the complete next page state after action execution
2. **Context Maintenance**: Track navigation state, focused elements, form values, and browsing history across turns
3. **Behavioral Fidelity**: Stay faithful to browser behavior, including navigation, loading completion, focus changes, and no-op actions
```

--------------------------------

### Web Search Input Formats

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/search/system_prompt.txt

Specifies the input formats for simulating a web search query, supporting both batch and single queries.

```json
{"queries": ["query1", "query2", ...]}}
```

```json
{"query": "single_query"}
```

--------------------------------

### Side-Effect Modeling: Working Directory Change

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Models the side effect of changing the current working directory.

```Shell
cd path
```

--------------------------------

### Keystrokes Semantics: Empty Action Array

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

An empty action array '[]' signals task completion within the evaluation environment.

```string
"keystrokes": "" 	→ types and executes
"keystrokes": "" 		→ types but does NOT execute
```

--------------------------------

### Web World Model Role

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/web/system_prompt.txt

Defines the core function of the Web World Model as a browser state simulator.

```text
You are a **Web World Model** — a precise browser state simulator. Your task is to predict the exact next page state after a browser action is executed.

Given:
1. **Historical Context** (Optional): Previous interactions in this session
2. **Current Page State**: The visible browser page state
3. **Browser Action**: A browser control action

Predict the **next page state** after the action is executed.
```

--------------------------------

### Special Action Types: Ctrl+C

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Interrupting a command with Ctrl+C ('C-c') results in displaying '^C', potentially a traceback, and returning to the prompt.

```string
Ctrl+C (`"C-c"`)
Show `^C`, possibly traceback, return to prompt
```

--------------------------------

### PyAutoGUI Mouse and Keyboard Actions

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/os/system_prompt.txt

Commonly used PyAutoGUI functions for simulating mouse clicks, movements, drags, scrolling, typing, and key presses. These are fundamental for automating GUI interactions.

```python
pyautogui.click(x: int, y: int, clicks: int = 1, button: Literal['left', 'middle', 'right'] = 'left')
```

```python
pyautogui.moveTo(x: int, y: int)
```

```python
pyautogui.dragTo(x: int, y: int, duration: float = ...)
```

```python
pyautogui.scroll(amount: int)
```

```python
pyautogui.write(text: str)
```

```python
pyautogui.press(key: str)
```

```python
pyautogui.hotkey(*keys: str)
```

--------------------------------

### Output Assembly: Command Output

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Represents the second part of assembled output: the standard output and standard error from command execution.

```Terminal
stdout and stderr from execution
```

--------------------------------

### Side-Effect Modeling: File Deletion

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Models the side effect of deleting a file.

```Shell
rm file
```

--------------------------------

### Special Action Types: Empty Array

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

An empty array '[]' as an action signifies task completion confirmation within the evaluation environment.

```string
Empty array (`[]`)
Evaluation-specific: task completion confirmation prompt
```

--------------------------------

### Keystrokes Semantics: Control Characters

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Supports tmux-style notation for common control characters like Ctrl+C, Ctrl+D, Ctrl+Z, and Ctrl+L.

```string
C-c 	→ Ctrl+C (SIGINT, interrupt)
C-d 	→ Ctrl+D (EOF, exit)
C-z 	→ Ctrl+Z (SIGTSTP, suspend)
C-l 	→ Ctrl+L (clear screen)
```

--------------------------------

### Output Formatting: Special Characters and Escapes

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Preserves special characters and escape sequences in the output.

```Terminal
Special characters and escape sequences
```

--------------------------------

### Keystrokes Semantics: Empty Keystrokes

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

An empty 'keystrokes' string combined with a 'duration' value indicates a wait period without any input.

```string
"keystrokes": "" with duration means wait without input
```

--------------------------------

### Side-Effect Modeling: Environment Variable Set

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/terminal/system_prompt.txt

Models the side effect of setting an environment variable.

```Shell
export VAR=value
```

--------------------------------

### JSON Handling in Python

Source: https://github.com/qwenlm/qwen-agentworld/blob/main/prompts/os/system_prompt.txt

Standard Python functions for loading JSON data from a file or string and for dumping Python objects into JSON format. Essential for data serialization and deserialization.

```python
json.load(...)
```

```python
json.dumps(...)
```