### Start Headless Server

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Starts the X server required for rendering in headless environments using the provided startx script.

```bash
python -m embodiedbench.envs.eb_alfred.scripts.startx 1
```

--------------------------------

### Cloud Instance Setup for ALFRED

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/README.md

Instructions for setting up ALFRED on headless machines like AWS or Google Cloud instances. This involves starting a tmux session, running an X server on a specified display, detaching from tmux, setting the DISPLAY environment variable, and checking the THOR simulation. Modifying `X_DISPLAY` in `gen/constants.py` might be necessary.

```bash
# start tmux session
tmux new -s startx 

# start X server on DISPLAY 0
# single X server should be sufficient for multiple instances of THOR
$ sudo python $ALFRED_ROOT/scripts/startx.py 0  # if this throws errors e.g "(EE) Server terminated with error (1)" or "(EE) already running ..." try a display > 0

# detach from tmux shell
# Ctrl+b then d

# set DISPLAY variable to match X server
$ export DISPLAY=:0

# check THOR
$ cd $ALFRED_ROOT
$ python scripts/check_thor.py
```

--------------------------------

### Install Planner Dependencies

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/gen/README.md

Installs the necessary system-level dependencies and compiles the Fast-Forward planner required for PDDL-based trajectory generation.

```bash
sudo apt-get install ffmpeg flex bison
cd $ALFRED_ROOT/gen/ff_planner
make
```

--------------------------------

### Initialize and Use EBNavigationEnv for Navigation Tasks

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Demonstrates how to initialize the EBNavigationEnv for discrete navigation tasks. It covers environment setup, available actions, resetting the environment, executing a step with reasoning, and accessing navigation metrics. Dependencies include the EBNavigationEnv class.

```python
from embodiedbench.envs.eb_navigation.EBNavEnv import EBNavigationEnv

# Initialize navigation environment
env = EBNavigationEnv(
    eval_set='base',           # Options: 'base', 'common_sense', 'complex_instruction',
                               # 'visual_appearance', 'long_horizon'
    exp_name='nav_experiment',
    down_sample_ratio=1.0,
    fov=100,                   # Field of view
    multiview=False,           # Enable top-down view
    boundingbox=False,         # Enable detection boxes
    multistep=False,           # Multi-step image history
    resolution=500
)

# Available discrete actions
# 0: Move forward 0.25m    4: Rotate right 90 degrees
# 1: Move backward 0.25m   5: Rotate left 90 degrees
# 2: Move right 0.25m      6: Tilt camera up 30 degrees
# 3: Move left 0.25m       7: Tilt camera down 30 degrees

# Reset environment
obs = env.reset()
print(f"Target instruction: {env.episode_language_instruction}")
print(f"Number of episodes: {env.number_of_episodes}")

# Execute navigation action
action = 0  # Move forward
reasoning = "Moving toward the target object"
obs, reward, done, info = env.step(action, reasoning, i_flag=1)

# Check navigation metrics
print(f"Distance to target: {info['distance']}")
print(f"Task success (reached target): {info['task_success']}")
print(f"Last action success: {info['last_action_success']}")

# Save image with optional bounding boxes
image_path = env.save_image()

env.close()
```

--------------------------------

### Verify Environment Installations

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Scripts to verify that the EB-ALFRED, EB-Habitat, and EB-Navigation environments are correctly installed and functional.

```bash
# Verify EB-ALFRED
conda activate embench
python -m embodiedbench.envs.eb_alfred.EBAlfEnv

# Verify EB-Habitat
conda activate embench
python -m embodiedbench.envs.eb_habitat.EBHabEnv

# Verify EB-Navigation
conda activate embench_nav
python -m embodiedbench.envs.eb_navigation.EBNavEnv
```

--------------------------------

### Docker Setup for ALFRED

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/README.md

Scripts for building and running Docker containers for the ALFRED environment. Modify `docker_build.py` and `docker_run.py` as needed. The 'Run (Local)' command activates the ALFRED environment and changes the directory. The 'Run (Headless)' commands are for VMs and cloud instances, requiring tmux, X server setup, and environment activation.

```bash
python scripts/docker_build.py
```

```bash
python scripts/docker_run.py
```

```bash
source ~/alfred_env/bin/activate
cd $ALFRED_ROOT
```

```bash
python scripts/docker_run.py --headless
```

```bash
# inside docker
tmux new -s startx  # start a new tmux session

# start nvidia-xconfig
sudo nvidia-xconfig -a --use-display-device=None --virtual=1280x1024

# start X server on DISPLAY 0
# single X server should be sufficient for multiple instances of THOR
sudo python ~/alfred/scripts/startx.py 0  # if this throws errors e.g "(EE) Server terminated with error (1)" or "(EE) already running ..." try a display > 0

# detach from tmux shell
# Ctrl+b then d

# source env
source ~/alfred_env/bin/activate

# set DISPLAY variable to match X server
export DISPLAY=:0

# check THOR
cd $ALFRED_ROOT
python scripts/check_thor.py
```

--------------------------------

### Initialize and Use VLMPlanner for Vision-Language Model Interaction

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Illustrates how to initialize and use the VLMPlanner for integrating Vision-Language Models (VLMs) with embodied environments. It covers defining actions, system prompts, few-shot examples, planner configuration (model, history, feedback), and obtaining actions based on observations and instructions. Dependencies include the VLMPlanner class and specified VLM models.

```python
from embodiedbench.planner.vlm_planner import VLMPlanner

# Define available actions for the environment
actions = [
    "find a mug",
    "pick up the mug",
    "find a table",
    "put down the object in hand"
]

# System prompt defining robot capabilities
system_prompt = """You are a robot operating in a home.
Given a task, accomplish it using the defined set of actions."""

# In-context examples for few-shot learning
examples = [
    "Task: Put a mug on the table\nPlan: find a mug -> pick up the mug -> find a table -> put down the object"
]

# Initialize planner
planner = VLMPlanner(
    model_name='gpt-4o-mini',  # or 'claude-3-5-sonnet-20241022', 'gemini-1.5-pro'
    model_type='remote',       # 'remote', 'local', or 'custom'
    actions=actions,
    system_prompt=system_prompt,
    examples=examples,
    n_shot=5,                  # Number of in-context examples
    obs_key='head_rgb',        # Observation key for images
    chat_history=False,        # Enable multi-turn conversation
    language_only=False,       # Text-only mode (no images)
    use_feedback=True,         # Include environment feedback
    multistep=0,               # Number of historical images
    tp=1                       # Tensor parallelism for local models
)

# Reset planner for new episode
planner.reset()

# Get action from observation and instruction
image_path = "path/to/observation.png"
instruction = "Put the mug on the kitchen table"
action, reasoning = planner.act(image_path, instruction)

```

--------------------------------

### Initialize and Use EBManEnv for Manipulation Tasks

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Shows how to initialize the EBManEnv for robotic arm manipulation tasks. It covers environment setup, task descriptions, the continuous action space format (position, orientation, gripper), executing a step, and checking manipulation results. Dependencies include the EBManEnv class.

```python
from embodiedbench.envs.eb_manipulation.EBManEnv import EBManEnv

# Initialize manipulation environment
env = EBManEnv(
    eval_set='base',           # Options: 'base', 'common_sense', 'complex',
                               # 'spatial', 'visual'
    render_mode='human',       # 'human' or 'rgb_array'
    img_size=(500, 500),
    down_sample_ratio=1.0,
    log_path=None,             # Custom log path
    selected_indexes=[]
)

# Reset and get task description
description, obs = env.reset()
print(f"Task: {description}")
# Example tasks: pick_cube_shape, stack_cubes_color,
#                place_into_shape_sorter_color, wipe_table_direction

# Action format: [X, Y, Z, Roll, Pitch, Yaw, Gripper]
# X, Y, Z: Position in voxel coordinates (0-100)
# Roll, Pitch, Yaw: Orientation (0-120, each unit = 3 degrees)
# Gripper: 0 = close, 1 = open
discrete_action = [50, 50, 30, 60, 60, 60, 1.0]  # Move to center, open gripper

obs, reward, terminate, info = env.step(discrete_action)

# Check manipulation results
print(f"Action success: {info['action_success']}")
print(f"Task success: {info['task_success']}")
print(f"Environment feedback: {info['env_feedback']}")

# Save multi-view images
image_paths = env.save_image(key=['front_rgb', 'wrist_rgb', 'overhead_rgb'])

env.close()
```

--------------------------------

### EmbodiedBench Main Execution with Hydra Configuration (Bash)

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

These bash commands demonstrate how to run EmbodiedBench evaluations using Hydra for configuration management. They show basic evaluation setups with different models (proprietary and open-source), advanced configuration options, and how to integrate with locally served models via LMDeploy.

```bash
python -m embodiedbench.main \
    env=eb-alf \
    model_name=gpt-4o \
    exp_name='gpt4o_baseline'
```

```bash
python -m embodiedbench.main \
    env=eb-hab \
    model_name=Qwen/Qwen2-VL-7B-Instruct \
    model_type=local \
    tp=1 \
    exp_name='qwen_local'
```

```bash
python -m embodiedbench.main \
    env=eb-nav \
    model_name=claude-3-5-sonnet-20241022 \
    down_sample_ratio=0.1 \
    language_only=False \
    chat_history=True \
    n_shots=5 \
    detection_box=True \
    resolution=500 \
    eval_sets=['base','common_sense'] \
    log_level=DEBUG \
    exp_name='detailed_eval'
```

```bash
# Model serving with LMDeploy (separate terminal)
lmdeploy serve api_server "OpenGVLab/InternVL2_5-8B" --server-port 8000 --tp 1

# Then run evaluation
export remote_url='http://localhost:8000/v1'
python -m embodiedbench.main \
    env=eb-hab \
    model_name=OpenGVLab/InternVL2_5-8B \
    exp_name='intern_served'
```

--------------------------------

### Define PDDL Goal

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/gen/README.md

Example of a PDDL goal definition for a pick-and-place task, specifying conditions for object placement and receptacle states.

```python
gdict["pick_and_place_simple"] = '''
    (:goal
        (and
            ;; make sure all the cabinets and doors are closed in the end
            (forall (?re # receptacle)
                (not (opened ?re))
            )
            
            ;; make sure some object {obj} exists inside some receptacle {recep}
            (exists (?r # receptacle)
                (exists (?o # object)
                    (and 
                        (inReceptacle ?o ?r) 
                        (objectType ?o {obj}Type) 
                        (receptacleType ?r {recep}Type)
                    )
                )
            )
        )
    )
)'''
```

--------------------------------

### Download ALFRED dataset using shell script

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/data/README.md

Commands to download different versions of the ALFRED dataset using the provided download_data.sh script. Users can choose between Lite, Modeling Quickstart, or Full datasets based on their storage and feature requirements.

```bash
# Download Lite version
cd $ALFRED_ROOT/data
sh download_data.sh json

# Download Modeling Quickstart version
sh download_data.sh json_feat

# Download Full dataset
sh download_data.sh full
```

--------------------------------

### Define Custom Task Class in EmbodiedBench

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_manipulation/tools/assets/task_template.txt

This snippet provides the boilerplate structure for creating a new task class. It includes methods for task initialization, episode setup, variation counting, and simulation stepping.

```python
from typing import List
from amsolver.backend.unit_tasks import Task

class CustomTask(Task):
    def init_task(self) -> None:
        pass

    def init_episode(self, index: int) -> List[str]:
        return ['']

    def variation_count(self) -> int:
        return 1

    def step(self) -> None:
        pass
```

--------------------------------

### RemoteModel API Wrapper for LLM Interaction (Python)

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

The RemoteModel class acts as a wrapper for interacting with various LLMs, including GPT-4, Claude, and Gemini. It handles API key setup, model initialization for different types (remote/local), and prepares messages including image data for multimodal models. The `respond` method sends the request and retrieves the model's output.

```python
from embodiedbench.planner.remote_model import RemoteModel
import os
import base64

os.environ['OPENAI_API_KEY'] = 'your-openai-key'
os.environ['ANTHROPIC_API_KEY'] = 'your-anthropic-key'
os.environ['GEMINI_API_KEY'] = 'your-gemini-key'

model_gpt = RemoteModel(
    model_name='gpt-4o',
    model_type='remote',
    language_only=False,
    task_type=None
)

model_claude = RemoteModel(
    model_name='claude-3-5-sonnet-20241022',
    model_type='remote',
    language_only=False
)

model_local = RemoteModel(
    model_name='Qwen/Qwen2-VL-7B-Instruct',
    model_type='local',
    language_only=False,
    tp=1
)

def encode_image(image_path):
    with open(image_path, "rb") as f:
        return base64.b64encode(f.read()).decode('utf-8')

base64_image = encode_image("observation.png")
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image_url",
                "image_url": {"url": f"data:image/png;base64,{base64_image}"}
            },
            {
                "type": "text",
                "text": "Describe this scene and plan actions to complete the task."
            }
        ]
    }
]

response = model_gpt.respond(messages)
print(f"Model output: {response}")
```

--------------------------------

### EB_AlfredEvaluator for Evaluation Pipeline Orchestration (Python)

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

The EB_AlfredEvaluator class manages the end-to-end evaluation process for EmbodiedBench tasks. It takes a configuration dictionary specifying model details, task parameters, and evaluation settings. The evaluator orchestrates environment setup, model calls, metric collection, and result aggregation, saving detailed logs and summary statistics.

```python
from embodiedbench.evaluator.eb_alfred_evaluator import EB_AlfredEvaluator

config = {
    'model_name': 'gpt-4o-mini',
    'model_type': 'remote',
    'down_sample_ratio': 1.0,
    'language_only': False,
    'n_shots': 10,
    'exp_name': 'baseline',
    'chat_history': False,
    'detection_box': False,
    'eval_sets': ['base'],
    'selected_indexes': [],
    'multistep': 0,
    'resolution': 500,
    'env_feedback': True,
    'tp': 1
}

evaluator = EB_AlfredEvaluator(config)
evaluator.check_config_valid()
evaluator.evaluate_main()
```

--------------------------------

### Initialize and Use EBHabEnv for Habitat Simulation

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

This Python code shows how to initialize and use the EBHabEnv, a wrapper for the Habitat framework for household rearrangement tasks. It covers environment reset, executing actions by index, checking task completion, and saving observation images.

```python
from embodiedbench.envs.eb_habitat.EBHabEnv import EBHabEnv

# Initialize Habitat environment
env = EBHabEnv(
    eval_set='long_horizon',   # Options: 'base', 'common_sense', 'complex_instruction',
                               # 'spatial_relationship', 'visual_appearance', 'long_horizon'
    exp_name='habitat_test',
    down_sample_ratio=1.0,
    start_epi_index=0,         # Starting episode index
    resolution=500,
    recording=False            # Enable video recording
)

# Reset and get initial observation
obs = env.reset()
print(f"Language goal: {env.episode_language_instruction}")
print(f"Action space: {env.language_skill_set}")
# Example actions: 'navigate to the table 1', 'pick up the apple',
#                  'place at the counter', 'open the refrigerator'

# Execute discrete action by index
action_idx = 0  # First action in skill set
obs, reward, done, info = env.step(action_idx, reasoning='Navigating to target')

# Check task completion
print(f"Predicate task success: {info['predicate_task_success']}")
print(f"Task progress: {info['task_progress']}")
print(f"Was action invalid: {info['was_prev_action_invalid']}")

# Save observation images
image_path = env.save_image(obs, key='head_rgb')

env.close()
```

--------------------------------

### Clone and Initialize Repository

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Commands to clone the EmbodiedBench repository and initialize Git LFS for handling large dataset files.

```bash
git clone git@github.com:EmbodiedBench/EmbodiedBench.git
cd EmbodiedBench
git lfs install
git lfs pull
```

--------------------------------

### Initialize and Use EBAlfEnv for Household Tasks

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

This Python code demonstrates how to initialize and interact with the EBAlfEnv, a Gym-style wrapper for the ALFRED environment. It covers resetting the environment, executing actions like 'find a mug', accessing task success and progress information, and saving logs.

```python
from embodiedbench.envs.eb_alfred.EBAlfEnv import EBAlfEnv

# Initialize environment with evaluation set
env = EBAlfEnv(
    eval_set='base',           # Options: 'base', 'common_sense', 'complex_instruction',
                               # 'spatial', 'visual_appearance', 'long_horizon'
    exp_name='my_experiment',
    down_sample_ratio=1.0,     # Use 0.1 for debugging (10% of dataset)
    selected_indexes=[],       # Specific episode indices to evaluate
    detection_box=False,       # Enable bounding box annotations
    resolution=500             # Image resolution
)

# Reset environment for new episode
obs = env.reset()
print(f"Instruction: {env.episode_language_instruction}")
print(f"Available actions: {len(env.language_skill_set)}")

# Execute actions in the environment
action = "find a mug"  # Can be string or action index
obs, reward, done, info = env.step(action, reasoning='Looking for the mug')

# Access environment information
print(f"Task success: {info['task_success']}")
print(f"Task progress: {info['task_progress']}")
print(f"Environment feedback: {info['env_feedback']}")

# Save observations and logs
image_path = env.save_image()
env.save_episode_log()

# Clean up
env.close()
```

--------------------------------

### Update Planner with Environment Feedback (Python)

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

This snippet demonstrates how to update a planner with feedback from the environment after an action has been executed. It formats the action and feedback into a dictionary and passes it to the planner's update_info method. This is crucial for iterative planning and learning.

```python
info = {
    'action_id': action[0] if isinstance(action, list) else action,
    'env_feedback': 'Action executed successfully'
}
planner.update_info(info)
```

--------------------------------

### Deploy Open-Source Models via Offline Execution

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Instructions for running models locally by setting the model type and tensor parallelism (tp) based on available GPU memory.

```bash
conda activate embench
python -m embodiedbench.main env=eb-alf model_name=Qwen/Qwen2-VL-7B-Instruct model_type=local exp_name='baseline' tp=1
python -m embodiedbench.main env=eb-hab model_name=OpenGVLab/InternVL2_5-8B model_type=local exp_name='baseline' tp=1
conda activate embench_nav
python -m embodiedbench.main env=eb-nav model_name=OpenGVLab/InternVL2_5-38B model_type=local exp_name='baseline' tp=4
conda activate embench_man 
python -m embodiedbench.main env=eb-man model_name=meta-llama/Llama-3.2-11B-Vision-Instruct model_type=local exp_name='baseline' tp=2
```

--------------------------------

### Run EmbodiedBench Evaluations using Hydra

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

This snippet shows how to run evaluations for EmbodiedBench using the Hydra configuration system. It demonstrates launching evaluations for different environments (EB-ALFRED, EB-Habitat, EB-Navigation, EB-Manipulation) with various models and experiment names.

```python
python -m embodiedbench.main env=eb-alf model_name=gpt-4o-mini exp_name='baseline'

# Available environments: eb-alf, eb-hab, eb-nav, eb-man
python -m embodiedbench.main env=eb-hab model_name=gpt-4o exp_name='habitat_test'
python -m embodiedbench.main env=eb-nav model_name=claude-3-5-sonnet-20241022 exp_name='nav_test'
python -m embodiedbench.main env=eb-man model_name=gemini-1.5-pro exp_name='manip_test'
```

--------------------------------

### Deploy Models via Online Serving with LMDeploy

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Standard approach for decoupling model execution from evaluation using LMDeploy to serve models via API.

```bash
conda env create -f conda_envs/lmdeploy.yaml
conda activate lmdeploy
pip install lmdeploy
lmdeploy serve api_server "OpenGVLab/InternVL2_5-8B" --server-port $port --tp 1
conda activate embench
export remote_url='IP_address:port/v1'
python -m embodiedbench.main env=eb-hab model_name=OpenGVLab/InternVL2_5-8B exp_name='baseline'
```

--------------------------------

### Deploy Unsupported Models via Custom Server

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

A flexible approach for deploying newer or unsupported models by running a custom Flask server and pointing the evaluation script to it.

```bash
pip install flask
CUDA_VISIBLE_DEVICES=${gpu_ids} python server.py
export server_url="IP_address:port/process"
python -m embodiedbench.main env=eb-hab model_name='microsoft/Phi-4-multimodal-instruct' model_type='custom' exp_name='new_model'
```

--------------------------------

### EBNavigationEnv

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Interface for the navigation environment using AI2THOR, providing discrete movement and rotation actions.

```APIDOC
## EBNavigationEnv

### Description
Provides low-level navigation control using AI2THOR's ManipulaTHOR with discrete movement and rotation actions.

### Method
N/A (Python Class Interface)

### Parameters
- **eval_set** (string) - Required - Options: 'base', 'common_sense', 'complex_instruction', 'visual_appearance', 'long_horizon'
- **fov** (int) - Optional - Field of view (default: 100)
- **resolution** (int) - Optional - Image resolution (default: 500)

### Request Example
env = EBNavigationEnv(eval_set='base', fov=100)

### Response
- **obs** (array) - Observation data
- **reward** (float) - Step reward
- **done** (bool) - Episode termination status
- **info** (dict) - Metadata including 'distance' and 'task_success'
```

--------------------------------

### VLMPlanner

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Interface for VLM-based planning, managing prompt construction and action parsing for embodied agents.

```APIDOC
## VLMPlanner

### Description
Handles interaction between Vision-Language Models and environments for multi-turn reasoning.

### Method
N/A (Python Class Interface)

### Parameters
- **model_name** (string) - Required - e.g., 'gpt-4o-mini'
- **actions** (list) - Required - List of valid action strings
- **system_prompt** (string) - Required - Robot capability definition

### Request Example
planner = VLMPlanner(model_name='gpt-4o-mini', actions=actions)
action, reasoning = planner.act(image_path, instruction)

### Response
- **action** (string) - Selected action from defined list
- **reasoning** (string) - Rationale for the chosen action
```

--------------------------------

### EBManEnv

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Interface for the manipulation environment using CoppeliaSim, supporting continuous 7D action space.

```APIDOC
## EBManEnv

### Description
Provides robotic arm manipulation tasks with continuous gripper control.

### Method
N/A (Python Class Interface)

### Parameters
- **eval_set** (string) - Required - Options: 'base', 'common_sense', 'complex', 'spatial', 'visual'
- **render_mode** (string) - Optional - 'human' or 'rgb_array'

### Request Example
# Action: [X, Y, Z, Roll, Pitch, Yaw, Gripper]
action = [50, 50, 30, 60, 60, 60, 1.0]
obs, reward, terminate, info = env.step(action)

### Response
- **info** (dict) - Contains 'action_success', 'task_success', and 'env_feedback'
```

--------------------------------

### Create EmbodiedBench Conda Environments

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Commands to create the three distinct conda environments required for Habitat/Alfred, Navigation, and Manipulation tasks.

```bash
# Environment for Habitat and Alfred
conda env create -f conda_envs/environment.yaml
conda activate embench
pip install -e .

# Environment for EB-Navigation
conda env create -f conda_envs/environment_eb-nav.yaml
conda activate embench_nav
pip install -e .

# Environment for EB-Manipulation
conda env create -f conda_envs/environment_eb-man.yaml
conda activate embench_man
pip install -e .
```

--------------------------------

### Generate Trajectories

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/gen/README.md

Spawns multiple threads to sample tasks and generate expert trajectories. The script automatically handles failed executions and discards them.

```bash
cd $ALFRED_ROOT/gen
python scripts/generate_trajectories.py --save_path data/new_trajs --in_parallel --debug --num_threads 2
```

--------------------------------

### Run ALFRED Leaderboard Evaluation

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/README.md

This script evaluates a trained model on ALFRED's test sets and generates a JSON output file for submission to the leaderboard. It requires the model path, model type, and data path as input. Ensure you adhere to the specified input constraints (RGB and language instructions only) and evaluation limits (max_steps, max_fails).

```bash
cd $ALFRED_ROOT
python models/eval/leaderboard.py --model_path <model_path>/model.pth --model models.model.seq2seq_im_mask --data data/json_feat_2.1.0 --gpu --num_threads 5
```

--------------------------------

### Configure Chat History Truncation for EB-Navigation

Source: https://github.com/embodiedbench/embodiedbench/blob/master/README.md

Demonstrates how to enable or disable chat history truncation in navigation tasks to manage context window limits and improve model performance.

```bash
conda activate embench_nav
python -m embodiedbench.main env=eb-nav model_name=gpt-4o chat_history=True truncate=True exp_name='nav_with_truncation'
python -m embodiedbench.main env=eb-nav model_name=gpt-4o chat_history=True truncate=False exp_name='nav_standard_history'
python -m embodiedbench.main env=eb-nav model_name=gpt-4o chat_history=False exp_name='nav_no_history'
```

--------------------------------

### Extract ResNet Features using Python Script

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/data/README.md

This command-line script extracts ResNet features from raw image sequences within the EmbodiedBench dataset. It allows specifying data paths, batch size, GPU usage, the visual model, and the output filename. The extracted features are saved in .pt files.

```bash
python models/utils/extract_resnet.py --data data/full_2.1.0 --batch 32 --gpu --visual_model resnet18 --filename feat_conv.pt
```

--------------------------------

### Perform Replay Checks

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/gen/README.md

Validates that saved trajectories are reproducible by re-executing interaction masks and expert actions within the THOR simulator.

```bash
python scripts/replay_checks.py --data_path data/new_trajs --in_parallel
```

--------------------------------

### Result Summarization Utility (Python)

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

This snippet indicates the availability of a utility function for summarizing evaluation results. The `average_json_values` function is designed to aggregate metrics from multiple evaluation runs, providing a consolidated overview of performance.

```python
from embodiedbench.evaluator.summarize_result import average_json_values
```

--------------------------------

### Aggregate Evaluation Results

Source: https://context7.com/embodiedbench/embodiedbench/llms.txt

Aggregates JSON results from an evaluation directory into a summary file. It can select all metrics or specific ones like 'task_success'. This function is useful for consolidating evaluation data from multiple runs.

```python
average_json_values(
    json_dir='running/eb_alfred/gpt-4o-mini_baseline/base/results',
    target_file='*_final_res.json',
    output_file='summary.json',
    selected_key=None  # None for all metrics, or specific key like 'task_success'
)
```

--------------------------------

### Augment Trajectories

Source: https://github.com/embodiedbench/embodiedbench/blob/master/embodiedbench/envs/eb_alfred/gen/README.md

Processes existing datasets to extract additional sensor information such as higher resolution RGB images, depth maps, and instance segmentation masks.

```bash
python scripts/augment_trajectories.py --data_path data/json_2.1.0 --num_threads 2 --smooth_nav --time_delays
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.