### Setup MuJoCo Export Environment

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Prepare the environment for exporting scenes to MuJoCo and USD formats by running the setup script. This installs necessary dependencies in a dedicated virtual environment.

```sh
./scripts/setup_mujoco_export.sh
```

--------------------------------

### Install SAM3D Backend

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Run this script to install the SAM3D backend, which provides higher-quality asset generation but requires 32GB GPU memory. This installation includes cloning repositories, installing dependencies, and downloading model checkpoints.

```sh
bash scripts/install_sam3d.sh
```

--------------------------------

### Install Hunyuan3D-2 Backend

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Execute this script to install the Hunyuan3D-2 backend. This backend offers lower-quality asset generation but fits within 24GB GPU memory.

```sh
bash scripts/install_hunyuan3d.sh
```

--------------------------------

### Install uv dependency manager

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Installs the uv dependency manager using a script. Ensure uv is installed before proceeding with project dependency management.

```sh
curl -LsSf https://astral.sh/uv/install.sh | sh
```

--------------------------------

### Install pre-commit hooks

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Installs the pre-commit framework hooks for the project. These hooks run checks on code before commits to ensure code quality and consistency.

```sh
pre-commit install
```

--------------------------------

### Install Hunyuan3D-2 Submodules

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Run this command to initialize and update submodules required for Hunyuan3D-2.

```sh
git submodule update --init --recursive
```

--------------------------------

### Download AmbientCG Materials with Options

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

These examples demonstrate downloading AmbientCG materials with specific options, such as resolution, format, limiting the number of downloads, or performing a dry run.

```python
# Download specific resolution/format
python scripts/download_ambientcg.py -r 2K -f PNG --output data/materials

# Limit number of materials (for testing)
python scripts/download_ambientcg.py --limit 100 --output data/materials

# Dry run to see what would be downloaded
python scripts/download_ambientcg.py --dry-run
```

--------------------------------

### Install project dependencies with uv

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Installs all project dependencies, including development tools like pytest, into a virtual environment named .venv. Use --no-dev to exclude development dependencies.

```sh
uv sync
```

```sh
uv sync --no-dev
```

--------------------------------

### Install bubblewrap for multi-GPU rendering

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Installs bubblewrap on Ubuntu/Debian systems, which is required for distributing Blender rendering across multiple GPUs. This helps prevent Out-of-Memory errors during parallel scene generation.

```sh
# Ubuntu/Debian
sudo apt-get install bubblewrap
```

--------------------------------

### Resume Scene Generation From Later Stage

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Start the pipeline from a specified stage, loading previously saved state. This accelerates iteration on later stages of generation.

```sh
python main.py +name=my_experiment experiment.pipeline.start_stage=manipuland
```

--------------------------------

### Download SAM3D Checkpoints Manually

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Manually download SAM3D model weights if not using the install script. These checkpoints are required and mounted at runtime.

```sh
mkdir -p external/checkpoints
huggingface-cli download facebook/sam3 sam3.pt --local-dir external/checkpoints
huggingface-cli download facebook/sam-3d-objects \
    --repo-type model \
    --local-dir external/checkpoints/sam-3d-objects-download \
    --include "checkpoints/*"
mv external/checkpoints/sam-3d-objects-download/checkpoints/* external/checkpoints/
rm -rf external/checkpoints/sam-3d-objects-download
```

--------------------------------

### Download CoACD Variant Assets

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use this command to download the CoACD variant of ArtVIP assets, which can produce faster simulations. Ensure you have huggingface-cli installed.

```sh
huggingface-cli download nepfaff/scenesmith-preprocessed-data \
    artvip/artvip_coacd.tar.gz --repo-type dataset --local-dir .
mkdir -p data/artvip_sdf
tar xzf artvip/artvip_coacd.tar.gz -C data/artvip_sdf
rm -rf artvip
```

--------------------------------

### Run SceneSmith Services with Docker Compose

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Runs SceneSmith services using Docker Compose. This command starts an interactive shell within the container with all volumes and environment variables mounted from the host. Data directories must be mounted as volumes.

```bash
OPENAI_API_KEY="sk-..." docker compose run --rm scenesmith bash
```

--------------------------------

### Run SceneSmith Interactively in Docker

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Start an interactive bash shell within the SceneSmith Docker container, with all necessary volumes and environment variables mounted. This allows for running commands like 'python main.py'.

```sh
# Interactive shell with all volumes and env vars
docker compose run --rm scenesmith bash

# Then inside the container:
python main.py +name=my_experiment
```

--------------------------------

### Download VHACD Variant Assets

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use this command to download the VHACD variant of ArtVIP assets, which provides tighter collision geometries. Ensure you have huggingface-cli installed.

```sh
huggingface-cli download nepfaff/scenesmith-preprocessed-data \
    artvip/artvip_vhacd.tar.gz --repo-type dataset --local-dir .
mkdir -p data/artvip_sdf
tar xzf artvip/artvip_vhacd.tar.gz -C data/artvip_sdf
rm -rf artvip
```

--------------------------------

### Run Pytest Integration Tests

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Executes integration tests using pytest. These tests require a full GPU setup, Blender, and data. Options include running with testmon or running all tests without caching.

```bash
pytest tests/integration/ --testmon
```

```bash
pytest tests/
```

```bash
pytest tests/integration/test_mesh_processing_pipeline.py -v
```

--------------------------------

### Clone HSSD Models

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Clone the HSSD object library from Hugging Face. Ensure Git LFS is installed and you have accepted the HSSD license.

```sh
cd data
git lfs install
git clone git@hf.co:datasets/hssd/hssd-models
```

--------------------------------

### Initialize IndoorSceneGenerationExperiment

Source: https://context7.com/nepfaff/scenesmith/llms.txt

This Python code demonstrates how to initialize and run the `IndoorSceneGenerationExperiment` using Hydra. It registers resolvers, resolves configuration, builds the experiment, and executes defined tasks.

```python
from omegaconf import OmegaConf
from scenesmith.experiments.indoor_scene_generation import IndoorSceneGenerationExperiment

# Build via Hydra (standard usage in main.py)
import hydra
from omegaconf import DictConfig

@hydra.main(version_base=None, config_path="configurations", config_name="config")
def run(cfg: DictConfig):
    from scenesmith.experiments import build_experiment
    from scenesmith.utils.omegaconf import register_resolvers

    register_resolvers()
    OmegaConf.resolve(cfg)

    experiment = build_experiment(cfg=cfg)

    # Executes all tasks listed in experiment.tasks (default: ["generate_scenes"])
    for task in cfg.experiment.tasks:
        experiment.exec_task(task)

```

--------------------------------

### Run Full Scene Generation

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Execute the main script to generate a full scene experiment. This is the default behavior.

```sh
python main.py +name=my_experiment
```

--------------------------------

### Load Configuration and Generate Scenes

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Load experiment configuration using OmegaConf and initiate scene generation.

```python
cfg = OmegaConf.load("configurations/experiment/indoor_scene_generation.yaml")
experiment = IndoorSceneGenerationExperiment(cfg=cfg)
experiment.generate_scenes()
```

--------------------------------

### Initialize and Manage RoomScene

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Create a RoomScene instance with geometry and directory information. Supports adding, querying, moving, and removing objects. Use for managing the state of a single room.

```python
from pathlib import Path
from scenesmith.agent_utils.room import RoomScene, SceneObject, ObjectType, UniqueID
from pydrake.all import RigidTransform, RollPitchYaw
import numpy as np

# Create a room scene with pre-built room geometry
scene = RoomScene(
    room_geometry=room_geometry,   # RoomGeometry from floor plan agent
    scene_dir=Path("outputs/scene_000/room_main"),
    room_id="main",
    text_description="A cozy bedroom with hardwood floors.",
    action_log_path=Path("outputs/scene_000/room_main/action_log.json"),
)

# Add a wall (immutable room boundary object)
scene.add_object(wall_object)

# Query objects
furniture = scene.get_objects_by_type(ObjectType.FURNITURE)
manipulands = scene.get_manipulands()
objects_on_table = scene.get_objects_on_surface(surface_id=UniqueID("S_3"))

# Move an object to a new world-frame pose
new_pose = RigidTransform(rpy=RollPitchYaw(0, 0, 1.57), p=[1.5, 2.0, 0.0])
scene.move_object(object_id=UniqueID("chair_0"), new_transform=new_pose)

# Remove object
scene.remove_object(UniqueID("chair_0"))  # Returns True if removed
```

--------------------------------

### View Scene in MuJoCo Viewer

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Launch the MuJoCo viewer to visualize an exported scene. Ensure the scene XML file is correctly referenced.

```bash
python -m mujoco.viewer --mjcf=mujoco_export/scene.xml
```

--------------------------------

### Initialize Geometry Generation Server Programmatically

Source: https://github.com/nepfaff/scenesmith/blob/main/scenesmith/agent_utils/geometry_generation_server/README.md

Integrates the geometry generation server into applications. Instantiate the server with host, port, and optional backend configurations.

```python
from scenesmith.agent_utils.geometry_generation_server import GeometryGenerationServer

# Hunyuan3D backend
server = GeometryGenerationServer(host="127.0.0.1", port=7000)

# SAM3D backend
server = GeometryGenerationServer(
    host="127.0.0.1",
    port=7000,
    backend="sam3d",
    sam3d_config={
        "sam3_checkpoint": "external/checkpoints/sam3.pt",
        "sam3d_checkpoint": "external/checkpoints/pipeline.yaml",
    },
)

server.start()
server.wait_until_ready()
# ... use server via GeometryGenerationClient ...
server.stop()
```

--------------------------------

### Visualize Scene State with Drake Model Visualizer

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Use Drake's interactive 3D viewer to inspect scene states at various pipeline stages. Scenes use 'package://' URIs for portability. Ensure ROS_PACKAGE_PATH is set correctly for multi-room scenes.

```bash
# View intermediate state (e.g., after furniture stage)
python -m pydrake.visualization.model_visualizer \
    outputs/2025-12-21/10-30-45/scene_000/room_main/scene_renders/furniture/renders_001/scene.dmd.yaml
```

```bash
# View final combined house scene (multi-room)
export ROS_PACKAGE_PATH=/path/to/outputs/2025-12-21/10-30-45/scene_000:$ROS_PACKAGE_PATH
python -m pydrake.visualization.model_visualizer \
    outputs/2025-12-21/10-30-45/scene_000/combined_house/house.dmd.yaml
```

--------------------------------

### Run Geometry Generation Server Standalone

Source: https://github.com/nepfaff/scenesmith/blob/main/scenesmith/agent_utils/geometry_generation_server/README.md

Launches the geometry generation server in standalone mode for testing or microservice deployment. Specify the backend and checkpoint paths if using SAM3D.

```bash
python -m scenesmith.agent_utils.geometry_generation_server.standalone_server

python -m scenesmith.agent_utils.geometry_generation_server.standalone_server \
  --backend sam3d \
  --sam3-checkpoint external/checkpoints/sam3.pt \
  --sam3d-checkpoint external/checkpoints/pipeline.yaml
```

--------------------------------

### Export Scene to USD Format

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Export a scene to USD format, which will create a 'usd/' subdirectory within the specified output directory. This is useful for integration with USD-compatible tools.

```bash
python scripts/export_scene_to_mujoco.py --sdf /path/to/model.sdf -o mujoco_export --usd
```

--------------------------------

### Download ArtVIP Assets

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Download preprocessed ArtVIP assets, which include SDFormat conversions, collision geometries, and CLIP embeddings, available on HuggingFace.

```sh
cd data
git lfs install
git clone git@hf.co:datasets/artvip/artvip-assets
```

--------------------------------

### Download AmbientCG PBR Materials

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use this script to download free CC0 PBR materials from AmbientCG. The output directory can be specified.

```python
python scripts/download_ambientcg.py --output data/materials
```

--------------------------------

### View and Simulate MuJoCo Scenes

Source: https://context7.com/nepfaff/scenesmith/llms.txt

View exported scenes using MuJoCo's interactive viewer or load them programmatically in Python to run simulations. Requires the scene to be exported first.

```bash
# View in MuJoCo interactive viewer
python -m mujoco.viewer --mjcf=/tmp/mujoco_scene/scene.xml
```

```python
# Load programmatically and run simulation
import mujoco
model = mujoco.MjModel.from_xml_path("/tmp/mujoco_scene/scene.xml")
data = mujoco.MjData(model)
for _ in range(1000):
    mujoco.mj_step(model, data)
print(f"Simulation complete: {model.nbody} bodies, {model.njnt} joints")
```

--------------------------------

### Convert PartNet-Mobility in Parallel

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use a wrapper script for parallel conversion of PartNet-Mobility data to SDF format, specifying the number of parallel jobs.

```sh
bash scripts/convert_partnet_parallel.sh data/partnet-mobility-v0 data/partnet_mobility_sdf 8
```

--------------------------------

### Generate Scenes using Prompts

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Run the standard scene generation pipeline using prompts generated by `generate_prompts.py`. This command integrates with the SceneSmith generation process.

```bash
python main.py +name=eval experiment.csv_path=outputs/eval_run/prompts.csv
```

--------------------------------

### Run Scene Generation

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Execute the main scene generation script. Specify a run name using the `+name` argument. Scene prompts and floor plan modes can be configured in the associated YAML files.

```sh
python main.py +name=run_name
```

--------------------------------

### Run Scene Generation with SceneSmith

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Use the main.py script to run scene generation. Specify a run name and optionally override experiment parameters like prompts or number of workers. Supports parallel generation and GPU restriction.

```bash
python main.py +name=my_bedroom_run
```

```bash
python main.py +name=living_room \
    "experiment.prompts=['A modern living room with a sectional sofa and coffee table.']"
```

```bash
python main.py +name=batch_run \
    experiment.csv_path=outputs/eval_run/prompts.csv
```

```bash
python main.py +name=parallel_run experiment.num_workers=4
```

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 python main.py +name=multi_gpu_run experiment.num_workers=4
```

```bash
LOGLEVEL=DEBUG python main.py +name=debug_run
```

--------------------------------

### Export Scene to MuJoCo and USD

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Export scenes from a directory to MuJoCo's MJCF format or USD. Options include welding furniture, excluding floor/walls, and exporting single SDF models. Ensure the MuJoCo venv is activated first.

```bash
# Activate the dedicated MuJoCo venv first (avoids bpy/pxr conflicts)
./scripts/setup_mujoco_export.sh
source .mujoco_venv/bin/activate
```

```bash
# Export a full scene directory
python scripts/export_scene_to_mujoco.py \
    outputs/2025-12-21/10-30-45/scene_000 \
    --output /tmp/mujoco_scene
```

```bash
# Export with furniture having freejoints (default) vs. static
python scripts/export_scene_to_mujoco.py \
    outputs/2025-12-21/10-30-45/scene_000 \
    --weld-furniture    # make furniture static
```

```bash
# Export without floor/walls (manipulands only)
python scripts/export_scene_to_mujoco.py \
    outputs/2025-12-21/10-30-45/scene_000 \
    --no-floor-plan
```

```bash
# Also export to USD (for Isaac Sim)
python scripts/export_scene_to_mujoco.py \
    outputs/2025-12-21/10-30-45/scene_000 \
    --output /tmp/mujoco_scene \
    --usd
```

```bash
# Export a single SDF model for inspection (e.g., verify articulated joints)
python scripts/export_scene_to_mujoco.py \
    --sdf data/artvip_sdf/cabinet_001/model.sdf \
    --output /tmp/cabinet_test
```

--------------------------------

### Authenticate with HuggingFace CLI

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Log in to your HuggingFace account using the CLI. This is required for downloading model checkpoints for SAM3D.

```sh
huggingface-cli login
```

--------------------------------

### Download Pre-computed AmbientCG Embeddings

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Download pre-computed CLIP embeddings for AmbientCG materials using huggingface-cli. This is recommended for faster retrieval.

```sh
huggingface-cli download nepfaff/scenesmith-preprocessed-data \
    --repo-type dataset \
    --include "ambientcg/embeddings/**" \
    --local-dir data/scenesmith-preprocessed-data
mv data/scenesmith-preprocessed-data/ambientcg/embeddings data/materials/embeddings
rm -rf data/scenesmith-preprocessed-data
```

--------------------------------

### Visualize Portable House Scene

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

View portable house scenes using Drake's model visualizer by setting the ROS_PACKAGE_PATH environment variable. This allows scenes to be moved or shared without breaking file paths.

```sh
# View a portable house scene
export ROS_PACKAGE_PATH=/path/to/outputs/YYYY-MM-DD/HH-MM-SS/scene_000:$ROS_PACKAGE_PATH
python -m pydrake.visualization.model_visualizer \
    outputs/YYYY-MM-DD/HH-MM-SS/scene_000/combined_house_after_furniture/house.dmd.yaml
```

--------------------------------

### Test Asset Retrieval

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Run this script to test the asset retrieval functionality. Provide the source, a query string, the number of top results to retrieve, and an output path for inspection.

```python
python scripts/test_asset_retrieval.py \
    --source partnet_mobility \
    --query "wooden cabinet with drawers" \
    --top-k 5 \
    --output-path output/retrieval_test
```

--------------------------------

### Generate Scene Prompts for Robot Evaluation

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use this script to convert a human task description into diverse scene prompts. The output includes scene prompts and task metadata, which are essential for the robot evaluation pipeline.

```bash
python scripts/robot_eval/generate_prompts.py \
    --task "Find a fruit and place it on the kitchen table" \
    --output-dir outputs/eval_run \
    --num-prompts 5
```

--------------------------------

### Build SceneSmith Docker Image

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Builds the Docker image for SceneSmith. This image includes all necessary dependencies for GPU-dependent services.

```bash
docker build -t scenesmith .
```

--------------------------------

### Robot Evaluation Pipeline: Generate Prompts

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Stage 1 of the robot evaluation pipeline. Converts a task description into scene prompts using an LLM. Specify the task, output directory, and number of prompts.

```bash
# Stage 1: Convert task → scene prompts
python scripts/robot_eval/generate_prompts.py \
    --task "Find a fruit and place it on the kitchen table" \
    --output-dir outputs/eval_run \
    --num-prompts 10
# Outputs: outputs/eval_run/prompts.csv, outputs/eval_run/task_metadata.yaml
```

--------------------------------

### Compute CLIP Embeddings for Materials and Articulated Objects

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Scripts to compute CLIP embeddings for retrieval. Supports AmbientCG materials, ArtVIP articulated furniture, and PartNet-Mobility datasets. The ArtVIP script includes an option to save renders for visual inspection.

```bash
python scripts/compute_ambientcg_embeddings.py --materials-dir data/materials
```

```bash
python scripts/compute_articulated_embeddings.py \
    --source artvip \
    --data-path data/artvip_sdf \
    --output-path data/artvip_sdf/embeddings \
    --keep-renders  # save renders for visual inspection
```

```bash
python scripts/compute_articulated_embeddings.py \
    --source partnet_mobility \
    --data-path data/partnet_mobility_sdf \
    --output-path data/partnet_mobility_sdf/embeddings
```

--------------------------------

### Enable SAM3D Backend in Configuration

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Configure SceneSmith to use the SAM3D backend by setting the 'backend' option in your agent configuration files. SAM3D is the backend used in the paper.

```yaml
asset_manager:
  backend: "sam3d"
```

--------------------------------

### Programmatically Register Scene Package

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Load portable house scenes in Python scripts by programmatically registering the scene package with Drake's parser. This ensures correct path resolution for scene assets.

```python
from pydrake.multibody.parsing import Parser

parser = Parser(plant)
parser.package_map().Add("scene", "/path/to/scene_000")
```

--------------------------------

### Set Required Environment Variables

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Define these environment variables for both local and Docker usage. OPENAI_API_KEY is mandatory for GPT-5 agents and default image generation. GOOGLE_API_KEY is optional for Gemini image generation, and OPENAI_TRACING_KEY is optional for separate tracing accounts.

```sh
# Required: OpenAI API key for GPT-5 agents and default image generation
export OPENAI_API_KEY="your-openai-key"

# Optional: Google API key for Gemini image generation backend
# Only required if using image_generation.backend: "gemini" in config
export GOOGLE_API_KEY="your-google-key"

# Optional: Separate API key for OpenAI Agents tracing
# Allows traces to appear on a different account than is used for billing
export OPENAI_TRACING_KEY="your-tracing-key"
```

--------------------------------

### Programmatic Scene Loading with Drake

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Load and parse scene models programmatically in Python using Drake's MultibodyPlant and Parser. Configure package maps for scene portability.

```python
# Programmatic loading in Python
from pydrake.multibody.parsing import Parser
from pydrake.multibody.plant import MultibodyPlant
import pydrake.all as drake

plant = MultibodyPlant(time_step=0.001)
parser = Parser(plant)
parser.package_map().Add("scene", "/path/to/outputs/scene_000")
parser.AddModels("outputs/scene_000/combined_house/house.dmd.yaml")
plant.Finalize()
print(f"Loaded {plant.num_bodies()} bodies, {plant.num_joints()} joints")
```

--------------------------------

### Export Standalone SDF Model to MuJoCo

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Export a standalone SDF model directly to MuJoCo format. Provide the path to the SDF file and the output directory.

```bash
python scripts/export_scene_to_mujoco.py --sdf /path/to/model.sdf -o mujoco_export
```

--------------------------------

### Branch Experiment from Previous Run

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Create a new experiment that branches from a previous run's saved state. This is useful for A/B testing configurations or debugging specific stages.

```sh
# First run: generate floor plans and furniture
python main.py +name=base_run experiment.pipeline.stop_stage=furniture

# Branch 1: add manipulands with default config
python main.py +name=branch_1 \
  experiment.pipeline.start_stage=manipuland \
  experiment.pipeline.resume_from_path=outputs/2025-12-21/10-30-45

# Branch 2: add manipulands with different config
python main.py +name=branch_2 \
  experiment.pipeline.start_stage=manipuland \
  experiment.pipeline.resume_from_path=outputs/2025-12-21/10-30-45 \
  manipuland_agent.some_param=different_value
```

--------------------------------

### Enable HSSD in Configuration

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Configure SceneSmith to use the HSSD asset strategy by modifying the experiment configuration file.

```yaml
# In your experiment config file
asset_manager:
  strategy: "hssd"  # Use HSSD retrieval instead of generation
```

--------------------------------

### Export Scene to MuJoCo

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use this command to export a generated scene to the MuJoCo format. Specify the output directory for the exported files.

```bash
python scripts/export_scene_to_mujoco.py outputs/YYYY-MM-DD/HH-MM-SS/scene_000 \
    -o mujoco_export
```

--------------------------------

### Activate MuJoCo Export Environment

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Activate the virtual environment created for MuJoCo and USD export. This ensures that the correct dependencies are available for the conversion process.

```sh
source .mujoco_venv/bin/activate
```

--------------------------------

### Download AmbientCG PBR Materials

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Downloads PBR materials from AmbientCG. Supports specifying resolution and file format. A dry-run option is available to preview downloads without actual downloading.

```bash
python scripts/download_ambientcg.py \
    --output data/materials \
    -r 2K -f PNG
```

```bash
python scripts/download_ambientcg.py --dry-run --limit 20
```

--------------------------------

### Run SceneSmith Unit Tests in Docker

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Execute the unit tests for SceneSmith within the Docker container.

```sh
# Unit tests
docker compose run --rm scenesmith pytest tests/unit/ -x
```

--------------------------------

### Run Scene Generation within Docker Container

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Executes scene generation using the `main.py` script inside a Docker container. This can be done interactively or as a one-off command.

```bash
python main.py +name=my_experiment
```

```bash
docker compose run --rm scenesmith \
    python main.py +name=docker_run \
    "experiment.prompts=['A modern kitchen with an island and bar stools.']"
```

--------------------------------

### Enable Objaverse in Configuration

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Configure SceneSmith to use Objaverse as the general asset source in your experiment configuration.

```yaml
# In your experiment config file
asset_manager:
  general_asset_source: "objaverse"  # Use Objaverse retrieval
```

--------------------------------

### Enable Hunyuan3D-2 Backend in Configuration

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Configure SceneSmith to use the Hunyuan3D-2 backend by setting the 'backend' option in your agent configuration files.

```yaml
asset_manager:
  backend: "hunyuan3d"
```

--------------------------------

### Test Asset Retrieval

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Utility script to test CLIP-based asset retrieval before full scene generation. Renders multi-view images of retrieved objects for visual inspection. Supports ArtVIP/PartNet-Mobility.

```bash
# Test articulated furniture retrieval (ArtVIP/PartNet-Mobility)
python scripts/test_asset_retrieval.py \
    --source artvip \
    --query "wooden cabinet with two drawers" \
    --top-k 5 \
    --output-path output/retrieval_test
```

--------------------------------

### Extract and Propagate Support Surfaces

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Extracts support surfaces from a furniture object using HSM face-clustering and propagates the results to all geometrically identical instances in the scene. Requires `SupportSurfaceExtractionConfig` for filtering and backend selection.

```python
from scenesmith.agent_utils.room import (
    RoomScene, SceneObject, SupportSurface, UniqueID,
    extract_and_propagate_support_surfaces,
)
from scenesmith.agent_utils.support_surface_extraction import SupportSurfaceExtractionConfig
import numpy as np

config = SupportSurfaceExtractionConfig(
    min_area_m2=0.01,          # Filter tiny surfaces < 1cm²
    recompute_hssd_surfaces=False,  # Use pre-validated HSSD surfaces when available
)

# Extract surfaces for a table; propagates to all tables with same geometry
surfaces = extract_and_propagate_support_surfaces(
    scene=scene,
    furniture_object=table_object,
    config=config,
)
print(f"Found {len(surfaces)} support surfaces")
for surf in surfaces:
    print(f"  Surface {surf.surface_id}: area={surf.area:.3f}m²")

# Convert 2D surface-local pose to 3D world pose (for placing a book on the table)
surface = table_object.support_surfaces[0]
world_pose = surface.to_world_pose(
    position_2d=np.array([0.1, -0.05]),  # [x, y] on surface in surface frame
    rotation_2d=0.3,                      # yaw rotation on surface
    z_offset=0.0,
)

# Check if a 2D point (surface frame) is within the surface convex hull
is_valid = surface.contains_point_2d(np.array([0.1, -0.05]))

# Inverse: given a world pose, get back surface-relative 2D coordinates
pos_2d, rot_2d = surface.from_world_pose(world_pose)
```

--------------------------------

### Run SceneSmith Smoke Test in Docker

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Perform a basic smoke test within the SceneSmith Docker container to verify CUDA availability and SceneSmith import.

```sh
# Smoke test
docker compose run --rm scenesmith \
    python -c "import torch; print(torch.cuda.is_available()); import scenesmith"
```

--------------------------------

### Test Asset and Material Retrieval

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Scripts to test object and material retrieval functionalities. The asset retrieval script outputs PNG images for each candidate, while the material retrieval script generates preview renders in the specified output directory.

```bash
python scripts/test_asset_retrieval.py \
    --source hssd \
    --query "office desk" \
    --top-k 5 \
    --output-path output/hssd_test
```

```bash
python scripts/test_material_retrieval.py \
    --materials-dir data/materials \
    --query "rough dark wood floor" \
    --top-k 5 \
    --output-path output/material_test
```

--------------------------------

### Convert PartNet-Mobility to SDF

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Convert PartNet-Mobility dataset to SDF format using a Python script. Specify input and output directories.

```python
python scripts/convert_partnet_mobility.py \
    --input data/partnet-mobility-v0 \
    --output data/partnet_mobility_sdf
```

--------------------------------

### Visualize Intermediate Scene State

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Use Drake's model visualizer to interactively inspect intermediate scene states saved as .dmd.yaml files. This helps in understanding object placement and scene structure at different generation checkpoints.

```sh
python -m pydrake.visualization.model_visualizer \
    outputs/YYYY-MM-DD/HH-MM-SS/scene_000/room_*/scene_renders/furniture/renders_001/scene.dmd.yaml
```

--------------------------------

### Activate virtual environment

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Activates the Python virtual environment created by uv, typically named .venv. This command must be run in the shell session where you intend to use the project's dependencies.

```sh
source .venv/bin/activate
```

--------------------------------

### Compute PartNet-Mobility CLIP Embeddings with Renders

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Compute CLIP embeddings for PartNet-Mobility data and optionally keep rendered images for inspection. Use the --keep-renders flag.

```python
python scripts/compute_articulated_embeddings.py \
    --source partnet_mobility \
    --data-path data/partnet_mobility_sdf \
    --output-path data/partnet_mobility_sdf/embeddings \
    --keep-renders
```

--------------------------------

### Test Material Retrieval

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Test the retrieval of AmbientCG materials. Provide the materials directory, a query string, and the number of top results. Optionally, specify an output path for preview images.

```python
python scripts/test_material_retrieval.py \
    --materials-dir data/materials \
    --query "red brick wall" \
    --top-k 5

# Save preview images for inspection
python scripts/test_material_retrieval.py \
    --materials-dir data/materials \
    --query "wooden floor" \
    --top-k 5 \
    --output-path output/material_test
```

--------------------------------

### Docker Smoke Test and Unit Tests

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Performs a smoke test to verify CUDA availability and SceneSmith import within the Docker container. Also runs the unit test suite using pytest.

```bash
docker compose run --rm scenesmith \
    python -c "import torch; print(torch.cuda.is_available()); import scenesmith; print('OK')"
```

```bash
docker compose run --rm scenesmith pytest tests/unit/ -x
```

--------------------------------

### Generate Geometries using Client

Source: https://github.com/nepfaff/scenesmith/blob/main/scenesmith/agent_utils/geometry_generation_server/README.md

Uses the GeometryGenerationClient to send requests for 2D to 3D geometry conversion. Supports batch requests with backend selection and streaming results for parallel processing.

```python
from scenesmith.agent_utils.geometry_generation_server import (
    GeometryGenerationClient,
    GeometryGenerationServerRequest
)

client = GeometryGenerationClient()

# For batch requests with backend selection
requests = [
    GeometryGenerationServerRequest(
        image_path="/path/to/chair.png",
        output_dir="/path/to/output",
        prompt="Modern wooden chair",
        backend="sam3d",  # or "hunyuan3d"
        sam3d_config={
            "sam3_checkpoint": "external/checkpoints/sam3.pt",
            "sam3d_checkpoint": "external/checkpoints/pipeline.yaml",
            "mode": "foreground",  # or "text"
        },
    ),
]

# Process results as they stream back (enables parallel GPU/CPU processing)
for index, response in client.generate_geometries(requests):
    print(f"Asset {index} completed: {response.geometry_path}")
    # Can start Drake conversion here while GPU processes next asset
```

--------------------------------

### Export Scene to MuJoCo

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Converts a Drake SDFormat scene to MuJoCo MJCF format, including mesh conversion and joint mapping. Supports optional USD export for Isaac Sim.

```bash
# No code provided in the source for this snippet.
```

--------------------------------

### Download Objaverse Data

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Download ObjectThor data, which includes assets, annotations, and pre-computed CLIP features. This is a large download.

```sh
bash scripts/download_objaverse_data.sh
```

--------------------------------

### Compute AmbientCG Embeddings

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Alternatively, compute CLIP embeddings for AmbientCG materials yourself using this script. Ensure the materials directory is correctly specified.

```python
python scripts/compute_ambientcg_embeddings.py --materials-dir data/materials
```

--------------------------------

### Preprocess Objaverse Data

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Prepare Objaverse data for retrieval by running a Python script. This generates averaged CLIP embeddings and a metadata index.

```python
python scripts/prepare_objaverse.py
```

--------------------------------

### Robot Evaluation Pipeline: Generate Scenes

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Stage 2 of the robot evaluation pipeline. Generates scenes based on prompts from Stage 1. This uses the standard scene generation pipeline.

```bash
# Stage 2: Generate scenes (standard pipeline, reads prompts.csv)
python main.py +name=robot_eval \
    experiment.csv_path=outputs/eval_run/prompts.csv
```

--------------------------------

### Control SceneSmith Pipeline Stages

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Control the execution of pipeline stages using `start_stage` and `stop_stage`. This allows for partial generation, reviewing intermediate results, or resuming from specific checkpoints. State is automatically checkpointed.

```bash
python main.py +name=layout_only \
    experiment.pipeline.stop_stage=floor_plan
```

```bash
python main.py +name=furniture_only \
    experiment.pipeline.stop_stage=furniture
```

```bash
python main.py +name=resume_wall \
    experiment.pipeline.start_stage=wall_mounted
```

```bash
python main.py +name=branch_a \
    experiment.pipeline.start_stage=manipuland \
    experiment.pipeline.resume_from_path=outputs/2025-12-21/10-30-45
```

```bash
python main.py +name=branch_b \
    experiment.pipeline.start_stage=manipuland \
    experiment.pipeline.resume_from_path=outputs/2025-12-21/10-30-45 \
    manipuland_agent.asset_manager.general_asset_source=hssd
```

```bash
python main.py +name=parallel_rooms \
    experiment.pipeline.parallel_rooms=true \
    experiment.pipeline.max_parallel_rooms=4
```

--------------------------------

### Configure Articulated Retrieval Sources

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

This YAML snippet shows how to enable and configure articulated asset sources within the agent configuration. Adjust `enabled`, `data_path`, and `embeddings_path` as needed.

```yaml
# In configurations/furniture_agent/base_furniture_agent.yaml
asset_manager:
  router:
    enabled: true
  articulated:
    use_top_k: 5  # Number of CLIP candidates before bbox ranking
    sources:
      partnet_mobility:
        enabled: false  # Disabled by default (low quality)
        data_path: data/partnet_mobility_sdf
        embeddings_path: data/partnet_mobility_sdf/embeddings
      artvip:
        enabled: true  # Enabled by default
        data_path: data/artvip_sdf
        embeddings_path: data/artvip_sdf/embeddings
```

--------------------------------

### Run Unit Tests with Testmon

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Execute unit tests for the project using pytest. The `--testmon` flag enables test caching for faster feedback during development. Use `-x` to stop on the first test failure.

```sh
pytest tests/unit/ --testmon
```

--------------------------------

### Run Integration Tests with Testmon

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Execute integration tests for the project using pytest. The `--testmon` flag enables test caching for faster feedback. Some heavy integration tests may be skipped in CI and require local execution.

```sh
pytest tests/integration/ --testmon
```

--------------------------------

### Run Pytest Unit Tests

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Executes unit tests using pytest. Options include running with testmon for continuous testing, verbose output, stopping on the first failure, or running specific test modules.

```bash
pytest tests/unit/ --testmon
```

```bash
pytest tests/unit/ -v -x
```

```bash
pytest tests/unit/test_room_placement.py -v
```

```bash
pytest tests/unit/test_asset_manager.py -v
```

```bash
pytest tests/unit/test_furniture_agent.py -v
```

--------------------------------

### Compute Articulated Embeddings

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

This script computes CLIP embeddings for articulated assets. Specify the source directory, data path, and output path for the embeddings.

```python
python scripts/compute_articulated_embeddings.py \
    --source artvip \
    --data-path data/artvip_sdf \
    --output-path data/artvip_sdf/embeddings
```

--------------------------------

### Configure Multi-GPU for Scene Generation

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Control GPU usage for scene generation by setting the `CUDA_VISIBLE_DEVICES` environment variable. This allows specifying which GPUs to use or forcing single-GPU mode.

```sh
# Use only GPUs 0, 1, 2, 3 (4 workers)
CUDA_VISIBLE_DEVICES=0,1,2,3 python main.py +name=my_experiment

# Force single GPU mode
CUDA_VISIBLE_DEVICES=0 python main.py +name=my_experiment

# Use all available GPUs (default - no env var needed)
python main.py +name=my_experiment
```

--------------------------------

### RoomScene Class

Source: https://context7.com/nepfaff/scenesmith/llms.txt

The RoomScene class is the central state container for a single room. It manages SceneObject instances, generates simulation directives, and supports serialization for checkpointing.

```APIDOC
## RoomScene

**`scenesmith/agent_utils/room.py`**

The central state container for a single room. Manages all `SceneObject` instances, generates Drake simulation directives, and supports serialization/deserialization for checkpointing. Accessed by all placement agents.

### Methods

- **`__init__(room_geometry, scene_dir, room_id, text_description, action_log_path)`**: Initializes a RoomScene.
- **`add_object(object)`**: Adds a SceneObject to the room.
- **`get_objects_by_type(object_type)`**: Retrieves all objects of a specific type.
- **`get_manipulands()`**: Retrieves all manipulable objects.
- **`get_objects_on_surface(surface_id)`**: Retrieves objects placed on a specific surface.
- **`move_object(object_id, new_transform)`**: Moves an object to a new pose.
- **`remove_object(object_id)`**: Removes an object from the scene.
- **`to_drake_directive(...)`**: Generates Drake simulation directives.
- **`to_state_dict()`**: Serializes the scene state to a dictionary.
- **`restore_from_state_dict(state_dict)`**: Restores the scene from a dictionary.

### Example Usage

```python
from pathlib import Path
from scenesmith.agent_utils.room import RoomScene, SceneObject, ObjectType, UniqueID
from pydrake.all import RigidTransform, RollPitchYaw
import numpy as np

# Create a room scene with pre-built room geometry
scene = RoomScene(
    room_geometry=room_geometry,   # RoomGeometry from floor plan agent
    scene_dir=Path("outputs/scene_000/room_main"),
    room_id="main",
    text_description="A cozy bedroom with hardwood floors.",
    action_log_path=Path("outputs/scene_000/room_main/action_log.json"),
)

# Add a wall (immutable room boundary object)
scene.add_object(wall_object)

# Query objects
furniture = scene.get_objects_by_type(ObjectType.FURNITURE)
manipulands = scene.get_manipulands()
objects_on_table = scene.get_objects_on_surface(surface_id=UniqueID("S_3"))

# Move an object to a new world-frame pose
new_pose = RigidTransform(rpy=RollPitchYaw(0, 0, 1.57), p=[1.5, 2.0, 0.0])
scene.move_object(object_id=UniqueID("chair_0"), new_transform=new_pose)

# Remove object
scene.remove_object(UniqueID("chair_0"))  # Returns True if removed

# Generate Drake Model Directives YAML string for simulation/visualization
drake_yaml = scene.to_drake_directive(
    weld_furniture=True,          # Furniture welded to world frame
    weld_room_geometry=True,
    base_dir=Path("outputs/scene_000/room_main"),   # Enables package:// URIs
    include_object_types=[ObjectType.FURNITURE, ObjectType.MANIPULAND],
)
# Save to .dmd.yaml for Drake or pydrake.visualization.model_visualizer

# Checkpoint: serialize full scene state to JSON-serializable dict
state = scene.to_state_dict()
# state["objects"] is a dict of object_id -> serialized SceneObject

# Restore scene from a previous checkpoint
with open("outputs/scene_000/room_main/scene_state.json") as f:
    import json
    saved_state = json.load(f)
scene.restore_from_state_dict(saved_state)
print(f"Restored {len(scene.objects)} objects")
```
```

--------------------------------

### Generate Drake Directives from RoomScene

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Generate a Drake Model Directives YAML string for simulation and visualization. Use `base_dir` to enable package:// URIs and `include_object_types` to filter objects.

```python
# Generate Drake Model Directives YAML string for simulation/visualization
drake_yaml = scene.to_drake_directive(
    weld_furniture=True,          # Furniture welded to world frame
    weld_room_geometry=True,
    base_dir=Path("outputs/scene_000/room_main"),   # Enables package:// URIs
    include_object_types=[ObjectType.FURNITURE, ObjectType.MANIPULAND],
)
```

--------------------------------

### Download HSSD Preprocessed Data

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Download preprocessed data for HSSD, including CLIP indices and embeddings, and pre-validated support surfaces. This can be done via a script or manually.

```sh
bash scripts/download_hssd_data.sh
```

```sh
# Download CLIP indices
wget https://github.com/3dlg-hcvc/hsm/releases/latest/download/data.zip
unzip data.zip -d data/preprocessed

# Download pre-validated support surfaces
wget https://github.com/3dlg-hcvc/hsm/releases/latest/download/support-surfaces.zip
unzip support-surfaces.zip -d data/hssd-models
```

--------------------------------

### Robot Evaluation Pipeline: Extract Poses

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Stage 3 of the robot evaluation pipeline. Extracts robot-executable poses from a generated scene. Requires scene state, DMD file, scene directory, and task description.

```bash
# Stage 3: Extract robot-executable poses from a generated scene
python scripts/robot_eval/policy_interface.py \
    --scene-state outputs/2025-12-21/10-30-45/scene_002/combined_house/house_state.json \
    --dmd outputs/2025-12-21/10-30-45/scene_002/combined_house/house.dmd.yaml \
    --scene-dir outputs/2025-12-21/10-30-45/scene_002 \
    --task "Find a speaker and place it on the bed" \
    --output-json robot_commands.json
# robot_commands.json contains ranked (target, reference) pose bindings
```

--------------------------------

### Compute PartNet-Mobility CLIP Embeddings

Source: https://github.com/nepfaff/scenesmith/blob/main/README.md

Compute CLIP embeddings for PartNet-Mobility data to enable text-based retrieval. Specify source, data path, and output path.

```python
python scripts/compute_articulated_embeddings.py \
    --source partnet_mobility \
    --data-path data/partnet_mobility_sdf \
    --output-path data/partnet_mobility_sdf/embeddings
```

--------------------------------

### Asset Manager Configuration

Source: https://context7.com/nepfaff/scenesmith/llms.txt

YAML configuration for the asset manager, controlling 3D asset sourcing, validation, and generation strategies. Supports various backends like SAM3D, Hunyuan3D, and HSSD retrieval.

```yaml
# In configurations/furniture_agent/base_furniture_agent.yaml
asset_manager:
  # Switch between on-demand generation and CLIP-based retrieval
  general_asset_source: "generated"  # or "hssd", "objaverse"
  backend: "sam3d"                   # or "hunyuan3d" (lower quality, fits 24GB)

  # LLM-based asset routing: analyzes each request and picks strategy
  router:
    enabled: true
    parallel_workers: 10     # Concurrent asset generation threads
    strategies:
      generated:
        enabled: true
        max_retries: 2
      articulated:
        enabled: true          # Cabinets, drawers from ArtVIP/PartNet-Mobility
        max_retries: 3
      thin_covering:
        enabled: true          # Rugs, yoga mats as flat textured planes
        thickness_m: 0.003
        texture_scale: 0.5     # Meters per tile

  # SAM3D model checkpoints
  sam3d:
    sam3_checkpoint: "external/checkpoints/sam3.pt"
    sam3d_checkpoint: "external/checkpoints/pipeline.yaml"
    mode: "foreground"         # "foreground" or "object_description"

  # HSSD retrieval
  hssd:
    data_path: "data/hssd-models"
    preprocessed_path: "data/preprocessed"
    use_top_k: 5

  # Articulated sources
  articulated:
    sources:
      artvip:
        enabled: true
        data_path: "data/artvip_sdf"
        embeddings_path: "data/artvip_sdf/embeddings"
      partnet_mobility:
        enabled: false         # Low quality dataset; disabled by default

# OpenAI model for all agents
openai:
  model: "gpt-5.2"
  reasoning_effort:
    planner: "low"
    designer: "high"
    critic: "high"
```

--------------------------------

### Checkpoint and Restore RoomScene State

Source: https://context7.com/nepfaff/scenesmith/llms.txt

Serialize the full scene state to a dictionary for checkpointing or restore the scene from a saved state dictionary. Paths in the serialized state are relative to `scene_dir` for portability.

```python
# Checkpoint: serialize full scene state to JSON-serializable dict
state = scene.to_state_dict()
# state["objects"] is a dict of object_id -> serialized SceneObject

# Restore scene from a previous checkpoint
with open("outputs/scene_000/room_main/scene_states/scene_after_furniture/scene_state.json") as f:
    import json
    saved_state = json.load(f)
scene.restore_from_state_dict(saved_state)
print(f"Restored {len(scene.objects)} objects")
```