### Setup Volume Grid Library Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/README.md Executes a setup script to install the volume grid library required for running example tour agents. This script is located within the agents/sg directory. ```bash cd agents/sg ./setup.sh ``` -------------------------------- ### Manual Traffic Simulation Setup Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt This code demonstrates setting up and running a standalone traffic simulation for dry-run testing without physics. It involves creating a LocalMap and a TrafficManager instance, then scheduling simulation steps. Dependencies include 'modules.traffic_manager.traffic_manager.TrafficManager' and 'modules.traffic_manager.localmap.LocalMap'. ```python from modules.traffic_manager.traffic_manager import TrafficManager as TM from modules.traffic_manager.localmap import LocalMap # Create standalone traffic simulation local_map = LocalMap( file_path="assets/scenes/NY/road_data/road_data.xodr", terrain_height_path=None, ref_lat=40.7484, ref_lon=-73.9857 ) traffic_sim = TM( env=None, scene_name='NY', vehicle_number=20, pedestrian_number=50, dry_run=True, # No physics, just position updates logger=None ) # Run traffic simulation for _ in range(100): traffic_sim.schedule() positions = traffic_sim.get_vehicles_pos() ``` -------------------------------- ### Install Genesis Package Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/README.md Installs the Genesis package in editable mode from its source directory. This allows for development and testing of the package. ```bash cd Genesis pip install -e . ``` -------------------------------- ### AgentProcess Management Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Demonstrates the creation and management of an AgentProcess for multi-process execution. It shows how to instantiate an agent process with a custom agent class, start the process, update it with observations, retrieve actions, and handle conversational aspects. ```python # Create agent process for multi-process execution agent_process = AgentProcess( agent_cls=CustomAgent, name="Agent_1", pose=[0, 0, 0, 0, 0, 0], info={'cash': 500, 'held_objects': [None, None]}, sim_path='output/curr_sim', multi_process=True, logging_level='INFO' ) # Start agent process agent_process.start() # Update agent with observations agent_process.update(obs) # Get agent action action = agent_process.act() # Handle conversation agent_process.request_chat("What is your plan?") utterance = agent_process.get_utterance(step=0) ``` -------------------------------- ### Run Detroit Scene with Traffic (Bash) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Starts a simulation in the 'DETROIT' scene with traffic elements using 'env.py'. This configuration includes 10 agents, specifies traffic vehicle and avatar numbers, and enables debug mode for traffic management. ```bash python env.py --head_less \ --backend gpu \ --scene DETROIT \ --num_agents 10 \ --config agents_num_15 \ --tm_vehicle_num 20 \ --tm_avatar_num 30 \ --enable_tm_debug ``` -------------------------------- ### Simulation Loop Action Handling (Python) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt An example of a simulation loop that processes agent observations and determines their next actions. It checks the current action status of each agent, handling ongoing actions, collisions by retrying with a turn, or planning new actions based on observations. ```python for step in range(max_steps): agent_actions = {} for i in range(env.num_agents): if obs[i]['action_status'] == 'ONGOING': agent_actions[i] = None # Wait for current action to complete elif obs[i]['action_status'] == 'COLLIDE': # Handle collision - turn and retry agent_actions[i] = {'type': 'turn_left', 'arg1': 30} else: # Plan next action based on observations agent_actions[i] = plan_action(obs[i]) agent_actions['agent_list_to_update'] = list(range(env.num_agents)) obs, _, _, _ = env.step(agent_actions) ``` -------------------------------- ### Resolve GLIBCXX Version Error Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/README.md Installs the libstdcxx-ng package from conda-forge to resolve 'version GLIBCXX_3.4.32 not found' errors, which indicate a mismatch in the GCC runtime library. ```bash conda install -c conda-forge libstdcxx-ng ``` -------------------------------- ### AvatarController Humanoid Actions Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Manages humanoid avatar movement, animations, and interactions. Supports high-level actions like walking, running, sitting, object manipulation, and vehicle entry/exit. Provides methods to get avatar state and render camera views. ```python from modules.avatar import AvatarController import numpy as np # AvatarController is created internally by VicoEnv # Access avatar through environment avatar = env.agents[0] # First agent (if not a robot) # Basic movement actions avatar.walk(distance=5.0, speed=1.0) # Walk 5 meters at normal speed avatar.run(distance=10.0, speed=2.0) # Run 10 meters avatar.turn_left(angle=90, turn_frame_limit=15) # Turn left 90 degrees avatar.turn_right(angle=45, turn_frame_limit=15) # Turn right 45 degrees # Look at a specific position avatar.move_forward(distance=3.0, speed=1.0) # State changes avatar.sleep() # Enter sleep state avatar.wake() # Wake up from sleep avatar.sit(position=np.array([10.0, 5.0, 0.0])) # Sit at position avatar.stand_up() # Stand up from sitting # Object interaction avatar.pick(hand_id=0, obj=target_entity) # Pick up object with right hand avatar.put(hand_id=0, location=np.array([1, 0, 0])) # Put object down avatar.drink(hand_id=0) # Drinking animation avatar.eat(hand_id=0) # Eating animation # Vehicle interaction avatar.enter_bus(bus_entity) # Enter public bus avatar.exit_bus() # Exit from bus avatar.enter_bike(hand_id=0, bike_entity) # Mount a bicycle avatar.exit_bike(hand_id=0) # Dismount bicycle # Play custom animations avatar.play_animation(name="wave") # Play wave animation # Get avatar state pose = avatar.get_global_pose() # [x, y, z, roll, pitch, yaw] xy = avatar.get_global_xy() # (x, y) position height = avatar.get_global_height() # z height # Render ego-view camera rgb, depth, segmentation, extrinsics = avatar.render_ego_view( depth=True, segmentation=True ) # Check if avatar is idle is_idle = avatar.spare() # Get current action status status = avatar.action_status() # SUCCEED, FAIL, ONGOING, COLLIDE ``` -------------------------------- ### Initialize SimpleVicoEnv for Demo Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Initializes a SimpleVicoEnv for demonstration purposes. It configures simulation parameters such as scene path, output directory, backend, resolution, and rendering options. The environment is set up for GPU acceleration and includes collision detection and third-person cameras. ```python env = SimpleVicoEnv( config_path='assets/scenes/demos/1', # Demo configuration path output_dir='output/demo', backend=gs.gpu, seed=0, resolution=512, skip_avatar_animation=False, enable_collision=True, enable_third_person_cameras=True, load_indoor_objects=True, use_luisa_renderer=False, # Use LuisaRender for ray tracing dt_sim=0.01, head_less=True ) # Reset and get initial stateobs = env.reset() # Define agent actions for demo agent_actions = { 0: {'type': 'move_forward', 'arg1': 2.0}, # Move 2 meters forward 1: {'type': 'turn_left', 'arg1': 45.0}, # Turn left 45 degrees } # Run simulation steps for step in range(100): obs, reward, done, info = env.step(agent_actions) # Actions are consumed, set new actions or None to idle agent_actions = {i: None for i in range(env.num_agents)} # Close and generate video from demo camera frames env.close() ``` -------------------------------- ### Run Simple Demo Environment (Bash) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Executes a simple demo environment using 'simple_env.py'. This command specifies a configuration path, an output directory, runs on GPU, sets a step limit, loads indoor objects, and overwrites existing output. ```bash python simple_env.py \ --config_path assets/scenes/demos/1 \ --output_dir output/demo \ --backend gpu \ --step_limit 500 \ --load_indoor_objects \ --overwrite ``` -------------------------------- ### Run Basic Tour Agent Simulation (Bash) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Launches a basic tour agent simulation using the 'env.py' script. This command configures the simulation to run headlessly on a GPU, skip avatar animations, use the 'NY' scene, set a resolution of 512, employ 15 agents with a 'tour_agent' type, and run for 100 seconds with specific logging and overwrite settings. ```bash python env.py --head_less \ --backend gpu \ --skip_avatar_animation \ --scene NY \ --resolution 512 \ --num_agents 15 \ --config agents_num_15 \ --agent_type tour_agent \ --max_seconds 100 \ --save_per_seconds 1 \ --enable_gt_segmentation \ --logging_level info \ --overwrite ``` -------------------------------- ### Navigate and Visualize with Volume Grid Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Demonstrates navigation on a volume grid using A* pathfinding, retrieving occupancy maps for visualization, and checking for obstacles. It also covers saving and loading the volume grid and configuring its parameters for different environments. ```python current_position = np.array([10.0, 5.0]) goal_position = np.array([50.0, 30.0]) goal_bbox = np.array([[45, 25], [55, 35]]) # Bounding box [[min], [max]] # Find path using A* on volume grid path = sg_builder.volume_grid_builder.navigate( current_position, goal_bbox, previous_path=None # Provide for path smoothing ) if path is not None: next_waypoint = path[min(5, len(path) - 1)] # Get occupancy map for visualization occ_map, _ = sg_builder.volume_grid_builder.get_occ_map( agent_pose[:3], output_path="debug_occ_map.png" # Optional: save visualization ) # Check for obstacles has_obstacle = sg_builder.volume_grid_builder.has_obstacle(bbox) # Save and load volume grid sg_builder.volume_grid_builder.save("volume_grid.pkl") sg_builder.volume_grid_builder.load("volume_grid.pkl") # Volume grid configuration outdoor_config = VolumeGridBuilderConfig( voxel_size=0.1, # 10cm voxels nav_grid_size=0.5, # 50cm navigation grid depth_bound=30.0 # Maximum depth in meters ) indoor_config = VolumeGridBuilderConfig( voxel_size=0.025, # 2.5cm voxels for indoor detail nav_grid_size=0.2, # 20cm navigation grid depth_bound=30.0 ) ``` -------------------------------- ### Initialize SimpleVicoEnv for Demonstrations Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Initializes the SimpleVicoEnv, a lightweight environment class for basic demonstrations and testing within the Virtual Community platform. This class supports custom scene loading and avatar control, offering a simpler alternative to the full VicoEnv. ```python import genesis as gs from simple_env import SimpleVicoEnv ``` -------------------------------- ### Run Robot Simulation (Bash) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Initiates a simulation involving robots using the 'env.py' script. This command runs headlessly on a GPU in the 'NY' scene with 15 agents, using a robot-specific configuration. It enables collision detection, third-person cameras, and runs for 500 seconds. ```bash python env.py --head_less \ --backend gpu \ --scene NY \ --resolution 512 \ --num_agents 15 \ --config agents_num_15_robot \ --agent_type tour_agent \ --enable_collision \ --enable_third_person_cameras \ --max_seconds 500 ``` -------------------------------- ### Access and Control Robot in Virtual Environment Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt This snippet demonstrates how to access a robot from the environment, retrieve its observations, perform control actions, step the simulation, and get/reset its pose. It also covers rendering the robot's ego view. Dependencies include 'torch' and 'numpy'. ```python robot = env.robots[0] # First robot (Go2) # Get robot observations robot_obs, additional_obs = robot.get_observations() # Perform control action action = { 'type': 'control', 'control': torch.zeros(12) # Joint position targets } robot.perform_control(action) # Step the robot simulation robot.step(actions=None, perform_physics_step=True) # Get robot pose pose = robot.get_global_pose() # [x, y, z, qw, qx, qy, qz] xy = robot.get_global_xy() # numpy array [x, y] height = robot.get_global_height() # Reset robot position robot.reset( position=np.array([10.0, 5.0, 1.0]), rotation=np.eye(3) # 3x3 rotation matrix ) # Render robot ego view rgb, depth, seg, transform = robot.render_ego_view( depth=True, segmentation=True ) ``` -------------------------------- ### Initialize and Run VicoEnv Simulation Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Initializes the VicoEnv, the main simulation environment for Virtual Community, with a comprehensive set of parameters. It then runs a simulation loop, collecting observations, rewards, and status updates, and finally closes the environment. This snippet demonstrates setting up physics, agents, scenes, and rendering options. ```python import genesis as gs from env import VicoEnv # Initialize the environment with full configuration env = VicoEnv( seed=0, precision='32', logging_level='info', backend=gs.gpu, # Use GPU backend for physics head_less=True, # Run without visualization window resolution=512, # Camera resolution challenge='full', # Full simulation mode num_agents=15, # Number of agents in simulation config_path='output/NY_agents_num_15/curr_sim', # Configuration directory scene='NY', # Scene name (NY or DETROIT) enable_indoor_scene=True, # Load indoor environments enable_indoor_objects=True, # Enable indoor object interactions enable_outdoor_objects=True, # Enable outdoor objects (benches, etc.) outdoor_objects_max_num=10, # Maximum outdoor objects to load enable_collision=True, # Enable physics collision detection enable_decompose=False, # Decompose meshes for collision skip_avatar_animation=False, # Skip detailed avatar animations enable_gt_segmentation=True, # Enable ground truth segmentation no_load_scene=False, # Load the 3D scene output_dir='output', # Output directory for renders enable_third_person_cameras=True, # Enable third-person view cameras enable_demo_camera=False, # Enable demo camera no_traffic_manager=False, # Enable traffic system tm_vehicle_num=5, # Number of traffic vehicles tm_avatar_num=10, # Number of pedestrians enable_tm_debug=False, # Traffic manager debug mode save_per_seconds=10, # Save interval for images defer_chat=True, # Defer chat generation debug=False, dt_sim=0.01, # Physics timestep batch_renderer=False # Use batch rendering ) # Reset environment and get initial observations obs = env.reset() # Main simulation loop for step in range(1000): # Collect actions from all agents agent_actions = {i: None for i in range(env.num_agents)} agent_actions['agent_list_to_update'] = list(range(env.num_agents)) # Step the environment obs, reward, done, info = env.step(agent_actions) # Access observations for each agent for agent_id in range(env.num_agents): rgb = obs[agent_id].get('rgb') # Ego-view RGB image depth = obs[agent_id].get('depth') # Depth map pose = obs[agent_id].get('pose') # Agent position [x, y, z, roll, pitch, yaw] accessible = obs[agent_id].get('accessible_places') # List of accessible places # Clean up env.close() ``` -------------------------------- ### Robot Controller Initialization Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Supports various robot types including Go2 quadruped, H1 humanoid, Google Robot, Husky, and Drone. Controllers manage locomotion, perception, and control policies. Robots are initialized internally by VicoEnv based on specified agent IDs and types. ```python from modules.robot import Go2Controller, H1Controller, DroneController import torch import numpy as np # Robot controllers are created internally by VicoEnv when robot_agent_id_list is specified # Configuration example for robots config = { 'robot_agent_id_list': [0, 1], 'robot_types': ['go2', 'h1'], } ``` -------------------------------- ### Manage Autonomous Vehicles with VehicleController Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Illustrates how to use the VehicleController class to manage autonomous vehicles, including basic movement, turning (immediate and scheduled), stopping, retrieving vehicle state, resetting position, rendering camera views, and creating custom vehicles. ```python from modules.vehicle import VehicleController import numpy as np import genesis.utils.geom as geom_utils # VehicleController is created internally by VicoEnv vehicle = env.vehicles[0] # First vehicle # Basic vehicle movement vehicle.move_forward( target_pos=np.array([100.0, 50.0, 0.0]), speed=10.0 # m/s ) # Turning vehicle.turn_left(angle=30) # Immediate turn vehicle.turn_right(angle=30) # Scheduled turning (smooth animation) vehicle.turn_left_schedule(angle=45, speed=60) # 60 deg/s vehicle.turn_right_schedule(angle=45, speed=60) # Stop vehicle vehicle.stop() # Get vehicle state pose = vehicle.get_global_pose() # [x, y, z, roll, pitch, yaw] xy = vehicle.get_global_xy() # (x, y) tuple height = vehicle.get_global_height() # Check if vehicle is idle is_idle = vehicle.spare() # Reset vehicle position vehicle.reset( global_trans=np.array([50.0, 30.0, 0.0]), global_rot=np.eye(3) # Identity rotation matrix ) # Render vehicle camera view rgb, depth, seg, transform = vehicle.render_ego_view( depth=True, segmentation=True ) # Vehicle step (called internally) vehicle.step() # Create custom vehicle (internal to VicoEnv) custom_vehicle = env.add_vehicle( name="delivery_truck", vehicle_asset_path="ViCo/cars/Car/truck.glb", ego_view_options={ "res": (512, 512), "fov": 90, "GUI": False }, position=np.array([0.0, 0.0, 0.0]), rotation=np.array([0.0, 0.0, 0.0]), dt=0.01, forward_speed_m_per_s=15, angular_speed_deg_per_s=45, terrain_height_path="ViCo/scene/v1/NY/height_field.npz" ) ``` -------------------------------- ### Configure Robot Types and Position Offsets Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Defines configurations for various robot types, including file paths to their settings, and specifies position offsets for height adjustments. This is useful for initializing or managing different robot models in the simulation. ```python # Robot types and their configurations ROBOT_CONFIGS = { "go2": "go2/cfgs.pkl", # Unitree Go2 quadruped "h1": "h1/cfgs.pkl", # Unitree H1 humanoid "google_robot": "google_robot/cfgs.pkl", "drone": "drone/cfgs.pkl", "husky": "husky/cfgs.pkl" # Clearpath Husky } # Position offsets for different robots (z-height adjustment) ROBOT_POSITION_OFFSETS = { "go2": (0, 0, 0.8), "h1": (0, 0, 1.4), "google_robot": (0, 0, 0.8), "drone": (0, 0, 3.0), "husky": (0, 0, 0.9) } ``` -------------------------------- ### TourAgent Navigation with A* Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Implements an autonomous navigation agent using A* pathfinding on a volume grid. It tours buildings by setting goals and processing observations to determine movement actions. Requires scene graph and environment metadata for spatial memory. ```python from agents.tour_agent import TourAgent from agents.scene_graph import SceneGraph import numpy as np class NavigationDemo: def __init__(self, env): self.env = env # Create tour agent with spatial memory self.agent = TourAgent( name="Navigator", pose=[0, 0, 0, 0, 0, 0], info={'cash': 500, 'held_objects': [None, None]}, sim_path='output/curr_sim', tour_spatial_memory=env.building_metadata, # Building locations no_react=False, debug=True, logger=None ) def navigate_to_building(self, building_name): """Navigate to a specific building""" if building_name in self.agent.tour_spatial_memory: building_data = self.agent.tour_spatial_memory[building_name] self.agent.curr_goal_name = building_name self.agent.curr_goal_pos = np.array([ building_data['places'][0]['location'][0], building_data['places'][0]['location'][1] ]) self.agent.curr_goal_bbox = building_data.get("bounding_box") def step(self, obs): """Process observation and get navigation action""" # Update scene graph with visual observations self.agent._process_obs(obs) # Get navigation action action = self.agent._act(obs) return action # Example usage with environment demo = NavigationDemo(env) demo.navigate_to_building("Empire State Building") while True: obs = env.step(agent_actions)[0] action = demo.step(obs[0]) if action is None: break # Reached destination agent_actions = {0: action, 'agent_list_to_update': [0]} ``` -------------------------------- ### Initialize and Update SceneGraph for Spatial Memory Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt This snippet details the initialization and updating of a SceneGraph, which builds a volumetric representation of the environment using RGB-D observations. It includes setting up storage paths and configuring the graph's field of view and update interval. Dependencies include 'agents.scene_graph.SceneGraph', 'agents.sg.builder.builder', 'numpy', and 'os'. ```python from agents.scene_graph import SceneGraph from agents.sg.builder.builder import Builder, BuilderConfig, VolumeGridBuilderConfig import numpy as np import os # Create scene graph for navigation storage_path = "output/scene_graphs/agent_1" os.makedirs(storage_path, exist_ok=True) scene_graph = SceneGraph( storage_path=storage_path, detect_interval=1, # Process every frame fov=90.0, # Field of view in degrees debug=True, logger=None ) # Update scene graph with observations obs = { 'rgb': rgb_image, # (H, W, 3) uint8 'depth': depth_map, # (H, W) float32 'extrinsics': camera_transform, # (4, 4) camera matrix 'current_place': None, # Current indoor location or None 'pose': [x, y, z, roll, pitch, yaw] # Agent pose } scene_graph.update(obs) # Get current scene graph builder sg_builder = scene_graph.get_sg() ``` -------------------------------- ### Agent Data Loading and Initialization Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/tools/template.html Handles the asynchronous loading of configuration and simulation step data. It fetches JSON files for configuration, all steps, and individual agent scratch data, then populates the UI with agent details. ```javascript function loadAgents() { const sceneNameMap = { NY: "New York City", DETROIT: "Detroit City" }; fetch("$ConfigPath$") .then((response) => response.json()) .then((config) => { configData = config; fetch("$OutputFolderPath$/steps/all_steps.json") .then((response) => response.json()) .then((all_steps) => { allStepsData = all_steps; const agentNames = configData.agent_names || []; agentNames.forEach((name) => { fetch("$CurrSimFolderPath$" + "/" + name + "/scratch.json") .then((response) => response.json()) .then((scratch) => { const safeId = name.replace(/\s+/g, "_"); document.getElementById("age__" + safeId).innerText = scratch.age || "Unknown"; document.getElementById("personality__" + safeId).innerText = scratch.innate || "Unknown"; document.getElementById("bio__" + safeId).innerText = scratch.learned || "Unknown"; document.getElementById("current_goal__" + safeId).innerText = scratch.currently || "Unknown"; }) .catch((err) => { console.log(err); }); const safeId = name.replace(/\s+/g, "_"); const detailPanel = createDetailPanel(name); document.getElementById("character-details-container") .appendChild(detailPanel); }); agentNamesGlobal = agentNames.slice(); const groups = configData.groups || {}; const locatorColors = configData.locator_colors || []; const sceneName = document.getElementById("scene-name"); sceneName.innerText = "at " + sceneNameMap[configData.sim_name.split("_")[0]]; sceneName.style.color = "red"; sceneName.style.fontFamily = "georgia"; sceneName.style.fontStyle = "italic"; sceneName.style.fontWeight = "bold"; createAllGroupVisuals(groups); const detailsContainer = document.getElementById("character-details-container"); agentNames.forEach((name) => { const detailPanel = createDetailPanel(name); detailsContainer.appendChild(detailPanel); }); const randomFive = pickRandom(agentNamesGlobal, 5); randomFive.forEach((a) => selectedAgents.add(a)); placeAgentSprites(agentNames, locatorColors); fetch("$GlobalCameraParameterPath$") .then((response) => response.json()) .then((camParams) => { cameraParamsGlobal = camParams; if (agentNamesGlobal.length > 0) { const firstAgent = agentNamesGlobal[0]; currentAgentForDetail = firstAgent; const safeId = firstAgent.replace(/\s+/g, "_"); const firstPanel = document.getElementById("on_screen_det_content-" + safeId); if (firstPanel) firstPanel.style.display = "block"; } fetch("$OutputFolderPath$/steps/conversation_cumulative_lookup.json") .then((response) => response.json()) .then((conversationCumulativeLookup) => { allStepsConvo = new ConvoStepLookup(conversationCumulativeLookup); fetch("$TpegoFolderPath$/ego_check_exists_lookup.json") .then((response) => response.json()) .then((ego_check_exists_lookup) => { egoCheckExistsLookup = new TpegoStepLookup(ego_check_exists_lookup); updateStepData(); }); }); }); }) .catch((err) => { console.log(err); }); }) .catch((err) => { console.log(err); }); } ``` -------------------------------- ### Create Conda Environment Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/README.md Creates a new Conda environment from a specified YAML file. This is typically used to set up project-specific dependencies. ```bash conda env create -f env.yaml ``` -------------------------------- ### Initialize UI Event Listeners Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/tools/template.html Sets up event listeners for various UI elements upon DOMContentLoaded. This includes toggling a README overlay, handling play/pause controls for the simulation, and updating the simulation time based on user interaction with a progress bar. It also initiates the loading of agent data. ```javascript window.addEventListener("DOMContentLoaded", () => { document.getElementById("readme-toggle-btn").addEventListener("click", () => { document.getElementById("readme-overlay").style.display = "block"; }); document.getElementById("readme-close-btn").addEventListener("click", () => { document.getElementById("readme-overlay").style.display = "none"; }); loadAgents(); document.getElementById("play_button").addEventListener("click", () => { if (!isPlaying) { isPlaying = true; let playSpeed = parseFloat(document.getElementById("play-speed").value); currentPlaySpeed = playSpeed; // if (!playSpeed || playSpeed <= 0) playSpeed = 1 // intervalId = setInterval(tick, 1000 / playSpeed) intervalId = setInterval(tick, 1000); } }); document.getElementById("pause_button").addEventListener("click", () => { isPlaying = false; clearInterval(intervalId); }); progressBar.addEventListener("input", () => { const maxSimTime = Math.floor(configData.step / $FPS$); const newTime = Math.floor((progressBar.value / 100) * maxSimTime); currentSimTime = newTime; updateStepData(); }); }); ``` -------------------------------- ### Avatar Action Definitions (Python) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Defines a dictionary of available avatar actions, each with a type and optional arguments. These actions control avatar movement, state, interactions with objects and vehicles, communication, and more. Arguments vary based on the action type, specifying distances, angles, positions, or identifiers. ```python actions = { # Movement 'move_forward': {'type': 'move_forward', 'arg1': 5.0}, # distance in meters 'turn_left': {'type': 'turn_left', 'arg1': 45.0}, # angle in degrees 'turn_right': {'type': 'turn_right', 'arg1': 45.0}, # angle in degrees 'teleport': {'type': 'teleport', 'arg1': [x, y]}, # position [x, y] 'look_at': {'type': 'look_at', 'arg1': [x, y, z]}, # target position # State changes 'sleep': {'type': 'sleep'}, 'wake': {'type': 'wake'}, 'stand': {'type': 'stand'}, 'sit': {'type': 'sit', 'arg1': [[x, y, z]]}, # seat position # Object interaction 'pick': {'type': 'pick', 'arg1': 0, 'arg2': [x, y, z]}, # hand_id, object position 'put': {'type': 'put', 'arg1': 0, 'arg2': [x, y, z]}, # hand_id, drop position 'drink': {'type': 'drink', 'arg1': 0}, # hand_id 'eat': {'type': 'eat', 'arg1': 0}, # hand_id # Vehicle interaction 'enter_bus': {'type': 'enter_bus'}, 'exit_bus': {'type': 'exit_bus'}, 'enter_bike': {'type': 'enter_bike'}, 'exit_bike': {'type': 'exit_bike'}, # Building interaction 'enter': {'type': 'enter', 'arg1': 'Coffee Shop'}, # place name 'force_enter': {'type': 'force_enter', 'arg1': 'open space'}, # force entry # Communication 'converse': {'type': 'converse', 'arg1': 'Hello!', 'arg2': 10}, # message, range # Animation 'play_animation': {'type': 'play_animation', 'arg1': 'wave'}, # animation name # Financial 'exchange': {'type': 'exchange', 'arg1': 'Agent Name', 'arg2': 50}, # target, amount # Robot control (for robot agents) 'control': {'type': 'control', 'control': torch.tensor([...])}, # joint targets # No action 'wait': {'type': 'wait'} } ``` -------------------------------- ### CustomAgent Class Implementation Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Implements a CustomAgent class inheriting from the base Agent class. This class defines custom logic for processing observations, generating actions based on target positions and environment states, and handling chat interactions. It includes parameters for movement speed and initializes agent-specific attributes like target_position. ```python from agents.agent import Agent, AgentProcess import numpy as np class CustomAgent(Agent): def __init__(self, name, pose, info, sim_path, no_react=False, debug=False, logger=None): super().__init__(name, pose, info, sim_path, no_react, debug, logger) self.target_position = None self.WALK_SPEED = 1.0 # m/s self.BIKE_SPEED = 3.0 # m/s def _process_obs(self, obs): """Process observations each turn""" self.current_rgb = obs.get('rgb') self.current_depth = obs.get('depth') self.accessible_places = obs.get('accessible_places', []) def _act(self, obs): """Generate action based on observations""" if obs['current_place'] is not None: # Exit indoor location return {'type': 'enter', 'arg1': 'open space'} if self.target_position is not None: # Calculate direction to target current_pos = np.array(self.pose[:2]) direction = self.target_position - current_pos distance = np.linalg.norm(direction) if distance < 1.0: self.target_position = None return {'type': 'wait'} target_angle = np.arctan2(direction[1], direction[0]) current_angle = self.pose[-1] angle_diff = target_angle - current_angle if abs(angle_diff) > np.deg2rad(15): if angle_diff > 0: return {'type': 'turn_left', 'arg1': np.rad2deg(angle_diff)} else: return {'type': 'turn_right', 'arg1': np.rad2deg(-angle_diff)} return {'type': 'move_forward', 'arg1': min(distance, self.WALK_SPEED)} return {'type': 'wait'} def chat(self, content): """Generate utterance for conversation""" return f"{self.name} says: Hello!" ``` -------------------------------- ### Run Tour Agent Simulation Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/README.md Executes a shell script to run the simulation with tour agents in the New York Scene. This script is located in the scripts directory. ```bash ./scripts/run_tour_agent.sh ``` -------------------------------- ### Manage Autonomous Traffic with TrafficManager Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt This snippet shows how to use the TrafficManager to handle autonomous vehicles and pedestrians. It covers retrieving positions, interacting with bus systems and shared bicycles, and stepping the traffic simulation. Dependencies include 'modules.traffic_manager', 'modules.Bus', and 'modules.SharedBicycles'. ```python from modules.traffic_manager import TrafficManager from modules import Bus, SharedBicycles traffic_manager = env.traffic_manager # Get current vehicle positions vehicle_positions = traffic_manager.get_vehicles_pos() pedestrian_positions = traffic_manager.get_pedestrians_pos() # Access bus system bus = traffic_manager.bus current_stop = bus.current_stop_name is_stopped = bus.stop_at_this_step bus_pose = bus.bus.get_global_pose() # Get bus schedule information bus_route = env.transit_info["bus"]["stops"] bus_frequency = env.transit_info["bus"]["frequency"] # Access shared bicycle system bicycles = traffic_manager.shared_bicycles nearest_bike, bike_idx = bicycles.get_nearest_bicycle(agent_xy=(10.0, 5.0)) if nearest_bike is not None: # Start using bicycle bicycles.start_timer(bike_idx, env.curr_time) # End usage and calculate cost cost = bicycles.end_timer(bike_idx, env.curr_time) # Traffic manager step (called internally by env.step()) traffic_manager.step() # Check if all traffic agents are idle all_idle = traffic_manager.spare() ``` -------------------------------- ### Action Status Definitions (Python) Source: https://context7.com/umass-embodied-agi/virtual-community/llms.txt Defines enumeration values for the status of an avatar's action. These statuses indicate whether an action is initializing, has succeeded, failed, is ongoing, or has resulted in a collision. These are used to manage agent behavior within the simulation loop. ```python from modules.avatar.utils import ActionStatus status_values = { ActionStatus.INIT: "INIT", # Initial state ActionStatus.SUCCEED: "SUCCEED", # Action completed successfully ActionStatus.FAIL: "FAIL", # Action failed ActionStatus.ONGOING: "ONGOING", # Action in progress ActionStatus.COLLIDE: "COLLIDE" # Collision detected } ``` -------------------------------- ### Create Agent Detail Panel Element (JavaScript) Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/tools/template.html Constructs a DOM element for displaying detailed information about an agent. This includes an image and a placeholder for the agent's name. ```javascript function createDetailPanel(agentName) { const safeId = agentName.replace(/\s+/g, "_ "); const container = document.createElement("div"); container.className = "media"; container.id = `on_screen_det_content-${safeId}`; container.style.backgroundColor = "#EEEEEE"; container.style.padding = "1em 1em 1em 1em"; container.style.borderRadius = "1em"; container.style.display = "none"; const link = document.createElement("a"); const detailImg = document.createElement("img"); detailImg.src = `$AvatarImgsPath$/${agentName}.png`; detailImg.style.width = "5em"; link.appendChild(detailImg); link.style.marginLeft = "1em"; const mediaBody = document.createElement("div"); mediaBody.className = "media-body"; const rowDiv = document.createElement("div"); rowDiv.className = "row"; const nameCol = document.createElement("div"); nameCol.className = "col-md-6 d-flex align-items-center"; const h2 = document.createElement("h2"); h2.id = `name__${safeId}`; h2.style.marginTop = "0.7em"; h2.style.fontSize = "1.85em"; h2.innerHTML = ``` -------------------------------- ### Load Agent First-Person and Third-Person Views Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/tools/template.html Fetches and displays the first-person (ego) and third-person (tp) view images for a selected agent at a given simulation step. It checks for the existence of the image files and updates the `src` attribute of the corresponding image elements. If an image is not found, the `src` is set to an empty string. ```javascript if (currentAgentForDetail !== "") { const safeId = currentAgentForDetail.replace(/\s+/g, "_"); /* ----------- First person view ----------- */ const dvEgo = document.getElementById(`detail-tpego-${safeId}`); const egoStep = egoCheckExistsLookup.getValue(currentAgentForDetail, actualStep); const egoPath = `$TpegoFolderPath$/${currentAgentForDetail}/rgb_${egoStep.toString().padStart(6, "0")}.png`; fetch(egoPath, { method: "HEAD" }).then((r) => { dvEgo.src = r.ok ? egoPath : ""; }).catch(() => { dvEgo.src = ""; }); /* ----------- Third person view ------------ */ const dvTP = document.getElementById(`detail-tp-${safeId}`); const tpPath = egoPath.replace("/ego/", "/tp/"); // same step, other folder fetch(tpPath, { method: "HEAD" }).then((r) => { dvTP.src = r.ok ? tpPath : ""; }).catch(() => { dvTP.src = ""; }); } ``` -------------------------------- ### Generate Character Information JSON Source: https://github.com/umass-embodied-agi/virtual-community/blob/master/assets/prompt_character_gen_generate_instructions.txt This snippet defines the structure for generating detailed character information in JSON format. It specifies nine keys for each character: 'age', 'values', 'learned', 'working_place', 'living_place', 'currently', 'lifestyle', 'known_places', and 'initial_cash_value'. The 'learned' field requires grounding to available scene places for non-famous characters. ```json { "age": "copy from initial information of the character", "values": "contains a list of 2-3 Schwartz Basic Values from ['self-direction', 'stimulation', 'hedonism', 'achievement', 'power', 'security', 'conformity', 'tradition', 'benevolence', 'universalism']", "learned": "for famous person like Elon Musk, retrieve from wikipedia, e.g. I am a businessman and investor known for my key roles in the space company SpaceX and the automotive company Tesla, Inc. Other involvements include ownership of X Corp., the company that operates the social media platform X (formerly known as Twitter), and my role in the founding of the Boring Company, xAI, Neuralink, and OpenAI. I am one of the wealthiest individuals in the world. For non-famous person, describe the primary role first and the primary role must be grounded to the scene's available places! This means that if the character has a job or he or she's a student, the working place must exist in the scene. And be specific on the working place (for example, a local firm is not allowed, it must be a firm exists in the scene)! For examples, research scientist at xxx company (xxx company must exist in the scene's available places), doctor at xxx hospital (xxx hospital must exist in the scene's available places), hotel manager at xxx hotel (xxx hotel must exist in the scene's available places), undergraduate student at xxx college (xxx college must exist in the scene's available places), professor at xxx college (must be grounded similarly), cashier at xxx shop (xxx shop must exist in the scene). Only for visitors or tourists at the $scene_name$, working place is not needed. Then decribe the hobbies and other traits. For example in the MIT campus, I am an undergraduate student at MIT College of Computing studying computer science and I like streaming games on Twitch. I love to connect with people and explore new ideas. Places like 'MIT College of Computing' must exist in the scene places", "working_place": "for non-famous character, choose an appropriate place for working in the scene if the character has a job or is a student, else return null. This should be based on the 'learned' information. If the character has a job or is a student, it must have a valid working_place, cannot be null. For famous character, if the 'learned' information doesn't reveal a working place, it's okay to be null", "living_place": "choose an appropriate place of coarse type accommodation for the character to live, for example, most characters like should live in the apartment and house of type real_estate_agency because they are local, only students should live in the dormitory or apartment/house but not hotel, and only visitors and guests live in a temporary place like the hotel and lodging", "currently": "describe the current goal for the person, think about what he or she might want to do in $scene_name$. For famous person like Elon Musk, e.g. I will collaborate with The Picower Institute for Learning and Memory for a project in Neuralink. A requirement is that places like 'The Picower Institute for Learning and Memory' must exist in the scene. For ordinary person like that student mentioned above, e.g. I want to get an A in the Advanced Data Structures course. Another requirement is that the current goal must be related to the person's 'learned' key generated.", "lifestyle": "I go to bed around midnight, wake up around 08:00, eat dinner around 18:00", "known_places": "places that the character may know in the scene including any place mentioned in the 'learned' and 'currently' information, working place if there's a working place", "initial_cash_value": "given character information previously generated, return a reasonable integer from 100-1000 as the initial cash value for the character" } ```