### Install Setuptools and Pip for Gym 0.21.0
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Run these commands to fix installation errors when installing Gym version 0.21.0.
```bash
pip install setuptools==65.5.0 pip==21
```
--------------------------------
### Install FurnitureBench via PyPI
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_sim.rst.txt
Install FurnitureBench directly from the Python Package Index (PyPI). This is a convenient way to get the latest stable release.
```bash
pip install furniture-bench
```
--------------------------------
### Example: Download All Furniture Data (Medium Randomness)
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/dataset.rst.txt
Example command to download data for all furniture types with 'medium' randomness.
```bash
python furniture_bench/scripts/download_dataset.py --untar --randomness med --furniture all --out_dir ./furniture_dataset
```
--------------------------------
### Example: Download Lamp Data (Low Randomness)
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/dataset.rst.txt
Example command to download data for the 'lamp' furniture with 'low' randomness.
```bash
python furniture_bench/scripts/download_dataset.py --untar --randomness low --furniture lamp --out_dir ./furniture_dataset
```
--------------------------------
### Install Model Dependencies
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/training_and_testing.rst.txt
Run this script to install necessary dependencies for training and testing. Navigate to the furniture-bench directory first.
```bash
cd
./install_model_deps.sh
```
--------------------------------
### Upgrade Pip, Wheel, and Setuptools for Installation
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Run these commands to resolve 'python_requires' errors during local installation.
```bash
pip install --upgrade pip wheel
pip install setuptools==58
pip install --upgrade pip==22.2.2
```
--------------------------------
### Install FurnitureBench from Source
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_sim.rst.txt
Clone the furniture-bench repository and install it from source using pip. This method allows for direct modifications to the library.
```bash
# Install FurnitureBench from source
git clone https://github.com/clvrai/furniture-bench
cd furniture-bench
pip install -e .
```
--------------------------------
### Check JAX Installation
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/training_and_testing.rst.txt
Verify that JAX is installed correctly by performing a basic operation.
```python
python -c "import jax.numpy as jnp; jnp.ones((3,)); print('JAX installed successfully')"
```
--------------------------------
### Install Isaac Gym
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_sim.rst.txt
Install Isaac Gym from its source directory. Ensure you have navigated to the correct path.
```bash
cd
pip install -e python
```
--------------------------------
### Install Oculus Reader Package
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/teleoperation.html
Install the oculus-reader package if not using Docker and ADB/git-lfs are already installed.
```bash
pip install git+https://github.com/rail-berkeley/oculus_reader.git
```
--------------------------------
### Example nvcc -V output
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
This is an example of the expected output when verifying a CUDA installation using 'nvcc -V'.
```text
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__8_16:49:14_PDT_2022
Cuda compilation tools, release 11.7, V11.7.99
Build cuda_11.7.r11.7/compiler.31442593_0
```
--------------------------------
### Install Project Requirements
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Installs the necessary Python packages for the project, including dependencies for implicit Q-learning and the R3M and VIP libraries.
```bash
pip install -r implicit_q_learning/requirements.txt
pip install -e r3m
pip install -e vip
```
--------------------------------
### Data File Structure Example
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/teleoperation.rst.txt
Illustrates the directory and file structure for saved demonstration data, including .pkl files and various camera image recordings.
```text
|- 2023-01-16-10:48:51
|- 2023-01-16-10:48:51.pkl # Demonstration data (224x224 images, actions, rewards, etc.)
|- 2023-01-16-10:48:51_color_image1.mp4 # Wrist camera RGB images (1280x720)
|- 2023-01-16-10:48:51_color_image2.mp4 # Front camera RGB images (1280x720)
|- 2023-01-16-10:48:51_color_image3.mp4 # Rear camera RGB images (1280x720)
|- 2023-01-16-10:48:51_depth_image1 # Wrist camera depth images (1280x720)
|- 2023-01-16-10:48:51_depth_image2 # Front camera depth images (1280x720)
|- 2023-01-16-10:48:51_depth_image3 # Rear camera depth images (1280x720)
```
--------------------------------
### Run FurnitureSim with Assembled Furniture
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Shows how to launch FurnitureSim with pre-assembled furniture models using a command-line script. This is useful for testing or specific scenarios where starting from an assembled state is desired.
```bash
python -m furniture_bench.scripts.run_sim_env --furniture --init-assembled
```
--------------------------------
### Install Gym Version 0.21.0
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Install this specific version of Gym to resolve observation space errors.
```bash
pip install gym==0.21.0
```
--------------------------------
### Check PyTorch Installation
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/training_and_testing.rst.txt
Verify that PyTorch is installed correctly and check if GPU support is available.
```python
python -c "import torch; print(f'PyTorch installed successfully, with GPU support = {torch.cuda.is_available()}')"
```
--------------------------------
### Launch FurnitureSim Docker Container
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_sim.html
Execute the launch script to start the FurnitureSim Docker container with GPU support.
```bash
./launch_client.sh --sim-gpu
```
--------------------------------
### Install gdown
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/dataset.html
Installs the gdown package, a command-line tool for downloading files from Google Drive.
```bash
pip install gdown
```
--------------------------------
### Install Oculus Reader Package
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/teleoperation.rst.txt
Install the oculus-reader package from GitHub using pip. This is required if you are not using Docker.
```bash
pip install git+https://github.com/rail-berkeley/oculus_reader.git
```
--------------------------------
### FurnitureSim env.step() Example
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Illustrates how to use the `env.step()` function in FurnitureSim, including generating actions and processing observations, rewards, and done signals. This is essential for running simulation episodes.
```python
# Input
action: torch.Tensor or np.ndarray (shape: [num_envs, action_dim]) # Action space is 8-dimensional (3D EE delta position, 4D EE delta rotation (quaternion), and 1D gripper.Range to [-1, 1].
# Output
obs: Dictionary of observations. The keys are specified in obs_keys. The default keys are: ['color_image1', 'color_image2', 'robot_state'].
reward: torch.Tensor or np.ndarray (shape: [num_envs, 1])
done: torch.Tensor or np.ndarray (shape: [num_envs, 1])
info: Dictionary of additional information.
env = gym.make(
"FurnitureSim-v0",
furniture='one_leg',
num_envs=1,
)
ac = torch.tensor(env.action_space.sample()).float().to('cuda') # (1, 8) torch.Tensor
ob, rew, done, _ = env.step(ac)
print(ob.keys()) # ['color_image1', 'color_image2', 'robot_state']
print(ob['robot_state'].keys()) # ['ee_pos', 'ee_quat', 'ee_pos_vel', 'ee_ori_vel', 'gripper_width']
print(ob['color_image1'].shape) # Wrist camera of shape (1, 224, 224, 3)
print(ob['color_image2'].shape) # Front camera os shape (1, 224, 224, 3)
print(rew.shape) # (1, 1)
print(done.shape) # (1, 1)
```
--------------------------------
### Clone Furniture Bench Repository
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Clones the Furniture Bench repository from GitHub and navigates into the directory. This is the first step for server installation.
```bash
git clone https://github.com/clvrai/furniture-bench.git
cd furniture-bench
```
--------------------------------
### Install Python 3.8 Development Files
Source: https://clvrai.github.io/furniture-bench/docs/references/troubleshooting.html
Install Python 3.8 development files to resolve 'libpython3.8m.so.1.0' import errors.
```bash
sudo apt update
sudo apt-get install software-properties-common -y
sudo add-apt-repository ppa:deadsnakes/ppa -y
sudo apt update
sudo apt install python3.8-dev -y
```
--------------------------------
### Start Teleoperation Data Collection
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/teleoperation.html
Execute this script on the client computer to start teleoperation and data collection. Ensure the server daemon is running first. Mount storage if using Docker to prevent data loss.
```bash
python furniture_bench/scripts/collect_data.py --furniture --out-data-path
```
--------------------------------
### Install Obstacle
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Use the calibration tool to determine the correct pose for the 3D printed obstacle before attaching it to the table.
```python
python furniture_bench/scripts/calibration.py --target obstacle
```
--------------------------------
### Teleoperation with Real-World FurnitureBench Environment
Source: https://clvrai.github.io/furniture-bench
This example shows how to set up teleoperation for the real-world FurnitureBench environment using an Oculus controller and keyboard. It initializes the environment and a device interface to read inputs and convert them into robot actions.
```python
import furniture_bench
import gym
# Create a real-world environment.
env = gym.make("FurnitureBench-v0", furniture="chair")
# Create an input device interface for Oculus + keyboard.
device = furniture_bench.device.make_device()
obs, done = env.reset(), False
while not done:
# Read Oculus and keyboard inputs and convert into a robot action.
action = device.get_action()
obs, rew, done, info = env.step(action)
```
--------------------------------
### Test Environment for One-Leg Assembly
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Verify the environment setup by running a test with a pre-trained policy for the one-leg assembly task. This script checks camera calibration and alignment of key components.
```python
python furniture_bench/scripts/calibration.py --target one_leg
```
--------------------------------
### Example: Evaluate BC Policy with ResNet18
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/training_and_testing.html
An example command to evaluate a pre-trained BC policy using a ResNet18 encoder in the simulation environment.
```python
# E.g., evaluate a pre-trained BC policy with ResNet18 encoder
python -m run env.id=FurnitureSim-v0 env.furniture=one_leg run_prefix=one_leg_full_bc_resnet18_low_sim_1000 init_ckpt_path=checkpoints/ckpt/one_leg_full_bc_resnet18_low_sim_1000/ckpt_00000000050.pt rolf.encoder_type=resnet18 is_train=False gpu=0 env.randomness=low
```
--------------------------------
### Launch FurnitureSim Docker Image
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_sim.rst.txt
Launch the FurnitureSim client Docker image with GPU support. This command starts the simulation environment within a Docker container.
```bash
./launch_client.sh --sim-gpu
```
--------------------------------
### Start Data Collection Script
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/teleoperation.rst.txt
Execute the data collection script to begin teleoperation. Specify the furniture type and the output data path.
```bash
python furniture_bench/scripts/collect_data.py --furniture --out-data-path
```
--------------------------------
### Install Python 3.8 Development Headers
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Execute these commands to fix 'libpython3.8m.so.1.0' import errors.
```bash
sudo apt update
sudo apt-get install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa -y
sudo apt update
sudo apt install python3.8-dev
```
--------------------------------
### Rclone Configuration: New Remote Setup
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/dataset.rst.txt
Instructions for setting up a new remote in rclone for Google Drive. This involves naming the remote and selecting the drive type.
```bash
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
--------------------
Enter name for new remote.
name> furniture
--------------------
Choose a number from below, or type in your own value
Storage> 18
--------------------
Two double "Enter" to skip client_id and client_secret
--------------------
Choose a number from below, or type in your own value.
Press Enter to leave empty.
scope> 2
--------------------
Enter a value. Press Enter to leave empty.
service_account_file> "Enter"
--------------------
Edit advanced config?
y) Yes
n) No (default)
y/n> n
--------------------
Use web browser to automatically authenticate rclone with remote?
* Say Y if the machine running rclone has a web browser you can use
* Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes (default)
n) No
y/n> n
--------------------
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
rclone authorize "drive" ""
Then paste the result.
Enter a value.
config_token>
*Writer's note*
# Copy and past `rclone authorize ""` in a machine with web browser
# Login to your Google account
# Allow rclone to access your Google Drive
# Past the result to `config_token` in the terminal
--------------------
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No (default)
y/n> n
--------------------
Keep this "furniture" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
--------------------
Current remotes:
Name Type
==== ====
furniture drive
```
--------------------------------
### Example lsusb output for device detection
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Sample output from 'lsusb' command showing connected devices and their IDs.
```text
Bus 002 Device 006: ID 8086:0b07 Intel Corp. Intel(R) RealSense(TM) Depth Camera 435
Bus 002 Device 007: ID 8086:0b07 Intel Corp. Intel(R) RealSense(TM) Depth Camera 435
Bus 004 Device 008: ID 2833:0183 GenesysLogic USB3.2 Hub
Bus 004 Device 002: ID 05e3:0625 Genesys Logic, Inc. USB3.2 Hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
```
--------------------------------
### Start Data Collection with Keyboard
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/teleoperation.rst.txt
Execute the data collection script using only keyboard input for teleoperation. This is an alternative to using VR controllers.
```bash
python furniture_bench/scripts/collect_data.py --furniture --out-data-path --device keyboard
```
--------------------------------
### Initialize FurnitureSim Environment
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Demonstrates how to create a FurnitureSim environment using `gym.make` with various configuration options. This is useful for setting up simulation parameters like observation keys, image resizing, and headless mode.
```python
import furniture_bench
import gym
env = gym.make(
"FurnitureSim-v0",
furniture, # Specifies the type of furniture [lamp | square_table | desk | drawer | cabinet | round_table | stool | chair | one_leg].
num_envs=1, # Number of parallel environments.
obs_keys=None, # List of observations.
concat_robot_state=False, # Whether to return robot_state in a vector or dictionary.
resize_img=True, # If true, images are resized to 224 x 224.
headless=False, # If true, simulation runs without GUI.
compute_device_id=0, # GPU device ID for simulation.
graphics_device_id=0, # GPU device ID for rendering.
init_assembled=False, # If true, the environment is initialized with assembled furniture.
np_step_out=False, # If true, env.step() returns Numpy arrays.
channel_first=False, # If true, images are returned in channel first format.
randomness="low", # Level of randomness in the environment [low | med | high].
high_random_idx=-1, # Index of the high randomness level (range: [0-2]). Default -1 will randomly select the index within the range.
save_camera_input=False, # If true, the initial camera inputs are saved.
record=False, # If true, videos of the wrist and front cameras' RGB inputs are recorded.
max_env_steps=3000, # Maximum number of steps per episode.
act_rot_repr='quat' # Representation of rotation for action space. Options are 'quat' and 'axis'.
)
```
--------------------------------
### Initialize FurnitureBench Environment
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_bench.rst.txt
Demonstrates how to create a FurnitureBench environment with custom configurations. Images are resized to 224x224 by default.
```python
import furniture_bench
import gym
env = gym.make(
"FurnitureBench-v0",
furniture=..., # Specifies the name of furniture [lamp | square_table | desk | drawer | cabinet | round_table | stool | chair | one_leg].
resize_img=True, # If true, images are resized to 224 x 224.
manual_done=False, # If true, the episode ends only when the user presses the 'done' button.
with_display=True, # If true, camera inputs are rendered on environment steps.
draw_marker=False, # If true and with_display is also true, the AprilTag marker is rendered on display.
manual_label=False, # If true, manual labeling of the reward is allowed.
from_skill=0, # Skill index to start from (range: [0-5)). Index `i` denotes the completion of ith skill and commencement of the (i + 1)th skill.
to_skill=-1, # Skill index to end at (range: [1-5]). Should be larger than `from_skill`. Default -1 expects the full task from `from_skill` onwards.
randomness="low", # Level of randomness in the environment [low | med | high].
high_random_idx=-1, # Index of the high randomness level (range: [0-2]). Default -1 will randomly select the index within the range.
visualize_init_pose=True, # If true, the initial pose of furniture parts is visualized.
record=False, # If true, the video of the agent's observation is recorded.
manual_reset=True # If true, a manual reset of the environment is allowed.
)
```
--------------------------------
### Install gdown
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/dataset.rst.txt
Install the gdown package, which is required for downloading datasets via the provided script.
```bash
pip install gdown
```
--------------------------------
### Initialize FurnitureBench Environment
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/furniture_bench.html
Demonstrates how to create a FurnitureBench environment with various configuration options. Use this to set up the environment for data collection or specific task configurations.
```python
import furniture_bench
import gym
env = gym.make(
"FurnitureBench-v0",
furniture=..., # Specifies the name of furniture [lamp | square_table | desk | drawer | cabinet | round_table | stool | chair | one_leg].
resize_img=True, # If true, images are resized to 224 x 224.
manual_done=False, # If true, the episode ends only when the user presses the 'done' button.
with_display=True, # If true, camera inputs are rendered on environment steps.
draw_marker=False, # If true and with_display is also true, the AprilTag marker is rendered on display.
manual_label=False, # If true, manual labeling of the reward is allowed.
from_skill=0, # Skill index to start from (range: [0-5)). Index `i` denotes the completion of ith skill and commencement of the (i + 1)th skill.
to_skill=-1, # Skill index to end at (range: [1-5]). Should be larger than `from_skill`. Default -1 expects the full task from `from_skill` onwards.
randomness="low", # Level of randomness in the environment [low | med | high].
high_random_idx=-1, # Index of the high randomness level (range: [0-2]). Default -1 will randomly select the index within the range.
visualize_init_pose=True, # If true, the initial pose of furniture parts is visualized.
record=False, # If true, the video of the agent's observation is recorded.
manual_reset=True # If true, a manual reset of the environment is allowed.
)
```
--------------------------------
### Launch Client with GPU, FurnitureSim for FR3, and Local Build
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Use this command to launch the client with a GPU-enabled Docker image for Franka Research 3 (FR3) that has been locally built.
```bash
./launch_client.sh --sim-gpu --built_12_2_research3
```
--------------------------------
### Configure FurnitureSim Environment
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/furniture_sim.html
Demonstrates how to create and configure a FurnitureSim environment using gym.make. It shows various arguments for customization, such as furniture type, number of environments, image resizing, and headless mode.
```python
import furniture_bench
import gym
env = gym.make(
"FurnitureSim-v0",
furniture, # Specifies the type of furniture [lamp | square_table | desk | drawer | cabinet | round_table | stool | chair | one_leg].
num_envs=1, # Number of parallel environments.
obs_keys=None, # List of observations.
concat_robot_state=False, # Whether to return robot_state in a vector or dictionary.
resize_img=True, # If true, images are resized to 224 x 224.
headless=False, # If true, simulation runs without GUI.
compute_device_id=0, # GPU device ID for simulation.
graphics_device_id=0, # GPU device ID for rendering.
init_assembled=False, # If true, the environment is initialized with assembled furniture.
np_step_out=False, # If true, env.step() returns Numpy arrays.
channel_first=False, # If true, images are returned in channel first format.
randomness="low", # Level of randomness in the environment [low | med | high].
high_random_idx=-1, # Index of the high randomness level (range: [0-2]). Default -1 will randomly select the index within the range.
save_camera_input=False, # If true, the initial camera inputs are saved.
record=False, # If true, videos of the wrist and front cameras' RGB inputs are recorded.
max_env_steps=3000, # Maximum number of steps per episode.
act_rot_repr='quat' # Representation of rotation for action space. Options are 'quat' and 'axis'.
)
```
--------------------------------
### Demonstration File Structure
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/dataset.html
This structure details the contents of each .pkl demonstration file, including furniture name, observations (images and robot state), actions, rewards, and skill completion flags.
```python
'furniture': Furniture name, e.g., 'lamp'
'observations': List of observation dicts
{
'color_image1': Wrist camera image (224, 224, 3)
'color_image2': Front camera image (224, 224, 3)
'robot_state': {
'ee_pos': EEF position (3,)
'ee_quat': EEF orientation (4,)
'ee_pos_vel': EEF linear velocity (3,)
'ee_ori_vel': EEF angular velocity (3,)
'joint_positions': Joint positions (7,)
'joint_velocities': Joint velocities (7,)
'joint_torques': Joint torques (7,)
'gripper_width': Gripper width (1,)
}
}
'actions': List of 8-D actions
'rewards': List of rewards (1 if a furniture part is assembled; otherwise, 0)
'skills': List of skill completion flags (1 if a skill is completed; otherwise, 0)
```
--------------------------------
### Run FurnitureSim and Save Camera Input
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/furniture_sim.html
Launches the FurnitureSim environment, initializes it assembled, and saves the camera inputs from the first frame of each episode.
```bash
python -m furniture_bench.scripts.run_sim_env --furniture --init-assembled --save-camera-input
```
--------------------------------
### Launch Client with GPU, FurnitureSim, and Docker Hub Image
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Use this command to launch the client with a GPU-enabled Docker image that includes FurnitureSim and is pulled from Docker Hub.
```bash
./launch_client.sh --sim-gpu --pulled
```
--------------------------------
### Install JAX with CUDA Support
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Resolve RuntimeError related to JAX computation by installing a specific version of JAX with CUDA and cuDNN support. This is often necessary for GPU acceleration.
```bash
conda install -c anaconda cudnn=8.2.1
pip install -U jax[cuda11_cudnn82] -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
```
--------------------------------
### Interact with FurnitureBench Environment
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/furniture_bench.html
Shows how to sample an action, step the environment, and access the observation dictionary. The observation includes various camera feeds, robot states, and part poses.
```python
import furniture_bench
import gym
env = gym.make(
"FurnitureBench-v0",
furniture='one_leg',
)
ac = env.action_space.sample() # np.ndarray shape (8,)
ob, rew, done, _ = env.step(ac)
print(ob.keys()) # ['color_image1', 'color_image2', 'color_imag3', 'depth_image1', 'depth_image2', 'depth_image3', 'robot_state', 'parts_poses']
print(ob['robot_state'].keys()) # ['ee_pos', 'ee_quat', 'ee_pos_vel', 'ee_ori_vel', 'gripper_width', 'joint_positions', 'joint_velocities', 'joint_torques']
print(ob['color_image1'].shape) # Wrist camera of shape (224, 224, 3)
print(ob['depth_image1'].shape) # Wrist depth image of shape (224, 224)
```
--------------------------------
### Verify CUDA installation with nvcc
Source: https://clvrai.github.io/furniture-bench/docs/_sources/references/troubleshooting.rst.txt
Check if the NVIDIA CUDA compiler (nvcc) is accessible and report its version.
```bash
nvcc -V
```
--------------------------------
### Launch Server with Local Build
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Launch the server Docker container using a locally built image.
```bash
./launch_server.sh --built # (case 2) Local build.
```
--------------------------------
### Unlink Brew Openssl
Source: https://clvrai.github.io/furniture-bench/docs/references/troubleshooting.html
Unlinks the currently installed openssl package using Homebrew to resolve potential conflicts.
```bash
brew unlink openssl@1.1
```
--------------------------------
### Launch Client with GPU and Local Build
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Use this command to launch the client with a GPU-enabled Docker image that has been locally built.
```bash
./launch_client.sh --gpu --built
```
--------------------------------
### Launch Client with CPU and Docker Hub Image
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Use this command to launch the client with a CPU-enabled Docker image pulled from Docker Hub.
```bash
./launch_client.sh --cpu --pulled
```
--------------------------------
### Evaluate Pre-trained Policy
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Executes the evaluation script for the pre-trained policy. This script will guide through environment initialization and then run the policy.
```bash
./evaluate.sh --low
```
--------------------------------
### Preprocess Data for Training
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/training_and_testing.rst.txt
Convert raw demonstration data into a format suitable for BC and IQL training. Specify input and output data paths.
```python
python furniture_bench/scripts/preprocess_data.py --in-data-path --out-data-path
```
```python
# E.g., convert data in `scripted_sim_demo/one_leg` and store in `scripted_sim_demo/one_leg_processed`
python furniture_bench/scripts/preprocess_data.py --in-data-path scripted_sim_demo/one_leg --out-data-path scripted_sim_demo/one_leg_processed
```
--------------------------------
### Test Robot Reset Pose
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Execute this script in a client Docker container to verify the software setup by checking if the robot moves to its reset pose.
```python
python furniture_bench/scripts/reset.py
```
--------------------------------
### Configure Docker Image for FurnitureSim
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_sim.html
Specify the Docker image to use, either by pulling the latest version from Docker Hub or building it locally.
```bash
# Case 1: pull from Docker Hub
export CLIENT_DOCKER=furniturebench/client-gpu:latest
# Case 2: build locally
DOCKER_BUILDKIT=1 docker build -t client-gpu . -f docker/client_gpu.Dockerfile
export CLIENT_DOCKER=client-gpu
```
--------------------------------
### Configure rclone for Google Drive
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/dataset.html
Interactive setup for rclone to connect to Google Drive. This involves naming the remote, selecting the storage type, and authenticating.
```bash
No remotes found, make a new one?
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
--------------------
Enter name for new remote.
name> furniture
--------------------
Choose a number from below, or type in your own value
Storage> 18
--------------------
Two double "Enter" to skip client_id and client_secret
--------------------
Choose a number from below, or type in your own value.
Press Enter to leave empty.
scope> 2
--------------------
Enter a value. Press Enter to leave empty.
service_account_file> "Enter"
--------------------
Edit advanced config?
y) Yes
n) No (default)
y/n> n
--------------------
Use web browser to automatically authenticate rclone with remote?
* Say Y if the machine running rclone has a web browser you can use
* Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.
y) Yes (default)
n) No
y/n> n
--------------------
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
rclone authorize "drive" ""
Then paste the result.
Enter a value.
config_token>
*Writer's note*
# Copy and past `rclone authorize "drive" ""` in a machine with web browser
# Login to your Google account
# Allow rclone to access your Google Drive
# Past the result to `config_token` in the terminal
--------------------
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No (default)
y/n> n
--------------------
Keep this "furniture" remote?
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y
--------------------
Current remotes:
Name Type
==== ====
furniture drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
--------------------
```
--------------------------------
### Evaluate Pre-trained IQL Policy
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/training_and_testing.rst.txt
Execute this command to evaluate a pre-trained Implicit Q-Learning policy in FurnitureSim. Ensure you are in the furniture-bench directory and specify the correct checkpoint step and run name.
```bash
cd
python implicit_q_learning/test_offline.py --env_name=FurnitureSimImageFeature-v0/one_leg --config=implicit_q_learning/configs/furniture_config.py --ckpt_step=1000000 --run_name one_leg_full_iql_r3m_low_sim_1000 --randomness low
```
--------------------------------
### Set Isaac Gym Path Environment Variable
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Optionally sets the ISAAC_GYM_PATH environment variable to the absolute path of the Isaac Gym installation, if FurnitureSim is to be used.
```bash
export ISAAC_GYM_PATH=
```
--------------------------------
### Create Simulation FurnitureSim Environment
Source: https://clvrai.github.io/furniture-bench
This snippet demonstrates how to create a simulation environment using FurnitureSim for a specified furniture type. It requires the furniture_bench and gym libraries.
```python
import furniture_bench
import gym
# Create a simulation environment.
env = gym.make("FurnitureSim-v0", furniture="one_leg")
obs, done = env.reset(), False
while not done:
action = env.action_space.sample()
obs, rew, done, info = env.step(action)
```
--------------------------------
### Launch Client Docker Image
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_bench.rst.txt
Execute the launch script for the client Docker image. Specify whether to use GPU or CPU, and whether the image was built locally or pulled from Docker Hub.
```bash
# GPU image + locally built
./launch_client.sh --gpu --built
# CPU image + pulled from Docker Hub
./launch_client.sh --cpu --pulled
# GPU image with FurnitureSim + pulled from Docker Hub
./launch_client.sh --sim-gpu --pulled
```
--------------------------------
### Set Environment Variables for Docker
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_sim.rst.txt
Set the absolute paths for the furniture-bench repository and Isaac Gym installation. These are required for the Docker environment to locate necessary files.
```bash
# Set the absolute path to the furniture-bench repo
export FURNITURE_BENCH=
# Set the absolute path to Isaac Gym
export ISAAC_GYM_PATH=
```
--------------------------------
### Create Real-World FurnitureBench Environment
Source: https://clvrai.github.io/furniture-bench
Use this snippet to create an instance of the real-world FurnitureBench environment for a specific furniture type. It requires the furniture_bench and gym libraries.
```python
import furniture_bench
import gym
# Create a real-world environment.
env = gym.make("FurnitureBench-v0", furniture="one_leg")
obs, done = env.reset(), False
while not done:
action = env.action_space.sample()
obs, rew, done, info = env.step(action)
```
--------------------------------
### Run Camera April Script in Client Container
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Execute this command within the client Docker container to verify camera installation and connections. Ensure cameras are correctly displayed and oriented.
```bash
cd /furniture-bench
python furniture_bench/scripts/run_cam_april.py
```
--------------------------------
### Launch FurnitureSim with High-Resolution Camera Input
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Launches the FurnitureSim environment with the 'FurnitureSimFull-v0' ID, enabling high-resolution camera inputs. This is useful for capturing detailed visual data during simulation.
```bash
python -m furniture_bench.scripts.run_sim_env --furniture --init-assembled --save-camera-input --env-id FurnitureSimFull-v0 --high-res
```
--------------------------------
### Start Teleoperation with Keyboard Control
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/teleoperation.html
Use this command to enable keyboard-only teleoperation. Oculus Quest 2 may prompt for connection permission; ensure 'Allow' is pressed if controllers are unresponsive.
```bash
python furniture_bench/scripts/collect_data.py --furniture --out-data-path --device keyboard
```
--------------------------------
### Launch FurnitureSim with Automated Assembly Script
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Launches the FurnitureSim environment and utilizes a scripted policy for automated assembly of specified furniture types. This is useful for generating scripted demonstrations.
```bash
python -m furniture_bench.scripts.run_sim_env --furniture --scripted
```
--------------------------------
### Launch Server with Local Build for FR3
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Launch the server Docker container using a locally built image specifically for the Franka Research 3 (FR3) robot.
```bash
./launch_server.sh --built-research3 # (case 3) Local build for Franka Research 3 (FR3)
```
--------------------------------
### Visualize Robot Trajectory from PKL File
Source: https://clvrai.github.io/furniture-bench/docs/tutorials/furniture_bench.html
Displays the robot's trajectory saved in a .pkl file, showing front and wrist camera views. Requires pre-recorded trajectory files.
```python
python furniture_bench/scripts/show_trajectory.py --data-path 00149.pkl
```
--------------------------------
### Launch FurnitureSim for Teleoperation
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Launches the FurnitureSim environment configured for teleoperation. By default, it supports both keyboard and Oculus Quest 2 input devices. Specify '--input-device' to use only one.
```bash
python -m furniture_bench.scripts.collect_data --furniture --out-data-path --is-sim
```
--------------------------------
### Launch Server Docker Container
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_bench.rst.txt
Launch the server Docker container. Use `--pulled` to pull the image or `--built` to use a locally built image.
```bash
./launch_server.sh --pulled # (case 1) Docker pull.
```
```bash
./launch_server.sh --built # (case 2) Local build.
```
--------------------------------
### Collect Demonstration Data with Scripted Policy
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_sim.rst.txt
Collects demonstration data using a scripted policy in a simulated environment. Specify the furniture type, output path, and compute/graphics device IDs. Use --headless for environments without a display.
```bash
python -m furniture_bench.scripts.collect_data --furniture --scripted --is-sim --out-data-path --compute-device-id --graphics-device-id --num-demos --headless
```
```bash
# E.g., collect 100 demonstrations for one_leg assembly
python -m furniture_bench.scripts.collect_data --furniture one_leg --scripted --is-sim --out-data-path scripted_sim_demo --compute-device-id 0 --graphics-device-id 0 --num-demos 100 --headless
```
--------------------------------
### FurnitureBench env.step() API Usage
Source: https://clvrai.github.io/furniture-bench/docs/_sources/tutorials/furniture_bench.rst.txt
Shows how to interact with the environment using the step function. The observation dictionary contains various image and state data.
```python
import furniture_bench
import gym
import numpy as np
env = gym.make(
"FurnitureBench-v0",
furniture='one_leg',
)
ac = env.action_space.sample() # np.ndarray shape (8,)
ob, rew, done, _ = env.step(ac)
print(ob.keys()) # ['color_image1', 'color_image2', 'color_imag3', 'depth_image1', 'depth_image2', 'depth_image3', 'robot_state', 'parts_poses']
print(ob['robot_state'].keys()) # ['ee_pos', 'ee_quat', 'ee_pos_vel', 'ee_ori_vel', 'gripper_width', 'joint_positions', 'joint_velocities', 'joint_torques']
print(ob['color_image1'].shape) # Wrist camera of shape (224, 224, 3)
print(ob['depth_image1'].shape) # Wrist depth image of shape (224, 224)
```
--------------------------------
### Set Furniture Bench Environment Variables
Source: https://clvrai.github.io/furniture-bench/docs/_sources/getting_started/installing_furniture_bench.rst.txt
Configure essential environment variables for the Furniture Bench client. Set the absolute path to your cloned repository and optionally to Isaac Gym if using FurnitureSim.
```bash
export FURNITURE_BENCH=
# (Optional) If you want to use FurnitureSim
export ISAAC_GYM_PATH=
# (Optional) Environment variable for extra mounting (e.g., for data collection)
export HOST_DATA_MOUNT=
export CONTAINER_DATA_MOUNT=
```
--------------------------------
### Build Server Docker Image
Source: https://clvrai.github.io/furniture-bench/docs/getting_started/installing_furniture_bench.html
Builds the server Docker image from the source code using Docker BuildKit.
```bash
DOCKER_BUILDKIT=1 docker build -t server . -f docker/server.Dockerfile
```