### Install VoD Prediction Toolkit

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/vod.md

Install the `vod-devkit` from GitHub for access to examples and source code, or from PyPI for a simpler installation.

```bash
git clone git@github.com:tudelft-iv/view-of-delft-prediction-devkit.git
cd vod-devkit
pip install -e .
```

```bash
pip install vod-devkit
```

--------------------------------

### Install nuScenes Toolkit

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuscenes.md

Install the nuScenes devkit from GitHub for access to examples and source code, or use pip for a simpler installation.

```bash
git clone git@github.com:nutonomy/nuscenes-devkit.git
cd nuscenes-devkit/setup
pip install -e .
```

```bash
pip install nuscenes-devkit
```

--------------------------------

### Install nuplan-devkit from GitHub

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuplan.md

Install the nuplan-devkit by cloning the GitHub repository and installing dependencies. This method is recommended for accessing examples and source code. Additional requirements like pytorch-lightning may also be needed.

```bash
git clone git@github.com:nutonomy/nuplan-devkit.git
cd nuplan-devkit
pip install -r requirements.txt
pip install -e .

pip install pytorch-lightning
```

--------------------------------

### Install ScenarioNet and Dependencies

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Installs ScenarioNet and its dependency, the MetaDrive Simulator, using conda and pip. Verifies installation by running example scripts.

```bash
conda create -n scenarionet python=3.9
conda activate scenarionet

git clone https://github.com/metadriverse/metadrive.git
cd metadrive
pip install -e .

cd ..
git clone https://github.com/metadriverse/scenarionet.git
cd scenarionet
pip install -e .

python -m metadrive.examples.profile_metadrive
python -m scenarionet.list
```

--------------------------------

### Install TensorFlow and Protobuf

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/example.md

Install TensorFlow and Protobuf for Waymo data parsing. Use conda for protobuf if pip installation fails.

```bash
pip install tensorflow==2.11.0
conda install protobuf==3.20
```

--------------------------------

### Install MetaDrive and ScenarioNet

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/simulation.ipynb

Installs the MetaDrive and ScenarioNet libraries. This step is typically required when running in a Colab environment or for a fresh local installation.

```python
#@title Collect the MetaDrive & ScenarioNet
# NOTE: If you are running this notebook locally with installtion finished, this step is not required.
RunningInCOLAB = 'google.colab' in str(get_ipython()) # Detect if it is running in Colab
if RunningInCOLAB:
    %pip install git+https://github.com/metadriverse/metadrive.git
    %pip install git+https://github.com/metadriverse/scenarionet.git
```

--------------------------------

### Install nuplan-devkit from PyPI

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuplan.md

Install the nuplan-devkit directly from the Python Package Index (PyPI) using pip. This is a simpler installation method.

```bash
pip install nuplan-devkit
```

--------------------------------

### Install ScenarioNet and MetaDrive Simulator

Source: https://github.com/metadriverse/scenarionet/blob/main/README.md

Instructions for setting up a Conda environment and installing MetaDrive Simulator and ScenarioNet from source. Ensure you are in the desired host folder before cloning.

```bash
# create environment
conda create -n scenarionet python=3.9
conda activate scenarionet

# Install MetaDrive Simulator
cd ~/  # Go to the folder you want to host these two repos.
git clone https://github.com/metadriverse/metadrive.git
cd metadrive
pip install -e.

# Install ScenarioNet
cd ~/  # Go to the folder you want to host these two repos.
git clone https://github.com/metadriverse/scenarionet.git
cd scenarionet
pip install -e .
```

--------------------------------

### Build Documentation Locally

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/README.md

Run this script to build the documentation locally. Ensure you are in the scenarionet directory and have the necessary dependencies installed.

```bash
cd scenarionet
pip install -e .[doc]
cd documentation
make html
```

--------------------------------

### Clone and Install ScenarioNet

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Clone the ScenarioNet repository using SSH and install it in editable mode. This is the recommended method for installing ScenarioNet.

```bash
git clone git@github.com:metadriverse/scenarionet.git
cd scenarionet
pip install -e .
```

--------------------------------

### Clone and Install MetaDrive from Source (HTTPS)

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Clone the MetaDrive repository using HTTPS and install it in editable mode. This method is also recommended for development.

```bash
git clone https://github.com/metadriverse/metadrive.git
cd metadrive
pip install -e.
```

--------------------------------

### Alternative MetaDrive Installation

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Install MetaDrive directly from its GitHub repository using pip without cloning. This is an alternative to cloning the repository and installing from source.

```bash
pip install git+https://github.com/metadriverse/metadrive.git
```

--------------------------------

### Alternative ScenarioNet Installation

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Install ScenarioNet directly from its GitHub repository using pip without cloning. This method is an alternative if you do not need to access the source code.

```bash
pip install git+https://github.com/metadriverse/scenarionet.git
```

--------------------------------

### Clone and Install MetaDrive from Source (SSH)

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Clone the MetaDrive repository using SSH and install it in editable mode. This method is recommended for development as it exposes the source code.

```bash
git clone git@github.com:metadriverse/metadrive.git
cd metadrive
pip install -e.
```

--------------------------------

### Verify MetaDrive Installation

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Run this command to check if MetaDrive has been successfully installed. Ensure you are not in a directory containing a subfolder named './metadrive'.

```bash
python -m metadrive.examples.profile_metadrive
```

--------------------------------

### Run Scenario Simulation Check

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use this command to verify that all scenarios in a database can be loaded and run within the simulator. This is recommended before starting closed-loop training or testing. Specify the database path and optionally an error file path for failed scenarios.

```text
python -m scenarionet.check_simulation [-h] --database_path DATABASE_PATH
                       [--error_file_path ERROR_FILE_PATH] [--overwrite]
                       [--num_workers NUM_WORKERS] [--random_drop]
```

--------------------------------

### Install MetaDrive Stable Version

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Install the latest stable version of MetaDrive using pip. This method hides the source code and is suitable for users who do not need to modify the simulator.

```bash
pip install "metadrive-simulator>=0.4.1.1"
```

--------------------------------

### Generate PG Database

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/PG.md

Run this command to generate PG scenarios. Specify the output directory and the desired number of scenarios. Ensure MetaDrive is installed.

```bash
python -m scenarionet.convert_pg -d /path/to/pg_database --num_scenarios 1000
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/install.md

Create a new conda environment named 'scenarionet' with Python 3.9 and activate it. This ensures a clean environment for installations.

```bash
conda create -n scenarionet python==3.9
conda activate scenarionet
```

--------------------------------

### Waymo Convertor Example Signature

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/new_data.md

Example signature for a convertor function, demonstrating input types (scenario object and version string) and the expected output type (ScenarioDescription).

```python
convert_waymo(scenario:scenario_pb2.Scenario(), version:str)->metadrive.scenario.ScenarioDescription
```

--------------------------------

### Download Waymo Motion Dataset

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/waymo.md

Download the entire Waymo motion dataset or specific parts using gsutil. Ensure you have gsutil installed and configured.

```bash
gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario" .
```

```bash
gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s" .
```

--------------------------------

### Split Database with Scenarionet

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use the split command to extract a subset of scenarios from an existing database to create a new one, typically for training/testing sets. Specify the source database with --from and the destination with --to. Control the number of scenarios with --num_scenarios, starting index with --start_index, and selection method with --random.

```bash
python -m scenarionet.split [-h] --from FROM --to TO [--num_scenarios NUM_SCENARIOS]
            [--start_index START_INDEX] [--random] [--exist_ok]
            [--overwrite]

Build a new database containing a subset of scenarios from an existing
database.

optional arguments:
  -h, --help            show this help message and exit
  --from FROM           Which database to extract data from.
  --to TO               The name of the new database. It will create a new
                        directory to store dataset_summary.pkl and
                        dataset_mapping.pkl. If exists_ok=True, those two .pkl
                        files will be stored in an existing directory and turn
                        that directory into a database.
  --num_scenarios NUM_SCENARIOS
                        how many scenarios to extract (default: 30)
  --start_index START_INDEX
                        which index to start
  --random              If set to true, it will choose scenarios randomly from
                        all_scenarios[start_index:]. Otherwise, the scenarios
                        will be selected sequentially
  --exist_ok            Still allow to write, if the to_folder exists already.
                        This write will only create two .pkl files and this
                        directory will become a database.
  --overwrite           When exists ok is set but summary.pkl and map.pkl
                        exists in existing dir, whether to overwrite both
                        files
```

--------------------------------

### Split Database into Test Set

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/example.md

Generate a test set from a source database, specifying the number of scenarios and the starting index for selection. This ensures the test set contains distinct scenarios from the training set.

```bash
python -m scenarionet.split --from /path/toexp_converted/ --to /path/to/exp_test --num_scenarios 21 --start_index 40
```

--------------------------------

### Convert VoD Dataset to ScenarioNet

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use this script to convert the View-of-Delft (VoD) dataset into ScenarioNet's scenario descriptions. Ensure `vod-devkit` is installed and VoD source data is downloaded.

```text
python -m scenarionet.convert_vod [-h] [--database_path DATABASE_PATH]
                  [--dataset_name DATASET_NAME]
                  [--split
{v1.0-trainval,v1.0-test,train,train_val,val,test}]
                  [--dataroot DATAROOT] [--map_radius MAP_RADIUS]
                  [--future FUTURE] [--past PAST] [--overwrite]
                  [--num_workers NUM_WORKERS]

Build database from VOD scenarios

optional arguments:
  -h, --help            show this help message and exit
  --database_path DATABASE_PATH, -d DATABASE_PATH
                        directory, The path to place the data
  --dataset_name DATASET_NAME, -n DATASET_NAME
                        Dataset name, will be used to generate scenario files
  --split
    {v1.0-trainval,v1.0-test,train,train_val,val,test}
                        Which splits of VOD data should be used. If set to
                        ['v1.0-trainval', 'v1.0-test'], it will
                        convert the full log into scenarios with 20 second episode
                        length. If set to ['train', 'train_val', 'val', 'test'],
                        it will convert segments used for VOD prediction challenge
                        to scenarios, resulting in more converted scenarios.
                        Generally, you should choose this parameter from
                        ['v1.0-trainval', 'v1.0-test'] to get complete
                        scenarios for planning unless you want to use the
                        converted scenario files for prediction task.
  --dataroot DATAROOT   The path of vod data
  --map_radius MAP_RADIUS The size of map
  --future FUTURE       3 seconds by default. How many future seconds to
                        predict. Only available if split is chosen from
                        ['train', 'train_val', 'val', 'test']
  --past PAST           0.5 seconds by default. How many past seconds are
                        used for prediction. Only available if split is
                        chosen from ['train', 'train_val', 'val', 'test']
  --overwrite           If the database_path exists, whether to overwrite it
  --num_workers NUM_WORKERS number of workers to use
```

--------------------------------

### Generate Procedural Scenarios

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Generates synthetic scenarios using MetaDrive's procedural generation. Specify the output database directory. Use --num_scenarios to set the count, --num_workers for parallel generation, and --start_index to control the starting scenario index.

```bash
python -m scenarionet.convert_pg \
    -d /path/to/pg_database
```

```bash
python -m scenarionet.convert_pg \
    -d /path/to/pg_large_database \
    --num_scenarios 1000 \
    --num_workers 8
```

```bash
python -m scenarionet.convert_pg \
    -d /path/to/pg_extended \
    --num_scenarios 500 \
    --start_index 1000 \
    --num_workers 4
```

--------------------------------

### Call read_dataset_summary Function

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Example of calling the `read_dataset_summary` function with a dataset path. It returns a summary dictionary, a list of scenario IDs, and a mapping of scenario IDs to their file locations.

```python
dataset_summary, scenario_ids, mapping = read_dataset_summary(dataset_path=waymo_data)
```

--------------------------------

### Convert Waymo Dataset

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use this command to convert Waymo dataset scenarios into ScenarioNet's internal description format. Specify paths for the database and raw data, and optionally set the number of workers and overwrite behavior. Control which files are processed using start index and number of files.

```bash
python -m scenarionet.convert_waymo [-h] [--database_path DATABASE_PATH]
                        [--dataset_name DATASET_NAME] [--version VERSION]
                        [--overwrite] [--num_workers NUM_WORKERS]
                        [--raw_data_path RAW_DATA_PATH]
                        [--start_file_index START_FILE_INDEX]
                        [--num_files NUM_FILES]
```

--------------------------------

### Initialize Google Cloud SDK

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/waymo.md

Log in to your Google Cloud account using the gcloud init command to access and download data from Google Cloud Storage.

```bash
gcloud init
```

--------------------------------

### Split Database into Training Set

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/example.md

Create a training set from a source database by specifying the number of scenarios to include. The `--from` and `--to` arguments define the source and destination directories, respectively.

```bash
python -m scenarionet.split --from /path/to/exp_converted/ --to /path/to/exp_train --num_scenarios 40
```

--------------------------------

### Replay Scenarios with Semantic Rendering (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Launches the MetaDrive simulator to replay scenarios using semantic rendering mode.

```bash
python -m scenarionet.sim \
    -d /path/to/database \
    --render semantic
```

--------------------------------

### Configure Environment and Import Modules

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Imports necessary modules from MetaDrive and sets the SDL video driver to 'dummy' to prevent the pygame window from appearing. Includes utility functions for closing the engine and pulling assets.

```python
#@title Make some configurations and import some modules
from metadrive.engine.engine_utils import close_engine
close_engine()
from metadrive.pull_asset import pull_asset
pull_asset(False)
# NOTE: usually you don't need the above lines. It is only for avoiding a potential bug when running on colab

from metadrive.engine.asset_loader import AssetLoader
from metadrive.policy.replay_policy import ReplayEgoCarPolicy
from metadrive.envs.scenario_env import ScenarioEnv
import os

os.environ["SDL_VIDEODRIVER"] = "dummy" # Hide the pygame window
```

--------------------------------

### Configure Simulation Environment

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/simulation.ipynb

Sets up configurations and imports necessary modules for the MetaDrive simulation. It includes options for 3D rendering and specifies paths for Waymo and nuScenes datasets.

```python
#@title Make some configurations and import some modules
from metadrive.engine.engine_utils import close_engine
close_engine()
from metadrive.pull_asset import pull_asset
pull_asset(False)
# NOTE: usually you don't need the above lines. It is only for avoiding a potential bug when running on colab

from metadrive.engine.asset_loader import AssetLoader
from metadrive.policy.replay_policy import ReplayEgoCarPolicy
from metadrive.envs.scenario_env import ScenarioEnv
import os

threeD_render=False # turn on this to enable 3D render. It only works when you have a screen and not running on Colab.
threeD_render=threeD_render and not RunningInCOLAB
os.environ["SDL_VIDEODRIVER"] = "dummy" # Hide the pygame window
waymo_data =  AssetLoader.file_path(AssetLoader.asset_path, "waymo", unix_style=False) # Use the built-in datasets with simulator
nuscenes_data =  AssetLoader.file_path(AssetLoader.asset_path, "nuscenes", unix_style=False) # Use the built-in datasets with simulator
```

--------------------------------

### Extract Traffic Function Example

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/new_data.md

Example function for extracting traffic data from NuPlan scenarios. This function is used to convert frame-centric object history to an object-centric format required by ScenarioNet.

```python
extract_traffic(scenario: NuPlanScenario, center)
```

--------------------------------

### Copy Database with Raw Data (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Creates a standalone copy of the database, including raw scenario data instead of virtual mapping.

```bash
python -m scenarionet.cp \
    --from /path/to/source_database \
    --to /path/to/standalone_copy \
    --copy_raw_data
```

--------------------------------

### Count Scenarios in Database (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Use this command to get the total number of scenarios stored in a Scenarionet database.

```bash
python -m scenarionet.num -d /path/to/database
```

--------------------------------

### Get Scenario Keys

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Retrieve the available keys from the scenario object, which represent different aspects of the scenario data.

```python
scenario.keys()
```

--------------------------------

### Replay Scenarios with 2D Renderer (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Launches the MetaDrive simulator to replay scenarios from a database using a 2D top-down rendering mode.

```bash
python -m scenarionet.sim \
    -d /path/to/database \
    --render 2D
```

--------------------------------

### Replay Scenarios with 3D Renderer (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Launches the MetaDrive simulator to replay scenarios from a database using a 3D rendering mode.

```bash
python -m scenarionet.sim \
    -d /path/to/database \
    --render 3D
```

--------------------------------

### Build VoD Database

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/vod.md

Convert raw VoD data into scenario format and build the database using the `scenarionet.convert_vod` module. Specify the database path, the data split, and the data root if it differs from the default.

```bash
python -m scenarionet.convert_vod -d /path/to/your/database --split v1.0-trainval --dataroot /vod/data/path
```

--------------------------------

### Replay Scenarios with Advanced Rendering (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Enables advanced rendering features in the MetaDrive simulator for replaying scenarios. Requires specific hardware (RTX 2060+).

```bash
python -m scenarionet.sim \
    -d /path/to/database \
    --render advanced
```

--------------------------------

### ScenarioEnv with Replay Policy (Python API)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Initializes the ScenarioEnv for reinforcement learning using ReplayEgoCarPolicy, where the ego vehicle follows a recorded trajectory. Rendering is disabled by default.

```python
from metadrive.envs.scenario_env import ScenarioEnv
from metadrive.policy.replay_policy import ReplayEgoCarPolicy

# Create environment with replay policy (ego follows recorded trajectory)
env = ScenarioEnv({
    "manual_control": False,
    "reactive_traffic": False,
    "use_render": False,  # Set True for 3D rendering
    "agent_policy": ReplayEgoCarPolicy,
    "data_directory": "/path/to/database",
    "num_scenarios": 10,  # Number of scenarios to load
})

# Run simulation
for scenario_idx in range(3):
    obs, info = env.reset(seed=scenario_idx)
    frames = []
    done = False

    while not done:
        # For replay policy, action is ignored
        action = [0.0, 0.0]  # [throttle, steering]
        obs, reward, terminated, truncated, info = env.step(action)

        # Capture frame for visualization
        frame = env.render(mode="top_down", film_size=(1200, 1200))
        frames.append(frame)

        done = terminated or truncated

    print(f"Scenario {scenario_idx}: {len(frames)} steps")

env.close()
```

--------------------------------

### Create Scenario Environment

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/simulation.ipynb

Initializes a ScenarioEnv environment for simulating scenarios. It is configured for non-manual control, no reactive traffic, and uses ReplayEgoCarPolicy with a single Waymo scenario.

```python
env = ScenarioEnv(
    {
        "manual_control": False,
        "reactive_traffic": False,
        "use_render": threeD_render,
        "agent_policy": ReplayEgoCarPolicy,
        "data_directory": waymo_data,
        "num_scenarios": 1
    }
)
```

--------------------------------

### Replay Scenarios with 2D Rendering

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/example.md

Load a database into the MetaDrive simulator and replay scenarios using the 2D renderer. This is useful for quick visualization and debugging.

```bash
python -m scenarionet.sim -d /path/to/exp_converted --render 2D
```

--------------------------------

### Replay Specific Scenario by Index (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Replays a single scenario from the database at a specified index using the 2D renderer.

```bash
python -m scenarionet.sim \
    -d /path/to/database \
    --render 2D \
    --scenario_index 5
```

--------------------------------

### ScenarioEnv with Interactive Control (Python API)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Sets up ScenarioEnv for interactive control where traffic reacts to the ego vehicle. 3D rendering is enabled by default. Suitable for RL training.

```python
from metadrive.envs.scenario_env import ScenarioEnv
import numpy as np

# Create environment with interactive control
env = ScenarioEnv({
    "manual_control": False,
    "reactive_traffic": True,  # Traffic reacts to ego
    "use_render": True,  # Enable 3D rendering
    "data_directory": "/path/to/database",
    "num_scenarios": 50,
    "start_scenario_index": 0,
})

# Training loop example
for episode in range(100):
    obs, info = env.reset(seed=episode % 50)
    total_reward = 0

    for step in range(1000):
        # Action: [throttle/brake (-1 to 1), steering (-1 to 1)]
        action = np.array([0.5, 0.0])  # Replace with your policy

        obs, reward, terminated, truncated, info = env.step(action)
        total_reward += reward

        if terminated or truncated:
            break

    print(f"Episode {episode}: reward={total_reward:.2f}, steps={step}")

env.close()
```

--------------------------------

### Import ScenarioNet Reading Utilities

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Imports the `read_dataset_summary` and `read_scenario` functions from the ScenarioNet library, which are essential for accessing and processing scenario data.

```python
from scenarionet import read_dataset_summary, read_scenario
```

--------------------------------

### Replay Scenarios with 3D Rendering

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/example.md

Load a database into the MetaDrive simulator and replay scenarios using the 3D renderer for a more immersive experience. Ensure your system meets the requirements for 3D rendering.

```bash
python -m scenarionet.sim -d /path/to/exp_converted --render 3D
```

--------------------------------

### Copy Database (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Copies a Scenarionet database, preserving virtual mapping. Use the --remove_source flag to move the database instead.

```bash
python -m scenarionet.cp \
    --from /path/to/source_database \
    --to /path/to/copied_database
```

```bash
python -m scenarionet.cp \
    --from /path/to/source_database \
    --to /path/to/new_location \
    --remove_source
```

--------------------------------

### Split Scenario Databases

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Extracts subsets of scenarios from a database to create training, test, or validation sets. Specify the source database with --from, the destination with --to, and the number of scenarios with --num_scenarios. Options include setting a --start_index and creating a --random split.

```bash
python -m scenarionet.split \
    --from /path/to/full_database \
    --to /path/to/train_set \
    --num_scenarios 40
```

```bash
python -m scenarionet.split \
    --from /path/to/full_database \
    --to /path/to/test_set \
    --num_scenarios 20 \
    --start_index 40
```

```bash
python -m scenarionet.split \
    --from /path/to/full_database \
    --to /path/to/val_set \
    --num_scenarios 100 \
    --random
```

```bash
python -m scenarionet.split \
    --from /path/to/source_db \
    --to /path/to/existing_dir \
    --num_scenarios 50 \
    --exist_ok
```

--------------------------------

### Read a Scenario File

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Use the `read_scenario` function to load a scenario from a pkl file. Ensure you have the dataset path, mapping, and scenario file name.

```python
scenario_file_name = scenario_ids[0]

scenario = read_scenario(dataset_path=waymo_data, mapping=mapping, scenario_file_name=scenario_file_name)
```

--------------------------------

### Simulate Scenarios

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Load a database into the simulator to replay scenarios. Various render modes are available for visualization. Refer to the Simulation section or MetaDrive documentation for more details.

```text
python -m scenarionet.sim [-h] --database_path DATABASE_PATH
          [--render {none,2D,3D,advanced,semantic}]
          [--scenario_index SCENARIO_INDEX]

Load a database to simulator and replay scenarios

optional arguments:
  -h, --help            show this help message and exit
  --database_path DATABASE_PATH, -d DATABASE_PATH
                        The path of the database
  --render {none,2D,3D,advanced,semantic}
  --scenario_index SCENARIO_INDEX
                        Specifying a scenario to run
```

--------------------------------

### Custom Scenario Filtering (Python API)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Demonstrates the initialization of ScenarioFilter for programmatic filtering of scenarios based on various criteria. Requires importing additional modules.

```python
from scenarionet.builder.filters import ScenarioFilter
from scenarionet import read_dataset_summary, read_scenario
```

--------------------------------

### Convert nuPlan Dataset to ScenarioNet Format

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Converts nuPlan dataset (.db files) to ScenarioNet format. This command initiates the conversion process for nuPlan data.

```bash
# Convert nuPlan dataset (.db files) to ScenarioNet format.

```

--------------------------------

### Build nuScenes Database

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuscenes.md

Convert raw nuScenes data into a ScenarioNet-compatible database format. Specify the data root if it's not the default.

```python
python -m scenarionet.convert_nuscenes -d /path/to/your/database --split v1.0-mini --dataroot /nuscens/data/path
```

--------------------------------

### Count Scenarios in a Database

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/example.md

Use this command to determine the total number of scenarios present in a given database directory. Ensure the path to the database is correctly specified.

```bash
python -m scenarionet.num -d /path/to/exp_converted/
```

--------------------------------

### Generate Database from Error File

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use this script to create a new database by either excluding or including only broken scenarios detected by other tools. This is useful for debugging or creating clean datasets.

```text
python -m scenarionet.generate_from_error_file [-h] --database_path DATABASE_PATH --file
                               FILE [--overwrite] [--broken]

Generate a new database excluding or only including the failed scenarios
detected by 'check_simulation' and 'check_existence'

optional arguments:
  -h, --help            show this help message and exit
  --database_path DATABASE_PATH, -d DATABASE_PATH
                        The path of the newly generated database
  --file FILE, -f FILE  The path of the error file, should be xyz.json
  --overwrite           If the database_path exists, overwrite it
  --broken              By default, only successful scenarios will be picked
                        to build the new database. If turn on this flog, it
                        will generate database containing only broken
                        scenarios.
```

--------------------------------

### Build Argoverse 2 Database

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/argoverse2.md

Use this command to build the Argoverse 2 database from raw data. Ensure you replace the placeholder paths with your actual database and raw data locations.

```bash
python -m scenarionet.convert_argoverse2 -d /path/to/your/database –raw_data_path /path/to/your/raw_data
```

--------------------------------

### Simulate Multiple nuScenes Scenarios

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/simulation.ipynb

Loads multiple nuScenes scenarios into the simulator and simulates/visualizes a specified number of them. Ensure the ScenarioEnv is properly configured with data directory and desired number of scenarios.

```python
env = ScenarioEnv(
    {
        "manual_control": False,
        "reactive_traffic": False,
        "use_render": False,
        "agent_policy": ReplayEgoCarPolicy,
        "data_directory": nuscenes_data, # use nuscenes data
        "num_scenarios": 10, # load 10 scenarios
    }
)

for seed in range(2): # only simulate the first 2 scenarios
    print("\nSimulate Scenario: {}".format(seed))
    o, _ = env.reset(seed=seed)
    frames = []
    for i in range(1, 100000):
        o, r, tm, tc, info = env.step([1.0, 0.])
        frames.append(env.render(mode="top_down",film_size=(4000, 4000), screen_size=(500, 500)))
        if tm or tc:
            make_GIF(frames, name="scenario_{}.gif".format(seed))
            break

env.close()
```

--------------------------------

### Convert NuPlan Raw Data to Database

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuplan.md

Use this command to convert raw NuPlan data, specifically from the 'mini' split, into the internal database format. The --raw_data_path argument points to the directory containing the .db files.

```bash
python -m scenarionet.convert_nuplan -d /path/to/your/database --raw_data_path ~/nuplan/dataset/nuplan-v1.1/splits/mini
```

--------------------------------

### Check Scenario Simulation (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Tests if all scenarios in the database can be successfully loaded and simulated. Errors are saved to a JSON file. Supports multi-worker processing.

```bash
python -m scenarionet.check_simulation \
    -d /path/to/database \
    --error_file_path /path/to/sim_errors.json \
    --num_workers 4
```

--------------------------------

### Convert Procedurally Generated Scenarios to ScenarioNet

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use this script to build a database from procedurally generated (PG) scenarios. These scenarios are created using rule-based generation and an IDM policy for the ego car.

```text
python -m scenarionet.convert_pg [-h] [--database_path DATABASE_PATH]
                     [--dataset_name DATASET_NAME] [--version VERSION]
                     [--overwrite] [--num_workers NUM_WORKERS]
                     [--num_scenarios NUM_SCENARIOS]
                     [--start_index START_INDEX]

Build database from synthetic or procedurally generated scenarios

optional arguments:
  -h, --help            show this help message and exit
  --database_path DATABASE_PATH, -d DATABASE_PATH
                        directory, The path to place the data
  --dataset_name DATASET_NAME, -n DATASET_NAME
                        Dataset name, will be used to generate scenario files
  --version VERSION, -v VERSION
                        version
  --overwrite           If the database_path exists, whether to overwrite it
  --num_workers NUM_WORKERS
                        number of workers to use
  --num_scenarios NUM_SCENARIOS
                        how many scenarios to generate (default: 30)
  --start_index START_INDEX
                        which index to start
```

--------------------------------

### Organize VoD Data

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/vod.md

Organize the downloaded VoD data into the specified directory structure. The `dataroot` should be set to `/vod/data/path` by default, or specified when calling VoD APIs if stored elsewhere.

```default
/vod/data/path/
├── maps/
|   └──expansion/
├── v1.0-trainval/
|   ├──attribute.json
|   ├──calibrated_sensor.json
|   ├──map.json
|   ├──log.json
|   ├──ego_pose.json
|   └──...
└── v1.0-test/
```

--------------------------------

### List Available Operations

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Lists all available operations with detailed descriptions. Use this to discover what actions can be performed.

```APIDOC
## List Available Operations

### Description
Lists all available operations with detailed descriptions.

### Method
Command Line

### Endpoint
N/A

### Parameters
None

### Request Example
```bash
python -m scenarionet.list
```

### Response
N/A (Outputs to console)
```

--------------------------------

### Convert nuPlan Dataset to ScenarioNet

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Converts nuPlan datasets to ScenarioNet format. Specify the output database directory and the raw data path. Use --num_workers for parallel conversion.

```bash
python -m scenarionet.convert_nuplan \
    -d /path/to/nuplan_database \
    --raw_data_path ~/nuplan/dataset/nuplan-v1.1/splits/mini
```

```bash
python -m scenarionet.convert_nuplan \
    -d /path/to/nuplan_full \
    --raw_data_path ~/nuplan/dataset/nuplan-v1.1/splits/train_boston \
    --num_workers 16
```

--------------------------------

### NuPlan Dataset Folder Structure

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuplan.md

This illustrates the expected directory structure for the NuPlan dataset, including maps, splits, and sensor blobs.

```text
~/nuplan
├── exp
│   └── ${USER}
│       ├── cache
│       │   └── <cached_tokens>
│       └── exp
│           └── my_nuplan_experiment
└── dataset
    ├── maps
    │   ├── nuplan-maps-v1.0.json
    │   ├── sg-one-north
    │   │   └── 9.17.1964
    │   │       └── map.gpkg
    │   ├── us-ma-boston
    │   │   └── 9.12.1817
    │   │       └── map.gpkg
    │   ├── us-nv-las-vegas-strip
    │   │   └── 9.15.1915
    │   │       └── map.gpkg
    │   └── us-pa-pittsburgh-hazelwood
    │       └── 9.17.1937
    │           └── map.gpkg
    └── nuplan-v1.1
        ├── splits
        │     ├── mini
        │     │    ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
        │     │    ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
        │     │    ├── ...
        │     │    └── 2021.10.11.08.31.07_veh-50_01750_01948.db
        │     └── train_boston
        │          ├── 2021.05.12.22.00.38_veh-35_01008_01518.db
        │          ├── 2021.06.09.17.23.18_veh-38_00773_01140.db
        │          ├── ...
        │          └── 2021.10.11.08.31.07_veh-50_01750_01948.db
        └── sensor_blobs
              ├── 2021.05.12.22.00.38_veh-35_01008_01518
              │    ├── CAM_F0
              │    │     ├── c082c104b7ac5a71.jpg
              │    │     ├── af380db4b4ca5d63.jpg
              │    │     ├── ...
              │    │     └── 2270fccfb44858b3.jpg
              │    ├── CAM_B0
              │    ├── CAM_L0
              │    ├── CAM_L1
              │    ├── CAM_L2
              │    ├── CAM_R0
              │    ├── CAM_R1
              │    ├── CAM_R2
              │    └──MergedPointCloud
              │         ├── 03fafcf2c0865668.pcd
              │         ├── 5aee37ce29665f1b.pcd
              │         ├── ...
              │         └── 5fe65ef6a97f5caf.pcd
              │
              ├── 2021.06.09.17.23.18_veh-38_00773_01140
              ├── ...
              └── 2021.10.11.08.31.07_veh-50_01750_01948
```

--------------------------------

### Run and Record Waymo Scenario Simulation

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/simulation.ipynb

Runs a Waymo scenario simulation for a specified number of steps, capturing frames for visualization. The simulation uses the configured environment and ReplayEgoCarPolicy.

```python
# @title Run Simulation

o, _ = env.reset(seed=0)
frames = []
for i in range(1, 100000):
    o, r, tm, tc, info = env.step([1.0, 0.])
    frames.append(env.render(mode="top_down",film_size=(1200, 1200)))
    if tm or tc:
        break
env.close()

make_GIF(frames)
# visualization
IImage(open("demo.gif", 'rb').read())
```

--------------------------------

### Merge Databases with Scenarionet

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Use the merge command to combine multiple existing databases into a larger one. Specify the destination database path with --to and the source databases with --from. Optional flags like --exist_ok, --overwrite, --filter_moving_dist, and --sdc_moving_dist_min can modify merge behavior.

```bash
python -m scenarionet.merge [-h] --to DATABASE_PATH --from FROM [FROM ...]
                [--exist_ok] [--overwrite] [--filter_moving_dist]
                [--sdc_moving_dist_min SDC_MOVING_DIST_MIN]

Merge a list of databases. e.g. scenario.merge --from db_1 db_2 db_3...db_n
--to db_dest

optional arguments:
-h, --help            show this help message and exit
--database_path DATABASE_PATH, -d DATABASE_PATH, --to DATABASE_PATH
                    The name of the new combined database. It will create
                    a new directory to store dataset_summary.pkl and
                    dataset_mapping.pkl. If exists_ok=True, those two .pkl
                    files will be stored in an existing directory and turn
                    that directory into a database.
--from FROM [FROM ...]
                    Which datasets to combine. It takes any number of
                    directory path as input
--exist_ok            Still allow to write, if the dir exists already. This
                    write will only create two .pkl files and this
                    directory will become a database.
--overwrite           When exists ok is set but summary.pkl and map.pkl
                    exists in existing dir, whether to overwrite both
                    files
--filter_moving_dist  add this flag to select cases with SDC moving dist >
                    sdc_moving_dist_min
--sdc_moving_dist_min SDC_MOVING_DIST_MIN
                    Selecting case with sdc_moving_dist > this value. We
                    will add more filter conditions in the future.
```

--------------------------------

### Generate Database of Broken Scenarios (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Creates a new database containing only the scenarios identified as broken in the provided error file, useful for debugging.

```bash
python -m scenarionet.generate_from_error_file \
    -d /path/to/broken_scenarios \
    --file /path/to/errors.json \
    --broken
```

--------------------------------

### Convert nuPlan Dataset

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

This command converts nuPlan dataset scenarios into ScenarioNet's internal description format. It requires the 'nuplan-devkit' and source data. Options include specifying database and raw data paths, dataset name, version, and number of workers. A `--test` flag is available for converting a single log.

```bash
python -m scenarionet.convert_nuplan [-h] [--database_path DATABASE_PATH]
                     [--dataset_name DATASET_NAME] [--version VERSION]
                     [--overwrite] [--num_workers NUM_WORKERS]
                     [--raw_data_path RAW_DATA_PATH] [--test]
```

--------------------------------

### Set NuPlan Environment Variables

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/nuplan.md

Add these export commands to your ~/.bashrc file to ensure the nuplan-devkit can locate the dataset and experiment directories.

```bash
export NUPLAN_DATA_ROOT="$HOME/nuplan/dataset"
export NUPLAN_MAPS_ROOT="$HOME/nuplan/dataset/maps"
export NUPLAN_EXP_ROOT="$HOME/nuplan/exp"
```

--------------------------------

### List Waymo Dataset Files

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Constructs the file path for the Waymo dataset within the MetaDrive asset directory and lists its contents. This helps in understanding the file structure of the Waymo scenarios.

```python
waymo_data =  AssetLoader.file_path(AssetLoader.asset_path, "waymo", unix_style=False) # Use the built-in datasets with simulator
os.listdir(waymo_data) # there are 3 waymo scenario file with a 'dataset_summary.pkl'
```

--------------------------------

### Check Scenario Existence

Source: https://github.com/metadriverse/scenarionet/blob/main/documentation/operations.md

Verify if scenarios in a database are runnable and exist on your machine. This tool checks if scenario files can be loaded, and generates an error file if broken scenarios are found. The error file can then be used with `generate_from_error_file`.

```text
python -m scenarionet.check_existence [-h] --database_path DATABASE_PATH
                          [--error_file_path ERROR_FILE_PATH] [--overwrite]
                          [--num_workers NUM_WORKERS] [--random_drop]

Check if the database is intact and all scenarios can be found and recorded in
internal scenario description

optional arguments:
  -h, --help            show this help message and exit
  --database_path DATABASE_PATH, -d DATABASE_PATH
                        Dataset path, a directory containing summary.pkl and
                        mapping.pkl
  --error_file_path ERROR_FILE_PATH
                        Where to save the error file. One can generate a new
                        database excluding or only including the failed
                        scenarios.For more details, see operation
                        'generate_from_error_file'
  --overwrite           If an error file already exists in error_file_path, whether to overwrite it
  --num_workers NUM_WORKERS
                        number of workers to use
  --random_drop         Randomly make some scenarios fail. for test only!
```

--------------------------------

### Display Dataset Summary

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Prints the `dataset_summary` dictionary, which contains metadata for each scenario, including coordinate system, timestamps, and object information.

```python
dataset_summary
```

--------------------------------

### List nuScenes Dataset Files

Source: https://github.com/metadriverse/scenarionet/blob/main/tutorial/read_established_scenarionet_dataset.ipynb

Constructs the file path for the nuScenes dataset within the MetaDrive asset directory and lists its contents. This is useful for examining the file structure of nuScenes scenarios.

```python
nuscenes_data =  AssetLoader.file_path(AssetLoader.asset_path, "nuscenes", unix_style=False) # Use the built-in datasets with simulator
os.listdir(nuscenes_data) # there are 10 nuscenes scenario file with a 'dataset_summary.pkl' and a 'dataset_summary.pkl'
```

--------------------------------

### Check for Overlapping Scenarios (CLI)

Source: https://context7.com/metadriverse/scenarionet/llms.txt

Compares two databases to identify overlapping scenarios. Use --show_id to display the IDs of overlapping scenarios.

```bash
python -m scenarionet.check_overlap \
    --d_1 /path/to/train_set \
    --d_2 /path/to/test_set
```

```bash
python -m scenarionet.check_overlap \
    --d_1 /path/to/db1 \
    --d_2 /path/to/db2 \
    --show_id
```