Trajectron++

https://github.com/stanfordasl/trajectron-plus-plus
Admin
Trajectron++ is a project for dynamically-feasible trajectory forecasting using heterogeneous data,...
Tokens:15,318
Snippets:115
Trust Score:9
License:MIT
Update:3 months ago
Show doc for...
Tokens:
Raw
### Train Model With Dynamics Integration (ETH Scene)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Example command to train a model with dynamics integration for the ETH - University scene. The primary difference from the non-dynamics version is the sourced model configuration file.

```bash
python train.py --eval_every 10 --vis_every 1 --train_data_dict eth_train.pkl --eval_data_dict eth_val.pkl --offline_scene_graph yes --preprocess_workers 5 --log_dir ../experiments/pedestrians/models --log_tag _eth_ar3 --train_epochs 100 --augment --conf ../experiments/pedestrians/models/eth_attention_radius_3/config.json
```

--------------------------------

### Train Model Without Dynamics Integration (ETH Scene)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Example command to train a model without dynamics integration for the ETH - University scene. It specifies training data, evaluation data, preprocessing options, logging details, and model configuration.

```bash
python train.py --eval_every 10 --vis_every 1 --train_data_dict eth_train.pkl --eval_data_dict eth_val.pkl --offline_scene_graph yes --preprocess_workers 5 --log_dir ../experiments/pedestrians/models --log_tag _eth_vel_ar3 --train_epochs 100 --augment --conf ../experiments/pedestrians/models/eth_vel/config.json
```

--------------------------------

### Evaluate Pedestrian Model (ETH - With Dynamics)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Example command to evaluate a model with dynamics integration on the ETH - University scene. This command specifies the model, checkpoint, data path, output settings, and node type.

```bash
python evaluate.py --model models/eth_attention_radius_3 --checkpoint 100 --data ../processed/eth_test.pkl --output_path results --output_tag eth_12 --node_type PEDESTRIAN
```

--------------------------------

### Select Scene and Get Scene Name

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Selects a specific scene from the loaded evaluation scenes and retrieves its name.

```python
scene = eval_scenes[25]
scene.name
```

--------------------------------

### Create and Activate Conda Environment

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Creates a new conda environment named 'trajectron++' with Python 3.6 and installs dependencies from requirements.txt.

```bash
conda create --name trajectron++ python=3.6 -y
source activate trajectron++
pip install -r requirements.txt
```

--------------------------------

### Evaluate Pedestrian Model (ETH - No Dynamics)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Example command to evaluate a model without dynamics integration on the ETH - University scene. This command specifies the model, checkpoint, data path, output settings, and node type.

```bash
python evaluate.py --model models/eth_vel --checkpoint 100 --data ../processed/eth_test.pkl --output_path results --output_tag eth_vel_12 --node_type PEDESTRIAN
```

--------------------------------

### Install Conda Environment as IPython Kernel

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Registers the current conda environment as an IPython kernel for use with Jupyter notebooks.

```bash
python -m ipykernel install --user --name trajectronpp --display-name "Python 3.6 (Trajectron++)"
```

--------------------------------

### Initialize and Update Submodules

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Initializes and updates submodules separately after a normal clone.

```bash
git submodule init # Initializing our local configuration file
git submodule update # Fetching all of the data from the submodules at the specified commits
```

--------------------------------

### Configure CPU Training for Trajectron++

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

To train Trajectron++ on a CPU, comment out line 38 in `train.py` and append `--device cpu` to your training command. This ensures the script uses the CPU instead of a GPU.

```bash
--device cpu
```

--------------------------------

### Process nuScenes Mini Dataset

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Navigates to the nuScenes experiments directory and processes the mini dataset. Ensure the dataset is extracted in the 'experiments/nuScenes' directory.

```bash
cd experiments/nuScenes
python process_data.py --data=./v1.0-mini --version="v1.0-mini" --output_path=../processed
```

--------------------------------

### Quantitative Performance Analysis for NuScenes Models

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Quantitative.ipynb

This script iterates through different models and prediction horizons (PH) to calculate and print mean FDE, ADE, and KDE metrics. It reads performance data from CSV files, aggregates it into a pandas DataFrame, and computes the mean for each metric at specified time steps.

```python
for model in ['int_ee_me_ped', 'vel_ee_ped']:
    print(f"FDE Results for: {model}")
    for ph in [2, 4, 6, 8]:
        print(f"-----------------PH: {ph} -------------------")
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_fde_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"FDE Mean @{ph*0.5}s: {perf_df['value'][perf_df['metric'] == 'fde'].mean()}")
        del perf_df    
              
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_ade_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"ADE Mean @{ph*0.5}s: {perf_df['value'][perf_df['metric'] == 'ade'].mean()}")
        del perf_df
              
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_kde_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"KDE Mean @{ph*0.5}s: {perf_df['value'][perf_df['metric'] == 'kde'].mean()}")
        del perf_df  

        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_fde_most_likely_z.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"FDE @{ph*0.5}s: {perf_df['value'][perf_df['metric'] == 'fde'].mean()}")      
        del perf_df
              
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_ade_most_likely_z.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"ADE @{ph*0.5}s: {perf_df['value'][perf_df['metric'] == 'ade'].mean()}")      
        del perf_df
        print("----------------------------------------------")
    print("")
```

--------------------------------

### Train Model with Dynamics Integration on nuScenes

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command for training a Trajectron++ model with dynamics integration on the nuScenes dataset. Requires specifying the configuration file for this model type.

```bash
python train.py --eval_every 1 --vis_every 1 --conf ../experiments/nuScenes/models/int_ee/config.json --train_data_dict nuScenes_train_full.pkl --eval_data_dict nuScenes_val_full.pkl --offline_scene_graph yes --preprocess_workers 10 --batch_size 256 --log_dir ../experiments/nuScenes/models --train_epochs 20 --node_freq_mult_train --log_tag _int_ee --augment
```

--------------------------------

### Process Full nuScenes Dataset

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Navigates to the nuScenes experiments directory and processes the full dataset. Ensure the dataset is extracted in the 'experiments/nuScenes' directory.

```bash
cd experiments/nuScenes
python process_data.py --data=./v1.0 --version="v1.0-trainval" --output_path=../processed
```

--------------------------------

### Load nuScenes SDK and Map API

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Initializes the nuScenes map API by specifying the data path and devkit path. This is required for accessing map information.

```python
nuScenes_data_path = # Data Path to nuScenes data set 
nuScenes_devkit_path = './devkit/python-sdk/'
sys.path.append(nuScenes_devkit_path)
from nuscenes.map_expansion.map_api import NuScenesMap
nusc_map = NuScenesMap(dataroot=nuScenes_data_path, map_name='boston-seaport')
```

--------------------------------

### Quantitative Analysis Loop for NuScenes Models

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Quantitative.ipynb

This script iterates through different models and prediction horizons, loading performance data from CSV files and printing key metrics such as FDE and KDE. It requires 'pandas' and 'glob' libraries.

```python
for model in ['int_ee_me_no_ego', 'robot']:
    print(f"FDE Results for: {model}")
    for ph in [2, 4, 6, 8]:
        print(f"-----------------PH: {ph} -------------------")
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}_{ph}_fde_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        
        print(f"FDE Mean @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'full'].mean()}")
        del perf_df      
        
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}_{ph}_rv_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"RB Viols @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'full'].sum() / (len(perf_df['value'][perf_df['type'] == 'full'].index)*2000)}")
        del perf_df

        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_fde_most_likely_z.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"FDE @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'ml'].mean()}")      
        del perf_df
              
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_kde_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"KDE @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'full'].mean()}")      
        print("----------------------------------------------")
        del perf_df
    print("")
```

--------------------------------

### Define Dataset and Algorithm Names

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Sets up lists and variables for dataset names and the algorithm identifier used in the analysis.

```python
dataset_names = ['eth', 'hotel', 'univ', 'zara1', 'zara2', 'Average']
alg_name = "Ours"
```

--------------------------------

### Train Base Model on nuScenes

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Use this command to train the base Trajectron++ model on the nuScenes dataset. Ensure you are in the `trajectron/` directory.

```bash
python train.py --eval_every 1 --vis_every 1 --conf ../experiments/nuScenes/models/vel_ee/config.json --train_data_dict nuScenes_train_full.pkl --eval_data_dict nuScenes_val_full.pkl --offline_scene_graph yes --preprocess_workers 10 --batch_size 256 --log_dir ../experiments/nuScenes/models --train_epochs 20 --node_freq_mult_train --log_tag _vel_ee --augment
```

--------------------------------

### Evaluate Hotel Dataset Model

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/eval_vel_cmds.txt

Run evaluation for the Hotel dataset using a specified model and checkpoint. Ensure the output path and tag are set for results.

```bash
python evaluate.py --model ./models/hotel_vel --checkpoint 100 --data ../processed/hotel_test.pkl --output_path ./results/ --output_tag hotel_vel_12 --node_type PEDESTRIAN
```

--------------------------------

### Clone Repository with Submodules

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Clones the Trajectron++ repository and its submodules in one command.

```bash
git clone --recurse-submodules <repository cloning URL>
```

--------------------------------

### Process Pedestrian Datasets

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Navigates to the pedestrian experiments directory and processes the raw data into a model-compatible format. This process takes approximately 10-15 minutes.

```bash
cd experiments/pedestrians
python process_data.py
```

--------------------------------

### Evaluate Zara2 Dataset Model

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/eval_vel_cmds.txt

Run evaluation for the Zara2 dataset using a specified model and checkpoint. Ensure the output path and tag are set for results.

```bash
python evaluate.py --model ./models/zara2_vel --checkpoint 100 --data ../processed/zara2_test.pkl --output_path ./results/ --output_tag zara2_vel_12 --node_type PEDESTRIAN
```

--------------------------------

### Evaluate Zara1 Dataset Model

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/eval_vel_cmds.txt

Run evaluation for the Zara1 dataset using a specified model and checkpoint. Ensure the output path and tag are set for results.

```bash
python evaluate.py --model ./models/zara1_vel --checkpoint 100 --data ../processed/zara1_test.pkl --output_path ./results/ --output_tag zara1_vel_12 --node_type PEDESTRIAN
```

--------------------------------

### Quantitative Analysis Loop

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Quantitative.ipynb

Iterates through predefined models and prediction horizons to calculate and print performance metrics from CSV files. Requires results to be saved in 'results/' directory with specific naming conventions.

```python
for model in ['int_ee', 'int_ee_me', 'vel_ee', 'robot']:
    print(f"FDE Results for: {model}")
    for ph in [2, 4, 6, 8]:
        print(f"-----------------PH: {ph} -------------------")
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}_{ph}_fde_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        
        print(f"FDE Mean @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'full'].mean()}")
        del perf_df      
        
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}_{ph}_rv_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"RB Viols @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'full'].sum() / (len(perf_df['value'][perf_df['type'] == 'full'].index)*2000)}")
        del perf_df

        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_fde_most_likely_z.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"FDE @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'ml'].mean()}")      
        del perf_df
              
        perf_df = pd.DataFrame()
        for f in glob.glob(f"results/{model}*_{ph}_kde_full.csv"):
            dataset_df = pd.read_csv(f)
            dataset_df['model'] = model
            perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
        print(f"KDE @{ph*0.5}s: {perf_df['value'][perf_df['type'] == 'full'].mean()}")      
        print("----------------------------------------------")
        del perf_df
    print("")
```

--------------------------------

### Run Base Model Online

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to run the base Trajectron++ model in online mode. Ensure the specified log directory, data directory, and configuration file are correctly set.

```bash
python test_online.py --log_dir=../experiments/nuScenes/models --data_dir=../experiments/processed --conf=config.json --eval_data_dict=nuScenes_test_mini_full.pkl
```

--------------------------------

### Run Model with Dynamics, Maps, and Robot Future Online

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to run Trajectron++ with dynamics integration, map encoding, and robot future prediction enabled for online execution. This utilizes the `--map_encoding` and `--incl_robot_node` flags.

```bash
python test_online.py --log_dir=../experiments/nuScenes/models --data_dir=../experiments/processed --conf=config.json --eval_data_dict=nuScenes_test_mini_full.pkl --map_encoding --incl_robot_node
```

--------------------------------

### Train Model with Dynamics Integration and Maps on nuScenes

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Use this command to train a Trajectron++ model that includes dynamics integration and map encoding on the nuScenes dataset. Note the `--map_encoding` flag.

```bash
python train.py --eval_every 1 --vis_every 1 --conf ../experiments/nuScenes/models/int_ee_me/config.json --train_data_dict nuScenes_train_full.pkl --eval_data_dict nuScenes_val_full.pkl --offline_scene_graph yes --preprocess_workers 10 --batch_size 256 --log_dir ../experiments/nuScenes/models --train_epochs 20 --node_freq_mult_train --log_tag _int_ee_me --map_encoding --augment
```

--------------------------------

### Evaluate University Dataset Model

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/eval_vel_cmds.txt

Run evaluation for the University dataset using a specified model and checkpoint. Ensure the output path and tag are set for results.

```bash
python evaluate.py --model ./models/univ_vel --checkpoint 100 --data ../processed/univ_test.pkl --output_path ./results/ --output_tag univ_vel_12 --node_type PEDESTRIAN
```

--------------------------------

### Evaluate ETH Dataset Model

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/eval_vel_cmds.txt

Run evaluation for the ETH dataset using a specified model and checkpoint. Ensure the output path and tag are set for results.

```bash
python evaluate.py --model ./models/eth_vel --checkpoint 100 --data ../processed/eth_test.pkl --output_path ./results/ --output_tag eth_vel_12 --node_type PEDESTRIAN
```

--------------------------------

### Run Model with Dynamics Integration and Maps Online

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to execute Trajectron++ with dynamics integration and map encoding enabled for online processing. The `--map_encoding` flag is crucial for this configuration.

```bash
python test_online.py --log_dir=../experiments/nuScenes/models --data_dir=../experiments/processed --conf=config.json --eval_data_dict=nuScenes_test_mini_full.pkl --map_encoding
```

--------------------------------

### Base Model Training Command

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

This is a base command for training a model on pedestrian datasets. Replace placeholders like '<dataset>' and '<desired tag>' with specific values.

```bash
python train.py --eval_every 10 --vis_every 1 --train_data_dict <dataset>_train.pkl --eval_data_dict <dataset>_val.pkl --offline_scene_graph yes --preprocess_workers 5 --log_dir ../experiments/pedestrians/models --log_tag <desired tag> --train_epochs 100 --augment --conf <desired model configuration>
```

--------------------------------

### Load Model for Prediction

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Loads a pre-trained prediction model and its associated hyperparameters from a specified directory. This is the initial step before making any predictions.

```python
model_dir = os.path.join(log_dir, 'int_ee') 
eval_stg_nm, hyp = load_model(model_dir, eval_env, ts=12)
```

--------------------------------

### Print KDE NLL for Datasets

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Calculates and prints the KDE Negative Log Likelihood for each dataset. It iterates through dataset names, printing the mean 'value' from the performance DataFrame for the specified algorithm name ('Ours' in this context). It handles both individual datasets and an 'Average' case.

```python
for dataset in dataset_names:
    if dataset != 'Average':
        print('KDE NLL for ' + pretty_dataset_name(dataset))
        #print(f"SGAN: {-lls_df[(lls_df['method'] == 'sgan') & (lls_df['dataset'] == dataset)]['log-likelihood'].mean()}")
        #print(f"Trajectron: {-lls_df[(lls_df['method'] == 'Trajectron')  & (lls_df['dataset'] == dataset)]['log-likelihood'].mean()}")
        print(f"{alg_name}: {perf_df[(perf_df['method'] == alg_name) & (perf_df['dataset'] == dataset)]['value'].mean()}")
    else:
        print('KDE NLL for ' + pretty_dataset_name(dataset))
        #print(f"SGAN: {-lls_df[(lls_df['method'] == 'sgan')]['log-likelihood'].mean()}")
        #print(f"Trajectron: {-lls_df[(lls_df['method'] == 'Trajectron')]['log-likelihood'].mean()}")
        print(f"{alg_name}: {perf_df[(perf_df['method'] == alg_name)]['value'].mean()}")
```

--------------------------------

### Train Model with Robot Future Prediction on nuScenes

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command for training a Trajectron++ model that incorporates robot future prediction, dynamics integration, and map encoding on the nuScenes dataset. Includes `--incl_robot_node` and `--map_encoding` flags.

```bash
python train.py --eval_every 1 --vis_every 1 --conf ../experiments/nuScenes/models/robot/config.json --train_data_dict nuScenes_train_full.pkl --eval_data_dict nuScenes_val_full.pkl --offline_scene_graph yes --preprocess_workers 10 --batch_size 256 --log_dir ../experiments/nuScenes/models --train_epochs 20 --node_freq_mult_train --log_tag _robot --incl_robot_node --map_encoding
```

--------------------------------

### Load Trained Trajectron++ Model

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Loads a pre-trained Trajectron++ model and its associated hyperparameters from a specified directory.

```python
model_dir = os.path.join(log_dir, 'int_ee_me') 
eval_stg, hyp = load_model(model_dir, eval_env, ts=12)
```

--------------------------------

### Import Libraries

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Quantitative.ipynb

Imports necessary libraries for data manipulation, file system interaction, and plotting.

```python
import pandas as pd
import numpy as np
import glob
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
```

--------------------------------

### Evaluate nuScenes Vehicle Model (+Dynamics Integration, Maps, Robot Future)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to evaluate a nuScenes vehicle forecasting model with dynamics integration, map information, and robot future context. This includes specifying the model, checkpoint, data, output path, tag, node type, and prediction horizon.

```bash
python evaluate.py --model models/robot --checkpoint=12 --data ../processed/nuScenes_test_full.pkl --output_path results --output_tag robot --node_type VEHICLE --prediction_horizon 6
```

--------------------------------

### Import Libraries

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Imports necessary libraries for data manipulation, plotting, and analysis.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import glob
import numpy as np
import seaborn as sns
import pandas as pd

from collections import OrderedDict
```

--------------------------------

### Load Model for Velocity and Map Prediction

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Loads a pre-trained Trajectron++ model that utilizes velocity output and map context for evaluation. Ensure the 'log_dir' and 'eval_env' are correctly set.

```python
model_dir = os.path.join(log_dir, 'me_vel') 
eval_stg_vel_map, hyp = load_model(model_dir, eval_env, ts=12)
```

--------------------------------

### Import Libraries for Trajectron++

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Imports necessary Python libraries for data manipulation, deep learning, and visualization, including custom Trajectron++ modules.

```python
%matplotlib inline
import sys
sys.path.append('../../trajectron')
import os
import numpy as np
import torch
import dill
import json
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
import matplotlib.patheffects as pe
from helper import *
import visualization
```

--------------------------------

### Evaluate Pedestrian Model (General Command)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

General command to evaluate a trained pedestrian model. Replace placeholders with your specific model directory, checkpoint, dataset, and desired output tags.

```bash
python evaluate.py --model <model directory> --checkpoint <epoch number> --data ../processed/<dataset>_test.pkl --output_path results --output_tag <dataset>_<vel if no integration>_12 --node_type PEDESTRIAN
```

--------------------------------

### Evaluate nuScenes Pedestrian Model (+Dynamics Integration, Maps)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to evaluate a nuScenes pedestrian forecasting model with dynamics integration and map information. This includes specifying the model, checkpoint, data, output path, tag, node type, and prediction horizon.

```bash
python evaluate.py --model models/int_ee_me --checkpoint=12 --data ../processed/nuScenes_test_full.pkl --output_tag int_ee_me_ped --node_type PEDESTRIAN --prediction_horizon 6
```

--------------------------------

### Load Model for Velocity Prediction

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Loads a pre-trained Trajectron++ model for evaluation, specifically configured for velocity output. Ensure the log directory and model name are correctly specified.

```python
model_dir = os.path.join(log_dir, 'ee_vel') 
eval_stg_vel, hyp = load_model(model_dir, eval_env, ts=12)
```

--------------------------------

### Evaluate nuScenes Pedestrian Model (Base)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to evaluate the base nuScenes pedestrian forecasting model. This includes specifying the model, checkpoint, data, output path, tag, node type, and prediction horizon.

```bash
python evaluate.py --model models/vel_ee --checkpoint=12 --data ../processed/nuScenes_test_full.pkl --output_tag vel_ee_ped --node_type PEDESTRIAN --prediction_horizon 6
```

--------------------------------

### Evaluate nuScenes Vehicle Model (+Dynamics Integration)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to evaluate a nuScenes vehicle forecasting model with dynamics integration. This includes specifying the model, checkpoint, data, output path, tag, node type, and prediction horizon.

```bash
python evaluate.py --model models/int_ee --checkpoint=12 --data ../processed/nuScenes_test_full.pkl --output_path results --output_tag int_ee --node_type VEHICLE --prediction_horizon 6
```

--------------------------------

### Load and Process 'Ours' Method Results

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads CSV files containing performance metrics for the 'Ours' method, aggregates them into a pandas DataFrame, and renames columns for consistency.

```python
# Load Ours
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}*attention_radius_3*fde_most_likely.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = alg_name
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
perf_df = perf_df.rename(columns={"metric": "error_type", "value": "error_value"})
```

--------------------------------

### Load Processed nuScenes Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Loads pre-processed nuScenes scene data from a pickle file. This data is used for evaluation.

```python
with open('../processed/nuScenes_test_full.pkl', 'rb') as f:
    eval_env = dill.load(f, encoding='latin1')
eval_scenes = eval_env.scenes
```

--------------------------------

### Set Prediction Horizon and Model Directory

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Sets the prediction horizon (ph) and the directory where the trained models are stored.

```python
ph = 6
log_dir = './models'
```

--------------------------------

### Evaluate nuScenes Vehicle Model (Base)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to evaluate the base nuScenes vehicle forecasting model. This includes specifying the model, checkpoint, data, output path, tag, node type, and prediction horizon.

```bash
python evaluate.py --model models/vel_ee --checkpoint=12 --data ../processed/nuScenes_test_full.pkl --output_path results --output_tag vel_ee --node_type VEHICLE --prediction_horizon 6
```

--------------------------------

### Evaluate nuScenes Vehicle Model (+Dynamics Integration, Maps)

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Command to evaluate a nuScenes vehicle forecasting model with dynamics integration and map information. This includes specifying the model, checkpoint, data, output path, tag, node type, and prediction horizon.

```bash
python evaluate.py --model models/int_ee_me --checkpoint=12 --data ../processed/nuScenes_test_full.pkl --output_path results --output_tag int_ee_me --node_type VEHICLE --prediction_horizon 6
```

--------------------------------

### Generate Validation Set for nuScenes

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

Modifies the 'process_data.py' script to include validation set generation for the nuScenes dataset.

```python
val_scene_names = val_scenes
```

--------------------------------

### Load ADE Performance Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads CSV files containing ADE performance data for Trajectron++. Similar to the FDE loading process, this snippet iterates through datasets and glob patterns to collect performance metrics.

```python
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}_vel_12*ade_most_likely.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = 'Trajectron++'
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
```

--------------------------------

### Print ADE Best of 20 Results

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Prints the mean ADE Best of 20 values for each dataset and the overall average. Requires the 'perf_df' DataFrame to be populated with ADE results and 'pretty_dataset_name' function to be defined.

```python
for dataset in dataset_names:
    print('ADE Best of 20 for ' + pretty_dataset_name(dataset))
    if dataset != 'Average':
        print(f"Trajectron++: {perf_df[(perf_df['method'] == alg_name) & (perf_df['dataset'] == dataset)]['value'].mean()}")
    else:
        print(f"Trajectron++: {perf_df[(perf_df['method'] == alg_name)]['value'].mean()}")
```

--------------------------------

### Plotting Style Configuration

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Defines visual elements for plots, including markers, line styles, and color palettes for different datasets and areas.

```python
mean_markers = 'X'
marker_size = 7
line_colors = ['#1f78b4','#33a02c','#fb9a99','#e31a1c']
area_colors = ['#80CBE5','#ABCB51', '#F05F78']
area_rgbs = list()
for c in area_colors:
    area_rgbs.append([int(c[i:i+2], 16) for i in (1, 3, 5)])
```

--------------------------------

### Print FDE Best of 20 Results

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Prints the mean FDE Best of 20 values for each dataset and the overall average. Requires the 'perf_df' DataFrame to be populated with FDE results and 'pretty_dataset_name' function to be defined.

```python
for dataset in dataset_names:
    print('FDE Best of 20 for ' + pretty_dataset_name(dataset))
    if dataset != 'Average':
        print(f"Trajectron++: {perf_df[(perf_df['method'] == alg_name) & (perf_df['dataset'] == dataset)]['value'].mean()}")
    else:
        print(f"Trajectron++: {perf_df[(perf_df['method'] == alg_name)]['value'].mean()}")
```

--------------------------------

### Generate Predictions and Visualize on Map

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Performs predictions using the loaded model for a given scene and timestep, then visualizes these predictions overlaid on a rendered map patch. It includes plotting ground truth and predicted trajectories.

```python
ph = 6
with torch.no_grad():
    timestep = np.array([2])
    predictions = eval_stg.predict(scene,
                                   timestep,
                                   ph,
                                   num_samples=500)

    predictions_mm = eval_stg.predict(scene,
                                      timestep,
                                      ph,
                                      num_samples=1,
                                      z_mode=True,
                                      gmm_mode=True)

    # Plot predicted timestep for random scene in map
    my_patch = (x_min, y_min, x_max, y_max)
    fig, ax = nusc_map.render_map_patch(my_patch, layers, figsize=(10, 10), alpha=0.1, render_egoposes_range=False)

    ax.plot([], [], 'ko-',
            zorder=620,
            markersize=4,
            linewidth=2, alpha=0.7, label='Ours (MM)')

    ax.plot([],
            [],
            'w--o', label='Ground Truth',
            linewidth=3,
            path_effects=[pe.Stroke(linewidth=4, foreground='k'), pe.Normal()])

    plot_vehicle_nice(ax,
                      predictions,
                      scene.dt,
                      max_hl=10,
                      ph=ph,
                      map=None, x_min=x_min, y_min=y_min)

    plot_vehicle_mm(ax,
                    predictions_mm,
                    scene.dt,
                    max_hl=10,
                    ph=ph,
                    map=None, x_min=x_min, y_min=y_min)

    ax.set_ylim((1385, 1435))
    ax.set_xlim((850, 900))
    leg = ax.legend(loc='upper right', fontsize=20, frameon=True)
    ax.axis('off')
    for lh in leg.legendHandles:
        lh.set_alpha(.5)
    ax.get_legend().remove()
    fig.show()
    fig.savefig('plots/qual_nuScenes_map_pos.pdf', dpi=300, bbox_inches='tight')
```

--------------------------------

### Generate and Visualize Predictions

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Generates predictions for a given scene and timestep, including both standard and GMM/Z mode predictions. Visualizes these predictions on a map patch, comparing them with ground truth and plotting vehicle trajectories.

```python
ph = 6
with torch.no_grad():
    timestep = np.array([2])
    predictions = eval_stg_vel.predict(scene,
                                   timestep,
                                   ph,
                                   num_samples=500)

    predictions_mm = eval_stg_vel.predict(scene,
                                      timestep,
                                      ph,
                                      num_samples=1,
                                      z_mode=True,
                                      gmm_mode=True)

    # Plot predicted timestep for random scene in map
    my_patch = (x_min, y_min, x_max, y_max)
    fig, ax = nusc_map.render_map_patch(my_patch, layers, figsize=(10, 10), alpha=0.1, render_egoposes_range=False)

    ax.plot([], [], 'ko-',
            zorder=620,
            markersize=4,
            linewidth=2, alpha=0.7, label='Ours (MM)')

    ax.plot([],
            [],
            'w--o', label='Ground Truth',
            linewidth=3,
            path_effects=[pe.Stroke(linewidth=4, foreground='k'), pe.Normal()])

    plot_vehicle_nice(ax,
                      predictions,
                      scene.dt,
                      max_hl=10,
                      ph=ph,
                      map=None, x_min=x_min, y_min=y_min)

    plot_vehicle_mm(ax,
                    predictions_mm,
                    scene.dt,
                    max_hl=10,
                    ph=ph,
                    map=None, x_min=x_min, y_min=y_min)

    ax.set_ylim((1385, 1435))
    ax.set_xlim((850, 900))
    leg = ax.legend(loc='upper right', fontsize=20, frameon=True)
    ax.axis('off')
    for lh in leg.legendHandles:
        lh.set_alpha(.5)
    ax.get_legend().remove()
    fig.show()
    fig.savefig('plots/qual_nuScenes_no_map_vel.pdf', dpi=300, bbox_inches='tight')
```

--------------------------------

### Generate and Plot Predictions with Map Context

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Generates predictions for a given scene using the loaded model, including both standard and GMM/Z mode predictions. Visualizes these predictions on a map patch, along with ground truth and a custom legend.

```python
ph = 6
with torch.no_grad():
    timestep = np.array([2])
    predictions = eval_stg_vel_map.predict(scene,
                                   timestep,
                                   ph,
                                   num_samples=500)

    predictions_mm = eval_stg_vel_map.predict(scene,
                                      timestep,
                                      ph,
                                      num_samples=1,
                                      z_mode=True,
                                      gmm_mode=True)

    # Plot predicted timestep for random scene in map
    my_patch = (x_min, y_min, x_max, y_max)
    fig, ax = nusc_map.render_map_patch(my_patch, layers, figsize=(10, 10), alpha=0.1, render_egoposes_range=False)

    ax.plot([], [], 'ko-',
            zorder=620,
            markersize=4,
            linewidth=2, alpha=0.7, label='Ours (MM)')

    ax.plot([],
            [],
            'w--o', label='Ground Truth',
            linewidth=3,
            path_effects=[pe.Stroke(linewidth=4, foreground='k'), pe.Normal()])

    plot_vehicle_nice(ax,
                      predictions,
                      scene.dt,
                      max_hl=10,
                      ph=ph,
                      map=None, x_min=x_min, y_min=y_min)

    plot_vehicle_mm(ax,
                    predictions_mm,
                    scene.dt,
                    max_hl=10,
                    ph=ph,
                    map=None, x_min=x_min, y_min=y_min)

    ax.set_ylim((1385, 1435))
    ax.set_xlim((850, 900))
    leg = ax.legend(loc='upper right', fontsize=20, frameon=True)
    ax.axis('off')
    for lh in leg.legendHandles:
        lh.set_alpha(.5)
    ax.get_legend().remove()
    fig.show()
    fig.savefig('plots/qual_nuScenes_map_vel.pdf', dpi=300, bbox_inches='tight')
```

--------------------------------

### Load ADE Best of 20 Performance Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads 'Best of 20' ADE performance data from CSV files for various datasets. It aggregates the data into a single DataFrame and assigns dataset and method names. Ensure the 'results' directory contains the relevant CSV files.

```python
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}_vel_12*ade_best_of.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = alg_name
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
```

--------------------------------

### Perform and Visualize Predictions

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Generates trajectory predictions for a given scene and visualizes them. It includes predictions with and without GMM/Z-mode, and plots them on a map patch. Requires scene, map rendering objects, and plotting functions.

```python
ph = 6
with torch.no_grad():
    timestep = np.array([2])
    predictions = eval_stg_nm.predict(scene,
                                   timestep,
                                   ph,
                                   num_samples=500)

    predictions_mm = eval_stg_nm.predict(scene,
                                      timestep,
                                      ph,
                                      num_samples=1,
                                      z_mode=True,
                                      gmm_mode=True)

    # Plot predicted timestep for random scene in map
    my_patch = (x_min, y_min, x_max, y_max)
    fig, ax = nusc_map.render_map_patch(my_patch, layers, figsize=(10, 10), alpha=0.1, render_egoposes_range=False)

    ax.plot([], [], 'ko-',
            zorder=620,
            markersize=4,
            linewidth=2, alpha=0.7, label='Ours (MM)')

    ax.plot([],
            [],
            'w--o', label='Ground Truth',
            linewidth=3,
            path_effects=[pe.Stroke(linewidth=4, foreground='k'), pe.Normal()])

    plot_vehicle_nice(ax,
                      predictions,
                      scene.dt,
                      max_hl=10,
                      ph=ph,
                      map=None, x_min=x_min, y_min=y_min)

    plot_vehicle_mm(ax,
                    predictions_mm,
                    scene.dt,
                    max_hl=10,
                    ph=ph,
                    map=None, x_min=x_min, y_min=y_min)

    ax.set_ylim((1385, 1435))
    ax.set_xlim((850, 900))
    leg = ax.legend(loc='upper right', fontsize=20, frameon=True)
    ax.axis('off')
    for lh in leg.legendHandles:
        lh.set_alpha(.5)
    ax.get_legend().remove()
    fig.show()
    fig.savefig('plots/qual_nuScenes_no_map_pos.pdf', dpi=300, bbox_inches='tight')
```

--------------------------------

### Modify train.py for Ego-Vehicle Exclusion

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/README.md

To train a model without the ego-vehicle, modify line 132 in `train.py` to set `return_robot=False`. This is an alternative to using the base model configuration.

```python
return_robot=False)
```

--------------------------------

### Define Map Layers for Rendering

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Specifies which layers from the nuScenes map should be rendered in the visualization.

```python
layers = ['drivable_area',
          'road_segment',
          'lane',
          'ped_crossing',
          'walkway',
          'stop_line',
          'road_divider',
          'lane_divider']
```

--------------------------------

### Load and Aggregate Performance Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads performance data from CSV files for multiple datasets and appends them into a single DataFrame. It renames columns for consistency.

```python
# Load Ours
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}*attention_radius_3*ade_most_likely.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = alg_name
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
perf_df = perf_df.rename(columns={"metric": "error_type", "value": "error_value"})
#perf_df.head()
```

--------------------------------

### Load Performance Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads performance data from CSV files for different datasets and methods. It appends dataset and method information to each DataFrame before combining them.

```python
# Load Ours
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}_vel_12*kde_full.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = alg_name
        perf_df = perf_df.append(dataset_df, ignore_index=True)
        del perf_df['Unnamed: 0']
#perf_df.head()
```

--------------------------------

### Load FDE Best of 20 Performance Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads 'Best of 20' FDE performance data from CSV files for various datasets. It aggregates the data into a single DataFrame and assigns dataset and method names. Ensure the 'results' directory contains the relevant CSV files.

```python
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}_vel_12*fde_best_of.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = alg_name
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
```

--------------------------------

### Dataset Name Formatting

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Provides a function to format dataset names for consistent display in plots and tables.

```python
def pretty_dataset_name(dataset_name):
    if dataset_name == 'eth':
        return 'ETH - Univ'
    elif dataset_name == 'hotel':
        return 'ETH - Hotel'
    elif dataset_name == 'univ':
        return 'UCY - Univ'
    elif dataset_name == 'zara1':
        return 'UCY - Zara 1'
    elif dataset_name == 'zara2':
        return 'UCY - Zara 2'
    else:
        return dataset_name
```

--------------------------------

### Load FDE Performance Data

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Loads CSV files containing FDE performance data for Trajectron++. It iterates through specified datasets and glob patterns to find relevant files, appends them to a DataFrame, and cleans up unnecessary columns.

```python
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}_vel_12*fde_most_likely.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = 'Trajectron++'
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
```

--------------------------------

### Load and Aggregate ADE Results

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Aggregates ADE results from multiple CSV files into a single pandas DataFrame. Ensure that the 'results' directory contains the expected CSV files and that 'dataset_names' and 'alg_name' are defined.

```python
perf_df = pd.DataFrame()
for dataset in dataset_names:
    for f in glob.glob(f"results/{dataset}_12*ade_best_of.csv"):
        print(f)
        dataset_df = pd.read_csv(f)
        dataset_df['dataset'] = dataset
        dataset_df['method'] = alg_name
        perf_df = perf_df.append(dataset_df, ignore_index=True, sort=False)
        del perf_df['Unnamed: 0']
```

--------------------------------

### Print FDE Most Likely Results

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Prints the calculated FDE most likely results for each dataset and the overall average. It displays the mean 'value' for the 'Trajectron++' method, grouped by dataset.

```python
for dataset in dataset_names:
    print('FDE Most Likely for ' + pretty_dataset_name(dataset))
    if dataset != 'Average':
        print(f"{alg_name}: {perf_df[(perf_df['method'] == 'Trajectron++') & (perf_df['dataset'] == dataset)]['value'].mean()}")
    else:
        print(f"{alg_name}: {perf_df[(perf_df['method'] == 'Trajectron++')]['value'].mean()}")
```

--------------------------------

### Define Region of Interest (ROI) for Map

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/nuScenes/NuScenes Qualitative.ipynb

Defines the bounding box coordinates (x_min, y_min, x_max, y_max) for the region of interest on the nuScenes map.

```python
# Define ROI in nuScenes Map
x_min = 773.0
x_max = 1100.0
y_min = 1231.0
y_max = 1510.0
```

--------------------------------

### Clean Up Variables

Source: https://github.com/stanfordasl/trajectron-plus-plus/blob/master/experiments/pedestrians/Result Analysis.ipynb

Removes previously loaded dataframes to free up memory before loading new data.

```python
del perf_df
del errors_df
```
Tokens:
Raw