### Setup Conda Environment and Install Dependencies

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Sets up the necessary conda environment and installs core dependencies like nerfstudio and gsplat. This is crucial for running the project's functionalities.

```bash
conda activate nerfstudio
pip install gsplat
pip install gym
pip install transformers
```

--------------------------------

### Clone Repository with Git

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

This command clones the GRaD-Nav repository from GitHub. It's the initial step to get the project's source code.

```bash
git clone https://github.com/Qianzhong-Chen/grad_nav.git
```

--------------------------------

### Train GRaD-Nav Multi-Gate

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Starts the training for the GRaD-Nav model in a multi-gate environment. Training configurations and log outputs are managed through command-line arguments.

```python
python examples/train_gradnav.py --cfg examples/cfg/gradnav/drone_multi_gate.yaml --logdir examples/logs/DroneMultiGate/gradnav
```

--------------------------------

### Critic Network for Value Estimation (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Implements a critic network using MLPs to estimate state values, essential for advantage computation in reinforcement learning. It includes initialization, forward pass for value estimation, and setup for training using TD-lambda targets.

```python
import torch
import torch.nn as nn
from models import model_utils

# Initialize critic network
privilege_obs_dim = 67  # Privileged observations (full state info)
cfg_network = {
    'critic_mlp': {
        'units': [512, 256, 128],
        'activation': 'elu'
    }
}

critic = models.critic.CriticMLP(privilege_obs_dim, cfg_network, device='cuda:0')

# Compute state values
privilege_obs = torch.randn(128, privilege_obs_dim).cuda()
values = critic(privilege_obs)
# values.shape: (128, 1) - estimated state values

# Training critic with TD-lambda targets
import torch.optim as optim

critic_optimizer = optim.AdamW(critic.parameters(), lr=1e-4, weight_decay=5e-3)

# Compute TD-lambda targets (done in GradNav.compute_target_values())
rewards = torch.randn(32, 128).cuda()  # (steps, envs)
next_values = torch.randn(32, 128).cuda()
done_mask = torch.zeros(32, 128).cuda()
gamma = 0.99
lam = 0.95

# Compute target values using TD-lambda
target_values = torch.zeros(32, 128).cuda()
Ai = torch.zeros(128).cuda()
Bi = torch.zeros(128).cuda()
lam_factor = torch.ones(128).cuda()

for i in reversed(range(32)):
    lam_factor = lam_factor * lam * (1. - done_mask[i]) + done_mask[i]
    Ai = (1.0 - done_mask[i]) * (lam * gamma * Ai + gamma * next_values[i] + 
          (1. - lam_factor) / (1. - lam) * rewards[i])
    Bi = gamma * (next_values[i] * done_mask[i] + Bi * (1.0 - done_mask[i])) + rewards[i]
    target_values[i] = (1.0 - lam) * Ai + lam_factor * Bi

```

--------------------------------

### Load and Initialize PPO Trainer (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Loads the configuration for the Proximal Policy Optimization (PPO) algorithm from a YAML file and initializes the PPO trainer. This sets up the necessary parameters for training a baseline PPO agent.

```python
import yaml
import torch
import algorithms.ppo as ppo

# Load PPO configuration
with open('examples/cfg/ppo/drone_ppo.yaml', 'r') as f:
    cfg_train = yaml.load(f, Loader=yaml.SafeLoader)

cfg_train["params"]["general"] = {
    "cfg": "examples/cfg/ppo/drone_ppo.yaml",
    "logdir": "examples/logs/DronePPO/ppo",
    "device": torch.device("cuda:0"),
    "seed": 0,
    "train": True,
    "checkpoint": "Base"
}

# Initialize PPO trainer
ppo_trainer = ppo.PPO(cfg_train)

# PPO training loop:
# 1. Collect trajectories without gradients through environment
# 2. Compute GAE advantages
# 3. Update actor with clipped surrogate objective
# 4. Update critic with TD-lambda targets
# 5. Update VAE for observation history encoding

ppo_trainer.train()
```

--------------------------------

### Load and Initialize GRaD-Nav++ Trainer (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Loads configuration for GRaD-Nav++ with a Vision-Language Model (VLM) and Mixture-of-Experts (MoE) architecture from a YAML file. It then initializes the trainer, setting up parameters for logging, device, and training mode.

```python
import yaml
import torch
import algorithms.gradnav_vla_moe as gradnav_vla

# Load GRaD-Nav++ configuration
with open('examples/cfg/gradnav_vla_moe/drone_long_task.yaml', 'r') as f:
    cfg_train = yaml.load(f, Loader=yaml.SafeLoader)

cfg_train["params"]["general"] = {
    "cfg": "examples/cfg/gradnav_vla_moe/drone_long_task.yaml",
    "logdir": "examples/logs/DroneVLALongTaskEnv/gradnav_MoE",
    "device": torch.device("cuda:0"),
    "seed": 0,
    "train": True,
    "render": False,
    "checkpoint": "Base"
}

# Initialize GRaD-Nav++ trainer with MoE architecture
traj_optimizer = gradnav_vla.GradNavVLAMoE(cfg_train)

# Train multi-task navigation policy
traj_optimizer.train()
```

--------------------------------

### Train Baseline PPO Model

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Trains a baseline Proximal Policy Optimization (PPO) model for drone navigation. The configuration and log directory are provided.

```python
python examples/train_ppo.py --cfg examples/cfg/ppo/drone_ppo.yaml --logdir examples/logs/DronePPO/ppo
```

--------------------------------

### Test Baseline PPO Model

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests the trained baseline PPO model for drone navigation. This command loads the specified checkpoint and enables rendering.

```python
python examples/train_ppo.py --cfg examples/cfg/ppo/drone_ppo.yaml --checkpoint examples/logs/DronePPO/ppo/<map_name>/<time_stamp>/best_policy.pt --play --render
```

--------------------------------

### Train Baseline BPTT Model

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Trains a baseline Backpropagation Through Time (BPTT) model for long trajectory drone navigation. Configuration and logging parameters are specified.

```python
python examples/train_bptt.py --cfg examples/cfg/bptt/drone_long_traj.yaml --logdir examples/logs/DroneLongTraj/bptt
```

--------------------------------

### Drone Environment Step Function (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Executes a single simulation step for a quadrotor in a 3D environment. It integrates dynamics, 3DGS rendering, and reward computation. Inputs include actions and VAE latent vectors, with outputs covering observations, rewards, and termination flags.

```python
import torch
from envs.drone_long_traj import DroneLongTrajEnv

# Initialize environment
env = DroneLongTrajEnv(
    num_envs=128,
    device='cuda:0',
    render=False,
    episode_length=600,
    stochastic_init=True,
    map_name='gate_mid',
    env_hyper={
        'SINGLE_VISUAL_INPUT_SIZE': 16,
        'HISTORY_BUFFER_NUM': 5,
        'LATENT_VECT_NUM': 24
    }
)

# Reset environment
obs, privilege_obs = env.initialize_trajectory()
# obs.shape: (128, 57) - partial observations for policy
# privilege_obs.shape: (128, 67) - full state for critic

# Prepare VAE latent vector
vae_info = torch.randn(128, 24).cuda()  # From VAE encoder

# Execute action
actions = torch.randn(128, 4).cuda()  # [roll_rate, pitch_rate, yaw_rate, thrust]
actions = torch.tanh(actions)  # Normalize to [-1, 1]

obs, privilege_obs, obs_hist, obs_vel, reward, done, extras = env.step(actions, vae_info)

# Returns:
# - obs: (128, 57) partial observations
# - privilege_obs: (128, 67) full state observations
# - obs_hist: (128, 285) concatenated history for VAE (5 * 57)
# - obs_vel: (128, 3) linear velocity for VAE training
# - reward: (128,) shaped rewards including:
#   * survive_reward: 8.0 per step
#   * obstacle_reward: distance-based collision avoidance
#   * waypoint_reward: exponential proximity to navigation waypoints
#   * target_reward: velocity alignment with reference trajectory
#   * pose_penalty: deviation from upright orientation
#   * action_penalty: body rate and thrust regularization
#   * smooth_penalty: action smoothness
# - done: (128,) episode termination flags
# - extras: dict with obs_before_reset for bootstrap value estimation

# Check termination conditions:
# - Body rate exceeds 15 rad/s
# - Height outside [0, 4] meters
# - Position outside map boundaries
# - Episode length exceeds limit
# - NaN/Inf in observations

```

--------------------------------

### Download and Unzip Data

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Downloads and unzips required data for the project, including point cloud and 3DGS data, using gdown. Ensure you are in the correct directory before executing.

```bash
cd grad_nav/envs/assets
gdown --id 1xIDb1-HFkniBagvMbWDgfy4okKTckwra # point cloud data
unzip point_cloud.zip
gdown --id 1skbXqg__Ew3ytNmqe_xiaqHpexZnsfdw # 3DGS data
unzip gs_data.zip
cd ../../
```

--------------------------------

### Quadrotor Dynamics Simulation (PyTorch)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Simulates advanced quadrotor dynamics including inner-loop attitude control and outer-loop position control using PyTorch. It initializes a QuadrotorSimulator with various physical and control parameters and runs the simulation forward by a specified time step with given control inputs.

```python
import torch
from envs.assets.quadrotor_dynamics_advanced import QuadrotorSimulator

# Initialize quadrotor simulator
num_drones = 128
mass = torch.full((num_drones,), 1.1).cuda()  # kg
max_thrust = torch.full((num_drones,), 26.0).cuda()  # Newtons
link_length = 0.15  # meters (motor arm length)

inertia_diag = torch.tensor([0.01, 0.012, 0.025]).cuda()  # kg*m^2
inertia = torch.diag(inertia_diag).unsqueeze(0).repeat(num_drones, 1, 1)

# PD controller gains for attitude control
Kp = torch.tensor([1.0, 1.2, 2.5]).unsqueeze(0).repeat(num_drones, 1).cuda()
Kd = torch.tensor([0.001, 0.001, 0.002]).unsqueeze(0).repeat(num_drones, 1).cuda()

quad_sim = QuadrotorSimulator(
    mass=mass,
    inertia=inertia,
    link_length=link_length,
    Kp=Kp,
    Kd=Kd,
    freq=200.0,  # Inner loop control frequency (Hz)
    max_thrust=max_thrust,
    total_time=0.05,  # Simulation duration (seconds)
    rotor_noise_std=0.01,
    br_noise_std=0.01
)

# Current state
position = torch.randn(num_drones, 3).cuda()  # (x, y, z) meters
velocity = torch.randn(num_drones, 3).cuda()  # m/s
orientation = torch.randn(num_drones, 4).cuda()  # Quaternion (w, x, y, z)
orientation = torch.nn.functional.normalize(orientation, dim=1)
angular_velocity = torch.randn(num_drones, 3).cuda()  # rad/s

# Control input from policy
body_rate_cmd = torch.randn(num_drones, 3).cuda() * 0.5  # Desired body rates (rad/s)
thrust_cmd = torch.rand(num_drones, 1).cuda() * 0.5 + 0.25  # Normalized thrust [0, 1]

control_input = (body_rate_cmd, thrust_cmd)

# Run simulation forward by dt=0.05s
new_pos, new_vel, new_ang_vel, new_quat, new_lin_acc, new_ang_acc = quad_sim.run_simulation(
    position=position,
    velocity=velocity,
    orientation=orientation,  # Input: (w, x, y, z)
    angular_velocity=angular_velocity,
    control_input=control_input
)

# Returns:
# - new_pos: (128, 3) updated positions
# - new_vel: (128, 3) updated linear velocities
# - new_ang_vel: (128, 3) updated angular velocities
# - new_quat: (128, 4) updated orientations (w, x, y, z)
# - new_lin_acc: (128, 3) linear accelerations
# - new_ang_acc: (128, 3) angular accelerations

# Note: Simulation includes:
# - Gravity and aerodynamic drag
# - Gyroscopic effects from rotors
# - Motor dynamics with first-order lag
# - Attitude PD controller tracking body rate commands
# - Process noise for robustness

```

--------------------------------

### Test Baseline BPTT Model

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests the trained baseline BPTT model for long trajectory drone navigation. It requires the configuration and checkpoint path, and enables play/render.

```python
python examples/train_bptt.py --cfg examples/cfg/bptt/drone_long_traj.yaml --checkpoint examples/logs/DroneLongTraj/bptt/<map_name>/<time_stamp>/best_policy.pt --play --render
```

--------------------------------

### Train Critic Network and Optimizer Step (PyTorch)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Trains a critic network by calculating the loss between predicted and target values, performing backpropagation, and updating network weights using an optimizer. It includes gradient clipping for stable training.

```python
predicted_values = critic(privilege_obs[:32])
critic_loss = ((predicted_values.squeeze(-1) - target_values[0]) ** 2).mean()
critic_optimizer.zero_grad()
critic_loss.backward()
nn.utils.clip_grad_norm_(critic.parameters(), max_norm=1.0)
critic_optimizer.step()
```

--------------------------------

### Compute and Optimize VAE Loss (PyTorch)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Calculates the VAE loss, including reconstruction and KL divergence, and optimizes the VAE model. It shows how to use the VAE's loss function and update the model weights using an AdamW optimizer with gradient clipping.

```python
# Compute VAE loss for training
next_obs = torch.randn(128, num_obs).cuda()
loss_dict = vae.loss_fn(obs_history, next_obs)

vae_loss = loss_dict['loss'].mean()
recons_loss = loss_dict['recons_loss'].mean()
kld_loss = loss_dict['kld_loss'].mean()

# Optimize VAE
vae_optimizer = torch.optim.AdamW(vae.parameters(), lr=5e-4, weight_decay=1e-2)
vae_optimizer.zero_grad()
vae_loss.backward()
torch.nn.utils.clip_grad_norm_(vae.parameters(), max_norm=1.0)
vae_optimizer.step()
```

--------------------------------

### Drone Long Trajectory Configuration (YAML)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Configuration file for GradNav's DroneLongTrajEnv, specifying environment parameters, network architectures, VAE settings, and training hyperparameters such as learning rates, discount factors, and rollout lengths.

```yaml
params:
  diff_env:
    name: DroneLongTrajEnv
    stochastic_env: True
    episode_length: 600
    MM_caching_frequency: 16  # 3DGS model caching frequency

  network:
    actor: ActorStochasticMLP
    actor_mlp:
      units: [512, 256, 128]
      activation: elu
    critic: CriticMLP
    critic_mlp:
      units: [512, 256, 128]
      activation: elu

  vae:
    kl_weight: 1.0
    encoder_units: [256, 256, 256]
    decoder_units: [32, 64, 128, 256]

  env_hyper:
    SINGLE_VISUAL_INPUT_SIZE: 16
    HISTORY_BUFFER_NUM: 5
    LATENT_VECT_NUM: 24

  config:
    name: drone_long_traj
    map_name: gate_mid  # Options: gate_mid, gate_right
    actor_learning_rate: 1e-4
    critic_learning_rate: 1e-4
    vae_learning_rate: 5e-4
    lr_schedule: cosine  # Options: constant, linear, cosine
    target_critic_alpha: 0.2  # Target network update rate
    obs_rms: True  # Observation normalization
    ret_rms: False  # Return normalization
    critic_iterations: 16
    critic_method: td-lambda  # Options: td-lambda, one-step
    lambda: 0.95  # TD-lambda parameter
    num_batch: 4
    gamma: 0.99  # Discount factor
    betas: [0.7, 0.95]  # Adam optimizer betas
    steps_num: 32  # Rollout length
    grad_norm: 1.0  # Gradient clipping threshold
    truncate_grads: True
    num_actors: 128  # Parallel environments
    save_interval: 50
    max_epochs: 600
    domain_randomization: True  # Randomize mass, inertia, thrust

    # Low Pass Filter for action smoothing
    LPF_train: True
    LPF_val: 0.5

    player:
      determenistic: True
      games_num: 1
      num_actors: 1
      print_stats: True
      LPF_eval: True
      LPF_val: 0.5

```

--------------------------------

### Train GRaD-Nav Algorithm with PyTorch

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Trains a drone navigation policy using gradient-based optimization with differentiable dynamics and 3D Gaussian Splatting rendering. This script loads a YAML configuration, initializes the GRaD-Nav trainer, and executes the training loop. The training process involves differentiable forward simulation, actor loss computation, VAE updates, critic network training, and policy saving based on episode reward.

```python
import yaml
import torch
import algorithms.gradnav as gradnav

# Load training configuration
with open('examples/cfg/gradnav/drone_long_traj.yaml', 'r') as f:
    cfg_train = yaml.load(f, Loader=yaml.SafeLoader)

# Configure general parameters
cfg_train["params"]["general"] = {
    "cfg": "examples/cfg/gradnav/drone_long_traj.yaml",
    "logdir": "examples/logs/DroneLongTraj/gradnav",
    "device": torch.device("cuda:0"),
    "seed": 0,
    "train": True,
    "render": False,
    "checkpoint": "Base",
    "no_time_stamp": False
}

# Initialize GRaD-Nav trainer
traj_optimizer = gradnav.GradNav(cfg_train)

# Train the policy
# - Runs differentiable forward simulation through environment
# - Computes actor loss using discounted rewards and value estimates
# - Updates VAE for history encoding
# - Trains critic network using TD-lambda returns
# - Saves best policy based on episode reward
traj_optimizer.train()

# Example output during training:
# iter 100: ep loss -1250.32, ep discounted loss -980.15, vae loss 0.45,
# ep len 485.2, fps total 2840.15, value loss 12.34,
# grad norm before clip 2.15, grad norm after clip 1.00
```

--------------------------------

### Test Trained Drone Navigation Policy with PyTorch

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Evaluates a trained drone navigation policy in test environments with rendering enabled. This script loads a YAML configuration, sets test-specific parameters including a checkpoint path, and initializes the GRaD-Nav trainer. The `play` method is then called to run the evaluation, generating trajectory visualizations such as ego-video, trajectory plots, and pose data.

```python
import yaml
import torch
import algorithms.gradnav as gradnav

# Load test configuration
with open('examples/cfg/gradnav/drone_long_traj.yaml', 'r') as f:
    cfg_test = yaml.load(f, Loader=yaml.SafeLoader)

# Set test mode parameters
cfg_test["params"]["general"] = {
    "cfg": "examples/cfg/gradnav/drone_long_traj.yaml",
    "checkpoint": "examples/logs/DroneLongTraj/gradnav/gate_mid/2024_01_15_10_30_45/best_policy.pt",
    "device": torch.device("cuda:0"),
    "seed": 0,
    "train": False,
    "play": True,
    "render": True  # Enable visualization output
}

# Reduce number of parallel actors for testing
cfg_test["params"]["config"]["num_actors"] = 1

# Load trained policy and evaluate
traj_optimizer = gradnav.GradNav(cfg_test)
traj_optimizer.play(cfg_test)

# Outputs trajectory visualizations to:
# examples/outputs/drone_long_traj/gate_mid/<timestamp>/
# - ego_video.mp4: First-person view from 3DGS rendering
# - traj_plot.png: Top-down trajectory with waypoints
# - z_plot.png: Height over time
# - pose_plot.png: Roll, pitch, yaw angles
# - velo_plot.png: Linear velocities
# - body_rate_plot.png: Body rate commands and actual rates
```

--------------------------------

### Actor Network for Continuous Action Space (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Implements a stochastic actor network using MLPs for continuous action spaces, defining a diagonal Gaussian policy. It supports forward passes for sampling actions during training and for obtaining mean actions during evaluation, as well as evaluating log probabilities for PPO.

```python
import torch
import torch.nn as nn
from torch.distributions.normal import Normal
from models import model_utils

# Initialize actor networkobs_dim = 57  # Observation dimension (position, velocity, visual features, latent)
action_dim = 4  # Actions: [roll_rate, pitch_rate, yaw_rate, thrust]
cfg_network = {
    'actor_mlp': {
        'units': [512, 256, 128],
        'activation': 'elu'
    },
    'actor_logstd_init': -1.0
}

actor = models.actor.ActorStochasticMLP(obs_dim, action_dim, cfg_network, device='cuda:0')

# Forward pass during training
obs = torch.randn(128, obs_dim).cuda()  # Batch of observations
deterministic = False  # Sample from policy distribution

action = actor(obs, deterministic=deterministic)
# action.shape: (128, 4) - sampled actions from Gaussian distribution

# Forward pass during evaluation (deterministic)
action_mean = actor(obs, deterministic=True)
# action_mean.shape: (128, 4) - mean of policy distribution

# Evaluate log probabilities for PPO
action_samples = torch.randn(128, action_dim).cuda()
log_probs = actor.evaluate_actions_log_probs(obs, action_samples)
# log_probs.shape: (128, 4) - log probability of each action dimension

# Actor network structure:
# Input: obs (57-dim) -> Linear(57, 512) -> ELU -> LayerNorm(512)
#                     -> Linear(512, 256) -> ELU -> LayerNorm(256)
#                     -> Linear(256, 128) -> ELU -> LayerNorm(128)
#                     -> Linear(128, 4) -> mu_net output
# Learnable parameter: logstd (4-dim) for diagonal covariance

```

--------------------------------

### 3D Gaussian Splatting Rendering (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Renders RGB images and depth maps from a 3D Gaussian Splatting scene representation. It takes camera poses as input and outputs rendered images. This functionality is used for visual simulation and potentially for generating training data.

```python
import torch
from utils.gs_local import get_gs
from pathlib import Path

# Initialize 3DGS model
gs_dir = Path("envs/assets/gs_data")
map_name = "sv_1007_gate_mid"
resolution_quality = 0.4  # Lower = faster, higher = better quality

gs = get_gs(map_name, gs_dir, resolution_quality)

# Prepare camera poses for batch rendering
num_cameras = 128
positions = torch.randn(num_cameras, 3).cuda()  # Camera positions
orientations = torch.randn(num_cameras, 4).cuda()  # Quaternions (x, y, z, w)
orientations = torch.nn.functional.normalize(orientations, dim=1)

# 3DGS coordinate system requires transformation
gs_origin_offset = torch.tensor([[-6.0, 0., 0.]]).repeat(num_cameras, 1).cuda()
gs_positions = positions + gs_origin_offset
gs_positions[:, 1] = -gs_positions[:, 1]  # Flip Y
gs_positions[:, 2] = -gs_positions[:, 2]  # Flip Z

# Prepare pose input: [x, y, z, roll, pitch, yaw, qx, qy, qz, qw]
gs_pose = torch.cat([
    gs_positions,
    torch.zeros(num_cameras, 3).cuda(),  # Euler angles (computed internally)
    orientations
], dim=-1)

# Render RGB and depth
depth_map, rgb_image = gs.render(gs_pose)
# depth_map.shape: (128, H, W, 1) - depth in meters
# rgb_image.shape: (128, H, W, 3) - RGB values [0, 1]

```

--------------------------------

### Process Depth Map for Obstacle Detection (PyTorch)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Processes a depth map to identify minimum depth values in the upper half of the image, useful for obstacle detection in drone navigation. It utilizes PyTorch tensor operations and assumes input tensors are in the format (batch, H, W, channels).

```python
batch, H, W, ch = depth_map.shape
depth_upper = depth_map[:, 0:int(H/2), :, :]  # Focus on upper half
min_depth = torch.abs(torch.amin(depth_upper, dim=(1,2,3)))
# min_depth.shape: (128,) - minimum depth in front of drone
```

--------------------------------

### Initialize and Encode History with VAE (PyTorch)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Initializes a Variational Autoencoder (VAE) for encoding observation history into a compact latent representation. It demonstrates how to encode flattened observation history and extract the sampled latent vector, mean, and log variance.

```python
import torch
from models.vae import VAE

# Initialize VAE
num_obs = 57
num_history = 5
num_latent = 24
kld_weight = 1.0

vae = VAE(
    num_obs=num_obs,
    num_history=num_history,
    num_latent=num_latent,
    kld_weight=kld_weight,
    activation='elu',
    decoder_hidden_dims=[32, 64, 128, 256],
    encoder_hidden_dims=[256, 256, 256],
    device='cuda:0'
).cuda()

# Encode observation history
obs_history = torch.randn(128, num_obs * num_history).cuda()
latent_vector, (mu, logvar) = vae.forward(obs_history)
```

--------------------------------

### Test GRaD-Nav++ VLA Multi Map (All Tasks)

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests all tasks for the GRaD-Nav++ VLA model in a multi-map environment using an automated script. Requires the checkpoint path and enables play/render.

```python
python auto_test_vla_multi_map_moe.py --checkpoint examples/logs/DroneVLAMultiMapEnv/gradnav_MoE/<map_name>/<time_stamp>/final_policy.pt --play --render
```

--------------------------------

### Test GRaD-Nav++ VLA Long Task (All Tasks)

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests all tasks for the GRaD-Nav++ VLA model in a long task environment using an automated testing script. Requires the checkpoint path.

```python
python auto_test_vla_long_task_moe.py --checkpoint examples/logs/DroneVLALongTaskEnv/gradnav_MoE/<map_name>/<time_stamp>/final_policy.pt --play --render
```

--------------------------------

### Test GRaD-Nav Multi-Gate

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests the GRaD-Nav model in a multi-gate scenario. This command loads a specific checkpoint and enables playback and rendering.

```python
python examples/train_gradnav.py --cfg examples/cfg/gradnav/drone_multi_gate.yaml --checkpoint examples/logs/DroneMultiGate/gradnav/<map_name>/<time_stamp>/best_policy.pt --play --render
```

--------------------------------

### Test GRaD-Nav++ VLA Long Task (Single Task)

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests a single task for the GRaD-Nav++ VLA model in a long task environment. Requires specifying the configuration and checkpoint.

```python
python examples/train_gradnav_vla_moe.py --cfg examples/cfg/gradnav_vla_moe/drone_long_task.yaml --checkpoint examples/logs/DroneVLALongTaskEnv/gradnav_MoE/<map_name>/<time_stamp>/final_policy.pt --play --render
```

--------------------------------

### Test GRaD-Nav++ VLA Multi Map (Single Task)

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Tests a single task for the GRaD-Nav++ VLA model in a multi-map environment. The command specifies the configuration and checkpoint for testing.

```python
python examples/train_gradnav_vla_moe.py --cfg examples/cfg/gradnav_vla_moe/drone_multi_map.yaml --checkpoint examples/logs/DroneVLAMultiMapEnv/gradnav_MoE/<map_name>/<time_stamp>/final_policy.pt --play --render
```

--------------------------------

### Train GRaD-Nav Long Trajectory

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Initiates the training process for the GRaD-Nav model on long trajectories using a specified configuration file. Logs are saved to the specified log directory.

```python
python examples/train_gradnav.py --cfg examples/cfg/gradnav/drone_long_traj.yaml --logdir examples/logs/DroneLongTraj/gradnav
```

--------------------------------

### Point Cloud Collision Detection (Python)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Calculates distances to obstacles using point cloud data for reward shaping and collision detection. It takes drone positions and orientations as input, outputting nearest obstacle distances. This is crucial for reward calculation and triggering safety-related resets.

```python
import torch
from utils.point_cloud_util import ObstacleDistanceCalculator
from pathlib import Path

# Load point cloud from PLY file
ply_file = Path("envs/assets/point_cloud/sv_1007_gate_mid.ply")
point_cloud = ObstacleDistanceCalculator(ply_file=ply_file)

# Compute distances for batch of drone positions and orientations
num_drones = 128
drone_positions = torch.randn(num_drones, 3).cuda()  # (x, y, z) in meters
drone_orientations = torch.randn(num_drones, 4).cuda()  # Quaternion (x, y, z, w)
drone_orientations = torch.nn.functional.normalize(drone_orientations, dim=1)

# Get nearest obstacle distances
distances = point_cloud.compute_nearest_distances(
    drone_positions,
    drone_orientations
)
# distances.shape: (128,) - distance to nearest obstacle in meters

# Use in reward computation
obst_threshold = 0.5  # Start penalizing below 0.5m
ostacle_strength = 1.0
obst_reward = torch.where(
    distances < obst_threshold,
    distances * obstacle_strength,  # Linear penalty
    torch.zeros_like(distances)
)

# Collision detection for early termination
obst_collision_limit = 0.20  # 20cm safety margin
collision_mask = distances < obst_collision_limit
# Reset environments where collision occurred

```

--------------------------------

### Train GRaD-Nav++ VLA Multi Map

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Trains the GRaD-Nav++ model for multi-map environments using VLA. This command specifies the configuration and log directory for multi-map training.

```python
python examples/train_gradnav_vla_moe.py --cfg examples/cfg/gradnav_vla_moe/drone_multi_map.yaml --logdir examples/logs/DroneVLAMultiMapEnv/gradnav_MoE
```

--------------------------------

### Extract Visual Features with CNN (PyTorch)

Source: https://context7.com/qianzhong-chen/grad_nav/llms.txt

Extracts compact visual features from RGB images using a pre-trained CNN (VisualPerceptionNet). The process involves reshaping the input tensor, resizing it to a fixed dimension, and passing it through the network. Assumes PyTorch tensors and CUDA availability.

```python
from models.squeeze_net import VisualPerceptionNet

visual_net = VisualPerceptionNet(visual_feature_size=16).cuda()
rgb_tensor = rgb_image.permute(0, 3, 1, 2)  # (B, C, H, W)
resize = torch.nn.AdaptiveAvgPool2d((224, 224))
rgb_resized = resize(rgb_tensor)
visual_features = visual_net(rgb_resized)
# visual_features.shape: (128, 16) - compact visual representation
```

--------------------------------

### Test GRaD-Nav Long Trajectory

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Executes the trained GRaD-Nav model for long trajectory testing. It requires a checkpoint path and enables rendering for visualization.

```python
python examples/train_gradnav.py --cfg examples/cfg/gradnav/drone_long_traj.yaml --checkpoint examples/logs/DroneLongTraj/gradnav/<map_name>/<time_stamp>/best_policy.pt --play --render
```

--------------------------------

### Train GRaD-Nav++ VLA Long Task

Source: https://github.com/qianzhong-chen/grad_nav/blob/main/README.md

Trains the GRaD-Nav++ model for long tasks using a Vision-Language Model (VLA) approach. The configuration and logging paths are specified.

```python
python examples/train_gradnav_vla_moe.py --cfg examples/cfg/gradnav_vla_moe/drone_long_task.yaml --logdir examples/logs/DroneVLALongTaskEnv/gradnav_MoE
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.