UniDriveVLA

https://github.com/xiaomi-research/unidrivevla
Admin
UniDriveVLA is a unified vision-language-action model for autonomous driving that integrates...
Tokens:188,355
Snippets:2,194
Trust Score:8.4
License:Apache-2.0
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install example-specific requirements

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/README.md

Command to install dependencies for a specific example folder.

```bash
pip install -r requirements.txt
```

--------------------------------

### nuScenes Setup

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/docs/installation.md

Install nuScenes-specific requirements and build custom operations.

```bash
pip install -r requirements/requirements_nusc.txt
```

```bash
cd nuScenes/projects/mmdet3d_plugin/ops
pip install -e . --no-build-isolation
cd ../../../..
```

--------------------------------

### Install Neptune Client

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/README.md

Installation commands for the Neptune client library.

```bash
pip install neptune
```

```bash
conda install -c conda-forge neptune
```

--------------------------------

### Running Examples on Remote Hardware

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/README.md

Commands to install Runhouse and execute example scripts on various remote hardware configurations.

```bash
# First install runhouse:
pip install runhouse

# For an on-demand V100 with whichever cloud provider you have configured:
python run_on_remote.py \
    --example pytorch/text-generation/run_generation.py \
    --model_type=gpt2 \
    --model_name_or_path=openai-community/gpt2 \
    --prompt "I am a language model and"

# For byo (bring your own) cluster:
python run_on_remote.py --host <cluster_ip> --user <ssh_user> --key_path <ssh_key_path> \
  --example <example> <args>

# For on-demand instances
python run_on_remote.py --instance <instance> --provider <provider> \
  --example <example> <args>
```

--------------------------------

### Install Transformers from source

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/README.md

Commands to clone the repository and install the library in a virtual environment.

```bash
git clone https://github.com/huggingface/transformers
cd transformers
pip install .
```

--------------------------------

### Install Documentation Builder

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/CONTRIBUTING.md

Command to install the documentation builder tool.

```bash
pip install hf-doc-builder
```

--------------------------------

### Install Cookiecutter

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_example_script/README.md

Install the cookiecutter tool required to generate the example script template.

```bash
pip install cookiecutter
```

--------------------------------

### Install Accelerate

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/multiple-choice/README.md

Install the Accelerate library required for custom training loops.

```bash
pip install git+https://github.com/huggingface/accelerate
```

--------------------------------

### Run example tests with pytest

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/CONTRIBUTING.md

Install requirements and execute tests for a specific example directory.

```bash
pip install -r examples/xxx/requirements.txt  # only needed the first time
python -m pytest -n auto --dist=loadfile -s -v ./examples/pytorch/text-classification
```

--------------------------------

### Setup MMDetection3D from Scratch

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

A complete script to create a conda environment and install MMDetection3D along with its dependencies.

```shell
conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab

# install latest PyTorch prebuilt with the default prebuilt CUDA version (usually the latest)
conda install -c pytorch pytorch torchvision -y

# install mmcv
pip install mmcv-full

# install mmdetection
pip install git+https://github.com/open-mmlab/mmdetection.git

# install mmsegmentation
pip install git+https://github.com/open-mmlab/mmsegmentation.git

# install mmdetection3d
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
pip install -v -e .
```

--------------------------------

### Set Up Development Environment

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/open_model_proposals/ADD_BIG_BIRD.md

Commands to create a virtual environment and install the Transformers library in editable mode with development dependencies.

```bash
python -m venv .env
source .env/bin/activate
pip install -e ".[dev]"
```

--------------------------------

### Build and Install GPU Version

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/build.md

Execute the build and installation commands for the GPU-enabled version.

```shell
# build
python setup.py build_ext # if success, cl will be launched to compile ops
# install
python setup.py develop
```

--------------------------------

### Install and Login to Weights & Biases

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/README.md

Commands to install the W&B package and authenticate the environment.

```bash
pip install wandb
```

```bash
wandb login
```

--------------------------------

### Generate Example Script Template

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_example_script/README.md

Run the cookiecutter command within the examples folder to generate the pre-filled template structure.

```bash
cookiecutter ../templates/adding_a_new_example_script/
```

--------------------------------

### Install ms-swift Environment

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/docs/vlm_pretraining.md

Install the ms-swift library within the qwenvl3 directory to enable Megatron-LM backend support.

```bash
cd qwenvl3
pip install ms-swift
# Ensure swift/cli/_megatron/sft.py is available
```

--------------------------------

### Install mmcv-full with Custom PyPI Source

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/installation.md

Use a specific PyPI mirror to speed up the installation of dependencies.

```bash
mim install mmcv-full -i https://pypi.tuna.tsinghua.edu.cn/simple
```

--------------------------------

### Install and configure Accelerate

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/README.md

Commands to install the development version of Accelerate and configure the environment for distributed training.

```bash
pip install git+https://github.com/huggingface/accelerate
```

```bash
accelerate config
```

```bash
accelerate test
```

```bash
accelerate launch path_to_script.py --args_to_script
```

--------------------------------

### Install pre-commit hooks

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/CONTRIBUTING.md

Install and configure pre-commit to ensure code style compliance.

```shell
pip install -U pre-commit
pre-commit install
```

--------------------------------

### Install and Configure Transformers

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/docs/installation.md

Install the required transformers version and apply the custom model files for Qwen3-VL support.

```bash
pip install transformers==4.57.1
```

```bash
CONDA_PREFIX=/path/to/anaconda3/ or /path/to/anaconda3/envs/env_name/
TRANSFORMERS_DIR=${CONDA_PREFIX}/lib/python3.9/site-packages/transformers/
cp -r qwenvl3/transformers_replace/models ${TRANSFORMERS_DIR}
```

--------------------------------

### Install Training Dependencies

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/docs/installation.md

Install core training libraries including DeepSpeed and Flash Attention.

```bash
pip install deepspeed==0.14.4
pip install peft
pip install timm
pip install qwen_vl_utils
```

```bash
pip install flash-attn --no-build-isolation
```

--------------------------------

### Install quality and testing dependencies

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/AGENTS.md

Commands to install the necessary tools for code style enforcement and testing within the environment.

```bash
pip install -e .[quality]
```

```bash
pip install -e .[testing]
```

```bash
pip install torch accelerate
```

--------------------------------

### Example Input IDs

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/ADD_NEW_MODEL_PROPOSAL_TEMPLATE.md

An example of an array of integers representing input IDs for a model.

```python
input_ids = [0, 4, 4, 3, 2, 4, 1, 7, 19]
```

--------------------------------

### Install Vulkan Dependencies

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/Bench2Drive/bench2drive/tools/check_carla.md

Installs necessary Vulkan tools and geometry libraries for troubleshooting startup failures.

```bash
apt install vulkan-tools # 	vulkaninfo
apt install libgeos-dev
```

--------------------------------

### Define and Load a Basic Config

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/understand_mmcv/config.md

Example of a simple Python configuration file and how to load it using Config.fromfile.

```python
a = 1
b = dict(b1=[0, 1, 2], b2=None)
c = (1, 2)
d = 'string'
```

```python
>>> cfg = Config.fromfile('test.py')
>>> print(cfg)
>>> dict(a=1,
...      b=dict(b1=[0, 1, 2], b2=None),
...      c=(1, 2),
...      d='string')
```

--------------------------------

### Clone and Install BigBird Repository

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/open_model_proposals/ADD_BIG_BIRD.md

Commands to clone the original BigBird repository and install it in editable mode.

```bash
git clone https://github.com/google-research/bigbird.git 
cd big_bird
pip install -e .
```

--------------------------------

### Install Unit Test Dependencies

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/CONTRIBUTING.md

Commands to install necessary system dependencies for running unit tests on Linux or Windows.

```shell
# Linux
sudo apt-get update -y
sudo apt-get install -y libturbojpeg
sudo apt-get install -y ffmpeg

# Windows
conda install ffmpeg
```

--------------------------------

### Validate MMCV installation

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/build.md

Run the installation check script to verify the environment setup.

```powershell
(mmcv) PS C:\Users\xxx\mmcv> python .dev_scripts/check_installation.py
```

--------------------------------

### Inference Execution Examples

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/1_exist_data_model.md

Practical examples for testing specific models like VoteNet, SECOND, and PointPillars on various datasets.

```shell
python tools/test.py configs/votenet/votenet_8x8_scannet-3d-18class.py \
    checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth \
    --show --show-dir ./data/scannet/show_results
```

```shell
python tools/test.py configs/votenet/votenet_8x8_scannet-3d-18class.py \
    checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth \
    --eval mAP
    --eval-options 'show=True' 'out_dir=./data/scannet/show_results'
```

```shell
python tools/test.py configs/votenet/votenet_8x8_scannet-3d-18class.py \
    checkpoints/votenet_8x8_scannet-3d-18class_20200620_230238-2cea9c3a.pth \
    --eval mAP
```

```shell
./tools/slurm_test.sh ${PARTITION} ${JOB_NAME} configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py \
    checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20200620_230238-9208083a.pth \
    --out results.pkl --eval mAP
```

```shell
./tools/slurm_test.sh ${PARTITION} ${JOB_NAME} configs/pointpillars/hv_pointpillars_fpn_sbn-all_4x8_2x_nus-3d.py \
    checkpoints/hv_pointpillars_fpn_sbn-all_4x8_2x_nus-3d_20200620_230405-2fa62f3d.pth \
    --format-only --eval-options 'jsonfile_prefix=./pointpillars_nuscenes_results'
```

--------------------------------

### Configure and Test Accelerate

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/multiple-choice/README.md

Set up the environment and verify readiness for distributed training.

```bash
accelerate config
```

```bash
accelerate test
```

--------------------------------

### Clone and Install Original Model Repository

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/ADD_NEW_MODEL_PROPOSAL_TEMPLATE.md

Commands to clone the original model repository and install it in editable mode for debugging.

```bash
git clone [link to original repo].git 
cd [lowercase name of model]
pip install -e .
```

--------------------------------

### Implement benchmark with MetricsRecorder

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/benchmark/README.md

Example implementation showing how to initialize a benchmark and record device and model measurements using MetricsRecorder.

```py
from benchmarks_entrypoint import MetricsRecorder
import psycopg2

def run_benchmark(logger: Logger, branch: str, commit_id: str, commit_msg: str, num_tokens_to_generate=100):
  metrics_recorder = MetricsRecorder(psycopg2.connect("dbname=metrics"), logger, branch, commit_id, commit_msg)
  benchmark_id = metrics_recorder.initialise_benchmark({"gpu_name": gpu_name, "model_id": model_id})
    # To collect device measurements
    metrics_recorder.collect_device_measurements(
        benchmark_id, cpu_util, mem_megabytes, gpu_util, gpu_mem_megabytes
    )
    # To collect your model measurements
    metrics_recorder.collect_model_measurements(
        benchmark_id,
        {
            "model_load_time": model_load_time,
            "first_eager_forward_pass_time_secs": first_eager_fwd_pass_time,
            "second_eager_forward_pass_time_secs": second_eager_fwd_pass_time,
            "first_eager_generate_time_secs": first_eager_generate_time,
            "second_eager_generate_time_secs": second_eager_generate_time,
            "time_to_first_token_secs": time_to_first_token,
            "time_to_second_token_secs": time_to_second_token,
            "time_to_third_token_secs": time_to_third_token,
            "time_to_next_token_mean_secs": mean_time_to_next_token,
            "first_compile_generate_time_secs": first_compile_generate_time,
            "second_compile_generate_time_secs": second_compile_generate_time,
            "third_compile_generate_time_secs": third_compile_generate_time,
            "fourth_compile_generate_time_secs": fourth_compile_generate_time,
        },
    )
```

--------------------------------

### Example Tensor Output

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/open_model_proposals/ADD_BIG_BIRD.md

A sample tensor output used for reference during the porting process.

```bash
tensor([[-0.0818,  0.2207, -0.0749, -0.0030,  0.0045, -0.1569, -0.1598,  0.0212,
         -0.2077,  0.2157],
        [ 0.1044,  0.0201,  0.0990,  0.2482,  0.3116,  0.2509,  0.2866, -0.2190,
          0.2166, -0.0212],
        [-0.2000,  0.1107, -0.1999, -0.3119,  0.1559,  0.0993,  0.1776, -0.1950,
         -0.1023, -0.0447],
        [-0.0888, -0.1092,  0.2281,  0.0336,  0.1817, -0.0115,  0.2096,  0.1415,
         -0.1876, -0.2467],
        [ 0.2208, -0.2352, -0.1426, -0.2636, -0.2889, -0.2061, -0.2849, -0.0465,
          0.2577,  0.0402],
        [ 0.1502,  0.2465,  0.2566,  0.0693,  0.2352, -0.0530,  0.1859, -0.0604,
          0.2132,  0.1680],
        [ 0.1733, -0.2407, -0.1721,  0.1484,  0.0358, -0.0633, -0.0721, -0.0090,
          0.2707, -0.2509],
        [-0.1173,  0.1561,  0.2945,  0.0595, -0.1996,  0.2988, -0.0802,  0.0407,
          0.1829, -0.1568],
        [-0.1164, -0.2228, -0.0403,  0.0428,  0.1339,  0.0047,  0.1967,  0.2923,
          0.0333, -0.0536],
        [-0.1492, -0.1616,  0.1057,  0.1950, -0.2807, -0.2710, -0.1586,  0.0739,
          0.2220,  0.2358]]).
```

--------------------------------

### Run a forward pass with a pretrained checkpoint

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/ADD_NEW_MODEL_PROPOSAL_TEMPLATE.md

A pseudocode example demonstrating how to load a model and perform a forward pass with dummy input IDs.

```python
model = [camelcase name of model]Model.load_pretrained_checkpoint("/path/to/checkpoint/")
input_ids = [0, 4, 5, 2, 3, 7, 9]  # vector of input ids
original_output = model.predict(input_ids)
```

--------------------------------

### Start Training Workflow

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/understand_mmcv/runner.md

Execute the training process using the configured workflow.

```python
# workflow is typically set as: workflow = [('train', 1)]
# here the training begins.
runner.run(data_loaders, cfg.workflow)
```

--------------------------------

### Install MMDetection3D

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

Final installation command for MMDetection3D.

```shell
pip install -v -e .  # or "python setup.py develop"
```

--------------------------------

### Example of multi-dimensional float array output

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/open_model_proposals/ADD_BIG_BIRD.md

Representation of layer output values as a multi-dimensional float array.

```bash
[[
 [-0.1465, -0.6501,  0.1993,  ...,  0.1451,  0.3430,  0.6024],
 [-0.4417, -0.5920,  0.3450,  ..., -0.3062,  0.6182,  0.7132],
 [-0.5009, -0.7122,  0.4548,  ..., -0.3662,  0.6091,  0.7648],
 ...,
 [-0.5613, -0.6332,  0.4324,  ..., -0.3792,  0.7372,  0.9288],
 [-0.5416, -0.6345,  0.4180,  ..., -0.3564,  0.6992,  0.9191],
 [-0.5334, -0.6403,  0.4271,  ..., -0.3339,  0.6533,  0.8694]]],
```

--------------------------------

### Display Seq2SeqTrainer help options

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/legacy/seq2seq/README.md

Run this command to view all available command-line arguments for the fine-tuning script.

```bash
python finetune_trainer.py --help
```

--------------------------------

### Install MMDetection3D

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/docs/installation.md

Install the MMDetection3D dependency from the third-party directory.

```bash
cd third_party/mmdetection3d-1.0.0rc6
pip install -e . --no-build-isolation
cd ../..
```

--------------------------------

### Bench2Drive Setup

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/docs/installation.md

Configure dependencies for Bench2Drive training and evaluation, including CARLA 0.9.15.

```bash
pip install -r requirements/requirements_b2d.txt
```

```bash
conda create -n unidrivevla_eval python=3.8
conda activate unidrivevla_eval
pip install -r requirements/requirements_b2d.txt
```

```bash
cd third_party/mmcv-1.7.2
export MMCV_WITH_OPS=1
export FORCE_CUDA=1
export MMCV_NO_Compiler_CHECK=1
pip install -r requirements.txt
python setup.py build_ext --inplace
pip install -e .
cd ../..

cd third_party/mmdetection3d-1.0.0rc6
pip install -e .
cd ../..
```

```bash
cd third_party/transformers-4.57.1
pip install -e .
cd ../..
```

```bash
mkdir -p carla && cd carla
wget https://carla-releases.s3.us-east-005.backblazeb2.com/Linux/CARLA_0.9.15.tar.gz
wget https://carla-releases.s3.us-east-005.backblazeb2.com/Linux/AdditionalMaps_0.9.15.tar.gz

tar -xvf CARLA_0.9.15.tar.gz
cd Import && tar -xvf ../AdditionalMaps_0.9.15.tar.gz
cd .. && bash ImportAssets.sh
```

```bash
export CARLA_ROOT=/path/to/unidrivevla/carla
echo "$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.15-py3.7-linux-x86_64.egg" >> $CONDA_PREFIX/lib/python3.9/site-packages/carla.pth
```

--------------------------------

### Configure and Launch Training with Accelerate

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/semantic-segmentation/README.md

Commands to set up the environment, verify configuration, and initiate the training process for semantic segmentation.

```bash
accelerate config
```

```bash
accelerate test
```

```bash
accelerate launch run_semantic_segmentation_no_trainer.py --output_dir segformer-finetuned-sidewalk --with_tracking --push_to_hub
```

--------------------------------

### Install MMSegmentation

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

Commands to install MMSegmentation either directly or from source.

```shell
pip install mmsegmentation
```

```shell
git clone https://github.com/open-mmlab/mmsegmentation.git
cd mmsegmentation
git checkout v0.20.0  # switch to v0.20.0 branch
pip install -e .  # or "python setup.py develop"
```

--------------------------------

### Initialize ClearML CLI

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/README.md

Run the initialization command to configure ClearML credentials locally.

```bash
clearml-init
```

--------------------------------

### Install MMDetection

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

Commands to install MMDetection either directly or from source.

```shell
pip install mmdet
```

```shell
git clone https://github.com/open-mmlab/mmdetection.git
cd mmdetection
git checkout v2.24.0  # switch to v2.24.0 branch
pip install -r requirements/build.txt
pip install -v -e .  # or "python setup.py develop"
```

--------------------------------

### Install libcst dependency

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/modular-transformers/README.md

Required installation step before using the modular converter.

```bash
pip install libcst
```

--------------------------------

### Setup CARLA 0.9.15 Environment

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/Bench2Drive/bench2drive/README.md

Commands to download, extract, and configure the CARLA simulator environment.

```bash
    mkdir carla
    cd carla
    wget https://carla-releases.s3.us-east-005.backblazeb2.com/Linux/CARLA_0.9.15.tar.gz
    tar -xvf CARLA_0.9.15.tar.gz
    cd Import && wget https://carla-releases.s3.us-east-005.backblazeb2.com/Linux/AdditionalMaps_0.9.15.tar.gz
    cd .. && bash ImportAssets.sh
    export CARLA_ROOT=YOUR_CARLA_PATH
    echo "$CARLA_ROOT/PythonAPI/carla/dist/carla-0.9.15-py3.7-linux-x86_64.egg" >> YOUR_CONDA_PATH/envs/YOUR_CONDA_ENV_NAME/lib/python3.7/site-packages/carla.pth # python 3.8 also works well, please set YOUR_CONDA_PATH and YOUR_CONDA_ENV_NAME
```

--------------------------------

### Install spconv2.0

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

Commands to install spconv2.0 for optimized GPU memory usage.

```bash
pip install cumm-cuxxx
pip install spconv-cuxxx
```

--------------------------------

### Install mmcv-full

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/build.md

Commands to compile and install mmcv-full with MLU support enabled.

```bash
cd mmcv
export MMCV_WITH_OPS=1
export FORCE_MLU=1
python setup.py install
```

--------------------------------

### Install PyTorch

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/build.md

Install the appropriate PyTorch version based on hardware requirements.

```powershell
# CUDA version
(mmcv) PS C:\Users\xxx> conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
# CPU version
(mmcv) PS C:\Users\xxx> conda install install pytorch torchvision cpuonly -c pytorch
```

--------------------------------

### Configure and Launch Accelerate Training

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/object-detection/README.md

Commands to set up the environment and execute the training script for object detection.

```bash
accelerate config
```

```bash
accelerate test
```

```bash
accelerate launch run_object_detection_no_trainer.py \
    --model_name_or_path "facebook/detr-resnet-50" \
    --dataset_name cppe-5 \
    --output_dir "detr-resnet-50-finetuned" \
    --num_train_epochs 100 \
    --image_square_size 600 \
    --per_device_train_batch_size 8 \
    --per_device_eval_batch_size 8 \
    --checkpointing_steps epoch \
    --learning_rate 5e-5 \
    --ignore_mismatched_sizes \
    --with_tracking \
    --push_to_hub
```

--------------------------------

### Configure Accelerate Environment

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/instance-segmentation/README.md

Initial setup command to configure the training environment for Accelerate.

```bash
accelerate config
```

--------------------------------

### Verify PyTorch Installation

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/installation.md

Check if PyTorch is correctly installed and identify its version.

```bash
python -c 'import torch;print(torch.__version__)'
```

--------------------------------

### Install ClearML package

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/README.md

Install the necessary ClearML Python package via pip.

```bash
pip install clearml
```

--------------------------------

### Build and install MMCV-full for Ascend NPU from source

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/build.md

Commands to pull the source code and compile MMCV with NPU support enabled.

```bash
git pull https://github.com/open-mmlab/mmcv/tree/master
```

```bash
MMCV_WITH_OPS=1 MAX_JOBS=8 FORCE_NPU=1 python setup.py build_ext
```

```bash
MMCV_WITH_OPS=1 FORCE_NPU=1 python setup.py develop
```

--------------------------------

### Install Comet ML

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/README.md

Installation commands for Comet ML using pip or conda.

```bash
pip install comet_ml
```

```bash
conda install -c comet_ml -c anaconda -c conda-forge comet_ml
```

--------------------------------

### Initialize a Model and Access Configuration

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/templates/adding_a_new_model/open_model_proposals/ADD_BIG_BIRD.md

Demonstrates loading a pre-trained model and accessing its associated configuration object.

```python
# assuming that `brand_new_bert` belongs to the organization `brandy`
model = BrandNewBertModel.from_pretrained("brandy/brand_new_bert")
model.config  # model has access to its config
```

--------------------------------

### Install git-lfs

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/pytorch/audio-classification/README.md

Installs the git-lfs package required for managing large model files.

```bash
$ apt install git-lfs
```

--------------------------------

### Render Documentation

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/CONTRIBUTING.md

Commands to install documentation dependencies and build the HTML documentation.

```shell
pip install -r requirements/docs.txt
cd docs/zh_cn/
# or docs/en
make html
# check file in ./docs/zh_cn/_build/html/index.html
```

--------------------------------

### Install Minkowski Engine

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

Commands to install the Minkowski Engine sparse convolution backend.

```shell
conda install openblas-devel -c anaconda
pip install -U git+https://github.com/NVIDIA/MinkowskiEngine -v --no-deps --install-option="--blas_include_dirs=/opt/conda/include" --install-option="--blas=openblas"
```

--------------------------------

### Install PyTorch

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmdetection3d-1.0.0rc6/docs/en/getting_started.md

Commands to install PyTorch and torchvision for either GPU or CPU platforms.

```shell
conda install pytorch torchvision -c pytorch
```

```shell
conda install pytorch torchvision cpuonly -c pytorch
```

--------------------------------

### Sequential Model Initialization

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/zh_cn/understand_mmcv/cnn.md

Demonstrates initializing a Sequential model with individual layer configurations and shows that inner configurations override global ones.

```python
init_cfg1 = dict(type='Constant', layer='Conv1d', val=0., bias=1.)
init_cfg2 = dict(type='Constant', layer='Conv2d', val=2., bias=3.)
model1 = FooConv1d(init_cfg1)
model2 = FooConv2d(init_cfg2)
seq_model = Sequential(model1, model2)
seq_model.init_weights()
```

```python
model1 = FooConv1d(init_cfg1)
model2 = FooConv2d(init_cfg2)
init_cfg = dict(type='Constant', layer=['Conv1d', 'Conv2d'], val=4., bias=5.)
seq_model = Sequential(model1, model2, init_cfg=init_cfg)
seq_model.init_weights()
```

--------------------------------

### Install Build Dependencies

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/build.md

Install optional and runtime dependencies required for compilation and execution.

```powershell
(mmcv) PS C:\Users\xxx\mmcv> pip install -r requirements/optional.txt
```

```shell
(mmcv) PS C:\Users\xxx\mmcv> pip install -r requirements/runtime.txt
```

--------------------------------

### Install MMCV via pip

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/docs/en/get_started/installation.md

Use this command to install the MMCV package in your environment.

```bash
pip install mmcv
```

--------------------------------

### Install mmcv

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/mmcv-1.7.2/README.md

Standard command to install the lite version of MMCV using OpenMIM.

```bash
pip install -U openmim
mim install mmcv
```

--------------------------------

### Setup mosesdecoder and wmt16-scripts

Source: https://github.com/xiaomi-research/unidrivevla/blob/main/third_party/transformers-4.57.1/examples/legacy/seq2seq/romanian_postprocessing.md

Clone the necessary repositories to the home directory to prepare the postprocessing environment.

```bash
cd $HOME
git clone git@github.com:moses-smt/mosesdecoder.git
cd mosesdecoder  
git clone git@github.com:rsennrich/wmt16-scripts.git
```
Tokens:
Raw