NVIDIA AI Enterprise VMware Deployment Guide

https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/
Admin
NVIDIA AI Enterprise is a software platform designed to enable the deployment and management of AI...
Tokens:25,508
Snippets:256
Trust Score:10
Update:1 year ago
Show doc for...
Tokens:
Raw
### Verify Docker Installation

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Runs the 'hello-world' Docker image to confirm that Docker Engine is installed and functioning correctly.

```bash
sudo docker run hello-world
```

--------------------------------

### Combined Startup Script Example

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

An example of a combined startup script that executes multiple commands or scripts in sequence during container initialization. This is useful for complex setups.

```bash
#!/bin/bash

echo "Starting services..."
/etc/my_init.d/service1.sh
/etc/my_init.d/service2.sh
echo "Services started."
```

--------------------------------

### Start and Enable Docker Service

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Ensures the Docker service is running and configured to start automatically on system boot, a prerequisite for Kubernetes installation.

```bash
sudo systemctl start docker && sudo systemctl enable docker
```

--------------------------------

### Install NVIDIA Driver

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Executes the NVIDIA driver installer script using the 'sh' command. It's recommended to accept default options during the installation process.

```bash
sudo sh ./NVIDIA-Linux-x86_64-550.127.05-grid.run
```

--------------------------------

### Install NVIDIA Driver with .run file on Ubuntu

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Instructions for installing the NVIDIA driver on Ubuntu using the .run file provided by NVIDIA. This method allows for direct installation of the driver without relying on package managers.

```bash
# Example commands to install NVIDIA driver on Ubuntu using .run file
# Download the appropriate .run file from NVIDIA's driver download page.
# Ensure you are in the directory where the .run file is downloaded.
chmod +x NVIDIA-Linux-x86_64-*.run
sudo ./NVIDIA-Linux-x86_64-*.run
```

--------------------------------

### Install NVIDIA Driver Dependencies

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Installs essential development libraries and tools required for compiling and installing the NVIDIA drivers. This includes elfutils-libelf-devel, tar, bzip2, make, gcc, gcc-c++, and pciutils.

```bash
dnf install elfutils-libelf-devel.x86_64
dnf install -y tar bzip2 make automake gcc gcc-c++ pciutils libglvnd-devel
```

--------------------------------

### Set Execute Permissions and Run NVIDIA Driver Installer

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Changes the permissions of the downloaded NVIDIA driver .run file to make it executable and then runs the installer script using 'sh'.

```bash
cd vgpu-guest-driver-5_v5.2/
sudo chmod +x NVIDIA-Linux-x86_64-550.127.05-grid.run
sudo sh ./NVIDIA-Linux-x86_64-550.127.05-grid.run
```

--------------------------------

### Install Docker Dependencies (Ubuntu)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Installs necessary packages on Ubuntu to allow the apt package index to use repositories over HTTPS, a prerequisite for installing Docker.

```bash
sudo apt-get update
sudo apt-get install -y \
 apt-transport-https \
 ca-certificates \
 curl \
 gnupg-agent \
 software-properties-common
```

--------------------------------

### Install Kubernetes Components (Ubuntu)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Installs specific versions of kubelet, kubeadm, and kubectl on Ubuntu Xenial, and holds them to prevent automatic updates.

```bash
sudo apt-get update
sudo apt-get install -y -q kubelet=1.21.1-00 kubeadm=1.21.1-00 kubeadm=1.21.1-00
sudo apt-mark hold kubelet kubeadm kubectl
```

--------------------------------

### Install RHEL Kernel Development Packages

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Installs the kernel-devel and kernel-headers packages specific to the currently running kernel version. These are required by the NVIDIA driver for proper installation and compatibility.

```bash
dnf install -y kernel-devel-$(uname -r) kernel-headers-$(uname -r)
```

--------------------------------

### Update Ubuntu System and Install Build Tools

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Updates the package list and installs the 'build-essential' package, which includes the gcc compiler and make tool, required for compiling kernel modules.

```bash
sudo apt-get update
sudo apt-get install build-essential
```

--------------------------------

### Create Systemd Service for Triton (Systemd Unit File)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Defines a systemd service to automatically start the Triton Inference Server. It specifies the service description and the command to execute, which is the startup script.

```systemd
[Unit]
Description=Starts Triton server

[Service]
ExecStart=/home/nvidia/startup.sh

[Install]
WantedBy=multi-user.target
```

--------------------------------

### Install NGC CLI

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Steps to install the NVIDIA GPU Cloud (NGC) Command Line Interface (CLI). The NGC CLI is used to download and manage AI models, frameworks, and other resources from the NGC registry.

```bash
# Example command to download and install NGC CLI
# Visit the NGC CLI Install page for the most up-to-date instructions.
curl -fsSL https://raw.githubusercontent.com/NVIDIA/ngc-cli/main/scripts/install.sh | bash
```

--------------------------------

### Update RHEL Kernel and Install Sources

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Updates the system to the latest kernel and installs the corresponding kernel development packages. This ensures that the installed kernel matches the available kernel source files, a requirement for NVIDIA driver installation.

```bash
dnf install -y kernel kernel-core kernel-modules
dnf install -y kernel-devel-$(uname -r) kernel-headers-$(uname -r)
```

--------------------------------

### Create Triton Startup Script (Bash)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

This script is designed to run the Triton Inference Server within a Docker container. It removes existing containers, starts a new one with GPU support and port mappings, and mounts a local directory for models.

```bash
#!/bin/bash
docker rm -f $(docker ps -a -q)
docker run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8000:8000 -p8001:8001 -p8002:8002 --name triton_server_cont -v /home/nvidia/triton:/models nvcr.io/nvidia/tritonserver:<CONTAINER-TAG> tritonserver --model-store=/models --strict-model-config=false --log-verbose=1
```

--------------------------------

### Install TAO Toolkit

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Provides steps for installing the TAO Toolkit, a framework for training and deploying computer vision models. This often involves downloading and setting up a specific environment.

```bash
bash ngc patch deploy --container tao-toolkit --version 5.0
```

--------------------------------

### Reboot System and Verify NVIDIA Driver Installation

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Reboots the system to apply the driver changes and then uses the 'nvidia-smi' command to verify that the NVIDIA driver is successfully installed and the vGPU device is recognized.

```bash
sudo reboot
nvidia-smi
```

--------------------------------

### Install NVIDIA Driver with .run file on RHEL

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Instructions for installing the NVIDIA driver on Red Hat Enterprise Linux (RHEL) using the .run file provided by NVIDIA. This process involves disabling Nouveau and ensuring compatibility.

```bash
# Example commands to install NVIDIA driver on RHEL using .run file
# Ensure Nouveau driver is disabled first.
# Download the appropriate .run file from NVIDIA's driver download page.
chmod +x NVIDIA-Linux-x86_64-*.run
sudo ./NVIDIA-Linux-x86_64-*.run
```

--------------------------------

### Check Podman version on RHEL 8

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-runtimes

This command is used to verify the installed version of Podman after installation. It displays details such as the version number, API version, Go version, and build information.

```bash
podman version
```

--------------------------------

### Startup Scripts for Individual Containers

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Demonstrates how to create and use startup scripts for individual containers, allowing custom initialization or commands to run when a container starts.

```bash
echo "#!/bin/bash\npython /app/my_script.py" > /etc/my_init.d/my_container_init.sh
```

--------------------------------

### Install container-tools module on RHEL 8

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-runtimes

This command installs the 'container-tools' module on RHEL 8, which includes Podman and its dependencies. It ensures that Podman is set up correctly for container management.

```bash
sudo dnf module install -y container-tools
```

--------------------------------

### Install Docker Engine (Ubuntu)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Installs Docker Engine Community Edition and containerd.io on Ubuntu Bionic, specifying exact versions for stability.

```bash
sudo apt-get update
sudo apt-get install -y docker-ce=5:19.03.12~3-0~ubuntu-bionic docker-ce-cli=5:19.03.12~3-0~ubuntu-bionic containerd.io
```

--------------------------------

### Make Startup Script Executable (Bash)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Grants execute permissions to the `startup.sh` script, allowing it to be run as a command or service. This is a standard Linux permission change.

```bash
chmod +x ~/startup.sh
```

--------------------------------

### Startup Scripts for Triton Inference Server

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Specific instructions or examples for using startup scripts with the NVIDIA Triton Inference Server, perhaps for pre-loading models or configuring parameters.

```bash
docker run --gpus all -v /path/to/models:/models nvcr.io/nvidia/tritonserver:<yy.mm>-py3 
    tritonserver --model-repository=/models --strict-model-config=false
```

--------------------------------

### Start Multiple AI Containers with Jupyter Notebooks

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

This script automates the startup of multiple NVIDIA AI Enterprise containers, including PyTorch, TensorFlow (versions 1 and 2), and RAPIDS, each with its own Jupyter notebook instance. It ensures all containers are run in detached mode, maps ports, and binds local datasets to the container workspace. Replace the placeholder for the container tag and use absolute paths for the dataset volumes.

```shell
#!/bin/bash
docker rm -f $(docker ps -a -q)
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 --name pytorch_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvidia/pytorch:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8888
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8889:8889 --name tensorflow1_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvidia/tensorflow:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8889
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8890:8890 --name tensorflow2_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvidia/tensorflow:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8890
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8891:8891 --name rapids_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvidia/nvidia-rapids-:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8891
```

--------------------------------

### Manage Jupyter Systemd Service (Systemctl Commands)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Commands to start a systemd service and enable it to run on system reboot. These commands are used after creating the jupyter.service file.

```bash
sudo systemctl start jupyter.service
sudo systemctl enable jupyter.service
```

--------------------------------

### Install EPEL and DKMS

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Installs the Extra Packages for Enterprise Linux (EPEL) repository and the Dynamic Kernel Module System (DKMS). EPEL provides additional packages, and DKMS is crucial for building kernel modules, including NVIDIA drivers.

```bash
dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
dnf install dkms
```

--------------------------------

### Enable and Start nginx Service

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/tanzu

Enables the nginx service to start on boot and then starts the nginx service immediately. This makes the web server operational.

```bash
systemctl enable nginx
systemctl start nginx
```

--------------------------------

### Install NVIDIA Network Operator

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

This Helm command installs the NVIDIA Network Operator in the 'network-operator' namespace. It applies the custom configuration from `network-operator-values.yaml` and waits for the installation to complete.

```bash
helm install -f ./network-operator-values.yaml -n network-operator --create-namespace --wait network-operator mellanox/network-operator
```

--------------------------------

### Download NVIDIA vGPU Guest Driver from NGC

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Downloads a specific version of the NVIDIA vGPU Guest Driver from the NGC registry using the NGC CLI.

```bash
ngc registry resource download-version "nvidia/vgpu/vgpu-guest-driver-5:5.2"
```

--------------------------------

### Register RHEL Machine

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Registers the current machine to RHEL using the subscription-manager. This is a prerequisite for installing packages and updates from Red Hat repositories.

```bash
subscription-manager register
```

--------------------------------

### Configure Docker Stable Repository (Ubuntu)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Sets up the stable Docker repository for Ubuntu, enabling the installation of specific Docker Engine versions.

```bash
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
```

--------------------------------

### Install GPU Operator with RDMA Enabled

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Installs the GPU Operator with RDMA support enabled. This command assumes the Network Operator is already installed and requires the driver.rdma.enabled flag to be set to true.

```bash
helm install --wait gpu-operator nvaie/gpu-operator -n gpu-operator --set driver.rdma.enabled=true
```

--------------------------------

### Make NVIDIA Driver Executable

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Changes the file permissions of the NVIDIA driver .run file to make it executable. This is a necessary step before running the installer script.

```bash
sudo chmod +x NVIDIA-Linux-x86_64-550.127.05-grid.run
```

--------------------------------

### Create Systemd Service for Jupyter (Systemd Unit File)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Defines a systemd service to manage the execution of a startup script for Jupyter Notebook. It specifies the description and the command to execute.

```systemd
[Unit]
Description=Starts Jupyter server

[Service]
ExecStart=/home/nvidia/startup.sh #Use your home path

[Install]
WantedBy=multi-user.target
```

--------------------------------

### Install GPU Operator

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Installs the GPU Operator using Helm. The --wait flag ensures the command waits for resources to be ready. The -n gpu-operator flag specifies the namespace.

```bash
helm install --wait --generate-name nvaie/gpu-operator -n gpu-operator
```

--------------------------------

### Add NVIDIA Helm Repository

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

This command adds the Mellanox Helm repository, which is necessary for installing the NVIDIA Network Operator. Ensure Helm is installed prior to executing this command.

```bash
helm repo add mellanox https://mellanox.github.io/network-operator
```

--------------------------------

### Check container-tools module availability on RHEL 8

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-runtimes

This command checks for the availability of the 'container-tools' module on RHEL 8 systems. It helps identify different versions of the module that can be installed, including Podman, buildah, and skopeo.

```bash
sudo dnf module list | grep container-tools
```

--------------------------------

### Install nginx on Ubuntu

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/tanzu

Installs the nginx web server on an Ubuntu system. This is a prerequisite for serving local package mirrors.

```bash
apt-get update
apt-get install nginx
```

--------------------------------

### Manage Triton Systemd Service (Systemctl Commands)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Starts the Triton systemd service and configures it to launch automatically upon system reboot. These commands ensure the Triton server is active after VM restarts.

```bash
sudo systemctl start triton.service
sudo systemctl enable triton.service
```

--------------------------------

### Install NVIDIA Driver on Ubuntu using .run file

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs the NVIDIA driver on an Ubuntu system using the .run installation file. Ensure you have the .run file downloaded before proceeding.

```bash
sudo bash NVIDIA-Linux-x86_64-*.run
```

--------------------------------

### Create Triton Inference Server Startup Script and Systemd Service

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-advanced-framework

This snippet covers the process of creating a startup script for Triton Inference Server. It includes pulling the latest Triton container, defining the script to run the container with specific configurations and port mappings, making the script executable, and setting up a systemd service to manage its execution on boot.

```shell
mkdir ~/triton
sudo docker pull nvcr.io/nvaie/tritonserver-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG>
vim ~/startup.sh
#!/bin/bash
docker rm -f $(docker ps -a -q)
docker run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8000:8000 -p8001:8001 -p8002:8002 --name triton_server_cont -v $HOME/triton_models:/models nvcr.io/nvaie/tritonserver-<NVAIE-MAJOR-VERSION>:<NVAIE-CONTAINER-TAG> tritonserver --model-store=/models --strict-model-config=false --log-verbose=1
chmod +x ~/startup.sh
sudo vim /etc/systemd/system/triton.service
[Unit]
Description=Starts Triton server

[Service]
ExecStart=/home/nvidia/startup.sh

[Install]
WantedBy=multi-user.target

sudo systemctl start triton.service
sudo systemctl enable triton.service
```

--------------------------------

### Enable Startup Script Execution (Bash)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Grants execute permissions to a shell script. This is a prerequisite for running the script as a service. It requires the absolute path to the script.

```bash
chmod +x /home/nvidia/startup.sh
```

--------------------------------

### Reboot System

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Restarts the virtual machine. A reboot is typically required after installing hardware drivers to allow the system to load and initialize them correctly.

```bash
sudo reboot
```

--------------------------------

### Install NVIDIA AI Enterprise Host Software (VIB) on ESXi

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/vgpu

Installs the NVIDIA AI Enterprise Host Software package on an ESXi host using the `esxcli` command. Requires the absolute path to the VIB file. Note that a reboot is recommended for the VIB to load correctly, even if the command indicates otherwise.

```shell
[root@esxi:~] esxcli software vib install -v directory/NVIDIA-AIE_ESXi_6.7.0_Driver_470.105-1OEM.670.0.0.8169922.vib
Installation Result Message: Operation finished successfully.
Reboot Required: false
VIBs Installed: NVIDIA-AIE_ESXi_6.7.0_Driver_470.105-1OEM.670.0.0.8169922
VIBs Removed:
VIBs Skipped: 
```

--------------------------------

### Pull TAO Toolkit PyTorch Docker Image

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/installing-ai

Pulls the Docker image for the TAO Toolkit with PyTorch support. Ensure Docker is installed.

```shell
sudo docker pull nvcr.io/nvidia/tao-toolkit-pyt:<NVAIE-CONTAINER-TAG>
```

--------------------------------

### RAPIDS Container Startup Script

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-advanced-framework

A bash script to stop all running Docker containers and then start a new container for the RAPIDS AI framework. It maps the local dataset directory to the container's workspace and exposes the Jupyter notebook port.

```bash
#!/bin/bash
docker rm -f $(docker ps -a -q)
docker run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvidia/rapidsai/notebooks:<CONTAINER-TAG> jupyter-notebook --allow-root --ip='0.0.0.0'
```

--------------------------------

### Install NVIDIA AI Enterprise Host Software (VIB)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Steps to prepare, upload, install, update, verify, and uninstall the NVIDIA AI Enterprise Host Software VIB on VMware ESXi hosts.

```bash
esxcli software vib install -v=<vib_file_path>
```

```bash
esxcli software vib update -v=<vib_file_path>
```

```bash
esxcli software vib list | grep NVIDIA
```

```bash
esxcli software vib remove -n <vib_name>
```

--------------------------------

### Install NGC Catalog CLI

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Instructions for installing the NGC Catalog Command Line Interface (CLI), which is used to manage NVIDIA NGC container images and software.

```bash
bash ngc-cli-installer.sh
```

--------------------------------

### Log into NGC Container Registry

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-installing-ai

This command logs you into the NVIDIA NGC container registry using a special username '$oauthtoken' and your API key for authentication. Ensure your API key is kept secret.

```bash
docker login nvcr.io
```

--------------------------------

### Verify NGC Catalog CLI Installation

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-prereqs

This command verifies that the NGC Catalog CLI has been successfully installed. It should output the installed version number.

```bash
ngc --version
```

--------------------------------

### Install NVIDIA Triton Inference Server

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Provides instructions or commands for installing the NVIDIA Triton Inference Server, a software package that simplifies the deployment of trained AI models at scale.

```bash
docker run --gpus all -p 8000:8000 -p 8001:8001 -p 8002:8002 nvcr.io/nvidia/tritonserver:<yy.mm>-py3
```

--------------------------------

### Create Dataset Directory

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-advanced-framework

This command creates a directory named 'dataset' in the user's home directory. This directory is intended to store datasets for use with Jupyter notebooks.

```bash
mkdir ~/dataset
```

--------------------------------

### Reload System Daemon

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Reloads the systemd manager configuration to apply changes made during the installation of Kubernetes components.

```bash
sudo systemctl daemon-reload
```

--------------------------------

### Label Kubernetes Nodes

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

This command labels all Kubernetes nodes to exclude the 'master' role for the node controller. This is a preparatory step before installing the network operator.

```bash
kubectl label nodes --all node-role.kubernetes.io/master- --overwrite
```

--------------------------------

### Enable Startup Script

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Details the process of enabling a startup script within a container or system, ensuring it runs automatically upon boot or container start.

```bash
chmod +x /etc/my_init.d/my_startup_script.sh
update-rc.d my_startup_script.sh defaults
```

--------------------------------

### Install and Configure apt-mirror for Ubuntu Package Mirroring

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/tanzu

Installs the apt-mirror package and configures it to create a local mirror of Ubuntu repositories. This is crucial for air-gapped or restricted internet environments to provide necessary packages like kernel headers and images for GPU operator deployments.

```bash
$ apt install apt-mirror
#configure mirror.list
vim /etc/apt/mirror.list
############# config ##################
#
# set base_path /var/spool/apt-mirror
#
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads 20
set _tilde 0
#
############## end config ##############
clean http://archive.ubuntu.com/ubuntu
deb http://archive.ubuntu.com/ubuntu focal-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-proposed main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-backports main restricted universe multiverse





$ /usr/bin/apt-mirror
```

--------------------------------

### Test Docker and NVIDIA Container Runtime

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

A command to test if the Docker installation and NVIDIA Container Runtime are correctly configured and can access the GPU. This confirms the setup for GPU-accelerated containers.

```bash
docker run --rm --gpus all nvidia/cuda:11.0.0-base-ubuntu20.04 nvidia-smi
```

--------------------------------

### Install NVIDIA Driver on Ubuntu using .run file

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Instructions for installing the NVIDIA AI Enterprise software driver on an Ubuntu virtual machine using the .run file. This process typically involves downloading the driver from NGC and executing the installer.

```bash
sudo sh NVIDIA-Linux-x86_64-XXX.XX.run
```

--------------------------------

### Add Kubernetes GPG Key and Repository (Ubuntu)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Adds the necessary GPG key for the Kubernetes apt repository and configures the repository source for Kubernetes components on Ubuntu Xenial.

```bash
sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo mkdir -p /etc/apt/sources.list.d/
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
```

--------------------------------

### Pull TAO Toolkit TensorFlow Docker Image

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/installing-ai

Pulls the Docker image for the TAO Toolkit with TensorFlow support. Docker must be installed.

```shell
sudo docker pull nvcr.io/nvidia/tao-toolkit-tf:<NVAIE-CONTAINER-TAG>
```

--------------------------------

### Download NVIDIA AI Enterprise Host Software (VMware ESXi)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/software

This command downloads the NVIDIA AI Enterprise Host Software VIB for VMware ESXi from the NGC catalog. Replace 'X:X.X' with the specific version you need.

```bash
ngc registry resource download-version "nvidia/vgpu/vgpu-host-driver-X:X.X"
```

--------------------------------

### Add and Update Helm Repository for NVIDIA AI Enterprise

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Adds the NVIDIA AI Enterprise Helm repository and updates it. Requires an NGC API key for authentication. This is a prerequisite for installing the GPU Operator.

```bash
helm repo add nvaie https://helm.ngc.nvidia.com/nvaie --username='$oauthtoken' --password=api-key && helm repo update
```

--------------------------------

### Install Helm Package Manager

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Downloads, extracts, and installs Helm 3.5.4 on a Linux system. Helm is a package manager for Kubernetes that simplifies the deployment and management of applications. This process involves fetching the archive, unpacking it, moving the binary to a system path, and cleaning up temporary files.

```shell
wget https://get.helm.sh/helm-v3.5.4-linux-amd64.tar.gz
tar -zxvf helm-v3.5.4-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm
rm -rf helm-v3.5.4-linux-amd64.tar.gz linux-amd64/
```

--------------------------------

### Configure Licensed Client on Linux

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Steps for configuring a licensed client on a Linux system to connect to the NVIDIA License System (NLS). This enables the client to obtain and utilize NVIDIA AI Enterprise licenses.

```bash
# Example commands for configuring NLS licensed client on Linux
# Ensure you have the NLS client utility installed.
# Refer to NVIDIA License System documentation for specific commands.
# Example: Copying the client configuration token
# cp <token_file_path> /path/to/nls/client/config/
```

--------------------------------

### Install PyTorch

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs PyTorch, a popular open-source machine learning framework, optimized for NVIDIA GPUs. Ensure CUDA toolkit compatibility.

```bash
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch
```

--------------------------------

### Pull TAO Toolkit LM Docker Image

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/installing-ai

Pulls the Docker image for the TAO Toolkit Language Model. Requires Docker to be installed and configured.

```shell
sudo docker pull nvcr.io/nvidia/tao-toolkit-lm:<NVAIE-CONTAINER-TAG>
```

--------------------------------

### Install NVIDIA RAPIDS

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Instructions for installing NVIDIA RAPIDS, a suite of open-source software libraries and APIs for running data science and analytics workflows entirely on GPUs.

```bash
bash conda install -c rapidsai -c conda-forge 
    cudatoolkit=11.0 
    rapids=0.20
```

--------------------------------

### Install NVIDIA Driver on RHEL using .run file

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs the NVIDIA driver on a Red Hat Enterprise Linux (RHEL) system using the .run installation file. It's recommended to have the .run file downloaded beforehand.

```bash
sudo sh NVIDIA-Linux-x86_64-*.run
```

--------------------------------

### Install NVIDIA TensorRT

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs NVIDIA TensorRT, a platform for high-performance deep learning inference. This typically involves downloading and installing a package or using NGC.

```bash
sudo dpkg -i nv-tensorrt-repo-ubuntu2004-8.x.x.x-1_amd64.deb
```

--------------------------------

### Create Triton Inference Server Directory (Bash)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Creates a directory named 'triton' within the user's home directory. This directory is intended to host AI models for the Triton Inference Server.

```bash
mkdir ~/triton
```

--------------------------------

### Install TensorFlow

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs TensorFlow, another widely-used open-source machine learning library, with GPU support. Verify CUDA and cuDNN compatibility.

```bash
pip install tensorflow[and-cuda]
```

--------------------------------

### Install Helm Package Manager

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs the Helm package manager, a tool for managing Kubernetes applications. Helm simplifies the deployment and management of applications on Kubernetes clusters.

```bash
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
```

--------------------------------

### Install NVIDIA GPU Operator with Helm

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/tanzu

Installs the NVIDIA GPU Operator using a Helm chart. Requires Helm and the GPU Operator chart to be available. Installs into the 'gpu-operator' namespace.

```bash
helm install --wait gpu-operator ./gpu-operator -n gpu-operator
```

--------------------------------

### Install NVIDIA Container Toolkit for MIG-Enabled vGPU

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-gpu

Installs the NVIDIA Container Toolkit (`nvidia-docker2`) which is necessary to run containers on MIG-enabled vGPUs. This process involves adding the NVIDIA Docker repository, updating package lists, and installing the package, followed by a Docker service restart.

```shell
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
&& distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
&& curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list \
&& sudo apt-get update

sudo apt-get install -y nvidia-docker2 \
&& sudo systemctl restart docker
```

--------------------------------

### Install Kubernetes CLI Tools (kubectl)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs the kubectl command-line tool, which is used to interact with Kubernetes clusters. This is a prerequisite for managing TKG clusters and deploying operators.

```bash
tanzu plugin install kubectl
```

--------------------------------

### Combined Startup Script for AI Enterprise Containers

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-advanced-framework

This script automates the startup of multiple NVIDIA AI Enterprise containers, including PyTorch, TensorFlow (1 and 2), and RAPIDS, each with its own Jupyter notebook instance. It ensures all containers are run in detached mode, with GPU access, and specific port mappings. Replace `<NVAIE-CONTAINER-TAG>` with the correct tag for each container. Note that a home directory path like `/home/nvidia/dataset` should be used instead of `$HOME`.

```shell
#!/bin/bash
docker rm -f $(docker ps -a -q)
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 --name pytorch_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/pytorch:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8888
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8889:8889 --name tensorflow1_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/tensorflow:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8889
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8890:8890 --name tensorflow2_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/tensorflow:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8890
docker run -d --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8891:8891 --name rapids_cont -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvaie/nvidia-rapids:<NVAIE-CONTAINER-TAG> jupyter-notebook --allow-root --NotebookApp.token='' --ip='0.0.0.0' --port 8891
```

--------------------------------

### Configure apt to use local mirror

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/tanzu

Modifies the apt sources list to point to a local directory serving as a package mirror. This allows installing packages without internet access.

```bash
vim /etc/apt/sources.list
###### Ubuntu Main Repos
deb file:///var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/
/ubuntu/ focal main restricted universe multiverse
```

--------------------------------

### Configure NGC CLI

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

This command configures the NGC CLI by prompting the user for their API key, desired output format, organization, team, and ACE. It saves the configuration to the user's home directory.

```bash
ngc config set

Enter API key [no-apikey]. Choices: [<VALID_APIKEY>, 'no-apikey']:

Enter CLI output format type [ascii]. Choices: [ascii, csv, json]: ascii

Enter org [no-org]. Choices: ['no-org']:

Enter team [no-team]. Choices: ['no-team']:

Enter ace [no-ace]. Choices: ['no-ace']:

Successfully saved NGC configuration to /home/$username/.ngc/config
```

--------------------------------

### Enable Jupyter Startup Script Execution and Systemd Service

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-advanced-framework

This snippet demonstrates how to grant execution permissions to a startup script for Jupyter Notebook, create a systemd service to manage the script, and start/enable the service for automatic execution on reboot. It involves using `chmod`, `vim`, and `systemctl` commands.

```shell
chmod +x /home/nvidia/startup.sh
sudo vim /etc/systemd/system/jupyter.service
[Unit]
Description=Starts Jupyter server

[Service]
ExecStart=/home/nvidia/startup.sh #Use your home path

[Install]
WantedBy=multi-user.target

sudo systemctl start jupyter.service
sudo systemctl enable jupyter.service
journalctl -f -u jupyter.service
```

--------------------------------

### Validate Network Operator with GPUDirect RDMA

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/kubernetes

Executes a command to list Mellanox NICs and their status, useful for validating the Network Operator setup for GPUDirect RDMA. It uses kubectl exec to run 'ibdev2netdev' inside a pod.

```bash
kubectl exec -it $(kubectl get pods -n nvidia-network-operator-resources | grep mofed | awk '{print $1}') -n nvidia-network-operator-resources -- ibdev2netdev
```

--------------------------------

### Install NVIDIA Container Toolkit

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Steps to install the NVIDIA Container Toolkit, which allows Docker containers to access NVIDIA GPUs. This is essential for running AI workloads in containers.

```bash
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-container-toolkit/gpgkey | sudo apt-key add -
sudo curl -s -L https://nvidia.github.io/nvidia-container-toolkit/$distribution/nvidia-container-toolkit.list | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
```

--------------------------------

### Configure NVIDIA runtime for rootless containers

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-runtimes

This command modifies the NVIDIA container runtime configuration file to enable rootless container execution. It changes the 'no-cgroups' setting to 'true' in the config.toml file, which is necessary for privileged users to avoid conflicts.

```bash
sudo sed -i 's/^#no-cgroups = false/no-cgroups = true/;' /etc/nvidia-container-runtime/config.toml
```

--------------------------------

### Pull TensorFlow Container using Docker

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-installing-ai

This command pulls the TensorFlow Docker container, part of the open-source machine learning platform. Users should replace '<NVAIE-CONTAINER-TAG>' with the appropriate tag corresponding to their NVIDIA AI Enterprise installation.

```docker
sudo docker pull nvcr.io/nvidia/tensorflow:<NVAIE-CONTAINER-TAG>
```

--------------------------------

### Generate NGC API Key

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/first-vm

Guide on generating an API key from the NVIDIA NGC (NVIDIA GPU Cloud) private registry. This key is required for authentication when accessing NGC resources via the CLI or other tools.

```bash
# Access the NGC sign-in page to generate your API key.
# Typically involves navigating to your profile settings after logging in.
# Example CLI command using the generated API key:
# ngc config set --apikey <your_api_key>
```

--------------------------------

### Find NVIDIA VIB for Uninstallation

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/vgpu

This command lists all installed software packages on the ESXi host and filters for NVIDIA-related drivers, which is the first step before uninstalling the NVIDIA AI Enterprise Host Software.

```bash
esxcli software vib list | grep -i nvidia
```

--------------------------------

### TensorFlow 1 Container Startup Script

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-advanced-framework

A bash script designed to stop all currently running Docker containers and launch a new TensorFlow 1 container. It configures GPU access, shared memory, memory locking, and maps a local directory for datasets.

```bash
#!/bin/bash
docker rm -f $(docker ps -a -q)
docker run --gpus all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -p8888:8888 -v /home/nvidia/dataset:/workspace/dataset nvcr.io/nvidia/tensorflow:<CONTAINER-TAG> jupyter-notebook --allow-root --ip='0.0.0.0'
```

--------------------------------

### Get Kubernetes Cluster Contexts

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/tanzu

Retrieves a list of available Kubernetes contexts that can be used to switch between different clusters or namespaces.

```bash
kubectl config get-contexts
```

--------------------------------

### Startup Scripts for Jupyter

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Shows how to configure startup scripts specifically for Jupyter environments, enabling custom configurations or pre-loading of libraries.

```python
import os

def configure_jupyter():
    jupyter_dir = os.path.expanduser('~/.jupyter')
    if not os.path.exists(jupyter_dir):
        os.makedirs(jupyter_dir)
    with open(os.path.join(jupyter_dir, 'jupyter_notebook_config.py'), 'a') as f:
        f.write('c.NotebookApp.ip = "0.0.0.0"\n')

configure_jupyter()
```

--------------------------------

### Set up Web Server for Local Package Repository

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

This section details the setup of a web server to host a local package repository for NVIDIA AI Enterprise components. This typically involves configuring a web server like Nginx or Apache to serve files from a specific directory.

```Shell
# Example for Nginx setup (conceptual)
# Ensure Nginx is installed and running
# Create a directory for the repository
sudo mkdir -p /var/www/html/nvidia-ai-enterprise-repo
# Download or copy NVIDIA AI Enterprise packages to this directory
# Configure Nginx to serve this directory
# Example Nginx config snippet:
# server {
#     listen 80;
#     server_name your_server_ip;
#     root /var/www/html/nvidia-ai-enterprise-repo;
#     location / {
#         autoindex on;
#     }
# }
# sudo systemctl restart nginx
```

--------------------------------

### Download NVIDIA vGPU Guest Driver for Ubuntu (NGC CLI)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/software

This command downloads a specific version of the NVIDIA vGPU Guest Driver for Ubuntu from the NGC registry. It utilizes the NGC CLI and requires the exact version string.

```bash
ngc registry resource download-version "nvidia/vgpu/vgpu-guest-driver-X:X.X"
```

--------------------------------

### Install GPU Operator using Helm

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs the NVIDIA GPU Operator using Helm, applying the customized configurations and pointing to the local image registry. This command assumes Helm is configured and the repository is added.

```bash
helm install gpu-operator <chart-path> -n gpu-operator --create-namespace -f values.yaml
```

--------------------------------

### Configure NVIDIA vGPU Guest Driver

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Information on installing and configuring the NVIDIA vGPU Guest Driver within virtual machines for GPU virtualization.

```bash
bash NVIDIA-Linux-x86_64-XXX.XX-vgpu-driver.run
```

--------------------------------

### Install Docker and NVIDIA Container Toolkit

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs Docker and the NVIDIA Container Toolkit, which allows Docker containers to access NVIDIA GPUs. This is essential for running GPU-accelerated applications in containers.

```bash
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
```

--------------------------------

### Pull PyTorch Container using Docker

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/cpu-installing-ai

This Docker command fetches the PyTorch container image, which is a GPU-accelerated tensor computational framework. Customize '<NVAIE-CONTAINER-TAG>' with the specific version tag for your NVIDIA AI Enterprise deployment.

```docker
sudo docker pull nvcr.io/nvidia/pytorch:<NVAIE-CONTAINER-TAG>
```

--------------------------------

### Mounting NFS ISO Data Store

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Steps to mount a Network File System (NFS) data store for ISO images within VMware vSphere, often used for OS installations.

```bash
esxcli storage nfs add --host <nfs_server_ip> --share <nfs_share_path> --volume-name <datastore_name>
```

```bash
esxcli storage nfs list
```

--------------------------------

### Install NVIDIA Network Operator

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/index

Installs the NVIDIA Network Operator using Helm. This operator manages the configuration and deployment of NVIDIA networking components within a Kubernetes cluster.

```bash
helm install --create-namespace nvidia-network-operator nvidia/network-operator -n gpu-operator
```

--------------------------------

### Reboot ESXi Host via Command Prompt

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/vgpu

Reboots the ESXi host from the command prompt. This action does not return a response and will not refresh the vSphere Web Client UI.

```shell
reboot
```

--------------------------------

### View Jupyter Service Logs (Journalctl Command)

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-framework

Retrieves logs for a specific systemd service, typically used to find the authentication token for Jupyter Notebook. The -f flag follows the logs in real-time.

```bash
journalctl -f -u jupyter.service
```

--------------------------------

### Test NVIDIA Container Toolkit Installation

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/advance-gpu

Tests the installation of the NVIDIA Container Toolkit on a VM by running a Docker container with NVIDIA runtime support. It verifies that the container can access and utilize the MIG-enabled vGPU, using a specific MIG device format for `NVIDIA_VISIBLE_DEVICES`.

```shell
sudo docker run --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=MIG-GPU-786035d5-1e85-11b2-9fec-ac9c9a792daf/0/0 nvidia/cuda nvidia-smi
```

--------------------------------

### Verify NVIDIA GPU Communication with nvidia-smi on ESXi

Source: https://docs.nvidia.com/ai-enterprise/deployment/vmware/latest/vgpu

Confirms that the NVIDIA driver can communicate with the physical GPUs installed in the ESXi system. The `nvidia-smi` command provides detailed information about GPU utilization and status.

```shell
nvidia-smi
```
Tokens:
Raw