Jetson AI Lab Tutorials

https://www.jetson-ai-lab.com/tutorials/
Admin
Jetson AI Lab Tutorials provides step-by-step guides to deploy, experiment, and run cutting-edge AI...
Tokens:22,340
Snippets:234
Trust Score:9.4
Update:9 months ago
Show doc for...
Tokens:
Raw
### Install aiohttp Module

Source: https://www.jetson-ai-lab.com/tutorials/nanoowl

This command installs the `aiohttp` Python module, which is a dependency for running the NanoOWL tree demonstration example.

```bash
pip install aiohttp
```

--------------------------------

### Clone and Setup jetson-containers

Source: https://www.jetson-ai-lab.com/tutorials/nanoowl

This snippet shows how to clone the jetson-containers repository and execute the installation script. This is a prerequisite for running NanoOWL and other jetson-containers applications.

```bash
git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh
```

--------------------------------

### Install SDK Manager on Ubuntu 20.04

Source: https://www.jetson-ai-lab.com/tutorials/initial-setup-sdk-manager

Installs the NVIDIA SDK Manager on an Ubuntu 20.04 system. The process includes downloading a .deb package, refreshing the package index, and installing the sdkmanager package. Root privileges are necessary for these operations.

```bash
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install sdkmanager
```

--------------------------------

### Install SDK Manager on Ubuntu 22.04

Source: https://www.jetson-ai-lab.com/tutorials/initial-setup-sdk-manager

Installs the NVIDIA SDK Manager on an Ubuntu 22.04 system. This involves downloading a .deb package, updating package lists, and then installing the sdkmanager package. It requires administrative privileges for package installation.

```bash
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install sdkmanager
```

--------------------------------

### Install HuggingFace Hub and Login

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Installs the HuggingFace Hub library and logs into HuggingFace. This is necessary for accessing gated repositories, such as Llama models, which require authentication.

```bash
pip install huggingface_hub
huggingface-cli login
```

--------------------------------

### Serve Llama-3.1-8B-Instruct (FP16 Baseline)

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Starts the vLLM server for the FP16 version of the Llama-3.1-8B-Instruct model. This serves as the baseline for evaluating performance and quality before applying quantization techniques.

```bash
vllm serve meta-llama/Llama-3.1-8B-Instruct
```

--------------------------------

### Launch SDK Manager

Source: https://www.jetson-ai-lab.com/tutorials/initial-setup-sdk-manager

Launches the NVIDIA SDK Manager application after it has been installed. This command assumes the SDK Manager executable is in the system's PATH. It's typically used after completing the installation steps.

```bash
sdkmanager
```

--------------------------------

### Install QSPI Updater Package (Shell)

Source: https://www.jetson-ai-lab.com/tutorials/initial-setup-jetson-orin-nano

Installs the `nvidia-l4t-jetson-orin-nano-qspi-updater` package, which contains the necessary tools to update the Quad Serial Peripheral Interface (QSPI) flash memory on the Jetson Orin Nano Developer Kit. This requires an active internet connection.

```shell
sudo apt-get install nvidia-l4t-jetson-orin-nano-qspi-updater
```

--------------------------------

### Install Docker and nvidia-container on JetPack 6.x

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This script updates package lists, installs `nvidia-container` and `curl`, downloads and installs Docker using the official convenience script, enables the Docker service, and configures the `nvidia-ctk` to use Docker as its runtime. This is specifically for JetPack 6.x where Docker is not automatically installed with `nvidia-container`.

```bash
sudo apt update
sudo apt install -y nvidia-container curl
curl https://get.docker.com | sh && sudo systemctl --now enable docker
sudo nvidia-ctk runtime configure --runtime=docker
```

--------------------------------

### Install nvidia-container Package

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

Installs the `nvidia-container` package using apt. This is a prerequisite for using NVIDIA container runtimes with Docker. Note that on JetPack 6.x, Docker may need to be installed separately.

```bash
sudo apt update
sudo apt install -y nvidia-container
```

--------------------------------

### Launch NanoOWL Tree Demo

Source: https://www.jetson-ai-lab.com/tutorials/nanoowl

This command launches the NanoOWL tree prediction demo. It requires navigating to the example directory, specifying the camera index and resolution, and providing the path to the TensorRT engine file.

```python
cd examples/tree_demo
python3 tree_demo.py --camera 0 --resolution 640x480 \
    ../../data/owl_image_encoder_patch32.engine
```

--------------------------------

### Launch vLLM Server for a Model

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Starts the vLLM inference server, making a specified model available via an OpenAI-compatible API. This command loads the model and prepares it for requests. Startup time can vary significantly based on model size and hardware.

```bash
vllm serve openai/gpt-oss-120b
```

--------------------------------

### Clone and Start Live VLM WebUI Container (Shell)

Source: https://www.jetson-ai-lab.com/tutorials/live-vlm-webui

Clones the Live VLM WebUI repository from GitHub and starts the application using a provided script. This assumes Ollama or another VLM backend is already configured.

```shell
git clone https://github.com/nvidia-ai-iot/live-vlm-webui.git
cd live-vlm-webui
./scripts/start_container.sh
```

--------------------------------

### Monitor Disk Usage

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command starts a utility that monitors and displays disk usage information, updating every second. It is useful for observing changes in disk space as Docker operations occur.

```bash
watch -n1 df
```

--------------------------------

### Configure Docker Default Runtime using jq

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command uses `jq` to add or update the `default-runtime` setting in `/etc/docker/daemon.json` to `nvidia`. It first installs `jq`, then pipes the modified JSON to a temporary file, and finally moves the temporary file to replace the original `daemon.json`.

```bash
sudo apt install -y jq
sudo jq '. + {"default-runtime": "nvidia"}' /etc/docker/daemon.json | \
  sudo tee /etc/docker/daemon.json.tmp && \
  sudo mv /etc/docker/daemon.json.tmp /etc/docker/daemon.json
```

--------------------------------

### Restart Docker Daemon

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

These commands reload the systemd daemon to recognize configuration changes, restart the Docker service, and then display the Docker service logs to verify it started correctly. Root privileges are required.

```bash
sudo systemctl daemon-reload && \
    sudo systemctl restart docker && \
    sudo journalctl -u docker
```

--------------------------------

### Install Ollama on Jetson

Source: https://www.jetson-ai-lab.com/tutorials/genai-on-jetson-llms-vlms

Installs the Ollama application on a Jetson device using a curl script. This is the recommended method for a quick start with LLMs and VLMs.

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

--------------------------------

### Format SSD and Create Mount Point

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This sequence of commands formats the identified SSD device (e.g., `/dev/nvme0n1`) with the ext4 filesystem, creates a mount point directory (e.g., `/ssd`), and then mounts the SSD to that directory.

```bash
sudo mkfs.ext4 /dev/nvme0n1
sudo mkdir /ssd
sudo mount /dev/nvme0n1 /ssd
```

--------------------------------

### Verify SSD Identification with lspci

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command lists all PCI devices connected to the system. It is used to verify if the Jetson identifies the newly installed NVMe SSD as a memory controller on the PCI bus.

```bash
lspci
```

--------------------------------

### Identify SSD Device Name with lsblk

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

The `lsblk` command lists block devices. This is used to identify the device name assigned to the NVMe SSD (e.g., `nvme0n1`) after physical installation.

```bash
lsblk
```

--------------------------------

### Run Ollama CLI Example

Source: https://www.jetson-ai-lab.com/tutorials/ollama

Demonstrates how to interact with the Ollama service using its command-line interface. This is a basic command to check if the Ollama service is running and accessible.

```bash
ollama
```

--------------------------------

### Install Ollama and Download VLM Model (Shell)

Source: https://www.jetson-ai-lab.com/tutorials/live-vlm-webui

Installs Ollama, a tool for serving large language models, and downloads a recommended VLM model. Includes a specific version for Jetson Thor users experiencing issues with newer Ollama versions.

```shell
# Ollama installer for Linux/Mac
curl -fsSL https://ollama.com/install.sh | sh

# Download recommended model (lightweight)
ollama pull gemma3:4b

# Or other vision models
# ollama pull llama3.2-vision:11b
# ollama pull qwen2.5-vl:7b
```

```shell
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.12.9 sh
```

--------------------------------

### List Docker Images

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command lists all Docker images currently stored on the system. It can be used in conjunction with disk monitoring to confirm that images are being stored in the new Docker root directory on the SSD.

```bash
docker image ls
```

--------------------------------

### Verify SSD and Docker Configuration

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This set of commands verifies that the SSD is recognized, checks overall disk space, confirms the Docker root directory setting, lists the contents of the Docker directory on the SSD, displays its size, and verifies the default runtime. These are crucial for confirming the successful migration and configuration.

```bash
# Check SSD is recognized
sudo blkid | grep nvme

# Check disk space
df -h

# Check Docker root directory
docker info | grep Root

# List Docker directory on SSD
sudo ls -l /ssd/docker/

# Check Docker size
sudo du -chs /ssd/docker/

# Verify nvidia runtime
docker info | grep -e "Runtime" -e "Root"
```

--------------------------------

### Start vLLM Container with NVIDIA NGC

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Launches a Docker container for vLLM from NVIDIA's NGC registry, pre-configured for model serving. It mounts host directories for model and cache storage and utilizes NVIDIA's runtime for GPU acceleration. Ensure sufficient shared memory (`--shm-size`) is allocated.

```bash
docker run --rm -it \
  --network host \
  --shm-size=16g \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  --runtime=nvidia \
  --name=vllm \
  -v $HOME/data/models/huggingface:/root/.cache/huggingface \
  -v $HOME/data/vllm_cache:/root/.cache/vllm \
  nvcr.io/nvidia/vllm:25.09-py3
```

--------------------------------

### Start vLLM Server on Jetson Orin

Source: https://www.jetson-ai-lab.com/tutorials/genai-on-jetson-llms-vlms

Starts the vLLM inference server on a Jetson Orin device, serving a specified Hugging Face model. It uses Docker with NVIDIA runtime, mounts a cache directory, and exposes the model via the network. Adjust the image for Jetson Thor.

```bash
docker run --pull=always --rm -it \
  --network host \
  --shm-size=16g \
  --ulimit memlock=-1 \
  --ulimit stack=67108864 \
  --runtime=nvidia \
  --name=vllm \
  -v $HOME/data/models/huggingface:/root/.cache/huggingface \
  ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
  vllm serve RedHatAI/Qwen3-8B-quantized.w4a16
```

--------------------------------

### Pull Docker Image

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command downloads the specified Docker image (`ubuntu:22.04`) from a container registry. Observing disk usage on the SSD during this process verifies that Docker is using the new storage location.

```bash
docker pull ubuntu:22.04
```

--------------------------------

### Configure SSD Auto-mount with fstab

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This process involves finding the UUID of the SSD using `lsblk -f` and then manually editing the `/etc/fstab` file to add an entry that ensures the SSD is automatically mounted upon system boot. Replace the placeholder UUID with the actual UUID obtained from `lsblk -f`.

```bash
lsblk -f
sudo vi /etc/fstab
UUID=************-****-****-****-******** /ssd/ ext4 defaults 0 2
```

--------------------------------

### Test vLLM API Endpoints

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Provides example `curl` commands to interact with the running vLLM server. It demonstrates how to list available models and send a chat completion request, verifying the server's functionality and model response.

```bash
# Check available models
curl http://localhost:8000/v1/models

# Test chat completion
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [{"role": "user", "content": "Hello! Tell me about Jetson Thor."}],
    "max_tokens": 100
  }'
```

--------------------------------

### Reload Docker Daemon and Restart Docker

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

These commands reload the Docker daemon configuration to recognize changes made to `daemon.json` and then restart the Docker service to apply these changes.

```bash
sudo systemctl daemon-reload && sudo systemctl restart docker
```

--------------------------------

### Serve a model with vLLM

Source: https://www.jetson-ai-lab.com/tutorials/genai-on-jetson-llms-vlms

Starts a vLLM server inside a container to serve a specified model checkpoint. Replace 'MODEL_CHECKPOINT' with a Hugging Face repository ID.

```bash
vllm serve MODEL_CHECKPOINT
```

--------------------------------

### Interact with GPT OSS Model in Ollama

Source: https://www.jetson-ai-lab.com/tutorials/ollama

This example demonstrates a user interaction with the 'gpt-oss:20b' model within the Ollama container. It shows a prompt, the model's thinking process, and the generated response, along with performance metrics like token evaluation rate. Users can type '/bye' to exit the session.

```bash
root@c11344f6bbbc:/# ollama run --verbose gpt-oss:20b
>>> why is the sky blue in one sentence
Thinking...
We need to answer: "why is the sky blue in one sentence". Just one sentence. Provide explanation: Rayleigh scattering of sunlight,
shorter wavelengths scatter more. We'll produce a single sentence. Let's give a concise explanation.
...done thinking.

The sky looks blue because the Earth's atmosphere scatters shorter-wavelength (blue) light from the sun more efficiently than longer
wavelengths, a phenomenon called Rayleigh scattering.

total duration:       3.504445244s
load duration:        225.399151ms
prompt eval count:    76 token(s)
prompt eval duration: 673.487645ms
prompt eval rate:     112.85 tokens/s
eval count:           88 token(s)
eval duration:        2.603822053s
eval rate:            33.80 tokens/s
>>> Send a message (/? for help)
```

--------------------------------

### Move Docker Directory to SSD

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This sequence of commands first checks the size of the current Docker directory, creates a new directory on the SSD, copies the Docker data to the SSD using rsync for efficiency, and then verifies the size of the data on the SSD. It requires root privileges.

```bash
sudo du -csh /var/lib/docker/ && \
    sudo mkdir /ssd/docker && \
    sudo rsync -axPS /var/lib/docker/ /ssd/docker/ && \
    sudo du -csh  /ssd/docker/
```

--------------------------------

### Run a Model with Ollama CLI

Source: https://www.jetson-ai-lab.com/tutorials/ollama

Shows how to run a specific large language model (LLM) using the Ollama command-line interface. This example uses the `gpt-oss:20b` model, demonstrating the ease of deploying and interacting with LLMs locally.

```bash
ollama run gpt-oss:20b
```

--------------------------------

### Manually Configure Docker Default Runtime

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This method involves manually editing the `/etc/docker/daemon.json` file to specify the NVIDIA container runtime as the default. It includes the necessary configuration for the `nvidia` runtime and sets it as the `default-runtime`.

```json
{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia"
}
```

--------------------------------

### Serve Llama Model with vLLM

Source: https://www.jetson-ai-lab.com/tutorials/genai-benchmarking

This command serves the Meta-Llama-3.1-8B-Instruct-quantized.w4a16 model using vLLM. It configures the attention backend, port, host, and various performance-related parameters like sequence length and GPU memory utilization. Ensure vLLM is installed and the model is accessible.

```bash
VLLM_ATTENTION_BACKEND=FLASHINFER vllm serve "RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w4a16" \
--port "8000" \
--host "0.0.0.0" \
--trust_remote_code \
--swap-space 16 \
--max-seq-len 32000 \
--max-model-len 32000 \
--tensor-parallel-size 1 \
--max-num-seqs 1024 \
--gpu-memory-utilization 0.8
```

--------------------------------

### Run Ollama Docker Container on Jetson Orin

Source: https://www.jetson-ai-lab.com/tutorials/ollama

Starts the Ollama server using the `jetson-containers` utility for Jetson Orin devices. This command simplifies the process of running the Ollama container, automatically handling image tagging and network configuration.

```bash
# Option 1: models cached under jetson-containers/data
jetson-containers run --name ollama $(autotag ollama)
```

--------------------------------

### Start Open WebUI with Docker

Source: https://www.jetson-ai-lab.com/tutorials/genai-on-jetson-llms-vlms

Launches the Open WebUI application in detached mode using Docker. It maps a local directory for persistent data and sets the OpenAI API base URL. This is the first step to interact with your LLM via a web interface.

```bash
docker run -d \
  --network=host \
  -v ${HOME}/open-webui:/app/backend/data \
  -e OPENAI_API_BASE_URL=http://0.0.0.0:8000/v1 \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main
```

--------------------------------

### Configure Docker Data Root

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This snippet shows how to edit the Docker daemon configuration file (`/etc/docker/daemon.json`) to specify a new data root directory on the SSD. It also configures the NVIDIA container runtime. This requires root privileges and a text editor like `vi`.

```json
{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia",
    "data-root": "/ssd/docker"
}
```

--------------------------------

### Restart Docker and Add User to Docker Group

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

Restarts the Docker service to apply changes and adds the current user to the `docker` group. This allows the user to run Docker commands without `sudo`. The `newgrp docker` command is used to apply the group changes immediately in the current shell session.

```bash
sudo systemctl restart docker
sudo usermod -aG docker $USER
newgrp docker
```

--------------------------------

### Serve Llama-3.1-8B-Instruct (FP8 Quantization)

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Launches the vLLM server with an FP8 quantized version of the Llama-3.1-8B-Instruct model. This demonstrates the performance benefits of FP8 quantization while aiming to maintain similar output quality to FP16.

```bash
vllm serve nvidia/Llama-3.1-8B-Instruct-FP8
```

--------------------------------

### Serve FP4 Quantized Model with vLLM

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Launches a vLLM server with a specified FP4 quantized model. This command is used to serve models that have undergone FP4 quantization for reduced memory footprint and faster inference.

```bash
vllm serve nvidia/Llama-3.1-8B-Instruct-FP4
```

--------------------------------

### Programmatic VLM Prompting with Image Analysis in Python

Source: https://www.jetson-ai-lab.com/tutorials/live-vlm-webui

Demonstrates how to programmatically send prompts and images to an OpenAI-compatible VLM backend using `aiohttp`. It handles base64 encoding of images and sends requests to a local Ollama instance.

```python
import aiohttp
import base64

async def analyze_image(image_path, prompt):
    with open(image_path, 'rb') as f:
        image_data = base64.b64encode(f.read()).decode('utf-8')

    async with aiohttp.ClientSession() as session:
        async with session.post(
            'http://localhost:11434/v1/chat/completions',
            json={
                'model': 'gemma3:4b',
                'messages': [{
                    'role': 'user',
                    'content': [
                        {'type': 'text', 'text': prompt},
                        {'type': 'image_url', 'image_url': {'url': f'data:image/jpeg;base64,{image_data}'}}
                    ]
                }]
            }
        ) as resp:
            result = await resp.json()
            return result['choices'][0]['message']['content']
```

--------------------------------

### Stop Docker Service

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command stops the Docker service to allow for safe modification of its data directory. It requires root privileges.

```bash
sudo systemctl stop docker
```

--------------------------------

### Serve FP4 Model with Speculative Decoding using vLLM

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Launches a vLLM server with an FP4 model and enables speculative decoding using a smaller 'draft' model. This configuration aims to further accelerate inference by having a draft model predict tokens ahead of the main model.

```bash
vllm serve nvidia/Llama-3.1-8B-Instruct-FP4 \
  --trust_remote_code \
  --speculative-config '{"method":"eagle3","model":"yuhuili/EAGLE3-LLaMA3.1-Instruct-8B","num_speculative_tokens":5}'
```

--------------------------------

### Rename Old Docker Directory

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command renames the original Docker data directory to `docker.old` as a backup before restarting the Docker service with the new configuration. It requires root privileges.

```bash
sudo mv /var/lib/docker /var/lib/docker.old
```

--------------------------------

### Configure Tokenizer Encodings for GPT-OSS Models

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Sets up the necessary tokenizer files (`.tiktoken`) required for GPT-OSS models within the vLLM container. It downloads the encoding files and sets the environment variable to point to their location, enabling correct text processing.

```bash
mkdir /etc/encodings
wget https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken -O /etc/encodings/cl100k_base.tiktoken
wget https://openaipublic.blob.core.windows.net/encodings/o200k_base.tiktoken -O /etc/encodings/o200k_base.tiktoken
export TIKTOKEN_ENCODINGS_BASE=/etc/encodings
```

--------------------------------

### Run NanoOWL Container

Source: https://www.jetson-ai-lab.com/tutorials/nanoowl

This command uses `jetson-containers run` and `autotag` to pull or build a compatible container image for NanoOWL. It sets the working directory to `/opt/nanoowl`.

```bash
jetson-containers run --workdir /opt/nanoowl $(autotag nanoowl)
```

--------------------------------

### Verify Pre-downloaded Model Weights

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Checks if the specified 120B model weights are already present in the Hugging Face cache within the container. This avoids unnecessary re-downloading and confirms the model is ready for serving. It lists files and shows disk usage.

```bash
ls -la /root/.cache/huggingface/hub/models--openai--gpt-oss-120b/
du -h /root/.cache/huggingface/hub/models--openai--gpt-oss-120b/
```

--------------------------------

### Change Ownership of SSD Mount Point

Source: https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup

This command changes the ownership of the SSD mount point directory (e.g., `/ssd`) to the current user, allowing the user to read and write files without needing root privileges.

```bash
sudo chown ${USER}:${USER} /ssd
```

--------------------------------

### Project Structure of Live VLM WebUI

Source: https://www.jetson-ai-lab.com/tutorials/live-vlm-webui

Outlines the directory and file structure for the Live VLM WebUI project. It categorizes files into source code, scripts, Docker configurations, documentation, and project metadata.

```text
live-vlm-webui/
├── src/
│   └── live_vlm_webui/           # Main Python package
│       ├── __init__.py           # Package initialization
│       ├── server.py             # WebRTC server with WebSocket
│       ├── video_processor.py   # Frame processing & VLM integration
│       ├── gpu_monitor.py        # GPU/system monitoring (Jetson support)
│       ├── vlm_service.py        # VLM API client
│       └── static/
│           └── index.html        # Frontend web UI
│
├── scripts/                      # Utility scripts
│   ├── start_server.sh          # Quick start with SSL
│   ├── start_container.sh       # Docker launcher (auto-detection)
│   ├── stop_container.sh        # Stop Docker container
│   └── generate_cert.sh         # SSL certificate generator
│
├── docker/                       # Docker configurations
│   ├── Dockerfile.jetson-orin   # Jetson Orin (JetPack 6)
│   ├── Dockerfile.jetson-thor   # Jetson Thor (JetPack 7)
│   └── docker-compose.yml       # Multi-service stack
│
├── docs/                         # Documentation
│   ├── setup/                   # Setup guides
│   ├── usage/                   # Usage guides
│   └── troubleshooting.md       # Common issues & solutions
│
├── pyproject.toml               # Python package configuration
├── requirements.txt             # Python dependencies
└── README.md                    # Main documentation
```

--------------------------------

### Check Docker Daemon Configuration

Source: https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025

Displays the content of the Docker daemon configuration file. This is used to verify that the NVIDIA runtime is correctly configured for Docker, which is essential for GPU acceleration.

```bash
cat /etc/docker/daemon.json
```
Tokens:
Raw