### ImportNvtxt Info Command Example Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html An example of running the 'info' command and its output, showing analysis start and end times in nanoseconds. ```bash ImportNvtxt info Report.nsys-rep Analysis start (ns) 83501026500000 Analysis end (ns) 83506375000000 ``` -------------------------------- ### Basic Container Launch for Preflight Check Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Perform a basic container launch to verify the Pyxis/Enroot setup before adding Nsight Systems specific configurations. This command ensures the container can be started successfully. ```bash srun --container-image=nvcr.io#nvidia/pytorch: /bin/true ``` -------------------------------- ### Install Nsight Systems CLI on Ubuntu (Minimal Container) Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs the Nsight Systems command-line interface on Ubuntu, suitable for minimal container setups. Requires root privileges. ```bash apt update apt install -y --no-install-recommends gnupg echo "deb http://developer.download.nvidia.com/devtools/repos/ubuntu$(source /etc/lsb-release; echo "$DISTRIB_RELEASE" | tr -d .)/$(dpkg --print-architecture) /" | tee /etc/apt/sources.list.d/nvidia-devtools.list apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub apt update apt install nsight-systems-cli ``` -------------------------------- ### Example: Flash 6.0.9.1 QNX Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example command to run the Docker container for flashing version 6.0.9.1 of DRIVE OS on QNX. ```bash sudo docker run --rm --privileged --net host \ -v /dev/bus/usb:/dev/bus/usb \ -v /tmp:/drive_flashing \ -it nvcr.io/{MY_NGC_ORG}/driveos-pdk/drive-agx-orin-qnx-aarch64-pdk-build-x86:6.0.9.1-latest ``` -------------------------------- ### Example: Flash 6.0.8.0 QNX Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example command to run the Docker container for flashing version 6.0.8.0 of DRIVE OS on QNX. ```bash sudo docker run --rm --privileged --net host \ -v /dev/bus/usb:/dev/bus/usb \ -v /tmp:/drive_flashing \ -it nvcr.io/{MY_NGC_ORG}/driveos-pdk/drive-agx-orin-qnx-aarch64-pdk-build-x86:6.0.8.0-0003 ``` -------------------------------- ### Example: Flash 6.0.8.0 Linux Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example command to run the Docker container for flashing version 6.0.8.0 of DRIVE OS on Linux. ```bash sudo docker run --rm --privileged --net host \ -v /dev/bus/usb:/dev/bus/usb \ -v /tmp:/drive_flashing \ -it nvcr.io/{MY_NGC_ORG}/driveos-pdk/drive-agx-orin-linux-aarch64-pdk-build-x86:6.0.8.0-0003 ``` -------------------------------- ### Install Nsight Systems on Compute Nodes Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example path for installing Nsight Systems on Slurm compute nodes. Ensure the path is identical and readable across all nodes and within the container's mount namespace. ```bash /opt/nvidia/nsight-systems/2026.2.1 ``` -------------------------------- ### Install Nsight Systems CLI on CentOS/RHEL (Minimal Container) Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs the Nsight Systems command-line interface on CentOS/RHEL, suitable for minimal container setups. Requires root privileges. ```bash rpm --import https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* dnf install -y 'dnf-command(config-manager)' dnf config-manager --add-repo "https://developer.download.nvidia.com/devtools/repos/rhel$(source /etc/os-release; echo ${VERSION_ID%%.*})/$(rpm --eval '%{_arch}' | sed s/aarch/arm/)/" dnf install -y nsight-systems-cli ``` -------------------------------- ### Launch Nsight Systems GUI Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Execute this command to start the Nsight Systems graphical user interface directly. The executable is located in the Host sub-directory of your Nsight Systems installation. ```bash nsys-ui ``` -------------------------------- ### Nsight Systems CLI GPU Video Devices Example Output Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example output from the '--gpu-video-devices help' command, showing supported and unsupported GPUs for video accelerator tracing. ```text Possible --gpu-video-devices values are: 0: NVIDIA GeForce RTX 3070 PCI[0000:65:00.0] all: Select all supported GPUs none: Disable GPU video accelerator tracing [Default] Some GPUs don't support video accelerator tracing: Quadro P620 PCI[0000:04:00.0] (reason = Arch Pascal < Turing) ``` -------------------------------- ### Install Nsight Systems GUI on Ubuntu (Desktop) Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs the full Nsight Systems package, including the GUI, on Ubuntu desktop systems. Requires root privileges. ```bash sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo add-apt-repository "deb https://developer.download.nvidia.com/devtools/repos/ubuntu$(source /etc/lsb-release; echo "$DISTRIB_RELEASE" | tr -d .)/$(dpkg --print-architecture)/ /" sudo apt install nsight-systems ``` -------------------------------- ### One-Step Dependency Installation for Nsight Systems Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs all Nsight Systems CLI and GUI dependencies using pip. Ensure your virtual environment is activated before running. ```bash python3 -m pip install -r nsys_recipe/requirements/dask.txt -r nsys_recipe/requirements/common.txt -r nsys_recipe/requirements/jupyter.txt ``` -------------------------------- ### Automated Nsight Systems Installation (Linux) Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Enables automated installation of Nsight Systems on Linux hosts by accepting the EULA. Use `--quiet` for silent installation. ```bash sudo apt install nsight-systems --accept ``` -------------------------------- ### Install Nsight Systems Dependencies with Root Privileges Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Run this script to install all required libraries in system directories for Nsight Systems GUI on Ubuntu and CentOS when root privileges are available. ```bash [installation_path]/host-linux-[arch]/Scripts/DependenciesInstaller/install-dependencies.sh ``` -------------------------------- ### Example Nsight Systems Plugin Manifest File Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html This is an example of a plugin manifest file in YAML format. It specifies the plugin name, executable path, library path, target environment, and a description. ```yaml PluginName: unique_name ExecutablePath: bin/plugin LibraryPath: libPlugin.so TargetEnvironment: {KEY: VALUE} Description: This is an example plugin manifest. ``` -------------------------------- ### Install Nsight Systems GUI on CentOS/RHEL (Desktop) Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs the full Nsight Systems package, including the GUI, on CentOS/RHEL desktop systems. Requires root privileges. ```bash sudo rpm --import https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub sudo dnf install -y 'dnf-command(config-manager)' sudo dnf config-manager --add-repo "https://developer.download.nvidia.com/devtools/repos/rhel$(source /etc/os-release; echo ${VERSION_ID%%.*})/$(rpm --eval '%{_arch}' | sed s/aarch/arm/)/" sudo dnf install nsight-systems ``` -------------------------------- ### Install Netcat on Ubuntu Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Installs the Netcat utility on Ubuntu-based systems. Netcat is a required dependency for Nsight Systems on the target device. ```bash sudo apt-get install netcat-openbsd ``` -------------------------------- ### Install xcb-util-cursor on RHEL/CentOS/Fedora Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html This command installs the necessary xcb-cursor package on RHEL-based systems to resolve issues with the Qt xcb platform plugin. ```bash sudo dnf install -y xcb-util-cursor ``` -------------------------------- ### NVTX Capture with Specific Message and Any Domain Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Launches an application, profiling starts when the first range with the message 'profiler' is opened in any domain. ```bash nsys launch -w true -p profiler@* ./app ``` -------------------------------- ### Install Nsight Systems Dependencies Without Root Privileges Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Execute this script to install Nsight Systems dependencies into a specified directory when root privileges are not available. Ensure the target directory is user-writable. ```bash [installation_path]/host-linux-[arch]/Scripts/DependenciesInstaller/install-dependencies-without-root.sh [dependencies_path] ``` -------------------------------- ### NVTX Capture with Specific Message and Domain Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Launches an application, profiling starts when the first range with the message 'profiler' is opened in the 'service' domain. ```bash nsys launch -w true -p profiler@service ./app ``` -------------------------------- ### Launch Chrome for WebGL Profiling Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Use this command structure to profile WebGL content in Chrome with Nsight Systems. Ensure Chrome is installed in the default location. ```bash "C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --inprocess-gpu --no-sandbox --disable-gpu-watchdog --use-angle=gl https://webglsamples.org/aquarium/aquarium.html ``` -------------------------------- ### Format and Mount Power-Safe File System Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Commands to format a specified partition with a power-safe file system and then mount it. This example targets the `vblk_ufs40` partition. ```bash mkqnx6fs /dev/vblk_ufs40 -q mount -o rw /dev/vblk_ufs40 / ``` -------------------------------- ### QNX Kernel Symbols Configuration Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example JSON configuration for QNX kernel symbols, mapping guest VMs to their respective symbol files. ```json { "guest_cfg": [ { "guest_id": 0, "guest_name": "Guest VM 0", "symbols": "/root/symbols/procnto-smp-instr-safety.guest_vm.bin.sym" }, { "guest_id": 1, "guest_name": "Update service", "symbols": "/root/symbols/procnto-smp-instr-safety.update_vm.bin.sym" }, { "guest_id": 2, "guest_name": "Resource Manager Server" }, { "guest_id": 3, "guest_name": "Storage Server" }, { "guest_id": 4, "guest_name": "Ethernet Server" }, { "guest_id": 5, "guest_name": "Debug Server" } ], "symbol_files": { "Sidekick": "/root/symbols/sidekick.unstripped" } } ``` -------------------------------- ### Flash Drive Orin p3663 Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example of running the flash script for a Drive Orin p3663 board. ```bash ./flash.py /dev/ttyUSB1 p3663 ``` -------------------------------- ### ImportNvtxt Help Command Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Prints the help message for the ImportNvtxt utility. Use this to get an overview of available commands and options. ```bash -h [ --help ] ``` -------------------------------- ### Check QNX Filesystem Writable Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Use this command to verify if the target QNX filesystem is writable before connecting Nsight Systems. This is necessary for installing target-side binaries. ```bash echo XX > /xx && ls -l /xx ``` -------------------------------- ### Enable NVML Metrics Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Add this option to nsys profile or start commands to enable NVML power and temperature sampling. Arguments can be specified as needed. ```bash --enable nvml_metrics[,arg1[=value1],arg2[=value2], ...] ``` -------------------------------- ### NVTX Capture with Specific Message in Default Domain Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Launches an application, profiling starts when the first range with the message 'profiler' is opened in the default domain. ```bash nsys launch -w true -p profiler ./app ``` -------------------------------- ### Print Tool Version Information Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Use this command to display the installed version of the Nsight Systems CLI tool. This is a simple query to verify the tool's version. ```bash nsys -v ``` -------------------------------- ### Launch Nsight Systems GUI with Dependencies Environment Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Source this script to set up the environment for Nsight Systems GUI after installing dependencies without root privileges. Then, launch the UI. ```bash source [installation_path]/host-linux-[arch]/Scripts/DependenciesInstaller/setup-dependencies-environment.sh [dependencies_path] && [installation_path]/host-linux-[arch]/nsys-ui ``` -------------------------------- ### Filter Trace Data with grep Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html This example demonstrates how to export trace data to a SQLite file and then process it using a command-line utility like grep to filter specific information. The output name starting with '@' indicates a command to be executed. ```bash nsys export --stats=cuda_api_sum --output=@grep -E (-|Name|cudaFree) test.sqlite ``` -------------------------------- ### Install Pip and Venv on Ubuntu Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs pip and venv for Python 3 on Ubuntu systems. Run 'sudo apt-get update' first on a fresh install. ```bash sudo apt-get update sudo apt-get install python3-pip sudo apt-get install python3-venv ``` -------------------------------- ### Create XHV Directory (QNX) Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Creates the necessary directory structure and copies configuration files for XHV sampling on QNX systems. ```bash cd /drive_flashing mkdir -p xhv/hypervisor/configs/t234ref-release/pct/qnx xhv/schemas cp -rv /drive/drive-foundation/virtualization/hypervisor/t23x/configs/t234ref-release/pct/p3710-10-a03/qnx/pct.json ./xhv/hypervisor/configs/t234ref-release/pct/qnx/ cp -rv /drive/drive-foundation/schemas/event ./xhv/schemas/ ``` -------------------------------- ### Launch Application with Nsight Systems Launcher Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Use the Nsight Systems launcher binary to start your application. The application will daemonize and wait for a SIGUSR1 signal to resume execution after Nsight Systems attaches. ```bash $ /opt/nvidia/nsight_systems/launcher ./my-binary --arguments ``` -------------------------------- ### Validate GPUDirect Storage Installation Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Run the `gdscheck.py` tool to verify that GPUDirect Storage is installed correctly and that the intended filesystem type is supported. This script is typically found in the CUDA installation directory. ```bash /usr/local/cuda/gds/tools/gdscheck.py -p ``` -------------------------------- ### Interactive CLI: Launch App, Start Collection Manually Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Launch an application and then manually start the data collection. This method allows for control over when profiling begins after the application has already started running. ```bash nsys launch -w true [application-arguments] ``` ```bash nsys start ``` -------------------------------- ### Run nvtx_cpu_topdown Recipe Source: https://docs.nvidia.com/nsight-systems/AnalysisGuide/index.html This sequence of commands sets up a directory for reports, collects CPU profiling data using a provided script, and then runs the nvtx_cpu_topdown recipe on the collected data. Ensure the path to collect_cpu_topdown.sh is correct for your system. ```bash mkdir reports && cd reports /CpuProfiling/collect_cpu_topdown.sh ./myApp nsys recipe nvtx_cpu_topdown --input . ``` -------------------------------- ### Create Recipe Directory and Copy Files Source: https://docs.nvidia.com/nsight-systems/AnalysisGuide/index.html Use these Linux commands to set up the directory structure and copy necessary files for a new custom recipe. Ensure you replace placeholders with your specific directory and recipe names. ```bash > cd /target-linux-x64/python/packages/nsys_recipe > mkdir new_metric_util_map > cp gpu_metric_util_map/metadata.json new_metric_util_map/metadata.json > cp gpu_metric_util_map/heatmap.ipynb new_metric_util_map/heatmap.ipynb > cp gpu_metric_util_map/gpu_metric_util_map.py new_metric_util_map/new_metric_util_map.py ``` -------------------------------- ### Install libxcb-cursor0 on Debian/Ubuntu Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html This command installs the necessary libxcb-cursor0 package on Debian/Ubuntu systems to resolve issues with the Qt xcb platform plugin. ```bash sudo apt-get install -y libxcb-cursor0 ``` -------------------------------- ### Verify Nsight Systems Installation Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Command to verify the Nsight Systems installation on a compute node. This checks if the `nsys` executable is accessible and reports its version. ```bash $ /opt/nvidia/nsight-systems/2026.2.1/bin/nsys --version ``` -------------------------------- ### Profile Local and NVMe-oF Volumes Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Nsight Systems command line for profiling local storage and NVMe-oF device performance. Use '--storage-metrics' and '--storage-devices=all'. ```bash ./nsys profile --storage-metrics --storage-devices=all ``` -------------------------------- ### Install Nsight Systems Dependencies on Offline Machine Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Installs Nsight Systems dependencies from a tarball on a machine without internet access. Ensure the tarball has been transferred. ```bash tar -xvfz recipe-deps.tar.gz python3 -m pip install recipe-deps/* --no-index ``` -------------------------------- ### Display Nsight Systems CLI Help Source: https://docs.nvidia.com/nsight-systems/InstallationGuide/index.html Run this command to view a list of available options and commands for the Nsight Systems CLI. For comprehensive documentation, refer to the User Guide. ```bash nsys --help ``` -------------------------------- ### List Available Metric Sets Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html To list the available metric sets for your selected GPUs, use the 'help' option for --gpu-metrics-set. ```bash $ nsys profile --gpu-metrics-devices=all --gpu-metrics-set=help ``` -------------------------------- ### Enable GPU Context Switch Tracing Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Use the `--gpuctxsw` option when running Nsight Systems from the command line to enable GPU context switch tracing. ```bash nsys profile --gpuctxsw true ``` -------------------------------- ### Bind Mount Host Install into Enroot Container Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html This configuration binds the Nsight Systems installation from the host to the container, making it accessible without modifying the container image. It ensures read-only access and prevents execution from the mount point. ```bash /opt/nvidia/nsight-systems/2026.2.1 /opt/nvidia/nsight-systems/2026.2.1 none bind,ro,nosuid,nodev 0 0 ``` -------------------------------- ### Python Equivalent for Finding Innermost NVTX Range Source: https://docs.nvidia.com/nsight-systems/AnalysisGuide/index.html A Python function to find the innermost NVTX range that encompasses a given kernel's start and end times. It iterates through provided NVTX ranges and returns the text of the latest starting range that fully contains the kernel. ```python # Python equivalent: def find_innermost_nvtx_range(kernel_start, kernel_end, nvtx_ranges): # Find NVTX ranges that completely contain the kernel containing_ranges = [] for nvtx in nvtx_ranges: if nvtx['start'] <= kernel_start and nvtx['end'] >= kernel_end: containing_ranges.append(nvtx) # Return the innermost (latest starting) range if containing_ranges: return max(containing_ranges, key=lambda x: x['start'])['text'] return None ``` -------------------------------- ### Delayed Start Profiling Run Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Starts profiling after a specified delay (20 seconds) and sets an environment variable for the target application. Collection ends upon application exit. This command traces CUDA, OpenGL, NVTX, and OS runtime libraries, and collects CPU sampling and thread scheduling data. ```bash nsys profile -e TEST_ONLY=0 -y 20 [application-arguments] ``` -------------------------------- ### Run gpu_vram_usage_trace Recipe Source: https://docs.nvidia.com/nsight-systems/AnalysisGuide/index.html Execute the gpu_vram_usage_trace recipe using the Nsight Systems CLI. This recipe analyzes VRAM usage patterns from a collected report. ```bash nsys recipe gpu_vram_usage_trace --input [report file path] ``` -------------------------------- ### Flash Firespray p3710 Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Example of running the flash script for a Firespray p3710 board. ```bash ./flash.py /dev/ttyACM1 p3710 ``` -------------------------------- ### Interactive CLI: Start Collection, Launch App, Stop Manually Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Initiate an interactive CLI session to start collecting data from the beginning of an application's execution. The collection is set to stop manually. This sequence is suitable for applications that may run for extended periods, but users should be aware of potential storage issues if collections are not stopped. ```bash nsys start --stop-on-exit=false ``` ```bash nsys launch --trace=cuda,nvtx --sample=none [application-arguments] ``` ```bash nsys stop ``` -------------------------------- ### Docker/Podman Run Command for Nsight Systems Profiling Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html This command demonstrates how to run a Docker or Podman container with Nsight Systems bind-mounted and configured for profiling. It includes necessary GPU access and capability flags. ```bash docker run --rm --gpus all \ --cap-add=SYS_ADMIN \ -v /opt/nvidia/nsight-systems/2026.2.1:/opt/nvidia/nsight-systems/2026.2.1:ro \ my-workload:latest \ /opt/nvidia/nsight-systems/2026.2.1/bin/nsys profile \ -t cuda,nvtx,mpi \ -o /reports/run_%p \ python train.py ``` -------------------------------- ### Create XHV Directory (Linux) Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Creates the necessary directory structure and copies configuration files for XHV sampling on Linux systems. ```bash cd /drive_flashing mkdir -p xhv/hypervisor/configs/t234ref-release/pct/linux xhv/schemas cp -rv /drive/drive-foundation/virtualization/hypervisor/t23x/configs/t234ref-release/pct/p3710-10-a03/linux/pct.json ./xhv/hypervisor/configs/t234ref-release/pct/linux/ cp -rv /drive/drive-foundation/schemas/event ./xhv/schemas/ ``` -------------------------------- ### OpenMP Event Class Values Source: https://docs.nvidia.com/nsight-systems/AnalysisGuide/index.html Defines event class values for OpenMP, including start and finish events. ```text 78 - TRACE_PROCESS_EVENT_OPENMP 79 - TRACE_PROCESS_EVENT_OPENMP_START 80 - TRACE_PROCESS_EVENT_OPENMP_FINISH ``` -------------------------------- ### Run Nsight Systems Recipe CLI Source: https://docs.nvidia.com/nsight-systems/AnalysisGuide/index.html Use the 'recipe' CLI command to execute advanced analysis recipes. Specify arguments and the recipe name, followed by any recipe-specific arguments. ```bash nsys recipe [args] [recipe args] ``` -------------------------------- ### Start CUDA Profiling Source: https://docs.nvidia.com/nsight-systems/UserGuide/index.html Use cudaProfilerStart() to begin data collection in a CUDA application. Include cuda_profiler_api.h to use this function. ```c++ #include // ... cudaProfilerStart(); ```