NVIDIA Nsight Compute

https://docs.nvidia.com/nsight-compute/NsightCompute/
Admin
NVIDIA Nsight Compute is an interactive kernel profiler for CUDA applications, providing detailed...
Tokens:53,697
Snippets:495
Trust Score:10
Update:1 week ago
Show doc for...
Tokens:
Raw
### Multiple configurations in one option (incorrect)

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

This example shows that passing multiple configurations in a single include option does not work as expected.

```bash
ncu --call-stack-type native --native-include "FileA.cpp@FuncA,FileB.cpp@FuncB" CuApp.exe
```

--------------------------------

### Example output from ncu profiling

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Shows the output from the compute profiler, including kernel profiling progress and the report file name.

```text
[Vector addition of 1144477 elements]
==PROF== Connected to process 5268
Copy input data from the host memory to the CUDA device
CUDA kernel launch A with 4471 blocks of 256 threads
==PROF== Profiling "vectorAdd_A" - 0: 0%....50%....100% - 46 passes
CUDA kernel launch B with 4471 blocks of 256 threads
==PROF== Profiling "vectorAdd_B" - 1: 0%....50%....100% - 46 passes
Copy output data from the CUDA device to the host memory
Done
==PROF== Disconnected from process 5268
==PROF== Report: profile.ncu-rep
```

--------------------------------

### Using --communicator shmem for single-node profiling

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Command line example for profiling a multi-process workload on a single node using shared memory communication. The --communicator shmem option is used with --communicator-shmem-num-peers set to the number of processes.

```bash
ncu --communicator shmem --communicator-shmem-num-peers 2 -f -o report -k regex:nccl.\* torchrun --nnodes=1 --nproc\_per\_node=2 pytorch\_app.py
```

--------------------------------

### Config file sample for Launch-and-attach and Import modes

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Sample INI-style config file showing parameters for Launch-and-attach and Import general modes. The -c and --section options are set under each mode.

```ini
[Launch-and-attach]
-c = 1
--section = LaunchStats, Occupancy

[Import]
--open-in-ui
-c = 1
--section = LaunchStats, Occupancy
```

--------------------------------

### Match all kernels except those starting with foo

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Matches all kernels except those starting with "foo". The expression is quoted for shells like Linux bash.

```text
--kernel-id::regex:'^(?!foo)':
```

--------------------------------

### Multiple include options for multiple configurations

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Use the option multiple times to pass multiple configurations, as shown with two --native-include options.

```bash
ncu --call-stack-type native --native-include @FuncA --native-include @FileA.cpp@FuncB CuApp.exe
```

--------------------------------

### Limit synchronization with NVTX filtering

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Use --lockstep-nvtx-include and --lockstep-nvtx-exclude to limit synchronization to specific kernel ranges. This example includes only kernels with 'nccl/' in their NVTX range.

```bash
mpirun ncu --lockstep-kernel-launch --lockstep-nvtx-include nccl/ -o report
```

--------------------------------

### Config file syntax for options with multiple values

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Shows the general syntax for options that can be passed multiple times, written once with comma-separated values.

```ini
[]
=,,...
```

--------------------------------

### Filter ranges with yes flag and NVTX

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Matches the 3rd NVTX push/pop range A from the 2nd start/stop range, with NVTX numbering starting from 1 inside every start/stop range.

```text
--range-filter yes:2:3 --nvtx-include A/
```

--------------------------------

### Slurm script for automating Nsight Compute profiling

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Example Slurm script that automates profiling with Nsight Compute. It extracts the node list from SLURM_JOB_NODELIST, determines the head node address, and runs ncu with TCP communication settings.

```bash
#!/bin/bash
nodes=( $( scontrol show hostnames $SLURM\_JOB\_NODELIST ) )
nodes\_array=($nodes)
head\_node=${nodes\_array\[0\]}
head\_node\_addr=$(srun --nodes=1 --ntasks=1 -w "$head\_node" hostname)
srun ncu \\
  --communicator=tcp \\
  --communicator-tcp-hostname="$head\_node\_addr" \\
  --communicator-tcp-num-peers="$SLURM\_NTASKS" \\
  --lockstep-kernel-launch \\./application
```

--------------------------------

### Config file with clubbed modes

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Sample config file after clubbing common parameters under [Launch-and-attach, import] and keeping mode-specific options under [Import].

```ini
[Launch-and-attach, import]
-c = 1
--section = LaunchStats, Occupancy

[Import]
--open-in-ui
```

--------------------------------

### Launch ncu to collect basic profile for all kernels

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Launches the target application with the command line profiler, collecting the basic set for all kernel launches. Results are written to profile.nsight-cuprof by default.

```bash
$ ncu -o profile CuVectorAddMulti.exe
```

--------------------------------

### Use regular expression with escaped brackets

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Profiles kernels wrapped inside start/end ranges with matching regex '[0-9]rangeA' and 'RangeC[0-9]' of domain with matching regex '[0-9]domainA'. Escaping of '[' and ']' is required when they are at the start or end of the range part.

```bash
ncu --nvtx --nvtx-include "regex:[0-9]domainA@/[0-9]rangeA,RangeC[0-9/]" CuNvtx.exe
```

--------------------------------

### Execute wrapper script with mpirun

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Execute the wrapper script with mpirun to profile rank 0 only.

```bash
mpirun [mpi arguments]./wrap.sh [app arguments]
```

--------------------------------

### NVTX include with Push/Pop configuration using slash

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Use --nvtx-include with a trailing '/' to specify a Push/Pop configuration for the range 'A_range'.

```bash
--nvtx-inlcude "A\_range/"
```

--------------------------------

### Include kernels by native function and file

Source: https://docs.nvidia.com/nsight-compute/NsightComputeCli/index.html

Use --call-stack-type native with --native-include to profile kernels launched via a specific function in a native file.

```bash
ncu --call-stack-type native --native-include ModuleA@FileA.cpp@FuncA CuApp.exe
```
Tokens:
Raw