### Container Runtime Initialization Error Example (Bash) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/wsl-user-guide/index This example demonstrates a common error encountered when running Docker containers with GPU support on WSL 2. It shows the command used and the resulting error output, indicating issues with the NVIDIA Container Toolkit or driver setup. ```bash $ sudo docker run --gpus all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\\\n\\\"\"": unknown. ERRO[0000] error waiting for container: context canceled ``` -------------------------------- ### Setup CUDA Network Repository for Ubuntu 18.04 Source: https://docs.nvidia.com/cuda/archive/11.0_GA/wsl-user-guide/index These commands set up the NVIDIA CUDA network repository for Ubuntu 18.04 within WSL. This allows for the installation of the CUDA Toolkit via the apt package manager. ```bash $ apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub $ sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 /" > /etc/apt/sources.list.d/cuda.list' $ apt-get update ``` -------------------------------- ### Manage Nsight Eclipse Plugins using Script (Shell) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nsightee-plugins-install-guide/index This script manages the installation or uninstallation of Nsight Eclipse Plugins. It requires an action ('install' or 'uninstall') and the Eclipse installation directory as parameters. The script is typically located in the CUDA toolkit's bin directory. ```shell Usage: ./nsight_ee_plugins_manage.sh : 'install' or 'uninstall' : eclipse installation directory ``` ```shell $ /usr/local/cuda-11.0/bin/nsight_ee_plugins_manage.sh install ``` ```shell $ /usr/local/cuda-11.0/bin/nsight_ee_plugins_manage.sh uninstall ``` -------------------------------- ### Linking and Usage Examples Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvjpeg/index This section provides examples of how to link the nvJPEG library (both shared and static) and demonstrates command-line usage for decoding JPEG files. ```APIDOC ## Examples of nvJPEG Usage The nvJPEG library package includes header files and library binaries (static and shared). ### Linking the Library - **Shared Library (`libnvjpeg.so`)**: Dependencies are statically linked within the shared library. ```bash g++ -I/include -lnvjpeg -L/lib64 my_example.cpp -o my_example ``` - **Static Library (`libnvjpeg_static.a`)**: Requires explicit linking of other dependencies like `dl`, `rt`, and `pthread`. ```bash g++ -I/include -lnvjpeg_static -ldl -lrt -pthread -L/lib64 my_example.cpp -o my_example ``` ### Compiling Example Applications To compile an example application (e.g., `nvjpeg_example.cpp`) assuming CUDA 9.0 is installed at `/usr/local/cuda-9.0`: ```bash g++ -O3 -m64 nvjpeg_example.cpp -I../include -lnvjpeg -L../lib64 -I/usr/local/cuda-9.0/include -ldl -lrt -pthread -lcudart -L/usr/local/cuda-9.0/lib64 -Wl,-rpath=../lib64 -Wl,-rpath=/usr/local/cuda-9.0/lib64 -o nvjpeg_example ``` ### Decoding JPEG Files - **Decode a single image**: Specify input image, output format, and output directory. ```bash ./nvjpeg_example -i /tmp/my_image.jpg -fmt rgb -o /tmp ``` - **Decode multiple images using batched API**: Use batching, pipelining, and specify batch size for processing images in a folder. ```bash ./nvjpeg_example -i /tmp/my_images/ -fmt rgb -b 32 -pipelined -batched -o /tmp ``` For a full list of command-line parameters and their descriptions, run: ```bash ./nvjpeg_example -h ``` ``` -------------------------------- ### Build Instructions for CUDA and NVRTC Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvrtc/index Provides build commands for compiling the CUDA and NVRTC example code on Windows, Linux, and Mac OS X using their respective compilers. ```shell * Windows: cl.exe lowered-name.cpp /Felowered-name \ /I "%CUDA_PATH%"\include \ "%CUDA_PATH%"\lib\x64\nvrtc.lib "%CUDA_PATH%"\lib\x64\cuda.lib * Linux: g++ lowered-name.cpp -o lowered-name \ -I $CUDA_PATH/include \ -L $CUDA_PATH/lib64 \ -lnvrtc -lcuda \ -Wl,-rpath,$CUDA_PATH/lib64 * Mac OS X: clang++ lowered-name.cpp -o lowered-name \ -I $CUDA_PATH/include \ -L $CUDA_PATH/lib \ -lnvrtc -framework CUDA \ -Wl,-rpath,$CUDA_PATH/lib ``` -------------------------------- ### NVBLAS Configuration File Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvblas/index An example NVBLAS configuration file demonstrating various settings such as log file, CPU BLAS library path, and GPU device list. This file is used to configure NVBLAS behavior for BLAS calls. ```text # This is the configuration file to use NVBLAS Library # Setup the environment variable NVBLAS_CONFIG_FILE to specify your own config file. # By default, if NVBLAS_CONFIG_FILE is not defined, # NVBLAS Library will try to open the file "nvblas.conf" in its current directory # Example : NVBLAS_CONFIG_FILE /home/cuda_user/my_nvblas.conf # The config file should have restricted write permissions accesses # Specify which output log file (default is stderr) NVBLAS_LOGFILE nvblas.log # Enable trace log of every intercepted BLAS calls NVBLAS_TRACE_LOG_ENABLED #Put here the CPU BLAS fallback Library of your choice #It is strongly advised to use full path to describe the location of the CPU Library NVBLAS_CPU_BLAS_LIB /usr/lib/libopenblas.so #NVBLAS_CPU_BLAS_LIB /libmkl_rt.so # List of GPU devices Id to participate to the computation # Use ALL if you want all your GPUs to contribute # Use ALL0, if you want all your GPUs of the same type as device 0 to contribute # However, NVBLAS consider that all GPU have the same performance and PCI bandwidth ``` -------------------------------- ### Compile NVJPEG Example (g++) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvjpeg/index Compilation command for a general NVJPEG example, demonstrating linking with CUDA toolkit libraries and NVJPEG. It includes optimization flags, include/library paths, and linker options for runtime paths. ```bash g++ -O3 -m64 nvjpeg_example.cpp -I../include -lnvjpeg -L../lib64 -I/usr/local/cuda-9.0/include -ldl -lrt -pthread -lcudart -L/usr/local/cuda-9.0/lib64 -Wl,-rpath=../lib64 -Wl,-rpath=/usr/local/cuda-9.0/lib64 -o nvjpeg_example ``` -------------------------------- ### Build Instructions for Dynamic Parallelism Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvrtc/index Provides compilation commands for the dynamic parallelism example on Windows, Linux, and macOS. These commands leverage the NVRTC and CUDA libraries, assuming the CUDA_PATH environment variable is set. ```shell // Windows: cl.exe dynamic-parallelism.cpp /Fedynamic-parallelism ^ /I "%CUDA_PATH%\include" ^ "%CUDA_PATH%"\lib\x64\nvrtc.lib "%CUDA_PATH%"\lib\x64\cuda.lib // Linux: g++ dynamic-parallelism.cpp -o dynamic-parallelism \ -I $CUDA_PATH/include \ -L $CUDA_PATH/lib64 \ -lnvrtc -lcuda \ -Wl,-rpath,$CUDA_PATH/lib64 // Mac OS X: clang++ dynamic-parallelism.cpp -o dynamic-parallelism \ -I $CUDA_PATH/include \ -L $CUDA_PATH/lib \ -lnvrtc -framework CUDA \ -Wl,-rpath,$CUDA_PATH/lib ``` -------------------------------- ### NVTX Usage Example in C Source: https://docs.nvidia.com/cuda/archive/11.0_GA/profiler-users-guide/index This example demonstrates the use of marker events, range events, and resource naming using the NVTX API in C. It shows how to integrate NVTX into an application for capturing and visualizing events. ```C void Wait(int waitMilliseconds) { nvtxNameOsThread("MAIN"); nvtxRangePush(__FUNCTION__); nvtxMark("Waiting..."); Sleep(waitMilliseconds); nvtxRangePop(); } int main(void) { nvtxNameOsThread("MAIN"); nvtxRangePush(__FUNCTION__); Wait(); nvtxRangePop(); } ``` -------------------------------- ### nvprof API Trace Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/profiler-users-guide/index This example demonstrates how to use 'nvprof' to print the API trace of a CUDA application. It shows the command-line usage and the expected output format, including timings for CUDA runtime and driver API calls. ```bash $nvprof --print-api-trace matrixMul ==27722== NVPROF is profiling process 27722, command: matrixMul ``` ```text ==27722== Profiling application: matrixMul [Matrix Multiply Using CUDA] - Starting... GPU Device 0: "GeForce GT 640M LE" with compute capability 3.0 MatrixA(320,320), MatrixB(640,320) Computing result using CUDA Kernel... done Performance= 35.35 GFlop/s, Time= 3.708 msec, Size= 131072000 Ops, WorkgroupSize= 1024 threads/block Checking computed result for correctness: OK Note: For peak performance, please refer to the matrixMulCUBLAS example. ==27722== Profiling result: Start Duration Name 108.38ms 6.2130us cuDeviceGetCount 108.42ms 840ns cuDeviceGet 108.42ms 22.459us cuDeviceGetName 108.45ms 11.782us cuDeviceTotalMem 108.46ms 945ns cuDeviceGetAttribute 149.37ms 23.737us cudaLaunch (void matrixMulCUDA(float*, float*, float*, int, int) [2198]) 149.39ms 6.6290us cudaEventRecord 149.40ms 1.10156s cudaEventSynchronize <...more output...> 1.25096s 21.543us cudaEventElapsedTime 1.25103s 1.5462ms cudaMemcpy 1.25467s 153.93us cudaFree 1.25483s 75.373us cudaFree 1.25491s 75.564us cudaFree 1.25693s 10.901ms cudaDeviceReset Note: Due to the way the profiler is setup, the first "cuInit()" driver API call is never traced. ``` -------------------------------- ### Fortran 77 Application Executing on the Host Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Example Fortran 77 code demonstrating cuBLAS integration. Shows how to initialize cuBLAS, modify matrix data using cuBLAS functions, and properly shutdown. Requires compilation with appropriate Fortran-to-C wrappers. ```Fortran ! Example B.1. Fortran 77 Application Executing on the Host ! ---------------------------------------------------------- subroutine modify ( m, ldm, n, p, q, alpha, beta ) implicit none integer ldm, n, p, q real*4 m (ldm, *) , alpha , beta external cublas_sscal call cublas_sscal (n-p+1, alpha , m(p,q), ldm) call cublas_sscal (ldm-p+1, beta, m(p,q), 1) return end program matrixmod implicit none integer M,N parameter (M=6, N=5) real*4 a(M,N) integer i, j external cublas_init external cublas_shutdown do j = 1, N do i = 1, M a(i, j) = (i-1)*M + j enddo enddo call cublas_init call modify ( a, M, N, 2, 3, 16.0, 12.0 ) call cublas_shutdown do j = 1 , N do i = 1 , M write(*,"(F7.0$)") a(i,j) enddo write (*,*) "" enddo stop end ``` -------------------------------- ### nvvp Command Line Usage for Sessions - Bash Source: https://docs.nvidia.com/cuda/archive/11.0_GA/profiler-users-guide/index The Visual Profiler (nvvp) can be launched from the command line to start new executable sessions or import single/multi-process nvprof profile files. Requires NVIDIA CUDA toolkit installation with Visual Profiler. Inputs are executable names/arguments or .nvprof files; outputs a new profiling session in nvvp. Limitations: Supports specific nvprof export formats only. ```bash nvvp executableName [[executableArguments]...] ``` ```bash nvvp data.nvprof ``` ```bash nvvp data1.nvprof data2.nvprof ... ``` -------------------------------- ### Example: Create Tiled Groups Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cuda-c-programming-guide/index Demonstrates the creation of two `thread_block_tile` instances with different sizes (32 and 4). The example shows how to specify the tile size as a template parameter to `tiled_partition` and how provenance can be encoded in the type or stored in the handle. ```C++ /// The following code will create two sets of tiled groups, of size 32 and 4 respectively: /// The latter has the provenance encoded in the type, while the first stores it in the handle thread_block block = this_thread_block(); thread_block_tile<32> tile32 = tiled_partition<32>(block); thread_block_tile<4, thread_block> tile4 = tiled_partition<4>(block); ``` -------------------------------- ### SIMD Video Instructions: vset2 Examples Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index Provides examples of using the vset2 instruction for SIMD comparison, demonstrating different data types and comparison operators, including the optional '.add' modifier for accumulation. ```asm vset2.s32.u32.lt r1, r2, r3, r0; vset2.u32.u32.ne.add r1, r2, r3, r0; ``` -------------------------------- ### Examples of texture attribute queries Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index These examples demonstrate querying various texture and sampler attributes using the txq instruction. They include querying width, filter mode (unified mode), address mode (independent mode), and level-specific width. ```ptx txq.width.b32 %r1, [tex_A]; txq.filter_mode.b32 %r1, [tex_A]; // unified mode txq.addr_mode_0.b32 %r1, [smpl_B]; // independent mode txq.level.width.b32 %r1, [tex_A], %r_lod; ``` -------------------------------- ### WMMA Load Instruction Example (CUDA C++) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index This CUDA C++ example demonstrates the `wmma.load.a.sync.aligned.row.m16n16k16.f16` instruction. It shows how to load matrix fragments from memory, specifying alignment and layout. The example highlights the relationship between fragment size, stride, and memory alignment. ```CUDA C++ wmma.load.a.sync.aligned.row.m16n16k16.f16 {x0,...,x7}, [p], s; ``` -------------------------------- ### Multi-Sample Texture Example - CUDA Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index Illustrates a multi-sample texture fetch in unified texturing mode. The example includes a sample index and three scalar values. ```CUDA tex.2dms.v4.s32.s32 {r0,r1,r2,r3}, [tex_ms,{sample,r6,r7,r8}]; ``` -------------------------------- ### PTX Call Instruction Examples Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index Demonstrates direct and indirect 'call' instructions in PTX. Includes examples using jump tables, .calltargets, and .callprototype directives for managing function calls, including parameter passing and return values. ```ptx // examples of direct call call init; // call function 'init' call.uni g, (a); // call function 'g' with parameter 'a' @p call (d), h, (a, b); // return value into register d // call-via-pointer using jump table .func (.reg .u32 rv) foo (.reg .u32 a, .reg .u32 b) ... .func (.reg .u32 rv) bar (.reg .u32 a, .reg .u32 b) ... .func (.reg .u32 rv) baz (.reg .u32 a, .reg .u32 b) ... .global .u32 jmptbl[5] = { foo, bar, baz }; ... @p ld.global.u32 %r0, [jmptbl+4]; @p ld.global.u32 %r0, [jmptbl+8]; call (retval), %r0, (x, y), jmptbl; // call-via-pointer using .calltargets directive .func (.reg .u32 rv) foo (.reg .u32 a, .reg .u32 b) ... .func (.reg .u32 rv) bar (.reg .u32 a, .reg .u32 b) ... .func (.reg .u32 rv) baz (.reg .u32 a, .reg .u32 b) ... ... @p mov.u32 %r0, foo; @q mov.u32 %r0, baz; Ftgt: .calltargets foo, bar, baz; call (retval), %r0, (x, y), Ftgt; // call-via-pointer using .callprototype directive .func dispatch (.reg .u32 fptr, .reg .u32 idx) { ... Fproto: .callprototype _ (.param .u32 _, .param .u32 _); call %fptr, (x, y), Fproto; ... ``` -------------------------------- ### Examples of type checking predicates Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index These examples show how to use istypep to check if registers point to specific opaque variable types such as texture references, sampler references, and surface references. ```ptx istypep.texref istex, tptr; istypep.samplerref issampler, sptr; istypep.surfref issurface, surfptr; ``` -------------------------------- ### CUDA Data Prefetching Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cuda-c-programming-guide/index This C++ code example demonstrates how to use cudaMemPrefetchAsync to prefetch data to the GPU before a kernel launch and then prefetch it back to the CPU. This strategy aims to minimize data transfer overhead and avoid page faults during kernel execution. ```cpp void foo(cudaStream_t s) { char *data; cudaMallocManaged(&data, N); init_data(data, N); // execute on CPU cudaMemPrefetchAsync(data, N, myGpuId, s); // prefetch to GPU mykernel<<<..., s>>>(data, N, 1, compare); // execute on GPU cudaMemPrefetchAsync(data, N, cudaCpuDeviceId, s); // prefetch to CPU cudaStreamSynchronize(s); use_data(data, N); cudaFree(data); } ``` -------------------------------- ### Initialize OpenGL Context and Window with GLUT Source: https://docs.nvidia.com/cuda/archive/11.0_GA/optimus-developer-guide/index This code initializes a GLUT-based OpenGL context and window, setting up the display mode and window size. It also sets the clear color for the back buffer. ```c++ // Create GL context glutInit(&argc, argv); glutInitDisplayMode(GLUT_RGBA | GLUT_ALPHA | GLUT_DOUBLE | GLUT_DEPTH); glutInitWindowSize(window_width, window_height); glutCreateWindow("OpenGL Application"); // default initialization of the back buffer gglClearColor(0.5, 0.5, 0.5, 1.0); ``` -------------------------------- ### Get cuBLAS Logger Callback Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index This function retrieves a pointer to the user-defined callback function previously installed using cublasSetLoggerCallback, or a null pointer if no callback is set. ```C cublasStatus_t cublasGetLoggerCallback( cublasLogCallback* userCallback); ``` -------------------------------- ### CUDA Initialization and Kernel Launch (C++) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cuda-c-programming-guide/index This code snippet demonstrates the basic initialization steps required for using the CUDA driver API, including device enumeration, context creation, module loading, memory allocation, and kernel launching. ```C++ int main(){ int N = ...; size_t size = N * sizeof(float); // Allocate input vectors h_A and h_B in host memory float* h_A = (float*)malloc(size); float* h_B = (float*)malloc(size); // Initialize input vectors ... // Initialize cuInit(0); // Get number of devices supporting CUDA int deviceCount = 0; cuDeviceGetCount(&deviceCount); if (deviceCount == 0) { printf("There is no device supporting CUDA.\n"); exit (0); } // Get handle for device 0 CUdevice cuDevice; cuDeviceGet(&cuDevice, 0); // Create context CUcontext cuContext; cuCtxCreate(&cuContext, 0, cuDevice); // Create module from binary file CUmodule cuModule; cuModuleLoad(&cuModule, "VecAdd.ptx"); // Allocate vectors in device memory CUdeviceptr d_A; cuMemAlloc(&d_A, size); CUdeviceptr d_B; cuMemAlloc(&d_B, size); CUdeviceptr d_C; cuMemAlloc(&d_C, size); // Copy vectors from host memory to device memory cuMemcpyHtoD(d_A, h_A, size); cuMemcpyHtoD(d_B, h_B, size); // Get function handle from module CUfunction vecAdd; cuModuleGetFunction(&vecAdd, cuModule, "VecAdd"); // Invoke kernel int threadsPerBlock = 256; int blocksPerGrid = (N + threadsPerBlock - 1) / threadsPerBlock; void* args[] = { &d_A, &d_B, &d_C, &N }; cuLaunchKernel(vecAdd, blocksPerGrid, 1, 1, threadsPerBlock, 1, 1, 0, 0, args, 0); ... } ``` -------------------------------- ### Initialize CUDA Surfaces and Launch Copy Kernel Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cuda-c-programming-guide/index Host code demonstrating CUDA array allocation, surface object creation, kernel launch configuration, and resource cleanup. Sets up two CUDA arrays with surface load/store capability, copies host data to device memory, creates surface objects for kernel access, launches a 2D grid of threads, and -------------------------------- ### Get cuBLAS library stream Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Gets the cuBLAS library stream, which is being used to execute all calls to the cuBLAS library functions. If not set, all kernels use the default NULL stream. ```C cublasStatus_t cublasGetStream(cublasHandle_t handle, cudaStream_t *streamId) ``` -------------------------------- ### SIMD Video Instructions: vop2 Examples Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index Demonstrates the syntax for various vop2 instructions, including add, subtract, and minimum operations, with optional saturation modifiers. ```asm vadd2.s32.s32.u32.sat r1, r2, r3, r1; vsub2.s32.s32.s32.sat r1.h0, r2.h10, r3.h32, r1; vmin2.s32.u32.u32.add r1.h10, r2.h00, r3.h22, r1; ``` -------------------------------- ### Install CUDA Toolkit Meta-Package for WSL 2 Source: https://docs.nvidia.com/cuda/archive/11.0_GA/wsl-user-guide/index This command installs the CUDA Toolkit version 11.0 for use within WSL 2. It's important to use the cuda-toolkit- meta-package to avoid installing the Linux NVIDIA driver, which is already handled by the Windows driver. ```bash $ apt-get install -y cuda-toolkit-11-0 ``` -------------------------------- ### cuBLAS Application with 1-based Indexing in C Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index This example illustrates a C application using cuBLAS for scaling a submatrix with 1-based indexing. It initializes a 6x5 matrix with values, allocates device memory, transfers data, modifies elements using cublasSscal with alpha=16.0 and beta=12.0 starting from position (2,3), and prints the updated matrix. Dependencies: cuda_runtime.h, cublas_v2.h; limitations: fixed dimensions, error handling via status checks without retries. ```C //Example 1. Application Using C and cuBLAS: 1-based indexing //----------------------------------------------------------- #include #include #include #include #include "cublas_v2.h" #define M 6 #define N 5 #define IDX2F(i,j,ld) ((((j)-1)*(ld))+((i)-1)) static __inline__ void modify (cublasHandle_t handle, float *m, int ldm, int n, int p, int q, float alpha, float beta){ cublasSscal (handle, n-q+1, &alpha, &m[IDX2F(p,q,ldm)], ldm); cublasSscal (handle, ldm-p+1, &beta, &m[IDX2F(p,q,ldm)], 1); } int main (void){ cudaError_t cudaStat; cublasStatus_t stat; cublasHandle_t handle; int i, j; float* devPtrA; float* a = 0; a = (float *)malloc (M * N * sizeof (*a)); if (!a) { printf ("host memory allocation failed"); return EXIT_FAILURE; } for (j = 1; j <= N; j++) { for (i = 1; i <= M; i++) { a[IDX2F(i,j,M)] = (float)((i-1) * M + j); } } cudaStat = cudaMalloc ((void**)&devPtrA, M*N*sizeof(*a)); if (cudaStat != cudaSuccess) { printf ("device memory allocation failed"); return EXIT_FAILURE; } stat = cublasCreate(&handle); if (stat != CUBLAS_STATUS_SUCCESS) { printf ("CUBLAS initialization failed\n"); return EXIT_FAILURE; } stat = cublasSetMatrix (M, N, sizeof(*a), a, M, devPtrA, M); if (stat != CUBLAS_STATUS_SUCCESS) { printf ("data download failed"); cudaFree (devPtrA); cublasDestroy(handle); return EXIT_FAILURE; } modify (handle, devPtrA, M, N, 2, 3, 16.0f, 12.0f); stat = cublasGetMatrix (M, N, sizeof(*a), devPtrA, M, a, M); if (stat != CUBLAS_STATUS_SUCCESS) { printf ("data upload failed"); cudaFree (devPtrA); cublasDestroy(handle); return EXIT_FAILURE; } cudaFree (devPtrA); cublasDestroy(handle); for (j = 1; j <= N; j++) { for (i = 1; i <= M; i++) { printf ("%7.0f", a[IDX2F(i,j,M)]); } printf ("\n"); } free(a); return EXIT_SUCCESS; } ``` -------------------------------- ### Get Block Dimension for cuBLASXt Tiling Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Retrieves the current block dimension used for tiling matrices in cuBLASXt math functions. ```c cublasXtGetBlockDim(cublasXtHandle_t handle, int *blockDim) ``` -------------------------------- ### Get cuBLAS library property Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Returns the value of the requested property in memory pointed to by value. Refer to libraryPropertyType for supported types. ```C cublasStatus_t cublasGetProperty(libraryPropertyType type, int *value) ``` -------------------------------- ### CUDA Hello World with Dynamic Parallelism Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cuda-c-programming-guide/index A simple C++/CUDA program demonstrating dynamic parallelism. It includes a parent kernel that launches a child kernel and synchronizes with it, showcasing inter-kernel communication and execution flow on the device. ```cuda #include __global__ void childKernel() { printf("Hello "); } __global__ void parentKernel() { // launch child childKernel<<<1,1>>>(); if (cudaSuccess != cudaGetLastError()) { return; } // wait for child to complete if (cudaSuccess != cudaDeviceSynchronize()) { return; } printf("World!\n"); } int main(int argc, char *argv[]) { // launch parent parentKernel<<<1,1>>>(); if (cudaSuccess != cudaGetLastError()) { return 1; } // wait for parent to complete if (cudaSuccess != cudaDeviceSynchronize()) { return 2; } return 0; } ``` -------------------------------- ### Get cuBLAS library version Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Returns the version number of the cuBLAS library. The version is stored in the memory pointed to by the version parameter. ```C cublasStatus_t cublasGetVersion(cublasHandle_t handle, int *version) ``` -------------------------------- ### Fortran 77 cuBLAS Matrix Modification Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index This Fortran 77 code snippet demonstrates how to modify a matrix using the cuBLAS library. It includes device memory allocation, data transfer, matrix scaling operations, and memory deallocation. The example is designed to be compiled with ARCH_64 defined for 64-bit systems. ```fortran ! Example B.2. Same Application Using Non-thunking cuBLAS Calls !------------------------------------------------------------- #define IDX2F (i,j,ld) ((((j)-1)*(ld))+((i)-1)) subroutine modify ( devPtrM, ldm, n, p, q, alpha, beta ) implicit none integer sizeof_real parameter (sizeof_real=4) integer ldm, n, p, q #if ARCH_64 integer*8 devPtrM #else integer*4 devPtrM #endif real*4 alpha, beta call cublas_sscal ( n-p+1, alpha, 1 devPtrM+IDX2F(p, q, ldm)*sizeof_real, 2 ldm) call cublas_sscal(ldm-p+1, beta, 1 devPtrM+IDX2F(p, q, ldm)*sizeof_real, 2 1) return end program matrixmod implicit none integer M,N,sizeof_real #if ARCH_64 integer*8 devPtrA #else integer*4 devPtrA #endif parameter(M=6,N=5,sizeof_real=4) real*4 a(M,N) integer i,j,stat external cublas_init, cublas_set_matrix, cublas_get_matrix external cublas_shutdown, cublas_alloc integer cublas_alloc, cublas_set_matrix, cublas_get_matrix do j=1,N do i=1,M a(i,j)=(i-1)*M+j enddo enddo call cublas_init stat= cublas_alloc(M*N, sizeof_real, devPtrA) if (stat.NE.0) then write(*,*) "device memory allocation failed" call cublas_shutdown stop endif stat = cublas_set_matrix(M,N,sizeof_real,a,M,devPtrA,M) if (stat.NE.0) then call cublas_free( devPtrA ) write(*,*) "data download failed" call cublas_shutdown stop endif call modify(devPtrA, M, N, 2, 3, 16.0, 12.0) stat = cublas_get_matrix(M, N, sizeof_real, devPtrA, M, a, M ) if (stat.NE.0) then call cublas_free ( devPtrA ) write(*,*) "data upload failed" call cublas_shutdown stop endif call cublas_free ( devPtrA ) call cublas_shutdown do j = 1 , N do i = 1 , M write (*," (F7.0$)") a(i,j) enddo write (*,*) "" enddo stop end ``` -------------------------------- ### Set cuBLAS Logger Callback Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index This function installs a custom user-defined callback function for logging using the cuBLAS C public API. ```C cublasStatus_t cublasSetLoggerCallback( cublasLogCallback userCallback); ``` -------------------------------- ### Get cuBLAS Math Mode Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index The cublasGetMathMode function retrieves the currently configured math mode used by cuBLAS routines. It returns CUBLAS_STATUS_SUCCESS upon successful retrieval. ```C cublasStatus_t cublasGetMathMode(cublasHandle_t handle, cublasMath_t *mode); ``` -------------------------------- ### cublasgetriBatched Two-Step Usage Example Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Complete usage example showing the required two-step process for matrix inversion. First performs in-place LU decomposition using cublasDgetrfBatched, then performs out-ofplace inversion using cublasDgetriBatched. Input matrix A must be factorized first before inversion. Memory space of Carray cannot overlap with Aarray to avoid undefined behavior. ```C // step 1: perform in-place LU decomposition, P*A = L*U. // Aarray[i] is n*n matrix A[i] cublasDgetrfBatched(handle, n, Aarray, lda, PivotArray, infoArray, batchSize); // check infoArray[i] to see if factorization of A[i] is successful or not. // Array[i] contains LU factorization of A[i] // step 2: perform out-of-place inversion, Carray[i] = inv(A[i]) cublasDgetriBatched(handle, n, Aarray, lda, PivotArray, Carray, ldc, infoArray, batchSize); // check infoArray[i] to see if inversion of A[i] is successful or not. ``` -------------------------------- ### Node-wide process profiling with nvprof Source: https://docs.nvidia.com/cuda/archive/11.0_GA/profiler-users-guide/index Shows how to profile all processes on a node using nvprof's --profile-all-processes flag. Output files use %h and %p placeholders for hostname and process ID. ```bash nvprof --profile-all-processes -o output.%h.%p ``` -------------------------------- ### NPP Multiple Output Destination Pointers Example (C/C++) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/npp/npps_conventions_lb Shows the naming convention for multiple destination signal pointers when an NPP primitive writes to more than one output. Pointers are sequentially numbered starting from pDst1. ```c // Example for a primitive with two output signals: NppStatus nppsMultiDestPrimitive(const Npp32s * pSrc, Npp32s * pDst1, Npp32s * pDst2, ...) ``` -------------------------------- ### Get cuBLAS pointer mode Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Obtains the pointer mode used by the cuBLAS library. The pointer mode determines whether scalar values are passed by reference on the host or device. ```C cublasStatus_t cublasGetPointerMode(cublasHandle_t handle, cublasPointerMode_t *mode) ``` -------------------------------- ### Initialize nvJPEG Library with Default Settings Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvjpeg/index Simplified initialization function that creates nvJPEG library handle with library-selected default codec implementations and memory allocators. This is the recommended initialization method that eliminates the need to specify backend type or memory allocation strategy. ```c nvjpegStatus_t nvjpegCreateSimple(nvjpegHandle_t *handle); ``` -------------------------------- ### NPP Multiple Input Source Pointers Example (C/C++) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/npp/npps_conventions_lb Demonstrates how multiple input source signal pointers are named in NPP primitives when an algorithm requires more than one input. Pointers are sequentially numbered starting from pSrc1. ```c // Example for a primitive with two input signals: NppStatus nppsMultiSourcePrimitive(const Npp32s * pSrc1, const Npp32s * pSrc2, ...) ``` -------------------------------- ### PTX Ret Instruction Examples Source: https://docs.nvidia.com/cuda/archive/11.0_GA/parallel-thread-execution/index Illustrates the 'ret' instruction in PTX for returning from a function. Shows basic return and conditional return usage. ```ptx ret; @p ret; ``` -------------------------------- ### Get cuBLASLt Matrix Transform Descriptor Attribute Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Retrieves the value of a queried attribute from a matrix transform descriptor. It requires the descriptor, attribute type, a buffer for the output, and buffer size. Returns the attribute value and size written. ```c cublasStatus_t cublasLtMatrixTransformDescGetAttribute( cublasLtMatrixTransformDesc_t transformDesc, cublasLtMatrixTransformDescAttributes_t attr, const void *buf, size_t sizeInBytes, size_t *sizeWritten); ``` -------------------------------- ### Compile NPP Applications on Linux Source: https://docs.nvidia.com/cuda/archive/11.0_GA/npp/index Shows compilation commands for NPP applications using both nvcc and g++ compilers. Demonstrates dynamic linking against shared libraries and static linking with required dependencies like cuLIBOS, pthread, and dl. Includes include paths and library directory flags for CUDA toolkit integration. ```bash nvcc foo.c -lnppc -lnppicc -o foo ``` ```bash nvcc foo.c -lnppc_static -lnppicc_static -lculibos -o foo ``` ```bash g++ foo.c -lnppc_static -lnppicc_static -lculibos -lcudart_static -lpthread -ldl -I /include -L /lib64 -o foo ``` -------------------------------- ### Get cuBLASLt Algorithm Configuration Attribute Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Retrieves the value of a specified configuration attribute from a cuBLASLt matrix multiplication algorithm descriptor. The function supports buffer size verification and returns the actual bytes written or required for the attribute value. ```c cublasStatus_t cublasLtMatmulAlgoConfigGetAttribute( cublasLtMatmulAlgo_t *algo, cublasLtMatmulAlgoConfigAttributes_t attr, void *buf, size_t sizeInBytes, size_t *sizeWritten); ``` -------------------------------- ### CUDA Driver API launch and host-device synchronization (C++) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/nvrtc/index Launches a kernel using the CUDA Driver API and synchronizes the context afterward. Demonstrates cuLaunchKernel with grid/block dimensions, shared memory, stream, and argument list, followed by cuCtxSynchronize to ensure host waits for kernel completion. ```cpp // Assuming 'kernel' is a CUfunction obtained via cuModuleGetFunction using the lowered name // and 'args' is a properly constructed array of kernel arguments: CUDA_SAFE_CALL(cuLaunchKernel( kernel, 1, 1, 1, // grid dimensions (x, y, z) 1, 1, 1, // block dimensions (x, y, z) 0, // shared memory bytes nullptr, // CUstream args, // argument array nullptr // extra (optional) )); // Ensure host waits for kernel completion before accessing any results. CUDA_SAFE_CALL(cuCtxSynchronize()); ``` -------------------------------- ### Thrust Transformation Algorithms Demo Source: https://docs.nvidia.com/cuda/archive/11.0_GA/thrust/index Comprehensive example demonstrating various Thrust transformation operations including sequence generation, negation, modulus, filling, and replacement. Uses device vectors and showcases built-in functors from thrust/functional.h for common arithmetic operations. ```cpp #include #include #include #include #include #include #include #include int main(void) { // allocate three device_vectors with 10 elements thrust::device_vector X(10); thrust::device_vector Y(10); thrust::device_vector Z(10); // initialize X to 0,1,2,3, .... thrust::sequence(X.begin(), X.end()); // compute Y = -X thrust::transform(X.begin(), X.end(), Y.begin(), thrust::negate()); // fill Z with twos thrust::fill(Z.begin(), Z.end(), 2); // compute Y = X mod 2 thrust::transform(X.begin(), X.end(), Z.begin(), Y.begin(), thrust::modulus()); // replace all the ones in Y with tens thrust::replace(Y.begin(), Y.end(), 1, 10); // print Y thrust::copy(Y.begin(), Y.end(), std::ostream_iterator(std::cout, "\n")); return 0; } ``` -------------------------------- ### Get cublasLt Matrix Layout Attribute (C) Source: https://docs.nvidia.com/cuda/archive/11.0_GA/cublas/index Retrieves the value of a specified attribute from a cublasLt matrix layout descriptor. It requires a valid matrix layout, attribute type, buffer for the output, buffer size, and a pointer to store the size written. Returns a status code indicating success or failure. ```c cublasStatus_t cublasLtMatrixLayoutGetAttribute( cublasLtMatrixLayout_t matLayout, cublasLtMatrixLayoutAttribute_t attr, void *buf, size_t sizeInBytes, size_t *sizeWritten); ```