GPU Burn

https://github.com/wilicc/gpu-burn
Admin
GPU Burn is a multi-GPU CUDA stress test tool designed to put a heavy load on NVIDIA GPUs.
Tokens:2,763
Snippets:19
Trust Score:7.4
License:BSD-2-Clause
Update:8 months ago
Show doc for...
Tokens:
Raw
### gpu-burn Docker Deployment Examples

Source: https://context7.com/wilicc/gpu-burn/llms.txt

Provides examples for building and running the gpu-burn Docker image. It covers basic image building, running stress tests with GPU access, specifying custom durations and precision, and building with specific CUDA versions and distributions.

```bash
# Clone and build the Docker image
git clone https://github.com/wilicc/gpu-burn
cd gpu-burn
docker build -t gpu_burn .

# Run the stress test with GPU access (default 60 seconds)
docker run --rm --gpus all gpu_burn

# Run for a custom duration
docker run --rm --gpus all gpu_burn ./gpu_burn 300

# Run with double precision
docker run --rm --gpus all gpu_burn ./gpu_burn -d 120

# Build with specific CUDA version and distro
docker build \
  --build-arg CUDA_VERSION=12.0.1 \
  --build-arg IMAGE_DISTRO=ubuntu22.04 \
  -t gpu_burn:cuda12 .

# Run specific GPU with custom memory allocation
docker run --rm --gpus '"device=0"' gpu_burn ./gpu_burn -m 80% 180
```

--------------------------------

### gpu-burn Command-Line Output Examples

Source: https://context7.com/wilicc/gpu-burn/llms.txt

Examples of output from the gpu_burn tool, showing progress reporting, error counts, GPU temperatures, and final test summaries. This includes both successful and faulty GPU test outcomes.

```bash
# Example output from gpu_burn 120
GPU 0: NVIDIA GeForce RTX 3080, 10240MB
Using compare file: compare.ptx
Burning for 120 seconds.
Initialized device 0 with 10018 MB of memory (9756 MB available, using 8780 MB of it), using FLOATS
Results are 268435456 bytes each, thus performing 32 iterations

# Progress output format:
# [progress%]  proc'd: [iterations] ([Gflop/s]) - ... errors: [count] - ... temps: [°C] - ...
10.0%  proc'd: 1024 (2150 Gflop/s) - errors: 0 - temps: 67 C -
20.0%  proc'd: 2048 (2145 Gflop/s) - errors: 0 - temps: 71 C -
...
100.0% proc'd: 10240 (2148 Gflop/s) - errors: 0 - temps: 74 C -

# Final summary
Killing processes with SIGTERM (soft kill)
done

Tested 1 GPUs:
    GPU 0: OK

# Error output example (faulty GPU):
Tested 2 GPUs:
    GPU 0: OK
    GPU 1: FAULTY
```

--------------------------------

### gpu-burn Building from Source Examples

Source: https://context7.com/wilicc/gpu-burn/llms.txt

Illustrates Makefile commands for building gpu-burn from source. It shows how to specify compute capability, custom CUDA and GCC paths, compiler flags, linker flags, and NVCC flags for customized builds.

```bash
# Standard build (uses default CUDA path and compute capability 7.5)
make

# Build for specific compute capability (e.g., 8.6 for RTX 30xx)
make COMPUTE=86

# Specify custom CUDA toolkit path
make CUDAPATH=/usr/local/cuda-12.0

# Specify custom gcc path
make CCPATH=/usr/local/bin

# Add custom compiler flags
make CFLAGS="-Wall -Werror"

# Add custom linker flags
make LDFLAGS="-L/opt/cuda/lib64"

# Add custom nvcc flags
make NVCCFLAGS="-ccbin /usr/bin/g++-11"

# Build Docker image with custom tag and CUDA version
make IMAGE_NAME=myregistry/gpu-burn CUDA_VERSION=12.0.1 IMAGE_DISTRO=ubuntu22.04 image

# Clean build artifacts
make clean
```

--------------------------------

### Docker Build and Run gpu-burn

Source: https://github.com/wilicc/gpu-burn/blob/master/README.md

This snippet demonstrates how to clone the gpu-burn repository, build a Docker image, and run the container to execute the stress test on all available GPUs. It requires Docker to be installed.

```plain
git clone https://github.com/wilicc/gpu-burn
cd gpu-burn
docker build -t gpu_burn .
docker run --rm --gpus all gpu_burn
```

--------------------------------

### gpu-burn Command Line Interface Examples

Source: https://context7.com/wilicc/gpu-burn/llms.txt

Demonstrates various command-line options for the gpu_burn executable, including setting test duration, precision (single/double), memory usage percentage or absolute value, GPU selection, tensor core usage, and custom compare kernel files.

```bash
# Basic usage - run stress test for 60 seconds on all GPUs
gpu_burn 60

# Run for 1 hour using double-precision floating point
gpu_burn -d 3600

# Use 50% of available GPU memory instead of default 90%
gpu_burn -m 50% 300

# Use exactly 4096 MB of GPU memory
gpu_burn -m 4096 120

# List all available GPUs in the system
gpu_burn -l

# Run stress test only on GPU index 2
gpu_burn -i 2 60

# Use tensor cores if available (Volta and newer)
gpu_burn -tc 120

# Combine options: doubles + tensor cores on GPU 0 for 30 minutes
gpu_burn -d -tc -i 0 1800

# Custom compare kernel file
gpu_burn -c /path/to/compare.ptx 60

# Set SIGTERM timeout threshold to 60 seconds
gpu_burn -stts 60 300
```

--------------------------------

### Build gpu-burn from Source

Source: https://github.com/wilicc/gpu-burn/blob/master/README.md

Instructions for compiling gpu-burn using `make`. It covers basic build, cleaning artifacts, and overriding default build configurations like Compute Capability, CFLAGS, LDFLAGS, NVCCFLAGS, CUDAPATH, and CCPATH.

```makefile
make
make clean
make COMPUTE=<compute capability value>
make CFLAGS=-Wall
make LDFLAGS=-lmylib
make NVCCFLAGS=-ccbin <path to host compiler>
make CUDAPATH=/usr/local/cuda-<version>
make CCPATH=/usr/local/bin
```

--------------------------------

### Build Docker Image with Custom Options

Source: https://github.com/wilicc/gpu-burn/blob/master/README.md

This snippet shows how to build a Docker image for gpu-burn with custom settings for CUDA version, distribution, and image name. This is useful for specific deployment environments or version control.

```makefile
make IMAGE_NAME=myregistry.private.com/gpu-burn CUDA_VERSION=12.0.1 IMAGE_DISTRO=ubuntu22.04 image
```

--------------------------------

### GPU_Test Class Core Operations (C++)

Source: https://context7.com/wilicc/gpu-burn/llms.txt

Demonstrates the core operations of the `GPU_Test` template class in C++. This includes initialization with specific GPU and precision, memory allocation with different strategies (percentage, absolute, default), performing matrix computations, validating results, and querying memory information.

```cpp
// Internal architecture - GPU_Test initialization
// Each GPU gets its own CUDA context and cuBLAS handle
GPU_Test<float> test(0, false, false, "compare.ptx");  // GPU 0, floats, no tensors

// Memory allocation uses 90% of available GPU memory by default
// Matrices A and B are copied to device, C stores results
test.initBuffers(hostA, hostB);  // Uses USEMEM (90%) of free memory
test.initBuffers(hostA, hostB, 1024*1024*1024);  // Use exactly 1GB
test.initBuffers(hostA, hostB, -50);  // Use 50% of available memory

// Computation performs SIZE*SIZE matrix multiplications
// SIZE = 8192, resulting in ~1.1 trillion operations per multiply
test.compute();  // Executes SGEMM or DGEMM based on template type

// Comparison validates results using compare.ptx CUDA kernel
// Detects computational errors within EPSILON threshold
test.compare();

// Error reporting returns count of faulty matrix elements
unsigned long long errors = test.getErrors();

// Query memory information
size_t total = test.totalMemory();  // Total GPU memory
size_t avail = test.availMemory();  // Available GPU memory
```

--------------------------------

### gpu-burn Command-Line Usage

Source: https://github.com/wilicc/gpu-burn/blob/master/README.md

This section outlines the command-line interface for the `gpu_burn` executable. It details various options for controlling memory usage, precision (doubles, Tensor cores), GPU selection, and displaying help.

```plain
GPU Burn
Usage: gpu_burn [OPTIONS] [TIME]

-m X   Use X MB of memory
-m N%  Use N% of the available GPU memory
-d     Use doubles
-tc    Try to use Tensor cores (if available)
-l     List all GPUs in the system
-i N   Execute only on GPU N
-h     Show this help message

Example:
gpu_burn -d 3600
```

--------------------------------

### CUDA Kernel for Comparing Matrix Results

Source: https://context7.com/wilicc/gpu-burn/llms.txt

The compare.cu CUDA kernel validates computational accuracy by comparing matrix results across iterations. It detects bit flips or computation errors that may indicate hardware faults. It uses EPSILON thresholds for error detection and marks faulty elements. A double-precision version is also available.

```cuda
// EPSILON thresholds for error detection
#define EPSILON 0.001f      // For single-precision floats
#define EPSILOND 0.0000001  // For double-precision

// Kernel compares each element across all iterations
// If results differ beyond epsilon, element is marked faulty
extern "C" __global__ void compare(float *C, int *faultyElems, size_t iters) {
    size_t iterStep = blockDim.x * blockDim.y * gridDim.x * gridDim.y;
    size_t myIndex = (blockIdx.y * blockDim.y + threadIdx.y) * 
                     gridDim.x * blockDim.x + 
                     blockIdx.x * blockDim.x + threadIdx.x;

    int myFaulty = 0;
    for (size_t i = 1; i < iters; ++i)
        if (fabsf(C[myIndex] - C[myIndex + i * iterStep]) > EPSILON)
            myFaulty++;

    atomicAdd(faultyElems, myFaulty);
}

// Double-precision version for -d flag
extern "C" __global__ void compareD(double *C, int *faultyElems, size_t iters);
```
Tokens:
Raw