CUDA Python

https://github.com/nvidia/cuda-python
Admin
CUDA Python provides Pythonic access to NVIDIA's CUDA platform, offering components like cuda.core...
Tokens:50,684
Snippets:814
Trust Score:9.5
License:Apache-2.0
Update:1 month ago
Show doc for...
Tokens:
Raw
### Define kernel and host setup

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Example structure for a CUDA kernel and host-side resource creation.

```python
simpleCubemapTexture = """\
extern "C"
__global__ void transformKernel(float *g_odata, int width, cudaTextureObject_t tex)
{
    ...
}
"""

def main():
    ...
    d_data = checkCudaErrors(cudart.cudaMalloc(size))
    width = 64
    tex = checkCudaErrors(cudart.cudaCreateTextureObject(texRes, texDescr, None))
    ...
```

--------------------------------

### Install from Source

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/install.rst

Commands to clone the repository and perform a standard or editable installation.

```console
$ git clone https://github.com/NVIDIA/cuda-python.git
$ cd cuda-python/cuda_pathfinder
$ pip install .
```

```console
$ pip install -v -e .
```

--------------------------------

### Install via PyPI

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/install.rst

Standard installation command for the package using pip.

```console
$ pip install -U cuda-pathfinder
```

--------------------------------

### Download and install nightly builds

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/install.rst

Use the GitHub CLI to download and install the latest CI-built wheels for development testing.

```console
$ # Find the latest successful CI run on main:
$ RUN_ID=$(gh run list -R NVIDIA/cuda-python -w ci.yml -b main -s success -L1 --json databaseId -q '.[0].databaseId')

$ # Download the wheel (pick your Python version and platform):
$ gh run download "$RUN_ID" -R NVIDIA/cuda-python -p "cuda-core-python312-linux-64-*"

$ # Install the downloaded wheel:
$ pip install cuda-core-python312-linux-64-*/cuda_core*.whl[cu13]
```

--------------------------------

### Development setup with uv

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/install.rst

Set up an editable development environment using the uv package manager.

```console
$ git clone https://github.com/NVIDIA/cuda-python.git
$ cd cuda-python/cuda_core
$ uv venv
$ source .venv/bin/activate   # On Windows: .venv\Scripts\activate
$ uv pip install -e .[cu13] --group test
```

```console
$ python -m pytest tests
```

--------------------------------

### Perform an editable install

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/install.rst

Install the module in editable mode for development and testing purposes.

```console
$ pip install -v -e .
```

--------------------------------

### Install via Conda

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/install.rst

Installation command for the package using the conda-forge channel.

```console
$ conda install -c conda-forge cuda-pathfinder
```

--------------------------------

### Install cuda-python via PyPI

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/install.rst

Standard installation command for the cuda-python package.

```console
$ pip install -U cuda-python
```

--------------------------------

### Install cuda.core

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Use pip to install the core package with CUDA 13 support.

```console
$ pip install "cuda-core[cu13]"
```

--------------------------------

### Development setup with pixi

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/install.rst

Run tests in a reproducible environment using pixi.

```console
$ git clone https://github.com/NVIDIA/cuda-python.git
$ cd cuda-python
$ pixi run -e cu13 test-core
```

```console
$ pixi run -e cu13 test
```

--------------------------------

### Install cuda-core from source

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/install.rst

Commands to clone the repository and install the package using pip. Ensure a full clone is performed to maintain git tags for versioning.

```console
$ git clone https://github.com/NVIDIA/cuda-python.git
$ cd cuda-python/cuda_core
$ pip install .
```

--------------------------------

### cuda.bindings.runtime.cudaProfilerStart

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Starts the CUDA profiler.

```APIDOC
## cuda.bindings.runtime.cudaProfilerStart

### Description
Enables profiling for the current process. This function starts the collection of profiling data.
```

--------------------------------

### Install nightly builds from GitHub Actions

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/install.rst

Commands to fetch and install the latest development wheels from the NVIDIA/cuda-python repository.

```console
$ # Find the latest successful CI run on main:
$ RUN_ID=$(gh run list -R NVIDIA/cuda-python -w ci.yml -b main -s success -L1 --json databaseId -q '.[0].databaseId')

$ # Download the wheel (pick your Python version and platform):
$ gh run download "$RUN_ID" -R NVIDIA/cuda-python -p "cuda-bindings-python312-cuda13*-linux-64-*"

$ # Install the downloaded wheel:
$ pip install cuda-bindings-python312-cuda13*-linux-64-*/cuda_bindings*.whl[all]
```

--------------------------------

### Install cuda.core via PyPI

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/install.rst

Install the package using pip with the appropriate CUDA version extras.

```console
$ pip install cuda-core[cu12]
```

--------------------------------

### Install cuda-python with optional dependencies

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/install.rst

Installs the package along with all optional CUDA Toolkit components.

```console
$ pip install -U cuda-python[all]
```

--------------------------------

### Install cuda-python with optional dependencies

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/release/11.8.6-notes.rst

Use this command to install the meta package along with all optional dependencies including NVRTC.

```shell
pip install cuda-python[all]
```

--------------------------------

### Install cuda-python via Conda

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/install.rst

Standard installation command using the conda-forge channel.

```console
$ conda install -c conda-forge cuda-python
```

--------------------------------

### Install cuda.core via Conda

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/install.rst

Install the package from the conda-forge channel specifying the CUDA version.

```console
$ conda install -c conda-forge cuda-core cuda-version=12
```

--------------------------------

### Define and Allocate Kernel Arguments

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Setup custom data types and allocate device memory for kernel arguments using NumPy and cudart.

```python
# Define a custom type
testStruct = np.dtype([("value", np.int32)], align=True)

# Allocate device memory
pInt = checkCudaErrors(cudart.cudaMalloc(np.dtype(np.int32).itemsize))
pFloat = checkCudaErrors(cudart.cudaMalloc(np.dtype(np.float32).itemsize))
pStruct = checkCudaErrors(cudart.cudaMalloc(testStruct.itemsize))

# Collect all input kernel arguments into a single tuple for further processing
kernelValues = (
    np.array(1, dtype=np.uint32),
    np.array([pInt], dtype=np.intp),
    np.array(123.456, dtype=np.float32),
    np.array([pFloat], dtype=np.intp),
    np.array([5], testStruct),
    np.array([pStruct], dtype=np.intp),
)
```

--------------------------------

### get_cuda_path_or_home

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/api.rst

Retrieves the path to the CUDA installation or the home directory.

```APIDOC
## get_cuda_path_or_home()

### Description
Returns the path to the CUDA installation or the CUDA home directory based on environment variables or system defaults.
```

--------------------------------

### Install cuda-python with specific CUDA version via Conda

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/install.rst

Installs the package while pinning the CUDA Toolkit version.

```console
$ conda install -c conda-forge cuda-python cuda-version=13
```

--------------------------------

### Define a CUDA kernel for argument testing

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Example kernel definition containing primitive types and a custom structure for testing argument passing.

```python
kernel_string = """
typedef struct {
    int value;
} testStruct;

extern "C" __global__
void testkernel(int i, int *pi,
                float f, float *pf,
                testStruct s, testStruct *ps)
{
    *pi = i;
    *pf = f;
    ps->value = s.value;
}
"""
```

--------------------------------

### Coredump Attribute Management

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Functions for getting and setting coredump attributes and registering callbacks for coredump events.

```APIDOC
## cuCoredumpGetAttribute
Retrieves the value of a specific coredump attribute.

## cuCoredumpGetAttributeGlobal
Retrieves the value of a global coredump attribute.

## cuCoredumpSetAttribute
Sets the value of a specific coredump attribute.

## cuCoredumpSetAttributeGlobal
Sets the value of a global coredump attribute.

## cuCoredumpRegisterStartCallback
Registers a callback function to be executed when a coredump starts.

## cuCoredumpRegisterCompleteCallback
Registers a callback function to be executed when a coredump completes.

## cuCoredumpDeregisterStartCallback
Deregisters a previously registered coredump start callback.

## cuCoredumpDeregisterCompleteCallback
Deregisters a previously registered coredump completion callback.
```

--------------------------------

### Prepare kernel arguments and configuration

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/getting-started.rst

Retrieve the kernel from the module and set up input/output arrays and launch configuration parameters.

```python
ker = mod.get_kernel("vector_add<float>")

# Prepare input/output arrays (using CuPy)
size = 50000
rng = cp.random.default_rng()
a = rng.random(size, dtype=cp.float32)
b = rng.random(size, dtype=cp.float32)
c = cp.empty_like(a)

# Configure launch parameters
block = 256
grid = (size + block - 1) // block
config = LaunchConfig(grid=grid, block=block)
```

--------------------------------

### Launch kernel and synchronize

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/getting-started.rst

Execute the kernel using the launch function with device pointers and synchronize the stream.

```python
launch(s, config, ker, a.data.ptr, b.data.ptr, c.data.ptr, cp.uint64(size))
s.sync()
```

--------------------------------

### Copy Data and Launch Kernels

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Perform host-to-device transfers, launch kernels with specific configurations, and synchronize streams.

```python
dbuf.copy_from(host, stream=stream)          # H2D

block = 256
grid = (n + block - 1) // block
config = LaunchConfig(grid=grid, block=block)
launch(stream, config, scale, dbuf, np.float32(3.0), np.uint64(n))

dbuf.copy_to(host, stream=stream)            # D2H
stream.sync()                                # wait for all of the above

assert np.array_equal(host_np, np.arange(n, dtype=np.float32) * 3.0)
```

--------------------------------

### Initialize Device and Stream

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/getting-started.rst

Create a Device object and a corresponding Stream for execution, ensuring the device is set as current.

```python
import cupy as cp
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch

dev = Device()
dev.set_current()
s = dev.create_stream()
```

--------------------------------

### Launch Kernel and Synchronize

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Executes the kernel on the device and copies results back to the host, followed by stream synchronization.

```python
checkCudaErrors(driver.cuLaunchKernel(
   kernel,
   NUM_BLOCKS,  # grid x dim
   1,  # grid y dim
   1,  # grid z dim
   NUM_THREADS,  # block x dim
   1,  # block y dim
   1,  # block z dim
   0,  # dynamic shared memory
   stream,  # stream
   args.ctypes.data,  # kernel arguments
   0,  # extra (ignore)
))

checkCudaErrors(driver.cuMemcpyDtoHAsync(
   hOut.ctypes.data, dOutclass, bufferSize, stream
))
checkCudaErrors(driver.cuStreamSynchronize(stream))
```

--------------------------------

### Initialize and query a GPU device

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Create a device object and set it as the current context before performing GPU operations.

```python
dev = Device()        # current device (device 0 if none is current); Device(1) picks a specific GPU
dev.set_current()     # initialize CUDA and make this device current

print(dev.name)                 # e.g. 'NVIDIA GB10'
print(dev.compute_capability)   # ComputeCapability(major=12, minor=1)
print(dev.arch)                 # '121'  (handy for building kernels below)
```

--------------------------------

### Launch Kernel with Buffer Protocol

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Execute the kernel using the constructed parameters array via the cuLaunchKernel API.

```python
checkCudaErrors(cuda.cuLaunchKernel(
    kernel,
    1, 1, 1,  # grid dim
    1, 1, 1,  # block dim
    0, stream,  # shared mem and stream
    kernelParams=kernelParams,
    extra=0,
))
```

--------------------------------

### Initialize Host Data

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Sets up host-side NumPy arrays for SAXPY operations.

```python
NUM_THREADS = 512  # Threads per block
NUM_BLOCKS = 32768  # Blocks per grid

a = np.array([2.0], dtype=np.float32)
n = np.array(NUM_THREADS * NUM_BLOCKS, dtype=np.uint32)
bufferSize = n * a.itemsize

hX = np.random.rand(n).astype(dtype=np.float32)
hY = np.random.rand(n).astype(dtype=np.float32)
hOut = np.zeros(n).astype(dtype=np.float32)
```

--------------------------------

### Launch a CUDA kernel

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Executes a kernel using the prepared parameter tuple.

```python
checkCudaErrors(cuda.cuLaunchKernel(
    kernel,
    1, 1, 1,  # grid dim
    1, 1, 1,  # block dim
    0, stream,  # shared mem and stream
    kernelParams=(kernelValues, kernelTypes),
    extra=0,
))
```

--------------------------------

### Create a GPU Stream

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Initialize a stream to manage ordered GPU operations.

```python
stream = dev.create_stream()
```

--------------------------------

### Define and allocate kernel arguments

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Demonstrates defining a custom ctypes structure and allocating device memory for kernel parameters.

```python
# Define a custom type
class testStruct(ctypes.Structure):
    _fields_ = [("value", ctypes.c_int)]

# Allocate device memory
pInt = checkCudaErrors(cudart.cudaMalloc(ctypes.sizeof(ctypes.c_int)))
pFloat = checkCudaErrors(cudart.cudaMalloc(ctypes.sizeof(ctypes.c_float)))
pStruct = checkCudaErrors(cudart.cudaMalloc(ctypes.sizeof(testStruct)))

# Collect all input kernel arguments into a single tuple for further processing
kernelValues = (
    1,
    pInt,
    123.456,
    pFloat,
    testStruct(5),
    pStruct,
)
kernelTypes = (
    ctypes.c_int,
    ctypes.c_void_p,
    ctypes.c_float,
    ctypes.c_void_p,
    None,
    ctypes.c_void_p,
)
```

--------------------------------

### Green Contexts API

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Functions for creating and managing lightweight green contexts to partition and utilize GPU resources.

```APIDOC
## cuGreenCtxCreate
Creates a new green context based on a provided resource descriptor.

## cuGreenCtxStreamCreate
Creates a stream associated with a specific green context for kernel launches and operations.
```

--------------------------------

### Profile CUDA applications

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Use the NVIDIA Nsight Systems CLI to profile CUDA application performance.

```shell
nsys profile -s none -t cuda --stats=true <executable>
```

--------------------------------

### Initialization

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Function to initialize the CUDA driver API.

```APIDOC
## cuda.bindings.driver.cuInit

### Description
Initializes the CUDA driver API.
```

--------------------------------

### Initialize CUDA and Compile PTX

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Initializes the driver, retrieves device attributes, and compiles a program into PTX format using NVRTC.

```python
# Initialize CUDA Driver API
checkCudaErrors(driver.cuInit(0))

# Retrieve handle for device 0
cuDevice = checkCudaErrors(driver.cuDeviceGet(0))

# Derive target architecture for device 0
major = checkCudaErrors(driver.cuDeviceGetAttribute(driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, cuDevice))
minor = checkCudaErrors(driver.cuDeviceGetAttribute(driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, cuDevice))
arch_arg = bytes(f'--gpu-architecture=compute_{major}{minor}', 'ascii')

# Create program
prog = checkCudaErrors(nvrtc.nvrtcCreateProgram(str.encode(saxpy), b"saxpy.cu", 0, [], []))

# Compile program
opts = [b"--fmad=false", arch_arg]
checkCudaErrors(nvrtc.nvrtcCompileProgram(prog, 2, opts))

# Get PTX from compilation
ptxSize = checkCudaErrors(nvrtc.nvrtcGetPTXSize(prog))
ptx = b" " * ptxSize
checkCudaErrors(nvrtc.nvrtcGetPTX(prog, ptx))
```

--------------------------------

### cudaGreenCtxCreate

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Creates a green context with specified resources.

```APIDOC
## cudaGreenCtxCreate

### Description
Creates a green context with the specified resources and returns an execution context handle.

### Signature
`cudaGreenCtxCreate(resources, ...)`
```

--------------------------------

### Process.restore

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/api.rst

Restores a CPU process image from a checkpoint, optionally remapping GPUs.

```APIDOC
## Process.restore(gpu_mapping=None)

### Description
Restores the CPU process image. Optionally remaps GPUs by providing a mapping from checkpointed GPU UUIDs to target GPU UUIDs.

### Parameters
- **gpu_mapping** (dict) - Optional - A dictionary mapping checkpointed GPU UUIDs to target GPU UUIDs.
```

--------------------------------

### Modifying CULaunchConfig Attributes

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/tips_and_tricks.rst

Demonstrates the correct approach to updating attributes for extension types like CULaunchConfig by using explicit setters instead of in-place modifications.

```python
cfg = cuda.CUlaunchConfig()

cfg.numAttrs += 1
attr = cuda.CUlaunchAttribute()

...

# This works. We are passing the new attribute to the setter
drv_cfg.attrs = [attr]

# This does not work. We are only modifying the returned attribute in place
drv_cfg.attrs.append(attr)
```

--------------------------------

### Prepare Kernel Arguments

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Constructs a contiguous array of pointers to be passed as arguments to the kernel.

```python
args = [a, dX, dY, dOut, n]
args = np.array([arg.ctypes.data for arg in args], dtype=np.uint64)
```

--------------------------------

### Create a NumPy View of Host Memory

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Use DLPack to create a zero-copy NumPy view of a host buffer.

```python
host_np = np.from_dlpack(host).view(np.float32)   # writable view, zero-copy
host_np[:] = np.arange(n, dtype=np.float32)
```

--------------------------------

### Primitive Interoperability in Structure Members

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/release/11.6.0-notes.rst

Demonstrates how structure members accept both raw values and wrapped primitive classes.

```python
>>> waitParams = cuda.CUstreamMemOpWaitValueParams_st()
>>> waitParams.value64 = 1
>>> waitParams.value64
<cuuint64_t 1>
>>> waitParams.value64 = cuda.cuuint64_t(2)
>>> waitParams.value64
<cuuint64_t 2>
```

--------------------------------

### Measure Execution Time with Events

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Use timing-enabled events to measure the duration of kernel launches.

```python
start = dev.create_event({"timing_enabled": True})
end = dev.create_event({"timing_enabled": True})

stream.record(start)
for _ in range(100):
    launch(stream, config, scale, dbuf, np.float32(1.0), np.uint64(n))
stream.record(end)
end.sync()

print(f"100 launches took {end - start:.4f} ms")
```

--------------------------------

### Import cuda.core modules

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Standard imports required for device management, kernel compilation, and launching.

```python
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch
```

--------------------------------

### Manage Multiple Streams

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Create independent streams to allow concurrent GPU work.

```python
stream_a = dev.create_stream()
stream_b = dev.create_stream()

buf_a = dev.allocate(nbytes, stream=stream_a)
buf_b = dev.allocate(nbytes, stream=stream_b)
```

--------------------------------

### load_nvidia_dynamic_lib

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/api.rst

Loads a specified NVIDIA dynamic library.

```APIDOC
## load_nvidia_dynamic_lib(libname)

### Description
Attempts to load a dynamic library from the supported NVIDIA library list.
```

--------------------------------

### Configure Per-Thread Default Stream

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/release/11.6.0-notes.rst

Set the environment variable to toggle between legacy and per-thread default stream behavior.

```shell
export CUDA_PYTHON_CUDA_PER_THREAD_DEFAULT_STREAM=1
```

--------------------------------

### Compile CUDA kernel with Program

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/getting-started.rst

Compile the CUDA C++ kernel into a CUBIN using Program options and specify kernel template instantiations.

```python
program_options = ProgramOptions(std="c++17", arch=f"sm_{dev.arch}")
prog = Program(code, code_type="c++", options=program_options)
mod = prog.compile("cubin", name_expressions=("vector_add<float>",))
```

--------------------------------

### find_nvidia_binary_utility

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/api.rst

Locates a specific CUDA binary utility.

```APIDOC
## find_nvidia_binary_utility(utility_name)

### Description
Finds the path to a supported CUDA binary utility.
```

--------------------------------

### Perform array scaling with cuda.core

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Demonstrates scaling a buffer using a CUDA stream and verifying the result.

```python
# Works on a CuPy array, a PyTorch tensor, or anything else with DLPack/CAI:
buf = cp.ones(n, dtype=cp.float32)
dev.sync()
scale_array(buf, stream, scale, 2.0)
assert bool((buf == 2).all())
```

--------------------------------

### Allocate Device Memory and Transfer Data

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Allocates device memory and performs asynchronous host-to-device data transfers using streams.

```python
dXclass = checkCudaErrors(driver.cuMemAlloc(bufferSize))
dYclass = checkCudaErrors(driver.cuMemAlloc(bufferSize))
dOutclass = checkCudaErrors(driver.cuMemAlloc(bufferSize))

stream = checkCudaErrors(driver.cuStreamCreate(0))

checkCudaErrors(driver.cuMemcpyHtoDAsync(
   dXclass, hX.ctypes.data, bufferSize, stream
))
checkCudaErrors(driver.cuMemcpyHtoDAsync(
   dYclass, hY.ctypes.data, bufferSize, stream
))
```

--------------------------------

### Manage Asynchronous Stream Operations

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Demonstrates using multiple streams to overlap independent GPU work and synchronizing them with wait operations.

```python
host_np[:] = np.arange(n, dtype=np.float32)   # known input
buf_a.copy_from(host, stream=stream_a)
buf_b.copy_from(host, stream=stream_b)

# Independent work on each stream: free to run concurrently.
launch(stream_a, config, scale, buf_a, np.float32(2.0), np.uint64(n))
launch(stream_b, config, scale, buf_b, np.float32(5.0), np.uint64(n))

# stream_b now needs stream_a's result, so it waits before reading it.
stream_b.wait(stream_a)
buf_b.copy_from(buf_a, stream=stream_b)   # safe: buf_a is ready

buf_b.copy_to(host, stream=stream_b)
stream_b.sync()
assert np.array_equal(host_np, np.arange(n, dtype=np.float32) * 2.0)
```

--------------------------------

### Library Management

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Functions for loading, unloading, and querying libraries and kernels.

```APIDOC
### Library Management

- `cudaLibraryLoadData`: Loads library data.
- `cudaLibraryLoadFromFile`: Loads a library from a file.
- `cudaLibraryUnload`: Unloads a library.
- `cudaLibraryGetKernel`: Retrieves a kernel from a library.
- `cudaLibraryGetGlobal`: Retrieves a global symbol from a library.
```

--------------------------------

### Driver Entry Point Access

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Functions to access driver entry points from the CUDA runtime.

```APIDOC
### Driver Entry Point Access

- `cudaGetDriverEntryPoint`: Retrieves a driver entry point.
- `cudaGetDriverEntryPointByVersion`: Retrieves a driver entry point by version.
```

--------------------------------

### cudaDeviceGetDevResource

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Queries device resource information.

```APIDOC
## cudaDeviceGetDevResource

### Description
Queries the device for resource information, such as the maximum concurrency limit for workqueues.

### Signature
`cudaDeviceGetDevResource(device, resourceType, ...)`
```

--------------------------------

### Perform CUDA process checkpointing

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/api.rst

Demonstrates the coordinator-side sequence for locking and checkpointing a target CUDA process by PID.

```python
import os

from cuda.core import checkpoint

target_pid = os.getpid()  # or the PID of another CUDA process
process = checkpoint.Process(target_pid)
process.lock(timeout_ms=5000)
process.checkpoint()
```

--------------------------------

### External Resource Interoperability

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Functions for importing and managing external memory and semaphores.

```APIDOC
## External Resource Interoperability Functions

- `cuImportExternalMemory`
- `cuExternalMemoryGetMappedBuffer`
- `cuExternalMemoryGetMappedMipmappedArray`
- `cuDestroyExternalMemory`
- `cuImportExternalSemaphore`
- `cuSignalExternalSemaphoresAsync`
- `cuWaitExternalSemaphoresAsync`
- `cuDestroyExternalSemaphore`
```

--------------------------------

### cudaDeviceGetExecutionCtx

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Retrieves the primary execution context for a device.

```APIDOC
## cudaDeviceGetExecutionCtx

### Description
Returns the execution context that corresponds to the primary context of the specified device.

### Signature
`cudaDeviceGetExecutionCtx(device)`
```

--------------------------------

### Primitive Interoperability in Function Signatures

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/release/11.6.0-notes.rst

Shows that function arguments accept both raw values and their corresponding type-wrapped instances.

```python
>>> cudart.cudaStreamQuery(cudart.cudaStreamNonBlocking)
(<cudaError_t.cudaSuccess: 0>,)
>>> cudart.cudaStreamQuery(cudart.cudaStream_t(cudart.cudaStreamNonBlocking))
(<cudaError_t.cudaSuccess: 0>,)
```

--------------------------------

### CUjit_option Enumeration

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

A collection of configuration options for the JIT compiler and linker in the CUDA driver.

```APIDOC
## CUjit_option

### Description
Options used to configure the behavior of the CUDA JIT compiler and linker.

### Attributes
- **CU_JIT_OPTIMIZATION_LEVEL** (unsigned int) - Level of optimizations (0-4, default 4).
- **CU_JIT_TARGET_FROM_CUCONTEXT** (void) - Determines target based on current attached context.
- **CU_JIT_TARGET** (unsigned int) - Target specified by CUjit_target.
- **CU_JIT_FALLBACK_STRATEGY** (unsigned int) - Fallback strategy if cubin is not found.
- **CU_JIT_GENERATE_DEBUG_INFO** (int) - Whether to create debug information (0: false).
- **CU_JIT_LOG_VERBOSE** (int) - Generate verbose log messages (0: false).
- **CU_JIT_GENERATE_LINE_INFO** (int) - Generate line number information (0: false).
- **CU_JIT_CACHE_MODE** (unsigned int) - Cache mode configuration based on CUjit_cacheMode_enum.
- **CU_JIT_GLOBAL_SYMBOL_NAMES** (const char **) - Array of device symbol names for relocation.
- **CU_JIT_GLOBAL_SYMBOL_ADDRESSES** (void **) - Array of host addresses for symbol relocation.
- **CU_JIT_GLOBAL_SYMBOL_COUNT** (unsigned int) - Number of entries in symbol arrays.
```

--------------------------------

### cuda.bindings.cufile API Functions

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/cufile.rst

The module provides a comprehensive set of functions to manage GPUDirect Storage operations, including driver control, buffer registration, and I/O execution.

```APIDOC
## Function Reference

### Driver Management
- `driver_open()`: Opens the cuFile driver.
- `driver_close()`: Closes the cuFile driver.
- `driver_get_properties()`: Retrieves driver properties.
- `driver_set_poll_mode(mode)`: Configures driver polling mode.
- `driver_set_max_direct_io_size(size)`: Sets the maximum direct I/O size.
- `driver_set_max_cache_size(size)`: Sets the maximum cache size.
- `driver_set_max_pinned_mem_size(size)`: Sets the maximum pinned memory size.

### Handle and Buffer Registration
- `handle_register(fd)`: Registers a file descriptor.
- `handle_deregister(handle)`: Deregisters a file handle.
- `buf_register(ptr, size)`: Registers a memory buffer for I/O.
- `buf_deregister(ptr)`: Deregisters a memory buffer.

### I/O Operations
- `read(handle, buf, size, offset)`: Performs a synchronous read.
- `write(handle, buf, size, offset)`: Performs a synchronous write.
- `read_async(handle, buf, size, offset, stream)`: Performs an asynchronous read.
- `write_async(handle, buf, size, offset, stream)`: Performs an asynchronous write.

### Batch I/O
- `batch_io_set_up(count)`: Sets up batch I/O operations.
- `batch_io_submit(batch_id)`: Submits a batch of I/O operations.
- `batch_io_get_status(batch_id)`: Retrieves the status of a batch.
- `batch_io_cancel(batch_id)`: Cancels a batch operation.
- `batch_io_destroy(batch_id)`: Destroys a batch context.
```

--------------------------------

### Device Management

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Functions for querying and managing CUDA devices.

```APIDOC
## Device Management Functions

- **cuDeviceGet**: Returns a handle to a compute device.
- **cuDeviceGetCount**: Returns the number of compute devices.
- **cuDeviceGetName**: Returns an identifier string for the device.
- **cuDeviceGetUuid**: Returns the UUID of the device.
- **cuDeviceGetLuid**: Returns the LUID of the device.
- **cuDeviceTotalMem**: Returns the total amount of memory on the device.
- **cuDeviceGetAttribute**: Returns information about the device.
- **cuDeviceSetMemPool**: Sets the current memory pool for the device.
- **cuDeviceGetMemPool**: Returns the current memory pool for the device.
- **cuDeviceGetDefaultMemPool**: Returns the default memory pool for the device.
```

--------------------------------

### Version Management

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Functions to retrieve version information for the CUDA driver and runtime.

```APIDOC
## cudaDriverGetVersion
## cudaRuntimeGetVersion
## getLocalRuntimeVersion

### Description
These functions provide access to the version numbers of the installed CUDA driver and the linked CUDA runtime.
```

--------------------------------

### Importing cuda.bindings and NumPy

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Initial imports required to access the Driver API, NVRTC, and NumPy for host data management.

```python
from cuda.bindings import driver, nvrtc
import numpy as np
```

--------------------------------

### Allocate Host and Device Memory

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Allocate page-locked host memory and device memory using a stream.

```python
import numpy as np

from cuda.core import PinnedMemoryResource

n = 1024
nbytes = n * np.dtype(np.float32).itemsize

pinned = PinnedMemoryResource()
host = pinned.allocate(nbytes, stream=stream)   # host-accessible buffer
dbuf = dev.allocate(nbytes, stream=stream)      # device buffer

print(host.size, host.is_host_accessible)    # 4096 True
print(dbuf.is_device_accessible)             # True
```

--------------------------------

### Retrieve Device Pointers

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Converts device memory handles into integer pointers for kernel argument preparation.

```python
dX = np.array([int(dXclass)], dtype=np.uint64)
dY = np.array([int(dYclass)], dtype=np.uint64)
dOut = np.array([int(dOutclass)], dtype=np.uint64)
```

--------------------------------

### Load PTX Module and Function

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Loads a PTX module into the driver and retrieves a specific function handle.

```python
# Load PTX as module data and retrieve function
ptx = np.char.array(ptx)
# Note: Incompatible --gpu-architecture would be detected here
module = checkCudaErrors(driver.cuModuleLoadData(ptx.ctypes.data))
kernel = checkCudaErrors(driver.cuModuleGetFunction(module, b"saxpy"))
```

--------------------------------

### nvfatbin API Functions

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/nvfatbin.rst

The nvfatbin module exposes several functions for fatbinary manipulation. Note that add_tile_ir is only available in CUDA 13.1+.

```APIDOC
## Functions

- **cuda.bindings.nvfatbin.create()**: Creates a new fatbinary handle.
- **cuda.bindings.nvfatbin.destroy(handle)**: Destroys the specified fatbinary handle.
- **cuda.bindings.nvfatbin.add_ptx(handle, ptx, name)**: Adds PTX code to the fatbinary.
- **cuda.bindings.nvfatbin.add_cubin(handle, cubin, name)**: Adds CUBIN code to the fatbinary.
- **cuda.bindings.nvfatbin.add_ltoir(handle, ltoir, name)**: Adds LTOIR code to the fatbinary.
- **cuda.bindings.nvfatbin.add_reloc(handle, reloc, name)**: Adds relocatable code to the fatbinary.
- **cuda.bindings.nvfatbin.add_tile_ir(handle, tile_ir, name)**: Adds Tile IR code to the fatbinary (CUDA 13.1+).
- **cuda.bindings.nvfatbin.size(handle)**: Returns the size of the fatbinary.
- **cuda.bindings.nvfatbin.get(handle)**: Retrieves the fatbinary data.
- **cuda.bindings.nvfatbin.get_error_string(result)**: Returns a string representation of a Result error code.
- **cuda.bindings.nvfatbin.version()**: Returns the version of the nvfatbin library.
```

--------------------------------

### Create CUDA Context

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Creates a CUDA context on the specified device, which is required before executing any GPU operations.

```python
# Create context
context = checkCudaErrors(driver.cuCtxCreate(0, cuDevice))
```

--------------------------------

### cuda.bindings.runtime.cudaDeviceAttr

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

A collection of attributes representing hardware and runtime capabilities of a CUDA device.

```APIDOC
## cuda.bindings.runtime.cudaDeviceAttr

### Description
This class provides constants representing various device attributes that can be queried to understand the specific capabilities of a GPU device.

### Attributes
- **cudaDevAttrComputeCapabilityMinor** - Minor compute capability version number
- **cudaDevAttrMaxTexture1DMipmappedWidth** - Maximum mipmapped 1D texture width
- **cudaDevAttrStreamPrioritiesSupported** - Device supports stream priorities
- **cudaDevAttrGlobalL1CacheSupported** - Device supports caching globals in L1
- **cudaDevAttrLocalL1CacheSupported** - Device supports caching locals in L1
- **cudaDevAttrMaxSharedMemoryPerMultiprocessor** - Maximum shared memory available per multiprocessor in bytes
- **cudaDevAttrMaxRegistersPerMultiprocessor** - Maximum number of 32-bit registers available per multiprocessor
- **cudaDevAttrManagedMemory** - Device can allocate managed memory on this system
- **cudaDevAttrIsMultiGpuBoard** - Device is on a multi-GPU board
- **cudaDevAttrMultiGpuBoardGroupID** - Unique identifier for a group of devices on the same multi-GPU board
- **cudaDevAttrHostNativeAtomicSupported** - Link between the device and the host supports native atomic operations
- **cudaDevAttrSingleToDoublePrecisionPerfRatio** - Ratio of single precision performance to double precision performance
- **cudaDevAttrPageableMemoryAccess** - Device supports coherently accessing pageable memory
- **cudaDevAttrConcurrentManagedAccess** - Device can coherently access managed memory concurrently with the CPU
- **cudaDevAttrComputePreemptionSupported** - Device supports Compute Preemption
- **cudaDevAttrCanUseHostPointerForRegisteredMem** - Device can access host registered memory at the same virtual address as the CPU
- **cudaDevAttrCooperativeLaunch** - Device supports launching cooperative kernels
- **cudaDevAttrMaxSharedMemoryPerBlockOptin** - The maximum optin shared memory per block
- **cudaDevAttrCanFlushRemoteWrites** - Device supports flushing of outstanding remote writes
- **cudaDevAttrHostRegisterSupported** - Device supports host memory registration
- **cudaDevAttrPageableMemoryAccessUsesHostPageTables** - Device accesses pageable memory via the host's page tables
- **cudaDevAttrDirectManagedMemAccessFromHost** - Host can directly access managed memory on the device without migration
- **cudaDevAttrMaxBlocksPerMultiprocessor** - Maximum number of blocks per multiprocessor
- **cudaDevAttrMaxPersistingL2CacheSize** - Maximum L2 persisting lines capacity setting in bytes
- **cudaDevAttrMaxAccessPolicyWindowSize** - Maximum value of cudaAccessPolicyWindow.num_bytes
- **cudaDevAttrReservedSharedMemoryPerBlock** - Shared memory reserved by CUDA driver per block in bytes
- **cudaDevAttrSparseCudaArraySupported** - Device supports sparse CUDA arrays and sparse CUDA mipmapped arrays
```

--------------------------------

### find_nvidia_header_directory

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_pathfinder/docs/source/api.rst

Locates the directory containing NVIDIA C/C++ headers.

```APIDOC
## find_nvidia_header_directory(header_name)

### Description
Searches for the specified NVIDIA header directory within the system paths.
```

--------------------------------

### cuda.bindings.runtime.cudaDeviceAttr

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

A collection of attributes representing hardware specifications and capabilities of a CUDA device.

```APIDOC
## cuda.bindings.runtime.cudaDeviceAttr

### Description
This enumeration provides access to various hardware attributes of a CUDA device, such as memory limits, grid/block dimensions, and architectural features.

### Attributes
- **cudaDevAttrMaxBlockDimZ** - Maximum block dimension Z
- **cudaDevAttrMaxGridDimX** - Maximum grid dimension X
- **cudaDevAttrMaxGridDimY** - Maximum grid dimension Y
- **cudaDevAttrMaxGridDimZ** - Maximum grid dimension Z
- **cudaDevAttrMaxSharedMemoryPerBlock** - Maximum shared memory available per block in bytes
- **cudaDevAttrTotalConstantMemory** - Memory available on device for constant variables in a CUDA C kernel in bytes
- **cudaDevAttrWarpSize** - Warp size in threads
- **cudaDevAttrMaxPitch** - Maximum pitch in bytes allowed by memory copies
- **cudaDevAttrMaxRegistersPerBlock** - Maximum number of 32-bit registers available per block
- **cudaDevAttrClockRate** - Peak clock frequency in kilohertz
- **cudaDevAttrTextureAlignment** - Alignment requirement for textures
- **cudaDevAttrGpuOverlap** - Device can possibly copy memory and execute a kernel concurrently
- **cudaDevAttrMultiProcessorCount** - Number of multiprocessors on device
- **cudaDevAttrKernelExecTimeout** - Specifies whether there is a run time limit on kernels
- **cudaDevAttrIntegrated** - Device is integrated with host memory
- **cudaDevAttrCanMapHostMemory** - Device can map host memory into CUDA address space
- **cudaDevAttrComputeMode** - Compute mode
- **cudaDevAttrMaxTexture1DWidth** - Maximum 1D texture width
- **cudaDevAttrMaxTexture2DWidth** - Maximum 2D texture width
- **cudaDevAttrMaxTexture2DHeight** - Maximum 2D texture height
- **cudaDevAttrMaxTexture3DWidth** - Maximum 3D texture width
- **cudaDevAttrMaxTexture3DHeight** - Maximum 3D texture height
- **cudaDevAttrMaxTexture3DDepth** - Maximum 3D texture depth
- **cudaDevAttrMaxTexture2DLayeredWidth** - Maximum 2D layered texture width
- **cudaDevAttrMaxTexture2DLayeredHeight** - Maximum 2D layered texture height
- **cudaDevAttrMaxTexture2DLayeredLayers** - Maximum layers in a 2D layered texture
- **cudaDevAttrSurfaceAlignment** - Alignment requirement for surfaces
- **cudaDevAttrConcurrentKernels** - Device can possibly execute multiple kernels concurrently
- **cudaDevAttrEccEnabled** - Device has ECC support enabled
- **cudaDevAttrPciBusId** - PCI bus ID of the device
- **cudaDevAttrPciDeviceId** - PCI device ID of the device
- **cudaDevAttrTccDriver** - Device is using TCC driver model
- **cudaDevAttrMemoryClockRate** - Peak memory clock frequency in kilohertz
- **cudaDevAttrGlobalMemoryBusWidth** - Global memory bus width in bits
- **cudaDevAttrL2CacheSize** - Size of L2 cache in bytes
- **cudaDevAttrMaxThreadsPerMultiProcessor** - Maximum resident threads per multiprocessor
```

--------------------------------

### nvjitlink Functions

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/nvjitlink.rst

The nvjitlink module exposes several functions for managing the linking process, including creating and destroying linkers, adding input data or files, completing the link, and retrieving results or logs.

```APIDOC
## nvjitlink Functions

### Overview
The following functions are available in `cuda.bindings.nvjitlink`:

- `create()`: Initializes a new linker instance.
- `destroy(handle)`: Destroys a linker instance.
- `add_data(handle, input_type, data, name)`: Adds data to the linker.
- `add_file(handle, input_type, file_path)`: Adds a file to the linker.
- `complete(handle)`: Completes the linking process.
- `get_linked_cubin_size(handle)`, `get_linked_cubin(handle)`: Retrieve linked CUBIN data.
- `get_linked_ptx_size(handle)`, `get_linked_ptx(handle)`: Retrieve linked PTX data.
- `get_linked_ltoir_size(handle)`, `get_linked_ltoir(handle)`: Retrieve linked LTOIR data.
- `get_error_log_size(handle)`, `get_error_log(handle)`: Retrieve error logs.
- `get_info_log_size(handle)`, `get_info_log(handle)`: Retrieve info logs.
- `version()`: Returns the nvJitLink version.
```

--------------------------------

### Prepare kernel arguments with NumPy

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Converts CUDA types to NumPy integer arrays for kernel parameter passing.

```python
kernelValues = (
    np.array([d_data], dtype=np.intp),
    np.array(width, dtype=np.uint32),
    np.array([int(tex)], dtype=np.intp),
)
kernelArgs = np.array([arg.ctypes.data for arg in kernelValues], dtype=np.intp)
```

--------------------------------

### Prepare kernel arguments with ctypes

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/overview.rst

Uses None type for automatic pointer extraction from Python classes implementing getPtr().

```python
kernelValues = (
    d_data,
    width,
    tex,
)
kernelTypes = (
    ctypes.c_void_p,
    ctypes.c_int,
    None,
)
kernelArgs = (kernelValues, kernelTypes)
```

--------------------------------

### Interoperate with CuPy arrays

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Access CuPy device pointers via .data.ptr and ensure synchronization before reading data.

```python
# CuPy exposes its device pointer as .data.ptr
import cupy as cp

a = cp.ones(n, dtype=cp.float32)
dev.sync()  # CuPy fills "a" on its own stream; sync before our stream reads it
launch(stream, config, scale, a.data.ptr, np.float32(2.0), np.uint64(a.size))
stream.sync()
assert bool((a == 2).all())
```

--------------------------------

### register_events

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/api_nvml.rst

Registers for system events.

```APIDOC
## register_events()

### Description
Registers the application to receive notifications for specific system events.
```

--------------------------------

### Memory Management Functions

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Functions for allocating, freeing, and managing device and host memory.

```APIDOC
## Memory Allocation and Deallocation

- `cuda.bindings.runtime.cudaMalloc`
- `cuda.bindings.runtime.cudaMallocManaged`
- `cuda.bindings.runtime.cudaMallocHost`
- `cuda.bindings.runtime.cudaMallocPitch`
- `cuda.bindings.runtime.cudaMallocArray`
- `cuda.bindings.runtime.cudaMalloc3D`
- `cuda.bindings.runtime.cudaMalloc3DArray`
- `cuda.bindings.runtime.cudaMallocMipmappedArray`
- `cuda.bindings.runtime.cudaFree`
- `cuda.bindings.runtime.cudaFreeHost`
- `cuda.bindings.runtime.cudaFreeArray`
- `cuda.bindings.runtime.cudaFreeMipmappedArray`
```

--------------------------------

### Compile a CUDA kernel

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Define CUDA C++ source code and compile it into a callable kernel object at runtime.

```python
code = r"""
extern "C" __global__
void scale(float* data, float factor, size_t n) {
    size_t i = threadIdx.x + blockIdx.x * blockDim.x;
    if (i < n)
        data[i] *= factor;
}
"""

opts = ProgramOptions(std="c++17", arch=f"sm_{dev.arch}")
prog = Program(code, code_type="c++", options=opts)
mod = prog.compile("cubin")          # "cubin" | "ptx" | "ltoir"
scale = mod.get_kernel("scale")
```

--------------------------------

### get_minimal_required_cuda_ver_from_ptx_ver

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/utils.rst

Determines the minimum required CUDA version based on a specified PTX version.

```APIDOC
## get_minimal_required_cuda_ver_from_ptx_ver(ptx_ver)

### Description
Calculates the minimum CUDA version required to support a specific PTX version.

### Parameters
- **ptx_ver** (int/float) - The PTX version to check.
```

--------------------------------

### get_ptx_ver

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/utils.rst

Retrieves the current PTX version information.

```APIDOC
## get_ptx_ver()

### Description
Returns the PTX version currently in use or supported by the environment.
```

--------------------------------

### Create array-library-agnostic views

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_core/docs/source/10_minutes_to_cuda_core.rst

Use the args_viewable_as_strided_memory decorator to handle various array types that support DLPack or the CUDA Array Interface.

```python
from cuda.core.utils import StridedMemoryView, args_viewable_as_strided_memory

@args_viewable_as_strided_memory((0,))
def scale_array(arr, work_stream, kern, factor):
    view = arr.view(work_stream.handle)
    assert isinstance(view, StridedMemoryView)
    assert view.is_device_accessible
    size = view.shape[0]
    cfg = LaunchConfig(grid=(size + 255) // 256, block=256)
    launch(work_stream, cfg, kern, view.ptr, np.float32(factor), np.uint64(size))
    work_stream.sync()
```

--------------------------------

### Version Management

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Function to retrieve the CUDA driver version.

```APIDOC
## cuda.bindings.driver.cuDriverGetVersion

### Description
Returns the version of the installed CUDA driver.
```

--------------------------------

### Context Management

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/driver.rst

Functions for managing CUDA contexts, including primary context operations.

```APIDOC
## Context Management Functions

- **cuCtxCreate**: Creates a new CUDA context.
- **cuCtxDestroy**: Destroys a CUDA context.
- **cuCtxPushCurrent**: Pushes a context to the current thread.
- **cuCtxPopCurrent**: Pops the current context from the thread.
- **cuCtxSetCurrent**: Sets the current context.
- **cuCtxGetCurrent**: Returns the current context.
- **cuCtxSynchronize**: Blocks until the device has completed all preceding tasks.
- **cuDevicePrimaryCtxRetain**: Retains the primary context for a device.
- **cuDevicePrimaryCtxRelease**: Releases the primary context for a device.
```

--------------------------------

### cuda.bindings.runtime.cudaMallocAsync

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Allocates memory asynchronously on the device.

```APIDOC
## cuda.bindings.runtime.cudaMallocAsync

### Description
Allocates memory asynchronously on the device using the specified stream.

### Method
Function Call

### Parameters
- **devPtr** (void**) - Pointer to the allocated memory
- **size** (size_t) - Size of the allocation
- **hStream** (cudaStream_t) - Stream to perform the allocation on
```

--------------------------------

### Memory Copy and Set Operations

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Functions for performing memory copies and setting memory values on device and host.

```APIDOC
## Memory Operations

- `cuda.bindings.runtime.cudaMemcpy`
- `cuda.bindings.runtime.cudaMemcpyAsync`
- `cuda.bindings.runtime.cudaMemcpy2D`
- `cuda.bindings.runtime.cudaMemcpy3D`
- `cuda.bindings.runtime.cudaMemset`
- `cuda.bindings.runtime.cudaMemsetAsync`
- `cuda.bindings.runtime.cudaMemset2D`
- `cuda.bindings.runtime.cudaMemset3D`
```

--------------------------------

### cuda.bindings.runtime.cudaGetKernel

Source: https://github.com/nvidia/cuda-python/blob/main/cuda_bindings/docs/source/module/runtime.rst

Retrieves a kernel handle from the CUDA runtime.

```APIDOC
## cuda.bindings.runtime.cudaGetKernel

### Description
Retrieves a kernel handle from the CUDA runtime for use in subsequent operations.
```
Tokens:
Raw