NVIDIA cuSPARSELt

https://docs.nvidia.com/cuda/cusparselt/
Admin
NVIDIA cuSPARSELt is a high-performance CUDA library for sparse matrix-matrix multiplication...
Tokens:21,219
Snippets:185
Trust Score:10
Update:5 months ago
Show doc for...
Tokens:
Raw
### cuSPARSELt Matrix Multiplication Workflow

Source: https://docs.nvidia.com/cuda/cusparselt/getting_started.html

Demonstrates the essential steps for performing sparse matrix multiplication using cuSPARSELt, including initialization, descriptor setup, plan creation, pruning, compression, and execution.

```c++
#include <cusparseLt.h> // cusparseLt header

// Device pointers and coefficient definitions
float alpha = 1.0f;
float beta = 0.0f;
__half* dA = ...
__half* dB = ...
__half* dC = ...

//--------------------------------------------------------------------------
// cusparseLt data structures and handle initialization
cusparseLtHandle_t handle;
cusparseLtMatDescriptor_t matA, matB, matC;
cusparseLtMatmulDescriptor_t matmul;
cusparseLtMatmulAlgSelection_t alg_sel;
cusparseLtMatmulPlan_t plan;
cudaStream_t stream = nullptr;
cusparseLtInit(&handle);

//--------------------------------------------------------------------------
// matrix descriptor initialization
cusparseLtStructuredDescriptorInit(&handle, &matA, num_A_rows, num_A_cols,
 lda, alignment, type, order,
 CUSPARSELT_SPARSITY_50_PERCENT);
cusparseLtDenseDescriptorInit(&handle, &matB, num_B_rows, num_B_cols, ldb,
 alignment, type, order);
cusparseLtDenseDescriptorInit(&handle, &matC, num_C_rows, num_C_cols, ldc,
 alignment, type, order);

//--------------------------------------------------------------------------
// matmul, algorithm selection, and plan initialization
cusparseLtMatmulDescriptorInit(&handle, &matmul, opA, opB, &matA, &matB,
 &matC, &matC, compute_type);
cusparseLtMatmulAlgSelectionInit(&handle, &alg_sel, &matmul,
 CUSPARSELT_MATMUL_ALG_DEFAULT);
cusparseLtMatmulPlanInit(&handle, &plan, &matmul, &alg_sel);

//--------------------------------------------------------------------------
// Prune the A matrix (in-place) and check the correctness
cusparseLtSpMMAPrune(&handle, &matmul, dA, dA, CUSPARSELT_PRUNE_SPMMA_TILE, 
 stream);
int *d_valid;
cudaMalloc((void**) &d_valid, sizeof(d_valid));
cusparseLtSpMMAPruneCheck(&handle, &matmul, dA, &d_valid, stream);

int is_valid;
cudaMemcpyAsync(&is_valid, d_valid, sizeof(d_valid), cudaMemcpyDeviceToHost, 
 stream);
cudaStreamSynchronize(stream);
if (is_valid != 0) {
 std::printf("!!!! The matrix has been pruned in a wrong way. "
 "cusparseLtMatmul will not provided correct results\n");
 return EXIT_FAILURE;
}

//--------------------------------------------------------------------------
// Matrix A compression
size_t compressed_size;
cusparseLtSpMMACompressedSize(&handle, &plan, &compressed_size);
cudaMalloc((void**) &dA_compressed, compressed_size);

cusparseLtSpMMACompress(&handle, &plan, dA, dA_compressed, stream);

//--------------------------------------------------------------------------
// Allocate workspace
size_t workspace_size;
void* d_workspace = nullptr;

cusparseLtMatmulGetWorkspace(&handle, &plan, &workspace_size);
cudaMalloc((void**) &d_workspace, workspace_size);

//--------------------------------------------------------------------------
// Perform the matrix multiplication
int num_streams = 0;
cudaStream_t* streams = nullptr;

cusparseLtMatmul(&handle, &plan, &alpha, dA_compressed, dB, &beta, dC, dD,
 d_workspace, streams, num_streams);

//--------------------------------------------------------------------------
// Destroy descriptors, plan and handle
cusparseLtMatDescriptorDestroy(&matA);
cusparseLtMatDescriptorDestroy(&matB);
cusparseLtMatDescriptorDestroy(&matC);
cusparseLtMatmulPlanDestroy(&plan);
cusparseLtMatmulAlgSelectionDestroy(&alg_sel);
cusparseLtDestroy(&handle);

```

--------------------------------

### Compile cuSPARSELt Sample Code (Windows)

Source: https://docs.nvidia.com/cuda/cusparselt/getting_started.html

Compiles a sample C++ file using nvcc, linking against the cuSPARSELt shared library. Ensure CUSPARSELT_DIR is set.

```bash
setx PATH "%CUSPARSELT_DIR%\lib:%PATH%"
nvcc.exe matmul_example.cpp -I "%CUSPARSELT_DIR%\include" -lcusparseLt -lcuda -o matmul.exe
```

--------------------------------

### Compile cuSPARSELt Sample Code with Static Library (Windows)

Source: https://docs.nvidia.com/cuda/cusparselt/getting_started.html

Compiles a sample C++ file using nvcc, linking against the cuSPARSELt static library. This requires specific linker flags.

```bash
nvcc.exe matmul_example.cpp -I %CUSPARSELT_DIR%\include \
 -Xlinker=/WHOLEARCHIVE:"%CUSPARSELT_DIR%\lib\cusparseLt_static.lib" \
 -Xlinker=/FORCE -lcuda -o matmul.exe
```

--------------------------------

### cusparseLtMatmulPlanInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes a matrix multiplication plan with the given handle, descriptor, and algorithm selection.

```APIDOC
## cusparseLtMatmulPlanInit

### Description
Initializes a matrix multiplication plan.

### Signature
cusparseStatus_t
cusparseLtMatmulPlanInit(const cusparseLtHandle_t* handle, cusparseLtMatmulPlan_t* plan, const cusparseLtMatmulDescriptor_t* matmulDescr, const cusparseLtMatmulAlgSelection_t* algSelection)

### Parameters
#### In Parameters
- **handle** (const cusparseLtHandle_t*) - cuSPARSELt library handle.
- **matmulDescr** (const cusparseLtMatmulDescriptor_t*) - Matrix multiplication descriptor.
- **algSelection** (const cusparseLtMatmulAlgSelection_t*) - Algorithm selection descriptor.

#### Out Parameters
- **plan** (cusparseLtMatmulPlan_t*) - Matrix multiplication plan.

### Return Value
See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.
```

--------------------------------

### Compile cuSPARSELt Sample with Static Library

Source: https://docs.nvidia.com/cuda/cusparselt/getting_started.html

Compile a C++ sample using nvcc, linking against the static version of the cuSPARSELt library. This requires specifying the static library file path directly using the -Xlinker flag.

```bash
nvcc matmul_example.cpp -I${CUSPARSELT_DIR}/include \
 -Xlinker=${CUSPARSELT_DIR}/lib64/libcusparseLt_static.a \
 -o matmul_static -ldl -lcuda
```

--------------------------------

### Compile cuSPARSELt Sample with Shared Library

Source: https://docs.nvidia.com/cuda/cusparselt/getting_started.html

Compile a C++ sample using nvcc, linking against the cuSPARSELt shared library. Ensure the include and library paths are correctly specified.

```bash
nvcc matmul_example.cpp -I {CUSPARSELT_PATH}/include/ -l cusparse -l cusparseLt -L ${CUSPARSELT_PATH}/lib/
```

--------------------------------

### cusparseLtMatmulPlanInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes a plan for sparse matrix-matrix multiplication, configuring the operation based on input matrices and desired properties.

```APIDOC
## cusparseLtMatmulPlanInit

### Description
Initializes and configures a plan for sparse matrix-matrix multiplication (SpMMA). This function sets up the necessary structures and parameters for an efficient SpMMA operation based on the properties of the input matrices and the desired output format.

### Usage
Call this function to create a SpMMA plan before performing matrix multiplication. The plan guides the subsequent search and execution phases.
```

--------------------------------

### cusparseLtMatmulAlgSelectionInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes the algorithm selection descriptor for matrix multiplication.

```APIDOC
## cusparseLtMatmulAlgSelectionInit

### Description
The function initializes the _algorithm selection_ descriptor.

### Parameters
#### Path Parameters
- **handle** (const cusparseLtHandle_t*) - IN - cuSPARSELt library handle
- **algSelection** (cusparseLtMatmulAlgSelection_t*) - OUT - Algorithm selection descriptor
- **matmulDescr** (const cusparseLtMatmulDescriptor_t*) - IN - Matrix multiplication descriptor
- **alg** (cusparseLtMatmulAlg_t) - IN - Algorithm mode. Possible value: `CUSPARSELT_MATMUL_ALG_DEFAULT`.

### Return Value
cusparseStatus_t - See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.
```

--------------------------------

### cusparseLtInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes the cuSPARSELt library handle, allocating necessary resources. This must be called before any other cuSPARSELt functions.

```APIDOC
## `cusparseLtInit`

### Description
The function initializes the cuSPARSELt library handle (`cusparseLtHandle_t`) which holds the cuSPARSELt library context. It allocates light hardware resources on the host, and must be called prior to making any other cuSPARSELt library calls. Calling any cusparseLt function which uses `cusparseLtHandle_t` without a previous call of `cusparseLtInit()` will return an error.

The cuSPARSELt library context is tied to the current CUDA device. To use the library on multiple devices, one cuSPARSELt handle should be created for each device.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters Table
| Parameter | Memory | In/Out |
| --- | --- | --- |
| `handle` | Host | OUT |

### Return Value
- `cusparseStatus_t`: Status of the operation.

### See Also
- [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t)
```

--------------------------------

### cusparseLtMatmulDescriptorInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes the matrix multiplication descriptor used for cuSPARSELt operations.

```APIDOC
## cusparseLtMatmulDescriptorInit

### Description
The function initializes the _matrix multiplication_ descriptor.

### Function Signature
```c
cusparseStatus_t
cusparseLtMatmulDescriptorInit(const cusparseLtHandle_t* handle,
 cusparseLtMatmulDescriptor_t* matmulDescr,
 cusparseOperation_t opA,
 cusparseOperation_t opB,
 const cusparseLtMatDescriptor_t* matA,
 const cusparseLtMatDescriptor_t* matB,
 const cusparseLtMatDescriptor_t* matC,
 const cusparseLtMatDescriptor_t* matD,
 cusparseComputeType computeType)
```

### Parameters
#### Input Parameters
- **handle** (const cusparseLtHandle_t*) - Host memory - IN - cuSPARSELt library handle.
- **opA** (cusparseOperation_t) - Host memory - IN - Operation applied to the matrix `A`. Possible Values: `CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`.
- **opB** (cusparseOperation_t) - Host memory - IN - Operation applied to the matrix `B`. Possible Values: `CUSPARSE_OPERATION_NON_TRANSPOSE`, `CUSPARSE_OPERATION_TRANSPOSE`.
- **matA** (const cusparseLtMatDescriptor_t*) - Host memory - IN - Structured or dense matrix descriptor `A`.
- **matB** (const cusparseLtMatDescriptor_t*) - Host memory - IN - Structured or dense matrix descriptor `B`.
- **matC** (const cusparseLtMatDescriptor_t*) - Host memory - IN - Dense matrix descriptor `C`.
- **matD** (const cusparseLtMatDescriptor_t*) - Host memory - IN - Dense matrix descriptor `D`.
- **computeType** (cusparseComputeType) - Host memory - IN - Compute precision. Possible Values: `CUSPARSE_COMPUTE_32I`, `CUSPARSE_COMPUTE_32F`, `CUSPARSE_COMPUTE_16F`.

#### Output Parameters
- **matmulDescr** (cusparseLtMatmulDescriptor_t*) - Host memory - OUT - Matrix multiplication descriptor.

### Notes
- The structured matrix descriptor can be used for `matA` or `matB` but not both.
- See [cusparseLtMatmul()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltmatmul-label) for the supported data types.
- `CUDA_R_8I`, `CUDA_R_8F_E4M3`,`CUDA_R_8F_E5M2` and `CUDA_R_4F_E2M1` data types only support specific operation and order combinations for `opA/opB` and `orderA/orderB`.
- `C` and `D` must have the same leading dimension and memory layout.
- The maximum number of elements for each dimension (rows and columns) of matrices `C` and `D` is limited to 2097120.
- The minimum number of elements for the continuous dimension of matrices `C` and `D` is 64 if input A/B=`E4M3` and output D=`E4M3` with scale=`VEC64_UE8M0`.

### Return Value
See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.
```

--------------------------------

### cusparseLtMatmulSearch

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Searches for the optimal configuration for a sparse matrix-matrix multiplication operation based on an initialized plan.

```APIDOC
## cusparseLtMatmulSearch

### Description
Performs a search to find the most efficient algorithm and configuration for a sparse matrix-matrix multiplication (SpMMA) based on a previously initialized plan. This function optimizes the SpMMA operation for the given hardware and matrix properties.

### Usage
This function is called after `cusparseLtMatmulPlanInit` to determine the best execution strategy for the SpMMA operation.
```

--------------------------------

### cusparseLtStructuredDescriptorInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes the descriptor for a structured sparse matrix, specifying its dimensions, memory layout, data type, and sparsity.

```APIDOC
## cusparseLtStructuredDescriptorInit

### Description
Initializes the descriptor of a _structured_ matrix.

### Parameters
#### Path Parameters
- **handle** (cusparseLtHandle_t*) - IN - cuSPARSELt library handle
- **matDescr** (cusparseLtMatDescriptor_t*) - OUT - Sparse matrix description
- **rows** (int64_t) - IN - Number of rows
- **cols** (int64_t) - IN - Number of columns
- **ld** (int64_t) - IN - Leading dimension (≥ rows if column-major, ≥ cols if row-major)
- **alignment** (uint32_t) - IN - Memory alignment in bytes (Multiple of 16)
- **valueType** (cudaDataType) - IN - Data type of the matrix (`CUDA_R_32F`, `CUDA_R_16F`, `CUDA_R_16BF`, `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`, `CUDA_R_4F_E2M1`)
- **order** (cusparseOrder_t) - IN - Memory layout (`CUSPARSE_ORDER_COL`, `CUSPARSE_ORDER_ROW`)
- **sparsity** (cusparseLtSparsity_t) - IN - Matrix sparsity ratio (`CUSPARSELT_SPARSITY_50_PERCENT`)

### Constraints
- `rows`, `cols`, and `ld` must be a multiple of 32 if `valueType` is `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`, or `CUDA_R_4F_E2M1`
- `rows`, `cols`, and `ld` must be a multiple of 16 if `valueType` is `CUDA_R_16F`, or `CUDA_R_16BF`
- `rows`, `cols`, and `ld` must be a multiple of 8 if `valueType` is `CUDA_R_32F`

### Return Value
- `cusparseStatus_t` - Status of the operation.
```

--------------------------------

### cusparseLtDenseDescriptorInit

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Initializes the descriptor for a dense matrix. This function sets up the necessary metadata for representing a dense matrix within the cuSPARSELt library.

```APIDOC
## cusparseLtDenseDescriptorInit

### Description
The function initializes the descriptor of a _dense_ matrix.

### Parameters
#### Path Parameters
- `handle` (const cusparseLtHandle_t*) - IN - cuSPARSELt library handle
- `matDescr` (cusparseLtMatDescriptor_t*) - OUT - Dense matrix description
- `rows` (int64_t) - IN - Number of rows
- `cols` (int64_t) - IN - Number of columns
- `ld` (int64_t) - IN - Leading dimension (≥ rows if column-major, ≥ cols if row-major)
- `alignment` (uint32_t) - IN - Memory alignment in bytes (Multiple of 16)
- `valueType` (cudaDataType) - IN - Data type of the matrix (e.g., `CUDA_R_32F`, `CUDA_R_16F`)
- `order` (cusparseOrder_t) - IN - Memory layout (`CUSPARSE_ORDER_COL`, `CUSPARSE_ORDER_ROW`)

### Return Value
- `cusparseStatus_t` - Status of the operation. See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.

### Constraints
- `rows`, `cols`, and `ld` must be a multiple of:
  - 16 if `valueType` is `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`, or `CUDA_R_4F_E2M1`
  - 8 if `valueType` is `CUDA_R_16F` or `CUDA_R_16BF`
  - 4 if `valueType` is `CUDA_R_32F`
```

--------------------------------

### cusparseLtMatmulSearch

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Evaluates all available algorithms for a given matrix multiplication plan and selects the fastest one. This function is intended for auto-tuning purposes when the same operation is repeated multiple times.

```APIDOC
## cusparseLtMatmulSearch

### Description
Evaluates all available algorithms for the matrix multiplication described by `plan` and automatically updates the `cusparseLtMatmulAlgSelection_t` used to initialize the `plan` by selecting the fastest one. The functionality is intended to be used for auto-tuning purposes when the same operation is repeated multiple times over different inputs.

The function behaves similarly to [cusparseLtMatmul()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltmatmul-label), with the difference that `d_D` values may accumulate if the operation is performed in-place (`d_C=d_D`).

* The function is _NOT_ asynchronous with respect to `streams[0]` (_blocking call_)
* The number of iterations for the evaluation can be set by using [cusparseLtMatmulAlgSetAttribute()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltmatmulalgsetattribute-label) with [`CUSPARSELT_MATMUL_SEARCH_ITERATIONS`](https://docs.nvidia.com/cuda/cusparselt/types.html.md#cusparseltmatmulalgattribute-t).
* The selected algorithm id can be retrieved by using [cusparseLtMatmulAlgGetAttribute()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltmatmulalggetattribute-label) with [`CUSPARSELT_MATMUL_ALG_CONFIG_ID`](https://docs.nvidia.com/cuda/cusparselt/types.html.md#cusparseltmatmulalgattribute-t).
* The function also searches for optimal combination of [Split-K parameters](https://docs.nvidia.com/cuda/cusparselt/types.html.md#cusparseltmatmulalgattribute-t-label). The selected values can be retrieved by using [cusparseLtMatmulAlgGetAttribute()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltmatmulalggetattribute-label).

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters Table
| Parameter | Memory | In/Out | Description |
| --- | --- | --- | --- |
| [`handle`](https://docs.nvidia.com/cuda/cusparselt/types.html.md#cusparselthandle-t) | Host | IN | cuSPARSELt library handle |
| [`plan`](https://docs.nvidia.com/cuda/cusparselt/types.html.md#cusparseltmatmulplan-t) | Host | OUT | Matrix multiplication plan |
| `alpha` | Host | IN | scalar/vector of scalars used for multiplication (`float` data type) |
| `d_A` | Device | IN | Pointer to the structured or dense matrix `A` |
| `d_B` | Device | IN | Pointer to the structured or dense matrix `B` |
| `beta` | Host | IN | scalar/vector of scalars used for multiplication (`float` data type). It can have a `NULL` value only if `CUSPARSELT_MATMUL_ALPHA_VECTOR_SCALING` is set without `CUSPARSELT_MATMUL_BETA_VECTOR_SCALING` |
| `d_C` | Device | IN | Pointer to the dense matrix `C` |
| `d_D` | Device | OUT | Pointer to the dense matrix `D` |
| `workspace` | Device | IN | Pointer to workspace |
| `streams` | Host | IN | Pointer to CUDA stream array for the computation |
| `numStreams` | Host | IN | Number of CUDA streams in `streams` |

### Request Example
```c
cusparseStatus_t cusparseLtMatmulSearch(const cusparseLtHandle_t* handle,
 cusparseLtMatmulPlan_t* plan,
 const void* alpha,
 const void* d_A,
 const void* d_B,
 const void* beta,
 const void* d_C,
 void* d_D,
 void* workspace,
 cudaStream_t* streams,
 int32_t numStreams);
```

### Response
#### Success Response (200)
- `cusparseStatus_t`: Status of the operation. (e.g., `CUSPARSE_STATUS_SUCCESS`)

#### Response Example
```json
{
  "status": "CUSPARSE_STATUS_SUCCESS"
}
```
```

--------------------------------

### Update Library Path on Linux

Source: https://docs.nvidia.com/cuda/cusparselt/getting_started.html

Set the LD_LIBRARY_PATH environment variable to include the cuSPARSELt library directory. This is necessary for the system to find the cuSPARSELt shared libraries at runtime.

```bash
export LD_LIBRARY_PATH=${CUSPARSELT_DIR}/lib64:${LD_LIBRARY_PATH}
```

--------------------------------

### cusparseLtSpMMACompressedSize

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Calculates the required size for storing a compressed sparse matrix, used in conjunction with sparse matrix-matrix multiplication.

```APIDOC
## cusparseLtSpMMACompressedSize

### Description
Determines the amount of memory required to store a sparse matrix in a compressed format, typically used as a prerequisite for compression operations within sparse matrix-matrix multiplication (SpMMA).

### Usage
Use this function to allocate the correct buffer size before compressing a sparse matrix for SpMMA.
```

--------------------------------

### cusparseLtMatmulGetWorkspace

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Determines the required workspace size for a given matrix multiplication plan and selected algorithm.

```APIDOC
## cusparseLtMatmulGetWorkspace

### Description
The function determines the required workspace size associated to the selected algorithm.

### Signature
cusparseStatus_t
cusparseLtMatmulGetWorkspace(const cusparseLtHandle_t* handle, const cusparseLtMatmulPlan_t* plan, size_t* workspaceSize)

### Parameters
#### In Parameters
- **handle** (const cusparseLtHandle_t*) - cuSPARSELt library handle.
- **plan** (const cusparseLtMatmulPlan_t*) - Matrix multiplication plan.

#### Out Parameters
- **workspaceSize** (size_t*) - Workspace size in bytes.

### Return Value
See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.
```

--------------------------------

### cusparseLtSpMMAPrune

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Prunes a dense matrix according to a specified algorithm. This function supports asynchronous execution and deterministic results.

```APIDOC
## cusparseLtSpMMAPrune

### Description
Prunes a dense matrix `d_in` according to the specified algorithm `pruneAlg`.

### Signature
cusparseStatus_t
cusparseLtSpMMAPrune(const cusparseLtHandle_t* handle,
 const cusparseLtMatmulDescriptor_t* matmulDescr,
 const void* d_in,
 void* d_out,
 cusparseLtPruneAlg_t pruneAlg,
 cudaStream_t stream)

### Parameters
#### Host Parameters
- **handle** (const cusparseLtHandle_t*) - IN - cuSPARSELt library handle
- **matmulDescr** (const cusparseLtMatmulDescriptor_t*) - IN - Matrix multiplication descriptor
- **stream** (cudaStream_t) - IN - CUDA stream for the computation

#### Device Parameters
- **d_in** (const void*) - IN - Pointer to the dense matrix
- **d_out** (void*) - OUT - Pointer to the pruned matrix
- **pruneAlg** (cusparseLtPruneAlg_t) - IN - Pruning algorithm. Possible Values: `CUSPARSELT_PRUNE_SPMMA_TILE`, `CUSPARSELT_PRUNE_SPMMA_STRIP`

### Properties
- Supports asynchronous execution with respect to `stream`.
- Provides deterministic (bit-wise) results for each run.
- Requires no extra storage.
- Supports CUDA graph capture.
- Supports Hardware Memory Compression.
```

--------------------------------

### cusparseLtGetVersion

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Retrieves the version number of the cuSPARSELt library.

```APIDOC
## `cusparseLtGetVersion`

### Description
This function returns the version number of the cuSPARSELt library.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters Table
| Parameter | Memory | In/Out |
| --- | --- | --- |
| `handle` | Host | IN |
| `version` | Host | OUT |

### Return Value
- `cusparseStatus_t`: Status of the operation.

### See Also
- [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t)
```

--------------------------------

### cusparseLtSpMMACompressedSize

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Provides the size of the compressed matrix to be allocated before calling `cusparseLtSpMMACompress()`. This is used to determine the necessary memory for compressed sparse matrices.

```APIDOC
## cusparseLtSpMMACompressedSize

### Description
Provides the size of the _compressed_ matrix to be allocated before calling [cusparseLtSpMMACompress()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltspmmacompress-label).

### Signature
cusparseStatus_t
cusparseLtSpMMACompressedSize(const cusparseLtHandle_t* handle,
 const cusparseLtMatmulPlan_t* plan,
 size_t* compressedSize,
 size_t* compressBufferSize)

### Parameters
#### Host Parameters
- **handle** (const cusparseLtHandle_t*) - IN: cuSPARSELt library handle
- **plan** (const cusparseLtMatmulPlan_t*) - IN: Matrix plan descriptor
- **compressedSize** (size_t*) - OUT: Size in bytes for the compressed matrix
- **compressBufferSize** (size_t*) - OUT: Size in bytes for the buffer needed for the matrix compression

### Return Value
See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.
```

--------------------------------

### cusparseLtMatmul

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Computes the matrix multiplication of matrices A and B to produce the output matrix D, with optional addition of matrix C. Supports structured sparsity in one of the input matrices.

```APIDOC
## cusparseLtMatmul

### Description
Computes the matrix multiplication of matrices `A` and `B` to produce the the output matrix `D`, according to the following operation: D = Activation(α op(A) ⋅ op(B) + β op(C)). One and only one input matrix A or B should have structured sparsity.

### Method
`cusparseLtMatmul`

### Parameters
#### Input Parameters
- **handle** (`const cusparseLtHandle_t*`) - IN - cuSPARSELt library handle.
- **plan** (`const cusparseLtMatmulPlan_t*`) - IN - Matrix multiplication plan.
- **alpha** (`const void*`) - IN - Scalar or vector of scalars used for multiplication. Can be host or device pointer.
- **d_A** (`const void*`) - IN - Pointer to the structured or dense matrix `A`.
- **d_B** (`const void*`) - IN - Pointer to the structured or dense matrix `B`.
- **beta** (`const void*`) - IN - Scalar or vector of scalars used for multiplication. Can be host or device pointer. Can be NULL if alpha is used for scaling and beta scaling is not required.
- **d_C** (`const void*`) - IN - Pointer to the dense matrix `C`.
- **workspace** (`void*`) - IN - Pointer to workspace.
- **streams** (`cudaStream_t*`) - IN - Pointer to CUDA stream array for the computation.
- **numStreams** (`int32_t`) - IN - Number of CUDA streams in `streams`.

#### Output Parameters
- **d_D** (`void*`) - OUT - Pointer to the dense matrix `D`.

### Data Types Supported
| Input A/B | Input C | Output D | Compute |
|---|---|---|---|
| `CUDA_R_32F` | `CUDA_R_32F` | `CUDA_R_32F` | `CUSPARSE_COMPUTE_32F` |
| `CUDA_R_16F` | `CUDA_R_16F` | `CUDA_R_16F` | `CUSPARSE_COMPUTE_32F` |
| `CUDA_R_16BF` | `CUDA_R_16BF` | `CUDA_R_16BF` | `CUSPARSE_COMPUTE_32F` |
| `CUDA_R_8I` | `CUDA_R_8I` | `CUDA_R_8I` | `CUSPARSE_COMPUTE_32I` |
| `CUDA_R_32I` | `CUDA_R_32I` | `CUDA_R_32I` | `CUSPARSE_COMPUTE_32I` |
| `CUDA_R_16F` | `CUDA_R_16F` | `CUDA_R_16F` | `CUSPARSE_COMPUTE_16F` |
| `CUDA_R_16BF` | `CUDA_R_16BF` | `CUDA_R_16BF` | `CUSPARSE_COMPUTE_16F` |
| `CUDA_R_8F_E4M3` | `CUDA_R_8F_E4M3` | `CUDA_R_8F_E4M3` | `CUSPARSE_COMPUTE_32F` |
| `CUDA_R_8F_E5M2` | `CUDA_R_8F_E5M2` | `CUDA_R_8F_E5M2` | `CUSPARSE_COMPUTE_32F` |

**Note**: The function currently only supports the case where `D` has the same shape of `C`.
```

--------------------------------

### cusparseLtSpMMACompress

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Compresses a dense matrix into a format suitable for sparse matrix multiplication using cuSPARSELt. The input dense matrix must be pruned according to specific constraints based on the operation and data type.

```APIDOC
## cusparseLtSpMMACompress

### Description
Compresses a dense matrix `d_dense` into a compressed format intended for use as an operand in `cusparseLtMatmul()` or `cusparseLtMatmulSearch()`. The input matrix must be pruned beforehand.

### Function Signature
cusparseStatus_t
cusparseLtSpMMACompress(const cusparseLtHandle_t* handle,
 const cusparseLtMatmulPlan_t* plan,
 const void* d_dense,
 void* d_compressed,
 void* d_compressed_buffer,
 cudaStream_t stream)

### Parameters
#### Host Parameters
- **handle** (const cusparseLtHandle_t*) - IN - cuSPARSELt library handle.
- **plan** (const cusparseLtMatmulPlan_t*) - IN - Matrix multiplication plan.
- **stream** (cudaStream_t) - IN - CUDA stream for the computation.

#### Device Parameters
- **d_dense** (const void*) - IN - Pointer to the dense matrix.
- **d_compressed** (void*) - OUT - Pointer to the compressed matrix.
- **d_compressed_buffer** (void*) - OUT - Pointer to temporary buffer for the compression.

### Constraints for Pruned Input Matrix
Pruned data must respect constraints based on the operation (`op`) and data type:

*   For `op = CUSPARSE_NON_TRANSPOSE`:
    *   `CUDA_R_4F_E2M1`: Each row must have at least 4 paired non-zero values every eight elements.
    *   `CUDA_R_16F`, `CUDA_R_16BF`, `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`: Each row must have at least two non-zero values every four elements.
    *   `CUDA_R_32F`: Each row must have at least one non-zero value every two elements.

*   For `op = CUSPARSE_TRANSPOSE`:
    *   `CUDA_R_4F_E2M1`: Each column must have at least 4 paired non-zero values every eight elements.
    *   `CUDA_R_16F`, `CUDA_R_16BF`, `CUDA_R_8I`, `CUDA_R_8F_E4M3`, `CUDA_R_8F_E5M2`: Each column must have at least two non-zero values every four elements.
    *   `CUDA_R_32F`: Each column must have at least one non-zero value every two elements.

### Properties
- Supports asynchronous execution with respect to `stream`.
- Provides deterministic results.

### Notes
- `cusparseLtSpMMACompress()` must be called after the algorithm ID is updated.
- The correctness of the pruning result can be checked with `cusparseLtSpMMAPruneCheck()`.
- Pruning with `cusparseLtSpMMAPrune()` is guaranteed to be correct.
- `int8`, `e4m3`, and `e5m2` kernels should run at high SM clocks for maximizing performance.
- Supports CUDA graph capture and Hardware Memory Compression.
```

--------------------------------

### cusparseLtSpMMACompress2

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

This function is deprecated and should not be used in new code. It was related to sparse matrix-matrix multiplication compression.

```APIDOC
## cusparseLtSpMMACompress2 [DEPRECATED]

### Description
This function is deprecated. It was used for compressing sparse matrices in the context of sparse matrix-matrix multiplication.

### Note
Users are advised to migrate to newer, supported functions as this API will be removed in a future release.
```

--------------------------------

### cusparseLtSpMMACompressedSize2

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Provides the size of the compressed matrix to be allocated before calling `cusparseLtSpMMACompress2()`. This function is deprecated and has similar properties to `cusparseLtSpMMACompressedSize()`.

```APIDOC
## cusparseLtSpMMACompressedSize2 [DEPRECATED]

### Description
Provides the size of the _compressed_ matrix to be allocated before calling [cusparseLtSpMMACompress2()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltspmmacompress2-label). It has to be called after [cusparseLtMatmulPlanInit](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltmatmulplaninit-label).

### Signature
cusparseStatus_t
cusparseLtSpMMACompressedSize2(const cusparseLtHandle_t* handle,
 const cusparseLtMatDescriptor_t* sparseMatDescr,
 size_t* compressedSize,
 size_t* compressBufferSize)

### Parameters
#### Host Parameters
- **handle** (const cusparseLtHandle_t*) - IN: cuSPARSELt library handle
- **sparseMatDescr** (const cusparseLtMatDescriptor_t*) - IN: Structured (sparse) matrix descriptor
- **compressedSize** (size_t*) - OUT: Size in bytes of the compressed matrix
- **compressBufferSize** (size_t*) - OUT: Size in bytes for the buffer needed for the matrix compression

### Notes
The function has the same properties of [cusparseLtSpMMACompressedSize()](https://docs.nvidia.com/cuda/cusparselt/functions.html.md#cusparseltspmmacompressedsize-label)
```

--------------------------------

### cusparseLtMatmulPlanDestroy

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Releases the resources used by an instance of the matrix multiplication plan. This function should be called as the last operation with a specific plan instance.

```APIDOC
## cusparseLtMatmulPlanDestroy

### Description
The function releases the resources used by an instance of the matrix multiplication plan. This function is the last call with a specific plan instance.

Calling any cusparseLt function which uses `cusparseLtMatmulPlan_t` after `cusparseLtMatmulPlanDestroy()` will return an error.

### Signature
cusparseStatus_t
cusparseLtMatmulPlanDestroy(const cusparseLtMatmulPlan_t* plan)

### Parameters
#### In Parameters
- **plan** (const cusparseLtMatmulPlan_t*) - Matrix multiplication plan.

### Return Value
See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.
```

--------------------------------

### cusparseLtGetProperty

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Retrieves the value of a requested library property. This function is useful for querying version information or other library-specific settings.

```APIDOC
## cusparseLtGetProperty

### Description
The function returns the value of the requested property. Refer to `libraryPropertyType` for supported types.

### Parameters
#### Path Parameters
- `propertyType` (libraryPropertyType) - IN - Requested property
- `value` (int*) - OUT - Value of the requested property

### Return Value
- `cusparseStatus_t` - Status of the operation. See [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t) for the description of the return status.

### Library Property Types
- `MAJOR_VERSION`: Enumerator to query the major version
- `MINOR_VERSION`: Enumerator to query the minor version
- `PATCH_LEVEL`: Number to identify the patch level
```

--------------------------------

### cusparseLtMatmulAlgGetAttribute

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Returns the value of the queried attribute belonging to algorithm selection descriptor.

```APIDOC
## cusparseLtMatmulAlgGetAttribute

### Description
The function returns the value of the queried attribute belonging to algorithm selection descriptor.

### Parameters
#### Path Parameters
- **handle** (const cusparseLtHandle_t*) - IN - cuSPARSELt library handle
- **algSelection** (const cusparseLtMatmulAlgSelection_t*) - IN - Algorithm selection descriptor
- **attribute** (cusparseLtMatmulAlgAttribute_t) - IN - The attribute that will be retrieved by this function. Possible Values: `CUSPARSELT_MATMUL_ALG_CONFIG_ID`, `CUSPARSELT_MATMUL_ALG_CONFIG_MAX_ID`, `CUSPARSELT_MATMUL_SEARCH_ITERATIONS`, `CUSPARSELT_MATMUL_SPLIT_K`, `CUSPARSELT_MATMUL_SPLIT_K_MODE`, `CUSPARSELT_MATMUL_SPLIT_K_BUFFERS`
- **data** (void*) - OUT - Memory address containing the attribute value retrieved by this function
- **dataSize** (size_t) - IN - Size in bytes of the attribute value used for verification

### Return Value
- cusparseStatus_t - Status of the operation.
```

--------------------------------

### cusparseLtGetErrorString

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Retrieves the description string for a given cuSPARSELt error code. Returns an unrecognized error message if the code is not valid.

```APIDOC
## `cusparseLtGetErrorString()`

### Description
Returns the description string for an error code. If the error code is not recognized, “unrecognized error code” is returned.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters Table
| Parameter | Memory | In/Out |
| --- | --- | --- |
| `status` | Host | IN |

### Return Value
- `const char*`: Pointer to a NULL-terminated string representing the error description.

### See Also
- [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t)
```

--------------------------------

### cusparseLtDestroy

Source: https://docs.nvidia.com/cuda/cusparselt/functions.html

Releases hardware resources used by the cuSPARSELt library for a given handle. This should be the last call made with a specific handle.

```APIDOC
## `cusparseLtDestroy`

### Description
The function releases hardware resources used by the cuSPARSELt library. This function is the last call with a particular handle to the cuSPARSELt library.

Calling any cusparseLt function which uses `cusparseLtHandle_t` after `cusparseLtDestroy()` will return an error.

### Parameters
#### Path Parameters
- None

#### Query Parameters
- None

#### Request Body
- None

### Parameters Table
| Parameter | Memory | In/Out |
| --- | --- | --- |
| `handle` | Host | IN |

### Return Value
- `cusparseStatus_t`: Status of the operation.

### See Also
- [cusparseStatus_t](https://docs.nvidia.com/cuda/cusparse/index.html.md#cusparseStatus_t)
```
Tokens:
Raw