NVIDIA RAFT

https://docs.nvidia.com/raft/latest/
Admin
NVIDIA RAFT provides CUDA-accelerated algorithms and primitives for machine learning and information...
Tokens:240,742
Snippets:1,621
Trust Score:10
Update:3 days ago
Show doc for...
Tokens:
Raw
### Install RAFT headers

Source: https://docs.nvidia.com/raft/latest/build/index.html

Download dependencies and install RAFT headers using the build script.

```bash
./build.sh libraft
```

--------------------------------

### Install pre-commit via pip

Source: https://docs.nvidia.com/raft/latest/contributing/index.html

Install the pre-commit tool using the pip package manager.

```bash
pip install pre-commit
```

--------------------------------

### Configure and install RAFT with CMake directly

Source: https://docs.nvidia.com/raft/latest/build/index.html

Example of invoking cmake directly for fine-grained control. Installs RAFT into the current Conda environment using CMAKE_INSTALL_PREFIX.

```bash
cd cpp
mkdir build
cd build
cmake -D BUILD_TESTS=ON -DRAFT_COMPILE_LIBRARY=ON -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX ../
make -j<parallel_level> install

```

--------------------------------

### Usage example for map_offset

Source: https://docs.nvidia.com/raft/latest/cpp_api/linalg_map_reduce/index.html

Example demonstrating the use of map_offset to compute squares of input values.

```cpp
#include <raft/core/device_mdarray.hpp>
#include <raft/core/resources.hpp>
#include <raft/core/operators.hpp>
#include <raft/linalg/map.cuh>

auto input = raft::make_device_vector<int>(res, n);
... fill input ..
auto squares = raft::make_device_vector<int>(res, n);
raft::linalg::map_offset(res, squares.view(), raft::sq_op{}, input.view());
```

--------------------------------

### Install pre-commit via conda

Source: https://docs.nvidia.com/raft/latest/contributing/index.html

Install the pre-commit tool using the conda package manager.

```bash
conda install -c conda-forge pre-commit
```

--------------------------------

### map_offset usage example

Source: https://docs.nvidia.com/raft/latest/cpp_api/linalg_map_reduce/index.html

Usage example showing how to create a device vector and apply the square operator using map_offset. Requires including raft/core/device_mdarray.hpp, raft/core/resources.hpp, raft/core/operators.hpp, and raft/linalg/map.cuh.

```cpp
#include <raft/core/device_mdarray.hpp>
#include <raft/core/resources.hpp>
#include <raft/core/operators.hpp>
#include <raft/linalg/map.cuh>

auto squares = raft::make_device_vector<int>(handle, n);
raft::linalg::map_offset(res, squares.view(), raft::sq_op{});
```

--------------------------------

### Install RAFT headers with Mamba

Source: https://docs.nvidia.com/raft/latest/build/index.html

Use these commands to install only the RAFT C++ headers for specific CUDA versions.

```bash
# CUDA 13
mamba install -c rapidsai -c conda-forge libraft-headers cuda-version=13.3

# CUDA 12
mamba install -c rapidsai -c conda-forge libraft-headers cuda-version=12.9
```

--------------------------------

### Install RAFT via pip

Source: https://docs.nvidia.com/raft/latest/build/index.html

Commands to install pylibraft and raft-dask for specific CUDA versions.

```bash
# CUDA 13
pip install pylibraft-cu13
pip install raft-dask-cu13

# CUDA 12
pip install pylibraft-cu12
pip install raft-dask-cu12
```

--------------------------------

### Create a device_span from rmm::device_uvector

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_span

Example showing how to initialize a device_span using data from an rmm::device_uvector.

```cpp
rmm::device_uvector<float> uvec(10, cuda::stream_ref{cudaStream_t{cudaStreamDefault}});
auto view = device_span<float>{uvec.data(), uvec.size()};
```

--------------------------------

### Install pre-commit git hooks

Source: https://docs.nvidia.com/raft/latest/contributing/index.html

Set up pre-commit hooks to run automatically upon every git commit.

```bash
pre-commit install
```

--------------------------------

### Install RAFT packages with Mamba

Source: https://docs.nvidia.com/raft/latest/build/index.html

Use these commands to install the full suite of RAFT packages, including raft-dask and pylibraft, for specific CUDA versions. Mamba is the recommended package manager.

```bash
# CUDA 13
mamba install -c rapidsai -c conda-forge raft-dask pylibraft cuda-version=13.3

# CUDA 12
mamba install -c rapidsai -c conda-forge raft-dask pylibraft cuda-version=12.9
```

--------------------------------

### Download RAFT dependencies

Source: https://docs.nvidia.com/raft/latest/build/index.html

Use the -n flag to download dependencies without installing RAFT headers.

```bash
./build.sh libraft -n
```

--------------------------------

### Dispatching with memory_type_dispatcher

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer/index.html

This example demonstrates using raft::memory_type_dispatcher to handle different memory types, which implicitly constructs an mdbuffer.

```cpp
void foo_device(raft::resources const& res, raft::device_matrix_view<float> data) {
// Implement foo solely for device data
};

// Call foo with data of any memory type:
template <typename mdspan_type>
void foo(raft::resources const& res, mdspan_type data) {
raft::memory_type_dispatcher(res,
    [&res](raft::device_matrix_view<float> dev_data) {foo_device(res, dev_data);},
    data
  );
}
```

--------------------------------

### Select-K for Dense Matrices

Source: https://docs.nvidia.com/raft/latest/cpp_api/matrix_ordering/index.html

Example usage of the select_k function for dense row-major matrices to find the k smallest values in each row.

```cpp
using namespace raft;
// get a 2D row-major array of values to search through
auto in_values = {... input device_matrix_view<const float, int64_t, row_major> ...}
// prepare output arrays
auto out_extents = make_extents<int64_t>(in_values.extent(0), k);
auto out_values  = make_device_mdarray<float>(handle, out_extents);
auto out_indices = make_device_mdarray<int64_t>(handle, out_extents);
// search `k` smallest values in each row
matrix::select_k<float, int64_t>(
  handle, in_values, std::nullopt, out_values.view(), out_indices.view(), true);
```

--------------------------------

### Build documentation

Source: https://docs.nvidia.com/raft/latest/build

Build the documentation along with the required C++ and Python packages.

```bash
./build.sh libraft pylibraft raft-dask docs --compile-lib
```

--------------------------------

### Build Python libraries manually with setup.py

Source: https://docs.nvidia.com/raft/latest/build

Manually build and install Python libraries using setup.py within their respective directories.

```bash
cd python/raft-dask
python setup.py build_ext --inplace
python setup.py install

cd python/pylibraft
python setup.py build_ext --inplace
python setup.py install
```

--------------------------------

### Build documentation with build.sh

Source: https://docs.nvidia.com/raft/latest/build/index.html

Builds documentation along with C++ and Python packages. Requires C++ and Python libraries to be built and installed first.

```bash
./build.sh libraft pylibraft raft-dask docs --compile-lib

```

--------------------------------

### launch_kernel()

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Launches a kernel.

```APIDOC
## launch_kernel()

### Description
Launches a kernel.

### Method
Function

### Endpoint
launch_kernel()

```

--------------------------------

### Build Python libraries manually with setup.py

Source: https://docs.nvidia.com/raft/latest/build/index.html

Alternative to build.sh for building and installing raft-dask and pylibraft manually using setup.py. Run from each package directory.

```bash
cd python/raft-dask
python setup.py build_ext --inplace
python setup.py install

cd python/pylibraft
python setup.py build_ext --inplace
python setup.py install

```

--------------------------------

### view()

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitset

Creates a device-usable view of the bitset.

```APIDOC
## view()

### Description
Creates a device-usable view of the bitset.

### Method
Member function

### Returns
A bitset_view<bitset_t, index_t> object.
```

--------------------------------

### unravel_index usage example

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdspan/index.html

Example showing how to convert a linear index into a coordinate using unravel_index and apply it to an mdspan view.

```cpp
auto m = make_host_matrix<float>(7, 6);
auto m_v = m.view();
auto coord = unravel_index(2, m.extents(), typename decltype(m)::layout_type{});
std::apply(m_v, coord) = 2;
```

--------------------------------

### view()

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitset

Returns a lightweight device-usable view of the bitset.

```APIDOC
## view()

### Description
Returns a lightweight device-usable view of the bitset.

### Returns
- **raft::core::bitset_view<bitset_t, index_t>** - A view of the bitset.
```

--------------------------------

### Convert Bitset to CSR Matrix

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitset

Example demonstrating how to initialize a bitset in device memory and convert it into a Compressed Sparse Row (CSR) matrix format using the to_csr method.

```cpp
#include <raft/core/resource/cuda_stream.hpp>
#include <raft/sparse/convert/csr.cuh>
#include <rmm/device_uvector.hpp>

using bitset_t = uint32_t;
using index_t  = int;
using value_t  = float;

raft::resources handle;
auto stream    = resource::get_cuda_stream(handle);
index_t n_rows = 3;
index_t n_cols = 30;

// Compute bitset size and initialize device memory
index_t bitset_size = (n_cols + sizeof(bitset_t) * 8 - 1) / (sizeof(bitset_t) * 8);
rmm::device_uvector<bitset_t> bitset_d(bitset_size, stream);
std::vector<bitset_t> bitset_h = {
  bitset_t(0b11001010),
};  // Example bitset, with 4 non-zero entries.

raft::copy(bitset_d.data(), bitset_h.data(), bitset_h.size(), stream);

// Create bitset view and CSR matrix
auto bitset_view = raft::core::bitset_view<bitset_t, index_t>(bitset_d.data(), n_cols);
auto csr = raft::make_device_csr_matrix<value_t, index_t>(handle, n_rows, n_cols, 4 * n_rows);

// Convert bitset to CSR
bitset_view.to_csr(handle, csr);
resource::sync_stream(handle);

// Results:
// csr.indptr  = [0, 4, 8, 12];
// csr.indices = [1, 3, 6, 7,
//                1, 3, 6, 7,
//                1, 3, 6, 7];
// csr.values  = [1, 1, 1, 1,
//                1, 1, 1, 1,
//                1, 1, 1, 1];
```

--------------------------------

### get_elements

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Function to get elements.

```APIDOC
## get_elements

### Description
Function to get elements.

### Signature
`get_elements()`
```

--------------------------------

### get_input_class_cardinality()

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Gets the cardinality of input classes.

```APIDOC
## get_input_class_cardinality()

### Description
Gets the cardinality of input classes.

### Method
Function

### Endpoint
get_input_class_cardinality()

```

--------------------------------

### raft::launch_on

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Provides configuration for launching kernels on a specific device.

```APIDOC
## raft::launch_on

### Description
Provides configuration for launching kernels on a specific device.

### Method
Type

### Endpoint
raft::launch_on

### Parameters
- location
- config
- flags

```

--------------------------------

### structure_view

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Function to get a structure view.

```APIDOC
## structure_view

### Description
Function to get a structure view.

### Signature
`structure_view()`
```

--------------------------------

### cooperative()

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Launches a cooperative kernel.

```APIDOC
## cooperative()

### Description
Launches a cooperative kernel.

### Method
Function

### Endpoint
cooperative()

```

--------------------------------

### get_n_rows

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Function to get the number of rows.

```APIDOC
## get_n_rows

### Description
Function to get the number of rows.

### Signature
`get_n_rows()`
```

--------------------------------

### Compile C++ benchmarks

Source: https://docs.nvidia.com/raft/latest/build/index.html

Build the primitives microbenchmarks.

```bash
./build.sh libraft bench-prims
```

--------------------------------

### get_workspace_resource_ref

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_resources/index.html

Gets the workspace as a non-owning device_async_resource_ref.

```APIDOC
## get_workspace_resource_ref

### Description
Get the workspace as a non-owning device_async_resource_ref.

### Function Signature
`rmm::device_async_resource_ref get_workspace_resource_ref(_resources const &res_)`

### Parameters
- **res** (_resources const &) - RAFT resources object for managing resources.

### Returns
- rmm::device_async_resource_ref - Non-owning reference to the workspace device memory resource.
```

--------------------------------

### view

Source: https://docs.nvidia.com/raft/latest/cpp_api

Function to get a view of the sparse matrix.

```APIDOC
## view

### Description
Function to get a view of the sparse matrix.

### Signature
`view()`
```

--------------------------------

### cooperative()

Source: https://docs.nvidia.com/raft/latest/cpp_api

Configures cooperative kernel launch.

```APIDOC
## cooperative()

### Description
Configures cooperative kernel launch.

### Method
Function

### Signature
cooperative()

### Parameters
No parameters documented.

### Returns
No return value documented.
```

--------------------------------

### qr_get_qr()

Source: https://docs.nvidia.com/raft/latest/cpp_api

Gets the QR matrices from QR decomposition.

```APIDOC
## qr_get_qr()

### Description
Gets the QR matrices from QR decomposition.

### Method
Function

### Signature
`qr_get_qr()`
```

--------------------------------

### Create mdbuffer for device memory and check ownership

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer/index.html

Shows how to wrap an mdspan in an mdbuffer to guarantee device accessibility. If the data is already on device, no copy is performed; otherwise a copy is made. Use is_owning() to determine if a copy was needed, and copy back if so.

```cpp
template <typename mdspan_type>
void foo_device(raft::resources const& res, mdspan_type data) {
auto buf = raft::mdbuffer{res, raft::mdbuffer{data}, raft::memory_type::device};
// Data in buf is now guaranteed to be accessible from device.
// If it was already accessible from device, no copy was performed. If it
// was not, a copy was performed.

  some_kernel<<<...>>>(buf.view<raft::memory_type::device>());

// It is sometimes useful to know whether or not a copy was performed to
// e.g. determine whether the transformed data should be copied back to its original
// location. This can be checked via the `is_owning()` method.
if (buf.is_owning()) {
raft::copy(res, data, buf.view<raft::memory_type::device>());
  }
}

```

--------------------------------

### Use ccache with build.sh

Source: https://docs.nvidia.com/raft/latest/build/index.html

Enable ccache or sccache to speed up frequent rebuilds.

```bash
./build.sh libraft --cache-tool=ccache
```

--------------------------------

### get_nnz

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Function to get the number of non-zero elements.

```APIDOC
## get_nnz

### Description
Function to get the number of non-zero elements.

### Signature
`get_nnz()`
```

--------------------------------

### get_type

Source: https://docs.nvidia.com/raft/latest/cpp_api/index.html

Function to get the datatype_t for a given type.

```APIDOC
## get_type

### Description
Function to get the datatype_t for a given type.

### Overloads
- `get_type()`
- `get_type< char >()`
- `get_type< uint8_t >()`
- `get_type< int >()`
- `get_type< uint32_t >()`
- `get_type< int64_t >()`
- `get_type< uint64_t >()`
- `get_type< float >()`
- `get_type< double >()`
```

--------------------------------

### get_large_workspace_resource_ref

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_resources/index.html

Gets the large workspace as a non-owning device_async_resource_ref.

```APIDOC
## get_large_workspace_resource_ref

### Description
Get the large workspace as a non-owning device_async_resource_ref.

### Method
inline function

### Signature
inline rmm::device_async_resource_ref get_large_workspace_resource_ref(_resources const &res_)

### Parameters
- **res** (resources) - Required - raft resources object for managing resources

### Returns
- non-owning reference to the large workspace device memory resource
```

--------------------------------

### get_workspace_total_bytes

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_resources/index.html

Gets the total size of the workspace resource.

```APIDOC
## get_workspace_total_bytes

### Description
Get the total size of the workspace resource.

### Function Signature
`size_t get_workspace_total_bytes(_resources const &res_)`

### Parameters
- **res** (_resources const &) - RAFT resources object for managing resources.

### Returns
- size_t - The total size of the workspace resource.
```

--------------------------------

### bitset_view::data

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitset

Get the device pointer to the bitset.

```APIDOC
## bitset_view::data() -> bitset_t*

### Description
Get the device pointer to the bitset.

### Returns
- **bitset_t*** - Device pointer to the bitset
```

--------------------------------

### make_device_matrix_view()

Source: https://docs.nvidia.com/raft/latest/cpp_api

Creates a device matrix view.

```APIDOC
## make_device_matrix_view()

### Description
Creates a device matrix view.

### Method
Function

### Signature
`make_device_matrix_view()`
```

--------------------------------

### get_n_cols()

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitmap/index.html

Get the total number of columns in the bitmap.

```APIDOC
## get_n_cols() const

### Description
Get the total number of columns.

### Parameters
None.

### Returns
- **index_t** - The total number of columns.
```

--------------------------------

### get_n_rows()

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitmap/index.html

Get the total number of rows in the bitmap.

```APIDOC
## get_n_rows() const

### Description
Get the total number of rows.

### Parameters
None.

### Returns
- **index_t** - The total number of rows.
```

--------------------------------

### Build Python libraries with build.sh

Source: https://docs.nvidia.com/raft/latest/build/index.html

Use build.sh to compile and install pylibraft and raft-dask. raft-dask depends on pylibraft, so include both when building raft-dask.

```bash
# to build pylibraft
./build.sh libraft pylibraft --compile-lib
# to build raft-dask (depends on pylibraft)
./build.sh libraft pylibraft raft-dask --compile-lib

```

--------------------------------

### memory_type_from_variant_index()

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer

Function to get the memory type from a variant index.

```APIDOC
## memory_type_from_variant_index()

### Description
Function to get the memory type from a variant index.

```

--------------------------------

### variant_index_from_memory_type()

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer

Function to get the variant index from a memory type.

```APIDOC
## variant_index_from_memory_type()

### Description
Function to get the variant index from a memory type.

```

--------------------------------

### Create temporary device buffer with make_temporary_device_buffer

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_temporary_device_buffer

Creates a temporary device buffer. Setting write_back to true allows copying writes back to the original pointer if it was in host memory.

```cpp
#include <raft/core/resources.hpp>

raft::resources handle;

// Initialize raft::device_mdarray and raft::extents
// Can be either raft::device_mdarray or raft::host_mdarray
auto exts  = raft::make_extents<int>(5);
auto array = raft::make_device_mdarray<int, int>(handle, exts);

auto d_buf = raft::make_temporary_device_buffer(handle, array.data_handle(), exts);
```

--------------------------------

### bitset()

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitset

Constructs a bitset object. Two overloads are available: one that takes a size, and another that takes a size and a default value.

```APIDOC
## bitset()

### Description
Constructs a bitset object. Two overloads are available: one that takes a size, and another that takes a size and a default value.

### Method
Constructor

### Parameters
#### Overload 1
- **size** (index_t) - Required - Number of bits in the bitset

#### Overload 2
- **size** (index_t) - Required - Number of bits in the bitset
- **default_value** (bool) - Required - Default value for the bits (true or false)
```

--------------------------------

### memory_type_from_variant_index()

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer/index.html

Function to get the memory type from a variant index.

```APIDOC
## memory_type_from_variant_index()

### Description
Function to get the memory type from a variant index.
```

--------------------------------

### variant_index_from_memory_type()

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer/index.html

Function to get the variant index from a memory type.

```APIDOC
## variant_index_from_memory_type()

### Description
Function to get the variant index from a memory type.
```

--------------------------------

### device_resources_snmg(const device_resources_snmg &world_)

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_resources

SNMG resources instance copy constructor.

```APIDOC
## Constructor: device_resources_snmg(const device_resources_snmg &world_)

### Description
SNMG resources instance copy constructor.

### Method
Constructor

### Parameters
- **world** (const device_resources_snmg&) - [in] A SNMG resources instance

### Returns
A copy of the device_resources_snmg object.

```

--------------------------------

### bitmap_view(bitmap_t*, index_t, index_t, index_t)

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitmap

Constructor that creates a bitmap view from a device raw pointer. It takes the pointer, number of rows, number of columns, and an optional original_nbits parameter.

```APIDOC
## bitmap_view(bitmap_t*, index_t, index_t, index_t)

### Description
Creates a bitmap view from a device raw pointer.

### Method
Constructor

### Signature
bitmap_view(bitmap_t *bitmap_ptr_, index_t rows_, index_t cols_, index_t original_nbits = 0)

### Parameters
- **bitmap_ptr** (bitmap_t*) - Required - Device raw pointer
- **rows** (index_t) - Required - Number of rows in the matrix
- **cols** (index_t) - Required - Number of columns in the matrix
- **original_nbits** (index_t) - Optional - Original number of bits used when the bitmap was created, to handle potential mismatches of data types
```

--------------------------------

### default_container_policy_variant

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_mdbuffer/index.html

Utility to get the default container policy variant.

```APIDOC
## default_container_policy_variant

### Description
Utility to get the default container policy variant.
```

--------------------------------

### bitset_view::size

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitset

Get the number of bits of the bitset representation.

```APIDOC
## bitset_view::size() const -> index_t

### Description
Get the number of bits of the bitset representation.

### Returns
- **index_t** - Number of bits of the bitset representation
```

--------------------------------

### Create write-back temporary device buffer with make_writeback_temporary_device_buffer

Source: https://docs.nvidia.com/raft/latest/cpp_api/mdspan_temporary_device_buffer

Creates a temporary device buffer that provides a writeable raft::device_mdspan via the view() method with write_back set to true.

```cpp
#include <raft/core/resources.hpp>

raft::resources handle;

// Initialize raft::host_mdarray and raft::extents
// Can be either raft::device_mdarray or raft::host_mdarray
auto exts  = raft::make_extents<int>(5);
auto array = raft::make_host_mdarray<int, int>(handle, exts);

auto d_buf = raft::make_writeback_temporary_device_buffer(handle, array.data_handle(), exts);
```

--------------------------------

### get_n_cols() const

Source: https://docs.nvidia.com/raft/latest/cpp_api/core_bitmap

Get the total number of columns in the bitmap.

```APIDOC
## get_n_cols() const

### Description
Get the total number of columns.

### Method
get_n_cols

### Signature
index_t get_n_cols() const

### Returns
- **index_t** - The total number of columns
```
Tokens:
Raw