NVIDIA RMM

https://docs.nvidia.com/rmm/latest/
Admin
NVIDIA RMM is a CUDA memory management library that provides configurable device memory allocation,...
Tokens:158,303
Snippets:1,450
Trust Score:10
Update:4 days ago
Show doc for...
Tokens:
Raw
### device_buffer usage examples

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers/index.html

Demonstrates various ways to allocate, copy, move, and resize device_buffer objects.

```cpp
// Allocates at least 100 bytes of device memory using the default memory
// resource and default stream.
device_buffer buff(100);

// Allocates at least 100 bytes using the custom memory resource and
// specified stream
custom_memory_resource mr;
auto stream = cuda::stream_ref{cudaStream_t{cudaStreamDefault}};
device_buffer custom_buff(100, stream, &mr);

// Deep copies `buff` into a new device buffer using the specified stream
device_buffer buff_copy(buff, stream);

// Moves the memory in `from_buff` to `to_buff`. Deallocates previously allocated
// to_buff memory on `to_buff.stream()`.
device_buffer to_buff(std::move(from_buff));

// Deep copies `buff` into a new device buffer using the specified stream
device_buffer buff_copy(buff, stream);

// Shallow copies `buff` into a new device_buffer, `buff` is now empty
device_buffer buff_move(std::move(buff));

// Default construction. Buffer is empty
device_buffer buff_default{};

// If the requested size is larger than the current size, resizes allocation to the new size and
// deep copies any previous contents. Otherwise, simply updates the value of `size()` to the
// newly requested size without any allocations or copies. Uses the specified stream.
buff_default.resize(100, stream);
```

--------------------------------

### Set debug logging level

Source: https://docs.nvidia.com/rmm/latest/python/rmm/index.html

Example of setting the debug logging level to WARN.

```python
>>> import rmm
>>> rmm.set_logging_level(rmm.logging_level.WARN) # set logging level to warn
```

--------------------------------

### Adapting a stream-ordered allocator

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources/index.html

Example showing how to use stream_allocator_adaptor to wrap a stream-ordered allocator for use with standard allocator interfaces.

```cpp
my_stream_ordered_allocator<int> a{...};
cuda::stream_ref s = // create stream;

auto adapted = stream_allocator_adaptor(a, s);

// Allocates storage for `n` int's on stream `s`
int * p = std::allocator_traits<decltype(adapted)>::allocate(adapted, n);
```

--------------------------------

### rmm.DeviceBuffer.nbytes

Source: https://docs.nvidia.com/rmm/latest/python/rmm

Gets the size of the buffer in bytes.

```APIDOC
## rmm.DeviceBuffer.nbytes

### Description
Gets the size of the buffer in bytes.
```

--------------------------------

### rmm.DeviceBuffer.nbytes

Source: https://docs.nvidia.com/rmm/latest/python/rmm/index.html

Gets the size of the buffer in bytes.

```APIDOC
## rmm.DeviceBuffer.nbytes

### Description
Gets the size of the buffer in bytes.

### Returns
- **int** - The size of the buffer in bytes.
```

--------------------------------

### Implementing custom memory resources

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

Demonstrates the minimal migration for custom memory resources to comply with the CCCL resource concept, requiring both async and synchronous interfaces.

```cpp
// 26.04
class my_resource : public rmm::mr::device_memory_resource {
  void* do_allocate(std::size_t bytes, rmm::cuda_stream_view stream) override {
    return upstream_->allocate(bytes, stream);
  }
  void do_deallocate(void* p, std::size_t bytes, rmm::cuda_stream_view stream) override {
    upstream_->deallocate(p, bytes, stream);
  }
  bool do_is_equal(device_memory_resource const& other) const noexcept override {
    return this == &other;
  }
  rmm::mr::device_memory_resource* upstream_;
};

// 26.06
class my_resource {
 public:
  explicit my_resource(cuda::mr::any_resource<cuda::mr::device_accessible> upstream)
    : upstream_(std::move(upstream)) {}

  // Async (stream-ordered) interface
  void* allocate(cuda::stream_ref stream, std::size_t bytes,
                 std::size_t alignment = rmm::CUDA_ALLOCATION_ALIGNMENT) {
    return upstream_.allocate(stream, bytes, alignment);
  }
  void deallocate(cuda::stream_ref stream, void* p, std::size_t bytes,
                  std::size_t alignment = rmm::CUDA_ALLOCATION_ALIGNMENT) noexcept {
    upstream_.deallocate(stream, p, bytes, alignment);
  }

  // Synchronous interface — required by the CCCL resource concept
  void* allocate_sync(std::size_t bytes,
                      std::size_t alignment = rmm::CUDA_ALLOCATION_ALIGNMENT) {
    return upstream_.allocate_sync(bytes, alignment);
  }
  void deallocate_sync(void* p, std::size_t bytes,
                       std::size_t alignment = rmm::CUDA_ALLOCATION_ALIGNMENT) noexcept {
    upstream_.deallocate_sync(p, bytes, alignment);
  }

  bool operator==(my_resource const& other) const noexcept {
    return this == &other;
  }
  bool operator!=(my_resource const& other) const noexcept {
    return !(*this == other);
  }

  constexpr friend void get_property(my_resource const&, cuda::mr::device_accessible) noexcept {}

 private:
  cuda::mr::any_resource<cuda::mr::device_accessible> upstream_;
};
static_assert(cuda::mr::resource_with<my_resource, cuda::mr::device_accessible>);
```

--------------------------------

### Copy host data to device with copy_host_to_ptr

Source: https://docs.nvidia.com/rmm/latest/python/pylibrmm/index.html

Shows how to copy host data into a device buffer pointer.

```python
>>> import rmm
>>> db = rmm.DeviceBuffer(size=10)
>>> hb = b"abc"
>>> rmm.pylibrmm.device_buffer.copy_host_to_ptr(hb, db.ptr)
>>> hb = db.copy_to_host()
>>> print(hb)
array([97, 98, 99,  0,  0,  0,  0,  0,  0,  0], dtype=uint8)
```

--------------------------------

### cuda_stream::value()

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams

Gets the value of the wrapped CUDA stream.

```APIDOC
## cuda_stream::value()

### Description
Gets the value of the wrapped CUDA stream.

### Returns
- **cudaStream_t** - The wrapped CUDA stream.
```

--------------------------------

### rmm::cuda_stream::value()

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams/index.html

Get the value of the wrapped CUDA stream.

```APIDOC
## value()

### Description
Get the value of the wrapped CUDA stream.

### Returns
- cudaStream_t - The wrapped CUDA stream.

### Example
```cpp
cudaStream_t raw = stream.value();
```
```

--------------------------------

### Allocating and initializing a device_uvector

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers/index.html

Demonstrates how to allocate uninitialized device memory using a custom resource and stream, followed by an explicit initialization using thrust::uninitialized_fill.

```cpp
auto mr = new my_custom_resource();
auto s = cuda::stream_ref{cudaStream_t{cudaStreamDefault}};

// Allocates *uninitialized* device memory on stream `s` sufficient for 100 ints using the
// supplied resource `mr`
rmm::device_uvector<int> uv(100, s, mr);

// Initializes all elements to 0 on stream `s`
thrust::uninitialized_fill(thrust::cuda::par.on(s), uv.begin(), uv.end(), 0);
```

--------------------------------

### Get the current logging level

Source: https://docs.nvidia.com/rmm/latest/python/rmm/index.html

Retrieves the current debug logging level.

```python
>>> import rmm
>>> rmm.get_logging_level() # get current logging level
<logging_level.INFO: 2>
```

--------------------------------

### Implement a CallbackMemoryResource

Source: https://docs.nvidia.com/rmm/latest/python/mr

Demonstrates how to define custom allocation and deallocation functions and use them to set the current device memory resource.

```python
>>> import rmm
>>> base_mr = rmm.mr.CudaMemoryResource()
>>> def allocate_func(size, stream):
...     print(f"Allocating {size} bytes")
...     return base_mr.allocate(size, stream)
...
>>> def deallocate_func(ptr, size, stream):
...     print(f"Deallocating {size} bytes")
...     return base_mr.deallocate(ptr, size, stream)
...
>>> rmm.mr.set_current_device_resource(
    rmm.mr.CallbackMemoryResource(allocate_func, deallocate_func)
)
>>> dbuf = rmm.DeviceBuffer(size=256)
Allocating 256 bytes
>>> del dbuf
Deallocating 256 bytes
```

--------------------------------

### Migrating owning_wrapper to value-based storage

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

Shows the transition from using rmm::mr::owning_wrapper to storing resources by value with internal shared ownership.

```cpp
// 26.04
auto cuda_mr = std::make_shared<rmm::mr::cuda_memory_resource>();
auto pool = rmm::mr::make_owning_wrapper<rmm::mr::pool_memory_resource>(
    cuda_mr, pool_size);

// 26.06 — resources are value types with shared ownership
rmm::mr::cuda_memory_resource cuda_mr;
rmm::mr::pool_memory_resource pool{cuda_mr, pool_size};
// pool is stored by value; its state is kept alive by internal shared ownership
```

--------------------------------

### get_per_device_resource_ref

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources/index.html

Get the device_async_resource_ref for the specified device. Returns a device_async_resource_ref for the specified device, initially referencing a cuda_memory_resource.

```APIDOC
## Function: get_per_device_resource_ref

### Description
Get the device_async_resource_ref for the specified device. Returns a device_async_resource_ref for the specified device. The initial resource_ref references a cuda_memory_resource.

### Signature
inline device_async_resource_ref get_per_device_resource_ref(_cuda_device_id device_id_);

### Parameters
- **device_id** (_cuda_device_id) - The id of the target device. device_id.value() must be in the range [0, cudaGetDeviceCount()), otherwise behavior is undefined.

### Returns
The current device_async_resource_ref for device device_id.

### Thread Safety
This function is thread-safe with respect to concurrent calls to set_per_device_resource, get_per_device_resource_ref, get_current_device_resource_ref, set_current_device_resource, reset_per_device_resource, and reset_current_device_resource. Concurrent calls to any of these functions will result in a valid state, but the order of execution is undefined.

### Notes
The returned device_async_resource_ref should only be used when CUDA device device_id is the current device (e.g. set using cudaSetDevice()). The behavior of a device_async_resource_ref is undefined if used while the active CUDA device is a different device from the one that was active when the memory resource was created.
```

--------------------------------

### Enable statistics tracking with context manager and globally

Source: https://docs.nvidia.com/rmm/latest/user_guide/guide/index.html

Shows how to enable RMM statistics tracking using the statistics() context manager or enable_statistics() globally, and demonstrates that the current device resource becomes a StatisticsResourceAdaptor.

```python
>>> import rmm
>>> import rmm.statistics

>>> # We start with the default CUDA memory resource
>>> rmm.mr.get_current_device_resource()
<rmm.pylibrmm.memory_resource.CudaMemoryResource object at 0x7fa0da48a8e0>

>>> # When using statistics, we get a StatisticsResourceAdaptor with the context
>>> with rmm.statistics.statistics():
...     rmm.mr.get_current_device_resource()
<rmm.pylibrmm.memory_resource.StatisticsResourceAdaptor object at 0x7fa0dd6e4a40>

>>> # We can also enable statistics globally
>>> rmm.statistics.enable_statistics()
>>> print(rmm.mr.get_current_device_resource())
<rmm.pylibrmm.memory_resource.StatisticsResourceAdaptor object at 0x7f9a11340a40>
```

--------------------------------

### Create DeviceBuffer from host memory

Source: https://docs.nvidia.com/rmm/latest/user_guide/guide/index.html

Initialize a DeviceBuffer by copying data from a host NumPy array.

```python
>>> import rmm
>>> import numpy as np
>>> a = np.array([1, 2, 3], dtype='float64')
>>> buf = rmm.DeviceBuffer.to_device(a.view("uint8"))  # to_device expects an unsigned 8-bit dtype
>>> buf.size
24
```

--------------------------------

### cuda_stream_pool::get_pool_size()

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams

Gets the number of streams in the pool. This function is thread safe with respect to other calls to the same function.

```APIDOC
## cuda_stream_pool::get_pool_size()

### Description
Gets the number of streams in the pool. This function is thread safe with respect to other calls to the same function.

### Returns
- **std::size_t** - The number of streams in the pool.
```

--------------------------------

### Construct a DeviceBuffer

Source: https://docs.nvidia.com/rmm/latest/python/rmm/index.html

Initialize a new DeviceBuffer with a specified size.

```python
>>> import rmm
>>> db = rmm.DeviceBuffer(size=5)
```

--------------------------------

### Set flush level for debug logger

Source: https://docs.nvidia.com/rmm/latest/python/rmm/index.html

Example of setting the flush level for the debug logger to WARN.

```python
>>> import rmm
>>> rmm.flush_on(rmm.logging_level.WARN) # set flush level to warn
```

--------------------------------

### Copy device data to host with copy_ptr_to_host

Source: https://docs.nvidia.com/rmm/latest/python/pylibrmm/index.html

Demonstrates copying data from a device pointer into a host-side buffer.

```python
>>> import rmm
>>> db = rmm.DeviceBuffer.to_device(b"abc")
>>> hb = bytearray(db.nbytes)
>>> rmm.pylibrmm.device_buffer.copy_ptr_to_host(db.ptr, hb)
>>> print(hb)
bytearray(b'abc')
```

--------------------------------

### rmm.mr.get_per_device_resource(device)

Source: https://docs.nvidia.com/rmm/latest/python/mr

Gets the default memory resource for the specified device. The returned resource should only be used while that device is active.

```APIDOC
## rmm.mr.get_per_device_resource(device)

### Description
Get the default memory resource for the specified device.

### Method
Function call

### Endpoint
rmm.mr.get_per_device_resource(device)

### Parameters
#### Path Parameters
- **device** (int) - Required - The ID of the device for which to get the memory resource.

### Returns
- **DeviceMemoryResource** - The memory resource for the specified device.

### Notes
If the returned memory resource is used when a different device is the active CUDA device, behavior is undefined.
```

--------------------------------

### Pass Upstream Resources by Value

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

Constructors now accept cuda::mr::any_resource instead of raw pointers. Resources can be passed directly.

```cpp
// 26.04 - pass a device_memory_resource pointer
rmm::mr::cuda_memory_resource cuda_mr;
rmm::mr::pool_memory_resource<rmm::mr::cuda_memory_resource> pool{&cuda_mr, pool_size};

// 26.06 — pass the resource directly
rmm::mr::cuda_memory_resource cuda_mr;
rmm::mr::pool_memory_resource pool{cuda_mr, pool_size};
```

--------------------------------

### rmm::cuda_stream_pool::get_pool_size()

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams/index.html

Get the number of streams in the pool. This function is thread safe with respect to other calls to the same function.

```APIDOC
## get_pool_size()

### Description
Get the number of streams in the pool. This function is thread safe with respect to other calls to the same function.

### Returns
- std::size_t - The number of streams in the pool.

### Example
```cpp
std::size_t size = pool.get_pool_size();
```
```

--------------------------------

### Create a DeviceBuffer

Source: https://docs.nvidia.com/rmm/latest/user_guide/guide/index.html

Create an untyped, uninitialized device memory allocation by specifying the size in bytes.

```python
>>> import rmm
>>> buf = rmm.DeviceBuffer(size=100)
```

--------------------------------

### cuda_stream_pool::get_stream()

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams

Gets a cuda::stream_ref of a stream in the pool. This function is thread safe with respect to other calls to the same function.

```APIDOC
## cuda_stream_pool::get_stream()

### Description
Gets a cuda::stream_ref of a stream in the pool. This function is thread safe with respect to other calls to the same function.

### Returns
- **cuda::stream_ref**
```

--------------------------------

### Update Per-Device Resource Management

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

Comparison of per-device resource management between 26.04 and 26.06, showing the shift from raw pointers to owning any_resource types and the use of RAII-style restoration.

```cpp
// 26.04
rmm::mr::pool_memory_resource<rmm::mr::cuda_memory_resource> pool{&cuda_mr, size};
rmm::mr::set_current_device_resource(&pool);
auto* mr = rmm::mr::get_current_device_resource();

// 26.06
rmm::mr::pool_memory_resource pool{cuda_mr, size};
auto prev = rmm::mr::set_current_device_resource(pool);
auto ref = rmm::mr::get_current_device_resource_ref();
// When done, restore: rmm::mr::set_current_device_resource(std::move(prev));
// Or reset to default: rmm::mr::reset_current_device_resource();
```

--------------------------------

### RMMNumbaManager.get_ipc_handle(memory: MemoryPointer) -> IpcHandle

Source: https://docs.nvidia.com/rmm/latest/python/allocators/index.html

Get an IPC handle for the MemoryPointer memory with offset modified by the RMM memory pool.

```APIDOC
## RMMNumbaManager.get_ipc_handle(memory: MemoryPointer) -> IpcHandle

### Description
Get an IPC handle for the MemoryPointer memory with offset modified by the RMM memory pool.

### Parameters
- **memory** (MemoryPointer) - Required - The memory pointer for which to get an IPC handle.

### Returns
- **IpcHandle** - The IPC handle for the memory.
```

--------------------------------

### device_uvector constructor (size, stream, mr)

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers/index.html

Constructs a new device_uvector with sufficient uninitialized storage for the specified number of elements. The elements are uninitialized, and reading them before initialization is undefined behavior.

```APIDOC
## Constructor: device_uvector(size_type size, cuda::stream_ref stream, cuda::mr::any_resource<cuda::mr::device_accessible> mr = mr::get_current_device_resource_ref())

### Description
Constructs a new device_uvector with sufficient uninitialized storage for `size` elements. Elements are uninitialized; reading an element before it is initialized results in undefined behavior.

### Parameters
- **size** (size_type) - Required - The number of elements to allocate storage for.
- **stream** (cuda::stream_ref) - Required - The stream on which to perform the allocation.
- **mr** (cuda::mr::any_resource<cuda::mr::device_accessible>) - Optional - The resource used to allocate the device storage. Defaults to the current device resource.

### Throws
- **rmm::bad_alloc** - If the provided memory resource cannot allocate with alignment to satisfy the alignment requirements of the value type.
- **rmm::invalid_argument** - If the requested size overflows the maximum allocation size.

### Example
```cpp
auto mr = new my_custom_resource();
auto s = cuda::stream_ref{cudaStream_t{cudaStreamDefault}};
rmm::device_uvector<int> uv(100, s, mr);
```
```

--------------------------------

### Create a DeviceBuffer with to_device

Source: https://docs.nvidia.com/rmm/latest/python/pylibrmm/index.html

Creates a new DeviceBuffer containing a copy of the provided host data.

```python
>>> import rmm
>>> db = rmm.pylibrmm.device_buffer.to_device(b"abc")
>>> print(bytes(db))
b'abc'
```

--------------------------------

### get_current_device_resource_ref

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources/index.html

Gets the device_async_resource_ref for the current device. Returns the resource reference set for the current device, initially referencing a cuda_memory_resource.

```APIDOC
## get_current_device_resource_ref

### Description
Gets the device_async_resource_ref for the current device. Returns the device_async_resource_ref set for the current device. The initial resource_ref references a cuda_memory_resource. The current device is the device returned by cudaGetDevice. This function is thread-safe with respect to concurrent calls to set_per_device_resource, get_per_device_resource_ref, get_current_device_resource_ref, set_current_device_resource, reset_per_device_resource, and reset_current_device_resource. Concurrent calls to any of these functions will result in a valid state, but the order of execution is undefined.

### Method
N/A (C++ inline function)

### Signature
inline device_async_resource_ref get_current_device_resource_ref()

### Parameters
None

### Returns
device_async_resource_ref active for the current device

### Notes
The returned device_async_resource_ref should only be used with the current CUDA device. Changing the current device (e.g. using cudaSetDevice()) and then using the returned resource_ref can result in undefined behavior. The behavior of a device_async_resource_ref is undefined if used while the active CUDA device is a different device from the one that was active when the memory resource was created.
```

--------------------------------

### rmm::cuda_stream_pool::get_stream()

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams/index.html

Get a cuda::stream_ref of a stream in the pool. This function is thread safe with respect to other calls to the same function.

```APIDOC
## get_stream()

### Description
Get a cuda::stream_ref of a stream in the pool. This function is thread safe with respect to other calls to the same function.

### Returns
- cuda::stream_ref

### Example
```cpp
auto stream_ref = pool.get_stream();
```
```

--------------------------------

### bad_alloc::what()

Source: https://docs.nvidia.com/rmm/latest/cpp/errors

Returns the explanatory string for the bad_alloc exception.

```APIDOC
## bad_alloc::what()

### Description
Returns the explanatory string for the bad_alloc exception.

### Returns
- **const char*** - The explanatory string
```

--------------------------------

### void* data() noexcept

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers/index.html

Returns a pointer to the device memory allocation.

```APIDOC
## void* data() noexcept

### Description
Pointer to the device memory allocation.

### Returns
Pointer to the device memory allocation.
```

--------------------------------

### BinningMemoryResource

Source: https://docs.nvidia.com/rmm/latest/python/mr

A memory resource that bins allocations by size. It inherits from UpstreamResourceAdaptor and provides methods to add bins, allocate, deallocate, and get the upstream resource.

```APIDOC
## BinningMemoryResource

### Description
A memory resource that bins allocations by size. It inherits from UpstreamResourceAdaptor and provides methods to add bins, allocate, deallocate, and get the upstream resource.

### Constructor
`rmm.mr.BinningMemoryResource(upstream_mr, min_size_exponent=-1, max_size_exponent=-1)`

#### Parameters
- **upstream_mr** (DeviceMemoryResource) - Required - The upstream memory resource.
- **min_size_exponent** (int8_t) - Optional - Minimum size exponent.
- **max_size_exponent** (int8_t) - Optional - Maximum size exponent.

### Attributes
- **bin_mrs** (list) - The list of binned memory resources.
- **upstream_mr** - The upstream memory resource.

### Methods

#### add_bin

##### Description
Adds a bin of the specified maximum allocation size to this memory resource. If specified, uses bin_resource for allocation for this bin. If not specified, creates and uses a FixedSizeMemoryResource for allocation for this bin. Allocations smaller than allocation_size and larger than the next smaller bin size will use this fixed-size memory resource.

##### Parameters
- **allocation_size** (size_t) - Required - The maximum allocation size in bytes for the created bin.
- **bin_resource** (DeviceMemoryResource) - Optional - The resource to use for this bin.

#### allocate

##### Description
Allocate `nbytes` bytes of memory.

##### Parameters
- **nbytes** (size_t) - Required - The size of the allocation in bytes.
- **stream** (Stream) - Optional - Optional stream for the allocation.

##### Raises
- **MemoryError** - If allocation fails.

##### Notes
- Allocations are always requested with 256-byte alignment.
- On integrated memory systems, attempting to allocate more memory than available can cause the process to be killed by the operating system instead of raising a catchable `MemoryError`.

#### deallocate

##### Description
Deallocate memory pointed to by `ptr` of size `nbytes`.

##### Parameters
- **ptr** (uintptr_t) - Required - Pointer to be deallocated.
- **nbytes** (size_t) - Required - Size of the allocation in bytes.
- **stream** (Stream) - Optional - Optional stream for the deallocation.

#### get_upstream

##### Description
Returns the upstream memory resource.

##### Returns
- **DeviceMemoryResource** - The upstream memory resource.
```

--------------------------------

### rmm.DeviceBuffer.copy_from_host()

Source: https://docs.nvidia.com/rmm/latest/python/rmm

Copies from a buffer on host to self.

```APIDOC
## rmm.DeviceBuffer.copy_from_host(ary, stream=DEFAULT_STREAM)

### Description
Copy from a buffer on host to self.

### Parameters
- **ary** (bytes-like) - Required - Buffer to copy from.
- **stream** (Stream) - Optional - CUDA stream to use for copying, defaults to the default stream.

### Notes
If stream is the default stream, it is synchronized after the copy. If a non-default stream is provided, the copy is asynchronous. The caller must keep ary alive and unmodified until the stream is synchronized.

### Example
```python
>>> import rmm
>>> db = rmm.DeviceBuffer(size=10)
>>> hb = b"abcdef"
>>> db.copy_from_host(hb)
>>> hb = db.copy_to_host()
>>> print(hb)
array([97, 98, 99,  0,  0,  0,  0,  0,  0,  0], dtype=uint8)
```
```

--------------------------------

### void* data()

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers

Returns a pointer to the device memory allocation.

```APIDOC
## void* data()

### Description
Returns a pointer to the device memory allocation.

### Returns
Pointer to the device memory allocation.
```

--------------------------------

### StatisticsResourceAdaptor.allocation_counts

Source: https://docs.nvidia.com/rmm/latest/python/mr

Gets the current, peak, and total allocated bytes and number of allocations. Returns a dictionary with keys: current_bytes, current_count, peak_bytes, peak_count, total_bytes, total_count.

```APIDOC
## allocation_counts

### Description
Gets the current, peak, and total allocated bytes and number of allocations. The dictionary keys are `current_bytes`, `current_count`, `peak_bytes`, `peak_count`, `total_bytes`, and `total_count`.

### Returns
- **dict** - Dictionary containing allocation counts and bytes.
```

--------------------------------

### begin() noexcept

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers

Returns an iterator to the first element. If the vector is empty, begin() equals end().

```APIDOC
## begin() noexcept

### Description
Returns an iterator to the first element. If the vector is empty, then `begin() == end()`.

### Returns
- **iterator** - Iterator to the first element.
```

--------------------------------

### cuda_async_memory_resource

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources/index.html

Memory resource that uses cudaMallocAsync and cudaFreeAsync for allocation and deallocation. Provides constructors, assignment operators, and a method to get the underlying CUDA pool handle.

```APIDOC
## cuda_async_memory_resource

### Description
Memory resource that uses `cudaMallocAsync`/`cudaFreeAsync` for allocation/deallocation.

### Constructors

#### cuda_async_memory_resource()
Default constructor.

#### cuda_async_memory_resource(cuda_async_memory_resource const&)
Default copy constructor.

#### cuda_async_memory_resource(cuda_async_memory_resource &&)
Default move constructor.

### Assignment Operators

#### operator=(cuda_async_memory_resource const&)
Default copy assignment operator.

**Returns:**
- `cuda_async_memory_resource&` - Reference to the assigned object

#### operator=(cuda_async_memory_resource &&)
Default move assignment operator.

**Returns:**
- `cuda_async_memory_resource&` - Reference to the assigned object

### Destructor

#### ~cuda_async_memory_resource() = default

### Friend Functions

#### get_property(cuda_async_memory_resource const&, cuda::mr::device_accessible)
Enables the `cuda::mr::device_accessible` property.

### Methods

#### pool_handle() const noexcept
Returns the underlying native handle to the CUDA pool.

**Returns:**
- `cudaMemPool_t` - Handle to the underlying CUDA pool
```

--------------------------------

### cuda_stream_pool::get_stream(stream_id)

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams

Gets a cuda::stream_ref of the stream associated with stream_id. Equivalent values of stream_id return a stream_ref to the same underlying stream. This function is thread safe with respect to other calls to the same function.

```APIDOC
## cuda_stream_pool::get_stream(stream_id)

### Description
Gets a cuda::stream_ref of the stream associated with stream_id. Equivalent values of stream_id return a stream_ref to the same underlying stream. This function is thread safe with respect to other calls to the same function.

### Parameters
- **stream_id** (std::size_t) - Required - Unique identifier for the desired stream. Note: stream_id is wrapped around the pool size, therefore any size_t value is allowed.

### Returns
- **cuda::stream_ref**
```

--------------------------------

### rmm.DeviceBuffer.copy_to_host()

Source: https://docs.nvidia.com/rmm/latest/python/rmm

Copies from a DeviceBuffer to a buffer on host.

```APIDOC
## rmm.DeviceBuffer.copy_to_host(ary=None, stream=DEFAULT_STREAM)

### Description
Copy from a DeviceBuffer to a buffer on host.

### Parameters
- **ary** (bytes-like) - Optional - Buffer to write into.
- **stream** (Stream) - Optional - CUDA stream to use for copying, defaults to the default stream.

### Example
```python
>>> import rmm
>>> db = rmm.DeviceBuffer.to_device(b"abc")
>>> hb = bytearray(db.nbytes)
>>> db.copy_to_host(hb)
>>> print(hb)
bytearray(b'abc')
>>> hb = db.copy_to_host()
>>> print(hb)
bytearray(b'abc')
```
```

--------------------------------

### rmm::cuda_stream_pool::get_stream(stream_id)

Source: https://docs.nvidia.com/rmm/latest/cpp/cuda_streams/index.html

Get a cuda::stream_ref of the stream associated with stream_id. Equivalent values of stream_id return a stream_ref to the same underlying stream. This function is thread safe with respect to other calls to the same function.

```APIDOC
## get_stream(stream_id)

### Description
Get a cuda::stream_ref of the stream associated with stream_id. Equivalent values of stream_id return a stream_ref to the same underlying stream. This function is thread safe with respect to other calls to the same function.

### Parameters
- **stream_id** (std::size_t) - Required - Unique identifier for the desired stream. Note: stream_id is wrapped around the pool size, therefore any size_t value is allowed.

### Returns
- cuda::stream_ref

### Example
```cpp
auto stream_ref = pool.get_stream(0);
```
```

--------------------------------

### rmm.pylibrmm.device_buffer.to_device()

Source: https://docs.nvidia.com/rmm/latest/python/pylibrmm

Creates a new DeviceBuffer with a copy of the host data.

```APIDOC
## rmm.pylibrmm.device_buffer.to_device()

### Description
Return a new `DeviceBuffer` with a copy of the data.

### Method
`to_device(const unsigned char[::1] b, Stream stream=DEFAULT_STREAM) -> DeviceBuffer`

### Parameters
- **b** (buffer) - Required - Host data to copy to device.
- **stream** (Stream) - Optional - CUDA stream to use for copying. Defaults to the default stream.

### Returns
- **DeviceBuffer** - Device buffer containing a copy of the host data.

### Notes
If `stream` is the default stream, it is synchronized after the copy. If a non-default `stream` is provided, the copy is asynchronous. The caller must keep `b` alive and unmodified until the stream is synchronized.

### Example
```
>>> import rmm
>>> db = rmm.pylibrmm.device_buffer.to_device(b"abc")
>>> print(bytes(db))
b'abc'
```
```

--------------------------------

### Get log filenames for each device

Source: https://docs.nvidia.com/rmm/latest/python/mr

Shows how to use rmm.get_log_filenames() after reinitializing RMM with logging enabled for devices 0 and 1. The output is a dictionary mapping device IDs to log file paths, with the suffix .dev{id} added to the log file name.

```python
>>> import rmm
>>> rmm.reinitialize(devices=[0, 1], logging=True, log_file_name="rmm.log")
>>> rmm.get_log_filenames()
{0: '/home/user/workspace/rapids/rmm/python/rmm.dev0.log',
 1: '/home/user/workspace/rapids/rmm/python/rmm.dev1.log'}
```

--------------------------------

### Nested memory profiling with rmm.statistics.profiler

Source: https://docs.nvidia.com/rmm/latest/user_guide/guide/index.html

Use rmm.statistics.profiler as a context manager to profile nested code blocks. The example creates two DeviceBuffers inside outer and inner profiler contexts, then prints the report from rmm.statistics.default_profiler_records.report(). The report shows ncalls, memory_peak, and memory_total for each profiled block.

```python
>>> with rmm.statistics.profiler(name="outer"):
...     buf1 = rmm.DeviceBuffer(size=10)
...     with rmm.statistics.profiler(name="inner"):
...         buf2 = rmm.DeviceBuffer(size=10)
>>> print(rmm.statistics.default_profiler_records.report())
Memory Profiling
================

Legends:
  ncalls       - number of times the function or code block was called
  memory_peak  - peak memory allocated in function or code block (in bytes)
  memory_total - total memory allocated in function or code block (in bytes)

Ordered by: memory_peak

ncalls     memory_peak    memory_total  filename:lineno(function)
     1           1,008           1,008  <ipython-input-4-865fbe04e29f>:1(f)
     1              32              32  my code block
     1              32              32  outer
     1              16              16  inner
```

--------------------------------

### host_device_async_resource_ref

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources

Alias for an asynchronous resource reference with both host and device accessibility.

```APIDOC
## host_device_async_resource_ref

### Description
Alias for a `cuda::mr::resource_ref` with the properties `cuda::mr::host_accessible` and `cuda::mr::device_accessible`.

### Type
using host_device_async_resource_ref = cuda::mr::resource_ref<cuda::mr::host_accessible, cuda::mr::device_accessible>
```

--------------------------------

### Making custom resources copyable for any_resource

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

Illustrates how to wrap non-copyable members in std::shared_ptr to satisfy the copyability requirement of cuda::mr::any_resource.

```cpp
// Won't work — shared_mutex is not copyable
class my_tracking_resource {
  std::shared_mutex mutex_;                // NOT copyable
  std::unordered_set<void*> allocations_;  // copyable, but expensive
  // ...
};

// Fix: wrap shared state in shared_ptr
class my_tracking_resource {
  struct shared_state {
    std::shared_mutex mutex;
    std::unordered_set<void*> allocations;
  };
  std::shared_ptr<shared_state> state_{std::make_shared<shared_state>()};
  // Now the resource is copyable; copies share the tracking state
  // ...
};
```

--------------------------------

### device_buffer(void const* source_data, std::size_t size, cuda::stream_ref stream, cuda::mr::any_resource<cuda::mr::device_accessible> mr = mr::get_current_device_resource_ref())

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers/index.html

Constructs a new device buffer by copying from a raw pointer to an existing host or device memory allocation. The copy is performed on the specified stream, and the caller is responsible for synchronization. The buffer is guaranteed to have an alignment of at least rmm::CUDA_ALLOCATION_ALIGNMENT.

```APIDOC
## device_buffer(void const* source_data, std::size_t size, cuda::stream_ref stream, cuda::mr::any_resource<cuda::mr::device_accessible> mr = mr::get_current_device_resource_ref())

### Description
Constructs a new device buffer by copying from a raw pointer to an existing host or device memory allocation. The copy is performed on the specified stream, and the caller is responsible for synchronization. The buffer is guaranteed to have an alignment of at least rmm::CUDA_ALLOCATION_ALIGNMENT.

### Parameters
- **source_data** (void const*) - Required - Pointer to the host or device memory to copy from.
- **size** (std::size_t) - Required - Size in bytes to copy.
- **stream** (cuda::stream_ref) - Required - CUDA stream on which memory may be allocated if the memory resource supports streams.
- **mr** (cuda::mr::any_resource<cuda::mr::device_accessible>) - Optional - Memory resource to use for the device memory allocation. Defaults to mr::get_current_device_resource_ref().

### Throws
- rmm::bad_alloc - If creating the new allocation fails.
- rmm::logic_error - If source_data is null, and size != 0.
- rmm::cuda_error - If copying from the device memory fails.
```

--------------------------------

### Copy a DeviceBuffer

Source: https://docs.nvidia.com/rmm/latest/python/rmm/index.html

Create a deep copy of an existing DeviceBuffer.

```python
>>> import rmm
>>> db = rmm.DeviceBuffer.to_device(b"abc")
>>> db_copy = db.copy()
>>> db.copy_to_host()
array([97, 98, 99], dtype=uint8)
>>> db_copy.copy_to_host()
array([97, 98, 99], dtype=uint8)
>>> assert db is not db_copy
>>> assert db.ptr != db_copy.ptr
```

--------------------------------

### rmm::bad_alloc::what()

Source: https://docs.nvidia.com/rmm/latest/cpp/errors/index.html

Returns the explanatory string for the bad_alloc exception.

```APIDOC
## what()

### Description
The explanatory string.

### Returns
The explanatory string

```

--------------------------------

### ProfilerContext.__call__

Source: https://docs.nvidia.com/rmm/latest/python/statistics/index.html

Use the profiler as a decorator. Wraps a function to profile its memory usage.

```APIDOC
## ProfilerContext.__call__(func)

### Description
Use the profiler as a decorator. Wraps the given function to profile memory allocations.

### Method
Method call

### Endpoint
ProfilerContext.__call__(func)

### Parameters
- **func** (Callable) - Required - The function to profile.

### Returns
The wrapped function.
```

--------------------------------

### using host_device_async_resource_ref

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources/index.html

Alias for a cuda::mr::resource_ref with the properties cuda::mr::host_accessible and cuda::mr::device_accessible.

```APIDOC
## Type Alias: host_device_async_resource_ref

### Description
Alias for a cuda::mr::resource_ref with the properties cuda::mr::host_accessible and cuda::mr::device_accessible.

### Signature
using host_device_async_resource_ref = cuda::mr::resource_ref<cuda::mr::host_accessible, cuda::mr::device_accessible>;
```

--------------------------------

### Implement Custom Resources with CCCL

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

Replace inheritance from device_memory_resource with direct implementation of the CCCL resource concept.

```cpp
// 26.04
#include <rmm/mr/device_memory_resource.hpp>

class my_resource : public rmm::mr::device_memory_resource {
  void* do_allocate(std::size_t bytes, rmm::cuda_stream_view stream) override { ... }
  void do_deallocate(void* ptr, std::size_t bytes, rmm::cuda_stream_view stream) override { ... }
};

// 26.06
#include <cuda/memory_resource>

class my_resource {
 public:
  void* allocate(cuda::stream_ref stream, std::size_t bytes, std::size_t alignment = 256) { ... }
  void deallocate(cuda::stream_ref stream, void* ptr, std::size_t bytes,
                  std::size_t alignment = 256) noexcept { ... }
  bool operator==(my_resource const&) const noexcept { ... }
  bool operator!=(my_resource const& other) const noexcept { return !(*this == other); }

  // Declare properties via ADL friend:
  constexpr friend void get_property(my_resource const&, cuda::mr::device_accessible) noexcept {}
};
static_assert(cuda::mr::resource_with<my_resource, cuda::mr::device_accessible>);
```

--------------------------------

### Configure PoolMemoryResource with CudaMemoryResource

Source: https://docs.nvidia.com/rmm/latest/user_guide/guide/index.html

Create a memory pool using CudaMemoryResource as the upstream resource.

```python
>>> import rmm
>>> pool = rmm.mr.PoolMemoryResource(
...     rmm.mr.CudaMemoryResource(),
...     initial_pool_size="1GiB", # equivalent to initial_pool_size=2**30
...     maximum_pool_size="4GiB"
... )
>>> rmm.mr.set_current_device_resource(pool)
```

--------------------------------

### front_element

Source: https://docs.nvidia.com/rmm/latest/cpp/data_containers/index.html

Returns the first element from device memory. Synchronizes the stream.

```APIDOC
## front_element

### Description
Returns the first element.

### Method
N/A (C++ member function)

### Parameters
#### Parameters
- **stream** (_cuda::stream_ref) - Required - The stream on which to perform the copy

### Throws
- **rmm::out_of_range** - exception if the vector is empty.

### Returns
The value of the first element
```

--------------------------------

### Define Custom Cython Resource Wrappers

Source: https://docs.nvidia.com/rmm/latest/migration_guide_2606/index.html

For custom shared_resource-based types, define an inline C++ helper that returns an optional device_async_resource_ref.

```cython
# my_project/my_resource.pxd
from libcpp.optional cimport optional
from rmm.librmm.memory_resource cimport (
    any_resource,
    device_accessible,
    device_async_resource_ref,
)

cdef extern from "<my_project/my_resource.hpp>" nogil:
    cdef cppclass cpp_MyResource "my_project::MyResource":
        cpp_MyResource(any_resource[device_accessible] upstream) except +
        # ...

cdef extern from *:
    """
    #include <optional>
    #include <rmm/resource_ref.hpp>
    #include <my_project/my_resource.hpp>
    std::optional<rmm::device_async_resource_ref>
    make_my_resource_ref(my_project::MyResource& r) {
        return std::optional<rmm::device_async_resource_ref>(
            rmm::device_async_resource_ref(r));
    }
    """
    optional[device_async_resource_ref] make_my_resource_ref(
        cpp_MyResource&) except +
```

```cython
# my_project/my_resource.pyx
self.c_obj.reset(new cpp_MyResource(upstream_mr.get_mr()))
self.c_ref = make_my_resource_ref(deref(self.c_obj))
```

--------------------------------

### host_device_resource_ref

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources

Alias for a synchronous resource reference with both host and device accessibility.

```APIDOC
## host_device_resource_ref

### Description
Alias for a `cuda::mr::synchronous_resource_ref` with the properties `cuda::mr::host_accessible` and `cuda::mr::device_accessible`.

### Type
using host_device_resource_ref = cuda::mr::synchronous_resource_ref<cuda::mr::host_accessible, cuda::mr::device_accessible>
```

--------------------------------

### system_memory_resource constructor

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resources/index.html

Default constructor for system_memory_resource.

```APIDOC
## system_memory_resource()

### Description
Default constructor.
```

--------------------------------

### tracking_resource_adaptor::get_outstanding_allocations_str

Source: https://docs.nvidia.com/rmm/latest/cpp/memory_resources/memory_resource_adaptors/index.html

Returns a string describing all outstanding allocations.

```APIDOC
## tracking_resource_adaptor::get_outstanding_allocations_str

### Description
Gets a string describing all outstanding allocations (pointer, size, optional stack).

### Method
std::string get_outstanding_allocations_str() const

### Returns
std::string describing outstanding allocations
```

--------------------------------

### rmm.DeviceBuffer.copy_from_device()

Source: https://docs.nvidia.com/rmm/latest/python/rmm

Copies from a buffer on device to self.

```APIDOC
## rmm.DeviceBuffer.copy_from_device(cuda_ary, stream=DEFAULT_STREAM)

### Description
Copy from a buffer on host to self.

### Parameters
- **cuda_ary** (object) - Required - Object to copy from that has __cuda_array_interface__.
- **stream** (Stream) - Optional - CUDA stream to use for copying, defaults to the default stream.

### Example
```python
>>> import rmm
>>> db = rmm.DeviceBuffer(size=5)
>>> db2 = rmm.DeviceBuffer.to_device(b"abc")
>>> db.copy_from_device(db2)
>>> hb = db.copy_to_host()
>>> print(hb)
array([97, 98, 99,  0,  0], dtype=uint8)
```
```
Tokens:
Raw