### CMake Project Setup and Package Finding

Source: https://github.com/rocm/rocshmem/blob/develop/examples/CMakeLists.txt

This snippet demonstrates the initial CMake configuration, including setting the minimum required version, including setup scripts, defining the project name and version, and finding essential packages like MPI, HIP, and rocshmem. It ensures that the rocshmem library is available for linking.

```CMake
cmake_minimum_required(VERSION 3.16.3 FATAL_ERROR)

include(${CMAKE_SOURCE_DIR}/cmake/setup_project.cmake)
project(rocshmem_examples VERSION 1.0.0 LANGUAGES CXX)

find_package(MPI REQUIRED)
find_package(hip REQUIRED PATHS /opt/rocm)
if(NOT TARGET roc::rocshmem)
  find_package(rocshmem REQUIRED PATHS /opt/rocm)
endif()
```

--------------------------------

### Run rocSHMEM Example

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Demonstrates how to execute a rocSHMEM example using mpirun. This command maps processes by NUMA, utilizes UCX for PML and OSC, and runs a specific rocSHMEM test executable.

```bash
mpirun --map-by numa --mca pml ucx --mca osc ucx -np 2 ./build/examples/rocshmem_getmem_test
```

--------------------------------

### CMake Executable Target Creation and Linking

Source: https://github.com/rocm/rocshmem/blob/develop/examples/CMakeLists.txt

This snippet shows how to iterate through a list of example source files, extract the executable name from each, and create a corresponding executable target. It then links each executable to the necessary libraries, including MPI and rocshmem, enabling the examples to run.

```CMake
set(EXAMPLE_SOURCES
  rocshmem_allreduce_test.cc
  rocshmem_alltoall_test.cc
  rocshmem_broadcast_test.cc
  rocshmem_getmem_test.cc
  rocshmem_put_signal_test.cc
  rocshmem_init_attr_test.cc
)

foreach(SOURCE_FILE IN LISTS EXAMPLE_SOURCES)
  get_filename_component(EXECUTABLE_NAME ${SOURCE_FILE} NAME_WE)

  add_executable(${EXECUTABLE_NAME} ${SOURCE_FILE})

  target_link_libraries(
    ${EXECUTABLE_NAME}
    PRIVATE
      MPI::MPI_CXX
      roc::rocshmem
  )
endforeach()
```

--------------------------------

### rocSHMEM Project Setup and Versioning

Source: https://github.com/rocm/rocshmem/blob/develop/CMakeLists.txt

Initializes the rocSHMEM project with CMake, sets the project name and version, and includes necessary ROCm build tools and scripts for package creation and target installation. It parses the version string from the header file.

```CMake
include(${CMAKE_SOURCE_DIR}/cmake/setup_project.cmake)

## Setup VERSION
file(READ include/rocshmem/rocshmem.hpp header_text)
if("${header_text}" MATCHES "constexpr char VERSION\[\] *= \"([0-9]+)\.([0-9]+)\.([0-9]+)\";")
  set(VERSION_STRING ${CMAKE_MATCH_1}.${CMAKE_MATCH_2}.${CMAKE_MATCH_3})
else()
  message(FATAL_ERROR "Failed to parse Version")
endif()
message(STATUS "rocSHMEM Version: " "${VERSION_STRING}")

project(rocshmem VERSION ${VERSION_STRING} LANGUAGES CXX)

find_package(ROCmCMakeBuildTools PATHS /opt/rocm)
include(ROCMCreatePackage)
include(ROCMInstallTargets)
include(ROCMCheckTargetIds)

rocm_setup_version(VERSION ${VERSION_STRING})
```

--------------------------------

### Install Dependencies using Script

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

Installs necessary dependencies for rocSHMEM using a provided script. This is an alternative to manually building UCX and Open MPI. Users should review the script for compatibility with their system.

```bash
export BUILD_DIR=/path/to/not_rocshmem_src_or_build/dependencies
/path/to/rocshmem_src/scripts/install_dependencies.sh
```

--------------------------------

### Build and Install Open MPI with UCX and ROCm Support

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

Builds and installs Open MPI version 5.0.7 or later with support for UCX and ROCm. This is necessary for inter-node communication with rocSHMEM. The process includes cloning the Open MPI repository, configuring it with ROCm and UCX paths, and then compiling and installing.

```bash
git clone --recursive https://github.com/open-mpi/ompi.git -b v5.0.x
cd ompi
./autogen.pl
./configure --prefix=<prefix_dir> --with-rocm=<rocm_path> --with-ucx=<ucx_path>
make -j 8
make -j 8 install
```

--------------------------------

### Add Test and Example Subdirectories

Source: https://github.com/rocm/rocshmem/blob/develop/CMakeLists.txt

Includes subdirectories for tests and examples. The examples subdirectory is conditionally added only if the BUILD_EXAMPLES variable is set to true.

```cmake
###############################################################################
# TEST SUBDIRECTORIES
###############################################################################
add_subdirectory(tests)

if (BUILD_EXAMPLES)
  add_subdirectory(examples)
endif()
```

--------------------------------

### Build and Install UCX with ROCm Support

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

Builds and installs UCX (Unified Communication X) version 1.17.0 or later with ROCm support. This is a prerequisite for ROCm-aware Open MPI and rocSHMEM. The process involves cloning the UCX repository, configuring it with ROCm paths, and then compiling and installing.

```bash
git clone https://github.com/ROCm/ucx.git -b v1.17.x
cd ucx
./autogen.sh
./configure --prefix=<prefix_dir> --with-rocm=<rocm_path> --enable-mt
make -j 8
make -j 8 install
```

--------------------------------

### Set up Project and Define rocshmem Project

Source: https://github.com/rocm/rocshmem/blob/develop/tests/functional_tests/CMakeLists.txt

Initializes the CMake build system and defines the project name and version for rocshmem. It includes a setup script for project configuration.

```CMake
cmake_minimum_required(VERSION 3.16.3 FATAL_ERROR)

###############################################################################
# PROJECT
###############################################################################
include(${CMAKE_SOURCE_DIR}/cmake/setup_project.cmake)
project(rocshmem_functional_tests VERSION 1.0.0 LANGUAGES CXX)
```

--------------------------------

### HIP Host API Example - rocSHMEM

Source: https://github.com/rocm/rocshmem/blob/develop/docs/introduction.rst

Illustrates how to use rocSHMEM host APIs from the CPU to manage GPU-side communication. These functions typically initiate or synchronize GPU operations.

```HIP
/* Example of a __host__ function using rocSHMEM */
void host_communication_routine(...) {
    // Example: Initialize rocSHMEM
    rocshmem_init();
    // ... setup communication ...
    // Example: Launch a kernel that uses rocSHMEM
    my_kernel<<<gridDim, blockDim>>>(...);
    // Example: Synchronize host with device operations
    hipDeviceSynchronize();
    // ... cleanup ...
    rocshmem_finalize();
}
```

--------------------------------

### Initialize rocSHMEM Library (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Initializes the rocSHMEM library and its underlying transport layer on the host. Requires `hipSetDevice` to be called beforehand to associate the PE with a specific device.

```cpp
__host__ void rocshmem_init(void)
```

--------------------------------

### Finalize rocSHMEM Library (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Finalizes the rocSHMEM library on the host, releasing associated resources.

```cpp
__host__ void rocshmem_finalize(void)
```

--------------------------------

### Custom Installation Path for rocSHMEM

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Demonstrates how to specify a custom installation path when building rocSHMEM using the `ipc_single` configuration script. This allows users to install the library in a non-default location.

```bash
../scripts/build_configs/ipc_single /path/to/install
```

--------------------------------

### Install rocSHMEM via apt (Ubuntu)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

Installs the rocSHMEM development package on Ubuntu systems. This method requires ROCm 6.4 or later and assumes manual building of dependencies like Open MPI and UCX with accelerator support.

```bash
apt install rocshmem-dev
```

--------------------------------

### Build HTML Documentation (macOS)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/README.md

Builds HTML documentation locally on macOS. Requires Doxygen and Sphinx-doc to be installed via Homebrew, and Python packages listed in requirements.txt.

```shell
brew install doxygen sphinx-doc
pip3.10 install -r ./requirements.txt
python3.10 -m sphinx -T -E -b html -d _build/doctrees -D language=en . _build/html
open _build/html/index.html
```

--------------------------------

### Set Unique ID Arguments for rocSHMEM Initialization (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Initializes the `rocshmem_init_attr_t` structure with rank, total PE count, and a unique ID, preparing for group-based initialization.

```cpp
__host__ int rocshmem_set_attr_uniqueid_args(int rank, int nranks, rocshmem_uniqueid_t *uid, rocshmem_init_attr_t *attr)
```

--------------------------------

### CMake Project Setup for rocshmem Unit Tests

Source: https://github.com/rocm/rocshmem/blob/develop/tests/unit_tests/CMakeLists.txt

This snippet sets up the CMake project for rocshmem unit tests, specifying the minimum required CMake version, project name, version, and language. It also includes a setup script for project configuration.

```CMake
cmake_minimum_required(VERSION 3.16.3 FATAL_ERROR)

###############################################################################
# PROJECT
###############################################################################
include(${CMAKE_SOURCE_DIR}/cmake/setup_project.cmake)
project(rocshmem_unit_tests VERSION 1.0.0 LANGUAGES CXX)
```

--------------------------------

### Set up rocSHMEM development environment

Source: https://github.com/rocm/rocshmem/blob/develop/CONTRIBUTING.md

Instructions for forking the rocSHMEM repository, cloning it locally, and adding the original repository as a remote named 'mainline'. This setup is crucial for tracking upstream changes and contributing effectively.

```git
git remote add mainline https://github.com/ROCm/rocSHMEM.git
git checkout dev
```

--------------------------------

### Build rocSHMEM with RO and IPC Backends

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

Builds and installs rocSHMEM with both the RO (Reverse Offload) for inter-node and IPC (Inter-Process Communication) for intra-node communication backends enabled. This configuration uses IPC for same-host communication and RO for cross-host communication.

```bash
git clone git@github.com:ROCm/rocSHMEM.git
cd rocSHMEM
mkdir build
cd build
../scripts/build_configs/ro_ipc
```

--------------------------------

### Initialize rocSHMEM with Attributes (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Initializes the rocSHMEM runtime and transport layer using specified flags and attributes. Supports initialization with a unique ID or an MPI communicator.

```cpp
__host__ int rocshmem_init_attr(unsigned int flags, rocshmem_init_attr_t *attr)
```

--------------------------------

### Get Unique ID for rocSHMEM Group (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Retrieves a unique identifier for a rocSHMEM group, used for communication coordination.

```cpp
__host__ int rocshmem_get_uniqueid(rocshmem_uniqueid_t *uid)
```

--------------------------------

### Initialize rocSHMEM Resources (Device Work-Group)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Initializes device-side rocSHMEM resources for a work-group. Must be called collectively by all threads in the work-group before any other rocSHMEM functions are invoked.

```cpp
__device__ void rocshmem_wg_init(void)
```

--------------------------------

### Build PDF Documentation (macOS)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/README.md

Builds PDF documentation on macOS, requiring a LaTeX installation. It involves installing Python dependencies and using sphinx-build to generate the LaTeX output.

```shell
pip3.10 install -r ./requirements.txt
sphinx-build -M latexpdf . _build
open _build/latex/rocshmem.pdf
```

--------------------------------

### Customize ROCshmem Installation Path

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

This snippet demonstrates how to change the default installation directory for ROCshmem. By providing a desired path as a parameter to the build script, users can relocate the library to a custom location.

```bash
../scripts/build_configs/ro_ipc /path/to/install
```

--------------------------------

### Install rocSHMEM Dependencies Script

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Invokes a script to install rocSHMEM dependencies. The script's behavior and configuration options are platform-dependent, and it's recommended to run it from outside the rocSHMEM source or build directories.

```bash
BUILD_DIR=/path/to/not_rocshmem_src_or_build/dependencies /path/to/rocshmem_src/sripts/install_dependencies.sh
```

--------------------------------

### Build rocSHMEM with IPC Only Backend

Source: https://github.com/rocm/rocshmem/blob/develop/docs/install.rst

Builds and installs rocSHMEM with only the IPC (Inter-Process Communication) backend enabled for GPU-to-GPU communication on the same node. This configuration is similar to the default build in ROCm 6.4.

```bash
git clone git@github.com:ROCm/rocSHMEM.git
cd rocSHMEM
mkdir build
cd build
../scripts/build_configs/ipc_single
```

--------------------------------

### Run rocSHMEM Application with mpiexec

Source: https://github.com/rocm/rocshmem/blob/develop/docs/compile_and_run.rst

Launches a rocSHMEM application using mpiexec, specifying the number of processes and MPI configurations for optimal GPU communication. This example launches two instances of a rocshmem_getmem_test application.

```bash
mpiexec --map-by numa --mca pml ucx --mca osc ucx -np 2 ./build/examples/rocshmem_getmem_test
```

--------------------------------

### Query Total Number of PEs (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Queries the total number of processing elements (PEs) available in the rocSHMEM environment. Can be called before `rocshmem_init`.

```cpp
__host__ int rocshmem_n_pes(void)
```

--------------------------------

### Finalize rocSHMEM Resources (Device Work-Group)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Finalizes device-side rocSHMEM resources for a work-group. Must be called collectively by all threads in the work-group if `rocshmem_wg_init` was previously called.

```cpp
__device__ void rocshmem_wg_finalize(void)
```

--------------------------------

### HIP Device API Example - rocSHMEM

Source: https://github.com/rocm/rocshmem/blob/develop/docs/introduction.rst

Demonstrates the usage of rocSHMEM device APIs within HIP kernels. These APIs are designed for communication between GPU threads and require careful parameter consistency across threads in a wavefront or workgroup.

```HIP
/* Example of a __device__ function using rocSHMEM */
__global__ void my_kernel(...) {
    // ... computation ...
    // Example: Symmetric put operation
    rocshmem_put(&data_on_gpu_A, &data_to_send, size, target_gpu_id);
    // Example: Wavefront barrier
    rocshmem_wg_barrier_all(); 
    // ... more computation ...
}
```

--------------------------------

### Set Dependency Paths for rocSHMEM Build

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Configures environment variables to help the rocSHMEM build process locate dependencies like OpenMPI and UCX when they are installed in non-standard paths. It also sets the ROCm installation path.

```bash
MPI_ROOT=/path/to/openmpi UCX_ROOT=/path/to/ucx CMAKE_PREFIX_PATH="/path/to/rocm:$CMAKE_PREFIX_PATH" ../script/build_configs/ipc_single /path/to/install
```

--------------------------------

### Query Caller's PE ID (Host)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Queries the processing element (PE) identifier of the calling process on the host. Can be called before `rocshmem_init`.

```cpp
__host__ int rocshmem_my_pe(void)
```

--------------------------------

### Build ROCm-Aware UCX

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Clones the UCX repository, configures it with ROCm support and multi-threading enabled, and then builds and installs it. Requires UCX version 1.17.0 or later.

```bash
git clone https://github.com/openucx/ucx.git -b v1.17.x
cd ucx
./autogen.sh
./configure --prefix=<ucx_install_dir> --with-rocm=<rocm_path> --enable-mt
make -j 8
make -j 8 install
```

--------------------------------

### Build ROCm-Aware Open MPI with UCX

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Clones the Open MPI repository recursively, configures it with ROCm and UCX support, and then builds and installs it. Requires Open MPI version 5.0.6 or later.

```bash
git clone --recursive https://github.com/open-mpi/ompi.git -b v5.0.x
cd ompi
./autogen.pl
./configure --prefix=<ompi_install_dir> --with-rocm=<rocm_path> --with-ucx=<ucx_install_dir>
make -j 8
make -j 8 install
```

--------------------------------

### Query Total Number of PEs (Device Context)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Queries the total number of processing elements (PEs) for a specific device-side context. Can be called per thread without performance penalty.

```cpp
__device__ int rocshmem_n_pes(void)
__device__ int rocshmem_ctx_n_pes(rocshmem_ctx_t ctx)
```

--------------------------------

### Query Caller's PE ID (Device Context)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/init.rst

Queries the processing element (PE) identifier of the calling thread within a specific device-side context. Can be called per thread without performance penalty.

```cpp
__device__ int rocshmem_my_pe(void)
__device__ int rocshmem_ctx_my_pe(rocshmem_ctx_t ctx)
```

--------------------------------

### rocSHMEM CMake Configuration Options

Source: https://github.com/rocm/rocshmem/blob/develop/CMakeLists.txt

Defines various boolean options to customize the rocSHMEM build. These options control features such as debugging, profiling, memory allocation strategies (device and host), thread usage, and build targets like tests and examples.

```CMake
option(DEBUG "Enable debug trace" OFF)
option(PROFILE "Enable statistics and timing support" OFF)
option(USE_RO "Enable RO conduit." ON)
option(USE_IPC "Enable IPC support (using HIP)" OFF)
option(USE_THREADS "Enable workgroup threads to share network queues" OFF)
option(USE_WF_COAL "Enable wavefront message coalescing" OFF)
option(USE_HEAP_DEVICE_FINEGRAIN "Heap uses GPU memory in finegrain mode" ON)
option(USE_HEAP_DEVICE_UNCACHED "Heap uses GPU memory in uncached mode" OFF)
option(USE_HEAP_DEVICE_COARSEGRAIN "Heap uses GPU memory in coarsegrain mode" OFF)
option(USE_HEAP_MANAGED "Heap uses managed memory" OFF)
option(USE_HEAP_HOST_HIP "Heap uses pinned host memory allocated with hip api" OFF)
option(USE_HEAP_HOST "Heap uses host memory allocated with malloc/free" OFF)
option(USE_ALLOC_DLMALLOC "Enable dlmalloc device memory allocator" ON)
option(USE_ALLOC_POW2BINS "Enable legacy Pow2Bins device memory allocator" OFF)
option(USE_FUNC_CALL "Force compiler to use function calls on library API" OFF)
option(USE_SHARED_CTX "Request support for shared ctx between WG" OFF)
option(USE_SINGLE_NODE "Enable single node support only." OFF)
option(USE_HDP_FLUSH "Force flush the HDP cache." OFF)
option(USE_HDP_FLUSH_HOST_SIDE "Use a polling thread to flush the HDP cache on the host." OFF)

option(BUILD_FUNCTIONAL_TESTS "Build the functional tests" OFF)
option(BUILD_EXAMPLES "Build the examples" ON)
option(BUILD_UNIT_TESTS "Build the unit tests" OFF)
option(BUILD_TESTS_ONLY "Build only tests. Used to link agains rocSHMEM in a ROCm Release" OFF)
option(BUILD_TOOLS "Build binary tools (e.g., rocshmem_info)" ON)

option(BUILD_LOCAL_GPU_TARGET_ONLY "Build only for GPUs detected on this machine" OFF)
option(BUILD_CODE_COVERAGE "Build with code coverage flags (gcc only)" OFF)
```

--------------------------------

### CMake: Get Git Hash and Set Compile Definitions

Source: https://github.com/rocm/rocshmem/blob/develop/src/tools/CMakeLists.txt

This snippet demonstrates how to execute a bash command to get the current Git commit hash and use it to set compile definitions for the 'rocshmem_info' executable. It also strips whitespace from the hash and processes offload targets.

```cmake
execute_process (
  COMMAND bash -c "git -C ${PROJECT_SOURCE_DIR} rev-parse HEAD;"
  OUTPUT_VARIABLE GIT_HASH
)

string(STRIP "${GIT_HASH}" GIT_HASH)
string(REPLACE ";" " " ROCSHMEM_OFFLOAD_TARGETS "${COMPILING_TARGETS}")

add_executable(rocshmem_info rocshmem_info.cpp)

target_compile_definitions(
  rocshmem_info
  PRIVATE
  ROCSHMEM_INSTALL_PREFIX="${CMAKE_INSTALL_PREFIX}"
  ROCSHMEM_GIT_HASH="${GIT_HASH}"
  ROCSHMEM_OFFLOAD_TARGETS="${ROCSHMEM_OFFLOAD_TARGETS}"
)
```

--------------------------------

### Generate rocSHMEM Code Coverage Report

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Builds rocSHMEM, runs tests, and generates a code coverage report. This involves changing to the build directory, creating it if necessary, and running a helper script.

```bash
cd rocSHMEM
mkdir build && cd build
../scripts/build_configs/codecov
```

--------------------------------

### Update PATH and LD_LIBRARY_PATH

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Sets the PATH and LD_LIBRARY_PATH environment variables to include the bin and lib directories of the installed Open MPI and UCX, respectively. This is crucial for using the compiled libraries.

```bash
export PATH=<ompi_install_dir>/bin:$PATH
export LD_LIBRARY_PATH=<ompi_install_dir>/lib:<ucx_install_dir>/lib:$LD_LIBRARY_PATH
```

--------------------------------

### Compile rocSHMEM Application with hipcc

Source: https://github.com/rocm/rocshmem/blob/develop/docs/compile_and_run.rst

Compiles a rocSHMEM application source file using hipcc, including necessary include paths for rocSHMEM and MPI. This step creates an object file.

```bash
hipcc -c -fgpu-rdc -x hip rocshmem_allreduce_test.cc \
    -I/opt/rocm/include \
    -I$ROCSHMEM_INSTALL_DIR/include \
    -I$OPENMPI_UCX_INSTALL_DIR/include/
```

--------------------------------

### Link rocSHMEM Application with hipcc

Source: https://github.com/rocm/rocshmem/blob/develop/docs/compile_and_run.rst

Links the compiled object file to create an executable, including the rocSHMEM static library and the MPI shared library. It specifies library paths and necessary ROCm libraries.

```bash
hipcc -fgpu-rdc --hip-link rocshmem_allreduce_test.o -o rocshmem_allreduce_test \
    $ROCSHMEM_INSTALL_DIR/lib/librocshmem.a \
    $OPENMPI_UCX_INSTALL_DIR/lib/libmpi.so \
    -L/opt/rocm/lib -lamdhip64 -lhsa-runtime64
```

--------------------------------

### CMake: Set Include Directories and Link Libraries

Source: https://github.com/rocm/rocshmem/blob/develop/src/tools/CMakeLists.txt

This snippet shows how to configure the include directories and link the 'rocshmem' library to the 'rocshmem_info' executable. This ensures that the compiler can find the necessary header files and that the executable is linked against the rocshmem library.

```cmake
target_include_directories(
  rocshmem_info
  PRIVATE
    roc::rocshmem
)

target_link_libraries(
  rocshmem_info
  PRIVATE
    roc::rocshmem
)
```

--------------------------------

### Add ROCSHMEM Source Files (CMake)

Source: https://github.com/rocm/rocshmem/blob/develop/src/sync/CMakeLists.txt

This CMake snippet demonstrates how to add source files to a project. It uses the `target_sources` command to include `abql_block_mutex.cpp` as a private source file for the `${PROJECT_NAME}` target.

```cmake
target_sources(
  ${PROJECT_NAME}
  PRIVATE
    abql_block_mutex.cpp
)
```

--------------------------------

### Build rocSHMEM IPC Backend (Single-Node)

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Creates an out-of-source build for the IPC backend of rocSHMEM, suitable for single-node use cases. This involves creating a build directory and executing a build configuration script.

```bash
mkdir build
cd build
../scripts/build_configs/ipc_single
```

--------------------------------

### Get Number of PEs in a rocSHMEM Team (C++)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/teams.rst

Queries the total number of processing elements (PEs) present in a given rocSHMEM team. This is useful for determining the size of the communication group. It takes the team handle as input and returns the number of PEs.

```cpp
__host__ int rocshmem_team_n_pes(rocshmem_team_t team)
```

--------------------------------

### Adding Executable and Source Files for rocshmem Unit Tests

Source: https://github.com/rocm/rocshmem/blob/develop/tests/unit_tests/CMakeLists.txt

This CMake code defines the main executable for the unit tests and adds all the necessary C++ source files. It also includes target-specific include directories.

```CMake
add_executable(${PROJECT_NAME} "")

target_include_directories(
  ${PROJECT_NAME}
  PRIVATE
    ${CMAKE_CURRENT_SOURCE_DIR}
)

target_sources(
  ${PROJECT_NAME}
  PRIVATE
    shmem_gtest.cpp
    heap_memory_gtest.cpp
    hipmalloc_gtest.cpp
    bin_gtest.cpp
    binner_gtest.cpp
    #bitwise_gtest.cpp # Test is disabled because of compilation errors
    address_record_gtest.cpp
    index_strategy_gtest.cpp
    single_heap_gtest.cpp
    symmetric_heap_gtest.cpp
    pow2_bins_gtest.cpp
    dlmalloc_gtest.cpp
    remote_heap_info_gtest.cpp
    mpi_instance_gtest.cpp
    abql_block_mutex_gtest.cpp
    notifier_gtest.cpp
    free_list_gtest.cpp
    wavefront_size_gtest.cpp
    atomic_wf_queue_gtest.cpp
)

if (USE_IPC)
  target_sources(
    ${PROJECT_NAME}
    PRIVATE
      ipc_impl_simple_coarse_gtest.cpp
      ipc_impl_simple_fine_gtest.cpp
      ipc_impl_tiled_fine_gtest.cpp
  )
endif()
```

--------------------------------

### Get PE ID in a rocSHMEM Team (C++)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/teams.rst

Retrieves the PE ID of the calling process within a specified rocSHMEM team. This function is essential for identifying the rank of a process within a communication group. It takes the team handle as input and returns the PE ID.

```cpp
__host__ int rocshmem_team_my_pe(rocshmem_team_t team)
```

--------------------------------

### Split rocSHMEM Team with Strided Pattern (C++)

Source: https://github.com/rocm/rocshmem/blob/develop/docs/api/teams.rst

Creates a new rocSHMEM team by splitting an existing team based on a strided pattern. This allows for creating subgroups within a larger communication group. It requires the parent team, start index, stride, size, configuration, and a pointer to store the new team handle.

```cpp
__host__ int rocshmem_team_split_strided(rocshmem_team_t parent_team, int start, int stride, int size, const rocshmem_team_config_t *config, long config_mask, rocshmem_team_t *new_team)
```

--------------------------------

### Run rocSHMEM Unit Tests

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Executes the unit tests for rocSHMEM. This script runs all unit tests, specifying the test executable.

```bash
./scripts/unit_tests/driver.sh ./build/tests/unit_tests/rocshmem_unit_tests all
```

--------------------------------

### Integrating Google Test Framework with CMake

Source: https://github.com/rocm/rocshmem/blob/develop/tests/unit_tests/CMakeLists.txt

This CMake snippet demonstrates how to fetch and integrate the Google Test framework (release-1.12.0) into the project. It configures Google Test to be built as a shared library and links it to the test executable.

```CMake
###############################################################################
# GTEST DEPENDENCY
###############################################################################
include(FetchContent)

FetchContent_Declare(
    googletest
    PREFIX extern/googletest
    GIT_REPOSITORY https://github.com/google/googletest.git
    GIT_TAG release-1.12.0
)

set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
set(BUILD_GMOCK OFF CACHE BOOL "" FORCE)
set(BUILD_GTEST ON CACHE BOOL "" FORCE)
set(INSTALL_GTEST OFF CACHE BOOL "" FORCE)

FetchContent_MakeAvailable(googletest)

target_link_libraries(
  ${PROJECT_NAME}
  PRIVATE
    gtest
    gtest_main
)
```

--------------------------------

### Add Executable and Source Files for rocshmem Tests

Source: https://github.com/rocm/rocshmem/blob/develop/tests/functional_tests/CMakeLists.txt

Defines the main executable for the rocshmem functional tests and adds all the necessary C++ source files to be compiled. It also sets up include directories for the project.

```CMake
add_executable(${PROJECT_NAME} "")

target_include_directories(
  ${PROJECT_NAME}
  PRIVATE
    ${CMAKE_CURRENT_SOURCE_DIR}
)

target_sources(
  ${PROJECT_NAME}
  PRIVATE
    barrier_all_tester.cpp
    sync_tester.cpp
    test_driver.cpp
    tester.cpp
    tester_arguments.cpp
    ping_pong_tester.cpp
    ping_all_tester.cpp
    primitive_tester.cpp
    primitive_mr_tester.cpp
    default_ctx_primitive_tester.cpp
    team_ctx_primitive_tester.cpp
    team_ctx_infra_tester.cpp
    amo_bitwise_tester.cpp
    amo_extended_tester.cpp
    amo_standard_tester.cpp
    random_access_tester.cpp
    shmem_ptr_tester.cpp
    signaling_operations_tester.cpp
    signaling_operations_tester.hpp
    workgroup_primitives.cpp
    empty_tester.cpp
    wavefront_primitives.cpp
)
```

--------------------------------

### Build rocSHMEM RO/IPC Backend (Multi-Node)

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Creates an out-of-source build for the Reverse Offload (RO) backend of rocSHMEM, designed for multi-node use cases. This configuration can also leverage IPC for intra-node operations.

```bash
mkdir build
cd build
../scripts/build_configs/ro_ipc
```

--------------------------------

### Create rocshmem Library and Dependencies

Source: https://github.com/rocm/rocshmem/blob/develop/CMakeLists.txt

Defines the rocshmem library and its alias. It finds and links necessary packages such as MPI, HIP, and hsa-runtime64. It also configures C++ threading preferences and includes a custom configuration header.

```cmake
if (NOT BUILD_TESTS_ONLY)
  add_library(${PROJECT_NAME})
  add_library(roc::${PROJECT_NAME} ALIAS ${PROJECT_NAME})
  add_subdirectory(src)

  #############################################################################
  # PACKAGE DEPENDENCIES
  #############################################################################
  find_package(MPI REQUIRED)
  find_package(hip REQUIRED PATHS /opt/rocm)
  find_package(hsa-runtime64 REQUIRED)

  set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
  set(THREADS_PREFER_PTHREAD_FLAG TRUE)
  find_package(Threads REQUIRED)

  configure_file(cmake/rocshmem_config.h.in include/rocshmem/rocshmem_config.h)

  #############################################################################
  # LINKING AND INCLUDE DIRECTORIES
  #############################################################################
  target_compile_options(
    ${PROJECT_NAME}
    PUBLIC
      ${offload_flags}
      -fgpu-rdc
  )

  target_include_directories(
    ${PROJECT_NAME}
    PUBLIC
      $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
      $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include>            # rocshmem_config.h
      $<BUILD_INTERFACE:${CMAKE_BINARY_DIR}/include/rocshmem>   # rocshmem_config.h from rocshmem.hpp
      $<INSTALL_INTERFACE:include>
  )

  target_link_libraries(
    ${PROJECT_NAME}
    PUBLIC
      Threads::Threads
      MPI::MPI_CXX
      hip::device
      hip::host
      hsa-runtime64::hsa-runtime64
      -fgpu-rdc
  )

  #############################################################################
  # INSTALL
  #############################################################################
  include(ROCMInstallTargets)
  include(ROCMCreatePackage)

  rocm_install(TARGETS rocshmem)

  rocm_install(
    DIRECTORY ${CMAKE_SOURCE_DIR}/include/
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
  )

  rocm_install(
    FILES "${CMAKE_BINARY_DIR}/include/rocshmem/rocshmem_config.h"
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/rocshmem
  )

  rocm_install(
    PROGRAMS "${CMAKE_BINARY_DIR}/src/tools/rocshmem_info"
    DESTINATION ${CMAKE_INSTALL_BINDIR}
  )

  rocm_package_add_dependencies(
    DEPENDS
      hsa-rocr
      hip-runtime-amd
      rocm-dev
  )

  rocm_export_targets(
    TARGETS roc::rocshmem
    NAMESPACE roc::
  )

  rocm_create_package(
    NAME "rocSHMEM"
    DESCRIPTION "ROCm OpenSHMEM (rocSHMEM)"
    MAINTAINER "rocSHMEM Maintainer <rocshmem-maintainer@amd.com>"
  )
endif (NOT BUILD_TESTS_ONLY)
```

--------------------------------

### Link rocSHMEM Application with hipcc

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Links the compiled object file (`rocshmem_allreduce_test.o`) into an executable (`rocshmem_allreduce_test`) using `hipcc`. It specifies the rocSHMEM library, MPI library, and necessary ROCm libraries.

```c++
# Link
hipcc -fgpu-rdc --hip-link rocshmem_allreduce_test.o -o rocshmem_allreduce_test \
  $ROCSHMEM_INSTALL_DIR/lib/librocshmem.a \
  $OPENMPI_UCX_INSTALL_DIR/lib/libmpi.so \
  -L/opt/rocm/lib -lamdhip64 -lhsa-runtime64
```

--------------------------------

### Run rocSHMEM Functional Tests

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Executes the functional tests for rocSHMEM. This script runs all functional tests, specifying the test executable and a directory for log output.

```bash
./scripts/functional_tests/driver.sh ./build/tests/functional_tests/rocshmem_functional_tests all <log_directory>
```

--------------------------------

### Compile rocSHMEM Application with hipcc

Source: https://github.com/rocm/rocshmem/blob/develop/README.md

Compiles a C++ source file (`rocshmem_allreduce_test.cc`) for a rocSHMEM application using `hipcc`. It includes necessary header directories for rocSHMEM, ROCm, and OpenMPI/UCX.

```c++
# Compile
hipcc -c -fgpu-rdc -x hip rocshmem_allreduce_test.cc \
  -I/opt/rocm/include \
  -I$ROCSHMEM_INSTALL_DIR/include \
  -I$OPENMPI_UCX_INSTALL_DIR/include/
```

--------------------------------

### Add ROCSHMEM Target Sources (CMake)

Source: https://github.com/rocm/rocshmem/blob/develop/src/ipc/CMakeLists.txt

This CMake code snippet defines the source files for the ROCSHMEM project. It uses the `target_sources` command to add private source files, including those related to IPC device context, IPC host context, backend IPC, IPC teams, and device collective operations.

```cmake
target_sources(
  ${PROJECT_NAME}
  PRIVATE
    context_ipc_device.cpp
    context_ipc_host.cpp
    backend_ipc.cpp
    ipc_team.cpp
    context_ipc_device_coll.cpp
)
```