### Install hermes-shm for Developers Source: https://github.com/grc-iit/hermes-shm/blob/main/readme.md Install dependencies and compile the library from source. ```bash git clone https://github.com/grc-iit/grc-repo.git spack repo add grc-repo spack install hermes_shm +nocompile spack load hermes_shm +nocompile ``` ```bash git clone https://github.com/grc-iit/hermes-shm.git cd hermes-shm mkdir build cd build cmake ../ -DHSHM_ENABLE_CUDA=OFF -DHSHM_ENABLE_ROCM=OFF make -j8 ``` -------------------------------- ### Install hermes-shm for Users Source: https://github.com/grc-iit/hermes-shm/blob/main/readme.md Use Spack to install the library as a user. ```bash git clone https://github.com/grc-iit/grc-repo.git spack repo add grc-repo spack install hermes_shm ``` -------------------------------- ### Install hermes_shm via Spack Source: https://github.com/grc-iit/hermes-shm/wiki/Home Standard installation procedure for users to set up the environment and install the library. ```bash # Install spack cd ${HOME} git clone https://github.com/spack/spack cd spack git checkout v0.18.1 echo . `pwd`/share/spack/setup-env.sh >> ~/.bashrc source ~/.bashrc # Add our repo and install hermes_shm cd ${HOME} git clone https://github.com/lukemartinlogan/hermes_shm.git cd hermes_shm spack repo add scripts/hermes_shm spack install hermes_shm spack load hermes_shm ``` -------------------------------- ### Install and Configure Project Files Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Installs headers and generates configuration files for the project build system. ```cmake install(DIRECTORY include DESTINATION ${CMAKE_INSTALL_PREFIX}) configure_file( ${CMAKE_CURRENT_SOURCE_DIR}/include/hermes_shm/constants/settings.h_templ ${CMAKE_CURRENT_SOURCE_DIR}/include/hermes_shm/constants/settings.h @ONLY ) configure_file( ${CMAKE_CURRENT_SOURCE_DIR}/CMake/HermesShmConfig.cmake ${PROJECT_BINARY_DIR}/CMakeFiles/HermesShmConfig.cmake @ONLY ) configure_file( ${CMAKE_CURRENT_SOURCE_DIR}/CMake/HermesShmCommonConfig.cmake ${PROJECT_BINARY_DIR}/CMakeFiles/HermesShmCommonConfig.cmake @ONLY ) install( FILES ${PROJECT_BINARY_DIR}/CMakeFiles/HermesShmConfig.cmake ${PROJECT_BINARY_DIR}/CMakeFiles/HermesShmCommonConfig.cmake DESTINATION ${CMAKE_INSTALL_PREFIX}/cmake ) install(EXPORT ${HSHM_EXPORTED_TARGETS} FILE ${HSHM_EXPORTED_TARGETS}CoreConfig.cmake NAMESPACE hshm:: DESTINATION cmake ) ``` -------------------------------- ### Define Installation Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/data_structures/serialize/shm/CMakeLists.txt Specifies the destination directories for library, archive, and runtime files during installation. ```cmake install(TARGETS test_shm_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Define Installation Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/encrypt/CMakeLists.txt Specifies destination directories for the test executable during the installation process. ```cmake install(TARGETS test_encrypt_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Setup Memory Backend and Allocator Source: https://context7.com/grc-iit/hermes-shm/llms.txt Demonstrates how to create and attach to a shared memory backend and allocator. Rank 0 creates the backend and allocator, while other ranks attach to the existing ones. Requires MPI for process synchronization. ```cpp #include #include #include "hermes_shm/hermes_shm.h" struct CustomHeader { int data_; }; int main(int argc, char **argv) { int rank; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Common allocator information shared across processes std::string shm_url = "test_allocators"; hipc::allocator_id_t alloc_id(0, 1); auto mem_mngr = HERMES_MEMORY_MANAGER; hipc::Allocator *alloc; CustomHeader *header; // Rank 0 creates the backend and allocator if (rank == 0) { // Create a 64 megabyte allocatable region using POSIX shared memory mem_mngr->CreateBackend( MEGABYTES(64), shm_url); // Create a stack allocator over the 64MB region alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(CustomHeader)); // Get and initialize the custom header header = alloc->GetCustomHeader(); header->data_ = 10; } MPI_Barrier(MPI_COMM_WORLD); // Other ranks attach to the existing backend if (rank != 0) { mem_mngr->AttachBackend(hipc::MemoryBackendType::kPosixShmMmap, shm_url); alloc = mem_mngr->GetAllocator(alloc_id); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Verify header is accessible from all processes assert(header->data_ == 10); if (rank == 0) { std::cout << "Shared memory access verified!" << std::endl; } MPI_Finalize(); return 0; } ``` -------------------------------- ### Install hermes_shm for development Source: https://github.com/grc-iit/hermes-shm/wiki/Home Installation steps for contributors requiring manual re-compilation of the library. ```bash # Install spack cd ${HOME} git clone https://github.com/spack/spack cd spack git checkout v0.18.1 echo . `pwd`/share/spack/setup-env.sh >> ~/.bashrc source ~/.bashrc # Add our repo and install the hermes_shm dependencies cd ${HOME} git clone https://github.com/lukemartinlogan/hermes_shm.git cd hermes_shm spack repo add scripts/hermes_shm spack install hermes_shm spack load --only dependencies hermes_shm # Build hermes_shm mkdir build cd build cmake ../ make -j8 ``` -------------------------------- ### Define Installation Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/backend/CMakeLists.txt Specifies the installation paths for the test executable across library, archive, and runtime directories. ```cmake install(TARGETS test_backend_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Install Interceptor Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/interceptor/CMakeLists.txt Specifies installation paths for the defined libraries and executables. ```cmake install(TARGETS test_interceptor test_interceptor_lib test_interceptor_my_lib test_interceptor_stat test_interceptor_lib_stat test_interceptor_my_lib_stat LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Create and Use Shared Memory List with MPI Source: https://github.com/grc-iit/hermes-shm/wiki/2.-Containers Demonstrates initializing and populating a shared-memory list across MPI processes. This example assumes a single process initializes and others read. ```cpp #include #include #include "hermes_shm/data_structures/thread_unsafe/list.h" struct CustomHeader { hipc::TypedPointer> obj_; }; int main(int argc, char **argv) { int rank; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Common allocator information std::string shm_url = "test_allocators"; hipc::allocator_id_t alloc_id(0, 1); auto mem_mngr = HERMES_MEMORY_MANAGER; hipc::Allocator *alloc; CustomHeader *header; // Create backend + allocator if (rank == 0) { // Create a 64 megabyte allocatable region mem_mngr->CreateBackend( MEGABYTES(64), shm_url); // Create a memory allocator over the 64MB region alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(CustomHeader)); // Get the custom header from the allocator header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Attach backend + find allocator if (rank != 0) { mem_mngr->AttachBackend(hipc::MemoryBackendType::kPosixShmMmap, shm_url); alloc = mem_mngr->GetAllocator(alloc_id); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Create the list hipc::list obj; if (rank == 0) { // Initialize in shared memory obj.shm_init(alloc); // Save the list inside the allocator's header header->obj_ = obj.GetShmPointer(); // Emplace 1024 elements for (int i = 0; i < 1024; ++i) { obj.emplace_back(10); } } MPI_Barrier(MPI_COMM_WORLD); // Find the list in shared memory if (rank != 0) { obj << header->obj_; } // Read list on all ranks for (hipc::ShmRef x : obj) { assert(*x == 10); } MPI_Barrier(MPI_COMM_WORLD); // Finalize if (rank == 0) { std::cout << "COMPLETE!" << std::endl; } MPI_Finalize(); } ``` -------------------------------- ### Install CUDA Executable Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/cuda/CMakeLists.txt Installs the 'test_memory_cuda_exec' target. Libraries are installed to ${HSHM_INSTALL_LIB_DIR}, archives to ${HSHM_INSTALL_LIB_DIR}, and runtime binaries to ${HSHM_INSTALL_BIN_DIR}. ```cmake install(TARGETS test_memory_cuda_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### CMake Install Rules Source: https://github.com/grc-iit/hermes-shm/blob/main/benchmark/data_structure/CMakeLists.txt Configures the installation of the 'benchmark_data_structures' target, specifying its destination directories for libraries, archives, and runtime binaries. ```cmake #----------------------------------------------------------------------------- # Add Target(s) to CMake Install #----------------------------------------------------------------------------- install(TARGETS benchmark_data_structures EXPORT ${HERMES_EXPORTED_TARGETS} LIBRARY DESTINATION ${HERMES_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HERMES_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HERMES_INSTALL_BIN_DIR}) ``` -------------------------------- ### Install MPI Test Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/allocators_mpi/CMakeLists.txt Defines installation paths for the MPI test executable. ```cmake if (HSHM_ENABLE_MPI) install(TARGETS test_allocator_mpi_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) endif() ``` -------------------------------- ### Install test_compress_exec Target Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/compress/CMakeLists.txt Specifies installation directories for the test executable across different target types. ```cmake install(TARGETS test_compress_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Install Test Executable Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/data_structures/serialize/cereal/CMakeLists.txt Specifies installation paths for the test executable across library, archive, and runtime directories. ```cmake install(TARGETS test_cereal_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Initialize MPI and Get Rank Source: https://github.com/grc-iit/hermes-shm/wiki/1.-Memory-Backends-and-Allocators Initializes the MPI environment and retrieves the unique rank for the current process. Rank 0 is designated as the root process. ```cpp int main(int argc, char **argv) { int rank; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); ``` -------------------------------- ### Configure Build with Install RPATH Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Enables the use of the install RPATH and sets it to include the Hermes-SHM library directory, ensuring runtime library loading. ```cmake list(APPEND CMAKE_INSTALL_RPATH "${HSHM_INSTALL_LIB_DIR}") set(CMAKE_BUILD_WITH_INSTALL_RPATH ON) SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE) ``` -------------------------------- ### Define Project and Conditional OpenMP Build Source: https://github.com/grc-iit/hermes-shm/blob/main/benchmark/data_structures/CMakeLists.txt Defines the project name and conditionally includes directories and adds an executable if OpenMP is enabled. This setup is for building the benchmark executable with specific source files and linking against host libraries and dependencies. ```cmake project(hermes_shm) if (HSHM_ENABLE_OPENMP) include_directories( ${Boost_INCLUDE_DIRS} ) include_directories( ${TEST_MAIN} ) add_executable(benchmark_data_structures_exec ${TEST_MAIN}/main.cc test_init.cc atomic.cc ref.cc string.cc list.cc vector.cc unordered_map.cc queue.cc lock.cc ) add_dependencies(benchmark_data_structures_exec hermes_shm_host) target_link_libraries(benchmark_data_structures_exec hermes_shm_host Catch2::Catch2 ${OpenMP_LIBS} ${Boost_LIBRARIES}) endif() ``` -------------------------------- ### Install Hermes SHM Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/src/CMakeLists.txt Configures CMake to install the built hermes_shm targets and their dependencies to specified directories. This ensures the library is available for use in other projects. ```cmake # ----------------------------------------------------------------------------- # Add Target(s) to CMake Install # ----------------------------------------------------------------------------- install(TARGETS ${HSHM_LIBS} rocm_gpu_lib_deps rocm_host_lib_deps rocm_host_exec_deps rocm_gpu_exec_deps cuda_gpu_lib_deps cuda_gpu_exec_deps host_deps EXPORT ${HSHM_EXPORTED_TARGETS} LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) ``` -------------------------------- ### Shared Memory Mutex Implementation Source: https://context7.com/grc-iit/hermes-shm/llms.txt Demonstrates the setup and usage of hshm::Mutex for exclusive locking in shared memory. Includes manual lock/unlock, try-lock, and RAII-style ScopedMutex. ```cpp #include "hermes_shm/thread/lock/mutex.h" struct SharedData { hshm::Mutex mutex; int counter; }; int main() { auto mem_mngr = HERMES_MEMORY_MANAGER; // Setup shared memory std::string shm_url = "test_mutex"; hipc::allocator_id_t alloc_id(0, 1); mem_mngr->CreateBackend(MEGABYTES(64), shm_url); auto alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(SharedData)); SharedData *data = alloc->GetCustomHeader(); data->mutex.Init(); data->counter = 0; // Manual lock/unlock uint32_t owner_id = 1; // Thread/process identifier for debugging data->mutex.Lock(owner_id); data->counter++; data->mutex.Unlock(); // Try lock (non-blocking) if (data->mutex.TryLock(owner_id)) { // Got the lock data->counter++; data->mutex.Unlock(); } else { // Lock not available } // Scoped mutex (RAII pattern - recommended) { hshm::ScopedMutex lock(data->mutex, owner_id); // Lock is held within this scope data->counter++; // Can manually unlock early if needed lock.Unlock(); // Can re-lock if needed lock.Lock(owner_id); } // Lock automatically released when scope exits return 0; } ``` -------------------------------- ### Initialize Backend and Allocator on Rank-0 Source: https://github.com/grc-iit/hermes-shm/wiki/1.-Memory-Backends-and-Allocators On the rank-0 process, this code creates a 64MB PosixShmMmap backend and a StackAllocator. It then retrieves and sets a custom header. An MPI barrier ensures all processes wait for initialization. ```cpp if (rank == 0) { // Create a 64 megabyte allocatable region mem_mngr->CreateBackend( MEGABYTES(64), shm_url); // Create a memory allocator over the 64MB region alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(CustomHeader)); // Get the custom header from the allocator header = alloc->GetCustomHeader(); // Set custom header to 10 header->data_ == 10; } MPI_Barrier(MPI_COMM_WORLD); ``` -------------------------------- ### Initialize and Use hipc::list in MPI Source: https://context7.com/grc-iit/hermes-shm/llms.txt Demonstrates setting up shared memory, creating a hipc::list, populating it with elements, and accessing it from multiple MPI ranks. Ensure proper MPI initialization and finalization. ```cpp #include #include #include "hermes_shm/data_structures/ipc/list.h" struct CustomHeader { hipc::TypedPointer> obj_; }; int main(int argc, char **argv) { int rank; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); std::string shm_url = "test_list"; hipc::allocator_id_t alloc_id(0, 1); auto mem_mngr = HERMES_MEMORY_MANAGER; hipc::Allocator *alloc; CustomHeader *header; // Setup shared memory if (rank == 0) { mem_mngr->CreateBackend(MEGABYTES(64), shm_url); alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(CustomHeader)); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); if (rank != 0) { mem_mngr->AttachBackend(hipc::MemoryBackendType::kPosixShmMmap, shm_url); alloc = mem_mngr->GetAllocator(alloc_id); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Create and populate the list on rank 0 hipc::list lst; if (rank == 0) { lst.shm_init(alloc); header->obj_ = lst.GetShmPointer(); // Add elements to back for (int i = 0; i < 1024; ++i) { lst.emplace_back(10); } // Add elements to front lst.emplace_front(5); // Insert at specific position auto it = lst.begin(); ++it; // Move to second position lst.emplace(it, 7); } MPI_Barrier(MPI_COMM_WORLD); // Deserialize list in other ranks if (rank != 0) { lst << header->obj_; } // All ranks can read the list std::cout << "Rank " << rank << " list size: " << lst.size() << std::endl; // Access front and back int first = lst.front(); int last = lst.back(); // Iterate and verify for (auto &x : lst) { // Process each element } // Find and erase elements (on rank 0 only for safety) if (rank == 0) { auto it = lst.find(7); if (it != lst.end()) { lst.erase(it); } // Clear entire list // lst.clear(); } MPI_Barrier(MPI_COMM_WORLD); MPI_Finalize(); return 0; } ``` -------------------------------- ### Initialize and Use hipc::string Source: https://context7.com/grc-iit/hermes-shm/llms.txt Shows how to create, resize, compare, and access shared memory strings, including conversion to and from standard C++ strings. ```cpp #include "hermes_shm/data_structures/ipc/string.h" int main() { auto mem_mngr = HERMES_MEMORY_MANAGER; // Setup allocator std::string shm_url = "test_string"; hipc::allocator_id_t alloc_id(0, 1); mem_mngr->CreateBackend(MEGABYTES(64), shm_url); auto alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, 0); // Create string from const char* hipc::string str1("Hello, World!"); // Create string from std::string std::string std_str = "Shared Memory"; hipc::string str2(std_str); // Create string with explicit allocator hipc::string str3(alloc, "Allocated string"); // Create string with specific length hipc::string str4("Hello", 5); // Get string size size_t len = str1.size(); // Access as C-string const char* c_str = str1.c_str(); const char* data = str1.data(); // Convert to std::string std::string converted = str1.str(); // Character access char first = str1[0]; // Resize string str1.resize(100); // String comparison if (str1 == str2) { /* equal */ } if (str1 != str2) { /* not equal */ } if (str1 < str2) { /* lexicographically less */ } // Compare with std::string if (str1 == "Hello, World!") { /* equal */ } // Assignment str1 = "New content"; str1 = std_str; // Hash for use in containers size_t hash = str1.Hash(); // Output stream support std::cout << str1 << std::endl; return 0; } ``` -------------------------------- ### Configure Test Executable and Dependencies Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/singleton/CMakeLists.txt Creates an executable for testing and links the necessary libraries and dependencies. ```cmake add_executable(test_singleton ${TEST_MAIN}/main.cc test_init.cc test_singleton.cc) add_dependencies(test_singleton my_lib1 my_lib2) target_link_libraries(test_singleton PRIVATE my_lib1 my_lib2 Catch2::Catch2) ``` -------------------------------- ### Initialize and Use hipc::unordered_map Source: https://context7.com/grc-iit/hermes-shm/llms.txt Demonstrates setting up a shared memory hash map, populating it across MPI ranks, and performing basic operations like find, access, and erase. ```cpp #include #include #include "hermes_shm/data_structures/ipc/unordered_map.h" #include "hermes_shm/data_structures/ipc/string.h" struct CustomHeader { hipc::TypedPointer> obj_; }; int main(int argc, char **argv) { int rank; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); std::string shm_url = "test_map"; hipc::allocator_id_t alloc_id(0, 1); auto mem_mngr = HERMES_MEMORY_MANAGER; hipc::Allocator *alloc; CustomHeader *header; // Setup shared memory if (rank == 0) { mem_mngr->CreateBackend(MEGABYTES(64), shm_url); alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(CustomHeader)); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); if (rank != 0) { mem_mngr->AttachBackend(hipc::MemoryBackendType::kPosixShmMmap, shm_url); alloc = mem_mngr->GetAllocator(alloc_id); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Create and populate the map on rank 0 hipc::unordered_map map; if (rank == 0) { // Initialize with custom parameters: // num_buckets=100, max_capacity=4/5, growth=5/4 map.shm_init(alloc, 100); header->obj_ = map.GetShmPointer(); // Emplace key-value pairs (overwrites existing) map.emplace(1, "hello"); map.emplace(2, "world"); map.emplace(3, "shared memory"); // Try emplace (does not overwrite existing) map.try_emplace(1, "this won't replace hello"); } MPI_Barrier(MPI_COMM_WORLD); // Deserialize map in other ranks if (rank != 0) { map << header->obj_; } // All ranks can read the map std::cout << "Rank " << rank << " map size: " << map.size() << std::endl; std::cout << "Rank " << rank << " buckets: " << map.get_num_buckets() << std::endl; // Find and access elements auto it = map.find(1); if (!it.is_end()) { auto &pair = *it; std::cout << "Key: " << pair.GetKey() << ", Value: " << pair.GetVal().str() << std::endl; } // Access with operator[] (throws if key not found) hipc::string &val = map[2]; // Iterate over all entries for (auto &entry : map) { int key = entry.GetKey(); hipc::string &value = entry.GetVal(); } // Erase elements (on rank 0 only) if (rank == 0) { map.erase(3); // Erase by key map.erase(it); // Erase by iterator map.clear(); // Clear entire map } MPI_Barrier(MPI_COMM_WORLD); MPI_Finalize(); return 0; } ``` -------------------------------- ### Define CMake Project and Libraries Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/singleton/CMakeLists.txt Initializes the project and defines static libraries for the build system. ```cmake project(hermes_shm) add_library(my_lib1 STATIC my_lib1.cc) add_library(my_lib2 STATIC my_lib2.cc) ``` -------------------------------- ### CMake Build Configuration Source: https://github.com/grc-iit/hermes-shm/blob/main/benchmark/data_structure/CMakeLists.txt Sets the minimum CMake version, project name, and C++ standard. Include directories for Boost and test main are also specified. ```cmake cmake_minimum_required(VERSION 3.10) project(hermes_shm) set(CMAKE_CXX_STANDARD 17) include_directories( ${Boost_INCLUDE_DIRS} ) include_directories( ${TEST_MAIN} ) ``` -------------------------------- ### Create and Use Shared Memory Vector with MPI Source: https://github.com/grc-iit/hermes-shm/wiki/2.-Containers Demonstrates initializing, resizing, and accessing a shared-memory vector across MPI processes. Requires manual synchronization for concurrent writes. ```cpp #include #include #include "hermes_shm/data_structures/thread_unsafe/vector.h" struct CustomHeader { hipc::TypedPointer> obj_; }; int main(int argc, char **argv) { int rank; MPI_Init(&argc, &argv); MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Common allocator information std::string shm_url = "test_allocators"; hipc::allocator_id_t alloc_id(0, 1); auto mem_mngr = HERMES_MEMORY_MANAGER; hipc::Allocator *alloc; CustomHeader *header; // Create backend + allocator if (rank == 0) { // Create a 64 megabyte allocatable region mem_mngr->CreateBackend( MEGABYTES(64), shm_url); // Create a memory allocator over the 64MB region alloc = mem_mngr->CreateAllocator( shm_url, alloc_id, sizeof(CustomHeader)); // Get the custom header from the allocator header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Attach backend + find allocator if (rank != 0) { mem_mngr->AttachBackend(hipc::MemoryBackendType::kPosixShmMmap, shm_url); alloc = mem_mngr->GetAllocator(alloc_id); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); // Create the vector hipc::vector obj; if (rank == 0) { // Initialize in shared memory obj.shm_init(alloc); // Resize to 1024 ints. Each int will be set to 10. obj.resize(1024, 10); // Save the vector inside the allocator's header header->obj_ = obj.GetShmPointer(); } MPI_Barrier(MPI_COMM_WORLD); // Find the vector in shared memory if (rank != 0) { obj << header->obj_; } // Read vector on all ranks for (hipc::ShmRef x : obj) { assert(*x == 10); } MPI_Barrier(MPI_COMM_WORLD); // Finalize if (rank == 0) { std::cout << "COMPLETE!" << std::endl; } MPI_Finalize(); } ``` -------------------------------- ### Run Tests Source: https://github.com/grc-iit/hermes-shm/blob/main/readme.md Execute the test suite using ctest. ```bash ctest ``` ```bash ctest -VV -R test_mpsc_queue_mpi ``` -------------------------------- ### Configure Build and Test for test_compress_exec Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/compress/CMakeLists.txt Defines the executable, links necessary libraries, and registers the test case with CTest. ```cmake add_executable(test_compress_exec ${TEST_MAIN}/main.cc test_init.cc test_compress.cc) add_dependencies(test_compress_exec hermes_shm_host) target_link_libraries(test_compress_exec hermes_shm_host Catch2::Catch2) add_test(NAME test_compress COMMAND ${CMAKE_BINARY_DIR}/bin/test_compress_exec "~[error=FatalError]") ``` -------------------------------- ### Configure Tests and Benchmarks Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Conditionally builds unit tests and benchmarks based on project configuration flags. ```cmake set(TEST_MAIN ${HSHM_ROOT}/test/unit) enable_testing() if(BUILD_HSHM_TESTS) message("Building HSHM unit tests") add_subdirectory(test) endif() if(BUILD_HSHM_BENCHMARKS) message("Building HSHM benchmarks") add_subdirectory(benchmark) endif() ``` -------------------------------- ### Define Test Executable and Dependencies Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/backend/CMakeLists.txt Configures the test_backend_exec executable with required source files and links against Hermes SHM and MPI libraries. ```cmake add_executable(test_backend_exec ${TEST_MAIN}/main_mpi.cc test_init.cc backend.cc memory_slots.cc memory_manager.cc) add_dependencies(test_backend_exec hermes_shm_host) target_link_libraries(test_backend_exec hermes_shm_host Catch2::Catch2 ${MPI_LIBS}) ``` -------------------------------- ### Implement SHM Move Constructor Source: https://github.com/grc-iit/hermes-shm/wiki/3.-Custom-Containers Initializes a new container by transferring ownership from another instance. Handles cases where allocators match or differ. ```cpp MyClass(ShmHeader *header, Allocator *alloc, MyClass &&other) noexcept { shm_init_header(header, alloc); if (alloc_ == other.alloc_) { list_ = hipc::make_ref>(header_->list_ar_, alloc_, std::forward(other)); other.SetNull(); } else { shm_strong_copy_construct(other); other.shm_destroy(); } } ``` -------------------------------- ### Project and Executable Definition Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/thread/CMakeLists.txt Defines the project name and adds an executable for thread testing. It includes source files and dependencies. ```cmake project(hermes_shm) if (HSHM_ENABLE_OPENMP) #------------------------------------------------------------------------------ # Build Tests #------------------------------------------------------------------------------ add_executable(test_thread_exec ${TEST_MAIN}/main.cc test_init.cc test_lock.cc) add_dependencies(test_thread_exec hermes_shm_host) target_link_libraries(test_thread_exec hermes_shm_host $<$:thallium> Catch2::Catch2 ${OpenMP_LIBS}) #------------------------------------------------------------------------------ # Test Cases #------------------------------------------------------------------------------ add_test(NAME test_thread COMMAND ${CMAKE_BINARY_DIR}/bin/test_thread_exec) #------------------------------------------------------------------------------ # Install Targets #------------------------------------------------------------------------------ install(TARGETS test_thread_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) #----------------------------------------------------------------------------- # Coverage #----------------------------------------------------------------------------- if(HSHM_ENABLE_COVERAGE) set_coverage_flags(test_thread_exec) endif() endif() ``` -------------------------------- ### Register Test Cases Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/backend/CMakeLists.txt Registers individual test cases using the test_backend_exec binary, utilizing mpirun for parallel execution where required. ```cmake add_test(NAME test_memory_slots COMMAND mpirun -n 2 ${CMAKE_BINARY_DIR}/bin/test_backend_exec "MemorySlot") add_test(NAME test_reserve COMMAND ${CMAKE_BINARY_DIR}/bin/test_backend_exec "BackendReserve") add_test(NAME test_memory_manager COMMAND mpirun -n 2 ${CMAKE_BINARY_DIR}/bin/test_backend_exec "MemoryManager") ``` -------------------------------- ### Initialize a Custom SHM Container Source: https://github.com/grc-iit/hermes-shm/wiki/3.-Custom-Containers Defines the ShmHeader POD structure and the corresponding ShmContainer class with an initialization constructor. ```cpp template<> struct ShmHeader { SHM_CONTAINER_HEADER_TEMPLATE(ShmHeader) ShmArchive> list_ar_; /** Called during both copy and move */ void strong_copy(const ShmHeader &other) { length_ = other.length_; text_ = other.text_; } }; #define CLASS_NAME MyClass #define TYPED_CLASS MyClass #define TYPED_HEADER ShmHeader class MyClass : public ShmContainer { SHM_CONTAINER_TEMPLATE((CLASS_NAME), (TYPED_CLASS), (TYPED_HEADER)) // This macro provides the following two class variables: // ShmHeader *header_; // Allocator *alloc_; hipc::Ref> list_; MyClass(ShmHeader *header, Allocator *alloc) { shm_init_header(header, alloc); list_ = hipc::make_ref>(header_->list_ar_, alloc_); } }; ``` -------------------------------- ### Project and Executable Definition Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/data_structures/containers_mpi/CMakeLists.txt Defines the project name and an executable target for MPI tests. Includes source files and dependencies. ```cmake project(hermes_shm) if (HSHM_ENABLE_MPI) #------------------------------------------------------------------------------ # Test Cases #------------------------------------------------------------------------------ add_executable(test_data_structure_mpi_exec ${TEST_MAIN}/main_mpi.cc test_init.cc mpsc_queue.cc ) add_dependencies(test_data_structure_mpi_exec hermes_shm_host) target_link_libraries(test_data_structure_mpi_exec hermes_shm_host Catch2::Catch2 ${MPI_LIBS} ${OpenMP_LIBS}) message(STATUS "MPI_LIBS: ${MPI_LIBS}") #------------------------------------------------------------------------------ # Build Tests #------------------------------------------------------------------------------ # LIST TESTS add_test(NAME test_mpsc_queue_mpi COMMAND mpirun -n 2 ${CMAKE_BINARY_DIR}/bin/test_data_structure_mpi_exec "TestMpscQueueMpi") #------------------------------------------------------------------------------ # Install Targets #------------------------------------------------------------------------------ install(TARGETS test_data_structure_mpi_exec LIBRARY DESTINATION ${HSHM_INSTALL_LIB_DIR} ARCHIVE DESTINATION ${HSHM_INSTALL_LIB_DIR} RUNTIME DESTINATION ${HSHM_INSTALL_BIN_DIR}) #----------------------------------------------------------------------------- # Coverage #----------------------------------------------------------------------------- if(HSHM_ENABLE_COVERAGE) set_coverage_flags(test_data_structure_mpi_exec) endif() endif() ``` -------------------------------- ### Serialize and Deserialize via Pointers Source: https://github.com/grc-iit/hermes-shm/wiki/3.-Custom-Containers Demonstrates converting a container to a process-independent pointer and back using stream operators. ```cpp Pointer p; auto x = make_uptr(); x >> p; uptr y; y << p; ``` -------------------------------- ### Define Build Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Adds subdirectories for the main library and defines custom targets for linting and preamble checks. ```cmake add_subdirectory(src) if(NOT IS_HSHM_MAIN) add_custom_target(lint COMMAND bash ${HSHM_ROOT}/scripts/lint.sh ${HSHM_ROOT}) add_custom_target(preamble COMMAND python3 ${HSHM_ROOT}/scripts/preamble.py ${HSHM_ROOT}) endif() ``` -------------------------------- ### Verify Custom Header and Finalize Source: https://github.com/grc-iit/hermes-shm/wiki/1.-Memory-Backends-and-Allocators Verifies that the custom header's data is correctly set to 10 across all processes. Finally, it prints a completion message and finalizes the MPI environment. ```cpp // Verify header is equal to 10 in all processes assert(header->data_ == 10); // Finalize if (rank == 0) { std::cout << "COMPLETE!" << std::endl; } MPI_Finalize(); } ``` -------------------------------- ### Configure CMake for hermes-shm Source: https://github.com/grc-iit/hermes-shm/blob/main/readme.md Link against the appropriate library version based on hardware support. ```cmake find_package(HermesShm CONFIG REQUIRED) message(STATUS "found hermes_shm.h at ${HermesShm_INCLUDE_DIRS}") target_link_libraries(hshm::cxx) ``` ```cmake find_package(HermesShm CONFIG REQUIRED) message(STATUS "found hermes_shm.h at ${HermesShm_INCLUDE_DIRS}") target_link_libraries(hshm::cudacxx) ``` ```cmake find_package(HermesShm CONFIG REQUIRED) message(STATUS "found hermes_shm.h at ${HermesShm_INCLUDE_DIRS}") target_link_libraries(hshm::rocmcxx_gpu) ``` -------------------------------- ### Define and Configure Test Executable in CMake Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/data_structures/serialize/cereal/CMakeLists.txt Configures a test executable by linking necessary libraries, setting dependencies, and registering the test with CTest. ```cmake set (LIBS hermes_shm_host Catch2::Catch2 ${MPI_LIBS} ${OpenMP_LIBS}) add_executable(test_cereal_exec ${TEST_MAIN}/main.cc test_init.cc test_cereal.cc) add_dependencies(test_cereal_exec hermes_shm_host) target_link_libraries(test_cereal_exec ${LIBS}) add_test(NAME test_cereal COMMAND ${CMAKE_BINARY_DIR}/bin/test_cereal_exec) ``` -------------------------------- ### Define Hermes SHM Library for Host Source: https://github.com/grc-iit/hermes-shm/blob/main/src/CMakeLists.txt Configures the build for the hermes_shm library targeting the host system. It includes source files and links against host dependencies. ```cmake project(hermes_shm) # ----------------------------------------------------------------------------- # Build HSHM # ----------------------------------------------------------------------------- set(HSHM_LIBS "") # BUILD HSHM FOR HOST ONLY set(SRC_FILES # system_info.cc # NOTE(llogan): Commented out because ROCm is annoying and buggy memory_manager.cc) add_library(hermes_shm_host ${SRC_FILES}) target_link_libraries(hermes_shm_host PUBLIC host_deps) list(APPEND HSHM_LIBS hermes_shm_host) add_library(cxx INTERFACE) target_link_libraries(cxx INTERFACE hermes_shm_host) target_include_directories(cxx INTERFACE ${HSHM_INSTALL_INCLUDE_DIR}) target_link_directories(cxx INTERFACE ${HSHM_INSTALL_LIB_DIR}) list(APPEND HSHM_LIBS cxx) ``` -------------------------------- ### Attach to Backend in Other Ranks Source: https://github.com/grc-iit/hermes-shm/wiki/1.-Memory-Backends-and-Allocators Processes with ranks other than 0 attach to the existing shared memory backend using the specified URL. They then retrieve the allocator and its custom header. An MPI barrier synchronizes all processes. ```cpp if (rank != 0) { mem_mngr->AttachBackend(hipc::MemoryBackendType::kPosixShmMmap, shm_url); alloc = mem_mngr->GetAllocator(alloc_id); header = alloc->GetCustomHeader(); } MPI_Barrier(MPI_COMM_WORLD); ``` -------------------------------- ### Define Host Dependencies Interface Library Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Creates an INTERFACE library for host dependencies, allowing other targets to link against common libraries and include directories. ```cmake add_library(host_deps INTERFACE) if(NOT YAML_CPP_LIBRARIES) set(YAML_CPP_LIBRARIES yaml-cpp) endif() target_link_libraries(host_deps INTERFACE ${YAML_CPP_LIBRARIES} ${REAL_TIME_FLAGS} ${SERIALIZATION_LIBS} ${COMPRESS_LIBS} ${ENCRYPT_LIBS} ${Boost_LIBRARIES} ${ELF_LIBS} ) target_link_directories(host_deps INTERFACE ${COMPRESS_LIB_DIRS} ${ENCRYPT_LIB_DIRS} ${Boost_LIBRARY_DIRS} ${ELF_LIB_DIRS} ${YAML_CPP_LIBRARY_DIR}) target_include_directories(host_deps INTERFACE ${COMPRESS_INCLUDES} ${ENCRYPT_INCLUDES} ${Boost_INCLUDE_DIRS} ${ELF_INCLUDES} ${YAML_CPP_INCLUDE_DIR}) if(HSHM_ENABLE_PTHREADS) target_link_libraries(host_deps INTERFACE pthread) endif() ``` -------------------------------- ### Define MemoryBackend Structure Source: https://github.com/grc-iit/hermes-shm/wiki/1.-Memory-Backends-and-Allocators Defines the structure for a Memory Backend, holding header, data pointer, size, and flags. ```cpp class MemoryBackend { public: MemoryBackendHeader *header_; char *data_; size_t data_size_; bitfield32_t flags_; } ``` -------------------------------- ### Define CUDA GPU Library Dependencies Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Sets up an INTERFACE library for CUDA GPU dependencies, linking against host dependencies and defining CUDA-specific compile definitions and options. ```cmake add_library(cuda_gpu_lib_deps INTERFACE) add_library(hshm::cuda_gpu_lib_deps ALIAS cuda_gpu_lib_deps) if(HSHM_ENABLE_CUDA) target_link_libraries(cuda_gpu_lib_deps INTERFACE host_deps) target_compile_definitions(cuda_gpu_lib_deps INTERFACE HSHM_ENABLE_CUDA) target_compile_options(cuda_gpu_lib_deps INTERFACE $<$:--expt-relaxed-constexpr>) endif() ``` -------------------------------- ### Configure CMake Build and Test Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/data_structures/serialize/shm/CMakeLists.txt Defines the test executable, links required libraries, and registers the test with CTest. ```cmake set (LIBS hermes_shm_host Catch2::Catch2 ${MPI_LIBS} ${OpenMP_LIBS}) add_executable(test_shm_exec ${TEST_MAIN}/main.cc test_init.cc test_shm.cc) add_dependencies(test_shm_exec hermes_shm_host) target_link_libraries(test_shm_exec ${LIBS}) add_test(NAME test_shm COMMAND ${CMAKE_BINARY_DIR}/bin/test_shm_exec) ``` -------------------------------- ### Configure CMake Build and Test Targets Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/encrypt/CMakeLists.txt Defines the test executable, links dependencies, and registers the test with CTest. ```cmake add_executable(test_encrypt_exec ${TEST_MAIN}/main.cc test_init.cc test_encrypt.cc) add_dependencies(test_encrypt_exec hermes_shm_host) target_link_libraries(test_encrypt_exec hermes_shm_host Catch2::Catch2) add_test(NAME test_encrypt COMMAND ${CMAKE_BINARY_DIR}/bin/test_encrypt_exec "~[error=FatalError]") ``` -------------------------------- ### Configure Benchmark Executable in CMake Source: https://github.com/grc-iit/hermes-shm/blob/main/benchmark/allocator/CMakeLists.txt Defines the benchmark_allocators_exec target and links necessary dependencies when OpenMP is enabled. ```cmake project(hermes_shm) if (HSHM_ENABLE_OPENMP) include_directories( ${Boost_INCLUDE_DIRS} ) include_directories( ${TEST_MAIN} ) add_executable(benchmark_allocators_exec ${TEST_MAIN}/main.cc test_init.cc allocator.cc ) add_dependencies(benchmark_allocators_exec hermes_shm_host) target_link_libraries(benchmark_allocators_exec hermes_shm_host Catch2::Catch2 ${MPI_LIBS} ${OpenMP_LIBS} ${Boost_LIBRARIES}) endif() ``` -------------------------------- ### Define ROCm GPU Execution Dependencies Source: https://github.com/grc-iit/hermes-shm/blob/main/CMakeLists.txt Sets up an INTERFACE library for ROCm GPU execution dependencies, linking against ROCm GPU library dependencies and specific ROCm libraries. ```cmake add_library(rocm_gpu_exec_deps INTERFACE) add_library(hshm::rocm_gpu_exec_deps ALIAS rocm_gpu_exec_deps) if(HSHM_ENABLE_ROCM) target_link_libraries(rocm_gpu_exec_deps INTERFACE rocm_gpu_lib_deps amdhip64 amd_comgr) endif() ``` -------------------------------- ### Add Executable Target Source: https://github.com/grc-iit/hermes-shm/blob/main/benchmark/data_structure/CMakeLists.txt Defines the 'benchmark_data_structures' executable and lists its source files. It also establishes dependencies on other targets. ```cmake add_executable(benchmark_data_structures ${TEST_MAIN}/main.cc test_init.cc atomic.cc ref.cc string.cc list.cc vector.cc unordered_map.cc queue.cc lock.cc ) add_dependencies(benchmark_data_structures hermes_shm_data_structures) ``` -------------------------------- ### Define Interceptor Libraries and Executables Source: https://github.com/grc-iit/hermes-shm/blob/main/test/unit/interceptor/CMakeLists.txt Defines static and shared libraries for interceptor testing and links them to test executables. ```cmake project(hermes_shm) add_library(test_interceptor_lib_stat STATIC test_lib.cc) add_library(test_interceptor_my_lib_stat STATIC test_my_lib.cc) add_library(test_interceptor_lib SHARED test_lib.cc) add_library(test_interceptor_my_lib SHARED test_my_lib.cc) add_executable(test_interceptor test_interceptor.cc) add_dependencies(test_interceptor test_interceptor_lib) target_link_libraries(test_interceptor PRIVATE test_interceptor_lib) add_executable(test_interceptor_stat test_interceptor.cc) add_dependencies(test_interceptor_stat test_interceptor_lib_stat) target_link_libraries(test_interceptor_stat PRIVATE test_interceptor_lib_stat) ```