### Install Application Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Installs the compiled application executable to the 'bin' directory in the runtime destination. ```cmake install(TARGETS Hi5Application RUNTIME DESTINATION bin) ``` -------------------------------- ### Build and install HDF5 manually Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Configure, build, and install HDF5 from source using CMake. Specify the installation prefix and build type. Use '--parallel [NPROC]' for parallel builds. ```bash cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../hdf5-v1.14.0 -B build . ``` ```bash cmake --build build --parallel [NPROC] ``` ```bash cmake --install build ``` -------------------------------- ### Find Package with Components Example Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/dependent_library/CMakeLists.txt Demonstrates how to find a package with and without optional components. ```cmake # Without Boost: find_package(Hi5Dependent REQUIRED) target_link_libraries(foo PUBLIC Hi5Dependent::Read Hi5Dependent::Write) # With Boost: find_package(Hi5Dependent REQUIRED COMPONENTS boost) target_link_libraries(foo PUBLIC Hi5Dependent::Read Hi5Dependent::Write) target_link_libraries(foo PUBLIC Hi5Dependent::Boost) ``` -------------------------------- ### Install HDF5 on ArchLinux Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Install HDF5 using pacman. Use 'hdf5-openmpi' for HDF5 with OpenMPI support. ```bash sudo pacman -S hdf5 ``` -------------------------------- ### Manually Install HighFive Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Clone the HighFive repository, checkout a specific version, and then use CMake to configure, build, and install it. Remember to initialize submodules if prompted. ```bash git clone --recursive https://github.com/highfive-devs/HighFive.git cd HighFive git checkout v2.8.0 ``` ```bash git submodule update --init --recursive ``` ```bash cmake -DCMAKE_INSTALL_PREFIX=build/install -B build . cmake --build build --parallel cmake --install build ``` -------------------------------- ### Install HDF5 C library on Ubuntu Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Install the HDF5 C library without MPI support using apt. For MPI support, install 'libhdf5-openmpi-dev'. ```bash sudo apt-get install libhdf5-dev ``` -------------------------------- ### Compile and Run Tests Source: https://github.com/highfive-devs/highfive/blob/main/doc/developer_guide.md Compile the project with examples and unit tests using CMake and run tests. ```bash cmake -B build -DCMAKE_BUILD_TYPE={Debug,Release} . cmake --build build --parallel ctest --test-dir build ``` -------------------------------- ### Compile and Verify HDF5 Installation Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Use this C++ code with CMake to confirm HDF5 is correctly installed and discoverable by your build system. Ensure your CMakeLists.txt correctly finds and links against the HDF5 library. ```cpp #include int main() { auto file = H5Fcreate("foo.h5", H5F_ACC_EXCL, H5P_DEFAULT, H5P_DEFAULT); H5Fclose(file); return 0; } ``` ```cmake cmake_minimum_required(VERSION 3.19) project(Dummy) find_package(HDF5 REQUIRED) add_executable(dummy dummy.cpp) target_link_libraries(dummy HDF5::HDF5) ``` -------------------------------- ### Create and activate Spack environment Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Create a new Spack environment named 'useful', activate it, add HDF5, and then install it. Use 'hdf5+mpi' for MPI support. ```bash spack env create useful ``` ```bash spack env activate -p useful ``` ```bash spack add hdf5 ``` ```bash spack install --jobs NPROC ``` -------------------------------- ### Get Spack HDF5 installation location Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Retrieve the installation directory for HDF5 managed by Spack. ```bash spack location --install-dir hdf5 ``` -------------------------------- ### Format Codebase Source: https://github.com/highfive-devs/highfive/blob/main/doc/developer_guide.md Format the entire codebase using the provided script, which installs and uses clang-format. ```bash bin/format.sh ``` -------------------------------- ### Clone and activate Spack Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Obtain Spack by cloning its repository and activate it by sourcing the setup script. This makes the 'spack' command available. ```bash git clone https://github.com/spack/spack.git ``` ```bash source spack/share/spack/setup-env.sh ``` -------------------------------- ### Install HighFive using Spack Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Leverage Spack to manage HighFive installation, which automatically handles its HDF5 dependency. Activate a Spack environment and add HighFive to your package list. ```bash spack env activate -p useful spack add highfive spack install --jobs NPROC ``` -------------------------------- ### Target Include Directories Configuration Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/dependent_library/CMakeLists.txt Configures public include directories for targets, supporting both build and install interfaces. ```cmake foreach(target IN LISTS Hi5DependentAllTargets) target_include_directories(${target} PUBLIC $ PUBLIC $ ) # Remember to pick one. Writing out all variation serves testing and # demonstration purposes only. Minimizing lines of code is probably a good # strategy. if( ${INTEGRATION_STRATEGY} STREQUAL "Include" OR ${INTEGRATION_STRATEGY} STREQUAL "bailout") target_link_libraries(${target} PUBLIC HighFive::Include) find_package(HDF5 REQUIRED) target_link_libraries(${target} PUBLIC HDF5::HDF5) find_package(MPI REQUIRED) target_link_libraries(${target} PUBLIC MPI::MPI_C MPI::MPI_CXX) elseif(${INTEGRATION_STRATEGY} STREQUAL "short") target_link_libraries(${target} PUBLIC HighFive) elseif(${INTEGRATION_STRATEGY} STREQUAL "full") target_link_libraries(${target} PUBLIC HighFive::HighFive) endif() if(USE_STATIC_HDF5) find_package(ZLIB REQUIRED) target_link_libraries(${target} PUBLIC ZLIB::ZLIB) endif() endforeach() ``` -------------------------------- ### Compile and Verify HighFive Installation Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Use this C++ code with CMake to verify your HighFive installation. Ensure that CMake correctly finds both HighFive and its HDF5 dependency, and that the executable links properly. ```cpp #include int main() { auto file = HighFive::File("foo.h5", HighFive::File::Create); return 0; } ``` ```cmake cmake_minimum_required(VERSION 3.19) project(UseHighFive) find_package(HighFive REQUIRED) add_executable(dummy dummy.cpp) target_link_libraries(dummy HighFive) ``` -------------------------------- ### CMake Integration for HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Demonstrates different ways to integrate HighFive into a CMake project, including system-wide installation, vendoring, and manual HDF5 management. ```cmake # Option 1 – system-wide installation find_package(HighFive REQUIRED) target_link_libraries(my_app PRIVATE HighFive::HighFive) # Option 2 – vendored (e.g., third_party/HighFive as a git submodule) add_subdirectory(third_party/HighFive) target_link_libraries(my_app PRIVATE HighFive::HighFive) # Option 3 – "bailout": let the project manage HDF5 itself set(HIGHFIVE_FIND_HDF5 Off) find_package(HighFive REQUIRED) target_link_libraries(my_app PRIVATE HighFive::Include) # no automatic HDF5 link find_package(HDF5 REQUIRED) target_link_libraries(my_app PRIVATE HDF5::HDF5) # Optional dependency (Boost, Eigen, xtensor) – include the header + link yourself find_package(Boost REQUIRED) target_link_libraries(my_app PUBLIC Boost::headers) # In C++: #include ``` -------------------------------- ### Find and Link Boost with CMake Source: https://github.com/highfive-devs/highfive/blob/main/README.md Example CMake code to find and link the Boost library, which is an optional dependency for HighFive. ```cmake find_package(Boost REQUIRED) target_link_libraries(foo PUBLIC Boost::headers) ``` -------------------------------- ### Memory Layout Test Example Source: https://github.com/highfive-devs/highfive/blob/main/doc/developer_guide.md Demonstrates a typical memory layout assumption test for types like std::vector>. ```cpp auto array = std::vector>(n); double * ptr = (double*) array.data(); ``` -------------------------------- ### Tell CMake where to find HDF5 Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Use CMAKE_PREFIX_PATH to specify the HDF5 installation location when it's not in a default directory. ${HDF5_ROOT} should point to the directory containing 'include' and 'lib'. ```bash cmake -DCMAKE_PREFIX_PATH="${HDF5_ROOT}" ... ``` -------------------------------- ### HighFive v3 CMake Integration - System Library Source: https://github.com/highfive-devs/highfive/blob/main/doc/migration_guide.md When HighFive is installed as a system-wide library, use `find_package(HighFive)` and `target_link_libraries` to link your application with HighFive and HDF5. ```cmake find_package(HighFive) target_link_libraries(app PUBLIC HighFive::HighFive) ``` -------------------------------- ### Find HighFive via find_package Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/test_dependent_library/CMakeLists.txt Finds an externally installed HighFive library using `find_package`. This is used when HighFive is installed like regular software. ```cmake find_package(HighFive REQUIRED) ``` -------------------------------- ### Opening and Creating HDF5 Files with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Shows how to open and create HDF5 files using `HighFive::File` with various access modes. Includes retrieving file metadata and explicit flushing. ```cpp #include using namespace HighFive; // Create / overwrite a file File file("data.h5", File::Truncate); // Open read-only File ro("data.h5", File::ReadOnly); // Open for read-write, create if absent File rw("data.h5", File::OpenOrCreate); // Retrieve file metadata std::cout << "file: " << file.getName() << "\n"; // "data.h5" std::cout << "size: " << file.getFileSize() << " bytes\n"; std::cout << "free space: "<< file.getFreeSpace() << " bytes\n"; file.flush(); // explicit flush before out-of-scope ``` -------------------------------- ### Create and Write HDF5 File with HighFive Source: https://github.com/highfive-devs/highfive/blob/main/README.md Demonstrates opening a file with truncation, creating a dataset, and writing data using the HighFive library. Ensure the file is closed properly to flush writes. ```c++ // Open (or create) a file: HighFive::File file(filename, HighFive::File::Truncate); // ... write the data to disk std::vector data(50, 1); auto dataset = file.createDataSet("grp/data", data); // ... and read it back, automatically allocating memory: data = dataset.read>(); ``` ```c++ #include using namespace HighFive; std::string filename = "/tmp/new_file.h5"; { // We create an empty HDF5 file, by truncating an existing // file if required: File file(filename, File::Truncate); std::vector data(50, 1); file.createDataSet("grp/data", data); } { // We open the file as read-only: File file(filename, File::ReadOnly); auto dataset = file.getDataSet("grp/data"); // Read back, with allocating: auto data = dataset.read>(); // Because `data` has the correct size, this will // not cause `data` to be reallocated: dataset.read(data); } ``` -------------------------------- ### Application Options Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Defines options for static HDF5 linking and simulating Boost usage. ```cmake option(USE_STATIC_HDF5 "Link against static HDF5" OFF) option(USE_BOOST "Simulates an application using Boost" OFF) ``` -------------------------------- ### Project and C++ Standard Configuration Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/dependent_library/CMakeLists.txt Sets up the CMake project and enforces C++14 standard. ```cmake cmake_minimum_required(VERSION 3.14) project(Hi5Dependent VERSION 0.1) if(NOT DEFINED CMAKE_CXX_STANDARD) set(CMAKE_CXX_STANDARD 14) set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_EXTENSIONS OFF) endif() ``` -------------------------------- ### Integration Strategy and Options Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/dependent_library/CMakeLists.txt Configures integration strategy and optional dependencies like static HDF5 and Boost. ```cmake set(INTEGRATION_STRATEGY "short" CACHE STRING "Use 'Include' for HighFive::Include, 'full' for HighFive::HighFive, 'short' for HighFive.") option(USE_STATIC_HDF5 "Link against static HDF5" OFF) option(USE_BOOST "Build '${PROJECT_NAME}' with optional Boost dependency." OFF) ``` -------------------------------- ### Creating and Writing Datasets with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Demonstrates creating HDF5 datasets using `NodeTraits::createDataSet` on `File` or `Group` objects. Shows automatic type/dataspace inference and explicit configuration. ```cpp #include using namespace HighFive; File file("write_demo.h5", File::Truncate); // Write a 1-D vector of int — type and shape inferred automatically std::vector v(20); for (int i = 0; i < 20; ++i) v[i] = i; file.createDataSet("grp/integers", v); // Write a 2-D vector of double std::vector> mat = {{1., 2., 4.}, {8., 16., 32.}}; file.createDataSet("grp/matrix", mat); // Explicit type + explicit DataSpace DataSet ds = file.createDataSet("scalars", DataSpace(std::vector{10, 10})); // Expected layout on disk: // /grp/integers – 1-D int[20] // /grp/matrix – 2-D double[2][3] // /scalars – 2-D float[10][10] ``` -------------------------------- ### Slice-Based Selection with ProductSet Source: https://context7.com/highfive-devs/highfive/llms.txt Illustrates `ProductSet` for building hyperslabs from per-axis slice descriptors, supporting contiguous ranges, lists of indices, and unions of ranges, similar to NumPy-style indexing. ```cpp #include using namespace HighFive; File file("slices.h5", File::Truncate); std::vector> grid(9, std::vector(7)); for (int i = 0; i < 9; ++i) for (int j = 0; j < 7; ++j) grid[i][j] = (i + 1) + 0.1 * (j + 1); auto ds = file.createDataSet("dset", grid); using container_type = std::vector>; // rows 1–3, columns 2–3 → shape [3][2] auto xslice = std::array{2, 4}; auto yslice = std::array{1, 4}; auto block = ds.select(ProductSet(yslice, xslice)).read(); // Non-contiguous rows {1, 2, 8}, columns 2–3 auto rows = std::vector{1, 2, 8}; auto nc = ds.select(ProductSet(rows, xslice)).read(); // Union of row-ranges {0:2, 5:9}, columns 2–3 auto row_union = std::vector>{{0, 2}, {5, 9}}; auto uc = ds.select(ProductSet(row_union, xslice)).read(); ``` -------------------------------- ### Enable Testing and Add Test Case Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/test_dependent_library/CMakeLists.txt Enables the testing framework and adds a test case that runs the compiled executable. ```cmake enable_testing() add_test(NAME run_test_hi5_dependent COMMAND ${CMAKE_CURRENT_BINARY_DIR}/test_hi5_dependent) ``` -------------------------------- ### Including Optional Dependencies in C++ Source: https://github.com/highfive-devs/highfive/blob/main/doc/migration_guide.md To use optionally supported containers like `boost::multi_array`, include the corresponding header (e.g., ``) in your C++ code. ```cpp #include // Code the reads or write `boost::multi_array`. ``` -------------------------------- ### Load HDF5 module on a cluster Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Use the 'module load' command to access HDF5 on a cluster. Check 'module avail' or 'module show hdf5' for specific instructions. ```bash module load hdf5 ``` -------------------------------- ### Enable Testing and Add Test Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Enables testing for the project and adds a test case that runs the compiled application. ```cmake enable_testing() add_test(NAME test_hi5_application COMMAND ${CMAKE_CURRENT_BINARY_DIR}/Hi5Application) ``` -------------------------------- ### Configure Doxygen Documentation Build Source: https://github.com/highfive-devs/highfive/blob/main/doc/CMakeLists.txt This snippet finds Doxygen, configures the Doxyfile, and adds a custom target for building documentation. It includes a fallback message if Doxygen is not found. ```cmake find_package(Doxygen) if(Doxygen_FOUND) configure_file(${CMAKE_CURRENT_SOURCE_DIR}/Doxyfile ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile @ONLY) add_custom_target(doc COMMAND Doxygen::doxygen ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile) else() message(STATUS " Documentation (doc) cannot be built since Doxygen is not available.") endif() ``` -------------------------------- ### Configure Parallel MPI Tests Source: https://github.com/highfive-devs/highfive/blob/main/tests/unit/CMakeLists.txt Sets up executables and linking for parallel tests using MPI. It includes patching the Catch2 test discovery script to incorporate MPI execution commands. ```cmake set(tests_parallel_src "tests_high_five_parallel.cpp") ## parallel MPI tests add_executable(tests_parallel_bin ${tests_parallel_src}) target_link_libraries(tests_parallel_bin HighFive HighFiveWarnings Catch2::Catch2) target_link_libraries(tests_parallel_bin HighFiveOptionalDependencies) # We need to patch in a call to `mpirun` or equivalent when using # parallel tests. Somehow, this is not foreseen in Catch2, modify the # test detection script and fail if the execution method needs updating. set(original_catch_script "${_CATCH_DISCOVER_TESTS_SCRIPT}") set(patched_catch_script "${CMAKE_CURRENT_BINARY_DIR}/patched_catch_test_discovery.cmake") file(READ "${original_catch_script}" original_catch_script_contents) string(REGEX REPLACE "(add_command\(add_test.*TEST_EXECUTOR})" "\1 ${MPIEXEC_EXECUTABLE} ${MPIEXEC_NUMPROC_FLAG} 2" modified_catch_script_contents "${original_catch_script_contents}") if(original_catch_script_contents STREQUAL modified_catch_script_contents) message(FATAL_ERROR "Failed to modify Catch2 test execution") endif() file(WRITE "${patched_catch_script}" "${modified_catch_script_contents}") set(_CATCH_DISCOVER_TESTS_SCRIPT "${patched_catch_script}") catch_discover_tests(tests_parallel_bin) set(_CATCH_DISCOVER_TESTS_SCRIPT "${original_catch_script}") ``` -------------------------------- ### Link Boost and Define Compile Definition Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Finds the Boost library and links against its headers. Also defines a preprocessor macro to indicate Boost's presence. ```cmake find_package(Boost REQUIRED) target_link_libraries(Hi5Application PUBLIC Boost::headers) target_compile_definitions(Hi5Application PUBLIC HI5_APPLICATION_HAS_BOOST=1) ``` -------------------------------- ### Resize Extendible Dataset in HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Demonstrates how to resize a dataset that was created with an unlimited dimension and chunking. This allows for dynamic growth of the dataset after initial creation. Ensure the dataset is created with `DataSpace::UNLIMITED` and `Chunking` properties. ```cpp #include using namespace HighFive; File file("extendible.h5", File::Truncate); // Initial shape {4, 5}, unlimited in second dimension DataSpace ds_space({4, 5}, {17, DataSpace::UNLIMITED}); DataSetCreateProps props; props.add(Chunking(std::vector{2, 2})); DataSet ds = file.createDataSet("dset", ds_space, create_datatype(), props); // Write initial data into [0:3][0:1] double t1[3][1] = {{2.0}, {2.0}, {4.0}}; ds.select({0, 0}, {3, 1}).write(t1); // Grow and fill new area ds.resize({4, 6}); double t2[1][3] = {{4.0, 8.0, 6.0}}; ds.select({3, 3}, {1, 3}).write(t2); // Read entire (resized) dataset std::vector> result; ds.read(result); // result[0] == {2, 0, 0, 0, 0, 0} // result[3] == {0, 0, 0, 4, 8, 6} ``` -------------------------------- ### Write and Read Data with DataSet Source: https://context7.com/highfive-devs/highfive/llms.txt Demonstrates writing a std::vector to an HDF5 dataset and then reading it back. Supports both allocating and non-allocating reads. ```cpp #include using namespace HighFive; const std::string filename = "rw_demo.h5"; // --- Write --- { File file(filename, File::Truncate); std::vector data(50, 1); file.createDataSet("dset", data); } // --- Read (allocating) --- { File file(filename, File::ReadOnly); DataSet ds = file.getDataSet("dset"); // allocating read — returns a new std::vector auto data = ds.read>(); // non-allocating read into pre-sized container std::vector buf(50); ds.read(buf); std::cout << "first: " << data[0] << ", last: " << data[49] << "\n"; // Output: first: 1, last: 1 } ``` -------------------------------- ### Set CMake Minimum Version and Project Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Sets the minimum required CMake version and defines the project name and version. ```cmake cmake_minimum_required(VERSION 3.14) project(Hi5Application VERSION 0.1) ``` -------------------------------- ### Add Executable and Link Libraries for Base Tests Source: https://github.com/highfive-devs/highfive/blob/main/tests/unit/CMakeLists.txt Defines an executable for each base test and links necessary HighFive and Catch2 libraries. This is used for standard unit testing. ```cmake add_executable(${test_name} "${test_name}.cpp") target_link_libraries(${test_name} HighFive HighFiveWarnings HighFiveFlags Catch2::Catch2WithMain) target_link_libraries(${test_name} HighFiveOptionalDependencies) catch_discover_tests(${test_name}) ``` -------------------------------- ### SWMR Writer with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Allows one writer process to extend a chunked dataset while readers observe the growing data. The writer must use FileVersionBounds and call File::startSWMRWrite() before the write loop. Flush after each write. ```cpp // ---- WRITER ---- #include using namespace HighFive; int main() { FileAccessProps fapl; fapl.add(FileVersionBounds(H5F_LIBVER_LATEST, H5F_LIBVER_LATEST)); File file("swmr.h5", File::Truncate, fapl); DataSetCreateProps props; props.add(Chunking({1})); DataSet ds = file.createDataSet( "data", DataSpace({0ul}, {DataSpace::UNLIMITED}), props); file.startSWMRWrite(); // no new datasets/groups/attributes after this for (int i = 0; i < 10; ++i) { ds.resize({size_t(i + 1)}); ds.select({size_t(i)}, {1ul}).write(i); ds.flush(); // must flush after each write in SWMR } } ``` -------------------------------- ### ProductSet Source: https://context7.com/highfive-devs/highfive/llms.txt Builds a hyperslab selection from per-axis slice descriptors, supporting contiguous ranges, lists of indices, and unions of ranges, similar to NumPy-style indexing. ```APIDOC ## ProductSet ### Description Constructs a hyperslab selection for a dataset using slice descriptors for each axis. Supports various indexing schemes like contiguous ranges, individual indices, and unions of ranges. ### Method `ProductSet(slice_descriptor1, slice_descriptor2, ...)` ### Parameters - `slice_descriptor` (various types): Describes the selection for a single axis. Can be: - `std::array{start, stop}` for a contiguous range [start, stop). - `std::vector` for a list of individual indices. - `std::vector>` for a union of ranges. ### Request Example ```cpp #include using namespace HighFive; File file("slices.h5", File::Truncate); std::vector> grid(9, std::vector(7)); for (int i = 0; i < 9; ++i) for (int j = 0; j < 7; ++j) grid[i][j] = (i + 1) + 0.1 * (j + 1); auto ds = file.createDataSet("dset", grid); using container_type = std::vector>; // rows 1–3, columns 2–3 -> shape [3][2] auto xslice = std::array{2, 4}; auto yslice = std::array{1, 4}; auto block = ds.select(ProductSet(yslice, xslice)).read(); // Non-contiguous rows {1, 2, 8}, columns 2–3 auto rows = std::vector{1, 2, 8}; auto nc = ds.select(ProductSet(rows, xslice)).read(); // Union of row-ranges {0:2, 5:9}, columns 2–3 auto row_union = std::vector>{{0, 2}, {5, 9}}; auto uc = ds.select(ProductSet(row_union, xslice)).read(); ``` ``` -------------------------------- ### Include Optional Dependency Headers in C++ Source: https://github.com/highfive-devs/highfive/blob/main/README.md Include specific headers for optional dependencies like Boost or Eigen when using them with HighFive. ```cpp #include #include ``` -------------------------------- ### Set C++ Standard and Project Details Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/test_dependent_library/CMakeLists.txt Configures the minimum CMake version, project name, and C++ standard to be used. Ensures C++14 is required and extensions are disabled. ```cmake cmake_minimum_required(VERSION 3.14) project(TestHi5Dependent VERSION 0.1) if(NOT DEFINED CMAKE_CXX_STANDARD) set(CMAKE_CXX_STANDARD 14) set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_EXTENSIONS OFF) endif() ``` -------------------------------- ### Simple Data Dump and Load with H5Easy Source: https://github.com/highfive-devs/highfive/blob/main/README.md Utilizes the H5Easy interface for a simplified way to dump scalar variables or supported containers to an HDF5 file and load them back. This is a convenient alternative for common use cases. ```cpp #include int main() { H5Easy::File file("example.h5", H5Easy::File::Overwrite); int A = 42; H5Easy::dump(file, "/path/to/A", A); A = H5Easy::load(file, "/path/to/A"); } ``` -------------------------------- ### Use H5Easy for One-Liner Dump/Load API Source: https://context7.com/highfive-devs/highfive/llms.txt The `H5Easy` API simplifies data I/O with minimal `dump` and `load` calls. It supports scalars, STL containers, and integrates with Eigen, XTensor, and OpenCV when their headers are included before `H5Easy.hpp`. Use `DumpMode::Overwrite` to replace existing datasets and `dumpAttribute` for metadata. ```cpp #include int main() { H5Easy::File file("easy.h5", H5Easy::File::Overwrite); // Scalar H5Easy::dump(file, "/scalars/pi", 3.14159); double pi = H5Easy::load(file, "/scalars/pi"); // STL vector std::vector v{1., 2., 3.}; H5Easy::dump(file, "/data/v", v); v = H5Easy::load>(file, "/data/v"); // Overwrite existing dataset H5Easy::dump(file, "/data/v", std::vector{4., 5., 6.}, H5Easy::DumpMode::Overwrite); // Attributes alongside datasets H5Easy::dumpAttribute(file, "/data/v", "units", std::string("m/s")); H5Easy::dumpAttribute(file, "/data/v", "temperature", 293.15); // Extendible dataset: append scalars at index 0, 1, 2, ... H5Easy::dump(file, "/log/step", 10, {0}); H5Easy::dump(file, "/log/step", 20, {1}); int s0 = H5Easy::load(file, "/log/step", {0}); // 10 // Shape/size queries auto shape = H5Easy::getShape(file, "/data/v"); // {3} auto size = H5Easy::getSize(file, "/data/v"); // 3 std::cout << "size=" << size << "\n"; } ``` -------------------------------- ### Include HighFive via FetchContent Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/test_dependent_library/CMakeLists.txt Declares and makes HighFive available using FetchContent. Requires HIGHFIVE_GIT_REPOSITORY and HIGHFIVE_GIT_TAG environment variables to be set. ```cmake include(FetchContent) FetchContent_Declare(HighFive GIT_REPOSITORY $ENV{HIGHFIVE_GIT_REPOSITORY} GIT_TAG $ENV{HIGHFIVE_GIT_TAG} ) FetchContent_MakeAvailable(HighFive) ``` -------------------------------- ### Define Read Library Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/dependent_library/CMakeLists.txt Defines the STATIC library for reading data. ```cmake add_library(${PROJECT_NAME}Read STATIC "src/hi5_dependent/read_vector.cpp") add_library(${PROJECT_NAME}::Read ALIAS ${PROJECT_NAME}Read) set_target_properties(${PROJECT_NAME}Read PROPERTIES EXPORT_NAME Read) ``` -------------------------------- ### SWMR Reader with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Opens a file in SWMR read mode to observe data being extended by a writer. Use File::ReadSWMR and call ds.refresh() before reading to see the latest data. ```cpp // ---- READER (separate process) ---- // File file("swmr.h5", File::ReadOnly | File::ReadSWMR); // DataSet ds = file.getDataSet("data"); // ds.refresh(); // refresh before reading to see latest data // auto dims = ds.getDimensions(); // int val; ds.select({dims[0]-1}, {1}).squeezeMemSpace({0}).read(val); ``` -------------------------------- ### Partial Selection with DataSet::select Source: https://context7.com/highfive-devs/highfive/llms.txt Demonstrates using `DataSet::select` to define a rectangular subregion (hyperslab) for read/write operations. This enables partial data access within a dataset. ```cpp #include using namespace HighFive; File file("select_demo.h5", File::Truncate); // Create a 2×5 dataset std::vector> values = {{1, 2, 4, 8, 16}, {32, 64, 128, 256, 512}}; DataSet ds = file.createDataSet("dset", DataSpace::From(values)); ds.write(values); // Read a 2×2 sub-block starting at offset (0,2) → {{4,8},{128,256}} std::vector> sub; ds.select({0, 2}, {2, 2}).read(sub); for (auto& row : sub) { for (auto v : row) std::cout << v << " "; std::cout << "\n"; } // Output: // 4 8 // 128 256 ``` -------------------------------- ### Link ZLIB for Static HDF5 Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Finds and links the ZLIB library when static HDF5 is used. This is often a dependency for static builds. ```cmake find_package(ZLIB REQUIRED) target_link_libraries(${target} PUBLIC ZLIB::ZLIB) ``` -------------------------------- ### Link HighFive (Short Integration) Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Links the application directly against the HighFive target. This is a common and often backwards-compatible approach. ```cmake target_link_libraries(Hi5Application PUBLIC HighFive) ``` -------------------------------- ### Update Submodules Source: https://github.com/highfive-devs/highfive/blob/main/doc/developer_guide.md Update and initialize submodules if they are out of sync after a checkout. ```bash git submodule update --init --recursive ``` -------------------------------- ### Configuring CMake for Optional Dependencies Source: https://github.com/highfive-devs/highfive/blob/main/doc/migration_guide.md When using optional dependencies like Boost, ensure your CMake code finds the dependency (e.g., `find_package(Boost)`) and links it to your target (e.g., `target_link_libraries(app PUBLIC boost::boost)`). ```cmake find_package(Boost) target_link_libraries(add PUBLIC boost::boost) ``` -------------------------------- ### Define Application Executable Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Defines the main executable target for the application. ```cmake add_executable(Hi5Application "hi5_application.cpp") ``` -------------------------------- ### Create In-Memory HDF5 File with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Subclass `HighFive::File` to enable operating on HDF5 data stored in RAM. This requires providing an HDF5 identifier from `H5LTopen_file_image`. Useful for unit tests or serving HDF5 over a network buffer. ```cpp #include #include #include class InMemoryFile : public HighFive::File { public: explicit InMemoryFile(std::vector buffer) : _buffer(std::move(buffer)) { _hid = H5LTopen_file_image( _buffer.data(), _buffer.size() * sizeof(_buffer[0]), H5LT_FILE_IMAGE_DONT_RELEASE | H5LT_FILE_IMAGE_DONT_COPY); } private: std::vector _buffer; }; int main() { // Write a normal file first { HighFive::File f("tmp.h5", HighFive::File::Truncate); f.createDataSet("v", std::vector{1.0, 2.0, 3.0}); } // Load raw bytes auto buf = std::vector(1ul << 20); auto* fp = std::fopen("tmp.h5", "r"); std::fread(buf.data(), 1, buf.size(), fp); std::fclose(fp); // Open from buffer InMemoryFile h5(std::move(buf)); auto v = h5.getDataSet("v").read>(); // v == {1.0, 2.0, 3.0} } ``` -------------------------------- ### Clone Repository with Submodules Source: https://github.com/highfive-devs/highfive/blob/main/doc/developer_guide.md Clone the HighFive repository, ensuring all submodules are initialized recursively. ```bash git clone --recursive git@github.com:highfive-devs/HighFive.git ``` -------------------------------- ### Parallel HDF5 (MPI-IO) with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Enables collective or independent MPI-IO using MPIOFileAccess and requests collective I/O for optimized bandwidth with UseCollectiveIO. Requires HDF5 built with parallel support. ```cpp #include #include using namespace HighFive; int main(int argc, char** argv) { MPI_Init(&argc, &argv); int rank, size; MPI_Comm_rank(MPI_COMM_WORLD, &rank); MPI_Comm_size(MPI_COMM_WORLD, &size); try { FileAccessProps fapl; fapl.add(MPIOFileAccess{MPI_COMM_WORLD, MPI_INFO_NULL}); fapl.add(MPIOCollectiveMetadata{}); File file("parallel.h5", File::Truncate, fapl); auto group = file.createGroup("grp"); // One row per rank, two columns DataSet ds = group.createDataSet( "data", DataSpace(std::vector{size_t(size), 2})); std::array row{rank * 1.0, rank * 2.0}; DataTransferProps xfer; xfer.add(UseCollectiveIO{}); // Each rank writes its own row ds.select({size_t(rank), 0}, {1, 2}) .squeezeMemSpace({0}) .write(row, xfer); file.flush(); // Each rank reads the next rank's row (wrapping) size_t next = (rank + 1) % size; ds.select({next, 0}, {1, 2}) .squeezeMemSpace({0}) .read(row, xfer); } catch (Exception& e) { std::cerr << e.what() << "\n"; MPI_Abort(MPI_COMM_WORLD, 1); } MPI_Finalize(); } ``` -------------------------------- ### Find and Link Hi5Dependent with Boost Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/test_dependent_library/CMakeLists.txt Finds the Hi5Dependent library with its Boost component and links 'Read', 'Write', and 'Boost' targets to the executable. This is used when USE_BOOST is defined. ```cmake find_package(Hi5Dependent REQUIRED COMPONENTS boost) target_link_libraries(test_hi5_dependent PUBLIC Hi5Dependent::Read Hi5Dependent::Write) target_link_libraries(test_hi5_dependent PUBLIC Hi5Dependent::Boost) ``` -------------------------------- ### Integrate Eigen Matrices and Arrays with HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Include `` after Eigen headers to serialize `Eigen::Matrix`, `Eigen::Array`, `Eigen::Vector`, and `Eigen::Map` directly into HDF5 datasets. Ensure correct `Eigen::RowMajor` or `Eigen::ColMajor` specification when reading back. ```cpp #include #include int main() { HighFive::File file("eigen.h5", HighFive::File::Truncate); Eigen::MatrixXd A(4, 3); A << 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12; // Write file.createDataSet("mat", A); // Read back as a fixed-size row-major matrix using Matrix43d = Eigen::Matrix; auto B = file.getDataSet("mat").read(); std::cout << B << "\n"; // 1 2 3 // 4 5 6 // 7 8 9 // 10 11 12 } ``` -------------------------------- ### Clone HDF5 source code Source: https://github.com/highfive-devs/highfive/blob/main/doc/installation.md Obtain the HDF5 source code using git and check out a specific version. ```bash git clone https://github.com/HDFGroup/hdf5 ``` ```bash cd hdf5 ``` ```bash git checkout hdf5-1_14_0 ``` -------------------------------- ### Organize Datasets into Hierarchies with Groups in HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Shows how to create and manage hierarchical structures (groups) within an HDF5 file using HighFive. Groups can contain other groups and datasets, similar to directories in a file system. Supports creating, accessing, listing children, and renaming objects within groups. ```cpp #include using namespace HighFive; File file("groups.h5", File::Truncate); // Create nested groups Group g1 = file.createGroup("experiment/run_001"); g1.createDataSet("temperature", std::vector{20.1f, 20.5f, 21.0f}); g1.createDataSet("pressure", std::vector{1.01f, 1.02f, 1.03f}); // Enumerate children for (const auto& name : file.getGroup("experiment").listObjectNames()) std::cout << "child: " << name << "\n"; // child: run_001 // Rename / move an object (creates intermediate groups if needed) file.rename("/experiment/run_001", "/archive/run_001"); ``` -------------------------------- ### Configure C++ Standard Source: https://github.com/highfive-devs/highfive/blob/main/tests/cmake_integration/application/CMakeLists.txt Ensures a specific C++ standard (14) is used, with required settings. ```cmake if(NOT DEFINED CMAKE_CXX_STANDARD) set(CMAKE_CXX_STANDARD 14) set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_EXTENSIONS OFF) endif() ``` -------------------------------- ### Read/Write Raw Pointers with DataSet Source: https://context7.com/highfive-devs/highfive/llms.txt Utilizes `write_raw` and `read_raw` for direct I/O with pre-allocated flat buffers, bypassing the inspector layer. The caller must ensure sufficient buffer size. ```cpp #include using namespace HighFive; File file("raw_ptr.h5", File::Truncate); // Write from a flat row-major buffer representing a 3×5 matrix std::vector dims{3, 5}; std::vector flat(15); for (size_t i = 0; i < 3; ++i) for (size_t j = 0; j < 5; ++j) flat[j + i * 5] = double(j) + 100.0 * double(i); auto ds = file.createDataSet("matrix", DataSpace(dims)); ds.write_raw(flat.data()); // Read back via raw pointer auto ds2 = file.getDataSet("matrix"); auto rdims = ds2.getDimensions(); // {3, 5} std::vector out(rdims[0] * rdims[1]); ds2.read_raw(out.data()); std::cout << out[0] << " " << out[6] << "\n"; // 0 101 ``` -------------------------------- ### Map C Structs to HDF5 Compound Types in HighFive Source: https://context7.com/highfive-devs/highfive/llms.txt Demonstrates how to map C/C++ structs to HDF5 compound datatypes using HighFive. The `HIGHFIVE_REGISTER_TYPE` macro and a custom creation function enable transparent reading and writing of struct data to HDF5 datasets. Committing the type to the file is optional but recommended for self-description. ```cpp #include struct Particle { double x, y, z; float mass; }; HighFive::CompoundType create_compound_Particle() { return { {"x", HighFive::create_datatype()}, {"y", HighFive::create_datatype()}, {"z", HighFive::create_datatype()}, {"mass", HighFive::create_datatype()}, }; } HIGHFIVE_REGISTER_TYPE(Particle, create_compound_Particle) int main() { HighFive::File file("particles.h5", HighFive::File::Truncate); // Commit the type to the file (optional but useful for self-description) create_compound_Particle().commit(file, "Particle"); std::vector particles = {{0.1, 0.2, 0.3, 1.0f}, {1.0, 2.0, 3.0, 2.5f}}; file.createDataSet("particles", particles); auto back = file.getDataSet("particles").read>(); std::cout << back[1].x << "\n"; // 1.0 } ```