### Configure Bitshuffle Installation Options Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/bitshuffle/README.rst Copy the example setup configuration file and edit it to customize installation options, such as OpenMP multi-threading and target microarchitecture. ```bash cp setup.cfg.example setup.cfg ``` -------------------------------- ### Install SZ Example Executables Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/SZ/example/CMakeLists.txt Installs the defined SZ example executables to the binary directory of the installation. ```cmake install (TARGETS testint_compress testint_decompress testfloat_compress testfloat_decompress testdouble_compress testdouble_decompress sz RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}) ``` -------------------------------- ### Project and Options Setup Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/HDF5-Blosc2/CMakeLists.txt Initializes the CMake project and defines build options for tests and the HDF5 plugin. The plugin installation path can be configured. ```cmake cmake_minimum_required(VERSION 2.8.10) cmake_policy(SET CMP0074 NEW) project(blosc2_hdf5) include(ExternalProject) # options option(BUILD_TESTS "Build test programs form the blosc2 filter" ON) option(BUILD_PLUGIN "Build dynamically loadable plugin for HDF5 version > 1.8.11" ON) if(BUILD_PLUGIN) set(PLUGIN_INSTALL_PATH "/usr/local/hdf5/lib/plugin" CACHE PATH "Where to install the dynamic HDF5-plugin") endif(BUILD_PLUGIN) ``` -------------------------------- ### Build and Install H5Z-ZFP Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-ZFP/docs/installation.md After configuring the build with CMake, use 'make install' to compile and install the H5Z-ZFP filter and library. ```bash make install ``` -------------------------------- ### Install FCIDECOMP JPEGLS Plugin Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/fcidecomp/INSTALL.md Install the fcicomp-jpegls plugin using the provided install script. Requires super-user privileges. ```bash sudo ./gen/install.sh fcicomp-jpegls ``` -------------------------------- ### Install ZLIB Library and Headers Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/SZ/zlib/CMakeLists.txt Specifies the installation rules for the ZLIB library and its header files. Libraries are installed to the configured library directory, and headers to the include directory. ```cmake install(TARGETS ZLIB EXPORT ZLIBConfig LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) install(FILES zlib.h zconf.h DESTINATION include) ``` -------------------------------- ### Install hdf5plugin from source Source: https://github.com/silx-kit/hdf5plugin/blob/main/README.rst Install hdf5plugin from source to recompile HDF5 plugins. This may enable performance improvements if AVX2 and OpenMP are available. Use the [--user] flag for user-specific installation. ```bash pip install hdf5plugin --no-binary hdf5plugin [--user] ``` -------------------------------- ### Install Docker Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc2/internal-complibs/zlib-ng-2.0.7/arch/s390/README.md Installs the Docker package using dnf. This is a prerequisite for setting up the CI builder. ```bash sudo dnf install docker ``` -------------------------------- ### Install Utilities Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc2/internal-complibs/zlib-ng-2.0.7/CMakeLists.txt Installs the `minigzip` and `minideflate` executables to specified runtime and library destinations if `INSTALL_UTILS` is enabled. ```cmake if(INSTALL_UTILS) install(TARGETS minigzip minideflate RUNTIME DESTINATION "${BIN_INSTALL_DIR}" ARCHIVE DESTINATION "${LIB_INSTALL_DIR}" LIBRARY DESTINATION "${LIB_INSTALL_DIR}") endif() ``` -------------------------------- ### Install FCIDECOMP HDF5 Plugin Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/fcidecomp/INSTALL.md Install the fcicomp-H5Zjpegls plugin. Requires super-user privileges. ```bash sudo ./gen/install.sh fcicomp-H5Zjpegls ``` -------------------------------- ### Configure and Install Package Configuration Files Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/snappy/CMakeLists.txt Configures and installs the package configuration files for Snappy, including the main Config.cmake and ConfigVersion.cmake files. This enables CMake to find the installed Snappy package. ```cmake include(CMakePackageConfigHelpers) configure_package_config_file( "cmake/${PROJECT_NAME}Config.cmake.in" "${PROJECT_BINARY_DIR}/cmake/${PROJECT_NAME}Config.cmake" INSTALL_DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}" ) write_basic_package_version_file( "${PROJECT_BINARY_DIR}/cmake/${PROJECT_NAME}ConfigVersion.cmake" COMPATIBILITY SameMajorVersion ) install( EXPORT SnappyTargets NAMESPACE Snappy:: DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}" ) install( FILES "${PROJECT_BINARY_DIR}/cmake/${PROJECT_NAME}Config.cmake" "${PROJECT_BINARY_DIR}/cmake/${PROJECT_NAME}ConfigVersion.cmake" DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/${PROJECT_NAME}" ) ``` -------------------------------- ### nccopy Examples Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-SPERR/README.md Examples demonstrating how to use `cd_values` with the `nccopy` command-line tool for SPERR compression. ```APIDOC ## nccopy Examples ### Description These examples show how to apply SPERR compression to specific variables using `nccopy` with pre-generated `cd_values`. ### Usage with `nccopy -F` #### Basic Compression (No Missing Values Handling) Compresses `VAR0` using fixed-rate compression with a bitrate of 3.3, without rank order swap. ```Bash nccopy -F "VAR0, 268651725u" ``` #### Compression with Missing Values Handling (Default) Compresses `VAR0` with the same parameters as above, explicitly indicating no special handling for missing values. ```Bash nccopy -F "VAR0, 268651725u, 0" ``` #### Compression with NaN Handling Compresses `VAR1` using fixed-rate compression (bitrate 3.3), assuming `VAR1` might contain NaN values. ```Bash nccopy -F "VAR1, 268651725u, 1" ``` #### Compression with Special Value Handling Compresses `VAR2` using fixed-rate compression (bitrate 3.3), assuming `VAR2` might contain special values like `1e35`. ```Bash nccopy -F "VAR2, 268651725u, 2" ``` ### Parameters for `nccopy -F` * **Variable Name**: The name of the variable to compress. * **cd_values**: The `unsigned int` generated by `generate_cd_values` or `H5Z_SPERR_make_cd_values`. * **Missing Value Handling Code** (Optional): * `0`: No special handling. * `1`: Handle NaNs. * `2`: Handle special values (e.g., `1e35`). ``` -------------------------------- ### Installation Configuration Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/zfp/python/CMakeLists.txt Installs the 'zfpy' target to the standard Python site-packages directory. ```cmake # Install to the typical python module directory set(python_install_lib_dir "lib/python${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR}/site-packages/") install(TARGETS zfpy LIBRARY DESTINATION ${python_install_lib_dir}) ``` -------------------------------- ### Enable and Start QEMU User Static Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc2/internal-complibs/zlib-ng-2.0.7/arch/s390/README.md Enables and starts the qemu-user-static service for x86_64 emulation support. This is necessary for the CI builder. ```bash sudo systemctl enable --now qemu-user-static ``` -------------------------------- ### Install Blosc wheel package Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/COMPILING_WITH_WHEELS.rst Command to install the Blosc Python wheel package using pip. This example shows the installation process and output, including dependency resolution and successful installation. ```bash $ pip install blosc (base) Collecting blosc Downloading blosc-1.10.0-cp37-cp37m-macosx_10_9_x86_64.whl (2.2 MB) |████████████████████████████████| 2.2 MB 4.7 MB/s Installing collected packages: blosc Attempting uninstall: blosc Found existing installation: blosc 1.10.0 Uninstalling blosc-1.10.0: Successfully uninstalled blosc-1.10.0 Successfully installed blosc-1.10.0 ``` -------------------------------- ### Table Y Example (3-bit lookup) Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/internal-complibs/zlib-1.3.1/doc/algorithm.txt This is an example of another secondary lookup table (Table Y) used for symbols starting with '111'. It uses 3 bits for decoding. ```text 000: F,2 001: F,2 010: G,2 011: G,2 100: H,2 101: H,2 110: I,3 111: J,3 ``` -------------------------------- ### Build Example Executable Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/internal-complibs/zlib-1.3.1/CMakeLists.txt Builds the 'example' executable from test/example.c and links it against the zlib library. It also registers it as a test case. ```cmake if(ZLIB_BUILD_EXAMPLES) add_executable(example test/example.c) target_link_libraries(example zlib) add_test(example example) ``` -------------------------------- ### Table X Example (2-bit lookup) Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/internal-complibs/zlib-1.3.1/doc/algorithm.txt This is an example of a secondary lookup table (Table X) used when the initial lookup requires further processing. It uses 2 bits to decode symbols starting with '110'. ```text 00: C,1 01: C,1 10: D,2 11: E,2 ``` -------------------------------- ### Basic Compression and Decompression Examples Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/zfp/docs/source/zfpcmd.rst Illustrates fundamental usage patterns for reading from and writing to files, including standard input/output. ```bash -i file ``` ```bash -z file ``` ```bash -i ifile -z zfile ``` ```bash -z zfile -o ofile ``` ```bash -i ifile -o ofile ``` -------------------------------- ### EUMETSAT Data Tailor Plugin Setup Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/fcidecomp/documentation/notebooks/00_FCIDECOMP_example_uses.ipynb Imports xarray and the epct.api module for using the EUMETSAT Data Tailor. Checks if the plugin is correctly installed using api.info(). ```python import xarray as xr from epct import api ``` ```python api.info() ``` -------------------------------- ### Compression with Statistics and Standard I/O Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/zfp/docs/source/zfpcmd.rst Demonstrates how to enable statistics output and use standard input/output for compression and decompression pipelines. ```bash -i file -s ``` ```bash -i - -o - -s ``` -------------------------------- ### Prepare and Build FCIDECOMP from Official Delivery Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/fcidecomp/src/fcidecomp/SCF_fcidecomp/generation.txt Steps to prepare the build environment by creating a generation folder, extracting source and COTS archives, setting environment variables, and building the product. Ensure you replace placeholders with actual version and release information. ```bash ENVGEN= mkdir -p $ENVGEN tar zxvf FCIDECOMP_FCIDECOMP_SOURCES--.tar.gz -C $ENVGEN tar zxvf FCIDECOMP_FCIDECOMP_COTS--.tar.gz -C $ENVGEN export FCICOMP_COTS_ROOT=$ENVGEN/FCIDECOMP_COTS cd $ENVGEN/FCIDECOMP_SOURCES/ ./build.sh release ``` -------------------------------- ### Compile LZF Filter as Shared Library Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/bitshuffle/lzf/README.txt Example command to build the LZF filter as a shared library. The resulting library must be installed where the runtime linker can find it. Using -O1 or higher is strongly recommended for performance. ```bash $ gcc -O2 -lhdf5 -fPIC -shared lzf/*.c lzf_filter.c -o liblzf_filter.so ``` -------------------------------- ### Install H5Z-ZFP using Spack Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-ZFP/docs/installation.md This command installs H5Z-ZFP using Spack. Ensure you have Spack installed and configured. ```bash spack install h5z-zfp ``` -------------------------------- ### Install Spack and H5Z-ZFP Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-ZFP/docs/installation.md Steps to clone Spack, checkout a specific release, set up the environment, and then install H5Z-ZFP. ```bash git clone https://github.com/llnl/spack.git cd spack git checkout releases/v0.20 . ./share/spack/setup-env.sh spack install h5z-zfp ``` -------------------------------- ### Install Headers Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/zfp/CMakeLists.txt Installs the header files. If BUILD_CFP is enabled, all headers are installed; otherwise, headers in the 'cfp' directory are excluded. ```cmake if(BUILD_CFP) install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}) else() install(DIRECTORY include/ DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} PATTERN "cfp" EXCLUDE) endif() ``` -------------------------------- ### Build Example64 Executable with 64-bit File Offset Support Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/internal-complibs/zlib-1.3.1/CMakeLists.txt Builds the 'example64' executable from test/example.c, linking it against zlib and enabling 64-bit file offset support via a compile flag. It is also registered as a test case. ```cmake if(HAVE_OFF64_T) add_executable(example64 test/example.c) target_link_libraries(example64 zlib) set_target_properties(example64 PROPERTIES COMPILE_FLAGS "-D_FILE_OFFSET_BITS=64") add_test(example64 example64) ``` -------------------------------- ### Build 3D Example Executable Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-SPERR/utilities/CMakeLists.txt Configures the build for the 'example-3d' executable, linking it with HDF5 and the math library. ```cmake add_executable( example-3d example-3d.c ) include_directories( example-3d ${HDF5_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/include ) target_link_libraries( example-3d PUBLIC ${HDF5_LIBRARIES} PUBLIC "m" ) ``` -------------------------------- ### Build zfp with CMake and Examples Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/zfp/docs/source/installation.rst Builds the zfp library and its examples using CMake. Ensure you are in the zfp source directory. ```bash cmake -DBUILD_EXAMPLES=ON .. ``` -------------------------------- ### Install C-Blosc2 via pip Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc2/COMPILING_WITH_WHEELS.rst Install the latest C-Blosc2 binaries, including plugins, by installing the blosc2 Python package. ```console $ pip install blosc2 Collecting blosc2 Downloading blosc2-0.1.8-cp37-cp37m-manylinux2010_x86_64.whl (3.3 MB) |████████████████████████████████| 3.3 MB 4.7 MB/s Installing collected packages: blosc2 Successfully installed blosc2-0.1.8 ``` -------------------------------- ### Configure and Install Pkgconfig File Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/CMakeLists.txt Configures the blosc.pc.in file and installs it to the pkgconfig directory if BLOSC_INSTALL is enabled. ```cmake configure_file( "${CMAKE_CURRENT_SOURCE_DIR}/blosc.pc.in" "${CMAKE_CURRENT_BINARY_DIR}/blosc.pc" @ONLY) install(FILES "${CMAKE_CURRENT_BINARY_DIR}/blosc.pc" DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig" COMPONENT DEV) ``` -------------------------------- ### Install Build Tools on Ubuntu 20.04 Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/fcidecomp/INSTALL.md Install git, cmake, and gcc for building FCIDECOMP on Ubuntu 20.04. ```bash sudo apt install -y git sudo apt-get install -y cmake gcc=4:9.3.0-1ubuntu2 g++ ``` -------------------------------- ### Enable and Start Actions Runner Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc2/internal-complibs/zlib-ng-2.0.7/arch/s390/README.md Enables and starts the actions-runner service. This command ensures the runner starts automatically on boot. ```bash sudo systemctl enable --now actions-runner ``` -------------------------------- ### Install HDF5Plugin from Source with OpenMP Disabled Source: https://github.com/silx-kit/hdf5plugin/blob/main/doc/install.md Install HDF5Plugin from source while explicitly disabling OpenMP optimizations by setting the HDF5PLUGIN_OPENMP environment variable to False. This command can be used with pip install hdf5plugin or when installing from the source directory. ```bash HDF5PLUGIN_OPENMP=False pip install hdf5plugin --no-binary hdf5plugin ``` ```bash HDF5PLUGIN_OPENMP=False pip install . ``` -------------------------------- ### Build 2D Example Executable Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-SPERR/utilities/CMakeLists.txt Configures the build for the 'example-2d' executable, linking it with HDF5 and the math library. ```cmake add_executable( example-2d example-2d.c ) include_directories( example-2d ${HDF5_INCLUDE_DIR} ${CMAKE_SOURCE_DIR}/include ) target_link_libraries( example-2d PUBLIC ${HDF5_LIBRARIES} PUBLIC "m" ) ``` -------------------------------- ### Compile Example for ARM with Cross-Compiler Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc2/README_ARM.rst Use this command to compile a C example for ARM platforms when running on an Intel machine with a cross-compiler. Ensure the NEON flags are included for optimized performance. ```console $ arm-linux-gnueabihf-gcc -static -mfpu=neon -flax-vector-conversions -DSHUFFLE_NEON_ENABLED -O3 simple.c ../blosc/blosc.c ../blosc/blosclz.c ../blosc/shuffle.c ../blosc/shuffle-generic.c ../blosc/bitshuffle-generic.c ../blosc/shuffle-neon.c ../blosc/bitshuffle-neon.c -I../blosc -o simple -lpthread ``` -------------------------------- ### Install Zstd Library and Headers Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/SZ/zstd/CMakeLists.txt Installs the Zstd library targets (shared/static) and header files to their designated locations in the installation directory. ```cmake install(TARGETS zstd EXPORT ZSTDConfig LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} ) install(FILES deprecated/zbuff.h dictBuilder/zdict.h zstd.h common/zstd_errors.h DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}/ ) export(TARGETS zstd FILE ZSTDConfig.cmake) ``` -------------------------------- ### Install CharLS with vcpkg on Windows Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/charls/README.md Use vcpkg to install the CharLS library on Windows systems. This command installs both the default and 64-bit versions. ```powershell PS> vcpkg install charls charls:x64-windows ``` -------------------------------- ### Build, Test, and Install Blosc Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/README.md Build the Blosc library, run tests, and install it using CMake commands. ```bash cmake --build . ``` ```bash ctest ``` ```bash cmake --build . --target install ``` -------------------------------- ### Find installed packages using Spack Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/H5Z-ZFP/docs/installation.md Commands to locate the installed HDF5 library and the H5Z-ZFP plugin within the Spack installation directory. ```bash spack find -vp hdf5 ``` ```bash spack find -vp h5z-zfp ``` -------------------------------- ### Install SPERR Library (Static) Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/SPERR/CMakeLists.txt Installs the SPERR library as a static library. It specifies the installation destinations for the archive files and public headers. ```cmake install( TARGETS SPERR ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} ) ``` -------------------------------- ### Compile example.c with Blosc (Windows) Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/c-blosc/COMPILING_WITH_WHEELS.rst Compiles the 'simple.c' example file using the MSVC compiler and the Blosc library. This demonstrates a practical application of the compilation command. ```console > cl simple.c %WHEEL_DIR%\lib\blosc.lib /Ox /Fesimple.exe /I%WHEEL_DIR%\include /MT /link/NODEFAULTLIB:MSVCRT Microsoft (R) C/C++ Optimizing Compiler Version 19.10.25017 for x86 Copyright (C) Microsoft Corporation. All rights reserved. simple.c Microsoft (R) Incremental Linker Version 14.10.25017.0 Copyright (C) Microsoft Corporation. All rights reserved. /out:simple.exe simple.obj /NODEFAULTLIB:MSVCRT .\miniconda3\lib\blosc.lib ``` -------------------------------- ### Install SPERR Library (Shared) Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/SPERR/CMakeLists.txt Installs the SPERR library as a shared library. It specifies the installation destinations for the library files and public headers. ```cmake install( TARGETS SPERR LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR} ) ``` -------------------------------- ### Install caterva and Fetch Data Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/HDF5-Blosc2/data/README.md Install the 'caterva' library and then run the 'fetch_data_air.py' script to acquire benchmark data. Ensure you have Python installed. ```bash pip install caterva ``` ```bash python fetch_data_air.py ``` -------------------------------- ### Installation of Filter Header and Library Source: https://github.com/silx-kit/hdf5plugin/blob/main/lib/HDF5-Blosc2/CMakeLists.txt Installs the Blosc2 filter header file and the filter shared library. These components are installed for development purposes. ```cmake # install install(FILES src/blosc2_filter.h DESTINATION include COMPONENT HDF5_FILTER_DEV) install(TARGETS blosc2_filter_shared DESTINATION lib COMPONENT HDF5_FILTER_DEV) ``` -------------------------------- ### Install hdf5plugin using pip Source: https://github.com/silx-kit/hdf5plugin/blob/main/README.rst Install the hdf5plugin package using pip. The optional [--user] flag installs it for the current user only. ```bash pip install hdf5plugin [--user] ```