### Compile and Install BLAKE3 with CMake (Recent) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Use this command for recent CMake versions to configure, build, and install the BLAKE3 library. Ensure CMake version 3.9 or higher is installed. ```bash cmake -S c -B c/build "-DCMAKE_INSTALL_PREFIX=/usr/local" cmake --build c/build --target install ``` -------------------------------- ### Install BLAKE3 Library and Targets Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Installs the blake3 header file, library targets (archive, library, runtime), and export set for CMake. ```cmake install(FILES blake3.h DESTINATION "${CMAKE_INSTALL_INCLUDEDIR}") install(TARGETS blake3 EXPORT blake3-targets ARCHIVE DESTINATION "${CMAKE_INSTALL_LIBDIR}" LIBRARY DESTINATION "${CMAKE_INSTALL_LIBDIR}" RUNTIME DESTINATION "${CMAKE_INSTALL_BINDIR}" ) install(EXPORT blake3-targets NAMESPACE BLAKE3:: DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/blake3" ) ``` -------------------------------- ### Build Example TBB Application Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Build an example application that uses mmap and blake3_hasher_update_tbb for large-file performance. Link against C++ standard library, tbb, and the BLAKE3 object files. ```bash gcc -O3 -o example_tbb -lstdc++ -ltbb -DBLAKE3_USE_TBB blake3_tbb.o example_tbb.c blake3.c \ blake3_dispatch.c blake3_portable.c blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S \ blake3_avx2_x86-64_unix.S blake3_avx512_x86-64_unix.S ``` -------------------------------- ### Conditional Include for Examples Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Includes example scripts if the BLAKE3_EXAMPLES variable is set. ```cmake if(BLAKE3_EXAMPLES) include(BLAKE3/Examples) endif() ``` -------------------------------- ### Install b3sum from local path Source: https://github.com/blake3-team/blake3/blob/master/b3sum/README.md Install b3sum directly from the source directory using Cargo. This is useful for development or testing. ```bash cargo install --path . ``` -------------------------------- ### Compile BLAKE3 Example Program Source: https://github.com/blake3-team/blake3/blob/master/c/README.md This bash command compiles the example BLAKE3 C program on an x86_64 Unix-like system. It includes optimization flags and links all necessary BLAKE3 source files. ```bash gcc -O3 -o example example.c blake3.c blake3_dispatch.c blake3_portable.c \ blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S \ blake3_avx512_x86-64_unix.S ``` -------------------------------- ### Compile and Install BLAKE3 with CMake (Older) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md This sequence of commands is for older CMake versions. It involves manually creating a build directory and navigating into it before running CMake. ```bash cd c mkdir build cd build cmake .. "-DCMAKE_INSTALL_PREFIX=/usr/local" cmake --build . --target install ``` -------------------------------- ### Install b3sum using Cargo Source: https://github.com/blake3-team/blake3/blob/master/b3sum/README.md Install the b3sum utility using the Cargo package manager. This command assumes Rust and Cargo are already installed. ```bash cargo install b3sum ``` -------------------------------- ### Configure Pkg-Config File Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Configures and installs the libblake3.pc file for pkg-config support. ```cmake join_pkg_config_field(", " PKG_CONFIG_REQUIRES) join_pkg_config_field(" " PKG_CONFIG_LIBS) join_pkg_config_field(" " PKG_CONFIG_CFLAGS) join_paths(PKG_CONFIG_INSTALL_LIBDIR "${prefix}" "${CMAKE_INSTALL_LIBDIR}") join_paths(PKG_CONFIG_INSTALL_INCLUDEDIR "${prefix}" "${CMAKE_INSTALL_INCLUDEDIR}") configure_file(libblake3.pc.in libblake3.pc @ONLY) install(FILES "${CMAKE_BINARY_DIR}/libblake3.pc" DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig" ) ``` -------------------------------- ### Configure Package Configuration Files Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Configures the blake3 package configuration files (config.cmake and config-version.cmake) and installs them. ```cmake include(CMakePackageConfigHelpers) configure_package_config_file(blake3-config.cmake.in "${CMAKE_CURRENT_BINARY_DIR}/blake3-config.cmake" INSTALL_DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/blake3" ) write_basic_package_version_file( "${CMAKE_CURRENT_BINARY_DIR}/blake3-config-version.cmake" VERSION ${libblake3_VERSION} COMPATIBILITY SameMajorVersion ) install(FILES "${CMAKE_CURRENT_BINARY_DIR}/blake3-config.cmake" "${CMAKE_CURRENT_BINARY_DIR}/blake3-config-version.cmake" DESTINATION "${CMAKE_INSTALL_LIBDIR}/cmake/blake3" ) ``` -------------------------------- ### Example BLAKE3 Hashing Program Source: https://github.com/blake3-team/blake3/blob/master/c/README.md This C program demonstrates how to hash bytes from standard input using the BLAKE3 algorithm and print the resulting hash in hexadecimal format. It requires including the 'blake3.h' header and links to standard C libraries. ```c #include "blake3.h" #include #include #include #include int main(void) { // Initialize the hasher. blake3_hasher hasher; blake3_hasher_init(&hasher); // Read input bytes from stdin. unsigned char buf[65536]; while (1) { ssize_t n = read(STDIN_FILENO, buf, sizeof(buf)); if (n > 0) { blake3_hasher_update(&hasher, buf, n); } else if (n == 0) { break; // end of file } else { fprintf(stderr, "read failed: %s\n", strerror(errno)); return 1; } } // Finalize the hash. BLAKE3_OUT_LEN is the default output length, 32 bytes. uint8_t output[BLAKE3_OUT_LEN]; blake3_hasher_finalize(&hasher, output, BLAKE3_OUT_LEN); // Print the hash as hexadecimal. for (size_t i = 0; i < BLAKE3_OUT_LEN; i++) { printf("%02x", output[i]); } printf("\n"); return 0; } ``` -------------------------------- ### Set BLAKE3 Target Include Directories Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Configures include directories for the BLAKE3 target, making the current source directory available during build and the install directory available during installation. ```cmake target_include_directories(blake3 PUBLIC $ $ ) ``` -------------------------------- ### Hash Input All at Once in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md Use `blake3::hash` for a simple, one-shot hashing of input bytes. No setup is required beyond adding the `blake3` crate as a dependency. ```rust let hash1 = blake3::hash(b"foobarbaz"); ``` -------------------------------- ### Finalize BLAKE3 Hash with Seek Offset Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Finalizes the BLAKE3 hash computation, allowing specification of a seek offset for the starting byte position in the output stream. This enables efficient streaming of large outputs without large memory allocations. ```c void blake3_hasher_finalize_seek( const blake3_hasher *self, uint64_t seek, uint8_t *out, size_t out_len); ``` -------------------------------- ### Hash Input Incrementally in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md For larger inputs or streaming data, use `blake3::Hasher` to update the hash incrementally. Call `finalize()` to get the final hash. This method is equivalent to the one-shot `blake3::hash`. ```rust let mut hasher = blake3::Hasher::new(); hasher.update(b"foo"); hasher.update(b"bar"); hasher.update(b"baz"); let hash2 = hasher.finalize(); ``` -------------------------------- ### Key Derivation Initialization (C String Context) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a BLAKE3 hasher for key derivation using a null-terminated C string as context. ```APIDOC ## Key Derivation Initialization (C String Context) ### Description Initializes a `blake3_hasher` in the key derivation mode. The context string is given as an initialization parameter, and afterwards input key material should be given with `blake3_hasher_update`. The context string is a null-terminated C string which should be **hardcoded, globally unique, and application-specific**. The context string should not include any dynamic input like salts, nonces, or identifiers read from a database at runtime. A good default format for the context string is `"[application] [commit timestamp] [purpose]"`, e.g., `"example.com 2019-12-25 16:18:03 session tokens v1"`. This function is intended for application code written in C. For language bindings, see `blake3_hasher_init_derive_key_raw` below. ### Method `void blake3_hasher_init_derive_key( blake3_hasher *self, const char *context); ` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response None ``` -------------------------------- ### Initialize BLAKE3 Hasher for Key Derivation (C String) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a BLAKE3 hasher for key derivation using a null-terminated C string context. The context should be hardcoded, globally unique, and application-specific. ```c void blake3_hasher_init_derive_key( blake3_hasher *self, const char *context); ``` -------------------------------- ### Key Derivation Initialization (Raw Context) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a BLAKE3 hasher for key derivation using a byte array and its length as context. Intended for language bindings. ```APIDOC ## Key Derivation Initialization (Raw Context) ### Description As `blake3_hasher_init_derive_key` above, except that the context string is given as a pointer to an array of arbitrary bytes with a provided length. This is intended for writing language bindings, where C string conversion would add unnecessary overhead and new error cases. Unicode strings should be encoded as UTF-8. Application code in C should prefer `blake3_hasher_init_derive_key`, which takes the context as a C string. If you need to use arbitrary bytes as a context string in application code, consider whether you're violating the requirement that context strings should be hardcoded. ### Method `void blake3_hasher_init_derive_key_raw( blake3_hasher *self, const void *context, size_t context_len); ` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response None ``` -------------------------------- ### Initialize BLAKE3 Hasher for Key Derivation (Raw Bytes) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a BLAKE3 hasher for key derivation using raw bytes for the context. This is intended for language bindings to avoid C string conversion overhead. Unicode strings should be UTF-8 encoded. ```c void blake3_hasher_init_derive_key_raw( blake3_hasher *self, const void *context, size_t context_len); ``` -------------------------------- ### Initialize BLAKE3 Hasher in Keyed Mode Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Use this function to initialize a BLAKE3 hasher for keyed hashing. The key must be exactly 32 bytes. ```c void blake3_hasher_init_keyed( blake3_hasher *self, const uint8_t key[BLAKE3_KEY_LEN]); ``` -------------------------------- ### Build b3sum from source Source: https://github.com/blake3-team/blake3/blob/master/b3sum/README.md Build the b3sum binary from the current directory using Cargo. The release build will be located in `./target/release/b3sum`. ```bash cargo build --release ``` -------------------------------- ### Enable TBB Parallelism and Fetch TBB with CMake Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile the BLAKE3 library with oneTBB parallelism enabled and configure CMake to fetch TBB if it's not found on the system. Requires a C++20 capable compiler. ```bash cmake -S c -B c/build "-DCMAKE_INSTALL_PREFIX=/usr/local" -DBLAKE3_USE_TBB=1 -DBLAKE3_FETCH_TBB=1 ``` -------------------------------- ### Run BLAKE3 Unit Tests Source: https://github.com/blake3-team/blake3/blob/master/CONTRIBUTING.md Execute the project's unit tests using Cargo. Ensure your code changes are covered by new tests. ```bash cargo test ``` -------------------------------- ### Build Shared Library with x86 Assembly Implementations Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile a shared BLAKE3 library on x86_64 Linux using GCC, linking against assembly implementations for SSE2, SSE4.1, AVX2, and AVX-512 instruction sets. This method is generally preferred for performance. ```bash gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c \ blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S blake3_avx2_x86-64_unix.S \ blake3_avx512_x86-64_unix.S ``` -------------------------------- ### Keyed Hashing Initialization Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a BLAKE3 hasher in keyed hashing mode. Requires a 32-byte key. ```APIDOC ## Keyed Hashing Initialization ### Description Initializes a `blake3_hasher` in the keyed hashing mode. The key must be exactly 32 bytes. ### Method `void blake3_hasher_init_keyed( blake3_hasher *self, const uint8_t key[BLAKE3_KEY_LEN]);` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response None ``` -------------------------------- ### Build Shared Library with x86 Intrinsics Implementations Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile a shared BLAKE3 library using GCC and C intrinsics for SSE2, SSE4.1, AVX2, and AVX-512. Each intrinsic implementation is compiled separately and then linked. ```bash gcc -c -fPIC -O3 -msse2 blake3_sse2.c -o blake3_sse2.o gcc -c -fPIC -O3 -msse4.1 blake3_sse41.c -o blake3_sse41.o gcc -c -fPIC -O3 -mavx2 blake3_avx2.c -o blake3_avx2.o gcc -c -fPIC -O3 -mavx512f -mavx512vl blake3_avx512.c -o blake3_avx512.o gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c \ blake3_avx2.o blake3_avx512.o blake3_sse41.o blake3_sse2.o ``` -------------------------------- ### Build Shared Library with ARM NEON Support Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile a shared BLAKE3 library on ARM Linux with NEON support enabled. This command assumes GCC and requires the NEON implementation file `blake3_neon.c`. ```bash gcc -shared -O3 -o libblake3.so -DBLAKE3_USE_NEON=1 blake3.c blake3_dispatch.c \ blake3_portable.c blake3_neon.c ``` -------------------------------- ### Hashing Files with b3sum Source: https://github.com/blake3-team/blake3/blob/master/b3sum/what_does_check_do.md Demonstrates the basic usage of `b3sum` to hash multiple files and generate a checkfile. This output format is consumed by `b3sum --check`. ```bash $ echo hi > a $ echo lo > b $ mkdir c $ echo stuff > c/d $ b3sum a b c/d 0b8b60248fad7ac6dfac221b7e01a8b91c772421a15b387dd1fb2d6a94aee438 a 6ae4a57bbba24f79c461d30bcb4db973b9427d9207877e34d2d74528daa84115 b 2d477356c962e54784f1c5dc5297718d92087006f6ee96b08aeaf7f3cd252377 c/d ``` -------------------------------- ### Build Shared Library with Portable Code Only (x86) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile a shared BLAKE3 library on x86, disabling all SIMD instruction sets (SSE2, SSE4.1, AVX2, AVX-512) to use only the portable C implementation. This is useful for maximum compatibility. ```bash gcc -shared -O3 -o libblake3.so -DBLAKE3_NO_SSE2 -DBLAKE3_NO_SSE41 -DBLAKE3_NO_AVX2 \ -DBLAKE3_NO_AVX512 blake3.c blake3_dispatch.c blake3_portable.c ``` -------------------------------- ### Initialize BLAKE3 Hasher Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a `blake3_hasher` structure in the default hashing mode. This function must be called before any other operations on the hasher. ```c void blake3_hasher_init( blake3_hasher *self); ``` -------------------------------- ### Initialize Hasher Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Initializes a `blake3_hasher` struct in the default hashing mode. ```APIDOC ## `blake3_hasher_init` ### Description Initialize a `blake3_hasher` in the default hashing mode. ### Method `void blake3_hasher_init(blake3_hasher *self);` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### Configure BLAKE3 Shared Library Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Sets up compilation definitions for shared libraries (DLLs) and adds the DLL definition to PKG_CONFIG_CFLAGS if building a shared library. ```cmake set(PKG_CONFIG_CFLAGS) if (BUILD_SHARED_LIBS) target_compile_definitions(blake3 PUBLIC BLAKE3_DLL PRIVATE BLAKE3_DLL_EXPORTS ) list(APPEND PKG_CONFIG_CFLAGS -DBLAKE3_DLL) endif() ``` -------------------------------- ### Hash a file with b3sum Source: https://github.com/blake3-team/blake3/blob/master/b3sum/README.md Use this command to calculate the BLAKE3 hash of a specified file. ```bash b3sum foo.txt ``` -------------------------------- ### Configure BLAKE3 AMD64 Assembly SIMD Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Enables BLAKE3 assembly sources for AMD64 architecture. Requires BLAKE3_AMD64_ASM_SOURCES to be defined. Initializes the ASM_MASM language for MSVC. ```cmake if(BLAKE3_SIMD_TYPE STREQUAL "amd64-asm") if (NOT DEFINED BLAKE3_AMD64_ASM_SOURCES) message(FATAL_ERROR "BLAKE3_SIMD_TYPE is set to 'amd64-asm' but no assembly sources are available for the target architecture.") endif() set(BLAKE3_SIMD_AMD64_ASM ON) if(MSVC) enable_language(ASM_MASM) endif() target_sources(blake3 PRIVATE ${BLAKE3_AMD64_ASM_SOURCES}) ``` -------------------------------- ### Fetch TBB using CMake FetchContent Source: https://github.com/blake3-team/blake3/blob/master/c/dependencies/tbb/CMakeLists.txt This snippet uses FetchContent to download the TBB library from a GitHub repository. It's designed for CMake versions 3.11 and later and includes options to control build behavior. ```cmake find_package(TBB 2021.11.0 QUIET) if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.11) include(FetchContent) if(NOT TBB_FOUND AND BLAKE3_FETCH_TBB) set(CMAKE_C_STANDARD 99) set(CMAKE_C_EXTENSIONS OFF) set(CMAKE_CXX_STANDARD 20) set(CMAKE_CXX_EXTENSIONS ON) option(TBB_TEST OFF "") option(TBBMALLOC_BUILD OFF "") mark_as_advanced(TBB_TEST) mark_as_advanced(TBBMALLOC_BUILD) FetchContent_Declare( TBB GIT_REPOSITORY https://github.com/uxlfoundation/oneTBB GIT_TAG 0c0ff192a2304e114bc9e6557582dfba101360ff # v2022.0.0 GIT_SHALLOW TRUE ) FetchContent_MakeAvailable(TBB) endif() endif() ``` -------------------------------- ### Compile BLAKE3 TBB Support Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile the optional C++ support file for BLAKE3 multithreading with oneTBB. Ensure exceptions and RTTI are disabled for compilation. ```bash g++ -c -O3 -fno-exceptions -fno-rtti -DBLAKE3_USE_TBB -o blake3_tbb.o blake3_tbb.cpp ``` -------------------------------- ### Build Shared Library with Portable Implementation Only Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile a shared BLAKE3 library using GCC, linking only the portable C implementation. This is a fallback for architectures without specific SIMD support or when SIMD is explicitly disabled. ```bash gcc -shared -O3 -o libblake3.so blake3.c blake3_dispatch.c blake3_portable.c ``` -------------------------------- ### Verifying Hashes with b3sum --check Source: https://github.com/blake3-team/blake3/blob/master/b3sum/what_does_check_do.md Shows how `b3sum --check` verifies file integrity using a previously generated checkfile. It exits with status zero on success and prints 'OK' for each file. ```bash $ b3sum a b c/d | b3sum --check a: OK b: OK c/d: OK ``` -------------------------------- ### Generate Feature Summary Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Generates and displays a summary of enabled features. ```cmake feature_summary(WHAT ENABLED_FEATURES) ``` -------------------------------- ### Define BLAKE3 Library Target Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Defines the BLAKE3 static library target using the core C source files and creates an alias for it. ```cmake # library target add_library(blake3 blake3.c blake3_dispatch.c blake3_portable.c ) add_library(BLAKE3::blake3 ALIAS blake3) ``` -------------------------------- ### Set BLAKE3 Target Properties Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Sets version, SOVERSION, and visibility properties for the BLAKE3 target. Ensures C extensions are off and sets the C standard to 99. ```cmake set_target_properties(blake3 PROPERTIES VERSION ${PROJECT_VERSION} SOVERSION 0 C_VISIBILITY_PRESET hidden C_EXTENSIONS OFF ) ``` -------------------------------- ### Integrate oneTBB for BLAKE3 Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Finds and integrates the oneTBB library for parallel processing with BLAKE3. Adds TBB sources, links TBB library, and defines BLAKE3_USE_TBB. Includes warnings if TBB is not found. ```cmake if(BLAKE3_USE_TBB) find_package(TBB 2021.11.0 QUIET) if(NOT TBB_FOUND AND NOT TARGET TBB::tbb) message(WARNING "oneTBB not found; disabling BLAKE3_USE_TBB\n" "Enable BLAKE3_FETCH_TBB to automatically fetch and build oneTBB" ) set(BLAKE3_USE_TBB OFF) else() target_sources(blake3 PRIVATE blake3_tbb.cpp) target_link_libraries(blake3 PUBLIC # Make shared TBB a transitive dependency. The consuming program is technically not required # to link TBB in order for libblake3 to function but we do this in order to prevent the # possibility of multiple separate TBB runtimes being linked into a final program in case # the consuming program also happens to already use TBB. TBB::tbb) target_compile_definitions(blake3 PUBLIC BLAKE3_USE_TBB) endif() if (CMAKE_SIZEOF_VOID_P EQUAL 8) set(TBB_PC_NAME tbb) else() set(TBB_PC_NAME tbb32) endif() list(APPEND PKG_CONFIG_REQUIRES "${TBB_PC_NAME} >= ${TBB_VERSION}") list(APPEND PKG_CONFIG_CFLAGS -DBLAKE3_USE_TBB) include(CheckCXXSymbolExists) check_cxx_symbol_exists(_LIBCPP_VERSION "version" BLAKE3_HAVE_LIBCPP) check_cxx_symbol_exists(__GLIBCXX__ "version" BLAKE3_HAVE_GLIBCXX) if(BLAKE3_HAVE_GLIBCXX) list(APPEND PKG_CONFIG_LIBS -lstdc++) elseif(BLAKE3_HAVE_LIBCPP) list(APPEND PKG_CONFIG_LIBS -lc++) endif() endif() ``` -------------------------------- ### Benchmark b3sum performance Source: https://github.com/blake3-team/blake3/blob/master/b3sum/README.md Compare the hashing speed of b3sum against other tools like openssl by hashing a large file. Ensure the file is created before timing. ```bash # Create a 1 GB file. head -c 1000000000 /dev/zero > /tmp/bigfile # Hash it with SHA-256. time openssl sha256 /tmp/bigfile # Hash it with BLAKE3. time b3sum /tmp/bigfile ``` -------------------------------- ### Configure BLAKE3 NEON Intrinsics SIMD Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Enables BLAKE3 NEON intrinsic sources. Defines BLAKE3_USE_NEON and optionally sets COMPILE_FLAGS if BLAKE3_CFLAGS_NEON is defined. ```cmake elseif(BLAKE3_SIMD_TYPE STREQUAL "neon-intrinsics") set(BLAKE3_SIMD_NEON_INTRINSICS ON) target_sources(blake3 PRIVATE blake3_neon.c ) target_compile_definitions(blake3 PRIVATE BLAKE3_USE_NEON=1 ) if (DEFINED BLAKE3_CFLAGS_NEON) set_source_files_properties(blake3_neon.c PROPERTIES COMPILE_FLAGS "${BLAKE3_CFLAGS_NEON}") endif() ``` -------------------------------- ### Handling File Changes with b3sum --check Source: https://github.com/blake3-team/blake3/blob/master/b3sum/what_does_check_do.md Illustrates `b3sum --check` behavior when files are modified or deleted. It exits with a non-zero status and reports 'FAILED' for discrepancies. ```bash $ b3sum a b c/d > checkfile $ rm b $ echo more stuff >> c/d $ b3sum --check checkfile a: OK b: FAILED (No such file or directory (os error 2)) c/d: FAILED ``` -------------------------------- ### Set BLAKE3 C++ Standard Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Compiles the BLAKE3 library using C++20 standard if CMake version is 3.12 or greater. Otherwise, it's handled later via BLAKE3_CMAKE_CXXFLAGS_*. ```cmake target_compile_features(blake3 PUBLIC c_std_99) if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.12) target_compile_features(blake3 PUBLIC cxx_std_20) # else: add it further below through `BLAKE3_CMAKE_CXXFLAGS_*` endif() ``` -------------------------------- ### Keyed Hash Input Incrementally in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md For incremental keyed hashing (MAC computation), initialize `blake3::Hasher` with `new_keyed` and use `update` and `finalize`. This is useful for streaming data that needs to be authenticated. ```rust let mut hasher = blake3::Hasher::new_keyed(&example_key); hasher.update(b"example input"); let mac2 = hasher.finalize(); ``` -------------------------------- ### Configure BLAKE3 No SIMD Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Disables all SIMD optimizations for BLAKE3. Defines preprocessor macros to indicate no SIMD support. ```cmake elseif(BLAKE3_SIMD_TYPE STREQUAL "none") target_compile_definitions(blake3 PRIVATE BLAKE3_USE_NEON=0 BLAKE3_NO_SSE2 BLAKE3_NO_SSE41 BLAKE3_NO_AVX2 BLAKE3_NO_AVX512 ) ``` -------------------------------- ### Add Feature Information Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Adds information about a specific feature, its corresponding CMake variable, and a description. ```cmake add_feature_info("AMD64 assembly" BLAKE3_SIMD_AMD64_ASM "The library uses hand written amd64 SIMD assembly.") add_feature_info("x86 SIMD intrinsics" BLAKE3_SIMD_X86_INTRINSICS "The library uses x86 SIMD intrinsics.") add_feature_info("NEON SIMD intrinsics" BLAKE3_SIMD_NEON_INTRINSICS "The library uses NEON SIMD intrinsics.") add_feature_info("oneTBB parallelism" BLAKE3_USE_TBB "The library uses oneTBB parallelism.") ``` -------------------------------- ### Configure BLAKE3 x86 Intrinsics SIMD Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Enables BLAKE3 x86 intrinsic sources for SSE2, SSE4.1, AVX2, and AVX512. Requires specific compiler flags to be defined. Sets compile flags for each source file. ```cmake elseif(BLAKE3_SIMD_TYPE STREQUAL "x86-intrinsics") if (NOT DEFINED BLAKE3_CFLAGS_SSE2 OR NOT DEFINED BLAKE3_CFLAGS_SSE4.1 OR NOT DEFINED BLAKE3_CFLAGS_AVX2 OR NOT DEFINED BLAKE3_CFLAGS_AVX512) message(FATAL_ERROR "BLAKE3_SIMD_TYPE is set to 'x86-intrinsics' but no compiler flags are available for the target architecture.") endif() set(BLAKE3_SIMD_X86_INTRINSICS ON) target_sources(blake3 PRIVATE blake3_avx2.c blake3_avx512.c blake3_sse2.c blake3_sse41.c ) set_source_files_properties(blake3_avx2.c PROPERTIES COMPILE_FLAGS "${BLAKE3_CFLAGS_AVX2}") set_source_files_properties(blake3_avx512.c PROPERTIES COMPILE_FLAGS "${BLAKE3_CFLAGS_AVX512}") set_source_files_properties(blake3_sse2.c PROPERTIES COMPILE_FLAGS "${BLAKE3_CFLAGS_SSE2}") set_source_files_properties(blake3_sse41.c PROPERTIES COMPILE_FLAGS "${BLAKE3_CFLAGS_SSE4.1}") ``` -------------------------------- ### CMake Function: Join PkgConfig Field Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt In-place rewrites a string list by joining its elements with a separator. Intended for pkg-config fields. ```cmake function(join_pkg_config_field sep requires) set(_requires "${${requires}}") list(LENGTH "${requires}" len) set(idx 1) foreach(req IN LISTS _requires) string(APPEND acc "${req}") if(idx LESS len) string(APPEND acc "${sep}") endif() math(EXPR idx "${idx} + 1") endforeach() set("${requires}" "${acc}" PARENT_SCOPE) endfunction() ``` -------------------------------- ### Reset BLAKE3 Hasher to Initial State Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Resets the BLAKE3 hasher to its initial state, equivalent to re-initializing it. ```c void blake3_hasher_reset( blake3_hasher *self); ``` -------------------------------- ### Creating a File with a Newline Character Source: https://github.com/blake3-team/blake3/blob/master/b3sum/what_does_check_do.md Uses Python to create a file named 'x\nx' which contains a newline character. This demonstrates how to produce file paths that require special handling. ```python open("x\nx", "w") ``` -------------------------------- ### Set BLAKE3 C++ Standard Flags Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Sets C++ standard flags for BLAKE3 compilation, supporting C++20 for GNU and MSVC compilers. Uses cache variables to store flags. ```cmake if(BLAKE3_USE_TBB) # Define some scratch variables for building appropriate flags per compiler if(CMAKE_VERSION VERSION_LESS 3.12) set(APPEND BLAKE3_CXX_STANDARD_FLAGS_GNU -std=c++20) set(APPEND BLAKE3_CXX_STANDARD_FLAGS_MSVC /std:c++20) endif() set(BLAKE3_CXXFLAGS_GNU "-fno-exceptions;-fno-rtti;${BLAKE3_CXX_STANDARD_FLAGS_GNU}" CACHE STRING "C++ flags used for compiling private BLAKE3 library components with GNU-like compiler frontends.") set(BLAKE3_CXXFLAGS_MSVC "/EHs-c-;/GR-;${BLAKE3_CXX_STANDARD_FLAGS_MSVC}" CACHE STRING "C++ flags used for compiling private BLAKE3 library components with MSVC-like compiler frontends.") if(BLAKE3_CXX_COMPILER_FRONTEND_VARIANT STREQUAL "GNU") target_compile_options(blake3 PRIVATE $<$:${BLAKE3_CXXFLAGS_GNU}>) elseif(BLAKE3_CXX_COMPILER_FRONTEND_VARIANT STREQUAL "MSVC") target_compile_options(blake3 PRIVATE $<$:${BLAKE3_CXXFLAGS_MSVC}>) endif() # Undefine scratch variables unset(BLAKE3_CXX_STANDARD_FLAGS_GNU) unset(BLAKE3_CXX_STANDARD_FLAGS_MSVC) unset(BLAKE3_CXXFLAGS_GNU) unset(BLAKE3_CXXFLAGS_MSVC) endif() ``` -------------------------------- ### Derive Keys with Context in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md Employ `blake3::derive_key` to generate unique subkeys for different purposes from input key material, using a hardcoded, application-specific context string. The context string must be unique and globally defined for security. ```rust const EMAIL_CONTEXT: &str = "BLAKE3 example 2020-01-07 17:10:44 email key"; const API_CONTEXT: &str = "BLAKE3 example 2020-01-07 17:11:21 API key"; let input_key_material = b"usually at least 32 random bytes, not a password"; let email_key = blake3::derive_key(EMAIL_CONTEXT, input_key_material); let api_key = blake3::derive_key(API_CONTEXT, input_key_material); ``` -------------------------------- ### Build Shared Library Disabling ARM NEON Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Compile a shared BLAKE3 library on ARM Linux explicitly disabling NEON instructions. This ensures only the portable implementation is used. ```bash gcc -shared -O3 -o libblake3.so -DBLAKE3_USE_NEON=0 blake3.c blake3_dispatch.c \ blake3_portable.c ``` -------------------------------- ### Keyed Hash Input All at Once in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md Use `blake3::keyed_hash` to compute a Message Authentication Code (MAC) using a 256-bit key. This is suitable for verifying data integrity and authenticity when a secret key is available. ```rust let example_key = [42u8; 32]; let mac1 = blake3::keyed_hash(&example_key, b"example input"); ``` -------------------------------- ### Update BLAKE3 Hasher with Input Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Adds input data to the BLAKE3 hasher. This function can be called multiple times to incrementally hash data. It is designed for single-threaded use; multithreaded updates are handled by `blake3_hasher_update_tbb`. ```c void blake3_hasher_update( blake3_hasher *self, const void *input, size_t input_len); ``` -------------------------------- ### Update BLAKE3 Hasher with Parallel Processing (oneTBB) Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Adds input to the hasher using oneTBB for parallel processing of large inputs. This function is only enabled when compiled with BLAKE3_USE_TBB and oneTBB is detected. It can be slower than `blake3_hasher_update` for inputs under 128 KiB. ```c void blake3_hasher_update_tbb( blake3_hasher *self, const void *input, size_t input_len); ``` -------------------------------- ### CMake Function: Join Paths Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt A CMake function to join path segments, similar to Python's os.path.join. Does not support Windows paths. ```cmake function(join_paths joined_path first_path_segment) set(temp_path "${first_path_segment}") foreach(current_segment IN LISTS ARGN) if(NOT ("${current_segment}" STREQUAL "")) if(IS_ABSOLUTE "${current_segment}") set(temp_path "${current_segment}") else() set(temp_path "${temp_path}/${current_segment}") endif() endif() endforeach() set(${joined_path} "${temp_path}" PARENT_SCOPE) endfunction() ``` -------------------------------- ### Handle Unknown SIMD Type Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Generates a fatal error if BLAKE3_SIMD_TYPE is set to an unrecognized value. ```cmake else() message(FATAL_ERROR "BLAKE3_SIMD_TYPE is set to an unknown value: '${BLAKE3_SIMD_TYPE}'") endif() ``` -------------------------------- ### Creating a file with invalid UTF-8 name in Python Source: https://github.com/blake3-team/blake3/blob/master/b3sum/what_does_check_do.md Shows how to create a file with a name containing bytes that are not valid UTF-8. This is possible on some systems like Linux, illustrating a scenario where Unicode conversion fails. ```python >>> open(b"y\xFFy", "w") ``` -------------------------------- ### Extended Output Hashing in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md Obtain an extended output (XOF) from a `blake3::Hasher` using `finalize_xof()`. The resulting `OutputReader` implements `Read` and `Seek`, allowing you to read arbitrary amounts of hash output. Ensure the hasher is finalized before reading. ```rust let mut output = [0; 1000]; let mut output_reader = hasher.finalize_xof(); output_reader.fill(&mut output); assert_eq!(hash1, output[..32]); ``` -------------------------------- ### Print Hash as Hex in Rust Source: https://github.com/blake3-team/blake3/blob/master/README.md The `blake3::Hash` type implements `Display`, allowing it to be easily printed as a hexadecimal string using `println!` or similar formatting macros. ```rust println!("{}", hash1); ``` -------------------------------- ### Configure GNU/Clang Compiler Flags for SIMD Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Sets compiler flags for SSE2, SSE4.1, AVX2, and AVX512 for GNU, Clang, and AppleClang compilers. It also defines assembly source files based on whether the system is Windows or Unix. ```cmake elseif(CMAKE_C_COMPILER_ID STREQUAL "GNU" OR CMAKE_C_COMPILER_ID STREQUAL "Clang" OR CMAKE_C_COMPILER_ID STREQUAL "AppleClang") set(BLAKE3_CFLAGS_SSE2 "-msse2" CACHE STRING "the compiler flags to enable SSE2") set(BLAKE3_CFLAGS_SSE4.1 "-msse4.1" CACHE STRING "the compiler flags to enable SSE4.1") set(BLAKE3_CFLAGS_AVX2 "-mavx2" CACHE STRING "the compiler flags to enable AVX2") set(BLAKE3_CFLAGS_AVX512 "-mavx512f -mavx512vl" CACHE STRING "the compiler flags to enable AVX512") if (WIN32 OR CYGWIN) set(BLAKE3_AMD64_ASM_SOURCES blake3_avx2_x86-64_windows_gnu.S blake3_avx512_x86-64_windows_gnu.S blake3_sse2_x86-64_windows_gnu.S blake3_sse41_x86-64_windows_gnu.S ) elseif(UNIX) set(BLAKE3_AMD64_ASM_SOURCES blake3_avx2_x86-64_unix.S blake3_avx512_x86-64_unix.S blake3_sse2_x86-64_unix.S blake3_sse41_x86-64_unix.S ) endif() if (CMAKE_SYSTEM_PROCESSOR IN_LIST BLAKE3_ARMv8_NAMES AND NOT CMAKE_SIZEOF_VOID_P EQUAL 8) # 32-bit ARMv8 needs NEON to be enabled explicitly set(BLAKE3_CFLAGS_NEON "-mfpu=neon" CACHE STRING "the compiler flags to enable NEON") endif() endif() ``` -------------------------------- ### Conditional Include for Testing Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Includes testing scripts if the BLAKE3_TESTING variable is set. ```cmake if(BLAKE3_TESTING) include(BLAKE3/Testing) endif() ``` -------------------------------- ### BLAKE3 Hasher Struct Source: https://github.com/blake3-team/blake3/blob/master/c/README.md The `blake3_hasher` struct represents the state of an incremental BLAKE3 hashing process. It can accept any number of updates and does not allocate heap memory. ```APIDOC ## The Struct ```c typedef struct { // private fields } blake3_hasher; ``` ### Description An incremental BLAKE3 hashing state, which can accept any number of updates. This implementation doesn't allocate any heap memory, but `sizeof(blake3_hasher)` itself is relatively large, currently 1912 bytes on x86-64. This size can be reduced by restricting the maximum input length, as described in Section 5.4 of [the BLAKE3 spec](https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blake3.pdf), but this implementation doesn't currently support that strategy. ``` -------------------------------- ### BLAKE3 Hasher Struct Definition Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Defines the structure for an incremental BLAKE3 hashing state. This structure holds the internal state and does not allocate heap memory. Its size can be reduced by limiting the maximum input length as per the BLAKE3 specification. ```c typedef struct { // private fields } blake3_hasher; ``` -------------------------------- ### Update Hasher with Input Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Adds input data to the hasher. This function can be called multiple times to incrementally hash data. It is a single-threaded operation. ```APIDOC ## `blake3_hasher_update` ### Description Add input to the hasher. This can be called any number of times. This function is always single-threaded; for multithreading see `blake3_hasher_update_tbb` below. ### Method `void blake3_hasher_update(blake3_hasher *self, const void *input, size_t input_len);` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### Reset Hasher Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Resets the hasher to its initial state, allowing it to be reused without re-initialization. ```APIDOC ## Reset Hasher ### Description Reset the hasher to its initial state, prior to any calls to `blake3_hasher_update`. Currently this is no different from calling `blake3_hasher_init` or similar again. ### Method `void blake3_hasher_reset( blake3_hasher *self); ` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response None ``` -------------------------------- ### Threaded Update Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Adds input to the hasher using oneTBB for parallel processing of large inputs. Provides the same result as `blake3_hasher_update`. ```APIDOC ## Threaded Update ### Description Add input to the hasher, using [oneTBB] to process large inputs using multiple threads. This can be called any number of times. This gives the same result as `blake3_hasher_update` above. NOTE: This function is only enabled when the library is compiled with CMake option `BLAKE3_USE_TBB` and when the oneTBB library is detected on the host system. See the building instructions for further details. To get any performance benefit from multithreading, the input buffer needs to be large. As a rule of thumb on x86_64, `blake3_hasher_update_tbb` is _slower_ than `blake3_hasher_update` for inputs under 128 KiB. That threshold varies quite a lot across different processors, and it's important to benchmark your specific use case. Hashing large files with this function usually requires [memory-mapping](https://en.wikipedia.org/wiki/Memory-mapped_file), since reading a file into memory in a single-threaded loop takes longer than hashing the resulting buffer. Note that hashing a memory-mapped file with this function produces a "random" pattern of disk reads, which can be slow on spinning disks. Again it's important to benchmark your specific use case. This implementation doesn't require configuration of thread resources and will use as many cores as possible by default. More fine-grained control of resources is possible using the [oneTBB] API. ### Method `void blake3_hasher_update_tbb( blake3_hasher *self, const void *input, size_t input_len); ` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response None ``` -------------------------------- ### Finalize BLAKE3 Hash Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Finalizes the BLAKE3 hashing process and returns the computed hash. The output length can be specified in bytes. This function does not modify the hasher, allowing for multiple finalizations or further updates. ```c void blake3_hasher_finalize( const blake3_hasher *self, uint8_t *out, size_t out_len); ``` -------------------------------- ### Mark Advanced CMake Variables Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Marks several BLAKE3-related CMake variables as advanced, meaning they will be hidden from the main CMake GUI interface unless explicitly shown. This helps keep the interface clean. ```cmake mark_as_advanced(BLAKE3_CFLAGS_SSE2 BLAKE3_CFLAGS_SSE4.1 BLAKE3_CFLAGS_AVX2 BLAKE3_CFLAGS_AVX512 BLAKE3_CFLAGS_NEON) ``` ```cmake mark_as_advanced(BLAKE3_AMD64_ASM_SOURCES) ``` -------------------------------- ### Finalize Hasher Source: https://github.com/blake3-team/blake3/blob/master/c/README.md Finalizes the hashing process and returns the computed hash digest. The output length can be specified in bytes. The hasher remains unmodified, allowing for further updates and finalizations. ```APIDOC ## `blake3_hasher_finalize` ### Description Finalize the hasher and return an output of any length, given in bytes. This doesn't modify the hasher itself, and it's possible to finalize again after adding more input. The constant `BLAKE3_OUT_LEN` provides the default output length, 32 bytes, which is recommended for most callers. See the [Security Notes](#security-notes) below. ### Method `void blake3_hasher_finalize(const blake3_hasher *self, uint8_t *out, size_t out_len);` ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (200) None #### Response Example None ``` -------------------------------- ### Respect C_EXTENSIONS OFF Policy Source: https://github.com/blake3-team/blake3/blob/master/c/CMakeLists.txt Ensures C_EXTENSIONS OFF is respected without overriding CMAKE_C_STANDARD, particularly when CMAKE_C_STANDARD is not defined by the user or toolchain. ```cmake if (NOT POLICY CMP0128 AND NOT DEFINED CMAKE_C_STANDARD) set_target_properties(blake3 PROPERTIES C_STANDARD 99) endif() ``` -------------------------------- ### Python UnicodeDecodeError Source: https://github.com/blake3-team/blake3/blob/master/b3sum/what_does_check_do.md Demonstrates a UnicodeDecodeError when attempting to decode a byte that is invalid in UTF-8. This highlights a limitation when dealing with arbitrary byte sequences as filepaths. ```python >>> b"\xFF".decode("UTF-8") UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte ```