### Install Python Package Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/python-package/README.md Clone the repository, navigate to the example directory, and install the package using pip. ```bash git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow/examples/python-package python -m pip install . ``` -------------------------------- ### Install nanoarrow from GitHub Source: https://github.com/apache/arrow-nanoarrow/blob/main/r/README.md Install the development version of the nanoarrow package from GitHub. Ensure 'remotes' is installed first if necessary. ```r # install.packages("remotes") remotes::install_github("apache/arrow-nanoarrow/r") ``` -------------------------------- ### Install Fedora Prerequisites Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Installs required packages on Fedora systems for building nanoarrow. ```bash dnf install -y git cmake R gnupg curl python3-devel python3-virtualenv awk ``` -------------------------------- ### Conda Environment Setup for Verification Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Create and activate a conda environment, set channel priority, and install necessary tools like compilers, git, cmake, and R packages for verification. ```bash conda create --name nanoarrow-verify-rc conda activate nanoarrow-verify-rc conda config --set channel_priority strict conda install -c conda-forge compilers git cmake # For R (see below about potential interactions with system R # before installing via conda on MacOS) conda install -c conda-forge r-testthat r-hms r-blob r-pkgbuild r-bit64 ``` -------------------------------- ### Install Debian/Ubuntu Prerequisites Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Installs necessary build tools and development libraries on Debian/Ubuntu systems for compiling nanoarrow. ```bash apt-get update && apt-get install -y git g++ cmake r-base gnupg curl python3-dev python3-venv ``` -------------------------------- ### Build the minimal Meson example Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/meson-minimal/README.md Clone the repository, set up the Meson build directory, and compile the project. ```shell git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow/examples/meson-minimal Meson setup builddir && cd builddir Meson compile ``` -------------------------------- ### Install Nanoarrow Wrap Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/python-package/README.md Create a subprojects directory and install the nanoarrow wrap definition file using Meson. ```bash mkdir -p subprojects meson wrap install nanoarrow ``` -------------------------------- ### Build nanoarrow IPC Example with CMake Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-ipc/README.md Clone the repository, navigate to the example directory, create a build directory, and then configure and build the project using CMake. ```bash git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow/examples/cmake-ipc mkdir build && cd build cmake .. cmake --build . ``` -------------------------------- ### Install Arch Linux Prerequisites Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Installs necessary development tools and libraries on Arch Linux using pacman. ```bash pacman -Sy git gcc make cmake r-base gnupg curl python ``` -------------------------------- ### Run the minimal Meson example application Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/meson-minimal/README.md Execute the compiled application from the build directory to parse arguments into an int32 array. ```shell ./example_meson_minimal_app ``` -------------------------------- ### Basic CMake Project Setup Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-scenarios/CMakeLists.txt Initializes a CMake project with C/C++ support and sets the C++ standard. This is a foundational step for any CMake project. ```cmake cmake_minimum_required(VERSION 3.12) project(NanoArrowExampleCMakeMinimal LANGUAGES C CXX) set(CMAKE_CXX_STANDARD 11) ``` -------------------------------- ### Install Alpine Linux Prerequisites Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Installs required packages on Alpine Linux using apk add. ```bash apk add bash linux-headers git cmake R R-dev g++ gnupg curl python3-dev ``` -------------------------------- ### Install Nanoarrow Device Headers Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Installs the nanoarrow device header files to the include directory. ```cmake install(FILES src/nanoarrow/nanoarrow_device.h src/nanoarrow/nanoarrow_device.hpp DESTINATION include/nanoarrow) ``` -------------------------------- ### Install nanoarrow Source: https://github.com/apache/arrow-nanoarrow/blob/main/python/README.ipynb Install nanoarrow from PyPI or conda-forge. Development versions are also available. ```shell pip install nanoarrow conda install nanoarrow -c conda-forge ``` ```shell pip install --extra-index-url https://pypi.fury.io/arrow-nightlies/ \ --prefer-binary --pre nanoarrow ``` -------------------------------- ### Setup nanoarrow build with Meson Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Initializes the build directory for nanoarrow using the Meson build system. This is an experimental feature. ```sh meson setup builddir cd builddir ``` -------------------------------- ### Run the vendored minimal application Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/vendored-minimal/README.md Execute the compiled application. This example parses command-line arguments into an int32 array and prints the resulting array length or any errors. ```bash ./example_vendored_minimal_app # 1 # 2 # 3 ``` -------------------------------- ### Install nanoarrow with Meson WrapDB Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/meson-minimal/README.md Use `meson wrap install` to add nanoarrow as a dependency to your Meson project. ```shell mkdir subprojects meson wrap install nanoarrow ``` -------------------------------- ### Run Python Benchmarks with ASV Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/benchmarks/README.md Installs ASV, generates fixtures, and runs Python benchmarks. Requires pyarrow. ```shell # pip install asv python generate-fixtures.py # requires pyarrow asv run ``` -------------------------------- ### Install Nanoarrow Testing Header Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Installs the nanoarrow testing header file to the include directory. ```cmake install(FILES src/nanoarrow/nanoarrow_testing.hpp DESTINATION include/nanoarrow) ``` -------------------------------- ### Build nanoarrow Project with CMake Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-minimal/README.md Clone the nanoarrow repository, navigate to the example directory, create a build folder, and then configure and build the project using CMake. ```bash git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow/examples/cmake-minimal mkdir build && cd build cmake .. cmake --build . ``` -------------------------------- ### Install nlohmann_json Target Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Installs the nlohmann_json target, making it available for use after installation. This includes runtime, library, and archive destinations. ```cmake install(TARGETS nlohmann_json EXPORT nanoarrow-exports RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) ``` -------------------------------- ### Install nanoarrow Python Package Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Install the nanoarrow Python bindings using pip or conda. ```sh pip install nanoarrow ``` ```sh conda install nanoarrow -c conda-forge ``` -------------------------------- ### Install Targets for Export Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt This code block handles the installation of targets, exporting them under the 'nanoarrow-exports' configuration. It specifies runtime, library, and archive destinations based on whether the target is a shared library and if shared installation is enabled. ```cmake # Ensure target is added to nanoarrow-exports if(((target_type STREQUAL "SHARED_LIBRARY") AND NANOARROW_INSTALL_SHARED) OR NOT (target_type STREQUAL "SHARED_LIBRARY")) install(TARGETS ${target} EXPORT nanoarrow-exports RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) endif() ``` -------------------------------- ### Editable Install with Debug Build Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/python-package/README.md Perform an editable install with custom build options, specifying the build directory and build type. ```bash python -m pip install -e . -Cbuilddir=builddir -Csetup-args="-Dbuildtype=debug" ``` -------------------------------- ### Install nanoarrow R Package Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Install the nanoarrow R package from CRAN. ```r install.packages("nanoarrow") ``` -------------------------------- ### Install macOS Dependencies with Homebrew Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Install CMake and GnuPG on macOS using Homebrew for release candidate verification. ```bash brew install cmake gnupg ``` -------------------------------- ### Configure IPC Libraries and Include Directories Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Sets up static and shared libraries for nanoarrow_ipc and configures include directories for build and installation. ```cmake install(FILES src/nanoarrow/nanoarrow_ipc.h src/nanoarrow/nanoarrow_ipc.hpp src/nanoarrow/ipc/flatcc_generated.h DESTINATION include/nanoarrow) if(BUILD_SHARED_LIBS) add_library(nanoarrow::nanoarrow_ipc ALIAS nanoarrow_ipc_shared) else() add_library(nanoarrow::nanoarrow_ipc ALIAS nanoarrow_ipc_static) endif() foreach(target "nanoarrow_ipc_static" "nanoarrow_ipc_shared") target_compile_definitions(${target} PRIVATE ${NANOARROW_IPC_EXTRA_FLAGS}) target_include_directories(${target} PUBLIC $ $ $ $) endforeach() ``` -------------------------------- ### Build nanoarrow Documentation Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/README.md Build the nanoarrow documentation using Docker Compose or by installing prerequisites and running the build script. After building, use sphinx-build to iterate on changes. ```bash docker compose run --rm docs ``` ```bash ci/scripts/build-docs.sh ``` ```bash cd docs sphinx-build source _build/html ``` -------------------------------- ### Build and Upload Python Package Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Commands to clean, build, and upload the Python package distribution to PyPI. Ensure you have twine installed and a PyPI upload token. ```shell rm -rf python/dist mkdir python/dist pushd python/dist gh run download 15963020465 rm -rf release-wheels-pyodide popd twine upload python/dist/**/*.tar.gz python/dist/**/*.whl ``` -------------------------------- ### CMakeLists.txt for C++ Project Setup Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md Configures a C++ project using CMake, including fetching and integrating the nanoarrow library. This file is essential for building the project with CMake and IDEs like VSCode. ```cmake project(linesplitter) set(CMAKE_CXX_STANDARD 11) include(FetchContent) FetchContent_Declare( nanoarrow URL https://github.com/apache/arrow-nanoarrow/releases/download/apache-arrow-nanoarrow-0.2.0/apache-arrow-nanoarrow-0.2.0.tar.gz URL_HASH SHA512=38a100ae5c36a33aa330010eb27b051cff98671e9c82fff22b1692bb77ae61bd6dc2a52ac6922c6c8657bd4c79a059ab26e8413de8169eeed3c9b7fdb216c817)FetchContent_MakeAvailable(nanoarrow) add_library(linesplitter linesplitter.cc) target_link_libraries(linesplitter PRIVATE nanoarrow_static) ``` -------------------------------- ### Import nanoarrow Source: https://github.com/apache/arrow-nanoarrow/blob/main/python/README.ipynb Import the nanoarrow package to start using its functionalities. ```python import nanoarrow as na ``` -------------------------------- ### Use nanoarrow as a Meson Subproject Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Integrate nanoarrow into a Meson build system by installing it as a subproject and declaring it as a dependency. ```meson nanoarrow_dep = dependency('nanoarrow') example_exec = executable('example_meson_minimal_app', 'src/app.cc', dependencies: [nanoarrow_dep]) ``` -------------------------------- ### Run nanoarrow Tests Source: https://github.com/apache/arrow-nanoarrow/blob/main/python/README.ipynb Install test dependencies and run the test suite using pytest. Ensure CMake is installed as it's required for vendored nanoarrow synchronization. ```shell # Install dependencies pip install -e ".[test]" # Run tests pytest -vvx ``` -------------------------------- ### Build nanoarrow Python Package Source: https://github.com/apache/arrow-nanoarrow/blob/main/python/README.ipynb Build the nanoarrow Python package using setuptools. This command installs the package in editable mode, useful during development. ```shell git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow/python # Build dependencies: # pip install meson meson-python cython pip install -e . --no-build-isolation ``` -------------------------------- ### Build nanoarrow with CMake, Tests, and Arrow C++ Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Enables building nanoarrow with tests and integrates with Arrow C++ for more comprehensive testing. Requires Arrow C++ to be installed. ```sh mkdir build && cd build cmake .. -DNANOARROW_BUILD_TESTS=ON -DNANOARROW_BUILD_TESTS_WITH_ARROW=ON cmake --build . ``` -------------------------------- ### Print Schema using schema_printer Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/python-package/README.md Demonstrates printing an Arrow schema using the schema_printer module after installing the package. ```python >>> import pyarrow as pa >>> import schema_printer >>> schema = pa.schema([("some int", pa.int32()), ("some_string", pa.string())]) >>> schema some int: int32 some_string: string >>> schema_printer.print_schema(schema) Field: some int, Type: int32 Field: some_string, Type: string ``` -------------------------------- ### Configuring Nanoarrow Dependency Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-scenarios/CMakeLists.txt Sets up how Nanoarrow is found or fetched. It supports finding an installed Nanoarrow package or fetching it directly from source using FetchContent. ```cmake option(FIND_NANOARROW "Find an existing nanoarrow" ON) # When adding nanoarrow's CMake directory to a CMake project that contains a library # intended for use by others, set NANOARROW_NAMESPACE to rename symbols in the # nanoarrow library such that they do not collide with other libraries that may also # link to their own copy of nanoarrow. You may wish to include the namespace only # on release builds, since the namespace implementation obscures inline help # available in many text editors. set(NANOARROW_NAMESPACE "ExampleCmakeScenarios") include(FetchContent) if(FIND_NANOARROW) # Users should pin to a specific version of nanoarrow if they choose # to find nanoarrow rather than pinning to a specific version via # FetchContent. find_package(nanoarrow REQUIRED) else() # We need all these components for our test set(NANOARROW_DEVICE ON) set(NANOARROW_IPC ON) set(NANOARROW_TESTING ON) # Using SOURCE_DIR here such that CI accurately reflects the state of the # source; however, using GIT_TAG or URL of a pinned version is a more realistic # user pattern. fetchcontent_declare(nanoarrow SOURCE_DIR "${CMAKE_CURRENT_LIST_DIR}/../..") fetchcontent_makeavailable(nanoarrow) endif() ``` -------------------------------- ### Configure nanoarrow build options with Meson Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Configures build options for nanoarrow using Meson, such as enabling tests and benchmarks. Use `-Dtest_with_arrow=enabled` if Arrow C++ is installed. ```sh meson configure -Dtests=enabled -Dbenchmarks=enabled ``` -------------------------------- ### Linking Nanoarrow Libraries to Executable Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-scenarios/CMakeLists.txt Links the Nanoarrow libraries to the main application executable. This example demonstrates conditional linking based on the build type (static, shared, or default). ```cmake add_executable(minimal_cpp_app src/app.cpp) if(TEST_BUILD_TYPE STREQUAL "static") target_link_libraries(minimal_cpp_app nanoarrow::nanoarrow_static nanoarrow::nanoarrow_device_static nanoarrow::nanoarrow_ipc_static nanoarrow::nanoarrow_testing_static) elseif(TEST_BUILD_TYPE STREQUAL "shared") target_link_libraries(minimal_cpp_app nanoarrow::nanoarrow_shared nanoarrow::nanoarrow_device_shared nanoarrow::nanoarrow_ipc_shared nanoarrow::nanoarrow_testing_shared) else() target_link_libraries(minimal_cpp_app nanoarrow::nanoarrow nanoarrow::nanoarrow_device nanoarrow::nanoarrow_ipc nanoarrow::nanoarrow_testing) endif() ``` -------------------------------- ### Run Source Script Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Execute the source script to prepare release artifacts. This requires the version and release candidate number as arguments and the .env file to be configured. ```bash # 03-source.sh $0 dev/release/03-source.sh 0.8.0 0 ``` -------------------------------- ### Run Signing Script Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Execute the signing script with the specified version and release candidate number. Ensure the .env file is configured with APACHE_USERNAME. ```bash dev/release/02-sign.sh 0.8.0 0 ``` -------------------------------- ### Run the CMake Minimal Application Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-minimal/README.md After building the project, execute the application from the build directory. The application parses command-line arguments into an int32 array and prints the resulting length or any errors. ```bash ./example_cmake_minimal_app # 1 # 2 # 3 ``` -------------------------------- ### Clone and Prepare Documentation Site Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Clones the asf-site branch and downloads release documentation archives. This is a prerequisite for updating the project's website. ```shell git clone -b asf-site --single-branch https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow ``` ```shell curl -L https://github.com/apache/arrow-nanoarrow/releases/download/apache-arrow-nanoarrow-0.8.0/docs.tgz \ -o docs.tgz ``` ```shell tar -xvzf docs.tgz mv nanoarrow-docs 0.8.0 ``` ```shell rm -rf latest tar -xvzf docs.tgz mv nanoarrow-docs latest ``` -------------------------------- ### Build and link the vendored library and application Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/vendored-minimal/README.md Compile the bundled nanoarrow source and your library code, then link them with your application. This ensures nanoarrow is statically linked. ```bash cd src cc -c library.c nanoarrow.c ar rcs libexample_vendored_minimal_library.a library.o nanoarrow.o cc -o example_vendored_minimal_app app.c libexample_vendored_minimal_library.a ``` -------------------------------- ### Load nanoarrow package Source: https://github.com/apache/arrow-nanoarrow/blob/main/r/README.md Load the nanoarrow package into your R session to start using its functions. ```r library(nanoarrow) ``` -------------------------------- ### Compile Vendored IPC Library and Application Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/vendored-ipc/README.md Compile the vendored nanoarrow and IPC C sources into a static library, then link it with the application code to create an executable. ```bash cd src cc -c library.c nanoarrow.c flatcc.c nanoarrow_ipc.c -I. ar rcs libexample_vendored_ipc_library.a library.o nanoarrow.o nanoarrow_ipc.o flatcc.o cc -o example_vendored_ipc_app app.c libexample_vendored_ipc_library.a ``` -------------------------------- ### Build Metal Implementation Library Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Builds a static library for the Metal device implementation, linking against Metal frameworks and setting include directories. ```cmake set(NANOARROW_DEVICE_INCLUDE_METAL "${CMAKE_BINARY_DIR}/metal-cpp") add_library(nanoarrow_metal_impl STATIC src/nanoarrow/device/metal_impl.cc) target_link_libraries(nanoarrow_metal_impl PRIVATE ${METAL_LIBRARY} ${FOUNDATION_LIBRARY} ${QUARTZ_CORE_LIBRARY}) target_include_directories(nanoarrow_metal_impl PRIVATE ${NANOARROW_DEVICE_INCLUDE_METAL}) install(TARGETS nanoarrow_metal_impl EXPORT nanoarrow-exports RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}) ``` -------------------------------- ### Generate Full C Library Benchmark Report Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/benchmarks/README.md Generates fixtures, runs all C library benchmarks, and renders a markdown report. Requires pyarrow and doxygen. ```shell python generate-fixtures.py # requires pyarrow ./benchmark-run-all.sh cd apidoc && doxygen && cd .. quarto render benchmark-report.qmd ``` -------------------------------- ### Package Configuration File Generation Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Generates package configuration files (nanoarrow-config-version.cmake and nanoarrow-config.cmake) for both build and install trees. Captures BUILD_SHARED_LIBS at configure time. ```cmake # Generate package files for the build and install trees. include(CMakePackageConfigHelpers) # Capture BUILD_SHARED_LIBS at configure time for use in the config file if(BUILD_SHARED_LIBS) set(NANOARROW_BUILD_SHARED_LIBS "ON") else() set(NANOARROW_BUILD_SHARED_LIBS "OFF") endif() foreach(tree_type BUILD INSTALL) if(tree_type STREQUAL "BUILD") set(install_location ".") else() set(install_location "${CMAKE_INSTALL_LIBDIR}/cmake/nanoarrow") endif() set(build_location "${PROJECT_BINARY_DIR}/${install_location}") write_basic_package_version_file( "${build_location}/nanoarrow-config-version.cmake" VERSION ${nanoarrow_VERSION} # After 1.0.0, we can use `SameMajorVersion` here. COMPATIBILITY ExactVersion) configure_package_config_file("${CMAKE_CURRENT_LIST_DIR}/cmake/config.cmake.in" "${build_location}/nanoarrow-config.cmake" INSTALL_DESTINATION "${install_location}") if(tree_type STREQUAL "BUILD") export(EXPORT nanoarrow-exports FILE "${build_location}/nanoarrow-targets.cmake" NAMESPACE nanoarrow::) else() install(DIRECTORY "${build_location}/" DESTINATION "${install_location}") install(EXPORT nanoarrow-exports DESTINATION "${install_location}" FILE "nanoarrow-targets.cmake" NAMESPACE nanoarrow::) endif() endforeach() ``` -------------------------------- ### Prepare Release Candidate Script Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Executes the script to prepare a release candidate, updating version numbers, changelog, and creating a git tag. Ensure you are in the repository root. ```bash # from the repository root # 01-prepare.sh dev/release/01-prepare.sh . 0.7.0 0.8.0 0.9.0 0 ``` -------------------------------- ### Configure C++ Standard and Find Metal Frameworks Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Sets the C++ standard to 17 and finds required Metal-related system libraries (Metal, Foundation, QuartzCore). ```cmake if(NOT DEFINED CMAKE_CXX_STANDARD) set(CMAKE_CXX_STANDARD 17) endif() set(CMAKE_CXX_STANDARD_REQUIRED ON) find_library(METAL_LIBRARY Metal REQUIRED) message(STATUS "Metal framework found at '${METAL_LIBRARY}'") find_library(FOUNDATION_LIBRARY Foundation REQUIRED) message(STATUS "Foundation framework found at '${FOUNDATION_LIBRARY}'") find_library(QUARTZ_CORE_LIBRARY QuartzCore REQUIRED) message(STATUS "CoreFoundation framework found at '${QUARTZ_CORE_LIBRARY}'") ``` -------------------------------- ### Run C Library Benchmarks Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/benchmarks/README.md Build and run C library benchmarks for a specific configuration. Requires CMake and ctest. ```shell mkdir build && cd build cmake .. --preset local cmake --build . ctest ``` -------------------------------- ### CMakeLists.txt for googletest Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md CMake configuration to fetch and build googletest, define an executable for tests, and link against the library and GTest. ```cmake FetchContent_Declare( googletest URL https://github.com/google/googletest/archive/refs/tags/v1.13.0.zip ) FetchContent_MakeAvailable(googletest) enable_testing() add_executable(linesplitter_test linesplitter_test.cc) target_link_libraries(linesplitter_test linesplitter GTest::gtest_main) include(GoogleTest) gtest_discover_tests(linesplitter_test) ``` -------------------------------- ### Run Build Script Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-scenarios/README.md Execute the build script to test various nanoarrow integration scenarios with CMake. ```bash #!/bin/bash set -e # Clean previous builds rm -rf build mkdir build cd build # Build with nanoarrow fetched from source cmake .. -DNANOARROW_FETCH_FROM_SOURCE=ON cmake --build . # Build with nanoarrow preinstalled (assuming it's in CMAKE_PREFIX_PATH or system path) # You might need to install nanoarrow first, e.g., using 'cmake --install build --prefix /path/to/install' # cmake .. -DNANOARROW_FETCH_FROM_SOURCE=OFF # cmake --build . # Example of building with a specific install prefix # cmake .. -DCMAKE_INSTALL_PREFIX=/tmp/nanoarrow_install -DNANOARROW_FETCH_FROM_SOURCE=ON # cmake --build . --target install # cmake .. -DCMAKE_PREFIX_PATH=/tmp/nanoarrow_install -DNANOARROW_FETCH_FROM_SOURCE=OFF # cmake --build . cd .. echo "Build script finished successfully." ``` -------------------------------- ### Windows Environment Variables for Verification Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Set environment variables on Windows to specify CMake options for building with shared CRT and the R installation directory for release candidate verification. ```bash # Pass location of R to the verification script export NANOARROW_CMAKE_OPTIONS="-Dgtest_force_shared_crt=ON" export R_HOME="/c/Program Files/R/R-4.5.0" ``` -------------------------------- ### Fetch and Use nanoarrow with CMake Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Integrate the nanoarrow C library into a CMake project using FetchContent. This downloads and builds nanoarrow as part of your project. ```cmake fetchcontent_declare(nanoarrow URL "https://www.apache.org/dyn/closer.lua?action=download&filename=arrow/apache-arrow-nanoarrow-0.6.0/apache-arrow-nanoarrow-0.6.0.tar.gz") fetchcontent_makeavailable(nanoarrow) ``` -------------------------------- ### Upload Artifacts and Create GitHub Release Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Execute the upload script to handle uploading artifacts to Subversion and creating a GitHub release. Requires PMC member credentials and configured APACHE_USERNAME. ```bash dev/release/post-01-upload.sh 0.8.0 0 ``` -------------------------------- ### Fetch and Build nanoarrow with IPC Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-ipc/CMakeLists.txt Fetches the nanoarrow library and enables the IPC extension. It's important that the fetched nanoarrow version matches the one your application links against. ```cmake include(FetchContent) project(NanoArrowExampleCMakeIpc) # See examples/cmake-minimal for why this is a good idea set(NANOARROW_NAMESPACE "ExampleCmakeIpc") set(NANOARROW_IPC ON CACHE INTERNAL "Turn on Ipc") fetchcontent_declare(# The name 'nanoarrow' is important here: it allows the IPC extension # to link to the same version of nanoarrow as your library/application nanoarrow # See examples/cmake-minimal for how to specify a GIT repository or URL SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/../..) fetchcontent_makeavailable(nanoarrow) target_compile_options(nanoarrow_static PUBLIC -DNANOARROW_IPC=ON) unset(NANOARROW_IPC) ``` -------------------------------- ### Verify nanoarrow Release Candidate Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Clone the repository, navigate to the release directory, and run the verification script with the version and release candidate number. ```bash git clone https://github.com/apache/arrow-nanoarrow.git arrow-nanoarrow cd arrow-nanoarrow/dev/release ./verify-release-candidate.sh 0.8.0 0 ``` -------------------------------- ### Configure Device Tests Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Sets up executables for nanoarrow device tests, linking necessary libraries and defining compile flags. It supports conditional builds for Metal and CUDA backends. ```cmake if(NANOARROW_DEVICE) enable_testing() add_executable(nanoarrow_device_test src/nanoarrow/device/device_test.cc) add_executable(nanoarrow_device_hpp_test src/nanoarrow/device/device_hpp_test.cc) target_link_libraries(nanoarrow_device_test ${NANOARROW_TEST_DEVICE_LIB} gtest_main nanoarrow_coverage_config) target_link_libraries(nanoarrow_device_hpp_test ${NANOARROW_TEST_DEVICE_LIB} gtest_main nanoarrow_coverage_config) if(Arrow_FOUND) target_compile_definitions(nanoarrow_device_test PRIVATE -DNANOARROW_BUILD_TESTS_WITH_ARROW) target_compile_definitions(nanoarrow_device_hpp_test PRIVATE -DNANOARROW_BUILD_TESTS_WITH_ARROW) endif() include(GoogleTest) gtest_discover_tests(nanoarrow_device_test) gtest_discover_tests(nanoarrow_device_hpp_test) if(NANOARROW_DEVICE_WITH_METAL) add_executable(nanoarrow_device_metal_test src/nanoarrow/device/metal_test.cc) target_link_libraries(nanoarrow_device_metal_test ${NANOARROW_TEST_DEVICE_LIB} ${NANOARROW_TEST_LIB} gtest_main nanoarrow_coverage_config) target_include_directories(nanoarrow_device_metal_test PRIVATE ${NANOARROW_DEVICE_INCLUDE_METAL}) if(Arrow_FOUND) target_compile_definitions(nanoarrow_device_metal_test PRIVATE -DNANOARROW_BUILD_TESTS_WITH_ARROW) endif() gtest_discover_tests(nanoarrow_device_metal_test) endif() if(NANOARROW_DEVICE_WITH_CUDA) add_executable(nanoarrow_device_cuda_test src/nanoarrow/device/cuda_test.cc) target_link_libraries(nanoarrow_device_cuda_test ${NANOARROW_TEST_DEVICE_LIB} ${NANOARROW_TEST_LIB} CUDA::cuda_driver gtest_main nanoarrow_coverage_config) if(Arrow_FOUND) target_compile_definitions(nanoarrow_device_cuda_test PRIVATE -DNANOARROW_BUILD_TESTS_WITH_ARROW) endif() gtest_discover_tests(nanoarrow_device_cuda_test) endif() endif() endif() ``` -------------------------------- ### Create low-level C structures Source: https://github.com/apache/arrow-nanoarrow/blob/main/python/README.ipynb Create low-level wrappers for Arrow C interface structures using nanoarrow. ```python # You can create these using nanoarrow.c_schema(), nanoarrow.c_array(), and nanoarrow.c_array_stream(). ``` -------------------------------- ### Run C Library Verification Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Execute the verification script with specific environment variables to test only the C library, including bundled components. Requires CMake and Python. ```bash TEST_DEFAULT=0 TEST_C=1 TEST_C_BUNDLED=1 ./verify-release-candidate.sh 0.8.0 0 ``` -------------------------------- ### Download and Extract metal-cpp for Metal Device Support Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Downloads and extracts the metal-cpp library if it does not exist. This is a prerequisite for Metal device integration. ```cmake if(NANOARROW_DEVICE_WITH_METAL) if(NOT EXISTS "${CMAKE_BINARY_DIR}/metal-cpp") message(STATUS "Fetching metal-cpp") file(DOWNLOAD "https://developer.apple.com/metal/cpp/files/metal-cpp_macOS12_iOS15.zip" "${CMAKE_BINARY_DIR}/metal-cpp.zip") file(ARCHIVE_EXTRACT INPUT ${CMAKE_BINARY_DIR}/metal-cpp.zip DESTINATION ${CMAKE_BINARY_DIR}) endif() ``` -------------------------------- ### Basic googletest Stub Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md A minimal googletest test case to set up the testing framework. Includes necessary headers and a placeholder test. ```cpp #include #include "nanoarrow/nanoarrow.hpp" #include "linesplitter.h" TEST(Linesplitter, LinesplitterRoundtrip) { EXPECT_EQ(4, 4); } ``` -------------------------------- ### Fetch Google Benchmark Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/benchmarks/CMakeLists.txt Fetches the Google Benchmark library version 1.8.3 using its GitHub release. Ensures the benchmark framework is available for building tests. ```cmake set(BENCHMARK_ENABLE_TESTING OFF) fetchcontent_declare(benchmark URL https://github.com/google/benchmark/archive/refs/tags/v1.8.3.zip URL_HASH SHA256=abfc22e33e3594d0edf8eaddaf4d84a2ffc491ad74b6a7edc6e7a608f690e691 ) fetchcontent_makeavailable(benchmark) ``` -------------------------------- ### Clone nanoarrow Repository Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/README.md Clone the nanoarrow repository to your local machine. This is typically the first step before building documentation. ```bash git clone https://github.com/apache/arrow-nanoarrow.git ``` -------------------------------- ### Run nanoarrow tests and benchmarks with Meson Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Executes the test suite and benchmarks for nanoarrow using Meson. Options include running tests under Valgrind or with verbose benchmark output. ```sh meson test nanoarrow: # default test run ``` ```sh meson test nanoarrow: --wrap valgrind # run tests under valgrind ``` ```sh meson test nanoarrow: --benchmark --verbose # run benchmarks ``` -------------------------------- ### Define and Link Executable Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-minimal/CMakeLists.txt Defines a C executable and links it against the previously defined library. This sets up the main application that will utilize the functionality provided by the library. ```cmake add_executable(example_cmake_minimal_app src/app.c) target_link_libraries(example_cmake_minimal_app example_cmake_minimal_library) ``` -------------------------------- ### Compile nanoarrow with Meson Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Compiles the nanoarrow library using the Meson build system after configuration. ```sh meson compile ``` -------------------------------- ### C: Manual Memory Management for Arrow Structures Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md Demonstrates manual memory management for ArrowSchema and ArrowArray in C, including explicit release calls on failure. This pattern is verbose and error-prone. ```c struct ArrowSchema schema; struct ArrowArray array; // Ok: if this returns, array was not initialized NANOARROW_RETURN_NOT_OK(ArrowSchemaInitFromType(&schema, NANOARROW_TYPE_STRING)); // Verbose: if this fails, we need to release schema before returning // or it will leak. int code = ArrowArrayInitFromSchema(&array, &schema, NULL); if (code != NANOARROW_OK) { ArrowSchemaRelease(&schema); return code; } ``` -------------------------------- ### Upload Python Packages to PyPI Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Upload Python package source distributions and wheels to PyPI using twine. Download all assets from the build run and then execute the upload command. ```shell # pip install twine ``` -------------------------------- ### Build nanoarrow with CMake Source: https://github.com/apache/arrow-nanoarrow/blob/main/README.md Standard build process for the nanoarrow C library using CMake. Ensure you are in the build directory before running these commands. ```sh mkdir build && cd build cmake .. cmake --build . ``` -------------------------------- ### Sign Release Assets Script Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Calls the script to sign release assets. Requires a .env file with GPG_KEY_ID set. ```bash # The following command is a placeholder and requires the dev/release/.env file to be configured. # 02-sign.sh ``` -------------------------------- ### Run Python Verification Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/release/README.md Execute the verification script with specific environment variables to test only the Python bindings. Requires Python. ```bash TEST_DEFAULT=0 TEST_PYTHON=1 ./verify-release-candidate.sh 0.8.0 0 ``` -------------------------------- ### Define Metal Device Sources and Definitions Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Sets variables for Metal device source files, preprocessor definitions, and libraries. ```cmake set(NANOARROW_DEVICE_SOURCES_METAL src/nanoarrow/device/metal.cc) set(NANOARROW_DEVICE_DEFS_METAL "NANOARROW_DEVICE_WITH_METAL") set(NANOARROW_DEVICE_LIBS_METAL nanoarrow_metal_impl) ``` -------------------------------- ### C: Basic ArrowSchema Initialization Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md Initializes an ArrowSchema for a non-nullable string type. Demonstrates manual error checking. ```c int init_string_non_null(struct ArrowSchema* schema) { int code = ArrowSchemaInitFromType(&schema, NANOARROW_TYPE_STRING); if (code != NANOARROW_OK) { return code; } schema->flags &= ~ARROW_FLAG_NULLABLE; return NANOARROW_OK; } ``` -------------------------------- ### Configure Include Directories and Compile Definitions for Nanoarrow Device Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt Sets include directories and compile definitions for nanoarrow device static and shared libraries. This ensures proper header inclusion and definition of preprocessor macros for CUDA and Metal. ```cmake foreach(target "nanoarrow_device_static" "nanoarrow_device_shared") target_include_directories(${target} PUBLIC $ $ $ $) target_compile_definitions(${target} PRIVATE ${NANOARROW_DEVICE_DEFS_METAL} ${NANOARROW_DEVICE_DEFS_CUDA}) endforeach() ``` -------------------------------- ### C: ArrowSchema Initialization with NANOARROW_RETURN_NOT_OK Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md Initializes an ArrowSchema for a non-nullable string type using the NANOARROW_RETURN_NOT_OK macro for concise error handling. Assumes internal functions return int or ArrowError*. ```c int init_string_non_null(struct ArrowSchema* schema) { NANOARROW_RETURN_NOT_OK(ArrowSchemaInitFromType(&schema, NANOARROW_TYPE_STRING)); schema->flags &= ~ARROW_FLAG_NULLABLE; return NANOARROW_OK; } ``` -------------------------------- ### Include Guard for Header Files Source: https://github.com/apache/arrow-nanoarrow/blob/main/docs/source/getting-started/cpp.md Use #pragma once to ensure that a header file is included only once in a source file. ```cpp #pragma once ``` -------------------------------- ### Generate Bundled IPC Sources with Namespace Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/vendored-ipc/README.md Use this Python script to generate vendored C sources for nanoarrow and nanoarrow_ipc.c, applying a custom symbol namespace and including FlatBuffers support. ```bash git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow/examples/vendored-ipc python ../../ci/scripts/bundle.py \ --source-output-dir=src \ --include-output-dir=src \ --symbol-namespace=MyProject \ --with-ipc \ --with-flatcc ``` -------------------------------- ### Fetch nanoarrow using FetchContent Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/cmake-minimal/CMakeLists.txt Declares and makes available the nanoarrow library using FetchContent. This is useful for integrating nanoarrow into an existing CMake project. ```cmake include(FetchContent) project(NanoArrowExampleCMakeMinimal) # When adding nanoarrow's CMake directory to a CMake project that contains a library # intended for use by others, set NANOARROW_NAMESPACE to rename symbols in the # nanoarrow library such that they do not collide with other libraries that may also # link to their own copy of nanoarrow. You may wish to include the namespace only # on release builds, since the namespace implementation obscures inline help # available in many text editors. set(NANOARROW_NAMESPACE "ExampleCmakeMinimal") fetchcontent_declare(nanoarrow_example_cmake_minimal # We use SOURCE_DIR here to point to the version of nanoarrow represented # by this checkout of the repo; however, you can use any of the methods # supported by FetchContent_Declare, e.g.: # GIT_REPOSITORY https://github.com/apache/arrow-nanoarrow.git # GIT_TAG some_commit_hash # GIT_SHALLOW TRUE SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/../..) fetchcontent_makeavailable(nanoarrow_example_cmake_minimal) ``` -------------------------------- ### Bundle nanoarrow with a custom namespace Source: https://github.com/apache/arrow-nanoarrow/blob/main/examples/vendored-minimal/README.md Use the provided Python script to bundle nanoarrow's C source and header files, applying a custom symbol namespace to prevent conflicts. ```bash git clone https://github.com/apache/arrow-nanoarrow.git cd arrow-nanoarrow cd examples/vendored-minimal python ../../ci/scripts/bundle.py \ --source-output-dir=src \ --include-output-dir=src \ --symbol-namespace=ExampleVendored ``` -------------------------------- ### Configure Testing Environment Source: https://github.com/apache/arrow-nanoarrow/blob/main/CMakeLists.txt This section sets up the testing environment when NANOARROW_BUILD_TESTS is enabled. It defines memory checking options, includes the CTest module, and selects appropriate library targets (shared or static) based on NANOARROW_TEST_LINKAGE_SHARED. ```cmake if(NANOARROW_BUILD_TESTS) set(MEMORYCHECK_COMMAND_OPTIONS "--leak-check=full --suppressions=${CMAKE_CURRENT_LIST_DIR}/valgrind.supp --error-exitcode=1" ) include(CTest) if(NANOARROW_TEST_LINKAGE_SHARED) set(NANOARROW_TEST_LIB nanoarrow_shared) set(NANOARROW_TEST_IPC_LIB nanoarrow_ipc_shared) set(NANOARROW_TEST_DEVICE_LIB nanoarrow_device_shared) set(NANOARROW_TEST_TESTING_LIB nanoarrow_testing_shared) else() set(NANOARROW_TEST_LIB nanoarrow_static) set(NANOARROW_TEST_IPC_LIB nanoarrow_ipc_static) set(NANOARROW_TEST_DEVICE_LIB nanoarrow_device_static) set(NANOARROW_TEST_TESTING_LIB nanoarrow_testing_static) endif() if(NANOARROW_BUILD_TESTS_WITH_ARROW) find_package(Arrow REQUIRED) message(STATUS "Arrow version: ${ARROW_VERSION}") message(STATUS "Arrow SO version: ${ARROW_FULL_SO_VERSION}") # Give caller the option to link a static version of Arrow C++ if(NANOARROW_ARROW_STATIC) set(NANOARROW_ARROW_TARGET arrow_static) else() set(NANOARROW_ARROW_TARGET arrow_shared) endif() # Arrow >= 10.0.0 requires C++17; GTest requires C++11. # Leave the option open to use an older version of Arrow # to make it easier to test on old Linux (e.g., Centos7) if(${ARROW_VERSION} VERSION_GREATER_EQUAL "10.0.0") set(CMAKE_CXX_STANDARD 17) else() set(CMAKE_CXX_STANDARD 11) endif() set(CMAKE_CXX_STANDARD_REQUIRED ON) endif() add_subdirectory("thirdparty/googletest") endif() ``` -------------------------------- ### Fetch nanoarrow with IPC enabled Source: https://github.com/apache/arrow-nanoarrow/blob/main/dev/benchmarks/CMakeLists.txt Fetches the nanoarrow library, enabling the IPC feature. Handles different scenarios based on whether a source directory or URL is provided, and adjusts for older nanoarrow versions that packaged IPC separately. ```cmake if(IS_DIRECTORY "${NANOARROW_BENCHMARK_SOURCE_URL}") fetchcontent_declare(nanoarrow SOURCE_DIR "${NANOARROW_BENCHMARK_SOURCE_URL}") fetchcontent_makeavailable(nanoarrow) elseif(NOT "${NANOARROW_BENCHMARK_SOURCE_URL}" STREQUAL "") fetchcontent_declare(nanoarrow URL "${NANOARROW_BENCHMARK_SOURCE_URL}") fetchcontent_makeavailable(nanoarrow) # nanoarrow 0.4.0 and 0.5.0 packaged IPC in a subdirectory if(NANOARROW_VERSION VERSION_LESS "0.6.0") fetchcontent_declare(nanoarrow_ipc URL "${NANOARROW_BENCHMARK_SOURCE_URL}" SOURCE_SUBDIR extensions/nanoarrow_ipc) fetchcontent_makeavailable(nanoarrow_ipc) endif() endif() ```