### Instantiate flatdata_make_example for Examples

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/examples/CMakeLists.txt

Calls the flatdata_make_example CMake function to set up build targets for various flatdata examples.

```cmake
flatdata_make_example(binary_layout)
```

```cmake
flatdata_make_example(fibonacci)
```

```cmake
flatdata_make_example(graph)
```

```cmake
flatdata_make_example(coappearances/coappearances)
```

--------------------------------

### Install and Run flatdata-writer

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Install the flatdata-py library with writer support and run the writer tool to create a flatdata archive.

```sh
pip3 install flatdata-py[writer]
flatdata-writer --schema archive.flatdata --output-dir testdir --json-file data.json --resource-name resourcename
```

--------------------------------

### Install and Run flatdata-inspector

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Install the flatdata-py library with inspector support and run the inspector tool with a specified archive path.

```sh
pip3 install flatdata-py[inspector]  # the inspector feature requires IPython
flatdata-inspector -p /path/to/my/flatdata.archive
```

--------------------------------

### Install flatdata-generator CLI

Source: https://context7.com/heremaps/flatdata/llms.txt

Install the Flatdata code generation tool using pip. This command installs the generator and its dependencies.

```sh
# Install the generator
pip3 install flatdata-generator
```

--------------------------------

### Install and Use flatdata-generator

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-generator/README.md

Install the generator using pip and use it to generate a header-only C++ library from a schema file.

```sh
# installation
pip3 install flatdata-generator

# example: generate a header-only C++ library
flatdata-generator -s locations.flatdata -g cpp -O locations.hpp
```

--------------------------------

### Project Setup and Module Inclusion

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Initializes the CMake project and includes custom modules for code generation.

```cmake
cmake_minimum_required(VERSION 3.2)
project(flatdata-cpp)

list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/cmake")
include(flatdata/GenerateSource)
```

--------------------------------

### Conditional Installation of Targets

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Checks if an install target is available and, if so, installs the benchmark executable and a run script with their dependencies.

```cmake
include(InstallTarget OPTIONAL RESULT_VARIABLE HAS_INSTALL_TARGET)
if(HAS_INSTALL_TARGET)
    install_with_depends(flatdata_benchmark bin lib flatdata_benchmark_component)
    install_with_depends(run.py bin lib flatdata_benchmark_component)
endif()
```

--------------------------------

### Flatdata Archive Example

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

An example archive definition demonstrating various resource types including single structures, vectors, multivectors, raw data, and nested archives.

```cpp
archive ExampleArchive {
    single_structure : StructureType;
    vector_of_stuff : vector< StructureType >;
    what_an_archive_without_a_map : multivector< 40, StructureA, StructureB, StructureC >;
    strings_forever : raw_data;
    lets_get_some_structure : archive OtherArchive;
}
```

--------------------------------

### Include Directories Configuration

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Adds include directories for build, installation, and Boost headers to the 'flatdata' target.

```cmake
target_include_directories(flatdata
    PUBLIC $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
    PUBLIC $<INSTALL_INTERFACE:include>
    PUBLIC ${Boost_INCLUDE_DIRS})
```

--------------------------------

### Flatdata Structure Example

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

An example of a structure definition with fields specifying their type and bit width.

```cpp
struct Structure {
    field1 : u32 : 29;
    field2 : u8 : 2;
}
```

--------------------------------

### Testing and Subdirectory Inclusion

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Enables testing and includes the 'test', 'benchmark', and 'examples' subdirectories.

```cmake
enable_testing()
add_subdirectory(test)
add_subdirectory(benchmark)
add_subdirectory(examples)
```

--------------------------------

### Serialize Data with flatdata-cpp

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/README.md

Example of writing data to a flatdata archive using C++. Requires a generated header file.

```cpp
// Compile with: c++ -std=c++11 writer.cpp -Iflatdata-cpp/include -Lbuild/flatdata-cpp -lflatdata -lboost_system -lboost_filesystem -o writer
#include "locations.hpp"
int main() {
  auto storage = flatdata::FileResourceStorage::create("locations.archive");  // create storage
  auto builder = loc::LocationsBuilder::open(std::move(storage));             // create builder
  auto pois = builder.start_pois();

  uint32_t x, y;
  while(std::cin >> x >> y) {
      loc::PointMutator poi = pois.grow();
      poi.x = x;
      poi.y = y;
  }
  pois.close();  // flush not yet written data to disk
}
```

--------------------------------

### Read Data with flatdata-cpp

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/README.md

Example of reading data from a flatdata archive using C++. Requires a generated header file and a previously created archive.

```cpp
// Compile with: c++ -std=c++11 reader.cpp -Iflatdata-cpp/include -Lbuild/flatdata-cpp -lflatdata -lboost_system -lboost_filesystem -o reader
#include "locations.hpp"
#include <iostream>
int main() {
    auto storage = flatdata::FileResourceStorage::create("locations.archive");  // open storage
    auto archive = loc::Locations::open(std::move(storage));              // create archive
    for (loc::Point point : archive.pois()) {                             // iterate through pois
        std::cout << point.to_string() << std::endl;
    }
    return 0;
}
```

--------------------------------

### Conditional Installation of Test Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Conditionally installs the flatdata_test executable if the 'InstallTarget' component is available. This ensures the test executable is installed only when explicitly requested.

```cmake
include(InstallTarget OPTIONAL RESULT_VARIABLE HAS_INSTALL_TARGET)
if(HAS_INSTALL_TARGET)
    install_with_depends(flatdata_test bin lib flatdata_test_component)
endif()
```

--------------------------------

### Python: Opening and Reading a Flatdata Archive

Source: https://context7.com/heremaps/flatdata/llms.txt

Open a Flatdata archive in Python using the generated `Archive` subclass. This example shows how to access metadata, iterate through vector resources, and print DataFrame summaries. Requires `FileResourceStorage` and a generated Python module.

```python
import flatdata
from flatdata.lib.file_resource_storage import FileResourceStorage

# Generated module from: flatdata-generator --gen py --schema graph.flatdata --output-file graph.py
import graph as co

# Open archive from a directory of memory-mapped files
storage = FileResourceStorage("path/to/graph_archive")
archive = co.Graph(storage)

# Access metadata (single-struct resource)
strings_raw = bytes(archive.strings[:])
meta = archive.meta
title = strings_raw[meta.title_ref:strings_raw.index(b'\x00', meta.title_ref)].decode()

# Iterate vertices (Vector resource)
for vertex in archive.vertices:
    name = strings_raw[vertex.name_ref:strings_raw.index(b'\x00', vertex.name_ref)].decode()

# Print a DataFrame summary of all resources
print(archive.to_data_frame())
# Columns: Name | Type | Optional | SizeInBytes | Size

# Inspect schema
print(archive.schema())
print(archive.resource_schema("vertices"))
```

--------------------------------

### Define flatdata_make_example CMake Function

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/examples/CMakeLists.txt

Defines a CMake function to generate C++ code from flatdata schemas and build executable examples.

```cmake
function (flatdata_make_example PATH)
    get_filename_component(EXAMPLE_DIR ${PATH} DIRECTORY)
    get_filename_component(EXAMPLE_NAME ${PATH} NAME)

    flatdata_generate_source(generate_flatdata_${EXAMPLE_NAME}
        ${CMAKE_CURRENT_SOURCE_DIR}/../../examples/${PATH}.flatdata
        ${CMAKE_CURRENT_BINARY_DIR}/generated/${PATH}.hpp)

    add_executable(${EXAMPLE_NAME} ${PATH}.cpp)
    add_dependencies(${EXAMPLE_NAME} generate_flatdata_${EXAMPLE_NAME})

    target_include_directories(${EXAMPLE_NAME}
        PRIVATE ${Boost_INCLUDE_DIRS}
        PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/generated/${EXAMPLE_DIR}
        PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../vendor)

    target_link_libraries(${EXAMPLE_NAME} flatdata)
endfunction(flatdata_make_example)
```

--------------------------------

### Rust: Reading a Flatdata Archive

Source: https://context7.com/heremaps/flatdata/llms.txt

Open and read data from a Flatdata archive using Rust. This example demonstrates accessing vector and multivector resources with zero-copy, memory-mapped reads. Requires `FileResourceStorage` and the generated archive type.

```rust
// Reading an archive in Rust
use flatdata::FileResourceStorage;

fn main() {
    let storage = FileResourceStorage::new("path/to/archive").unwrap();
    let graph = coappearances::Graph::open(storage).unwrap();

    // Access a vector resource — zero-copy, memory-mapped
    for vertex in graph.vertices() {
        println!("name_ref={}", vertex.name_ref());
    }

    // Access a multivector resource
    for (i, items) in graph.vertices_data().iter().enumerate() {
        for item in items {
            match item {
                coappearances::VerticesDataItem::Nickname(n) => println!("  nickname ref={}", n.ref_()),
                coappearances::VerticesDataItem::Description(d) => println!("  desc ref={}", d.ref_()),
            }
        }
    }
}
```

--------------------------------

### Define Flatdata Schema

Source: https://github.com/heremaps/flatdata/blob/master/README.md

Define a flatdata archive schema using the simple schema language. This example shows a struct 'Point' and a vector of 'Point' within an archive named 'Locations'.

```cpp
namespace loc {
    struct Point {
        x : u32 : 32;
        y : u32 : 32;
    }
    archive Locations {
        pois : vector< Point >; 
    }
}
```

--------------------------------

### Simplify Index Ranges with @range

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Use @range to simplify accessing contiguous ranges of data within vectors, particularly when a field in one structure points to the start of a range in another. It adds a range attribute and hides sentinel values.

```cpp
struct Node {
    ...
    @range(edges_range)
    first_edge_ref : u32;
}

struct Edge {
    ...
}

archive Archive {
    // contains sentinel
    @explicit_reference( Nodes.first_edge_ref, edges )
    nodes : vector< Nodes >;
    edges : vector< Edges >;
}
```

--------------------------------

### Run flatdata-writer from Source

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Execute the flatdata-writer tool directly from the flatdata-py source directory to create a flatdata archive.

```sh
./writer.py --schema archive.flatdata --output-dir testdir --json-file data.json --resource-name resourcename
```

```sh
python3 -m flatdata.lib.writer --schema archive.flatdata --output-dir testdir --json-file data.json --resource-name resourcename
```

--------------------------------

### Build flatdata-cpp

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/README.md

Steps to build the flatdata-cpp library using CMake and make.

```shell
pip3 install -r flatdata-generator/requirements.txt
cd flatdata-cpp
mkdir build && cd build
cmake ..
make
make test # optional
```

--------------------------------

### Writing FlatData Archives with Python Writer

Source: https://context7.com/heremaps/flatdata/llms.txt

Demonstrates how to write a FlatData archive using the Writer class, which accepts a schema string and output path. Shows how to set resources and finish the writing process. The written archive can then be opened for reading.

```python
from flatdata.lib.flatdata_writer import Writer

SCHEMA = """
    namespace example {
        struct Point { x : u32 : 16; y : u32 : 16; }
        archive Locations {
            points : vector< Point >; 
        }
    "

# Writer generates a builder on-the-fly from the schema string
writer = Writer(archive_schema=SCHEMA, path="/tmp/output", archive_name="Locations")

# Write a vector resource as a list of dicts (one dict per struct)
writer.set("points", [
    {"x": 10, "y": 20},
    {"x": 30, "y": 40},
    {"x": 50, "y": 60},
])

writer.finish()   # flushes all resources and closes storage

# The archive can now be opened for reading
from flatdata.lib.file_resource_storage import FileResourceStorage
import example as ex   # generated reader module

storage = FileResourceStorage("/tmp/output/Locations")
archive = ex.Locations(storage)
for pt in archive.points:
    print(pt.x, pt.y)
```

--------------------------------

### Run flatdata-inspector from Source

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Execute the flatdata-inspector tool directly from the flatdata-py source directory.

```sh
./inspector.py
```

```sh
python3 -m flatdata.lib.inspector
```

--------------------------------

### Configuring Benchmark Include Directories

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Specifies private include directories for the benchmark executable, including Google Test, Boost, generated code, and vendor directories.

```cmake
target_include_directories(flatdata_benchmark
    PRIVATE ${GTEST_INCLUDE_DIRS}
    PRIVATE ${Boost_INCLUDE_DIRS}
    PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/generated
    PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../vendor)
```

--------------------------------

### Creating the Benchmark Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Defines the main executable for the benchmark, including all discovered C++ source files and adding a dependency on the generated code.

```cmake
add_executable(flatdata_benchmark ${BENCHMARK_FLATDATA_SOURCES})
add_dependencies(flatdata_benchmark generate_flatdata_benchmark_code)
```

--------------------------------

### Source File Globbing and Library Creation

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Collects all source files from the 'src' directory and creates a static library named 'flatdata'.

```cmake
file(GLOB FLATDATA_SOURCE
    "src/*.h"
    "src/*.inl"
    "src/*.cpp")

add_library(flatdata STATIC ${FLATDATA_SOURCE})
```

--------------------------------

### Discovering Benchmark Source Files

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Uses CMake's file(GLOB ...) command to find all .cpp files in the current directory for the benchmark executable.

```cmake
file(GLOB BENCHMARK_FLATDATA_SOURCES "*.cpp")
```

--------------------------------

### Working with RawData for Untyped Byte Blobs in Python

Source: https://context7.com/heremaps/flatdata/llms.txt

Illustrates how to use the RawData class to access an untyped byte blob, commonly used for string pools. Shows byte-level access, slicing, and extracting null-terminated strings using helper methods. Requires an initialized archive and storage.

```python
storage = FileResourceStorage("path/to/archive")
archive = co.Graph(storage)
strings = archive.strings   # RawData instance

# Total length in bytes
print(len(strings))

# Byte-level access (returns memoryview)
byte = strings[0]
region = strings[4:12]

# Extract a single null-terminated UTF-8 string by offset
title = strings.sub_str(archive.meta.title_ref)     # reads until '\x00'
print(f"Title: {title}")

# Extract a null-terminated list (list separator = '\x00\x00')
tags = strings.sub_str_list(some_offset)

# Extract a fixed-count array of null-separated strings
names = strings.sub_str_array(offset=0, size=5)
```

--------------------------------

### Mark Resource as Optional with @optional

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Apply @optional to a resource to allow the archive to be opened successfully even if the resource is missing. Any resource type can be optional.

```cpp
archive Archive {
    @optional
    resource: vector< SomeStructure >; 
}
```

--------------------------------

### Link Libraries Configuration

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Links the 'flatdata' target against Boost libraries and the Threads library initialization.

```cmake
target_link_libraries(flatdata
  ${Boost_LIBRARIES}
  ${CMAKE_THREAD_LIBS_INIT})
```

--------------------------------

### C++ Standard and Dependency Finding

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Sets the C++ standard to C++17 and finds required Boost and Threads libraries.

```cmake
set(CMAKE_CXX_STANDARD 17)

find_package(Boost COMPONENTS system filesystem REQUIRED)
find_package(Threads REQUIRED)
```

--------------------------------

### Linking Benchmark Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Links the flatdata library to the benchmark executable.

```cmake
target_link_libraries(flatdata_benchmark
    flatdata)
```

--------------------------------

### Use Local and Fully-Qualified Paths for Entity Referencing

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Reference entities using either a local path within the current namespace or a fully-qualified path. Local paths are resolved within the current scope.

```cpp
namespace N {
    struct T {
        ...
    }

    archive Archive {
        // Local path
        resource: vector< T >
        // Fully-qualified path
        another_resource: vector< .N.T >
    }
}
```

--------------------------------

### Accessing Heterogeneous Data with Python Multivector

Source: https://context7.com/heremaps/flatdata/llms.txt

Demonstrates how to access and iterate through data stored in a Multivector, which holds variable-length lists of typed elements per index. Requires an initialized archive and storage.

```python
storage = FileResourceStorage("path/to/archive")
archive = co.Graph(storage)
vdata = archive.vertices_data  # Multivector instance

# Access bucket at index 0 — list of heterogeneous Structure objects
items = vdata[0]
for item in items:
    # item is one of the declared element types: Nickname, Description, etc.
    print(type(item).__name__, item.ref)

# Iterate all buckets
for i, bucket in enumerate(vdata):
    if bucket:
        print(f"vertex {i}: {[type(b).__name__ for b in bucket]}")

# Slicing
for bucket in vdata[10:20]:
    print(bucket)

print(f"Total buckets: {len(vdata)}")
```

--------------------------------

### Setting Target Folder Property

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Organizes the flatdata_benchmark target within the CMake build system's folder structure under 'flatdata'.

```cmake
set_property(TARGET flatdata_benchmark PROPERTY FOLDER "flatdata")
```

--------------------------------

### Python: Vectorized Access and Iteration

Source: https://context7.com/heremaps/flatdata/llms.txt

Demonstrates various ways to access data from a Flatdata `Vector` in Python, including element-wise iteration, indexing, slicing, and vectorized access via NumPy and Pandas DataFrames. This allows for efficient bulk data retrieval.

```python
from flatdata.lib.file_resource_storage import FileResourceStorage
import graph as co          # generated module
import numpy as np

storage = FileResourceStorage("path/to/archive")
archive = co.Graph(storage)
vertices = archive.vertices   # a Vector instance

# Element access (Python object per element — per-field bit unpacking)
v = vertices[0]
print(v.name_ref, v.is_active)

# Negative indexing
last = vertices[-1]

# Slicing — returns _VectorSlice, lazily iterable
for v in vertices[10:20]:
    print(v.name_ref)

# Single-column vectorized access — returns pandas DataFrame
df_names = vertices.name_ref          # DataFrame with one column 'name_ref'
count = len(df_names[df_names['name_ref'] > 1000])

# Full vectorized bulk read — NumPy structured array with all fields
arr = vertices.to_numpy()             # shape (N,), dtype with named fields
fast_count = int(np.sum(arr['is_active'] == 1))

# Slice + vectorized
arr_slice = vertices[1000:2000].to_numpy()
df_slice  = vertices[::10].to_data_frame()

print(f"Total vertices: {len(vertices)}, size: {vertices.size_in_bytes()} bytes")
```

--------------------------------

### Accessing and Converting FlatData Structures in Python

Source: https://context7.com/heremaps/flatdata/llms.txt

Explains how to access fields within a FlatData Structure instance and convert it to various Python native types like dictionaries, lists, tuples, and NumPy arrays. Also shows how to inspect the structure's schema.

```python
# Accessing structure fields
vertex = archive.vertices[42]
print(vertex.name_ref)      # int — unpacked from bit field
print(vertex.is_active)     # int (0 or 1)

# Convert a structure to Python native types
d = vertex.as_dict()        # {"name_ref": 1234, "is_active": 1, "kind": 0}
lst = vertex.as_list()      # [1234, 1, 0]
tpl = vertex.as_tuple()     # (1234, 1, 0)

# NumPy single-element structured array
arr = vertex.as_nparray()   # shape (1,), dtype with named fields

# Pretty JSON repr
print(repr(vertex))
# {
#     "name": "Vertex",
#     "attributes": {"name_ref": 1234, "is_active": 1, "kind": 0}
# }

# Inspect schema of the struct type
print(vertex.schema())
```

--------------------------------

### Generate Python Module with flatdata-generator

Source: https://context7.com/heremaps/flatdata/llms.txt

Generate a Python module from a Flatdata schema. Use the `--schema` and `--output-file` flags to specify the input schema and output Python file.

```sh
# Generate a Python module
flatdata-generator --gen py --schema locations.flatdata --output-file locations.py
```

--------------------------------

### Configure Test Executable Include Directories

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Sets the include directories for the flatdata_test executable. This ensures that generated headers and vendor code are accessible during compilation.

```cmake
target_include_directories(flatdata_test
    PRIVATE ${Boost_INCLUDE_DIRS}
    PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/generated
    PRIVATE ${CMAKE_SOURCE_DIR}/../vendor
)
```

--------------------------------

### Generate Flatdata Code with build.rs

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-rs/README.md

Use this build script to automatically generate Rust code from your flatdata schema. Ensure the schema path is correct and handle potential errors during generation.

```rust
fn main() {
    flatdata::generate(
        "path/to/my_schema.flatdata",
        &std::env::var("OUT_DIR").unwrap(),
    )
    .expect("generator failed");
}
```

--------------------------------

### Create Flatdata Test Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Defines the main test executable for the Flatdata project. It includes all C++ source files and specifies dependencies on generated code.

```cmake
file(GLOB TEST_FLATDATA_SOURCES "*.cpp" "../vendor/catch_amalgamated.cpp")

add_executable(flatdata_test ${TEST_FLATDATA_SOURCES})
add_dependencies(flatdata_test generate_flatdata_test_code generate_flatdata_test_code2 generate_flatdata_test_case_ranges)
```

--------------------------------

### Run Tests with pytest

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Execute the test suite for the flatdata-py project using pytest.

```sh
python3 -m pytest
```

--------------------------------

### Low-Level Data Access Functions

Source: https://context7.com/heremaps/flatdata/llms.txt

Functions for bit-precise binary I/O on arbitrary buffers, used as building blocks for higher-level data structures.

```APIDOC
## Low-Level Data Access

### Description
`read_value` and `write_value` are the lowest-level Python functions for bit-precise binary I/O on arbitrary `bytes`, `bytearray`, or `memoryview` buffers. They are building blocks used by the generated `Structure` and resource classes. `read_value` constructs a reader closure via `make_field_reader` and invokes it once; for repeated reads of the same field layout, `make_field_reader` should be used directly to amortize closure creation.

### Functions
- **`read_value(data: bytes | bytearray | memoryview, offset_bits: int, num_bits: int, is_signed: bool) -> int`**: Reads a specified number of bits from a buffer at a given bit offset, interpreting them as signed or unsigned.
- **`write_value(data: bytearray, offset_bits: int, num_bits: int, is_signed: bool, value: int)`**: Writes a value into a bytearray at a specified bit offset and width, interpreting it as signed or unsigned.
- **`make_field_reader(offset_bits: int, num_bits: int, is_signed: bool) -> callable`**: Creates a reusable reader closure for a specific field layout, optimizing repeated reads of the same field structure.

### Example Usage
```python
from flatdata.lib.data_access import read_value, write_value, make_field_reader

# --- read_value(data, offset_bits, num_bits, is_signed) -> int ---
# Read 8 unsigned bits starting at bit offset 0
val = read_value(b"\x2a", 0, 8, False)         # 42

# Read a 16-bit signed value from a little-endian buffer starting at bit 8
buf = bytes([0x00, 0xff, 0x7f])
val = read_value(buf, 8, 16, True)             # 32767

# Read a single bit (boolean)
flag = read_value(b"\x04", 2, 1, False)        # 1  (bit 2 of 0b00000100)

# --- write_value(data: bytearray, offset_bits, num_bits, is_signed, value) ---
buf = bytearray(4)
write_value(buf, 0, 16, False, 1000)           # write 1000 into bits 0-15
write_value(buf, 16, 8, True, -5)              # write -5 into bits 16-23
print(list(buf))                               # [232, 3, 251, 0]

# --- make_field_reader(offset_bits, num_bits, is_signed) -> callable ---
# Pre-build a reusable reader closure for a specific field layout
read_speed = make_field_reader(offset_bits=16, num_bits=12, is_signed=False)
record = b"\x00\x00\x40\x01\x00\x00"
speed = read_speed(record, 0)                  # reads 12 bits from offset 16

# Reuse across many records (avoids rebuilding closure each time)
records = [b"\x00\x00\x80\x00\x00\x00", b"\x00\x00\xc0\x00\x00\x00"]
speeds = [read_speed(r, 0) for r in records]
```
```

--------------------------------

### Read Flatdata Archive in C++

Source: https://context7.com/heremaps/flatdata/llms.txt

Open a flatdata archive using FileResourceStorage and the generated reader class. Access data via typed ArrayView/MultiArrayView accessors. Handle potential errors during opening.

```cpp
// --- Reading ---
auto read_storage = flatdata::FileResourceStorage::create("output_dir");
auto graph = co::Graph::open(read_storage);
if (!graph) {
    std::cerr << graph.describe() << std::endl;
    return 1;
}

// Random access to a vector
auto vertices_view = graph.vertices();
std::cout << "Count: " << vertices_view.size() << "\n";
auto first = vertices_view[0];
std::cout << "name_ref: " << first.name_ref << "\n";

// Iterate edges (range-for works on ArrayView)
for (auto edge : graph.edges()) {
    std::cout << edge.a_ref << " <-> " << edge.b_ref << "\n";
}

// Access multivector with typed visitor
auto visitor = flatdata::make_overload(
    [](co::Nickname n)    { std::cout << "nickname ref=" << n.ref << "\n"; },
    [](co::Description d) { std::cout << "desc ref="    << d.ref << "\n"; }
);
graph.vertices_data().for_each(0, visitor);

// Optional sub-archive
if (auto stats = graph.statistics()) {
    std::cout << "max_degree: " << stats->invariants().max_degree << "\n";
}

```

--------------------------------

### Generate C++ Header with flatdata-generator

Source: https://context7.com/heremaps/flatdata/llms.txt

Generate a header-only C++ library from a Flatdata schema file. Use the `-s` flag for the schema and `-g cpp` for the C++ generator.

```sh
# Generate a header-only C++ library from a schema
flatdata-generator -s locations.flatdata -g cpp -O locations.hpp
```

--------------------------------

### Visualize Schema with flatdata-generator

Source: https://context7.com/heremaps/flatdata/llms.txt

Generate a Dot graph visualization of the Flatdata schema. This is useful for understanding schema relationships.

```sh
# Visualise the schema as a Dot graph
flatdata-generator -s locations.flatdata -g dot -O schema.dot
```

--------------------------------

### Include Generated Flatdata Code

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-rs/README.md

Include the code generated by `build.rs` into your crate. This allows you to use the data structures and accessors defined in your flatdata schema.

```rust
#![allow(dead_code)]
include!(concat!(env!("OUT_DIR"), "path/to/my_schema.rs"));
// re-export if desired
pub use my_schema::*;
```

--------------------------------

### Write Flatdata Archive in C++

Source: https://context7.com/heremaps/flatdata/llms.txt

Use ArchiveBuilder to stream element-by-element construction into a FileResourceStorage. Ensure strings are set correctly before the builder's destructor finalizes the archive.

```cpp
#include "coappearances.hpp"  // generated from coappearances.flatdata
#include <flatdata/flatdata.h>

// --- Writing ---
auto storage = flatdata::FileResourceStorage::create("output_dir");
auto builder = co::GraphBuilder::open(std::move(storage));

auto vertices = builder.start_vertices();
auto v = vertices.grow();
v.name_ref = 0;       // offset into strings blob
vertices.close();

builder.set_strings({strings.data(), strings.size()});
// builder destructor closes and finalizes the archive

```

--------------------------------

### Link Libraries for Flatdata Test Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Links the necessary libraries to the flatdata_test executable. This includes the main flatdata library and system threading libraries.

```cmake
target_link_libraries(flatdata_test
    flatdata
    ${CMAKE_THREAD_LIBS_INIT}
)
```

--------------------------------

### Generate Python Module from Schema

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Generate a Python module for reading a flatdata archive based on a schema file.

```sh
flatdata-generator --gen py --schema locations.flatdata --output-file locations.py
```

--------------------------------

### Flatdata Schema Comments

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Demonstrates the use of C++ style comments (single-line // and multi-line /** */) within Flatdata schemas. These comments can be preserved in generated code.

```cpp
/// A single secret. Might be important
struct Secret { importance : u64 : 64; }

/**
    * Very important archive
    */
archive TheBookOfSecrets {
    // More important secret
    secret1 : Secret;
    // Less important secret
    secret2 : Secret;
}
```

--------------------------------

### Generate C++ Header

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/README.md

Generates a C++ header file from a flatdata schema using the flatdata generator.

```shell
./generator --gen cpp --schema locations.flatdata --output-file locations.hpp
```

--------------------------------

### Generate Rust Source with flatdata-generator

Source: https://context7.com/heremaps/flatdata/llms.txt

Generate Rust source code from a Flatdata schema. The output can be used directly or integrated with build.rs.

```sh
# Generate Rust source (also available via build.rs integration)
flatdata-generator -s locations.flatdata -g rust -O src/locations.rs
```

--------------------------------

### Flatdata Archive Definition

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Defines an archive, which serves as the entry point to the data. It lists the resources contained within the archive.

```cpp
archive <name> {
    <resource> : <type>;
    ...
}
```

--------------------------------

### Python Low-Level Data Access - read_value and write_value

Source: https://context7.com/heremaps/flatdata/llms.txt

Utilize read_value and write_value for bit-precise binary I/O on byte buffers. read_value constructs a reader closure, while write_value modifies the buffer in place. make_field_reader is useful for amortizing closure creation for repeated reads of the same field layout.

```python
from flatdata.lib.data_access import read_value, write_value, make_field_reader

# --- read_value(data, offset_bits, num_bits, is_signed) -> int ---
# Read 8 unsigned bits starting at bit offset 0
val = read_value(b"\x2a", 0, 8, False)         # 42

# Read a 16-bit signed value from a little-endian buffer starting at bit 8
buf = bytes([0x00, 0xff, 0x7f])
val = read_value(buf, 8, 16, True)             # 32767

# Read a single bit (boolean)
flag = read_value(b"\x04", 2, 1, False)        # 1  (bit 2 of 0b00000100)

# --- write_value(data: bytearray, offset_bits, num_bits, is_signed, value) ---
buf = bytearray(4)
write_value(buf, 0, 16, False, 1000)           # write 1000 into bits 0-15
write_value(buf, 16, 8, True, -5)              # write -5 into bits 16-23
print(list(buf))                               # [232, 3, 251, 0]

# --- make_field_reader(offset_bits, num_bits, is_signed) -> callable ---
# Pre-build a reusable reader closure for a specific field layout
read_speed = make_field_reader(offset_bits=16, num_bits=12, is_signed=False)
record = b"\x00\x00\x40\x01\x00\x00"
speed = read_speed(record, 0)                  # reads 12 bits from offset 16

# Reuse across many records (avoids rebuilding closure each time)
records = [b"\x00\x00\x80\x00\x00\x00", b"\x00\x00\xc0\x00\x00\x00"]
speeds = [read_speed(r, 0) for r in records]
```

--------------------------------

### Generating Flatdata Benchmark Code

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

Invokes the flatdata_generate_source macro to generate C++ header files from a .flatdata schema definition.

```cmake
flatdata_generate_source(generate_flatdata_benchmark_code
    ${CMAKE_CURRENT_SOURCE_DIR}/graph.flatdata
    ${CMAKE_CURRENT_BINARY_DIR}/generated/graph.hpp)
```

--------------------------------

### Rust build.rs: Generate Flatdata Bindings

Source: https://context7.com/heremaps/flatdata/llms.txt

Invoke the Flatdata generator during the Rust build process. This ensures that Rust bindings are regenerated whenever the schema file changes. Requires the schema file path and the output directory.

```rust
// build.rs — invoke the generator at compile time
fn main() {
    flatdata::generate(
        "src/schema/my_data.flatdata",
        &std::env::var("OUT_DIR").unwrap(),
    )
    .expect("flatdata generator failed");
}
```

--------------------------------

### Set Compile Definitions for Test Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Configures compile-time definitions for the flatdata_test executable. This specific definition sets the console width for the Catch2 testing framework.

```cmake
target_compile_definitions(flatdata_test PRIVATE CATCH_CONFIG_CONSOLE_WIDTH=120)
```

--------------------------------

### Iterate and Filter Elements (Python)

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Iterate over elements in a flatdata archive and count those meeting a specific condition. This method has per-element overhead.

```python
count = sum(1 for x in archive.links if x.speed_limit > 100)
```

--------------------------------

### Target Properties Configuration

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Configures the 'flatdata' target to produce position-independent code, which is essential for shared libraries.

```cmake
set_target_properties(flatdata PROPERTIES POSITION_INDEPENDENT_CODE ON)
```

--------------------------------

### Rust: Include Generated Flatdata Code

Source: https://context7.com/heremaps/flatdata/llms.txt

Include the code generated by flatdata::generate into your Rust crate. This makes the archive, builder, vector, and multivector types available as ordinary Rust structs. Ensure the generated file path is correctly concatenated.

```rust
#![allow(dead_code)]
include!(concat!(env!("OUT_DIR"), "/my_data.rs"));
pub use my_data::*;
```

--------------------------------

### Flatdata Schema Structure Definition with Bit Fields

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Defines a structure with fields, each having a type and a specific bit width. Flatdata handles packing and alignment.

```cpp
struct <Name> {
    <field> : <type> : <width>;
    ...
}
```

--------------------------------

### Define Flatdata Source Files for Generation

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Uses the flatdata_generate_source macro to generate C++ header files from .flatdata schema definitions. This is typically used for creating data structures and serialization code.

```cmake
flatdata_generate_source(generate_flatdata_test_code
    ${CMAKE_CURRENT_SOURCE_DIR}/test_structures.flatdata
    ${CMAKE_CURRENT_BINARY_DIR}/generated/test_structures.hpp)
```

```cmake
flatdata_generate_source(generate_flatdata_test_code2
    ${CMAKE_CURRENT_SOURCE_DIR}/test_structures2.flatdata
    ${CMAKE_CURRENT_BINARY_DIR}/generated/test_structures2.hpp)
```

```cmake
flatdata_generate_source(generate_flatdata_test_case_ranges
    ${CMAKE_CURRENT_SOURCE_DIR}/../../test_cases/archives/ranges.flatdata
    ${CMAKE_CURRENT_BINARY_DIR}/generated/ranges.hpp)
```

--------------------------------

### Flatdata Schema Constant Definition

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Defines a constant with a specified type and value. Use this for fixed values within your schema.

```cpp
const <type> <name> = <value>;
```

--------------------------------

### Flatdata Schema: Archives and Resources

Source: https://context7.com/heremaps/flatdata/llms.txt

Define archives, the smallest unit of data, which group named resources. Resources can be single structs, vectors, multivectors, raw data, or nested archives, and can be marked optional.

```flatdata
namespace coappearances {

    struct Meta   { title_ref : u32 : 32; author_ref : u32 : 32; }
    struct Vertex { name_ref  : u32 : 32; }
    struct Nickname    { ref  : u32 : 32; }
    struct Description { ref  : u32 : 32; }

    archive Graph {
        meta            : Meta;                                   // single instance
        vertices        : vector< Vertex >;                       // random-access vector
        vertices_data   : multivector< 40, Nickname, Description >;  // per-vertex variants
        edges           : vector< Edge >;
        chapters        : vector< Chapter >;
        strings         : raw_data;                               // zero-terminated string pool

        @optional
        statistics      : archive Statistics;                     // optional sub-archive
    }

    archive Statistics {
        invariants     : Invariants;
        vertex_degrees : vector< Degree >;
    }
}
```

--------------------------------

### FileResourceStorage

Source: https://context7.com/heremaps/flatdata/llms.txt

The standard storage backend for reading Flatdata archives in Python. It maps each resource file to memory using Python's `mmap` module, providing zero-copy byte-level access.

```APIDOC
## FileResourceStorage

### Description
`FileResourceStorage` is the standard storage backend for reading Flatdata archives in Python. It maps each resource file to memory using Python's `mmap` module, providing zero-copy byte-level access. The `get(key)` method resolves a resource name to a file path within the archive directory; if the path is a directory, it returns a nested `FileResourceStorage` for sub-archives.

### Methods
- **`__init__(path: str)`**: Initializes the storage with the path to the archive directory.
- **`get(key: str, is_optional: bool = False)`**: Retrieves a resource by key. Returns `mmap.mmap` for raw resources, a nested `FileResourceStorage` for sub-archives, or `None` if `is_optional` is `True` and the resource is not found.
- **`ls()`**: Lists available resources in the directory.

### Example Usage
```python
from flatdata.lib.file_resource_storage import FileResourceStorage

# Point to the directory containing resource files
storage = FileResourceStorage("/path/to/my_archive")

# Retrieve a raw mmap'd resource by name
raw = storage.get("vertices")         # returns mmap.mmap
schema = storage.get("vertices.schema")  # returns mmap.mmap of the schema file

# List available resources in the directory
print(storage.ls())

# Sub-archive resources are returned as nested FileResourceStorage
sub_storage = storage.get("statistics")  # returns FileResourceStorage

# Optional resources return None instead of raising MissingResourceError
maybe = storage.get("optional_resource", is_optional=True)  # None if absent
```
```

--------------------------------

### Set Target Folder for Test Executable

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Organizes the flatdata_test target within the CMake build system's folder structure. This helps in managing targets in larger projects.

```cmake
set_property(TARGET flatdata_test PROPERTY FOLDER "flatdata")
```

--------------------------------

### Python FileResourceStorage - Memory-Mapped File Access

Source: https://context7.com/heremaps/flatdata/llms.txt

Use FileResourceStorage to access resources within an archive directory. It provides memory-mapped access to files and supports nested archives. Optional resources can be retrieved without raising an error if they are missing.

```python
from flatdata.lib.file_resource_storage import FileResourceStorage

# Point to the directory containing resource files
storage = FileResourceStorage("/path/to/my_archive")

# Retrieve a raw mmap'd resource by name
raw = storage.get("vertices")         # returns mmap.mmap
schema = storage.get("vertices.schema")  # returns mmap.mmap of the schema file

# List available resources in the directory
print(storage.ls())

# Sub-archive resources are returned as nested FileResourceStorage
sub_storage = storage.get("statistics")  # returns FileResourceStorage

# Optional resources return None instead of raising MissingResourceError
maybe = storage.get("optional_resource", is_optional=True)  # None if absent
```

--------------------------------

### Mark Field as Optional with @optional

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Use @optional(<name>) on a field to designate a special constant value as a sentinel, making the entire field optional. Many backends use native optional types for such fields.

```cpp
struct MyStruct {
    @optional( SENTINEL_VALUE )
    my_field : i32;
}
```

--------------------------------

### Add Test to CTest

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/test/CMakeLists.txt

Registers the flatdata_test executable as a test case within CTest, allowing it to be discovered and run using the 'ctest' command.

```cmake
add_test(NAME flatdata_test COMMAND flatdata_test)
```

--------------------------------

### Conditional Execution for Non-UNIX Systems

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/benchmark/CMakeLists.txt

This snippet ensures that the subsequent CMake commands are only executed on UNIX-like systems, returning early otherwise.

```cmake
if (NOT ${UNIX})
    return()
endif()
```

--------------------------------

### Vectorized Access to Data (NumPy)

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Access data fields using vectorized methods for bulk operations. Use `vector.field_name` for single column access returning a pandas DataFrame, or `vector.to_numpy()` for a full NumPy structured array.

```python
# single column access, returns a pandas DataFrame
df = archive.links.speed_limit
count = len(df[df['speed_limit'] > 100])

# full NumPy structured array with all fields
arr = archive.links.to_numpy()
count = int(np.sum(arr['speed_limit'] > 100))
```

--------------------------------

### Flatdata Schema: Bit-packed Structures

Source: https://context7.com/heremaps/flatdata/llms.txt

Define compact data structures with arbitrary-width bitfields. Fields can use fewer bits than their underlying type, and Flatdata handles alignment and masking.

```flatdata
// Bit-packed structure: total size = 29+2 = 31 bits, stored in 4 bytes
struct Vertex {
    name_ref     : u32 : 29;  // 29-bit offset into a strings blob
    is_active    : bool : 1;
    kind         : u8   : 2;  // 2-bit enum-like field
}

// An edge connecting two 16-bit vertex references plus a 16-bit count
struct Edge {
    a_ref  : u16 : 16;
    b_ref  : u16 : 16;
    count  : u32 : 16;
    @range( chapters_range )
    first_chapter_ref : u32 : 16;  // @range exposes a derived range attribute
}

// Enum definition: underlying type u8 stored in 3 bits
enum ChapterKind : u8 : 3 {
    MINOR  = 0,
    MAJOR  = 1,
    EPILOG = 2,
}

// Struct using an enum field
struct Chapter {
    major : u8      : 6;
    minor : u8      : 6;
    kind  : ChapterKind : 3;
}
```

--------------------------------

### Vectorized Slicing (NumPy and Pandas)

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-py/README.md

Utilize slicing with vectorized access methods to retrieve subsets of data as NumPy arrays or pandas DataFrames.

```python
# slices work too
arr = archive.links[1000:2000].to_numpy()
df = archive.links[::10].to_data_frame()
```

--------------------------------

### Flatdata Schema Enumeration Definition

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Defines an enumeration over a basic type. Values can be auto-assigned or manually specified. Ensure values fit the underlying type and avoid duplicates.

```cpp
enum <name> : <type> [ : bits ] {
    <value name> [= value],
    ...
}
```

--------------------------------

### Conditional Compilation for Debug Statistics

Source: https://github.com/heremaps/flatdata/blob/master/flatdata-cpp/CMakeLists.txt

Enables the DEBUG_DATA_ACCESS_STATISTICS compile definition if the WITH_DEBUG_DATA_ACCESS_STATISTICS option is set.

```cmake
if(WITH_DEBUG_DATA_ACCESS_STATISTICS)
  message(STATUS "WITH_DEBUG_DATA_ACCESS_STATISTICS enabled")
  target_compile_definitions(flatdata PUBLIC DEBUG_DATA_ACCESS_STATISTICS)
endif()
```

--------------------------------

### Group Implicitly Bound Resources with @bound_implicitly

Source: https://github.com/heremaps/flatdata/blob/master/docs/schema-language.md

Employ @bound_implicitly to group multiple resources that represent a single entity, promoting data locality. This decoration also assigns a name to the entity.

```cpp
@bound_implicitly( transactions: keys, transaction_data )
archive Archive {
    keys: vector< Key >
    transaction_data : vector< Transaction >
}
```

--------------------------------

### Vectorized Field Read

Source: https://context7.com/heremaps/flatdata/llms.txt

Efficiently reads a single bit-packed field from all elements of a vector at once, returning a NumPy array.

```APIDOC
## read_field_vectorized

### Description
`read_field_vectorized` reads a single bit-packed field from all elements of a vector at once, returning a NumPy array. It accepts a 2D `uint8` NumPy array of shape `(n_elements, struct_size_bytes)` and produces a `uint64` or `int64` array of the field values, handling arbitrary bit offsets and widths up to 64 bits. This is the engine behind `Vector.to_numpy()` and column access like `vector.field_name`.

### Parameters
- **`data`** (`np.ndarray`): A 2D NumPy array of `uint8` with shape `(n_elements, struct_size_bytes)`.
- **`field_offset_bits`** (`int`): The starting bit offset of the field within each element's structure.
- **`field_width_bits`** (`int`): The number of bits the field occupies.
- **`is_signed`** (`bool`): Whether the field should be interpreted as signed.

### Returns
- **`np.ndarray`**: A NumPy array of `uint64` or `int64` containing the extracted field values.

### Example Usage
```python
import numpy as np
from flatdata.lib.data_access import read_field_vectorized

# Simulate a packed vector: 3 elements, each 4 bytes (32-bit struct)
# Field layout: bits 0-15 = x (u16), bits 16-31 = y (u16)
raw_bytes = np.array([
    [0x0A, 0x00, 0x14, 0x00],   # x=10,  y=20
    [0x1E, 0x00, 0x28, 0x00],   # x=30,  y=40
    [0x32, 0x00, 0x3C, 0x00],   # x=50,  y=60
], dtype=np.uint8)

x_values = read_field_vectorized(raw_bytes, field_offset_bits=0,  field_width_bits=16, is_signed=False)
y_values = read_field_vectorized(raw_bytes, field_offset_bits=16, field_width_bits=16, is_signed=False)

print(x_values)  # array([10, 30, 50], dtype=uint64)
print(y_values)  # array([20, 40, 60], dtype=uint64)

# Signed field example — a 5-bit signed field at bit offset 3
signed_raw = np.array([[0b00011000, 0x00]], dtype=np.uint8)  # bits 3-7 = 0b00011 = 3
result = read_field_vectorized(signed_raw, field_offset_bits=3, field_width_bits=5, is_signed=True)
print(result)  # array([3], dtype=int64)
```
```