### Install nanoparquet

Source: https://nanoparquet.r-lib.org/index.html

Install the package from CRAN.

```R
install.packages("nanoparquet")
```

--------------------------------

### Load Libraries for Benchmarking

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Loads necessary R libraries for data manipulation, table creation, and table styling. Ensure these libraries are installed before running.

```r
library(dplyr)
library(gt)
library(gtExtras)
```

--------------------------------

### Specify Non-Default Parquet Schema

Source: https://nanoparquet.r-lib.org/reference/nanoparquet-types.html

Use `parquet_schema()` to define custom Parquet types for specific columns when writing data frames. This example shows how to write a factor column named 'name' as an ENUM type.

```R
write_parquet(..., schema = parquet_schema(name = "ENUM"))
```

--------------------------------

### Load pillar package

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Initializes the pillar package for improved data frame printing.

```R
library(pillar)
```

--------------------------------

### Create Nanoparquet Options

Source: https://nanoparquet.r-lib.org/reference/parquet_options.html

Defines the configuration list for nanoparquet operations, allowing customization of class assignment, compression levels, and metadata handling.

```R
parquet_options(
  class = getOption("nanoparquet.class", "tbl"),
  compression_level = getOption("nanoparquet.compression_level", NA_integer_),
  keep_row_groups = FALSE,
  num_rows_per_row_group = getOption("nanoparquet.num_rows_per_row_group", 10000000L),
  use_arrow_metadata = getOption("nanoparquet.use_arrow_metadata", TRUE),
  write_arrow_metadata = getOption("nanoparquet.write_arrow_metadata", TRUE),
  write_data_page_version = getOption("nanoparquet.write_data_page_version", 1L),
  write_minmax_values = getOption("nanoparquet.write_minmax_values", TRUE)
)
```

--------------------------------

### Nanoparquet Options Configuration

Source: https://nanoparquet.r-lib.org/reference/parquet_options.html

Configure and retrieve nanoparquet options for reading and writing Parquet files.

```APIDOC
## Function: parquet_options

### Description
Create a list of nanoparquet options.

### Usage
```R
parquet_options(
  class = getOption("nanoparquet.class", "tbl"),
  compression_level = getOption("nanoparquet.compression_level", NA_integer_),
  keep_row_groups = FALSE,
  num_rows_per_row_group = getOption("nanoparquet.num_rows_per_row_group", 10000000L),
  use_arrow_metadata = getOption("nanoparquet.use_arrow_metadata", TRUE),
  write_arrow_metadata = getOption("nanoparquet.write_arrow_metadata", TRUE),
  write_data_page_version = getOption("nanoparquet.write_data_page_version", 1L),
  write_minmax_values = getOption("nanoparquet.write_minmax_values", TRUE)
)
```

### Arguments 

* `class` (character) - The extra class or classes to add to data frames created in `read_parquet()`. By default nanoparquet adds the `"tbl"` class, so data frames are printed differently if the pillar package is loaded.

* `compression_level` (integer) - The compression level in `write_parquet()`. `NA` is the default, and it specifies the default compression level of each method. `Inf` always selects the highest possible compression level. More details:
  * Snappy does not support compression levels currently.
  * GZIP supports levels from 0 (uncompressed), 1 (fastest), to 9 (best). The default is 6.
  * ZSTD allows positive levels up to 22 currently. 20 and above require more memory. Negative levels are also allowed, the lower the level, the faster the speed, at the cost of compression. Currently the smallest level is -131072. The default level is 3.

* `keep_row_groups` (logical) - This option is used when appending to a Parquet file with `append_parquet()`. If `TRUE` then the existing row groups of the file are always kept as is and nanoparquet creates new row groups for the new data. If `FALSE` (the default), then the last row group of the file will be overwritten if it is smaller than the default row group size, i.e. `num_rows_per_row_group`.

* `num_rows_per_row_group` (integer) - The number of rows to put into a row group, if row groups are not specified explicitly. It should be an integer scalar. Defaults to 10 million.

* `use_arrow_metadata` (logical) - `TRUE` or `FALSE`. If `TRUE`, then `read_parquet()` and `read_parquet_schema()` will make use of the Apache Arrow metadata to assign R classes to Parquet columns. This is currently used to detect factor columns, and to detect "difftime" columns.
If this option is `FALSE`:
  * "factor" columns are read as character vectors.
  * "difftime" columns are read as real numbers, meaning one of seconds, milliseconds, microseconds or nanoseconds. Impossible to tell which without using the Arrow metadata.

* `write_arrow_metadata` (logical) - Whether to add the Apache Arrow types as metadata to the file `write_parquet()`.

* `write_data_page_version` (integer) - Data version to write by default. Possible values are 1 and 2. Default is 1.

* `write_minmax_values` (logical) - Whether to write minimum and maximum values per row group, for data types that support this in `write_parquet()`. However, nanoparquet currently does not support minimum and maximum values for the `DECIMAL`, `UUID` and `FLOAT16` logical types and the `BOOLEAN`, `BYTE_ARRAY` and `FIXED_LEN_BYTE_ARRAY` primitive types if they are writing without a logical type. Currently the default is `TRUE`.

### Value
List of nanoparquet options.

### Examples
```R
if (FALSE) {
# the effect of using Arrow metadata
tmp <- tempfile(fileext = ".parquet")
d <- data.frame(
  fct = as.factor("a"),
  dft = as.difftime(10, units = "secs")
)
write_parquet(d, tmp)
read_parquet(tmp, options = parquet_options(use_arrow_metadata = TRUE))
read_parquet(tmp, options = parquet_options(use_arrow_metadata = FALSE))
}
```
```

--------------------------------

### Print benchmark results

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Displays the full benchmark results table.

```R
print(results, n = Inf)
```

--------------------------------

### Load and display session information

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Loads the sessioninfo package and prints the current R session details.

```R
library(sessioninfo)
si
```

--------------------------------

### Compare Arrow Metadata Usage

Source: https://nanoparquet.r-lib.org/reference/parquet_options.html

Demonstrates how toggling the use_arrow_metadata option affects the reading of factor and difftime columns from a Parquet file.

```R
if (FALSE) {
# the effect of using Arrow metadata
tmp <- tempfile(fileext = ".parquet")
d <- data.frame(
  fct = as.factor("a"),
  dft = as.difftime(10, units = "secs")
)
write_parquet(d, tmp)
read_parquet(tmp, options = parquet_options(use_arrow_metadata = TRUE))
read_parquet(tmp, options = parquet_options(use_arrow_metadata = FALSE))
}
```

--------------------------------

### Generate Parquet Writing Performance Table

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Creates a formatted table summarizing Parquet writing performance using the 'gt' package. It includes headers, row grouping by data size, column hiding, alignment, byte formatting, and bar plots for speedup.

```R
pq_tab_write |>
  gt() |>
  tab_header(title = "Parquet implementations, writing") |>
  tab_options(table.align = "left") |>
  tab_row_group(md("**small data**"), rows = `data size` == "small", "s") |>
  tab_row_group(md("**medium data**"), rows = `data size` == "medium", "m") |>
  tab_row_group(md("**large data**"), rows = `data size` == "large", "l") |>
  row_group_order(c("s", "m", "l")) |>
  cols_hide(columns = c(`data size`, rawtime)) |>
  cols_align(columns = time, align = "right") |>
  fmt_bytes(columns = c(memory, `file size`)) |>
  gt_plt_bar(column = `speedup from CSV`)
```

--------------------------------

### Benchmark Execution Script

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Executes performance benchmarks for multiple data variants and sizes, caching results in a Parquet file.

```R
if (file.exists(file.path(me, "results.parquet"))) {
  results <- nanoparquet::read_parquet(file.path(me, "results.parquet"))
} else {
  results <- NULL
  lapply(data_sizes, function(s) {
    lapply(variants, function(v) {
      r <- if (v == "readr" && s == "large") {
        measure(v, s)
      } else {
        measure(v, s)
        measure(v, s)
        measure(v, s)
      }
      results <<- rbind(results, r)
    })
  })
  nanoparquet::write_parquet(results, file.path(me, "results.parquet"))
}
```

--------------------------------

### Parquet Schemas and Type Mappings

Source: https://nanoparquet.r-lib.org/reference/index.html

Functions for inferring, creating, and understanding Parquet schemas and type mappings.

```APIDOC
## infer_parquet_schema()

### Description
Infer Parquet schema of a data frame.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A

## nanoparquet-types

### Description
nanoparquet's type maps.

### Method
N/A

### Endpoint
N/A

## parquet-encodings

### Description
Parquet encodings.

### Method
N/A

### Endpoint
N/A

## parquet_schema()

### Description
Create a Parquet schema.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A
```

--------------------------------

### Generate Parquet Reading Performance Table

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Creates a formatted table summarizing Parquet reading performance using the 'gt' package. It includes headers, row grouping by data size, column hiding, alignment, byte formatting, and bar plots for speedup.

```R
pq_tab_read |>
  gt() |>
  tab_header(title = "Parquet implementations, reading") |>
  tab_options(table.align = "left") |>
  tab_row_group(md("**small data**"), rows = `data size` == "small", "s") |>
  tab_row_group(md("**medium data**"), rows = `data size` == "medium", "m") |>
  tab_row_group(md("**large data**"), rows = `data size` == "large", "l") |>
  row_group_order(c("s", "m", "l")) |>
  cols_hide(columns = c(`data size`, rawtime)) |>
  cols_align(columns = time, align = "right") |> 
  fmt_bytes(columns = memory) |>
  gt_plt_bar(column = `speedup from CSV`)
```

--------------------------------

### Nanoparquet Options

Source: https://nanoparquet.r-lib.org/reference/index.html

Function to access and manage nanoparquet options.

```APIDOC
## parquet_options()

### Description
Nanoparquet options.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A
```

--------------------------------

### Prepare Data for Parquet Writing Comparison

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Filters and transforms results data to prepare for a comparative table of Parquet writing performance. It renames columns, calculates memory usage, determines speedup factors, and formats time for display.

```R
pq_tab_write <- results |>
  filter(software %in% c("nanoparquet", "arrow", "duckdb", "readr")) |>
  filter(direction == "write") |>
  rename(`data size` = data_size, time = time_elapsed, `file size` = file_size) |>
  mutate(memory = mem_max_after - mem_before) |>
  mutate(base = tail(time, 1), .by = `data size`) |>
  mutate(speedup = base / time, x = round(speedup, digits = 1)) |>
  select(`data size`, software, time, x, speedup, memory, `file size`) |>
  mutate(rawtime = time, time = prettyunits::pretty_sec(time)) |>
  filter(software %in% c("nanoparquet", "arrow", "duckdb", "readr")) |>
  mutate(software = case_when(
    software ==  "arrow" ~ "Arrow",
    software == "duckdb" ~ "DuckDB",
    .default = software
  )) |>
  rename(`speedup from CSV` = speedup)
```

--------------------------------

### Debugging Parquet Files

Source: https://nanoparquet.r-lib.org/reference/index.html

Utility functions for debugging Parquet files and for nanoparquet developers.

```APIDOC
## read_parquet_pages()

### Description
Metadata of all pages of a Parquet file.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A

## read_parquet_page()

### Description
Read a page from a Parquet file.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A
```

--------------------------------

### Write Parquet Files

Source: https://nanoparquet.r-lib.org/index.html

Write data frames to Parquet format and inspect the schema before writing.

```R
nanoparquet::write_parquet(mtcars, "mtcars.parquet")
```

```R
nanoparquet::infer_parquet_schema(mtcars)
```

--------------------------------

### Glimpse Flights Data Structure

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Provides a concise summary of the nycflights13::flights dataset, including the number of rows, columns, and the data type of each column.

```r
dplyr::glimpse(nycflights13::flights)
```

--------------------------------

### Prepare Data for Parquet Reading Comparison

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Filters and transforms results data to prepare for a comparative table of Parquet reading performance. It renames columns, calculates memory usage, determines speedup factors, and formats time for display.

```R
pq_tab_read <- results |>
  filter(software %in% c("nanoparquet", "arrow", "duckdb", "readr")) |>
  filter(direction == "read") |>
  rename(`data size` = data_size, time = time_elapsed) |>
  mutate(memory = mem_max_after - mem_before) |>
  mutate(base = tail(time, 1), .by = `data size`) |>
  mutate(speedup = base / time, x = round(speedup, digits = 1)) |>
  select(`data size`, software, time, x, speedup, memory) |>
  mutate(rawtime = time, time = prettyunits::pretty_sec(time)) |>
  filter(software %in% c("nanoparquet", "arrow", "duckdb")) |> 
  mutate(software = case_when(
    software ==  "arrow" ~ "Arrow",
    software == "duckdb" ~ "DuckDB",
    .default = software
  )) |>
  rename(`speedup from CSV` = speedup)
```

--------------------------------

### Read Parquet File Schema

Source: https://nanoparquet.r-lib.org/reference/read_parquet_schema.html

Use this function to read the schema of a Parquet file. It works even if `read_parquet()` cannot read the file due to unsupported schema, encoding, or compression.

```R
read_parquet_schema(file, options = parquet_options())
```

--------------------------------

### Cite nanoparquet in BibTeX format

Source: https://nanoparquet.r-lib.org/authors.html

Use this BibTeX entry to cite the nanoparquet package in LaTeX documents.

```bibtex
@Manual{,
  title = {nanoparquet: Read and Write 'Parquet' Files},
  author = {Gábor Csárdi and Hannes Mühleisen},
  year = {2025},
  note = {R package version 0.4.3},
  url = {https://github.com/r-lib/nanoparquet},
}
```

--------------------------------

### Read Parquet Files

Source: https://nanoparquet.r-lib.org/index.html

Read a single Parquet file or a collection of files from a directory.

```R
df <- nanoparquet::read_parquet("example.parquet")
```

```R
nanoparquet::read_parquet_schema("example.parquet")
```

```R
df <- data.table::rbindlist(lapply(
  Sys.glob("some-folder/part-*.parquet"),
  nanoparquet::read_parquet
))
```

--------------------------------

### Create a Parquet Schema

Source: https://nanoparquet.r-lib.org/reference/parquet_schema.html

Defines how to create a Parquet schema to specify the structure for writing data frames to Parquet files. It supports various primitive and logical Parquet types, including parameterized types and repetition specifications.

```APIDOC
## POST /parquet_schema

### Description
Creates a Parquet schema to specify how to write a data frame to a Parquet file with `write_parquet()`.

### Method
POST

### Endpoint
/parquet_schema

### Parameters
#### Arguments
- **...** (any) - Used to specify Parquet type specifications. Can be a character scalar or a list. Parameterized types need to be specified as a list. Primitive Parquet types can be specified as a string or a list. For backwards compatibility, a file name can be supplied, in which case `parquet_schema` behaves as `read_parquet_schema()`.

### Possible Types:
- **Special Type:**
  - `"AUTO"`: Maps R type to Parquet automatically.
- **Primitive Parquet Types:**
  - `"BOOLEAN"`
  - `"INT32"`
  - `"INT64"`
  - `"INT96"`
  - `"FLOAT"`
  - `"DOUBLE"`
  - `"BYTE_ARRAY"`
  - `"FIXED_LEN_BYTE_ARRAY"`: Requires `type_length` (integer 0 to 2^31-1).
- **Parquet Logical Types:**
  - `"STRING"`
  - `"ENUM"`
  - `"UUID"`
  - `"INTEGER"`: Requires `bit_width` (8, 16, 32, 64) and `is_signed` (TRUE/FALSE).
  - `"INT"`: Alias for `"INTEGER"`.
  - `"DECIMAL"`: Requires `precision` and `primitive_type`. Optional `scale` (defaults to 0).
  - `"FLOAT16"`
  - `"DATE"`
  - `"TIME"`: Requires `is_adjusted_utc` (TRUE/FALSE) and `unit` ("MILLIS", "MICROS", "NANOS").
  - `"TIMESTAMP"`: Requires `is_adjusted_utc` (TRUE/FALSE) and `unit` ("MILLIS", "MICROS", "NANOS").
  - `"JSON"`
  - `"BSON"`

*Note: `MAP`, `LIST`, and `UNKNOWN` logical types are not currently supported.*

### Converted Types (Deprecated Shortcuts):
- `INT_8`, `INT_16`, `INT_32`, `INT_64`: Shortcuts for signed integers.
- `TIME_MICROS`, `TIME_MILLIS`: Shortcuts for `TIME` with UTC adjustment.
- `TIMESTAMP_MICROS`, `TIMESTAMP_MILLIS`: Shortcuts for `TIMESTAMP` with UTC adjustment.
- `UINT_8`, `UINT_16`, `UINT_32`, `UINT_64`: Shortcuts for unsigned integers.

### Missing Values
- `repetition_type`: Can be `"REQUIRED"`, `"OPTIONAL"`, or `"REPEATED"`. `"REPEATED"` is not supported in `write_parquet()`.

### Request Example
```json
{
  "c1": "INT32",
  "c2": {"type": "INT", "bit_width": 64, "is_signed": true},
  "c3": {"type": "STRING", "repetition_type": "OPTIONAL"}
}
```

### Response
#### Success Response (200)
Returns a data frame with schema details, including columns: `file_name`, `name`, `r_type`, `type`, `type_length`, `repetition_type`, `converted_type`, `logical_type`, `num_children`, `scale`, `precision`, `field_id`.

#### Response Example
```json
{
  "file_name": null,
  "name": "c1",
  "r_type": null,
  "type": "INT32",
  "type_length": null,
  "repetition_type": "REQUIRED",
  "converted_type": null,
  "logical_type": null,
  "num_children": null,
  "scale": null,
  "precision": null,
  "field_id": null
}
```
```

--------------------------------

### Inspect Parquet Metadata

Source: https://nanoparquet.r-lib.org/index.html

Retrieve metadata, schema, and file information from a Parquet file.

```R
nanoparquet::read_parquet_info("mtcars.parquet")
nanoparquet::read_parquet_schema("mtcars.parquet")
nanoparquet::read_parquet_metadata("mtcars.parquet")
```

--------------------------------

### Read Parquet Metadata Functions

Source: https://nanoparquet.r-lib.org/reference/read_parquet_metadata.html

Use these functions to retrieve metadata from a Parquet file path. read_parquet_metadata allows for custom options, while parquet_metadata provides a simplified interface.

```R
read_parquet_metadata(file, options = parquet_options())

parquet_metadata(file)
```

--------------------------------

### Generate and Display Data Set Information

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Reads data set information from a Parquet file if it exists, otherwise generates it using specified functions and writes it back. Displays the information using the gt package.

```r
if (file.exists(file.path(me, "data-info.parquet"))) {
  info_tab <- nanoparquet::read_parquet(file.path(me, "data-info.parquet"))
} else {
  get_data_info <- function(x) {
    list(dim = dim(x), size = object.size(x))
  }
  info <- lapply(data_sizes, function(s) get_data_info(gen_data(s)))
  info_tab <- data.frame(
    check.names = FALSE,
    name = data_sizes,
    rows = sapply(info, "[[", "dim")[1,],
    columns = sapply(info, "[[", "dim")[2,],
    "size in memory" = sapply(info, "[[", "size")
  )
  nanoparquet::write_parquet(info_tab, file.path(me, "data-info.parquet"))
}
info_tab |
  gt() |
  tab_header(title = "Data sets") |
  tab_options(table.align = "left") |
  fmt_integer() |
  fmt_bytes(columns = "size in memory")
```

--------------------------------

### read_parquet_info / parquet_info

Source: https://nanoparquet.r-lib.org/reference/read_parquet_info.html

Retrieves a summary of a Parquet file, including file size, row counts, and version information.

```APIDOC
## read_parquet_info(file)

### Description
Returns a summary of a Parquet file as a data frame.

### Arguments
- **file** (string) - Required - Path to a Parquet file.

### Value
A data frame with the following columns:
- **file_name** (string) - File name.
- **num_cols** (integer) - Number of (leaf) columns.
- **num_rows** (integer) - Number of rows.
- **num_row_groups** (integer) - Number of row groups.
- **file_size** (integer) - File size in bytes.
- **parquet_version** (string) - Parquet version.
- **created_by** (string) - Name of the software that created the file (NA if not available).
```

--------------------------------

### Read Parquet file summary in R

Source: https://nanoparquet.r-lib.org/reference/read_parquet_info.html

Use these functions to obtain a data frame containing metadata about a Parquet file.

```R
read_parquet_info(file)

parquet_info(file)
```

--------------------------------

### Read Parquet vs CSV Data

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Filters and processes results to compare reading performance of nanoparquet and CSV readers. It calculates speedup and memory usage, then formats the output into a gt table for visualization.

```R
csv_tab_read <- results |>
  filter(software %in% c("nanoparquet", "data.table", "data.table.gz", "readr")) |>
  filter(direction == "read") |>
  mutate(software = case_when(
    software ==  "data.table.gz" ~ "data.table (compressed)",
    .default = software
  )) |>
  rename(`data size` = data_size, time = time_elapsed) |>
  mutate(memory = mem_max_after - mem_before) |>
  mutate(base = tail(time, 1), .by = `data size`) |>
  mutate(speedup = base / time, x = round(speedup, digits = 1)) |>
  select(`data size`, software, time, x, speedup, memory) |>
  mutate(rawtime = time, time = prettyunits::pretty_sec(time)) |>
  rename(`speedup from CSV` = speedup)

csv_tab_read |>
  gt() |>
  tab_header(title = "Parquet vs CSV, reading") |>
  tab_options(table.align = "left") |>
  tab_row_group(md("**small data**"), rows = `data size` == "small", "s") |>
  tab_row_group(md("**medium data**"), rows = `data size` == "medium", "m") |>
  tab_row_group(md("**large data**"), rows = `data size` == "large", "l") |>
  row_group_order(c("s", "m", "l")) |>
  cols_hide(columns = c(`data size`, rawtime)) |>
  cols_align(columns = time, align = "right") |>
  fmt_bytes(columns = memory) |>
  gt_plt_bar(column = `speedup from CSV`)
```

--------------------------------

### Read Parquet file page metadata

Source: https://nanoparquet.r-lib.org/reference/read_parquet_pages.html

Retrieves metadata for all pages in a specified Parquet file.

```R
read_parquet_pages(file)
```

```R
file_name <- system.file("extdata/userdata1.parquet", package = "nanoparquet")
nanoparquet:::read_parquet_pages(file_name)
```

--------------------------------

### Write Parquet vs CSV Data

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Filters and processes results to compare writing performance of nanoparquet and CSV writers. It calculates speedup, memory usage, and file size, then formats the output into a gt table for visualization.

```R
csv_tab_write <- results |>
  filter(software %in% c("nanoparquet", "data.table", "data.table.gz", "readr")) |>
  filter(direction == "write") |>
  mutate(software = case_when(
    software ==  "data.table.gz" ~ "data.table (compressed)",
    .default = software
  )) |>
  rename(`data size` = data_size, time = time_elapsed, `file size` = file_size) |>
  mutate(memory = mem_max_after - mem_before) |>
  mutate(base = tail(time, 1), .by = `data size`) |>
  mutate(speedup = base / time, x = round(speedup, digits = 1)) |>
  select(`data size`, software, time, x, speedup, memory, `file size`) |>
  mutate(rawtime = time, time = prettyunits::pretty_sec(time)) |>
  rename(`speedup from CSV` = speedup)

csv_tab_write |>
  gt() |>
  tab_header(title = "Parquet vs CSV, writing") |>
  tab_options(table.align = "left") |>
  tab_row_group(md("**small data**"), rows = `data size` == "small", "s") |>
  tab_row_group(md("**medium data**"), rows = `data size` == "medium", "m") |>
  tab_row_group(md("**large data**"), rows = `data size` == "large", "l") |>
  row_group_order(c("s", "m", "l")) |>
  cols_hide(columns = c(`data size`, rawtime)) |>
  cols_align(columns = time, align = "right") |>
  fmt_bytes(columns = c(memory, `file size`)) |>
  gt_plt_bar(column = `speedup from CSV`)
```

--------------------------------

### Display Head of Flights Data

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Shows the first few rows of the nycflights13::flights dataset to provide a quick look at its structure and content.

```r
head(nycflights13::flights)
```

--------------------------------

### Read Parquet Page Metadata and Content

Source: https://nanoparquet.r-lib.org/reference/read_parquet_page.html

Use internal functions to inspect page metadata and retrieve specific page data from a Parquet file.

```R
file_name <- system.file("extdata/userdata1.parquet", package = "nanoparquet")
nanoparquet:::read_parquet_pages(file_name)
```

```R
options(max.print = 100)  # otherwise long raw vector
nanoparquet:::read_parquet_page(file_name, 4L)
```

--------------------------------

### Read a Parquet page

Source: https://nanoparquet.r-lib.org/reference/read_parquet_page.html

Reads a single page from a Parquet file at the specified byte offset.

```R
read_parquet_page(file, offset)
```

--------------------------------

### Write Parquet Files

Source: https://nanoparquet.r-lib.org/reference/index.html

Functions for writing R data frames to Parquet files.

```APIDOC
## append_parquet()

### Description
Append a data frame to an existing Parquet file.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A

## write_parquet()

### Description
Write a data frame to a Parquet file.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A
```

--------------------------------

### read_parquet

Source: https://nanoparquet.r-lib.org/reference/read_parquet.html

Reads a Parquet file and converts it into an R data frame.

```APIDOC
## read_parquet

### Description
Converts the contents of the named Parquet file to an R data frame.

### Method
Function Call

### Parameters
#### Arguments
- **file** (string/connection) - Required - Path to a Parquet file or an R connection.
- **col_select** (numeric vector/character vector) - Optional - Columns to read. Can be indices or column names.
- **options** (list) - Optional - Nanoparquet options, see parquet_options().

### Value
A data.frame with the file's contents.

### Request Example
read_parquet("data.parquet", col_select = c("id", "first_name"))
```

--------------------------------

### Read Session Info from RDS

Source: https://nanoparquet.r-lib.org/articles/benchmarks.html

Reads session information from an RDS file if it exists, otherwise captures and saves it. This is useful for reproducibility in benchmarking.

```r
if (file.exists(file.path(me, "sessioninfo.rds"))) {
  si <- readRDS(file.path(me, "sessioninfo.rds"))
} else {
  si <- sessioninfo::session_info()
  saveRDS(si, file.path(me, "sessioninfo.rds"))
}
```

--------------------------------

### Read Parquet Page Metadata

Source: https://nanoparquet.r-lib.org/reference/read_parquet_pages.html

This function retrieves metadata for all pages within a specified Parquet file. It provides detailed information about each page, including its file name, row group, column, type, offsets, sizes, compression, encoding, and value counts.

```APIDOC
## GET /read_parquet_pages

### Description
Retrieves metadata for all pages of a Parquet file.

### Method
GET

### Endpoint
/read_parquet_pages

### Parameters
#### Query Parameters
- **file** (string) - Required - Path to a Parquet file.

### Response
#### Success Response (200)
- **file_name** (string) - The name of the Parquet file.
- **row_group** (integer) - The ID of the row group the page belongs to.
- **column** (integer) - The ID of the column the page belongs to.
- **page_type** (string) - The type of the page (e.g., 'DATA_PAGE', 'DICTIONARY_PAGE').
- **page_header_offset** (integer) - The offset of the page header in the file.
- **uncompressed_page_size** (integer) - The uncompressed size of the page (excluding header).
- **compressed_page_size** (integer) - The compressed size of the page (excluding header).
- **crc** (integer) - The checksum of the page, if present (can be NA).
- **num_values** (integer) - The number of data values in the page, including NULLs.
- **encoding** (string) - The encoding used for the page data.
- **definition_level_encoding** (string) - The encoding used for definition levels.
- **repetition_level_encoding** (string) - The encoding used for repetition levels.
- **data_offset** (integer) - The offset of the actual data within the file.
- **page_header_length** (integer) - The size of the page header in bytes.

### Request Example
```json
{
  "file": "/path/to/your/file.parquet"
}
```

### Response Example
```json
{
  "file_name": "/home/runner/work/_temp/userdata1.parquet",
  "row_group": 0,
  "column": 0,
  "page_type": "DATA_PAGE",
  "page_header_offset": 4,
  "uncompressed_page_size": 6737,
  "compressed_page_size": 6737,
  "crc": null,
  "num_values": 1000,
  "encoding": "PLAIN",
  "definition_level_encoding": "RLE",
  "repetition_level_encoding": "RLE",
  "data_offset": 6741,
  "page_header_length": 4
}
```
```

--------------------------------

### Write Data Frame to Parquet File

Source: https://nanoparquet.r-lib.org/reference/write_parquet.html

Use this function to write an R data frame to a Parquet file. Row names are ignored and must be added as a column if needed. The file path can be ":raw:" to return the data as a raw vector.

```R
write_parquet(
  x, 
  file, 
  schema = NULL, 
  compression = c("snappy", "gzip", "zstd", "uncompressed"), 
  encoding = NULL, 
  metadata = NULL, 
  row_groups = NULL, 
  options = parquet_options()
)
```

```R
if (FALSE) {
# add row names as a column, because `write_parquet()` ignores them.
mtcars2 <- cbind(name = rownames(mtcars), mtcars)
write_parquet(mtcars2, "mtcars.parquet")
}
```

--------------------------------

### Write Data Frame to Parquet

Source: https://nanoparquet.r-lib.org/reference/write_parquet.html

This section details the `write_parquet` function, which writes the contents of an R data frame into a Parquet file.

```APIDOC
## POST /write_parquet

### Description
Writes the contents of an R data frame into a Parquet file.

### Method
POST

### Endpoint
/write_parquet

### Parameters
#### Path Parameters
- **file** (string) - Required - Path to the output file. If this is the string ":raw:", then the data frame is written to a memory buffer, and the memory buffer is returned as a raw vector.

#### Query Parameters
- **schema** (ParquetSchema) - Optional - Parquet schema. Specify a schema to tweak the default nanoparquet R -> Parquet type mappings. Use `parquet_schema()` to create a schema that you can use here, or `read_parquet_schema()` to use the schema of a Parquet file.
- **compression** (string) - Optional - Compression algorithm to use. Currently "snappy" (the default), "gzip", "zstd", and "uncompressed" are supported.
- **encoding** (string or list) - Optional - Encoding to use. Possible values:
  * If `NULL`, the appropriate encoding is selected automatically: `RLE` or `PLAIN` for `BOOLEAN` columns, `RLE_DICTIONARY` for other columns with many repeated values, and `PLAIN` otherwise.
  * If It is a single (unnamed) character string, then it'll be used for all columns.
  * If it is an unnamed character vector of encoding names of the same length as the number of columns in the data frame, then those encodings will be used for each column.
  * If it is a named character vector, then the named must be unique and each name must match a column name, to specify the encoding of that column. The special empty name (`""`) applies to the rest of the columns. If there is no empty name, the rest of the columns will use the default encoding.
If `NA_character_` is specified for a column, the default encoding is used for the column.
If a specified encoding is invalid for a certain column type, or nanoparquet does not implement it, `write_parquet()` throws an error.
Currently `write_parquet()` supports the following encodings:
  * `PLAIN` for all column types,
  * `PLAIN_DICTIONARY` and `RLE_DICTIONARY` for all column types,
  * `RLE` for BOOLEAN columns.
See parquet-encodings for more about encodings.
- **metadata** (named character vector or data frame) - Optional - Additional key-value metadata to add to the file. This must be a named character vector, or a data frame with columns character columns called `key` and `value`.
- **row_groups** (integer vector) - Optional - Row groups of the Parquet file. If `NULL`, then the `num_rows_per_row_group` option is used from the `options` argument, see `parquet_options()`. Otherwise it must be an integer vector, specifying the starts of the row groups.
- **options** (NanoparquetOptions) - Optional - Nanoparquet options, see `parquet_options()`.

#### Request Body
- **x** (data.frame) - Required - Data frame to write.

### Request Example
```json
{
  "x": {},
  "file": "mtcars.parquet",
  "schema": null,
  "compression": "snappy",
  "encoding": null,
  "metadata": null,
  "row_groups": null,
  "options": {}
}
```

### Response
#### Success Response (200)
- **raw vector** (raw) - The Parquet file as a raw vector if `file` is ":raw:". Otherwise, `NULL`.

#### Response Example
```json
null
```
```

--------------------------------

### Map R and Parquet Data Types

Source: https://nanoparquet.r-lib.org/reference/parquet_column_types.html

This function maps R data types to Parquet types for writing, or Parquet file types to R types for reading. Note: This function is deprecated.

```APIDOC
## GET /parquet_column_types

### Description
Maps R data types to Parquet types for writing, or Parquet file types to R types for reading. This function is deprecated and users should use `read_parquet_schema()` for files and `infer_parquet_schema()` for data frames.

### Method
GET

### Endpoint
/parquet_column_types

### Parameters
#### Query Parameters
- **x** (file path or data frame) - Required - Path to a Parquet file, or a data frame.
- **options** (parquet_options) - Optional - Nanoparquet options, see `parquet_options()`.

### Response
#### Success Response (200)
- **file_name** (string) - The name of the file.
- **name** (string) - The name of the column.
- **type** (string) - The low-level Parquet data type.
- **r_type** (string) - The R type that corresponds to the Parquet type. Might be `NA` if `read_parquet()` cannot read this column.
- **repetition_type** (string) - Indicates if the column is `REQUIRED` or `OPTIONAL`. `REPEATED` columns are not supported.
- **logical_type** (list) - Parquet logical type in a list column. Contains at least a `type` entry and potentially others like `bit_width`, `is_signed`.

### Request Example
```json
{
  "example": "request body"
}
```

### Response Example
```json
{
  "file_name": "example.parquet",
  "name": "column1",
  "type": "INT64",
  "r_type": "integer",
  "repetition_type": "REQUIRED",
  "logical_type": {"type": "INT"}
}
```
```

--------------------------------

### Read Parquet Metadata

Source: https://nanoparquet.r-lib.org/reference/read_parquet_metadata.html

Functions to retrieve metadata from a Parquet file, useful for inspecting files even when data reading is not supported.

```APIDOC
## read_parquet_metadata

### Description
Reads the metadata of a Parquet file. This function is designed to work on files even if `read_parquet()` is unable to read the data due to unsupported schemas, encodings, or compression.

### Arguments
- **file** (string) - Required - Path to a Parquet file.
- **options** (list) - Optional - Options that potentially alter the default Parquet to R type mappings, see `parquet_options()`.

## parquet_metadata

### Description
An alternative function to retrieve metadata from a Parquet file.

### Arguments
- **file** (string) - Required - Path to a Parquet file.
```

--------------------------------

### Define a Parquet schema

Source: https://nanoparquet.r-lib.org/reference/parquet_schema.html

Use this function to specify column types when writing data frames to Parquet files.

```R
parquet_schema(...)
```

```R
parquet_schema(
  c1 = "INT32",
  c2 = list("INT", bit_width = 64, is_signed = TRUE),
  c3 = list("STRING", repetition_type = "OPTIONAL")
)
#> # A data frame: 3 × 12
#>   file_name name  r_type type       type_length repetition_type
#> * <chr>     <chr> <chr>  <chr>            <int> <chr>          
#> 1 NA        c1    NA     INT32               NA NA             
#> 2 NA        c2    NA     INT64               NA NA             
#> 3 NA        c3    NA     BYTE_ARRAY          NA OPTIONAL       
#> # ℹ 6 more variables: converted_type <chr>, logical_type <I<list>>,
#> #   num_children <int>, scale <int>, precision <int>, field_id <int>
```

--------------------------------

### Append Data Frame to Parquet File

Source: https://nanoparquet.r-lib.org/reference/append_parquet.html

Appends a data frame to an existing Parquet file. The schema of the data frame must be compatible with the schema of the file. This operation is not atomic and may corrupt the file if interrupted.

```APIDOC
## APPEND PARQUET

### Description
Appends a data frame to an existing Parquet file. The schema of the data frame must be compatible with the schema of the file.

### Usage
```R
append_parquet(x, file, compression = c("snappy", "gzip", "zstd", "uncompressed"), encoding = NULL, row_groups = NULL, options = parquet_options())
```

### Arguments

*   **x** (data frame) - Data frame to append.
*   **file** (string) - Path to the output file.
*   **compression** (string) - Compression algorithm to use for the newly written data. See `write_parquet()`.
*   **encoding** (string) - Encoding to use for the newly written data. It does not have to be the same as the encoding of data in `file`. See `write_parquet()` for possible values.
*   **row_groups** (list) - Row groups of the new, extended Parquet file. `append_parquet()` can only change the last existing row group, and if `row_groups` is specified, it has respect this. (It is simpler to specify `num_rows_per_row_group` in `options`, see `parquet_options()` instead of `row_groups`. Only use `row_groups` if you need complete control.)
*   **options** (object) - Nanoparquet options, for the new data, see `parquet_options()`. The `keep_row_groups` option also affects whether `append_parquet()` overwrites existing row groups in `file`.

### Warning
This function is **not** atomic! If it is interrupted, it may leave the file in a corrupt state. To work around this create a copy of the original file, append the new data to the copy, and then rename the new, extended file to the original one.

### About row groups
A Parquet file may be partitioned into multiple row groups, and indeed most large Parquet files are. `append_parquet()` is only able to update the existing file along the row group boundaries. There are two possibilities:
  * `append_parquet()` keeps all existing row groups in `file`, and creates new row groups for the new data. This mode can be forced by the `keep_row_groups` option in `options`, see `parquet_options()`.
  * Alternatively, `write_parquet` will overwrite the _last_ row group in file, with its existing contents plus the (beginning of) the new data. This mode makes more sense if the last row group is small, because many small row groups are inefficient.

By default `append_parquet` chooses between the two modes automatically, aiming to create row groups with at least `num_rows_per_row_group` (see `parquet_options()`) rows. You can customize this behavior with the `keep_row_groups` options and the `row_groups` argument.

### See also
`write_parquet()`.
```

--------------------------------

### Read Parquet Files

Source: https://nanoparquet.r-lib.org/reference/index.html

Function to read data from Parquet files into an R data frame.

```APIDOC
## read_parquet()

### Description
Read a Parquet file into a data frame.

### Method
Not specified (likely a function call in R)

### Endpoint
N/A
```

--------------------------------

### Read Parquet File into R Data Frame

Source: https://nanoparquet.r-lib.org/reference/read_parquet.html

Use `read_parquet` to convert a Parquet file into an R data frame. Specify columns to read using `col_select` and customize options with `parquet_options()`.

```R
file_name <- system.file("extdata/userdata1.parquet", package = "nanoparquet")
parquet_df <- nanoparquet::read_parquet(file_name)
print(str(parquet_df))
```

--------------------------------

### Append Data Frame to Parquet File

Source: https://nanoparquet.r-lib.org/reference/append_parquet.html

Appends a data frame to an existing Parquet file. The schema must be compatible. This function is not atomic and may corrupt the file if interrupted.

```R
append_parquet(
  x,
  file,
  compression = c("snappy", "gzip", "zstd", "uncompressed"),
  encoding = NULL,
  row_groups = NULL,
  options = parquet_options()
)
```