Pyrosm

https://pyrosm.readthedocs.io/en/stable/
Admin
Pyrosm is a Python library for reading OpenStreetMap data from PBF files into Geopandas...
Tokens:68,782
Snippets:648
Trust Score:9.6
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install Black

Source: https://pyrosm.readthedocs.io/en/stable/_sources/contributions.rst

Install the Black formatter via pip.

```bash
pip install black
```

--------------------------------

### List downloads output

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Example output from listing downloaded files.

```text
['/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/Helsinki.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/Paris.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/estonia-latest.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/finland-latest.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/malta-latest.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/newyorkcity.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/south-america-latest.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/spain-latest.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/tallinn-estonia.osm.pbf',
 '/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/us-west-latest.osm.pbf']
```

--------------------------------

### Install Benchmark Dependencies

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Commands to install Python, Java, and R dependencies required for the benchmark suite.

```bash
# Python tools (conda-forge recommended for the geo-stack)
# ipywidgets is needed so QuackOSM/DuckDB can manage their progress bar inside Jupyter.
mamba install -c conda-forge pyrosm osmnx quackosm pyosmium osmium-tool geopandas matplotlib pandas ipywidgets

# Osmosis (Java command-line tool) + a Java runtime
mamba install -c conda-forge openjdk
# then download Osmosis from https://github.com/openstreetmap/osmosis/releases
# and make sure the `osmosis` launcher is on your PATH.

# osmextract is an R package (run via the companion osmextract_benchmark_scaling.ipynb).
# Install R + sf (GDAL) + the Jupyter R kernel from conda-forge, then osmextract from CRAN:
mamba install -c conda-forge r-base r-sf r-irkernel
R -e 'install.packages("osmextract", repos="https://cloud.r-project.org")'
R -e 'IRkernel::installspec()'   # registers the "R" Jupyter kernel
```

--------------------------------

### Install Pyrosm

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

The recommended way to install the library is via pip.

```bash
pip install pyrosm
```

--------------------------------

### Download and crop output

Source: https://pyrosm.readthedocs.io/en/stable/downloading_data.html

Example output showing the file paths for cropped and uncropped downloads.

```text
Geofabrik extract covering the area: 'England' (id: england)
Cropped filename: /var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/bbox_-0.245_50.798_-0.016_50.892.osm.pbf
Geofabrik extract covering the area: 'England' (id: england)
No cropping: /var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/england-latest.osm.pbf
```

--------------------------------

### Example benchmark output logs

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/osmextract_benchmark_scaling.html

Sample console output showing the detection of the input file and the version information for the packages used.

```text
The input place was matched with: Helsinki

The chosen file was already detected in the download directory. Skip downloading.
```

```text
osmextract 0.6.0 | sf 1.1.0 
PBF: /Users/tenkanh2/Library/Application Support/org.R-project.R/R/osmextract/bbbike_Helsinki.osm.pbf
```

--------------------------------

### Example benchmark output

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Displays the console output showing the performance results for the PBF cropping task.

```text
Task 3 - crop Finland to the Helsinki region (buildings = count read back):

  pyrosm      98.6 s     59.6 MB   208,632 buildings
  pyosmium   226.3 s     59.6 MB   208,632 buildings
  osmium-tool   14.8 s     59.7 MB   208,632 buildings
```

--------------------------------

### Setup and Benchmark Utilities for OSMExtract

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/osmextract_benchmark_scaling.html

Initializes the environment, defines PBF resolution logic, and provides functions to measure peak memory and execution time for osmextract tasks.

```R
# Buildings setup: read the manifest, define the per-area benchmark (time + peak memory, measured
# in a separate process like the Python tools), and accumulate results so each area can be run --
# and re-run -- in its own cell below.
areas <- read.csv("scaling_areas.csv", stringsAsFactors = FALSE)
worker <- normalizePath("bench_worker.R", mustWork = FALSE)
if (!file.exists(worker)) stop("bench_worker.R must sit next to this notebook")
time_flag <- if (Sys.info()[["sysname"]] == "Darwin") "-l" else "-v"
time_bin <- if (file.exists("/usr/bin/time")) "/usr/bin/time" else "time"
rscript <- file.path(R.home("bin"), "Rscript")
if (!file.exists(rscript)) rscript <- Sys.which("Rscript")
if (!nzchar(rscript) || !file.exists(rscript)) stop("Could not find Rscript for this R installation")


find_python <- function() {
    candidates <- c(Sys.which("python"), file.path(dirname(dirname(R.home())), "bin", "python"))
    candidates <- candidates[nzchar(candidates) & file.exists(candidates)]
    if (length(candidates) == 0) return(NA_character_)
    candidates[1]
}


rebuild_kamppi_pbf <- function() {
    out <- file.path(tempdir(), "kamppi.osm.pbf")
    if (file.exists(out)) return(normalizePath(out))
    if (!exists("pbf") || !file.exists(pbf)) {
        stop("Kamppi PBF is missing and the Helsinki PBF from oe_get() is not available; re-run the first cell")
    }
    python <- find_python()
    if (is.na(python)) {
        stop("Kamppi PBF is missing and Python was not found; re-run the setup cell in benchmarks_scaling.ipynb")
    }
    script <- paste(
        "from pyrosm import OSM",
        "import sys",
        "OSM(sys.argv[1], bounding_box=[24.920, 60.162, 24.942, 60.172]).to_pbf(sys.argv[2])",
        sep = "\n"
    )
    log <- suppressWarnings(system2(python, c("-c", script, pbf, out), stdout = TRUE, stderr = TRUE))
    if (!file.exists(out)) {
        detail <- paste(tail(log[nzchar(log)], 8), collapse = " | ")
        stop(sprintf("Could not rebuild Kamppi PBF: %s", detail))
    }
    out
}


resolve_area_pbf <- function(area_label, pbf_path) {
    if (file.exists(pbf_path)) return(normalizePath(pbf_path))
    if (identical(area_label, "Kamppi")) return(rebuild_kamppi_pbf())
    stop(sprintf("PBF for '%s' not found: %s. Re-run the setup cell in benchmarks_scaling.ipynb.",
                 area_label, pbf_path))
}


parse_peak_rss_mb <- function(output_lines) {
    text <- paste(output_lines, collapse = "\n")
    # macOS `/usr/bin/time -l`: "<bytes>  maximum resident set size"
    m <- regmatches(text, regexpr("[0-9]+ +maximum resident set size", text))
    if (length(m) == 1) return(as.numeric(sub(" .*", "", m)) / 1e6)         # bytes -> MB
    # GNU `/usr/bin/time -v`: "Maximum resident set size (kbytes): <kb>"
    m <- regmatches(text, regexpr("Maximum resident set size \\(kbytes\\): [0-9]+", text))
    if (length(m) == 1) return(as.numeric(sub(".*: ", "", m)) / 1e3)        # kB -> MB
    NA_real_
}


bench_oe <- function(worker_task, csv_task, area, pbf, repeats) {
    runtimes <- numeric(0); peaks <- numeric(0); n <- NA_integer_; status <- "ok"
    for (i in seq_len(repeats)) {
        out <- tryCatch(suppressWarnings(system2(time_bin, c(time_flag, rscript, worker, worker_task, pbf),
                                                stdout = TRUE, stderr = TRUE)),
                        error = function(e) {
                            status <<- paste("error:", conditionMessage(e)); character(0)
                        })
        if (length(out) == 0) break
        data_line <- grep("^[0-9.]+,[0-9]+$", out, value = TRUE)  # "seconds,features" from the worker
        if (length(data_line) == 0) {
            detail <- paste(tail(out[nzchar(out)], 8), collapse = " | ")
            status <- if (nzchar(detail)) paste("error:", detail) else "error: no worker output"
            break
        }
        parts <- as.numeric(strsplit(data_line[length(data_line)], ",")[[1]])
        runtimes <- c(runtimes, parts[1]); n <- as.integer(parts[2])
        peaks <- c(peaks, parse_peak_rss_mb(out))
    }
    if (length(runtimes) == 0)
        return(data.frame(task = csv_task, area = area, tool = "osmextract",
                          seconds = NA_real_, peak_mb = NA_real_, features = NA_integer_,
                          status = status, stringsAsFactors = FALSE))
    data.frame(task = csv_task, area = area, tool = "osmextract",
               seconds = round(median(runtimes), 2), peak_mb = round(median(peaks), 1),
               features = n, status = "ok", stringsAsFactors = FALSE)
}
```

--------------------------------

### Benchmark Output Log

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Example output showing file paths and sizes for the defined benchmark areas.

```text
Region PBF : /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/Helsinki.osm.pbf (62.9 MB)
Finland PBF: /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/finland-latest.osm.pbf (725.8 MB)

Buildings area ladder:
  Kamppi           /var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/kamppi.osm.pbf  (    6.1 MB)
  Helsinki region  /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/Helsinki.osm.pbf  (   62.9 MB)
  New York City    /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/newyorkcity.osm.pbf  (  151.6 MB)
  Paris            /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/Paris.osm.pbf  (  238.4 MB)
  Finland          /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/finland-latest.osm.pbf  (  725.8 MB)
  Spain            /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/spain-latest.osm.pbf  ( 1456.1 MB)
  South America    /private/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/south-america-latest.osm.pbf  ( 4027.3 MB)
```

--------------------------------

### Benchmark Output Example

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Sample output showing performance metrics for the Kamppi area.

```text
Kamppi (6 MB, up to 10x, timeout 120s):
  pyrosm (in-memory)         0.95 s        190 MB       4,717 lines
  pyrosm (out-of-core)       0.92 s        205 MB       4,717 lines
  quackosm (with columns)    13.84 s       1173 MB       4,711 lines
  quackosm (no columns)     13.93 s        538 MB       4,711 lines
  pyosmium                   0.74 s        169 MB       4,771 lines
  osmium-tool                1.00 s       1541 MB       4,717 lines
  osmnx                     26.11 s        481 MB      13,112 lines
```

--------------------------------

### Install pyrosm via conda-forge

Source: https://pyrosm.readthedocs.io/en/stable/_sources/faq.md

Recommended installation command using mamba or conda.

```bash
mamba install -c conda-forge pyrosm
```

--------------------------------

### Benchmark results output

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/osmextract_benchmark_scaling.html

Example output format showing task, area, tool, execution time, peak memory, feature count, and status.

```text
   task   area       tool seconds peak_mb features status
1 roads Kamppi osmextract    0.07   350.9     4716     ok
```

```text
   task            area       tool seconds peak_mb features status
1 roads Helsinki region osmextract    3.14  1022.3   296845     ok
```

```text
   task          area       tool seconds peak_mb features status
1 roads New York City osmextract    6.65  1584.9   669037     ok
```

```text
   task  area       tool seconds peak_mb features status
1 roads Paris osmextract    7.34  1503.3   620227     ok
```

```text
   task    area       tool seconds peak_mb features status
1 roads Finland osmextract   32.57  4657.9  2178539     ok
```

```text
   task  area       tool seconds peak_mb features status
1 roads Spain osmextract  114.19 10010.6  5716619     ok
```

--------------------------------

### Parallel Cropping Benchmark Output

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Example console output showing the performance results of the parallel cropping tasks.

```text
Task 3 in parallel - more workers / threads:

  pyrosm, 10 workers (fast-mode)   24.9 s     83.7 MB   208,632 buildings
  pyrosm, 10 workers (compact)   64.5 s     59.6 MB   208,632 buildings
  pyosmium, 10 workers    228.0 s     59.6 MB   208,632 buildings
  osmium-tool, 10 threads    2.9 s     59.7 MB   208,632 buildings
```

--------------------------------

### Example output of PBF export

Source: https://pyrosm.readthedocs.io/en/stable/saving_and_cropping.html

Displays the console output showing file sizes and layer counts after performing the export operations.

```text
whole dataset   54.4 MiB
buildings only  12.8 MiB
combined: 176900 buildings, 255130 network edges
```

--------------------------------

### Benchmark output for Kamppi

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Example output showing performance metrics for the Kamppi area.

```text
Kamppi (6 MB, up to 10x, timeout 120s):
  pyrosm (in-memory)         0.72 s        179 MB         665 polys
  pyrosm (out-of-core)       0.83 s        177 MB         665 polys
  quackosm (with columns)    10.81 s        705 MB         663 polys
  quackosm (no columns)     10.25 s        525 MB         663 polys
  pyosmium                   0.53 s        144 MB         663 polys
  osmium-tool                0.65 s       1203 MB         619 polys
  osmnx                      4.38 s        307 MB         666 polys
```

--------------------------------

### Install optional dependency for out-of-core reading

Source: https://pyrosm.readthedocs.io/en/stable/_sources/installation.ipynb

Required for caching layer results to GeoParquet files when using the out-of-core engine.

```bash
$ pip install pyarrow
```

--------------------------------

### Install optional dependencies for graph export

Source: https://pyrosm.readthedocs.io/en/stable/_sources/installation.ipynb

Required for exporting street networks to graphs using specific backends.

```bash
$ pip install igraph networkx pandarm
```

--------------------------------

### Setup benchmark worker and area manifest

Source: https://pyrosm.readthedocs.io/en/stable/_sources/benchmarks/osmextract_benchmark_scaling.ipynb

Loads the area manifest and configures the external worker process for measuring execution time and memory usage.

```r
# Buildings setup: read the manifest, define the per-area benchmark (time + peak memory, measured
# in a separate process like the Python tools), and accumulate results so each area can be run --
# and re-run -- in its own cell below.
areas <- read.csv("scaling_areas.csv", stringsAsFactors = FALSE)
worker <- normalizePath("bench_worker.R", mustWork = FALSE)
if (!file.exists(worker)) stop("bench_worker.R must sit next to this notebook")
time_flag <- if (Sys.info()[["sysname"]] == "Darwin") "-l" else "-v"
time_bin <- if (file.exists("/usr/bin/time")) "/usr/bin/time" else "time"
rscript <- file.path(R.home("bin"), "Rscript")
if (!file.exists(rscript)) rscript <- Sys.which("Rscript")
if (!nzchar(rscript) || !file.exists(rscript)) stop("Could not find Rscript for this R installation")


find_python <- function() {
    candidates <- c(Sys.which("python"), file.path(dirname(dirname(R.home())), "bin", "python"))
    candidates <- candidates[nzchar(candidates) & file.exists(candidates)]
    if (length(candidates) == 0) return(NA_character_)
    candidates[1]
}


rebuild_kamppi_pbf <- function() {

```

--------------------------------

### Initialize osmextract benchmark environment

Source: https://pyrosm.readthedocs.io/en/stable/_sources/benchmarks/osmextract_benchmark_scaling.ipynb

Configures the R environment, sets download timeouts, and retrieves the Helsinki PBF file for benchmarking.

```r
# Same input as the Python benchmark (get_data("helsinki")): the latest Helsinki extract from
# BBBike. osmextract fetches it itself via oe_get(provider = "bbbike") -- one of its core
# capabilities. download_only + skip_vectortranslate just download the .osm.pbf and return its
# path; it is used for the version banner and as a fallback source for clipping Kamppi.
suppressPackageStartupMessages(library(osmextract))
REPEATS <- 3L
options(timeout = max(600, getOption("timeout")))  # the ~62 MB download can exceed R's 60 s default

pbf <- oe_get("Helsinki", provider = "bbbike",
              download_only = TRUE, skip_vectortranslate = TRUE, quiet = FALSE)

cat("osmextract", as.character(packageVersion("osmextract")),
    "| sf", as.character(packageVersion("sf")), "\nPBF:", pbf, "\n")
```

--------------------------------

### Install pyrosm via package managers

Source: https://pyrosm.readthedocs.io/en/stable/_sources/installation.ipynb

Use these commands to install pyrosm from the conda-forge channel or via pip.

```bash
$ mamba install -c conda-forge pyrosm
```

```bash
$ micromamba install -c conda-forge pyrosm
```

```bash
$ pip install pyrosm
```

--------------------------------

### Graph Simplification Output

Source: https://pyrosm.readthedocs.io/en/stable/graphs.html

Example output showing the reduction in graph complexity.

```text
Full:       768 nodes, 1490 edges
Simplified: 268 nodes, 577 edges
```

--------------------------------

### Build documentation locally

Source: https://pyrosm.readthedocs.io/en/stable/_sources/contributions.rst

Generate HTML documentation from the docs folder using Sphinx.

```bash
make html
```

--------------------------------

### Initialize OSM reader with history file

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Download a sample history file and initialize the OSM reader. The reader automatically detects the .osh.pbf format.

```python
from pyrosm import OSM, get_data

# Download a small sample history file for Helsinki
fp = get_data("helsinki_history_pbf")

# Initialise the reader; pyrosm detects the .osh.pbf file as a history file
osm = OSM(fp)
```

--------------------------------

### Get boundary geometry

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Retrieve the boundary of a specific area to use as a spatial filter.

```python
# Get the borough of Camden as our bounding box
bounding_box = osm.get_boundaries(name="London Borough of Camden")
bounding_box.plot()
```

--------------------------------

### Initialize Benchmark Environment

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Imports and configuration constants used to manage benchmark execution and output readability.

```python
import os
import time
import shutil
import statistics
import subprocess
import tempfile
import warnings
from collections import namedtuple
from pathlib import Path

import pandas as pd
import geopandas as gpd
import matplotlib.pyplot as plt

warnings.filterwarnings("ignore")  # keep the benchmark output readable

# Median window for the single-area road task. The buildings task sets its own per-area
# repeats via the AREAS ladder below (small areas 10x; the big country files 3x).
REPEATS = 3
# Cropping reads/writes a multi-hundred-MB file; once is enough.
CROP_REPEATS = 1
```

--------------------------------

### Geofabrik extract URL output

Source: https://pyrosm.readthedocs.io/en/stable/downloading_data.html

Example output showing the URL of the identified Geofabrik extract.

```text
'https://download.geofabrik.de/europe/united-kingdom/england-latest.osm.pbf'
```

--------------------------------

### Perform Editable Build

Source: https://pyrosm.readthedocs.io/en/stable/_sources/contributions.rst

Command to install pyrosm in editable mode, compiling Cython extensions in-place.

```bash
pip install -e . --no-build-isolation
```

--------------------------------

### Initialize OSM reader with test data

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Retrieve a test PBF dataset path using get_data and initialize the OSM reader instance.

```python
import pyrosm

# Get filepath to test PBF dataset
fp = pyrosm.get_data("test_pbf")
print("Filepath to test data:", fp)

# Initialize the OSM object 
osm = pyrosm.OSM(fp)

# See the type
print("Type of 'osm' instance: ", type(osm))
```

--------------------------------

### Access to_graph documentation

Source: https://pyrosm.readthedocs.io/en/stable/graphs.html

Displays help documentation for the to_graph method.

```python
from pyrosm import OSM, get_data
osm = OSM(get_data("test_pbf"))

# To see all available parameters and their explanations, simply call help
help(osm.to_graph)
```

--------------------------------

### Filter Tags to Keep

Source: https://pyrosm.readthedocs.io/en/stable/_sources/tags_and_columns.ipynb

Limit the columns created from OSM tags by specifying a subset in the get method.

```python
from pyrosm import OSM, get_data

osm = OSM(get_data("test_pbf"))
buildings = osm.get_buildings(tags_to_keep=["building", "addr:city"])
buildings.columns.tolist()
```

--------------------------------

### Initialize OSM with out-of-core engine

Source: https://pyrosm.readthedocs.io/en/stable/api/pyrosm.OSM.html

Demonstrates initializing the OSM reader with the out-of-core engine and parallel workers, requiring a main guard on certain operating systems.

```python
if __name__ == "__main__":
    osm = OSM(fp, engine="out_of_core", workers="auto")
    buildings = osm.get_buildings()
```

--------------------------------

### Create Development Environment

Source: https://pyrosm.readthedocs.io/en/stable/_sources/contributions.rst

Commands to create a development environment using mamba or micromamba from provided configuration files.

```bash
mamba env create -f ci/314-conda.yaml
```

```bash
micromamba create -f ci/314-conda.yaml
```

--------------------------------

### Initialize OSM reader with dataset

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Download and initialize the OSM reader for a specific region.

```python
from pyrosm import OSM, get_data

# Download a dataset for Greater London (update if exists in the temp already)
fp = get_data("Greater London", update=True)
osm = OSM(fp)
```

--------------------------------

### Geofabrik extract lookup output

Source: https://pyrosm.readthedocs.io/en/stable/downloading_data.html

Example output showing the identified Geofabrik extract for a given bounding box.

```text
Geofabrik extract covering the area: 'England' (id: england)
```

--------------------------------

### Initialize OSM and Retrieve Network Data

Source: https://pyrosm.readthedocs.io/en/stable/graphs.html

Load Helsinki region data and extract the driving network and restaurant POIs.

```python
from pyrosm import OSM, get_data
import numpy as np
import matplotlib.pyplot as plt
osm = OSM(get_data("helsinki"))
nodes, edges = osm.get_network(network_type="driving", nodes=True)
restaurants = osm.get_pois(custom_filter={"amenity": ["restaurant"]})
```

--------------------------------

### Activate Development Environment

Source: https://pyrosm.readthedocs.io/en/stable/_sources/contributions.rst

Command to activate the created development environment.

```bash
mamba activate test
```

--------------------------------

### Compact and repack PBF output

Source: https://pyrosm.readthedocs.io/en/stable/_sources/saving_and_cropping.ipynb

Demonstrates using compact and repack flags to reduce the output file size at the cost of processing speed.

```python
default = OSM(fp, bounding_box=bbox).to_pbf()
compact = OSM(fp, bounding_box=bbox).to_pbf(compact=True)
repack = OSM(fp, bounding_box=bbox).to_pbf(repack=True)

for label, path in [("default", default), ("compact", compact), ("repack", repack)]:
    size_mb = os.path.getsize(path) / (1024 * 1024)
    print(f"{label:8s} {size_mb:.1f} MiB")
```

--------------------------------

### Initialize and Run OSMExtract Benchmarks

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/osmextract_benchmark_scaling.html

Initializes the results data frame and defines helper functions to execute and persist benchmark results for specific areas.

```R
if (!exists("oe_scaling_results")) oe_scaling_results <- data.frame()


# Upsert one (task, area) result into oe_scaling_results and write the CSV immediately -- so each
# tool/area is exported as it finishes, and re-running an area replaces just that row.
run_oe_for <- function(worker_task, csv_task, area_label) {
    row <- areas[areas$area == area_label, ]
    if (nrow(row) == 0) stop(sprintf("area '%s' not in scaling_areas.csv", area_label))
    pbf_path <- resolve_area_pbf(row$area[1], row$path[1])
    result <- bench_oe(worker_task, csv_task, row$area[1], pbf_path, row$repeats[1])
    keep <- !(oe_scaling_results$task == csv_task & oe_scaling_results$area == area_label)
    oe_scaling_results <<- rbind(oe_scaling_results[keep, ], result)
    write.csv(oe_scaling_results, "osmextract_scaling_results.csv", row.names = FALSE)
    print(result)
}


run_oe_buildings <- function(area_label) run_oe_for("buildings", "buildings", area_label)
run_oe_network   <- function(area_label) run_oe_for("network", "roads", area_label)
```

--------------------------------

### Initialize Pyrosm with an OSM file

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

The primary entry point for using the library is creating an OSM object with a path to a PBF file.

```python
from pyrosm import OSM

# Initialize the OSM object with the path to your PBF file
fp = "path/to/your/data.osm.pbf"
osm = OSM(fp)
```

--------------------------------

### Download and crop OSM data by bounding box

Source: https://pyrosm.readthedocs.io/en/stable/downloading_data.html

Demonstrates downloading data with default cropping enabled and disabling cropping to keep the full extract.

```python
# Download the covering extract and crop it to the box
fp = get_data_by_bbox([-0.245, 50.798, -0.016, 50.892])
print("Cropped filename:", fp)

# crop=False keeps the full extract instead
fp = get_data_by_bbox([-0.245, 50.798, -0.016, 50.892], crop=False)
print("No cropping:", fp)
```

--------------------------------

### Configure and execute building parsing benchmarks

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Defines the reader configurations and the execution logic for benchmarking tools against PBF files of varying sizes.

```python
SCALE_READERS = [
    # label,                    worker tool,                     only on these areas (None = all)
    ("pyrosm (in-memory)",      "pyrosm-in-memory",            {"Kamppi", "Helsinki region", "New York City", "Paris", "Finland"}),
    ("pyrosm (out-of-core)",    "pyrosm-out-of-core",          None),
    ("quackosm (with columns)", "quackosm-with-columns",       None),
    ("quackosm (no columns)",   "quackosm-no-columns",         None),
    ("pyosmium",                "pyosmium",                    {"Kamppi", "Helsinki region", "New York City", "Paris"}),
    ("osmium-tool",             "osmium-tool",                 {"Kamppi", "Helsinki region", "New York City", "Paris", "Finland"}),
    ("osmnx",                   "osmnx",                       {"Kamppi", "Helsinki region", "New York City", "Paris"}),
]
SMALL_AREAS = {"Kamppi", "Helsinki region"}
AREA = {area.label: area for area in AREAS}

# OSMnx queries Overpass for a bounding box rather than reading the file; use the same extent as
# the local data (Kamppi's clip box; Helsinki's data extent, read once and cached).
_OSMNX_BBOX = {"Kamppi": KAMPPI_BBOX}


def _osmnx_bbox(area):
    if area.label not in _OSMNX_BBOX:
        _OSMNX_BBOX[area.label] = list(OSM(area.pbf).get_buildings().total_bounds)
    return _OSMNX_BBOX[area.label]


def run_buildings_for_area(area):
    """Benchmark every applicable tool on one area and record the rows. Re-running replaces this
    area's earlier buildings rows, so each area cell can be run on its own."""
    print(f"{area.label} ({os.path.getsize(area.pbf)/1e6:.0f} MB, up to {area.repeats}x, "
          f"timeout {area.timeout}s):")
    for label, worker_tool, only_areas in SCALE_READERS:
        if only_areas is not None and area.label not in only_areas:
            continue
        extra_args = _osmnx_bbox(area) if worker_tool == "osmnx" else None
        if worker_tool == "osmnx":
            reps = 1                                     # one live Overpass query is enough
        elif worker_tool in ["pyosmium", "osmium-tool"] and area.label not in SMALL_AREAS:
            reps = 1                                     # cap the slow streaming reader on big files
        else:
            reps = area.repeats
        runs = []
        for _ in range(reps):
            run = monitor_parse(worker_tool, area.pbf, area.timeout, extra_args=extra_args)
            runs.append(run)
            if run["status"] != "ok":
                break   # don't repeat an OOM / timeout / crash
        ok_runs = [r for r in runs if r["status"] == "ok"]
        if ok_runs:
            row = {"task": "buildings", "area": area.label, "tool": label,
                   "seconds": round(statistics.median(r["seconds"] for r in ok_runs), 2),
                   "peak_mb": round(statistics.median(r["peak_mb"] for r in ok_runs), 1),
                   "features": ok_runs[-1]["features"], "status": "ok"}
            print(f"  {label:22} {row['seconds']:8.2f} s   {row['peak_mb']:8.0f} MB   "
                  f"{row['features']:>9,} polys")
        else:
            row = {"task": "buildings", "area": area.label, "tool": label, "seconds": None,
                   "peak_mb": max(r["peak_mb"] for r in runs), "features": None,
                   "status": runs[-1]["status"]}
            print(f"  {label:22} {runs[-1]['status']:>22}   {row['peak_mb']:8.0f} MB peak")
        record_result(row)
```

--------------------------------

### OSM Class Constructor

Source: https://pyrosm.readthedocs.io/en/stable/_sources/reference.rst

The OSM class is the main entry point for the library. It is initialized with a path to an OpenStreetMap PBF file.

```APIDOC
## OSM(filepath)

### Description
Initializes the OSM reader with a PBF file.

### Parameters
- **filepath** (str) - Required - Path to the PBF file.
```

--------------------------------

### Import math library

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Imports the standard math library.

```python
import math
```

--------------------------------

### Display benchmark output

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Prints a confirmation message for the loaded osmextract data.

```text
Loaded 12 osmextract row(s) from osmextract_scaling_results.csv
```

--------------------------------

### Initialize Pyrosm and extract network

Source: https://pyrosm.readthedocs.io/en/stable/graphs.html

Load OSM data and retrieve walkable roads as nodes and edges.

```python
from pyrosm import OSM, get_data
import osmnx as ox

# Initialize the reader
osm = OSM(get_data("helsinki_pbf"))

# Get all walkable roads and the nodes 
nodes, edges = osm.get_network(nodes=True)

# Check first rows in the edge 
edges.head()
```

--------------------------------

### Initialize the OSM reader object

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Import the pyrosm library and access the help documentation for the OSM reader initialization.

```python
# Import the library
import pyrosm

# Print information about the basic usage of the `OSM` reader object
help(pyrosm.OSM.__init__)
```

--------------------------------

### OSM Class Constructor

Source: https://pyrosm.readthedocs.io/en/stable/reference.html

Initializes the OpenStreetMap PBF reader object.

```APIDOC
## OSM(filepath, bounding_box=None, keep_metadata=False, ...)

### Description
Constructs an OSM reader object from a PBF file.

### Parameters
- **filepath** (str) - Required - Path to the PBF file.
- **bounding_box** (list/tuple) - Optional - Bounding box to filter data.
- **keep_metadata** (bool) - Optional - Whether to keep OSM metadata.
```

--------------------------------

### Initialize OSM reader and inspect columns

Source: https://pyrosm.readthedocs.io/en/stable/_sources/tags_and_columns.ipynb

Initializes the OSM reader with test data and displays existing columns and unique tag attributes.

```python
from pyrosm import OSM, get_data
# Get test data 
fp = get_data("test_pbf")

# Initialize the reader
osm = OSM(fp)
            
buildings = osm.get_buildings()

# Print info
print("Existing columns:\n", buildings.columns)
print("\nAdditional attributes in the 'tags': \n", buildings.tags.unique())
```

--------------------------------

### Initialize OSM with bounding box

Source: https://pyrosm.readthedocs.io/en/stable/reading_osm_data.html

Apply a geometric filter to the OSM reader instance.

```python
# Get the shapely geometry from GeoDataFrame
bbox_geom = bounding_box['geometry'].values[0]

# Initiliaze with bounding box
osm = OSM(fp, bounding_box=bbox_geom)
```

--------------------------------

### Download Data with Flexible Naming

Source: https://pyrosm.readthedocs.io/en/stable/downloading_data.html

Demonstrates that the library can resolve variations in place name formatting.

```python
# Passing names in slightly different style does not matter
fp = get_data("Rio de Janeiro")
print(fp)
```

```text
Downloaded Protobuf data 'RiodeJaneiro.osm.pbf' (35.3 MB) to:
'/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/RiodeJaneiro.osm.pbf'
/var/folders/f2/pgp09jl542zffhtrt2hx8zhh0000gp/T/pyrosm/RiodeJaneiro.osm.pbf
```

--------------------------------

### Read OSM data from a file

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

Initialize the OSM reader with a path to an OSM PBF file.

```python
from pyrosm import OSM

# Initialize the OSM reader
# Replace 'path/to/your/file.osm.pbf' with the actual path to your file
osm = OSM("path/to/your/file.osm.pbf")
```

--------------------------------

### Fork and Clone the Repository

Source: https://pyrosm.readthedocs.io/en/stable/_sources/contributions.rst

Commands to clone the repository and link it to the upstream project.

```bash
git clone git@github.com:your-user-name/pyrosm.git pyrosm-yourname
cd pyrosm-yourname
git remote add upstream https://github.com/pyrosm/pyrosm.git
```

--------------------------------

### Initialize Pyrosm Reader

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

Initialize the OSM reader with a path to a PBF file.

```python
from pyrosm import OSM

# Initialize the reader
osm = OSM("path/to/your/data.osm.pbf")
```

--------------------------------

### Read OSM data from a file

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

Initialize the OSM reader with a path to an OpenStreetMap PBF file.

```python
from pyrosm import OSM

# Initialize the OSM object with the path to the PBF file
osm = OSM("path/to/your/data.osm.pbf")
```

--------------------------------

### Configure visualization styling

Source: https://pyrosm.readthedocs.io/en/stable/benchmarks/benchmarks_scaling.html

Sets up matplotlib parameters for consistent chart styling and defines color constants for tool highlighting.

```python
# ---- Styling for the result charts ----
plt.rcParams.update({
    "figure.facecolor": "white", "savefig.facecolor": "white",
    "savefig.dpi": 200, "savefig.bbox": "tight",
    "axes.edgecolor": "#888888", "axes.linewidth": 0.8,
    "axes.spines.top": False, "axes.spines.right": False,
    "axes.grid": True, "axes.axisbelow": True,
    "grid.color": "#ECECF1", "grid.linewidth": 1.0,
    "axes.titlesize": 13, "axes.titleweight": "bold",
    "axes.labelsize": 11, "axes.labelcolor": "#333333",
    "xtick.color": "#333333", "ytick.color": "#333333",
})

# pyrosm is highlighted; every other tool shares one muted colour.
PYROSM_COLOR = "#2A849D"
OTHER_COLOR = "#BC778F"

def bar_colors(tools):
    return [PYROSM_COLOR if tool == "pyrosm" else OTHER_COLOR for tool in tools]

# Result figures are written here (relative to this notebook).
FIGURE_DIR = "figures"
os.makedirs(FIGURE_DIR, exist_ok=True)
```

--------------------------------

### Run osmextract for specific area

Source: https://pyrosm.readthedocs.io/en/stable/_sources/benchmarks/osmextract_benchmark_scaling.ipynb

Retrieves area configuration and prepares for benchmark execution.

```R
# Upsert one (task, area) result into oe_scaling_results and write the CSV immediately -- so each
# tool/area is exported as it finishes, and re-running an area replaces just that row.
run_oe_for <- function(worker_task, csv_task, area_label) {
    row <- areas[areas$area == area_label, ]
    if (nrow(row) == 0) stop(sprintf("area '%s' not in scaling_areas.csv", area_label))
    pbf_path <- resolve_area_pbf(row$area[1], row$path[1])
```

--------------------------------

### OSM.list_downloads()

Source: https://pyrosm.readthedocs.io/en/stable/api/pyrosm.OSM.list_downloads.html

Lists the PBF files downloaded by get_data in the default download directory.

```APIDOC
## OSM.list_downloads()

### Description
List the PBF files downloaded by `get_data` in the default download directory (`<tempdir>/pyrosm`).

### Returns
- **list of str**: The downloaded files’ paths.
```

--------------------------------

### Initialize OSM reader and extract driving network

Source: https://pyrosm.readthedocs.io/en/stable/quickstart.html

Initializes the OSM reader with a file path and retrieves the driving network. Use the out-of-core engine for large datasets to manage memory usage.

```python
%matplotlib inline
osm = OSM(fp)
drive = osm.get_network("driving")
drive.head(3)
```

--------------------------------

### Initialize Pyrosm with OSM data

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

Load an OSM PBF file into a Pyrosm object to begin processing.

```python
from pyrosm import OSM

# Initialize the OSM object with the path to your PBF file
osm = OSM("path/to/your/data.osm.pbf")
```

--------------------------------

### Execute osmextract benchmark

Source: https://pyrosm.readthedocs.io/en/stable/_sources/benchmarks/osmextract_benchmark_scaling.ipynb

Runs a benchmark task multiple times and returns a data frame containing median runtime, peak memory, and feature count.

```R
bench_oe <- function(worker_task, csv_task, area, pbf, repeats) {
    runtimes <- numeric(0); peaks <- numeric(0); n <- NA_integer_; status <- "ok"
    for (i in seq_len(repeats)) {
        out <- tryCatch(suppressWarnings(system2(time_bin, c(time_flag, rscript, worker, worker_task, pbf),
                                                stdout = TRUE, stderr = TRUE)),
                        error = function(e) {
                            status <<- paste("error:", conditionMessage(e)); character(0)
                        })
        if (length(out) == 0) break
        data_line <- grep("^[0-9.]+,[0-9]+$", out, value = TRUE)  # "seconds,features" from the worker
        if (length(data_line) == 0) {
            detail <- paste(tail(out[nzchar(out)], 8), collapse = " | ")
            status <- if (nzchar(detail)) paste("error:", detail) else "error: no worker output"
            break
        }
        parts <- as.numeric(strsplit(data_line[length(data_line)], ",")[[1]])
        runtimes <- c(runtimes, parts[1]); n <- as.integer(parts[2])
        peaks <- c(peaks, parse_peak_rss_mb(out))
    }
    if (length(runtimes) == 0)
        return(data.frame(task = csv_task, area = area, tool = "osmextract",
                          seconds = NA_real_, peak_mb = NA_real_, features = NA_integer_,
                          status = status, stringsAsFactors = FALSE))
    data.frame(task = csv_task, area = area, tool = "osmextract",
               seconds = round(median(runtimes), 2), peak_mb = round(median(peaks), 1),
               features = n, status = "ok", stringsAsFactors = FALSE)
}
```

--------------------------------

### Read OSM data from a file

Source: https://pyrosm.readthedocs.io/en/stable/_sources/quickstart.ipynb

Initialize the OSM object by pointing to a PBF file.

```python
from pyrosm import OSM

# Initialize the OSM object
# Provide the path to the PBF file
fp = "path/to/your/data.osm.pbf"
osm = OSM(fp)
```

--------------------------------

### Apply a custom filter to OSM data

Source: https://pyrosm.readthedocs.io/en/stable/custom_filter.html

Demonstrates initializing the OSM reader and applying a custom filter to extract specific building types.

```python
from pyrosm import OSM, get_data
# Get test data 
fp = get_data("test_pbf")

# Initialize the reader
osm = OSM(fp)

# Read buildings with custom filter
my_filter = {"building": ["residential", "retail"]}
buildings = osm.get_buildings(custom_filter=my_filter)

# Plot
title = "Filtered buildings: " + ", ".join(buildings["building"].unique())
ax = buildings.plot(column="building", cmap="RdBu", legend=True)
ax.set_title(title);
```

--------------------------------

### OSM.to_pbf()

Source: https://pyrosm.readthedocs.io/en/stable/api/pyrosm.OSM.to_pbf.html

Crops the source PBF file by the object's bounding box and writes a valid, re-readable *.osm.pbf file to disk.

```APIDOC
## OSM.to_pbf(output_path=None, keep_relations=True, workers=1, compact=False, repack=False)

### Description
Crop the source PBF by this object’s bounding_box and write a valid, re-readable *.osm.pbf to disk. The process streams the source file blob-by-blob to maintain memory efficiency.

### Parameters
- **output_path** (str, optional) - Where to write the cropped PBF. If None, a temporary file is created.
- **keep_relations** (bool) - Whether to include relations referencing kept nodes or ways. Defaults to True.
- **workers** (int) - Number of worker processes for parallel processing. Defaults to 1.
- **compact** (bool) - Whether to prune the string table in each output block to reduce file size. Defaults to False.
- **repack** (bool) - Whether to re-chunk elements into canonical, densely packed blocks. Defaults to False.

### Returns
- **str** - The path of the written PBF file.
```
Tokens:
Raw