### Install OvertureMaestro as Python Module Source: https://github.com/kraina-ai/overturemaestro/blob/main/README.md Use this command to install the OvertureMaestro library for use as a Python module. ```bash pip install overturemaestro ``` -------------------------------- ### Install OvertureMaestro with CLI Source: https://github.com/kraina-ai/overturemaestro/blob/main/README.md Use this command to install OvertureMaestro with its command-line interface (CLI) capabilities. ```bash pip install overturemaestro[cli] ``` -------------------------------- ### Import necessary libraries for OvertureMaestro Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Imports all required libraries for OvertureMaestro and related data processing tasks. Ensure these libraries are installed before use. ```python import itertools import time from pathlib import Path import duckdb import matplotlib.pyplot as plt import matplotlib.ticker as tkr import pandas as pd import psutil import pyarrow.compute as pc import pyarrow.dataset as ds import pyarrow.fs as fs import seaborn.objects as so from cpuinfo import get_cpu_info from overturemaps.core import geodataframe as om_gdf from psutil._common import bytes2human from pypalettes import load_cmap from seaborn import axes_style import overturemaestro as om from overturemaestro.release_index import download_existing_release_index ``` -------------------------------- ### Get Possible Column Names by Theme and Type Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Import and use `get_all_possible_column_names` to preview the final column list. Specify `theme` and `type` to filter the results. ```python from overturemaestro.advanced_functions.wide_form import get_all_possible_column_names get_all_possible_column_names(theme="base", type="water") ``` -------------------------------- ### Get CPU and System Information Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Retrieves detailed CPU information including brand, core count, and advertised frequency, along with total system RAM. Requires 'psutil' and 'get_cpu_info' libraries. ```python cpu_info = get_cpu_info() cpu_name = cpu_info["brand_raw"] cpu_cores = cpu_info["count"] cpu_freq = cpu_info["hz_advertised_friendly"] total_ram = bytes2human(psutil.virtual_memory().total) title_second_row = f"Run on {cpu_name} ({cpu_cores} cores) with {total_ram} total memory" cmap = load_cmap("thunder_city2") ``` -------------------------------- ### Benchmark examples dictionary Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb A dictionary mapping benchmark names to their corresponding Python functions. This is used to easily iterate and run different benchmarking methods. ```python BENCHMARK_EXAMPLES = { "OvertureMaps (PyArrow)": overturemaps_example, "DuckDB": duckdb_example, "DuckDB with file index": duckdb_with_index_example, "PyArrow with file index": pyarrow_index, "OvertureMaestro": overturemaestro_example, } ``` -------------------------------- ### List Available Theme/Type Pairs with Specific Release Version Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Get a list of all available theme and type combinations for a specified Overture Maps release version. This function may download release indexes if they are not cached. ```python om.get_available_theme_type_pairs(overture_maps_release) ``` -------------------------------- ### Save and Read Benchmark Results Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Saves benchmark results to a Parquet file and then reads them back. Ensure the 'pyarrow' library is installed for Parquet support. ```python pd.DataFrame(results).to_parquet(f"{bbox_type}_bboxes.parquet") results_df = pd.read_parquet(f"{bbox_type}_bboxes.parquet") ``` -------------------------------- ### Get Newest Release Version Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Obtain the newest available release version of Overture Maps data. ```python overture_maps_release = om.get_newest_release_version() ``` -------------------------------- ### Get All Available Release Versions Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Retrieve a list of all available release versions for Overture Maps data. This function downloads release indexes from the Overture Maps releases GitHub repository if not cached locally. ```python om.get_available_release_versions() ``` -------------------------------- ### Get All Possible Column Names with Maximal Depth Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Call `get_all_possible_column_names()` without arguments to retrieve a complete list of all possible columns with the maximum hierarchy depth. ```python columns = get_all_possible_column_names() len(columns) ``` -------------------------------- ### Get Newest Available Release Version Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Fetch the identifier for the most recent release version of Overture Maps data. ```python om.get_newest_release_version() ``` -------------------------------- ### Get Column Names with Specific Hierarchy Depth Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Specify the `hierarchy_depth` parameter when calling `get_all_possible_column_names` to control the level of detail in the returned column names. ```python get_all_possible_column_names(theme="buildings", type="building", hierarchy_depth=1) ``` -------------------------------- ### DuckDB query helper function Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb A helper function to query Parquet files using DuckDB, filtering by a bounding box. It selects a specified column and all other columns, excluding the specified one. Ensure DuckDB is installed and Parquet files are accessible. ```python def _from_duckdb(path_or_paths, column, bbox): q = f""" SET s3_region='us-west-2'; SELECT {column}, * EXCLUDE ({column}) FROM read_parquet({path_or_paths}) WHERE bbox.xmin < {bbox[2]} AND bbox.xmax > {bbox[0]} AND bbox.ymin < {bbox[3]} AND bbox.ymax > {bbox[1]}; """ rows = duckdb.sql(q) col = [r[0] for r in rows.fetchall()] return col ``` -------------------------------- ### Display OvertureMaestro Help Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Displays the full description of all arguments for the OvertureMaestro command using the --help or -h parameter. ```bash ! OvertureMaestro --help ``` -------------------------------- ### List Available Theme/Type Pairs (Implicit Release Version) Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Retrieve a list of all available theme and type combinations, implicitly using the newest release version if not specified. ```python om.get_available_theme_type_pairs() ``` -------------------------------- ### Import OvertureMaestro Library Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Import the OvertureMaestro library for use in your Python scripts. ```python import overturemaestro as om ``` -------------------------------- ### Run OvertureMaestro CLI Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Executes the OvertureMaestro CLI without any arguments. This will typically result in an error if a PBF file path is not provided. ```bash ! OvertureMaestro ``` -------------------------------- ### Display available theme type pairs Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Use the `--show-theme-type-pairs` flag to list all available theme type pairs in OvertureMaestro. ```bash ! OvertureMaestro --show-theme-type-pairs ``` -------------------------------- ### Execute OvertureMaestro with WKT Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Downloads building data filtered by a Well-Known Text (WKT) geometry definition. The operation is performed silently, and the results are saved to a Parquet file. ```bash ! OvertureMaestro buildings building --geom-filter-wkt 'POLYGON ((7.414 43.735, 7.414 43.732, 7.419 43.732, 7.419 43.735, 7.414 43.735))' \ --silent --output files/wkt_example.parquet ``` -------------------------------- ### Download existing release index Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Downloads the release index for a specific version. This is a prerequisite for some benchmarking functions that rely on indexed data. ```python download_existing_release_index("2024-09-18.0") ``` -------------------------------- ### Preview First 10 Columns Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb After retrieving all possible column names, you can preview a subset, such as the first 10, using standard Python slicing. ```python columns[:10] ``` -------------------------------- ### Execute OvertureMaestro with BBox Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Uses the OvertureMaestro command-line interface to download building data within a specified bounding box. The `--silent` flag suppresses progress output, and the results are saved to a Parquet file. ```bash ! OvertureMaestro buildings building --geom-filter-bbox 7.416486,43.730886,7.421931,43.733507 --silent --output files/bbox_example.parquet ``` -------------------------------- ### Render extracted rivers to ASCII Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Use the `pixel-map` tool to render the extracted Wroclaw river data into an ASCII black and white representation. ```bash ! pixel-map files/wroclaw_rivers_example.parquet --width 82 --height 23 --renderer ascii-bw ``` -------------------------------- ### Download Multiple Data Types in Wide Format Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Downloads data for specified multiple theme/type pairs (e.g., water and land cover) in the wide format and displays the top 10 most common features. ```python from overturemaestro.advanced_functions import ( convert_geometry_to_wide_form_geodataframe_for_all_types, convert_geometry_to_wide_form_geodataframe_for_multiple_types, ) two_datasets_gdf = convert_geometry_to_wide_form_geodataframe_for_multiple_types( [("base", "water"), ("base", "land_cover")], amsterdam ) two_datasets_gdf.drop(columns="geometry").sum().sort_values(ascending=False).head(10) ``` -------------------------------- ### Download All Datasets in Wide Format Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Downloads all available datasets in the wide format for a given area and displays the top 10 most common features, with an option to skip sorting for performance. ```python all_datasets_gdf = convert_geometry_to_wide_form_geodataframe_for_all_types( amsterdam, sort_result=False # we skip sorting the result here for faster execution ) all_datasets_gdf.drop(columns="geometry").sum().sort_values(ascending=False).head(10) ``` -------------------------------- ### Convert Geometry to Parquet (Sorted) Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Convert building geometry data for Paris to a Parquet file with sorting enabled. Sorting by geometry using Hilbert curve can reduce file size and improve spatial operation speed. ```python sorted_pq = om.convert_geometry_to_parquet( theme="buildings", type="building", geometry_filter=om.geocode_to_geometry("Paris"), sort_result=True, ignore_cache=True, ) ``` -------------------------------- ### Run benchmarking loop Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Iterates through different bounding box sizes, themes, types, and benchmark methods to execute and time the data retrieval functions. It measures elapsed time and network I/O (bytes sent/received) for each run. ```python for bbox_type, _bboxes in bboxes.items(): if Path(f"{bbox_type}_bboxes.parquet").exists(): print(bbox_type, "exists") continue results = [] for ov_theme, ov_type in theme_types: for bbox_name, bbox in _bboxes.items(): for benchmark_example_name, function in BENCHMARK_EXAMPLES.items(): print(ov_theme, ov_type, bbox_name, benchmark_example_name) net_io_start = psutil.net_io_counters() start_time = time.time() function(ov_theme, ov_type, bbox) elapsed_time = time.time() - start_time net_io_end = psutil.net_io_counters() bytes_sent = net_io_end.bytes_sent - net_io_start.bytes_sent bytes_recv = net_io_end.bytes_recv - net_io_start.bytes_recv print(f"Elapsed Time: {elapsed_time:.4f} seconds") print( f"Bytes Sent: {bytes2human(bytes_sent)}, Bytes Received: {bytes2human(bytes_recv)}" ) results.append( { "theme_type": f"{ov_theme}_{ov_type}", "bbox": bbox_name, ``` -------------------------------- ### Include All Possible Columns in Wide Form Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Use `include_all_possible_columns=True` to retain all potential columns, ensuring schema consistency across regions. This is the default behavior. ```python convert_geometry_to_wide_form_geodataframe( "base", "infrastructure", amsterdam, include_all_possible_columns=True # default value ) ``` -------------------------------- ### Execute OvertureMaestro with H3 Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Retrieves building data using an H3 spatial index as the geometry filter. The command suppresses progress output and saves the filtered data to a Parquet file. ```bash ! OvertureMaestro buildings building --geom-filter-index-h3 893969a4037ffff --silent --output files/h3_example.parquet ``` -------------------------------- ### Load and Convert Water Data to Wide Format Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Fetches the Overture Maps schema for water data and converts downloaded water features for a specified area into the wide format. ```python import requests import yaml response = requests.get( "https://raw.githubusercontent.com/OvertureMaps/schema/refs/tags/v1.4.0/schema/base/water.yaml", allow_redirects=True, ) water_schema = yaml.safe_load(response.content.decode("utf-8")) water_schema ``` ```python amsterdam = geocode_to_geometry("Amsterdam") original_data = convert_geometry_to_geodataframe("base", "water", amsterdam) wide_data = convert_geometry_to_wide_form_geodataframe("base", "water", amsterdam) ``` -------------------------------- ### Define small and big bounding boxes Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Sets up dictionaries containing bounding boxes for different geographical locations, categorized as 'small' and 'big' areas. These are used for testing data retrieval with varying spatial extents. ```python small_bboxes = { "London": (-0.120, 51.498, -0.090, 51.508), "Boston": (-71.068, 42.353, -71.058, 42.363), "Tokyo": (139.708, 35.643, 139.720, 35.650), } big_bboxes = { "London": (-0.521, 51.388, 0.200, 51.640), "Boston": (-71.104, 42.325, -71.002, 42.383), "Tokyo": (139.326, 35.552, 139.969, 35.806), } bboxes = { "small": small_bboxes, "big": big_bboxes, } ``` -------------------------------- ### Load Wroclaw museums with high confidence Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Extract museum data from Wroclaw within a bounding box, filtering by a high confidence score and accessing nested category fields. ```bash ! OvertureMaestro places place --filter "categories,primary = museum" --filter "confidence > 0.9" \ --geom-filter-bbox "17.010921,51.093406,17.054266,51.122229" \ --output files/wroclaw_museums_example.parquet ``` -------------------------------- ### Compare Sorted and Unsorted Parquet File Sizes Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Read and plot unsorted and sorted Parquet files, displaying their sizes and the percentage reduction achieved by sorting. ```python import geopandas as gpd from matplotlib import pyplot as plt fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(20, 10)) gpd.read_parquet(unsorted_pq).reset_index().reset_index().plot(column="index", ax=ax1, cmap="jet") gpd.read_parquet(sorted_pq).reset_index().reset_index().plot(column="index", ax=ax2, cmap="jet") unsorted_size = unsorted_pq.stat().st_size sorted_size = sorted_pq.stat().st_size ax1.set_title(f"Unsorted: {unsorted_size} bytes") ax2.set_title( f"Sorted: {sorted_size} bytes ({100 - (100 * sorted_size) / unsorted_size:.2f}% reduction)" ) plt.show() ``` -------------------------------- ### Inspect Theme Type Classifications Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Iterates through and prints the hierarchy columns defined for various theme types in OvertureMaestro's wide form classification. ```python from overturemaestro.advanced_functions.wide_form import THEME_TYPE_CLASSIFICATION for (theme_value, type_value), definition in sorted(THEME_TYPE_CLASSIFICATION.items()): print(theme_value, type_value, definition.hierachy_columns) ``` -------------------------------- ### Display Python Class Bases Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/class.html Renders the base classes of a Python class. Ensure 'config.show_bases' is true to display this information. ```python {% for expression in class.bases -%} `{% include "expression.html" with context %}`{% if not loop.last %}, {% endif %} {% endfor -%} ``` -------------------------------- ### Convert Geometry to Parquet (Unsorted) Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Convert building geometry data for Paris to a Parquet file without sorting the result. Sorting is disabled to potentially speed up acquisition at the cost of file size. ```python unsorted_pq = om.convert_geometry_to_parquet( theme="buildings", type="building", geometry_filter=om.geocode_to_geometry("Paris"), sort_result=False, ignore_cache=True, ) ``` -------------------------------- ### Extract Wroclaw rivers with PyArrow filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Extract river data for a specific bounding box in Wroclaw using a PyArrow filter and save to Parquet. ```bash ! OvertureMaestro base water --filter "subtype = river" \ --geom-filter-bbox "17.010921,51.093406,17.054266,51.122229" \ --output files/wroclaw_rivers_example.parquet ``` -------------------------------- ### Define OvertureMaps and OvertureMaestro data retrieval functions Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Provides functions to retrieve data using OvertureMaps' geodataframe module and OvertureMaestro's conversion utility. These functions are useful for direct data access and format conversion. ```python def overturemaps_example(theme, type, bbox) -> None: om_gdf(type, bbox=bbox) def overturemaestro_example(theme, type, bbox) -> None: om.convert_bounding_box_to_parquet( release="2024-09-18.0", theme=theme, type=type, bbox=bbox, ignore_cache=True, ) ``` -------------------------------- ### Format Bytes for Display Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb A helper function to format byte sizes into a human-readable string. It handles negative inputs by returning an empty string. ```python def _sizeof_fmt(x, pos): if x < 0: return "" return bytes2human(x) ``` -------------------------------- ### Generate bbox index function Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Imports and calls the main function from 'generate_bbox_index' to generate bounding box indices for specified themes and types. This is typically run once to prepare index files. ```python from generate_bbox_index import main as generate_bbox_index_fn for ov_theme, ov_type in theme_types: generate_bbox_index_fn(ov_theme, ov_type) ``` -------------------------------- ### Download Buildings with Geom Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Downloads building data within a specified bounding box using the OvertureMaestro CLI. The output is cached by default. ```bash ! OvertureMaestro buildings building --geom-filter-bbox 7.416486,43.730886,7.421931,43.733507 ``` -------------------------------- ### Display Python Class Docstring Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/class.html Includes the docstring for a Python class. If 'config.merge_init_into_class' is true, it also includes the __init__ method's docstring if present. ```python {% with docstring_sections = class.docstring.parsed %} {% include "docstring.html" with context %} {% endwith %} {% if config.merge_init_into_class %} {% if "__init__" in class.all_members and class.all_members["__init__"].has_docstring %} {% with docstring_sections = class.all_members["__init__"].docstring.parsed %} {% include "docstring.html" with context %} {% endwith %} {% endif %} {% endif %} ``` -------------------------------- ### Display Python Class Source Code Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/class.html Renders the source code of a Python class. If 'config.merge_init_into_class' is true, it displays the __init__ method's source code instead. Includes line numbers and highlights the code. ```python {% with init = class.all_members["__init__"] %} Source code in `{%- if init.relative_filepath.is_absolute() -%} {{ init.relative_package_filepath }} {%- else -%} {{ init.relative_filepath }} {%- endif -%}` {{ init.source|highlight(language="python", linestart=init.lineno, linenums=True) }} {% endwith %} ``` ```python Source code in `{%- if class.relative_filepath.is_absolute() -%} {{ class.relative_package_filepath }} {%- else -%} {{ class.relative_filepath }} {%- endif -%}` {{ class.source|highlight(language="python", linestart=class.lineno, linenums=True) }} ``` -------------------------------- ### Execute OvertureMaestro with Geocode Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Executes OvertureMaestro using a geocoding string to filter data for a specific location. Output is saved to a Parquet file, and progress messages are silenced. ```bash ! OvertureMaestro buildings building --geom-filter-geocode 'Monaco-Ville, Monaco' --silent --output files/geocode_example.parquet ``` -------------------------------- ### Query Wroclaw museums with DuckDB Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Use DuckDB to query the extracted Wroclaw museum data, selecting primary names and rounded confidence scores. ```bash ! ./duckdb :memory: "SELECT names['primary'], ROUND(confidence, 4) confidence FROM 'files/wroclaw_museums_example.parquet'"; ``` -------------------------------- ### DuckDB benchmarking functions Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Implements two functions for benchmarking data retrieval using DuckDB: one directly querying Parquet files and another utilizing a file index for potentially faster lookups. These functions return the count of matching IDs. ```python def duckdb_with_index_example(theme, type, bbox): filenames = _from_duckdb(f"'_index_{theme}_{type}.parquet'", "filename", bbox) duckdb_list = "[{}]".format(",".join([f"'s3://{f}'" for f in filenames])) ids = _from_duckdb(duckdb_list, "id", bbox) return len(ids) def duckdb_example(theme, type, bbox): ids = _from_duckdb( f"'s3://overturemaps-us-west-2/release/2024-09-18.0/theme={theme}/type={type}/*.parquet'", "id", bbox, ) return len(ids) ``` -------------------------------- ### Execute OvertureMaestro with S2 Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Filters Overture Maps data using an S2 spatial index. The command-line execution silences progress messages and directs the output to a specified Parquet file. ```bash ! OvertureMaestro buildings building --geom-filter-index-s2 12cdc28dc --silent --output files/s2_example.parquet ``` -------------------------------- ### Extract Wroclaw rail segments with PyArrow filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Extract rail transportation data for a specific bounding box in Wroclaw using a PyArrow filter and save to Parquet. ```bash ! OvertureMaestro transportation segment --filter "subtype = rail" \ --geom-filter-bbox "17.010921,51.093406,17.054266,51.122229" \ --output files/wroclaw_rail_example.parquet ``` -------------------------------- ### Convert Bounding Box to GeoDataFrame (Buildings) Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Retrieve building data within a specified bounding box as a GeoDataFrame. If no release version is specified, the newest available version is used. ```python buildings_gdf = om.convert_bounding_box_to_geodataframe( theme="buildings", type="building", bbox=london_bbox ) ``` ```python buildings_gdf ``` -------------------------------- ### Import Advanced Functions Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Import necessary functions for geometry conversion and geocoding, including the wide form specific geometry conversion. ```python from overturemaestro import convert_geometry_to_geodataframe, geocode_to_geometry from overturemaestro.advanced_functions import convert_geometry_to_wide_form_geodataframe ``` -------------------------------- ### PyArrow benchmarking function with file index Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Benchmarks data retrieval using PyArrow with a file index. It first retrieves filenames from an index file and then fetches IDs from the corresponding Parquet files using an S3 filesystem. Note: The index is currently at a non-public location. ```python def pyarrow_index(theme, type, bbox): # The index is currently behind a non-public location filenames = _from_pyarrow(f"_index_{theme}_{type}.parquet", "filename", bbox) ids = _from_pyarrow( filenames, "id", bbox, filesystem=fs.S3FileSystem(anonymous=True, region="us-west-2") ) return len(ids) ``` -------------------------------- ### Execute OvertureMaestro with GeoHash Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Downloads building data filtered by a GeoHash index. The `--silent` flag is used to suppress terminal output, and the results are stored in a Parquet file. ```bash ! OvertureMaestro buildings building --geom-filter-index-geohash spv2bcs --silent --output files/geohash_example.parquet ``` -------------------------------- ### Query Wroclaw rail data with DuckDB Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Use DuckDB to query the extracted Wroclaw rail data, limiting the output to one record. ```bash ! ./duckdb :memory: ".mode line" "FROM 'files/wroclaw_rail_example.parquet' LIMIT 1"; ``` -------------------------------- ### Set Output File Path Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Specifies the output file path for the generated GeoParquet file using the -o or --output option. ```bash ! OvertureMaestro buildings building --geom-filter-bbox 7.416486,43.730886,7.421931,43.733507 -o monaco_buildings.parquet ``` -------------------------------- ### Analyze Wide Format Data Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Demonstrates how to analyze the wide format data by summing boolean flags to count features per category, excluding the geometry column. ```python wide_data.drop(columns="geometry").sum().sort_values(ascending=False) ``` -------------------------------- ### Execute OvertureMaestro with GeoJSON Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Filters Overture Maps data using a GeoJSON geometry definition. The command-line arguments include the GeoJSON string and output file path, with silent mode enabled. ```bash ! OvertureMaestro buildings building \ --geom-filter-geojson '{\"type\":\"Feature\",\"geometry\":{\"coordinates\":[[[7.416,43.734],[7.416,43.731],[7.421,43.731],[7.421,43.734],[7.416,43.734]]],\"type\":\"Polygon\"}}' \ --silent --output files/geojson_example.parquet ``` -------------------------------- ### Apply PyArrow Filters for Places Data Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Downloads 'places' data with custom PyArrow filters to include only features with non-null categories and a minimal confidence level of 0.75. This prepares data for wide form conversion. ```python import pyarrow.compute as pc category_not_null_filter = pc.invert(pc.field("categories").is_null()) minimal_confidence_filter = pc.field("confidence") >= pc.scalar(0.75) combined_filter = category_not_null_filter & minimal_confidence_filter original_places_data = convert_geometry_to_geodataframe( "places", "place", amsterdam, pyarrow_filter=combined_filter, columns_to_download=["id", "geometry", "categories", "confidence"], ) original_places_data ``` -------------------------------- ### Extend Console Width Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Sets the COLUMNS environment variable to 160 to extend the default console width for better output formatting. ```python import os os.environ["COLUMNS"] = "160" ``` -------------------------------- ### Include Python Docstring Sections Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/function.html Includes parsed docstring sections for a Python function. This assumes a 'docstring.html' template exists to render the actual content. ```html {% with docstring_sections = function.docstring.parsed %} {% include "docstring.html" with context %} {% endwith %} ``` -------------------------------- ### Convert Bounding Box to GeoDataFrame (Roads) with Release Version Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Retrieve road segment data within a specified bounding box using an explicit release version. ```python roads_gdf = om.convert_bounding_box_to_geodataframe( release=overture_maps_release, theme="transportation", type="segment", bbox=london_bbox ) ``` ```python roads_gdf ``` -------------------------------- ### Import Geometry Parsers and GeoPandas Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Imports necessary classes for parsing different geometry formats and the GeoPandas library for geospatial data manipulation. ```python import geopandas as gpd from overturemaestro.cli import ( BboxGeometryParser, GeocodeGeometryParser, GeohashGeometryParser, GeoJsonGeometryParser, H3GeometryParser, S2GeometryParser, WktGeometryParser, ) ``` -------------------------------- ### Plotting Benchmark Results Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb Generates and displays benchmark results plots for different bounding box types. It customizes plot themes, facets, scales, and legends, and saves the plots to PNG files. Requires pandas, matplotlib, and altair_data_server. ```python cmap = load_cmap("OKeeffe") for bbox_type in bboxes.keys(): results_df = pd.read_parquet(f"{bbox_type}_bboxes.parquet") f = plt.figure(figsize=(15, 10)) sf1, sf2 = f.subfigures(2, 1) theme_dict = {**axes_style("whitegrid"), "grid.linestyle": ":"} ( so.Plot(results_df, x="bbox", y="time", color="benchmark") .theme(theme_dict) .facet(col="theme_type") .add(so.Bar(), so.Agg(), so.Dodge()) # .scale(color="colorblind") .scale(color=cmap.colors) .on(sf1) .label(color=str.capitalize) .plot() ) ( so.Plot(results_df, x="bbox", y="bytes_recv", color="benchmark") .theme(theme_dict) .facet(col="theme_type") .add(so.Bar(), so.Agg(), so.Dodge()) # .scale(color="colorblind") .scale(color=cmap.colors) .on(sf2) .label(color=str.capitalize) .plot() ) l2 = f.legends.pop(1) l1 = f.legends.pop(0) f.legend( l2.legend_handles, [t.get_text() for t in l1.texts], loc="center right", bbox_to_anchor=(1.06, 0.5), ) sf2.axes[0].yaxis.set_major_formatter(tkr.FuncFormatter(_sizeof_fmt)) sf1.axes[0].set_ylabel("Time (s)\n(lower is better)") sf2.axes[0].set_ylabel("Data downloaded\n(lower is better)") for sf in [sf1, sf2]: for _ax in sf.axes: _ax.set_xlabel("Bounding Box") title_first_row = f"Benchmark - {bbox_type.capitalize()} bounding boxes (2024-09-18.0 release)" plt.suptitle(f"{title_first_row}\n{title_second_row}", y=1.01) # # add arrows # fig_arrow( # head_position=(0.16, 0.65), # tail_position=(0.12, 0.71), # width=2, # radius=0.3, # color="darkred", # fill_head=False, # mutation_scale=2, # ) # # add arrows plt.savefig(f"benchmark_bbox_{bbox_type.lower()}.png", bbox_inches="tight") plt.show() ``` -------------------------------- ### Convert Bounding Box to GeoDataFrame (Water) with PyArrow Filter Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Retrieve water data within a specified bounding box, applying a PyArrow filter to acquire only river subtypes. ```python water_gdf = om.convert_bounding_box_to_geodataframe( theme="base", type="water", bbox=london_bbox, pyarrow_filter=[("subtype", "=", "river")] ) ``` ```python water_gdf ``` -------------------------------- ### Inspect Geo Data with Pixel Map Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Inspects the generated GeoParquet file using the pixel-map tool, displaying geo data in the terminal with specified dimensions and renderer. ```bash ! pixel-map monaco_buildings.parquet --width 82 --height 23 --renderer ascii-bw ``` -------------------------------- ### Plot London features Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Visualize extracted building, place, water, and road data for the City of London using matplotlib. ```python import matplotlib.pyplot as plt from overturemaestro import geocode_to_geometry geometry_filter = gpd.GeoSeries([geocode_to_geometry("City of London")], crs=4326) bounds = geometry_filter.total_bounds geometry_types = ["water", "roads", "buildings", "places"] colors = ["#118AB2", "#073B4C", "#06D6A0", "#FFD166"] fig, axs = plt.subplot_mosaic( """ aa aa bp rw """, figsize=(10, 12), layout="constrained", ) main_ax = axs["a"] main_ax.set_title("City of London") for geometry_type, color in zip(geometry_types, colors): filename = f"files/london_{{geometry_type}}_example.parquet" gdf = gpd.read_parquet(filename, columns=["geometry"]) for ax in (main_ax, axs[geometry_type[0]]): gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.8, color=color) for geometry_type in geometry_types: axs[geometry_type[0]].set_title(geometry_type.capitalize()) for key, ax in axs.items(): ax.set_xlim([bounds[0] - 0.001, bounds[2] + 0.001]) ax.set_ylim([bounds[1] - 0.001, bounds[3] + 0.001]) if key == "a": continue geometry_filter.plot( ax=ax, color=(0, 0, 0, 0), zorder=2, edgecolor="#EF476F", linewidth=1.5, ) ax.set_axis_off() ``` -------------------------------- ### Prune Columns in Wide Form Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Set `include_all_possible_columns=False` to include only columns derived from features that actually exist in the data, reducing the overall column count. ```python convert_geometry_to_wide_form_geodataframe( "base", "infrastructure", amsterdam, include_all_possible_columns=False ) ``` -------------------------------- ### Define theme-type pairs Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb A list of tuples, where each tuple represents a theme and its corresponding data type for Overture Maps. This is used to iterate through different data categories during benchmarking. ```python theme_types = [ ("buildings", "building"), ("transportation", "segment"), ("places", "place"), ("base", "water"), ] ``` -------------------------------- ### Format Python Class Signature Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/class.html Formats the signature of a Python class, optionally merging the __init__ method's signature. Useful for displaying class definitions. ```python {{ class_name|highlight(language="python", linenums=False) }} ``` -------------------------------- ### Display Python Function Source Code Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/function.html Renders the source code of a Python function, including line numbers and syntax highlighting. It determines the relative file path for display. ```html {{ lang.t("Source code in") }} `{%- if function.relative_filepath.is_absolute() -%} {{ function.relative_package_filepath }} {%- else -%} {{ function.relative_filepath }} {%- endif -%}` {{ function.source|highlight(language="python", linestart=function.lineno, linenums=True) }} ``` -------------------------------- ### Find First Feature with Alternate Categories Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Identifies the index and categories of the first feature in the 'places' dataset that has at least one alternate category. Useful for inspecting specific feature structures. ```python first_index = ( # Find first object with at least one alternate category original_places_data[original_places_data.categories.str.get("alternate").str.len() > 1] .iloc[0] .name ) first_index, original_places_data.loc[first_index].categories ``` -------------------------------- ### Conditional Heading Rendering for Python Functions Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/function.html Conditionally renders the heading for a Python function based on configuration. It can display the full path or just the name and includes options for separate signatures. ```html {% with obj = function, html_id = function.path %} {% if root %} {% set show_full_path = config.show_root_full_path %} {% set root_members = True %} {% elif root_members %} {% set show_full_path = config.show_root_members_full_path or config.show_object_full_path %} {% set root_members = False %} {% else %} {% set show_full_path = config.show_object_full_path %} {% endif %} {% set function_name = function.path if show_full_path else function.name %} {% set heading_classes = "doc doc-heading" if config.separate_signature else "doc doc-heading-code" %} {% if not root or config.show_root_heading %} {% filter heading(heading_level, role="function", id=html_id, class=heading_classes, toc_label=function.name ~ "()") %} {% block heading scoped %} {% if config.separate_signature %} {{ function_name }} {% else %} {%+ filter format_signature(function, config.line_length, crossrefs=config.signature_crossrefs) %} {{ function_name }} {% endfilter %} {% endif %} {% endblock heading %} {% endfilter %} {% block labels scoped %} {% with labels = function.labels %} {% include "labels.html" with context %} {% endwith %} {% endblock labels %} {% block signature scoped %} {% if config.separate_signature %} {% filter format_signature(function, config.line_length, crossrefs=config.signature_crossrefs) %} {{ function.name }} {% endfilter %} {% endif %} {% endblock signature %} {% else %} {% if config.show_root_toc_entry %} {% filter heading(heading_level, role="function", id=html_id, toc_label=function.name, hidden=True) %} {% endfilter %} {% endif %} {% set heading_level = heading_level - 1 %} {% endif %} ``` -------------------------------- ### Force Regeneration of GeoParquet File Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Forces the regeneration of the GeoParquet file by using the --ignore-cache flag, bypassing the default caching mechanism. ```bash ! OvertureMaestro buildings building --geom-filter-bbox 7.416486,43.730886,7.421931,43.733507 --ignore-cache ``` -------------------------------- ### Plotting GeoDataFrames Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Visualize buildings, roads, and water data from different GeoDataFrames on a single plot with custom styling and axis settings. ```python ax = buildings_gdf.plot(color="#f7cda9", zorder=3) roads_gdf.plot(ax=ax, color="#cc7578", zorder=1) water_gdf.plot(ax=ax, color="#97b5bf", zorder=2) ax.set_xlim([london_bbox[0], london_bbox[2]]) ax.set_ylim([london_bbox[1], london_bbox[3]]) ax.set_axis_off() ``` -------------------------------- ### Format Python Function Signature with Class Name Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/class.html Formats the signature of a Python function, prepending the class name. This is used when merging the __init__ method's signature into the class definition. ```python {% with function = class.all_members["__init__"] %} {%+ filter format_signature(function, config.line_length, crossrefs=config.signature_crossrefs) %} {{ class_name }} {% endfilter %} {% endwith %} ``` -------------------------------- ### Extract London water features Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Extract water feature data for the City of London using a geocode filter and save it to a Parquet file. ```bash ! OvertureMaestro base water --geom-filter-geocode "City of London" --silent --output files/london_water_example.parquet ``` -------------------------------- ### Limit Hierarchy Depth to Zero for All Types Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts geometry data for all types to wide form with a hierarchy depth of 0. This results in a list of theme/type pairs. ```python limited_depth_all_gdf = convert_geometry_to_wide_form_geodataframe_for_all_types( amsterdam, hierarchy_depth=0 ) limited_depth_all_gdf.drop(columns="geometry").sum() ``` -------------------------------- ### Plot and Compare Filtered Results Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Reads Parquet files generated by different geometry filters and plots them using Matplotlib and GeoPandas. This visualization allows for a direct comparison of the data retrieved by each filter type. ```python import matplotlib.patches as mpatches import matplotlib.pyplot as plt fig, axs = plt.subplots(2, 4, sharex=True, sharey=True, figsize=(10, 5)) for idx, (geometry_type, geometry) in enumerate(zip(geometry_types, geometries)): ax = axs[idx // 4, idx % 4] gdf = gpd.read_parquet(f"files/{geometry_type.lower()}_example.parquet") gdf.plot(ax=ax, markersize=1, zorder=1, alpha=0.8) gdf.boundary.plot(ax=ax, markersize=0, zorder=1, alpha=0.8) gpd.GeoSeries([geometry], crs=4326).plot( ax=ax, color=(0, 0, 0, 0), zorder=2, hatch="///", edgecolor="orange", linewidth=1.5, ) ax.set_title(geometry_type) axs[1, 3].set_axis_off() blue_patch = mpatches.Patch(color="C0", alpha=0.8, label="OSM features") orange_patch = mpatches.Patch( facecolor=(0, 0, 0, 0), edgecolor="orange", hatch="///", linewidth=1.5, label="Geometry filter" ) fig.legend(handles=[blue_patch, orange_patch], bbox_to_anchor=(0.98, 0.35)) fig.tight_layout() ``` -------------------------------- ### Convert Places Data to Wide Form Using Primary Category Only Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts 'places' geometry data to wide form, using only the primary category for each feature. This simplifies the representation by excluding alternative categories. ```python primary_only_wide_form_places_data = convert_geometry_to_wide_form_geodataframe( "places", "place", amsterdam, places_use_primary_category_only=True, ) primary_only_wide_form_places_data.loc[first_index].drop("geometry").sort_values(ascending=False) ``` -------------------------------- ### Inspect Wide Form Places Data for a Specific Feature Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Displays the wide form attributes for a specific feature, sorted by value in descending order. This helps visualize which categories are present for that feature. ```python wide_form_places_data.loc[first_index].drop("geometry").sort_values(ascending=False) ``` -------------------------------- ### Limit Hierarchy Depth for Multiple Datasets Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts geometry data for multiple theme/type pairs to wide form, allowing individual hierarchy depth limits for each pair. The list of depths must match the number of theme/type pairs. ```python limited_depth_multiple_gdf = convert_geometry_to_wide_form_geodataframe_for_multiple_types( [("places", "place"), ("base", "land_cover"), ("base", "water")], amsterdam, hierarchy_depth=[1, None, 0], ) limited_depth_multiple_gdf.drop(columns="geometry").sum() ``` -------------------------------- ### Parse and Visualize Geometries Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Parses various geometry strings into GeoDataFrames and visualizes them on a map using GeoPandas. This helps in understanding the spatial extent of each filter. ```python geometry_types = ["BBox", "Geocode", "GeoJSON", "WKT", "H3", "GeoHash", "S2"] geometries = [ BboxGeometryParser().convert(bbox_string), GeocodeGeometryParser().convert(geocode_string), GeoJsonGeometryParser().convert(geojson_string), WktGeometryParser().convert(wkt_string), H3GeometryParser().convert(h3_string), GeohashGeometryParser().convert(geohash_string), S2GeometryParser().convert(s2_string), ] gpd.GeoDataFrame( data=dict(type=geometry_types), geometry=geometries, crs=4326, ).explore(column="type", tiles="CartoDB positron") ``` -------------------------------- ### Convert Places Data to Wide Form Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts 'places' geometry data to wide form, preserving primary and alternative categories. Features with categories will have 'True' values in corresponding columns. ```python wide_form_places_data = convert_geometry_to_wide_form_geodataframe("places", "place", amsterdam) wide_form_places_data ``` -------------------------------- ### Extract London buildings Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Extract building data for the City of London using a geocode filter and save it to a Parquet file. ```bash ! OvertureMaestro buildings building --geom-filter-geocode "City of London" --silent --output files/london_buildings_example.parquet ``` -------------------------------- ### PyArrow data retrieval helper function Source: https://github.com/kraina-ai/overturemaestro/blob/main/dev/benchmark_methods.ipynb A helper function to retrieve data from Parquet files using PyArrow, applying a bounding box filter. It can optionally use a specified filesystem, such as S3. ```python def _from_pyarrow(path_or_paths, column, bbox, filesystem=None): # Fetch a single column from the path or paths and apply the bbox filter dataset = ds.dataset(path_or_paths, filesystem=filesystem) batches = dataset.to_batches(filter=_filter(bbox)) return list(itertools.chain(*(b[column].to_pylist() for b in batches))) ``` -------------------------------- ### Limit Hierarchy Depth for Places Data to 1 Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts 'places' geometry data to wide form, limiting the hierarchy depth to 1. This results in fewer columns, representing broader category groupings. ```python convert_geometry_to_wide_form_geodataframe("places", "place", amsterdam, hierarchy_depth=1) ``` -------------------------------- ### Set Minimal Confidence for Places Data Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts 'places' geometry data to wide form, setting a higher minimal confidence threshold of 0.95. This filters out features with lower confidence scores. ```python convert_geometry_to_wide_form_geodataframe( "places", "place", amsterdam, places_minimal_confidence=0.95 ) ``` -------------------------------- ### Extract London places Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/command_line_interface.ipynb Extract place data for the City of London using a geocode filter and save it to a Parquet file. ```bash ! OvertureMaestro places place --geom-filter-geocode "City of London" --silent --output files/london_places_example.parquet ``` -------------------------------- ### Format Python Function Signature Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/class.html Formats the signature of a Python function, including cross-references if configured. Used when merging __init__ into the class signature. ```python {% with function = class.all_members["__init__"] %} {%+ filter format_signature(function, config.line_length, crossrefs=config.signature_crossrefs) %} {{ class.name }} {% endfilter %} {% endwith %} ``` -------------------------------- ### Define Bounding Box for London Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/basic_usage.ipynb Define a bounding box tuple representing the geographical area of interest for data retrieval. ```python london_bbox = (-0.120077, 51.498164, -0.090809, 51.508849) ``` -------------------------------- ### Format Python Function Signature Source: https://github.com/kraina-ai/overturemaestro/blob/main/docs/templates/python/material/function.html Formats a Python function's signature for display, respecting line length and cross-reference options. Used when signatures are displayed separately from the heading. ```html {% filter format_signature(function, config.line_length, crossrefs=config.signature_crossrefs) %} {{ function.name }} {% endfilter %} ``` -------------------------------- ### Limit Hierarchy Depth for Water Data Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Converts geometry data to wide form, limiting the hierarchy depth to 1 for water features. Useful for higher-level aggregations. ```python limited_depth_water_gdf = convert_geometry_to_wide_form_geodataframe( "base", "water", amsterdam, hierarchy_depth=1 ) limited_depth_water_gdf.drop(columns="geometry").sum() ``` -------------------------------- ### Sum of True Values in Primary Only Wide Form Places Data Source: https://github.com/kraina-ai/overturemaestro/blob/main/examples/advanced_functions/wide_form.ipynb Calculates and displays the sum of 'True' values across all columns for 'places' data converted using only the primary category. This shows category counts when alternatives are ignored. ```python primary_only_wide_form_places_data.drop(columns="geometry").sum().sort_values(ascending=False) ```