### Install from Source Source: https://ndpyramid.readthedocs.io/en/latest/_sources/install.md Clone the repository and perform an editable installation. ```python git clone https://github.com/carbonplan/ndpyramid cd ndpyramid python -m pip install -e . ``` -------------------------------- ### Install Optional Dependencies Source: https://ndpyramid.readthedocs.io/en/latest/_sources/install.md Install specific extras for regridding or resampling functionality, or install all available dependencies. ```shell python -m pip install "ndpyramid[xesmf]" # Install optional dependencies for regridding with ESMF python -m pip install "ndpyramid[dask]" # Install optional dependencies for resampling with pyresample and Dask python -m pip install "ndpyramid[complete]" # Install all optional dependencies ``` -------------------------------- ### Install via Pip Source: https://ndpyramid.readthedocs.io/en/latest/_sources/install.md Standard installation using the pip package manager. ```shell python -m pip install ndpyramid ``` -------------------------------- ### Install via Conda Source: https://ndpyramid.readthedocs.io/en/latest/_sources/install.md Use the conda-forge channel to install the package. ```shell conda install ndpyramid --channel conda-forge ``` -------------------------------- ### Resample Pyramid Example Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Demonstrates how to resample a pyramid structure. ```python import xarray as xr from ndpyramid import resample_pyramid # Assuming 'ds' is an existing pyramid dataset resampled_ds = resample_pyramid(ds, target_grid='some_grid') ``` -------------------------------- ### Resampling a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Example demonstrating how to resample a pyramid structure. ```python import xarray as xr from ndpyramid import resample_pyramid ds = xr.open_dataset('data.nc') resampled = resample_pyramid(ds, target_resolution=0.5) ``` -------------------------------- ### Pyramid Reprojection Example Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Basic implementation of pyramid reprojection. ```python import numpy as np from ndpyramid import pyramid_reproject # Example usage of pyramid_reproject # Assuming 'data' is an xarray DataArray reprojected = pyramid_reproject(data, target_grid) ``` -------------------------------- ### Basic Regridding Example Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-regrid.ipynb Demonstrates the fundamental usage of the regrid function within the pyramid-regrid library. ```python import numpy as np from pyramid_regrid import regrid # Define source and target grids source_grid = np.array([[0, 1], [1, 0]]) target_grid = np.array([[0, 0.5], [0.5, 0]]) # Perform regridding result = regrid(source_grid, target_grid) print(result) ``` -------------------------------- ### Basic Regridding Example Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-regrid.ipynb Demonstrates the fundamental usage of the pyramid-regrid library for regridding spatial data. ```python import numpy as np from pyramid_regrid import regrid # Example usage of regrid function data = np.random.rand(10, 10) regridded_data = regrid(data, target_shape=(20, 20)) ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Example showing how to reproject a pyramid structure. ```python from ndpyramid import reproject # Reproject the pyramid reprojected_pyramid = reproject(pyramid, target_crs='EPSG:4326') ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Example demonstrating how to perform a reprojection on a pyramid structure. ```python from ndpyramid import reproject # Example usage of reproject reproject(data, target_crs) ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Example demonstrating how to reproject a pyramid dataset to a new coordinate reference system. ```python import xarray as xr from ndpyramid import reproject # Load your dataset ds = xr.open_dataset('path/to/your/dataset.zarr') # Reproject the pyramid reprojected_ds = reproject(ds, target_crs='EPSG:3857') ``` -------------------------------- ### Pyramid Regridding Example Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-regrid.ipynb Basic usage of the pyramid regridding functionality. ```python import numpy as np from ndpyramid import pyramid_regrid # Example usage of pyramid_regrid data = np.random.rand(100, 100) regridded = pyramid_regrid(data, scale_factor=2) ``` -------------------------------- ### Configure storage and pyramid parameters Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Sets up constants for pyramid levels and tiles, and initializes storage backends for different data dimensions. ```python VERSION = 2 LEVELS = 6 PIXELS_PER_TILE = 128 S3 = False input_path = f"s3://carbonplan-maps/v{VERSION}/demo/raw" if S3: base = f"s3://carbonplan-maps/v{VERSION}/demo/" store_2d = base + "2d/tavg" store_3d = base + "3d/tavg-prec" store_3d_1var = base + "3d/tavg-month" store_4d = base + "4d/tavg-prec-month" else: import zarr store_2d = zarr.storage.MemoryStore() store_3d = zarr.storage.MemoryStore() store_3d_1var = zarr.storage.MemoryStore() store_4d = zarr.storage.MemoryStore() ``` -------------------------------- ### Create a Pyramid Project Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-create.ipynb Use this command to initialize a new project structure for a Pyramid application. ```bash pcreate -s starter myproject ``` -------------------------------- ### Load input dataset Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-create.html Loads the air_temperature tutorial dataset from xarray. ```python ds = xr.tutorial.load_dataset("air_temperature") ``` -------------------------------- ### Import required libraries Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-create.html Initial imports for dask, xarray, and the pyramid_create function. ```python import dask # noqa import xarray as xr from ndpyramid import pyramid_create ``` -------------------------------- ### Opening and preparing input dataset Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-resample.html Loads a chunked Zarr dataset and assigns a coordinate reference system for processing. ```python ds = xr.open_zarr( "https://rice1.osn.mghpcc.org/carbonplan/cp-cmip/version1/rechunked_data/GARD-SV/ScenarioMIP.CCCma.CanESM5.ssp245.r1i1p1f1.month.GARD-SV.tasmax.zarr" ) ds = ds.rio.write_crs("EPSG:4326") ds ``` -------------------------------- ### Import required libraries Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Initial imports for data handling and pyramid generation. ```python import matplotlib.pyplot as plt import pandas as pd import xarray as xr from ndpyramid import pyramid_reproject ``` -------------------------------- ### Configuring output storage Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-resample.html Sets up the destination for the generated pyramid, supporting either S3 object storage or an in-memory store. ```python S3 = False if S3: output = "s3://carbonplan-scratch/pyramid_comparison/resampled.zarr" else: import zarr output = zarr.storage.MemoryStore() ``` -------------------------------- ### Opening and preparing the input dataset Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-reproject.html Loads a CMIP6 dataset from a Zarr source and assigns a coordinate reference system using rioxarray. ```python # Open the dataset ds = xr.open_zarr( "https://rice1.osn.mghpcc.org/carbonplan/cp-cmip/version1/rechunked_data/GARD-SV/ScenarioMIP.CCCma.CanESM5.ssp245.r1i1p1f1.month.GARD-SV.tasmax.zarr" ) ds = ds.rio.write_crs("EPSG:4326") ds ``` -------------------------------- ### Configure output storage Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-create.html Sets the output destination to either an S3 bucket or an in-memory zarr store. ```python S3 = False if S3: output = "s3://carbonplan-scratch/pyramid_comparison/custom.zarr" else: import zarr output = zarr.storage.MemoryStore() ``` -------------------------------- ### Importing necessary modules Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-regrid.html Initial imports required for working with dask, xarray, and the pyramid_regrid function. ```python import dask # noqa import xarray as xr from ndpyramid.regrid import pyramid_regrid ``` -------------------------------- ### Create a 3D Zarr pyramid from single-variable GeoTIFFs Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Opens 12 monthly temperature GeoTIFFs, concatenates them along a month dimension, and writes the resulting pyramid to Zarr. ```python %%time # open and extract the input datasets ds_all = [] months = list(map(lambda d: d + 1, range(12))) for i in months: path = f"{input_path}/wc2.1_2.5m_tavg_{i:02g}.tif" ds = ( xr.open_dataarray(path, engine="rasterio") .to_dataset(name="tavg") .squeeze() .reset_coords(["band"], drop=True) ) ds_all.append(ds) ds = xr.concat(ds_all, pd.Index(months, name="month")) # create the pyramid dt = pyramid_reproject(ds, levels=LEVELS, other_chunks={"month": 12}, clear_attrs=True) # write the pyramid to zarr dt.to_zarr(store_3d_1var, zarr_format=2, consolidated=True) ``` -------------------------------- ### Resampling an image using a pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Demonstrates the basic workflow for creating a pyramid and resampling an image to a target size. ```python from ndpyramid import pyramid_resample # Example usage of pyramid resampling resampled_image = pyramid_resample(image, target_shape=(256, 256)) ``` -------------------------------- ### Resample an image using a pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Demonstrates how to use a pyramid to resample an image to a target size. ```python from ndpyramid import pyramid_resample # Resample an image to a target shape resampled_image = pyramid_resample(image, target_shape=(256, 256)) ``` -------------------------------- ### Create a 2D GeoTIFF pyramid Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Opens a single 2D GeoTIFF image, converts it to a dataset, and writes it as a Zarr pyramid. ```python %%time # input dataset path = f"{input_path}/wc2.1_2.5m_tavg_10.tif" # open and extract the input dataset ds = ( xr.open_dataarray(path, engine="rasterio") .to_dataset(name="tavg") .squeeze() .reset_coords(["band"], drop=True) ) # create the pyramid dt = pyramid_reproject(ds, levels=LEVELS, clear_attrs=True) # write the pyramid to zarr dt.to_zarr(store_2d, zarr_format=2, consolidated=True) ``` -------------------------------- ### Resample a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Demonstrates how to resample an existing pyramid structure. ```python import xarray as xr from ndpyramid import resample_pyramid # Load your pyramid dataset ds = xr.open_zarr('path/to/pyramid.zarr') # Resample the pyramid resampled_ds = resample_pyramid(ds, scale_factor=2) ``` -------------------------------- ### Open and plot pyramid results Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-resample.html Opens the generated Zarr DataTree and plots the first time step of the specified variable. ```python dt = xr.open_datatree(output, engine="zarr") dt["1"].ds.isel(time=0).tasmax.plot() ``` -------------------------------- ### Open and visualize the result Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-create.html Opens the generated datatree from the Zarr store and plots a specific level. ```python dt = xr.open_datatree(output, engine="zarr") dt["2"].ds.isel(time=0).air.plot() ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Demonstrates how to reproject a pyramid dataset to a new coordinate reference system. ```python from ndpyramid import reproject_pyramid reprojected_pyramid = reproject_pyramid(pyramid, target_crs='EPSG:4326') ``` -------------------------------- ### Resample Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Demonstrates how to resample an existing pyramid structure to a new resolution or grid. ```python import xarray as xr from ndpyramid import resample_pyramid # Assuming 'ds' is an existing pyramid dataset resampled_ds = resample_pyramid(ds, target_grid='some_grid_definition') ``` -------------------------------- ### Importing pyramid_reproject Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-reproject.html Initial imports required to use the pyramid_reproject functionality. ```python import dask # noqa import xarray as xr from ndpyramid import pyramid_reproject ``` -------------------------------- ### pyramid_reproject(ds, *, projection='web-mercator', levels=None, level_list=None, pixels_per_tile=128, other_chunks=None, resampling='average', extra_dim=None, clear_attrs=False) Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/reproject.html Creates a multiscale pyramid of a dataset via reprojection. ```APIDOC ## pyramid_reproject ### Description Create a multiscale pyramid of a dataset via reprojection. ### Parameters - **ds** (xarray.Dataset) - Required - The dataset to create a multiscale pyramid of. - **projection** (str) - Optional - The projection to use. Default is 'web-mercator'. - **levels** (int) - Optional - The number of (contiguous) levels to create starting at 0. Mutually exclusive with level_list. - **level_list** (Sequence[int]) - Optional - Explicit list of zoom levels to generate. Mutually exclusive with levels. - **pixels_per_tile** (int) - Optional - Number of pixels per tile, by default 128. - **other_chunks** (dict) - Optional - Chunks for non-spatial dims to pass to xr.Dataset.chunk. - **resampling** (str or dict) - Optional - Resampling method to use. Default is 'average'. If a dict, keys are variable names and values are resampling methods. - **extra_dim** (str) - Optional - Deprecated/ignored. - **clear_attrs** (bool) - Optional - Clear the attributes of the DataArrays within the multiscale pyramid. Default is False. ### Returns - **xr.DataTree** - The multiscale pyramid. ``` -------------------------------- ### Importing ndpyramid modules Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-resample.html Initial imports required to use the pyramid_resample functionality. ```python import dask # noqa import xarray as xr from ndpyramid import pyramid_resample ``` -------------------------------- ### Create a 4D Zarr pyramid from multi-variable GeoTIFFs Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Processes temperature and precipitation GeoTIFFs into a 4D dataset with month and band dimensions before creating a Zarr pyramid. ```python %%time # open and extract the input datasets ds1_all = [] ds2_all = [] months = list(map(lambda d: d + 1, range(12))) for i in months: path = f"{input_path}/wc2.1_2.5m_tavg_{i:02g}.tif" ds = ( xr.open_dataarray(path, engine="rasterio") .to_dataset(name="climate") .squeeze() .reset_coords(["band"], drop=True) ) ds1_all.append(ds) ds1 = xr.concat(ds1_all, pd.Index(months, name="month")) for i in months: path = f"{input_path}/wc2.1_2.5m_prec_{i:02g}.tif" ds = ( xr.open_dataarray(path, engine="rasterio") .to_dataset(name="climate") .squeeze() .reset_coords(["band"], drop=True) ) ds2_all.append(ds) ds2 = xr.concat(ds2_all, pd.Index(months, name="month")) ds2["climate"].values[ds2["climate"].values == ds2["climate"].values[0, 0, 0]] = ds1[ "climate" ].values[0, 0, 0] ds = xr.concat([ds1, ds2], pd.Index(["tavg", "prec"], name="band")) # create the pyramid dt = pyramid_reproject( ds, levels=LEVELS, extra_dim="band", other_chunks={"band": 2, "month": 12}, clear_attrs=True ) dt.ds.attrs # write the pyramid to zarr dt.to_zarr(store_4d, zarr_format=2, consolidated=True, mode="w") ``` -------------------------------- ### Resample a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Demonstrates the basic usage of the resample method on a pyramid object. ```python import xarray as xr from ndpyramid import pyramid_regrid # Assuming 'ds' is an xarray Dataset representing a pyramid # Resample the pyramid to a new resolution resampled_pyramid = pyramid_regrid(ds, target_resolution=0.5) ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Demonstrates how to reproject a pyramid object to a new coordinate reference system. ```python from ndpyramid import reproject # Assuming 'pyramid' is an existing ndpyramid object reprojected_pyramid = reproject(pyramid, target_crs='EPSG:3857') ``` -------------------------------- ### Generate pyramid with bilinear resampling Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-reproject.html Uses the bilinear resampling algorithm to generate a two-level pyramid and saves it to Zarr. ```python %%time levels = 2 reprojected_pyramid = pyramid_reproject(ds, levels=levels, resampling="bilinear") reprojected_pyramid.to_zarr(output, zarr_format=2, consolidated=True, mode="w") ``` -------------------------------- ### Generate and save pyramids Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-create.html Defines a custom coarsening function and uses pyramid_create to generate levels, then saves the result to Zarr. ```python def sel_coarsen(ds, factor, dims, **kwargs): return ds.sel(**{dim: slice(None, None, factor) for dim in dims}) factors = [4, 2, 1] pyramid = pyramid_create( ds, dims=("lat", "lon"), factors=factors, boundary="trim", func=sel_coarsen, method_label="slice_coarsen", type_label="pick", ) pyramid.to_zarr(output, zarr_format=2, consolidated=True, mode="w") ``` -------------------------------- ### Generate a target grid dataset Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/regrid.html Creates an xarray Dataset representing a grid at a specific zoom level for a given projection. ```python def make_grid_ds( level: int, pixels_per_tile: int = 128, projection: typing.Literal["web-mercator", "equidistant-cylindrical"] = "web-mercator", ) -> xr.Dataset: """Make a dataset representing a target grid Parameters ---------- level : int The zoom level to compute the grid for. Level zero is the furthest out zoom level pixels_per_tile : int, optional Number of pixels to include along each axis in individual tiles, by default 128 projection : str, optional The projection to use for the grid, by default 'equidistant-cylindrical' Returns ------- xr.Dataset Target grid dataset with the following variables: - "x": X coordinate in Web Mercator projection (grid cell center) - "y": Y coordinate in Web Mercator projection (grid cell center) - "lat": latitude coordinate (grid cell center) - "lon": longitude coordinate (grid cell center) - "lat_b": latitude bounds for grid cell - "lon_b": longitude bounds for grid cell """ projection_model = Projection(name=projection) dim = (2**level) * pixels_per_tile transform = projection_model.transform(dim=dim) if projection_model.name == "equidistant-cylindrical": title = "Equidistant Cylindrical Grid" elif projection_model.name == "web-mercator": title = "Web Mercator Grid" else: title = "Unknown Projection Grid" p = projection_model._proj grid_shape = (dim, dim) bounds_shape = (dim + 1, dim + 1) xs = np.empty(grid_shape) ys = np.empty(grid_shape) lat = np.empty(grid_shape) lon = np.empty(grid_shape) lat_b = np.zeros(bounds_shape) lon_b = np.zeros(bounds_shape) # calc grid cell center coordinates ii, jj = np.meshgrid(np.arange(dim) + 0.5, np.arange(dim) + 0.5) for i, j in itertools.product(range(grid_shape[0]), range(grid_shape[1])): locs = [ii[i, j], jj[i, j]] xs[i, j], ys[i, j] = transform * locs lon[i, j], lat[i, j] = p(xs[i, j], ys[i, j], inverse=True) # calc grid cell bounds iib, jjb = np.meshgrid(np.arange(dim + 1), np.arange(dim + 1)) for i, j in itertools.product(range(bounds_shape[0]), range(bounds_shape[1])): locs = [iib[i, j], jjb[i, j]] x, y = transform * locs lon_b[i, j], lat_b[i, j] = p(x, y, inverse=True) return xr.Dataset( { "x": xr.DataArray(xs[0, :], dims=["x"]), "y": xr.DataArray(ys[:, 0], dims=["y"]), "lat": xr.DataArray(lat, dims=["y", "x"]), "lon": xr.DataArray(lon, dims=["y", "x"]), "lat_b": xr.DataArray(lat_b, dims=["y_b", "x_b"]), "lon_b": xr.DataArray(lon_b, dims=["y_b", "x_b"]), }, attrs=dict(title=title, Conventions="CF-1.8"), ) ``` -------------------------------- ### Create a 3D GeoTIFF pyramid with multiple variables Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Combines two 2D GeoTIFF images into a single 3D array along the band dimension before creating a Zarr pyramid. ```python %%time # input datasets path1 = f"{input_path}/wc2.1_2.5m_tavg_10.tif" path2 = f"{input_path}/wc2.1_2.5m_prec_10.tif" # open and extract the input datasets ds1 = ( xr.open_dataarray(path1, engine="rasterio") .to_dataset(name="climate") .squeeze() .reset_coords(["band"], drop=True) ) ds2 = ( xr.open_dataarray(path2, engine="rasterio") .to_dataset(name="climate") .squeeze() .reset_coords(["band"], drop=True) ) ds2["climate"].values[ds2["climate"].values == ds2["climate"].values[0, 0]] = ds1["climate"].values[ 0, 0 ] ds = xr.concat([ds1, ds2], pd.Index(["tavg", "prec"], name="band")) # create the pyramid dt = pyramid_reproject(ds, levels=LEVELS, other_chunks={"band": 2}, clear_attrs=True) # write the pyramid to zarr dt.to_zarr(store_3d, zarr_format=2, consolidated=True) ``` -------------------------------- ### pyramid_create(ds, factors, dims, func, type_label='reduce', method_label=None, **kwargs) Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/create.html Creates a multiscale pyramid via a given function applied to an xarray Dataset. This is the generalized version of pyramid_coarsen. ```APIDOC ## pyramid_create ### Description Creates a multiscale pyramid via a given function applied to a dataset. This function serves as the generalized version of pyramid_coarsen. ### Parameters - **ds** (xarray.Dataset) - Required - The dataset to apply the function to. - **factors** (list[int]) - Required - The factors to coarsen by. - **dims** (list[str]) - Required - The dimensions to coarsen. - **func** (Callable) - Required - The function to apply to the dataset; must accept the `ds`, `factor`, and `dims` as positional arguments. - **type_label** (str) - Optional - The type label to use as metadata for the multiscales spec. Default is 'reduce'. - **method_label** (str) - Optional - The method label to use as metadata for the multiscales spec. Default is the name of the function. - **kwargs** (dict) - Optional - Additional keyword arguments to pass to the func. ### Returns - **DataTree** - Returns an xarray.DataTree containing the multiscale pyramid levels. ``` -------------------------------- ### Generate Weights Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/regrid.html Creates a multiscale weights pyramid for regridding, requiring an input dataset and optional configuration for levels and projection. ```python def generate_weights_pyramid( ds_in: xr.Dataset, levels: int | None = None, *, level_list: Sequence[int] | None = None, method: str = "bilinear", regridder_kws: dict | None = None, projection: typing.Literal["web-mercator", "equidistant-cylindrical"] = "web-mercator", ) -> xr.DataTree: """Helper function to generate weights for a multiscale regridder Parameters ---------- ds_in : xr.Dataset Input dataset to regrid levels : int, optional Number of contiguous levels (0..levels-1) to build. Ignored if ``level_list`` is provided. level_list : Sequence[int], optional Explicit list of zoom levels to build (sparse weights). Mutually exclusive with ``levels``. method : str, optional Regridding method. See :py:class:`~xesmf.Regridder` for valid options, by default 'bilinear' regridder_kws : dict Keyword arguments to pass to :py:class:`~xesmf.Regridder`. Default is `{'periodic': True}` projection : str, optional The projection to use for the grid, by default 'web-mercator' Returns ------- weights : xr.DataTree Multiscale weights """ import xesmf as xe regridder_kws = {} if regridder_kws is None else regridder_kws regridder_kws = {"periodic": True, **regridder_kws} if levels is not None and level_list is not None: raise ValueError("Specify only one of 'levels' or 'level_list'.") if level_list is not None: level_indices = sorted({int(i) for i in level_list}) else: if levels is None: raise ValueError("Must provide either 'levels' or 'level_list'.") level_indices = list(range(levels)) plevels = {} for level in level_indices: ds_out = make_grid_ds(level=level, projection=projection) regridder = xe.Regridder(ds_in, ds_out, method, **regridder_kws) ds = xesmf_weights_to_xarray(regridder) plevels[str(level)] = ds root_levels_attr: typing.Any = level_indices if level_list is not None else len(level_indices) root = xr.Dataset(attrs={"levels": root_levels_attr, "regrid_method": method}) plevels["/"] = root return xr.DataTree.from_dict(plevels) ``` -------------------------------- ### pyramid_create Source: https://ndpyramid.readthedocs.io/en/latest/_sources/generated/ndpyramid.pyramid_create.rst Creates a pyramid structure based on the provided parameters. ```APIDOC ## pyramid_create ### Description Creates a pyramid structure. This function is part of the ndpyramid library. ### Signature `ndpyramid.pyramid_create()` ``` -------------------------------- ### Generate pyramid with nearest resampling Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-reproject.html Uses the nearest resampling algorithm to generate a two-level pyramid and saves it to Zarr. ```python %%time levels = 2 reprojected_pyramid = pyramid_reproject(ds, levels=levels, resampling="nearest") reprojected_pyramid.to_zarr(output, zarr_format=2, consolidated=True, mode="w") ``` -------------------------------- ### pyramid_coarsen(ds, factors, dims, **kwargs) Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/coarsen.html Creates a multiscale pyramid via coarsening of an xarray dataset by given factors. ```APIDOC ## pyramid_coarsen(ds, factors, dims, **kwargs) ### Description Create a multiscale pyramid via coarsening of a dataset by given factors. ### Parameters - **ds** (xarray.Dataset) - Required - The dataset to coarsen. - **factors** (list[int]) - Required - The factors to coarsen by. - **dims** (list[str]) - Required - The dimensions to coarsen. - **kwargs** (dict) - Optional - Additional keyword arguments to pass to xarray.Dataset.coarsen. ### Returns - **xr.DataTree** - The resulting multiscale pyramid. ``` -------------------------------- ### Plot pyramid levels Source: https://ndpyramid.readthedocs.io/en/latest/examples/geotiff.html Iterates through pyramid levels to visualize the first time step of the climate data. ```python for level in range(LEVELS): fig, ax = plt.subplots() dt[str(level)].ds.isel(band=0, month=0).climate.plot() ax.set_title(f"Level {level}") ``` -------------------------------- ### pyramid_regrid Source: https://ndpyramid.readthedocs.io/en/latest/api.html Makes a pyramid using xesmf's regridders. ```APIDOC ## pyramid_regrid(ds[, projection, ...]) ### Description Make a pyramid using xesmf's regridders. ``` -------------------------------- ### pyramid_create Source: https://ndpyramid.readthedocs.io/en/latest/_sources/api.rst Create a pyramid structure from an xarray object. ```APIDOC ## pyramid_create ### Description Initializes and creates a pyramid structure from the provided xarray object. ``` -------------------------------- ### Jupyter Notebook Metadata Structure Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-resample.ipynb Displays the standard metadata structure for a Jupyter notebook file. ```json { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.7" } }, "nbformat": 4, "nbformat_minor": 4 } ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb This snippet demonstrates how to perform a reprojection on a pyramid object. ```python from ndpyramid import reproject # Example of reprojecting a pyramid reprojected_pyramid = reproject(pyramid, target_crs='EPSG:4326') ``` -------------------------------- ### pyramid_create Source: https://ndpyramid.readthedocs.io/en/latest/api.html Creates a multiscale pyramid via a given function applied to a dataset. ```APIDOC ## pyramid_create(ds, *, factors, dims, func[, ...]) ### Description Create a multiscale pyramid via a given function applied to a dataset. ``` -------------------------------- ### Reprojecting a Raster Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb This snippet demonstrates the basic usage of the reproject function to transform a raster dataset. ```python from pyramid.reproject import reproject # Reproject the raster to a new CRS reprojected_data = reproject(source_raster, target_crs) ``` -------------------------------- ### Generate Pyramids Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Generates a pyramid with specified levels using the nearest neighbor resampling algorithm. ```python %%time levels = 2 reprojected_pyramid = pyramid_reproject(ds, levels=levels, resampling="nearest") ``` -------------------------------- ### Zarr store structure for pyramid data Source: https://ndpyramid.readthedocs.io/en/latest/_sources/schema.md Visual representation of the Zarr store hierarchy showing the quadrupling of chunks as zoom levels increase. ```text / ├── .zmetadata ├── 0 │ ├── tavg │ └── 0.0 ├── 1 │ ├── tavg │ └── 0.0 │ └── 0.1 │ └── 1.0 │ └── 1.1 ├── 2 ... ``` -------------------------------- ### Inspect Time Index Source: https://ndpyramid.readthedocs.io/en/latest/examples/pyramid-resample.html Displays the DatetimeIndex for the time coordinate. ```python PandasIndex(DatetimeIndex(['2015-01-01', '2015-02-01', '2015-03-01', '2015-04-01', '2015-05-01', '2015-06-01', '2015-07-01', '2015-08-01', '2015-09-01', '2015-10-01', ... '2099-03-01', '2099-04-01', '2099-05-01', '2099-06-01', '2099-07-01', '2099-08-01', '2099-09-01', '2099-10-01', '2099-11-01', '2099-12-01'], dtype='datetime64[ns]', name='time', length=1020, freq=None)) ``` -------------------------------- ### Create a multiscale pyramid with pyramid_create Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/create.html Defines the pyramid_create function which generates a DataTree of multiscale levels based on a provided dataset and transformation function. ```python def pyramid_create( ds: xr.Dataset, *, factors: list[int], dims: list[str], func: Callable, type_label: str = "reduce", method_label: str | None = None, **kwargs, ): """Create a multiscale pyramid via a given function applied to a dataset. The generalized version of pyramid_coarsen. Parameters ---------- ds : xarray.Dataset The dataset to apply the function to. factors : list[int] The factors to coarsen by. dims : list[str] The dimensions to coarsen. func : Callable The function to apply to the dataset; must accept the `ds`, `factor`, and `dims` as positional arguments. type_label : str, optional The type label to use as metadata for the multiscales spec. The default is 'reduce'. method_label : str, optional The method label to use as metadata for the multiscales spec. The default is the name of the function. kwargs : dict Additional keyword arguments to pass to the func. """ # multiscales spec save_kwargs = locals() del save_kwargs["ds"] del save_kwargs["func"] del save_kwargs["type_label"] del save_kwargs["method_label"] attrs = { "multiscales": multiscales_template( datasets=[{"path": str(i)} for i in range(len(factors))], type=type_label, method=method_label or func.__name__, version=get_version(), kwargs=save_kwargs, ) } plevels = {str(key): func(ds, factor, dims, **kwargs) for key, factor in enumerate(factors)} plevels["/"] = xr.Dataset(attrs=attrs) return xr.DataTree.from_dict(plevels) ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb This snippet demonstrates how to reproject a pyramid object to a new coordinate reference system. ```python from ndpyramid import pyramid_reproject # Reproject the pyramid to the target CRS reprojected_pyramid = pyramid_reproject(pyramid, target_crs="EPSG:3857") ``` -------------------------------- ### Implement pyramid_coarsen function Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/coarsen.html The core implementation of the pyramid_coarsen function, which utilizes pyramid_create to generate a multiscale pyramid using a coarsening reduction method. ```python from __future__ import annotations # noqa: F401 import xarray as xr from .create import pyramid_create [docs] def pyramid_coarsen( ds: xr.Dataset, *, factors: list[int], dims: list[str], **kwargs ) -> xr.DataTree: """Create a multiscale pyramid via coarsening of a dataset by given factors Parameters ---------- ds : xarray.Dataset The dataset to coarsen. factors : list[int] The factors to coarsen by. dims : list[str] The dimensions to coarsen. kwargs : dict Additional keyword arguments to pass to xarray.Dataset.coarsen. """ def coarsen(ds: xr.Dataset, factor: int, dims: list[str], **kwargs): # merge dictionary via union operator kwargs |= {d: factor for d in dims} return ds.coarsen(**kwargs).mean() # type: ignore return pyramid_create( ds, factors=factors, dims=dims, func=coarsen, method_label="pyramid_coarsen", type_label="reduce", **kwargs, ) ``` -------------------------------- ### pyramid_resample Source: https://ndpyramid.readthedocs.io/en/latest/api.html Creates a multiscale pyramid of a dataset via resampling. ```APIDOC ## pyramid_resample(ds, *, x, y[, projection, ...]) ### Description Create a multiscale pyramid of a dataset via resampling. ``` -------------------------------- ### Reprojecting a Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Basic usage for reprojecting a pyramid structure. ```python from ndpyramid import reproject # Example usage reproject(data, target_crs) ``` -------------------------------- ### Perform Regridding Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-regrid.ipynb Basic usage of the regrid function to transform data. ```python from pyramid_regrid import regrid # Example usage result = regrid(data, target_grid) ``` -------------------------------- ### Reprojecting a dataset into a pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_sources/examples/pyramid-reproject.ipynb Uses the pyramid_reproject function to create a reprojected pyramid with specified levels and resampling method, then saves it to a Zarr store. ```python reprojected_pyramid.to_zarr(output, zarr_format=2, consolidated=True, mode="w") ``` ```python %%time levels = 2 reprojected_pyramid = pyramid_reproject(ds, levels=levels, resampling="bilinear") reprojected_pyramid.to_zarr(output, zarr_format=2, consolidated=True, mode="w") ``` -------------------------------- ### Regrid Pyramid Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/regrid.html Constructs a multiscale pyramid from an input dataset using xESMF regridders. ```python def pyramid_regrid( ds: xr.Dataset, projection: typing.Literal["web-mercator", "equidistant-cylindrical"] = "web-mercator", target_pyramid: xr.DataTree | None = None, levels: int | None = None, *, level_list: Sequence[int] | None = None, parallel_weights: bool = True, weights_pyramid: xr.DataTree | None = None, method: str = "bilinear", regridder_kws: dict | None = None, regridder_apply_kws: dict | None = None, other_chunks: dict | None = None, pixels_per_tile: int = 128, ) -> xr.DataTree: """Make a pyramid using xesmf's regridders Parameters ---------- ds : xr.Dataset Input dataset projection : str, optional Projection to use for the grid, by default 'web-mercator' target_pyramid : xr.DataTree, optional Target grids, if not provided, they will be generated, by default None levels : int, optional Number of contiguous levels to build (0..levels-1). Ignored if ``level_list`` provided. level_list : Sequence[int], optional Explicit list of zoom levels to build (sparse). Mutually exclusive with ``levels``. weights_pyramid : xr.DataTree, optional pyramid containing pregenerated weights parallel_weights : Bool Use dask to generate parallel weights method : str, optional Regridding method. See :py:class:`~xesmf.Regridder` for valid options, by default 'bilinear' regridder_kws : dict Keyword arguments to pass to regridder. Default is `{'periodic': True}` regridder_apply_kws : dict Keyword arguments such as `keep_attrs`, `skipna`, `na_thres` to pass to :py:meth:`~xesmf.Regridder.__call__`. Default is None other_chunks : dict Chunks for non-spatial dims to pass to :py:meth:`~xr.Dataset.chunk`. Default is None pixels_per_tile : int, optional Number of pixels per tile, by default 128 ``` -------------------------------- ### Create a Multiscale Pyramid with pyramid_resample Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/resample.html Defines the interface for creating a multiscale pyramid from an xarray dataset. It supports specifying zoom levels, projections, and resampling methods. ```python def pyramid_resample( ds: xr.Dataset, *, x: str, y: str, projection: ProjectionOptions = "web-mercator", levels: int | None = None, level_list: Sequence[int] | None = None, pixels_per_tile: int = 128, other_chunks: dict | None = None, resampling: ResamplingOptions | dict = "bilinear", clear_attrs: bool = False, ) -> xr.DataTree: """Create a multiscale pyramid of a dataset via resampling. Parameters ---------- ds : xarray.Dataset The dataset to create a multiscale pyramid of. y : string name of the variable to use as ``y`` axis of the CF area definition x : string name of the variable to use as ``x`` axis of the CF area definition projection : str, optional The projection to use. Default is ``web-mercator``. levels : int, optional Number of contiguous levels starting at 0 to create. Mutually exclusive with ``level_list``. level_list : Sequence[int], optional Explicit list of zoom levels to build (e.g. ``[4]``). Mutually exclusive with ``levels``. pixels_per_tile : int, optional Number of pixels per tile, by default 128 other_chunks : dict Chunks for non-spatial dims to pass to :py:meth:`~xr.Dataset.chunk`. Default is None resampling : str or dict, optional Pyresample resampling method to use (``bilinear`` or ``nearest``). Default is ``bilinear``. If a dict, keys are variable names and values are resampling methods. clear_attrs : bool, False Clear the attributes of the DataArrays within the multiscale pyramid. Default is False. Returns ------- xr.DataTree The multiscale pyramid. ``` -------------------------------- ### Create a multiscale pyramid with pyramid_reproject Source: https://ndpyramid.readthedocs.io/en/latest/_modules/ndpyramid/reproject.html Use this function to generate a multiscale pyramid from an xarray Dataset. It supports custom projections, specific zoom levels, and various resampling strategies. ```python def pyramid_reproject( ds: xr.Dataset, *, projection: ProjectionOptions = "web-mercator", levels: int | None = None, level_list: Sequence[int] | None = None, pixels_per_tile: int = 128, other_chunks: dict | None = None, resampling: str | dict = "average", extra_dim: str | None = None, clear_attrs: bool = False, ) -> xr.DataTree: """Create a multiscale pyramid of a dataset via reprojection. Parameters ---------- ds : xarray.Dataset The dataset to create a multiscale pyramid of. projection : str, optional The projection to use. Default is 'web-mercator'. levels : int, optional The number of (contiguous) levels to create starting at 0. Mutually exclusive with ``level_list``. If both ``levels`` and ``level_list`` are ``None`` then an attempt is made to infer the number of levels via ``get_levels`` (currently not implemented). level_list : Sequence[int], optional Explicit list of zoom levels to generate (e.g. ``[4]`` to only build Z4, or ``[2,4,6]`` for a sparse pyramid). Mutually exclusive with ``levels``. pixels_per_tile : int, optional Number of pixels per tile, by default 128 other_chunks : dict Chunks for non-spatial dims to pass to :py:meth:`~xr.Dataset.chunk`. Default is None resampling : str or dict, optional Resampling method to use. Default is 'average'. If a dict, keys are variable names and values are resampling methods. extra_dim : str, optional Deprecated/ignored. Extra dimensions are handled natively by odc-geo/xarray broadcasting. clear_attrs : bool, False Clear the attributes of the DataArrays within the multiscale pyramid. Default is False. Returns ------- xr.DataTree The multiscale pyramid. """ if levels is not None and level_list is not None: raise ValueError("Specify only one of 'levels' or 'level_list'.") if level_list is not None: # sanitize and sort unique levels level_indices = sorted({int(idx) for idx in level_list}) else: if not levels: levels = get_levels(ds) level_indices = list(range(int(levels))) projection_model = Projection(name=projection) save_kwargs = { # store the explicit list for reproducibility "levels": level_indices, "pixels_per_tile": pixels_per_tile, "projection": projection, "other_chunks": other_chunks, "resampling": resampling, "extra_dim": extra_dim, "clear_attrs": clear_attrs, } attrs = { "multiscales": multiscales_template( datasets=[ {"path": str(i), "level": i, "crs": projection_model._crs} for i in level_indices ], type="reduce", method="pyramid_reproject", version=get_version(), kwargs=save_kwargs, ) } plevels = { str(level): level_reproject( ds, projection=projection, level=level, pixels_per_tile=pixels_per_tile, resampling=resampling, extra_dim=extra_dim, clear_attrs=clear_attrs, ) for level in level_indices } # create the final multiscale pyramid plevels["/"] = xr.Dataset(attrs=attrs) pyramid = xr.DataTree.from_dict(plevels) pyramid = add_metadata_and_zarr_encoding( pyramid, levels=level_indices, pixels_per_tile=pixels_per_tile, other_chunks=other_chunks, projection=projection_model, ) return pyramid ``` -------------------------------- ### pyramid_resample Source: https://ndpyramid.readthedocs.io/en/latest/_sources/generated/ndpyramid.pyramid_resample.rst The pyramid_resample function allows users to perform resampling on pyramid data structures. ```APIDOC ## pyramid_resample ### Description Resamples a pyramid data structure to a new resolution or grid. ### Signature `ndpyramid.pyramid_resample(source, ...)` ``` -------------------------------- ### Generate Sparse Pyramids Source: https://ndpyramid.readthedocs.io/en/latest/_sources/schema.md Use the level_list parameter in pyramid functions to generate only specific zoom levels instead of the full range. ```python from ndpyramid import pyramid_reproject # Only build zoom level 5 pyr = pyramid_reproject(ds, level_list=[5]) # Build a sparse pyramid containing zoom levels 2, 4, and 6 pyr_sparse = pyramid_reproject(ds, level_list=[2, 4, 6]) ```