xdggs

https://github.com/xarray-contrib/xdggs
Admin
xdggs is an open-source Python package that provides tools for handling geospatial data using...
Tokens:5,996
Snippets:97
Trust Score:9.1
License:Apache-2.0
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install xdggs from source

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/getting_started/installation.md

Cloning the repository and installing locally for development or custom builds.

```shell
git clone https://github.com/xarray-contrib/xdggs.git
cd xdggs
pip install .
```

--------------------------------

### Install xdggs from GitHub

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/getting_started/installation.md

Direct installation from the project's GitHub repository using pip.

```shell
pip install "xdggs @ git+https://github.com/xarray-contrib/xdggs.git"
```

--------------------------------

### Install xdggs via pip

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/getting_started/installation.md

Standard installation method using the Python package index.

```shell
pip install xdggs
```

--------------------------------

### Importing required libraries

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Initial setup for H3 and xarray operations.

```python
import h3
import h3.api.numpy_int
import h3.unstable.vect
import numpy as np
import xarray as xr
```

--------------------------------

### Install xdggs via conda

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/getting_started/installation.md

Installation method for users managing environments with conda.

```shell
conda install -c conda-forge xdggs
```

--------------------------------

### Manage pre-commit hooks

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/contributor_guide/index.md

Install and manually trigger pre-commit hooks for local code validation.

```shell
pre-commit install
```

```shell
pre-commit run --all-files
```

--------------------------------

### Create documentation environment

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/contributor_guide/index.md

Commands to set up the environment required for building documentation.

```shell
mamba env create -f ci/docs.yml
mamba activate xdggs-docs
```

```shell
conda env create -f ci/docs.yml
conda activate xdggs-docs
```

--------------------------------

### Build documentation

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/contributor_guide/index.md

Commands to build and preview documentation locally.

```shell
sphinx-autobuild docs docs/_build/html --open-browser
```

```shell
sphinx-build -b html docs docs/_build/html
```

```shell
open docs/_build/html/index.html    # macOS
xdg-open docs/_build/html/index.html  # Linux
start docs/_build/html/index.html  # Windows
```

--------------------------------

### Loading and visualizing the dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Load the air temperature tutorial dataset and visualize the first time step.

```python
ds = xr.tutorial.load_dataset("air_temperature").load()
ds
```

```python
ds.air[0].plot()
```

--------------------------------

### Initialize xdggs and xarray

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Import the necessary libraries to begin working with xarray and xdggs.

```python
import xarray as xr

import xdggs
```

--------------------------------

### Initialize xdggs environment

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Imports necessary libraries and configures xarray display options for better readability.

```python
import numpy as np
import xarray as xr

import xdggs

xr.set_options(keep_attrs=True, display_expand_attrs=False, display_expand_indexes=True)
```

--------------------------------

### Load and prepare Healpix dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/healpix.ipynb

Load the tutorial dataset and remove unnecessary geographic coordinates.

```python
original_ds = xdggs.tutorial.open_dataset("air_temperature", "healpix").load()
air_temperature = original_ds.drop_vars(["lat", "lon"])
air_temperature
```

--------------------------------

### Create development environment

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/contributor_guide/index.md

Commands to create and configure a development environment using Mamba or Conda.

```shell
mamba create -n xdggs-dev python=3.12
mamba env update -n xdggs-dev -f ci/environment.yml
mamba activate xdggs-dev
pip install --no-deps -e .
```

```shell
conda create -n xdggs-dev python=3.12
conda env update -n xdggs-dev -f ci/environment.yml
conda activate xdggs-dev
pip install --no-deps -e .
```

--------------------------------

### Load and Prepare HEALPix Dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Open a NetCDF file, drop coordinate variables, and stack dimensions to prepare for indexing.

```python
ds = xr.open_dataset("data/healpix_nolotation.nc")

ds = (
    ds.load()
    .drop_vars(["latitude", "longitude"])
    .stack(cell=["x", "y"], create_index=False)
)

ds.cell_ids.attrs = {
    "grid_name": "healpix",
    "nside": 4096,
    "nest": True,
}

ds
```

--------------------------------

### Load and prepare H3 dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Load the H3 tutorial dataset and remove unnecessary geographic coordinates.

```python
original_ds = xdggs.tutorial.open_dataset("air_temperature", "h3").load()
air_temperature = original_ds.drop_vars(["lat", "lon"])
air_temperature
```

--------------------------------

### Run tests

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/contributor_guide/index.md

Execute the test suite using pytest.

```shell
pytest
```

--------------------------------

### Load H3 dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Opens a NetCDF file containing H3 grid data using the netcdf4 engine.

```python
ds = xr.open_dataset("data/h3.nc", engine="netcdf4")
ds
```

--------------------------------

### Visualize H3 data

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Use lonboard to explore the H3 dataset interactively.

```python
ds["air"].dggs.explore()
```

--------------------------------

### Inspecting shapes

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Verify that the index and coordinate shapes match.

```python
index.shape
```

```python
ds.lon.shape
```

--------------------------------

### Defining a bounding box

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Calculate the dataset bounds and create a shapely polygon for polyfilling.

```python
lon_min, lon_max = ds.lon.min().values.item(), ds.lon.max().values.item()
lat_min, lat_max = ds.lat.min().values.item(), ds.lat.max().values.item()
```

```python
import shapely

bbox_coords = [
    (lon_min - 360, lat_min),
    (lon_min - 360, lat_max),
    (lon_max - 360, lat_max),
    (lon_max - 360, lat_min),
    (lon_min - 360, lat_min),
]
bbox = shapely.Polygon(bbox_coords)
bbox
```

```python
bbox_coords
```

--------------------------------

### Set DGGSIndex on Xarray Dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/design_doc.md

Demonstrates how to configure a DGGSIndex on a coordinate, either via auto-detection or manual specification of the grid system and resolution.

```python
import xarray as xr
import xdggs

ds = xr.Dataset(coords={"cell": ("cell", [...], {"grid_name": "h3", "resolution": 3})})

# auto-detect grid system and parameters
ds.set_xindex("cell", xdggs.DGGSIndex)

# set the grid system and parameters manually
ds.set_xindex("cell", xdggs.H3Index, resolution=3)
```

--------------------------------

### Reconstruct coordinates and visualize DGGS data

Source: https://github.com/xarray-contrib/xdggs/blob/main/README.md

Demonstrates decoding DGGS coordinates, assigning latitude and longitude, and creating an interactive plot using xdggs.

```python
import xarray as xr
import xdggs

ds = xdggs.tutorial.open_dataset("air_temperature", "h3")

# Decode DGGS coordinates
ds_idx = ds.pipe(xdggs.decode)

# Assign geographical coordinates
ds_idx = ds_idx.dggs.assign_latlon_coords()

# Interactive visualization
ds_idx["air"].isel(time=0).compute().dggs.explore(center=0, cmap="viridis", alpha=0.5)
```

--------------------------------

### Verify derived data

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Compare the derived dataset with the original dataset.

```python
xr.testing.assert_allclose(derived_ds, original_ds)
```

--------------------------------

### Register a custom convention

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/user_guide/conventions.md

Defines a new convention class by inheriting from xdggs.conventions.Convention and implementing the required decode and encode methods.

```python
import xdggs
from collections.abc import Hashable
from typing import Any
from xdggs.grid import DGGSInfo


@xdggs.conventions.register_convention("my-convention")
class MyConvention(xdggs.conventions.Convention):
    def decode(
        self,
        obj: xr.Dataset,
        *,
        grid_info: dict[str, Any] | DGGSInfo | None,
        name: Hashable | None,
        index_options: dict[str, Any] | None,
    ) -> xr.Dataset:
        # decode
        pass

    def encode(
        self, obj: xr.Dataset, *, encoding: dict[str, Any] | None = None
    ) -> xr.Dataset:
        # encode
        pass
```

--------------------------------

### Verifying index coverage

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Check the ratio of unique H3 cells to original data points.

```python
index.shape = lon.shape

len(np.unique(index)) / lon.size
```

--------------------------------

### Broadcasting coordinates for H3 indexing

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Prepare coordinate arrays for H3 conversion at a specified resolution.

```python
resolution = 3

lon, lat = xr.broadcast(ds.lon, ds.lat)
```

--------------------------------

### Verify Selection Results

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Validate that the selection result matches the original dataset.

```python
xr.testing.assert_equal(result.drop_vars(["latitude", "longitude"]), ds)
```

--------------------------------

### Visualize Grid Cells

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Render the GeoDataFrame on an interactive map.

```python
df.explore(tiles="OpenStreetMap")
```

--------------------------------

### Query with Coordinate Arrays

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Use existing coordinate arrays to perform a selection on the DGGS index.

```python
result = ds_idx.dggs.sel_latlon(ds2.latitude, ds2.longitude)
result
```

--------------------------------

### Decode xdggs convention

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/user_guide/conventions.md

Demonstrates loading a dataset with the built-in xdggs convention and decoding it into an in-memory xarray index.

```python
import xdggs
import xarray as xr

xdggs_encoded = xdggs.tutorial.open_dataset("air_temperature", "h3").load()
display(xdggs_encoded)
decoded = xdggs_encoded.dggs.decode()
display(decoded)
```

--------------------------------

### Create DGGS Index

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Apply the DGGSIndex to the dataset using the cell_ids variable.

```python
ds_idx = ds.set_xindex("cell_ids", xdggs.DGGSIndex)

ds_idx
```

--------------------------------

### Derive cell boundaries

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Generate cell boundary polygons as shapely objects.

```python
cell_boundaries = ds.dggs.cell_boundaries()
cell_boundaries
```

--------------------------------

### Interpolating dataset to H3 grid

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Interpolate the original dataset onto the H3 grid points and save to NetCDF.

```python
coords = {"cell_ids": bbox_indexes}

# remember to re-add the 360 degree offset
dsi = ds.interp(
    lon=xr.DataArray(ll_points_lon_first[:, 0] + 360, dims="cell_ids", coords=coords),
    lat=xr.DataArray(ll_points_lon_first[:, 1], dims="cell_ids", coords=coords),
)
dsi
```

```python
dsi2 = dsi.drop_vars(["lon", "lat", "index"])
dsi2.cell_ids.attrs = {"grid_name": "h3", "resolution": resolution}
dsi2.to_netcdf("data/h3.nc", mode="w")
dsi2
```

--------------------------------

### Encode and decode using Zarr convention

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/user_guide/conventions.md

Uses the Zarr dggs convention to encode grid metadata into attributes and verifies the round-trip integrity.

```python
zarr_encoded = decoded.dggs.encode(convention="zarr")
display(zarr_encoded)
zarr_decoded = zarr_encoded.dggs.decode(convention="zarr")

xr.testing.assert_identical(zarr_decoded, decoded)
```

--------------------------------

### DGGS Accessor API Methods

Source: https://github.com/xarray-contrib/xdggs/blob/main/design_doc.md

Proposed methods for converting datasets to and from DGGS formats using the .dggs accessor.

```python
# "convert" directly from existing cell ids coordinate to DGGS
# basically an alias to ds.set_xindex(..., DGGSIndex)
ds.dggs.from_cell_ids(...)

# convert from lat/lon grid
ds.dggs.from_latlon_grid(...)

# convert from raster
ds.dggs.from_raster(...)

# convert from point data
ds.dggs.from_points(...)

# convert from point data (with aggregation)
ds.dggs.from_points_aggregate(...)

# convert from point data (with aggregation using Xarray API)
ds.dggs.from_points(...).groupby(...).mean()

# convert to lat/lon grid
ds.dggs.to_latlon_grid(...)

# convert to raster
ds.dggs.to_raster(...)

# convert to points (cell centroids)
ds.dggs.to_points(...)

# convert to polygons (cell boundaries)
ds.dggs.to_polygons(...)
```

--------------------------------

### Polyfilling the bounding box

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Generate H3 indices within the defined bounding box.

```python
# h3 wants lat first
bbox_coords_lat_first = [(lat, lon) for lon, lat in bbox_coords]
bbox_indexes = np.array(
    list(h3.api.basic_int.polyfill_polygon(bbox_coords_lat_first, resolution))
)
bbox_indexes.shape
```

--------------------------------

### Inspect H3 grid attributes

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Required attributes for cell_ids coordinate to enable grid-aware operations.

```python
{
    "grid_name": "h3",
    "level": 2,
}
```

--------------------------------

### Select DGGS Data by Cell IDs

Source: https://github.com/xarray-contrib/xdggs/blob/main/design_doc.md

Use Xarray's .sel() method to select data based on specific cell IDs or arrays of cell IDs.

```python
ds.sel(cell=value)
```

--------------------------------

### Select data by latitude and longitude

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Performs a spatial selection on the decoded dataset using specific latitude and longitude arrays.

```python
ds_idx.dggs.sel_latlon(np.array([37.0, 37.5]), np.array([299.3, 299.5]))
```

--------------------------------

### Perform selection using assigned coordinates

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Selects data from the indexed dataset using the latitude and longitude data from the assigned coordinate dataset.

```python
result = ds_idx.dggs.sel_latlon(ds2.latitude.data, ds2.longitude.data)
result
```

--------------------------------

### Verify dataset equality

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Validates that the processed result matches the original dataset using xarray's testing utility.

```python
xr.testing.assert_equal(result, ds)
```

--------------------------------

### Define grid attributes

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/healpix.ipynb

Required metadata attributes for the cell_ids coordinate.

```python
{
    "grid_name": "healpix",
    "level": 4,
    "indexing_scheme": "nested",
}
```

--------------------------------

### Assign latitude and longitude coordinates

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Adds latitude and longitude coordinate variables to the decoded dataset.

```python
ds2 = ds_idx.dggs.assign_latlon_coords()
ds2
```

--------------------------------

### Derive cell centers

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Reconstruct cell center coordinates from the grid object.

```python
cell_centers = ds.dggs.cell_centers()
cell_centers
```

--------------------------------

### Convert to GeoDataFrame

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Transform cell boundaries into a pandas GeoDataFrame for spatial analysis.

```python
df = (
    cell_boundaries.to_dataset(name="geometry")
    .to_pandas()
    .set_geometry("geometry", crs=4326)
    .set_index("cell_ids")
)
df
```

--------------------------------

### Select by Cell IDs

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Query the indexed dataset using specific cell identifiers.

```python
ds_idx.sel(cell_ids=[11320973, 11320975])
```

--------------------------------

### Assign derived coordinates

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Add derived cell centers back to the dataset.

```python
derived_ds = ds.assign_coords(
    cell_centers.rename_vars({"latitude": "lat", "longitude": "lon"}).coords
)
derived_ds
```

--------------------------------

### Encode and decode using CF convention

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/user_guide/conventions.md

Applies the CF convention encoding scheme to a dataset and verifies the round-trip integrity.

```python
cf_encoded = decoded.dggs.encode(convention="cf")
display(cf_encoded)
cf_decoded = cf_encoded.dggs.decode(convention="cf")

xr.testing.assert_identical(cf_decoded, decoded)
```

--------------------------------

### Decode H3 dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/docs/tutorials/h3.ipynb

Interpret cell IDs to create a grid object for grid-aware operations.

```python
ds = air_temperature.pipe(xdggs.decode)
ds
```

--------------------------------

### Access Cell Boundaries

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_healpy.ipynb

Retrieve the geometric boundaries of the grid cells from the indexed dataset.

```python
cell_boundaries = ds_idx.dggs.cell_boundaries
cell_boundaries
```

--------------------------------

### Decode H3 indices

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/example_h3.ipynb

Uses the pipe method to apply xdggs decoding to the dataset.

```python
ds_idx = ds.pipe(xdggs.decode)
ds_idx
```

--------------------------------

### Extract DGGS Cell Geometries

Source: https://github.com/xarray-contrib/xdggs/blob/main/design_doc.md

Methods for retrieving DGGS cell centroids, coordinates, boundaries, and envelopes as xarray DataArrays.

```python
# return a DataArray of DGGS cell centroids as shapely.POINT objects
ds.dggs.cell_centroids()

# return two DataArrays of DGGS cell centroids as lat/lon coordinates
ds.dggs.cell_centroids_coords()

# return a DataArray of DGGS cell boundaries as shapely.POLYGON objects
ds.dggs.cell_boundaries()

# return a DataArray of DGGS cell envelopes as shapely.POLYGON objects
ds.dggs.cell_envelopes()
```

--------------------------------

### Converting coordinates to H3 indices

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Perform the vector-based conversion of latitude and longitude to H3 indices.

```python
%%time
index = h3.unstable.vect.geo_to_h3(lat.data.ravel(), lon.data.ravel(), resolution)
```

--------------------------------

### Converting H3 indices to geographic points

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Retrieve the center coordinates for the generated H3 indices.

```python
ll_points = np.array([h3.api.numpy_int.h3_to_geo(i) for i in bbox_indexes])
ll_points_lon_first = ll_points[:, ::-1]
```

--------------------------------

### Adding H3 index to dataset

Source: https://github.com/xarray-contrib/xdggs/blob/main/examples/prepare_dataset_h3.ipynb

Assign the computed H3 indices as a new coordinate in the xarray dataset.

```python
ds.coords["index"] = ("lat", "lon"), index.transpose()
ds
```

```python
ds.index.plot()
```
Tokens:
Raw