### Testing Setup and Examples Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Provides setup instructions and example tests for the xpublish-edr API using pytest and TestClient. ```APIDOC ## Testing Setup and Examples ### Description This section outlines how to set up a testing environment for xpublish-edr and includes example tests for querying data and verifying metadata. ### Setup To run these tests, you need `pytest`, `xpublish`, `fastapi.testclient`, `xpublish_edr`, and `cf_xarray` installed. ### Fixture - **test_client**: A pytest fixture that creates an xpublish REST instance with the CfEdrPlugin enabled and returns a TestClient for making requests. ### Example Tests #### `test_position_query` - **Purpose:** Tests the `/datasets/air/edr/position` endpoint with basic query parameters and checks for expected content in the response. - **Request:** `GET /datasets/air/edr/position?coords=POINT(204 44)&f=csv` - **Assertions:** Status code is 200, response text contains 'air' and 'lat,lon'. #### `test_metadata` - **Purpose:** Tests the root dataset endpoint (`/datasets/air/edr/`) to verify metadata retrieval, including available data queries. - **Request:** `GET /datasets/air/edr/` - **Assertions:** Status code is 200, response is JSON, contains 'id' equal to 'air', and 'position', 'area', 'cube' in 'data_queries'. ### Test Code Example ```python import pytest import xpublish from fastapi.testclient import TestClient from xpublish_edr import CfEdrPlugin from cf_xarray.datasets import airds @pytest.fixture def test_client(): # Create xpublish instance with EDR plugin rest = xpublish.Rest( {"air": airds}, plugins={"edr": CfEdrPlugin()} ) return TestClient(rest.app) def test_position_query(test_client): response = test_client.get( "/datasets/air/edr/position?coords=POINT(204 44)&f=csv" ) assert response.status_code == 200 assert "air" in response.text assert "lat,lon" in response.text def test_metadata(test_client): response = test_client.get("/datasets/air/edr/") assert response.status_code == 200 data = response.json() assert data["id"] == "air" assert "position" in data["data_queries"] assert "area" in data["data_queries"] assert "cube" in data["data_queries"] ``` ``` -------------------------------- ### Local Development Environment Setup (Bash/Conda) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md This command demonstrates how to set up a local development environment for the xpublish-edr project using Conda. It mirrors the environment creation process used in the CI, ensuring consistency. This requires Conda to be installed on the local machine. ```bash conda create --name TEST python=3.8 --file requirements.txt --file requirements-dev.txt ``` -------------------------------- ### Travis CI Configuration for xpublish-edr (YAML) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md This YAML configuration defines the continuous integration pipeline for the xpublish-edr project using Travis CI. It includes environment setup, matrix builds for various Python versions and testing types (coding standards, tarball, docs), pre-installation steps using Conda, standard installation, and script execution for tests, packaging, and documentation deployment. Dependencies include Miniconda, pytest, check-manifest, twine, and doctr. ```yaml language: minimal sudo: false env: global: - secure: "TOKEN" matrix: fast_finish: true include: - name: "python-3.6" env: PY=3.6 - name: "python-3.8" env: PY=3.8 - name: "coding_standards" env: PY=3 - name: "tarball" env: PY=3 - name: "docs" env: PY=3 before_install: # Install miniconda and create TEST env. - | wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh bash miniconda.sh -b -p $HOME/miniconda export PATH="$HOME/miniconda/bin:$PATH" conda config --set always_yes yes --set changeps1 no --set show_channel_urls true conda update --quiet conda conda config --add channels conda-forge --force conda config --set channel_priority strict conda create --name TEST python=$PY --file requirements.txt --file requirements-dev.txt source activate TEST conda info --all install: - pip install -e . --no-deps --force-reinstall script: - if [[ $TRAVIS_JOB_NAME == python-* ]]; then cp -r tests/ /tmp ; pushd /tmp && pytest -n 2 -rxs --cov=xpublish_edr tests && popd ; fi - if [[ $TRAVIS_JOB_NAME == 'tarball' ]]; then pip wheel . -w dist --no-deps ; check-manifest --verbose ; twine check dist/* ; fi - if [[ $TRAVIS_JOB_NAME == 'coding_standards' ]]; then pytest --flake8 -m flake8 ; fi - | if [[ $TRAVIS_JOB_NAME == 'docs' ]]; then set -e cp notebooks/tutorial.ipynb docs/source/ pushd docs make clean html linkcheck popd if [[ -z "$TRAVIS_TAG" ]]; then python -m doctr deploy --build-tags --key-path github_deploy_key.enc --built-docs docs/_build/html dev else python -m doctr deploy --build-tags --key-path github_deploy_key.enc --built-docs docs/_build/html "version-$TRAVIS_TAG" python -m doctr deploy --build-tags --key-path github_deploy_key.enc --built-docs docs/_build/html . fi fi ``` -------------------------------- ### Install and Run Pre-commit Hooks (CLI) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md Commands to install pre-commit hooks locally, run all hooks on all files in the project, and bypass pre-commit checks during a git commit. These are essential for setting up and utilizing pre-commit in a development workflow. ```default pre-commit install pre-commit run --all-files git commit xpublish_edr/some-dot-pwhy.py --no-verify ``` -------------------------------- ### Package and Build Configuration with setup.cfg Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md The setup.cfg file configures package metadata, installation options, and tool-specific settings for building and checking the Python package. It specifies package name, description, author details, classifiers, dependencies, and exclusion patterns for source distributions. ```cfg [metadata] name = xpublish_edr description = My Awesome module author = AUTHOR NAME author_email = AUTHOR@EMAIL.COM url = https://github.com/ioos/ioos-python-package-skeleton long_description = file: README.md long_description_content_type = text/markdown license = BSD-3-Clause license_file = LICENSE.txt classifiers = Development Status :: 5 - Production/Stable Intended Audience :: Science/Research Operating System :: OS Independent License :: OSI Approved :: BSD License Programming Language :: Python Programming Language :: Python :: 3 Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 Programming Language :: Python :: 3.8 Topic :: Scientific/Engineering [options] zip_safe = False install_requires = numpy requests python_requires = >=3.6 packages = find: [sdist] formats = gztar [check-manifest] ignore = *.yml *.yaml .coveragerc docs docs/* *.enc notebooks notebooks/* tests tests/* [flake8] max-line-length = 105 select = C,E,F,W,B,B950 ignore = E203, E501, W503 exclude = xpublish_edr/_version.py ``` -------------------------------- ### LICENSE File Content Example (Text) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Provides a standard BSD-style license text for software distribution. It outlines the terms under which the software can be used, modified, and redistributed, including disclaimers of warranty. ```text Copyright 2017 AUTHOR NAME Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ``` -------------------------------- ### Travis CI Installation Steps (YAML) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Configures the `before_install` section of a Travis CI build. This involves downloading and setting up Miniconda, creating a specific Conda environment (TEST) with specified Python version and dependencies from `requirements.txt` and `requirements-dev.txt`, and activating the environment. ```yaml before_install: # Install miniconda and create TEST env. - | wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh bash miniconda.sh -b -p $HOME/miniconda export PATH="$HOME/miniconda/bin:$PATH" conda config --set always_yes yes --set changeps1 no --set show_channel_urls true conda update --quiet conda conda config --add channels conda-forge --force conda config --set channel_priority strict conda create --name TEST python=$PY --file requirements.txt --file requirements-dev.txt source activate TEST conda info --all install: - pip install -e . --no-deps --force-reinstall ``` -------------------------------- ### Project Dependencies Example (Text) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb A sample `requirements.txt` file listing project dependencies. It includes version specifiers and conditional dependencies based on Python version. Comments indicate potential conda-specific dependencies. ```text OWSLib>=0.8.3 Jinja2>=2.7.3 functools32==3.2.3-2; python_version < '3.2' #conda: libnetcdf #conda: libgdal ``` -------------------------------- ### Project Structure Example Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Illustrates a typical directory structure for a Python package, including directories for documentation, tests, and the package source code itself. It highlights the common practice of placing tests outside the main module directory. ```tree |-docs | |-source | | |-_static | |-build |-tests |-ioos_pkg_skeleton ``` -------------------------------- ### Error Handling Examples Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Demonstrates how to handle common API errors such as invalid coordinates, incorrect parameter names, and unsupported data formats. ```APIDOC ## Error Handling Examples ### Description This section provides examples of how to catch and interpret common error responses from the API when making requests with invalid parameters or formats. ### Method GET ### Endpoint `/datasets/{dataset_id}/edr/position` (examples use `/datasets/air/edr/position`) ### Error Scenarios #### Invalid WKT coordinates - **Status Code:** 422 - **Response Body:** `{"detail": "Could not parse coordinates to geometry, check the format of the 'coords' query parameter"}` - **Request Example:** ```python import requests base_url = "http://localhost:9000" response = requests.get( f"{base_url}/datasets/air/edr/position", params={"coords": "(71, 41)"} # Invalid WKT ) if response.status_code == 422: print(f"Invalid geometry: {response.json()['detail']}") ``` #### Invalid parameter name - **Status Code:** 404 - **Response Body:** `{"detail": "Error selecting from query: Invalid variable: 'temperature'"}` - **Request Example:** ```python import requests base_url = "http://localhost:9000" response = requests.get( f"{base_url}/datasets/air/edr/position", params={"coords": "POINT(204 44)", "parameter-name": "temperature"} ) if response.status_code == 404: print(f"Error: {response.json()['detail']}") ``` #### Invalid format - **Status Code:** 404 - **Response Body:** `{"detail": "invalid_format is not a valid format for EDR position queries. Get `./position/formats` for valid formats"}` - **Request Example:** ```python import requests base_url = "http://localhost:9000" response = requests.get( f"{base_url}/datasets/air/edr/position", params={"coords": "POINT(204 44)", "f": "invalid_format"} ) if response.status_code == 404: print(f"Format error: {response.json()['detail']}") ``` ``` -------------------------------- ### Calculate Meaning of Life with ioos_pkg_skeleton Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/tutorial.ipynb This Python code snippet demonstrates how to use the `meaning_of_life` function from the `ioos_pkg_skeleton` library to calculate a result based on an input integer. It imports the necessary library and calls the function. ```python import ioos_pkg_skeleton ioos_pkg_skeleton.meaning_of_life(3) ``` -------------------------------- ### GET /edr/position/formats Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Retrieve a list of supported output formats for position queries. This endpoint helps users identify available data export formats. ```APIDOC ## GET /edr/position/formats ### Description Retrieve list of supported output formats for position queries. ### Method GET ### Endpoint `/edr/position/formats` ### Response #### Success Response (200) - **cf_covjson** (string) - Description for CoverageJSON format. - **csv** (string) - Description for CSV format. - **geojson** (string) - Description for GeoJSON format. - **nc** (string) - Description for NetCDF format. - **parquet** (string) - Description for Parquet format. #### Response Example ```json { "cf_covjson": "Transform an xarray dataset to CoverageJSON using CF conventions", "csv": "Convert xarray Dataset to CSV format", "geojson": "Transform vectorized xarray dataset to GeoJSON", "nc": "Export xarray Dataset as NetCDF file", "parquet": "Convert xarray Dataset to Parquet format" } ``` ``` -------------------------------- ### Get Supported Output Formats (bash) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Retrieve a list of supported output formats for position queries via a curl command. The response is a JSON object mapping format names to their descriptions. ```bash curl http://localhost:9000/edr/position/formats # Response: { "cf_covjson": "Transform an xarray dataset to CoverageJSON using CF conventions", "csv": "Convert xarray Dataset to CSV format", "geojson": "Transform vectorized xarray dataset to GeoJSON", "nc": "Export xarray Dataset as NetCDF file", "parquet": "Convert xarray Dataset to Parquet format" } ``` -------------------------------- ### Retrieve Meaning of Life Information via URL with ioos_pkg_skeleton Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/tutorial.ipynb This Python code snippet shows how to retrieve information related to the 'meaning of life' using the `meaning_of_life_url` function from the `ioos_pkg_skeleton` library. It involves importing the library and calling the function, which returns a descriptive string. ```python import ioos_pkg_skeleton ioos_pkg_skeleton.meaning_of_life_url() ``` -------------------------------- ### GET /datasets/ensemble/edr/position Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Retrieves data subsets from ensemble datasets based on specified coordinates, datetime, band, and ensemble member range. ```APIDOC ## GET /datasets/ensemble/edr/position ### Description This endpoint allows users to select specific data based on spatial coordinates, time, a particular band, and a range of ensemble members. It's useful for analyzing specific slices of ensemble forecast data. ### Method GET ### Endpoint `/datasets/ensemble/edr/position` ### Parameters #### Query Parameters - **coords** (string) - Required - The geographic coordinates in WKT format (e.g., `POINT(-120 40)`). - **datetime** (string) - Required - The specific date and time for the data query (e.g., `2023-01-01`). - **band** (string) - Required - The identifier for the specific band of data to retrieve (e.g., `3`). - **member** (string) - Required - The range of ensemble members to include (e.g., `5/10`). - **f** (string) - Optional - The desired output format for the data (e.g., `nc` for NetCDF). ### Request Example ```python import requests response = requests.get( "http://localhost:9000/datasets/ensemble/edr/position", params={ "coords": "POINT(-120 40)", "datetime": "2023-01-01", "band": "3", "member": "5/10", "f": "nc" } ) ``` ### Response #### Success Response (200) - Returns NetCDF data for the specified band and ensemble members. ``` -------------------------------- ### Setup Test Client with xpublish and CfEdrPlugin Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt This snippet shows how to set up a testing environment for xpublish-edr using pytest and FastAPI's TestClient. It initializes an xpublish Rest instance with the CfEdrPlugin and sample data, then defines two test functions to verify position queries and metadata retrieval via the API. Dependencies include 'pytest', 'xpublish', 'fastapi.testclient', 'xpublish_edr', and 'cf_xarray.datasets'. ```python import pytest import xpublish from fastapi.testclient import TestClient from xpublish_edr import CfEdrPlugin from cf_xarray.datasets import airds @pytest.fixture def test_client(): # Create xpublish instance with EDR plugin rest = xpublish.Rest( {"air": airds}, plugins={"edr": CfEdrPlugin()} ) return TestClient(rest.app) def test_position_query(test_client): response = test_client.get( "/datasets/air/edr/position?coords=POINT(204 44)&f=csv" ) assert response.status_code == 200 assert "air" in response.text assert "lat,lon" in response.text def test_metadata(test_client): response = test_client.get("/datasets/air/edr/") assert response.status_code == 200 data = response.json() assert data["id"] == "air" assert "position" in data["data_queries"] assert "area" in data["data_queries"] assert "cube" in data["data_queries"] ``` -------------------------------- ### Position Query - Multiple Points (bash) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Query data at multiple geographic locations simultaneously using a curl command. This example retrieves data at multiple points in CSV format for the 'air' parameter. ```bash # Get data at multiple points in CSV format curl "http://localhost:9000/datasets/air/edr/position?coords=MULTIPOINT((202%2043),(205%2045))&f=csv¶meter-name=air" # Response (CSV): time,lat,lon,air 2013-01-01T00:00:00,43.0,202.0,241.2 2013-01-01T06:00:00,43.0,202.0,242.5 2013-01-01T00:00:00,45.0,205.0,243.1 2013-01-01T06:00:00,45.0,205.0,244.0 ``` -------------------------------- ### GET /datasets/{dataset_id}/edr/ Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Retrieve OGC EDR collection metadata for a specific dataset. This includes spatial and temporal extents, available parameters, and query capabilities. ```APIDOC ## GET /datasets/{dataset_id}/edr/ ### Description Retrieve OGC EDR collection metadata for a specific dataset including extents, parameters, and query capabilities. ### Method GET ### Endpoint `/datasets/{dataset_id}/edr/` ### Parameters #### Path Parameters - **dataset_id** (string) - Required - The identifier of the dataset to retrieve metadata for. ### Response #### Success Response (200) - **id** (string) - The unique identifier of the dataset. - **title** (string) - The title of the dataset. - **description** (string) - A detailed description of the dataset. - **extent** (object) - Spatial and temporal extents of the dataset. - **spatial** (object) - **bbox** (array) - Bounding box coordinates [[min_lon, min_lat, max_lon, max_lat]]. - **crs** (string) - Coordinate reference system identifier. - **temporal** (object) - **interval** (array) - Temporal interval [start_time, end_time]. - **values** (array) - List of available temporal values or ranges. - **crs** (array) - List of supported Coordinate Reference Systems. - **output_formats** (array) - List of supported output formats. - **data_queries** (object) - Available data query types (position, area, cube). - **parameter_names** (array) - List of parameter names available in the dataset. #### Response Example ```json { "id": "air", "title": "4x daily NMC reanalysis (1948)", "description": "Data is from NMC initialized reanalysis\n(4x/day)...", "extent": { "spatial": { "bbox": [[200.0, 15.0, 322.5, 75.0]], "crs": "EPSG:4326" }, "temporal": { "interval": ["2013-01-01T00:00:00", "2013-01-01T18:00:00"], "values": ["2013-01-01T00:00:00/2013-01-01T18:00:00"] } }, "crs": ["EPSG:4326"], "output_formats": ["cf_covjson", "csv", "geojson", "nc", "parquet"], "data_queries": { "position": {...}, "area": {...}, "cube": {...} }, "parameter_names": ["air"] } ``` ``` -------------------------------- ### Position Query - Single Point (bash) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Extract data at a specific geographic point using WKT POINT format via a curl command. This example retrieves temperature at a single point for all time steps and returns data in CoverageJSON format. ```bash # Get temperature at a single point, all time steps curl "http://localhost:9000/datasets/air/edr/position?coords=POINT(204%2044)" # Response (CoverageJSON): { "type": "Coverage", "domain": { "type": "Domain", "domainType": "Grid", "axes": { "x": {"values": [205.0]}, "y": {"values": [45.0]}, "t": {"values": ["2013-01-01T00:00:00", "2013-01-01T06:00:00", ...] } }, "parameters": { "air": { "type": "Parameter", "description": {"en": "Air temperature"}, "unit": {"label": {"en": "degK"}}, "observedProperty": {"id": "air_temperature"} } }, "ranges": { "air": { "type": "NdArray", "dataType": "float", "values": [241.2, 242.5, 243.1, 244.0] } } } ``` -------------------------------- ### GET /datasets/{dataset_id}/edr/position Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Perform a position query to extract data at specific geographic coordinates. Supports single points, multiple points, time ranges, and interpolation. ```APIDOC ## GET /datasets/{dataset_id}/edr/position ### Description Extract data at a specific geographic point or multiple points. Supports filtering by time, interpolation methods, and selecting specific parameters. ### Method GET ### Endpoint `/datasets/{dataset_id}/edr/position` ### Parameters #### Path Parameters - **dataset_id** (string) - Required - The identifier of the dataset. #### Query Parameters - **coords** (string) - Required - Well-Known Text (WKT) format for coordinates (e.g., `POINT(lon lat)` or `MULTIPOINT((lon1 lat1),(lon2 lat2))`). - **datetime** (string) - Optional - Specifies the time range or specific timestamp (e.g., `YYYY-MM-DDTHH:MM:SS/YYYY-MM-DDTHH:MM:SS` or a single timestamp). - **method** (string) - Optional - Interpolation method to use if `datetime` specifies a range (e.g., `linear`). - **f** (string) - Optional - Desired output format (e.g., `cf_covjson`, `csv`). Defaults to `cf_covjson`. - **parameter-name** (string or array) - Optional - The name(s) of the parameter(s) to retrieve. ### Response #### Success Response (200) The response format depends on the `f` query parameter. Examples include CoverageJSON, CSV, GeoJSON, etc. #### Response Example (CoverageJSON for single point) ```json { "type": "Coverage", "domain": { "type": "Domain", "domainType": "Grid", "axes": { "x": {"values": [205.0]}, "y": {"values": [45.0]}, "t": {"values": ["2013-01-01T00:00:00", "2013-01-01T06:00:00", ...] } } }, "parameters": { "air": { "type": "Parameter", "description": {"en": "Air temperature"}, "unit": {"label": {"en": "degK"}}, "observedProperty": {"id": "air_temperature"} } }, "ranges": { "air": { "type": "NdArray", "dataType": "float", "values": [241.2, 242.5, 243.1, 244.0] } } } ``` #### Response Example (CSV for multiple points) ```csv time,lat,lon,air 2013-01-01T00:00:00,43.0,202.0,241.2 2013-01-01T06:00:00,43.0,202.0,242.5 2013-01-01T00:00:00,45.0,205.0,243.1 2013-01-01T06:00:00,45.0,205.0,244.0 ``` #### Response Example (CSV with interpolation) ```csv time,lat,lon,air 2013-01-01T00:00:00,44.5,204.5,241.75 2013-01-01T06:00:00,44.5,204.5,242.85 2013-01-01T12:00:00,44.5,204.5,243.25 ``` ``` -------------------------------- ### Query with Custom Dimensions (Python) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Shows how to query data along arbitrary dimensions beyond the standard time, latitude, longitude, and elevation. This example assumes a dataset with dimensions including 'member'. ```python import requests # Dataset has dimensions: time, lat, lon, band, member ``` -------------------------------- ### Position Query with Time Range and Interpolation (bash) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Select data for a specific time period using linear interpolation via a curl command. This example retrieves interpolated temperature at exact coordinates for a given date range and returns data in CSV format. ```bash # Get interpolated temperature at exact coordinates for date range curl "http://localhost:9000/datasets/air/edr/position?coords=POINT(204.5%2044.5)&datetime=2013-01-01T00:00:00/2013-01-01T12:00:00&method=linear&f=csv" # Response: CSV with linearly interpolated values at exact coordinates time,lat,lon,air 2013-01-01T00:00:00,44.5,204.5,241.75 2013-01-01T06:00:00,44.5,204.5,242.85 2013-01-01T12:00:00,44.5,204.5,243.25 ``` -------------------------------- ### Query Specific Ensemble Member and Band in NetCDF Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt This snippet demonstrates how to query a specific band and an ensemble member range from a dataset using the xpublish-edr API. It makes a GET request and saves the NetCDF response to a file. Dependencies include the 'requests' library. ```python import requests response = requests.get( "http://localhost:9000/datasets/ensemble/edr/position", params={ "coords": "POINT(-120 40)", "datetime": "2023-01-01", "band": "3", "member": "5/10", # Range notation for member dimension "f": "nc" } ) # Returns NetCDF with data for band=3, members 5-10 with open("ensemble_subset.nc", "wb") as f: f.write(response.content) ``` -------------------------------- ### Handle API Errors for Invalid Coordinates, Parameters, and Formats Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt This Python code illustrates how to handle common API errors when querying xpublish-edr. It shows specific examples for invalid WKT coordinates, non-existent parameter names, and unsupported output formats, checking the response status code and extracting error messages from the JSON response. Dependencies include the 'requests' library. ```python import requests base_url = "http://localhost:9000" # Invalid WKT coordinates response = requests.get( f"{base_url}/datasets/air/edr/position", params={"coords": "(71, 41)"} ) if response.status_code == 422: print(f"Invalid geometry: {response.json()['detail']}") # Output: "Could not parse coordinates to geometry, check the format of the 'coords' query parameter" # Invalid parameter name response = requests.get( f"{base_url}/datasets/air/edr/position", params={"coords": "POINT(204 44)", "parameter-name": "temperature"} ) if response.status_code == 404: print(f"Error: {response.json()['detail']}") # Output: "Error selecting from query: Invalid variable: 'temperature'" # Invalid format response = requests.get( f"{base_url}/datasets/air/edr/position", params={"coords": "POINT(204 44)", "f": "invalid_format"} ) if response.status_code == 404: print(f"Format error: {response.json()['detail']}") # Output: "invalid_format is not a valid format for EDR position queries. Get `./position/formats` for valid formats" ``` -------------------------------- ### Get Collection Metadata (bash) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Retrieve OGC EDR collection metadata for a specific dataset using a curl command. The response includes dataset ID, title, description, spatial and temporal extents, CRS, supported output formats, query capabilities, and parameter names. ```bash curl http://localhost:9000/datasets/air/edr/ # Response includes: { "id": "air", "title": "4x daily NMC reanalysis (1948)", "description": "Data is from NMC initialized reanalysis\n(4x/day)...", "extent": { "spatial": { "bbox": [[200.0, 15.0, 322.5, 75.0]], "crs": "EPSG:4326" }, "temporal": { "interval": ["2013-01-01T00:00:00", "2013-01-01T18:00:00"], "values": ["2013-01-01T00:00:00/2013-01-01T18:00:00"] } }, "crs": ["EPSG:4326"], "output_formats": ["cf_covjson", "csv", "geojson", "nc", "parquet"], "data_queries": { "position": {...}, "area": {...}, "cube": {...} }, "parameter_names": ["air"] } ``` -------------------------------- ### Simplified setup.py with setuptools-scm (Python) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md A minimal `setup.py` file leveraging `setuptools-scm` for automated version management. It configures `setuptools` to generate the package version from source control tags and writes it to a specified file. ```python from setuptools import setup setup( use_scm_version={ "write_to": "xpublish_edr/_version.py", "write_to_template": '__version__ = "{version}" ', "tag_regex": r"^(?Pv)?(?P[^\+]+)(?P.*)?$", } ) ``` -------------------------------- ### Setuptools Configuration with SCM Versioning (Python) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Configures setuptools for package building, specifically integrating with Source Code Management (SCM) for versioning. It defines how the package version is written to a file and the regular expression used to parse version tags. ```python from setuptools import setup setup( use_scm_version={ "write_to": "ioos_pkg_skeleton/_version.py", "write_to_template": '__version__ = "{version}"', "tag_regex": r"^(?Pv)?(?P[^\+]+)(?P.*)?$", } ) ``` -------------------------------- ### Development Dependencies with requirements-dev.txt Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md The requirements-dev.txt file lists all dependencies necessary for development, including tools for code styling, testing, documentation generation, and building the package. This ensures a consistent development environment. ```default # code style/consistency black flake8 flake8-builtins flake8-comprehensions flake8-mutable flake8-print isort pylint pytest-flake8 # checks and tests check-manifest pytest pytest-cov pytest-xdist pre-commit # documentation doctr nbsphinx sphinx # build setuptools_scm twine wheel ``` -------------------------------- ### Travis CI: Build Documentation Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Sets up Travis CI to build project documentation when the job name is 'docs'. It copies a tutorial notebook, navigates to the docs directory, cleans and builds HTML documentation, and deploys it using 'doctr' based on whether a tag is present. ```yaml - | if [[ $TRAVIS_JOB_NAME == 'docs' ]]; then set -e cp notebooks/tutorial.ipynb docs/source/ pushd docs make clean html linkcheck popd if [[ -z "$TRAVIS_TAG" ]]; then python -m doctr deploy --build-tags --key-path github_deploy_key.enc --built-docs docs/_build/html dev else python -m doctr deploy --build-tags --key-path github_deploy_key.enc --built-docs docs/_build/html "version-$TRAVIS_TAG" python -m doctr deploy --build-tags --key-path github_deploy_key.enc --built-docs docs/_build/html . fi fi ``` -------------------------------- ### Pre-commit Configuration (.pre-commit-config.yaml) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md Configuration file for pre-commit, specifying various hooks from different repositories to enforce code style, check for errors, and manage dependencies. It includes hooks for trailing whitespace, AST checks, debug statements, end-of-file fixing, docstring validation, large file detection, requirements file fixing, sorting file contents, linting with flake8, sorting imports with isort, generating isort configuration, code formatting with black, and type checking with mypy. ```yaml repos: - repo: https://github.com/pre-commit/pre-commit-hooks rev: v3.1.0 hooks: - id: trailing-whitespace exclude: tests/data - id: check-ast - id: debug-statements - id: end-of-file-fixer - id: check-docstring-first - id: check-added-large-files - id: requirements-txt-fixer - id: file-contents-sorter files: requirements-dev.txt - repo: https://gitlab.com/pycqa/flake8 rev: 3.7.9 hooks: - id: flake8 exclude: docs/source/conf.py args: [--max-line-length=105, --ignore=E203,E501,W503, --select=select=C,E,F,W,B,B950] - repo: https://github.com/pre-commit/mirrors-isort rev: v4.3.21 hooks: - id: isort additional_dependencies: [toml] args: [--project=xpublish_edr, --multi-line=3, --lines-after-imports=2, --lines-between-types=1, --trailing-comma, --force-grid-wrap=0, --use-parentheses, --line-width=88] - repo: https://github.com/asottile/seed-isort-config rev: v2.1.1 hooks: - id: seed-isort-config - repo: https://github.com/psf/black rev: stable hooks: - id: black language_version: python3 - repo: https://github.com/pre-commit/mirrors-mypy rev: v0.770 hooks: - id: mypy exclude: docs/source/conf.py args: [--ignore-missing-imports] ``` -------------------------------- ### Source Distribution Manifest with MANIFEST.in Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md The MANIFEST.in file specifies additional files to be included in the source distribution (sdist) of the package. This is crucial for ensuring that all necessary files, such as README, LICENSE, and Python modules, are packaged correctly. ```default include *.txt include LICENSE # Please consider the Windows users and use .txt include README.md recursive-include xpublish_edr *.py ``` -------------------------------- ### Development Dependencies: requirements-dev.txt Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Lists the development dependencies required for the xpublish-edr project. This file includes tools for testing, linting, formatting, documentation, and packaging. ```text black check-manifest doctr flake8 flake8-builtins flake8-comprehensions flake8-mutable flake8-print isort nbsphinx pre-commit pylint pytest pytest-cov pytest-flake8 pytest-xdist setuptools_scm sphinx twine wheel ``` -------------------------------- ### PEP 518 Build System Requirements (TOML) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md Specifies the build system requirements for a Python package using PEP 518. This TOML file declares dependencies needed for building the package and the build backend to be used, typically `setuptools.build_meta`. ```toml [build-system] requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4", "cython", "numpy"] build-backend = "setuptools.build_meta" ``` -------------------------------- ### Pre-commit Configuration: .isort.cfg Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Configuration file for the 'isort' tool, specifying third-party libraries that should be recognized and grouped separately during import sorting. ```ini [settings] known_third_party = numpy,pytest,requests,setuptools ``` -------------------------------- ### Dependency Management with requirements.txt Source: https://github.com/xpublish-community/xpublish-edr/blob/main/docs/source/how2package4ioos.md The requirements.txt file lists the Python and non-Python dependencies required for the project. It supports both pip and conda dependencies, with conda dependencies indicated by a '#conda:' prefix. A separate requirements-dev.txt file is recommended for development-specific dependencies. ```default numpy requests #conda: libnetcdf #conda: libgdal ``` -------------------------------- ### GitHub Actions: Pre-commit Workflow Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Defines a GitHub Actions workflow to run pre-commit hooks on pull requests and pushes to the master branch. It checks out code, sets up Python, caches pre-commit dependencies, and executes the pre-commit action. ```yaml name: pre-commit on: pull_request: push: branches: [master] jobs: pre-commit: runs-on: ubuntu-latest steps: - uses: actions/checkout@v1 - uses: actions/setup-python@v1 - name: set PY run: echo "::set-env name=PY::$(python --version --version | sha256sum | cut -d' ' -f1)" - uses: actions/cache@v1 with: path: ~/.cache/pre-commit key: pre-commit|${{ env.PY }}|${{ hashFiles('.pre-commit-config.yaml') }} - uses: pre-commit/action@v1.0.0 ``` -------------------------------- ### Travis CI Build Matrix Configuration (YAML) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Defines the build matrix for Travis CI, specifying different Python versions and job types (e.g., python-3.6, python-3.8, coding_standards, tarball, docs) to be tested. It uses environment variables to control the build process. ```yaml language: minimal sudo: false env: global: - secure: "TOKEN" matrix: fast_finish: true include: - name: "python-3.6" env: PY=3.6 - name: "python-3.8" env: PY=3.8 - name: "coding_standards" env: PY=3 - name: "tarball" env: PY=3 - name: "docs" env: PY=3 ``` -------------------------------- ### Travis CI Test Execution Script (YAML) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Defines the `script` section for Travis CI builds. It executes tests using pytest for Python-specific jobs, including code coverage analysis. For 'tarball' jobs, it focuses on building wheels, checking manifest, and validating the distribution with twine. ```yaml script: - if [[ $TRAVIS_JOB_NAME == python-* ]]; then cp -r tests/ /tmp ; pushd /tmp && pytest -n 2 -rxs --cov=ioos_pkg_skeleton tests && popd ; fi - if [[ $TRAVIS_JOB_NAME == 'tarball' ]]; then pip wheel . -w dist --no-deps ; check-manifest --verbose ; twine check dist/* ; fi ``` -------------------------------- ### Travis CI: Test Coding Standards Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Configures Travis CI to run pytest with the flake8 marker when the job name is 'coding_standards'. This is intended for basic code style checks. ```yaml script: - if [[ $TRAVIS_JOB_NAME == 'coding_standards' ]]; then pytest --flake8 -m flake8 ; fi ``` -------------------------------- ### Custom Format Registration (Python) Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Illustrates how to extend xpublish-edr with custom output formats by defining a formatter function in a Python package and registering it via `pyproject.toml` entry points. ```python # In your custom package, create a formatter function # mypackage/formats.py import xarray as xr def to_custom_format(ds: xr.Dataset) -> dict: """Export dataset in custom JSON format""" return { "data": ds.to_dict(), "metadata": { "variables": list(ds.data_vars), "dims": dict(ds.dims) } } # Register via pyproject.toml entry points # [project.entry-points.xpublish_edr_position_formats] # custom = "mypackage.formats:to_custom_format" # Now available via API: # curl "http://localhost:9000/datasets/air/edr/position?coords=POINT(204%2044)&f=custom" ``` -------------------------------- ### MANIFEST.in File for Package Inclusion (Text) Source: https://github.com/xpublish-community/xpublish-edr/blob/main/notebooks/IOOS-Python-Package-Skeleton.ipynb Specifies additional files to include in the Python package distribution beyond the source code. This includes text files, licenses, and recursively includes all Python files within a specific directory. ```text include *.txt include LICENSE # Please consider the Windows users and use .txt include README.md recursive-include ioos_pkg_skeleton *.py ``` -------------------------------- ### Plugin Registration Source: https://context7.com/xpublish-community/xpublish-edr/llms.txt Register the CfEdrPlugin with xpublish to enable EDR endpoints. This involves loading a dataset and initializing the xpublish.Rest server with the plugin. ```APIDOC ## Plugin Registration Register the CfEdrPlugin with xpublish to enable EDR endpoints. ```python import xpublish import xarray as xr from xpublish_edr import CfEdrPlugin # Load your CF-compliant dataset ds = xr.tutorial.open_dataset("air_temperature") # Create REST API with EDR plugin rest = xpublish.Rest( {"air": ds}, plugins={"edr": CfEdrPlugin()} ) # Start the server rest.serve() ``` ```