### Setup Your Own Data Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst A guide on how to set up and use your own custom climate data with the climpred library. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('custom_data') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### List Available Example Datasets Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Displays the names of all example datasets available for loading within the climpred library. ```python load_dataset() ``` -------------------------------- ### Install herbie-data Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/Herbie.ipynb Installs the herbie-data library quietly. ```python !pip install herbie-data --quiet ``` -------------------------------- ### Set up Conda Environment and Install Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Commands to create a new conda environment using a provided yml file, activate it, and then perform an editable install of climpred. ```bash conda env create -f ci/requirements/climpred-dev.yml conda activate climpred-dev pip install -e . ``` -------------------------------- ### Install Dependencies Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Installs necessary Python packages for data analysis and forecast verification using conda and pip. ```conda #!conda install intake fsspec intake-xarray intake-thredds -c conda-forge -y ``` ```conda #!conda install eccodes cfgrib -c conda-forge -y ``` ```pip #!pip install climetlab --quiet ``` -------------------------------- ### Install climpred-dev Environment Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst This snippet shows how to create a conda development environment for climpred, ensuring all necessary dependencies are installed for running the examples. ```bash conda env create -f ci/requirements/climpred-dev.yml ``` -------------------------------- ### Import Libraries and Load Dataset Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Imports necessary libraries such as matplotlib, xarray, and climpred, and loads an example dataset. ```python %matplotlib inline import matplotlib.pyplot as plt import xarray as xr from climpred import HindcastEnsemble from climpred.tutorial import load_dataset import climpred ``` -------------------------------- ### Install and Use pre-commit Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Instructions for installing pre-commit and its hooks into the climpred repository. pre-commit automates code beautification, maintainability checks, and syntax error detection. ```bash pip install --user pre-commit pre-commit install ``` -------------------------------- ### Install Dependencies Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Installs necessary Python packages for data handling and analysis, including intake, fsspec, intake-xarray, and climetlab. ```python #!conda install intake fsspec intake-xarray -c conda-forge -y ``` ```python #!pip install climetlab #!pip install climetlab_s2s_ai_challenge ``` -------------------------------- ### Install climpred from GitHub (clone) Source: https://github.com/pangeo-data/climpred/blob/main/README.rst Installs the bleeding-edge version of climpred by cloning the repository from GitHub, navigating into the directory, and installing locally. ```bash git clone https://github.com/pangeo-data/climpred.git cd climpred python -m pip install . --upgrade ``` -------------------------------- ### Load CESM-DP-SST Dataset Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Loads the 'CESM-DP-SST' example dataset, which is a sample dataset for climate prediction analysis. ```python initialized = climpred.tutorial.load_dataset("CESM-DP-SST") ``` -------------------------------- ### Install climpred from GitHub (local clone) Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/index.rst Installs the bleeding-edge version of climpred by cloning the repository and installing locally. ```bash git clone https://github.com/pangeo-data/climpred.git cd climpred pip install . --upgrade ``` -------------------------------- ### climpred Documentation: Setup Your Own Raw Model Output Source: https://github.com/pangeo-data/climpred/blob/main/CHANGELOG.rst A new demo has been added to guide users on setting up their own raw model output that is compliant with climpred. This includes instructions for using `intake-esm` with climpred for CMORized output. ```python # Demo: Setup your own raw model output compliant to climpred # Link: examples/misc/setup_your_own_data.html # Demo: Using intake-esm with climpred for CMORized output # Link: examples/misc/setup_your_own_data.html#intake-esm-for-cmorized-output ``` -------------------------------- ### climpred Initialization and Setup Source: https://github.com/pangeo-data/climpred/blob/main/CHANGELOG.rst This snippet demonstrates the basic setup for using climpred, including importing necessary libraries and configuring matplotlib for plotting. It shows how to initialize the environment for climpred operations. ```python import climpred from climpred import HindcastEnsemble import matplotlib as mpl mpl.rcdefaults() mpl.use("Agg") # cut border when saving (for maps) mpl.rcParams["savefig.bbox"] = "tight" ``` -------------------------------- ### Install climpred from GitHub (direct) Source: https://github.com/pangeo-data/climpred/blob/main/README.rst Installs the bleeding-edge version of climpred directly from its GitHub repository using pip. ```bash pip install git+https://github.com/pangeo-data/climpred.git ``` -------------------------------- ### DCPP Dataset Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/initialized-datasets.rst Demonstrates how to use the DCPP dataset with climpred, referencing intake-esm for data loading and providing links to example notebooks. ```python with intake-esm ``` -------------------------------- ### Environment Setup for climpred with GPU Support Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/climpred_gpu.ipynb Installs necessary packages for climpred, including GPU-accelerated libraries like tensorflow-gpu, pytorch-gpu, cudatoolkit, and cupy, along with the cupy-xarray integration. This setup is crucial for leveraging GPU capabilities. ```bash #!conda create -n ML_gpu tensorflow-gpu pytorch-gpu xarray dask matplotlib nb_conda_kernels jupyterlab cudatoolkit cupy esmtools climpred -y #!pip install git+https://github.com/jacobtomlinson/cupy-xarray.git ``` -------------------------------- ### NMME Dataset Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/initialized-datasets.rst Shows how to work with the NMME dataset for seasonal predictions using climpred, with a link to seasonal examples. ```python `seasonal SubX `_ ``` -------------------------------- ### Install climpred from GitHub (direct URL) Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/index.rst Installs the bleeding-edge version of climpred directly from its GitHub repository using pip. ```bash pip install git+https://github.com/pangeo-data/climpred.git ``` -------------------------------- ### Get Initialized and Observation Coordinates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Retrieves and displays the coordinates for both the initialized and observation datasets within the HindcastEnsemble. This helps verify data alignment. ```python hindcast.get_initialized().coords hindcast.get_observations().coords ``` -------------------------------- ### NWP GEFS 6h Forecasts Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Demonstrates how to work with GEFS 6-hour forecasts using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('gefs') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### GEFS Dataset Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/initialized-datasets.rst Demonstrates the usage of the GEFS dataset for numerical weather prediction with climpred, linking to a specific example for GEFS NWP. ```python `GEFS NWP `_ ``` -------------------------------- ### SubX Dataset Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/initialized-datasets.rst Provides information on using the SubX dataset for subseasonal predictions with climpred, linking to relevant examples. ```python `subseasonal SubX `_ ``` -------------------------------- ### climpred Examples and Features Source: https://github.com/pangeo-data/climpred/blob/main/CHANGELOG.rst This section details various examples and new features added to the climpred library, including NWP, subseasonal forecasts, GPU acceleration, and handling biweekly aggregates. ```python Add GEFS example _ for numerical weather prediction. (:issue:`602`, :pr:`603`) `Aaron Spring`_. Add subseasonal daily ECMWF example __ using `climetlab `_ to access hindcasts from ECMWF cloud. (:issue:`587`, :pr:`603`) `Aaron Spring`_. Add subseasonal daily S2S example _ accessing `S2S `_ output on `IRIDL `_ with a cookie and working with "on-the-fly" reforecasts with ``hdate`` dimension. (:issue:`588`, :pr:`593`) `Aaron Spring`_. Added example climpred on GPU _. Running :py:meth:`.PerfectModelEnsemble.verify` on GPU with `cupy-xarray `_ finishes 10x faster. (:issue:`592`, :pr:`607`) `Aaron Spring`_. How to work with biweekly aggregates in ``climpred``, see `daily ECMWF example `__. (:issue:`625`, :pr:`630`) `Aaron Spring`_. ``` -------------------------------- ### Herbie NWP Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst An example showcasing the use of climpred with Herbie, likely for NWP data. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('herbie') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### climpred GPU Usage Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Illustrates how to utilize GPU acceleration with climpred for faster computations. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('gpu') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Perfect Model Predictability Demo Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst An example demonstrating perfect model predictability analysis with climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('perfect_model') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Install climpred with pip Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/index.rst Installs the latest release of climpred with all optional dependencies using pip. ```bash pip install climpred[complete] ``` -------------------------------- ### Daily S2S ECMWF Subseasonal Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst An example for processing daily S2S forecasts from ECMWF using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('s2s_ecmwf') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Weekly SubX Subseasonal Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Shows how to analyze weekly subseasonal forecasts from the SubX dataset with climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('subx_weekly') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### CESM-DPLE Dataset Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/initialized-datasets.rst Highlights the use of the CESM-DPLE dataset with climpred, noting its compatibility with standard climpred examples. ```python many standard climpred `examples `_ ``` -------------------------------- ### Seasonal ENSO SubX Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Shows how to analyze seasonal ENSO forecasts from the SubX dataset using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('subx_seasonal') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Daily SubX Subseasonal Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Illustrates how to use climpred for daily subseasonal forecasts from the SubX dataset. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('subx') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Build Documentation Locally Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Commands to update the conda environment and build the documentation locally using make html. The built documentation will be available in the docs/build/ directory. ```bash $ conda env update -f ci/requirements/climpred-dev.yml $ cd docs $ make html ``` -------------------------------- ### Verify Dimension Implications Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst This example demonstrates how to verify dimension implications in climate model outputs using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('verify_dim') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Fork and Clone Repository Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Steps to fork the climpred GitHub repository, clone it locally, and set up the upstream remote. It also includes creating a new branch for bug fixes or features. ```bash git clone git@github.com:YOUR_GITHUB_USERNAME/climpred.git cd climpred git remote add upstream git@github.com:pangeo-data/climpred.git # now, to fix a bug or add feature create your own branch off "main": git checkout -b your-bugfix-feature-branch-name main ``` -------------------------------- ### Significance Testing Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst An example showing how to perform significance testing on climate predictions with climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('significance') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Linting Notebooks Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Loads nb_black and lab_black extensions for code linting within a Jupyter Notebook environment. ```python # linting %load_ext nb_black %load_ext lab_black ``` -------------------------------- ### Monthly ENSO SubX Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Demonstrates climpred usage for monthly ENSO forecasts from the SubX dataset. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('subx_monthly') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Install climpred with conda Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/index.rst Installs the latest release of climpred from the conda-forge channel. ```bash conda install -c conda-forge climpred ``` -------------------------------- ### Load IPython Extensions Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Loads the nb_black and lab_black extensions for enhanced code formatting in IPython environments. ```python # linting %load_ext nb_black %load_ext lab_black ``` -------------------------------- ### Load IPython Extensions Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Loads IPython extensions for code formatting and styling within a Jupyter Notebook environment. ```python # linting %load_ext nb_black %load_ext lab_black ``` -------------------------------- ### S2S Dataset Examples Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/initialized-datasets.rst Illustrates how to access and use the S2S dataset for subseasonal predictions via IRIDL and climetlab, with links to specific examples. ```python `IRIDL `_ ``` ```python `EWC Cloud/climetlab `_ ``` -------------------------------- ### climpred New Example Pages Source: https://github.com/pangeo-data/climpred/blob/main/CHANGELOG.rst New example pages have been added to demonstrate subseasonal-to-seasonal (S2S) prediction using climpred. These examples cover calculating MJO index skill, ENSO skill as a function of lead time and initial month, and seasonal ENSO skill. ```python # Example: Calculate the skill of the MJO index as a function of lead time # Link: examples/subseasonal/daily-subx-example.html # Example: Calculate ENSO skill as a function of initial month vs. lead time # Link: examples/monseas/monthly-enso-subx-example.html ``` -------------------------------- ### Install climpred with pip Source: https://github.com/pangeo-data/climpred/blob/main/README.rst Installs the latest release of the climpred package, including optional dependencies for complete functionality, using pip. ```bash python -m pip install climpred[complete] ``` -------------------------------- ### Daily S2S IRIDL Subseasonal Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Demonstrates working with daily S2S (Subseasonal to Seasonal) forecasts from IRIDL using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('s2s_iridl') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Create Biweekly Observation Aggregates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Generates biweekly aggregates for observation data using a rolling 14-day mean. The time index is adjusted to represent the starting date of each 14-day period. ```python # 14D rolling mean obs_biweekly = obs.rolling(time=14, center=False).mean() obs_biweekly = obs_biweekly.isel(time=slice(13, None)).assign_coords( time=obs.time.isel(time=slice(None, -13)) ) # time represents first day of the biweekly aggregate ``` -------------------------------- ### Tropical Pacific SSTs Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst This example focuses on analyzing tropical Pacific Sea Surface Temperatures (SSTs) using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('ssts') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### Load Black Formatting Extensions Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/Herbie.ipynb Loads nb_black and lab_black extensions for code linting and formatting in notebooks. ```python # linting %load_ext nb_black %load_ext lab_black ``` -------------------------------- ### Diagnose Potential Predictability Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples.rst Demonstrates how to diagnose potential predictability of climate variables using climpred. ```python import climpred.tutorial import xarray as xr # Load example data ds = climpred.tutorial.load_dataset('diagnose_predictability') # Example analysis (replace with actual climpred usage) print(ds) ``` -------------------------------- ### List Available Datasets Source: https://github.com/pangeo-data/climpred/wiki/Sample-Data Retrieves and displays a list of all available example datasets that can be loaded by the climpred library. This function helps users understand the data sources available for analysis. ```python from climpred.loadutils import get_datasets get_datasets() >>> 'MPI-PM-DP-1D': decadal prediction ensemble area averages of SST/SSS/AMO. >>> 'MPI-PM-DP-3D': decadal prediction ensemble lat/lon/time of SST/SSS/AMO. >>> 'MPI-control-1D': area averages for the control run of SST/SSS. >>> 'MPI-control-3D': lat/lon/time for the control run of SST/SSS. >>> 'CESM-DP-SST': decadal prediction ensemble of global mean SSTs. >>> 'CESM-DP-SSS': decadal prediction ensemble of global mean SSS. >>> 'CESM-LE': uninitialized ensemble of global mean SSTs. >>> 'ERSST': observations of global mean SSTs. >>> 'FOSI-SST': reconstruction of global mean SSTs. >>> 'FOSI-SSS': reconstruction of global mean SSS. ``` -------------------------------- ### Create Biweekly Forecast Aggregates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Creates biweekly aggregates for forecast data by calculating the mean over specific lead time ranges (1-14 days, 14-28 days, 28-42 days) and concatenating them. The lead dimension is then updated to represent the start of each biweekly period. ```python # create 14D averages forecast_w12 = forecast.sel(lead=range(1, 14)).mean(dim="lead") forecast_w34 = forecast.sel(lead=range(14, 28)).mean(dim="lead") forecast_w56 = forecast.sel(lead=range(28, 42)).mean(dim="lead") forecast_biweekly = xr.concat([forecast_w12, forecast_w34, forecast_w56], dim="lead") forecast_biweekly["lead"] = [ 1, 14, 28, ] # lead represents first day of biweekly aggregate forecast_biweekly["lead"].attrs["units"] = "days" forecast_biweekly.coords ``` -------------------------------- ### Display Initialized Coordinates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Prints the coordinates of the initialized dataset. This is useful for understanding the structure and dimensions of the forecast data. ```python initialized.coords ``` -------------------------------- ### Git Commit and Push Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Standard commands for committing changes locally and pushing them to the remote repository. ```shell $ git commit -a -m "" $ git push -u ``` -------------------------------- ### Load Dataset Function Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/api/climpred.tutorial.load_dataset.rst The `load_dataset` function from the `climpred.tutorial` module is used to load a dataset. It is part of the tutorial section, suggesting its use for examples and learning. ```python import climpred.tutorial dataset = climpred.tutorial.load_dataset() ``` -------------------------------- ### Install climpred with conda Source: https://github.com/pangeo-data/climpred/blob/main/README.rst Installs the latest release of the climpred package from the conda-forge channel using conda. ```bash conda install -c conda-forge climpred ``` -------------------------------- ### Prepare Initial Forecast Data Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/Herbie.ipynb Takes the first forecast from the downloaded dataset and selects the 't2m' variable, preparing it as the initial forecast data. ```python # take the first with multiple members as forecast init = ds[0][["t2m"]] ``` -------------------------------- ### Running ASV Benchmarks Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Execute performance benchmarks using the asv tool. This command runs continuous benchmarking against upstream main and the current HEAD, reporting significant changes. ```shell $ asv continuous -f 1.1 upstream/main HEAD ``` -------------------------------- ### Initialize HindcastEnsemble for Smoothing Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Initializes a `HindcastEnsemble` with daily lead data, adds observations, and computes the ensemble, preparing it for subsequent smoothing operations. ```python fct = ( climpred.HindcastEnsemble(forecast_climetlab.drop_vars("valid_time")) # daily lead .add_observations(obs) .compute() ) ``` -------------------------------- ### HindcastEnsemble Initialization and Verification Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Demonstrates initializing a HindcastEnsemble with forecast data, adding observations, computing the ensemble, and then verifying it using RMSE with specific comparison and alignment parameters. It also shows plotting the results. ```python fct = ( climpred.HindcastEnsemble( forecast_climetlab.drop_vars("valid_time").isel(lead_time=range(4)) ) .add_observations(obs) .compute() ) metric_kwargs = dict( metric="rmse", comparison="e2o", alignment="same_inits", dim="init" ) skill = fct.verify(**metric_kwargs) skill[var].plot(col="lead", robust=True) ``` -------------------------------- ### Check cupy Installation Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/climpred_gpu.ipynb Verifies the installation of the cupy library using the conda package manager. This command is useful for confirming that cupy is available in the current environment. ```bash !conda list cupy ``` -------------------------------- ### Same Verifications Alignment Example Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/alignment.ipynb An example of using the 'same_verifs' alignment strategy in climpred's verify function. This strategy ensures that the verification dates remain constant across all leads. ```python skill = hindcast.verify( metric="acc", comparison="e2o", dim="init", alignment="same_verifs" ) ``` -------------------------------- ### Display Coordinates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Displays the coordinates of the hindcast dataset loaded using climetlab. ```python forecast_climetlab.coords ``` -------------------------------- ### GPU Limitations: Bootstrap Method Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/climpred_gpu.ipynb Demonstrates the usage of the `bootstrap` method within `PerfectModelEnsemble` on the GPU. This example is provided in the context of discussing potential GPU limitations, although the code itself shows a standard operation. ```python pm_gpu.bootstrap(metric="pearson_r", comparison="m2m", dim=["init", "member"], iterations=10)[v] ``` -------------------------------- ### Import Core Libraries Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/setup_your_own_data.ipynb Imports necessary libraries for data manipulation, plotting, and climpred functionality. ```python %matplotlib inline import matplotlib.pyplot as plt import numpy as np import xarray as xr import climpred ``` -------------------------------- ### Define Reforecast Dates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Generates a time series of dates for reforecasts, starting from January 2nd, 2020, with a weekly frequency. ```python dates = xr.cftime_range(start="2020-01-02", freq="7D", end="2020-12-31") ``` -------------------------------- ### Display First Few Rows of Catalog DataFrame Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/setup_your_own_data.ipynb Shows the beginning of the DataFrame after searching the catalog, providing a preview of the selected data. ```python cat.df.head() ``` -------------------------------- ### Create and Add Observations to Hindcast Ensemble Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Creates a HindcastEnsemble object from the initialized data and adds the processed observations to it. This prepares the data for comparative analysis. ```python hindcast = HindcastEnsemble(initialized) hindcast = hindcast.add_observations(obs) hindcast ``` -------------------------------- ### Import Libraries Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Imports core libraries for climate data analysis: climpred for prediction verification and xarray for labeled multi-dimensional array manipulation. ```python import climpred import xarray as xr ``` -------------------------------- ### Running Specific ASV Benchmark Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Execute a specific benchmark test within the ASV suite. This allows for targeted performance testing of individual components. ```shell $ asv continuous -f 1.1 upstream/main HEAD -b benchmarks_PredictionEnsemble.GenerateHindcastEnsembleSmall.time_bootstrap ``` -------------------------------- ### Define Forecast Parameters Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Sets up parameters for retrieving GEFS forecasts, including initialization times, dates, ensemble members, and forecast lead times. ```python inits_time = "0000" # get forecasts started at 00:00 inits = ["20200829", "20200830", "20200831"] # four initial dates members = range(5) # 5 members out of 20 leads = np.arange(0, 6 * 4 * 2 + 1, 6) # 6h lead forecasts, 9 leads upto 48h ``` -------------------------------- ### Import Libraries Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/Herbie.ipynb Imports necessary libraries: xarray for data manipulation, numpy for numerical operations, and climpred for forecast verification. ```python import xarray as xr import numpy as np import climpred # forecast verification ``` -------------------------------- ### Manual PyPI and TestPyPI Upload Source: https://github.com/pangeo-data/climpred/blob/main/HOWTORELEASE.rst Provides the commands to manually build the package and upload it to TestPyPI and PyPI. This is an alternative to the automated GitHub Actions workflow. ```shell $ git clean -xfd # remove any files not checked into git $ python -m build # build package $ python -m twine upload --repository-url https://test.pypi.org/legacy dist/* # register and push to testpypi $ python -m twine upload dist/* # register and push to pypi ``` -------------------------------- ### Generate API Documentation Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Command to generate API documentation using sphinx-autogen. This command should be run from the docs/source directory after adding new functions to api.rst. ```bash sphinx-autogen -o api api.rst ``` -------------------------------- ### Prepare for Forecast Verification Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Prepares the data for forecast verification using the climpred library's `HindcastEnsemble` class. This step involves setting coordinates for lead time. ```python # climpred.tutorial.load_dataset('s2s-ai-challenge') # HindcastEnsemble converts `lead_time` to `lead` due to standard_name 'forecast_period' ``` -------------------------------- ### Adjust Lead Dimension Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-subx-example.ipynb Adjusts the 'lead' dimension in the SubX dataset. SubX uses fractional leads (e.g., 0.5, 1.5), which are converted to integers starting from 0 to represent days since initialization. ```python fcstds["lead"] = (fcstds["lead"] - 0.5).astype("int") ``` -------------------------------- ### Configure climetlab/cdsapi Logging Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Configures the logging level for climetlab or cdsapi. This example sets the logger to ERROR level, suppressing informational messages. ```python # climetlab or cds enable logging.INFO import logging logger = logging.getLogger() logger.setLevel(logging.ERROR) ``` -------------------------------- ### reStructuredText Quick Reference Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Links to external resources for learning reStructuredText (rst), the markup language used for climpred's documentation. ```rst .. _rst-quickref: http://docutils.sourceforge.net/docs/user/rst/quickref.html .. _rst-cheatsheet: https://github.com/ralsina/rst-cheatsheet/blob/master/rst-cheatsheet.rst ``` -------------------------------- ### Tagging a Release Source: https://github.com/pangeo-data/climpred/blob/main/HOWTORELEASE.rst This snippet demonstrates how to tag a new release and push it to the upstream repository. It follows semantic versioning conventions. ```shell $ git tag -a v1.0.0 -m "Version 1.0.0" $ git push upstream main --tags ``` -------------------------------- ### Get Initialized Dates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/alignment.ipynb Prints the initialization dates from the hindcast object. ```python print(f"initialization dates: \n{hindcast.get_initialized().init.to_index()})") ``` -------------------------------- ### Download and Process Forecast Data Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Iterates through specified initializations, leads, and members to download GEFS forecast data using intake-xarray and cfgrib, applying filters for GRIB file parsing. Includes error handling for failed downloads. ```python for init in inits: for lead in leads: for member in members: try: url = f"https://www.ncei.noaa.gov/thredds/fileServer/model-gefs-003/202008/{init}/gens-a_3_{init}_{inits_time}_{str(lead).zfill(3)}_{str(member).zfill(2)}.grb2" # print(f'download init = {init}, lead = {lead}, member = {member}') intake_xarray.NetCDFSource( f"simplecache::{url}", xarray_kwargs=dict( engine="cfgrib", backend_kwargs=dict( filter_by_keys={ "typeOfLevel": "heightAboveGround", "cfVarName": "t2m", } ), ), ).to_dask() except Exception as e: print("failed", type(e).__name__, e) ``` -------------------------------- ### Changelog Entry Format Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Format for creating new entries in the CHANGELOG.rst file, detailing changes, pull request numbers, and author names. ```rst (:pr:`#`) ``_ ``` -------------------------------- ### Open GRIB Files with cfgrib Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Demonstrates how to open a GRIB file using intake-xarray and cfgrib, specifying backend arguments to filter by level type and variable name. This is necessary to correctly parse GRIB data. ```python # how to open grib files: https://github.com/ecmwf/cfgrib/issues/170 intake_xarray.NetCDFSource( "simplecache::https://www.ncei.noaa.gov/thredds/fileServer/model-gefs-003/202008/20200831/gens-a_3_20200831_1800_000_20.grb2", xarray_kwargs=dict( engine="cfgrib", backend_kwargs=dict( filter_by_keys={"typeOfLevel": "heightAboveGround", "shortName": "2t"} ), ), ).to_dask().coords ``` -------------------------------- ### Code Coverage and Reporting Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Measure code coverage during testing and report the results. This helps ensure that new contributions are adequately tested. ```shell coverage run --source climpred -m py.test coverage report coveralls ``` -------------------------------- ### Create HindcastEnsemble and Add Observations Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-subx-example.ipynb Initializes a climpred HindcastEnsemble object with the forecast data and adds the observed data to it. ```python hindcast = HindcastEnsemble(fcstds).add_observations(obsds) hindcast ``` -------------------------------- ### Install climpred Dependencies Source: https://github.com/pangeo-data/climpred/blob/main/requirements_upstream.txt This snippet lists the primary Python dependencies for the climpred project, specifying their installation source via git repositories. These packages cover data handling, numerical computation, and scientific visualization. ```python bias_correction @ git+https://github.com/pankajkarman/bias_correction bottleneck @ git+https://github.com/pydata/bottleneck cftime @ git+https://github.com/Unidata/cftime dask @ git+https://github.com/dask/dask nc-time-axis @ git+https://github.com/SciTools/nc-time-axis xarray @ git+https://github.com/pydata/xarray xclim @ git+https://github.com/Ouranosinc/xclim xrft @ git+https://github.com/xgcm/xrft xskillscore @ git+https://github.com/xarray-contrib/xskillscore ``` -------------------------------- ### Load and Prepare Dataset for GPU Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/climpred_gpu.ipynb Loads the same sample datasets as the CPU example but converts them to use CuPy arrays via `.as_cupy()` for GPU acceleration. This prepares the data for `PerfectModelEnsemble` operations on the GPU. ```python import cupy_xarray v = "tos" ds3d = load_dataset("MPI-PM-DP-3D")[v].as_cupy() ds3d.lead.attrs["unit"] = "years" control3d = load_dataset("MPI-control-3D")[v].as_cupy() pm_gpu = PerfectModelEnsemble(ds3d) pm_gpu = pm_gpu.add_control(control3d) ``` -------------------------------- ### Download Observations with climetlab Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Downloads observational data from the Copernicus Climate Data Store (CDS) using the climetlab library. It specifies the dataset, product type, time, grid, parameter, and date range for the download. The data is then converted to an xarray DataArray and squeezed. ```python obs = ( climetlab.load_source( "cds", "reanalysis-era5-single-levels", product_type="reanalysis", time=["00:00", "06:00", "12:00", "18:00"], grid=[1.0, 1.0], param="2t", date=[ "2020-08-29", "2020-08-30", "2020-08-31", "2020-09-01", "2020-09-02", "2020-09-03", ], ) .to_xarray() .squeeze(drop=True) ).drop_vars("valid_time") ``` -------------------------------- ### climpred Documentation Additions Source: https://github.com/pangeo-data/climpred/blob/main/CHANGELOG.rst Describes documentation additions for climpred, including an examples notebook for temporal and spatial smoothing, documentation for computing metrics over specified dimensions, an organized API documentation structure, a page detailing HindcastEnsemble and PerfectModelEnsemble objects, and new pages for publications and helpful links. ```python - Add examples notebook for `temporal and spatial smoothing `_. (:pr:`244`) `Aaron Spring`_ - Add documentation for computing a metric over a `specified dimension `_. (:pr:`244`) `Aaron Spring`_ - Update `API `_ to be more organized with individual function/class pages. (:pr:`243`) `Riley X. Brady`_. - Add `page `_ describing the :py:class:`.HindcastEnsemble` and :py:class:`.PerfectModelEnsemble` objects more clearly. (:pr:`243`) `Riley X. Brady`_ - Add page for `publications `_ and `helpful links `_. (:pr:`270`) `Riley X. Brady`_. ``` -------------------------------- ### Initialize Lead Attribute Units Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/quick-start.ipynb Sets the units for the 'lead' attribute in the initialized dataset to 'years'. This is a common step in preparing forecast data for analysis. ```python initialized["lead"].attrs["units"] = "years" ``` -------------------------------- ### Running Tests with Pytest Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/contributing.rst Execute all tests in the climpred package using pytest. This includes running doctests on modules, excluding specific test directories. ```shell pytest climpred pytest --doctest-modules climpred --ignore climpred/tests ``` -------------------------------- ### Combine Forecast and Observation Biweekly Aggregates Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb Initializes a `HindcastEnsemble` with the pre-aggregated biweekly forecast data and adds the biweekly observation data. ```python fct_biweekly = climpred.HindcastEnsemble(forecast_biweekly).add_observations( obs_biweekly ) ``` -------------------------------- ### Load Hindcast Data with Preprocessing Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/setup_your_own_data.ipynb Loads hindcast data using `load_hindcast`, specifying initializations, members, and a preprocessing function to handle data format. ```python # lead_offset because yearmean output %time ds = load_hindcast(inits=range(1961, 1965), members=range(1, 3), preprocess=preprocess_1var, get_path=get_path) ``` -------------------------------- ### Import Libraries and Load Data Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/smoothing.ipynb Imports necessary libraries (climpred, matplotlib, xarray) and loads sample sea surface temperature data for PerfectModelEnsemble and HindcastEnsemble examples. ```python %matplotlib inline from climpred import PerfectModelEnsemble, HindcastEnsemble from climpred.tutorial import load_dataset import matplotlib.pylab as plt import xarray as xr ``` ```python # Sea surface temperature v = "tos" initialized3d = load_dataset("MPI-PM-DP-3D")[v] control3d = load_dataset("MPI-control-3D")[v] ``` -------------------------------- ### Initialize HindcastEnsemble and Add Observations Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/Herbie.ipynb Initializes a climpred.HindcastEnsemble object with the prepared initial forecast data and adds the prepared observation data to it. The data is expanded to include 'time' and 'step' dimensions. ```python hindcast = climpred.HindcastEnsemble( init.expand_dims(["time", "step"]) ).add_observations(obs) hindcast ``` -------------------------------- ### Load Dataset from NetCDF Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Loads the previously saved GEFS forecast data from a NetCDF file into an xarray Dataset, making it ready for analysis and skill calculation. ```python init = xr.open_dataset("tmp_GEFS_a.nc") ``` -------------------------------- ### Initialize fsspec Caching Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/NWP/NWP_GEFS_6h_forecasts.ipynb Configures fsspec to cache downloaded files locally, improving performance for repeated access. It specifies the cache directory and ensures original filenames are preserved. ```python import intake import fsspec # caching downloads # specify caching location, where to store files to with their original names fsspec.config.conf["simplecache"] = { "cache_storage": "../my_caching_folder", "same_names": True, } import intake_xarray # access files hosted via THREDDS import climetlab # download ERA5 import xarray as xr import numpy as np import climpred # forecast verification ``` -------------------------------- ### Import Preprocessing Utilities Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/misc/setup_your_own_data.ipynb Imports specific preprocessing functions from climpred for loading hindcast data and setting integer time axes. ```python from climpred.preprocessing.shared import load_hindcast, set_integer_time_axis from climpred.preprocessing.mpi import get_path ``` -------------------------------- ### Verify and Plot Climate Forecasts Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/examples/subseasonal/daily-S2S-ECMWF.ipynb This snippet demonstrates how to verify climate model forecasts against observations using the 'rps' metric and 'm2o' comparison. It also shows how to select specific lead times and plot the results, with different category edges for observations and forecasts. ```python fct.smooth(dict(lead=14), how="mean").verify( metric="rps", comparison="m2o", alignment="same_inits", dim=["member", "init"], category_edges=(obs_edges, model_edges), ).sel(lead=["1.0-14.0", "14.0-27.0", "28.0-41.0"]).t2m.plot(col="lead", robust=True) ``` -------------------------------- ### Load climpred Tutorial Dataset and Set Up Plotting Source: https://github.com/pangeo-data/climpred/blob/main/docs/source/alignment.ipynb Loads a sample dataset using climpred's tutorial functions and configures matplotlib for plotting. It also imports necessary libraries for data analysis and handling warnings. ```python from climpred import HindcastEnsemble from climpred.tutorial import load_dataset from esmtools.stats import rm_trend import matplotlib.pyplot as plt plt.style.use("fivethirtyeight") %matplotlib inline import numpy as np import warnings ```