### Setup: Import Libraries and Initialize Environment Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/09_model_output.ipynb Imports necessary libraries and sets up the output directory for the notebook examples. It also loads the jupyter_black extension for code formatting. ```python import pathlib as pl from pprint import pprint import jupyter_black import numpy as np import pywatershed as pws import xarray as xr jupyter_black.load() # auto-format the code in this notebook ``` ```python nb_output_dir = pl.Path("./09_model_output") if not nb_output_dir.exists(): nb_output_dir.mkdir() ``` -------------------------------- ### Configure and Install Pywatershed Source: https://github.com/doi-usgs/pywatershed/blob/develop/prms_src/prms5.3.0/README.md Use these commands to set up the build directory and install Pywatershed. The prefix is set to the current directory, and libraries are installed in the 'bin' subdirectory. ```bash meson setup builddir --prefix=$(pwd) --libdir=bin ``` ```bash meson install -C builddir ``` -------------------------------- ### Import Libraries and Setup Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/make_starfit_parameters.ipynb Imports necessary libraries and sets up display options for pandas. This is a common setup for data analysis tasks. ```python import pathlib as pl import jupyter_black import pandas as pd import xarray as xr import pywatershed as pws jupyter_black.load() pd.set_option("display.max_rows", 500) ``` -------------------------------- ### Setup: Load Model Parameters and Configuration Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/09_model_output.ipynb Loads model parameters, including custom reservoir capacities, and sets up domain-specific control and parameter files for the simulation. ```python ex_output_dir = nb_output_dir / "flowgraph_example" if not ex_output_dir.exists(): ex_output_dir.mkdir() pkg_root = pws.constants.__pywatershed_root__ big_sandy_param_file = pkg_root / "data/big_sandy_starfit_parameters.nc" sf_params = pws.Parameters.from_netcdf(big_sandy_param_file, use_xr=True) sfp_ds = sf_params.to_xr_ds().copy() # We'll increase the reservoir capacity by 50% cap_mult = 1.5 sfp_ds["GRanD_CAP_MCM"] *= cap_mult sf_params_new = pws.Parameters.from_ds(sfp_ds) domain_dir = pws.utils.get_addtl_domains_dir("fgr_2yr") control_file = domain_dir / "nhm.control" # Number of days to run ndays_run = 365 * 2 params_file_channel = domain_dir / "parameters_PRMSChannel.nc" params_channel = pws.parameters.PrmsParameters.from_netcdf(params_file_channel) dis_file = domain_dir / "parameters_dis_hru.nc" dis_hru = pws.Parameters.from_netcdf(dis_file, encoding=False) dis_both_file = domain_dir / "parameters_dis_both.nc" dis_both = pws.Parameters.from_netcdf(dis_both_file, encoding=False) ``` -------------------------------- ### Setup model run parameters Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/08_restart_streamflow.ipynb Defines input directories, initial time, timestep, and the number of simulation steps. It also sets up an output directory for restart files. ```python input_dir = pws.utils.get_test_data_dir() / "drb_2yr/output/" domain_dir = pws.constants.__pywatershed_root__ / "data/drb_2yr" t0 = np.datetime64("1978-12-31T00:00:00") timestep = np.timedelta64(24, "h") n_steps = 5 nb_output_dir = pl.Path("./08_restart_streamflow") ``` -------------------------------- ### Install Pip Dependencies Source: https://github.com/doi-usgs/pywatershed/blob/develop/DEVELOPER.md Installs all project dependencies using pip. You can specify dependency groups like 'lint', 'test', or 'optional' for a more tailored installation. ```bash pip install ".[all]" ``` -------------------------------- ### Initialize Model and Run First Timestep Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/08_restart_streamflow.ipynb Sets up the model for the first timestep and runs it to create the initial restart file. Ensure the restart directory is clean before starting. ```python restart_dir = nb_output_dir / "restart_multi_run" if restart_dir.exists(): shutil.rmtree(restart_dir) nhm = pws.Model( nhm_processes, control=get_control(t0, t0 + timestep), parameters=get_params(), ) hnm.run(finalize=True) pprint(sorted(restart_dir.glob("*.nc"))) ``` -------------------------------- ### Import PyNHM and Setup Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/performance/pynhm_nhm_performance.ipynb Imports necessary libraries and defines constants for running the NHM model. Sets up paths for repository root, data directory, and output variables. ```python import pathlib as pl import shutil import pywatershed from pywatershed.base.control import Control from pywatershed.base.model import Model from pywatershed.utils.parameters import PrmsParameters repo_root = pywatershed.constants.__pywatershed_root__.parent data_dir = pl.Path("../../../data/") output_vars = [ 'albedo', 'gwres_flow', 'gwres_sink', 'hru_actet', 'hru_impervstor', 'hru_intcpevap', 'hru_rain', 'hru_snow', 'infil', 'pkwater_equiv', 'potet', 'snowcov_area', 'soil_moist', 'sroff', 'ssres_flow', 'swrad', 'tavgc', 'tmaxc', 'tminc', 'seg_outflow', ] ``` -------------------------------- ### Install Pre-commit Hooks Source: https://github.com/doi-usgs/pywatershed/blob/develop/DEVELOPER.md Install pre-commit hooks to automate checks before committing. This is recommended for maintaining repository integrity. ```shell pre-commit install ``` -------------------------------- ### Import Libraries and Setup Environment Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/07_mmr_to_mf6_chf_dfw.ipynb Imports necessary libraries for pywatershed, flopy, and plotting, and sets up the environment by adding the MF6 binary directory to the system path. It also ensures GIS files are downloaded. ```python import os import shutil import contextily as cx import flopy import geopandas as gpd import jupyter_black import numpy as np import pandas as pd import pint import pywatershed as pws import xarray as xr import hvplot.xarray # noqa import pywatershed as pws from pywatershed.utils.mmr_to_mf6_dfw import MmrToMf6Dfw jupyter_black.load() pws.utils.gis_files.download() # make sure we have the GIS files os.environ["PATH"] += os.pathsep + str(mf6_bin.parent) plot_height = 350 plot_width = 900 ``` -------------------------------- ### Model Setup and Flow Graph Plotting Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/06_flow_graph_starfit.ipynb Sets up a Pywatershed model and plots the channel flow graph. This code should be run if the simulation directory does not exist. ```python %%time if not run_dir.exists(): run_dir.mkdir() model = pws.Model(model_dict) model.processes["prms_channel_flow_graph"].plot() model.run() model.finalize() model.processes["prms_channel_flow_graph"].mass_budget ``` -------------------------------- ### Render PRMSGroundwater help documentation Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb Renders the documentation for the PRMSGroundwater class using pydoc. This is useful for understanding how the class is instantiated and its initial setup. ```python # this is equivalent to help() but we get the multiline string and just look at part of it prms_gw_help = pydoc.render_doc(pws.PRMSGroundwater, "Help on %s") ``` -------------------------------- ### Prepare and Run Open-Loop Model with Static Ag_frac Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/10_ag_irrigation_use.ipynb Sets up and runs an open-loop pywatershed model. This example demonstrates how to use a static 'ag_frac' value, loaded from a parameter file and written to a time-varying NetCDF file, for the agricultural fraction input. ```python if mk_run_dirs( [control_ol], exist_fail=run_dir_exist_fail, ): params_spinup_file = control_spinup_file.parent / control_ol.options.get( "parameter_file" ) params_nhm = pws.parameters.PrmsParameters.load(params_spinup_file) params_spinup = pws.parameters.PrmsParameters.load(params_spinup_file) # Here we take the static ag_frac from the parameter file and put it # into a time-varying input netcdf input file. ag_frac_static = xr.load_dataarray(domain_dir / "tmin.nc") ag_frac_static[:, :] = params_spinup.parameters["ag_frac"] ag_frac_static.name = "ag_frac" ag_frac_static_file = domain_dir / "ag_frac_static_2d.nc" if ag_frac_static_file.exists(): ag_frac_static_file.unlink() ag_frac_static.to_netcdf(ag_frac_static_file) # now use this static one as the one the run uses shutil.copy2(ag_frac_static_file, ag_frac_file) model_ol = pws.Model( wu_ol_processes, control=control_ol, parameters=params_ol ) model_ol.run(finalize=True) # If we left this around, the analysis model would use it because it would be # found before the ag_frac.param file we'll use below. if ag_frac_file.exists(): ag_frac_file.unlink() ``` -------------------------------- ### Create a Multi-Process Model Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/developer/02_process_models.ipynb Instantiate a Pywatershed model by providing a list of desired process classes and a control object. This example includes soil zone, groundwater, and channel processes. ```python control = pywatershed.Control.load(domain_dir / "control.test") control.config["input_dir"] = input_dir multi_proc_model = pywatershed.Model( [ pywatershed.PRMSSoilzone, pywatershed.PRMSGroundwater, pywatershed.PRMSChannel, ], control=control, parameters=params, ) ``` -------------------------------- ### Import necessary libraries and setup Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb Imports essential libraries for pywatershed development and analysis, including plotting and notebook formatting tools. It also downloads necessary GIS files and configures the notebook environment. ```python import pathlib as pl from pprint import pprint import pydoc import hvplot.xarray # noqa import jupyter_black import numpy as np import pywatershed as pws from pywatershed.utils import gis_files from tqdm.notebook import tqdm import xarray as xr gis_files.download() # this downloads GIS files specific to the example notebooks jupyter_black.load() # auto-format the code in this notebook ``` -------------------------------- ### Import Libraries and Setup Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/05_parameter_ensemble.ipynb Imports necessary libraries for pywatershed operations, including path manipulation, data handling with xarray, and parallel processing. It also sets up automatic code formatting for the notebook. ```python import os import pathlib as pl from pprint import pprint import shutil import jupyter_black import numpy as np import pywatershed as pws import xarray as xr jupyter_black.load() # auto-format the code in this notebook ``` -------------------------------- ### Get PRMS Output Times Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-graphs.ipynb Extracts time information from the PRMS output and prints the number of days to be processed. The 'idx0' variable is used to set a starting point for plotting. ```python idx0 = 730 #365 times = prms_out["time"] ndays = times.shape[0] print("Number of PRMS days to process {}".format(ndays)) ``` -------------------------------- ### Set up output directory Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/04_preprocess_atm.ipynb Creates a dedicated directory for notebook outputs. Removes the directory if it already exists to ensure a clean slate. ```python nb_output_dir = pl.Path("./04_preprocess_atm") if nb_output_dir.exists(): shutil.rmtree(nb_output_dir) nb_output_dir.mkdir() ``` -------------------------------- ### Initialize Output Directory and NetCDF Files Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb Set up the output directory and initialize NetCDF files for storing simulation results. Ensure the run directory exists before initialization. ```python run_dir = pl.Path("./00_processes") run_dir.mkdir(exist_ok=True) prms_gw.initialize_netcdf(run_dir) ``` -------------------------------- ### Install Pywatershed without Git Source: https://github.com/doi-usgs/pywatershed/blob/develop/README.md Use these commands to install pywatershed and its dependencies without using git. Ensure you have curl installed. Replace `` with 'main' for stable or 'develop' for development. ```bash curl -L -O https://raw.githubusercontent.com/DOI-USGS/pywatershed//environment_w_jupyter.yml mamba env create -f environment_w_jupyter.yml mamba activate pws pip install git+https://github.com/DOI-USGS/pywatershed.git@ ``` -------------------------------- ### Install pywatershed in Development Mode Source: https://github.com/doi-usgs/pywatershed/blob/develop/DEVELOPER.md Installs the pywatershed package in editable development mode. This command should be run after activating your Python environment. ```bash pip install -e . ``` -------------------------------- ### Set up model directories and parameters Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/prms/panel_errors.ipynb Configures the repository directory, domain directory, GIS directory, and run directory. It also loads PRMS parameters and control files. The code includes logic to re-run the model if the run directory does not exist, by creating necessary input directories and symlinking NetCDF files. ```python repo_dir = pywatershed.constants.__pywatershed_root__.parent domain_dir = repo_dir / f'test_data/{domain}/' gis_dir = repo_dir / f'examples/pynhm_gis/{domain}' run_dir = repo_dir / f"pynhm/analysis/runs/{domain}_{'_'.join([proc.__name__ for proc in processes])}" #change this input_dir_all = run_dir / 'inputs' parameter_file = domain_dir / 'myparam.param' control_file = domain_dir / 'control.test' parameters = pywatershed.PRMSParameters.load(parameter_file) control = pywatershed.Control.load(control_file, params=parameters) # to re-run, delete the run_dir run_flag = False if not run_dir.exists(): run_flag = True # Combine all PRMS inputs and outputs in one directory for pynhm input input_dir_all.mkdir(exist_ok=True, parents=True) for nc_file in domain_dir.glob('**/*.nc'): target = input_dir_all / (nc_file.name) if not target.exists(): target.symlink_to(nc_file) # instantiate the model model = pywatershed.Model(*processes, control=control, input_dir=input_dir_all, imbalance_behavior='warn', netcdf_output_dir=run_dir) if run_flag: model.run(finalize=True) ``` -------------------------------- ### Install Pywatershed with Git for Development Source: https://github.com/doi-usgs/pywatershed/blob/develop/README.md Clone the repository and install pywatershed in editable mode for development. This method allows for direct code modifications. Replace `` with 'main' for stable or 'develop' for development. ```bash git clone https://github.com/DOI-USGS/pywatershed.git cd pywatershed mamba env create -f environment_w_jupyter.yml mamba activate pws pip install -e . ``` -------------------------------- ### Clone pywatershed Repository and Install Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/developer/00_python_virtual_env.ipynb Clones the pywatershed repository if it doesn't exist, installs the package in editable mode using pip, and downloads and unzips necessary GIS files. This should be run within the activated 'pyws_nb' conda environment. ```bash # clone the repository if [ ! -e pynhm ]; then git clone https://github.com/DOI-USGS/pynhm || exit 3 fi # install pynhm cd pynhm || exit 4 pip install -e . || exit 5 # GIS files cd examples || exit 6 curl -LO https://github.com/DOI-USGS/pynhm/releases/download/v2022.0.1/pynhm_gis.zip || exit 7 unzip pynhm_gis.zip || exit 8 ``` -------------------------------- ### Configure input and output directories Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-api-pynhm.ipynb Sets up the output directory for simulation results and verifies the existence of all necessary input files and directories, including parameter files, control files, and climate data. ```python # This step is critical for MF6 initialization output_dir = run_dir / 'output' if not output_dir.exists(): output_dir.mkdir() input_dir = tgt_dir for ff in [param_file, control_file, input_dir, output_dir]: assert ff.exists() ``` -------------------------------- ### Get DataFrame Index Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-graphs.ipynb Retrieves the index from the sagehenStreamFlow DataFrame, likely used for time-based operations. ```python df_index = sagehenStreamFlow.index ``` -------------------------------- ### Prepare Output Directories Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/05_parameter_ensemble.ipynb Defines the directory for domain parameters and creates necessary output directories for the notebook, including a subdirectory for parameters. ```python domain_dir = pws.constants.__pywatershed_root__ / "data/drb_2yr" nb_output_dir = pl.Path("./05_parameter_ensemble") nb_output_dir.mkdir(exist_ok=True) (nb_output_dir / "params").mkdir(exist_ok=True) ``` -------------------------------- ### Helper function to get parameters Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/08_restart_streamflow.ipynb Loads and returns PRMS parameters from a specified parameter file. ```python def get_params(): return pws.parameters.PrmsParameters.load(domain_dir / "myparam.param") ``` -------------------------------- ### Set Up Budget Analysis Case Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/runoff_errors.ipynb Configures the specific time and location indices for the mass balance budget analysis. Asserts the existence of required input files. ```python budget_cases = [ ( "1979-01-11T00:00:00", [0, 1, 2, 3, 4, 5, 6, 9, 10, 11, 12, 13], ), ] case_ind = 0 budget_time = np.datetime64(budget_cases[case_ind][0]) budget_location_inds = budget_cases[case_ind][1] budget_comps = {} for term, vars in budget_terms.items(): print(term) for vv in vars: print(" ", vv) if term == "inputs": pws_file = input_dir_cp / f"{vv}.nc" else: pws_file = output_dir / f"{vv}.nc" assert (pws_file).exists() pws_ds = xr.open_dataset(pws_file)[vv].rename("pws") prms_file = input_dir_cp / f"{vv}.nc" assert (prms_file).exists() prms_ds = xr.open_dataset(prms_file)[vv].rename("prms") budget_comps[vv] = ( xr.merge([pws_ds, prms_ds]) .sel(time=budget_time) .isel(nhm_id=budget_location_inds) ) ``` -------------------------------- ### Get PRMS Output Shape Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-graphs.ipynb Retrieves and displays the shape of the 'ppt' (precipitation) array from the prms_out dictionary. ```python prms_out["ppt"].shape ``` -------------------------------- ### Get Test Data Directory Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/03_compare_pws_prms.ipynb Obtain the path to the test data directory for a specified domain name. ```python domain_dir = pws.utils.get_test_data_dir() / domain_name ``` -------------------------------- ### Configuration settings Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/runoff_errors.ipynb Sets up configuration variables for the simulation, including the domain name and output directory. ```python # Configuration domain_name = "drb_2yr" nb_output_dir = pl.Path("./runoff_errors") ``` -------------------------------- ### Get Process Variables Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb List the public variables maintained by a specific process. These are the outputs that can be written or accessed. ```python pws.PRMSGroundwater.get_variables() ``` -------------------------------- ### Initialize pywatershed constants and paths Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/snow_errors.ipynb Sets up the pywatershed root directory, domain directory, and creates the output directory. It also initializes constants for zero and epsilon values used in calculations. ```python pws_root = pws.constants.__pywatershed_root__ domain_dir = pws_root / f"../test_data/{domain_name}" nb_output_dir.mkdir(exist_ok=True) zero = pws.constants.zero epsilon64 = pws.constants.epsilon64 epsilon32 = pws.constants.epsilon32 ``` -------------------------------- ### Get Current Working Directory Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/performance/prms_5.2.1_performance.ipynb Prints the current working directory, useful for verifying file paths. ```python import os; os.getcwd() ``` -------------------------------- ### Get Process Parameters Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb Retrieve the parameters used by a specific process. This helps in understanding the configurable aspects of the process. ```python pws.PRMSGroundwater.get_parameters() ``` -------------------------------- ### Import necessary libraries and set up paths Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/performance/compile_performance.ipynb Imports pathlib and pywatershed, and defines the repository root and data directory paths. ```python import pathlib as pl import pywatershed repo_root = pywatershed.constants.__pywatershed_root__.parent data_dir = pl.Path("../../../data/") ``` -------------------------------- ### Get Process Dimensions Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb Query the dimensions associated with a specific process. Useful for understanding the spatial or temporal extents of the process. ```python pws.PRMSGroundwater.get_dimensions() ``` -------------------------------- ### Define simulation parameters and output directory Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/snow_errors.ipynb Sets up key parameters for the simulation, including the domain name and the output directory for results. It also defines flags to skip existing runs. ```python domain_name = "ucb_2yr" nb_output_dir = pl.Path("./snow_errors") skip_if_exists_prms_mixed = True skip_if_exists_prms_double = True skip_if_exists_pws = True ``` -------------------------------- ### Define model processes Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/08_restart_streamflow.ipynb Specifies the list of pywatershed processes to be included in the model simulation. This example uses PRMSGroundwater and PRMSChannel. ```python nhm_processes = [ pws.PRMSGroundwater, pws.PRMSChannel, ] ``` -------------------------------- ### Set Up Run Directory and Domain Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/10_ag_irrigation_use.ipynb Configures the run directory and specifies the additional domain files to be used. This sets up the environment for subsequent pywatershed analyses. ```python run_dir_exist_fail = False pkg_root = pws.constants.__pywatershed_root__ domain_dir = pws.utils.get_addtl_domains_dir("fgr_ag_2yr") ``` -------------------------------- ### Plotting a specific variable Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/prms/panel_errors.ipynb This is a commented-out example showing how to plot a specific variable, 'pkwater_equiv', for HRUs using the ProcessPlot object. ```python # pp.plot_hru('pkwater_equiv', model) # variable ``` -------------------------------- ### Initialize and Use PRMSGroundwater Module Source: https://context7.com/doi-usgs/pywatershed/llms.txt Shows how to initialize the PRMSGroundwater module, set its calculation method and imbalance behavior, and access groundwater variables like storage and baseflow. Ensure the control and parameter files are correctly loaded, and note the specified output directory for control options. ```python import pywatershed as pws pws_root = pws.constants.__pywatershed_root__ domain_dir = pws_root / "../test_data/drb_2yr" control = pws.Control.load_prms( domain_dir / "control.test", warn_unused_options=False ) control.options["input_dir"] = domain_dir / "output" dis_both = pws.Parameters.from_netcdf(domain_dir / "parameters_dis_both.nc") params_gw = pws.Parameters.from_netcdf(domain_dir / "parameters_PRMSGroundwater.nc") gw = pws.PRMSGroundwater( control=control, discretization=dis_both, parameters=params_gw, calc_method="numba", imbalance_behavior="warn", ) print("Groundwater variables:", pws.PRMSGroundwater.get_variables()) # ['gwres_flow', 'gwres_sink', 'gwres_stor', 'gwres_in'] control.advance() gw.advance() gw.calculate(1.0) print("GW storage [in]:", gw["gwres_stor"]) print("Baseflow [cfs]:", gw["gwres_flow"]) ``` -------------------------------- ### Set up Reduced Output Directory Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/01_multi-process_models.ipynb Creates a directory for reduced model output and resolves its path. This is the first step in configuring the model to write less data. ```python run_dir = pl.Path(nb_output_dir / "yaml_less_output").resolve() run_dir.mkdir(exist_ok=True) ``` -------------------------------- ### Import necessary libraries for pywatershed Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/snow_errors.ipynb Imports standard libraries for data manipulation, plotting, and pywatershed functionality. Ensure these are installed before running. ```python import pathlib as pl from pprint import pprint from shutil import rmtree, copy2 import hvplot.xarray # noqa import jupyter_black from IPython.display import display import numpy as np import pywatershed as pws import xarray as xr jupyter_black.load() ``` -------------------------------- ### Import Necessary Packages Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-graphs.ipynb Imports all required Python libraries for data manipulation, modeling, and plotting. Ensure these packages are installed before running. ```python import os import sys import datetime import pathlib as pl import numpy as np import matplotlib as mpl from matplotlib import gridspec import matplotlib.ticker as mticker import matplotlib.dates as mdates import matplotlib.pyplot as plt import flopy import geopandas as gpd import fiona import netCDF4 as nc import pandas as pd import hydrofunctions as hf %matplotlib inline ``` -------------------------------- ### Initialize and Use PRMSAtmosphere Module Source: https://context7.com/doi-usgs/pywatershed/llms.txt Demonstrates initializing the PRMSAtmosphere module, advancing the simulation, and accessing computed atmospheric variables like precipitation, temperature, and solar radiation. Ensure control and parameter files are correctly loaded. ```python import pywatershed as pws pws_root = pws.constants.__pywatershed_root__ domain_dir = pws_root / "../test_data/drb_2yr" control = pws.Control.load_prms( domain_dir / "control.test", warn_unused_options=False ) control.options["input_dir"] = domain_dir dis_hru = pws.Parameters.from_netcdf(domain_dir / "parameters_dis_hru.nc") params_a = pws.Parameters.from_netcdf(domain_dir / "parameters_PRMSAtmosphere.nc") atm = pws.PRMSAtmosphere( control=control, discretization=dis_hru, parameters=params_a, ) # Advance once — all time variables are computed immediately control.advance() atm.advance() atm.calculate(time_length=1.0) # Access computed atmospheric variables (per HRU arrays) print("HRU precip [in]:", atm["hru_ppt"]) # shape (nhru,) print("Min temp [°F]:", atm["tminf"]) # shape (nhru,) print("Shortwave rad:", atm["swrad"]) # shape (nhru,) print("Pot. ET [in]:", atm["potet"]) # shape (nhru,) print("Transp. flag:", atm["transp_on"]) # boolean shape (nhru,) # Full time-series of a variable is accessible via underscore prefix print("All-time swrad shape:", atm["_swrad"].shape) # (n_times, nhru) ``` -------------------------------- ### Import necessary libraries Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/06_flow_graph_starfit.ipynb Imports all required libraries for the notebook, including pyPRMS, pywatershed, and plotting tools. Ensure these are installed before running. ```python import pathlib as pl from pprint import pprint import folium import geopandas as gpd import jupyter_black import numpy as np import pandas as pd from tqdm.auto import tqdm import xarray as xr import hvplot.xarray # noqa, after xr import hvplot.pandas # noqa, after pandas import pyPRMS import pywatershed as pws from pywatershed.plot import DomainPlot from pywatershed.constants import zero ndays_run = 365 * 2 plot_height = 600 plot_width = 1000 pyprms_meta = pyPRMS.MetaData(verbose=False).metadata jupyter_black.load() ``` -------------------------------- ### Define Domain and Parameter File Paths Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/prms_parameter_discretization_separation.ipynb Set up the paths for the domain directory and the PRMS parameter file. ```python domain_name = "drb_2yr" domain_dir = pl.Path(f"../test_data/{domain_name}/") param_file = domain_dir/ "myparam.param" ``` -------------------------------- ### Initialize PRMSChannel Model Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/developer/02_process_models.ipynb Sets up the PRMSChannel model by loading control and parameter files, and defining input directories. This prepares the model for execution. ```python import pathlib as pl import pywatershed import tempfile pynhm_root = pywatershed.constants.__pywatershed_root__ domain = "drb_2yr" domain_dir = pynhm_root.parent / f"test_data/{domain}" input_dir = domain_dir / "output" run_dir = pl.Path(tempfile.mkdtemp()) control = pywatershed.Control.load(domain_dir / "control.test") control.config["input_dir"] = input_dir params = pywatershed.parameters.PrmsParameters.load( domain_dir / "myparam.param" ) streamflow_model = pywatershed.Model( [pywatershed.PRMSChannel], control=control, parameters=params ) ``` -------------------------------- ### Get PRMSSnow Mass Budget Terms Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/snow_errors.ipynb Retrieves the mass budget terms for the PRMSSnow model. This is used to identify variables for comparison. ```python budget_terms = pws.PRMSSnow.get_mass_budget_terms() ``` -------------------------------- ### Configure and Populate Run Dictionary Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/performance/pynhm_nhm_performance.ipynb Iterates through defined domain directories, calculation methods, and I/O options to create a dictionary of run configurations. It prepares output directories and control files for each scenario. ```python # Actually passing the control file is moot since it dosent control output # for pynhm, but... we'll use the control file name to key io in the model setup. control_dir = data_dir / "pynhm/performance_runs/PRMS/control_files" run_dict = {} for dom in dom_dirs: for meth in ['numpy', 'numba']: if dom == conus_dom_dir and meth == 'numpy': continue for io in ['yes', 'no']: run_dir = pynhm_run_dir / f"{dom.name}_io_{io}_method_{meth}" if io == 'yes': if run_dir.exists(): shutil.rmtree(run_dir) run_dir.mkdir(parents=True) src_dir = f"{dom}" n_hru_desc = 'multi' if 'hru_1' in dom.name: n_hru_desc = 'single' control_file = control_dir / f"control.{n_hru_desc}_hru_{io}_io" assert control_file.exists() run_dict[f"{run_dir}"] = { 'src': src_dir, 'io': io, 'method': meth, 'control_file': control_file } ``` -------------------------------- ### Import Necessary Packages Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-maps.ipynb Imports essential Python libraries for numerical operations, plotting, and model handling. Ensure these packages are installed before running. ```python import os import sys import pathlib as pl import numpy as np import matplotlib as mpl from matplotlib import gridspec import matplotlib.ticker as mticker import matplotlib.pyplot as plt import flopy import geopandas as gpd import fiona import netCDF4 as nc ``` -------------------------------- ### Prepare Submodel Run Directory Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/01_multi-process_models.ipynb Create a dedicated directory for running the submodel and prepare control options by deep copying from the main control object. ```python run_dir = pl.Path(nb_output_dir / "nhm_sub").resolve() run_dir.mkdir(exist_ok=True) control_cp = deepcopy(control) ``` -------------------------------- ### Import Libraries for Geospatial Plotting Source: https://github.com/doi-usgs/pywatershed/blob/develop/evaluation/performance/plot_4_performance_domains.ipynb Imports necessary libraries for plotting, geospatial data handling, and basemap integration. Ensure these libraries are installed. ```python %matplotlib inline import matplotlib.pyplot as plt import geopandas as gpd import contextily as cx ``` -------------------------------- ### Initialize PRMSGroundwater with Budget for mass balance Source: https://context7.com/doi-usgs/pywatershed/llms.txt Sets up control, parameters, and discretization for a PRMSGroundwater model. It initializes the groundwater model with a specified imbalance behavior ('warn') to track mass balance using the Budget class, which is automatically attached to ConservativeProcess subclasses. ```python import numpy as np import pywatershed as pws pws_root = pws.constants.__pywatershed_root__ domain_dir = pws_root / "../test_data/drb_2yr" control = pws.Control.load_prms( domain_dir / "control.test", warn_unused_options=False ) control.options["input_dir"] = domain_dir / "output" params_all = pws.parameters.PrmsParameters.load(domain_dir / "myparam.param") dis_both = pws.Parameters.from_netcdf(domain_dir / "parameters_dis_both.nc") gw = pws.PRMSGroundwater( control=control, discretization=dis_both, parameters=params_all, imbalance_behavior="warn", # warn on mass imbalance ) # The Budget is automatically attached to ConservativeProcess subclasses ``` -------------------------------- ### Get Maximum PRMS Output Values Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/runoff_errors.ipynb Retrieves the maximum output values from the PRMS model as a list, typically for normalization or comparison purposes. ```python output_max.prms.values.tolist() ``` -------------------------------- ### Create and Load Parameters with PrmsParameters Source: https://context7.com/doi-usgs/pywatershed/llms.txt Shows how to build `PrmsParameters` from scratch, load from PRMS native parameter files, and interact with NetCDF parameter files. Use `PrmsParameters` to configure the physical behavior of `Process` subclasses. ```python import numpy as np import pywatershed as pws # Build parameters from scratch params = pws.parameters.PrmsParameters( dims={"nsegment": 3}, coords={"nsegment": np.array([0, 1, 2])}, data_vars={ "tosegment": np.array([2, 3, 0]), # channel routing connectivity "seg_length": np.ones(3) * 1.0e3, # meters }, metadata={ "nsegment": {"dims": ["nsegment"]}, "tosegment": {"dims": ["nsegment"]}, "seg_length": {"dims": ["nsegment"]}, }, validate=True, ) print(params.dims) # mappingproxy({'nsegment': 3}) print(params.parameters) # all parameter names (coords + data_vars) print(params.get_param_values("seg_length")) # array([1000., 1000., 1000.]) print(params.get_dim_values("nsegment")) # 3 # Load from a PRMS native parameter file prms_params = pws.parameters.PrmsParameters.load( "path/to/myparam.param" ) # Load from / save to NetCDF pws_root = pws.constants.__pywatershed_root__ params_nc = pws.Parameters.from_netcdf( pws_root / "data/drb_2yr/parameters_dis_hru.nc", encoding=False, ) params_nc.to_nc4_ds("my_params.nc") # Convert to xarray Dataset for inspection xrds = params_nc.to_xr_ds() print(xrds) # Merge multiple parameter objects merged = pws.Parameters.merge(params_nc, prms_params) ``` -------------------------------- ### Get Process Mass Budget Terms Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/00_processes.ipynb Obtain the terms that constitute the mass budget for a specific process. This clarifies how mass is conserved or accounted for. ```python pws.PRMSGroundwater.get_mass_budget_terms() ``` -------------------------------- ### Get Maximum PWS Output Values Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/runoff_errors.ipynb Retrieves the maximum output values from the PWS (SWMM) model as a list, often used for normalization or comparison. ```python input_max.pws.values.tolist() ``` -------------------------------- ### Display help for Model class Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/01_multi-process_models.ipynb Renders and prints the first 22 lines of the documentation for the `pws.Model` class. This provides a concise overview of its instantiation methods and parameters. ```python # this is equivalent to help() but we get the multiline string and just look at part of it model_help = pydoc.render_doc(pws.Model, "Help on %s") # the first 22 lines of help(pws.Model) print("\n".join(model_help.splitlines()[0:22])) ``` -------------------------------- ### Set up pywatershed paths and imports Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/developer/01_automated_testing.ipynb Imports necessary libraries and defines paths for the pywatershed repository, test data, and test scripts. ```python import pywatershed import os import shutil import subprocess import pathlib as pl # set up paths for notebook # this is as pywatershed is installed pynhm_repo_root = pywatershed.constants.__pywatershed_root__.parent pynhm_test_data_dir = pynhm_repo_root / "test_data" pynhm_test_data_scripts_dir = pynhm_repo_root / "test_data/scripts" pynhm_autotest_dir = pynhm_repo_root / "autotest" ``` -------------------------------- ### Set Plotting Times Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-graphs.ipynb Selects the time range for plotting from the combined streamflow DataFrame, starting from 'idx0'. This allows focusing on a specific period of interest. ```python plt_times = sagehenStreamFlow.index[idx0:] plt_times.shape ``` -------------------------------- ### Launch Jupyter Lab Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/developer/00_python_virtual_env.ipynb Starts the jupyter-lab application, which will be used to run subsequent notebooks. Ensure the 'pyws_nb' conda environment is active before running this command. ```bash jupyter-lab || exit 9 ``` -------------------------------- ### Instantiate and Graph Multiple Models Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/10_ag_irrigation_use.ipynb Demonstrates creating and visualizing multiple pywatershed models with different configurations (NHM, OL, Analysis). This helps in comparing model structures and identifying changes in inputs and dependencies. ```python model_nhm_gr = pws.Model( nhm_processes, control=control_nhm, parameters=params_nhm, find_input_files=False, ) mk_model_graph(model_nhm_gr) model_ol_gr = pws.Model( wu_ol_processes, control=control_ol, parameters=params_ol, find_input_files=False, ) mk_model_graph(model_ol_gr) model_analysis_gr = pws.Model( wu_analysis_processes, control=control_analysis, parameters=params_analysis, find_input_files=False, ) mk_model_graph(model_analysis_gr) ``` -------------------------------- ### Construct and Manage Simulation Time with Control Source: https://context7.com/doi-usgs/pywatershed/llms.txt Demonstrates how to construct a `Control` object directly, load settings from a PRMS control file, and advance simulation time. Use this to manage simulation start/end times, time steps, and global options. ```python import pathlib as pl import numpy as np import pywatershed as pws # Construct directly control = pws.Control( start_time=np.datetime64("1979-01-01T00:00:00"), end_time=np.datetime64("1980-12-31T00:00:00"), time_step=np.timedelta64(24, "h"), options={ "input_dir": pl.Path("./input_data"), "netcdf_output_dir": pl.Path("./output"), "calc_method": "numba", "imbalance_behavior": "warn", "verbosity": 0, }, ) print(control.start_time) # numpy.datetime64('1979-01-01T00:00:00') print(control.end_time) # numpy.datetime64('1980-12-31T00:00:00') print(control.n_times) # 730 print(control.time_step) # numpy.timedelta64(24,'h') print(control.time_step_seconds) # 86400.0 # Advance time manually control.advance() print(control.current_time) # numpy.datetime64('1979-01-01T00:00:00') print(control.current_doy) # 1 print(control.current_dowy) # 93 (day of water year) print(control.current_month) # 1 # Load from a PRMS legacy control file pws_root = pws.constants.__pywatershed_root__ control_prms = pws.Control.load_prms( pws_root / "data/drb_2yr/control.test", warn_unused_options=False, ) control_prms.options["input_dir"] = pws_root / "data/drb_2yr/output" # Save and reload as YAML control.to_yaml("my_control.yaml") control_reloaded = pws.Control.from_yaml("my_control.yaml") ``` -------------------------------- ### Run NHM Model Configuration Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/10_ag_irrigation_use.ipynb Initializes and runs a pywatershed model using the NHM configuration. This is a standard setup for running the National Hydrologic Model components. ```python if mk_run_dirs( [control_nhm], exist_fail=run_dir_exist_fail, ): model_nhm = pws.Model( nhm_processes, control=control_nhm, parameters=params_nhm ) model_nhm.run(finalize=True) ``` -------------------------------- ### Inspect and Run a Single pywatershed Process Source: https://context7.com/doi-usgs/pywatershed/llms.txt Shows how to inspect the inputs, variables, and parameters of a `PRMSSnow` process and how to instantiate and run it standalone for testing or custom loops. Includes manual time stepping and NetCDF output initialization. ```python import numpy as np import pywatershed as pws pws_root = pws.constants.__pywatershed_root__ domain_dir = pws_root / "../test_data/drb_2yr" control = pws.Control.load_prms( domain_dir / "control.test", warn_unused_options=False ) control.options["input_dir"] = domain_dir dis_hru = pws.Parameters.from_netcdf(domain_dir / "parameters_dis_hru.nc") params_snow = pws.Parameters.from_netcdf(domain_dir / "parameters_PRMSSnow.nc") # Inspect what PRMSSnow needs before building a full model print("Inputs:", pws.PRMSSnow.get_inputs()) print("Variables:", pws.PRMSSnow.get_variables()) print("Parameters:", pws.PRMSSnow.get_parameters()) # Instantiate a single process for standalone use / testing snow = pws.PRMSSnow( control=control, discretization=dis_hru, parameters=params_snow, calc_method="numba", # or "numpy" restart_write=False, restart_read=False, ) # Initialize NetCDF output for this process alone snow.initialize_netcdf(output_dir="./snow_output", separate_files=False) # Manual time loop for a single process for _ in range(10): control.advance() snow.advance() snow.calculate(time_length=1.0) snow.output() print(f"{control.current_time}: SWE = {snow['snow_water_equiv'].mean():.3f}") snow.finalize() ``` -------------------------------- ### Get MODFLOW Output Times Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-postprocess-maps.ipynb Retrieves the simulation times from MODFLOW 6 cell-by-cell budget files. It filters times based on a previously determined index. ```python tobj = flopy.utils.CellBudgetFile("sagehenmodel/output/gwf_sagehen-gsf.sfr.cbc", precision="double") times = np.array(tobj.get_times())[idx0:] ndays = times.shape[0] print("Number of MODFLOW days to process {}".format(ndays)) ``` -------------------------------- ### Configure Pywatershed Parameters Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/03_compare_pws_prms.ipynb Set up domain name, calculation method, imbalance behavior, and flags for running PRMS and Pywatershed simulations. ```python domain_name: str = "drb_2yr" # must be present in test_data/domain_name calc_method: str = "numba" imbalance_behavior: str = None run_prms: bool = True ## always forced/overwrite run_pws: bool = True # run if the output does not exist on disk force_pws_run: bool = True # if it exists on disk, re-run it and overwrite? ``` -------------------------------- ### Create and Activate Conda Environment for pywatershed Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/developer/00_python_virtual_env.ipynb Sets up the 'pyws_nb' conda environment by downloading a dependency file, removing any existing environment with the same name, and creating a new one. It then activates the environment and checks versions of git, conda, pip, and python. ```bash # Customize this path to where you'd like have the repository as a sub directory demo_dir=../path/to/pynhm_demo mkdir -p $demo_dir || exit 1 cd $demo_dir || exit 2 # Conda required for the the following curl -LO https://raw.githubusercontent.com/DOI-USGS/pynhm/main/examples/examples_env.yml || exit 1 conda remove -y --name pyws_nb --all || exit 2 conda env create -f examples_env.yml || exit 1 rm examples_env.yml || exit 4 # Check we got everything conda activate pyws_nb git --version conda --version pip --version python --version ``` ```bash git version 2.37.3 conda 4.14.0 pip 22.2.2 Python 3.10.6 ``` -------------------------------- ### Get Output File Lists for Comparison Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/01_multi-process_models.ipynb Defines output directories for memory and YAML models and retrieves sorted lists of NetCDF files from each directory for comparison. ```python mem_out_dir = nb_output_dir / "nhm_memory" yaml_out_dir = nb_output_dir / "nhm_yaml" mem_files = sorted(mem_out_dir.glob("*.nc")) yaml_files = sorted(yaml_out_dir.glob("*.nc")) ``` -------------------------------- ### Initialize pywatershed Run Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/runoff_errors.ipynb Sets up the pywatershed model by defining the process, run directory, and control options. It copies necessary input files from the PRMS double-precision run. ```python process = [pws.PRMSRunoff] pws_run_dir = nb_output_dir / f"{domain_name}_pws_run" input_dir_cp = prms_dbl_run_dir / "inputs" ``` ```python skip_if_exists_pws = True control = pws.Control.load_prms(domain_dir / "nhm.control") output_dir = pws_run_dir / "output" control.options = control.options | { "input_dir": input_dir_cp, "imbalance_behavior": "error", "calc_method": "numpy", "netcdf_output_dir": output_dir, } del control.options["netcdf_output_dir"] params = pws.parameters.PrmsParameters.load(domain_dir / "myparam.param") ``` ```python if output_dir.exists() and skip_if_exists_pws: print( f"Output ({output_dir}) already exists and skip_if_exists=True. Using existing run." ) else: input_dir_cp.mkdir(exist_ok=True, parents=True) for ff in prms_dbl_run_dir.glob("*.nc"): copy2(ff, input_dir_cp / ff.name) for ff in (prms_dbl_run_dir / "output").glob("*.nc"): copy2(ff, input_dir_cp / ff.name) submodel = pws.Model( process, control=control, parameters=params, ) submodel.run(finalize=True) ``` -------------------------------- ### Declare humidity_cbh_flag control parameter in C Source: https://github.com/doi-usgs/pywatershed/blob/develop/humidity_bug_prms.md This C code snippet declares the `humidity_cbh_flag` control parameter, initializing it to 0 (OFF). This is part of the setup for control parameters in PRMS. ```c lval = (long *)umalloc (sizeof (long)); lval[0] = 0; decl_control_int_array ("humidity_cbh_flag", 1, lval); ``` -------------------------------- ### Initialize pynhm components Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-api-pynhm.ipynb Copies necessary parameter and control files for pynhm initialization. These files are essential for setting up the pynhm model simulation. ```python # Have to bring in or create some file # This parameter dictionary was adapted from PRMS6 parameter file param_file = root_dir / 'sagehen_params.pkl' _ = shutil.copy2(pynhm_dir / 'examples/sagehen/sagehen_params.pkl', param_file) # Control file control_file = root_dir / 'pywatershed.control' _ = shutil.copy2(pynhm_dir / 'examples/sagehen/pywatershed.control', control_file) ``` -------------------------------- ### Initialize pynhm parameters and control Source: https://github.com/doi-usgs/pywatershed/blob/develop/examples/sagehen/sagehen-api-pynhm.ipynb Loads pynhm parameters from a dictionary and initializes the control object using the specified control file. This prepares pynhm for simulation. ```python params = pywatershed.PrmsParameters(parameter_dict=param_dict) control = pywatershed.Control.load(control_file, params=params) ```