### Sphinx Documentation Structure Example Source: https://github.com/eiagov/bluesky/blob/main/docs/source/index.rst An example of how Sphinx organizes the documentation structure for the BlueSky project, showing packages, subpackages, and modules. ```text src (package)/ ├── integrator (subpackage)/ │ ├── input │ └── runner.py (module) └── models (subpackage)/ ├── electricity (package)/ │ └── scripts (subpackage)/ │ ├── electricity_model.py (module) │ └── preprocessor.py ├── hydrogen/ │ ├── utilities/ │ │ └── h2_functions.py │ ├── model/ │ │ └── h2_model.py │ └── etc. └── residential/ └── scripts/ ├── residential.py └── utilites.py ``` -------------------------------- ### Configuration Setup Functions Source: https://github.com/eiagov/bluesky/blob/main/docs/source/src.common.config_setup.rst Provides functions for setting up configuration, such as creating directories and temporal mappings. ```python def create_temporal_mapping(): """Creates a temporal mapping.""" pass def make_dir(path): """Creates a directory at the specified path.""" pass ``` -------------------------------- ### Configuration Setup Functions Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_sources/src.common.config_setup.rst.txt Provides functions for setting up configuration, such as creating directories and temporal mappings. ```python def create_temporal_mapping(): """Creates a temporal mapping.""" pass def make_dir(path): """Creates a directory at the specified path.""" pass ``` -------------------------------- ### Configuration Setup Classes Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_sources/src.common.config_setup.rst.txt Includes classes for managing configuration settings, paths, and datetime objects. ```python class Config_settings: """Manages configuration settings.""" pass class Path: """Represents a file path.""" pass class datetime: """Represents a datetime object.""" pass ``` -------------------------------- ### Configuration Setup Classes Source: https://github.com/eiagov/bluesky/blob/main/docs/source/src.common.config_setup.rst Includes classes for managing configuration settings, paths, and datetime objects. ```python class Config_settings: """Manages configuration settings.""" pass class Path: """Represents a file path.""" pass class datetime: """Represents a datetime object.""" pass ``` -------------------------------- ### Sphinx Documentation Structure Example Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_sources/index.rst.txt An example of how Sphinx organizes the documentation structure for the BlueSky project, showing packages, subpackages, and modules. ```text src (package)/ ├── integrator (subpackage)/ │ ├── input │ └── runner.py (module) └── models (subpackage)/ ├── electricity (package)/ │ └── scripts (subpackage)/ │ ├── electricity_model.py (module) │ └── preprocessor.py ├── hydrogen/ │ ├── utilities/ │ │ └── h2_functions.py │ ├── model/ │ │ └── h2_model.py │ └── etc. └── residential/ └── scripts/ ├── residential.py └── utilites.py ``` -------------------------------- ### Snakemake Configuration Example Source: https://github.com/eiagov/bluesky/blob/main/sample/baseload_data_pipeline/README.md An example of a `config.yaml` file for a Snakemake pipeline. Key parameters like `db_directory`, `noaa_start_year`, and `num_stations` are shown, with guidance on which parameters to modify for testing. ```yaml db_directory: "/path/to/your/personal/directory" noaa_start_year: "2019" num_stations: "2" # Other configurations should not be changed. ``` -------------------------------- ### Example Snakemake Execution Source: https://github.com/eiagov/bluesky/blob/main/sample/baseload_data_pipeline/README.md An example of executing a Snakemake workflow to generate the 'outputs/shapefiles/EIA_BA.gpkg' file using 4 cores. ```bash snakemake --cores 4 "outputs/shapefiles/EIA_BA.gpkg" ``` -------------------------------- ### Environment Setup (Windows) Source: https://github.com/eiagov/bluesky/blob/main/README.md This snippet shows the commands to set up the Conda environment for the BlueSky Prototype on a Windows system. It involves navigating to the project directory and running a setup script. ```bash cd path\to\your\BlueSkyPrototype .\envs\env-setup.bat ``` -------------------------------- ### Sphinx Documentation Navigation Links Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/index.md Provides example links to Sphinx-generated documentation pages for different modules and packages within the BlueSky project. ```text [`src.models`](src.models.md#module-src.models) [`src`](src.md#module-src) ``` -------------------------------- ### Common Utilities and Configuration Source: https://github.com/eiagov/bluesky/blob/main/docs/run_log/sphinx_build_html_log.txt Core utilities and configuration setup used across the project. ```python src.common.config_setup src.common.model src.common.utilities ``` -------------------------------- ### Logger Setup Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/genindex.html Sets up the logging mechanism for the project. This ensures that all operations are properly logged for debugging and monitoring. ```python from src.common.utilities import setup_logger # Example usage: # logger = setup_logger('my_module') # logger.info('This is an informational message.') ``` -------------------------------- ### Setup Logger Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/src.common.utilities.html Initializes the logging system and sets up the logger in the specified output directory. Requires a settings object that includes the output directory path. ```python src.common.utilities.setup_logger(_settings_) Parameters: output_dir (path): output directory path ``` -------------------------------- ### Toy Model Example Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/sensitivity/faster_sensitivity.html An example class demonstrating how to create a model and obtain its sensitivity matrix. It initializes a Pyomo model, converts it to an AutoSympy model, calculates the sensitivity matrix, and saves it to a CSV file. ```python class toy_model: """An example of the method in action that scales by the given 'n' value""" def __init__(self, n): self.n = n mod = self.create_model() sympy_model = AutoSympy(mod) sensitivity = sympy_model.get_sensitivity_matrix() np.savetxt('partials.csv', sensitivity.partials, '%d') # print(type(sensitivity.partials)) # print(sensitivity.partials) ``` -------------------------------- ### Common Utilities Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/modules.md Provides utility functions for configuration, directory management, argument parsing, and logger setup. ```python from src.common.config_setup import Config_settings from src.common.utilities import get_args, make_dir, scale_load, scale_load_with_enduses, setup_logger ``` -------------------------------- ### Logger Utilities Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_sources/src.common.utilities.rst.txt Provides functions for setting up and getting loggers, essential for application monitoring and debugging. ```python from logging import getLogger def getLogger(name: str): """Get a logger instance.""" pass def setup_logger(name: str, level: int = 20): """Setup a logger instance.""" pass ``` -------------------------------- ### Common Utilities Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/modules.html Provides essential utility functions for the BlueSky model, including argument parsing, directory creation, load scaling, and logger setup. ```python from src.common.utilities import get_args, make_dir, scale_load, scale_load_with_enduses, setup_logger # Example usage: args = get_args() make_dir("output_data") scaled_data = scale_load(input_data) logger = setup_logger(__name__) ``` -------------------------------- ### Common Utilities and Configuration Source: https://github.com/eiagov/bluesky/blob/main/docs/run_log/sphinx_build_mkdn_log.txt Details the common utilities and configuration setup used across the Bluesky project. This includes configuration management and general utility functions. ```python src.common.config_setup src.common.utilities ``` -------------------------------- ### Python Logging Example Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/src.sensitivity.speed_test.html Example of how to get a logger in Python. ```Python import logging logger = logging.getLogger('my_module') logger.setLevel(logging.INFO) # Example usage: logger.info('This is an informational message.') ``` -------------------------------- ### toy_model Function Example Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.sensitivity.faster_sensitivity.md An example function demonstrating the sensitivity analysis method in action. It scales by a given 'n' value. ```python def toy_model(n): """An example of the method in action that scales by the given 'n' value""" pass ``` -------------------------------- ### Run main script with help Source: https://github.com/eiagov/bluesky/blob/main/sample/electricity_data_pipeline/README.md Executes the main script with the --help flag to display command-line interface options and script description. This is a setup step to verify the script's functionality. ```bash python main.py --help ``` -------------------------------- ### Toy Model Example Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/src.sensitivity.faster_sensitivity.html An example class demonstrating a model that scales by a given 'n' value, with a method to create the model instance. ```APIDOC toy_model(_n_) An example of the method in action that scales by the given ‘n’ value create_model() Creates an instance of the toy_model. ``` -------------------------------- ### Logging Utilities Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.common.utilities.md Provides functions for setting up and retrieving loggers. `getLogger` returns a logger instance, while `setup_logger` initializes the logging system and configures the output directory. ```APIDOC getLogger([name]) Return a logger with the specified name, creating it if necessary. setup_logger(settings) Initiates logging and sets up the logger in the specified output directory. Parameters: output_dir (path): The path to the output directory. ``` -------------------------------- ### Utility Functions Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/src.common.config_setup.html Provides utility functions for managing model configurations and directories. `create_temporal_mapping` combines input mapping files for the electricity model, while `make_dir` generates output directories for model results. ```python def create_temporal_mapping(sw_temporal): """Combines the input mapping files within the electricity model to create a master temporal mapping dataframe.""" pass def make_dir(dir_name): """generates an output directory to write model results, output directory is the date/time at the time this function executes.""" pass ``` -------------------------------- ### Bluesky Initialization and Configuration Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/integrator/config_setup.html This snippet covers the core initialization logic for the Bluesky project. It sets up the output directory, configures various modules based on a provided configuration, and determines the run method based on the selected operational mode and module availability. It also handles temporal and spatial configurations, including year mapping and region selection. ```python import logging import pandas as pd import numpy as np from datetime import datetime # Assume make_dir, create_temporal_mapping, and logger are defined elsewhere # from utils import make_dir, create_temporal_mapping # logger = logging.getLogger(__name__) class Bluesky: def __init__(self, config, selected_mode, regions_ow=[], years_ow=[], test=False): # Initialize output directory OUTPUT_ROOT = f"{self.selected_mode}_{datetime.now().strftime('%Y_%m_%d_%H%Mh')}" # make_dir(OUTPUT_ROOT) with open('output_root.txt', 'w') as file: file.write(str(OUTPUT_ROOT)) self.OUTPUT_ROOT = OUTPUT_ROOT # Set modules from config self.electricity = config['electricity'] self.hydrogen = config['hydrogen'] self.residential = config['residential'] # Redirects and raises based on conditions if (not any((self.electricity, self.hydrogen, self.residential))) and ( self.selected_mode in ('unified-combo', 'gs-combo', 'standalone') ): raise ValueError('No modules turned on; check run_config.toml') # Single module case if [self.electricity, self.hydrogen, self.residential].count(True) == 1: print('you selected a combo mode, but only one module is turned on') self.run_method = 'run_standalone' # Combinations of Modules and Mode --> run guidance match self.selected_mode: case 'unified-combo': if self.hydrogen and self.residential and not self.electricity: print('not an available option, running default version') print( 'running unified-combo with electricity, hydrogen, and residential modules' ) self.electricity = True self.run_method = 'run_unified' self.method_options = {} case 'gs-combo': if self.hydrogen and self.residential and not self.electricity: print('not an available option, running default version') print('running gs-combo with electricity, hydrogen, and residential modules') self.electricity = True self.run_method = 'run_gs' self.method_options = { 'update_h2_price': self.hydrogen, 'update_elec_price': True, 'update_h2_demand': self.hydrogen, 'update_load': self.residential, 'update_elec_demand': False, } case 'standalone': self.run_method = 'run_standalone' case 'elec': self.run_method = 'run_elec_solo' case 'h2': self.run_method = 'run_h2_solo' case 'residential': self.run_method = 'run_residential_solo' case _: # logger.error('Unkown op mode specified... exiting') pass # Iterative Solve Configs self.tol = config['tol'] self.force_10 = config['force_10'] self.max_iter = config['max_iter'] # Spatial Configs if not test or len(regions_ow) == 0: # self.regions = list(pd.read_csv(config['regions']).dropna()['region']) pass # Placeholder for actual region loading else: self.regions = regions_ow # Temporal Configs self.sw_temporal = config['sw_temporal'] # self.cw_temporal = create_temporal_mapping(self.sw_temporal) # Temporal Configs - Years self.sw_agg_years = config['sw_agg_years'] # year_frame = pd.read_csv(config['years']) if not test or len(years_ow) == 0: # self.years = list(year_frame.dropna()['year']) pass # Placeholder for actual year loading else: self.years = years_ow if self.sw_agg_years and len(self.years) > 1: # self.start_year = year_frame['year'][0] # all_years_list = list(range(self.start_year, self.years[-1] + 1)) pass # Placeholder for year range calculation else: self.start_year = self.years[0] # all_years_list = self.years # solve_array = np.array(self.years) # mapped_list = [solve_array[solve_array >= year].min() for year in all_years_list] # self.year_map = pd.DataFrame({'y': all_years_list, 'Map_y': mapped_list}) # self.year_weights = ( # self.year_map.groupby(['Map_y']) # .agg('count') # .reset_index() # .rename(columns={'Map_y': 'y', 'y': 'year_weights'}) # ) # Electricity Configs self.sw_trade = config['sw_trade'] self.sw_expansion = config['sw_expansion'] self.sw_rm = config['sw_rm'] ``` -------------------------------- ### Pandas DataFrame Join Examples Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/pandas/core/frame.html Demonstrates joining a DataFrame with another DataFrame or Series using different strategies. Includes examples of 'm:1' validation. ```python >>> df = pd.DataFrame({'key': ['K0', 'K1', 'K1', 'K3', 'K0', 'K1'], ... 'A': ['A0', 'A1', 'A2', 'A3', 'A4', 'A5']}) >>> other = pd.DataFrame({'key': ['K0', 'K1', 'K2'], ... 'B': ['B0', 'B1', 'B2']}) >>> df key A 0 K0 A0 1 K1 A1 2 K1 A2 3 K3 A3 4 K0 A4 5 K1 A5 >>> df.join(other.set_index('key'), on='key', validate='m:1') key A B 0 K0 A0 B0 1 K1 A1 B1 2 K1 A2 B1 3 K3 A3 NaN 4 K0 A4 B0 5 K1 A5 B1 ``` -------------------------------- ### Config_settings Class API Documentation Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.common.config_setup.md Provides detailed API documentation for the Config_settings class, including its constructor and various validation methods for configuration parameters. It covers settings related to iterative solve, spatial, temporal, and electricity configurations. ```APIDOC Config_settings(config_path: Path, args: Namespace | None = None, test=False) Generates the model settings that are used to solve. Settings include: - Iterative Solve Config Settings - Spatial Config Settings - Temporal Config Settings - Electricity Config Settings - Other _additional_year_settings(name, value) Checks year related settings to see if values are within expected ranges and updates other settings linked to years if years is changed. Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _check_elec_expansion_settings(name, value) Checks that switches for reserve margin and learning are on only if expansion is on. Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _check_int(name, value) Checks if attribute is an integer Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _check_regions(name, value) Checks to see if region is between the current default values of 1 and 25. Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _check_res_settings(name, value) Checks if view year or region settings are subsets of year or region Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _check_true_false(name, value) Checks if attribute is either true or false Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _check_zero_one(name, value) Checks if attribute is either zero or one Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error _has_all_attributes(attrs: set) Determines if all attributes within the set exist or not Parameters: attrs (set): set of setting attributes Returns: True or False Return type: bool ``` -------------------------------- ### String Extraction Utility Source: https://github.com/eiagov/bluesky/blob/main/sample/aimms_sympy/aimms_sympy.ipynb A utility function `extract_between` that finds and returns the substring located between the first occurrence of a specified start character and the first occurrence of a specified end character after the start. ```python def extract_between(text, start_char, end_char): start_index = text.find(start_char) if start_index == -1: return '' start_index += len(start_char) end_index = text.find(end_char, start_index) if end_index == -1: return '' return text[start_index:end_index] ``` -------------------------------- ### toy_model Function for Sensitivity Analysis Example Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/src.sensitivity.faster_sensitivity.html The toy_model function demonstrates the sensitivity analysis method by scaling based on an input value 'n'. It can be used as an example or imported into other modules. ```python def toy_model(n): """An example of the method in action that scales by the given 'n' value""" pass ``` -------------------------------- ### Initialization of Configuration Parameters Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/common/config_setup.html Initializes various configuration parameters for the simulation, including iterative solve settings, spatial configurations, temporal configurations, electricity configurations, and hydrogen configurations. ```python # __INIT__: Iterative Solve Configs self.tol = config['tol'] self.force_10 = config['force_10'] self.max_iter = config['max_iter'] # __INIT__: Spatial Configs self.regions = config['regions'] # __INIT__: Temporal Configs self.sw_temporal = config['sw_temporal'] self.cw_temporal = create_temporal_mapping(self.sw_temporal) # __INIT__: Temporal Configs - self.sw_agg_years = config['sw_agg_years'] self.years = config['years'] if self.sw_agg_years == 1: self.start_year = config['start_year'] else: self.start_year = self.years[0] ############################################################################################ # __INIT__: Electricity Configs self.sw_trade = config['sw_trade'] self.sw_rm = config['sw_rm'] self.sw_ramp = config['sw_ramp'] self.sw_reserves = config['sw_reserves'] self.sw_learning = config['sw_learning'] self.sw_expansion = config['sw_expansion'] ############################################################################################ # __INIT__: Residential Configs self.scale_load = config['scale_load'] if not test: self.view_regions = config['view_regions'] self.view_years = config['view_years'] self.sensitivity = config['sensitivity'] self.change_var = config['change_var'] self.percent_change = config['percent_change'] self.complex = config['complex'] ############################################################################################ # __INIT__: Hydrogen Configs self.h2_data_folder = self.PROJECT_ROOT / config['h2_data_folder'] ``` -------------------------------- ### Snakemake Data Pipeline Example Source: https://github.com/eiagov/bluesky/blob/main/README.md An example data pipeline documented with Snakemake is used to create a geospatial crosswalk involving electricity operations and weather data. Snakemake helps organize, pre-process, and post-process model data. ```Python rule create_geospatial_crosswalk: input: electricity_ops = 'data/electricity_operations.csv', weather_data = 'data/weather_data.csv' output: crosswalk = 'results/geospatial_crosswalk.csv' shell: 'python scripts/create_crosswalk.py {input.electricity_ops} {input.weather_data} {output.crosswalk}' ``` -------------------------------- ### Settings Class Setup (config_setup.py) Source: https://github.com/eiagov/bluesky/blob/main/sample/integration_rebuild/README.md The Settings class in config_setup.py is responsible for creating instructions for building modules, setting up crosswalks for Interchange, and determining the solve algorithm based on configuration. It's recommended to instantiate this class via main.py to explore its attributes. ```python from common.config_setup import Settings # Instantiate settings using main.py (example) # settings = Settings() # Explore attributes to understand build functions # print(settings.solve_algorithm) ``` -------------------------------- ### Pandas pivot_table Examples Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/pandas/core/frame.html Demonstrates various ways to use the pivot_table function in pandas for data aggregation and reshaping. Includes examples for summing values, filling missing data, and calculating means across multiple columns. ```python import pandas as pd df = pd.DataFrame({ "A": ["foo", "foo", "foo", "foo", "foo", "bar", "bar", "bar", "bar"], "B": ["one", "one", "one", "two", "two", "one", "one", "two", "two"], "C": ["small", "large", "large", "small", "small", "large", "small", "small", "large"], "D": [1, 2, 2, 3, 3, 4, 5, 6, 7], "E": [2, 4, 5, 5, 6, 6, 8, 9, 9] }) # Aggregating values by taking the sum table = pd.pivot_table(df, values='D', index=['A', 'B'], columns=['C'], aggfunc="sum") print(table) # Filling missing values table = pd.pivot_table(df, values='D', index=['A', 'B'], columns=['C'], aggfunc="sum", fill_value=0) print(table) # Aggregating by taking the mean across multiple columns table = pd.pivot_table(df, values=['D', 'E'], index=['A', 'C'], aggfunc={'D': "mean", 'E': "mean"}) print(table) # Calculating multiple types of aggregations table = pd.pivot_table(df, values=['D', 'E'], index=['A', 'C'], aggfunc={'D': "mean", 'E': ["min", "max", "mean"]}) print(table) ``` -------------------------------- ### Config_settings Initialization Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/integrator/config_setup.html Initializes the Config_settings class, loading configurations from a TOML file and setting up operational modes and output paths. It handles default modes and arguments passed to the model. ```python from logging import getLogger import pandas as pd import numpy as np import tomllib from pathlib import Path from datetime import datetime import types import argparse from definitions import PROJECT_ROOT from src.integrator.utilities import create_temporal_mapping from src.integrator.utilities import make_dir logger = getLogger(__name__) class Config_settings: """Generates the model settings that are used to solve. Settings include: - Iterative Solve Config Settings - Spatial Config Settings - Temporal Config Settings - Electricity Config Settings - Other """ def __init__( self, config_path: Path, args: argparse.Namespace | None = None, test=False, years_ow=[], regions_ow=[], ): """Creates configuration object upon instantiation Parameters ---------- config_path : Path Path to run_config.toml args : Namespace Parsed arguments fed to main.py or other parsed object test : bool, optional _description_, by default False years_ow : list, optional _description_, by default [] regions_ow : list, optional _description_, by default [] Raises ------ ValueError No modules turned on; check run_config.toml ValueError sw_expansion: Must turn RM switch off if no expansion ValueError sw_expansion: Must turn learning switch off if no expansion """ # __INIT__: Grab arguments namespace and set paths self.args = args if not args: self.args = types.SimpleNamespace() self.args.op_mode = None self.args.debug = False self.PROJECT_ROOT = PROJECT_ROOT # __INIT__: Dump toml, sse args to set mode with open(config_path, 'rb') as src: config = tomllib.load(src) ############################################################################################ # Universal Configs # __INIT__: Default Solve Mode self.default_mode = config['default_mode'] # __INIT__: If no mode is specified read default mode from TOML if not self.args.op_mode: print('No mode arg passed, therefore...') self.selected_mode = self.default_mode print(f'using mode {self.default_mode} specified in run_config file') else: self.selected_mode = self.args.op_mode # __INIT__: Setting output paths # Setup the output directory and write out its name for other scripts to grab OUTPUT_ROOT = Path( PROJECT_ROOT / 'output' ``` -------------------------------- ### Get Logger Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.hydrogen.model.actions.md Returns a logger with the specified name, creating it if necessary. ```python def getLogger([name]): """Return a logger with the specified name, creating it if necessary.""" pass ``` -------------------------------- ### Code Documentation Link Source: https://github.com/eiagov/bluesky/blob/main/src/common/README.md Provides a link to the detailed code documentation, including instructions for compiling Sphinx. ```markdown [Code Documentation](/docs/README.md) ``` -------------------------------- ### Get Gas Price Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.hydrogen.utilities.h2_functions.md Retrieves the gas price for a specific region and year. ```APIDOC get_gas_price(hm: H2Model, region: str, year: int) Parameters: hm: H2Model - The H2 model instance. region: str - The region for which to get the gas price. year: int - The year for which to get the gas price. Returns: float - The gas price in the specified region and year. ``` -------------------------------- ### Get Electricity Consumption Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.hydrogen.utilities.h2_functions.md Retrieves the electricity consumption for a specific region and year. ```APIDOC get_electricty_consumption(hm: H2Model, region: str, year: int) Parameters: hm: H2Model - The H2 model instance. region: str - The region for which to get electricity consumption. year: int - The year for which to get electricity consumption. Returns: float - The electricity consumption for the specified region and year. ``` -------------------------------- ### Get Electricity Price Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.hydrogen.utilities.h2_functions.md Retrieves the electricity price for a specific region and year. ```APIDOC get_elec_price(hm: H2Model, region: str, year: int) Parameters: hm: H2Model - The H2 model instance. region: str - The region for which to get the electricity price. year: int - The year for which to get the electricity price. Returns: float - The electricity price in the specified region and year. ``` -------------------------------- ### Config_settings Class Initialization Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/common/config_setup.html Initializes the Config_settings class, loading configurations from a TOML file and setting up output directories. It parses arguments, determines the operational mode, and configures module-specific settings (electricity, hydrogen, residential). It includes validation for module activation and sets the run method based on the selected mode. ```python import pandas as pd import numpy as np import tomllib from pathlib import Path from datetime import datetime import types import argparse from definitions import PROJECT_ROOT from src.integrator.utilities import create_temporal_mapping from src.common.utilities import make_dir class Config_settings: """Generates the model settings that are used to solve. Settings include: \n - Iterative Solve Config Settings \n - Spatial Config Settings \n - Temporal Config Settings \n - Electricity Config Settings \n - Other """ def __init__(self, config_path: Path, args: argparse.Namespace | None = None, test=False): """Creates configuration object upon instantiation Parameters ---------- config_path : Path Path to run_config.toml args : Namespace Parsed arguments fed to main.py or other parsed object test : bool, optional Used only for unit testing in unit_tests directory, by default False Raises ------ ValueError No modules turned on; check run_config.toml ValueError sw_expansion: Must turn RM switch off if no expansion ValueError sw_expansion: Must turn learning switch off if no expansion """ # __INIT__: Grab arguments namespace and set paths self.missing_checks = set() self.test = test self.args = args if not args: self.args = types.SimpleNamespace() self.args.op_mode = None self.args.debug = False self.PROJECT_ROOT = PROJECT_ROOT # __INIT__: Dump toml, sse args to set mode with open(config_path, 'rb') as src: config = tomllib.load(src) ############################################################################################ # Universal Configs # __INIT__: Solve Mode self.default_mode = config['default_mode'] if not self.args.op_mode: print('No mode arg passed, therefore...') self.selected_mode = self.default_mode print(f'using mode {self.default_mode} specified in run_config file') else: self.selected_mode = self.args.op_mode # __INIT__: Setting output paths # Setup the output directory and write out its name for other scripts to grab if test: OUTPUT_ROOT = Path(PROJECT_ROOT, 'unit_tests', 'test_logs') else: output_folder_name = f"{self.selected_mode}_{datetime.now().strftime('%Y_%m_%d_%H%Mh')}" OUTPUT_ROOT = Path(PROJECT_ROOT / 'output' / output_folder_name) self.OUTPUT_ROOT = OUTPUT_ROOT make_dir(self.OUTPUT_ROOT) ##### ### __INIT__: Methods and Modules Configuration ##### # __INIT__: Set modules from config self.electricity = config['electricity'] self.hydrogen = config['hydrogen'] self.residential = config['residential'] # __INIT__: Redirects and raises based on conditions if (not any((self.electricity, self.hydrogen, self.residential))) and ( self.selected_mode in ('unified-combo', 'gs-combo', 'standalone') ): raise ValueError('No modules turned on; check run_config.toml') # __INIT__: Single module case if [self.electricity, self.hydrogen, self.residential].count(True) == 1: print('Only one module is turned on; running standalone mode') self.run_method = 'run_standalone' # __INIT__: Combinations of Modules and Mode --> run guidance match self.selected_mode: case 'unified-combo': # No elec case pass ``` -------------------------------- ### Config_settings API Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/src.common.config_setup.html API documentation for the Config_settings class within the src.common.config_setup module. Includes methods for attribute validation. ```APIDOC Config_settings: _check_zero_one(_name_: str, _value_: _type_) Parameters: name (str): attribute name value (_type_): attribute value Raises: TypeError: Error if value is not zero or one _has_all_attributes(attrs: set) Parameters: attrs (set): set of setting attributes Returns: bool: True if all attributes exist, False otherwise ``` -------------------------------- ### Run Unified Solve Method Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.integrator.unified.md Executes the unified solve method using configuration settings. It takes an instance of `Config_settings` which contains various options, mode, and settings for the run. ```python from src.common.config_setup import Config_settings def run_unified(settings: Config_settings): """ Runs unified solve method based on * **Parameters:** **settings** ([*Config_settings*](src.common.config_setup.md#src.common.config_setup.Config_settings)) – Instance of config_settings containing run options, mode and settings """ pass ``` -------------------------------- ### Initialize Residential Module Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/models/residential/scripts/residential.html Initializes the residential module, setting up demand calculations, loading base load and price data, and creating temporal mappings. It handles different scaling methods for load data and ensures base year and hour maps are set. ```python self.demand = self.consumption[self.BaseYr, self.reg, self.fuel] * self.QIndex # Convert equation into lambda function for use in python self.lambdifiedDemand = sp.lambdify( [ self.incomeIndex[self.year - 1, self.reg, self.fuel], self.i_lag[self.reg, self.fuel], self.income[self.year, self.reg, self.fuel], self.income[self.BaseYr, self.reg, self.fuel], self.i_elas[self.reg, self.fuel], self.priceIndex[self.year - 1, self.reg, self.fuel], self.p_lag[self.reg, self.fuel], self.price[self.year, self.reg, self.fuel], self.price[self.BaseYr, self.reg, self.fuel], self.p_elas[self.reg, self.fuel], self.year, self.LastHYr, self.LastMYr, self.trendGR[self.reg, self.fuel], self.consumption[self.BaseYr, self.reg, self.fuel], ], self.demand, ) # Set base Load values if they aren't already set if loadFile: self.loads['BaseLoads'] = pd.read_csv(loadFile).set_index( ['region', 'year', 'hour'], drop=False ) elif not self.loads: # read in load data from residential input directory self.load_scalar = settings.scale_load res_dir = Path(PROJECT_ROOT, 'input', 'residential') if self.load_scalar == 'annual': self.loads['BaseLoads'] = scale_load(res_dir) elif self.load_scalar == 'enduse': self.loads['BaseLoads'] = scale_load_with_enduses(res_dir) else: raise ValueError('load_scalar in TOML must be set to "annual" or "enduse"') self.loads['BaseLoads'].set_index(['region', 'year', 'hour'], drop=False, inplace=True) # Set base year if not already set if not self.baseYear: # TODO: How could this below work after y has been made part of the index? self.baseYear = self.loads['BaseLoads'].year.min() # Create hour map if not already set if self.hr_map.empty: # TODO: update config path reference self.hr_map = settings.cw_temporal self.hr_map.set_index('hour', inplace=True, drop=False) pass # Create base price values if they aren't already set if 'BasePrices' not in residentialModule.prices.keys(): path_to_price_data = PROJECT_ROOT / 'input/residential/BaseElecPrice.csv' residentialModule.prices['BasePrices'] = pd.read_csv(path_to_price_data).set_index( ['region', 'year', 'hour'], drop=False ) self.calibrate = calibrate # Make lambdifiedDemand function easier to use with named inputs and default values ``` -------------------------------- ### Code Documentation Access Source: https://github.com/eiagov/bluesky/blob/main/src/integrator/README.md Instructions for accessing and compiling code documentation. ```APIDOC Code Documentation: Navigate to the docs README for details on code documentation and instructions for locally compiling Sphinx to generate an HTML version of the documentation. Link: /docs/README.md ``` -------------------------------- ### Logger Utilities Source: https://github.com/eiagov/bluesky/blob/main/docs/source/src.common.utilities.rst Provides functions for setting up and getting loggers, essential for application monitoring and debugging. ```python from logging import getLogger def getLogger(name: str): """Get a logger instance.""" pass def setup_logger(name: str, level: int = 20): """Setup a logger instance.""" pass ``` -------------------------------- ### Get Logger Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.integrator.utilities.md Returns a logger instance with the specified name. If a logger with the given name does not exist, it will be created. ```python def getLogger(name): """Return a logger with the specified name, creating it if necessary.""" pass ``` -------------------------------- ### Setup Logger Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/common/utilities.html Initiates logging and sets up the logger in the specified output directory. It configures the logging level based on a debug flag and sets the format for log messages. It also suppresses warnings from 'pyomo', 'pandas', and 'matplotlib' libraries. ```python # Logger Setup [[docs]](../../../src.common.utilities.html#src.common.utilities.setup_logger) def setup_logger(settings): """initiates logging, sets up logger in the output directory specified Parameters ---------- output_dir : path output directory path """ # set up root logger output_dir = settings.OUTPUT_ROOT log_path = Path(output_dir) if not Path.is_dir(log_path): Path.mkdir(log_path) # logger level if settings.args.debug: loglevel = logging.DEBUG else: loglevel = logging.INFO # logger configs logging.basicConfig( filename=f'{output_dir}/run.log', encoding='utf-8', filemode='w', # format='[%(asctime)s][%(name)s]'+'[%(funcName)s][%(levelname)s] :: |%(message)s|', format='%(asctime)s | %(name)s | %(levelname)s :: %(message)s', datefmt='%d-%b-%y %H:%M:%S', level=loglevel, ) logging.getLogger('pyomo').setLevel(logging.WARNING) logging.getLogger('pandas').setLevel(logging.WARNING) logging.getLogger('matplotlib').setLevel(logging.WARNING) ``` -------------------------------- ### Python Configuration Initialization Source: https://github.com/eiagov/bluesky/blob/main/docs/build/html/_modules/src/integrator/config_setup.html Initializes project configurations, including various switches and data paths. It enforces constraints between switches, such as disabling RM and learning if expansion is off. It also loads residential and hydrogen-specific configurations. ```python self.sw_ramp = config['sw_ramp'] self.sw_reserves = config['sw_reserves'] self.sw_learning = config['sw_learning'] # __INIT__: throwing errors if certain combinations of switches if self.sw_expansion == 0: # expansion off if self.sw_rm == 1: raise ValueError('sw_expansion: Must turn RM switch off if no expansion') if self.sw_learning == 1: raise ValueError('sw_expansion: Must turn learning switch off if no expansion') ############################################################################################ # Residential Configs if not test or len(regions_ow) == 0: self.view_regions = config['view_regions'] self.view_years = config['view_years'] self.sensitivity = config['sensitivity'] self.change_var = config['change_var'] self.percent_change = config['percent_change'] self.complex = config['complex'] ############################################################################################ # Hydrogen Configs self.h2_data_folder = self.PROJECT_ROOT / config['h2_data_folder'] ``` -------------------------------- ### Get Production Cost Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.hydrogen.utilities.h2_functions.md Returns the production cost for a specific technology at a given hub and year. ```APIDOC get_production_cost(hm: H2Model, hub: str, tech: str, year: int) Parameters: hm: H2Model - The H2 model instance. hub: str - The hub where production occurs. tech: str - The technology type used for production. year: int - The year for which to get the production cost. Returns: float - The production cost of H2 for the specified technology, hub, and year. ``` -------------------------------- ### Display Help for Main Script Source: https://github.com/eiagov/bluesky/blob/main/sample/enduse_data_pipeline/README.md Displays the help message for the `main.py` script, outlining its functionality and available command-line arguments. This is useful for understanding the script's options before execution. ```bash python main.py --help ``` -------------------------------- ### Common Utilities and Configuration Source: https://github.com/eiagov/bluesky/blob/main/docs/run_log/sphinx_build_html_log.txt Details the common utilities and configuration setup within the Bluesky project. This includes general helper functions, configuration management, and data models used across different modules. ```rst src.common.rst src.common.config_setup.rst src.common.model.rst src.common.utilities.rst ``` -------------------------------- ### Get Electricity Consumption Rate Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.hydrogen.utilities.h2_functions.md Returns the electricity consumption rate for a given technology type. ```APIDOC get_electricity_consumption_rate(hm: H2Model, tech: str) Parameters: hm: H2Model - The H2 model instance. tech: str - The technology type. Returns: float - The electricity consumption rate in GWh per kg H2. ``` -------------------------------- ### Run Configuration File Source: https://github.com/eiagov/bluesky/blob/main/src/integrator/README.md Describes the role and location of the run_config.toml file for module integration and solving. ```APIDOC run_config.toml: - Location: Common folder. - Purpose: Contains settings for integrating and solving modules. - Content: Includes shared settings across modules and module-specific settings. ``` -------------------------------- ### Get Logger Source: https://github.com/eiagov/bluesky/blob/main/docs/build/markdown/src.models.electricity.scripts.runner.md Returns a logger with the specified name, creating it if it does not exist. This is a standard logging utility. ```python def getLogger(name: str = None): """Return a logger with the specified name, creating it if necessary.""" # Function implementation details... ``` -------------------------------- ### Integration Modules Source: https://github.com/eiagov/bluesky/blob/main/docs/run_log/sphinx_build_html_log.txt Documentation for the integration modules within Bluesky. This covers functionalities related to running simulations, progress plotting, and specific integration algorithms like Gauss-Seidel. ```rst src.integrator.rst src.integrator.gaussseidel.rst src.integrator.progress_plot.rst src.integrator.runner.rst src.integrator.unified.rst src.integrator.utilities.rst ```