Use this interactive widget to tune your custom RBC! Reference the building load profiles above and the questions below when deciding on how to charge/discharge your rule-based controlled batteries.
Use the sliders to set the hourly charge and discharge rate of the batteries. Positive values indicate charging and negative values indicate discharging the batteries
""" html_ui = HTML(value=questions, placeholder='Questions') sliders = [FloatSlider( value=0.0, min=-1.0, max=1.0, step=action_step, description=f'Hr: {h}-{h + hour_step - 1}', disabled=False, continuous_update=False, orientation='vertical', readout=True, readout_format='.2f', ) for h in hours] reset_button = Button( description='Reset', disabled=False, button_style='info', tooltip='Set all hour actions to 0.0', icon='' ) random_button = Button( description='Random', disabled=False, button_style='warning', tooltip='Select random hour actions', icon='' ) simulate_button = Button( description='Simulate', disabled=False, button_style='success', tooltip='Run simulation', icon='check' ) sliders_ui = HBox(sliders) buttons_ui = HBox([reset_button, random_button, simulate_button]) # run simulation so that the environment has results ``` -------------------------------- ### Display CLI Help for Simulation Source: https://github.com/citylearn-project/citylearn/blob/master/examples/cli.ipynb Use this command to view all available arguments and options for the simulation CLI. ```bash !python -m citylearn simulate -h ``` -------------------------------- ### Configure Environment Rendering and Simulation Loop Source: https://github.com/citylearn-project/citylearn/blob/master/docs/source/ui.rst Sets up the environment with end-of-episode rendering and executes a simulation loop to collect observations and rewards. ```python episode_time_steps=48, render_mode='end', render_directory=Path('outputs/ui_exports'), render_session_name='buffered_run', ) observations, _ = env.reset() while not env.terminated: actions = [env.action_space[0].sample()] observations, reward, terminated, truncated, info = env.step(actions) ``` -------------------------------- ### Initialize CityLearnEnv Source: https://context7.com/citylearn-project/citylearn/llms.txt Demonstrates various ways to initialize the CityLearnEnv, including using named datasets, schema filepaths, or custom parameters. Accesses environment properties after initialization. ```python from citylearn.citylearn import CityLearnEnv # Initialize using a named dataset (downloads automatically if not cached) env = CityLearnEnv('citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True) # Initialize using a schema filepath env = CityLearnEnv('path/to/schema.json', central_agent=False) # Initialize with custom parameters from citylearn.utilities import FileHandler schema = FileHandler.read_json('path/to/schema.json') env = CityLearnEnv( schema, root_directory='path/to/data', central_agent=True, simulation_start_time_step=0, simulation_end_time_step=8760, # One year of hourly data random_seed=42 ) # Access environment properties print(f"Buildings: {len(env.buildings)}") print(f"Observation space: {env.observation_space}") print(f"Action space: {env.action_space}") print(f"Time steps: {env.time_steps}") ``` -------------------------------- ### Set Simulation Period in Schema Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Randomly selects a start and end time step for the simulation based on a specified number of days. ```python def set_schema_simulation_period( schema: dict, count: int, seed: int ) -> Tuple[dict, int, int]: """Randomly select environment simulation start and end time steps that cover a specified number of days. Parameters ---------- schema: dict CityLearn dataset mapping used to construct environment. count: int Number of simulation days. seed: int Seed for pseudo-random number generator. Returns ------- schema: dict CityLearn dataset mapping with `simulation_start_time_step` and `simulation_end_time_step` key-values set. simulation_start_time_step: int The first time step in schema time series files to be read when constructing the environment. simulation_end_time_step: int The last time step in schema time series files to be read when constructing the environment. """ assert 1 <= count <= 365, 'count must be between 1 and 365.' # set random seed np.random.seed(seed) # use any of the files to determine the total # number of available time steps filename = schema['buildings'][building_name]['carbon_intensity'] filepath = os.path.join(root_directory, filename) time_steps = pd.read_csv(filepath).shape[0] # set candidate simulation start time steps # spaced by the number of specified days simulation_start_time_step_list = np.arange(0, time_steps, 24*count) ``` -------------------------------- ### Download and Get Dataset Source: https://github.com/citylearn-project/citylearn/blob/master/examples/load_environment.ipynb Download a specified dataset to a local directory for inspection or use. This method returns the filepath to the dataset's schema file. ```python from citylearn.data import DataSet schema_filepath = DataSet().get_dataset('citylearn_challenge_2020_climate_zone_1', directory='citylearn_dataset') print('Schema filepath:', schema_filepath) ``` -------------------------------- ### Get Available Dataset Names Source: https://github.com/citylearn-project/citylearn/blob/master/examples/load_environment.ipynb Retrieve a list of available named datasets that can be used to initialize the CityLearn environment. These datasets are pre-packaged with simulation data. ```python from citylearn.data import DataSet dataset_names = DataSet().get_dataset_names() print(dataset_names) ``` -------------------------------- ### Load Environment Using Schema Dictionary Source: https://github.com/citylearn-project/citylearn/blob/master/examples/load_environment.ipynb Initializes the environment by reading a JSON schema into a dictionary and manually setting the root directory. ```python from citylearn.citylearn import CityLearnEnv from citylearn.utilities import FileHandler schema_filepath = 'citylearn_dataset/citylearn_challenge_2020_climate_zone_1/schema.json' schema = FileHandler.read_json(schema_filepath) schema['root_directory'] = 'citylearn_dataset/citylearn_challenge_2020_climate_zone_1' env = CityLearnEnv(schema) ``` -------------------------------- ### Get KPIs as DataFrame Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Calculates and returns simulation KPIs in a pandas DataFrame. Useful for programmatic access and further processing of electricity consumption, cost, emissions, and load-related metrics. ```python def get_kpis(env: CityLearnEnv) -> pd.DataFrame: """Returns evaluation KPIs. Electricity consumption, cost and carbon emissions KPIs are provided at the building-level and average district-level. Average daily peak, ramping and (1 - load factor) KPIs are provided at the district level. Parameters ---------- env: CityLearnEnv CityLearn environment instance. Returns ------- kpis: pd.DataFrame KPI table. """ kpis = env.evaluate() # names of KPIs to retrieve from evaluate function kpi_names = { 'electricity_consumption_total': 'Consumption', 'cost_total': 'Cost', 'carbon_emissions_total': 'Emissions', 'daily_peak_average': 'Avg. daily peak', 'ramping_average': 'Ramping', 'monthly_one_minus_load_factor_average': '1 - load factor' } kpis = kpis[ (kpis['cost_function'].isin(kpi_names)) ].dropna() kpis['cost_function'] = kpis['cost_function'].map(lambda x: kpi_names[x]) # round up the values to 3 decimal places for readability kpis['value'] = kpis['value'].round(3) # rename the column that defines the KPIs kpis = kpis.rename(columns={'cost_function': 'kpi'}) return kpis ``` -------------------------------- ### Initialize CityLearn Environment Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Initialize a new CityLearn environment instance for use with RL agents. ```python sac_env = CityLearnEnv(schema) ``` -------------------------------- ### Update Simulation Time Steps in Schema Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Randomly selects a simulation start time step and calculates the corresponding end time step based on the number of days. Updates the schema with these simulation time steps. ```python # randomly select a simulation start time step simulation_start_time_step = np.random.choice( simulation_start_time_step_list, size=1 )[0] simulation_end_time_step = simulation_start_time_step + 24*count - 1 # update schema simulation time steps schema['simulation_start_time_step'] = simulation_start_time_step schema['simulation_end_time_step'] = simulation_end_time_step return schema, simulation_start_time_step, simulation_end_time_step ``` -------------------------------- ### Run Baseline Agent Simulation Source: https://github.com/citylearn-project/citylearn/blob/master/examples/quickstart.ipynb Simulate the CityLearn environment with a baseline agent where storage actions are 0.0 and heat pumps have no action. This demonstrates the environment's default behavior. ```python from citylearn.agents.base import BaselineAgent as Agent from citylearn.citylearn import CityLearnEnv # initialize env = CityLearnEnv('citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True) model = Agent(env) # step through environment and apply agent actions observations, _ = env.reset() while not env.terminated: actions = model.predict(observations) observations, reward, info, terminated, truncated = env.step(actions) ``` -------------------------------- ### Access CityLearn CLI Help Source: https://github.com/citylearn-project/citylearn/blob/master/examples/cli.ipynb Execute this command in a terminal or PowerShell to view the available CLI subcommands and options. ```python !python -m citylearn -h ``` -------------------------------- ### Initialize Environment with Schema Filepath Source: https://github.com/citylearn-project/citylearn/blob/master/examples/load_environment.ipynb Initialize the CityLearn environment using the filepath to a schema JSON file. This approach is ideal for custom datasets or when the schema needs to be managed manually. ```python from citylearn.citylearn import CityLearnEnv schema_filepath = 'citylearn_dataset/citylearn_challenge_2020_climate_zone_1/schema.json' env = CityLearnEnv(schema_filepath) ``` -------------------------------- ### Initialize CityLearn Environment Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Initializes an instance of the CityLearn environment by parsing the configured schema to the CityLearnEnv constructor. ```python env = CityLearnEnv(schema) ``` -------------------------------- ### CityLearn CLI Source: https://github.com/citylearn-project/citylearn/blob/master/docs/source/cli.rst The CityLearn CLI allows users to interact with the project via the command line. ```APIDOC ## CLI Usage ### Description The CityLearn command line interface provides access to the project's main functionality. ### Usage `python -m citylearn [arguments]` ``` -------------------------------- ### Initialize and Train Tabular Q-Learning Model Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Initializes the CustomTabularQLearning model with the environment, data loader, random seed, and defined hyperparameters. It then proceeds to train the model for the calculated number of episodes. ```python tql_model = CustomTabularQLearning( env=tql_env, loader=loader, random_seed=RANDOM_SEED, **tql_kwargs ) _ = tql_model.learn(episodes=tql_episodes) ``` -------------------------------- ### Initialize CityLearn Environment Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Initializes the environment instance required for the agent. ```python rbc_env = CityLearnEnv(schema) ``` -------------------------------- ### Environment Step and Reset Source: https://context7.com/citylearn-project/citylearn/llms.txt Illustrates the core Gymnasium methods for running simulations. `reset()` initializes the environment, and `step()` applies actions to advance the simulation, returning observations, rewards, and termination status. ```python from citylearn.citylearn import CityLearnEnv env = CityLearnEnv('citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True) # Reset environment and get initial observations observations, info = env.reset() # Run simulation loop total_reward = 0 while not env.terminated: # Sample random actions (replace with agent actions) actions = [env.action_space[0].sample()] # Step environment next_observations, rewards, terminated, truncated, info = env.step(actions) total_reward += sum(rewards) observations = next_observations print(f"Episode complete. Total reward: {total_reward}") print(f"Final time step: {env.time_step}") ``` -------------------------------- ### Initialize and Train SAC Agent Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Sets up the training loader and executes the learning process for the SAC agent using a custom callback. ```python print('Number of episodes to train:', sac_episodes) sac_modr_loader = get_loader(max=sac_total_timesteps) display(sac_modr_loader) # ----------------------- TRAIN AGENT ---------------------- sacr_callback = CustomCallback(env=sacr_env, loader=sac_modr_loader) sacr_model = sacr_model.learn( total_timesteps=sac_total_timesteps, callback=sacr_callback ) ``` -------------------------------- ### Configure Energy Model Components Source: https://context7.com/citylearn-project/citylearn/llms.txt Instantiate energy model classes like HeatPump and Battery to simulate building devices. These classes allow for parameter configuration and performance calculation. ```python from citylearn.energy_model import HeatPump, Battery, PV, StorageTank import numpy as np # Heat Pump configuration and usage heat_pump = HeatPump( nominal_power=5.0, # kW efficiency=0.25, # Carnot efficiency target_heating_temperature=45.0, # Celsius target_cooling_temperature=8.0 # Celsius ) # Calculate COP at different outdoor temperatures outdoor_temps = np.array([0, 10, 20, 30, 35]) heating_cop = heat_pump.get_cop(outdoor_temps, heating=True) cooling_cop = heat_pump.get_cop(outdoor_temps, heating=False) print("Heat Pump COP values:") for t, h, c in zip(outdoor_temps, heating_cop, cooling_cop): print(f" Outdoor {t}C: Heating COP={h:.2f}, Cooling COP={c:.2f}") # Battery configuration battery = Battery( capacity=10.0, # kWh nominal_power=5.0, # kW charge/discharge rate efficiency=0.95, initial_soc=0.5, # 50% initial state of charge depth_of_discharge=0.9 # Can discharge to 10% SoC ) ``` -------------------------------- ### Run CityLearn Neighborhood Simulation Source: https://github.com/citylearn-project/citylearn/blob/master/examples/quickstart.ipynb Initializes the environment and agent, executes the simulation loop, and evaluates performance metrics. ```python env = CityLearnEnv(neighborhood_build.schema_filepath, central_agent=True) model = Agent(env) observations, _ = env.reset() while not env.terminated: actions = model.predict(observations) observations, reward, info, terminated, truncated = env.step(actions) kpis = model.env.evaluate() kpis = kpis.pivot(index='cost_function', columns='name', values='value').round(3) kpis = kpis.dropna(how='all') display(kpis) ``` -------------------------------- ### Configure Buffered End-of-Run Export Source: https://github.com/citylearn-project/citylearn/blob/master/docs/source/ui.rst Initialize the environment with render_mode='end' to buffer simulation steps in memory and flush them to disk upon episode completion. ```python from pathlib import Path from citylearn.citylearn import CityLearnEnv schema = 'data/datasets/citylearn_challenge_2022_phase_all_plus_evs/schema.json' env = CityLearnEnv( schema, central_agent=True, episode_time_steps=48, ``` -------------------------------- ### Create Custom Tabular Q-Learning Agent Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Extend the base TabularQLearning class to include progress tracking and reward history. ```python class CustomTabularQLearning(TabularQLearning): def __init__( self, env: CityLearnEnv, loader: IntProgress, random_seed: int = None, **kwargs ): r"""Initialize CustomRBC. Parameters ---------- env: Mapping[str, CityLearnEnv] CityLearn environment instance. loader: IntProgress Progress bar. random_seed: int Random number generator reprocucibility seed for eqsilon-greedy action selection. kwargs: dict Parent class hyperparameters """ super().__init__(env=env, random_seed=random_seed, **kwargs) self.loader = loader self.reward_history = [] def next_time_step(self): if self.env.time_step == 0: self.reward_history.append(0) else: self.reward_history[-1] += sum(self.env.rewards[-1]) self.loader.value += 1 super().next_time_step() ``` -------------------------------- ### Define Observation and Action Bin Sizes Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Configure discretization bins for observations and actions across all buildings in the environment. ```python # define active observations and actions and their bin sizes observation_bins = {'hour': 24} action_bins = {'electrical_storage': 12} # initialize list of bin sizes where each building # has a dictionary in the list definining its bin sizes observation_bin_sizes = [] action_bin_sizes = [] for b in tql_env.buildings: # add a bin size definition for the buildings observation_bin_sizes.append(observation_bins) action_bin_sizes.append(action_bins) ``` -------------------------------- ### Initialize Environment with Named Dataset Source: https://github.com/citylearn-project/citylearn/blob/master/examples/load_environment.ipynb Initialize the CityLearn environment using a named dataset. This is a convenient way to load pre-defined environments for simulations. ```python from citylearn.citylearn import CityLearnEnv env = CityLearnEnv('citylearn_challenge_2020_climate_zone_1') ``` -------------------------------- ### Evaluate and Export KPIs Source: https://github.com/citylearn-project/citylearn/blob/master/docs/source/kpis.rst Demonstrates the transition from legacy evaluation methods to the V2 API required for the UI, and how to export the results. ```python from citylearn.citylearn import CityLearnEnv env = CityLearnEnv(schema, central_agent=True, render_mode='none') legacy_kpis = env.evaluate() # Legacy naming, kept for compatibility v2_kpis = env.evaluate_v2() # V2 naming used by the UI/export env.export_final_kpis(filepath='exported_kpis.csv') # Defaults to V2 ``` -------------------------------- ### HourRBC with Custom Action Map Simulation Source: https://context7.com/citylearn-project/citylearn/llms.txt Demonstrates the simulation using HourRBC with a custom action map. This allows for fine-grained control strategies for storage and HVAC devices based on specific hours. ```python from citylearn.agents.rbc import HourRBC from citylearn.citylearn import CityLearnEnv env = CityLearnEnv('citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True) # Define custom hour-based action map # Values for storage: positive = charge, negative = discharge (fraction of capacity) # Values for devices: power output as fraction of nominal power custom_action_map = { 'electrical_storage': { 1: 0.05, 2: 0.05, 3: 0.05, 4: 0.05, 5: 0.05, 6: 0.05, # Early morning: charge 7: -0.03, 8: -0.05, 9: -0.08, 10: -0.08, 11: -0.08, 12: -0.08, # Morning peak: discharge 13: 0.02, 14: 0.02, 15: 0.02, 16: -0.05, 17: -0.08, 18: -0.10, # Afternoon/evening peak 19: -0.08, 20: -0.05, 21: -0.03, 22: 0.03, 23: 0.04, 24: 0.05 # Night: charge }, 'cooling_device': {hour: 0.7 if 9 <= hour <= 18 else 0.3 for hour in range(1, 25)}, 'heating_device': {hour: 0.3 if 9 <= hour <= 18 else 0.7 for hour in range(1, 25)}, } agent = HourRBC(env, action_map=custom_action_map) observations, _ = env.reset() while not env.terminated: actions = agent.predict(observations) observations, _, _, _, _ = env.step(actions) print("Custom HourRBC simulation complete") kpis = env.evaluate() print(kpis.pivot(index='cost_function', columns='name', values='value').round(3)) ``` -------------------------------- ### Configure Per-Step Data Export Source: https://github.com/citylearn-project/citylearn/blob/master/docs/source/ui.rst Use render_mode='during' to write simulation data to disk at every step. This enables full time-series exploration in the UI. ```python from pathlib import Path from citylearn.citylearn import CityLearnEnv schema = 'data/datasets/citylearn_challenge_2022_phase_all_plus_evs/schema.json' env = CityLearnEnv( schema, central_agent=True, episode_time_steps=48, render_mode='during', render_directory=Path('outputs/ui_exports'), # optional custom base folder render_session_name='my_first_run', # optional custom subfolder ) observations, _ = env.reset() while not env.terminated: actions = [env.action_space[0].sample()] observations, reward, terminated, truncated, info = env.step(actions) # CSV rows are appended at each step when render_mode='during'. ``` -------------------------------- ### Access CityLearn Environment Properties Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Prints key properties of the initialized CityLearn environment, including the current time step, total time steps, central agent usage, and the number of buildings. ```python print('Current time step:', env.time_step) print('environment number of time steps:', env.time_steps) print('environment uses central agent:', env.central_agent) print('Number of buildings:', len(env.buildings)) ``` -------------------------------- ### Preview Building Data File Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Reads and displays the head of a building's energy simulation data. Change 'Building_1' to preview other buildings (1-17). Requires pandas and os modules. ```python root_directory = schema['root_directory'] # change the suffix number in the next code line to a # number between 1 and 17 to preview other buildings building_name = 'Building_1' filename = schema['buildings'][building_name]['energy_simulation'] filepath = os.path.join(root_directory, filename) building_data = pd.read_csv(filepath) display(building_data.head()) ``` -------------------------------- ### Evaluate CityLearn Environment Performance Source: https://context7.com/citylearn-project/citylearn/llms.txt Shows how to evaluate the environment's performance using various cost functions. It runs a simulation with a baseline agent and then accesses detailed metrics. ```python from citylearn.citylearn import CityLearnEnv from citylearn.agents.base import BaselineAgent env = CityLearnEnv('citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True) agent = BaselineAgent(env) # Run simulation observations, _ = env.reset() while not env.terminated: actions = agent.predict(observations) observations, _, _, _, _ = env.step(actions) # Evaluate performance kpis = env.evaluate() # Pivot for cleaner display kpis_pivot = kpis.pivot(index='cost_function', columns='name', values='value').round(3) print(kpis_pivot) # Access specific metrics district_metrics = kpis[kpis['name'] == 'District'] carbon_emissions = district_metrics[district_metrics['cost_function'] == 'carbon_emissions_total']['value'].values[0] print(f"District carbon emissions (normalized): {carbon_emissions}") ``` -------------------------------- ### Simulate Environment with Decentralized SAC Agents Source: https://github.com/citylearn-project/citylearn/blob/master/examples/quickstart.ipynb Initializes and trains a decentralized-independent SAC agent for a specified number of episodes. Ensure the CityLearn environment is correctly configured. ```python from citylearn.agents.sac import SAC as Agent from citylearn.citylearn import CityLearnEnv # initialize env = CityLearnEnv('citylearn_challenge_2023_phase_2_local_evaluation', central_agent=False) model = Agent(env) # train model.learn(episodes=2, deterministic_finish=True) ``` -------------------------------- ### Custom Reward Functions Source: https://context7.com/citylearn-project/citylearn/llms.txt Demonstrates using built-in and custom reward functions with CityLearn. Includes SolarPenaltyReward and a combined SolarPenaltyAndComfortReward with configurable parameters. ```python from citylearn.citylearn import CityLearnEnv from citylearn.reward_function import ( RewardFunction, MARL, SolarPenaltyReward, ComfortReward, SolarPenaltyAndComfortReward ) # Use built-in reward functions env = CityLearnEnv( 'citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True, reward_function=SolarPenaltyReward # Encourages solar self-consumption ) # Combined reward function with configurable parameters env_comfort = CityLearnEnv( 'citylearn_challenge_2023_phase_2_local_evaluation', central_agent=True, reward_function=SolarPenaltyAndComfortReward, reward_function_kwargs={ 'band': 2.0, # Comfort band in degrees Celsius 'lower_exponent': 2.0, # Penalty exponent for under-cooling/heating 'higher_exponent': 3.0, # Higher penalty for over-cooling/heating 'coefficients': (1.0, 0.5) # Weights for solar and comfort rewards } ) ``` -------------------------------- ### Inspect Reward Function Source: https://github.com/citylearn-project/citylearn/blob/master/examples/tutorial.ipynb Displays help documentation for the current reward function object. ```python help(sac_env.reward_function) ```