Sinergym

https://ugr-sail.github.io/sinergym/
Admin
Sinergym is a Gymnasium-based building energy simulation environment for developing and testing...
Tokens:43,681
Snippets:494
Trust Score:9.7
Update:5 months ago
Show doc for...
Tokens:
Raw
### Basic Sinergym Environment Usage

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/usage-example.html

This script demonstrates the fundamental steps to initialize a Sinergym environment, apply common wrappers, run a single episode with random actions, and log the episode's completion. It's a starting point for testing Sinergym installation and basic functionality.

```python
import logging

import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.logger import TerminalLogger
from sinergym.utils.wrappers import (
    CSVLogger,
    DatetimeWrapper,
    LoggerWrapper,
    NormalizeAction,
    NormalizeObservation,
)

# Logger
terminal_logger = TerminalLogger()
logger = terminal_logger.getLogger(name='MAIN', level=logging.INFO)

# Create environment and apply wrappers for normalization and logging
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
env = DatetimeWrapper(env)
env = NormalizeAction(env)
env = NormalizeObservation(env)
env = LoggerWrapper(env)
env = CSVLogger(env)

# Execute 1 episode
episodes = 1
for i in range(episodes):

    # Reset the environment to start a new episode
    obs, info = env.reset()
    truncated = terminated = False

    while not (terminated or truncated):

        # Random action selection
        a = env.action_space.sample()

        # Perform action and receive env information
        obs, reward, terminated, truncated, info = env.step(a)

    logger.info(f'Episode {env.get_wrapper_attr("episode")} finished.')

logger.info(
    'You can get a dictionary of the observation if you want to check something, independently of the changes made to the observation by the wrappers:'
)
logger.info(env.get_obs_dict(obs))

env.close()

```

--------------------------------

### Install Sinergym from Cloned Repository using Poetry

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Installs Sinergym from a local clone of the repository using Poetry, with support for optional extras. Use '--no-interaction' to prevent prompts.

```bash
$ poetry install --no-interaction --extras <optional_extras>

```

--------------------------------

### Import and Make a Sinergym Environment

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/change_environment.html

Import the necessary libraries and create a Sinergym environment using its ID. This is the basic step to start any simulation.

```python
import gymnasium as gym
import numpy as np

import sinergym

env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
```

--------------------------------

### Define the environment

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/rule_controller_example.html

Instantiate the Sinergym environment using gym.make. This example uses a 5-zone hot continuous environment.

```python
env = gym.make('Eplus-5zone-hot-continuous-v1')
```

--------------------------------

### Install Development Packages with Poetry

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Use this command to install all development packages when working with Sinergym. Ensure you have Poetry installed.

```bash
$ poetry install --no-interaction --with dev

```

--------------------------------

### Discrete Incremental Wrapper Example

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

This example shows the output of a discrete incremental wrapper during environment closure and simulation progress. It indicates the environment is closed and provides simulation progress details.

```text
[ENVIRONMENT] (INFO) : Environment closed. [Eplus-5zone-hot-continuous-v1]
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:00<00:00, 8961.81%/s, 100% completed]
```

--------------------------------

### Install Sinergym using pip

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Installs the Sinergym package and its core dependencies using pip.

```bash
$ pip install sinergym

```

--------------------------------

### Install Sinergym from Cloned Repository using pip

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Installs Sinergym from a local clone of the repository using pip, with support for optional extras.

```bash
$ pip install .[<optional_extras>]

```

--------------------------------

### Sinergym Environment and Model Initialization Log

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/sinergym_package.html

This log output shows the initialization process of a Sinergym environment and its associated model. It details the creation of the Gymnasium environment, the working directory, model configuration, and the setup of the run period and timestep.

```log
#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: demo-v1
#==============================================================================================#
[MODEL] (INFO) : Working directory created: /workspaces/sinergym/examples/demo-v1-res1
[MODEL] (INFO) : Model Config is correct.
[MODEL] (INFO) : Building model Output:Variable updated with defined variable names.
[MODEL] (INFO) : Updated building model Output:Meter with meter names.
[MODEL] (INFO) : Building configuration: runperiod updated to {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 1, 'end_month': 3, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 1}
[MODEL] (INFO) : Updated episode length (seconds): 5184000.0
[MODEL] (INFO) : Updated timestep size (seconds): 3600.0
[MODEL] (INFO) : Updated timesteps per episode: 1440
[MODEL] (INFO) : Runperiod established.
[MODEL] (INFO) : Episode length (seconds): 5184000.0
[MODEL] (INFO) : timestep size (seconds): 3600.0
[MODEL] (INFO) : timesteps per episode: 1440
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: demo-v1
#----------------------------------------------------------------------------------------------#
[MODEL] (INFO) : Episode directory created.
[MODEL] (INFO) : Weather file USA_PA_Pittsburgh-Allegheny.County.AP.725205_TMY3.epw used.
[MODEL] (INFO) : Adapting weather to building model.
[ENVIRONMENT] (INFO) : Saving episode output path in /workspaces/sinergym/examples/demo-v1-res1/episode-1/output.
[ENVIRONMENT] (DEBUG) : Path: /workspaces/sinergym/examples/demo-v1-res1/episode-1/output
[ENVIRONMENT] (DEBUG) : Waiting for finishing WARMUP process.
[ENVIRONMENT] (DEBUG) : WARMUP process finished.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[ENVIRONMENT] (INFO) : Episode 1 started.
[ENVIRONMENT] (DEBUG) : RESET observation received: {'month': 1, 'day_of_month': 1, 'hour': 0, 'outdoor_temperature': 7.2, 'htg_setpoint': 12.8, 'clg_setpoint': 40.0, 'air_temperature': 19.7479730060398, 'air_humidity': 54.12386596530383, 'HVAC_electricity_demand_rate': 87.36488027634958}
[ENVIRONMENT] (DEBUG) : RESET info received: {'time_elapsed(hours)': 1.5, 'month': 1, 'day': 1, 'hour': 0, 'is_raining': False, 'timestep': 0}
[ENVIRONMENT] (DEBUG) : STEP queues not receive value, simulation must be completed. changing TRUNCATED flag to TRUE
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:03<00:00, 30.73%/s, 100% completed]
[ENVIRONMENT] (INFO) : Environment closed. [demo-v1]
```

--------------------------------

### Install Sinergym with Optional Packages using pip

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Installs Sinergym along with specified optional packages using pip.

```bash
$ pip install sinergym[extras]

```

--------------------------------

### Configure WandBLogger

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

Example configuration for the WandBLogger, which integrates Sinergym with Weights & Biases for experiment tracking. Note that this example is commented out and requires specific parameters for entity, project name, and run name.

```python
# env = WandBLogger(env = Env,
#                 entity = <wandb_account_entity>,
#                 project_name = <wandb_project_name>,
#                 run_name = <run_name>
#                 group = 'Notebook_example',
#                 tags: ['tag1','tag2'],
#                 save_code = False,
#                 dump_frequency = 1000,
#                 artifact_save = True,
#                 artifact_type = 'output',
#                 excluded_info_keys = ['reward',
#                                   'action',
#                                   'timestep',
#                                   'month',
#                                   'day',
#                                   'hour',
#                                   'time_elapsed(hours)',
#                                   'reward_weight',
#                                   'is_raining'],
#                 excluded_episode_summary_keys = ['terminated',
#                                              'truncated']):

# env.reset()
```

--------------------------------

### Run All Sinergym Tests

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/tests.html

Execute all tests in the `tests/` directory to ensure the project is functioning correctly. This is the primary command for verifying installation and integrity.

```bash
pytest tests/ -vv
```

--------------------------------

### Install Development Packages with Pip

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Install Sinergym with extra development dependencies using pip. Note that not all development packages may be installed with this method.

```bash
$ pip install -e .[extras]

```

--------------------------------

### Install Pytest

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/tests.html

Install the pytest package if it's not already included in the project requirements. This command ensures you have the necessary testing framework.

```bash
pip install pytest
```

--------------------------------

### Multi-observation Wrapper Example

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

Demonstrates the usage of the Multi-observation wrapper by comparing observations before and after applying the wrapper. This wrapper stacks observations into a history queue.

```python
# Original environment
env = gym.make('Eplus-5zone-hot-discrete-v1')
obs, info = env.reset()
print('BEFORE MULTI OBSERVATION: ', obs)
```

--------------------------------

### EnergyPlus Simulation Output Log

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/default_building_control.html

This log output shows the initialization process of a Sinergym environment and the subsequent EnergyPlus simulation. It details the creation of the environment, model configuration, weather file adaptation, and the start of the simulation episode, including initial reward and progress updates.

```log
#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: Eplus-office-hot-continuous-v1
#==============================================================================================#
[MODEL] (INFO) : Working directory created: /workspaces/sinergym/examples/Eplus-office-hot-continuous-v1-res1
[MODEL] (INFO) : Model Config is correct.
[MODEL] (INFO) : Building model Output:Variable updated with defined variable names.
[MODEL] (INFO) : Updated building model Output:Meter with meter names.
[MODEL] (INFO) : Runperiod established.
[MODEL] (INFO) : Episode length (seconds): 31536000.0
[MODEL] (INFO) : timestep size (seconds): 900.0
[MODEL] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: Eplus-office-hot-continuous-v1
#----------------------------------------------------------------------------------------------#
[MODEL] (INFO) : Episode directory created.
[MODEL] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODEL] (INFO) : Adapting weather to building model.
[ENVIRONMENT] (INFO) : Saving episode output path in /workspaces/sinergym/examples/Eplus-office-hot-continuous-v1-res1/episode-1/output.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[ENVIRONMENT] (INFO) : Episode 1 started.
Reward:  -3.3966374173223297 {'time_elapsed(hours)': 0.5, 'month': 1, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [], 'timestep': 1, 'reward': -3.3966374173223297, 'energy_term': -0.008430450234156376, 'comfort_term': -3.3882069670881734, 'energy_penalty': -168.6090046831275, 'comfort_penalty': -6.776413934176347, 'total_power_demand': 168.6090046831275, 'total_temperature_violation': 6.776413934176347, 'reward_weight': 0.5}
Simulation Progress [Episode 1]:  10%|█         | 10/100 [00:03<00:41,  2.19%/s, 10% completed] Reward:  -28181.260229794305 {'time_elapsed(hours)': 744.25, 'month': 2, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [], 'timestep': 2976, 'reward': -11.282435670662087, 'energy_term': -0.010066938284793056, 'comfort_term': -11.272368732377295, 'energy_penalty': -201.33876569586113, 'comfort_penalty': -22.54473746475459, 'total_power_demand': 201.33876569586113, 'total_temperature_violation': 22.54473746475459, 'reward_weight': 0.5}

```

--------------------------------

### Apply Wrappers for Logging and Normalization

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/logging_unused_variables.html

Apply normalization and logging wrappers to the environment. This setup is necessary before applying ReduceObservationWrapper to log unused variables.

```python
import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.wrappers import (
    CSVLogger,
    LoggerWrapper,
    NormalizeAction,
    NormalizeObservation,
    ReduceObservationWrapper,
    WandBLogger,
)

# Creating environment and applying wrappers for normalization and logging
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
env = NormalizeAction(env)
env = NormalizeObservation(env)
env = LoggerWrapper(env)
print('###########################################################################')
print('Old observation space shape: ', env.observation_space.shape[0])
print('Old observation variables: ',
      env.get_wrapper_attr('observation_variables'))
print('###########################################################################')

# Here we can add the CSVLogger or WandBLogger
env = CSVLogger(env)
# env = WandBLogger(env,
#                   entity='alejandro-campoy',
#                   project_name='test-project',
```

--------------------------------

### Get Sinergym Version and Environment IDs

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/sinergym_package.html

Import the sinergym package to print its version and a list of available environment IDs.

```python
import sinergym

print(sinergym.__version__)
print(sinergym.ids())

```

--------------------------------

### Sinergym Environment Logging

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Illustrates log messages from the Sinergym environment during simulation. This includes starting new episodes, creating directories, and using weather files.

```text
[ENVIRONMENT] (INFO) : Starting a new episode... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40] [Episode 8]
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run8]
[MODELING] (INFO) : Weather file USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model. [USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run8/output]
[ENVIRONMENT] (INFO) : Episode 8 started.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
```

--------------------------------

### Example YAML Configuration for Environments

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/environments_registration.html

This YAML structure defines environment parameters including building file, weather specifications, and variable definitions. It serves as a template for custom environment creation.

```yaml
# Documentation: https://ugr-sail.github.io/sinergym/compilation/main/pages/environments_registration.html
# ---------------------------------------------------------------------------- #
#                        ID BASE FROM ENVIRONMNET NAMES                        #
# ---------------------------------------------------------------------------- #
id_base: 5zone

# ---------------------------------------------------------------------------- #
#                         BUILDING AND WEATHER SETTINGS                        #
# ---------------------------------------------------------------------------- #
building_file: 5ZoneAutoDXVAV.epJSON

# A environment alternative for each weather key
weather_specification:
  weather_files:
    - USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw
    - USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw
    - USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw
  keys:
    - hot
    - mixed
    - cool

# ---------------------------------------------------------------------------- #
#                         BUILDING EXTRA CONFIGURATION                         #
# ---------------------------------------------------------------------------- #
building_config: null


# ---------------------------------------------------------------------------- #
#                        WEATHER VARIABILITY (OPTIONAL)                        #

```

--------------------------------

### Initialize and Observe Environment Before Energy Cost Wrapper

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

Initializes a standard Sinergym environment and prints its observation variables and initial observation values before applying any wrappers. This serves as a baseline for comparison.

```python
# Original environment
env = gym.make('Eplus-5zone-hot-discrete-v1')
obs, info = env.reset()
print('OBSERVATION VARIABLES BEFORE ADDING ENERGY COST: 
',
      env.get_wrapper_attr('observation_variables'))
print('OBSERVATION VALUES BEFORE ADDING ENERGY COST: 
', obs)
```

--------------------------------

### Create a GCE VM with Sinergym Container

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/gcloudAPI.html

Creates a Google Compute Engine virtual machine instance pre-configured with the Sinergym container. Ensure the specified zone is available and you have sufficient quota. The `--boot-disk-size` is critical as the default 10GB is insufficient.

```bash
$ gcloud compute instances create-with-container sinergym \
    --container-image eu.gcr.io/sinergym/sinergym \
    --zone europe-west1-b \
    --container-privileged \
    --container-restart-policy never \
    --container-stdin \
    --container-tty \
    --boot-disk-size 20GB \
    --boot-disk-type pd-ssd \
    --machine-type n2-highcpu-8

```

--------------------------------

### Create Gymnasium environments

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Instantiates the training and evaluation environments using `gym.make`. The environment name includes the experiment configuration for tracking. Note the log output indicating successful creation and configuration.

```python
env = gym.make(environment, env_name=experiment_name)
eval_env = gym.make(environment, env_name=experiment_name+'_EVALUATION')
```

--------------------------------

### Install Development Packages in Dockerfile

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

This command is used within a Dockerfile to ensure all development modules are available in the container. It installs packages using Poetry.

```dockerfile
RUN poetry install --no-interaction --with dev

```

--------------------------------

### Serialize and Restore Environment with Wrappers

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/serialization.html

Demonstrates the process of creating an environment, applying wrappers, serializing wrapper information, and then restoring the environment with its wrappers. Wrapper information is saved separately and reapplied.

```python
import gymnasium as gym
from sinergym.utils.common import get_wrappers_info, apply_wrappers_info
from sinergym.utils.wrappers import NormalizeObservation, NormalizeAction
from sinergym.envs.eplus_env import EplusEnv
import yaml

# Environment creation and wrapping
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
env = NormalizeObservation(env)
env = NormalizeAction(env)

# Serialize wrapper information
# By default, this will be saved in the environment's output directory
wrappers_info = get_wrappers_info(env)
```

```python
# Load environment parameters
with open('<env_output_folder>/env_config.pyyaml', 'r') as f:
    env_params = yaml.load(f, Loader=yaml.FullLoader)

# Modify environment parameters if needed...

# Restore environment
env = EplusEnv.from_dict(env_params)

# Restore wrappers (from file or dictionary)
env = apply_wrappers_info(env, '<env_output_folder>/wrappers_config.pyyaml')
# Alternatively:
env = apply_wrappers_info(env, wrappers_info)
```

--------------------------------

### Initialize and Run Rule-Based Controller

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/rule_controller_example.html

This snippet shows how to initialize a rule-based controller with an environment, run a simulation episode, and log rewards and information per month.

```python
agent = MyRuleBasedController(env)

for i in range(1):
    obs, info = env.reset()
    rewards = []
    truncated = terminated = False
    current_month = 0
while not (terminated or truncated):
    action = agent.act(obs)
    obs, reward, terminated, truncated, info = env.step(action)
    rewards.append(reward)
    if info['month'] != current_month:  # display results every month
        current_month = info['month']
        print('Reward: ', sum(rewards), info)
print(
    'Episode ',
    i,
    'Mean reward: ',
    np.mean(rewards),
    'Cumulative reward: ',
    sum(rewards))
```

--------------------------------

### Initialize Environment and Observe Before Weather Forecasting

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

Initializes a discrete 5-zone hot environment and prints the observation variables and observation before applying any weather forecasting wrapper. This serves as a baseline for comparison.

```python
# Original environment
env = gym.make('Eplus-5zone-hot-discrete-v1')
obs, info = env.reset()
print('OBSERVATION VARIABLES BEFORE WEATHER FORECASTING: ', \
      env.get_wrapper_attr('observation_variables'))
print('OBSERVATION BEFORE WEATHER FORECASTING: ', obs)
```

--------------------------------

### Import Sinergym and Create Environment

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/basic_example.html

Imports necessary libraries and creates an instance of the 'Eplus-demo-v1' environment. Importing Sinergym implicitly defines available environments.

```python
import gymnasium as gym
import numpy as np

import sinergym

env = gym.make('Eplus-demo-v1')
```

--------------------------------

### Create a VM Instance Template with a Container

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/gcloudAPI.html

This command creates a reusable instance template for creating VMs with a container. It specifies the container image, service account, scopes, environment variables, and machine type. Ensure proper configuration of service accounts and scopes beforehand.

```bash
gcloud compute instance-templates create-with-container sinergym-template \
--container-image eu.gcr.io/sinergym/sinergym \
--container-privileged \
--service-account storage-account@sinergym.iam.gserviceaccount.com \
--scopes https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/devstorage.full_control \
--container-env=gce_zone=europe-west1-b,gce_project_id=sinergym \
--container-restart-policy never \
--container-stdin \
--container-tty \
--boot-disk-size 20GB \
--boot-disk-type pd-ssd \
--machine-type n2-highcpu-8
```

--------------------------------

### Initialize and Wrap Environment with Logging

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/logging_unused_variables.html

Initializes a Gymnasium environment and applies several wrappers, including CSVLogger and ReduceObservationWrapper. The order of wrappers is crucial for determining which variables are logged and how the observation space is transformed.

```python
env = sinergym.utils.wrappers.NormalizeAction(env)
env = sinergym.utils.wrappers.NormalizeObservation(env)
env = sinergym.utils.wrappers.LoggerWrapper(env)
env = sinergym.utils.wrappers.ReduceObservationWrapper(env,
                                                      obs_reduction=[
                                                          'outdoor_temperature', 'outdoor_humidity', 'air_temperature'])
print('###########################################################################')
print('Wrapped observation space shape: ', env.observation_space.shape[0])
print('Wrapped observation variables: ',
      env.get_wrapper_attr('observation_variables'))
print('###########################################################################')
```

--------------------------------

### Run Sinergym DRL Experiment with Docker

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Builds a Docker image with 'drl' extras and runs a sample PPO training agent. Ensure WANDB_API_KEY is set if WandB is used.

```bash
$ docker build -t example/sinergym:latest --build-arg SINERGYM_EXTRAS="drl" .
$ docker run -e WANDB_API_KEY=$WANDB_API_KEY -it --rm example/sinergym:latest python scripts/train/train_agent.py -conf scripts/train/train_agent_PPO.yaml

```

--------------------------------

### Initialize Environment with CustomLoggerWrapper and CSVLogger

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/personalize_loggerwrapper.html

This snippet shows how to instantiate a Sinergym environment, wrap it with CustomLoggerWrapper, and then further wrap it with CSVLogger to enable saving data to CSV files.

```python
env = gym.make('Eplus-demo-v1')
env = CustomLoggerWrapper(env)
env = CSVLogger(env)
```

--------------------------------

### Configure Docker Authentication for Google Cloud

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/gcloudAPI.html

Authenticates Docker with Google Cloud's container registry. Ensure you have the Google Cloud SDK installed and configured.

```bash
$ gcloud auth configure-docker

```

--------------------------------

### Sinergym Environment with Empty Action Space

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/default_building_control.html

This Python script demonstrates setting up a Sinergym environment with an empty action space. This approach allows Sinergym to manage the simulation and automatically configure necessary parameters, avoiding manual adjustments required when running EnergyPlus directly.

```python
import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.wrappers import LoggerWrapper

env = gym.make(
    'Eplus-office-hot-continuous-v1',
    actuators={},
    action_space=gym.spaces.Box(
        low=0,
        high=0,
        shape=(0,)))
env = LoggerWrapper(env)

for i in range(1):
    obs, info = env.reset()
    rewards = []
    truncated = terminated = False
    current_month = 0
    while not (terminated or truncated):
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        rewards.append(reward)
        if info['month'] != current_month:  # display results every month
            current_month = info['month']
            print('Reward: ', sum(rewards), info)
    print(
        'Episode ',
        i,
        'Mean reward: ',
        np.mean(rewards),
        'Cumulative reward: ',
        sum(rewards))
env.close()

```

--------------------------------

### EnergyPlus Simulation Log Output

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

This log output shows the initialization and execution of an EnergyPlus simulation within Sinergym, including environment setup and progress updates.

```log
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run11/USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3_Random_1.0_0.0_0.001.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run11/output', '/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run11/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 11 started.

```

```log
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
1 -1.9855085492179452
2 -2525.9765795852795-----------------------------------------------------------------------------------------| 9%
3 -4836.447992908011*******-----------------------------------------------------------------------------------| 16%
4 -7415.106488666047****************--------------------------------------------------------------------------| 25%
5 -10199.115705433655***********************------------------------------------------------------------------| 33%
6 -13226.748838894438*******************************----------------------------------------------------------| 41%
7 -14570.022002702428****************************************-------------------------------------------------| 50%
8 -16163.73052129073************************************************------------------------------------------| 58%
9 -17710.726350820067********************************************************---------------------------------| 66%
10 -19090.64622045218*****************************************************************------------------------| 75%
11 -22118.184102555933************************************************************************----------------| 83%
12 -24523.10955600609*********************************************************************************--------| 91%
Progress: |***************************************************************************************************| 99%
Episode  4 Mean reward:  -0.770449260651564 Cumulative reward:  -26996.542093230804
[WRAPPER CSVLogger] (INFO) : Environment closed, data updated in monitor and progress.csv.
[WRAPPER CSVLogger] (INFO) : Environment closed, data updated in monitor and progress.csv.
[ENVIRONMENT] (INFO) : Environment closed. [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]

```

--------------------------------

### Initialize MultiObjectiveReward Wrapper

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

Create a Sinergym environment and wrap it with MultiObjectiveReward to handle multiple reward terms. Ensure the specified reward terms are available in the environment's info dictionary.

```python
env = gym.make('Eplus-5zone-hot-discrete-v1')
env = MultiObjectiveReward(env, reward_terms=['energy_term', 'comfort_term'])
```

--------------------------------

### List and Enable Google Cloud Services

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/gcloudAPI.html

Lists available Google Cloud services and enables specific services required for Sinergym. This command requires appropriate permissions.

```bash
$ gcloud services list
$ gcloud services enable artifactregistry.googleapis.com \
                         cloudapis.googleapis.com \
                         cloudbuild.googleapis.com \
                         containerregistry.googleapis.com \
                         gmail.googleapis.com \
                         sql-component.googleapis.com \
                         sqladmin.googleapis.com \
                         storage-component.googleapis.com \
                         storage.googleapis.com \
                         cloudfunctions.googleapis.com \
                         pubsub.googleapis.com \
                         run.googleapis.com \
                         serviceusage.googleapis.com \
                         drive.googleapis.com \
                         appengine.googleapis.com

```

--------------------------------

### Configure WandB Logger in SB3

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Sets up a custom logger for Stable-Baselines3 that includes both human-readable console output and Weights & Biases (WandB) logging. Ensure WandB is installed and configured.

```python
if is_wrapped(env, WandBLogger):
    logger = SB3Logger(
        folder=None,
        output_formats=[
            HumanOutputFormat(
                sys.stdout,
                max_length=120),
            WandBOutputFormat()]
    )
    model.set_logger(logger)
```

--------------------------------

### Import necessary libraries for DRL

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Imports essential libraries for deep reinforcement learning, including Gymnasium, Stable Baselines 3, and Sinergym utilities. Ensure these are installed before running.

```python
import sys
from datetime import datetime

import gymnasium as gym
import numpy as np
import wandb
from stable_baselines3.common.callbacks import CallbackList
from stable_baselines3.common.logger import HumanOutputFormat
from stable_baselines3.common.logger import Logger as SB3Logger

import sinergym
from sinergym.utils.callbacks import *
from sinergym.utils.constants import *
from sinergym.utils.logger import WandBOutputFormat
from sinergym.utils.rewards import *
from sinergym.utils.wrappers import *
```

--------------------------------

### Sinergym Episode Logging

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Logs indicating the start of new episodes and the creation of episode directories. It also shows the saving of normalization calibration data and the update of CSV logs.

```bash
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER CSVLogger] (INFO) : End of episode detected, data updated in monitor and progress.csv.
[WRAPPER CSVLogger] (INFO) : End of episode detected, data updated in monitor and progress.csv.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40] [Episode 11]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run11]
[MODELING] (INFO) : Weather file USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model. [USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run11/output]
```

--------------------------------

### Open Bash Session in Sinergym Docker Container

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Starts an interactive bash session within the Sinergym Docker container, allowing for manual command execution and file management.

```bash
$ docker run -it <tag_name> /bin/bash

```

--------------------------------

### Define Discrete Action Space

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/environments_registration.html

Configure a discrete action space using gym.spaces.Discrete, specifying the number of possible discrete actions. This example defines an action space with 10 discrete actions.

```python
action_space_discrete: gym.spaces.Discrete(10)
```

--------------------------------

### Define environment and training configurations

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Sets up key training parameters such as the environment ID, number of episodes, and experiment name. The experiment name is dynamically generated with a timestamp for uniqueness.

```python
# Environment ID
environment = 'Eplus-5zone-mixed-continuous-stochastic-v1'

# Training episodes
episodes = 5

# Name of the experiment
experiment_date = datetime.today().strftime('%Y-%m-%d_%H:%M')
experiment_name = 'SB3_PPO-' + environment + \
    '-episodes-' + str(episodes)
experiment_name += '_' + experiment_date
```

--------------------------------

### Define Continuous Action Space

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/environments_registration.html

Configure a continuous action space using gym.spaces.Box, defining the low and high bounds and shape for the actions. This example defines an action space for two continuous variables.

```python
action_space: gym.spaces.Box(low=np.array([12.0, 23.25], dtype=np.float32), high=np.array([23.25, 30.0], dtype=np.float32), shape=(2,), dtype=np.float32)
```

--------------------------------

### List Available Environment IDs

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/environments.html

Use this script to print the Sinergym version and a list of all available environment IDs. It also shows how to make and inspect a specific environment.

```python
# This script is available in scripts/consult_environments.py
import sinergym
import gymnasium as gym

print(sinergym.__version__)
print(sinergym.ids())

# Make and consult environment
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
print(env.get_wrapper_attr('to_str')())
```

--------------------------------

### Get and Print Schedulers Information

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/getting_env_information.html

Load the Sinergym environment and retrieve scheduler information as a Python dictionary. Use `pprint` for a formatted output of the complex dictionary structure. This is useful for inspecting building control rules.

```python
from pprint import pprint

env = gym.make('Eplus-demo-v1')
schedulers = env.get_wrapper_attr('schedulers')
pprint(schedulers)
```

--------------------------------

### Import necessary libraries

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/rule_controller_example.html

Import all required libraries for Sinergym and environment interaction. Ensure 'sinergym' is imported.

```python
from datetime import datetime
from typing import Any, List, Sequence

import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.constants import YEAR
```

--------------------------------

### Implement a Custom Constant Reward Function

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/rewards.html

Define a custom reward function that always returns a constant value. This example inherits from `BaseReward` and overrides the `__call__` method to return -1.0. It also shows how to instantiate an environment with this custom reward.

```python
from sinergym.utils.rewards import BaseReward

class CustomReward(BaseReward):
    """Naive reward function."""
    def __init__(self, env):
        super(CustomReward, self).__init__(env)
    def __call__(self, obs_dict):
        return -1.0, {}

env = gym.make('Eplus-discrete-stochastic-mixed-v1', reward=CustomReward)
```

--------------------------------

### Import Sinergym and Gymnasium

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/wrappers_examples.html

Import necessary libraries for Sinergym and Gymnasium environments, including specific wrappers.

```python
import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.wrappers import *

```

--------------------------------

### EnergyPlus Simulation Log Output

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

This snippet shows typical log messages from an EnergyPlus simulation run by Sinergym. It includes environment status, model setup, weather file usage, and simulation progress. These logs are useful for debugging and understanding the simulation process.

```log
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40] [Episode 7]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run7]
[MODELING] (INFO) : Weather file USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model. [USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run7/output]

```

```log
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run7/USA_NY_New.York-J.F.Kennedy.Intl.AP.744860_TMY3_Random_1.0_0.0_0.001.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run7/output', '/workspaces/sinergym/examples/Eplus-env-SB3_PPO-Ezone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40-res1/Eplus-env-sub_run7/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 7 started.
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [SB3_PPO-Eplus-5zone-mixed-continuous-stochastic-v1-episodes-5_2024-08-07_15:40]
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
1 -1.6446008069134792
2 -2535.9359450953466-----------------------------------------------------------------------------------------| 9%
3 -4823.4260633241665******-----------------------------------------------------------------------------------| 16%
4 -7364.053050786045****************--------------------------------------------------------------------------| 25%

```

--------------------------------

### Create a VM with a Container

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/gcloudAPI.html

This command creates a new virtual machine instance pre-configured with a specified container image. The `--container-restart-policy never` option is crucial for correct operation.

```bash
gcloud compute instances create-with-container sinergym \
    --container-image eu.gcr.io/sinergym/sinergym \
    --zone europe-west1-b \
    --container-privileged \
    --container-restart-policy never \
    --container-stdin \
    --container-tty \
    --boot-disk-size 20GB \
    --boot-disk-type pd-ssd \
    --machine-type n2-highcpu-8
```

--------------------------------

### Initialize and Wrap Evaluation Environment

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Sets up the evaluation environment by applying normalization and logging wrappers. The CSVLogger wrapper is used here for local data storage.

```python
eval_env = NormalizeObservation(eval_env)
eval_env = NormalizeAction(eval_env)
eval_env = LoggerWrapper(eval_env)
eval_env = CSVLogger(eval_env)
```

--------------------------------

### Build Sinergym Docker Image with Optional Dependencies

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/installation.html

Builds a Docker image with specified optional dependencies like 'drl', 'notebooks', or 'gcloud'.

```bash
$ docker build -t <tag_name> --build-arg SINERGYM_EXTRAS="drl notebooks gcloud" .

```

--------------------------------

### Print All Environment Information

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/getting_env_information.html

Call `env.get_wrapper_attr('to_str')()` after `reset()` to display all environment details. This includes information about the building, weather, simulation settings, and available elements.

```python
env.get_wrapper_attr('to_str')()
```

--------------------------------

### Create and Wrap Gymnasium Environment for Evaluation

Source: https://ugr-sail.github.io/sinergym/compilation/v3.11.0/pages/notebooks/drl.html

Creates the Gymnasium environment and applies necessary wrappers for evaluation. It's essential to use the same wrappers as training, especially for observation and action space normalization, ensuring calibration data is used and updates are deactivated.

```python
evaluation_env = gym.make(environment, env_name=evaluation_name)
evaluation_env = NormalizeObservation(evaluation_env, mean=env.get_wrapper_attr(
    "mean"), var=env.get_wrapper_attr("var"), count=env.get_wrapper_attr("count"), automatic_update=False)
evaluation_env = NormalizeAction(evaluation_env)
evaluation_env = LoggerWrapper(evaluation_env)
evaluation_env = CSVLogger(evaluation_env)
# If you want to log the evaluation interactions to WandB, use WandBLogger too
```
Tokens:
Raw