### Install Pong Agent Dependencies

Source: https://github.com/opencog/rocca/blob/master/examples/pong/README.md

Installs the necessary Atari subpackage for gym and sets up ROMs. Requires gym[atari] and Atari ROMs.

```bash
pip install gym[atari]
```

```bash
unrar e Roms.rar
```

```bash
python -m atari_py.import_roms <ROM_FOLDER>
```

--------------------------------

### Run Jupyter Notebook Example

Source: https://github.com/opencog/rocca/blob/master/README.md

Launches a Jupyter Notebook server to run example experiments, such as the CartPole environment. This allows for interactive exploration and visualization of ROCCA's capabilities.

```bash
jupyter notebook 01_cartpole.ipynb
```

--------------------------------

### Run Rocca Example Script

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Command to execute the main Python script for the Rocca project, which likely orchestrates the interaction with Malmo and Minecraft.

```python
python collect_diamonds.py
```

--------------------------------

### Install Development Requirements

Source: https://github.com/opencog/rocca/blob/master/README.md

Installs the necessary dependencies for developing ROCCA, including tools for testing, linting, and building. This command should be run from the root folder of the project.

```bash
pip install -r requirements-dev.txt
```

--------------------------------

### OpenCog Temporal Pattern Matching Queries

Source: https://github.com/opencog/rocca/blob/master/doc/proto-agi-early-progress-report-and-planning.md

Examples of temporal query patterns for the OpenCog pattern matcher, illustrating how to query events at specific times or time intervals. These examples highlight the need for improved matching capabilities for time-based queries.

```APIDOC
Query Example 1:
  (Get (AtTime A (Plus (TimeNode "1" (TimeNode "10")))))

Query Example 2:
  (AtTime A (TimeNode "11"))
```

--------------------------------

### Launch Chase Malmo Application

Source: https://github.com/opencog/rocca/blob/master/examples/chase_malmo/README.md

Command to launch the chase malmo application. Ensure all dependencies are installed and Minecraft is launched prior to running this command.

```bash
python chase_malmo.py
```

--------------------------------

### Install ROCCA Package

Source: https://github.com/opencog/rocca/blob/master/README.md

Installs the ROCCA package in editable mode, allowing for development and direct use of the library. This command should be run from the root folder of the project.

```bash
pip install -e .
```

--------------------------------

### MineRL Environment Setup and Interaction

Source: https://github.com/opencog/rocca/blob/master/02_minerl_hello_world.ipynb

This snippet demonstrates the initial steps for using the MineRL library with OpenAI Gym. It covers importing libraries, creating a specific MineRL environment, resetting the environment to an initial state, and accessing the environment's action and observation spaces.

```python
import gym
import minerl
```

```python
env = gym.make('MineRLNavigateDense-v0')
```

```python
obs = env.reset()
```

```python
env.action_space
```

```python
env.observation_space
```

--------------------------------

### Open CartPole Jupyter Notebook

Source: https://github.com/opencog/rocca/blob/master/examples/cartpole/README.md

Launches the Jupyter notebook environment to interact with CartPole examples. This notebook provides an interactive way to explore the agent's behavior and data.

```bash
jupyter notebook ../../01_cartpole.ipynb
```

--------------------------------

### Start TensorBoard

Source: https://github.com/opencog/rocca/blob/master/README.md

Launches TensorBoard to visualize training metrics logged by experiments, such as rewards. It monitors the 'runs' directory for event files. Access the interface via http://localhost:6006.

```bash
tensorboard --logdir runs
```

--------------------------------

### Atomspace Explorer REST API Client

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Command to start the REST API client for atomspace-explorer, which is used to visualize mined cognitive schematics. This step is optional and requires setting a flag in the script.

```python
python start_rest_service.py
```

--------------------------------

### Initialize CartPole Environment

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Initializes the CartPole-v1 environment from the gym library. This is a standard setup for reinforcement learning tasks involving the CartPole balancing problem.

```python
env = gym.make("CartPole-v1")
```

--------------------------------

### Rocca Cognitive Schematic Example 2

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

A second parsed representation of a cognitive schematic, illustrating a conditional implication with actions like 'hold' and 'go_to' and a reward function.

```opencog-dsl
(BackPredictiveImplicationScopeLink (stv 1 0.00559006)
  (VariableSet
  ) ; [80075fffffff523a][3]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][3]
  ) ; [da5f815ba9d4009f][3]
  (BackSequentialAndLink
    (SLink
      (ZLink
      ) ; [800fbffffffe8ce4][3]
    ) ; [da5f815ba9d4009f][3]
    (AndLink (stv 0.1 0.0588235)
      (ExecutionLink
        (SchemaNode "go_to_house") ; [7e1737e3e117d059][3]
      ) ; [93427319ec122fff][3]
      (EvaluationLink (stv 0.32 0.0588235)
        (PredicateNode "hold") ; [4b1b7a8b0a4d2853][3]
        (ListLink
          (ConceptNode "self") ; [40b11d11524bd751][3]
          (ConceptNode "key") ; [4d0844146f96d3][3]
        ) ; [e7a9c95ae7484b28][3]
      ) ; [e6a8f21e6b37d8f0][3]
    ) ; [876645d6528a6fbb][3]
    (ExecutionLink
      (SchemaNode "go_to_diamonds") ; [7aee74cf6bad6442][3]
    ) ; [a2630f96ffbe0861][3]
  ) ; [c6e23df43eb8fbe6][3]
  (EvaluationLink (stv 0.1 0.0588235)
    (PredicateNode "Reward") ; [155bb4d713db0d51][3]
    (NumberNode "1") ; [2cf0956d543cff8e][3]
  ) ; [d3cee8bdda06ffcb][3]
) ; [f4dde218e5acedc6][3]
```

--------------------------------

### Build Library from Notebooks

Source: https://github.com/opencog/rocca/blob/master/README.md

Updates the Python library with code exported from Jupyter notebooks. This command is essential when making changes to notebook-based code that should be reflected in the installable package.

```bash
nbdev_build_lib
```

--------------------------------

### Monoaction Plan: Get Reward

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Represents a single action plan where the agent learns to achieve a reward. This plan requires the agent to be inside the house and then execute the 'go_to_diamonds' action.

```Scheme
(BackPredictiveImplicationScopeLink (stv 1 0.00621118)
  (VariableSet
  ) ; [80075fffffff523a][1]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][1]
  ) ; [da5f815ba9d4009f][1]
  (AndLink
    (EvaluationLink
      (PredicateNode "inside") ; [63398dcfcf85c8a3][1]
      (ListLink
        (ConceptNode "self") ; [40b11d11524bd751][1]
        (ConceptNode "house") ; [63eb9919f37daa5f][1]
      ) ; [aadca36fe9d1a468][1]
    ) ; [871c182b52e89756][1]
    (ExecutionLink
      (SchemaNode "go_to_diamonds") ; [7aee74cf6bad6442][1]
    ) ; [a2630f96ffbe0861][1]
  ) ; [b4373ec3773f1783][1]
  (EvaluationLink
    (PredicateNode "Reward") ; [155bb4d713db0d51][1]
    (NumberNode "1") ; [2cf0956d543cff8e][1]
  ) ; [d3cee8bdda06ffcb][1]
) ; [8a298365b46b204b][1]
```

--------------------------------

### Run Static Type Checking on Subfolder (mypy.sh)

Source: https://github.com/opencog/rocca/blob/master/README.md

Navigates to a subfolder (e.g., 'examples') and then executes the mypy static type checker for that specific directory. This allows for targeted type checking within parts of the project.

```shell
cd examples
../tests/mypy.sh
```

--------------------------------

### Rocca Cognitive Schematic Example 1

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

A parsed representation of a cognitive schematic, likely defining a sequence of actions and evaluations within the Rocca project's AI framework.

```opencog-dsl
(SLink
    (ZLink
    ) ; [800fbffffffe8ce4][3]
  ) ; [da5f815ba9d4009f][3]
  (BackSequentialAndLink
    (SLink
      (ZLink
      ) ; [800fbffffffe8ce4][3]
    ) ; [da5f815ba9d4009f][3]
    (AndLink (stv 0.18 0.0588235)
      (EvaluationLink (stv 0.64 0.0588235)
        (PredicateNode "outside") ; [72730412e28a734][3]
        (ListLink
          (ConceptNode "self") ; [40b11d11524bd751][3]
          (ConceptNode "house") ; [63eb9919f37daa5f][3]
        ) ; [aadca36fe9d1a468][3]
      ) ; [ca0c329fb1ab493b][3]
      (ExecutionLink
        (SchemaNode "go_to_key") ; [7f46c329a5e57604][3]
      ) ; [f8086e6fdf73cdf4][3]
    ) ; [ddda31153cf2aa6d][3]
    (ExecutionLink
      (SchemaNode "go_to_house") ; [7e1737e3e117d059][3]
    ) ; [93427319ec122fff][3]
  ) ; [d526b02a321df7c7][3]
  (EvaluationLink (stv 0.34 0.0588235)
    (PredicateNode "inside") ; [63398dcfcf85c8a3][3]
    (ListLink
      (ConceptNode "self") ; [40b11d11524bd751][3]
      (ConceptNode "house") ; [63eb9919f37daa5f][3]
    ) ; [aadca36fe9d1a468][3]
  ) ; [871c182b52e89756][3]
) ; [c4a99a11fb34aeb2][3]
```

--------------------------------

### Get MineRL Action Space

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Retrieves the action space definition for the initialized MineRL environment, detailing the possible actions the agent can take.

```python
env.action_space
```

--------------------------------

### Get MineRL Observation Space

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Retrieves the observation space definition for the initialized MineRL environment, detailing the format and range of observations the agent receives.

```python
env.observation_space
```

--------------------------------

### Clone Malmo Repository

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Instructions to clone the Microsoft Malmo repository from GitHub. This is the first step in setting up the project environment.

```bash
git clone https://github.com/Microsoft/malmo
```

--------------------------------

### Launch Minecraft Client

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Command to launch the Minecraft client, specifying the port for connection. This is required before running the project's Python script.

```bash
cd Minecraft
.launchClient.sh -port 10000
```

--------------------------------

### Run Pong Agent

Source: https://github.com/opencog/rocca/blob/master/examples/pong/README.md

Executes the Pong agent script.

```bash
python pong.py
```

--------------------------------

### Initialize MineRL Environment

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Creates an instance of the MineRLNavigateDense-v0 environment, which is used for navigation tasks where dense rewards are provided based on distance to the target.

```python
env = gym.make("MineRLNavigateDense-v0")
```

--------------------------------

### Initialize OpenCog AtomSpace

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Creates a new instance of the OpenCog AtomSpace, which serves as the central knowledge representation and reasoning engine.

```python
atomspace = AtomSpace()
```

--------------------------------

### Initialize LearningCartPoleAgent

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Sets up the AtomSpace and wraps the CartPole environment for a LearningCartPoleAgent. This prepares the environment and cognitive architecture for a learning agent.

```python
atomspace = AtomSpace()
set_default_atomspace(atomspace)
wrapped_env = CartPoleWrapper(env, atomspace)
```

--------------------------------

### Clone Malmo Repository

Source: https://github.com/opencog/rocca/blob/master/examples/chase_malmo/README.md

Instructions to clone the Microsoft Malmo repository from GitHub. This step is necessary to obtain the Malmo environment, although a pre-built version is included in the project.

```bash
git clone https://github.com/Microsoft/malmo
```

--------------------------------

### Initialize Navigation Agent

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Initializes a NavigateAgent instance, passing the environment, the defined action space, and the positive/negative goals. This sets up the agent for interaction and learning within the specified environment.

```python
agent = NavigateAgent(wrapped_env, action_space, pgoal, ngoal)
```

--------------------------------

### Training Loop with TensorBoardX

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Implements a training loop for the agent, managing learning and interaction phases over multiple epochs. It uses TensorBoardX for logging accumulated rewards and agent statistics, tracking progress and performance.

```python
from tensorboardX import SummaryWriter

tb_writer = SummaryWriter(comment="-minerl-navigate")

epochs = 1  # Number of epochs (learning / interacting episodes)
epoch_len = 200

for i in range(epochs):
    wrapped_env.restart()
    agent.reset_action_counter()
    accreward = agent.accumulated_reward  # Keep track of the reward before

    # Learning phase: discover patterns to make more informed decisions
    log_msg(agent_log, f"Learning phase started. ({i + 1}/{epochs})")
    agent.learn()

    # Run agent to accumulate percepta
    log_msg(agent_log, f"Interaction phase started. ({i + 1}/{epochs})")
    for j in range(epoch_len):
        done = agent.control_cycle()
        # wrapped_env.render() uncomment to see the rendered env
        time.sleep(0.01)
        log.info("cycle_count = {}".format(agent.cycle_count))
        if done:
            break

    new_reward = agent.accumulated_reward - accreward
    tb_writer.add_scalar("train/accumulated_reward", new_reward, agent.cycle_count)
    log_msg(
        agent_log, "Accumulated reward during {}th epoch = {}".format(i + 1, new_reward)
    )
    log_msg(
        agent_log,
        "Action counter during {}th epoch:\n{}".format(i + 1, agent.action_counter),
    )  # TODO: make the action counter look good

log_msg(agent_log, f"The average total reward over {epochs} trials (training): {agent.accumulated_reward / epochs}.")
```

--------------------------------

### Run CartPole Agent Script

Source: https://github.com/opencog/rocca/blob/master/examples/cartpole/README.md

Executes the main Python script for the CartPole agent. This command initiates the agent's operation and interaction with the CartPole-v1 environment.

```bash
python cartpole.py
```

--------------------------------

### Initialize FixedCartPoleAgent

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Sets up the AtomSpace and wraps the CartPole environment for use with a FixedCartPoleAgent. This prepares the environment and cognitive architecture for agent interaction.

```python
atomspace = AtomSpace()
set_default_atomspace(atomspace)
wrapped_env = CartPoleWrapper(env, atomspace)
```

--------------------------------

### Monoaction Plan: Hold Key

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Represents a single action plan where the agent learns to hold a key. It requires the agent to be outside the house and then execute the 'go_to_key' action.

```Scheme
(BackPredictiveImplicationScopeLink (stv 1 0.0111248)
  (VariableSet
  ) ; [80075fffffff523a][1]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][1]
  ) ; [da5f815ba9d4009f][1]
  (AndLink
    (EvaluationLink (stv 0.64 0.0588235)
      (PredicateNode "outside") ; [72730412e28a734][1]
      (ListLink
        (ConceptNode "self") ; [40b11d11524bd751][1]
        (ConceptNode "house") ; [63eb9919f37daa5f][1]
      ) ; [aadca36fe9d1a468][1]
    ) ; [ca0c329fb1ab493b][1]
    (ExecutionLink
      (SchemaNode "go_to_key") ; [7f46c329a5e57604][1]
    ) ; [f8086e6fdf73cdf4][1]
  ) ; [ddda31153cf2aa6d][1]
  (EvaluationLink (stv 0.32 0.0588235)
    (PredicateNode "hold") ; [4b1b7a8b0a4d2853][1]
    (ListLink
      (ConceptNode "self") ; [40b11d11524bd751][1]
      (ConceptNode "key") ; [4d0844146f96d3][1]
    ) ; [e7a9c95ae7484b28][1]
  ) ; [e6a8f21e6b37d8f0][1]
) ; [ff069058911f0233][1]
```

--------------------------------

### Environment and Logging Python Imports

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Imports standard Python libraries like 'gym' and 'time', along with logging utilities from OpenCog and TensorBoardX for experiment tracking. These are used for environment interaction, timing, logging, and visualization.

```python
import gym
import time
import logging

from opencog.logger import log
from rocca.agents.core import logger as ac_logger
from tensorboardX import SummaryWriter
```

--------------------------------

### Monoaction Plan: Enter House

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Represents a single action plan for the agent to enter the house. This plan requires the agent to be outside, holding a key, and then execute the 'go_to_house' action.

```Scheme
(BackPredictiveImplicationScopeLink (stv 1 0.00621118)
  (VariableSet
  ) ; [80075fffffff523a][1]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][1]
  ) ; [da5f815ba9d4009f][1]
  (AndLink
    (ExecutionLink
      (SchemaNode "go_to_house") ; [7e1737e3e117d059][1]
    ) ; [93427319ec122fff][1]
    (EvaluationLink (stv 0.64 0.0588235)
      (PredicateNode "outside") ; [72730412e28a734][1]
      (ListLink
        (ConceptNode "self") ; [40b11d11524bd751][1]
        (ConceptNode "house") ; [63eb9919f37daa5f][1]
      ) ; [aadca36fe9d1a468][1]
    ) ; [ca0c329fb1ab493b][1]
    (EvaluationLink (stv 0.32 0.0588235)
      (PredicateNode "hold") ; [4b1b7a8b0a4d2853][1]
      (ListLink
        (ConceptNode "self") ; [40b11d11524bd751][1]
        (ConceptNode "key") ; [4d0844146f96d3][1]
      ) ; [e7a9c95ae7484b28][1]
    ) ; [e6a8f21e6b37d8f0][1]
  ) ; [fbf4892ec0643e2c][1]
  (EvaluationLink
    (PredicateNode "inside") ; [63398dcfcf85c8a3][1]
    (ListLink
      (ConceptNode "self") ; [40b11d11524bd751][1]
      (ConceptNode "house") ; [63eb9919f37daa5f][1]
    ) ; [aadca36fe9d1a468][1]
  ) ; [871c182b52e89756][1]
) ; [a2984b519c1ddc4f][1]
```

--------------------------------

### Makefile for Development

Source: https://github.com/opencog/rocca/blob/master/README.md

Convenience targets provided by the Makefile for common development tasks. The 'make rocca' command typically builds the library and formats the code.

```bash
make rocca
```

--------------------------------

### Polyaction Plan: Sequential House Entry

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Represents a sequential action plan for the agent to enter the house. This plan involves being outside and executing 'go_to_key' followed by 'go_to_house'.

```Scheme
(BackPredictiveImplicationScopeLink (stv 1 0.00559006)
  (VariableSet

```

--------------------------------

### Run Unit Tests (pytest)

Source: https://github.com/opencog/rocca/blob/master/README.md

Executes the unit tests for the ROCCA project using the pytest framework. This command should be run from the root folder of the project to ensure all tests are discovered and executed.

```python
pytest
```

--------------------------------

### Run Chase Agent Script

Source: https://github.com/opencog/rocca/blob/master/examples/chase/README.md

Command to execute the Chase agent simulation. This script initiates the agent's learning process within its defined environment.

```bash
python chase.py
```

--------------------------------

### Define Agent Action Space

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Specifies the set of possible actions an agent can take, represented as ExecutionLinks with SchemaNodes and parameters. This includes movement, interaction, and camera control actions, crucial for defining the agent's behavior.

```python
action_space = {
    ExecutionLink(SchemaNode("attack"), NumberNode("0")),
    ExecutionLink(SchemaNode("attack"), NumberNode("1")),
    ExecutionLink(SchemaNode("forward"), NumberNode("0")),
    ExecutionLink(SchemaNode("forward"), NumberNode("1")),
    ExecutionLink(SchemaNode("back"), NumberNode("0")),
    ExecutionLink(SchemaNode("back"), NumberNode("1")),
    ExecutionLink(SchemaNode("left"), NumberNode("0")),
    ExecutionLink(SchemaNode("left"), NumberNode("1")),
    ExecutionLink(SchemaNode("right"), NumberNode("0")),
    ExecutionLink(SchemaNode("right"), NumberNode("1")),
    ExecutionLink(SchemaNode("jump"), NumberNode("0")),
    ExecutionLink(SchemaNode("jump"), NumberNode("1")),
    ExecutionLink(SchemaNode("sprint"), NumberNode("0")),
    ExecutionLink(SchemaNode("sprint"), NumberNode("1")),
    ExecutionLink(SchemaNode("sneak"), NumberNode("0")),
    ExecutionLink(SchemaNode("sneak"), NumberNode("1")),
    ExecutionLink(SchemaNode("place"), ConceptNode("dirt")),
    ExecutionLink(SchemaNode("place"), ConceptNode("none")),
    ExecutionLink(SchemaNode("camera"), ListLink(NumberNode("2.5"), NumberNode("0.0"))),
    ExecutionLink(
        SchemaNode("camera"), ListLink(NumberNode("0.0"), NumberNode("-1.5"))
    ),
}
```

--------------------------------

### Format Code with Black

Source: https://github.com/opencog/rocca/blob/master/README.md

Applies the Black code formatter to all Python files in the project, ensuring consistent code style. This command should be run from the project root.

```bash
black .
```

--------------------------------

### Run Static Type Checking (mypy.sh)

Source: https://github.com/opencog/rocca/blob/master/README.md

Executes the mypy static type checker for the entire ROCCA project. This script is run from the root folder and checks all Python files for type annotation compliance.

```shell
tests/mypy.sh
```

--------------------------------

### Initialize Cognitive Schematics with BackPredictiveImplicationScopeLink

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

This Python function initializes cognitive schematics for an agent. It defines several `BackPredictiveImplicationScopeLink` instances, each representing a specific rule or context-action mapping, and updates the agent's cognitive schematics with these definitions. The schematics involve evaluating a pole angle against thresholds and executing corresponding actions like 'Go Left' or 'Go Right'.

```python
def seed_with(agent, knowledge):
    # TODO: figure out how to pass atoms to be inserted.
    set_default_atomspace(agent.atomspace)

    angle = VariableNode("$angle")
    numt = TypeNode("NumberNode")
    time_offset = to_nat(1)
    pole_angle = PredicateNode("Pole Angle")
    go_right = SchemaNode("Go Right")
    go_left = SchemaNode("Go Left")
    reward = PredicateNode("Reward")
    epsilon = NumberNode("0.01")
    mepsilon = NumberNode("-0.01")
    unit = NumberNode("1")
    hTV = TruthValue(0.9, 0.1)  # High TV
    lTV = TruthValue(0.1, 0.1)  # Low TV

    # Right-Right (High TV)
    cs_rr = BackPredictiveImplicationScopeLink(
            TypedVariableLink(angle, numt),
            time_offset,
            AndLink(
                # Context
                EvaluationLink(pole_angle, angle),
                GreaterThanLink(angle, epsilon),
                # Action
                ExecutionLink(go_right),
            ),
            # Goal
            EvaluationLink(reward, unit),
            # TV
            tv=hTV,
        )

    # Left-Left (High TV)
    cs_ll = BackPredictiveImplicationScopeLink(
            TypedVariableLink(angle, numt),
            time_offset,
            AndLink(
                # Context
                EvaluationLink(pole_angle, angle),
                GreaterThanLink(mepsilon, angle),
                # Action
                ExecutionLink(go_left),
            ),
            # Goal
            EvaluationLink(reward, unit),
            # TV
            tv=hTV,
        )

    # Right-Left (Low TV)
    cs_rl = BackPredictiveImplicationScopeLink(
            TypedVariableLink(angle, numt),
            time_offset,
            AndLink(
                # Context
                EvaluationLink(pole_angle, angle),
                GreaterThanLink(angle, epsilon),
                # Action
                ExecutionLink(go_left),
            ),
            # Goal
            EvaluationLink(reward, unit),
            # TV
            tv=lTV,
        )
        
    # Left-Right (Low TV)
    cs_lr = BackPredictiveImplicationScopeLink(
            TypedVariableLink(angle, numt),
            time_offset,
            AndLink(
                # Context
                EvaluationLink(pole_angle, angle),
                GreaterThanLink(mepsilon, angle),
                # Action
                ExecutionLink(go_right),
            ),
            # Goal
            EvaluationLink(reward, unit),
            # TV
            tv=lTV,
        )

    # Ideally we want to return only relevant cognitive schematics
    # (i.e. with contexts probabilistically currently true) for
    # now however we return everything and let to the deduction
    # process deal with it, as it should be able to.
    agent.cognitive_schematics.update(set([cs_ll, cs_ll, cs_rl, cs_lr]))  # TODO: the code should update Python-side automatically.

```

--------------------------------

### Run LearningCartPoleAgent Training Loop

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Instantiates and trains a LearningCartPoleAgent over multiple epochs. Each epoch consists of a learning phase and an interaction phase, accumulating rewards and logging progress.

```python
agent = LearningCartPoleAgent(wrapped_env, atomspace, log_level="fine")
agent.delta = 1.0e-16

# seed_with(agent, ["fixme"])

tb_writer = SummaryWriter(comment="-cartpole-learning-seeded")
ac_logger.setLevel(logging.DEBUG)  # The agents.core logger

epochs = 1  # Number of epochs (learning / interacting episodes)
epoch_len = 200

for i in range(epochs):
    wrapped_env.restart()
    agent.reset_action_counter()
    accreward = agent.accumulated_reward  # Keep track of the reward before

    # Learning phase: discover patterns to make more informed decisions
    log_msg(agent_log, f"Learning phase started. ({i + 1}/{epochs})")
    agent.learn()
    
    # Run agent to accumulate percepta
    log_msg(agent_log, f"Interaction phase started. ({i + 1}/{epochs})")
    for j in range(epoch_len):
        done = agent.control_cycle()
        # wrapped_env.render() uncomment to see the rendered env
        time.sleep(0.01)
        log.debug("cycle_count = {}".format(agent.cycle_count))
        if done:
            break
        
    new_reward = agent.accumulated_reward - accreward
    tb_writer.add_scalar("train/accumulated_reward", new_reward, agent.cycle_count)
    log_msg(agent_log, "Accumulated reward during {}th epoch = {}".format(i + 1, new_reward))
    log_msg(agent_log, "Action counter during {}th epoch:\n{}".format(i + 1, agent.action_counter))  # TODO: make the action counter look good

log_msg(agent_log, f"The average total reward over {epochs} trials (training): {agent.accumulated_reward / epochs}.")
# TODO: add a separate testing loop and measure average total reward.
```

--------------------------------

### Atomese Plans for Chase Agent

Source: https://github.com/opencog/rocca/blob/master/examples/chase/README.md

Represents the learned optimal action plans for the Chase agent in Atomese. These plans describe sequences of actions based on agent and pellet positions to achieve rewards, demonstrating temporal deduction.

```scheme
(BackPredictiveImplicationScopeLink (stv 1 0.00669975)
  (VariableSet
  ) ; [80075fffffff523a][3]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][3]
  ) ; [da5f815ba9d4009f][3]
  (BackSequentialAndLink
    (SLink
      (ZLink
      ) ; [800fbffffffe8ce4][3]
    ) ; [da5f815ba9d4009f][3]
    (AndLink (stv 0.03 0.2)
      (EvaluationLink (stv 0.12 0.2)
        (PredicateNode "Pellet Position") ; [56e6ab0f525cb504][3]
        (ConceptNode "None") ; [68c616828fa0f8e6][3]
      ) ; [a609b2dbedb7904e][3]
      (ExecutionLink
        (SchemaNode "Go Right") ; [51c7a48fd94d12d8][3]
      ) ; [c29bf0559d1ad8ec][3]
      (EvaluationLink (stv 0.37 0.2)
        (PredicateNode "Agent Position") ; [3fdca752fd5e5335][3]
        (ConceptNode "Left Square") ; [586f8a0db3b1388a][3]
      ) ; [ec8ec4a729ccab8c][3]
    ) ; [f07c6a1c8318095c][3]
    (ExecutionLink
      (SchemaNode "Eat") ; [3fe4e22345c3679f][3]
    ) ; [9efce1dc8918c209][3]
  ) ; [b7ee6db6cd3a43bf][3]
  (EvaluationLink (stv 0.12 0.2)
    (PredicateNode "Reward") ; [155bb4d713db0d51][3]
    (NumberNode "1") ; [2cf0956d543cff8e][3]
  ) ; [d3cee8bdda06ffcb][3]
) ; [cac8b7b130c2115c][3]
```

```scheme
(BackPredictiveImplicationScopeLink (stv 1 0.00447761)
  (VariableSet
  ) ; [80075fffffff523a][3]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][3]
  ) ; [da5f815ba9d4009f][3]
  (BackSequentialAndLink
    (SLink
      (ZLink
      ) ; [800fbffffffe8ce4][3]
    ) ; [da5f815ba9d4009f][3]
    (AndLink (stv 0.02 0.2)
      (EvaluationLink (stv 0.12 0.2)
        (PredicateNode "Pellet Position") ; [56e6ab0f525cb504][3]
        (ConceptNode "None") ; [68c616828fa0f8e6][3]
      ) ; [a609b2dbedb7904e][3]
      (ExecutionLink
        (SchemaNode "Go Left") ; [7ca250f2efc2e872][3]
      ) ; [c7fb76d9605d5db5][3]
      (EvaluationLink (stv 0.63 0.2)
        (PredicateNode "Agent Position") ; [3fdca752fd5e5335][3]
        (ConceptNode "Right Square") ; [6dd382acb6aa376e][3]
      ) ; [c9fcc2094e0150df][3]
    ) ; [e1b8c69b9a37aa32][3]
    (ExecutionLink
      (SchemaNode "Eat") ; [3fe4e22345c3679f][3]
    ) ; [9efce1dc8918c209][3]
  ) ; [b470646c0c97b387][3]
  (EvaluationLink (stv 0.12 0.2)
    (PredicateNode "Reward") ; [155bb4d713db0d51][3]
    (NumberNode "1") ; [2cf0956d543cff8e][3]
  ) ; [d3cee8bdda06ffcb][3]
) ; [8f007faa792823de][3]
```

--------------------------------

### Wrap MineRL Environment

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Applies the MineRLWrapper to the standard MineRL environment, potentially adding custom functionalities or adapting its interface for the agent.

```python
wrapped_env = MineRLWrapper(env)
```

--------------------------------

### Polyaction Plan: Sequential Reward Acquisition

Source: https://github.com/opencog/rocca/blob/master/examples/collect_diamonds/README.md

Represents a sequential action plan for the agent to acquire a reward. This plan involves multiple steps: being outside, going to the key, going to the house, and finally going to the diamonds.

```Scheme
(BackPredictiveImplicationScopeLink (stv 1 0.00503106)
  (VariableSet
  ) ; [80075fffffff523a][3]
  (SLink
    (ZLink
    ) ; [800fbffffffe8ce4][3]
  ) ; [da5f815ba9d4009f][3]
  (BackSequentialAndLink
    (SLink
      (ZLink
      ) ; [800fbffffffe8ce4][3]
    ) ; [da5f815ba9d4009f][3]
    (BackSequentialAndLink
      (SLink
        (ZLink
        ) ; [800fbffffffe8ce4][3]
      ) ; [da5f815ba9d4009f][3]
      (AndLink (stv 0.18 0.0588235)
        (EvaluationLink (stv 0.64 0.0588235)
          (PredicateNode "outside") ; [72730412e28a734][3]
          (ListLink
            (ConceptNode "self") ; [40b11d11524bd751][3]
            (ConceptNode "house") ; [63eb9919f37daa5f][3]
          ) ; [aadca36fe9d1a468][3]
        ) ; [ca0c329fb1ab493b][3]
        (ExecutionLink
          (SchemaNode "go_to_key") ; [7f46c329a5e57604][3]
        ) ; [f8086e6fdf73cdf4][3]
      ) ; [ddda31153cf2aa6d][3]
      (ExecutionLink
        (SchemaNode "go_to_house") ; [7e1737e3e117d059][3]
      ) ; [93427319ec122fff][3]
    ) ; [d526b02a321df7c7][3]
    (ExecutionLink
      (SchemaNode "go_to_diamonds") ; [7aee74cf6bad6442][3]
    ) ; [a2630f96ffbe0861][3]
  ) ; [b0235094e5713353][3]
  (EvaluationLink (stv 0.1 0.0588235)
    (PredicateNode "Reward") ; [155bb4d713db0d51][3]
    (NumberNode "1") ; [2cf0956d543cff8e][3]
  ) ; [d3cee8bdda06ffcb][3]
) ; [bd6c515d92fe0be1][3]
```

--------------------------------

### Define OpenCog Goals

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Defines positive and negative goals for an agent using OpenCog's EvaluationLink, PredicateNode, and NumberNode. These goals represent desired outcomes or states for the agent's learning process.

```python
pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("100"))
ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
```

--------------------------------

### Set Default AtomSpace

Source: https://github.com/opencog/rocca/blob/master/03_minerl_navigate_agent.ipynb

Configures the previously created AtomSpace instance as the default for subsequent OpenCog operations, ensuring consistency.

```python
set_default_atomspace(atomspace)
```

--------------------------------

### Define LearningCartPoleAgent Class

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Defines the LearningCartPoleAgent, a specialized OpenCog agent for the CartPole environment. It initializes action spaces, goals, and customizes agent parameters for learning.

```python
#export

class LearningCartPoleAgent(OpencogAgent):
    def __init__(self, env: CartPoleWrapper, atomspace: AtomSpace, log_level="debug"):
        set_default_atomspace(atomspace)
        
        # Create Action Space. The set of allowed actions an agent can take.
        # TODO take care of action parameters.
        action_space = {ExecutionLink(SchemaNode(a)) for a in env.action_names}

        # Create Goal
        pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
        ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))

        # Call super ctor
        super().__init__(env, atomspace, action_space, pgoal, ngoal, log_level=log_level)

        # Overwrite some OpencogAgent parameters
        self.monoaction_general_succeedent_mining = False
        self.polyaction_mining = False
        self.temporal_deduction = False
```

--------------------------------

### OpenCog and ROCCA Python Imports

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Imports core components from the OpenCog framework and the ROCCA library, including PLN, type constructors, utilities, and environment wrappers. These imports are essential for setting up and running the CartPole agent.

```python
#export

# OpenCog
from opencog.pln import *
from opencog.type_constructors import *
from opencog.utilities import set_default_atomspace

# ROCCA
from rocca.envs.wrappers import CartPoleWrapper
from rocca.agents import OpencogAgent
from rocca.agents.utils import *

from rocca.utils import *
```

--------------------------------

### OpenCog Cognitive Schematic Construction

Source: https://github.com/opencog/rocca/blob/master/01_cartpole.ipynb

Demonstrates the construction of OpenCog cognitive schematics using various link types like BackPredictiveImplicationScopeLink, TypedVariableLink, AndLink, EvaluationLink, GreaterThanLink, and ExecutionLink. These schematics represent rules for agent behavior, mapping environmental states to actions.

```APIDOC
BackPredictiveImplicationScopeLink <tv>
  <vardecl>
  <expiry>
  And (or SimultaneousAnd?)
    <context>
    Execution
      <action>
      <input> [optional]
      <output> [optional]
  <goal>

TypedVariableLink
  VariableNode <variable_name>
  TypeNode <type_name>

AndLink
  <condition1>
  <condition2>
  ...

EvaluationLink
  PredicateNode <predicate_name>
  <value>

GreaterThanLink
  <operand1>
  <operand2>

ExecutionLink
  SchemaNode <schema_name>

PredicateNode <predicate_name>

VariableNode <variable_name>

NumberNode <number_value>

SchemaNode <schema_name>

TruthValue <tv_strength>, <tv_confidence>

Example: Go Right Rule
  BackPredictiveImplicationScopeLink (tv=TruthValue(0.9, 0.1))
    TypedVariableLink (Variable="$angle", Type="NumberNode")
    Time (1)
    AndLink
      EvaluationLink (Predicate="Pole Angle", Value=Variable("$angle"))
      GreaterThanLink (Variable("$angle"), Number("0.01"))
      ExecutionLink (Schema="Go Right")
    EvaluationLink (Predicate="Reward", Value=Number("1"))

Example: Go Left Rule
  BackPredictiveImplicationScopeLink (tv=TruthValue(0.9, 0.1))
    TypedVariableLink (Variable="$angle", Type="NumberNode")
    Time (1)
    AndLink
      EvaluationLink (Predicate="Pole Angle", Value=Variable("$angle"))
      GreaterThanLink (Number("-0.01"), Variable("$angle"))
      ExecutionLink (Schema="Go Left")
    EvaluationLink (Predicate="Reward", Value=Number("1"))

```