### Install Pandas

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Command to install the pandas library, which is required for reading Excel configuration files.

```bash
pip install pandas
```

--------------------------------

### Install Openpyxl

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Command to install the openpyxl library, necessary for supporting the Excel file format.

```bash
pip install openpyxl
```

--------------------------------

### Example XML File Structure

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Illustrates the typical structure and content of a generated task XML file.

```xml
<?xml version="1.0" encoding="utf-8"?>
<Level>
  <Slingshot x="-6.51" y="7.1" />
  <Birds>
    <Bird type="BirdRed"/>
  </Birds>
  <GameObjects>
    <Pig type="BasicSmall" material="" x="-2.5" y="1.0" rotation="0" />
    <Block type="RectBig" material="wood" x="0.0" y="0.5" rotation="0" />
    <Platform type="Platform" x="1.0" y="-2.0" rotation="0" scaleX="2.0" scaleY="1.0" />
  </GameObjects>
</Level>
```

--------------------------------

### Install Dependencies

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/README.md

Install necessary Python packages for NovPhy. This is required to resolve ModuleNotFoundError.

```bash
pip install openpyxl pandas
```

--------------------------------

### Example Config Dictionary

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/types.md

Illustrates the structure of the cleaned configuration data, mapping template names to their configuration details including reference points and restricted object types.

```python
cleaned_config_data = {
    '1_0_1': [[-6.51, 7.1], [-5.84, 6.91], [-5.274, 6.643], []],
    '1_1_1': [[...], [...], [...], ['wood circle', 'ice rectfat']]
}
```

--------------------------------

### Using Generation Utilities in Custom Code

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Demonstrates how to import and instantiate core classes from utility modules to generate levels and get spatial information.

```python
# Import core classes
from utils.data_classes import Block, Pig
from utils.generate_variations import GenerateLevels
from utils.constants import blocks, reachability_line

# Create instance
gen = GenerateLevels()

# Use methods
x, y = gen.get_reachable_location_using_reachability_line(-8, 5, -2, 4)
h_span, v_span = gen.get_horizontal_and_vertical_span(block)
```

--------------------------------

### Develop Custom Agent with Gym-like Environment

Source: https://github.com/phy-q/novphy/blob/main/README.md

Example of developing a custom agent using a gym-like environment. Initializes the environment with specific reward types and game speed, then proceeds to play through a list of levels.

```python
from SBAgent import SBAgent
from SBEnvironment.SBEnvironmentWrapper import SBEnvironmentWrapper

# for using reward as score and 50 times faster game play
env = SBEnvironmentWrapper(reward_type="score", speed=50)
level_list = [1, 2, 3]  # level list for the agent to play
dummy_agent = SBAgent(env=env, level_list=level_list)  # initialise agent
dummy_agent.state_representation_type = 'image'  # use symbolic representation as state and headless mode
env.make(agent=dummy_agent, start_level=dummy_agent.level_list[0],
         state_representation_type=dummy_agent.state_representation_type)  # initialise the environment

s, r, is_done, info = env.reset()  # get ready for running
for level_idx in level_list:
    is_done = False
    while not is_done:
        s, r, is_done, info = env.step([-100, -100])  # agent always shoots at -100,100 as relative to the slingshot

    env.current_level = level_idx+1  # update the level list once finished the level
    if env.current_level > level_list[-1]: # end the game when all game levels in the level list are played
        break
    s, r, is_done, info = env.reload_current_level() #go to the next level
```

--------------------------------

### Level Data Structure Example

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/types.md

Illustrates how to unpack the LevelData tuple into lists of blocks, pigs, and tnts.

```python
all_blocks, all_pigs, all_tnts = template_data[0], template_data[1], template_data[2]
```

--------------------------------

### Verify XML Structure

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Use this command to verify the XML structure of a novelty level file. Ensure you have Python installed.

```bash
python -m xml.etree.ElementTree output/novelty_level_0/type01/Levels/00001_type01_0_0_1.xml
```

--------------------------------

### Unit Test for Generation Module

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Example of unit testing for the GenerateLevels class, verifying the output of location generation methods.

```python
# Test generation
from utils.generate_variations import GenerateLevels
gen = GenerateLevels()
x, y = gen.get_location_in_reachability_line()
assert -7 < x < 10
```

--------------------------------

### XML Structure for Combined Evaluation Configuration

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Example of the XML output format for combined trial configurations. Each trial includes game level sets for training and testing, with specified limits and parameters.

```xml
<?xml version="1.0" encoding="utf-16"?>
<evaluation>
  <novelty_detection_measurement step="1" measure_in_training="True" measure_in_testing="True" />
  <trials>
    <trial id="0" number_of_executions="1" checkpoint_time_limit="200" checkpoint_interaction_limit="200" notify_novelty="False">
      <game_level_set mode="training" time_limit="10000" total_interaction_limit="60000" attempt_limit_per_level="1" allow_level_selection="False">
        <game_levels level_path="novelty_level_0/type01/Levels/00001_type01_0_0_1.xml" />
        <!-- More levels... -->
      </game_level_set>
    </trial>
    <!-- More trials... -->
  </trials>
</evaluation>
```

--------------------------------

### Get game state

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the current state of the game.

```APIDOC
## Get game state

### Description
Retrieves the current state of the game.

### Request
Message ID: 12
Format: [12]

### Return
One byte indicating the ordinal of the state:
- [0]: UNKNOWN
- [1]: MAIN_MENU
- [2]: EPISODE_MENU
- [3]: LEVEL_SELECTION
- [4]: LOADING
- [5]: PLAYING
- [6]: WON
- [7]: LOST
```

--------------------------------

### Example CSV Format for All Passed Non-Novel Levels

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/filter-utilities.md

Shows the format for `all_passed_non_novel.csv`, which lists the paths of all non-novel levels that have passed validation. This is used as a reference for creating curated copies.

```csv
path
novelty_level_0/type01/Levels/00001_type01_0_0_1.xml
novelty_level_0/type01/Levels/00002_type01_0_0_1.xml
...
```

--------------------------------

### Run Java Game Playing Interface

Source: https://github.com/phy-q/novphy/blob/main/README.md

Start the game playing interface by navigating to 'sciencebirdsgames/Linux' and running the JAR file. This is required before running Java heuristic agents.

```sh
java -jar game_playing_interface.jar
```

--------------------------------

### Generate Tasks (Custom Count)

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/README.md

Generate a specified number of variants per template. This example generates 100 variants, resulting in around 2,000 files.

```bash
cd tasks/task_generator
python generate_tasks.py 100            # Only 100 variants per template
# Output: ~2,000 files in ./output/
```

--------------------------------

### Unit Test for Data Classes

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Example of unit testing for the Block data class, verifying attribute assignments.

```python
# Test data_classes
from utils.data_classes import Block
b = Block(1, 'RectBig', 'wood', 0, 1, 45)
assert b.identifier == 1
assert b.type == 'RectBig'
```

--------------------------------

### Unit Test for Constants

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Example of unit testing for the constants module, checking for expected values in the 'blocks' dictionary.

```python
# Test constants
from utils.constants import blocks
assert 'RectBig' in blocks
assert len(blocks['RectBig']) == 2
```

--------------------------------

### Import Libraries for Analysis

Source: https://github.com/phy-q/novphy/blob/main/analysis/adaptation_plots.ipynb

Imports essential Python libraries for data manipulation, analysis, and plotting. Ensure these libraries are installed.

```python
import pandas as pd
import os
import plotly as plt
import plotly.graph_objs as go
import plotly.express as px
import numpy as np
from plotly.subplots import make_subplots
import sklearn.metrics as metrics
import seaborn as sns
import matplotlib
```

--------------------------------

### Example CSV Format for Levels to Delete

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/filter-utilities.md

Specifies the expected format for the `levels_to_delete.csv` file. Only the `level_filename` column is used by the script for identifying files to remove.

```csv
index, level_filename, failure_reason, notes
1, 00001_type01_0_0_1.xml, unsolvable, no valid trajectory
2, 00005_type02_0_1_1.xml, broken_geometry, overlapping objects
...
```

--------------------------------

### Python Plotting Example

Source: https://github.com/phy-q/novphy/blob/main/analysis/adaptation_plots.ipynb

A basic Python code snippet for generating plots, likely used in the analysis of agent behavior. Ensure necessary libraries like matplotlib or seaborn are imported.

```python
import matplotlib.pyplot as plt
import numpy as np

# Sample data
x = np.linspace(0, 10, 100)
y = np.sin(x)

# Create plot
plt.figure(figsize=(8, 6))
plt.plot(x, y, label='Sine Wave')
plt.title('Sample Plot')
plt.xlabel('X-axis')
plt.ylabel('Y-axis')
plt.legend()
plt.grid(True)
plt.show()
```

--------------------------------

### Get Location in Reachable Space (Alternative)

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

An alternative method for generating locations in reachable space with inverse Y constraint logic compared to `get_location_in_reachable_space`. Use when Y is limited in the lower half for X values greater than the midpoint.

```python
def get_location_in_reachable_space_2(self, x_min: float, x_max: float, 
                                      y_min: float, y_max: float) -> tuple
```

--------------------------------

### Prepare Test Configuration

Source: https://github.com/phy-q/novphy/blob/main/README.md

Prepare the test configuration by specifying the operating system. This script is located in the 'Utils' directory.

```bash
python PrepareTestConfig.py --os [Linux/MacOS]
```

--------------------------------

### Get the current level

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the index of the current level.

```APIDOC
## Get the current level

### Description
Retrieves the index of the current level.

### Request
Message ID: 14
Format: [14]

### Return
Four bytes array indicating the index of the current level: [level index]
```

--------------------------------

### Get the number of levels

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the total number of available levels.

```APIDOC
## Get the number of levels

### Description
Retrieves the total number of available levels.

### Request
Message ID: 15
Format: [15]

### Return
Four bytes array indicating the number of available levels: [number of level]
```

--------------------------------

### Inspect Generated XML Configuration

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

View the beginning of the generated XML configuration file, which defines trial and level parameters.

```bash
head -50 ./config_files/config.xml
```

--------------------------------

### Create Output Directory

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Ensure the output directory exists and has the correct permissions by creating it with 'mkdir -p' and verifying with 'ls -ld'. This prevents issues with generated files not being saved.

```bash
mkdir -p output/
ls -ld output/
```

--------------------------------

### Get Current Level Score

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the current score of the level. This can be requested at any time.

```APIDOC
## Get Current Level Score

### Description
Retrieves the current score of the level. This score can be requested at any time during the Playing, Won, or Lost states. It is particularly useful for agents that consider intermediate scores during training or reasoning. To obtain the winning score, ensure this command is executed when the game state is "WON".

### Method
Not specified (assumed to be a command code).

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None.

#### Query Parameters
None.

#### Request Body
None.

### Request Example
```json
[
  65
]
```

### Response
#### Success Response
- **Score** (4 bytes) - The current score of the level.

#### Response Example
```json
[
  [score]
]
```
```

--------------------------------

### Run Agent Evaluation

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Launch the agent evaluation using your agent framework, pointing to the generated configuration file. This step assesses the performance of agents on the generated tasks.

```bash
your-agent-framework ./config_files/config.xml
```

--------------------------------

### Get Noisy Symbolic Representation With Screenshot

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the noisy symbolic representation and a screenshot.

```APIDOC
## Get Noisy Symbolic Representation With Screenshot

### Description
Retrieves the noisy symbolic representation of the current state and a corresponding screenshot.

### Method
Not specified (assumed to be a command code).

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None.

#### Query Parameters
None.

#### Request Body
None.

### Request Example
```json
[
  63
]
```

### Response
#### Success Response
- **Noisy Symbolic Representation** (byte array) - The noisy symbolic representation of the state.
- **Screenshot** (image bytes) - The screenshot of the current state.
- **Symbolic representation byte array length** (4 bytes) - The length of the symbolic representation byte array.
- **Image width** (4 bytes) - The width of the image.
- **Image height** (4 bytes) - The height of the image.

#### Response Example
```json
[
  [symbolic representation byte array length],
  [Symbolic Representation bytes],
  [image width],
  [image height],
  [image bytes]
]
```
```

--------------------------------

### Constructor

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Initializes a new GenerateLevels instance. This is the entry point for using the class's functionalities.

```APIDOC
## Constructor

### Description
Initializes a new GenerateLevels instance. No parameters required.

### Method
__init__

### Parameters
None

### Example
```python
level_gen = GenerateLevels()
```
```

--------------------------------

### Get Symbolic Representation Without Screenshot

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves only the symbolic representation of the current state.

```APIDOC
## Get Symbolic Representation Without Screenshot

### Description
Retrieves only the symbolic representation of the current state, without a screenshot.

### Method
Not specified (assumed to be a command code).

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None.

#### Query Parameters
None.

#### Request Body
None.

### Request Example
```json
[
  62
]
```

### Response
#### Success Response
- **Symbolic Representation** (byte array) - The symbolic representation of the state.
- **Symbolic representation byte array length** (4 bytes) - The length of the symbolic representation byte array.

#### Response Example
```json
[
  [symbolic representation byte array length],
  [Symbolic Representation bytes]
]
```
```

--------------------------------

### Train Agent for Benchmark

Source: https://github.com/phy-q/novphy/blob/main/README.md

Execute the training script for benchmark scenarios. Ensure execution permissions are granted.

```sh
./TrainLearningAgent.sh benchmark
```

--------------------------------

### Get Symbolic Representation With Screenshot

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the symbolic representation of the current state along with a screenshot.

```APIDOC
## Get Symbolic Representation With Screenshot

### Description
Retrieves the symbolic representation of the current state and a corresponding screenshot.

### Method
Not specified (assumed to be a command code).

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None.

#### Query Parameters
None.

#### Request Body
None.

### Request Example
```json
[
  61
]
```

### Response
#### Success Response
- **Symbolic Representation** (byte array) - The symbolic representation of the state.
- **Screenshot** (image bytes) - The screenshot of the current state.
- **Symbolic representation byte array length** (4 bytes) - The length of the symbolic representation byte array.
- **Image width** (4 bytes) - The width of the image.
- **Image height** (4 bytes) - The height of the image.

#### Response Example
```json
[
  [symbolic representation byte array length],
  [Symbolic Representation bytes],
  [image width],
  [image height],
  [image bytes]
]
```
```

--------------------------------

### Get Location on Reachability Line

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Retrieves a random coordinate that lies directly on the defined reachability line.

```python
def get_location_in_reachability_line(self) -> tuple
```

--------------------------------

### Get Noisy Symbolic Representation Without Screenshot

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves only the noisy symbolic representation of the current state.

```APIDOC
## Get Noisy Symbolic Representation Without Screenshot

### Description
Retrieves only the noisy symbolic representation of the current state, without a screenshot.

### Method
Not specified (assumed to be a command code).

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None.

#### Query Parameters
None.

#### Request Body
None.

### Request Example
```json
[
  64
]
```

### Response
#### Success Response
- **Noisy Symbolic Representation** (byte array) - The noisy symbolic representation of the state.
- **Symbolic representation byte array length** (4 bytes) - The length of the symbolic representation byte array.

#### Response Example
```json
[
  [symbolic representation byte array length],
  [Symbolic Representation bytes]
]
```
```

--------------------------------

### GenerateVariations Class Instantiation and Main Method Call

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/overview.md

Shows how to instantiate the GenerateVariations class and initiate the level generation process by calling its main method.

```python
gen = GenerateVariations()
gen.main()

```

--------------------------------

### Module-Level Execution for Configuration Generation

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Demonstrates the module-level execution pattern in generate_configs.py, where code runs automatically upon import to process configuration files.

```python
# Read split configs
splitted_config_files = [...]

# Generate combined config
for split_config_file in splitted_config_files:
    # Combine normal and novel
    # Write config.xml and generated_config_info.csv
```

--------------------------------

### Run Configuration Generator

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Execute the Python script to generate configuration files. The output includes CSV and XML files.

```bash
python generate_configs.py
```

--------------------------------

### Execute Configuration Generation Script

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Command to run the configuration generation script. Ensure you are in the correct directory and that prerequisite split config files are present.

```bash
cd tasks/task_generator/utils/
python generate_configs.py
```

--------------------------------

### Run Small Test Case

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Execute a small test run with a specified number of tasks. This is useful for quick checks.

```bash
python generate_tasks.py 5
```

--------------------------------

### Backup and Filter Levels

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/filter-utilities.md

Demonstrates a safe workflow for filtering levels by first creating a backup of the original levels directory. This is crucial as filtering is a destructive operation.

```bash
# Backup original levels
cp -r generated_levels/ generated_levels.backup/

# Then run filter
python filter_levels.py
```

--------------------------------

### get_occupied_x_spans_above_y_axis

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Gets all horizontal intervals occupied by objects above a specified Y coordinate. This is useful for sky placement and avoids conflicts with the sky platform.

```APIDOC
## get_occupied_x_spans_above_y_axis

### Description
Gets all horizontal intervals occupied by objects above a Y coordinate (for sky placement). Skips the sky platform at Y=10.3 to avoid false conflicts.

### Method
GET (assumed, based on naming convention)

### Endpoint
(Not specified, likely an internal method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Request Example
```python
# Example usage within a class context
occupied_spans = self.get_occupied_x_spans_above_y_axis(template_data, 10.0)
```

### Response
#### Success Response (200)
- **occupied_spans** (list) - A list of [min_x, max_x] intervals occupied by objects above the specified Y-axis.
```

--------------------------------

### Input Format for Split Configuration Files

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Specifies the expected format for split configuration files, where each line contains a path to a level XML file. These files are used as input for generating combined configurations.

```text
novelty_level_0/type01/Levels/00001_type01_0_0_1.xml
novelty_level_0/type01/Levels/00002_type01_0_0_1.xml
novelty_level_0/type01/Levels/00003_type01_0_0_1.xml
novelty_level_0/type02/Levels/00001_type02_0_0_1.xml
...
```

--------------------------------

### Get Occupied Horizontal Spans Below Y-Axis

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Retrieve all horizontal intervals occupied by objects located below a specified Y-coordinate using `get_occupied_x_spans_below_y_axis`.

```python
level_gen.get_occupied_x_spans_below_y_axis(template_data, y_axis=-10.0)
```

--------------------------------

### main

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-variations.md

The main entry point that orchestrates the complete generation pipeline, from scanning input files to writing all generated level variations.

```APIDOC
## main

### Description
Main entry point that orchestrates the complete generation pipeline.

### Method
```python
def main(self) -> None
```

### Parameters
This method does not accept any parameters.

### Returns
- None

### Details
- Scans the input directory for all template files.
- Reads configuration from the Excel file.
- For each template file, it extracts game objects, derives the template name, generates level variations, and writes them to organized output folders.
- Processes files in directory order.

### Example
```python
if __name__ == "__main__":
    generator = GenerateVariations()
    generator.main()  # Generates all variations from ./input/*.xml
```
```

--------------------------------

### get_location_in_reachability_line

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Gets a random location directly on the reachability line. This is useful for placing objects or defining points that must lie on a specific path.

```APIDOC
## get_location_in_reachability_line

### Description
Gets a random location directly on the reachability line.

### Method
N/A (Python method)

### Parameters
None

### Returns
- **tuple** - Tuple of `(x: float, y: float)` on reachability line
```

--------------------------------

### Constructor

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-variations.md

Initializes a new GenerateVariations instance. Configuration is managed through instance methods after initialization.

```APIDOC
## Constructor

### Description
Initializes a new GenerateVariations instance with no parameters. All configuration is handled through instance methods.

### Method
__init__

### Parameters
None

### Returns
GenerateVariations instance

### Example
```python
generator = GenerateVariations()
```
```

--------------------------------

### Get Pass Rate DataFrames

Source: https://github.com/phy-q/novphy/blob/main/analysis/adaptation_plots.ipynb

Generates pass rate dataframes for various agents. This function is used to prepare data for further analysis and visualization.

```python
dqn_offline_1 = get_pass_rate_df(dqn_offline, "DQN Offline")
dqn_online_1 = get_pass_rate_df(dqn_online, "DQN Online")
dqn_adapt_1 = get_pass_rate_df(dqn_adapt, "DQN Adapt")

relational_offline_1 = get_pass_rate_df(relational_offline, "Relational Offline")
relational_online_1 = get_pass_rate_df(relational_online, "Relational Online")
relational_adapt_1 = get_pass_rate_df(relational_adapt, "Relational Adapt")

naive_adaptation_1 = get_pass_rate_df(naive_adaptation, "Naive Adapt")

datalab_1 = get_pass_rate_df(datalab, "Datalab")
eagles_wing_1 = get_pass_rate_df(eagles_wing, "Eagle's Wing")
pig_shooter_1 = get_pass_rate_df(pig_shooter, "Pig Shooter")
random_1 = get_pass_rate_df(random, "Random")
```

--------------------------------

### Complete Level Generation and Evaluation Pipeline

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/overview.md

Outlines the complete workflow for generating levels, validating them using a game simulator, filtering invalid levels, generating evaluation configurations, and running agent evaluations.

```bash
# 1. Generate all levels
cd tasks/task_generator
python generate_tasks.py 350

# 2. Validate externally (run game simulator)
# ... produces levels_to_delete.csv ...

# 3. Filter out invalid levels
cd utils
python filter_levels.py

# 4. Generate evaluation config
python generate_configs.py

# 5. Run agent evaluation
# ... agent_framework config.xml ...
```

--------------------------------

### Train and Test OpenAI Stable Baselines Agent for Benchmark

Source: https://github.com/phy-q/novphy/blob/main/README.md

Execute the script to train and test OpenAI Stable Baselines agents for benchmark scenarios.

```sh
./TrainAndTestOpenAIStableBaselines.sh benchmark
```

--------------------------------

### Get Block Spans

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Retrieve the horizontal and vertical spans of a given block using the GenerateLevels class. This is helpful for collision detection and spatial analysis.

```python
# Check collisions
h_span, v_span = level_gen.get_horizontal_and_vertical_span(block)
```

--------------------------------

### Full Pipeline: Generation, Validation, and Configuration

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/README.md

A multi-step workflow to generate tasks, validate them (assuming an external process creates levels_to_delete.csv), filter levels, and create evaluation configurations.

```bash
# 1. Generate
cd tasks/task_generator
python generate_tasks.py 350

# 2. Validate (external process produces levels_to_delete.csv)
# ... run game simulator ...

# 3. Filter
cd utils
python filter_levels.py

# 4. Create config
python generate_configs.py

# 5. Run evaluation
# ... your-agent-framework config_files/config.xml ...
```

--------------------------------

### File Paths for Configuration Generation

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Defines input and output file paths for the configuration generation utility. Ensure input files exist in the specified directory and output files will be created or overwritten.

```python
splitted_config_file_path = './config_files/splitted_configs/'
combined_config_file_path = './config_files/config.xml'
generated_config_info_file_path = './config_files/generated_config_info.csv'
```

--------------------------------

### Initialize GenerateVariations

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-variations.md

Instantiates the GenerateVariations class. Configuration is managed through instance methods after initialization.

```python
generator = GenerateVariations()
```

--------------------------------

### Delete Intermediate Novelty Levels

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Free up disk space by removing intermediate novelty level directories that are no longer needed. This example shows how to delete novelty level 5.

```bash
rm -rf output/novelty_level_5/
```

--------------------------------

### Generate Tasks (Basic)

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/README.md

Execute the basic task generation script. This will produce approximately 14,000 files in the ./output/ directory.

```bash
cd tasks/task_generator
python generate_tasks.py                # 350 variants per template
# Output: 14,000+ files in ./output/
```

--------------------------------

### Initialize GenerateLevels Instance

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Instantiate the GenerateLevels class to begin generating level variations. No parameters are required for initialization.

```python
from utils.generate_variations import GenerateLevels

level_gen = GenerateLevels()
```

--------------------------------

### Get my score

Source: https://github.com/phy-q/novphy/blob/main/README.md

Retrieves the best scores for all levels. This should be used carefully for training mode due to potentially large amounts of data. For winning states, use message ID 65.

```APIDOC
## Get my score

### Description
Retrieves the best scores for all levels. This should be used carefully for training mode due to potentially large amounts of data. For winning states, use message ID 65.

### Request
Message ID: 23
Format: [23]

### Return
Format: [number_of_levels][score_level_1]....[score_level_n]
- number_of_levels: 4 bytes
- score_level_x: 4 bytes for each level
```

--------------------------------

### Speed Up Generation with Fewer Variants

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Significantly reduce generation time by specifying a lower variant count for the generate_tasks.py script. This example uses 100 variants, which is faster than the default 350.

```bash
python generate_tasks.py 100
```

--------------------------------

### Module-Level Configuration Paths

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-variations.md

Configures the input and output folders for level generation, as well as the configuration file path. These variables should be set before calling the main generation function.

```python
level_input_folder = './input/'
level_output_folder = './output/'
config_file = 'template_constraints.xlsx'
```

--------------------------------

### Get Location in Unreachable Space

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Generates a location in an unreachable area, typically beyond a defined range. It samples Y from a reachable range and adjusts X based on Y to place targets far away.

```python
def get_location_in_unreachable_space(self, x_min_unreachable: float, x_max_unreachable: float, 
                                      y_min_reachable: float, y_max_reachable: float) -> tuple
```

--------------------------------

### Fully Zoom In

Source: https://github.com/phy-q/novphy/blob/main/README.md

Command to fully zoom in in the interface.

```APIDOC
## Fully Zoom In

### Description
Command to fully zoom in.

### Method
Not specified (assumed to be a command code).

### Endpoint
Not specified.

### Parameters
#### Path Parameters
None.

#### Query Parameters
None.

#### Request Body
None.

### Request Example
```json
[
  35
]
```

### Response
#### Success Response (OK/ERR)
- **Status** (string) - Indicates OK or ERR.

#### Response Example
```json
[
  1
]
```
```

--------------------------------

### Get Reachable Location (X, Y)

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Generates a random (X, Y) coordinate pair that is guaranteed to be reachable. It iteratively samples X and constrains Y based on the reachability line, retrying until a feasible location is found.

```python
def get_reachable_location_using_reachability_line(self, x_min: float, x_max: float, 
                                                    y_min: float, y_max: float) -> tuple:
```

```python
x, y = level_gen.get_reachable_location_using_reachability_line(
    x_min=-8, x_max=5, y_min=-2, y_max=4
)
print(f"Reachable location: ({x}, {y})")
```

--------------------------------

### Get Occupied X Spans Above Y Axis

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Retrieves horizontal intervals occupied by objects above a specified Y coordinate. It is used for sky placement and skips the sky platform at Y=10.3 to prevent false conflicts.

```python
def get_occupied_x_spans_above_y_axis(self, template_data: list, y_axis: float) -> list:
```

--------------------------------

### Instantiate GenerateLevels and Generate Positions

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Instantiate the GenerateLevels class and use its methods to find reachable locations within specified boundaries. This is useful for placing game elements.

```python
from utils.generate_variations import GenerateLevels

level_gen = GenerateLevels()

# Generate positions
x, y = level_gen.get_reachable_location_using_reachability_line(-8, 5, -2, 4)
```

--------------------------------

### Copy Normal Tasks to Game Engine

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Copies all generated tasks from the 'novelty_level_0' directory to a specified game engine path.

```bash
# Copy all novelty_level_0 (normal) tasks
cp -r output/novelty_level_0/ /path/to/game/levels/
```

--------------------------------

### Load Data from Multiple Paths

Source: https://github.com/phy-q/novphy/blob/main/analysis/finding_best_detection_approach.ipynb

Iterates through directories, loads data using the `make_data` function, and appends it to a list. This snippet demonstrates how to consolidate data from different sources.

```python
path1 = "../evaluationdata/detection_datasets/simple_moving_average/"
path2 = "../evaluationdata/detection_datasets/pre_assumed_moving_average/"

all_data = []
for file in os.listdir(path1):
    data = make_data(path1,file,method="sma")
    all_data.append(data)
    
for file in os.listdir(path2):
    data = make_data(path2,file,method="pma")
    all_data.append(data)
```

--------------------------------

### Get Reachable X Location

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Generates a random X coordinate within a specified range that is also reachable based on a precomputed reachability line for a given Y coordinate. It caps the maximum X value.

```python
def get_reachable_x_location_using_reachability_line(self, x_min: float, x_max: float, 
                                                      y_location: float) -> float:
```

```python
# Get X coordinate reachable at Y = 2.0
x = level_gen.get_reachable_x_location_using_reachability_line(-8, 5, 2.0)
```

--------------------------------

### Generate Default Tasks

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Runs the task generation script with default settings to create a large number of XML task files.

```bash
python generate_tasks.py
```

--------------------------------

### Train Agent within Template

Source: https://github.com/phy-q/novphy/blob/main/README.md

Execute the training script for within-template scenarios. Ensure execution permissions are granted.

```sh
./TrainLearningAgent.sh within_template
```

--------------------------------

### Get Location in Reachable Space

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-levels.md

Generates a random location within specified X and Y bounds, with Y constrained based on the X position relative to the midpoint. Use when Y needs to be limited in the lower half for X values less than or equal to the midpoint.

```python
def get_location_in_reachable_space(self, x_min: float, x_max: float, 
                                    y_min: float, y_max: float) -> tuple
```

--------------------------------

### Run Task Generation Module

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Execute the main task generation orchestrator. Defaults are used unless a variant count is provided.

```python
if __name__ == "__main__":
    generate_variations = GenerateVariations()
    generate_variations.main()
```

--------------------------------

### Run Task Generator

Source: https://github.com/phy-q/novphy/blob/main/README.md

Execute the task generator script by providing the number of tasks to generate as an argument. Ensure task templates are placed in the 'input' directory.

```bash
python generate_tasks.py <number of tasks to generate>
```

--------------------------------

### List Files in a Specific Novelty Level

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Lists the first 20 files within the 'Levels' subdirectory of a specific novelty level.

```bash
ls output/novelty_level_3/*/Levels/ | head -20
```

--------------------------------

### Navigate to Directory and Run Generation

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/README.md

Ensure you are in the correct directory before running the task generation script. This resolves FileNotFoundError.

```bash
cd /workspace/home/novphy/tasks/task_generator
python generate_tasks.py
```

--------------------------------

### Orchestrate Generation Pipeline

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-variations.md

The main entry point that orchestrates the complete generation pipeline. It scans input files, reads configurations, generates level variations, and writes them to organized output folders.

```python
def main(self) -> None
```

```python
if __name__ == "__main__":
    generator = GenerateVariations()
    generator.main()  # Generates all variations from ./input/*.xml
```

--------------------------------

### Do Screenshot

Source: https://github.com/phy-q/novphy/blob/main/README.md

Captures a screenshot of the current game view. This command only returns screenshots without symbolic representation.

```APIDOC
## Do Screenshot

### Description
Captures a screenshot of the current game view. This command only returns screenshots without symbolic representation.

### Request
Message ID: 11
Format: [11]

### Return
Format: [width][height][image bytes]
- width: 4 bytes
- height: 4 bytes
- image bytes: raw image data
```

--------------------------------

### Python Logic for Pairing Normal and Novel Levels

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Illustrates the core logic for combining normal and novel level paths into trial data. It randomly selects normal levels and appends a fixed count of novel levels.

```python
for split_config_file in splitted_config_files:
    if 'normal' in split_config_file:
        # 1. Read N normal level paths (random N between 1-40)
        trial_data = read_lines_from_file(splitted_config_file_path + split_config_file, count=random.randint(1, 40))
        
        # 2. Find corresponding novel file
        novel_file = split_config_file.replace('normal', 'novel')
        
        # 3. Read 40 novel level paths
        trial_data.extend(read_lines_from_file(splitted_config_file_path + novel_file, count=40))
        
        # 4. Store combined trial data
        combined_config_data.append(trial_data)
```

--------------------------------

### Instantiate a Block Object

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Create a new Block instance. Requires an identifier, type, material, and position. Scale defaults to 1.0.

```python
from utils.data_classes import Block
block = Block(1, 'RectBig', 'wood', 0.0, 1.5, 45.0)
```

--------------------------------

### Integration Test for Full Pipeline

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Command to run the full task generation pipeline and a check for the existence of an output file.

```bash
# Test full pipeline
python generate_tasks.py 1
# Check: output/novelty_level_0/type01/Levels/00001_type01_0_0_1.xml exists
```

--------------------------------

### Archive Old Generations

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Move the entire output directory to a backup location with a timestamped name to archive old generations and free up space.

```bash
mv output/ output_backup_$(date +%Y%m%d)/
```

--------------------------------

### Instantiate a Pig Object

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Create a new Pig instance. Requires an identifier, type, and position. Rotation defaults to 0.0.

```python
from utils.data_classes import Pig
pig = Pig(1, 'BasicSmall', -5.0, 2.0, 0.0)
```

--------------------------------

### Verify Generated File Size

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Check the size of generated XML files to ensure they are not too small or empty. This command lists files in the output directory and shows the first few.

```bash
ls -lah output/novelty_level_0/type01/Levels/ | head
```

--------------------------------

### Train and Test OpenAI Stable Baselines Agent within Template

Source: https://github.com/phy-q/novphy/blob/main/README.md

Execute the script to train and test OpenAI Stable Baselines agents for within-template scenarios.

```sh
./TrainAndTestOpenAIStableBaselines.sh within_template
```

--------------------------------

### Directory Structure for Output

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/overview.md

Illustrates the hierarchical organization of generated level files within the output directory, categorized by novelty and type.

```bash
output/
├── novelty_level_0/
│   ├── type01/
│   │   └── Levels/
│   │       ├── 00001_type01_0_0_1.xml
│   │       ├── 00002_type01_0_0_1.xml
│   │       └── ...
│   └── type02/
└── novelty_level_1/
    └── ...

```

--------------------------------

### Check Output Structure

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Verify the directory structure of the generated output. This helps ensure files are created in the expected locations.

```bash
find output/novelty_level_0 -type d
```

--------------------------------

### Data Preparation for Scenario-Specific Analysis

Source: https://github.com/phy-q/novphy/blob/main/analysis/detection_plots.ipynb

Prepares data for plotting scenario-specific CDT and detection delay, including calculating mean, standard deviation, and standard error.

```python
df_per_scenario_mean = all_detection_data.groupby(['Scenario Index','Scenario','Name']).mean()[['cdt','detection_delay']].reset_index()
df_per_scenario_mean['key'] = df_per_scenario_mean['Scenario']+"_"+df_per_scenario_mean['Name']
df_per_scenario_std = all_detection_data.groupby(['Scenario Index','Scenario','Name']).std()[['cdt','detection_delay']].reset_index()
df_per_scenario_std['key'] = df_per_scenario_std['Scenario']+"_"+df_per_scenario_std['Name']
df_per_scenario_std.rename(columns = {'cdt':'cdt_std', 'detection_delay':'detection_delay_std'}, inplace = True)

df_per_scenario = pd.merge(df_per_scenario_mean, df_per_scenario_std[['key','cdt_std','detection_delay_std']],
                            how='inner', on = 'key')
df_per_scenario['cdt_se'] = df_per_scenario['cdt_std']/np.sqrt(8)
df_per_scenario['dd_se'] = df_per_scenario['detection_delay_std']/np.sqrt(8)
```

--------------------------------

### Standard Generation Import

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/modules.md

Imports necessary classes and constants from the project's utility modules for task generation.

```python
from generate_tasks import GenerateVariations
from utils.data_classes import Block, Pig, Tnt, Bird
from utils.constants import blocks, GROUND_LEVEL
from utils.generate_variations import GenerateLevels
```

--------------------------------

### Utilities Module Dependencies

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/overview.md

Details the Python modules within the utilities directory, including their dependencies and input/output.

```text
utils/generate_configs.py
├── No internal dependencies
└── Input: Split config files
    Output: XML + CSV

utils/filter_levels.py
└── No dependencies (file I/O only)

```

--------------------------------

### Verify Generated CSV Output

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/api-reference/generate-configs.md

Check the contents of the generated CSV file, which lists trial configurations.

```bash
cat ./config_files/generated_config_info.csv
```

--------------------------------

### Compress Output Directory

Source: https://github.com/phy-q/novphy/blob/main/_autodocs/quickstart.md

Reduce disk space usage by compressing the entire output directory into a tar.gz archive. This command demonstrates compressing the 'output/' folder.

```bash
tar -czf output.tar.gz output/
```