### Serve API Command Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/endpoints.md

Example command to start the API server, specifying host and port configurations.

```bash
serve-api --host 0.0.0.0 --port 5000
```

--------------------------------

### Full Training Setup

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/modules.md

Configures and starts a full training pipeline, including environment loading, simulation setup, PPO agent creation, and the training loop. This setup requires detailed configuration for environment contracts, rewards, and agent parameters.

```python
import asyncio
from pvp_ml.env.pvp_env import PvpEnv
from pvp_ml.env.simulation import Simulation
from pvp_ml.ppo.ppo import PPO, PolicyParams, Meta
from pvp_ml.util.contract_loader import load_environment_contract
from pvp_ml.util.schedule import LinearSchedule, ConstantSchedule
from pvp_ml.util.remote_processor.remote_processor import create_remote_processor

# Load environment contract
env_meta = load_environment_contract("NhEnv")

# Start simulation
with Simulation(game_port=43595, remote_env_port=7070) as sim:
    sim.wait_until_loaded()
    
    # Create environment
    env = PvpEnv(
        env_name="NhEnv",
        remote_environment_port=sim.remote_env_port,
        default_reward=ConstantSchedule(0.0),
        win_reward=ConstantSchedule(1.0),
        damage_dealt_reward_scale=LinearSchedule(0.0, 0.1, 1000000)
    )
    
    # Create PPO agent
    policy_params = PolicyParams(
        max_sequence_length=4,
        actor_input_size=len(env_meta.observations),
        critic_input_size=len(env_meta.observations),
        action_head_sizes=[len(head.actions) for head in env_meta.actions]
    )
    
    ppo = PPO.create(policy_params, Meta(...))
    
    # Training loop
    async def train():
        obs, _ = await env.reset_async()
        while True:
            action, *_ = ppo.predict(obs, env.get_action_masks(), deterministic=False)
            obs, reward, done, truncated, info = await env.step_async(action)
            if done or truncated:
                obs, _ = await env.reset_async()
```

--------------------------------

### Clone the Repository

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/README.md

Clone the project repository to get started. Follow the README in each subproject for individual setup.

```bash
git clone https://github.com/Naton1/osrs-pvp-reinforcement-learning
```

--------------------------------

### Start Async Server with Client Handler

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Example of setting up an asyncio server that uses `handle_client` to process incoming newline-delimited JSON requests.

```python
# Server setup creates this as a callback for asyncio
# Client sends newline-delimited JSON requests
async with await asyncio.start_server(
    lambda r, w: handle_client(r, w, remote_processor),
    host, port
) as server:
    await server.serve_forever()
```

--------------------------------

### Start API Server with Specific Configuration

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Example of calling `run_api` with specific parameters for host, port, pool size, worker type, and device.

```python
await run_api(
    host="127.0.0.1",
    port=9999,
    remote_processor_pool_size=4,
    remote_processor_type="thread",
    remote_processor_kwargs={},
    device="cuda"
)
```

--------------------------------

### main

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

CLI entry point. Parses command-line arguments and starts the API server.

```APIDOC
## main

### Description
CLI entry point. Parses command-line arguments and starts the API server.

### Signature
```python
def main(argv: list[str]) -> None
```
```

--------------------------------

### Start Tensorboard

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/pvp-ml/README.md

Manually starts Tensorboard for visualizing training metrics. It typically launches automatically with training jobs.

```bash
train tensorboard
```

--------------------------------

### API Server Setup

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/modules.md

Initializes the remote processor and starts the API server. This is used to serve predictions from a trained model. Configuration includes host, port, and details for the remote processor.

```python
from pvp_ml.api import run_api
from pvp_ml.util.remote_processor.remote_processor import create_remote_processor
import asyncio

# Create processor
processor = await create_remote_processor(
    pool_size=4,
    processor_type="thread",
    device="cuda"
)

# Run API server
await run_api(
    host="0.0.0.0",
    port=9999,
    remote_processor_pool_size=4,
    remote_processor_type="thread",
    remote_processor_kwargs={},
    device="cuda"
)
```

--------------------------------

### Start Training Job

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/pvp-ml/README.md

Initiates a training job using a specified preset configuration. Replace <name-your-experiment> with a unique name for your experiment.

```bash
train --preset PastSelfPlay --name <name-your-experiment>
```

--------------------------------

### Training Configuration Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/README.md

Example YAML configuration for training reinforcement learning models. Adjust parameters like learning rate, batch sizes, and network architecture for optimal performance.

```yaml
env-name: NhEnv
ppo:
  learning-rate: 3e-4
  num-steps: 2048
  num-mini-batches: 32
policy:
  actor-hidden: [128, 128, 128]
  critic-hidden: [64, 64]
reward:
  default: 0.0
  win: 1.0
  lose: -1.0
num-steps: 10000000
```

--------------------------------

### Training Setup with Multiple Environments

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/simulation.md

Sets up the simulation for training with synchronization enabled and creates multiple PvpEnv instances connected to the same simulation.

```python
from pvp_ml.env.simulation import Simulation
from pvp_ml.env.pvp_env import PvpEnv

# Start simulation with training synchronization
with Simulation(
    game_port=43595,
    remote_env_port=7070,
    sync_training=True
) as sim:
    sim.wait_until_loaded()
    
    # Create multiple environments connected to same simulation
    envs = [
        PvpEnv(
            env_name="NhEnv",
            env_id=f"train_env_{i}",
            remote_environment_port=sim.remote_env_port
        )
        for i in range(4)
    ]
    
    # Use environments in training loop
```

--------------------------------

### Start ElvargClient

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/simulation-rsps/README.md

Change to the ElvargClient directory within the cloned upstream repository and start the client using Gradle. Requires Java 11.

```bash
cd elvarg-rsps/ElvargClient
./gradlew run
```

--------------------------------

### API Client Example (Python)

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/00-START-HERE.txt

Example of a Python client connecting to the API server to get model predictions. It sends an observation and action masks, then prints the predicted action. Ensure the server is running and accessible.

```python
import socket, json
sock = socket.socket()
sock.connect(("127.0.0.1", 9999))
request = {
  "model": "GeneralizedNh",
  "obs": [[1.0, 2.0, 3.0, ...]],
  "actionMasks": [[true, true, false, ...], ...]
}
sock.send((json.dumps(request) + "\n").encode())
response = json.loads(sock.recv(4096).decode())
print(response["action"])  # [0, 1, 2, ...]
```

--------------------------------

### Start Training Job with Preset and Name

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

Initiates a training job using a specified configuration preset and assigns a custom experiment name.

```bash
train --preset PastSelfPlay --name my-experiment --distribute 8
```

--------------------------------

### ExpressionSchedule Usage Examples

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Demonstrates ExpressionSchedule for linear reward growth and a conditional penalty that starts after a certain time.

```python
# Custom reward that increases with time
reward_schedule = ExpressionSchedule("t / 1000000")  # Linear growth: 0 at t=0, 1 at t=1M

# Conditional: switch behavior at milestone
penalty_schedule = ExpressionSchedule("0 if t < 50000 else min(0.1, (t - 50000) / 100000)")
```

--------------------------------

### Start and Connect to Remote Environment

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/errors.md

Ensures the simulation is running and waits for the server to be ready before initializing the PvpEnv. Use this to recover from connection timeouts.

```python
# Ensure simulation is running
from pvp_ml.env.simulation import Simulation

with Simulation(game_port=43595, remote_env_port=7070) as sim:
    sim.wait_until_loaded()  # Waits for server to be ready
    
    # Now create environment
    env = PvpEnv(
        env_name="NhEnv",
        remote_environment_host="localhost",
        remote_environment_port=7070
    )
```

--------------------------------

### Create Remote Processor and Preload Models

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Example of creating a remote processor and then preloading its models to ensure they are cached.

```python
async with await create_remote_processor(pool_size=4) as processor:
    await preload_models(processor)
    # Models are now cached in all 4 workers
```

--------------------------------

### Get Action Space Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Retrieves the Gymnasium action space for the environment. This is useful for understanding the structure of possible actions.

```python
env_meta = load_environment_contract("NhEnv")
action_space = env_meta.get_action_space()
# MultiDiscrete([4, 3, 3, 3, ...]) if NH has 11 action heads
```

--------------------------------

### Serve Models via API

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/00-START-HERE.txt

Command to start the API server for model serving. Specify host and port for the server to listen on.

```bash
serve-api --host 0.0.0.0 --port 9999
```

--------------------------------

### LinearSchedule Usage Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Example of setting up a LinearSchedule for a learning rate that decreases over 1 million steps.

```python
learning_rate = LinearSchedule(
    initial_value=1e-3,
    final_value=1e-5,
    change_over_time_steps=1000000
)
# At t=0: 1e-3
# At t=500000: 5e-6
# At t=1000000+: 1e-5
```

--------------------------------

### run_api

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Main async function that starts the API server.

```APIDOC
## run_api

### Description
Main async function that starts the API server.

### Signature
```python
async def run_api(
    host: str,
    port: int,
    remote_processor_pool_size: int,
    remote_processor_type: str,
    remote_processor_kwargs: dict[str, Any],
    device: str
) -> None
```

### Parameters
- **host** (str) - Bind address (default: 127.0.0.1)
- **port** (int) - Bind port (default: 9999)
- **remote_processor_pool_size** (int) - Number of worker threads/processes for inference
- **remote_processor_type** (str) - Worker type: "thread", "process", or "ray"
- **remote_processor_kwargs** (dict[str, Any]) - Additional kwargs for remote processor
- **device** (str) - Torch device: "cpu" or "cuda"

### Example
```python
await run_api(
    host="127.0.0.1",
    port=9999,
    remote_processor_pool_size=4,
    remote_processor_type="thread",
    remote_processor_kwargs={},
    device="cuda"
)
```
```

--------------------------------

### JumpSchedule Usage Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Example demonstrating JumpSchedule to manage reward levels across different stages of training, including constant and linearly increasing rewards.

```python
reward_schedule = JumpSchedule({
    "0": ConstantSchedule(0.0),        # Stage 1: t < 10000, reward = 0
    "10000": ConstantSchedule(1.0),    # Stage 2: 10000 <= t, reward = 1
    "50000": LinearSchedule(1.0, 2.0, 100000)  # Stage 3: 50000 <= t, linearly grow
})
```

--------------------------------

### Plugin Configuration Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/endpoints.md

Configure the system to use a scripted plugin instead of a trained model by specifying the plugin name in the configuration. Plugins only support the 'action' return field.

```json
{
  "model": "baseline",
  ...
}
```

--------------------------------

### Launch API Server

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Command-line tool for launching the API server. Use this to start the prediction service.

```bash
serve-api [OPTIONS]
```

--------------------------------

### Minimal Inference Setup

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/modules.md

Loads a pre-trained PPO model and sets up inference workers for making predictions. Requires the PPO model path and observation/action masks for prediction.

```python
from pvp_ml.ppo.ppo import PPO
from pvp_ml.util.remote_processor.remote_processor import create_remote_processor

# Load model
ppo = PPO.load("models/GeneralizedNh.zip")

# Create inference workers
processor = await create_remote_processor(pool_size=4, device="cuda")

# Get predictions
action, log_probs, *_ = ppo.predict(obs, action_masks)
```

--------------------------------

### Run API Server

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Main async function to start the API server with specified host, port, and remote processor configuration.

```python
async def run_api(
    host: str,
    port: int,
    remote_processor_pool_size: int,
    remote_processor_type: str,
    remote_processor_kwargs: dict[str, Any],
    device: str
) -> None:
    pass
```

--------------------------------

### FineTuning Configuration Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

This configuration is for fine-tuning a pretrained model against baseline or human-like opponents. It specifies an initial model to load and uses a lower learning rate suitable for fine-tuning.

```yaml
import:
  - Base

policy:
  # Load from existing checkpoint
  initial-model: models/GeneralizedNh.zip

ppo:
  learning-rate: 1e-4  # Lower LR for fine-tuning

target: baseline

num-steps: 500000

```

--------------------------------

### Optionally Run Simulation Server for Evaluation

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/evaluate.md

Context manager that conditionally starts a simulation server for evaluation. If not starting a simulation, it uses provided host and port details.

```python
# Start a simulation server for evaluation
with _maybe_run_simulation(
    run_simulation=True,
    run_simulation_game_port=43595,
    default_host="localhost",
    default_env_port=7070
) as (host, port):
    env = PvpEnv(
        env_name="NhEnv",
        remote_environment_host=host,
        remote_environment_port=port
    )
```

--------------------------------

### Start API Server for Model Serving

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/README.md

Launches the API server to serve trained models. It binds to all network interfaces on port 9999.

```bash
# Start server
serve-api --host 0.0.0.0 --port 9999
```

--------------------------------

### LogSchedule Usage Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Example of a LogSchedule for an epsilon decay, which decays exploration faster initially. The base can be customized.

```python
epsilon = LogSchedule(
    initial_value=0.1,
    final_value=0.01,
    change_over_time_steps=500000,
    base=2.0
)
# Decays exploration faster early, slower late
```

--------------------------------

### Start Training an RL Agent

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/README.md

Use this command to initiate the training process for a reinforcement learning agent. Ensure you are in the 'pvp-ml' directory and have the correct conda environment activated.

```bash
cd pvp-ml
conda activate ./env
train --preset PastSelfPlay --name my-experiment
```

--------------------------------

### ConstantSchedule Usage Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Demonstrates how to use ConstantSchedule to maintain a fixed reward value across different timesteps.

```python
reward_schedule = ConstantSchedule(1.0)
assert reward_schedule.value(0) == 1.0
assert reward_schedule.value(1000000) == 1.0
```

--------------------------------

### PastSelfPlay Configuration Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

This configuration is used for self-play training with a pool of past model checkpoints. It specifies the strategy as 'past' and lists the paths to the models to be used.

```yaml
import:
  - Base

ppo:
  learning-rate: 3e-4
  num-mini-batches: 32

self-play:
  strategy: past
  past-models:
    - models/GeneralizedNh_v1.zip
    - models/GeneralizedNh_v2.zip

num-steps: 10000000

```

--------------------------------

### Evaluation Setup with Disabled Synchronization

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/simulation.md

Configures the simulation for faster evaluation by disabling training synchronization. Includes a basic evaluation loop.

```python
# Start simulation with faster evaluation settings
with Simulation(
    game_port=43595,
    remote_env_port=7070,
    sync_training=False  # Disable synchronization for speed
) as sim:
    sim.wait_until_loaded()
    
    env = PvpEnv(
        env_name="NhEnv",
        remote_environment_port=sim.remote_env_port
    )
    
    # Run evaluation
    for _ in range(100):
        obs, _ = await env.reset_async()
        while not env.is_closed():
            action = predictor(obs, env.get_action_masks())
            obs, reward, done, truncated, info = await env.step_async(action)
            if done or truncated:
                break
```

--------------------------------

### Serve Models via API

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/pvp-ml/README.md

Starts a socket-based API to serve model predictions. By default, it listens on 127.0.0.1. Use --host to configure.

```bash
serve-api
```

--------------------------------

### Simulation Lifecycle Management

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/simulation.md

Manages the simulation lifecycle using a context manager. The server is automatically started and stopped.

```python
# Manage simulation lifecycle
with Simulation(game_port=43595, remote_env_port=7070) as sim:
    sim.wait_until_loaded()  # Block until server is ready
    # Create environment connected to this simulation
    env = PvpEnv(
        env_name="NhEnv",
        remote_environment_port=sim.remote_env_port
    )
```

--------------------------------

### Configure API Server with Process Pool and GPU Allocation

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

Starts the API server using a process-based remote processor pool and allocates fractional GPUs per worker.

```bash
serve-api \
  --host 0.0.0.0 \
  --port 9999 \
  --remote-processor-pool-size 4 \
  --remote-processor-type process \
  --device cuda \
  --remote-processor-kwargs '{"num_gpus_per_worker": 0.25}'
```

--------------------------------

### Distributed Training

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/pvp-ml/README.md

Starts a distributed training job using Ray. Specify the number of parallel rollouts or omit to use all available CPU cores.

```bash
train --preset <preset> --distribute <parallel-rollout-count>
```

--------------------------------

### Serve API with Remote Processor Pool

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/INDEX.txt

Start the API server for model serving. Specifies the host, port, and the size of the remote processor pool for inference.

```bash
serve-api --host 0.0.0.0 --port 9999 --remote-processor-pool-size 4
```

--------------------------------

### Configure PvpEnv with Scenario Rewards

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

Instantiate the PvpEnv with detailed reward configurations for various in-game scenarios, including outcomes, health management, damage, and prayer usage. This setup allows for fine-grained control over agent motivation.

```python
from pvp_ml.util.schedule import LinearSchedule, ConstantSchedule
from pvp_ml.env.pvp_env import PvpEnv

env = PvpEnv(
    env_name="NhEnv",
    env_id="train_0",
    
    # Outcome rewards
    default_reward=ConstantSchedule(0.0),
    win_reward=ConstantSchedule(1.0),
    lose_reward=ConstantSchedule(-1.0),
    tie_reward=ConstantSchedule(-0.2),
    
    # Health management rewards
    penalize_food_on_death=True,
    reward_target_food_on_death=True,
    penalize_wasted_food=True,
    
    # Damage rewards (annealed during training)
    reward_on_damage_generated=True,
    damage_dealt_reward_scale=LinearSchedule(0.0, 0.1, 1000000),
    damage_received_reward_scale=ConstantSchedule(0.0),
    
    # Prayer rewards
    protected_correct_prayer_reward=ConstantSchedule(0.05),
    attacked_wrong_prayer_reward=ConstantSchedule(0.02),
    
    # Connection settings
    remote_environment_host="localhost",
    remote_environment_port=7070,
    
    # Frame stacking
    stack_frames=4,
)
```

--------------------------------

### Create Conda Environment

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/pvp-ml/README.md

Creates a new conda environment from a specified YAML file. Ensure conda is installed and available in your PATH.

```bash
conda env create -p ./env -f environment.yml
```

--------------------------------

### reset_async

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/pvp_env.md

Resets the environment to its initial state, returning the starting observation and associated information. Supports optional parameters for configuring the reset, such as trained steps and agent.

```APIDOC
## reset_async

### Description
Reset the environment and return initial observation.

### Method
`async def reset_async(*, seed: int | None = None, options: ResetOptions | dict[str, Any] | None = None)`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **seed** (int | None) - Optional - Not supported (always None)
- **options** (ResetOptions | dict | None) - Optional - trained_steps, trained_rollouts, agent

### Response
#### Success Response
- **observation** (NDArray[np.float32]) - Initial observation
- **info** (dict[str, Any]) - Metadata

### Request Example
```python
obs, info = await env.reset_async(
    options={
        "trained_steps": 1000,
        "trained_rollouts": 10,
        "agent": "main"
    }
)
```
```

--------------------------------

### Wait Until Simulation Loaded

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/simulation.md

Blocks execution until the game server is ready to accept connections, with a configurable timeout. Raises TimeoutError if the server does not start within the specified duration.

```python
def wait_until_loaded(
    self,
    timeout_seconds: float = 60.0
) -> None
```

```python
sim = Simulation(game_port=43595)
try:
    sim.wait_until_loaded(timeout_seconds=120)
    # Server is now ready
except TimeoutError:
    logger.error("Server failed to start")
```

--------------------------------

### CLI Entry Point

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Parses command-line arguments and initiates the API server startup.

```python
def main(argv: list[str]) -> None:
    pass
```

--------------------------------

### Client Request to Model Serving API

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/README.md

Example of a Python client making a request to the model serving API to get a prediction. It sends observation and action mask data and prints the predicted action.

```python
import socket, json
sock = socket.socket()
sock.connect(("127.0.0.1", 9999))
request = {
    "model": "GeneralizedNh",
    "obs": [[obs_values...]],
    "actionMasks": [[mask_values...]]
}
sock.send((json.dumps(request) + "\n").encode())
response = json.loads(sock.recv(4096).decode())
print(response["action"])
```

--------------------------------

### CallableSchedule Usage Example

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Example of using CallableSchedule to implement a sinusoidal learning rate schedule using a lambda function.

```python
import math
# Sinusoidal learning rate
lr_schedule = CallableSchedule(
    lambda t: 1e-3 * (1 + math.cos(math.pi * t / 1000000)) / 2
)
```

--------------------------------

### Error Response Example: Unknown Model

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/endpoints.md

Example of an error response when an unknown model name is provided in the request. The server will close the connection.

```text
ValueError: Unknown model: {model_name}
```

--------------------------------

### _maybe_run_simulation

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/evaluate.md

A context manager that optionally starts a simulation server for evaluation purposes. It can either start a new simulation or connect to an existing one.

```APIDOC
## _maybe_run_simulation

### Description
Context manager that optionally starts a simulation server for evaluation.

### Signature
```python
@contextmanager
def _maybe_run_simulation(
    run_simulation: bool,
    run_simulation_game_port: int,
    default_host: str,
    default_env_port: int
) -> Iterator[tuple[str, int]]
```

### Parameters
#### Path Parameters
* None

#### Query Parameters
* None

#### Request Body
* None

### Parameters
- **run_simulation** (bool) - True = start simulation, False = use provided host/port
- **run_simulation_game_port** (int) - Port for game server (if starting simulation)
- **default_host** (str) - Host to connect to (if not starting simulation)
- **default_env_port** (int) - Remote env port to connect to (if not starting simulation)

### Yields
(host, port) tuple for connection

### Request Example
```python
# Start a simulation server for evaluation
with _maybe_run_simulation(
    run_simulation=True,
    run_simulation_game_port=43595,
    default_host="localhost",
    default_env_port=7070
) as (host, port):
    env = PvpEnv(
        env_name="NhEnv",
        remote_environment_host=host,
        remote_environment_port=port
    )
```

### Response
#### Success Response (200)
* None (This is a context manager, it yields a tuple)

#### Response Example
* None
```

--------------------------------

### Load and Use Environment Contract (Python)

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Demonstrates loading an environment contract using `load_environment_contract` and initializing a `PvpEnv` with the loaded metadata. Shows how to access action space and partially observable indices.

```python
from pvp_ml.util.contract_loader import load_environment_contract

# Load contract
meta = load_environment_contract("NhEnv")

# Create environment using contract data
env = PvpEnv(env_name="NhEnv")

# Access action space
action_space = meta.get_action_space()
num_action_heads = len(action_space.nvec)

# Check what observations are critic-visible
partial_indices = meta.get_partially_observable_indices()
```

--------------------------------

### RemoteProcessor Abstract Base Class

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/remote_processor.md

Abstract base class defining the interface for all remote processors. It includes abstract methods for prediction, closing resources, getting pool size, and getting the device.

```python
class RemoteProcessor(ABC):
    @abc.abstractmethod
    async def predict(
        self,
        process_id: int,
        model_path: str,
        observation: th.Tensor | None = None,
        action_masks: th.Tensor | None = None,
        deterministic: bool | th.Tensor = False,
        return_device: str | None = None,
        return_actions: bool = True,
        return_log_probs: bool = False,
        return_entropy: bool = False,
        return_values: bool = False,
        return_probs: bool = False,
        extensions: list[str] = []
    ) -> tuple[
        th.Tensor | None,
        th.Tensor | None,
        th.Tensor | None,
        th.Tensor | None,
        th.Tensor | None,
        list[Any]
    ]

    @abc.abstractmethod
    async def close(self) -> None:
        pass

    @abc.abstractmethod
    def get_pool_size(self) -> int:
        pass

    @abc.abstractmethod
    def get_device(self) -> str:
        pass

    async def __aenter__(self) -> "RemoteProcessor":
        return self

    async def __aexit__(self, *args: Any) -> None:
        await self.close()
```

--------------------------------

### Simulation Context Manager Support

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/simulation.md

Demonstrates the use of the Simulation class as an asynchronous context manager for automatic server startup and cleanup.

```python
async with Simulation(...) as sim:
    # Server is running
    pass
# Server is stopped and cleaned up
```

--------------------------------

### main_entry_point

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/api.md

Console script entry point (called as `serve-api` command).

```APIDOC
## main_entry_point

### Description
Console script entry point (called as `serve-api` command).

### Signature
```python
def main_entry_point() -> None
```
```

--------------------------------

### Instantiate PvpEnv with LinearSchedule

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/schedule.md

Demonstrates how to initialize a PvpEnv with a LinearSchedule for the default reward. Ensure LinearSchedule and ConstantSchedule are imported.

```python
from pvp_ml.env.pvp_env import PvpEnv
from pvp_ml.util.schedule import LinearSchedule

env = PvpEnv(
    env_name="NhEnv",
    default_reward=LinearSchedule(0.0, 1.0, 1000000),
    win_reward=ConstantSchedule(10.0)
)
```

--------------------------------

### Run ElvargServer

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/simulation-rsps/README.md

Navigate to the ElvargServer directory and launch the server using Gradle. Requires Java 17.

```bash
./gradlew run
```

--------------------------------

### Get Remote Environment Port

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/simulation.md

Retrieves the port number used for environment API connections from a Simulation instance.

```python
@property
def remote_env_port(self) -> int
```

--------------------------------

### Get Available Environment Types

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Retrieves a list of all available environment names, including built-in and custom registered environments.

```python
from contract_loader import get_env_types

available = get_env_types()
# ["NhEnv", "DharokEnv", "CustomNH"]
```

--------------------------------

### Configure Training and Evaluation Simulations

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

Instantiate Simulation objects for training with synchronization enabled or for faster evaluation without synchronization. Ensure correct ports are specified.

```python
from pvp_ml.env.simulation import Simulation

# Training simulation with synchronization
sim = Simulation(
    game_port=43595,
    remote_env_port=7070,
    sync_training=True
)

# Evaluation simulation (faster, no sync)
eval_sim = Simulation(
    game_port=43596,
    remote_env_port=7071,
    sync_training=False
)
```

--------------------------------

### Train a Model

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/00-START-HERE.txt

Command to initiate model training. Use --preset and --name to specify training configuration and experiment name.

```bash
train --preset PastSelfPlay --name my-exp
```

--------------------------------

### Initialize PvpEnv Environment

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/INDEX.txt

Instantiate the PvpEnv environment for reinforcement learning. This is typically used for training RL agents.

```python
from pvp_ml.env.pvp_env import PvpEnv
env = PvpEnv(env_name="NhEnv", remote_environment_port=7070)
```

--------------------------------

### Project File Structure Overview

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

This outlines the standard directory layout for the OSRS PvP Reinforcement Learning project, including configuration files, models, and experiment outputs.

```text
pvp-ml/
├── config/
│   ├── Base.yml
│   ├── PastSelfPlay.yml
│   ├── FineTuning.yml
│   └── CustomEnv.yml
├── models/
│   ├── GeneralizedNh.zip
│   ├── FineTunedNh.zip
│   └── [other model checkpoints]
├── experiments/
│   ├── PastSelfPlay_001/
│   │   ├── checkpoints/
│   │   ├── logs/
│   │   └── config.yml
│   └── [other experiments]
└── logs/
    └── [training logs]
```

--------------------------------

### Get Non-Constant Indices Method Signature

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Signature for retrieving indices of observations that vary. These are typically used for normalization in RL agents.

```python
def get_non_constant_indices(self) -> list[int]:
    pass
```

--------------------------------

### Full Training Configuration Structure

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

This is the complete structure for a training configuration file. It includes settings for the environment, PPO algorithm, neural network, reward function, opponent, self-play, training duration, and logging.

```yaml
# Environment
env-name: NhEnv

# PPO Algorithm
ppo:
  learning-rate: 3e-4
  gamma: 0.99
  gae-lambda: 0.95
  num-steps: 2048
  num-mini-batches: 32
  num-epochs: 3
  clip-ratio: 0.2
  entropy-weight: 0.001
  value-loss-weight: 0.5

# Neural Network Architecture
policy:
  feature-extractor-hidden: [128, 128]
  actor-hidden: [128, 128, 128]
  critic-hidden: [64, 64]
  share-feature-extractor: false
  autoregressive-actions: true

# Reward Function
reward:
  default: 0.0
  win: 1.0
  lose: -1.0
  tie: -0.2
  damage-dealt-scale: 0.0
  damage-received-scale: 0.0

# Opponent / Target
target: baseline  # Or model name, plugin name

# Self-Play
self-play:
  strategy: latest  # Or: past, exploiter, none
  past-models: []

# Training Duration
num-steps: 10000000
num-rollouts: 1000

# Logging
log-frequency: 100
checkpoint-frequency: 1000

```

--------------------------------

### Preloading Models

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/remote_processor.md

Load a model into all workers' memory before serving requests to reduce latency. This example also utilizes an async context manager.

```python
# Load model into all workers' memory before serving
async with create_remote_processor(pool_size=4) as processor:
    for worker_id in range(processor.get_pool_size()):
        await processor.predict(process_id=worker_id, model_path="models/agent.zip")
    # Model is now cached in all 4 workers
```

--------------------------------

### Create and Use PvpEnv for Training

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/README.md

Instantiate the PvpEnv with specific reward configurations and use it within a training loop for asynchronous environment interaction.

```python
from pvp_ml.env.pvp_env import PvpEnv
from pvp_ml.util.schedule import LinearSchedule

# Create environment with reward configuration
env = PvpEnv(
    env_name="NhEnv",
    default_reward=0.0,
    win_reward=1.0,
    lose_reward=-1.0,
    damage_dealt_reward_scale=LinearSchedule(0.0, 0.1, 1000000),
    remote_environment_port=7070
)

# In training loop:
obs, info = await env.reset_async()
while not env.is_closed():
    action = policy.predict(obs, env.get_action_masks())
    obs, reward, done, truncated, info = await env.step_async(action)
```

--------------------------------

### Get Partially Observable Indices Method Signature

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Signature for retrieving indices of observations visible only to the actor. This is crucial for partially observable environments.

```python
def get_partially_observable_indices(self) -> list[int]:
    pass
```

--------------------------------

### Get Observation Space Method Signature

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Signature for retrieving the Gymnasium observation space. This defines the shape and type of observations the environment provides.

```python
def get_observation_space(self) -> spaces.Box:
    pass
```

--------------------------------

### Project Structure Overview

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/README.md

This markdown snippet illustrates the directory structure of the OSRS PvP Reinforcement Learning project. It helps in understanding the organization of different modules and documentation files.

```markdown
```
├── README.md (this file)
├── api-reference/
│   ├── api.md
│   ├── pvp_env.md
│   ├── ppo.md
│   ├── schedule.md
│   ├── remote_processor.md
│   ├── contract_loader.md
│   ├── evaluate.md
│   └── simulation.md
├── types.md
├── configuration.md
├── endpoints.md
└── errors.md
```
```

--------------------------------

### PPO.create

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/ppo.md

Creates a new PPO model instance with specified policy parameters and metadata. This is used for initializing a model from scratch.

```APIDOC
## PPO.create

### Description
Creates a new PPO model instance with specified policy parameters and metadata.

### Method
Class Method

### Parameters
#### Path Parameters
- **policy_params** (dict) - Required - Parameters defining the policy network architecture and initial weights.
- **meta** (Meta object) - Required - Metadata object containing training configurations and other relevant information.

### Response
#### Success Response
- **PPO instance** - A newly created PPO model object.

### Request Example
```python
# Assuming Meta class and policy_params are defined elsewhere
new_model = PPO.create(policy_params, meta_data)
```

### Response Example
```python
# new_model is now an instance of the PPO class
```
```

--------------------------------

### PvpEnv Class Initialization

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/pvp_env.md

Initializes the PvpEnv environment with various configurable parameters to control the simulation's behavior, rewards, and agent interactions.

```APIDOC
## PvpEnv Class

### Description
Initializes the PvpEnv environment, which provides a Gymnasium-compatible interface for PvP fight simulations. It connects to a remote RSPS server via TCP socket.

### Method
__init__

### Parameters
#### Initialization Parameters:
- **env_name** (str) - Required - The name of the environment.
- **env_id** (str) - Optional - The ID of the environment, defaults to "PvpEnv".
- **default_reward** (Schedule[float]) - Optional - The default reward schedule, defaults to ConstantSchedule(0.0).
- **win_reward** (Schedule[float]) - Optional - The reward for winning, defaults to ConstantSchedule(1.0).
- **lose_reward** (Schedule[float]) - Optional - The reward for losing, defaults to ConstantSchedule(-1.0).
- **tie_reward** (Schedule[float]) - Optional - The reward for a tie, defaults to ConstantSchedule(-0.2).
- **safe_penalty** (Schedule[float]) - Optional - Penalty for being in a safe zone, defaults to ConstantSchedule(0.0).
- **death_match** (bool) - Optional - Whether the environment is a death match, defaults to True.
- **penalize_food_on_death** (bool) - Optional - Whether to penalize food on death, defaults to True.
- **reward_target_food_on_death** (bool) - Optional - Whether to reward for target's food on death, defaults to True.
- **penalize_wasted_food** (bool) - Optional - Whether to penalize wasted food, defaults to True.
- **reward_on_damage_generated** (bool) - Optional - Whether to reward for damage generated, defaults to True.
- **reward_heals** (bool) - Optional - Whether to reward for healing, defaults to True.
- **penalize_target_heals** (bool) - Optional - Whether to penalize target's heals, defaults to True.
- **target** (str) - Optional - The target type, defaults to "baseline".
- **desync_tolerance_seconds** (float) - Optional - Tolerance for desynchronization in seconds, defaults to 0.0.
- **truncate_on_desync** (bool) - Optional - Whether to truncate on desync, defaults to True.
- **damage_received_reward_scale** (Schedule[float]) - Optional - Scale for damage received reward, defaults to ConstantSchedule(0.0).
- **damage_dealt_reward_scale** (Schedule[float]) - Optional - Scale for damage dealt reward, defaults to ConstantSchedule(0.0).
- **stack_frames** (int | list[int]) - Optional - Number of frames to stack, defaults to 1.
- **target_frozen_tick_reward** (Schedule[float]) - Optional - Reward for target being frozen, defaults to ConstantSchedule(0.0).
- **player_frozen_tick_reward** (Schedule[float]) - Optional - Reward for player being frozen, defaults to ConstantSchedule(0.0).
- **protected_correct_prayer_reward** (Schedule[float]) - Optional - Reward for protecting correct prayer, defaults to ConstantSchedule(0.0).
- **protected_wrong_prayer_reward** (Schedule[float]) - Optional - Reward for protecting wrong prayer, defaults to ConstantSchedule(0.0).
- **attacked_correct_prayer_reward** (Schedule[float]) - Optional - Reward for being attacked with correct prayer, defaults to ConstantSchedule(0.0).
- **attacked_wrong_prayer_reward** (Schedule[float]) - Optional - Reward for being attacked with wrong prayer, defaults to ConstantSchedule(0.0).
- **protected_previous_correct_prayer_reward** (Schedule[float]) - Optional - Reward for protecting previous correct prayer, defaults to ConstantSchedule(0.0).
- **protected_previous_wrong_prayer_reward** (Schedule[float]) - Optional - Reward for protecting previous wrong prayer, defaults to ConstantSchedule(0.0).
- **attacked_previous_correct_prayer_reward** (Schedule[float]) - Optional - Reward for being attacked with previous correct prayer, defaults to ConstantSchedule(0.0).
- **attacked_previous_wrong_prayer_reward** (Schedule[float]) - Optional - Reward for being attacked with previous wrong prayer, defaults to ConstantSchedule(0.0).
- **attack_level_scale_reward** (Schedule[float]) - Optional - Reward scale based on attack level, defaults to ConstantSchedule(0.0).
- **strength_level_scale_reward** (Schedule[float]) - Optional - Reward scale based on strength level, defaults to ConstantSchedule(0.0).
- **defense_level_scale_reward** (Schedule[float]) - Optional - Reward scale based on defense level, defaults to ConstantSchedule(0.0).
- **ranged_level_scale_reward** (Schedule[float]) - Optional - Reward scale based on ranged level, defaults to ConstantSchedule(0.0).
- **magic_level_scale_reward** (Schedule[float]) - Optional - Reward scale based on magic level, defaults to ConstantSchedule(0.0).
- **reward_on_hit_with_boost_scale** (Schedule[float]) - Optional - Scale for reward on hit with boost, defaults to ConstantSchedule(0.0).
- **smite_damage_dealt_reward_multiplier** (Schedule[float]) - Optional - Multiplier for smite damage dealt reward, defaults to ConstantSchedule(0.0).
- **smite_damage_received_reward_multiplier** (Schedule[float]) - Optional - Multiplier for smite damage received reward, defaults to ConstantSchedule(0.0).
- **player_died_with_food_multiplier** (Schedule[float]) - Optional - Multiplier for player died with food, defaults to ConstantSchedule(1.0).
- **player_wasted_food_multiplier** (Schedule[float]) - Optional - Multiplier for player wasted food, defaults to ConstantSchedule(1.0).
- **custom_reward_fn** (Schedule[float]) - Optional - Custom reward function, defaults to ConstantSchedule(0.0).
- **no_prayer_tick_reward** (Schedule[float]) - Optional - Reward for no prayer tick, defaults to ConstantSchedule(0.0).
- **desync_tick_threshold** (int) - Optional - Threshold for desync ticks, defaults to 2.
- **action_mask_override** (Schedule[NDArray[np.bool_]] | None) - Optional - Override for action mask, defaults to None.
- **remote_environment_host** (str) - Optional - Host for the remote environment, defaults to "localhost".
- **remote_environment_port** (int) - Optional - Port for the remote environment, defaults to 7070.
- **noise_generator** (NoiseGenerator | None) - Optional - Noise generator, defaults to None.
- **reset_params** (dict[str, Any]) - Optional - Parameters for resetting the environment, defaults to {}.
- **include_target_obs_in_critic** (bool) - Optional - Whether to include target observations in the critic, defaults to False.
- **training** (bool) - Optional - Whether the environment is in training mode, defaults to False.
- **loop** (AbstractEventLoop | None) - Optional - Event loop for asynchronous operations, defaults to None.
```

--------------------------------

### Evaluate Reinforcement Learning Model

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/configuration.md

Use this command-line interface to evaluate trained reinforcement learning models. Specify the model path, environment name, number of episodes, and connection details for the game and environment servers.

```bash
eval \
  --model-path models/GeneralizedNh.zip \
  --env-name NhEnv \
  --num-episodes 100 \
  --run-simulation \
  --game-port 43595 \
  --remote-env-port 7070 \
  --deterministic
```

--------------------------------

### PvpEnv Class Initialization

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/pvp_env.md

Initializes the PvpEnv class with various parameters to configure the PvP simulation environment. Use this to set up rewards, penalties, and other simulation dynamics.

```python
class PvpEnv(AsyncIoEnv[NDArray[np.float32], NDArray[np.int32]]):
    def __init__(
        self,
        env_name: str,
        env_id: str = "PvpEnv",
        default_reward: Schedule[float] = ConstantSchedule(0.0),
        win_reward: Schedule[float] = ConstantSchedule(1.0),
        lose_reward: Schedule[float] = ConstantSchedule(-1.0),
        tie_reward: Schedule[float] = ConstantSchedule(-0.2),
        safe_penalty: Schedule[float] = ConstantSchedule(0.0),
        death_match: bool = True,
        penalize_food_on_death: bool = True,
        reward_target_food_on_death: bool = True,
        penalize_wasted_food: bool = True,
        reward_on_damage_generated: bool = True,
        reward_heals: bool = True,
        penalize_target_heals: bool = True,
        target: str = "baseline",
        desync_tolerance_seconds: float = 0.0,
        truncate_on_desync: bool = True,
        damage_received_reward_scale: Schedule[float] = ConstantSchedule(0.0),
        damage_dealt_reward_scale: Schedule[float] = ConstantSchedule(0.0),
        stack_frames: int | list[int] = 1,
        target_frozen_tick_reward: Schedule[float] = ConstantSchedule(0.0),
        player_frozen_tick_reward: Schedule[float] = ConstantSchedule(0.0),
        protected_correct_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        protected_wrong_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        attacked_correct_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        attacked_wrong_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        protected_previous_correct_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        protected_previous_wrong_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        attacked_previous_correct_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        attacked_previous_wrong_prayer_reward: Schedule[float] = ConstantSchedule(0.0),
        attack_level_scale_reward: Schedule[float] = ConstantSchedule(0.0),
        strength_level_scale_reward: Schedule[float] = ConstantSchedule(0.0),
        defense_level_scale_reward: Schedule[float] = ConstantSchedule(0.0),
        ranged_level_scale_reward: Schedule[float] = ConstantSchedule(0.0),
        magic_level_scale_reward: Schedule[float] = ConstantSchedule(0.0),
        reward_on_hit_with_boost_scale: Schedule[float] = ConstantSchedule(0.0),
        smite_damage_dealt_reward_multiplier: Schedule[float] = ConstantSchedule(0.0),
        smite_damage_received_reward_multiplier: Schedule[float] = ConstantSchedule(0.0),
        player_died_with_food_multiplier: Schedule[float] = ConstantSchedule(1.0),
        player_wasted_food_multiplier: Schedule[float] = ConstantSchedule(1.0),
        custom_reward_fn: Schedule[float] = ConstantSchedule(0.0),
        no_prayer_tick_reward: Schedule[float] = ConstantSchedule(0.0),
        desync_tick_threshold: int = 2,
        action_mask_override: Schedule[NDArray[np.bool_]] | None = None,
        remote_environment_host: str = "localhost",
        remote_environment_port: int = 7070,
        noise_generator: NoiseGenerator | None = None,
        reset_params: dict[str, Any] = {},
        include_target_obs_in_critic: bool = False,
        training: bool = False,
        loop: AbstractEventLoop | None = None
    )

```

--------------------------------

### Get Action Dependency Config Method Signature

Source: https://github.com/naton1/osrs-pvp-reinforcement-learning/blob/master/_autodocs/api-reference/contract_loader.md

Signature for parsing action dependencies into an internal format. This converts ID references to action indices for easier processing.

```python
def get_action_dependency_config(self) -> ActionDependencies:
    pass
```