### Generate Local Documentation

Source: https://github.com/opendilab/lightzero/blob/main/docs/README.md

Steps to install dependencies, compile, and serve the documentation locally. Ensure you are in the LightZero project directory before starting.

```bash
cd LightZero
pip install -r requirements-doc.txt
cd LightZero/docs/source
make live
```

--------------------------------

### Install LightZero from GitHub Source

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Install the latest LightZero in development from the GitHub source codes. This command clones the repository, navigates into the directory, and installs the package in editable mode.

```bash
git clone https://github.com/opendilab/LightZero.git
cd LightZero
pip3 install -e .
```

--------------------------------

### Run 1D and 2D Loss Landscape Examples

Source: https://github.com/opendilab/lightzero/blob/main/lzero/loss_landscape/README.md

Execute example scripts for visualizing 1D loss curves and 2D loss surfaces. These examples are useful for understanding the library's capabilities with CIFAR-10 data.

```bash
cd lzero/loss_landscape
python examples/example_1d.py
python examples/example_2d.py
```

--------------------------------

### Get Submodule Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/stochastic_muzero_model.md

Demonstrates how to retrieve a submodule using its fully-qualified string name. This method is efficient for checking submodule existence.

```python
a = A(
    (net_b): Module(
        (net_c): Module(
            (conv): Conv2d(16, 33, kernel_size=(3, 3), stride=(2, 2))
        )
        (linear): Linear(in_features=100, out_features=200, bias=True)
    )
)

# Check for 'net_b.linear'
submodule_linear = a.get_submodule("net_b.linear")

# Check for 'net_b.net_c.conv'
submodule_conv = a.get_submodule("net_b.net_c.conv")
```

--------------------------------

### Install LightZero from PyPI

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/installation/installation_and_quickstart.md

Install the latest stable version of LightZero using pip. Ensure you have Python 3.7 or higher.

```bash
pip install LightZero
```

--------------------------------

### Get Submodule Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/common.md

Demonstrates how to retrieve nested submodules using fully-qualified string names. This method is efficient for checking submodule existence.

```python
A(
    (net_b): Module(
        (net_c): Module(
            (conv): Conv2d(16, 33, kernel_size=(3, 3), stride=(2, 2))
        )
        (linear): Linear(in_features=100, out_features=200, bias=True)
    )
)

# To check whether or not we have the linear submodule, we would call get_submodule("net_b.linear").
# To check whether we have the conv submodule, we would call get_submodule("net_b.net_c.conv").
```

--------------------------------

### Install Panda3D-Simplepbr

Source: https://github.com/opendilab/lightzero/blob/main/zoo/metadrive/README.md

Installs the panda3d-simplepbr library, required for rendering in Metadrive. This command uses pip for installation.

```shell
pip install panda3d-simplepbr==0.9
```

--------------------------------

### Get Submodule Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/sampled_efficientzero_model.md

Demonstrates how to retrieve a nested submodule using its fully-qualified string name. This method is efficient for checking submodule existence.

```text
A(
    (net_b): Module(
        (net_c): Module(
            (conv): Conv2d(16, 33, kernel_size=(3, 3), stride=(2, 2))
        )
        (linear): Linear(in_features=100, out_features=200, bias=True)
    )
)
```

```text
get_submodule("net_b.linear")
```

```text
get_submodule("net_b.net_c.conv")
```

--------------------------------

### Environment Configuration Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/envs/customize_envs.md

Specify the registered environment's type and import path in the `create_config` section of your configuration file.

```python
create_config = dict(
    env=dict(
        type='my_custom_env',
        import_names=['zoo.board_games.my_custom_env.envs.my_custom_env'],
    ),
    ... 
)
```

--------------------------------

### Install Metadrive Simulator

Source: https://github.com/opendilab/lightzero/blob/main/zoo/metadrive/README.md

Installs the Metadrive simulator using pip. Ensure you have pip installed and a compatible Python version.

```shell
pip install metadrive-simulator==0.3.0.1
```

--------------------------------

### _init_multi_gpu_setting(model, bp_update_sync)

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Configures the multi-GPU data parallel training setup. It handles broadcasting model parameters at the start of training and prepares hooks for gradient all-reduction.

```APIDOC
## _init_multi_gpu_setting(model, bp_update_sync)

### Description
Initialize multi-gpu data parallel training setting, including broadcast model parameters at the beginning             of the training, and prepare the hook function to allreduce the gradients of model parameters.

### Method
None

### Endpoint
None

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **model** (Module) – The neural network model to be trained.
* **bp_update_sync** (bool) – Whether to synchronize update the model parameters after allreduce the                 gradients of model parameters. Async update can be parallel in different network layers like pipeline                 so that it can save time.
```

--------------------------------

### LightZero File Directory Structure Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/logs/logs.md

Illustrates the typical file organization for an experiment run with LightZero, showing log files, model checkpoints, and configuration files.

```markdown
cartpole_muzero
├── ckpt
│   ├── ckpt_best.pth.tar
│   ├── iteration_0.pth.tar
│   └── iteration_10000.pth.tar
├── log  
│   ├── buffer
│   │   └── buffer_logger.txt
│   ├── collector
│   │   └── collector_logger.txt
│   ├── evaluator
│   │   └── evaluator_logger.txt
│   ├── learner
│   │   └── learner_logger.txt
│   └── serial
│       └── events.out.tfevents.1626453528.CN0014009700M.local
├── formatted_total_config.py
└── total_config.py
```

--------------------------------

### Install Loss Landscape Library

Source: https://github.com/opendilab/lightzero/blob/main/lzero/loss_landscape/README.md

Install the necessary Python packages for the Loss Landscape library. This includes PyTorch, torchvision, and several data visualization and scientific computing libraries.

```bash
pip install torch torchvision h5py matplotlib scipy seaborn numpy
```

--------------------------------

### Run LightZero Docker Container

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Starts an interactive Docker container from the 'ubuntu-py38-lz:latest' image. The container will be removed automatically upon exit.

```bash
docker run -dit --rm ubuntu-py38-lz:latest /bin/bash
```

--------------------------------

### Example of Iterating Over Modules

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/stochastic_muzero_model_mlp.md

Demonstrates how to iterate over modules in a network using the `modules()` method. The example shows that duplicate modules are yielded only once.

```python
l = nn.Linear(2, 2)
net = nn.Sequential(l, l)
for idx, m in enumerate(net.modules()):
    print(idx, '->', m)
```

--------------------------------

### Example of using get_submodule

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/efficientzero_model.md

Demonstrates how to retrieve nested submodules using their fully-qualified string names. This method is efficient for checking submodule existence.

```python
A.get_submodule("net_b.linear")
A.get_submodule("net_b.net_c.conv")
```

--------------------------------

### Execute LightZero Example Script in Docker

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Runs a specific LightZero configuration script for CartPole MuZero within a Docker container. This assumes you are inside the container.

```bash
python ./LightZero/zoo/classic_control/cartpole/config/cartpole_muzero_config.py
```

--------------------------------

### Policy Collect Mode Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Demonstrates how to use the `collect_mode` interface of a Policy object to perform forward inference and environment stepping.

```python
>>> policy = Policy(cfg, model)
>>> policy_collect = policy.collect_mode
>>> obs = env_manager.ready_obs
>>> inference_output = policy_collect.forward(obs)
>>> next_obs, rew, done, info = env_manager.step(inference_output.action)
```

--------------------------------

### Policy Inference Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Demonstrates how to perform inference using a policy object in evaluation mode. Requires an initialized policy, model, and environment manager.

```python
>>> policy = Policy(cfg, model)
>>> policy_eval = policy.eval_mode
>>> obs = env_manager.ready_obs
>>> inference_output = policy_eval.forward(obs)
>>> next_obs, rew, done, info = env_manager.step(inference_output.action)
```

--------------------------------

### Launch Pooltool Interactive Interface

Source: https://github.com/opendilab/lightzero/blob/main/zoo/pooltool/README.md

Start the interactive command-line interface for Pooltool. Use the appropriate script for your operating system.

```bash
# Unix
run_pooltool
```

```bash
# Windows
run_pooltool.bat
```

--------------------------------

### lzero.entry.train_muzero.train_muzero

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/entry/index.md

The training entry point for MuZero and related MCTS+RL algorithms. It handles the setup and execution of the training loop.

```APIDOC
## lzero.entry.train_muzero.train_muzero

### Description
The training entry point for MuZero and related MCTS+RL algorithms. It handles the setup and execution of the training loop.

### Parameters
#### Parameters
- **input_cfg** (Tuple[dict, dict]) - Config in dict type. `Tuple[dict, dict]` type means [user_config, create_cfg].
- **seed** (int) - Random seed.
- **model** (Module | None) - Instance of torch.nn.Module.
- **model_path** (str | None) - The pretrained model path, which should point to the ckpt file of the pretrained model, and an absolute path is recommended. In LightZero, the path is usually something like `exp_name/ckpt/ckpt_best.pth.tar`.
- **max_train_iter** (int | None) - Maximum policy update iterations in training.
- **max_env_step** (int | None) - Maximum collected environment interaction steps.

### Returns
Converged policy.

### Return type
- policy (Policy)
```

--------------------------------

### Train Pong MuZero Agent

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Quick start command to train a MuZero agent for the Pong environment. Ensure you are in the 'LightZero' directory.

```bash
cd LightZero
python3 -u zoo/atari/config/atari_muzero_segment_config.py
```

--------------------------------

### Train Pong UniZero Agent

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Quick start command to train a UniZero agent for the Pong environment. Ensure you are in the 'LightZero' directory.

```bash
cd LightZero
python3 -u zoo/atari/config/atari_unizero_segment_config.py
```

--------------------------------

### Train TicTacToe MuZero Agent

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Quick start command to train a MuZero agent for the TicTacToe environment. Ensure you are in the 'LightZero' directory.

```bash
cd LightZero
python3 -u zoo/board_games/tictactoe/config/tictactoe_muzero_bot_mode_config.py
```

--------------------------------

### Initialize MuZero Collector and Evaluator

Source: https://github.com/opendilab/lightzero/blob/main/lzero/worker/README.md

Example of initializing a MuZeroCollector for data gathering and a MuZeroEvaluator for performance assessment. Ensure environment, policy, and logging objects are properly configured before instantiation.

```python
from lzero.worker import MuZeroCollector, MuZeroEvaluator

# Initialize collector
collector = MuZeroCollector(
    collect_print_freq=100,
    env=env_manager,
    policy=policy,
    tb_logger=tb_logger,
    exp_name='my_experiment',
    policy_config=policy_config
)

# Initialize evaluator
evaluator = MuZeroEvaluator(
    eval_freq=1000,
    n_evaluator_episode=10,
    env=eval_env,
    policy=policy,
    tb_logger=tb_logger,
    exp_name='my_experiment'
)
```

--------------------------------

### _init_multi_gpu_setting(model: Module, bp_update_sync: bool)

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Configures multi-GPU data parallel training. This method sets up parameter broadcasting at the start of training and prepares hooks for gradient all-reduction.

```APIDOC
## _init_multi_gpu_setting(model: Module, bp_update_sync: bool)

### Description
Initializes the settings for multi-GPU data parallel training. This includes broadcasting model parameters at the beginning of training and preparing hook functions to all-reduce the gradients of model parameters.

### Parameters
#### Path Parameters
- **model** (Module) - Required - The neural network model to be trained.
- **bp_update_sync** (bool) - Required - Specifies whether to synchronize model parameter updates after all-reducing gradients. Asynchronous updates can improve parallelism across different network layers, potentially saving time.

### Returns
- None
```

--------------------------------

### Compile Configuration and Initialize Components

Source: https://github.com/opendilab/lightzero/blob/main/zoo/classic_control/mountain_car/entry/visualize_mz_mtcar.ipynb

Compiles the main and create configurations, sets the device (CPU or CUDA), and initializes the environment manager, policy, and learner. Loads the pre-trained model state dictionary if provided.

```python
cfg, create_cfg = main_config, create_config
assert create_cfg.policy.type in ['efficientzero', 'muzero', 'stochastic_muzero', 'gumbel_muzero', 'sampled_efficientzero'], \
    "LightZero now only support the following algo.: 'efficientzero', 'muzero', 'stochastic_muzero', 'gumbel_muzero', 'sampled_efficientzero'"

if cfg.policy.cuda and torch.cuda.is_available():
    cfg.policy.device = 'cuda'
else:
    cfg.policy.device = 'cpu'

cfg = compile_config(cfg, seed=seed, env=None, auto=True, create_cfg=create_cfg, save_cfg=True)
# Create main components: env, policy
env_fn, collector_env_cfg, evaluator_env_cfg = get_vec_env_setting(cfg.env)
evaluator_env = create_env_manager(cfg.env.manager, [partial(env_fn, cfg=c) for c in evaluator_env_cfg])

evaluator_env.seed(cfg.seed, dynamic_seed=False)
set_pkg_seed(cfg.seed, use_cuda=cfg.policy.cuda)

policy = create_policy(cfg.policy, model=None, enable_field=['learn', 'collect', 'eval'])

# load pretrained model
if model_path is not None:
    policy.learn_mode.load_state_dict(torch.load(model_path, map_location=cfg.policy.device))

# Create worker components: learner, collector, evaluator, replay buffer, commander.
tb_logger = SummaryWriter(os.path.join('./{}/log/'.format(cfg.exp_name), 'serial'))
learner = BaseLearner(cfg.policy.learn.learner, policy.learn_mode, tb_logger, exp_name=cfg.exp_name)

# ==============================================================
# MCTS+RL algorithms related core code
```

--------------------------------

### Basic Usage Pattern for Training Entry Functions

Source: https://github.com/opendilab/lightzero/blob/main/lzero/entry/README.md

This snippet demonstrates the general pattern for calling training entry functions. It shows how to prepare configuration dictionaries and initiate training with optional parameters like seed, pre-initialized model, and training limits.

```python
from lzero.entry import train_muzero

# Prepare configuration
cfg = dict(...)  # User configuration
create_cfg = dict(...)  # Creation configuration

# Start training
policy = train_muzero(
    input_cfg=(cfg, create_cfg),
    seed=0,
    model=None,  # Optional: pre-initialized model
    model_path=None,  # Optional: pretrained model path
    max_train_iter=int(1e10),  # Maximum training iterations
    max_env_step=int(1e10),  # Maximum environment steps
)
```

--------------------------------

### Train MuZero with Wrapped Gym Environment

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/envs/customize_envs.md

Entry point for training MuZero using environments wrapped with LightZeroEnvWrapper. Assumes environment configuration is handled by train_muzero_with_gym_env.

```python
if __name__ == "__main__":
    """
    Overview:
        The ``train_muzero_with_gym_env`` entry means that the environment used in the training process is generated by wrapping the original gym environment with LightZeroEnvWrapper.
        Users can refer to lzero/envs/wrappers for more details.
    """
    from lzero.entry import train_muzero_with_gym_env
    train_muzero_with_gym_env([main_config, create_config], seed=0, max_env_step=max_env_step)
```

--------------------------------

### Initialize LightZeroEnvWrapper

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/envs/customize_envs.md

Initializes the LightZeroEnvWrapper by inheriting from gym.Wrapper and passing the original environment and configuration.

```python
class LightZeroEnvWrapper(gym.Wrapper):
    # overview comments
    def __init__(self, env: gym.Env, cfg: EasyDict) -> None:
        # overview comments
        super().__init__(env)
        ...
```

--------------------------------

### Verify Pooltool Installation

Source: https://github.com/opendilab/lightzero/blob/main/zoo/pooltool/README.md

Check if Pooltool is installed and accessible in your Python environment by printing its version.

```bash
python -c "import pooltool; print(pooltool.__version__)"
```

--------------------------------

### Configuration Parameters Overview

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/config/config.md

This section outlines the primary configuration parameters for LightZero, including settings for the policy model, learning process, data collection, evaluation, and other general training aspects. It highlights frequently changed areas with comments.

```python
# ==============================================================

```

--------------------------------

### __init__

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Initializes the policy instance based on the provided configuration and an optional model. It sets up different fields for learning, collecting, and evaluating.

```APIDOC
## __init__

### Description
Initializes policy instance according to input configures and model. This method will initialize different fields in policy, including `learn`, `collect`, `eval`. The `learn` field is used to train the policy, the `collect` field is used to collect data for training, and the `eval` field is used to evaluate the policy. The `enable_field` is used to specify which field to initialize, if it is None, then all fields will be initialized.

### Parameters
#### Parameters
- **cfg** (EasyDict) - The final merged config used to initialize policy.
- **model** (Module | None) - The neural network model used to initialize policy. If it is None, then the model will be created according to `default_model` method and `cfg.model` field. Otherwise, the model will be set to the `model` instance created by outside caller.
- **enable_field** (List[str] | None) - The field list to initialize. If it is None, then all fields will be initialized. Otherwise, only the fields in `enable_field` will be initialized, which is beneficial to save resources.

### Note
For the derived policy class, it should implement the `_init_learn`, `_init_collect`, `_init_eval` method to initialize the corresponding field.
```

--------------------------------

### MuZeroCollector.default_config

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/worker/index.md

Get collector’s default config. Merges collector’s default config with other default configs and user’s config to get the final config.

```APIDOC
## MuZeroCollector.default_config

### Description
Get collector’s default config. We merge collector’s default config with other default configs and user’s config to get the final config.

### Method
default_config

### Returns
#### Success Response
- **cfg** (EasyDict) - collector’s default config.
```

--------------------------------

### Running the Training Algorithm

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/config/config.md

Initiates the training process with specified configurations, seed, and maximum environment steps.

```python
if __name__ == "__main__":
    from lzero.entry import train_muzero
    train_muzero([main_config, create_config], seed=0, max_env_step=max_env_step)
```

--------------------------------

### Prepare Docker Build Context

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/tutorials/installation/installation_and_quickstart.md

Create a directory for the Docker build context, move the Dockerfile into it, and navigate into the directory. This prepares the environment for building the Docker image.

```bash
mkdir lightzero-docker
mv Dockerfile lightzero-docker/
cd lightzero-docker/
```

--------------------------------

### default_config

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/worker/index.md

Get the default configuration of the MuZeroEvaluator.

```APIDOC
## default_config MuZeroEvaluator

### Description
Get the default configuration of the MuZeroEvaluator.

### Method
GET

### Endpoint
/config/default

### Returns
An EasyDict object representing the default configuration.

### Return type
- cfg (EasyDict)
```

--------------------------------

### Get Number of Samples

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Returns the total number of samples collected or processed.

```python
def _get_n_sample() -> int | None:
    pass
```

--------------------------------

### sample

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/mcts/buffer/muzero_buffer.md

Samples data from the GameBuffer and prepares current and target batches for training. This method is essential for retrieving data to feed into the training process of MuZero models.

```APIDOC
## sample

### Description
Samples data from `GameBuffer` and prepares the current and target batch for training.

### Method
Not applicable (Python method)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **batch_size** (int) - Required - batch size.
* **policy** (MuZeroPolicy | EfficientZeroPolicy | SampledEfficientZeroPolicy) - Required - policy.

### Returns
List of train data, including current_batch and target_batch.

### Return type
- train_data (List)
```

--------------------------------

### Get Number of Episodes

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Returns the total number of episodes collected or processed.

```python
def _get_n_episode() -> int | None:
    pass
```

--------------------------------

### Get Batch Size

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Returns the batch size used for training or evaluation.

```python
def _get_batch_size() -> int | Dict[str, int]:
    pass
```

--------------------------------

### Policy Initialization and Evaluation Mode

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Demonstrates how to initialize a Policy object and use its evaluation mode for forward inference and environment interaction. Ensure 'cfg' and 'model' are defined and 'env_manager' is set up before use.

```python
policy = Policy(cfg, model)
policy_eval = policy.eval_mode
obs = env_manager.ready_obs
inference_output = policy_eval.forward(obs)
next_obs, rew, done, info = env_manager.step(inference_output.action)
```

--------------------------------

### __init__

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Initializes the policy instance with a given configuration and an optional model. It sets up the learn, collect, and eval fields based on the configuration. Fields to initialize can be specified, otherwise all fields are initialized.

```APIDOC
## __init__

### Description
Initializes the policy instance according to input configures and model. This method will initialize different fields in policy, including `learn`, `collect`, `eval`. The `learn` field is used to train the policy, the `collect` field is used to collect data for training, and the `eval` field is used to evaluate the policy. The `enable_field` is used to specify which field to initialize, if it is None, then all fields will be initialized.

### Parameters
* **cfg** (EasyDict) – The final merged config used to initialize policy.
* **model** (Module | None) – The neural network model used to initialize policy. If it is None, then the model will be created according to `default_model` method and `cfg.model` field. Otherwise, the model will be set to the `model` instance created by outside caller.
* **enable_field** (List[str] | None) – The field list to initialize. If it is None, then all fields will be initialized. Otherwise, only the fields in `enable_field` will be initialized, which is beneficial to save resources.

### Note
For the derived policy class, it should implement the `_init_learn`, `_init_collect`, `_init_eval` method to initialize the corresponding field.
```

--------------------------------

### Policy Eval Mode

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Demonstrates how to use the policy in evaluation mode to get inference outputs.

```APIDOC
## Policy Eval Mode

### Description
This section shows how to use the policy in evaluation mode.

### Usage Example
```pycon
>>> policy = Policy(cfg, model)
>>> policy_eval = policy.eval_mode
>>> obs = env_manager.ready_obs
>>> inference_output = policy_eval.forward(obs)
>>> next_obs, rew, done, info = env_manager.step(inference_output.action)
```
```

--------------------------------

### Find pybind11 Package

Source: https://github.com/opendilab/lightzero/blob/main/lzero/mcts/ctree/ctree_alphazero/CMakeLists.txt

Locates the pybind11 package, which is typically installed via pip. This is required for C++/Python interoperability.

```cmake
find_package(pybind11 CONFIG REQUIRED)
```

--------------------------------

### Strict Submodule Replacement Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/stochastic_muzero_model.md

Illustrates the behavior of `set_submodule` when `strict=True`. An AttributeError is raised if the target submodule does not exist.

```python
set_submodule("net_b.conv", nn.Conv2d(1, 1, 1), strict=True)
```

--------------------------------

### Get Target Observation Index

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Calculates the begin and end indices for the target observation at a given step `k`.

```python
def _get_target_obs_index_in_step_k(step):
    pass
```

--------------------------------

### EfficientZeroAgent.__init__

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/agent/index.md

Initializes the EfficientZeroAgent instance with environment parameters, model, and configuration. It allows for setting up the agent with a specific environment, random seed, experiment name, a PyTorch model, configuration settings, and optionally loading a pre-trained policy state dictionary.

```APIDOC
## EfficientZeroAgent.__init__

### Description
Initializes the EfficientZeroAgent instance with environment parameters, model, and configuration.

### Method
__init__

### Parameters
#### Path Parameters
- **env_id** (str | None) - Identifier for the environment to be used, registered in gym.
- **seed** (int) - Random seed for reproducibility. Defaults to 0.
- **exp_name** (str | None) - Name for the experiment. Defaults to None.
- **model** (Module | None) - PyTorch module to be used as the model. If None, a default model is created. Defaults to None.
- **cfg** (EasyDict | dict | None) - Configuration for the agent. If None, default configuration will be used. Defaults to None.
- **policy_state_dict** (str | None) - Path to a pre-trained model state dictionary. If provided, state dict will be loaded. Defaults to None.

### Notes
- If env_id is not specified, it must be included in cfg.
- The supported_env_list contains all the environment IDs that are supported by this agent.
```

--------------------------------

### train_muzero_with_gym_env

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/entry/index.md

Entry point for training MuZero, EfficientZero, and Sampled EfficientZero algorithms using gym environments. It handles environment creation, wrapping, and model training.

```APIDOC
## class lzero.entry.train_muzero_with_gym_env.train_muzero_with_gym_env

### Description
The train entry for MCTS+RL algorithms, including MuZero, EfficientZero, Sampled EfficientZero. We create a gym environment using env_id parameter, and then convert it to the format required by LightZero using LightZeroEnvWrapper class. Please refer to the get_wrappered_env method for more details.

### Parameters
* **input_cfg** (Tuple[dict, dict]) - Config in dict type. `Tuple[dict, dict]` type means [user_config, create_cfg].
* **seed** (int) - Random seed.
* **model** (torch.nn.Module | None) - Instance of torch.nn.Module.
* **model_path** (str | None) - The pretrained model path, which should point to the ckpt file of the pretrained model, and an absolute path is recommended. In LightZero, the path is usually something like `exp_name/ckpt/ckpt_best.pth.tar`.
* **max_train_iter** (int | None) - Maximum policy update iterations in training.
* **max_env_step** (int | None) - Maximum collected environment interaction steps.

### Returns
Converged policy.

### Return type
- policy (Policy)
```

--------------------------------

### State Dictionary Keys Example

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/efficientzero_model.md

Retrieves the keys of the state dictionary for a module. This is useful for inspecting the parameters and buffers within the model.

```python
module.state_dict().keys()
```

--------------------------------

### Get Default Policy Configuration

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Retrieves the default configuration settings for a policy. This is useful for understanding or initializing a policy with standard parameters.

```python
>>> cfg = Policy.default_config()
```

--------------------------------

### Build Docker Image for LightZero

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Builds a Docker image named 'ubuntu-py38-lz:latest' using the provided Dockerfile. Ensure you are in the directory containing the Dockerfile.

```bash
docker build -t ubuntu-py38-lz:latest -f ./Dockerfile .
```

--------------------------------

### MuZeroCollector._output_log

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/worker/index.md

Aggregates and logs collection statistics to the console, TensorBoard, and WandB. This method is only executed by the rank 0 process in a distributed setup.

```APIDOC
## MuZeroCollector._output_log

### Description
Aggregates and logs collection statistics to the console, TensorBoard, and WandB. This method is only executed by the rank 0 process in a distributed setup.

### Method
_output_log

### Parameters
#### Parameters
- **train_iter** (int) - Required - The current training iteration number, used as the logging step.
```

--------------------------------

### _init_learn

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Initializes the learn mode by setting up the learning model, optimizer, and MCTS utilities. This method is called during the `__init__` process.

```APIDOC
## _init_learn() -> None

### Description
Learn mode init method. Called by `self.__init__`. Initialize the learn model, optimizer and MCTS utils.

### Returns
None
```

--------------------------------

### Accessing Policy Learn Mode

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/policy/index.md

Demonstrates how to instantiate a Policy object and access its learn mode for training. This involves getting the learn mode interface and then using its forward method with data and retrieving the state dictionary.

```python
>>> policy = Policy(cfg, model)
>>> policy_learn = policy.learn_mode
>>> train_output = policy_learn.forward(data)
>>> state_dict = policy_learn.state_dict()
```

--------------------------------

### Train CartPole MuZero Agent

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Quick start command to train a MuZero agent for the CartPole environment. Ensure you are in the 'LightZero' directory.

```bash
cd LightZero
python3 -u zoo/classic_control/cartpole/config/cartpole_muzero_config.py
```

--------------------------------

### Planning for Sample Efficient Imitation Learning Code Repository

Source: https://github.com/opendilab/lightzero/blob/main/README.md

Code for 'Planning for Sample Efficient Imitation Learning'. This repository provides implementations for MCTS-based RL and imitation learning techniques.

```bash
git clone https://github.com/zhaohengyin/EfficientImitate
```

--------------------------------

### Get Specific Parameter by Target String

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/common.md

Retrieves a parameter using its fully-qualified string name. Throws an `AttributeError` if the target is invalid or not an `nn.Parameter`.

```python
>>> # xdoctest: +SKIP("undefined vars")
>>> model.get_parameter("layer1.bias")
```

--------------------------------

### Get Specific Buffer by Target String

Source: https://github.com/opendilab/lightzero/blob/main/docs/source/api_doc/model/common.md

Retrieves a buffer using its fully-qualified string name. Throws an `AttributeError` if the target is invalid or not a buffer.

```python
>>> # xdoctest: +SKIP("undefined vars")
>>> model.get_buffer("layer1.weight")
```