RLCard

https://rlcard.org
Admin
RLCard is a toolkit for reinforcement learning in card games with support for multiple game...
Tokens:66,284
Snippets:505
Trust Score:7.4
Update:4 months ago
Show doc for...
Tokens:
Raw
### Basic RLCard Blackjack Example

Source: https://rlcard.org/index.html

A short example demonstrating how to create a Blackjack environment, set up a random agent, and run a game. It prints environment properties like number of actions, players, and state/action shapes.

```python
import rlcard
from rlcard.agents import RandomAgent

env = rlcard.make('blackjack')
env.set_agents([RandomAgent(num_actions=env.num_actions)])

print(env.num_actions) # 2
print(env.num_players) # 1
print(env.state_shape) # [[2]]
print(env.action_shape) # [None]

trajectories, payoffs = env.run()

```

--------------------------------

### DMC Training Output Example

Source: https://rlcard.org/getting_started.html

This is an example of the expected output during DMC training, showing log directory creation, actor startup messages, and training statistics including loss and mean episode return.

```text
Creating log directory: experiments/dmc_result/doudizhu
Saving arguments to experiments/dmc_result/doudizhu/meta.json
Saving messages to experiments/dmc_result/doudizhu/out.log
Saving logs data to experiments/dmc_result/doudizhu/logs.csv
Saving logs' fields to experiments/dmc_result/doudizhu/fields.csv
[INFO:77533 utils:108 2022-03-23 14:31:30,859] Device cpu Actor 0 started.
[INFO:77544 utils:108 2022-03-23 14:31:32,631] Device cpu Actor 1 started.
[INFO:77554 utils:108 2022-03-23 14:31:34,505] Device cpu Actor 2 started.
[INFO:77564 utils:108 2022-03-23 14:31:36,183] Device cpu Actor 3 started.
[INFO:77574 utils:108 2022-03-23 14:31:37,967] Device cpu Actor 4 started.
Updated log fields: ['_tick', '_time', 'frames', 'mean_episode_return_0', 'loss_0', 'mean_episode_return_1', 'loss_1', 'mean_episode_return_2', 'loss_2']
[INFO:77516 trainer:335 2022-03-23 14:31:42,972] Saving checkpoint to experiments/dmc_result/doudizhu/model.tar
[INFO:77516 trainer:367 2022-03-23 14:31:43,065] After 9600 frames: @ 1884.4 fps Stats:
{'loss_0': 0.2543543875217438,
 'loss_1': 0.8054689764976501,
 'loss_2': 0.7721042633056641,
 'mean_episode_return_0': 0.2532467544078827,
 'mean_episode_return_1': 0.7515923380851746,
 'mean_episode_return_2': 0.753164529800415}
[INFO:77516 trainer:367 2022-03-23 14:31:48,070] After 19200 frames: @ 1918.3 fps Stats:
{'loss_0': 0.39971283078193665,
 'loss_1': 0.5237217545509338,
 'loss_2': 0.49323707818984985,
 'mean_episode_return_0': 0.3434908390045166,
 'mean_episode_return_1': 0.6602272987365723,
 'mean_episode_return_2': 0.6572840213775635}

```

--------------------------------

### Install RLCard with PyTorch Support

Source: https://rlcard.org/index.html

Installs the rlcard package with PyTorch implementation of training algorithms.

```bash
pip3 install rlcard[torch]

```

--------------------------------

### Train DQN or NFSP Agent

Source: https://rlcard.org/getting_started.html

This script trains a specified reinforcement learning agent (DQN or NFSP) on a chosen environment. It handles agent initialization, opponent setup, training loops, performance evaluation, and model saving. Ensure you have the necessary libraries installed and configure arguments like environment, algorithm, and logging directory.

```python
import os
import argparse

import torch

import rlcard
from rlcard.agents import RandomAgent
from rlcard.utils import (
    get_device,
    set_seed,
    tournament,
    reorganize,
    Logger,
    plot_curve,
)

def train(args):

    # Check whether gpu is available
    device = get_device()

    # Seed numpy, torch, random
    set_seed(args.seed)

    # Make the environment with seed
    env = rlcard.make(
        args.env,
        config={
            'seed': args.seed,
        }
    )

    # Initialize the agent and use random agents as opponents
    if args.algorithm == 'dqn':
        from rlcard.agents import DQNAgent
        agent = DQNAgent(
            num_actions=env.num_actions,
            state_shape=env.state_shape[0],
            mlp_layers=[64,64],
            device=device,
        )
    elif args.algorithm == 'nfsp':
        from rlcard.agents import NFSPAgent
        agent = NFSPAgent(
            num_actions=env.num_actions,
            state_shape=env.state_shape[0],
            hidden_layers_sizes=[64,64],
            q_mlp_layers=[64,64],
            device=device,
        )
    agents = [agent]
    for _ in range(1, env.num_players):
        agents.append(RandomAgent(num_actions=env.num_actions))
    env.set_agents(agents)

    # Start training
    with Logger(args.log_dir) as logger:
        for episode in range(args.num_episodes):

            if args.algorithm == 'nfsp':
                agents[0].sample_episode_policy()

            # Generate data from the environment
            trajectories, payoffs = env.run(is_training=True)

            # Reorganaize the data to be state, action, reward, next_state, done
            trajectories = reorganize(trajectories, payoffs)

            # Feed transitions into agent memory, and train the agent
            # Here, we assume that DQN always plays the first position
            # and the other players play randomly (if any)
            for ts in trajectories[0]:
                agent.feed(ts)

            # Evaluate the performance. Play with random agents.
            if episode % args.evaluate_every == 0:
                logger.log_performance(
                    episode,
                    tournament(
                        env,
                        args.num_eval_games,
                    )[0]
                )

        # Get the paths
        csv_path, fig_path = logger.csv_path, logger.fig_path

    # Plot the learning curve
    plot_curve(csv_path, fig_path, args.algorithm)

    # Save model
    save_path = os.path.join(args.log_dir, 'model.pth')
    torch.save(agent, save_path)
    print('Model saved in', save_path)

if __name__ == '__main__':
    parser = argparse.ArgumentParser("DQN/NFSP example in RLCard")
    parser.add_argument(
        '--env',
        type=str,
        default='leduc-holdem',
        choices=[
            'blackjack',
            'leduc-holdem',
            'limit-holdem',
            'doudizhu',
            'mahjong',
            'no-limit-holdem',
            'uno',
            'gin-rummy',
            'bridge',
        ],
    )
    parser.add_argument(
        '--algorithm',
        type=str,
        default='dqn',
        choices=[
            'dqn',
            'nfsp',
        ],
    )
    parser.add_argument(
        '--cuda',
        type=str,
        default='',
    )
    parser.add_argument(
        '--seed',
        type=int,
        default=42,
    )
    parser.add_argument(
        '--num_episodes',
        type=int,
        default=5000,
    )
    parser.add_argument(
        '--num_eval_games',
        type=int,
        default=2000,
    )
    parser.add_argument(
        '--evaluate_every',
        type=int,
        default=100,
    )
    parser.add_argument(
        '--log_dir',
        type=str,
        default='experiments/leduc_holdem_dqn_result/',
    )

    args = parser.parse_args()

    os.environ["CUDA_VISIBLE_DEVICES"] = args.cuda
    train(args)

```

--------------------------------

### Install RLCard from Source

Source: https://rlcard.org/index.html

Installs the rlcard package in editable mode after cloning the repository. Supports PyTorch installation.

```bash
cd rlcard
pip3 install -e .
pip3 install -e .[torch]

```

--------------------------------

### Install RLCard using Tsinghua Mirror

Source: https://rlcard.org/index.html

Installs the rlcard package using a mirror from Tsinghua University for faster downloads in China.

```bash
pip3 install rlcard -i https://pypi.tuna.tsinghua.edu.cn/simple

```

--------------------------------

### Run DQN on Blackjack

Source: https://rlcard.org/getting_started.html

This command executes the training script to train a DQN agent specifically on the Blackjack environment. It sets the environment to 'blackjack' and specifies a log directory for the results. This is a practical example to start with RLCard.

```bash
python3 examples/run_rl.py --env blackjack --algorithm dqn --log_dir experiments/blackjack_dqn_result/

```

--------------------------------

### Install RLCard Stable Version

Source: https://rlcard.org/index.html

Installs the stable version of the rlcard package using pip.

```bash
pip3 install rlcard

```

--------------------------------

### Execute Random Agent Script

Source: https://rlcard.org/getting_started.html

This command executes the random agent script for a specified environment. Ensure you have RLCard installed and the script is in the examples directory.

```bash
python3 examples/run_random.py --env leduc-holdem

```

--------------------------------

### Run Game

Source: https://rlcard.org/index.html

Executes a complete game from start to finish. It can be used for training or evaluation, calling either the agent's `step` or `eval_step` function accordingly.

```APIDOC
## env.run

### Description
Runs a complete game and returns trajectories and payoffs. This function can be called after `set_agents` has been invoked. If `is_training` is `True`, it uses the agent's `step` function; otherwise, it uses `eval_step`.

### Method
`env.run(is_training=False)`

### Parameters
#### Path Parameters
- **is_training** (boolean) - Optional - Default `False`. If `True`, the game runs in training mode, calling `agent.step()`. If `False`, it runs in evaluation mode, calling `agent.eval_step()`.
```

--------------------------------

### Install RLCard using Conda

Source: https://rlcard.org/index.html

Installs the rlcard package using the conda package manager. Note that conda installation only provides card environments.

```bash
conda install -c toubun rlcard

```

--------------------------------

### DoudizhuGame

Source: https://rlcard.org/rlcard.games.doudizhu.html

Provides game APIs for running Doudizhu and getting state information.

```APIDOC
## rlcard.games.doudizhu.game.DoudizhuGame

### Description
Provide game APIs for env to run doudizhu and get corresponding state information.

### Methods
#### `_static _get_num_actions()`
Return the total number of abstract actions.

Returns:
- the total number of abstract actions of doudizhu (int)

#### `get_num_players()`
Return the number of players in Doudizhu.

Returns:
- the number of players in doudizhu (int)

#### `get_player_id()`
Return current player’s id.

Returns:
- current player’s id (int)

#### `get_state(_player_id_)`
Return player’s state.

Parameters:
- **player_id** (int) – player id

Returns:
- The state of the player (dict)

#### `init_game()`
Initialize players and state.

Returns:
- first state in one game (dict)
- current player’s id (int)

#### `is_over()`
Judge whether a game is over.

Returns:
- True(over) / False(not over) (Bool)

#### `step(_action_)`
Perform one draw of the game.

Parameters:
- **action** (str) – specific action of doudizhu. Eg: ‘33344’

Returns:
- next player’s state (dict)
- next player’s id (int)

#### `step_back()`
Return to the previous state of the game.

Returns:
- True if the game steps back successfully (bool)
```

--------------------------------

### NolimitholdemEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Nolimit Hold'em environment. It provides methods to get payoffs and perfect information.

```APIDOC
## NolimitholdemEnv

### Description
Represents the Nolimit Hold'em environment. It provides methods to get payoffs and perfect information.

### Methods

#### get_payoffs()

* Description: Get the payoff of a game.
* Returns:
    * A list of payoffs.
* Return type:
    * list

#### get_perfect_information()

* Description: Get the perfect information of the current state.
* Returns:
    * A dictionary of all the perfect information of the current state.
* Return type:
    * dict
```

--------------------------------

### rlcard.utils.utils.init_standard_deck

Source: https://rlcard.org/rlcard.utils.html

Initializes a standard 52-card deck.

```APIDOC
## rlcard.utils.utils.init_standard_deck

### Description
Initialize a standard deck of 52 cards.

### Returns
A list of Card objects.

### Return Type
(list)
```

--------------------------------

### rlcard.games.uno.utils.init_deck()

Source: https://rlcard.org/rlcard.games.uno.html

Generates a standard Uno deck containing 108 cards.

```APIDOC
## rlcard.games.uno.utils.init_deck()

### Description
Generates a standard Uno deck consisting of 108 cards.

### Method
N/A (Function in rlcard.games.uno.utils)

### Parameters
None

### Response
#### Success Response
* **list**: A list of 108 strings, each representing a card in the Uno deck.

### Response Example
```python
deck = rlcard.games.uno.utils.init_deck()
print(len(deck))
# Output: 108
```
```

--------------------------------

### Get Legal Actions in No Limit Texas Hold'em

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Retrieves the list of legal actions available to the current player in a No Limit Texas Hold'em round. This is crucial for enforcing game rules and guiding player choices.

```python
def get_nolimit_legal_actions(self, players):
    """Obtain the legal actions for the current player

    Parameters:
        players (list): The players in the game

    Returns:
        list: A list of legal actions

    """
```

--------------------------------

### Create a Game Environment

Source: https://rlcard.org/index.html

Use `rlcard.make` to create a game environment. Optionally, provide a configuration dictionary for game-specific settings like the number of players.

```python
env = rlcard.make('leduc-holdem', config={'game_num_players': 3})
```

--------------------------------

### Setting up Agents in RLCard Environment

Source: https://rlcard.org/overview.html

This snippet shows how to set up multiple agents for a game environment in RLCard. Ensure that the agents provided are instances of the Agent class.

```python
env.set_agents([RandomAgent(), RandomAgent()])
```

--------------------------------

### Run a Complete Game

Source: https://rlcard.org/index.html

After setting agents, use `env.run()` to play a complete game. Set `is_training=True` to use the agent's `step` function, or `False` to use `eval_step`.

```python
trajectories, payoffs = env.run(is_training=False)
```

--------------------------------

### DQNAgentPettingZoo Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for DQNAgentPettingZoo, an adaptation of DQNAgent for PettingZoo environments.

```APIDOC
## DQNAgentPettingZoo Methods

### Description
An adaptation of DQNAgent for use with PettingZoo environments.

### Methods
- `eval_step(state)`: Perform an evaluation step.
- `feed(transition)`: Feed a transition into the agent.
- `step(state, action, reward, next_state, done)`: Perform a training step.
```

--------------------------------

### MahjongEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Mahjong environment. It provides a method to get payoffs.

```APIDOC
## MahjongEnv

### Description
Represents the Mahjong environment. It provides a method to get payoffs.

### Methods

#### get_payoffs()

* Description: Get the payoffs of players. Must be implemented in the child class.
* Returns:
    * A list of payoffs for each player.
* Return type:
    * list
```

--------------------------------

### Create Environment

Source: https://rlcard.org/index.html

Creates a game environment using the rlcard.make function. Optionally, configurations can be provided to customize the environment, such as setting a random seed or enabling step-back functionality.

```APIDOC
## rlcard.make

### Description
Creates a game environment. You can optionally specify configurations with a dictionary.

### Method
`rlcard.make(env_id, config={})`

### Parameters
#### Path Parameters
- **env_id** (string) - Required - The name of the environment to create (e.g., 'blackjack', 'leduc-holdem').
- **config** (dictionary) - Optional - A dictionary specifying environment configurations.
  - **seed** (any) - Optional - Default `None`. Sets a local random seed for the environment for reproducible results.
  - **allow_step_back** (boolean) - Optional - Default `False`. Set to `True` to allow the `step_back` function to traverse backward in the game tree.
  - **game_num_players** (integer) - Optional - Game-specific configuration. Currently supported in Blackjack to set the number of players.
```

--------------------------------

### GinRummyEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Gin Rummy environment. It provides a method to get payoffs.

```APIDOC
## GinRummyEnv

### Description
Represents the Gin Rummy environment. It provides a method to get payoffs.

### Methods

#### get_payoffs()

* Description: Get the payoffs of players. Must be implemented in the child class.
* Returns:
    * A list of payoffs for each player.
* Return type:
    * list
```

--------------------------------

### UnoCard.get_str()

Source: https://rlcard.org/rlcard.games.uno.html

Gets the string representation of an Uno card, including its color and trait.

```APIDOC
## UnoCard.get_str()

### Description
Gets the string representation of an Uno card.

### Method
N/A (Method of UnoCard class)

### Parameters
None

### Response
#### Success Response
- **str**: The string of card

### Response Example
```
"red 1"
```
```

--------------------------------

### NFSPAgentPettingZoo Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for NFSPAgentPettingZoo, an adaptation of NFSPAgent for PettingZoo environments.

```APIDOC
## NFSPAgentPettingZoo Methods

### Description
An adaptation of NFSPAgent for use with PettingZoo environments.

### Methods
- `eval_step(state)`: Perform an evaluation step.
- `feed(transition)`: Feed a transition into the agent.
- `step(state, action, reward, next_state, done)`: Perform a training step.
```

--------------------------------

### EnvRegistry Class

Source: https://rlcard.org/rlcard.envs.html

Utilities for registering and creating environment instances by their IDs.

```APIDOC
## class rlcard.envs.registration.EnvRegistry

Bases: object
Register an environment (game) by ID

### Methods

*   **make(_env_id_ , _config ={'allow_step_back': False, 'seed': None}_)**
    Create and environment instance.
    Parameters:
        *   **env_id** (_string_) – The name of the environment.
        *   **config** (_dict_) – A dictionary of the environment settings.

*   **register(_env_id_ , _entry_point_)**
    Register an environment.
    Parameters:
        *   **env_id** (_string_) – The name of the environent.
        *   **entry_point** (_string_) – A string the indicates the location of the envronment class.
```

--------------------------------

### Performing a Single Step in the Environment

Source: https://rlcard.org/overview.html

Given the current state, the environment takes one step forward, returning the next state and the next player. This is for advanced access to the environment.

```python
env.step(action)
```

--------------------------------

### BridgeEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Bridge environment. It provides methods to get payoffs and perfect information.

```APIDOC
## BridgeEnv

### Description
Represents the Bridge environment. It provides methods to get payoffs and perfect information.

### Methods

#### get_payoffs()

* Description: Get the payoffs of players.
* Returns:
    * A list of payoffs for each player.
* Return type:
    * list

#### get_perfect_information()

* Description: Get the perfect information of the current state.
* Returns:
    * A dictionary of all the perfect information of the current state.
* Return type:
    * dict
```

--------------------------------

### EnvRegistry Class Methods

Source: https://rlcard.org/doctree.html

Methods for registering and accessing game environments within RLCard.

```APIDOC
## EnvRegistry Class Methods

### Description
Manages the registration and retrieval of game environments. Allows developers to register custom environments or access predefined ones.

### Methods
- `EnvRegistry.make(env_id, **kwargs)`: Creates and returns an environment instance.
- `EnvRegistry.register(env_id, entry_point, **kwargs)`: Registers a new environment.
```

--------------------------------

### LimitholdemEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Limitholdem environment. It provides methods to get payoffs and perfect information.

```APIDOC
## LimitholdemEnv

### Description
Represents the Limitholdem environment. It provides methods to get payoffs and perfect information.

### Methods

#### get_payoffs()

* Description: Get the payoff of a game.
* Returns:
    * A list of payoffs.
* Return type:
    * list

#### get_perfect_information()

* Description: Get the perfect information of the current state.
* Returns:
    * A dictionary of all the perfect information of the current state.
* Return type:
    * dict
```

--------------------------------

### DouDizhuRuleModelV1 Methods

Source: https://rlcard.org/doctree.html

Methods for the DouDizhuRuleModelV1.

```APIDOC
## DouDizhuRuleModelV1.agents

### Description
Provides access to the agents used by the model.

### Method
Not specified (assumed to be an attribute access).

### Endpoint
Not applicable (SDK attribute).
```

--------------------------------

### LeducholdemEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Leduc Hold'em environment. It provides methods to get payoffs and perfect information.

```APIDOC
## LeducholdemEnv

### Description
Represents the Leduc Hold'em environment. It provides methods to get payoffs and perfect information.

### Methods

#### get_payoffs()

* Description: Get the payoff of a game.
* Returns:
    * A list of payoffs.
* Return type:
    * list

#### get_perfect_information()

* Description: Get the perfect information of the current state.
* Returns:
    * A dictionary of all the perfect information of the current state.
* Return type:
    * dict
```

--------------------------------

### Register New Environment

Source: https://rlcard.org/development.html

Shows how to register a newly developed environment with the RLCard toolkit. This involves making the environment discoverable by its name.

```python
import rlcard
rlcard.make(#the new evironment#)
```

--------------------------------

### Mahjong Game Get Player ID

Source: https://rlcard.org/rlcard.games.mahjong.html

Returns the ID of the current player in the Mahjong game.

```python
def get_player_id(self):
    pass
```

--------------------------------

### rlcard.utils.utils.init_54_deck

Source: https://rlcard.org/rlcard.utils.html

Initializes a standard deck of 52 cards plus two jokers (BJ and RJ).

```APIDOC
## rlcard.utils.utils.init_54_deck

### Description
Initialize a standard deck of 52 cards, BJ and RJ.

### Returns
A list of Card objects.

### Return Type
(list)
```

--------------------------------

### Mahjong Game Get Number of Players

Source: https://rlcard.org/rlcard.games.mahjong.html

Returns the number of players participating in the Mahjong game.

```python
def get_num_players(self):
    pass
```

--------------------------------

### DMCModelPettingZoo Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for DMCModelPettingZoo, a utility for DMC agents in PettingZoo environments.

```APIDOC
## DMCModelPettingZoo Methods

### Description
Provides utility methods for DMC models in PettingZoo environments.

### Methods
- `eval()`: Evaluate the model.
- `get_agent()`: Get the agent associated with the model.
- `get_agents()`: Get all agents associated with the model.
- `parameters()`: Get the model's parameters.
- `share_memory()`: Share the model's memory.
```

--------------------------------

### DoudizhuEnv

Source: https://rlcard.org/rlcard.envs.html

Represents the Dou Dizhu environment. It provides methods to get action features, payoffs, and perfect information.

```APIDOC
## DoudizhuEnv

### Description
Represents the Dou Dizhu environment. It provides methods to get action features, payoffs, and perfect information.

### Methods

#### get_action_feature(action)

* Description: For some environments such as DouDizhu, we can have action features.
* Parameters:
    * **action** (any) - The action to get features for.
* Returns:
    * The action features.
* Return type:
    * numpy.array

#### get_payoffs()

* Description: Get the payoffs of players. Must be implemented in the child class.
* Returns:
    * A list of payoffs for each player.
* Return type:
    * list

#### get_perfect_information()

* Description: Get the perfect information of the current state.
* Returns:
    * A dictionary of all the perfect information of the current state.
* Return type:
    * dict
```

--------------------------------

### RandomAgentPettingZoo Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for RandomAgentPettingZoo, a random agent for PettingZoo environments.

```APIDOC
## RandomAgentPettingZoo Methods

### Description
A random agent for use with PettingZoo environments.

### Methods
- `eval_step(state)`: Perform an evaluation step.
- `step(state, action, reward, next_state, done)`: Perform a step.
```

--------------------------------

### Mahjong Game Get State

Source: https://rlcard.org/rlcard.games.mahjong.html

Retrieves the current state for a specific player. Requires the player's ID as input.

```python
def get_state(self, player_id):
    pass
```

--------------------------------

### AveragePolicyNetwork Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for the AveragePolicyNetwork class, used in NFSP for policy approximation.

```APIDOC
## AveragePolicyNetwork Methods

### Description
Represents the average policy network used in NFSP for approximating the policy.

### Methods
- `checkpoint_attributes()`: Get attributes for checkpointing.
- `forward(state)`: Compute the output of the network for a given state.
- `from_checkpoint(path)`: Load network from a checkpoint file path.
```

--------------------------------

### Getting Payoffs at the End of a Game

Source: https://rlcard.org/overview.html

At the end of the game, this function can be called to obtain the payoffs for each player. This is useful for analyzing game outcomes.

```python
env.get_payoffs()
```

--------------------------------

### Run DMC on CPU

Source: https://rlcard.org/getting_started.html

This command executes the DMC training script on the Dou Dizhu environment using the CPU. It specifies the environment and an experiment ID for logging.

```bash
python3 examples/run_dmc.py --env doudizhu --xpid doudizhu

```

--------------------------------

### Load Pre-trained Model

Source: https://rlcard.org/development.html

Demonstrates how to load a pre-trained model for a specific game like Leduc Hold'em NFSP. This allows access to the game agents for further use.

```python
from rlcard import models
leduc_nfsp_model = models.load('leduc-holdem-nfsp')
```

--------------------------------

### Get Payoffs in No Limit Hold'em

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Returns the final payoffs for each player at the end of the game. Each element in the returned list corresponds to the payoff of a player.

```python
payoffs = game.get_payoffs()
```

--------------------------------

### Gin Rummy Judge Methods

Source: https://rlcard.org/rlcard.games.html

Provides methods for the Gin Rummy judge to determine legal actions and get specific card information.

```APIDOC
## GinRummyJudge Methods

### Description
Methods for the Gin Rummy judge to evaluate game states and actions.

### Methods
- `GinRummyJudge.get_legal_actions()`: Returns a list of legal actions for the current state.
- `get_going_out_cards()`: Retrieves cards that can be used to go out.
```

--------------------------------

### DQNAgent Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for the DQNAgent class, including methods for evaluation, feeding experiences, training, and managing checkpoints.

```APIDOC
## DQNAgent Methods

### Description
Provides methods for a Deep Q-Network (DQN) agent, including training, evaluation, and checkpoint management.

### Methods
- `checkpoint_attributes()`: Get attributes for checkpointing.
- `eval_step()`: Perform an evaluation step.
- `feed(time, state, action, reward, next_state, done)`: Feed a transition into the agent.
- `feed_memory(state, action, reward, next_state, done)`: Feed a transition into the memory.
- `from_checkpoint(path)`: Load agent from a checkpoint file path.
- `predict(state)`: Predict the Q-values for a given state.
- `save_checkpoint(path)`: Save the agent's checkpoint to a file path.
- `set_device(device)`: Set the computation device.
- `step(state, action, reward, next_state, done)`: Perform a training step.
- `train()`: Train the agent.
```

--------------------------------

### Mahjong Game Get Number of Actions

Source: https://rlcard.org/rlcard.games.mahjong.html

Returns the total number of possible actions in the Mahjong game. This is a static method and does not require any instance parameters.

```python
def get_num_actions():
    pass
```

--------------------------------

### RandomAgent

Source: https://rlcard.org/rlcard.agents.html

A general Random agent for running toy examples on card games. It provides methods for evaluation steps and taking steps during training data generation.

```APIDOC
## RandomAgent

### Description
A general Random agent for running toy examples on card games. It provides methods for evaluation steps and taking steps during training data generation.

### Methods

*   **eval_step(state)**: Predicts the action given the current state for evaluation. Since random agents are not trained, this function is equivalent to the step function.
    *   Parameters:
        *   **state** (dict): An dictionary representing the current state.
    *   Returns:
        *   action (int): The action predicted (randomly chosen) by the random agent.
        *   probs (list): The list of action probabilities.

*   **step(state)**: Predicts the action given the current state for generating training data.
    *   Parameters:
        *   **state** (dict): An dictionary representing the current state.
    *   Returns:
        *   action (int): The action predicted (randomly chosen) by the random agent.
```

--------------------------------

### DMCAgentPettingZoo Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for DMCAgentPettingZoo, an adaptation of DMCAgent for PettingZoo environments.

```APIDOC
## DMCAgentPettingZoo Methods

### Description
An adaptation of DMCAgent for use with PettingZoo environments.

### Methods
- `eval_step(state)`: Perform an evaluation step.
- `feed(transition)`: Feed a transition into the agent.
- `step(state, action, reward, next_state, done)`: Perform a training step.
```

--------------------------------

### GinRummyNoviceRuleModel Methods

Source: https://rlcard.org/doctree.html

Methods for the GinRummyNoviceRuleModel.

```APIDOC
## GinRummyNoviceRuleModel.agents

### Description
Provides access to the agents used by the model.

### Method
Not specified (assumed to be an attribute access).

### Endpoint
Not applicable (SDK attribute).
```

--------------------------------

### Mahjong Game Get Legal Actions

Source: https://rlcard.org/rlcard.games.mahjong.html

Returns a list of legal actions available for the current state of the game. Requires the current game state as input.

```python
def get_legal_actions(state):
    pass
```

--------------------------------

### DoudizhuPlayer Methods

Source: https://rlcard.org/rlcard.games.doudizhu.html

Provides methods for managing a player's hand, determining available actions, and performing actions within the Doudizhu game.

```APIDOC
## DoudizhuPlayer

### Description
Represents a player in the Doudizhu game, managing their hand, role, and actions.

### Methods

#### `available_actions(greater_player=None, judger=None)`

##### Description
Gets the available actions a player can make based on the game rules and the current state.

##### Parameters
* **greater_player** (DoudizhuPlayer object, optional) - The player who played the current biggest cards.
* **judger** (DoudizhuJudger object, optional) - The judger object to determine legal actions.

##### Returns
- A list of strings representing available actions (e.g., 'pass', '8', '9').

#### `play(action, greater_player=None)`

##### Description
Performs a specified action.

##### Parameters
* **action** (string) - The specific action to perform.
* **greater_player** (DoudizhuPlayer object, optional) - The player who played the current biggest cards.

##### Returns
- The player object who played the new biggest cards, or None if no new biggest cards were played.

#### `play_back()`

##### Description
Restores the player's recorded cards back to their current hand, used for undoing moves.

#### `set_current_hand(value)`

##### Description
Sets the player's current hand of cards.

##### Parameters
* **value** - The new hand of cards to set.
```

--------------------------------

### rlcard.utils.pettingzoo_utils.run_game_pettingzoo

Source: https://rlcard.org/rlcard.utils.html

Runs a game using PettingZoo environments.

```APIDOC
## rlcard.utils.pettingzoo_utils.run_game_pettingzoo

### Description
Runs a game using PettingZoo environments.

### Parameters
* **env** (Env class) - The PettingZoo environment.
* **num_games** (int) - The number of games to run.
```

--------------------------------

### Get Player State in No Limit Hold'em

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Retrieves the current state for a specific player. The state is represented as a dictionary and includes information relevant to the player's decision-making.

```python
player_state = game.get_state(player_id)
```

--------------------------------

### PettingZoo Agents Methods

Source: https://rlcard.org/doctree.html

Provides documentation for agents compatible with the PettingZoo environment API.

```APIDOC
## PettingZoo Agents Methods

### Description
Documentation for agent implementations designed to be compatible with the PettingZoo multi-agent environment API. This includes DQNAgentPettingZoo, NFSPAgentPettingZoo, and RandomAgentPettingZoo.

### Methods
- `DQNAgentPettingZoo.eval_step(state)`: Performs an evaluation step for the DQN agent in a PettingZoo environment.
- `DQNAgentPettingZoo.feed(transition)`: Feeds a transition into the DQN agent's memory.
- `DQNAgentPettingZoo.step(state)`: Takes a step in the environment using the DQN agent.
- `NFSPAgentPettingZoo.eval_step(state)`: Performs an evaluation step for the NFSP agent in a PettingZoo environment.
- `NFSPAgentPettingZoo.feed(transition)`: Feeds a transition into the NFSP agent's memory.
- `NFSPAgentPettingZoo.step(state)`: Takes a step in the environment using the NFSP agent.
- `RandomAgentPettingZoo.eval_step(state)`: Performs an evaluation step for the Random agent in a PettingZoo environment.
- `RandomAgentPettingZoo.step(state)`: Takes a step in the environment using the Random agent.
```

--------------------------------

### Get Legal Actions in No Limit Hold'em

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Retrieves the list of legal actions available to the current player in the No Limit Hold'em game. This is crucial for agents to determine valid moves.

```python
legal_actions = game.get_legal_actions()
```

--------------------------------

### Get Number of Actions in No Limit Hold'em

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Returns the total number of possible actions in No Limit Hold'em. This includes actions like call, raise (half pot, full pot, all-in), check, and fold.

```python
num_actions = game.get_num_actions()
```

--------------------------------

### Start New Bidding Round in No Limit Texas Hold'em

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Initiates a new bidding round, setting up initial chips for players. Special handling is included for the first round to establish big and small blinds.

```python
def start_new_round(self, game_pointer, raised=None):
    """Start a new bidding round

    Parameters:
        game_pointer (int): The game_pointer that indicates the next player
        raised (list): Initialize the chips for each player

    Note: For the first round of the game, we need to setup the big/small blind
    """
```

--------------------------------

### LeducHoldemRuleModelV2 Methods

Source: https://rlcard.org/doctree.html

Methods for the LeducHoldemRuleModelV2.

```APIDOC
## LeducHoldemRuleModelV2.agents

### Description
Provides access to the agents used by the model.

### Method
Not specified (assumed to be an attribute access).

### Endpoint
Not applicable (SDK attribute).
```

--------------------------------

### DoudizhuGame Methods

Source: https://rlcard.org/doctree.html

Core methods for managing the state and flow of a Dou Dizhu game.

```APIDOC
## DoudizhuGame

### Description
Manages the game logic for Dou Dizhu.

### Methods
#### `get_num_actions()`
Returns the number of possible actions.

#### `get_num_players()`
Returns the number of players.

#### `get_player_id()`
Returns the current player's ID.

#### `get_state()`
Returns the current game state.

#### `init_game()`
Initializes a new game.

#### `is_over()`
Checks if the game has ended.

#### `step()`
Executes a step in the game.

#### `step_back()`
Reverts the last step taken in the game.
```

--------------------------------

### Train CFR Agent on Leduc Hold'em

Source: https://rlcard.org/getting_started.html

This script demonstrates training a CFR agent on Leduc Hold'em using chance sampling. It includes environment setup, agent initialization, training loop, and performance evaluation.

```python
import os
import argparse

import rlcard
from rlcard.agents import (
    CFRAgent,
    RandomAgent,
)
from rlcard.utils import (
    set_seed,
    tournament,
    Logger,
    plot_curve,
)

def train(args):
    # Make environments, CFR only supports Leduc Holdem
    env = rlcard.make(
        'leduc-holdem',
        config={
            'seed': 0,
            'allow_step_back': True,
        }
    )
    eval_env = rlcard.make(
        'leduc-holdem',
        config={
            'seed': 0,
        }
    )

    # Seed numpy, torch, random
    set_seed(args.seed)

    # Initilize CFR Agent
    agent = CFRAgent(
        env,
        os.path.join(
            args.log_dir,
            'cfr_model',
        ),
    )
    agent.load()  # If we have saved model, we first load the model

    # Evaluate CFR against random
    eval_env.set_agents([
        agent,
        RandomAgent(num_actions=env.num_actions),
    ])

    # Start training
    with Logger(args.log_dir) as logger:
        for episode in range(args.num_episodes):
            agent.train()
            print('\rIteration {}'.format(episode), end='')
            # Evaluate the performance. Play with Random agents.
            if episode % args.evaluate_every == 0:
                agent.save() # Save model
                logger.log_performance(
                    episode,
                    tournament(
                        eval_env,
                        args.num_eval_games
                    )[0]
                )

        # Get the paths
        csv_path, fig_path = logger.csv_path, logger.fig_path
    # Plot the learning curve
    plot_curve(csv_path, fig_path, 'cfr')

if __name__ == '__main__':
    parser = argparse.ArgumentParser("CFR example in RLCard")
    parser.add_argument(
        '--seed',
        type=int,
        default=42,
    )
    parser.add_argument(
        '--num_episodes',
        type=int,
        default=5000,
    )
    parser.add_argument(
        '--num_eval_games',
        type=int,
        default=2000,
    )
    parser.add_argument(
        '--evaluate_every',
        type=int,
        default=100,
    )
    parser.add_argument(
        '--log_dir',
        type=str,
        default='experiments/leduc_holdem_cfr_result/',
    )

    args = parser.parse_args()

    train(args)

```

```bash
python3 examples/run_cfr.py

```

--------------------------------

### LeducHoldemRuleModelV1 Methods

Source: https://rlcard.org/doctree.html

Methods for the LeducHoldemRuleModelV1.

```APIDOC
## LeducHoldemRuleModelV1.agents

### Description
Provides access to the agents used by the model.

### Method
Not specified (assumed to be an attribute access).

### Endpoint
Not applicable (SDK attribute).
```

--------------------------------

### DoudizhuEnv Methods

Source: https://rlcard.org/doctree.html

Specific methods for the Dou Dizhu environment.

```APIDOC
## DoudizhuEnv Methods

### Description
Provides methods specific to the Dou Dizhu environment, including action feature, payoff, and perfect information retrieval.

### Methods
- `DoudizhuEnv.get_action_feature()`: Retrieves action features for Dou Dizhu.
- `DoudizhuEnv.get_payoffs()`: Gets the payoffs for the Dou Dizhu game.
- `DoudizhuEnv.get_perfect_information()`: Returns the perfect information of the Dou Dizhu state.
```

--------------------------------

### NFSPAgent Methods

Source: https://rlcard.org/rlcard.agents.html

Documentation for the NFSPAgent class, covering its training, evaluation, and checkpointing capabilities.

```APIDOC
## NFSPAgent Methods

### Description
Provides methods for the Neural Fictitious Self-Play (NFSP) agent, including training, evaluation, and checkpoint management.

### Methods
- `checkpoint_attributes()`: Get attributes for checkpointing.
- `eval_step()`: Perform an evaluation step.
- `feed(time, state, action, reward, next_state, done)`: Feed a transition into the agent.
- `from_checkpoint(path)`: Load agent from a checkpoint file path.
- `sample_episode_policy()`: Sample an action from the episode policy.
- `save_checkpoint(path)`: Save the agent's checkpoint to a file path.
- `set_device(device)`: Set the computation device.
- `step(state, action, reward, next_state, done)`: Perform a training step.
- `train_sl()`: Train the supervised learning component.
```

--------------------------------

### Run Random Agent in RLCard

Source: https://rlcard.org/getting_started.html

This script demonstrates how to use a random agent to play an RLCard environment. It sets up the environment, agents, and runs the game to generate trajectories and player wins.

```python
import argparse
import pprint

import rlcard
from rlcard.agents import RandomAgent
from rlcard.utils import set_seed

def run(args):
    # Make environment
    env = rlcard.make(
        args.env,
        config={
            'seed': 42,
        }
    )

    # Seed numpy, torch, random
    set_seed(42)

    # Set agents
    agent = RandomAgent(num_actions=env.num_actions)
    env.set_agents([agent for _ in range(env.num_players)])

    # Generate data from the environment
    trajectories, player_wins = env.run(is_training=False)
    # Print out the trajectories
    print('\nTrajectories:')
    print(trajectories)
    print('\nSample raw observation:')
    pprint.pprint(trajectories[0][0]['raw_obs'])
    print('\nSample raw legal_actions:')
    pprint.pprint(trajectories[0][0]['raw_legal_actions'])

if __name__ == '__main__':
    parser = argparse.ArgumentParser("Random example in RLCard")
    parser.add_argument(
        '--env',
        type=str,
        default='leduc-holdem',
        choices=[
            'blackjack',
            'leduc-holdem',
            'limit-holdem',
            'doudizhu',
            'mahjong',
            'no-limit-holdem',
            'uno',
            'gin-rummy',
            'bridge',
        ],
    )

    args = parser.parse_args()

    run(args)

```

--------------------------------

### Env Class Methods

Source: https://rlcard.org/doctree.html

Methods for interacting with RLCard environments, including state management, action execution, and game control.

```APIDOC
## Env Class Methods

### Description
Provides core functionalities for interacting with RLCard environments. Users can get game states, execute actions, reset the environment, and check game status.

### Methods
- `Env.get_action_feature()`: Retrieves action features.
- `Env.get_payoffs()`: Gets the payoffs for the current game state.
- `Env.get_perfect_information()`: Returns the perfect information of the current state.
- `Env.get_player_id()`: Gets the current player's ID.
- `Env.get_state()`: Retrieves the current state of the environment.
- `Env.is_over()`: Checks if the game is over.
- `Env.reset()`: Resets the environment to the initial state.
- `Env.run()`: Runs the environment with agents.
- `Env.seed()`: Seeds the environment's random number generator.
- `Env.set_agents()`: Sets the agents for the environment.
- `Env.step()`: Executes a step in the environment.
- `Env.step_back()`: Reverts the last step taken in the environment.
```

--------------------------------

### NolimitholdemRound.start_new_round

Source: https://rlcard.org/rlcard.games.nolimitholdem.html

Initiates a new bidding round in No Limit Hold'em, setting up blinds if necessary.

```APIDOC
## NolimitholdemRound.start_new_round

### Description
Initiates a new bidding round in No Limit Hold'em, setting up blinds if necessary.

### Method
NolimitholdemRound.start_new_round

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
* **game_pointer** (int) - The game_pointer that indicates the next player
* **raised** (list) - Initialize the chips for each player

### Notes
For the first round of the game, we need to setup the big/small blind
```

--------------------------------

### UNORuleModelV1 Methods

Source: https://rlcard.org/doctree.html

Methods for the UNORuleModelV1.

```APIDOC
## UNORuleModelV1.agents

### Description
Provides access to the agents used by the model.

### Method
Not specified (assumed to be an attribute access).

### Endpoint
Not applicable (SDK attribute).

## UNORuleModelV1.use_raw

### Description
Indicates whether to use raw data.

### Method
Not specified (assumed to be an attribute access).

### Endpoint
Not applicable (SDK attribute).
```

--------------------------------

### UNO Environment

Source: https://rlcard.org/rlcard.envs.html

Specific methods for the UNO environment.

```APIDOC
## UNO Environment

### `UnoEnv.get_payoffs()`

Returns the payoffs for the UNO game.

### `UnoEnv.get_perfect_information()`

Returns the perfect information representation of the UNO state.
```

--------------------------------

### Mahjong Game Initialize Game

Source: https://rlcard.org/rlcard.games.mahjong.html

Initializes a new game of Mahjong. This version supports two-player Mahjong and returns the initial game state and current player ID.

```python
def init_game(self):
    pass
```

--------------------------------

### Leducholdem Environment

Source: https://rlcard.org/rlcard.envs.html

Specific methods for the Leducholdem environment.

```APIDOC
## Leducholdem Environment

### `LeducholdemEnv.get_payoffs()`

Returns the payoffs for the Leducholdem game.

### `LeducholdemEnv.get_perfect_information()`

Returns the perfect information representation of the Leducholdem state.
```

--------------------------------

### Global Registration Functions

Source: https://rlcard.org/doctree.html

Top-level functions for environment registration and creation.

```APIDOC
## Global Registration Functions

### Description
Provides global functions for environment registration and creation, mirroring the `EnvRegistry` class methods.

### Functions
- `make(env_id, **kwargs)`: Creates and returns an environment instance.
- `register(env_id, entry_point, **kwargs)`: Registers a new environment.
```
Tokens:
Raw