### Install TensorTrade from GitHub

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/overview/getting_started.md

Installs the TensorTrade package directly from its GitHub repository. This is useful for developers who want the latest changes.

```bash
pip install git+https://github.com/tensortrade-org/tensortrade.git
```

--------------------------------

### Install TensorTrade with pip

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/overview/getting_started.md

Installs the TensorTrade package from the Python Package Index (PyPI). This is the recommended method for most users.

```bash
pip install tensortrade
```

--------------------------------

### Build TensorTrade Documentation with Docker

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/overview/getting_started.md

Executes a Docker command to build the documentation for TensorTrade. This command compiles the project's documentation files.

```bash
make run-docs
```

--------------------------------

### Run TensorTrade Test Suite with Docker

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/overview/getting_started.md

Executes a Docker command to run the test suite for TensorTrade. This command helps verify the functionality and stability of the library.

```bash
make run-tests
```

--------------------------------

### Install TensorTrade from GitHub

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/ledger_example.md

Installs the master branch of the TensorTrade library directly from its GitHub repository using pip. This ensures you have the latest version.

```bash
!pip install git+https://github.com/tensortrade-org/tensortrade.git -U
```

--------------------------------

### Run TensorTrade Jupyter Notebooks with Docker

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/overview/getting_started.md

Executes a Docker command to run Jupyter notebooks associated with TensorTrade. This command generates a URL to access the notebooks in a web browser.

```bash
make run-notebook
```

--------------------------------

### Instantiate and Interact with Trading Environment

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Creates an instance of the trading environment using a configuration and then resets the environment to start an episode.

```python
# Instantiate the environment
env = create_env(env_config_validation)

# Run until episode ends
done = False
obs = env.reset()
```

--------------------------------

### Configure Exchanges and Portfolio

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/ledger_example.ipynb

Sets up simulated exchanges (Bitfinex, Bitstamp) with streaming data for trading pairs and initializes a portfolio with multiple wallets holding different cryptocurrencies (USD, BTC, ETH, LTC) across these exchanges.

```python
bitfinex = Exchange("bitfinex", service=execute_order)(
    Stream.source(list(bitfinex_btc['close'][-100:]), dtype="float").rename("USD-BTC"),
    Stream.source(list(bitfinex_eth['close'][-100:]), dtype="float").rename("USD-ETH")
)

bitstamp = Exchange("bitstamp", service=execute_order)(
    Stream.source(list(bitstamp_btc['close'][-100:]), dtype="float").rename("USD-BTC"),
    Stream.source(list(bitstamp_eth['close'][-100:]), dtype="float").rename("USD-ETH"),
    Stream.source(list(bitstamp_ltc['close'][-100:]), dtype="float").rename("USD-LTC")
)

portfolio = Portfolio(USD, [
    Wallet(bitfinex, 10000 * USD),
    Wallet(bitfinex, 10 * BTC),
    Wallet(bitfinex, 5 * ETH),
    Wallet(bitstamp, 1000 * USD),
    Wallet(bitstamp, 5 * BTC),
    Wallet(bitstamp, 20 * ETH),
    Wallet(bitstamp, 3 * LTC)
])
```

--------------------------------

### Install TensorTrade and Dependencies

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Installs or upgrades the TensorTrade library along with essential dependencies like Ray (with default, rllib, tune, and serve features), Optuna, and TA. This ensures the environment is set up for efficient trading strategy development and execution.

```python
!pip install --upgrade ray[default,rllib,tune,serve] optuna tensortrade ta
```

--------------------------------

### Install TensorTrade Requirements

Source: https://github.com/tensortrade-org/tensortrade/blob/master/README.md

Installs the necessary Python dependencies for TensorTrade, either the base requirements or those needed for running examples.

```bash
pip install -r requirements.txt
pip install -r examples/requirements.txt
```

--------------------------------

### Create and Run a TensorTrade Trading Environment

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/ledger_example.ipynb

Constructs a TensorTrade trading environment using the configured portfolio and data feed. It employs a SimpleOrders action scheme and SimpleProfit reward scheme, then simulates trading steps by taking random actions until the environment is done.

```python
feed = DataFeed([
    Stream.source(list(bitstamp_eth['volume'][-100:]), dtype="float").rename("volume:/USD-ETH"),
    Stream.source(list(bitstamp_ltc['volume'][-100:]), dtype="float").rename("volume:/USD-LTC")
])

env = default.create(
    portfolio=portfolio,
    action_scheme=default.actions.SimpleOrders(),
    reward_scheme=default.rewards.SimpleProfit(),
    feed=feed
)

done = False
obs = env.reset()
while not done:
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)
```

--------------------------------

### Install TensorTrade

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/train_and_evaluate.ipynb

Installs the TensorTrade library from its GitHub repository using pip. This command ensures you have the latest version for your project.

```python
#!python3 -m pip install git+https://github.com/tensortrade-org/tensortrade.git
```

--------------------------------

### TensorTrade Process Setup Information

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This log entry shows an informational message from the TensorTrade project, indicating the completion of the 'Trainable.setup' phase and the time it took. It also provides a suggestion to use 'reuse_actors=True' for improving actor creation overheads if the trainable component is slow to initialize.

```python
2022-02-02 18:06:09,198	INFO trainable.py:127 -- Trainable.setup took 56.215 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
```

--------------------------------

### Import TensorTrade Modules

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/ledger_example.md

Imports necessary modules and classes from the TensorTrade library for environment setup, data handling, wallets, exchanges, and instruments. These are foundational for building trading strategies.

```python
import tensortrade.env.default as default

from tensortrade.feed.core import Stream, DataFeed
from tensortrade.data.cdd import CryptoDataDownload
from tensortrade.oms.wallets import Portfolio, Wallet
from tensortrade.oms.exchanges import Exchange
from tensortrade.oms.services.execution.simulated import execute_order
from tensortrade.oms.instruments import USD, BTC, ETH, LTC
```

--------------------------------

### TensorTrade DQNAgent Setup in Python

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/train_and_evaluate.ipynb

Demonstrates the setup of a DQNAgent in TensorTrade, including importing necessary modules, defining data streams, and initializing the agent with various reward functions and exchange options. It uses streams for price data and defines exchange options with commission.

```python
import tensortrade.env.default as default

from tensortrade.agents import DQNAgent
from tensortrade.feed.core import DataFeed, Stream
from tensortrade.feed.core.base import NameSpace
from tensortrade.env.default.actions import BSH
from tensortrade.env.default.rewards import RiskAdjustedReturns, SimpleProfit
from tensortrade.oms.exchanges import Exchange, ExchangeOptions
from tensortrade.oms.services.execution.simulated import execute_order
from tensortrade.oms.instruments import USD, BTC, ETH
from tensortrade.oms.wallets import Wallet, Portfolio
from tensortrade.oms.orders import TradeType

# TODO: adjust according to your commission percentage, if present
commission = 0.001
price = Stream.source(list(X_train["close"]), 
                      dtype="float").rename("USD-BTC")
#bitstamp_options = ExchangeOptions(commission=commission)
#bitstamp = Exchange("bitstamp", 
#                    service=execute_order,

```

--------------------------------

### Python Log Output: Training Setup and GPU Monitoring

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This log output indicates the completion of the trainable setup phase, noting the time taken and suggesting the use of 'reuse_actors=True' to optimize actor creation overhead. It also warns about the absence of 'gputil' for GPU monitoring.

```python
2022-02-02 16:02:22,667	INFO trainable.py:127 -- Trainable.setup took 56.400 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.

```

```python
2022-02-02 16:02:22,668	WARNING util.py:57 -- Install gputil for GPU system monitoring.

```

--------------------------------

### Python INFO: Trainable Setup Time

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This log entry provides information about the time taken for the `Trainable.setup` method to complete. It suggests that if the trainable is slow to initialize, setting `reuse_actors=True` can help reduce overheads associated with actor creation. This is a performance optimization tip for distributed training environments.

```python
class Trainable:
    def setup(self):
        # Simulate a slow initialization process
        import time
        time.sleep(60)
        print("Trainable setup complete.")

# Example usage:
trainable_instance = Trainable()
trainable_instance.setup()

# To potentially optimize:
# trainable_instance.setup(reuse_actors=True)

```

--------------------------------

### Build TensorTrade Documentation

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/README.md

Commands to build the TensorTrade documentation locally. Requires installing dependencies first. The output is generated in the 'build' directory.

```shell
make sync
make docs-build
```

```shell
make html
```

--------------------------------

### Ray Tune Performance Warning and Setup Information

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This snippet displays logging output from the Ray Tune framework, a popular library for hyperparameter tuning. It includes a warning about the event loop being backlogged, suggesting potential performance bottlenecks in result reporting, and an informational message about the Trainable.setup time, recommending `reuse_actors=True` for faster initialization.

```python
# Ray Tune performance warning and setup information
# WARNING ray_trial_executor.py:771 -- Over the last 60 seconds, the Tune event loop has been backlogged processing new results. Consider increasing your period of result reporting to improve performance.
# INFO trainable.py:127 -- Trainable.setup took 43.620 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
# WARNING util.py:57 -- Install gputil for GPU system monitoring.
```

--------------------------------

### Python WARNING: Install gputil for GPU Monitoring

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This warning message advises users to install the `gputil` library to enable GPU system monitoring. This is a common requirement for machine learning frameworks that leverage GPUs for accelerated computation, allowing users to track GPU utilization, memory, and temperature.

```bash
pip install gputil
```

--------------------------------

### TensorTrade Logging Configuration

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This snippet demonstrates how to configure logging for the TensorTrade project. It shows example log messages indicating informational and warning messages, along with process and timestamp information. It also mentions a dependency on 'gputil' for GPU monitoring.

```python
(PPO pid=20808) 2022-02-02 19:30:16,920	INFO trainable.py:127 -- Trainable.setup took 34.348 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
(PPO pid=20808) 2022-02-02 19:30:16,920	WARNING util.py:57 -- Install gputil for GPU system monitoring.

```

--------------------------------

### Install Ray and Symfit

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/tutorials/ray.md

Installs the necessary Ray and Symfit libraries using pip. Ray is used for distributed computing, and Symfit is for symbolic fitting.

```bash
!pip install ray==0.8.7
!pip install symfit
```

--------------------------------

### TensorTrade GPU Monitoring Suggestion

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This warning suggests installing the 'gputil' library for enhanced GPU system monitoring. It's a helpful tip for users who want to track GPU performance during training.

```python
(PPO pid=13340) 2022-02-02 16:58:27,751\tWARNING util.py:57 -- Install gputil for GPU system monitoring.
```

--------------------------------

### TensorTrade Trainable Setup Time

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Logs indicating the time taken for 'Trainable.setup' in TensorTrade. It suggests setting 'reuse_actors=True' if the trainable component is slow to initialize to reduce overhead.

```python
2022-02-02 18:52:38,458	INFO trainable.py:127 -- Trainable.setup took 53.457 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
```

--------------------------------

### Copy Ledger Transactions to Clipboard

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/ledger_example.ipynb

Exports the system's transaction ledger to a pandas DataFrame and copies it to the system clipboard for easy pasting into spreadsheet software. Requires pandas and a system with clipboard access.

```python
portfolio.ledger.as_frame().to_clipboard(index=False)
```

--------------------------------

### TensorTrade GPU Monitoring Warning

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

A warning from TensorTrade suggesting the installation of 'gputil' for GPU system monitoring. This is relevant for users who want to track GPU performance during training.

```python
2022-02-02 18:52:38,459	WARNING util.py:57 -- Install gputil for GPU system monitoring.
```

--------------------------------

### Install Ray with RLLib and Tune

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/train_and_evaluate_using_ray.md

Installs the Ray library with default features, RLLib for reinforcement learning, and Tune for hyperparameter tuning, specifying version 1.8.0.

```console
pip install ray[default,rllib,tune]==1.8.0
```

--------------------------------

### Add Technical Indicators and Define DataFeed

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/setup_environment_tutorial.md

Applies all technical analysis features from the 'ta' library to the Bitfinex BTC and ETH data. It then defines streams for these features and includes them in a namespace, preparing them for use in a TensorTrade DataFeed.

```python
# Add all features for bitstamp BTC & ETH
bitfinex_btc = bitfinex_data.loc[:, [name.startswith("BTC") for name in bitfinex_data.columns]]
bitfinex_eth = bitfinex_data.loc[:, [name.startswith("ETH") for name in bitfinex_data.columns]]

ta.add_all_ta_features(
    bitfinex_btc,
    colprefix="BTC:",
    **{k: "BTC:" + k for k in ['open', 'high', 'low', 'close', 'volume']}
)


with NameSpace("bitfinex"):
    bitfinex_streams = [Stream.source(list(bitfinex_btc[c]), dtype="float").rename(c) for c in bitfinex_btc.columns]
    bitfinex_streams += [
        Stream.source(list(bitfinex_eth[c]), dtype="float").rename(c) for c in bitfinex_eth.columns
    ]
```

--------------------------------

### Create PPO Agent with Dense Networks (Tensorforce)

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/agents/overview.md

Creates a PPO agent using the Tensorforce library, configuring it with a PPO agent type, Adam optimizer, and dense network layers. This example illustrates the flexibility in defining agent specifications and network architectures.

```python
from tensorforce.agents import Agent

agent_spec = {
    "type": "ppo_agent",
    "step_optimizer": {
        "type": "adam",
        "learning_rate": 1e-4
    },
    "discount": 0.99,
    "likelihood_ratio_clipping": 0.2,
}

network_spec = [
    dict(type='dense', size=64, activation="tanh"),
    dict(type='dense', size=32, activation="tanh")
]

agent = Agent.from_spec(spec=agent_spec,
                        kwargs=dict(network=network_spec,
                                    states=environment.states,
                                    actions=environment.actions))
```

--------------------------------

### TensorTrade Training Initialization Log

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This log entry indicates that the Trainable.setup process for the PPO agent took 50.076 seconds. It suggests that if initialization is slow, setting reuse_actors=True might help reduce overhead.

```python
(PPO pid=13340) 2022-02-02 16:58:27,750\tINFO trainable.py:127 -- Trainable.setup took 50.076 seconds. If your trainable is slow to initialize, consider setting reuse_actors=True to reduce actor creation overheads.
```

--------------------------------

### TensorTrade GPU Monitoring Warning

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This log snippet contains a warning message from the TensorTrade project suggesting the installation of 'gputil' for enhanced GPU system monitoring. This is relevant for users who want to track and manage GPU performance during training or inference.

```python
2022-02-02 18:06:09,199	WARNING util.py:57 -- Install gputil for GPU system monitoring.
```

--------------------------------

### Display Head of Bitfinex Data

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/setup_environment_tutorial.ipynb

Shows the first 5 rows of the `bitfinex_data` DataFrame, providing a preview of the downloaded and concatenated cryptocurrency price data, including columns for date, open, high, low, close, and volume for BTC and ETH.

```python
bitfinex_data.head()
```

--------------------------------

### Install ipywidgets

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/renderers_and_plotly_chart.md

Installs the ipywidgets library, which is required for interactive features in Jupyter environments.

```shell
!pip install ipywidgets
```

--------------------------------

### Install Data Fetching and TA Libraries

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/train_and_evaluate_using_ray.md

Installs yfinance for fetching stock data and pandas-ta for adding technical indicators. Note the '--pre' flag for pandas-ta due to its pre-release status.

```console
pip install yfinance==0.1.64
pip install pandas-ta==0.3.14b --pre
```

--------------------------------

### Prepare and Fetch Cryptocurrency Data (Python)

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Fetches hourly BTC data from Bitfinex, cleans it, and prepares it for analysis. Includes data cleaning, type conversion, sorting, and date formatting.

```python
from tensortrade.data.cdd import CryptoDataDownload

import pandas as pd
import numpy as np

def prepare_data(df):
    df['volume'] = np.int64(df['volume'])
    df['date'] = pd.to_datetime(df['date'])
    df.sort_values(by='date', ascending=True, inplace=True)
    df.reset_index(drop=True, inplace=True)
    df['date'] = df['date'].dt.strftime('%Y-%m-%d %I:%M %p')
    return df

def fetch_data():
    cdd = CryptoDataDownload()
    bitfinex_data = cdd.fetch("Bitfinex", "USD", "BTC", "1h")
    bitfinex_data = bitfinex_data[['date', 'open', 'high', 'low', 'close', 'volume']]
    bitfinex_data = prepare_data(bitfinex_data)
    return bitfinex_data

def load_csv(filename):
    df = pd.read_csv('data/' + filename, skiprows=1)
    df.drop(columns=['symbol', 'volume_btc'], inplace=True)

    # Fix timestamp from "2019-10-17 09-AM" to "2019-10-17 09-00-00 AM"
    df['date'] = df['date'].str[:14] + '00-00 ' + df['date'].str[-2:]

    return prepare_data(df)
```

--------------------------------

### PPO Training Results

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Details the performance metrics and statistics for a PPO agent's training run. Includes episode rewards, timesteps, and evaluation metrics.

```yaml
Result:\n<IPython.core.display.HTML object>\n
```

--------------------------------

### Import Libraries for TensorTrade

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/setup_environment_tutorial.md

Imports essential Python libraries for data manipulation (pandas), technical analysis (ta), and TensorTrade's core components like environment defaults, data downloading, streams, instruments, wallets, and exchanges.

```python
import ta

import pandas as pd
import tensortrade.env.default as default

from tensortrade.data.cdd import CryptoDataDownload
from tensortrade.feed.core import Stream, DataFeed, NameSpace
from tensortrade.oms.instruments import USD, BTC, ETH, LTC
from tensortrade.oms.wallets import Wallet, Portfolio
from tensortrade.oms.exchanges import Exchange
from tensortrade.oms.services.execution.simulated import execute_order
```

--------------------------------

### Installing TensorTrade Test Requirements

Source: https://github.com/tensortrade-org/tensortrade/blob/master/CONTRIBUTING.md

Installs the specific Python packages required for running TensorTrade's tests, including development and testing dependencies.

```bash
pip install -e .[tests]
```

--------------------------------

### Restore PPO Agent with Torch (TensorTrade)

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/agents/overview.md

Restores a PPO agent trained with PyTorch within the TensorTrade framework. Requires a checkpoint path for loading the agent's state. This example demonstrates agent restoration and is part of a larger RL workflow.

```python
agent = ppo.PPOTrainer(
    env="TradingEnv",
    config={
        "env_config": {
            "window_size": 25
        },
        "framework": "torch",
        "log_level": "DEBUG",
        "ignore_worker_failures": True,
        "num_workers": 1,
        "num_gpus": 0,
        "clip_rewards": True,
        "lr": 8e-6,
        "lr_schedule": [
            [0, 1e-1],
            [int(1e2), 1e-2],
            [int(1e3), 1e-3],
            [int(1e4), 1e-4],
            [int(1e5), 1e-5],
            [int(1e6), 1e-6],
            [int(1e7), 1e-7]
        ],
        "gamma": 0,
        "observation_filter": "MeanStdFilter",
        "lambda": 0.72,
        "vf_loss_coeff": 0.5,
        "entropy_coeff": 0.01
    }
)
agent.restore(checkpoint_path)
```

--------------------------------

### Running TensorTrade Test Suite

Source: https://github.com/tensortrade-org/tensortrade/blob/master/CONTRIBUTING.md

This command executes the entire test suite for the TensorTrade project locally. Ensure you have the project cloned and necessary dependencies installed.

```bash
py.test tests/
```

--------------------------------

### Ray Dashboard Initialization and Environment Registration (Python)

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Initializes the Ray cluster for distributed computing and registers the custom trading environment with Ray Tune. This allows for hyperparameter tuning and distributed training of trading agents.

```python
ray.init(num_cpus=6,
         include_dashboard=True,
         address=None,  # set `address=None` to train on laptop
         ignore_reinit_error=True)

register_env("TradingEnv", create_env)

```

--------------------------------

### Training Environment Configuration

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Specifies key configuration parameters for the training environment, such as window sizes for sample and reward calculation, maximum allowed loss, and the CSV filename for the training dataset.

```python
env_config_training = {
    "window_size": 14,  # The number of past samples to look at (hours)
    "reward_window_size": 7,  # And calculate reward based on the actions taken in the next n hours
    "max_allowed_loss": 0.10,  # If it goes past 90% loss during the iteration, we don't want to waste time on a "loser".
    "csv_filename": train_csv  # The variable that will be used to differentiate training and validation datasets
}
```

--------------------------------

### Advance DataFeed to Next Timestamp

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/setup_environment_tutorial.md

Advances the `DataFeed` to the next available timestamp, processing the data for all combined streams. This is a fundamental operation for iterating through time-series data in the `tensortrade` framework.

```python
feed.next()
```

--------------------------------

### Define TensorTrade Exchanges

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/setup_environment_tutorial.md

Defines exchange objects for Bitfinex and Bitstamp using a simulated execution service. Each exchange is configured with streams of price data for specific trading pairs (e.g., USD-BTC, USD-ETH, USD-LTC) derived from the fetched historical data.

```python
bitfinex = Exchange("bitfinex", service=execute_order)(
    Stream.source(list(bitfinex_data['BTC:close']), dtype="float").rename("USD-BTC"),
    Stream.source(list(bitfinex_data['ETH:close']), dtype="float").rename("USD-ETH")
)

bitstamp = Exchange("bitstamp", service=execute_order)(
    Stream.source(list(bitstamp_data['BTC:close']), dtype="float").rename("USD-BTC"),
    Stream.source(list(bitstamp_data['LTC:close']), dtype="float").rename("USD-LTC")
)
```

--------------------------------

### PPO Agent Training Output

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

Logs and output from a PPO agent during training, including system information, learner statistics, and sampler performance. Also contains runtime warnings from the 'ta' library.

```log
Output:\nResult for PPO_TradingEnv_c593ccd8:\n  agent_timesteps_total: 16000\n  custom_metrics: {}\n  date: 2022-02-02_16-57-21\n  done: true\n  episode_len_mean: 4073.5\n  episode_media: {}\n  episode_reward_max: -1541.2399999999898\n  episode_reward_mean: -1830.9300000000007\n  episode_reward_min: -2120.6200000000117\n  episodes_this_iter: 0\n  episodes_total: 2\n  evaluation:\n    custom_metrics: {}\n    episode_len_mean: 4412.5\n    episode_media: {}\n    episode_reward_max: 3798.1599999999835\n    episode_reward_mean: -4481.252999999969\n    episode_reward_min: -7106.719999999949\n    episodes_this_iter: 10\n    hist_stats:\n      episode_lengths:\n      - 5094\n      - 4467\n      - 4368\n      - 4345\n      - 4535\n      - 4361\n      - 4168\n      - 4339\n      - 4513\n      - 3935\n      episode_reward:\n      - 3798.1599999999835\n      - -4447.079999999978\n      - -5388.619999999995\n      - -5629.999999999987\n      - -4087.399999999978\n      - -5315.98999999996\n      - -6141.239999999954\n      - -5892.5999999999385\n      - -4601.039999999932\n      - -7106.719999999949\n    off_policy_estimator: {}\n    policy_reward_max: {}\n    policy_reward_mean: {}\n    policy_reward_min: {}\n    sampler_perf:\n      mean_action_processing_ms: 0.16577684639966186\n      mean_env_render_ms: 0.0\n      mean_env_wait_ms: 3.95665093818934\n      mean_inference_ms: 4.965813498513868\n      mean_raw_obs_processing_ms: 0.5631549629014174\n  experiment_id: 3387977cb59d48cc863c4f8ee378db29\n  hostname: 24f35851f6f8\n  info:\n    learner:\n      default_policy:\n        custom_metrics: {}\n        learner_stats:\n          allreduce_latency: 0.0\n          cur_kl_coeff: 0.4500000000000001\n          cur_lr: 9.800000000000001e-05\n          entropy: 0.5085176501863746\n          entropy_coeff: 0.09352884343369143\n          kl: 0.04640225437193721\n          policy_loss: -0.16024432633322416\n          total_loss: 0.04383458093475921\n          vf_explained_var: 0.7773565402594945\n          vf_loss: 0.4026546392549751\n        model: {}\n    num_agent_steps_sampled: 16000\n    num_agent_steps_trained: 16000\n    num_steps_sampled: 16000\n    num_steps_trained: 16000\n    num_steps_trained_this_iter: 0\n  iterations_since_restore: 4\n  node_ip: 172.28.0.2\n  num_healthy_workers: 2\n  off_policy_estimator: {}\n  perf:\n    cpu_util_percent: 99.77813211845101\n    ram_util_percent: 48.65125284738041\n  pid: 10913\n  policy_reward_max: {}\n  policy_reward_mean: {}\n  policy_reward_min: {}\n  sampler_perf:\n    mean_action_processing_ms: 0.26311805600181504\n    mean_env_render_ms: 0.0\n    mean_env_wait_ms: 6.662405975148401\n    mean_inference_ms: 7.366215165545714\n    mean_raw_obs_processing_ms: 0.8677475315777083\n  time_since_restore: 3298.4378027915955\n  time_this_iter_s: 618.5611255168915\n  time_total_s: 3298.4378027915955\n  timers:\n    learn_throughput: 22.162\n    learn_time_ms: 180486.125\n    load_throughput: 24738.233\n    load_time_ms: 161.693\n    sample_throughput: 5.869\n    sample_time_ms: 681591.098\n    update_time_ms: 17.714\n  timestamp: 1643821041\n  timesteps_since_restore: 0\n  timesteps_this_iter: 0\n  timesteps_total: 16000\n  training_iteration: 4\n  trial_id: c593ccd8\n  \n
```

```log
Output:\n(PPO pid=13340) 2022-02-02 16:57:38,193\tINFO ppo.py:167 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.\n(PPO pid=13340) 2022-02-02 16:57:38,193\tINFO trainer.py:745 -- Current log_level is ERROR. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.\n(RolloutWorker pid=13371) /usr/local/lib/python3.7/dist-packages/ta/trend.py:769: RuntimeWarning: invalid value encountered in double_scalars\n(RolloutWorker pid=13371)   dip[idx] = 100 * (self._dip[idx] / value)\n(RolloutWorker pid=13371) /usr/local/lib/python3.7/dist-packages/ta/trend.py:774: RuntimeWarning: invalid value encountered in double_scalars\n(RolloutWorker pid=13371)   din[idx] = 100 * (self._din[idx] / value)\n(RolloutWorker pid=13370) /usr/local/lib/python3.7/dist-packages/ta/trend.py:769: RuntimeWarning: invalid value encountered in double_scalars\n(RolloutWorker pid=13370)   dip[idx] = 100 * (self._dip[idx] / value)\n(RolloutWorker pid=13370) /usr/local/lib/python3.7/dist-packages/ta/trend.py:774: RuntimeWarning: invalid value encountered in double_scalars\n(RolloutWorker pid=13370)   din[idx] = 100 * (self._din[idx] / value)\n(PPO pid=13340) /usr/local/lib/python3.7/dist-packages/ta/trend.py:769: RuntimeWarning: invalid value encountered in double_scalars\n(PPO pid=13340)   dip[idx] = 100 * (self._dip[idx] / value)\n
```

--------------------------------

### Configuring and Loading RLlib PPO Agent Checkpoint

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This Python snippet demonstrates how to retrieve the best performing trial checkpoint and its configuration from an RLlib analysis object. It then updates the environment configuration for validation and prepares the agent for evaluation.

```python
import ray.rllib.agents.ppo as ppo

# Get checkpoint
checkpoints = analysis.get_trial_checkpoints_paths(
    trial=analysis.get_best_trial(checkpoint_metric, mode='max'),
    metric=checkpoint_metric
)
checkpoint_path = checkpoints[0][0]

env_config_validation = {
    "window_size": 14,  # The number of past samples to look at (hours)
    "reward_window_size": 7,  # And calculate reward based on the actions taken in the next n hours
    "max_allowed_loss": 1.0,  # Allow 100% loss during evaluation
    "csv_filename": valid_csv  # The variable that will be used to differentiate training and validation datasets
}

config = analysis.get_best_config(checkpoint_metric, mode='max')
config['env_config'] = env_config_validation
```

--------------------------------

### Initialize SimpleProfit Reward Scheme

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/components/reward_scheme.md

Demonstrates how to initialize the SimpleProfit reward scheme, a basic implementation that rewards the agent for profitable trades. It's a common starting point for defining reward mechanisms.

```python
from tensortrade.env.default.rewards import SimpleProfit

reward_scheme = SimpleProfit()
```

--------------------------------

### PPO Trainer Initialization

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This log message indicates that the Proximal Policy Optimization (PPO) trainer is initializing in multi-agent mode. It provides a recommendation to set 'simple_optimizer=True' if the default multi-GPU optimizer configuration causes issues.

```python
# (PPO pid=20808) 2022-02-02 19:29:43,391	INFO ppo.py:167 -- In multi-agent mode, policies will be optimized sequentially by the multi-GPU optimizer. Consider setting simple_optimizer=True if this doesn't work for you.
```

--------------------------------

### PPO Trading Environment Results Summary

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This detailed output provides a comprehensive summary of the PPO agent's performance in the TradingEnv. It includes agent timesteps, evaluation metrics like episode reward means and lengths, and performance timers.

```yaml
Output:\nResult for PPO_TradingEnv_5293e328:\n  agent_timesteps_total: 4000\n  custom_metrics: {}\n  date: 2022-02-02_17-00-04\n  done: false\n  episode_len_mean: .nan\n  episode_media: {}\n  episode_reward_max: .nan\n  episode_reward_mean: .nan\n  episode_reward_min: .nan\n  episodes_this_iter: 0\n  episodes_total: 0\n  evaluation:\n    custom_metrics: {}\n    episode_len_mean: 9725.1\n    episode_media: {}\n    episode_reward_max: 18409.100000000042\n    episode_reward_mean: 15470.147999999997\n    episode_reward_min: 11640.51999999997\n    episodes_this_iter: 10\n    hist_stats:\n      episode_lengths:\n      - 8994\n      - 9698\n      - 10147\n      - 9907\n      - 9720\n      - 9901\n      - 9720\n      - 9722\n      - 9721\n      - 9721\n      episode_reward:\n      - 11640.51999999997\n      - 15632.83999999997\n      - 18409.100000000042\n      - 16626.839999999993\n      - 15333.18000000001\n      - 16302.600000000006\n      - 15083.060000000003\n      - 15165.100000000006\n      - 15228.720000000005\n      - 15279.520000000004\n    off_policy_estimator: {}\n    policy_reward_max: {}\n    policy_reward_mean: {}\n    policy_reward_min: {}\n    sampler_perf:\n      mean_action_processing_ms: 0.1723678119973066\n      mean_env_render_ms: 0.0\n      mean_env_wait_ms: 5.388293071937147\n      mean_inference_ms: 5.146528417569002\n      mean_raw_obs_processing_ms: 0.6234363056693255\n  experiment_id: 037203a595b8489fa364049de1a8e6f2\n  hostname: 24f35851f6f8\n  info:\n    learner:\n      default_policy:\n        custom_metrics: {}\n        learner_stats:\n          allreduce_latency: 0.0\n          cur_kl_coeff: 0.20000000000000004\n          cur_lr: 0.10000000000000002\n          entropy: 0.6411102289794594\n          entropy_coeff: 0.01385129645863369\n          kl: 0.05455051675160759\n          policy_loss: -0.14480916934997914\n          total_loss: 0.6273927934347622\n          vf_explained_var: 0.22125849025223845\n          vf_loss: 0.8558271934909205\n        model: {}\n    num_agent_steps_sampled: 4000\n    num_agent_steps_trained: 4000\n    num_steps_sampled: 4000\n    num_steps_trained: 4000\n  iterations_since_restore: 1\n  node_ip: 172.28.0.2\n  num_healthy_workers: 2\n  off_policy_estimator: {}\n  perf:\n    cpu_util_percent: 99.77269293038316\n    ram_util_percent: 48.370210469508905\n  pid: 12621\n  policy_reward_max: {}\n  policy_reward_mean: {}\n  policy_reward_min: {}\n  sampler_perf: {}\n  time_since_restore: 1304.4702134132385\n  time_this_iter_s: 1304.4702134132385\n  time_total_s: 1304.4702134132385\n  timers:\n    learn_throughput: 23.571\n    learn_time_ms: 169699.034\n    load_throughput: 31066.042\n    load_time_ms: 128.758\n    sample_throughput: 91.635\n    sample_time_ms: 43651.273\n    update_time_ms: 14.615\n  timestamp: 1643821204\n  timesteps_since_restore: 0\n  timesteps_this_iter: 0\n  timesteps_total: 4000\n  training_iteration: 1\n  trial_id: 5293e328\n  
```

--------------------------------

### Initialize PPO2 Agent with MLP LN LSTM Policy (Stable Baselines)

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/agents/overview.md

Initializes a PPO2 agent using the MlpLnLstmPolicy from the Stable Baselines library. This code snippet shows how to set up the agent with a specified policy and learning rate for a given environment.

```python
from stable_baselines.common.policies import MlpLnLstmPolicy
from stable_baselines import PPO2

model = PPO2
policy = MlpLnLstmPolicy
params = { "learning_rate": 1e-5 }

agent = model(policy, environment, model_kwargs=params)
```

--------------------------------

### PPO Sampler Performance Metrics

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This section details the performance of the sampler used by the PPO agent. It includes average times for action processing, environment waiting, and inference, which are crucial for understanding the efficiency of data collection.

```python
sampler_perf:
    mean_action_processing_ms: 0.27595963245271704
    mean_env_render_ms: 0.0
    mean_env_wait_ms: 7.572952478533089
    mean_inference_ms: 8.31185730471847
    mean_raw_obs_processing_ms: 0.988462372156724
```

--------------------------------

### Inspect Transactions with ManagedRiskOrders

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/ledger_example.md

Demonstrates setting up a trading environment using the ManagedRiskOrders action scheme. It initializes a portfolio and simulates trading steps, then displays the transaction ledger. This scheme is suitable for managing risk within trading operations.

```python
portfolio = Portfolio(USD, [
    Wallet(bitfinex, 10000 * USD),
    Wallet(bitfinex, 0 * BTC),
    Wallet(bitfinex, 0 * ETH),
])

env = default.create(
    portfolio=portfolio,
    action_scheme=default.actions.ManagedRiskOrders(),
    reward_scheme=default.rewards.SimpleProfit(),
    feed=feed
)

done = False

while not done:
    action = env.action_space.sample()
    obs, reward, done, info = env.step(action)

portfolio.ledger.as_frame().head(20)
```

--------------------------------

### PPO Timers and Throughput

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This snippet details the timing metrics for PPO training, specifically focusing on learning and loading throughput and time. These metrics help in diagnosing performance bottlenecks during the training process.

```python
timers:
    learn_throughput: 21.038
    learn_time_ms: 190136.605
    load_throughput: 26575.93
    load_time_ms: 150.512
```

--------------------------------

### Restore Trained Ray Agent

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/agents/overview.md

Retrieves the checkpoint path for the best performing trial from a Ray Tune run and prepares it for restoring a trained agent. This allows for resuming training or using the agent for inference.

```python
import ray.rllib.agents.ppo as ppo

# Get checkpoint
checkpoints = analysis.get_trial_checkpoints_paths(
    trial=analysis.get_best_trial("episode_reward_mean"),
    metric="episode_reward_mean"
)
checkpoint_path = checkpoints[0][0]

```

--------------------------------

### Setup and Train DQN Agent with Visual Trades

Source: https://github.com/tensortrade-org/tensortrade/blob/master/docs/source/examples/renderers_and_plotly_chart.md

This snippet prepares a DQN agent for training and initiates the training process, noting that the generated chart will visually represent trades with arrows.

```python
from tensortrade.agents import DQNAgent

agent = DQNAgent(env)


```

--------------------------------

### PPO Training Iteration Summary

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This summary provides key metrics for a PPO training iteration. It includes the total number of agent steps sampled and trained, current iteration number, and overall time spent during training.

```python
agent_timesteps_total: 12000
  custom_metrics: {}
  date: 2022-02-02_19-29-48
  done: false
  episode_len_mean: 4262.0
  episode_media: {}
  episode_reward_max: 1528.7000000000162
  episode_reward_mean: -443.0099999999743
  episode_reward_min: -2414.719999999965
  episodes_this_iter: 2
  episodes_total: 2
  iterations_since_restore: 3
  num_agent_steps_sampled: 12000
  num_agent_steps_trained: 12000
  time_since_restore: 2229.5579912662506
  time_this_iter_s: 716.8894312381744
  time_total_s: 2229.5579912662506
```

--------------------------------

### PPO Evaluation Metrics

Source: https://github.com/tensortrade-org/tensortrade/blob/master/examples/use_lstm_rllib.ipynb

This comprehensive block outlines the evaluation results for the PPO agent. It includes mean and max episode rewards, episode lengths, and detailed statistics for sampled episodes, offering a performance benchmark.

```python
evaluation:
    custom_metrics: {}
    episode_len_mean: 4564.3
    episode_media: {}
    episode_reward_max: 1291.2599999999438
    episode_reward_mean: -1391.7140000000218
    episode_reward_min: -3483.740000000018
    episodes_this_iter: 10
    hist_stats:
      episode_lengths:
      - 4672
      - 4443
      - 4512
      - 4437
      - 4414
      - 4571
      - 4571
      - 4679
      - 4672
      - 4672
      episode_reward:
      - 1291.2599999999438
      - -1394.2600000000548
      - -886.9400000000551
      - -2853.000000000018
      - -3483.740000000018
      - -2510.959999999999
      - -1293.6800000000057
      - -819.2599999999984
      - -914.2800000000061
      - -1052.280000000006
```