### Agent Setup with Arms and Policy

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/quickstart.rst.txt

Initialize an agent with two arms, each using a NormalInverseGammaRegressor, and a ThompsonSampling policy. This setup is for learning the mean and variance of profit for each ad.

```python
import numpy as np
from bayesianbandits import Agent, Arm, NormalInverseGammaRegressor, ThompsonSampling

agent = Agent(
    arms=[
        Arm("ad_a", learner=NormalInverseGammaRegressor()),
        Arm("ad_b", learner=NormalInverseGammaRegressor()),
    ],
    policy=ThompsonSampling(),
)
```

--------------------------------

### Hybrid Bandit Example

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Demonstrates a basic hybrid bandit setup. This snippet is useful for understanding the core mechanics of hybrid bandits.

```python
from typing import List

import numpy as np


class HybridBandit:
    def __init__(self, n_arms: int, alpha: float = 1.0, beta: float = 1.0):
        self.n_arms = n_arms
        self.alpha = alpha
        self.beta = beta
        self.counts = np.zeros(n_arms, dtype=int)
        self.values = np.zeros(n_arms)

    def select_arm(self) -> int:
        # Sample from the Beta distribution for each arm
        sampled_theta = np.random.beta(self.alpha + self.counts, self.beta + self.counts)
        # Select the arm with the highest sampled theta
        return np.argmax(sampled_theta)

    def update(self, chosen_arm: int, reward: float):
        self.counts[chosen_arm] += 1
        n = self.counts[chosen_arm]
        old_value = self.values[chosen_arm]
        new_value = old_value + (reward - old_value) / n
        self.values[chosen_arm] = new_value


def run_hybrid_bandit(n_arms: int, n_steps: int, true_probs: List[float], alpha: float = 1.0, beta: float = 1.0) -> List[float]:
    bandit = HybridBandit(n_arms, alpha, beta)
    rewards = []
    for _ in range(n_steps):
        chosen_arm = bandit.select_arm()
        reward = np.random.binomial(1, true_probs[chosen_arm])
        bandit.update(chosen_arm, reward)
        rewards.append(reward)
    return rewards


if __name__ == '__main__':
    n_arms = 5
    n_steps = 1000
    true_probs = [0.1, 0.2, 0.3, 0.4, 0.5]
    rewards = run_hybrid_bandit(n_arms, n_steps, true_probs)
    print(f'Total reward: {sum(rewards)}')
    # Example with different alpha and beta values
    rewards_biased = run_hybrid_bandit(n_arms, n_steps, true_probs, alpha=2.0, beta=2.0)
    print(f'Total reward (biased alpha/beta): {sum(rewards_biased)}')
```

--------------------------------

### Install bayesian-bandits

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/quickstart.rst.txt

Install the library using pip. For CHOLMOD support, ensure SuiteSparse is installed and use the optional dependency.

```bash
pip install -U bayesianbandits
```

```bash
pip install -U bayesianbandits[cholmod]
```

--------------------------------

### Setup Agent with NormalInverseGammaRegressor

Source: https://bayesianbandits.readthedocs.io/en/latest/quickstart.html

Initialize an agent with two arms, each using a NormalInverseGammaRegressor to learn profit, and a ThompsonSampling policy.

```python
import numpy as np
from bayesianbandits import Agent, Arm, NormalInverseGammaRegressor, ThompsonSampling

agent = Agent(
    arms=[
        Arm("ad_a", learner=NormalInverseGammaRegressor()),
        Arm("ad_b", learner=NormalInverseGammaRegressor()),
    ],
    policy=ThompsonSampling(),
)

```

--------------------------------

### Initialize ContextualBandit with LinUCB

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the LinUCB policy. This is a common setup for evaluating contextual bandit algorithms.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.policies import LinUCB

boundit = ContextualBandit(policy=LinUCB(n_arms=5, n_features=10), n_arms=5)
```

--------------------------------

### Example Output of Performance Metrics (Text)

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/adversarial.html

This is an example output showing the calculated performance metrics for UCB and EXP3A against a biased opponent. It includes cumulative rewards, paper play rates, and rounds to reach specific play rate thresholds.

```text
=== Learning Performance Against Biased Opponent (70% Rock) ===

Final cumulative rewards (2000 rounds):
  Optimal (Always Paper):    1192
  UCB:                       1208 (101.3% of optimal)
  EXP3A:                      750 (62.9% of optimal)

Final 100 rounds Paper play rate:
  UCB:   100.0%
  EXP3A: 70.0%
  (Optimal against 70% Rock is to play Paper frequently)

Rounds to reach 50% Paper play rate:
  UCB:   0
  EXP3A: 0

Rounds to reach 60% Paper play rate:
  UCB:   0
  EXP3A: 0

Rounds to reach 70% Paper play rate:
  UCB:   0
  EXP3A: 0


```

--------------------------------

### Install Bayesian Bandits

Source: https://bayesianbandits.readthedocs.io/en/latest/quickstart.html

Install the library using pip. For CHOLMOD support, use the optional dependency.

```bash
pip install -U bayesianbandits

```

```bash
pip install -U bayesianbandits[cholmod]

```

--------------------------------

### Example Output of Best Parameters

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/delayed-reward.html

Illustrates the output format for the best parameters found by Optuna, specifically the optimal decay rate and its associated minimum regret value.

```text
({'decay_rate': 0.997843414009486}, 22657.478449147064)
```

--------------------------------

### Setup for BayesianGLM Contextual Bandit Simulation

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/rvga-glm.html

Initializes arms, featurizers, and agents for a contextual bandit simulation. Requires importing necessary classes from bayesian_bandits and numpy.

```python
from bayesian_bandits import (
    Arm,
    FunctionArmFeaturizer,
    LipschitzContextualAgent,
    ThompsonSampling,
)

N_ARMS = 5
N_FEATURES = 4  # intercept + 3 user features
P = N_ARMS * N_FEATURES

# Fixed true weights per arm
weight_rng = np.random.default_rng(7)
TRUE_WEIGHTS = weight_rng.normal(0.0, 1.0, (N_ARMS, N_FEATURES))


def block_sparse_featurizer(X, action_tokens):
    """Arm a gets the user features in the a-th block of a K*d vector."""
    n_contexts = X.shape[0]
    result = np.zeros((n_contexts, P, len(action_tokens)))
    for i, token in enumerate(action_tokens):
        result[:, token * N_FEATURES : (token + 1) * N_FEATURES, i] = X
    return result


arm_featurizer = FunctionArmFeaturizer(block_sparse_featurizer)


def make_agent(approximator, seed):
    return LipschitzContextualAgent(
        arms=[Arm(action_token=i) for i in range(N_ARMS)],
        learner=BayesianGLM(alpha=1.0, link="logit", approximator=approximator),
        policy=ThompsonSampling(),
        arm_featurizer=arm_featurizer,
        random_seed=np.random.default_rng(seed),
    )


```

--------------------------------

### Install CHOLMOD for Production Workloads

Source: https://bayesianbandits.readthedocs.io/en/latest/howto/sparse.html

Installs the bayesian-bandits library with CHOLMOD support, which is recommended for production workloads due to its efficiency with symmetric positive definite matrices.

```bash
pip install bayesianbandits[cholmod]

```

--------------------------------

### ArmFeaturizer.transform() Example

Source: https://bayesianbandits.readthedocs.io/en/latest/generated/bayesianbandits.ArmFeaturizer.html

Demonstrates the expected output shape and structure when transforming context features for multiple arms using discrete action tokens. The output is stacked, repeating context features for each arm.

```python
>>> import numpy as np
>>> # Discrete action tokens
>>> X = np.array([[1, 2], [3, 4]])  # 2 contexts, 2 features
>>> action_tokens = [0, 1, 2]  # 3 arms
>>> # After transform: 6 rows (2 contexts * 3 arms)
>>> # Rows 0-1: context features + arm 0 features
>>> # Rows 2-3: context features + arm 1 features
>>> # Rows 4-5: context features + arm 2 features

```

--------------------------------

### Setup Bayesian Bandit Agents

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Initializes agents for a contextual bandit problem using BayesianGLM with specified approximators and Thompson Sampling policy. Imports necessary classes and defines constants for the simulation.

```python
from bayesian_bandits import (
    Arm,
    FunctionArmFeaturizer,
    LipschitzContextualAgent,
    ThompsonSampling,
)

N_ARMS = 5
N_FEATURES = 4  # intercept + 3 user features
P = N_ARMS * N_FEATURES

# Fixed true weights per arm
weight_rng = np.random.default_rng(7)
TRUE_WEIGHTS = weight_rng.normal(0.0, 1.0, (N_ARMS, N_FEATURES))


def block_sparse_featurizer(X, action_tokens):
    """Arm a gets the user features in the a-th block of a K*d vector."""
    n_contexts = X.shape[0]
    result = np.zeros((n_contexts, P, len(action_tokens)))
    for i, token in enumerate(action_tokens):
        result[:, token * N_FEATURES : (token + 1) * N_FEATURES, i] = X
    return result


arm_featurizer = FunctionArmFeaturizer(block_sparse_featurizer)


def make_agent(approximator, seed):
    return LipschitzContextualAgent(
        arms=[Arm(action_token=i) for i in range(N_ARMS)],
        learner=BayesianGLM(alpha=1.0, link="logit", approximator=approximator),
        policy=ThompsonSampling(),
        arm_featurizer=arm_featurizer,
        random_seed=np.random.default_rng(seed),
    )
```

--------------------------------

### Initialize ContextualBandit with SoftmaxLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the SoftmaxLearner policy, which uses a softmax distribution for probabilistic arm selection.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import SoftmaxLearner

boundit = ContextualBandit(policy=SoftmaxLearner(n_arms=5, temperature=1.0), n_arms=5)
```

--------------------------------

### Delayed Reward Example

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/delayed-reward.ipynb.txt

Demonstrates a scenario with delayed rewards. This setup is crucial for understanding how agents learn in environments where immediate feedback is not always available.

```python
import numpy as np

# Parameters
num_actions = 5

# Simulate a delayed reward environment
# In this example, only action 3 gives a reward, but it's delayed.
# All other actions give no reward.
def get_reward(action, step):
    if action == 3 and step >= 5:
        return 1.0
    return 0.0

# Simulate agent interaction
num_steps = 10
action_taken = 3 # Agent chooses action 3

rewards = []
for step in range(num_steps):
    reward = get_reward(action_taken, step)
    rewards.append(reward)
    print(f"Step {step}: Action {action_taken}, Reward: {reward}")

print(f"Total reward: {sum(rewards)}")
```

--------------------------------

### Initialize ContextualBandit with DiscountedUCB1Learner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the DiscountedUCB1Learner policy, which adapts UCB1 for non-stationary settings.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import DiscountedUCB1Learner

boundit = ContextualBandit(policy=DiscountedUCB1Learner(n_arms=5, discount_factor=0.9), n_arms=5)
```

--------------------------------

### Import necessary libraries for BayesianGLM

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/rvga-glm.html

Imports libraries for plotting, numerical operations, logistic function, and BayesianGLM with its approximators. This setup is required for running the examples.

```python
import matplotlib.pyplot as plt
import numpy as np
from scipy.special import expit

from bayesian_bandits import BayesianGLM, LaplaceApproximator, RVGAApproximator

rng = np.random.default_rng(3)
```

--------------------------------

### Initialize ContextualBandit with SlidingWindowTSLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the SlidingWindowTSLearner policy, combining Thompson Sampling with a sliding window.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import SlidingWindowTSLearner

boundit = ContextualBandit(policy=SlidingWindowTSLearner(n_arms=5, window_size=100), n_arms=5)
```

--------------------------------

### Initialize Bayesian Bandits Components

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/delayed-reward.html

Sets up the necessary components for the simulation, including arms with specific price points and learners, and a contextual agent with a Thompson Sampling policy. Ensure all required imports are present.

```python
from bayesian_bandits import (
    Arm,
    ContextualAgent,
    NormalInverse রোগেরRegressor,
    ThompsonSampling,
)

est = NormalInverseGammaRegressor(mu=np.array([1000] + [0] * 10))
policy = ThompsonSampling()

arms = [
    Arm(
        Price.price_5,
        learner=NormalInverseGammaRegressor(mu=np.array([1000] + [0] * 10)),
    ),
    Arm(
        Price.price_8,
        learner=NormalInverseGammaRegressor(mu=np.array([1000] + [0] * 10)),
    ),
    Arm(
        Price.price_11,
        learner=NormalInverseGammaRegressor(mu=np.array([1000] + [0] * 10)),
    ),
    Arm(
        Price.price_14,
        learner=NormalInverseGammaRegressor(mu=np.array([1000] + [0] * 10)),
    ),
    Arm(
        Price.price_17,
        learner=NormalInverseGammaRegressor(mu=np.array([1000] + [0] * 10)),
    ),
]

agent = ContextualAgent(arms=arms, policy=policy, random_seed=111)

```

--------------------------------

### Install CHOLMOD for Production

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/howto/sparse.rst.txt

Command to install the bayesianbandits library with CHOLMOD support for optimized sparse matrix operations.

```bash
pip install bayesianbandits[cholmod]
```

--------------------------------

### Initialize Agent with ThompsonSampling

Source: https://bayesianbandits.readthedocs.io/en/latest/generated/bayesianbandits.ThompsonSampling.html

Demonstrates how to initialize an Agent with ThompsonSampling and multiple arms, each using a NormalInverseGammaRegressor learner.

```python
>>> from bayesianbandits import Agent, Arm, NormalInverseGammaRegressor
>>> from bayesianbandits import ThompsonSampling
>>> 
>>> arms = [
...     Arm(f"arm_{i}", learner=NormalInverseGammaRegressor())
...     for i in range(3)
... ]
>>> agent = Agent(arms, ThompsonSampling())
```

--------------------------------

### Delayed Reward Example

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/delayed-reward.ipynb.txt

This example illustrates a scenario with delayed reward, highlighting the need for agents to look ahead to understand the true value of their actions.

```python
import numpy as np

# Define the environment with delayed reward
def delayed_reward_env(state, action):
    if state == 0:
        if action == 0:  # Move right
            return 1, 1, False  # Reward=1, Next State=1, Done=False
        else:  # Stay
            return -1, 0, False # Reward=-1, Next State=0, Done=False
    elif state == 1:
        if action == 0:  # Move right
            return 10, 2, False # Reward=10, Next State=2, Done=False
        else:  # Move left
            return -1, 0, False # Reward=-1, Next State=0, Done=False
    else: # state == 2
        return 0, 0, True # Reward=0, Next State=0, Done=True

# Simple Q-learning agent
def q_learning(env, num_episodes=1000, alpha=0.1, gamma=0.9, epsilon=0.1):
    num_states = 3
    num_actions = 2
    q_table = np.zeros((num_states, num_actions))

    for episode in range(num_episodes):
        state = 0
        done = False
        while not done:
            # Epsilon-greedy action selection
            if np.random.uniform(0, 1) < epsilon:
                action = np.random.randint(num_actions)
            else:
                action = np.argmax(q_table[state, :])

            # Take action and observe reward and next state
            reward, next_state, done = env(state, action)

            # Q-learning update rule
            old_value = q_table[state, action]
            next_max = np.max(q_table[next_state, :])
            new_value = old_value + alpha * (reward + gamma * next_max - old_value)
            q_table[state, action] = new_value

            state = next_state
    return q_table

# Run the Q-learning agent
final_q_table = q_learning(delayed_reward_env)
print("Final Q-table:\n", final_q_table)

```

--------------------------------

### Initialize ContextualBandit with ThompsonSamplingLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the ThompsonSamplingLearner policy, which employs a Bayesian approach for exploration.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import ThompsonSamplingLearner

boundit = ContextualBandit(policy=ThompsonSamplingLearner(n_arms=5, n_features=10), n_arms=5)
```

--------------------------------

### Initialize EXP3A Policy and Agent

Source: https://bayesianbandits.readthedocs.io/en/latest/generated/bayesianbandits.EXP3A.html

Initialize the standard EXP3 policy with specified parameters and create a contextual agent. Use this for setting up the bandit problem with EXP3A.

```python
>>> # Initialize standard EXP3 policy
>>> policy = EXP3A(gamma=0.1, eta=2.0, ix_gamma=0.0)
>>>
>>> # Create agent
>>> agent = ContextualAgent(arms, policy)
```

--------------------------------

### Initialize ContextualBandit with NeuralUCB

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the NeuralUCB policy, which employs neural networks for reward modeling.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.policies import NeuralUCB

boundit = ContextualBandit(policy=NeuralUCB(n_arms=5, n_features=10, hidden_layers=[64, 32]), n_arms=5)
```

--------------------------------

### Initialize ContextualAgent with Arms and Policy

Source: https://bayesianbandits.readthedocs.io/en/latest/generated/bayesianbandits.ContextualAgent.html

Demonstrates how to create a ContextualAgent with two arms and a ThompsonSampling policy. Ensure to import necessary classes like Arm, NormalInverseGammaRegressor, ContextualAgent, and ThompsonSampling.

```python
from bayesianbandits import Arm, NormalInverseGammaRegressor
from bayesianbandits import ContextualAgent, ThompsonSampling
arms = [
    Arm(0, learner=NormalInverseGammaRegressor()),
    Arm(1, learner=NormalInverseGammaRegressor()),
]
agent = ContextualAgent(arms, ThompsonSampling(), random_seed=0)
```

--------------------------------

### EmpiricalBayesDirichletClassifier Example

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/math/dirichlet-eb.rst.txt

Demonstrates fitting the EmpiricalBayesDirichletClassifier with partial fitting and observing the convergence of the base measure.

```python
import numpy as np
from bayesianbandits import EmpiricalBayesDirichletClassifier

rng = np.random.default_rng(42)
true_alpha = np.array([3.0, 1.0])  # 75/25 split

# Wrong initial prior: heavily favors class 2
clf = EmpiricalBayesDirichletClassifier(
    {1: 1.0, 2: 5.0}, random_state=0
)

for g in range(200):
    theta = rng.dirichlet(true_alpha)
    obs = rng.choice([1, 2], size=rng.poisson(10) + 1, p=theta)
    clf.partial_fit(np.full((len(obs), 1), g), obs)

# Recovered base measure: m ≈ [0.75, 0.25]
s = sum(clf.alphas.values())
m = {k: v / s for k, v in clf.alphas.items()}
```

--------------------------------

### Agent (No Context)

Source: https://bayesianbandits.readthedocs.io/en/latest/introduction.html

A convenience wrapper for bandits with no context and independent arms. Each arm gets its own learner with an intercept-only model.

```python
Agent
```

--------------------------------

### Get Model Coefficients

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Retrieves the coefficients of the fitted RVGA GLM. These are crucial for understanding feature importance and model interpretability.

```python
coefficients = model.get_params()
```

--------------------------------

### Hybrid Bandit Example

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Demonstrates a basic hybrid bandit algorithm. This snippet is useful for understanding the core mechanics of hybrid bandits.

```python
from typing import List

import numpy as np

from banditpylib.arms import ConstantArm
from banditpylib.bandits import HybridBandit


def main():
  # Hybrid Bandit
  arms = [ConstantArm(0.1), ConstantArm(0.2), ConstantArm(0.3)]
  hybrid_bandit = HybridBandit(arms=arms)
  # Simulate 1000 rounds
  for _ in range(1000):
    # Choose arm
    chosen_arm = hybrid_bandit.sample_arm()
    # Get reward
    reward = chosen_arm.sample(1)[0]
    # Update bandit
    hybrid_bandit.update(chosen_arm.index, reward)
  # Print the chosen arm counts
  print(hybrid_bandit.counts)

if __name__ == "__main__":
  main()
```

--------------------------------

### Initialize ContextualBandit with EpsilonGreedyLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Creates a ContextualBandit instance configured with the EpsilonGreedyLearner policy, a simple algorithm balancing exploration and exploitation.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import EpsilonGreedyLearner

boundit = ContextualBandit(policy=EpsilonGreedyLearner(n_arms=5, epsilon=0.1), n_arms=5)
```

--------------------------------

### Example Output of Worst Decay Rate

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/delayed-reward.html

Shows the printed output for the worst decay rate identified during the Optuna optimization process.

```text
Worst decay rate: 0.964
```

--------------------------------

### Initialize LinUCBLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a LinUCBLearner, a common algorithm for contextual bandit problems. It requires specifying the number of arms and features.

```python
from banditpylib.learners import LinUCBLearner

learner = LinUCBLearner(n_arms=5, n_features=10)
```

--------------------------------

### Get Model Mean from RVGA GLM

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Retrieves the mean of the fitted RVGA GLM model. This represents the average predicted outcome.

```python
mean_val = rvga.get_mean()
```

--------------------------------

### Initialize ContextualBandit with TSLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the TSLearner policy, which implements Thompson Sampling for bandit problems.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import TSLearner

boundit = ContextualBandit(policy=TSLearner(n_arms=5), n_arms=5)
```

--------------------------------

### Initialize OFULearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Initializes an OFULearner (Optimism in the Face of Uncertainty), an algorithm for contextual bandits that uses confidence bounds to guide exploration.

```python
from banditpylib.learners import OFULearner

learner = OFULearner(n_arms=5, n_features=10, delta=0.1)
```

--------------------------------

### Initialize ContextualBandit with OFULearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Sets up a ContextualBandit environment using the OFULearner policy, which uses optimism in the face of uncertainty for exploration.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import OFULearner

boundit = ContextualBandit(policy=OFULearner(n_arms=5, n_features=10, delta=0.1), n_arms=5)
```

--------------------------------

### Initialize ContextualBandit with SlidingWindowUCB1Learner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Creates a ContextualBandit instance configured with the SlidingWindowUCB1Learner policy, using a sliding window for recent data focus.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import SlidingWindowUCB1Learner

boundit = ContextualBandit(policy=SlidingWindowUCB1Learner(n_arms=5, window_size=100), n_arms=5)
```

--------------------------------

### Fit a GLM with RVGA

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Demonstrates how to fit a Generalized Linear Model using the RVGA library. Ensure the 'rvga' package is installed and loaded.

```R
library(rvga)

# Example data (replace with your actual data)
x <- matrix(rnorm(100), ncol = 2)
y <- rnorm(50)

# Fit a GLM (e.g., Gaussian family, identity link)
model <- rvga(y, x, family = gaussian(), link = "identity")

# Print the model summary
print(model)
```

--------------------------------

### Example Output of Regret Summary

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/delayed-reward.html

Displays the expected output format for the regret summary table, showing strategies and their corresponding regret values.

```text
Strategy                                  Regret
--------------------------------------------------
Regret without drift (and no decay)      21751.4
Regret with drift and without decay      48109.2
Regret with drift and a 0.999 decay rate      21433.8
Regret with drift and optimal decay rate      22657.5
Regret with drift and worst decay rate     303682.4
--------------------------------------------------
```

--------------------------------

### Get Model Fitted Values from RVGA GLM

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Retrieves the fitted values (predicted values) of the RVGA GLM model for the training data.

```python
fitted_values = rvga.get_fitted_values()
```

--------------------------------

### Get Model Log-Likelihood from RVGA GLM

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Retrieves the log-likelihood of the fitted RVGA GLM model. This is a measure of how well the model fits the data.

```python
log_likelihood = rvga.get_log_likelihood()
```

--------------------------------

### Get Model Coefficients from RVGA GLM

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Retrieves the coefficients of the fitted RVGA GLM model. These indicate the strength and direction of relationships between variables.

```python
coefficients = rvga.get_coefficients()
```

--------------------------------

### Fit a Binomial GLM

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Example of fitting a Generalized Linear Model with a binomial family and logit link, suitable for binary classification or proportion data.

```R
library(rvga)

# Example binary data
x <- matrix(rnorm(100), ncol = 2)
y <- rbinom(50, 1, 0.5)

# Fit a Binomial GLM
model_binom <- rvga(y, x, family = binomial(), link = "logit")

# Print the model summary
print(model_binom)
```

--------------------------------

### Run EpsilonGreedyLearner with Data

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Runs the EpsilonGreedyLearner on a dataset for a specified number of steps. This demonstrates its exploration-exploitation strategy.

```python
from banditpylib.data import Dataset
from banditpylib.learners import EpsilonGreedyLearner

data = Dataset()
data.read_from_file("data.csv")
learner = EpsilonGreedyLearner(n_arms=5, epsilon=0.1)
learner.run(data, n_steps=100)
```

--------------------------------

### Initialize ContextualBandit with DiscountedThompsonSamplingLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Creates a ContextualBandit instance configured with the DiscountedThompsonSamplingLearner policy, suitable for non-stationary environments.

```python
from banditpylib.bandits import ContextualBandit
from banditpylib.learners import DiscountedThompsonSamplingLearner

boundit = ContextualBandit(policy=DiscountedThompsonSamplingLearner(n_arms=5, discount_factor=0.9), n_arms=5)
```

--------------------------------

### Non-linear Reward Function - Python

Source: https://bayesianbandits.readthedocs.io/en/latest/howto/reward-functions.html

Implement an asymmetric loss function that penalizes underperformance more than overperformance. Other examples include threshold utility and diminishing returns.

```python
def asymmetric_loss(samples, target=10.0, penalty=3.0):
    """Penalize undershoot 3x relative to overshoot."""
    diff = samples - target
    return np.where(diff < 0, penalty * diff, diff)

```

--------------------------------

### Initialize EpsilonGreedyLearner

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/hybrid-bandits.ipynb.txt

Initializes an EpsilonGreedyLearner, a simple yet effective bandit algorithm that balances exploration and exploitation.

```python
from banditpylib.learners import EpsilonGreedyLearner

learner = EpsilonGreedyLearner(n_arms=5, epsilon=0.1)
```

--------------------------------

### Get Model Residuals from RVGA GLM

Source: https://bayesianbandits.readthedocs.io/en/latest/_sources/notebooks/rvga-glm.ipynb.txt

Retrieves the residuals of the fitted RVGA GLM model. Residuals represent the differences between observed and predicted values.

```python
residuals = rvga.get_residuals()
```

--------------------------------

### Initialize Thompson Sampling Agents

Source: https://bayesianbandits.readthedocs.io/en/latest/notebooks/linear-bandits.html

Sets up contextual agents using Thompson Sampling with NormalInverseGammaRegressor and BayesianGLM. Ensure necessary imports and random seed are configured before use.

```python
from bayesianbandits import ThompsonSampling

# Create oracles for Thompson sampling experiments
normal_thompson_oracle = ClickthroughOracle()
glM_thompson_oracle = ClickthroughOracle()

# NormalInverseGammaRegressor with Thompson Sampling
normal_thompson_agent = ContextualAgent(
    arms=[
        Arm(Article.article_1, learner=NormalInverseGammaRegressor(lam=10.0)),
        Arm(Article.article_2, learner=NormalInverseGammaRegressor(lam=10.0)),
        Arm(Article.article_3, learner=NormalInverseGammaRegressor(lam=10.0)),
        Arm(Article.article_4, learner=NormalInverseGammaRegressor(lam=10.0)),
    ],
    policy=ThompsonSampling(),
    random_seed=rng,
)

# BayesianGLM with Thompson Sampling
glM_thompson_agent = ContextualAgent(
    arms=glm_arms_thompson,
    policy=ThompsonSampling(),
    random_seed=rng,
)
```