### Install Polars for Data Backend Example

Source: https://github.com/e10v/tea-tasting/blob/main/docs/multiple-testing.md

This command installs the Polars library, which is used as an example data backend in the tea-tasting guide for reproducible examples. Ensure you have pip or uv installed.

```bash
uv pip install polars
```

--------------------------------

### Install Dependencies for Tea-Tasting Examples

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

This snippet shows how to install the necessary Python packages, Polars and tqdm, which are required for running the simulated experiment examples in tea-tasting.

```bash
uv pip install polars tqdm
```

--------------------------------

### Launch Marimo Notebook Server

Source: https://github.com/e10v/tea-tasting/blob/main/examples/README.md

Starts the marimo notebook server using `uv run` to serve the examples located in the 'examples' directory. This allows users to access and run the example notebooks in their local environment.

```bash
uv run marimo edit examples
```

--------------------------------

### Install Ibis and DuckDB for Tea-Tasting

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

Installs the necessary packages, including Ibis with DuckDB support and Polars, to follow the examples in the guide. This command-line instruction ensures the environment is set up for database interaction and data manipulation.

```bash
uv pip install ibis-framework[duckdb] polars
```

--------------------------------

### Install Dependencies with Uv

Source: https://github.com/e10v/tea-tasting/blob/main/examples/README.md

Installs necessary Python packages including marimo, tea-tasting, polars, and ibis-framework with duckdb support using the `uv` package manager. This command sets up the environment for running the examples locally.

```bash
uv venv && uv pip install marimo tea-tasting polars ibis-framework[duckdb]
```

--------------------------------

### Install Polars for Tea-Tasting

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

This command installs the Polars library, which is used as an example data backend in this guide. Polars is a fast DataFrame library implemented in Rust using Apache Arrow.

```bash
uv pip install polars

```

--------------------------------

### Experiment Setup with Mean and RatioOfMeans Metrics (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Demonstrates how to set up an A/B test experiment using the Mean and RatioOfMeans metric classes. This example shows the instantiation of an experiment with different metric configurations, including specifying metric columns, alternative hypotheses, confidence levels, and distribution assumptions.

```python
>>> another_experiment = tt.Experiment(
...     sessions_per_user=tt.Mean("sessions", alternative="greater"),
...     orders_per_session=tt.RatioOfMeans("orders", "sessions", confidence_level=0.9),
...     orders_per_user=tt.Mean("orders", equal_var=True),
...     revenue_per_user=tt.Mean("revenue", use_t=False),
... )
```

--------------------------------

### Clone Repository and Install Example Dependencies

Source: https://github.com/e10v/tea-tasting/blob/main/docs/index.md

Clones the tea-tasting GitHub repository and installs necessary packages for running example notebooks locally. This includes marimo, tea-tasting itself, polars, and ibis with duckdb support.

```bash
git clone git@github.com:e10v/tea-tasting.git && cd tea-tasting

uv venv && uv pip install marimo tea-tasting polars ibis-framework[duckdb]
```

--------------------------------

### Clone Tea-Tasting Repository

Source: https://github.com/e10v/tea-tasting/blob/main/examples/README.md

Clones the tea-tasting repository from GitHub and navigates into the cloned directory. This is the initial step for setting up the examples in a local environment.

```bash
git clone git@github.com:e10v/tea-tasting.git && cd tea-tasting
```

--------------------------------

### Prepare Demo Data and Connect to DuckDB with Ibis

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

Generates example experimental data using `tt.make_users_data` and establishes a connection to an in-process DuckDB database using the Ibis API. The generated data is then loaded into a new table named 'users_data' within the DuckDB instance.

```python
>>> import ibis
>>> import polars as pl
>>> import tea_tasting as tt

>>> users_data = tt.make_users_data(seed=42)
>>> con = ibis.connect("duckdb://")
>>> con.create_table("users_data", users_data)
DatabaseTable: memory.main.users_data
  user     int64
  variant  int64
  sessions int64
  orders   int64
  revenue  float64

```

--------------------------------

### Install tea-tasting Package

Source: https://github.com/e10v/tea-tasting/blob/main/docs/index.md

Installs the tea-tasting Python package using uv pip. This is the primary method for adding the library to your project environment.

```bash
uv pip install tea-tasting
```

--------------------------------

### Get Global Configuration in Tea Tasting (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Retrieves global configuration values in the tea-tasting project. Can fetch a specific option by name or all options as a dictionary. Requires the `tea_tasting` library.

```python
import tea_tasting as tt

# Get a specific global option
print(tt.get_config("equal_var"))

# Get all global options
global_config = tt.get_config()
```

--------------------------------

### Customizing Experiment Result Keys (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Customizes the displayed keys for experiment results using the `with_keys` method or by passing keys to `to_string`. This example shows how to display only specific attributes like 'metric', 'control', 'treatment', 'effect_size', and 'effect_size_ci'.

```python
>>> result.with_keys((
...     "metric",
...     "control",
...     "treatment",
...     "effect_size",
...     "effect_size_ci",
... ))
            metric control treatment effect_size     effect_size_ci
 sessions_per_user    2.00      1.98     -0.0132  [-0.0750, 0.0485]
orders_per_session   0.266     0.289      0.0233 [-0.00246, 0.0491]
   orders_per_user   0.530     0.573      0.0427  [-0.0108, 0.0962]
  revenue_per_user    5.24      5.73       0.489     [-0.133, 1.11]


```

```python
>>> print(result.to_string(keys=(
...     "metric",
...     "control",
...     "treatment",
...     "effect_size",
...     "effect_size_ci",
... )))
            metric control treatment effect_size     effect_size_ci
 sessions_per_user    2.00      1.98     -0.0132  [-0.0750, 0.0485]
orders_per_session   0.266     0.289      0.0233 [-0.00246, 0.0491]
   orders_per_user   0.530     0.573      0.0427  [-0.0108, 0.0962]
  revenue_per_user    5.24      5.73       0.489     [-0.133, 1.11]


```

--------------------------------

### Basic tea-tasting experiment analysis

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Demonstrates the basic functionality of tea-tasting by creating synthetic user data, defining an experiment with various metrics, and performing the analysis. The output shows the calculated metrics for control and treatment groups, including relative effect size and p-values.

```python
>>> import tea_tasting as tt

>>> data = tt.make_users_data(seed=42)
>>> experiment = tt.Experiment(
...     sessions_per_user=tt.Mean("sessions"),
...     orders_per_session=tt.RatioOfMeans("orders", "sessions"),
...     orders_per_user=tt.Mean("orders"),
...     revenue_per_user=tt.Mean("revenue")
... )
>>> result = experiment.analyze(data)
>>> result
            metric control treatment rel_effect_size rel_effect_size_ci pvalue
 sessions_per_user    2.00      1.98          -0.66%      [-3.7%, 2.5%]  0.674
orders_per_session   0.266     0.289            8.8%      [-0.89%, 19%] 0.0762
   orders_per_user   0.530     0.573            8.0%       [-2.0%, 19%]  0.118
  revenue_per_user    5.24      5.73            9.3%       [-2.4%, 22%]  0.123

```

--------------------------------

### Basic A/B Test Analysis with tea-tasting

Source: https://github.com/e10v/tea-tasting/blob/main/README.md

This example demonstrates a basic A/B test analysis using the tea-tasting library. It defines an experiment with various metrics (Mean, RatioOfMeans) and analyzes generated user data, outputting a statistical summary including relative effect size and p-values.

```python
import tea_tasting as tt

data = tt.make_users_data(seed=42)
experiment = tt.Experiment(
    sessions_per_user=tt.Mean("sessions"),
    orders_per_session=tt.RatioOfMeans("orders", "sessions"),
    orders_per_user=tt.Mean("orders"),
    revenue_per_user=tt.Mean("revenue"),
)
result = experiment.analyze(data)
print(result)
```

--------------------------------

### Displaying Experiment Results (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Displays a formatted table of experiment results, including metrics like 'sessions_per_user', 'orders_per_session', 'orders_per_user', and 'revenue_per_user'. This output is generated by the `ExperimentResult` object itself or via the `to_string()` method.

```python
>>> result
            metric control treatment rel_effect_size rel_effect_size_ci pvalue
 sessions_per_user    2.00      1.98          -0.66%      [-3.7%, 2.5%]  0.674
orders_per_session   0.266     0.289            8.8%      [-0.89%, 19%] 0.0762
   orders_per_user   0.530     0.573            8.0%       [-2.0%, 19%]  0.118
  revenue_per_user    5.24      5.73            9.3%       [-2.4%, 22%]  0.123


```

--------------------------------

### Variance Reduction with CUPED/CUPAC in Tea-Tasting

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Demonstrates how to use CUPED/CUPAC for variance reduction with Mean and RatioOfMeans metrics in the tea-tasting library. This requires pre-experimental data as covariates. The covariates are specified using the 'covariate' parameter for Mean and 'numer_covariate'/'denom_covariate' for RatioOfMeans.

```python
>>> data_cuped = tt.make_users_data(seed=42, covariates=True)
>>> experiment_cuped = tt.Experiment(
...     sessions_per_user=tt.Mean("sessions", "sessions_covariate"),
...     orders_per_session=tt.RatioOfMeans(
...         numer="orders",
...         denom="sessions",
...         numer_covariate="orders_covariate",
...         denom_covariate="sessions_covariate",
...     ),
...     orders_per_user=tt.Mean("orders", "orders_covariate"),
...     revenue_per_user=tt.Mean("revenue", "revenue_covariate"),
... )
>>> result_cuped = experiment_cuped.analyze(data_cuped)
>>> result_cuped
            metric control treatment rel_effect_size rel_effect_size_ci  pvalue
 sessions_per_user    2.00      1.98          -0.68%      [-3.2%, 1.9%]   0.603
orders_per_session   0.262     0.293             12%        [4.2%, 21%] 0.00229
   orders_per_user   0.523     0.581             11%        [2.9%, 20%] 0.00733
  revenue_per_user    5.12      5.85             14%        [3.8%, 26%] 0.00674

```

--------------------------------

### Temporarily Set Global Configuration Context in Tea Tasting (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Applies global configuration settings temporarily within a specified context using a `with` statement. Settings revert to their previous values upon exiting the context. This is useful for running isolated experiments with specific configurations without affecting global defaults. Requires the `tea_tasting` library.

```python
import tea_tasting as tt

# Set configuration within a context
with tt.config_context(equal_var=True, use_t=False):
    experiment_within_context = tt.Experiment(
        sessions_per_user=tt.Mean("sessions"),
        orders_per_session=tt.RatioOfMeans("orders", "sessions"),
        orders_per_user=tt.Mean("orders"),
        revenue_per_user=tt.Mean("revenue"),
    )

# Verify global settings after context exit
print(
    f"global_config.equal_var: {tt.get_config('equal_var')}
"
    f"global_config.use_t: {tt.get_config('use_t')}

"
    f"orders_per_user_context.equal_var: {experiment_within_context.metrics['orders_per_user'].equal_var}
"
    f"orders_per_user_context.use_t: {experiment_within_context.metrics['orders_per_user'].use_t}"
)
```

--------------------------------

### Define tea-tasting experiment using a dictionary

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Defines an A/B test experiment using a dictionary for metrics and specifying the variant column name. This method is an alternative to using keyword arguments for metric definitions. It requires the `tea_tasting` library to be imported.

```python
>>> new_experiment = tt.Experiment(
...     {
...         "sessions per user": tt.Mean("sessions"),
...         "orders per session": tt.RatioOfMeans("orders", "sessions"),
...         "orders per user": tt.Mean("orders"),
...         "revenue per user": tt.Mean("revenue")
...     },
...     variant="variant"
... )

```

--------------------------------

### Analyze Power for Multiple Effect Sizes in Mean Metric

Source: https://github.com/e10v/tea-tasting/blob/main/docs/power-analysis.md

This example shows how to analyze statistical power for a Mean metric across different effect sizes and sample sizes. It configures the metric with a specific alpha and power, then uses solve_power to generate results for varying numbers of observations.

```python
>>> orders_per_user = tt.Mean("orders", alpha=0.1, power=0.7, n_obs=(10_000, 20_000))
>>> orders_per_user.solve_power(data, "rel_effect_size")
power effect_size rel_effect_size n_obs
  70%      0.0367            7.1% 10000
  70%      0.0260            5.0% 20000

```

--------------------------------

### Generate synthetic user data with tea-tasting

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Generates synthetic user data for A/B testing purposes using the `make_users_data` function. The output is a PyArrow Table by default, containing user identifiers, variant assignments, sessions, orders, and revenue. The `return_type` parameter can be used to specify other output formats like Pandas or Polars DataFrames.

```python
>>> data
pyarrow.Table
user: int64
variant: int64
sessions: int64
orders: int64
revenue: double
----
user: [[0,1,2,3,4,...,3995,3996,3997,3998,3999]]
variant: [[1,0,1,1,0,...,0,0,0,0,0]]
sessions: [[2,2,2,2,1,...,2,2,3,1,5]]
orders: [[1,1,1,1,1,...,0,0,0,0,2]]
revenue: [[9.17,6.43,7.94,15.93,7.14,...,0,0,0,0,17.16]]


```

--------------------------------

### Fetch and Display Head of Ibis Table Data

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

Fetches the first 5 rows of the 'data' Ibis Table, which represents the query result from DuckDB. This demonstrates how to preview the data directly within the Python environment, illustrating the structure and content of the experimental data.

```python
>>> ibis.options.interactive = True
>>> print(data.head(5))
┏━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓
┃ user  ┃ variant ┃ sessions ┃ orders ┃ revenue ┃
┡━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩
│ int64 │ int64   │ int64    │ int64  │ float64 │
├───────┼─────────┼──────────┼────────┼─────────┤
│     0 │       1 │        2 │      1 │    9.17 │
│     1 │       0 │        2 │      1 │    6.43 │
│     2 │       1 │        2 │      1 │    7.94 │
│     3 │       1 │        2 │      1 │   15.93 │
│     4 │       0 │        1 │      1 │    7.14 │
└───────┴─────────┴──────────┴────────┴─────────┘

>>> ibis.options.interactive = False

```

--------------------------------

### Set Global Configuration in Tea Tasting (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Sets global configuration options for the tea-tasting project. Changes made persist until explicitly changed again or until the program terminates. This function is crucial for modifying default statistical calculation behaviors like variance assumptions and distribution choices. It requires the `tea_tasting` library.

```python
import tea_tasting as tt

# Set specific global options
tt.set_config(equal_var=True, use_t=False)

# Example of using set_config with Experiment setup
experiment_with_config = tt.Experiment(
    sessions_per_user=tt.Mean("sessions"),
    orders_per_session=tt.RatioOfMeans("orders", "sessions"),
    orders_per_user=tt.Mean("orders"),
    revenue_per_user=tt.Mean("revenue"),
)

# Change configuration again
tt.set_config(equal_var=False, use_t=True)
orders_per_user = experiment_with_config.metrics["orders_per_user"]
print(
    f"orders_per_user.equal_var: {orders_per_user.equal_var}
"
    f"orders_per_user.use_t: {orders_per_user.use_t}"
)
```

--------------------------------

### Analyze Experiment with Three Variants (Specified Control)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

This Python code snippet demonstrates how to analyze an experiment with three variants using tea-tasting and Polars. It generates sample data with three variants, then analyzes the data comparing variant 0 (control) against variants 1 and 2.

```python
import polars as pl

# Assuming 'experiment' is an initialized tea-tasting experiment object

data_three_variants = pl.concat((
    tt.make_users_data(seed=42, return_type="polars"),
    tt.make_users_data(seed=21, return_type="polars")
        .filter(pl.col("variant").eq(1))
        .with_columns(variant=pl.lit(2, pl.Int64)),
))
results = experiment.analyze(data_three_variants, control=0, all_variants=True)
print(results)

```

--------------------------------

### Sample Ratio Mismatch Check with Tea-Tasting

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Utilizes the SampleRatio class in tea-tasting to detect mismatches in sample ratios between different variants of an A/B test. By default, it expects equal observations per variant. Custom ratios can be set using the 'ratio' parameter, and the statistical test method can be controlled via the 'method' parameter.

```python
>>> experiment_sample_ratio = tt.Experiment(
...     orders_per_user=tt.Mean("orders"),
...     revenue_per_user=tt.Mean("revenue"),
...     sample_ratio=tt.SampleRatio(),
... )
>>> result_sample_ratio = experiment_sample_ratio.analyze(data)
>>> result_sample_ratio
          metric control treatment rel_effect_size rel_effect_size_ci pvalue
 orders_per_user   0.530     0.573            8.0%       [-2.0%, 19%]  0.118
revenue_per_user    5.24      5.73            9.3%       [-2.4%, 22%]  0.123
    sample_ratio    2023      1977               -             [-, -]  0.477

```

--------------------------------

### Ibis: Group and Aggregate Data by Variant

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

This Python snippet uses the Ibis library to group data by the 'variant' column and calculate aggregate metrics such as mean sessions per user, orders per session, orders per user, and revenue per user. It demonstrates how Ibis translates these operations into SQL queries.

```python
>>> aggr_data = data.group_by("variant").aggregate(
...     sessions_per_user=data.sessions.mean(),
...     orders_per_session=data.orders.mean() / data.sessions.mean(),
...     orders_per_user=data.orders.mean(),
...     revenue_per_user=data.revenue.mean(),
... )
>>> aggr_data
r0 := SQLQueryResult
  query:
    select * from users_data
  schema:
    user     int64
    variant  int64
    sessions int64
    orders   int64
    revenue  float64
<BLANKLINE>
Aggregate[r0]
  groups:
    variant: r0.variant
  metrics:
    sessions_per_user:  Mean(r0.sessions)
    orders_per_session: Mean(r0.orders) / Mean(r0.sessions)
    orders_per_user:    Mean(r0.orders)
    revenue_per_user:   Mean(r0.revenue)

```

```python
>>> ibis.options.interactive = True
>>> print(aggr_data)  # doctest: +SKIP
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
```

--------------------------------

### Power Analysis for Experiment Planning (Python)

Source: https://context7.com/e10v/tea-tasting/llms.txt

Performs power analysis for experiment planning, allowing determination of required sample size, minimum detectable effect size, or statistical power. This example solves for the minimum detectable effect size given a sample size. Requires baseline data, optionally with covariates.

```Python
import tea_tasting as tt

# Generate baseline data (A/A test - no uplift)
data = tt.make_users_data(
    seed=42,
    sessions_uplift=0,
    orders_uplift=0,
    revenue_uplift=0,
    covariates=True,
)

# Solve for minimum detectable effect size at different sample sizes
orders_per_user = tt.RatioOfMeans(
    "orders",
    "sessions",
    "orders_covariate",
    "sessions_covariate",
    n_obs=(10_000, 20_000),
)
power_result = orders_per_user.solve_power(data)
print(power_result)
# power effect_size rel_effect_size n_obs
```

--------------------------------

### Track Simulation Progress with tqdm in Python

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

Illustrates how to integrate `tqdm` for progress visualization during experiment simulations. By passing `tqdm.tqdm` to the `progress` parameter, users can monitor the simulation's advancement in real-time, which is helpful for long-running simulations.

```python
>>> import tqdm

>>> results_progress = experiment.simulate(
...     data,
...     100,
...     seed=42,
...     progress=tqdm.tqdm,
... )  # doctest: +SKIP
100%|██████████████████████████████████████| 100/100 [00:01<00:00, 64.47it/s]

```

--------------------------------

### Analyze A/B Test Data with CUPED Variance Reduction (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

This snippet shows how to perform A/B test analysis using the CUPED method for variance reduction. It involves creating user data, registering it in a database, defining an experiment with various metrics, and analyzing the data to get results including effect sizes and p-values.

```python
>>> users_data_cuped = tt.make_users_data(seed=42, covariates=True)
>>> con.create_table("users_data_cuped", users_data_cuped)
DatabaseTable: memory.main.users_data_cuped
  user               int64
  variant            int64
  sessions           int64
  orders             int64
  revenue            float64
  sessions_covariate int64
  orders_covariate   int64
  revenue_covariate  float64

>>> data_cuped = con.sql("select * from users_data_cuped")
>>> experiment_cuped = tt.Experiment(
...     sessions_per_user=tt.Mean("sessions", "sessions_covariate"),
...     orders_per_session=tt.RatioOfMeans(
...         numer="orders",
...         denom="sessions",
...         numer_covariate="orders_covariate",
...         denom_covariate="sessions_covariate",
...     ),
...     orders_per_user=tt.Mean("orders", "orders_covariate"),
...     revenue_per_user=tt.Mean("revenue", "revenue_covariate"),
... )
>>> result_cuped = experiment_cuped.analyze(data_cuped)
>>> result_cuped
            metric control treatment rel_effect_size rel_effect_size_ci  pvalue
 sessions_per_user    2.00      1.98          -0.68%      [-3.2%, 1.9%]   0.603
orders_per_session   0.262     0.293             12%        [4.2%, 21%] 0.00229
   orders_per_user   0.523     0.581             11%        [2.9%, 20%] 0.00733
  revenue_per_user    5.12      5.85             14%        [3.8%, 26%] 0.00674

```

--------------------------------

### Prepare Data for A/A Testing (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

Prepares user data for A/A testing by ensuring no uplift is applied and dropping the 'variant' column. This step is crucial for setting up a baseline experiment.

```python
import polars as pl
import tea_tasting as tt

data = (
    tt.make_users_data(seed=42, orders_uplift=0, revenue_uplift=0)
    .drop_columns("variant")
)
print(data)
```

--------------------------------

### Analyze Experiment Data (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Analyzes experiment data using the `analyze` method of the `Experiment` class. This method takes data as input and returns an `ExperimentResult` object. By default, it assumes the variant with the lowest ID is the control.

```python
>>> new_result = experiment.analyze(data)

```

```python
>>> result_with_non_default_control = experiment.analyze(data, control=1)

```

--------------------------------

### Manage Global Configuration Defaults

Source: https://context7.com/e10v/tea-tasting/llms.txt

Access and view the current global default configuration for statistical tests and power analysis within the tea-tasting library. This allows users to understand or modify the default settings used for various analytical functions.

```python
import tea_tasting as tt

# View current configuration
config = tt.get_config()
print(config)

```

--------------------------------

### Basic A/B Test Analysis with tea-tasting

Source: https://github.com/e10v/tea-tasting/blob/main/docs/index.md

Demonstrates a basic A/B test analysis using the tea-tasting Python package. It shows how to define an experiment with various metrics (Mean, RatioOfMeans) and analyze sample data, then displays the results including effect sizes and p-values.

```python
>>> import tea_tasting as tt

>>> data = tt.make_users_data(seed=42)
>>> experiment = tt.Experiment(
...     sessions_per_user=tt.Mean("sessions"),
...     orders_per_session=tt.RatioOfMeans("orders", "sessions"),
...     orders_per_user=tt.Mean("orders"),
...     revenue_per_user=tt.Mean("revenue"),
... )
>>> result = experiment.analyze(data)
>>> result
            metric control treatment rel_effect_size rel_effect_size_ci pvalue
 sessions_per_user    2.00      1.98          -0.66%      [-3.7%, 2.5%]  0.674
orders_per_session   0.266     0.289            8.8%      [-0.89%, 19%] 0.0762
   orders_per_user   0.530     0.573            8.0%       [-2.0%, 19%]  0.118
  revenue_per_user    5.24      5.73            9.3%       [-2.4%, 22%]  0.123

```

--------------------------------

### Parallel Simulation with concurrent.futures.ProcessPoolExecutor in Python

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

This snippet demonstrates how to run simulations in parallel using `concurrent.futures.ProcessPoolExecutor`. It takes simulation data, number of runs, seed, treatment, and a mapping function as input. The `map_` parameter is set to `executor.map` for parallel processing, and `tqdm.tqdm` is used for progress visualization. Note: This code will not work in the marimo online playground due to WASM limitations.

```python
import concurrent.futures
import tqdm

# Assuming 'experiment' and 'data' are defined elsewhere
# For example:
# class MockExperiment:
#     def simulate(self, data, num_runs, seed, treat, map_, progress):
#         for i in range(num_runs):
#             # Simulate some work
#             pass
#         return [f"result_{i}" for i in range(num_runs)]
# 
# experiment = MockExperiment()
# data = [1, 2, 3]
# treat = lambda x: x * 2

with concurrent.futures.ProcessPoolExecutor() as executor:
    results_parallel = experiment.simulate(
        data,
        100,
        seed=42,
        treat=treat,
        map_=executor.map,
        progress=tqdm.tqdm,
    )  # doctest: +SKIP

```

--------------------------------

### Analyze Experiment with Three Variants (Default Control)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

This Python code snippet shows how to analyze an experiment with multiple variants without explicitly specifying a control. tea-tasting automatically designates the variant with the lowest ID as the control for each pairwise comparison.

```python
import polars as pl

# Assuming 'experiment' and 'data_three_variants' are initialized

results_all = experiment.analyze(data_three_variants, all_variants=True)
print(results_all)

```

--------------------------------

### Simulate Experiment with Data Generation Function in Python

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

Shows how to use a data generation function for dynamic input in `experiment.simulate`. The function receives a numpy random Generator and returns data compatible with tea-tasting. This is useful when data needs to be generated on-the-fly for each simulation iteration.

```python
>>> results_data_gen = experiment.simulate(tt.make_users_data, 100, seed=42)
>>> null_rejected(results_data_gen.to_polars())
shape: (5, 4)
┌────────────────────┬────────────────────┬────────────────────┬────────────────────┐
│ metric             ┆ null_rejected_0.01 ┆ null_rejected_0.02 ┆ null_rejected_0.05 │
│ ---                ┆ ---                ┆ ---                ┆ ---                │
│ str                ┆ f64                ┆ f64                ┆ f64                │
╞════════════════════╪════════════════════╪════════════════════╪════════════════════╡
│ sessions_per_user  ┆ 0.01               ┆ 0.01               ┆ 0.06               │
│ orders_per_session ┆ 0.27               ┆ 0.36               ┆ 0.54               │
│ orders_per_user    ┆ 0.24               ┆ 0.32               ┆ 0.49               │
│ revenue_per_user   ┆ 0.17               ┆ 0.26               ┆ 0.39               │
│ n_users            ┆ 0.01               ┆ 0.01               ┆ 0.04               │
└────────────────────┴────────────────────┴────────────────────┴────────────────────┘

```

--------------------------------

### View Specific Variant Pair Results

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

This Python code demonstrates how to access and view the analysis results for a specific pair of variants from the output of the `analyze` method. The results are stored in a dictionary-like object where keys are variant tuples (control, treatment).

```python
# Assuming 'results' is the output from experiment.analyze()

print(results[0, 1])

```

--------------------------------

### Accessing Metric Analysis Results (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/user-guide.md

Retrieves the analysis results for a specific metric from an `ExperimentResult` object. The `ExperimentResult` object is a mapping, allowing access to metric results using the metric name as a key. The output displays detailed statistics for the 'orders_per_user' metric.

```python
>>> import pprint

>>> pprint.pprint(result["orders_per_user"]._asdict())
{'control': 0.5304003954522986,
 'effect_size': 0.04269014577177832,
 'effect_size_ci_lower': -0.010800201598205515,
 'effect_size_ci_upper': 0.09618049314176216,
 'pvalue': np.float64(0.11773177998716214),
 'rel_effect_size': 0.08048664016431273,
 'rel_effect_size_ci_lower': -0.019515294044061937,
 'rel_effect_size_ci_upper': 0.1906880061278886,
 'statistic': 1.5647028839586707,
 'treatment': 0.5730905412240769}

```

--------------------------------

### Query Data from DuckDB Table using SQL and Ibis

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

Retrieves data from the 'users_data' table in the connected DuckDB database using a simple SQL SELECT statement through Ibis. This creates an Ibis Table object that represents the query result and can be further manipulated or inspected.

```python
>>> data = con.sql("select * from users_data")
>>> data
SQLQueryResult
  query:
    select * from users_data
  schema:
    user     int64
    variant  int64
    sessions int64
    orders   int64
    revenue  float64

```

--------------------------------

### Simulate Experiment with Treatment Function in Python

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

Demonstrates how to simulate an experiment with a custom treatment function using PyArrow. The `treat` function modifies the input PyArrow Table by increasing 'orders' and 'revenue' by 10%. This is useful for estimating statistical power by observing the proportion of rejected null hypotheses.

```python
>>> import pyarrow as pa
>>> import pyarrow.compute as pc

>>> def treat(data: pa.Table) -> pa.Table:
...     return (
...         data.drop_columns(["orders", "revenue"])
...         .append_column("orders", pc.multiply(data["orders"], pa.scalar(1.1)))
...         .append_column("revenue", pc.multiply(data["revenue"], pa.scalar(1.1)))
...     )
...
>>> results_treat = experiment.simulate(data, 100, seed=42, treat=treat)
>>> null_rejected(results_treat.to_polars())
shape: (5, 4)
┌────────────────────┬────────────────────┬────────────────────┬────────────────────┐
│ metric             ┆ null_rejected_0.01 ┆ null_rejected_0.02 ┆ null_rejected_0.05 │
│ ---                ┆ ---                ┆ ---                ┆ ---                │
│ str                ┆ f64                ┆ f64                ┆ f64                │
╞════════════════════╪════════════════════╪════════════════════╪════════════════════╡
│ sessions_per_user  ┆ 0.01               ┆ 0.02               ┆ 0.05               │
│ orders_per_session ┆ 0.23               ┆ 0.31               ┆ 0.42               │
│ orders_per_user    ┆ 0.21               ┆ 0.29               ┆ 0.4                │
│ revenue_per_user   ┆ 0.11               ┆ 0.16               ┆ 0.31               │
│ n_users            ┆ 0.01               ┆ 0.01               ┆ 0.04               │
└────────────────────┴────────────────────┴────────────────────┴────────────────────┘

```

--------------------------------

### Set Global Configuration in Tea-Tasting

Source: https://context7.com/e10v/tea-tasting/llms.txt

Configures the default parameters for all subsequent analyses within the tea-tasting library. This includes settings for hypothesis testing alternatives, confidence levels, variance assumptions, and power calculations. These defaults can be overridden by context managers or metric-specific settings.

```python
import tea_tasting as tt

# Set global configuration
tt.set_config(
    alternative="two-sided",
    confidence_level=0.95,
    equal_var=False,
    use_t=True,
    alpha=0.05,
    ratio=1,
    power=0.8,
)
```

--------------------------------

### Analyze experiment with Proportion and Mean metrics

Source: https://github.com/e10v/tea-tasting/blob/main/docs/custom-metrics.md

Sets up and analyzes a tea-tasting experiment using both a custom Proportion metric and a built-in Mean metric. The Proportion metric analyzes the proportion of users with orders, while the Mean metric calculates the average value for the same column. This demonstrates how to combine different metrics for comprehensive analysis.

```python
experiment_prop = tt.Experiment(
    prop_users_with_orders=Proportion("has_order"),
    mean_users_with_orders=tt.Mean("has_order", use_t=False),
)
experiment_prop.analyze(data)

```

--------------------------------

### Temporary Configuration Changes with Context Manager

Source: https://context7.com/e10v/tea-tasting/llms.txt

Applies temporary configuration settings within a specific block of code using a context manager. Settings within the `with` statement override global configurations for the duration of the block and revert to the original settings once the block is exited. This is useful for running analyses with different parameters without affecting the global state.

```python
import tea_tasting as tt

# Use context manager for temporary config changes
with tt.config_context(confidence_level=0.99, alpha=0.01):
    # Analysis with 99% confidence intervals
    experiment = tt.Experiment(
        orders_per_user=tt.Mean("orders"),
    )
    data = tt.make_users_data(seed=42)
    result = experiment.analyze(data)
    print(result)
# Config reverts after context exits
```

--------------------------------

### Prepare Data for Custom Metrics Analysis (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/custom-metrics.md

This snippet demonstrates how to import necessary libraries and prepare sample user data using PyArrow for custom metric analysis in Tea Tasting. It includes casting a boolean condition to an integer type.

```python
from typing import Literal, NamedTuple
import numpy as np
import pyarrow as pa
import pyarrow.compute as pc
import scipy.stats
import tea_tasting as tt
import tea_tasting.aggr
import tea_tasting.config
import tea_tasting.metrics
import tea_tasting.utils

data = tt.make_users_data(seed=42)
data = data.append_column(
    "has_order",
    pc.greater(data["orders"], 0).cast(pa.int64()),
)
print(data)
```

--------------------------------

### Analyze A/B Test Data with Polars DataFrame (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/data-backends.md

This snippet demonstrates how to analyze A/B test data using a Polars DataFrame. It converts an Arrow table to a Polars DataFrame and then uses the `analyze` method of an experiment object to compute the results. This is useful for users who prefer or already work with Polars.

```python
>>> data_polars = pl.from_arrow(users_data)
>>> experiment.analyze(data_polars)
            metric control treatment rel_effect_size rel_effect_size_ci pvalue
 sessions_per_user    2.00      1.98          -0.66%      [-3.7%, 2.5%]  0.674
orders_per_session   0.266     0.289            8.8%      [-0.89%, 19%] 0.0762
   orders_per_user   0.530     0.573            8.0%       [-2.0%, 19%]  0.118
  revenue_per_user    5.24      5.73            9.3%       [-2.4%, 22%]  0.123


```

--------------------------------

### Create and Analyze Experiment with Multiple Metrics (Python)

Source: https://context7.com/e10v/tea-tasting/llms.txt

Defines an experiment with multiple metrics (e.g., sessions per user, orders per session, revenue per user) and analyzes the results, providing confidence intervals and p-values. It uses the `tea_tasting` library and accepts a pandas DataFrame as input.

```Python
import tea_tasting as tt

# Create experiment with multiple metrics
experiment = tt.Experiment(
    sessions_per_user=tt.Mean("sessions"),
    orders_per_session=tt.RatioOfMeans("orders", "sessions"),
    orders_per_user=tt.Mean("orders"),
    revenue_per_user=tt.Mean("revenue"),
)

# Generate sample data
data = tt.make_users_data(seed=42)

# Analyze experiment
result = experiment.analyze(data)
print(result)
#             metric control treatment rel_effect_size rel_effect_size_ci pvalue
#  sessions_per_user    2.00      1.98          -0.66%      [-3.7%, 2.5%]  0.674
# orders_per_session   0.266     0.289            8.8%      [-0.89%, 19%] 0.0762
#    orders_per_user   0.530     0.573            8.0%       [-2.0%, 19%]  0.118
#   revenue_per_user    5.24      5.73            9.3%       [-2.4%, 22%]  0.123
```

--------------------------------

### Simulate A/A Experiments (Python)

Source: https://github.com/e10v/tea-tasting/blob/main/docs/simulated-experiments.md

Runs A/A tests by defining experiment metrics and simulating results using the Experiment.simulate method. It takes prepared data and the number of simulations as input, returning simulation results.

```python
experiment = tt.Experiment(
    sessions_per_user=tt.Mean("sessions"),
    orders_per_session=tt.RatioOfMeans("orders", "sessions"),
    orders_per_user=tt.Mean("orders"),
    revenue_per_user=tt.Mean("revenue"),
    n_users=tt.SampleRatio(),
)
results = experiment.simulate(data, 100, seed=42)
results_data = results.to_polars()
print(results_data.select(
    "metric",
    "control",
    "treatment",
    "rel_effect_size",
    "rel_effect_size_ci_lower",
    "rel_effect_size_ci_upper",
    "pvalue",
))  # doctest: +SKIP
```