### Development Installation

Source: https://github.com/biomedsciai/causallib/blob/master/docs/source/getting_started.md

Install causallib for development, including all dependencies and documentation tools. This involves cloning the repository and using pip's editable install.

```bash
git clone https://github.com/BiomedSciAI/causallib.git
cd causallib
pip install -e ".[dev,docs]"
```

--------------------------------

### Install causallib

Source: https://github.com/biomedsciai/causallib/blob/master/README.md

Command to install the package via pip.

```bash
pip install causallib
```

--------------------------------

### Benchmark X-Learner Cold Start

Source: https://github.com/biomedsciai/causallib/blob/master/examples/xlearner.ipynb

Measures the execution time and accuracy of an X-Learner model without warm start initialization.

```python
start = time.time()
xlearner_cold = XLearner(
    outcome_model=StratifiedStandardization(MLPRegressor()), effect_types="diff"
)
_ = xlearner_cold.fit(sim_data["X"], sim_data["a"], sim_data["y"])
end = time.time()
effect = xlearner_cold.estimate_effect(sim_data["X"], sim_data["a"], agg="population")
print(
    "Calculated ATE: {:.3f} Actual ATE: {:.3f}".format(
        effect.head().values[0], sim_data["tau"].mean()
    )
)
print("Run time (seconds): {:.2f}".format(end - start))
```

--------------------------------

### Importing IPW dependencies

Source: https://github.com/biomedsciai/causallib/blob/master/examples/ipw.ipynb

Initial setup for the IPW model, including dataset loading, the IPW estimator, and a scikit-learn logistic regression learner.

```python
%matplotlib inline
from causallib.datasets import load_nhefs
from causallib.estimation import IPW
from causallib.evaluation import evaluate
from sklearn.linear_model import LogisticRegression
from matplotlib import pyplot as plt
```

--------------------------------

### Benchmark X-Learner Warm Start

Source: https://github.com/biomedsciai/causallib/blob/master/examples/xlearner.ipynb

Measures the execution time and accuracy of an X-Learner model using warm start initialization to potentially reduce training time.

```python
start = time.time()
xlearner_warm = XLearner(
    outcome_model=StratifiedStandardization(MLPRegressor(warm_start=True)), effect_types="diff"
)
_ = xlearner_warm.fit(sim_data["X"], sim_data["a"], sim_data["y"])
end = time.time()
effect = xlearner_warm.estimate_effect(sim_data["X"], sim_data["a"], agg="population")
print(
    "Calculated ATE: {:.3f} Actual ATE: {:.3f}".format(
        effect.head().values[0], sim_data["tau"].mean()
    )
)
print("Run time (seconds): {:.2f}".format(end - start))
```

--------------------------------

### Install Pinned Dependencies

Source: https://github.com/biomedsciai/causallib/blob/master/docs/source/getting_started.md

Install specific versions of dependencies for development or reproducible environments using requirements files.

```bash
pip install -r requirements.txt -r docs/requirements.txt
```

--------------------------------

### Install Contrib Dependencies

Source: https://github.com/biomedsciai/causallib/blob/master/causallib/contrib/README.md

Install all contrib-related dependencies including PyTorch via the extra-requirements flag.

```shell
pip install causallib[contrib] -f https://download.pytorch.org/whl/torch_stable.html
```

--------------------------------

### Nuisance Model Setup

Source: https://github.com/biomedsciai/causallib/blob/master/examples/rlearner.ipynb

Sets up nuisance models for treatment and outcome prediction using Logistic Regression and GridSearchCV with RandomForestRegressor. This is a common setup for meta-learners.

```python
# nuisance models
treatment_model = LogisticRegression(max_iter=2000)
outcome_model = GridSearchCV(RandomForestRegressor(), 
                             {'max_depth':[2,4,6]}, n_jobs=-1, cv=3)
```

--------------------------------

### Install CausalLib with Contrib Extra

Source: https://github.com/biomedsciai/causallib/blob/master/docs/source/getting_started.md

Install causallib with the 'contrib' extra to use advanced methods. This command includes optional dependencies.

```bash
pip install causallib[contrib]
```

--------------------------------

### Quick Start: IPW Example

Source: https://github.com/biomedsciai/causallib/blob/master/docs/source/getting_started.md

A simple example demonstrating Inverse Propensity Weighting (IPW) to estimate causal effects. It loads data, fits an IPW model, estimates population outcomes, and calculates the Average Treatment Effect.

```python
from causallib.estimation import IPW
from causallib.datasets import load_nhefs
from sklearn.linear_model import LogisticRegression

# Load example data
data = load_nhefs()

# Create and fit IPW model
ipw = IPW(LogisticRegression())
ipw.fit(data.X, data.a)

# Estimate population outcomes
outcomes = ipw.estimate_population_outcome(data.X, data.a, data.y)
print(f"Outcome under control: {outcomes[0]:.2f}")
print(f"Outcome under treatment: {outcomes[1]:.2f}")

# Estimate treatment effect
effect = ipw.estimate_effect(outcomes[1], outcomes[0])
print(f"Average Treatment Effect: {effect['diff']:.2f}")
```

--------------------------------

### Initialize Matching Positivity Checker

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Setup for using the Matching class to identify samples that satisfy the positivity assumption.

```python
import pandas as pd
import numpy as np
import seaborn as sb

from causallib.positivity import Matching
from causallib.datasets import load_nhefs
data = load_nhefs(augment=False,onehot=False)
X, a, y = data.X, data.a, data.y
```

```python
m=Matching(with_replacement=False)
m.fit(X, a)
overlap=m.predict(X, a)
```

--------------------------------

### Importing CausalLib Dependencies

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching.ipynb

Initial setup for importing necessary modules and datasets for causal inference tasks.

```python
from causallib.estimation import IPW, Matching
import matplotlib.pyplot as plt
import seaborn as sb
import pandas as pd
import numpy as np
from causallib.evaluation.metrics import calculate_covariate_balance
from sklearn.linear_model import LogisticRegression
from causallib.preprocessing.transformers import PropensityTransformer, MatchingTransformer
from causallib.datasets import load_nhefs
%matplotlib inline
```

--------------------------------

### Run Inverse Probability Weighting (IPW) Model

Source: https://github.com/biomedsciai/causallib/blob/master/causallib/estimation/README.md

Example of initializing and fitting an IPW model using LogisticRegression. The learner can be any scikit-learn-compatible model.

```Python
from sklearn.linear_model import LogisticRegression
from causallib.estimation import IPW
from causallib.datasets.data_loader import fetch_smoking_weight

model = LogisticRegression()
ipw = IPW(learner=model)
data = fetch_smoking_weight()
ipw.fit(data.X, data.a)
ipw.estimate_population_outcome(data.X, data.a, data.y)
```

--------------------------------

### Import necessary libraries for causallib

Source: https://github.com/biomedsciai/causallib/blob/master/examples/xlearner.ipynb

Imports essential libraries including data loading, estimation methods, and evaluation tools from causallib, along with common data science libraries like numpy, pandas, and scikit-learn. This setup is required before analyzing datasets.

```python
import time
import warnings

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
import shap
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.neural_network import MLPRegressor

from causallib.datasets import load_nhefs
from causallib.estimation import Standardization, StratifiedStandardization, XLearner
from causallib.evaluation import evaluate

warnings.filterwarnings("ignore")
%matplotlib inline
```

--------------------------------

### Build HTML Documentation

Source: https://github.com/biomedsciai/causallib/blob/master/docs/README.md

Execute this command to build the HTML version of the documentation.

```bash
make html
```

--------------------------------

### Generate API Documentation

Source: https://github.com/biomedsciai/causallib/blob/master/docs/README.md

Run this command from the project's root directory to generate the source API documentation files.

```bash
sphinx-apidoc -o source ../causallib --separate --force
```

--------------------------------

### Initialize and Fit the Matching Estimator

Source: https://github.com/biomedsciai/causallib/blob/master/examples/lalonde_matching.ipynb

Sets up the `PropensityTransformer` and `Matching` objects. The `Matching` object is configured with the propensity transformer, replacement strategy, number of neighbors, and the chosen KNN backend. It is then fitted to the data.

```python
propensity_transform = PropensityTransformer(
    include_covariates=False, learner=learner())
matcher = Matching(propensity_transform=propensity_transform,
                   with_replacement=True, n_neighbors=1, knn_backend=knn_backend)
matcher.fit(X, a, y)
matcher.match(X, a)
```

--------------------------------

### Execute TMLE Flavor Comparison

Source: https://github.com/biomedsciai/causallib/blob/master/examples/TMLE.ipynb

Initializes the outcome and weight models and executes the comparison simulation.

```python
outcome_model = Standardization(make_pipeline(PolynomialFeatures(2), LassoCV(random_state=0)))
weight_model = IPW(LogisticRegression(penalty="none", random_state=0))

pred_ates, true_ates = compare_TMLE_flavors(outcome_model, weight_model, 50, 5000)
plot_tmle_flavor_comparison(pred_ates, true_ates)
```

--------------------------------

### Initialize Logistic Regression Learner

Source: https://github.com/biomedsciai/causallib/blob/master/examples/Dehejia_Wahba_replication.ipynb

Sets up a Logistic Regression model from scikit-learn for propensity score estimation. Regularization is turned off, and the solver is set to 'lbfgs' with an increased max_iter to ensure convergence.

```python
from sklearn.linear_model import LogisticRegression

learner = LogisticRegression(penalty='none',  # No regularization, new in scikit-learn 0.21.*
                             solver='lbfgs',
                             max_iter=500)    # Increaed to achieve convergence with 'lbfgs' solver
```

--------------------------------

### Initialize IPW Model with Clipping

Source: https://github.com/biomedsciai/causallib/blob/master/examples/ipw.ipynb

Instantiate an IPW model using a specified learner and define caliper values for trimming probabilities. Set `use_stabilized` to False for default behavior.

```python
clip_min = 0.2
clip_max = 0.8
ipw = IPW(learner, clip_min=clip_min, clip_max=clip_max, use_stabilized=False)
```

--------------------------------

### Initialize and fit Vanilla Doubly Robust model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/doubly_robust.ipynb

Configure an AIPW model using StratifiedStandardization and IPW, then fit it to the data.

```python
ipw = IPW(LogisticRegression(solver="liblinear"), clip_min=0.05, clip_max=0.95)
std = StratifiedStandardization(LinearRegression())
dr = AIPW(std, ipw)
dr.fit(data.X, data.a, data.y)
```

--------------------------------

### Estimate causal effects with IPW

Source: https://github.com/biomedsciai/causallib/blob/master/README.md

Example demonstrating the use of Inverse Probability Weighting (IPW) with a scikit-learn LogisticRegression model.

```python
from sklearn.linear_model import LogisticRegression
from causallib.estimation import IPW 
from causallib.datasets import load_nhefs

data = load_nhefs()
ipw = IPW(LogisticRegression())
ipw.fit(data.X, data.a)
potential_outcomes = ipw.estimate_population_outcome(data.X, data.a, data.y)
effect = ipw.estimate_effect(potential_outcomes[1], potential_outcomes[0])
```

--------------------------------

### Initialize and fit Doubly Robust IP-Feature model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/doubly_robust.ipynb

Configure a PropensityFeatureStandardization model using Standardization and IPW, then fit it to the data.

```python
ipw = IPW(LogisticRegression(solver="liblinear"))
std = Standardization(LinearRegression())
dr = PropensityFeatureStandardization(std, ipw)
dr.fit(data.X, data.a, data.y)
```

--------------------------------

### Import FaissNearestNeighbors

Source: https://github.com/biomedsciai/causallib/blob/master/causallib/contrib/README.md

Import the Faiss-powered nearest neighbors implementation for faster search.

```python
from causallib.contrib.faissknn import FaissNearestNeighbors
```

--------------------------------

### Initialize Causal Models

Source: https://github.com/biomedsciai/causallib/blob/master/examples/rlearner.ipynb

Define a dictionary of causal models including R-Learner, S-Learner, and T-Learner with specified outcome and treatment models.

```python
models = {
    'R-Learner': RLearner(outcome_model=outcome_model, 
                          treatment_model=treatment_model, 
                          effect_model=LinearRegression(fit_intercept=False), 
                          effect_covariates=list(), # homogeneous effect
                          n_splits=2, 
                          refit=False),
    'S-Learner': Standardization(outcome_model),
    'T-Learner': StratifiedStandardization(outcome_model),
}
```

--------------------------------

### Plot Survival Curves with Causallib MarginalSurvival

Source: https://github.com/biomedsciai/causallib/blob/master/examples/causal_survival_analysis.ipynb

Estimates and plots population-averaged survival curves using causallib's MarginalSurvival. This function is an alternative to lifelines when it's not installed.

```python
from causallib.survival.marginal_survival import MarginalSurvival

marginal_survival = MarginalSurvival()
marginal_survival.fit(X, a, t, y)
population_averaged_survival_curves = marginal_survival.estimate_population_outcome(X, a, t, y)

plot_survival_curves(
    population_averaged_survival_curves,
    labels=["non-quitters", "quitters"],
    title="Unadjusted survival of smoke quitters vs. non-quitters in a 10 years observation period",
)
```

--------------------------------

### Get Positivity Profile (One vs. Rest)

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Retrieves the positivity profile for the OneVersusRestPositivity model, showing which patients are considered positive for each treatment against all others. Displays the head of the resulting DataFrame.

```python
pos_one_vs_rest.positivity_profile(data['X'],data['a']).head()
```

--------------------------------

### Select Nearest Neighbors Backend (Faiss or Sklearn)

Source: https://github.com/biomedsciai/causallib/blob/master/examples/lalonde_matching.ipynb

Attempts to import and use the Faiss library for faster nearest neighbor searches. If Faiss is not available, it falls back to the default 'sklearn' backend.

```python
try:
    from causallib.contrib.faissknn import FaissNearestNeighbors
    knn_backend = FaissNearestNeighbors
except ImportError:
    knn_backend = "sklearn"
```

--------------------------------

### Load Lalonde Dataset for Matching

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Initializes the column structure for the Lalonde dataset to prepare for matching analysis.

```python
import pandas as pd
import numpy as np
columns = ["training",   # Treatment assignment indicator
           "age",        # Age of participant
           "education",  # Years of education
           "black",      # Indicate whether individual is black
           "hispanic",   # Indicate whether individual is hispanic
           "married",    # Indicate whether individual is married
           "no_degree",  # Indicate if individual has no high-school diploma
           "re74",       # Real earnings in 1974, prior to study participation
           "re75",       # Real earnings in 1975, prior to study participation
           "re78"]       # Real earnings in 1978, after study end

#treated = pd.read_csv("http://www.nber.org/~rdehejia/data/nswre74_treated.txt", 
#                      delim_whitespace=True, header=None, names=columns)
#control = pd.read_csv("http://www.nber.org/~rdehejia/data/nswre74_control.txt",
```

--------------------------------

### Analyze Categorical Feature Overlap with Treatment

Source: https://github.com/biomedsciai/causallib/blob/master/examples/MANAGE agricultural data.ipynb

Concatenate and display cross-tabulations for categorical features against the treatment variable. This helps identify features with significant overlap or lack thereof, guiding decisions on data stratification or exclusion.

```python
pd.concat({col: pd.crosstab(X[col], a) for col in X.select_dtypes('uint8').columns.values}, axis=0)
```

--------------------------------

### Initialize Causal Estimators

Source: https://github.com/biomedsciai/causallib/blob/master/examples/fast_food_employment_card_krueger.ipynb

Imports and configures various causal inference estimators from the causallib library for comparative analysis.

```python
from sklearn.linear_model import LogisticRegression, LinearRegression
from matplotlib import pyplot as plt
from causallib.estimation import (
    PropensityFeatureStandardization,
    WeightedStandardization,
    AIPW,
    IPW,
    MarginalOutcomeEstimator,
    Standardization,
    StratifiedStandardization,
    Matching,
    PropensityMatching,
)
from causallib.preprocessing.transformers import MatchingTransformer


def learner(): return LogisticRegression(
    solver="liblinear", class_weight="balanced", max_iter=5000)


def makeipw(): return IPW(learner=learner())
def makestd(): return StratifiedStandardization(learner=LinearRegression())


all_estimators = [
    MarginalOutcomeEstimator(learner=LinearRegression()),
    Matching(with_replacement=False),
```

--------------------------------

### Initialize IPW Estimator

Source: https://github.com/biomedsciai/causallib/blob/master/examples/nhefs.ipynb

Create an IPW instance using the previously defined learner.

```python
from causallib.estimation import IPW

ipw = IPW(learner)
```

--------------------------------

### Initialize UnivariateBoundingBox Data

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Load the NHEFS dataset and prepare the features and treatment variables for positivity analysis.

```python
import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
from causallib.positivity import UnivariateBoundingBox
from causallib.datasets import load_nhefs
data = load_nhefs(augment=False, onehot=False)
X, a, y = data.X, data.a, data.y
```

--------------------------------

### Configure NearestNeighbors with Custom Metric

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Initialize a NearestNeighbors backend using the custom log odds distance function.

```python
from causallib.estimation import PropensityMatching
from sklearn.neighbors import NearestNeighbors
from sklearn.linear_model import LogisticRegression

logodds_knn = NearestNeighbors(metric=logodds_distance)
```

--------------------------------

### Initialize IPW Model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/Bank-Marketing.ipynb

Configure the IPW estimator using a scikit-learn LogisticRegression model.

```python
from sklearn.linear_model import LogisticRegression
from causallib.estimation import IPW

lr = LogisticRegression(solver='lbfgs', max_iter=1000)
#lr = LogisticRegression(penalty='l1', solver='saga', max_iter=1000)
#lr = GradientBoostingClassifier()
ipw = IPW(lr)
```

--------------------------------

### Initialize PropensityTransformer

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching.ipynb

Instantiate a PropensityTransformer with a LogisticRegression learner. Set include_covariates to False to use only the learned propensity model.

```python
propensity_transform = PropensityTransformer(
    learner=LogisticRegression(
        solver="liblinear",
        class_weight="balanced"),
    include_covariates=False)
```

--------------------------------

### Train X-learner model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/xlearner.ipynb

Initializes and fits an X-learner estimator using GradientBoostingRegressor for both outcome and effect models.

```python
data = load_nhefs()
xlearner = XLearner(
    outcome_model=StratifiedStandardization(GradientBoostingRegressor()),
    effect_model=Standardization(GradientBoostingRegressor()),
    effect_types="diff",
)
_ = xlearner.fit(data["X"], data["a"], data["y"])
effect = xlearner.estimate_effect(data["X"], data["a"], agg="population")  #
print(f"{effect['diff']:.6f}")
```

--------------------------------

### Initialize and Fit MatchingTransformer

Source: https://github.com/biomedsciai/causallib/blob/master/examples/fast_food_employment_card_krueger.ipynb

Initialize MatchingTransformer for matching without replacement and fit it to the data. The propensity model is automatically fit during this step.

```python
mt = MatchingTransformer(with_replacement=False, propensity_transform=propensity_transform)
Xm, am, ym = mt.fit_transform(X, a, y)
print(f"Transforming data from {X.shape[0]} samples to {Xm.shape[0]} samples by matching without replacement.")
```

--------------------------------

### Import Matching and FaissNearestNeighbors

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Imports the necessary classes for performing matching and using the FAISS backend for nearest neighbor searches.

```python
from causallib.estimation import Matching
from causallib.contrib.faissknn import FaissNearestNeighbors
```

--------------------------------

### Configure and execute model comparison

Source: https://github.com/biomedsciai/causallib/blob/master/examples/causal_survival_analysis.ipynb

Initializes a dictionary of various survival models and passes it to the plotting function.

```python
MODELS_DICT = {
    "MarginalSurvival Kaplan-Meier": MarginalSurvival(survival_model=None),
    "MarginalSurvival LogisticRegression": MarginalSurvival(
        survival_model=LogisticRegression(max_iter=2000)
    ),
    "MarginalSurvival PiecewiseExponential": MarginalSurvival(
        survival_model=lifelines.PiecewiseExponentialFitter(breakpoints=range(1, 120, 10))
    ),
    "WeightedSurvival Kaplan-Meier": WeightedSurvival(
        weight_model=IPW(LogisticRegression(max_iter=2000)), survival_model=None
    ),
    "WeightedSurvival LogisticRegression": WeightedSurvival(
        weight_model=IPW(LogisticRegression(max_iter=2000)),
        survival_model=LogisticRegression(max_iter=2000),
    ),
    "WeightedSurvival WeibullFitter": WeightedSurvival(
        weight_model=IPW(LogisticRegression(max_iter=2000)),
        survival_model=lifelines.WeibullFitter(),
    ),
    "StandardizedSurvival LogisticRegression": StandardizedSurvival(
        survival_model=LogisticRegression(max_iter=2000)
    ),
    "StandardizedSurvival Cox": StandardizedSurvival(survival_model=lifelines.CoxPHFitter()),
    "WeightedStandardizedSurvival": WeightedStandardizedSurvival(
        weight_model=IPW(LogisticRegression(max_iter=2000)),
        survival_model=LogisticRegression(max_iter=2000),
    ),
}

plot_multiple_models(MODELS_DICT)
```

--------------------------------

### Initialize and fit Standardization model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/Bank-Marketing.ipynb

Uses GradientBoostingClassifier to fit a Standardization model for causal effect estimation.

```python
from sklearn.ensemble import GradientBoostingClassifier
from causallib.estimation import Standardization

#gb = LinearRegression()
gb = GradientBoostingClassifier()
std = Standardization(gb, predict_proba=True)
std.fit(X, a, y)
```

--------------------------------

### Perform Matching with Replacement and Constraints

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching.ipynb

Configures a Matching estimator with replacement, a neighbor limit, and a caliper constraint before fitting and matching.

```python
matcher = Matching(with_replacement=True, 
                   propensity_transform=propensity_transform, 
                   n_neighbors=3,
                   caliper=0.001).fit(X, a, y)
match_df_3 = matcher.match(X, a)
match_df_3
```

--------------------------------

### Run Simulation SI3

Source: https://github.com/biomedsciai/causallib/blob/master/examples/rlearner.ipynb

Execute simulation SI3 with randomized assignment and complex non-linear response, visualizing results with KDE and boxplots.

```python
data_generating_func = lambda seed: generate_data_SI3(n=300, d=20, seed=seed)
df_results = simulation_runner(data_generating_func)

# plot the estimated effects
fig, ax = plt.subplots(ncols=2, figsize=(14,5))
sns.kdeplot(data=df_results, x='effect', hue='method', ax=ax[0])
sns.boxplot(data=df_results, y='effect', x='method', linewidth=0.8, ax=ax[1]);
```

--------------------------------

### Instantiate and Fit IPW Model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/Bank-Marketing.ipynb

Initializes an IPW model using a logistic regression learner and fits it to the treatment data.

```python
lr = LogisticRegression(solver='lbfgs', max_iter=1000)
#lr = LogisticRegression(penalty='l1', solver='saga', max_iter=1000)
#lr = GradientBoostingClassifier()
ipw = IPW(lr)
ipw.fit(X, a)
```

--------------------------------

### Generate Simulated Data for X-Learner

Source: https://github.com/biomedsciai/causallib/blob/master/examples/xlearner.ipynb

Creates a synthetic dataset with linear response and imbalanced treatment assignment for benchmarking.

```python
def _generate_X(n, d, seed=0):
    rng = np.random.RandomState(seed)
    sigma = np.array([[0.7 ** (np.abs(i - j)) for i in range(d)] for j in range(d)])
    X = pd.DataFrame(rng.multivariate_normal(np.zeros(d), sigma, size=n))
    return X


def generate_data_SI1(n=500, d=20, seed=0, treat_fraction=0.02):
    """
    Simulation SI1 - linear response and imbalanced treatment assignment
    """
    rng = np.random.RandomState(seed)
    X = _generate_X(n, d, seed)
    a = pd.Series(rng.binomial(1, treat_fraction, n))

    beta = rng.uniform(-5, 5, size=(d,))
    mu_0 = np.dot(X, beta) + 5 * np.array(X.iloc[:, 0] > 0.5)
    mu_1 = mu_0 + 8 * np.array(X.iloc[:, 1] > 0.1)
    tau = mu_1 - mu_0
    y = a * mu_1 + (1 - a) * mu_0 + rng.normal(size=(n,))
    return {"X": X, "a": a, "y": y, "tau": tau, "mu_0": mu_0, "mu_1": mu_1}


# we create the data set here so we could accurately compare the run time.
sim_data = generate_data_SI1(n=50000, d=20, seed=0)
```

--------------------------------

### Import Adversarial Balancing

Source: https://github.com/biomedsciai/causallib/blob/master/causallib/contrib/README.md

Import the AdversarialBalancing model for causal inference.

```python
from causallib.contrib.adversarial_balancing import AdversarialBalancing
```

--------------------------------

### Fit and Predict Positivity Support

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Configure the bounding box with specific columns and quantile alpha, then fit the model to the data to predict support.

```python
uvbb = UnivariateBoundingBox(quantile_alpha=0.05, continuous_columns=["age", "smokeintensity", "smokeyrs", "wt71"])
uvbb.fit(X, a)
overlap = uvbb.predict(X, a)
```

--------------------------------

### Load Data and Libraries

Source: https://github.com/biomedsciai/causallib/blob/master/examples/standardization.ipynb

Imports necessary libraries and loads the NHEFS dataset for analysis. The dataset concerns the effect of smoking cessation on weight loss.

```python
%matplotlib inline
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import GradientBoostingRegressor

from causallib.datasets import load_nhefs
from causallib.estimation import Standardization, StratifiedStandardization
from causallib.evaluation import evaluate
```

```python
data = load_nhefs()
data.X.join(data.a).join(data.y).head()
```

--------------------------------

### Initialize IPW Causal Model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/Dehejia_Wahba_replication.ipynb

Instantiates the Inverse Treatment Probability Weighting (IPW) causal model from the causallib library, passing the configured logistic regression learner.

```python
from causallib.estimation import IPW

ipw = IPW(learner)
```

--------------------------------

### Initialize PropensityFeatureStandardization with Weight Covariates

Source: https://github.com/biomedsciai/causallib/blob/master/examples/doubly_robust.ipynb

Initializes a PropensityFeatureStandardization model, specifying weight covariates explicitly. If `outcome_covariates` is not provided, all covariates in `data.X` are used for the outcome model. The model is then fitted to the data.

```python
# Say smoke quitting does not depend on your weight and on your age
weight_covariates = [col for col in data.X.columns 
                     if not col.startswith("wt") and not col.startswith("age")]
ipw = IPW(LogisticRegression(solver="liblinear"))
std = Standardization(LinearRegression())
dr = PropensityFeatureStandardization(std, ipw, 
                                      weight_covariates=weight_covariates)
# By not specifying `outcome_covariates` the model will use all covariates
dr.fit(data.X, data.a, data.y);
```

--------------------------------

### Time Matching with FaissNearestNeighbors Backend (Instance)

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Measures the execution time for fitting a Matching model and estimating potential outcomes using an instance of FaissNearestNeighbors with a specific index type ('ivfflat'). This shows how to configure the FAISS backend for potentially different performance characteristics.

```python
%%timeit -n 2 -r 3 
m = Matching(knn_backend=FaissNearestNeighbors(index_type="ivfflat"))
m.fit(X,a,y)
y_potential_outcomes = m.estimate_population_outcome(X,a)
```

--------------------------------

### Initialize AIPW with Refitting Options

Source: https://github.com/biomedsciai/causallib/blob/master/examples/doubly_robust.ipynb

Initializes an AIPW model with specified outcome and weight models, including options for clipping weights. The `encode_treatment` parameter is set to `True` for the outcome model.

```python
ipw = IPW(LogisticRegression(solver="liblinear"), clip_min=0.05, clip_max=0.95)
std = Standardization(LinearRegression(), encode_treatment=True)
dr = AIPW(std, ipw)
```

--------------------------------

### Initialize CausalSimulator with No Data

Source: https://github.com/biomedsciai/causallib/blob/master/causallib/datasets/README.md

Initializes the CausalSimulator to generate synthetic data. Requires specifying graph topology, variable types, link functions, and other simulation parameters. Use this when you do not have an existing dataset to base the simulation on.

```Python
import numpy as np
from causallib.datasets import CausalSimulator
topology = np.zeros((4, 4), dtype=np.bool)  # topology[i,j] iff node j is a parent of node i
topology[1, 0] = topology[2, 0] = topology[2, 1] = topology[3, 1] = topology[3, 2] = True
var_types = ["hidden", "covariate", "treatment", "outcome"]
link_types = ['linear', 'linear', 'linear', 'linear']
prob_categories = [[0.25, 0.25, 0.5], None, [0.5, 0.5], None]
treatment_methods = "gaussian"
snr = 0.9
treatment_importance = 0.8
effect_sizes = None
outcome_types = "binary"

sim = CausalSimulator(topology=topology, prob_categories=prob_categories,
                      link_types=link_types, snr=snr, var_types=var_types,
                      treatment_importances=treatment_importance, 
                      outcome_types=outcome_types, 
                      treatment_methods=treatment_methods, 
                      effect_sizes=effect_sizes)
X, prop, (y0, y1) = sim.generate_data(num_samples=100)
```

--------------------------------

### Prepare Training, Validation, and Test Datasets

Source: https://github.com/biomedsciai/causallib/blob/master/examples/hemm_demo.ipynb

Splits the synthetic data into training, validation, and test sets. It also extracts features, outcomes, and treatment assignments.

```python
#Leave 30% of the Training Data as Validation
vsize = int(0.3*syn_data['TRAIN']['x'].shape[0])

#setting the Training Dataset
Xtr  = syn_data['TRAIN']['x'] [:-vsize,:, 0]
Ytr  = syn_data['TRAIN']['yf'][:-vsize  , 0]
Ttr  = syn_data['TRAIN']['t'] [:-vsize  , 0]

#setting the Dev/Val Dataset
Xdev = syn_data['TRAIN']['x'] [-vsize:,:, 0]
Ydev = syn_data['TRAIN']['yf'][-vsize:  , 0]
Tdev = syn_data['TRAIN']['t'] [-vsize:  , 0]

#setting the Test Dataset
Xte  = syn_data['TEST']['x'] [:,:, 0]
Yte  = syn_data['TEST']['yf'][:  , 0]
Tte  = syn_data['TEST']['t'] [:  , 0]

#Feature size
Xdim = Xtr.shape[1]

#Number of Components to Discover.
K = 3

#Initialize the model with Population means and Std. Devs.
#Empirically results in faster, and better convergence.
mu  = Xtr.mean(axis=0).reshape(1,-1)
std = Xtr.std(axis=0).reshape(1,-1)

#Set the Learning Rate for Adam Optimizer and Batch Size
learning_rate = 1e-4
batch_size    = 100

#Indicate the Outcome Distribution (Y), could be 'bin' for 
#Binary Outcomes or 'cont' for Continuous Outcomes.
response = 'bin'

#Indicate what kind of a model to be used to adjust for 
#confounding. You can also pass your own PyTorch Model!!!!
outcome_model='linear'

#Instantiate an HEMM model
model = HEMM(Xdim, K, homo=True, mu=mu, std=std, bc=2, lamb=0.0000,
                spread=.1,outcome_model=outcome_model,sep_heads=True,epochs=10,
                 learning_rate=learning_rate,weight_decay=0.0001,metric='LL', use_p_correction=False,
                 response=response,imb_fun=None,batch_size=batch_size )


#Fit the HEMM model on the Training Data, with Early Stopping
#on the Dev Set.
cd = model.fit(Xtr, Ttr,Ytr, validation_data=(Xdev, Tdev, Ydev))
```

--------------------------------

### Initialize and Fit WeightedStandardization

Source: https://github.com/biomedsciai/causallib/blob/master/examples/doubly_robust.ipynb

Initializes and fits a WeightedStandardization model using IPW for weights and Logistic Regression for the weight model, and Standardization with Linear Regression for the outcome model. This is used for doubly robust estimation.

```python
ipw = IPW(LogisticRegression(solver="liblinear"))
std = Standardization(LinearRegression())
dr = WeightedStandardization(std, ipw)
dr.fit(data.X, data.a, data.y)
```

--------------------------------

### Import CausalLib modules

Source: https://github.com/biomedsciai/causallib/blob/master/examples/doubly_robust.ipynb

Initial imports required for setting up causal models and loading datasets.

```python
%matplotlib inline
from sklearn.linear_model import LogisticRegression, LinearRegression
from causallib.datasets import load_nhefs
from causallib.estimation import IPW, Standardization, StratifiedStandardization
from causallib.estimation import AIPW, PropensityFeatureStandardization, WeightedStandardization
from causallib.evaluation import evaluate
```

--------------------------------

### Fit UnivariateBoundingBox Models

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Initializes and fits the UnivariateBoundingBox model with different thresholds to analyze feature supports.

```python
uvbbll = UnivariateBoundingBox(0.0, continuous_columns=["age", "re74", "re75", "education"]).fit(X_ll, a_ll)

plot_supports(uvbbll, ["age", "re74", "re75", "education"])
```

```python
uvbbll_q005 = UnivariateBoundingBox(0.05, continuous_columns=["age", "re74", "re75", "education"]).fit(X_ll, a_ll)

plot_supports(uvbbll, ["age", "re74", "re75", "education"])
```

--------------------------------

### Initialize PropensityTransformer

Source: https://github.com/biomedsciai/causallib/blob/master/examples/fast_food_employment_card_krueger.ipynb

Initialize PropensityTransformer to be used within MatchingTransformer. Set include_covariates to True to include covariates in the propensity score calculation.

```python
from causallib.preprocessing.transformers import PropensityTransformer
propensity_transform = PropensityTransformer(learner=learner(), include_covariates=True)
```

--------------------------------

### Initialize CausalSimulator

Source: https://github.com/biomedsciai/causallib/blob/master/examples/causal_simulator.ipynb

Create a CausalSimulator instance with specified topology, variable types, outcome types, link functions, and simulation parameters like SNR and treatment importance.

```python
outcome_types = "categorical"
link_types = ['linear'] * len(var_types)
prob_categories = pd.Series(data=[[0.5, 0.5] if typ in ["treatment", "outcome"] else None for typ in var_types],
                            index=var_types.index)
treatment_methods = "gaussian"
snr = 0.9
treatment_importance = 0.8
effect_sizes = None
sim = CausalSimulator(topology=topology.values, prob_categories=prob_categories,
                      link_types=link_types, snr=snr, var_types=var_types,
                      treatment_importances=treatment_importance,
                      outcome_types=outcome_types,
                      treatment_methods=treatment_methods,
                      effect_sizes=effect_sizes)
```

--------------------------------

### Load and Preprocess Lalonde Dataset

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Downloads multiple data files, concatenates them, and performs feature engineering to prepare the dataset for causal analysis.

```python
columns = ["training",   # Treatment assignment indicator
           "age",        # Age of participant
           "education",  # Years of education
           "black",      # Indicate whether individual is black
           "hispanic",   # Indicate whether individual is hispanic
           "married",    # Indicate whether individual is married
           "no_degree",  # Indicate if individual has no high-school diploma
           "re74",       # Real earnings in 1974, prior to study participation
           "re75",       # Real earnings in 1975, prior to study participation
           "re78"]       # Real earnings in 1978, after study end
file_names = ["http://www.nber.org/~rdehejia/data/nswre74_treated.txt",
              "http://www.nber.org/~rdehejia/data/nswre74_control.txt",
              "http://www.nber.org/~rdehejia/data/psid_controls.txt",
              "http://www.nber.org/~rdehejia/data/psid2_controls.txt",
              "http://www.nber.org/~rdehejia/data/psid3_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps2_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps3_controls.txt"]
files = [pd.read_csv(file_name, delim_whitespace=True,
                     header=None, names=columns) for file_name in file_names]
lalonde = pd.concat(files, ignore_index=True)
lalonde = lalonde.sample(frac=1.0, random_state=42)  # Shuffle
lalonde = lalonde.sample(frac=1.0, random_state=42)  # Shuffle
lalonde = lalonde.join(
    (lalonde[["re74", "re75"]] == 0).astype(int), rsuffix=("=0"))

print(lalonde.shape)

a_ll = lalonde.pop("training")
y_ll = lalonde.pop("re78")
X_ll = lalonde
```

--------------------------------

### Time Matching with FaissNearestNeighbors Backend (Class)

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Measures the execution time for fitting a Matching model and estimating potential outcomes using the FaissNearestNeighbors backend specified by its class name. This demonstrates the speed improvement over the 'sklearn' backend.

```python
%%timeit -n 2 -r 3 
m = Matching(knn_backend=FaissNearestNeighbors)
m.fit(X,a,y)
y_potential_outcomes = m.estimate_population_outcome(X,a)
```

--------------------------------

### Import Causallib and Scikit-learn Models

Source: https://github.com/biomedsciai/causallib/blob/master/examples/MANAGE agricultural data.ipynb

Import necessary classes from causallib for estimation and scikit-learn for the underlying model.

```python
from causallib.estimation import IPW
from sklearn.linear_model import LogisticRegression
```

--------------------------------

### Evaluate Propensity Model and Plot Results

Source: https://github.com/biomedsciai/causallib/blob/master/examples/MANAGE agricultural data.ipynb

Evaluate the propensity model using provided data and visualize the results. This helps in understanding the model's performance and identifying potential issues with data balancing.

```python
evaluation_results = evaluate(ipw, X, a, y)
evaluation_results.plot_all()
plt.tight_layout()
```

--------------------------------

### Initialize and Run TMLE Simulation

Source: https://github.com/biomedsciai/causallib/blob/master/examples/TMLE.ipynb

Initializes a TMLE object with specified outcome and weight models, then runs multiple simulations to collect results and the true ATE. Requires outcome and weight models to be defined and standardized.

```python
outcome_model = Standardization(outcome_model)
weight_model = IPW(weight_model)

tmle = TMLE(
    outcome_model=outcome_model,
    weight_model=weight_model,
    reduced=True
)

n_samples = 500
n_simulations = 20

results, true_ate = run_multiple_simulations(tmle, n_simulations, n_samples)
plot_multiple_simulations(results, true_ate);
```

--------------------------------

### Import R-Learner and Dependencies

Source: https://github.com/biomedsciai/causallib/blob/master/examples/rlearner.ipynb

Standard imports for data manipulation, visualization, machine learning models, and CausalLib components required to run R-Learner.

```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from collections import defaultdict
import seaborn as sns
sns.set_palette("deep")
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')

from sklearn.linear_model import LinearRegression, LogisticRegression, LassoCV
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import PolynomialFeatures
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import mean_squared_error


from causallib.estimation import StratifiedStandardization, Standardization
from causallib.estimation import RLearner
```

--------------------------------

### Initialize and fit StratifiedStandardization model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/Bank-Marketing.ipynb

Uses StratifiedStandardization to fit separate regressors for each treatment group.

```python
from causallib.estimation import StratifiedStandardization
std = StratifiedStandardization(gb, predict_proba=True)
std.fit(X, a, y)

outcomes = std.estimate_population_outcome(X, a).xs(1, level='y')
print(outcomes)
std.estimate_effect(outcomes[0], outcomes[1], effect_types=['diff','ratio'])
```

--------------------------------

### Cite the causallib package

Source: https://github.com/biomedsciai/causallib/blob/master/README.md

BibTeX reference for citing the causallib evaluation toolkit.

```bibtex
@article{causalevaluations,
  title={An Evaluation Toolkit to Guide Model Selection and Cohort Definition in Causal Inference},
  author={Shimoni, Yishai and Karavani, Ehud and Ravid, Sivan and Bak, Peter and Ng, Tan Hung and Alford, Sharon Hensley and Meade, Denise and Goldschmidt, Yaara},
  journal={arXiv preprint arXiv:1906.00442},
  year={2019}
}
```

--------------------------------

### Define Trimming and Visualization Functions

Source: https://github.com/biomedsciai/causallib/blob/master/examples/positivity.ipynb

Helper functions to create trimmed datasets and visualize observations in 2D space.

```python
def create_trimmed_data(X, a, threshold=[0.01,0.05,'crump']):
    filtered_series = dict()
    tr = Trimming()
    tr.fit(X, a)
    for th in threshold:
        filtered_series['filtered_' + str(th)] = tr.predict(X, a, threshold=th)
    df_filtered = pd.DataFrame.from_dict(filtered_series)

    df = pd.concat([X, a, df_filtered], axis=1)
    df.columns = ['X_1','X_2','a', *list(filtered_series.keys())]
    return df

def plot_trimmed_obs_2d_spcae(data, number_of_methods=3):
    fig, ax = plt.subplots(nrows=1, ncols=number_of_methods, figsize=(15,6),sharey=True)
    for ind, fliter_type in enumerate(data.columns[3:]):
        sns.scatterplot(data=data, x="X_1", y="X_2", hue="a", style=fliter_type, ax=ax[ind], style_order=[True, False])
        ax[ind].set_title('2d space with treatment assignment\nx - are observations that filtered out ({})'
                          .format(fliter_type), fontsize=11)
    plt.tight_layout()
```

--------------------------------

### Import necessary libraries

Source: https://github.com/biomedsciai/causallib/blob/master/examples/causal_simulator.ipynb

Import pandas for data manipulation and specific modules from causallib for datasets and simulation.

```python
import pandas as pd
from causallib.datasets import load_nhefs
from causallib.datasets import CausalSimulator
from causallib.datasets import generate_random_topology
```

--------------------------------

### Initialize IPW Estimator

Source: https://github.com/biomedsciai/causallib/blob/master/examples/MANAGE agricultural data.ipynb

Initialize the IPW (Inverse Probability Weighting) estimator with a specified logistic regression model.

```python
lr = LogisticRegression(solver='lbfgs', max_iter=500)
ipw = IPW(lr)
```

--------------------------------

### Initialize Logistic Regression Model

Source: https://github.com/biomedsciai/causallib/blob/master/examples/ipw.ipynb

Specify a scikit-learn compatible model with custom parameters for use in IPW. Ensure the solver is compatible with the chosen penalty.

```python
learner = LogisticRegression(penalty="l1", C=0.01, max_iter=500, solver='liblinear')
```

--------------------------------

### Load and Prepare LaLonde Dataset

Source: https://github.com/biomedsciai/causallib/blob/master/examples/lalonde.ipynb

Downloads multiple data files from the NBER repository, concatenates them into a single pandas DataFrame, and shuffles the rows.

```python
import pandas as pd

columns = ["training",   # Treatment assignment indicator
           "age",        # Age of participant
           "education",  # Years of education
           "black",      # Indicate whether individual is black
           "hispanic",   # Indicate whether individual is hispanic
           "married",    # Indicate whether individual is married
           "no_degree",  # Indicate if individual has no high-school diploma
           "re74",       # Real earnings in 1974, prior to study participation
           "re75",       # Real earnings in 1975, prior to study participation
           "re78"]       # Real earnings in 1978, after study end

#treated = pd.read_csv("http://www.nber.org/~rdehejia/data/nswre74_treated.txt", 
#                      delim_whitespace=True, header=None, names=columns)
#control = pd.read_csv("http://www.nber.org/~rdehejia/data/nswre74_control.txt",
#                      delim_whitespace=True, header=None, names=columns)
file_names = ["http://www.nber.org/~rdehejia/data/nswre74_treated.txt",
              "http://www.nber.org/~rdehejia/data/nswre74_control.txt",
              "http://www.nber.org/~rdehejia/data/psid_controls.txt",
              "http://www.nber.org/~rdehejia/data/psid2_controls.txt",
              "http://www.nber.org/~rdehejia/data/psid3_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps2_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps3_controls.txt"]
files = [pd.read_csv(file_name, delim_whitespace=True, header=None, names=columns) for file_name in file_names]
lalonde = pd.concat(files, ignore_index=True)
lalonde = lalonde.sample(frac=1.0, random_state=42)  # Shuffle

print(lalonde.shape)
lalonde.head()
```

--------------------------------

### Display First Few Rows of Dataset

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Displays the first few rows of the Lalonde DataFrame to give a preview of the data.

```python
lalonde.head()
```

--------------------------------

### Load and Prepare Lalonde Dataset

Source: https://github.com/biomedsciai/causallib/blob/master/examples/matching_with_custom_backends.ipynb

Loads data from multiple URLs, concatenates them into a single DataFrame, shuffles it, and prints its shape. This is a common first step for using the Lalonde dataset.

```python
file_names = ["http://www.nber.org/~rdehejia/data/nswre74_treated.txt",
              "http://www.nber.org/~rdehejia/data/nswre74_control.txt",
              "http://www.nber.org/~rdehejia/data/psid_controls.txt",
              "http://www.nber.org/~rdehejia/data/psid2_controls.txt",
              "http://www.nber.org/~rdehejia/data/psid3_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps2_controls.txt",
              "http://www.nber.org/~rdehejia/data/cps3_controls.txt"]
files = [pd.read_csv(file_name, delim_whitespace=True, header=None, names=columns) for file_name in file_names]
lalonde = pd.concat(files, ignore_index=True)
lalonde = lalonde.sample(frac=1.0, random_state=42)  # Shuffle

print(lalonde.shape)
lalonde.head()
```