### Install super_batch library with pip

Source: https://sparsesc.readthedocs.io/en/latest/azure_batch.html

Install the required super_batch library from GitHub. This is needed to run SparseSC with Azure Batch. The module has only been tested with Python 3.7, and you will also need pyyaml and psutil.

```bash
pip install git+https://github.com/jdthorpe/batch-config.git

```

--------------------------------

### Fit a SparseSC model and make predictions

Source: https://sparsesc.readthedocs.io/en/latest/fit.html

Use the fit() function to fit a synthetic control model, then use the model's predict() method to get in-sample predictions for the targets and predictions for additional held-out features.

```python
from SparseSC import fit

# Fit the model:
fitted_model = fit(features,targets,...)

# Get the fitted synthetic controls for `targets`:
in_sample_predictions = fitted_model.predict()

# Make predictions for a held out set of fetures (targets_hat) 
# using the fitted synthetic controls model:
additional_predictions = fitted_model.predict(targets_additional)
```

--------------------------------

### Reconnect to Azure Batch job and load results

Source: https://sparsesc.readthedocs.io/en/latest/azure_batch.html

Use this code to reconnect to a previously created Azure Batch job and download the results after a disconnection. It assumes you have recreated the batch_client as shown in the example above.

```python
batch_client.load_results()
fitted_model = aggregate_batch_results(batch_dir)

```

--------------------------------

### Fit SparseSC.estimate_effects and access results

Source: https://sparsesc.readthedocs.io/en/latest/estimate-effects.html

Example usage of the `estimate_effects()` function from the SparseSC package to conduct a differences-in-differences style analysis with sparse synthetic controls. The snippet shows how to fit the model, print a summary, extract the p-value and confidence interval of the average joint effect, and access the fitted synthetic controls model. The returned object is of class `SparseSCEstResults`.

```python
import SparseSC

# Fit the model:
fitted_estimates = SparseSC.estimate_effects(outcomes,unit_treatment_periods,covariates=X,fast=True,...)

# Print summary of the model including effect size estimates, 
# p-values, and confidendence intervals:
print(fitted_estimates)

# Extract model attributes:
fitted_estimates.pl_res_post.avg_joint_effect.p_value
fitted_estimates.pl_res_post.avg_joint_effect.CI

# access the fitted Synthetic Controls model:
fitted_model = fitted_estimates.fit

```

--------------------------------

### SparseSC.get_W

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Get W (np.ndarray 2D or pd.DataFrame depends on what was passed in) for one of the treatment periods.

```APIDOC
## SparseSC.get_W

### Description
Get W (np.ndarray 2D or pd.DataFrame depends on what was passed in) for one of the treatment periods.

### Parameters
- **treatment_period** (optional) - The treatment period to get W for.
```

--------------------------------

### Custom donor pool for interpolation bias example

Source: https://sparsesc.readthedocs.io/en/latest/overview.html

Uses calipers to restrict matching to units with a maximum difference of thresh on a key metric B (an N-vector), limiting interpolation bias. Combines the caliper restriction with the default donor pool.

```python
custom_donor_pool = (np.abs(B[:,np.newaxis] - B) < thresh)[:,control_units].astype('bool')
custom_donor_pool = np.logical_and(custom_donor_pool, base_custom_donor_pool)

```

--------------------------------

### Custom donor pool for interference example

Source: https://sparsesc.readthedocs.io/en/latest/overview.html

Constructs a custom donor pool from an adjacency matrix A to prevent treatment spillover to adjacent units (interference/SUTVA violations). It combines the adjacency-based restriction with the default donor pool.

```python
custom_donor_pool = (1 - A)[:,control_units].astype('bool')
custom_donor_pool = np.logical_and(custom_donor_pool, base_custom_donor_pool)

```

--------------------------------

### Prepare batch parameters with fit()

Source: https://sparsesc.readthedocs.io/en/latest/azure_batch.html

Create the parameters required for a batch job by calling fit() with a batchDir argument. The parameters files are stored in the specified directory.

```python
from SparseSC import fit
batch_dir = "/path/to/my/batch/data/"

# initialize the batch parameters in the directory `batch_dir`
fit(x, y, ... , batchDir = batch_dir)

```

--------------------------------

### Run unittest commands for SparseSC tests

Source: https://sparsesc.readthedocs.io/en/latest/dev_notes.html

Commands to run the built-in unittest suite from the repo root. The first runs a test file (requires Python >=3.5), the second runs a module, the third runs a class, and the fourth runs a single test function.

```bash
python -m unittest test/test_fit.py #file (only Python >=3.5)
python -m unittest test.test_fit #module
python -m unittest test.test_fit.TestFit #class
python -m unittest test.test_fit.TestFit.test_retrospective #function

```

--------------------------------

### SparseSC.__init__

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Initializes the SparseSC estimator with outcomes, fits, treatment period information, and placebo results. This constructor sets up the model for estimating treatment effects.

```APIDOC
## SparseSC.__init__

### Description
Initializes the SparseSC estimator with outcomes, fits, treatment period information, and placebo results. This constructor sets up the model for estimating treatment effects.

### Method
Constructor

### Parameters
- **outcomes** - Outcome for the whole sample
- **fits** (dictionary of period->SparseSCFit) - The fit() return objects
- **unit_treatment_periods** - Vector of treatment periods for each unit (use np.NaN if never treated, np.datetime64('NaT') if using DateTime)
- **unit_treatment_periods_idx** - The conversion of unit_treatment_periods to indexes
- **unit_treatment_periods_idx_fit** - The treatment period indexes passed to fit
- **T0** - Pre-history to match over
- **T1** - post-history to evaluate over
- **pl_res_pre** (PlaceboResults) - Statistics for the average fit of the treated units in the pre-period
- **pl_res_post** (PlaceboResults) - Statistics for the average treatment effect in the post-period
- **pl_res_post_scaled** (PlaceboResults) - Statistics for the average scaled treatment effect
- **max_n_pl** - Maximum number of placebo effects used for inference
- **covariates** (optional) - Nxk matrix of full baseline covariates
- **ind_CI** (optional, dictionary of period->CI_int) - Confidence intervals for SC predictions at the unit level
- **model_type** (optional) - Model type string
- **T2** (optional) - T2 (if prospective-type design)
- **pl_res_post_fit** (optional) - PlaceboResults for target period used for fit (if prospective-type design)
```

--------------------------------

### PlaceboResults.__init__

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Holds statistics for a vector of effects, including the full vector and two choices of aggregates (average and RMS).

```APIDOC
## PlaceboResults.__init__

### Description
Holds statistics for a vector of effects, including the full vector and two choices of aggregates (average and RMS).

### Parameters
- **effect_vec** (EstResultCI) - Statistics for a vector of time-specific effects.
- **avg_joint_effect** (EstResultCI) - Statistics for the average effect.
- **rms_joint_effect** (EstResultCI) - Statistics for the RMS effect.
- **N_placebo** (EstResultCI) - Number of placebos used for the statistics.
```

--------------------------------

### Execute Azure Batch job with super_batch Client

Source: https://sparsesc.readthedocs.io/en/latest/azure_batch.html

Creates a Batch configuration and runs the job. Credentials are taken from environment variables. The job name must be unique, and a timestamp is used. The pool is kept after the run to save time for multiple jobs. After running, results are aggregated into a fitted model.

```python
import os
from datetime import datetime
from super_batch import Client
from SparseSC.utils.AzureBatch import (
    DOCKER_IMAGE_NAME,
    create_job,
)
# Batch job names must be unique, and a timestamp is one way to keep it uniquie across runs
timestamp = datetime.utcnow().strftime("%H%M%S")
batch_dir = "/path/to/my/batch/data/"

batch_client = Client(
    # Name of the VM pool
    POOL_ID= name,
    # number of standard nodes
    POOL_NODE_COUNT=5,
    # number of low priority nodes
    POOL_LOW_PRIORITY_NODE_COUNT=5,
    # VM type
    POOL_VM_SIZE= "STANDARD_A1_v2",
    # Job ID.  Note that this must be unique.
    JOB_ID= name + timestamp,
    # Name of the storage container for storing parameters and results
    CONTAINER_NAME= name,
    # local directory with the parameters, and where the results will go
    BATCH_DIRECTORY= batch_dir,
    # Keep the pool around after the run, which saves time when doing
    # multiple batch jobs, as it typically takes a few minutes to spin up a
    # pool of VMs. (Optional. Default = False)
    DELETE_POOL_WHEN_DONE=False,
    # Keeping the job details can be useful for debugging:
    # (Optional. Default = False)
    DELETE_JOB_WHEN_DONE=False
)

create_job(batch_client, batch_dir)
# run the batch job
batch_client.run()

# aggregate the results into a fitted model instance
fitted_model = aggregate_batch_results(batch_dir)

```

--------------------------------

### Create Anaconda environment from YAML file

Source: https://sparsesc.readthedocs.io/en/latest/dev_notes.html

Creates a conda environment from the test/SparseSC_36.yml file. Use 'update' instead of 'create' to update an existing environment, but ensure the environment is not currently active to avoid potential bugs.

```bash
conda env create -f test/SparseSC_36.yml

```

--------------------------------

### Grid search over v_pen and w_pen using itertools.product

Source: https://sparsesc.readthedocs.io/en/latest/fit.html

Use this snippet to evaluate a grid of penalty parameters when both v_pen and w_pen are vectors. Passing vectors to both parameters directly raises an error, so this approach is required for an N x N grid.

```python
from intertools import product
fitted_models = [ fit(..., v_pen=v, w_pen=w) for v,w in product(v_pen,w_pen)]

```

--------------------------------

### SparseSCFit.summary

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

A summary of the model fit / penalty selection. This illustrates that (a) the grid function could / should be better, and (b) currently more than two iterations is typically useless.

```APIDOC
## SparseSCFit.summary

### Description
A summary of the model fit / penalty selection. This illustrates that (a) the grid function could / should be better, and (b) currently more than two iterations is typically useless.

### Method
Function call
```

--------------------------------

### fit()

Source: https://sparsesc.readthedocs.io/en/latest/fit.html

Fits a synthetic control model using provided penalty parameters. Supports passing v_pen and w_pen as floats or vectors, with optional grid search and cross-validation.

```APIDOC
## fit()

### Description
Fits a synthetic control model using penalty parameters for the V and W matrices. The function can accept single penalty values or perform a grid search over one penalty parameter while holding the other fixed, using cross-validation to select the optimal model.

### Method
Function call

### Endpoint
fit(v_pen, w_pen, ...)

### Parameters
#### Penalty Parameters
- **v_pen** (float or array-like) - Required - Penalty for the V matrix. If a single float, used directly. If an array, a grid search is performed over these values.
- **w_pen** (float or array-like) - Required - Penalty for the W matrix. If a single float, used directly. If an array, a grid search is performed over these values.

#### Selection Parameters
- **choice** (string or callable) - Optional - Model selection rule when searching over a vector of penalties. Options: "min" (default) selects the model with smallest out-of-sample error, "1se" implements the one-standard-error rule, or a custom function.

#### Grid Search Parameters
- **grid_length** (int) - Optional - Number of points in the search grid (default 20).
- **grid_min** (float) - Optional - Lower bound for the search grid.
- **grid_max** (float) - Optional - Upper bound for the search grid.
- **grid** (array-like) - Optional - Array of values between 0 and 1, multiplied by the relevant grid_max to determine the search grid.
- **stopping_rule** (parameter) - Optional - Determines how long the coordinate descent alternates between searching over V and W penalties.

#### Constraint Parameters
- **constrain** (string) - Optional - Constraint for the V matrix: "simplex" (default) or "orthant".

#### Fold Parameters
- **cv_fold** (int or list-of-lists) - Optional - Number of folds or list of unit indices for cross-validation folds.
- **gradient_fold** (int or list-of-lists) - Optional - Number of folds or list of unit indices for gradient descent folds.
- **cv_seed** (int) - Optional - Seed for cross-validation fold generation.
- **gradient_seed** (int) - Optional - Seed for gradient descent fold generation.

### Request Example
```python
from intertools import product
fitted_models = [ fit(..., v_pen=v, w_pen=w) for v,w in product(v_pen,w_pen)]
```

### Response
#### Success Response
Returns a fitted synthetic control model.

#### Error Handling
- Passing vectors to both `v_pen` and `w_pen` raises an error.

```

--------------------------------

### AA_results.__init__(diffs_pre, diffs_post, level=0.95, sym_CI=True)

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Constructs a simple results object from an AA test that has already been fit (model_type="full") and diffs constructed. Provides typical stats for simulations of estimation.

```APIDOC
## AA_results.__init__(diffs_pre, diffs_post, level=0.95, sym_CI=True)

### Description
Constructs a simple results object from an AA test that has already been fit (model_type="full") and diffs constructed. Provides typical stats for simulations of estimation.

### Method
Constructor

### Endpoint
AA_results.__init__

### Parameters
#### Path Parameters
- **diffs_pre** - Required - Pre-treatment diffs
- **diffs_post** - Required - vector of CI lower bounds
- **level** (float) - Optional - level (default: 0.95)
- **sym_CI** (bool) - Optional - Return symmetric CIs. Will also set "treatment" effect to 0 rather than mean of placebo diffs. (default: True)

### Returns
AA_results
```

--------------------------------

### CI_int.__init__

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Class to hold information for a confidence interval (for single point or for a vector).

```APIDOC
## CI_int.__init__

### Description
Class to hold information for a confidence interval (for single point or for a vector).

### Parameters
- **ci_low** (scalar, vector, or pd.Series) - Low-bound
- **ci_high** (scalar, vector, or pd.Series) - High-bound
- **level** (float) - Level (1-alpha) for the CI interval
```

--------------------------------

### fit() and fit_fast()

Source: https://sparsesc.readthedocs.io/en/latest/fit.html

These functions fit synthetic controls using the provided features and targets matrices. The model_type parameter selects among four model types, and the treated_units parameter indicates which units are treated.

```APIDOC
## fit() and fit_fast()

### Description
Fit synthetic controls using the provided features and targets matrices. The model_type parameter selects among four model types, and the treated_units parameter indicates which units are treated.

### Method
Function call

### Endpoint
fit(features, targets, model_type, treated_units)
fit_fast(features, targets, model_type, treated_units)

### Parameters
#### features
- **features** (matrix) - Required - Numeric matrix with one row per unit and one column per feature variable.

#### targets
- **targets** (matrix) - Required - Numeric matrix with one row per unit and one column per target variable.

#### model_type
- **model_type** (string) - Required - One of: "retrospective", "prospective", "prospective-restricted", or "full".

#### treated_units
- **treated_units** (vector) - Required for retrospective and prospective models; unused for full model - Indicates which units are treated (affected by intervention or designated to receive treatment).

### Request Example
```
fit(features = feature_matrix, targets = target_matrix, model_type = "retrospective", treated_units = c(1, 2, 3))
```

### Response
#### Success Response
Returns a fitted synthetic control model object.

#### Response Example
```
Synthetic control model object
```
```

--------------------------------

### SparseSC.utils.match_space.Fixed_V_factory

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns a MatchSpace function with user-supplied V over raw X. This is used to define a fixed match space.

```APIDOC
## SparseSC.utils.match_space.Fixed_V_factory

### Description
Returns a MatchSpace function with user-supplied V over raw X. This is used to define a fixed match space.

### Method
Function call

### Endpoint
`SparseSC.utils.match_space.Fixed_V_factory(V)`

### Parameters
- **V** - Required - V Matrix on the raw features

### Returns
A function with the signature MatchSpace fn, V vector, best_v_pen, V = function(X,Y)
```

--------------------------------

### SparseSCFit.show

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Display goodness of figures illustrating goodness of fit.

```APIDOC
## SparseSCFit.show

### Description
Display goodness of figures illustrating goodness of fit.

### Method
Function call
```

--------------------------------

### Fit and predict with SparseSC for anomaly detection

Source: https://sparsesc.readthedocs.io/en/latest/examples/prospective_anomaly_detection.html

Fits a synthetic control model for prospective anomaly detection using the `fit()` function from `SparseSC`. Divide the outcome matrix into pre-intervention features (`X`) and post-intervention targets (`Y`) at column `t`. Pass `model_type="full"` to allow every unit to serve as a donor unless `custom_donor_pool` is specified; the fit is computationally intensive. After fitting, use the `predict()` method on new observations, which is fast and suitable for real-time anomaly detection.

```python
from numpy import hstack
from SparseSC import fit

# Let X be the features plus some of the targets
X = hstack([features, targets[:,:t])

# And let Y be the remaining targets
Y = targets[:,t:]

# fit the model:
fitted_model = fit(X=X,
                   Y=Y,
                   model_type="full")

```

```python
synthetic_controls = fitted_model.predict(additional_observations)

```

--------------------------------

### SparseSC.fit

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Handy accessor if there is only one treatment-period.

```APIDOC
## SparseSC.fit

### Description
Handy accessor if there is only one treatment-period.
```

--------------------------------

### SparseSC.utils.match_space.MTLassoMixed_MatchSpace_factory

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns a MatchSpace function that will fit a MultiTaskLassoCV for Y ~ X with the penalization optimized to reduce errors on goal units.

```APIDOC
## SparseSC.utils.match_space.MTLassoMixed_MatchSpace_factory

### Description
Returns a MatchSpace function that will fit a MultiTaskLassoCV for Y ~ X with the penalization optimized to reduce errors on goal units.

### Method
Function

### Endpoint
SparseSC.utils.match_space.MTLassoMixed_MatchSpace_factory

### Parameters
#### Parameters
- **v_pens** (optional) - Penalties to evaluate (default is to automatically determine)
- **n_v_cv** (optional) - Number of Cross-Validation folds

### Returns
- MatchSpace fn
- V vector
- best_v_pen
- V
```

--------------------------------

### EstResultCI.__init__

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Holds an estimation result (effect + statistical significance).

```APIDOC
## EstResultCI.__init__

### Description
Holds an estimation result (effect + statistical significance).

### Parameters
- **effect** (scalar, vector, or pd.Series) - Effect
- **p** (scalar or vector) - p-value
- **ci** (CI_int) - Confidence interval
- **placebos** (matrix) - Full matrix of placebos
```

--------------------------------

### SparseSC.get_tr_time_info

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns treatment info: a) indexes for the time index (helpful if user used an np datetime index) b) treatment period used in the call to fit (helpful if model is a prospective type).

```APIDOC
## SparseSC.get_tr_time_info

### Description
Returns treatment info: a) indexes for the time index (helpful if user used an np datetime index) b) treatment period used in the call to fit (helpful if model is a prospective type).

### Parameters
- **treatment_period** - Treatment period in the user's view (could be TimeIndex)

### Returns
- (treatment_period_idx, treatment_period_idx_fit, treatment_period_fit)
```

--------------------------------

### SparseSC.utils.match_space.MTLassoCV_MatchSpace_factory

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns a MatchSpace function that will fit a MultiTaskLassoCV for Y ~ X. This is used to automatically determine the match space.

```APIDOC
## SparseSC.utils.match_space.MTLassoCV_MatchSpace_factory

### Description
Returns a MatchSpace function that will fit a MultiTaskLassoCV for Y ~ X. This is used to automatically determine the match space.

### Method
Function call

### Endpoint
`SparseSC.utils.match_space.MTLassoCV_MatchSpace_factory(v_pens=None, n_v_cv=5, sample_frac=1, Y_col_block_size=None, se_factor=None, normalize=True)`

### Parameters
- **v_pens** - Optional - Penalties to evaluate (default is to automatically determince)
- **n_v_cv** - Optional - Number of Cross-Validation folds
- **sample_frac** - Optional - Fraction of the data to sample
- **se_factor** - Optional - Allows taking a different penalty than the min mse. Similar to the lambda.1se rule, if not None, it will take the max lambda that has mse < min_mse + se_factor*(MSE standard error).

### Returns
MatchSpace fn, V vector, best_v_pen, V
```

--------------------------------

### estimate_effects()

Source: https://sparsesc.readthedocs.io/en/latest/estimate-effects.html

Documentation for the estimate_effects() function used to conduct DID style analyses with Sparse Synthetic Controls.

```APIDOC
## estimate_effects()

### Description
Conducts DID style analyses where counter-factual observations are constructed using Sparse Synthetic Controls. Returns an object of class `SparseSCEstResults`.

### Method
Function call

### Endpoint
`SparseSC.estimate_effects(outcomes, unit_treatment_periods, covariates=X, fast=True, ...)`

### Parameters
#### Required Parameters
- **outcomes** (numeric matrix) - Required - Matrix containing data on the target variables collected prior to (after) the treatment / intervention. One row per unit and one column per observation.
- **unit_treatment_periods** - Required - Indicates the periods when units are treated.

#### Optional Parameters
- **covariates** (matrix) - Optional - Matrix of additional features. One row per unit and one column per observation.
- **treated_units** (vector of booleans or integers) - Optional - Indicates the rows which belong to treated units.
- **level** - Optional - Confidence level for statistical inference.
- **max_n_pl** - Optional - Maximum number of simulations used to produce the placebo distribution.
- **fast** (boolean) - Optional - If True, uses a faster fitting method.
- **...** - Optional - Additional keyword arguments passed to `fit()`.

### Request Example
```python
import SparseSC
fitted_estimates = SparseSC.estimate_effects(outcomes, unit_treatment_periods, covariates=X, fast=True, ...)
```

### Response
#### Success Response
Returns an object of class `SparseSCEstResults` with attributes:
- **pl_res_post** - Placebo results post-treatment, containing `avg_joint_effect` with `p_value` and `CI`.
- **fit** - The fitted Synthetic Controls model.

#### Response Example
```python
print(fitted_estimates)
fitted_estimates.pl_res_post.avg_joint_effect.p_value
fitted_estimates.pl_res_post.avg_joint_effect.CI
fitted_model = fitted_estimates.fit
```
```

--------------------------------

### fit_synthetic_control (Slow, Joint)

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Fits a synthetic controls model using gradient descent in the covariate space. Returns a SparseSCFit object containing details of the fitted model.

```APIDOC
## fit_synthetic_control (Slow, Joint)

### Description
Fits a synthetic controls model using gradient descent in the covariate space. Returns a SparseSCFit object containing details of the fitted model.

### Method
Function call

### Parameters
- **method** (str or callable) - Optional - The method or function responsible for performing gradient descent in the covariate space. If a string, it is passed as the `method` argument to `scipy.optimize.minimize()`. Otherwise, `method` must be a function with a signature compatible with `scipy.optimize.minimize()` (`method(fun,x0,grad,**kwargs)`) which returns an object having `x` and `fun` attributes. Default = `SparseSC.optimizers.cd_line_search.cdl_search()`.
- **learning_rate** (float, Default = 0.2) - Optional - The initial learning rate which determines the initial step size, which is set to `learning_rate * null_model_error / gradient`. Must be between 0 and 1.
- **learning_rate_adjustment** (float, Default = 0.9) - Optional - Adjustment factor applied to the learning rate applied between iterations when the optimal step size returned by `scipy.optimize.line_search()` is greater less than 1, else the step size is adjusted by `1/learning_rate_adjustment`. Must be between 0 and 1.
- **tol** (float, Default = 0.0001) - Optional - Tolerance used for the stopping rule based on the proportion of the in-sample residual error reduced in the last step of the gradient descent.

### Returns
- **SparseSCFit** - A `SparseSCFit` object containing details of the fitted model.

### Raises
- **ValueError** - when `treated_units` is not None and not an `iterable`, or when model_type is not one of the allowed values.
```

--------------------------------

### SparseSC.get_V

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns V split across potential pre-treatment outcomes and baseline features.

```APIDOC
## SparseSC.get_V

### Description
Returns V split across potential pre-treatment outcomes and baseline features.

### Parameters
- **treatment_period** (optional) - The treatment period to get V for.
```

--------------------------------

### SparseSC.fit_fast.fit_fast

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Fits a synthetic controls model using a fast, separate approach. This function is the primary entry point for model fitting.

```APIDOC
## SparseSC.fit_fast.fit_fast

### Description
Fits a synthetic controls model using a fast, separate approach. This function is the primary entry point for model fitting.

### Method
Function call

### Endpoint
`SparseSC.fit_fast.fit_fast(features, targets, model_type='restrospective', treated_units=None, w_pens=None, custom_donor_pool=None, match_space_maker=None, w_pen_inner=True, avoid_NxN_mats=False, verbose=0, targets_aux=None, **kwargs)`

### Parameters
- **features** (matrix of floats) - Required - Matrix of features
- **targets** (matrix of floats) - Required - Matrix of targets
- **model_type** (str, default = `"retrospective"`) - Optional - Type of model being fit. One of `"retrospective"`, `"prospective"`, `"prospective-restricted"` or `"full"`
- **treated_units** (int []) - Optional - An iterable indicating the rows of X and Y which contain data from treated units.
- **w_pens** (float []) - Optional - Penalization values to try when searching for unit weights.
- **custom_donor_pool** (boolean, default = `None`) - Optional - By default all control units are allowed to be donors for all units. There are cases where this is not desired and so the user can pass in a matrix specifying a unit-specific donor pool (NxC matrix of booleans). Common reasons for restricting the allowability: (a) When we would like to reduce interpolation bias by restricting the donor pool to those units similar along certain features. (b) If units are not completely independent (for example there may be contamination between neighboring units). This is a violation of the Single Unit Treatment Value Assumption (SUTVA). Note: These are not used in the fitting stage (of V and penalties) just in final unit weight determination.
- **match_space_maker** - Optional - Function with signature MatchSpace_transformer, V_vector, best_v_pen, V desc = match_space_maker(X, Y, fit_model_wrapper) where we can call fit_model_wrapper(MatchSpace_transformer, V_vector). Default is MTLassoCV_MatchSpace_factory().
- **avoid_NxN_mats** (bool, default=False) - Optional - There are several points where typically a matrices on the order of NxN would be made (either N or N_c). With a large number of units these can be quite big. These can be avoided. One consequence is that the full unit-level weights will not be kept and just the built Synthetic Control outcome will be return.
- **verbose** (int, default=0) - Optional - Verbosity level. 0 means no printouts. 1 will note times of completing each of the 3 main stages and some loop progress bars. 2 will print memory snapshots (Optionally out to a file if the env var SparseSC_log_file is set).
- **kwargs** - Optional - Additional parameters so that one can easily switch between fit() and fit_fast()

### Returns
A `SparseSCFit` object containing details of the fitted model.

### Return Type
`SparseSCFit`

### Raises
- **ValueError** - when `treated_units` is not None and not an `iterable`, or when model_type is not one of the allowed values
```

--------------------------------

### SparseSCFit.sc_weights

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Getter for the sc_weights. By default, the trivial donors are included.

```APIDOC
## SparseSCFit.sc_weights

### Description
Getter for the sc_weights. By default, the trivial donors are included.

### Method
Property getter
```

--------------------------------

### SparseSC.get_sc

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns an NxT matrix of synthetic controls. For units not eligible (those previously treated or between treatment_period and treatment_period+T1) the result is left empty.

```APIDOC
## SparseSC.get_sc

### Description
Returns an NxT matrix of synthetic controls. For units not eligible (those previously treated or between treatment_period and treatment_period+T1) the result is left empty.

### Parameters
- **treatment_period** (optional) - The treatment period to get synthetic controls for.
```

--------------------------------

### SparseSC.utils.match_space.MTLSTMMixed_MatchSpace_factory

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Returns a MatchSpace function that will fit an LSTM of [X_fixed, X_time_varying, Y_pre] ~ Y with the hidden-layer size optimized to reduce errors on goal units.

```APIDOC
## SparseSC.utils.match_space.MTLSTMMixed_MatchSpace_factory

### Description
Returns a MatchSpace function that will fit an LSTM of [X_fixed, X_time_varying, Y_pre] ~ Y with the hidden-layer size optimized to reduce errors on goal units.

### Method
Function call

### Endpoint
`SparseSC.utils.match_space.MTLSTMMixed_MatchSpace_factory(T0=None, K_fixed=0, M_sizes=None, dropout_rate=0.2, epochs=2, verbose=0, hidden_length=100)`

### Parameters
- **T0** - Optional - length of Y_pre
- **K_fixed** - Optional - Number of fixed unit-covariates (rest will assume to be time-varying)
- **M_sizes** - Optional - list of sizes of hidden layer (match-space) sizes to try. Default is range(1, 2*int(np.log(Y.shape[0])))
- **dropout_rate** - Optional - 
- **epochs** - Optional - 
- **verbose** - Optional - 
- **hidden_length** - Optional - 

### Returns
MatchSpace fn, V vector, best_M_size, V
```

--------------------------------

### Default custom donor pool

Source: https://sparsesc.readthedocs.io/en/latest/overview.html

Default donor pool for all entry-points. It is an (N,N0) boolean matrix of allowable donors for each unit, excluding self-matching and restricting to control units.

```python
base_custom_donor_pool = (1-np.eye(N))[:,control_units].astype('bool')

```

--------------------------------

### SparseSCFit.get_weights

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Getter for the sc_weights. By default, the trivial donors are included.

```APIDOC
## SparseSCFit.get_weights

### Description
Getter for the sc_weights. By default, the trivial donors are included.

### Method
Function call

### Parameters
- **include_trivial_donors** (boolean) - Optional - Should donors for whom selected predictors and all targets equal to zero be included in the weights for non-trivial units. These units will typically have a weight of 1 / total number of units as they do not contribute to the gradient. Default = True.
```

--------------------------------

### SparseSC.estimate_effects.SparseSCEstResults

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Class that holds estimation info from the estimate_effects function. It has a CI attribute for p-value.

```APIDOC
## SparseSC.estimate_effects.SparseSCEstResults

### Description
Holds estimation info.

### Class
SparseSC.estimate_effects.SparseSCEstResults(outcomes, fits, unit_treatment_periods, unit_treatment_periods_idx, unit_treatment_periods_idx_fit, T0, T1, pl_res_pre, pl_res_post, pl_res_post_scaled, max_n_pl, covariates=None, ind_CI=None, model_type='retrospective', T2=None, pl_res_post_fit=None)

### Attributes
- **CI** - p-value for the current model if relevant, else None.
```

--------------------------------

### SparseSC.fit.fit

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Fits a synthetic controls model using a slow, joint optimization approach. This function takes features and targets matrices, along with various optional parameters to control the fitting process, including penalties, grid search, cross-validation, and donor pool restrictions.

```APIDOC
## SparseSC.fit.fit

### Description
Fits a synthetic controls model using a slow, joint optimization approach. This function takes features and targets matrices, along with various optional parameters to control the fitting process, including penalties, grid search, cross-validation, and donor pool restrictions.

### Method
Function call

### Endpoint
SparseSC.fit.fit(features, targets, treated_units=None, w_pen=None, v_pen=None, grid=None, grid_min=1e-06, grid_max=1, grid_length=20, stopping_rule=2, gradient_folds=10, w_pen_inner=False, match_space_maker=None, **kwargs)

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
- **features** (matrix of floats) - Required - Matrix of features
- **targets** (matrix of floats) - Required - Matrix of targets
- **model_type** (str, default = "retrospective") - Optional - Type of model being fit. One of "retrospective", "prospective", "prospective-restricted" or "full"
- **treated_units** (int [], Optional) - Optional - An iterable indicating the rows of X and Y which contain data from treated units.
- **w_pen** (float | float [], optional) - Optional - Penalty applied to the difference between the current weights and the null weights (1/n). default provided by :func:`w_pen_guestimate`.
- **v_pen** (float | float [], optional) - Optional - penalty (penalties) applied to the magnitude of the covariate weights. Defaults to `[ Lambda_c_max * g for g in grid]`, where Lambda_c_max is determined via `get_max_v_pen()`.
- **grid** (float | float [], optional) - Optional - only used when v_pen is not provided. Defaults to `np.exp(np.linspace(np.log(grid_min),np.log(grid_max),grid_length))`
- **grid_min** (float, default = 1e-6) - Optional - Lower bound for `grid` when `v_pen` and `grid` are not provided. Must be in the range `(0,1)`
- **grid_max** (float, default = 1) - Optional - Upper bound for `grid` when `v_pen` and `grid` are not provided. Must be in the range `(0,1]`
- **grid_length** (int, default = 20) - Optional - number of points in the `grid` parameter when `v_pen` and `grid` are not provided
- **stopping_rule** (int, float, or function) - Optional - A stopping rule less than one is interpreted as the percent improvement in the out-of-sample squared prediction error required between the current and previous iteration in order to continue with the coordinate descent. A stopping rule of one or greater is interpreted as the number of iterations of the coordinate descent (rounded down to the nearest Int). Alternatively, `stopping_rule` may be a function which will be passed the current model fit, the previous model fit, and the iteration number (depending on it's signature), and should return a truthy value if the coordinate descent should stop and a falsey value if the coordinate descent should stop.
- **choice** (str or function. default = "min") - Optional - Method for choosing from among the v_pen. Only used when v_pen is an iterable. Defaults to "min" which selects the v_pen parameter associated with the lowest cross validation error.
- **cv_folds** (int or (int [], int [])[], default = 10) - Optional - An integer number of Cross Validation folds passed to `sklearn.model_selection.KFold()`, or an explicit list of train validation folds. TODO: These folds are calculated with `KFold(...,shuffle=False)`, but instead, it should be assigned a random state.
- **gradient_folds** (int or (int [], int [])[]) - Optional - (default = 10) An integer number of Gradient folds passed to `sklearn.model_selection.KFold()`, or an explicit list of train validation folds, to be used model_type is one either `"foo"` `"bar"`.
- **cv_seed** (int, default = 10101) - Optional - passed to `sklearn.model_selection.KFold()` to allow for consistent cross validation folds across calls
- **gradient_seed** (int, default = 10101) - Optional - passed to `sklearn.model_selection.KFold()` to allow for consistent gradient folds across calls when model_type is one either `"foo"` `"bar"` with and gradient_folds is an integer.
- **progress** (boolean, default = True) - Optional - Controls the level of verbosity. If True, the messages indication the progress are printed to the console (stdout).
- **kwargs** - Optional - Additional arguments passed to the optimizer (i.e. `method` or scipy.optimize.minimize). See below.
- **custom_donor_pool** (boolean, default = None) - Optional - By default all control units are allowed to be donors for all units. There are cases where this is not desired and so the user can pass in a matrix specifying a unit-specific donor pool (NxC matrix of booleans). Common reasons for restricting the allowability: (a) When we would like to reduce interpolation bias by restricting the donor pool to those units similar along certain features. (b) If units are not completely independent (for example there may be contamination between neighboring units). This is a violation of the Single Unit Treatment Value Assumption (SUTVA). Note: These are not used in the fitting stage (of V and penalties) just in final unit weight determination.

### Request Example
No request body example provided.

### Response
#### Success Response (200)
No response schema provided.

#### Response Example
No response example provided.

```

--------------------------------

### SparseSC.p_value

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

p-value for the current model if relevant, else None.

```APIDOC
## SparseSC.p_value

### Description
p-value for the current model if relevant, else None.
```

--------------------------------

### SparseSC.estimate_effects.estimate_effects

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Determines statistical significance for average and individual treatment effects. It accepts outcomes, treatment periods, and optional parameters for matching and inference.

```APIDOC
## SparseSC.estimate_effects.estimate_effects

### Description
Determines statistical significance for average and individual effects.

### Method
Function call

### Endpoint
SparseSC.estimate_effects.estimate_effects(outcomes, unit_treatment_periods, T0=None, T1=None, covariates=None, max_n_pl=10000, ret_pl=False, ret_CI=False, level=0.95, fast=True, model_type='retrospective', T2=None, cf_folds=10, cf_seed=110011, **kwargs)

### Parameters
- **outcomes** (np.array or pd.DataFrame with shape (N, T)) - Required - Outcomes
- **unit_treatment_periods** (np.array or pd.Series with shape (N)) - Required - Vector of treatment periods for each unit (if a unit is never treated then use np.NaN if vector refers to time periods by numerical index and np.datetime64('NaT') if using DateTime to refer to time periods (and then Y must be pd.DataFrame with columns in DateTime too)) If using a prospective-based design this is the true treatment periods (and fit will be called with pseudo-treatment periods that are T1 periods earlier).
- **T0** (int, Optional (default is pre-period for first treatment)) - Optional - pre-history length to match over.
- **T1** (int, Optional (Default is post-period for last treatment)) - Optional - post-history length to fit over.
- **covariates** (np.array or pd.DataFrame with shape (N, K), Optional) - Optional - Additional pre-treatment features
- **max_n_pl** (int, Optional) - Optional - The full number of placebos is choose(N0,N1). If N1=1 then this is N0. This number grows quickly with N1 so we can set a maximum that we compute and this is drawn at random.
- **ret_pl** (bool) - Optional - Return the matrix of placebos (different from the SC of the controls when N1>1)
- **ret_CI** (bool) - Optional - Whether to return confidence intervals (requires more memory during execution)
- **level** (float (between 0 and 1)) - Optional - Level for confidence intervals
- **fast** (bool) - Optional - Whether to use the fast approximate solution (fit_fast() rather than fit())
- **model_type** (str) - Optional - Model type
- **T2** (int) - Optional - If model='prospective' then the period of which to evaluate the effect
- **cf_folds** (int) - Optional - Number of cross-fitting folds (inferred from context)
- **cf_seed** (int) - Optional - Seed for cross-fitting (inferred from context)
- **kwargs** - Optional - Additional parameters passed to fit() or fit_fast()

### Returns
An instance of SparseSCEstResults with the fitted results

### Raises
- **ValueError** - when invalid parameters are passed

### Keyword Args
Passed on to fit() or fit_fast()
```

--------------------------------

### SparseSCFit.predict

Source: https://sparsesc.readthedocs.io/en/latest/api_ref.html

Predict method for the fitted model. Returns a matrix of predicted outcomes.

```APIDOC
## SparseSCFit.predict

### Description
Predict method for the fitted model. Returns a matrix of predicted outcomes.

### Method
Function call

### Parameters
- **targets** (matrix of floats) - Optional - Matrix of targets.
- **include_trivial_donors** (boolean) - Optional - Should donors for whom selected predictors and all targets equal to zero be included in the weights for non-trivial units. These units will typically have a weight of 1 / total number of units as they do not contribute to the gradient. Default = False.

### Returns
- **matrix of floats** - Matrix of predicted outcomes.

### Raises
- **ValueError** - When `targets.shape[0]` is inconsistent with the fitted model.
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.