### Install Miceforest from GitHub

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Install the latest development version of miceforest directly from its GitHub repository. Ensure pip and git are installed via conda first if using conda.

```bash
pip install git+https://github.com/AnotherSamWilson/miceforest.git
```

--------------------------------

### Install Miceforest with Pip

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Install the miceforest package using pip. The --no-cache-dir flag is recommended for cleaner installations.

```bash
pip install miceforest --no-cache-dir
```

--------------------------------

### Example of Global LightGBM Parameters

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Shows how to set global LightGBM parameters such as 'boosting', 'num_iterations', and 'learning_rate' for the imputation kernel.

```python
kernel.mice(
    iterations=5,
    boosting='gbdt',
    num_iterations=100,
    max_depth=8,
    learning_rate=0.02,
    bagging_fraction=0.8
)
```

--------------------------------

### Tuning Parameters Timing Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Illustrates the logger's usage during parameter tuning, where timing is organized by 'Variable' and 'Iteration'.

```python
params = kernel.tune_parameters(verbose=True)

# Creates logger with:
# timed_levels = ["Variable", "Iteration"]
```

--------------------------------

### Accessing Package Version

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/module-structure.md

Demonstrates how to access the installed version of the miceforest package.

```python
import miceforest as mf
print(mf.__version__)
```

--------------------------------

### LGBParams Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Shows how to define a LightGBM parameter dictionary for use with the `mice` method. This includes common parameters like `boosting`, `num_iterations`, and `learning_rate`.

```python
lgb_params = {
    'boosting': 'gbdt',
    'num_iterations': 100,
    'max_depth': 8,
    'learning_rate': 0.05,
    'num_leaves': 64,
}
kernel.mice(iterations=5, **lgb_params)
```

--------------------------------

### Accessing All Loggers

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Provides an example of how to access all logger instances managed by the kernel and retrieve their timing summaries.

```python
loggers = kernel.loggers
for logger in loggers:
    print(f"Logger: {logger.name}")
    summary = logger.get_time_spend_summary()
    print(summary)
```

--------------------------------

### Impute New Data Timing Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Shows how the Logger is used internally when imputing new data, tracking events such as 'Getting Bachelor Features' and 'Mean Matching'.

```python
new_imputed = kernel.impute_new_data(new_df, verbose=True)

# Creates logger with:
# timed_levels = ["Dataset", "Iteration", "Variable", "Event"]
# where Event includes: "Getting Bachelor Features", "Mean Matching"
```

--------------------------------

### Set Start Time for Timer

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Records the start time for a specific timing key. This must be called before `record_time()` for the same key. An AssertionError is raised if a timer has already been started for this key.

```python
def set_start_time(self, time_key: Tuple) -> None
```

```python
logger = Logger(name="test", timed_levels=["Dataset", "Variable"])

# Timer key must have 2 elements to match timed_levels
logger.set_start_time((0, "age"))  # Dataset 0, Variable age

# Processing...

logger.record_time((0, "age"))
```

--------------------------------

### Example of Per-Variable Parameters

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Demonstrates how to apply specific LightGBM parameters like 'max_depth' and 'num_leaves' to individual variables ('age', 'salary') within the imputation process.

```python
kernel.mice(
    iterations=5,
    variable_parameters={
        'age': {
            'max_depth': 6,
            'num_leaves': 64,
            'learning_rate': 0.05,
        },
        'salary': {
            'num_iterations': 100,
            'min_data_in_leaf': 5,
        }
    }
)
```

--------------------------------

### set_start_time Method

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Records the start time for a given timing key. This method must be called before `record_time()` for the same key. It raises an AssertionError if a timer for the specified key has already been started.

```APIDOC
### set_start_time

```python
def set_start_time(self, time_key: Tuple) -> None
```

Record the start time for a timing key. Must be called before record_time().

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| time_key | Tuple | Key matching timed_levels structure |

**Raises:** AssertionError if timer already started for this key

**Example:**

```python
logger = Logger(name="test", timed_levels=["Dataset", "Variable"])

# Timer key must have 2 elements to match timed_levels
logger.set_start_time((0, "age"))  # Dataset 0, Variable age

# Processing...

logger.record_time((0, "age"))
```
```

--------------------------------

### ParameterTuningSpace Type Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Demonstrates setting hyperparameter tuning bounds using scalar, tuple (min, max), or list of discrete values for the `variable_parameters` argument in `tune_parameters()`.

```python
params = kernel.tune_parameters(
    variable_parameters={
        'age': {
            'learning_rate': 0.01,  # Fixed value
            'max_depth': (3, 8),     # Search between 3-8
            'num_leaves': [10, 25, 50],  # Try these values
        }
    }
)
```

--------------------------------

### MICE Variable Parameters Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/docs/ImputationKernel.md

Example of how to specify model parameters for specific variables within the MICE imputation process. This allows for fine-tuning imputation models on a per-variable basis.

```python
variable_parameters = {
    'column': {
        'min_sum_hessian_in_leaf: 25.0,
        'extra_trees': True,
    }
}
```

--------------------------------

### DatasetIndex Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Illustrates how to access specific imputed datasets using an integer index. The index ranges from 0 to `num_datasets`-1.

```python
kernel = ImputationKernel(df, num_datasets=5)
data0 = kernel.complete_data(dataset=0)
data4 = kernel.complete_data(dataset=4)
```

--------------------------------

### Analyze Performance with Miceforest

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Demonstrates how to use the Miceforest logger to analyze performance after an imputation. It shows how to get the timing summary and analyze it by variable, event, and identify slow steps.

```python
import miceforest as mf

kernel = mf.ImputationKernel(df, num_datasets=1)
kernel.mice(iterations=5, verbose=True)

# Get timing summary
logger = kernel.loggers[-1]
timing = logger.get_time_spend_summary()

# Total time per variable
print(timing.groupby(level='Variable').sum())

# Time distribution
print(timing.groupby(level='Event').sum())

# Identify slow steps
print(timing.sort_values(ascending=False).head(10))
```

--------------------------------

### Install Miceforest with Conda

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Install the miceforest package using conda from the conda-forge channel.

```bash
conda install -c conda-forge miceforest
```

--------------------------------

### IterationIndex Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Shows how to retrieve data from specific MICE iterations using an integer index. A positive integer specifies an iteration, while -1 refers to the latest iteration.

```python
kernel.mice(iterations=10)
# Iteration 10 is the latest

data_iter5 = kernel.complete_data(dataset=0, iteration=5)
data_latest = kernel.complete_data(dataset=0, iteration=-1)
```

--------------------------------

### MICE Timing Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Demonstrates how MICE operations internally use the Logger class to track performance across different levels like Dataset, Iteration, Variable, and Event.

```python
kernel.mice(iterations=5, verbose=True)

# Internally creates logger with:
# timed_levels = ["Dataset", "Iteration", "Variable", "Event"]
# where Event is one of: "Prepare XY", "Training", "Mean Matching"

summary = kernel.loggers[-1].get_time_spend_summary()
print(summary)
```

--------------------------------

### ExpandableParameter Type Examples

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Shows how to use scalar or dictionary forms for parameters like `mean_match_candidates` and `mean_match_strategy`. Scalar applies to all variables, while a dictionary allows per-variable specification.

```python
# Scalar form - applies to all variables
kernel = ImputationKernel(
    df,
    mean_match_candidates=10,  # All variables use 10
    mean_match_strategy='normal'  # All variables use 'normal'
)

# Dict form - per-variable specification
kernel = ImputationKernel(
    df,
    mean_match_candidates={'age': 5, 'salary': 10},  # salary gets 10, others default to 5
    mean_match_strategy={'age': 'fast', 'salary': 'normal'}
)
```

--------------------------------

### MultiIndexKey Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Demonstrates accessing, setting, and deleting imputation values using a `MultiIndexKey` tuple of `(variable, iteration, dataset)`.

```python
# Access imputation values
values = kernel['age', 3, 0]  # age variable, iteration 3, dataset 0

# Set values
kernel['age', 4, 0] = new_values

# Delete values
del kernel['age', 2, 0]
```

--------------------------------

### Predicting with a Trained Model

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Example of using a trained LightGBM Booster model to make predictions and retrieve feature importance.

```python
model = kernel.get_model(variable='age', dataset=0, iteration=-1)
predictions = model.predict(X_new)
feature_importance = model.feature_importance()
```

--------------------------------

### Customize LightGBM Objective and Boosting

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Create a new ImputationKernel and run MICE with custom objective functions for specific variables and a specified boosting method. This example uses a Poisson objective for 'sepal width (cm)' and 'gbdt' boosting.

```python
# Create kernel. 
cust_kernel = mf.ImputationKernel(
  iris_amp,
  num_datasets=1,
  random_state=1
)

cust_kernel.mice(
  iterations=1, 
  variable_parameters={'sepal width (cm)': {'objective': 'poisson'}},
  boosting = 'gbdt',
  min_sum_hessian_in_leaf=0.01
)
```

--------------------------------

### Defining Variable-Specific Tuning Parameters in Python

Source: https://github.com/anothersamwilson/miceforest/blob/master/docs/ImputationKernel.md

This example demonstrates how to define a custom tuning space for a specific variable ('column'). It shows how to set a fixed value for 'learning_rate', define a search range for 'min_sum_hessian_in_leaf', and specify a list of possible values for 'extra_trees'. Other variables will use the default search space.

```python
variable_parameters = {
    'column': {
        'learning_rate': 0.01,
        'min_sum_hessian_in_leaf': (0.1, 10),
        'extra_trees': [True, False]
    }
}
```

--------------------------------

### RandomState Type Union Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Demonstrates equivalent ways to specify a random state for reproducible imputation. Use `ensure_rng()` for conversion.

```python
from miceforest import ImputationKernel
import numpy as np

# All equivalent for reproducibility
kernel1 = ImputationKernel(df, random_state=42)
kernel2 = ImputationKernel(df, random_state=np.random.RandomState(42))
```

--------------------------------

### VariableSchema Type Examples

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/types.md

Illustrates the different forms of `variable_schema` for specifying variables to model and their predictors. Auto-mode, list of targets, or explicit mapping can be used.

```python
# All forms equivalent for same variable
df = pd.DataFrame({
    'age': [25, None, 35, 40],
    'salary': [50000, 60000, None, 80000],
    'region': ['A', 'B', 'C', 'D']
})

# Form 1: Auto-mode
kernel1 = ImputationKernel(df)

# Form 2: List form - model age and salary
kernel2 = ImputationKernel(df, variable_schema=['age', 'salary'])

# Form 3: Explicit form
kernel3 = ImputationKernel(df, variable_schema={
    'age': ['salary', 'region'],
    'salary': ['age', 'region']
})
```

--------------------------------

### Ampute Data Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/utility-functions.md

Demonstrates the usage of the `ampute_data` function, which is part of the main `miceforest` package entry point. This function is used to introduce missing values into a dataset.

```python
import miceforest as mf

# Exported function
df_amputed = mf.ampute_data(df, perc=0.2)
```

--------------------------------

### Customize LightGBM Parameters Per Variable

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Run the MICE algorithm with variable-specific parameters, overriding global settings for certain columns. This example shows setting n_estimators for the 'species' column and retrieving specific models.

```python
# Run the MICE algorithm for 2 more iterations on the kernel 
kernel.mice(
  iterations=1,
  variable_parameters={'species': {'n_estimators': 25}},
  n_estimators=50
)

# Let's get the actual models for these variables:
species_model = kernel.get_model(dataset=0,variable="species")
sepalwidth_model = kernel.get_model(dataset=0,variable="sepal width (cm)")

print(
  f"""Species used {str(species_model.params["num_iterations"])} iterations
Sepal Width used {str(sepalwidth_model.params["num_iterations"])} iterations
"""
)
```

--------------------------------

### Generate Bimodal and Skewed Data for PMM Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/README.md

This code constructs sample data with bimodal and skewed distributions, along with a uniform variable, to demonstrate the effects of predictive mean matching in miceforest.

```python
randst = np.random.RandomState(1991)
# random uniform variable
nrws = 1000
uniform_vec = randst.uniform(size=nrws)

def make_bimodal(mean1,mean2,size):
    bimodal_1 = randst.normal(size=nrws, loc=mean1)
    bimodal_2 = randst.normal(size=nrws, loc=mean2)
    bimdvec = []
    for i in range(size):
        bimdvec.append(randst.choice([bimodal_1[i], bimodal_2[i]]))
    return np.array(bimdvec)

# Make 2 Bimodal Variables
close_bimodal_vec = make_bimodal(2,-2,nrws)
far_bimodal_vec = make_bimodal(3,-3,nrws)


# Highly skewed variable correlated with Uniform_Variable
skewed_vec = np.exp(uniform_vec*randst.uniform(size=nrws)*3) + randst.uniform(size=nrws)*3
```

--------------------------------

### Customize LightGBM Parameters Per Variable

Source: https://github.com/anothersamwilson/miceforest/blob/master/README.md

Run the MICE algorithm with variable-specific parameters, overriding global settings for particular columns. This example shows setting a lower n_estimators for the 'species' column.

```python
# Run the MICE algorithm for 2 more iterations on the kernel 
kernel.mice(
  iterations=1,
  variable_parameters={'species': {'n_estimators': 25}},
  n_estimators=50
)

# Let's get the actual models for these variables:
species_model = kernel.get_model(dataset=0,variable="species")
sepalwidth_model = kernel.get_model(dataset=0,variable="sepal width (cm)")

print(
f"""Species used {str(species_model.params["num_iterations"])} iterations
Sepal Width used {str(sepalwidth_model.params["num_iterations"])} iterations
"""
)
```

--------------------------------

### Stratified Categorical Folds Example

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/utility-functions.md

Demonstrates how to create stratified folds for categorical data using `stratified_categorical_folds`. This is useful for ensuring that the distribution of categories is maintained across training and testing sets.

```python
import pandas as pd

cat = pd.Categorical(['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b'])
y = pd.Series(cat.codes)

folds = list(stratified_categorical_folds(y, nfold=2))
for train_idx, test_idx in folds:
    print(f"Train: {train_idx}, Test: {test_idx}")
```

--------------------------------

### Initialize ImputationKernel, Run MICE, and Complete Data

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/INDEX.md

Demonstrates the basic workflow of creating an ImputationKernel, running the MICE algorithm, and retrieving the completed data for a specific dataset and iteration.

```python
import miceforest as mf
import pandas as pd

# Create kernel
kernel = mf.ImputationKernel(
    df,
    num_datasets=3,
    variable_schema=['A', 'B'],  # Model A and B
    imputation_order='ascending',
    mean_match_candidates=5,
    random_state=42
)

# Run MICE
kernel.mice(
    iterations=5,
    verbose=True,
    variable_parameters={'A': {'max_depth': 6}}
)

# Get results
completed = kernel.complete_data(dataset=0, iteration=-1)
data_all = [kernel.complete_data(dataset=i) for i in range(3)]
```

--------------------------------

### Initialize and Run Imputation Kernel

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/README.md

Demonstrates the basic workflow of creating an ImputationKernel, running the MICE algorithm, and retrieving completed data.

```python
kernel = mf.ImputationKernel(
    df,
    num_datasets=3,
    variable_schema=['A', 'B'],
    random_state=42
)

# Run MICE
kernel.mice(iterations=5, verbose=True)

# Get completed data
completed = kernel.complete_data(dataset=0)
```

--------------------------------

### Install Plotnine for Visualization

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/errors.md

If you encounter an 'ImportError: plotnine must be installed to plot importance', install the 'plotnine' library using pip or conda to enable plotting functionalities.

```bash
pip install plotnine
# or
conda install -c conda-forge plotnine
```

--------------------------------

### Initialize Variable Parameters

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

An empty dictionary to hold per-variable LightGBM parameters. These parameters will override any global settings for the specified variables.

```python
variable_parameters: Dict[str, Any] = {}
```

--------------------------------

### Initialize ImputationKernel with Normal Strategy

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/errors.md

Alternatively, use the 'normal' strategy with zero candidates for mean matching when initializing the ImputationKernel.

```python
kernel = ImputationKernel(df, mean_match_strategy='normal', mean_match_candidates=0)
```

--------------------------------

### Create and Run Imputation Kernel

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Initializes an ImputationKernel with custom mean matching strategies and then performs imputation.

```python
cust_kernel = mf.ImputationKernel(
  iris_amp,
  num_datasets=1,
  random_state=1,
  mean_match_strategy={
      'sepal length (cm)': 'normal',
      'sepal width (cm)': 'shap',
      'species': 'fast',
  },
  mean_match_candidates={
      'petal length (cm)': 0,
      }
)

cust_kernel.mice(
  iterations=1,
)
```

--------------------------------

### plot_feature_importance

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Plots the feature importance as a heatmap using plotnine. Requires plotnine to be installed.

```APIDOC
## plot_feature_importance

### Description
Plots the feature importance as a heatmap using plotnine. Requires plotnine to be installed.

### Method
`plot_feature_importance`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **dataset** (int) - Required - Dataset to plot.
- **importance_type** (str) - Optional - Default: "split" - "split" or "gain".
- **normalize** (bool) - Optional - Default: True - Normalize importances.
- **iteration** (int) - Optional - Default: -1 - Iteration to plot.

### Returns
plotnine.ggplot object

### Raises
ImportError if plotnine not installed.

### Request Example
```python
fig = kernel.plot_feature_importance(dataset=0)
fig.show()
```
```

--------------------------------

### Get Non-Missing Values

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputed-data.md

Extracts all non-missing values for a given variable. This is an internal method.

```python
def _get_nonmissing_values(self, variable: str) -> pandas.Series:
    # ... function body ...
```

--------------------------------

### Get Non-Missing Index

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputed-data.md

Retrieves the row indices for a specified variable where the values are not missing. This is an internal method.

```python
def _get_nonmissing_index(self, variable: str) -> np.ndarray:
    # ... function body ...
```

--------------------------------

### ImputationKernel Initialization with Default Parameters

Source: https://github.com/anothersamwilson/miceforest/blob/master/docs/ImputationKernel.md

Initializes an ImputationKernel with default settings. All columns with missing values will have models trained using all other columns as features. Imputation order is ascending, and no data subsetting is applied.

```python
from miceforest import ImputationKernel

# Assuming 'data' is a pandas DataFrame
kernel = ImputationKernel(data=data)
```

--------------------------------

### Plot Feature Importance Heatmap

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Generate a heatmap visualization of feature importance using plotnine. Requires plotnine to be installed.

```python
def plot_feature_importance(
    self,
    dataset: int,
    importance_type: str = "split",
    normalize: bool = True,
    iteration: int = -1,
) -> plotnine.ggplot

```

```python
fig = kernel.plot_feature_importance(dataset=0)
fig.show()
```

--------------------------------

### Import Miceforest and Pandas

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/README.md

Imports the necessary libraries for using miceforest and pandas. This is a common starting point for many operations.

```python
import miceforest as mf
import pandas as pd

```

--------------------------------

### Initialize ImputationKernel with SHAP Strategy

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/errors.md

Use the SHAP strategy for imputation when initializing the ImputationKernel. This approach considers the number of candidates for mean matching.

```python
kernel = ImputationKernel(df, mean_match_strategy='shap', mean_match_candidates=5)
```

--------------------------------

### Create and Run Imputation Kernel

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Initializes an ImputationKernel with the amputed data and runs the MICE algorithm for a specified number of iterations. This is the core step for performing imputation.

```python
# Create kernel.
kds = mf.ImputationKernel(
  iris_amp,
  random_state=1991
)

# Run the MICE algorithm for 2 iterations
kds.mice(2)
```

--------------------------------

### Importing Utility Functions

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/utility-functions.md

Shows how to import various utility functions from the `miceforest.utils` module. These functions are available for direct use in your projects.

```python
from miceforest.utils import (
    ampute_data,
    ensure_rng,
    get_best_int_downcast,
    stratified_continuous_folds,
    stratified_categorical_folds,
    hash_numpy_int_array,
    logodds,
    logistic_function,
    stratified_subset,
)
```

--------------------------------

### Get Iteration Count

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputed-data.md

Retrieves the current iteration count for specified datasets and variables. Raises a ValueError if counts are inconsistent.

```python
iters = imputed_data.iteration_count(dataset=0, variable='A')
```

--------------------------------

### Initialize Kernel Specifying Target Variables

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Initializes the kernel by specifying a list of target variables to model. All other variables will be used as predictors.

```python
kernel = ImputationKernel(df, variable_schema=['age', 'income'])
```

--------------------------------

### Initialize ImputationKernel

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Instantiate the ImputationKernel with a pandas DataFrame and various configuration options. This is the primary entry point for MICE imputation.

```python
from miceforest import ImputationKernel
import pandas as pd

# Assuming 'data' is a pandas DataFrame with missing values
data = pd.DataFrame({
    'col1': [1, 2, None, 4, 5],
    'col2': [None, 'b', 'c', 'd', 'e'],
    'col3': [1.1, 2.2, 3.3, None, 5.5]
})

kernel = ImputationKernel(
    data=data,
    num_datasets=5,
    imputation_order="descending",
    mean_match_candidates=10,
    mean_match_strategy="shap",
    save_all_iterations_data=True,
    random_state=42
)
```

--------------------------------

### Initialize Kernel with Mixed Mean Match Strategies

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Applies different mean matching strategies per variable: 'shap' for 'age', 'normal' for 'salary', and 'fast' for the categorical variable 'region'.

```python
kernel = ImputationKernel(df, mean_match_strategy={
    'age': 'shap',
    'salary': 'normal',
    'region': 'fast'
})
```

--------------------------------

### get_feature_importance

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Gets the feature importance matrix for all models across specified datasets and iterations. Supports different importance types and normalization.

```APIDOC
## get_feature_importance

### Description
Gets the feature importance matrix for all models across specified datasets and iterations. Supports different importance types and normalization.

### Method
`get_feature_importance`

### Parameters
#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Parameters
- **dataset** (int) - Optional - Default: 0 - Which dataset's models to analyze.
- **iteration** (int) - Optional - Default: -1 - Iteration index. Use -1 for latest.
- **importance_type** (str) - Optional - Default: "split" - "split" (frequency of use) or "gain" (contribution to loss reduction).
- **normalize** (bool) - Optional - Default: True - If True, normalize importances per variable to sum to 1.

### Returns
pandas.DataFrame with imputed variables as rows, predictor variables as columns

### Request Example
```python
importance = kernel.get_feature_importance(dataset=0, normalize=True)
print(importance)
#       A    B    C
# A 0.00 0.50 0.50
# B 0.40 0.00 0.60
```
```

--------------------------------

### Get Time Spend Summary

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Retrieves the time spend summary from the last logger. This is useful for analyzing performance after an imputation operation.

```python
last_logger = kernel.loggers[-1]
summary = last_logger.get_time_spend_summary()
```

--------------------------------

### ImputationKernel with Data Subsetting and Mean Match Strategy

Source: https://github.com/anothersamwilson/miceforest/blob/master/docs/ImputationKernel.md

Initializes an ImputationKernel using a subset of data for training models and a specific mean matching strategy for categorical variables. This can speed up the process and manage memory for large datasets.

```python
from miceforest import ImputationKernel

mean_match_strategy = {
    'column_1': 'fast',
    'column_2': 'shap',
}
data_subset = 1000

kernel = ImputationKernel(data=data, data_subset=data_subset, mean_match_strategy=mean_match_strategy)
```

--------------------------------

### Get Iteration Count

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Retrieve the current iteration count for imputation across specified datasets and variables. Useful for monitoring the convergence of the imputation process.

```python
current_iters = kernel.iteration_count()
print(f"Completed {current_iters} iterations")
```

--------------------------------

### Get Bachelor Features

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputed-data.md

Fetches predictor features for rows that have missing values in a specified target variable. This method is used internally during the imputation process.

```python
def _get_bachelor_features(self, variable: str) -> pandas.DataFrame:
    # ... function body ...
```

--------------------------------

### Tune Kernel Parameters

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/README.md

Demonstrates how to tune the parameters of the ImputationKernel to optimize the imputation process.

```python
# Parameter tuning
params = kernel.tune_parameters(optimization_steps=10)
```

--------------------------------

### Load Data and Introduce Missing Values

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Loads the Iris dataset and introduces missing values using mf.ampute_data. This is a common setup for demonstrating imputation.

```python
import miceforest as mf
from sklearn.datasets import load_iris
import pandas as pd
import numpy as np

# Load data and introduce missing values
iris = pd.concat(load_iris(as_frame=True,return_X_y=True),axis=1)
iris.rename({"target": "species"}, inplace=True, axis=1)
iris['species'] = iris['species'].astype('category')
iris_amp = mf.ampute_data(iris,perc=0.25,random_state=1991)
```

--------------------------------

### ImputationKernel with Mean Match Candidates Configuration

Source: https://github.com/anothersamwilson/miceforest/blob/master/docs/ImputationKernel.md

Initializes an ImputationKernel with a specific number of mean match candidates for general imputation and a different number for a particular variable. Setting mean_match_candidates to 0 skips mean matching for that variable.

```python
from miceforest import ImputationKernel

mean_match_candidates = {
    'column_1': 0, # Skips mean matching for column_1
    'column_2': 10
}

kernel = ImputationKernel(data=data, mean_match_candidates=mean_match_candidates)
```

--------------------------------

### Plot Imputed Value Distributions

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Visualize the distributions of imputed values compared to original non-missing values for selected variables. Requires the 'plotnine' library to be installed.

```python
fig = kernel.plot_imputed_distributions()
fig.show()
```

--------------------------------

### Scikit-learn Pipeline Integration

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Integrate the ImputationKernel into scikit-learn pipelines for seamless preprocessing and modeling workflows. This example shows how to add the kernel as an imputation step before a model.

```python
from sklearn.pipeline import Pipeline

pipeline = Pipeline([
    ('impute', kernel),
    ('model', LogisticRegression())
])
```

--------------------------------

### Get Feature Importance Matrix

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Retrieve the feature importance matrix for all models within a specified dataset and iteration. Normalization can be applied to sum importances to 1.

```python
def get_feature_importance(
    self,
    dataset: int = 0,
    iteration: int = -1,
    importance_type: str = "split",
    normalize: bool = True,
) -> pandas.DataFrame

```

```python
importance = kernel.get_feature_importance(dataset=0, normalize=True)
print(importance)
#       A    B    C
# A 0.00 0.50 0.50
# B 0.40 0.00 0.60
```

--------------------------------

### Initialize Kernel with Multiple Imputations

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Sets the number of imputed datasets to generate to 5 for standard multiple imputation analysis.

```python
kernel = ImputationKernel(df, num_datasets=5)
```

--------------------------------

### Access Imputation Values

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputed-data.md

Demonstrates how to get, set, and delete imputation values using item access with a tuple key format (variable, iteration, dataset).

```python
# Get imputation values
values = imputed_data[variable, iteration, dataset]

# Set imputation values  
imputed_data[variable, iteration, dataset] = new_values

# Delete imputation values
del imputed_data[variable, iteration, dataset]

imputed_data = ImputedData(df, datasets=[0, 1])

# Get imputed values for variable 'age' at iteration 3, dataset 0
age_imps = imputed_data['age', 3, 0]

# Set new imputation values
imputed_data['age', 4, 0] = new_age_values

# Delete values to save memory
del imputed_data['age', 1, 0]
```

--------------------------------

### Initialize Kernel with Explicit Target-Predictor Mapping

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Initializes the kernel using an explicit dictionary mapping target variables to their specific predictor variables.

```python
kernel = ImputationKernel(df, variable_schema={
    'age': ['income', 'years_employed'],
    'income': ['age', 'education', 'years_employed'],
    'years_employed': ['age', 'income', 'education']
})
```

--------------------------------

### Get Time Spend Summary

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Retrieves all recorded times as a hierarchical pandas Series, indexed by the timed levels. This is useful for analyzing the performance of different operations.

```python
def get_time_spend_summary(self) -> pandas.Series
```

```python
logger = Logger(name="test", timed_levels=["Dataset", "Iteration", "Variable"])

# ... record various times ...

summary = logger.get_time_spend_summary()
print(summary)
# Output:
# Dataset  Iteration  Variable
# 0        1          age           0.234
#          1          salary        0.156
#          2          age           0.189
#          2          salary        0.145
```

--------------------------------

### Initialize Kernel with Per-Variable Mean Match Candidates

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Specifies different numbers of nearest-neighbor candidates for mean matching on a per-variable basis. 'region' is set to 0, disabling mean matching for it.

```python
kernel = ImputationKernel(df, mean_match_candidates={
    'age': 10,
    'salary': 5,
    'region': 0
})
```

--------------------------------

### Type Annotations in Miceforest

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/module-structure.md

Demonstrates common type hints used in Miceforest for clarity and static analysis, including Optional, Union, Dict, List, Tuple, and Literal.

```python
from typing import Any, Dict, Generator, List, Literal, Optional, Tuple, Union
import numpy as np
from pandas import DataFrame, Series
```

```python
# Common Patterns:
# Optional[X] = X or None
# Union[X, Y] = X or Y
# Dict[K, V] = dictionary with key type K, value type V
# List[X] = list of X
# Tuple[X, Y] = ordered pair of X and Y
# Literal["a", "b"] = one of the string values
```

--------------------------------

### In-Place Imputation Kernel Initialization

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Initialize the `ImputationKernel` with `copy_data=False` to perform imputation directly on the provided dataset, avoiding memory overhead from creating a copy. This is beneficial for large datasets.

```python
kernel_inplace = mf.ImputationKernel(
  iris_amp,
  num_datasets=1,
  copy_data=False,
  random_state=1,
)
kernel_inplace.mice(2)
```

--------------------------------

### Retrieve Trained LightGBM Model

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Get a specific trained LightGBM model for a given variable and dataset. Use -1 for iteration to retrieve the latest trained model.

```python
def get_model(
    self,
    variable: str,
    dataset: int,
    iteration: int = -1,
) -> lightgbm.Booster

```

```python
model = kernel.get_model(variable='A', dataset=0, iteration=-1)
print(model.feature_importance())
```

--------------------------------

### Get Hashed Seeds

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputed-data.md

Retrieves hashed seeds for the missing indices of a specific variable. This is used for deterministic imputation when a random_seed_array is provided. Returns None if no random_seed_array was configured.

```python
def _get_hashed_seeds(self, variable: str) -> Optional[np.ndarray]:
    # ... function body ...
```

--------------------------------

### Initialize Kernel with Scalar Mean Match Candidates

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Sets the number of nearest-neighbor candidates to sample from during mean matching to 5 for all variables. This is the default value.

```python
kernel = ImputationKernel(df, mean_match_candidates=5)
```

--------------------------------

### Tune Parameters and Build Model

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Performs parameter tuning on a dataset and then builds a mice model using the optimal parameters found. The optimal parameters are returned as a DataFrame.

```python
optimal_params = kernel.tune_parameters(
    dataset=0, 
    use_gbdt=True,
    num_iterations=500,
    random_state=1,
)
kernel.mice(1, variable_parameters=optimal_params)
pd.DataFrame(optimal_params)
```

--------------------------------

### Fix Incompatible Mean Matching Configuration

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/errors.md

When using mean_match_strategy='shap', mean_match_candidates must be greater than 0. This error indicates an unintentional setup where SHAP matching requires candidates.

```python
AssertionError: Failing because {col} mean_match_candidates == 0 and mean_match_strategy == shap. This implies an unintentional setup.
```

--------------------------------

### ImputationKernel Constructor

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Initializes the ImputationKernel with a dataset and configuration for MICE. It sets up parameters for imputation, model training, and data handling.

```APIDOC
## ImputationKernel Constructor

### Description
Initializes the ImputationKernel with a dataset and configuration for MICE. It sets up parameters for imputation, model training, and data handling.

### Parameters

#### Path Parameters
None

#### Query Parameters
None

#### Request Body
None

### Method Signature
```python
ImputationKernel(
    data: pandas.DataFrame,
    num_datasets: int = 1,
    variable_schema: Optional[Union[List[str], Dict[str, List[str]]]] = None,
    imputation_order: Literal["ascending", "descending", "roman", "latin"] = "ascending",
    mean_match_candidates: Union[int, Dict[str, int]] = 5,
    mean_match_strategy: Optional[Union[str, Dict[str, str]]] = "normal",
    data_subset: Union[int, Dict[str, int]] = 0,
    initialize_empty: bool = False,
    save_all_iterations_data: bool = True,
    copy_data: bool = True,
    random_state: Optional[Union[int, np.random.RandomState]] = None,
)
```

### Parameters Details

*   **data** (pandas.DataFrame) - Required - The dataset to impute. Must have a RangeIndex.
*   **num_datasets** (int) - Optional - Default: 1 - Number of imputed datasets to generate.
*   **variable_schema** (Optional[Union[List[str], Dict[str, List[str]]]]) - Optional - Default: None - Specifies feature-target relationships for model training. If None, all columns with missing values are modeled using all other columns. If List[str], models are built for specified variables. If Dict[str, List[str]], keys are target variables and values are their predictors.
*   **imputation_order** (Literal["ascending", "descending", "roman", "latin"]) - Optional - Default: "ascending" - Order to impute variables. Options: "ascending" (least to most missing), "descending" (most to least), "roman" (left to right), "latin" (right to left).
*   **mean_match_candidates** (Union[int, Dict[str, int]]) - Optional - Default: 5 - Number of candidate values to choose from during mean matching. Use 0 to skip mean matching. Can be specified per variable with a dict.
*   **mean_match_strategy** (Optional[Union[str, Dict[str, str]]]) - Optional - Default: "normal" - Strategy for mean matching. Options: "normal" (KNN on predictions), "fast" (probability-weighted for categorical), "shap" (KNN on SHAP values). Can be specified per variable.
*   **data_subset** (Union[int, Dict[str, int]]) - Optional - Default: 0 - Number of candidates to use for model training. 0 means no subsetting. Can be specified per variable. Saves memory and time.
*   **initialize_empty** (bool) - Optional - Default: False - If True, missing values are not filled before iteration 1 (LightGBM handles missing natively). If False, random initialization is used.
*   **save_all_iterations_data** (bool) - Optional - Default: True - If True, models and candidate predictions are saved for each iteration. Required for impute_new_data().
*   **copy_data** (bool) - Optional - Default: True - If True, dataset is copied. If False, data is modified in place.
*   **random_state** (Optional[Union[int, np.random.RandomState]]) - Optional - Default: None - Seed for reproducibility. If int, creates RandomState. If None, unseeded RNG.

### Attributes

*   **working_data** (pandas.DataFrame) - The dataset being imputed (copy or reference).
*   **shape** (tuple) - (n_rows, n_columns) of the dataset.
*   **datasets** (List[int]) - List of dataset indices.
*   **models** (Dict[Tuple[str, int, int], lightgbm.Booster]) - Trained models keyed by (variable, iteration, dataset).
*   **candidate_preds** (Dict[str, pandas.DataFrame]) - Predictions on candidate rows, used for mean matching.
*   **imputation_values** (Dict[str, pandas.DataFrame]) - Imputed values with MultiIndex (iteration, dataset).
*   **variable_schema** (Dict[str, List[str]]) - Target-predictor mappings.
*   **modeled_variables** (List[str]) - All variables with models trained.
*   **imputed_variables** (List[str]) - Variables with missing values to be imputed.
*   **imputation_order** (List[str]) - Order of variable imputation.
*   **model_training_order** (List[str]) - Order of model training (imputed vars then others).
*   **na_counts** (Dict[str, int]) - Count of missing values per variable.
*   **na_where** (Dict[str, np.ndarray]) - Row indices of missing values per variable.
*   **mean_match_candidates** (Dict[str, int]) - Mean match candidates per variable.
*   **mean_match_strategy** (Dict[str, str]) - Mean match strategy per variable.
*   **optimal_parameters** (Dict[str, Dict[str, Any]]) - Optimal parameters found via tune_parameters().
*   **loggers** (List[Logger]) - Loggers from MICE and other operations.
```

--------------------------------

### Customize LightGBM Parameters Globally

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Run the MICE algorithm for additional iterations, applying global LightGBM parameters like n_estimators.

```python
# Run the MICE algorithm for 1 more iteration on the kernel with new parameters
kernel.mice(iterations=1, n_estimators=50)
```

--------------------------------

### Prepare Data for Imputation with Variable Schema

Source: https://github.com/anothersamwilson/miceforest/blob/master/README_gen.ipynb

Prepares the dataset by ensuring a specific column ('sepal width (cm)') has no missing values before initializing the ImputationKernel. This is useful when you want to ensure certain columns are not imputed or are used as a reference.

```python
# Set petal length (cm) in our amputed data 
# to original values with no missing data.
iris_amp['sepal width (cm)'] = iris['sepal width (cm)'].copy()
iris_amp.isnull().sum()
```

--------------------------------

### Initialize Kernel with Default Variable Schema

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Initializes the kernel without specifying a variable schema, allowing all variables with missing values to be modeled using all other variables as predictors.

```python
kernel = ImputationKernel(df)
```

--------------------------------

### Initialize Kernel with Default Mean Match Strategy

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/configuration.md

Sets the mean matching strategy to 'normal' for all variables. This strategy uses K-NN on predictions and is suitable for all data types.

```python
kernel = ImputationKernel(df, mean_match_strategy='normal')
```

--------------------------------

### Perform MICE on Kernel Data

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/imputation-kernel.md

Use this method to perform Multiple Imputation by Chained Equations (MICE) on the kernel dataset. It modifies the kernel in place. Ensure you have a pandas DataFrame with missing values to start.

```python
import miceforest as mf
import pandas as pd

# Create sample data with missing values
df = pd.DataFrame({
    'A': [1, 2, None, 4],
    'B': [5, None, 7, 8],
    'C': [9, 10, 11, 12]
})

# Create kernel and run MICE
kernel = mf.ImputationKernel(df, num_datasets=3)
kernel.mice(iterations=5, verbose=True)

# Get completed data
completed = kernel.complete_data(dataset=0, iteration=-1)
```

--------------------------------

### Record Elapsed Time

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/logger.md

Records the elapsed time since `set_start_time()` was called for a given key. Time accumulates if the key is used multiple times. An AssertionError is raised if the timer was never started for this key.

```python
def record_time(self, time_key: Tuple) -> None
```

```python
logger = Logger(name="test", timed_levels=["Dataset", "Iteration", "Variable"])

logger.set_start_time((0, 1, "age"))
# ... do work ...
logger.record_time((0, 1, "age"))

logger.set_start_time((0, 1, "salary"))
# ... do work ...
logger.record_time((0, 1, "salary"))
```

--------------------------------

### ImputationKernel Initialization Source

Source: https://github.com/anothersamwilson/miceforest/blob/master/_autodocs/INDEX.md

Source code for the ImputationKernel constructor. This snippet shows the initialization logic for the kernel.

```python
miceforest/imputation_kernel.py:143-162    # ImputationKernel.__init__
```