hipe4ml

https://hipe4ml.github.io
Admin
hipe4ml is a Python package designed for heavy-ion physics that integrates machine learning...
Tokens:61,331
Snippets:290
Trust Score:9.7
Update:1 week ago
Show doc for...
Tokens:
Raw
### get_projection_variable

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Gets the name of the sliced variable.

```APIDOC
## get_projection_variable

### Description
Get the name of the sliced variable.

### Method
Method: get_projection_variable

### Parameters
None.

### Returns
- **out** (str) - Sliced variable.

```

--------------------------------

### entrypoint()

Source: https://hipe4ml.github.io/hipe4ml/index.html

This is the entrypoint function that can be called from the command line. It prints a greeting message.

```APIDOC
## entrypoint()

### Description
This is the entrypoint function that can be called from the command line. It prints a greeting message.

### Method
Function

### Signature
`def entrypoint()`

### Parameters
None

### Returns
None

### Example
```python
entrypoint()
```

### Output
```
Hello, world!
```
```

--------------------------------

### entrypoint function definition

Source: https://hipe4ml.github.io/hipe4ml/index.html

Defines the entrypoint function that prints 'Hello, world!' when called from the command line.

```python
def entrypoint():
    """ This is the entrypoint: call it from command line
    """
    print("Hello, world!\n")
```

--------------------------------

### get_projection_binning

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Gets the bins used for slicing the DataFrame.

```APIDOC
## get_projection_binning

### Description
Get the bins used for slicing the DataFrame.

### Method
Method: get_projection_binning

### Parameters
None.

### Returns
- **out** (type not specified) - Bins used for slicing the DataFrame.

```

--------------------------------

### TreeHandler.__init__

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Initializes a TreeHandler instance, opening the input file(s) and converting the selected tree leaves or parquet columns into a pandas DataFrame. If tree_name is not provided, the file is assumed to be a parquet file.

```APIDOC
## TreeHandler.__init__

### Description
Initializes a TreeHandler instance, opening the input file(s) and converting the selected tree leaves or parquet columns into a pandas DataFrame. If tree_name is not provided, the file is assumed to be a parquet file.

### Method
Constructor

### Signature
`TreeHandler(file_name=None, tree_name=None, column_names=None, folder_name=None, **kwds)`

### Parameters
- **file_name** (str or list of str) - Optional - Name of the input file where the data sit or list of input files.
- **tree_name** (str) - Optional - Name of the tree within the input file, must be the same for all files. If None, the method pandas.read_parquet is called.
- **column_names** (list) - Optional - List of the names of the branches that one wants to analyse. If not specified, all branches are converted.
- **folder_name** (str) - Optional - Name of the folder/folders within the input file. If the folder_name ends with a '*', all folders containing the string folder_name are read and merged into a single dataframe.
- **kwds** - Optional - Extra arguments passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods.

### Description of Parameters
- **file_name**: Name of the input file where the data sit or list of input files.
- **tree_name**: Name of the tree within the input file, must be the same for all files. If None the method pandas.read_parquet is called.
- **column_names**: List of the names of the branches that one wants to analyse. If column_names is not specified all the branches are converted.
- **folder_name**: Name of the folder/folders within the input file. If the folder_name ends with a '*' all the folders containing the string folder_name are read and merged into a single dataframe. Example: folder_name = "DF*" will read all the folders containing the string "DF" and merge them into a single dataframe.
- **kwds**: Extra arguments are passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods.
```

--------------------------------

### get_preselections

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Gets the preselections applied to the stored DataFrame.

```APIDOC
## get_preselections

### Description
Get the preselections applied to the stored DataFrame.

### Method
Method: get_preselections

### Parameters
None.

### Returns
- **out** (str) - String containing the cuts applied to the stored DataFrame.

```

--------------------------------

### get_data_frame

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Gets the pandas DataFrame stored in the TreeHandler.

```APIDOC
## get_data_frame

### Description
Get the pandas DataFrame stored in the TreeHandler.

### Method
Method: get_data_frame

### Parameters
None.

### Returns
- **out** (pandas.DataFrame) - DataFrame stored in the TreeHandler.

```

--------------------------------

### TreeHandler Constructor

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Initializes a TreeHandler instance with input file(s), optional tree name, column selection, and folder filtering.

```APIDOC
## TreeHandler Constructor

### Description
Initializes a TreeHandler object to store and manage data from a ROOT tree or a pandas DataFrame.

### Parameters
- **file_name** (`str` or `list` of `str`) - Required - Name of the input file where the data sit or list of input files.
- **tree_name** (`str`) - Optional - Name of the tree within the input file, must be the same for all files. If None, the method pandas.read_parquet is called.
- **column_names** (`list`) - Optional - List of the names of the branches that one wants to analyse. If not specified, all branches are converted.
- **folder_name** (`str`) - Optional - Name of the folder/folders within the input file. If the folder_name ends with a '_', all folders containing the string folder_name are read and merged into a single dataframe.
- **`**kwds`** - Optional - Extra arguments passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods.

### Notes
- If tree_name is not provided, file_name is assumed to be associated to a .parquet file.
- The class stores the data in a pandas DataFrame and optionally keeps slices for each bin.
```

--------------------------------

### print_summary()

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Prints information about the TreeHandler object and its data members, including file name, tree name, DataFrame head, preselections, sliced variable, and slices binning.

```APIDOC
## print_summary()

### Description
Prints information about the TreeHandler object and its data members, including file name, tree name, DataFrame head, preselections, sliced variable, and slices binning.

### Method
print_summary

### Parameters
None

### Returns
None

### Example
```python
handler.print_summary()
```
```

--------------------------------

### Train and test model with train_test_model

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Trains and tests the model using provided data and prints metrics: ROC AUC for classification, MSE for regression. 'data' is a list of training dataframe, training labels, test dataframe, test labels. Use 'average' and 'multi_class_opt' for multi-class AUC. 'output_margin' controls raw margin output; 'return_prediction' returns predictions; extra kwargs go to the model's fit method.

```python
def train_test_model(self, data, return_prediction=False, output_margin=False, average='macro',
                     multi_class_opt='raise', **kwargs):
    """
    Perform the training and the testing of the model. The model performance is estimated
    using the ROC AUC metric for classification and the MSE for regression.

    Parameters
    ----------------------------------------------
    data: list
        Contains respectively: training
        set dataframe, training label array,
        test set dataframe, test label array

    return_prediction: bool
        If True Model predictions on the test set are
        returned

    output_margin: bool
        Whether to output the raw untransformed margin value. If False model
        probabilities are returned. Not used when task_type is 'regression'.

    average: string
        Option for the average of ROC AUC scores used only in case of multi-classification.
        You can choose between 'macro' and 'weighted'. For more information see
        https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html#sklearn.metrics.roc_auc_score

    multi_class_opt: string
        Option to compute ROC AUC scores used only in case of multi-classification.
        The one-vs-one 'ovo' and one-vs-rest 'ovr' approaches are available

    **kwargs: dict
        Extra kwargs passed on to the model fit method

    Returns
    ---------------------------------------
    out: numpy array or None
        If return_prediction==True, Model predictions on the test set are
        returned

    """

    # get number of classes
    n_classes = len(np.unique(data[1]))
    self._n_classes = n_classes
    print('==============================')
    print(f"Training {self.model_string} model for {self._task_type}")
    if self._task_type == 'classification':
        print('Number of detected classes:', n_classes)

    # final training with the optimized hyperparams
    print('Training the model: ...')
    self.fit(data[0], data[1], **kwargs)
    print('Training the model: Done!')
    print('Testing the model: ...')
    y_pred = self.predict(data[2], output_margin=output_margin)
    if self._task_type == 'classification':
        roc_score = roc_auc_score(
            data[3], y_pred, average=average, multi_class=multi_class_opt)
        print(f'ROC_AUC_score: {roc_score:.6f}')
    else:
        mse_score = mean_squared_error(data[3], y_pred)
        print(f'Mean squared error: {mse_score:.6f}')
    print('Testing the model: Done!')
    print('==============================')
    if return_prediction:
        return y_pred
    return None
```

--------------------------------

### dump_original_model: Save trained model to pickle or .model file

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Saves the trained model to a pickle file, or to a .model file if xgb_format=True and the model is xgboost. If xgb_format=True with a non-xgboost model, it prints a warning and does not save the file.

```Python
def dump_original_model(self, filename, xgb_format=False):
    """
    Save the trained model into a pickle
    file. Only for xgboost models it is also given
    the possibility to save them into a .model file

    Parameters
    -----------------------------------------------------
    filename: str
        Name of the file in which the model is saved

    xgb_format : bool
        If True saves the xgboost model into a .model file
    """
    if xgb_format is False:
        with open(filename, "wb") as output_file:
            pickle.dump(self.model, output_file)
    else:
        if self.model_string == 'xgboost':
            self.model.save_model(filename)
        else:
            print("File not saved: only xgboost models support the .model extension")
```

--------------------------------

### TreeHandler Parameters

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Describes the constructor parameters for the TreeHandler class.

```APIDOC
## TreeHandler Parameters

### Description
Parameters for initializing a TreeHandler object.

### Parameters
- **base_file_name** (str) - Required - Base filename used to save the root files
- **path** (str) - Required - Base path of the output files
- **save_slices** (bool) - Required - If True and the slices are available, single root files for each bins are created
```

--------------------------------

### dump_original_model(filename, xgb_format=False)

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Saves the trained model into a pickle file. For xgboost models, it can also save into a .model file if xgb_format is True.

```APIDOC
## dump_original_model(filename, xgb_format=False)

### Description
Saves the trained model into a pickle file. For xgboost models, it can also save into a .model file if xgb_format is True.

### Method
Method of the ModelHandler class.

### Parameters
- **filename** (str) - Required - Name of the file in which the model is saved.
- **xgb_format** (bool) - Optional - If True saves the xgboost model into a .model file. Default is False.

### Behavior
- If xgb_format is False, the model is saved using pickle.dump.
- If xgb_format is True and the model is an xgboost model, it saves using the model's save_model method.
- If xgb_format is True and the model is not xgboost, it prints a message and does not save.
```

--------------------------------

### TreeHandler Constructor

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Initializes a TreeHandler object with input files, tree name, optional model handler, preselection cuts, output margin flag, and number of workers for reading data.

```APIDOC
## TreeHandler Constructor

### Description
Initializes a TreeHandler object with the specified parameters.

### Parameters
- **file_name** (`str` or `list` of `str`) - Required - Name of the input file where the data sit or list of input files.
- **tree_name** (`str`) - Required - Name of the tree within the input file, must be the same for all files.
- **model_handler** (`hipe4ml ModelHandler`) - Optional - Model handler to be applied as a preselection on the data contained in the original tree. A column named `model_output` is added to the tree_handler. In case of multi-classification a new column is added for each class with name: `model_output_{i}`.
- **preselection** (`str`) - Optional - String containing the cuts to be applied as preselection on the data contained in the original tree. The string syntax is the one required in the `pandas.DataFrame.query()` method. You can refer to variables in the environment by prefixing them with an '@' character like `@a + b`. You can apply ML based preselections like in the example below: - `"model_output > @score_cut"` # binary classification - `"model_output_0 > @score_cut[0] and model_output_1 <= @score_cut[1]"` # multi-classification.
- **output_margin** (`bool`) - Optional - Whether to predict the raw untransformed margin value. If False model probabilities are returned.
- **max_workers** (`int`) - Optional - Maximum number of workers employed to read the chunks. If `max_workers` is None or not given, it will default to the number of processors on the machine, multiplied by 5. If `max_workers==-1` the multi-threading computation is turned off. More details in: https://docs.python.org/3/library/concurrent.futures.html.
```

--------------------------------

### train_test_model(data, return_prediction=False, output_margin=False, average='macro', multi_class_opt='raise', **kwargs)

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Performs the training and testing of the model. The model performance is estimated using the ROC AUC metric for classification and the MSE for regression.

```APIDOC
## train_test_model(data, return_prediction=False, output_margin=False, average='macro', multi_class_opt='raise', **kwargs)

### Description
Perform the training and the testing of the model. The model performance is estimated using the ROC AUC metric for classification and the MSE for regression.

### Method
train_test_model

### Parameters
- **data** (list) - Required - Contains respectively: training set dataframe, training label array, test set dataframe, test label array
- **return_prediction** (bool) - Optional - If True Model predictions on the test set are returned
- **output_margin** (bool) - Optional - Whether to output the raw untransformed margin value. If False model probabilities are returned. Not used when task_type is 'regression'.
- **average** (string) - Optional - Option for the average of ROC AUC scores used only in case of multi-classification. You can choose between 'macro' and 'weighted'. For more information see https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html#sklearn.metrics.roc_auc_score
- **multi_class_opt** (string) - Optional - Option to compute ROC AUC scores used only in case of multi-classification. The one-vs-one 'ovo' and one-vs-rest 'ovr' approaches are available
- **kwargs** (dict) - Optional - Extra kwargs passed on to the model fit method

### Returns
- **out** (numpy array or None) - If return_prediction==True, Model predictions on the test set are returned
```

--------------------------------

### Apply ML model to data with apply_model_handler

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Applies a trained model to the data, adding a new column with the model output. Supports multi-class outputs and sliced data. If a list of model handlers is provided, applies to slices. The output_margin parameter controls raw margin output.

```python
def apply_model_handler(self, model_handler, output_margin=True, column_name=None):
    """
    Apply the ML model to data: a new column is added to the DataFrame
    If a list is given the application is performed on the slices.

    Parameters
    ------------------------------------------------
    model_handler: list or hipe4ml model_handler
        If a list of handlers(one for each bin) is provided, the ML
        model is applied to the slices

    output_margin: bool
        Whether to output the raw untransformed margin value.

    column_name: str
        Name of the new column with the model output
    """
    if isinstance(model_handler, list):
        n_class = model_handler[0].get_n_classes()
        sliced = True
    else:
        sliced = False
        n_class = model_handler.get_n_classes()
    if column_name is None:
        if n_class > 2:
            column_name = [f'model_output_{i_class}' for i_class in range(n_class)]
        else:
            column_name = "model_output"

    if sliced:
        for (mod_handl, sliced_df) in zip(model_handler, self._sliced_df_list):
            prediction = mod_handl.predict(sliced_df, output_margin)
            if n_class > 2:
                for i_class in range(n_class):
                    sliced_df[column_name[i_class]] = prediction[:, i_class]
            else:
                sliced_df[column_name] = prediction
        return

    prediction = model_handler.predict(self._full_data_frame, output_margin)
    if n_class > 2:
        for i_class in range(n_class):
            self._full_data_frame[column_name[i_class]] = prediction[:, i_class]
        return

    self._full_data_frame[column_name] = prediction
```

--------------------------------

### plot_output_train_test(model, data, bins=80, output_margin=True, labels=None, logscale=False, **kwds)

Source: https://hipe4ml.github.io/hipe4ml/plot_utils.html

Plots the model output distributions for each class and output both for training and test set. Supports binary and multi-class classification, with options for binning, margin output, custom labels, and log scale.

```APIDOC
## plot_output_train_test(model, data, bins=80, output_margin=True, labels=None, logscale=False, **kwds)

### Description
Plots the model output distributions for each class and output both for training and test set. Supports binary and multi-class classification, with options for binning, margin output, custom labels, and log scale.

### Parameters
- **model** (`hipe4ml model_handler`) - Required - The trained model handler.
- **data** (`list`) - Required - Contains respectively: training set dataframe, training label array, test set dataframe, test label array.
- **bins** (`int or sequence of scalars or str`) - Optional - If int, defines the number of equal-width bins (default 10). If sequence, defines bin edges. If string, defines the method used to calculate optimal bin width (see np.histogram_bin_edges).
- **output_margin** (`bool`) - Optional - Whether to output the raw untransformed margin value.
- **labels** (`list`) - Optional - Contains the labels to be displayed in the legend. If None, labels are class1, class2, ..., classN.
- **logscale** (`bool`) - Optional - Whether to plot the y axis in log scale.
- ****kwds** - Optional - Extra arguments passed on to plt.hist().

### Returns
- **out** (`matplotlib.figure.Figure` or list of them) - Model output distributions for each class.
```

--------------------------------

### optimize_params_optuna method

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Optimizes hyperparameters using Optuna. Requires data (train/test dataframes and label arrays), hyperparameter ranges (tuples or lists for ranges, constants otherwise), and a cross-validation scoring metric. Sets the model's hyperparameters to the best found. Optional parameters control the optimization direction, sampler, number of folds, and saving/resuming studies.

```python
def optimize_params_optuna(self, data, hyperparams_ranges, cross_val_scoring, nfold=5, direction='maximize',
                           optuna_sampler=None, resume_study=None, save_study=None, **kwargs):
    """
    Perform hyperparameter optimization of ModelHandler using the Optuna module.
    The model hyperparameters are automatically set as the ones that provided the
    best result during the optimization.

    Parameters
    ------------------------------------------------------
    data: list
        Contains respectively: training
        set dataframe, training label array,
        test set dataframe, test label array

    hyperparams_ranges: dict
        Hyperparameter ranges (in tuples or list). If a parameter is not
        in a tuple or a list it will be considered constant.
        Important: the type of the params must be preserved
        when passing the ranges.
        For example:
        dict={
            'max_depth':(10,100)
            'learning_rate': (0.01,0.03)
            'n_jobs': 8
        }

    cross_val_scoring: string, callable or None
        Score metrics used for the cross-validation.
        A string (see sklearn model evaluation documentation:
        https://scikit-learn.org/stable/modules/model_evaluation.html)
        or a scorer callable object / function with signature scorer(estimator, X, y)
        which should return only a single value.
        In binary classification 'roc_auc' is suggested.
        In multi-classification one between 'roc_auc_ovr', 'roc_auc_ovo',
        'roc_auc_ovr_weighted' and 'roc_auc_ovo_weighted' is suggested.
        For more information see
        https://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter

    direction: str
        The direction of optimization. Either 'maximize' or 'minimize'.
        (e.g. for the metric 'roc_auc' the direction is 'maximize')

    optuna_sampler: optuna.samplers.BaseSampler
        Sampler to be used for the optuna (maxi-)minimisation.
        If None, default TPESampler is used. For more information see:
        https://optuna.readthedocs.io/en/stable/reference/samplers.html

    nfold: int
        Number of folds to calculate the cross validation error

    resume_study: str
        A string indicating the filename of the study to be resumed.
        If None, the study is not resumed.

    save_study: str
        A string indicating the filename of the study. If None,
        the study is not saved into a file.

    **kwargs: dict
        Optuna study parameters

    Returns
    ------------------------------------------------------
    """
```

--------------------------------

### print_summary

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Prints information about the TreeHandler object and its data members, including file name, tree name, dataframe head, preselections, sliced variable, and slices binning.

```APIDOC
## print_summary

### Description
Prints information about the TreeHandler object and its data members.

### Method
print_summary

### Parameters
None

### Returns
None
```

--------------------------------

### TreeHandler Methods

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Lists the methods available in the TreeHandler class for users to call directly.

```APIDOC
## TreeHandler Methods

### Description
Methods available on the TreeHandler class for manipulating and processing data frames and root files.

### Methods
- `apply_model_handler`
- `apply_preselections`
- `eval_data_frame`
- `get_data_frame`
- `get_handler_from_large_file`
- `get_n_cand`
- `get_preselections`
- `get_projection_binning`
- `get_projection_variable`
- `get_slice`
- `get_sliced_df_list`
- `get_subset`
- `get_var_names`
- `print_summary`
- `set_data_frame`
- `shuffle_data_frame`
- `slice_data_frame`
- `write_df_to_parquet_files`
- `write_df_to_root_files`
```

--------------------------------

### load_model_handler method

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Loads a previously saved model handler from a pickle file. It restores the model, training columns, model parameters, module, number of classes, and task type. The filename must be a string pointing to the saved pickle file.

```python
def load_model_handler(self, filename):
    """
    Load a model handler saved into a pickle file

    Parameters
    -----------------------------------------------------
    filename: str
        Name of the file in which the model is saved
    """
    with open(filename, "rb") as input_file:
        loaded_model = pickle.load(input_file)
        self.model = loaded_model.get_original_model()
        self.training_columns = loaded_model.get_training_columns()
        self.model_params = loaded_model.get_model_params()
        self.model.set_params(**self.model_params)
        self.model_string = loaded_model.get_model_module()
        self._n_classes = loaded_model.get_n_classes()
        self._task_type = loaded_model.get_task_type()
```

--------------------------------

### set_training_columns(training_columns)

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Sets the features used for the training process. The training_columns parameter is a list of feature names.

```APIDOC
## set_training_columns(training_columns)

### Description
Set the features used for the training process.

### Method
set_training_columns

### Parameters
- **training_columns** (list) - Required - Contains the name of the features used for the training. Example: ['dEdx', 'pT', 'ct']

### Returns
None
```

--------------------------------

### get_model

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Returns the underlying machine learning model (XGBoost, LGBM, or sklearn model).

```APIDOC
## get_model

### Description
Returns the underlying machine learning model (XGBoost, LGBM, or sklearn model).

### Method
`get_model()`

### Returns
- **out** (XGBoost, LGBM, or sklearn model) - The trained model object.
```

--------------------------------

### TreeHandler class definition and initialization

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Defines the TreeHandler class with its __init__ method, which opens a ROOT tree or parquet file and converts it to a pandas DataFrame. Supports multiple files, column selection, and folder wildcards.

```python
class TreeHandler:
    """
    Class for storing and managing the data of a ROOT tree from a .root file
    or a pandas.DataFrame from a .parquet file
    """

    def __init__(self, file_name=None, tree_name=None, column_names=None, folder_name=None, **kwds):
        """
        Open the file in which the selected tree leaves are converted
        into pandas dataframe columns. If tree_name is not provided file_name is
        assumed to be associated to a .parquet file

        Parameters
        ------------------------------------------------
        file_name: str or list of str
            Name of the input file where the data sit or list of input files

        tree_name: str
            Name of the tree within the input file, must be the same for all files.
            If None the method pandas.read_parquet is called

        column_names: list
            List of the names of the branches that one wants to analyse. If column_names is
            not specified all the branches are converted

        folder_name: str
            Name of the folder/folders within the input file. If the folder_name ends with a '*' all the folders
            containing the string folder_name are read and merged into a single dataframe.
            Example: folder_name = "DF*" will read all the folders containing the string "DF" and
            merge them into a single dataframe.

        **kwds: extra arguments are passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods:
                https://uproot.readthedocs.io/en/latest/uproot.behaviors.TTree.TTree.html#uproot.behaviors.TTree.TTree.arrays
                https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_parquet.html#pandas.read_parquet
        """
        self._tree = tree_name
        self._full_data_frame = None
        self._preselections = None
        self._projection_variable = None
        self._projection_binning = None
        self._sliced_df_list = None
        if file_name is None:
            return

        self._full_data_frame = pd.DataFrame()
        self._files = file_name if isinstance(file_name, list) else [file_name]
        for file in self._files:
            if self._tree is None:  # read from a parquet file
                self._full_data_frame = pd.concat(
                [self._full_data_frame, pd.read_parquet(file, columns=column_names, **kwds)],
                ignore_index=True, copy=False)
                continue

            if folder_name is None:
                self._full_data_frame = pd.concat(
                    [self._full_data_frame,
                        uproot.open(f'{file}:{self._tree}').arrays(filter_name=column_names,
                        library='pd', **kwds)], ignore_index=True, copy=False)
                continue

            if folder_name[-1] != '*':
                self._full_data_frame = pd.concat(
                    [self._full_data_frame,
                        uproot.open(f'{file}:{folder_name}/{self._tree}').arrays(filter_name=column_names,
                        library='pd', **kwds)], ignore_index=True, copy=False)
                continue

            file_folders = uproot.open(file).keys()
            # check if there are multiple cycles of the same tree, keep only last one
            # first we sort to have as first one the last cycle
            file_folders.sort(reverse=True)
            file_folders_to_remove = []
            for ifolder, folder in enumerate(file_folders[1:]):
                obj_nocycle = folder.split(";")[0]
                if obj_nocycle in file_folders[ifolder]:
                    file_folders_to_remove.append(folder)
            for folder_to_remove in file_folders_to_remove:
                file_folders.remove(folder_to_remove)
            tree_path_list = []
            for folder in file_folders:
                if folder_name[:-1] in folder and self._tree in folder:
                    tree_path_list.append(folder)

            for tree_path in tree_path_list:
                print(f"Reading {file}:{tree_path}")
                self._full_data_frame = pd.concat(
                    [self._full_data_frame,
                        uproot.open(f'{file}:{tree_path}').arrays(filter_name=column_names,
                        library='pd', **kwds)], ignore_index=True, copy=False)
```

--------------------------------

### print_summary method source code

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Prints information about the TreeHandler object and its data members, including file name, tree name, DataFrame head, preselections, sliced variable, and slices binning.

```python
def print_summary(self):
    """
    Print information about the TreeHandler object and its
    data members
    """
    print("\nFile name: ", self._files)
    print("Tree name: ", self._tree)
    print("DataFrame head:\n", self._full_data_frame.head(5))
    print("\nPreselections:", self._preselections)
    print("Sliced variable: ", self._projection_variable)
    print("Slices binning: ", self._projection_binning)
```

--------------------------------

### apply_preselections

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Applies preselections to the data using a query string.

```APIDOC
## apply_preselections

### Description
Applies preselections to the data using a query string. The query syntax follows pandas.DataFrame.query(). If inplace is True, the preselected dataframe replaces the initial dataframe; otherwise, a copy is returned.

### Method
POST

### Endpoint
TreeHandler.apply_preselections(preselections, inplace=True, **kwds)

### Parameters
#### Path Parameters
- **preselections** (str) - Required - String containing the cuts to be applied as preselection. Syntax as in pandas.DataFrame.query().

#### Query Parameters
- **inplace** (bool) - Optional - If True, the preselected dataframe replaces the initial dataframe. Otherwise, return a copy. Default: True.
- **kwds** (dict) - Optional - Extra arguments passed to pandas.DataFrame.query().

### Request Example
```
{
  "preselections": "var1 > 0",
  "inplace": true
}
```

### Response
#### Success Response
- **out** (TreeHandler or None) - If inplace is True, returns None and the full DataFrame is replaced. Otherwise, returns a new TreeHandler with the preselected data.

#### Response Example
```
None
```
```

--------------------------------

### apply_preselections method implementation

Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html

Applies preselection cuts to the data using pandas.DataFrame.query. If inplace is True, the original DataFrame is modified and None is returned; otherwise a deep copy is made and returned. The preselection string uses pandas query syntax, with environment variables prefixed by '@'.

```python
def apply_preselections(self, preselections, inplace=True, **kwds):
    """
    Apply preselections to data

    Parameters
    ------------------------------------------------
    preselection: str
        String containing the cuts to be applied as preselection on the data contained in the original
        tree. The string syntax is the one required in the pandas.DataFrame.query() method.
        You can refer to variables in the environment by prefixing them with an ‘@’ character like @a + b.

    inplace: bool
        If True, the preselected dataframe replaces the initial dataframe. Otherwise return a copy of the
        preselected df

    **kwds: extra arguments are passed on to the pandas.DataFrame.query method:
            https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html#pandas.DataFrame.query

    Returns
    ------------------------------------------------
    out: TreeHandler or None
        If inplace == True return None is returned and the full DataFrame is replaced
    """
    if inplace:
        if self._preselections:
            self._preselections += " and " + preselections
        else:
            self._preselections = preselections
        self._full_data_frame.query(preselections, inplace=True, **kwds)
        return None

    new_hndl = copy.deepcopy(self)
    new_hndl._preselections = preselections  # pylint: disable=W0212
    new_hndl._full_data_frame.query(preselections, inplace=True, **kwds)  # pylint: disable=W0212
    return new_hndl
```

--------------------------------

### fit: Train model on training data with optional kwargs

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Fits the underlying model on the given training data. For classification tasks, it calculates and stores the number of classes. If training_columns is not set, it uses all columns from x_train. Extra keyword arguments are passed to the model's fit method.

```Python
def fit(self, x_train, y_train, **kwargs):
    """
    Fit Model

    Parameters
    ---------------------------
    x_train: array-like, sparse matrix
        Training data

    y_train: array-like, sparse matrix
        Target data

    **kwargs:
        Extra kwargs passed on to model.fit() method
    """
    if self._task_type == 'classification':
        n_classes = len(np.unique(y_train))
        self._n_classes = n_classes
    if self.training_columns is None:
        self.training_columns = list(x_train.columns)

    self.model.fit(x_train[self.training_columns], y_train, **kwargs)
```

--------------------------------

### ModelHandler constructor

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Initializes a ModelHandler instance with an optional input model, training columns, model parameters, and task type. Validates the task type and model module.

```APIDOC
## ModelHandler(input_model=None, training_columns=None, model_params=None, task_type='classification')

### Description
Creates a new ModelHandler instance wrapping an input model. Validates that the task type is either 'classification' or 'regression' and that the model module is one of 'xgboost', 'lightgbm', or 'sklearn'. If model_params are provided, they are set on the model.

### Parameters
- **input_model** (XGBoost, LightGBM or sklearn model) - Optional - The model to wrap.
- **training_columns** (list) - Optional - Names of features used for training. Example: ['dEdx', 'pT', 'ct']
- **model_params** (dict) - Optional - Model hyper-parameter values. Example (XGBoost): max_depth, learning_rate, n_estimators, gamma, min_child_weight, ...
- **task_type** (str) - Optional - Task type: 'classification' or 'regression'. Default: 'classification'

### Returns
- **ModelHandler** - The initialized ModelHandler instance.

### Raises
- **ValueError** - If task_type is not 'classification' or 'regression', or if the model module is not 'xgboost', 'lightgbm', or 'sklearn'.
```

--------------------------------

### get_n_classes

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Returns the number of classes.

```APIDOC
## get_n_classes

### Description
Returns the number of classes.

### Returns
- **out** (int) - Number of classes
```

--------------------------------

### Set training features with set_training_columns

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Assigns the feature names used in model training. The 'training_columns' parameter should be a list of feature names, e.g., ['dEdx', 'pT', 'ct']. These features are stored in the instance attribute 'training_columns'.

```python
def set_training_columns(self, training_columns):
    """
    Set the features used for the training process

    Parameters
    ------------------------------------
    training_columns: list
        Contains the name of the features used for the training.
        Example: ['dEdx', 'pT', 'ct']
    """
    self.training_columns = training_columns
```

--------------------------------

### get_original_model

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Returns the original unwrapped model.

```APIDOC
## get_original_model

### Description
Returns the original unwrapped model.

### Returns
- **out** (XGBoost, LGBM or sklearn model) - The original model object
```

--------------------------------

### get_training_columns()

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Returns the list of feature names used for training.

```APIDOC
## get_training_columns()

### Description
Returns the list of feature names used for training.

### Parameters
None

### Returns
- **out** (list) - Names of features used for training. Example: ['dEdx', 'pT', 'ct']
```

--------------------------------

### get_original_model()

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Returns the original unwrapped model.

```APIDOC
## get_original_model()

### Description
Returns the original unwrapped model object.

### Parameters
None

### Returns
- **out** (XGBoost, LGBM or sklearn model) - The original model.
```

--------------------------------

### fit(x_train, y_train, **kwargs)

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Fits the model to the training data.

```APIDOC
## fit(x_train, y_train, **kwargs)

### Description
Fits the model using the provided training data and target values.

### Parameters
- **x_train** (array-like, sparse matrix) - Required - Training data.
- **y_train** (array-like, sparse matrix) - Required - Target data.
- **kwargs** - Optional - Additional keyword arguments passed to the underlying model's fit method.

### Returns
- None
```

--------------------------------

### get_n_classes()

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Returns the number of classes.

```APIDOC
## get_n_classes()

### Description
Returns the number of classes.

### Parameters
None

### Returns
- **out** (int) - Number of classes.
```

--------------------------------

### set_training_columns(training_columns)

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Sets the features used for the training process.

```APIDOC
## set_training_columns(training_columns)

### Description
Sets the list of feature names used for training.

### Parameters
- **training_columns** (list) - Required - Names of features used for training. Example: ['dEdx', 'pT', 'ct']

### Returns
- None
```

--------------------------------

### fit(x_train, y_train, **kwargs)

Source: https://hipe4ml.github.io/hipe4ml/model_handler.html

Fits the model to the training data. It determines the number of classes for classification tasks and stores the training columns if not already set.

```APIDOC
## fit(x_train, y_train, **kwargs)

### Description
Fits the model to the training data. It determines the number of classes for classification tasks and stores the training columns if not already set.

### Method
Method of the ModelHandler class.

### Parameters
- **x_train** (array-like, sparse matrix) - Required - Training data.
- **y_train** (array-like, sparse matrix) - Required - Target data.
- **kwargs** - Optional - Extra kwargs passed on to model.fit() method.

### Behavior
- If the task type is classification, it computes the number of unique classes and stores it.
- If training_columns is None, it sets training_columns to the list of column names from x_train.
- Calls self.model.fit with the training columns and the provided kwargs.
```
Tokens:
Raw