### get_projection_variable Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Gets the name of the sliced variable. ```APIDOC ## get_projection_variable ### Description Get the name of the sliced variable. ### Method Method: get_projection_variable ### Parameters None. ### Returns - **out** (str) - Sliced variable. ``` -------------------------------- ### entrypoint() Source: https://hipe4ml.github.io/hipe4ml/index.html This is the entrypoint function that can be called from the command line. It prints a greeting message. ```APIDOC ## entrypoint() ### Description This is the entrypoint function that can be called from the command line. It prints a greeting message. ### Method Function ### Signature `def entrypoint()` ### Parameters None ### Returns None ### Example ```python entrypoint() ``` ### Output ``` Hello, world! ``` ``` -------------------------------- ### entrypoint function definition Source: https://hipe4ml.github.io/hipe4ml/index.html Defines the entrypoint function that prints 'Hello, world!' when called from the command line. ```python def entrypoint(): """ This is the entrypoint: call it from command line """ print("Hello, world!\n") ``` -------------------------------- ### get_projection_binning Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Gets the bins used for slicing the DataFrame. ```APIDOC ## get_projection_binning ### Description Get the bins used for slicing the DataFrame. ### Method Method: get_projection_binning ### Parameters None. ### Returns - **out** (type not specified) - Bins used for slicing the DataFrame. ``` -------------------------------- ### TreeHandler.__init__ Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Initializes a TreeHandler instance, opening the input file(s) and converting the selected tree leaves or parquet columns into a pandas DataFrame. If tree_name is not provided, the file is assumed to be a parquet file. ```APIDOC ## TreeHandler.__init__ ### Description Initializes a TreeHandler instance, opening the input file(s) and converting the selected tree leaves or parquet columns into a pandas DataFrame. If tree_name is not provided, the file is assumed to be a parquet file. ### Method Constructor ### Signature `TreeHandler(file_name=None, tree_name=None, column_names=None, folder_name=None, **kwds)` ### Parameters - **file_name** (str or list of str) - Optional - Name of the input file where the data sit or list of input files. - **tree_name** (str) - Optional - Name of the tree within the input file, must be the same for all files. If None, the method pandas.read_parquet is called. - **column_names** (list) - Optional - List of the names of the branches that one wants to analyse. If not specified, all branches are converted. - **folder_name** (str) - Optional - Name of the folder/folders within the input file. If the folder_name ends with a '*', all folders containing the string folder_name are read and merged into a single dataframe. - **kwds** - Optional - Extra arguments passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods. ### Description of Parameters - **file_name**: Name of the input file where the data sit or list of input files. - **tree_name**: Name of the tree within the input file, must be the same for all files. If None the method pandas.read_parquet is called. - **column_names**: List of the names of the branches that one wants to analyse. If column_names is not specified all the branches are converted. - **folder_name**: Name of the folder/folders within the input file. If the folder_name ends with a '*' all the folders containing the string folder_name are read and merged into a single dataframe. Example: folder_name = "DF*" will read all the folders containing the string "DF" and merge them into a single dataframe. - **kwds**: Extra arguments are passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods. ``` -------------------------------- ### get_preselections Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Gets the preselections applied to the stored DataFrame. ```APIDOC ## get_preselections ### Description Get the preselections applied to the stored DataFrame. ### Method Method: get_preselections ### Parameters None. ### Returns - **out** (str) - String containing the cuts applied to the stored DataFrame. ``` -------------------------------- ### get_data_frame Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Gets the pandas DataFrame stored in the TreeHandler. ```APIDOC ## get_data_frame ### Description Get the pandas DataFrame stored in the TreeHandler. ### Method Method: get_data_frame ### Parameters None. ### Returns - **out** (pandas.DataFrame) - DataFrame stored in the TreeHandler. ``` -------------------------------- ### TreeHandler Constructor Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Initializes a TreeHandler instance with input file(s), optional tree name, column selection, and folder filtering. ```APIDOC ## TreeHandler Constructor ### Description Initializes a TreeHandler object to store and manage data from a ROOT tree or a pandas DataFrame. ### Parameters - **file_name** (`str` or `list` of `str`) - Required - Name of the input file where the data sit or list of input files. - **tree_name** (`str`) - Optional - Name of the tree within the input file, must be the same for all files. If None, the method pandas.read_parquet is called. - **column_names** (`list`) - Optional - List of the names of the branches that one wants to analyse. If not specified, all branches are converted. - **folder_name** (`str`) - Optional - Name of the folder/folders within the input file. If the folder_name ends with a '_', all folders containing the string folder_name are read and merged into a single dataframe. - **`**kwds`** - Optional - Extra arguments passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods. ### Notes - If tree_name is not provided, file_name is assumed to be associated to a .parquet file. - The class stores the data in a pandas DataFrame and optionally keeps slices for each bin. ``` -------------------------------- ### print_summary() Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Prints information about the TreeHandler object and its data members, including file name, tree name, DataFrame head, preselections, sliced variable, and slices binning. ```APIDOC ## print_summary() ### Description Prints information about the TreeHandler object and its data members, including file name, tree name, DataFrame head, preselections, sliced variable, and slices binning. ### Method print_summary ### Parameters None ### Returns None ### Example ```python handler.print_summary() ``` ``` -------------------------------- ### Train and test model with train_test_model Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Trains and tests the model using provided data and prints metrics: ROC AUC for classification, MSE for regression. 'data' is a list of training dataframe, training labels, test dataframe, test labels. Use 'average' and 'multi_class_opt' for multi-class AUC. 'output_margin' controls raw margin output; 'return_prediction' returns predictions; extra kwargs go to the model's fit method. ```python def train_test_model(self, data, return_prediction=False, output_margin=False, average='macro', multi_class_opt='raise', **kwargs): """ Perform the training and the testing of the model. The model performance is estimated using the ROC AUC metric for classification and the MSE for regression. Parameters ---------------------------------------------- data: list Contains respectively: training set dataframe, training label array, test set dataframe, test label array return_prediction: bool If True Model predictions on the test set are returned output_margin: bool Whether to output the raw untransformed margin value. If False model probabilities are returned. Not used when task_type is 'regression'. average: string Option for the average of ROC AUC scores used only in case of multi-classification. You can choose between 'macro' and 'weighted'. For more information see https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html#sklearn.metrics.roc_auc_score multi_class_opt: string Option to compute ROC AUC scores used only in case of multi-classification. The one-vs-one 'ovo' and one-vs-rest 'ovr' approaches are available **kwargs: dict Extra kwargs passed on to the model fit method Returns --------------------------------------- out: numpy array or None If return_prediction==True, Model predictions on the test set are returned """ # get number of classes n_classes = len(np.unique(data[1])) self._n_classes = n_classes print('==============================') print(f"Training {self.model_string} model for {self._task_type}") if self._task_type == 'classification': print('Number of detected classes:', n_classes) # final training with the optimized hyperparams print('Training the model: ...') self.fit(data[0], data[1], **kwargs) print('Training the model: Done!') print('Testing the model: ...') y_pred = self.predict(data[2], output_margin=output_margin) if self._task_type == 'classification': roc_score = roc_auc_score( data[3], y_pred, average=average, multi_class=multi_class_opt) print(f'ROC_AUC_score: {roc_score:.6f}') else: mse_score = mean_squared_error(data[3], y_pred) print(f'Mean squared error: {mse_score:.6f}') print('Testing the model: Done!') print('==============================') if return_prediction: return y_pred return None ``` -------------------------------- ### dump_original_model: Save trained model to pickle or .model file Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Saves the trained model to a pickle file, or to a .model file if xgb_format=True and the model is xgboost. If xgb_format=True with a non-xgboost model, it prints a warning and does not save the file. ```Python def dump_original_model(self, filename, xgb_format=False): """ Save the trained model into a pickle file. Only for xgboost models it is also given the possibility to save them into a .model file Parameters ----------------------------------------------------- filename: str Name of the file in which the model is saved xgb_format : bool If True saves the xgboost model into a .model file """ if xgb_format is False: with open(filename, "wb") as output_file: pickle.dump(self.model, output_file) else: if self.model_string == 'xgboost': self.model.save_model(filename) else: print("File not saved: only xgboost models support the .model extension") ``` -------------------------------- ### TreeHandler Parameters Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Describes the constructor parameters for the TreeHandler class. ```APIDOC ## TreeHandler Parameters ### Description Parameters for initializing a TreeHandler object. ### Parameters - **base_file_name** (str) - Required - Base filename used to save the root files - **path** (str) - Required - Base path of the output files - **save_slices** (bool) - Required - If True and the slices are available, single root files for each bins are created ``` -------------------------------- ### dump_original_model(filename, xgb_format=False) Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Saves the trained model into a pickle file. For xgboost models, it can also save into a .model file if xgb_format is True. ```APIDOC ## dump_original_model(filename, xgb_format=False) ### Description Saves the trained model into a pickle file. For xgboost models, it can also save into a .model file if xgb_format is True. ### Method Method of the ModelHandler class. ### Parameters - **filename** (str) - Required - Name of the file in which the model is saved. - **xgb_format** (bool) - Optional - If True saves the xgboost model into a .model file. Default is False. ### Behavior - If xgb_format is False, the model is saved using pickle.dump. - If xgb_format is True and the model is an xgboost model, it saves using the model's save_model method. - If xgb_format is True and the model is not xgboost, it prints a message and does not save. ``` -------------------------------- ### TreeHandler Constructor Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Initializes a TreeHandler object with input files, tree name, optional model handler, preselection cuts, output margin flag, and number of workers for reading data. ```APIDOC ## TreeHandler Constructor ### Description Initializes a TreeHandler object with the specified parameters. ### Parameters - **file_name** (`str` or `list` of `str`) - Required - Name of the input file where the data sit or list of input files. - **tree_name** (`str`) - Required - Name of the tree within the input file, must be the same for all files. - **model_handler** (`hipe4ml ModelHandler`) - Optional - Model handler to be applied as a preselection on the data contained in the original tree. A column named `model_output` is added to the tree_handler. In case of multi-classification a new column is added for each class with name: `model_output_{i}`. - **preselection** (`str`) - Optional - String containing the cuts to be applied as preselection on the data contained in the original tree. The string syntax is the one required in the `pandas.DataFrame.query()` method. You can refer to variables in the environment by prefixing them with an '@' character like `@a + b`. You can apply ML based preselections like in the example below: - `"model_output > @score_cut"` # binary classification - `"model_output_0 > @score_cut[0] and model_output_1 <= @score_cut[1]"` # multi-classification. - **output_margin** (`bool`) - Optional - Whether to predict the raw untransformed margin value. If False model probabilities are returned. - **max_workers** (`int`) - Optional - Maximum number of workers employed to read the chunks. If `max_workers` is None or not given, it will default to the number of processors on the machine, multiplied by 5. If `max_workers==-1` the multi-threading computation is turned off. More details in: https://docs.python.org/3/library/concurrent.futures.html. ``` -------------------------------- ### train_test_model(data, return_prediction=False, output_margin=False, average='macro', multi_class_opt='raise', **kwargs) Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Performs the training and testing of the model. The model performance is estimated using the ROC AUC metric for classification and the MSE for regression. ```APIDOC ## train_test_model(data, return_prediction=False, output_margin=False, average='macro', multi_class_opt='raise', **kwargs) ### Description Perform the training and the testing of the model. The model performance is estimated using the ROC AUC metric for classification and the MSE for regression. ### Method train_test_model ### Parameters - **data** (list) - Required - Contains respectively: training set dataframe, training label array, test set dataframe, test label array - **return_prediction** (bool) - Optional - If True Model predictions on the test set are returned - **output_margin** (bool) - Optional - Whether to output the raw untransformed margin value. If False model probabilities are returned. Not used when task_type is 'regression'. - **average** (string) - Optional - Option for the average of ROC AUC scores used only in case of multi-classification. You can choose between 'macro' and 'weighted'. For more information see https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html#sklearn.metrics.roc_auc_score - **multi_class_opt** (string) - Optional - Option to compute ROC AUC scores used only in case of multi-classification. The one-vs-one 'ovo' and one-vs-rest 'ovr' approaches are available - **kwargs** (dict) - Optional - Extra kwargs passed on to the model fit method ### Returns - **out** (numpy array or None) - If return_prediction==True, Model predictions on the test set are returned ``` -------------------------------- ### Apply ML model to data with apply_model_handler Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Applies a trained model to the data, adding a new column with the model output. Supports multi-class outputs and sliced data. If a list of model handlers is provided, applies to slices. The output_margin parameter controls raw margin output. ```python def apply_model_handler(self, model_handler, output_margin=True, column_name=None): """ Apply the ML model to data: a new column is added to the DataFrame If a list is given the application is performed on the slices. Parameters ------------------------------------------------ model_handler: list or hipe4ml model_handler If a list of handlers(one for each bin) is provided, the ML model is applied to the slices output_margin: bool Whether to output the raw untransformed margin value. column_name: str Name of the new column with the model output """ if isinstance(model_handler, list): n_class = model_handler[0].get_n_classes() sliced = True else: sliced = False n_class = model_handler.get_n_classes() if column_name is None: if n_class > 2: column_name = [f'model_output_{i_class}' for i_class in range(n_class)] else: column_name = "model_output" if sliced: for (mod_handl, sliced_df) in zip(model_handler, self._sliced_df_list): prediction = mod_handl.predict(sliced_df, output_margin) if n_class > 2: for i_class in range(n_class): sliced_df[column_name[i_class]] = prediction[:, i_class] else: sliced_df[column_name] = prediction return prediction = model_handler.predict(self._full_data_frame, output_margin) if n_class > 2: for i_class in range(n_class): self._full_data_frame[column_name[i_class]] = prediction[:, i_class] return self._full_data_frame[column_name] = prediction ``` -------------------------------- ### plot_output_train_test(model, data, bins=80, output_margin=True, labels=None, logscale=False, **kwds) Source: https://hipe4ml.github.io/hipe4ml/plot_utils.html Plots the model output distributions for each class and output both for training and test set. Supports binary and multi-class classification, with options for binning, margin output, custom labels, and log scale. ```APIDOC ## plot_output_train_test(model, data, bins=80, output_margin=True, labels=None, logscale=False, **kwds) ### Description Plots the model output distributions for each class and output both for training and test set. Supports binary and multi-class classification, with options for binning, margin output, custom labels, and log scale. ### Parameters - **model** (`hipe4ml model_handler`) - Required - The trained model handler. - **data** (`list`) - Required - Contains respectively: training set dataframe, training label array, test set dataframe, test label array. - **bins** (`int or sequence of scalars or str`) - Optional - If int, defines the number of equal-width bins (default 10). If sequence, defines bin edges. If string, defines the method used to calculate optimal bin width (see np.histogram_bin_edges). - **output_margin** (`bool`) - Optional - Whether to output the raw untransformed margin value. - **labels** (`list`) - Optional - Contains the labels to be displayed in the legend. If None, labels are class1, class2, ..., classN. - **logscale** (`bool`) - Optional - Whether to plot the y axis in log scale. - ****kwds** - Optional - Extra arguments passed on to plt.hist(). ### Returns - **out** (`matplotlib.figure.Figure` or list of them) - Model output distributions for each class. ``` -------------------------------- ### optimize_params_optuna method Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Optimizes hyperparameters using Optuna. Requires data (train/test dataframes and label arrays), hyperparameter ranges (tuples or lists for ranges, constants otherwise), and a cross-validation scoring metric. Sets the model's hyperparameters to the best found. Optional parameters control the optimization direction, sampler, number of folds, and saving/resuming studies. ```python def optimize_params_optuna(self, data, hyperparams_ranges, cross_val_scoring, nfold=5, direction='maximize', optuna_sampler=None, resume_study=None, save_study=None, **kwargs): """ Perform hyperparameter optimization of ModelHandler using the Optuna module. The model hyperparameters are automatically set as the ones that provided the best result during the optimization. Parameters ------------------------------------------------------ data: list Contains respectively: training set dataframe, training label array, test set dataframe, test label array hyperparams_ranges: dict Hyperparameter ranges (in tuples or list). If a parameter is not in a tuple or a list it will be considered constant. Important: the type of the params must be preserved when passing the ranges. For example: dict={ 'max_depth':(10,100) 'learning_rate': (0.01,0.03) 'n_jobs': 8 } cross_val_scoring: string, callable or None Score metrics used for the cross-validation. A string (see sklearn model evaluation documentation: https://scikit-learn.org/stable/modules/model_evaluation.html) or a scorer callable object / function with signature scorer(estimator, X, y) which should return only a single value. In binary classification 'roc_auc' is suggested. In multi-classification one between 'roc_auc_ovr', 'roc_auc_ovo', 'roc_auc_ovr_weighted' and 'roc_auc_ovo_weighted' is suggested. For more information see https://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter direction: str The direction of optimization. Either 'maximize' or 'minimize'. (e.g. for the metric 'roc_auc' the direction is 'maximize') optuna_sampler: optuna.samplers.BaseSampler Sampler to be used for the optuna (maxi-)minimisation. If None, default TPESampler is used. For more information see: https://optuna.readthedocs.io/en/stable/reference/samplers.html nfold: int Number of folds to calculate the cross validation error resume_study: str A string indicating the filename of the study to be resumed. If None, the study is not resumed. save_study: str A string indicating the filename of the study. If None, the study is not saved into a file. **kwargs: dict Optuna study parameters Returns ------------------------------------------------------ """ ``` -------------------------------- ### print_summary Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Prints information about the TreeHandler object and its data members, including file name, tree name, dataframe head, preselections, sliced variable, and slices binning. ```APIDOC ## print_summary ### Description Prints information about the TreeHandler object and its data members. ### Method print_summary ### Parameters None ### Returns None ``` -------------------------------- ### TreeHandler Methods Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Lists the methods available in the TreeHandler class for users to call directly. ```APIDOC ## TreeHandler Methods ### Description Methods available on the TreeHandler class for manipulating and processing data frames and root files. ### Methods - `apply_model_handler` - `apply_preselections` - `eval_data_frame` - `get_data_frame` - `get_handler_from_large_file` - `get_n_cand` - `get_preselections` - `get_projection_binning` - `get_projection_variable` - `get_slice` - `get_sliced_df_list` - `get_subset` - `get_var_names` - `print_summary` - `set_data_frame` - `shuffle_data_frame` - `slice_data_frame` - `write_df_to_parquet_files` - `write_df_to_root_files` ``` -------------------------------- ### load_model_handler method Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Loads a previously saved model handler from a pickle file. It restores the model, training columns, model parameters, module, number of classes, and task type. The filename must be a string pointing to the saved pickle file. ```python def load_model_handler(self, filename): """ Load a model handler saved into a pickle file Parameters ----------------------------------------------------- filename: str Name of the file in which the model is saved """ with open(filename, "rb") as input_file: loaded_model = pickle.load(input_file) self.model = loaded_model.get_original_model() self.training_columns = loaded_model.get_training_columns() self.model_params = loaded_model.get_model_params() self.model.set_params(**self.model_params) self.model_string = loaded_model.get_model_module() self._n_classes = loaded_model.get_n_classes() self._task_type = loaded_model.get_task_type() ``` -------------------------------- ### set_training_columns(training_columns) Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Sets the features used for the training process. The training_columns parameter is a list of feature names. ```APIDOC ## set_training_columns(training_columns) ### Description Set the features used for the training process. ### Method set_training_columns ### Parameters - **training_columns** (list) - Required - Contains the name of the features used for the training. Example: ['dEdx', 'pT', 'ct'] ### Returns None ``` -------------------------------- ### get_model Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Returns the underlying machine learning model (XGBoost, LGBM, or sklearn model). ```APIDOC ## get_model ### Description Returns the underlying machine learning model (XGBoost, LGBM, or sklearn model). ### Method `get_model()` ### Returns - **out** (XGBoost, LGBM, or sklearn model) - The trained model object. ``` -------------------------------- ### TreeHandler class definition and initialization Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Defines the TreeHandler class with its __init__ method, which opens a ROOT tree or parquet file and converts it to a pandas DataFrame. Supports multiple files, column selection, and folder wildcards. ```python class TreeHandler: """ Class for storing and managing the data of a ROOT tree from a .root file or a pandas.DataFrame from a .parquet file """ def __init__(self, file_name=None, tree_name=None, column_names=None, folder_name=None, **kwds): """ Open the file in which the selected tree leaves are converted into pandas dataframe columns. If tree_name is not provided file_name is assumed to be associated to a .parquet file Parameters ------------------------------------------------ file_name: str or list of str Name of the input file where the data sit or list of input files tree_name: str Name of the tree within the input file, must be the same for all files. If None the method pandas.read_parquet is called column_names: list List of the names of the branches that one wants to analyse. If column_names is not specified all the branches are converted folder_name: str Name of the folder/folders within the input file. If the folder_name ends with a '*' all the folders containing the string folder_name are read and merged into a single dataframe. Example: folder_name = "DF*" will read all the folders containing the string "DF" and merge them into a single dataframe. **kwds: extra arguments are passed on to the uproot.TTree.arrays() or pandas.read_parquet() methods: https://uproot.readthedocs.io/en/latest/uproot.behaviors.TTree.TTree.html#uproot.behaviors.TTree.TTree.arrays https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_parquet.html#pandas.read_parquet """ self._tree = tree_name self._full_data_frame = None self._preselections = None self._projection_variable = None self._projection_binning = None self._sliced_df_list = None if file_name is None: return self._full_data_frame = pd.DataFrame() self._files = file_name if isinstance(file_name, list) else [file_name] for file in self._files: if self._tree is None: # read from a parquet file self._full_data_frame = pd.concat( [self._full_data_frame, pd.read_parquet(file, columns=column_names, **kwds)], ignore_index=True, copy=False) continue if folder_name is None: self._full_data_frame = pd.concat( [self._full_data_frame, uproot.open(f'{file}:{self._tree}').arrays(filter_name=column_names, library='pd', **kwds)], ignore_index=True, copy=False) continue if folder_name[-1] != '*': self._full_data_frame = pd.concat( [self._full_data_frame, uproot.open(f'{file}:{folder_name}/{self._tree}').arrays(filter_name=column_names, library='pd', **kwds)], ignore_index=True, copy=False) continue file_folders = uproot.open(file).keys() # check if there are multiple cycles of the same tree, keep only last one # first we sort to have as first one the last cycle file_folders.sort(reverse=True) file_folders_to_remove = [] for ifolder, folder in enumerate(file_folders[1:]): obj_nocycle = folder.split(";")[0] if obj_nocycle in file_folders[ifolder]: file_folders_to_remove.append(folder) for folder_to_remove in file_folders_to_remove: file_folders.remove(folder_to_remove) tree_path_list = [] for folder in file_folders: if folder_name[:-1] in folder and self._tree in folder: tree_path_list.append(folder) for tree_path in tree_path_list: print(f"Reading {file}:{tree_path}") self._full_data_frame = pd.concat( [self._full_data_frame, uproot.open(f'{file}:{tree_path}').arrays(filter_name=column_names, library='pd', **kwds)], ignore_index=True, copy=False) ``` -------------------------------- ### print_summary method source code Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Prints information about the TreeHandler object and its data members, including file name, tree name, DataFrame head, preselections, sliced variable, and slices binning. ```python def print_summary(self): """ Print information about the TreeHandler object and its data members """ print("\nFile name: ", self._files) print("Tree name: ", self._tree) print("DataFrame head:\n", self._full_data_frame.head(5)) print("\nPreselections:", self._preselections) print("Sliced variable: ", self._projection_variable) print("Slices binning: ", self._projection_binning) ``` -------------------------------- ### apply_preselections Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Applies preselections to the data using a query string. ```APIDOC ## apply_preselections ### Description Applies preselections to the data using a query string. The query syntax follows pandas.DataFrame.query(). If inplace is True, the preselected dataframe replaces the initial dataframe; otherwise, a copy is returned. ### Method POST ### Endpoint TreeHandler.apply_preselections(preselections, inplace=True, **kwds) ### Parameters #### Path Parameters - **preselections** (str) - Required - String containing the cuts to be applied as preselection. Syntax as in pandas.DataFrame.query(). #### Query Parameters - **inplace** (bool) - Optional - If True, the preselected dataframe replaces the initial dataframe. Otherwise, return a copy. Default: True. - **kwds** (dict) - Optional - Extra arguments passed to pandas.DataFrame.query(). ### Request Example ``` { "preselections": "var1 > 0", "inplace": true } ``` ### Response #### Success Response - **out** (TreeHandler or None) - If inplace is True, returns None and the full DataFrame is replaced. Otherwise, returns a new TreeHandler with the preselected data. #### Response Example ``` None ``` ``` -------------------------------- ### apply_preselections method implementation Source: https://hipe4ml.github.io/hipe4ml/tree_handler.html Applies preselection cuts to the data using pandas.DataFrame.query. If inplace is True, the original DataFrame is modified and None is returned; otherwise a deep copy is made and returned. The preselection string uses pandas query syntax, with environment variables prefixed by '@'. ```python def apply_preselections(self, preselections, inplace=True, **kwds): """ Apply preselections to data Parameters ------------------------------------------------ preselection: str String containing the cuts to be applied as preselection on the data contained in the original tree. The string syntax is the one required in the pandas.DataFrame.query() method. You can refer to variables in the environment by prefixing them with an ‘@’ character like @a + b. inplace: bool If True, the preselected dataframe replaces the initial dataframe. Otherwise return a copy of the preselected df **kwds: extra arguments are passed on to the pandas.DataFrame.query method: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.query.html#pandas.DataFrame.query Returns ------------------------------------------------ out: TreeHandler or None If inplace == True return None is returned and the full DataFrame is replaced """ if inplace: if self._preselections: self._preselections += " and " + preselections else: self._preselections = preselections self._full_data_frame.query(preselections, inplace=True, **kwds) return None new_hndl = copy.deepcopy(self) new_hndl._preselections = preselections # pylint: disable=W0212 new_hndl._full_data_frame.query(preselections, inplace=True, **kwds) # pylint: disable=W0212 return new_hndl ``` -------------------------------- ### fit: Train model on training data with optional kwargs Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Fits the underlying model on the given training data. For classification tasks, it calculates and stores the number of classes. If training_columns is not set, it uses all columns from x_train. Extra keyword arguments are passed to the model's fit method. ```Python def fit(self, x_train, y_train, **kwargs): """ Fit Model Parameters --------------------------- x_train: array-like, sparse matrix Training data y_train: array-like, sparse matrix Target data **kwargs: Extra kwargs passed on to model.fit() method """ if self._task_type == 'classification': n_classes = len(np.unique(y_train)) self._n_classes = n_classes if self.training_columns is None: self.training_columns = list(x_train.columns) self.model.fit(x_train[self.training_columns], y_train, **kwargs) ``` -------------------------------- ### ModelHandler constructor Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Initializes a ModelHandler instance with an optional input model, training columns, model parameters, and task type. Validates the task type and model module. ```APIDOC ## ModelHandler(input_model=None, training_columns=None, model_params=None, task_type='classification') ### Description Creates a new ModelHandler instance wrapping an input model. Validates that the task type is either 'classification' or 'regression' and that the model module is one of 'xgboost', 'lightgbm', or 'sklearn'. If model_params are provided, they are set on the model. ### Parameters - **input_model** (XGBoost, LightGBM or sklearn model) - Optional - The model to wrap. - **training_columns** (list) - Optional - Names of features used for training. Example: ['dEdx', 'pT', 'ct'] - **model_params** (dict) - Optional - Model hyper-parameter values. Example (XGBoost): max_depth, learning_rate, n_estimators, gamma, min_child_weight, ... - **task_type** (str) - Optional - Task type: 'classification' or 'regression'. Default: 'classification' ### Returns - **ModelHandler** - The initialized ModelHandler instance. ### Raises - **ValueError** - If task_type is not 'classification' or 'regression', or if the model module is not 'xgboost', 'lightgbm', or 'sklearn'. ``` -------------------------------- ### get_n_classes Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Returns the number of classes. ```APIDOC ## get_n_classes ### Description Returns the number of classes. ### Returns - **out** (int) - Number of classes ``` -------------------------------- ### Set training features with set_training_columns Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Assigns the feature names used in model training. The 'training_columns' parameter should be a list of feature names, e.g., ['dEdx', 'pT', 'ct']. These features are stored in the instance attribute 'training_columns'. ```python def set_training_columns(self, training_columns): """ Set the features used for the training process Parameters ------------------------------------ training_columns: list Contains the name of the features used for the training. Example: ['dEdx', 'pT', 'ct'] """ self.training_columns = training_columns ``` -------------------------------- ### get_original_model Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Returns the original unwrapped model. ```APIDOC ## get_original_model ### Description Returns the original unwrapped model. ### Returns - **out** (XGBoost, LGBM or sklearn model) - The original model object ``` -------------------------------- ### get_training_columns() Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Returns the list of feature names used for training. ```APIDOC ## get_training_columns() ### Description Returns the list of feature names used for training. ### Parameters None ### Returns - **out** (list) - Names of features used for training. Example: ['dEdx', 'pT', 'ct'] ``` -------------------------------- ### get_original_model() Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Returns the original unwrapped model. ```APIDOC ## get_original_model() ### Description Returns the original unwrapped model object. ### Parameters None ### Returns - **out** (XGBoost, LGBM or sklearn model) - The original model. ``` -------------------------------- ### fit(x_train, y_train, **kwargs) Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Fits the model to the training data. ```APIDOC ## fit(x_train, y_train, **kwargs) ### Description Fits the model using the provided training data and target values. ### Parameters - **x_train** (array-like, sparse matrix) - Required - Training data. - **y_train** (array-like, sparse matrix) - Required - Target data. - **kwargs** - Optional - Additional keyword arguments passed to the underlying model's fit method. ### Returns - None ``` -------------------------------- ### get_n_classes() Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Returns the number of classes. ```APIDOC ## get_n_classes() ### Description Returns the number of classes. ### Parameters None ### Returns - **out** (int) - Number of classes. ``` -------------------------------- ### set_training_columns(training_columns) Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Sets the features used for the training process. ```APIDOC ## set_training_columns(training_columns) ### Description Sets the list of feature names used for training. ### Parameters - **training_columns** (list) - Required - Names of features used for training. Example: ['dEdx', 'pT', 'ct'] ### Returns - None ``` -------------------------------- ### fit(x_train, y_train, **kwargs) Source: https://hipe4ml.github.io/hipe4ml/model_handler.html Fits the model to the training data. It determines the number of classes for classification tasks and stores the training columns if not already set. ```APIDOC ## fit(x_train, y_train, **kwargs) ### Description Fits the model to the training data. It determines the number of classes for classification tasks and stores the training columns if not already set. ### Method Method of the ModelHandler class. ### Parameters - **x_train** (array-like, sparse matrix) - Required - Training data. - **y_train** (array-like, sparse matrix) - Required - Target data. - **kwargs** - Optional - Extra kwargs passed on to model.fit() method. ### Behavior - If the task type is classification, it computes the number of unique classes and stores it. - If training_columns is None, it sets training_columns to the list of column names from x_train. - Calls self.model.fit with the training columns and the provided kwargs. ```