### Example Error Message for Node.js Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Example of an error message encountered when Node.js is missing during extension installation. ```bash jupyter labextension install jupyterlab-plotly An error occured. ValueError: Please install nodejs >=10.0.0 before continuing. nodejs may be installed using conda or directly from the nodejs website. See the log file for details: /tmp/jupyterlab-debug-y_g3xxpq.log ``` -------------------------------- ### Apply comprehensive postprocessing example Source: https://github.com/maif/shapash/blob/master/tutorial/postprocess/tuto-postprocess01.ipynb A complete example showing multiple postprocessing rules applied to a dataset. ```python postprocess = { 'Age': {'type': 'suffix', 'rule': ' years old' # Adding 'years old' at the end }, 'Sex': {'type': 'transcoding', 'rule': {'male': 'Man', 'female': 'Woman'} }, 'Pclass': {'type': 'regex', 'rule': {'in': ' class$', 'out': ''} # Deleting 'class' word at the end }, 'Fare': {'type': 'prefix', 'rule': '$' # Adding $ at the beginning }, 'Embarked': {'type': 'case', 'rule': 'upper' } } ``` -------------------------------- ### Complete Shapash Workflow Example Source: https://context7.com/maif/shapash/llms.txt End-to-end example demonstrating data loading, preprocessing, model training, Shapash explainer compilation, result exploration, and deployment. ```python from shapash import SmartExplainer from shapash.data.data_loader import data_loading from sklearn.model_selection import train_test_split from sklearn.ensemble import GradientBoostingRegressor from category_encoders import OrdinalEncoder # 1. Load data house_df, house_dict = data_loading('house_prices') X = house_df.drop('SalePrice', axis=1) y = house_df['SalePrice'] # 2. Split and encode X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) encoder = OrdinalEncoder().fit(X_train) X_train_enc = encoder.transform(X_train) X_test_enc = encoder.transform(X_test) # 3. Train model model = GradientBoostingRegressor(n_estimators=100, random_state=42) model.fit(X_train_enc, y_train) # 4. Create and compile SmartExplainer xpl = SmartExplainer( model=model, preprocessing=encoder, features_dict=house_dict, title_story='House Price Prediction' ) xpl.compile(x=X_test_enc, y_pred=model.predict(X_test_enc), y_target=y_test) # 5. Explore results fig = xpl.plot.features_importance() # Global importance fig = xpl.plot.local_plot(row_num=0) # Single prediction fig = xpl.plot.contribution_plot('OverallQual') # Feature contribution # 6. Export summary summary = xpl.to_pandas(max_contrib=5) # 7. Launch webapp app = xpl.run_app(port=8050) # 8. Generate report xpl.generate_report( output_file='house_price_report.html', project_info_file='project_info.yml', x_train=X_train, y_train=y_train, y_test=y_test, title_story='House Price Model Report' ) # 9. Create production predictor predictor = xpl.to_smartpredictor() predictor.save('house_price_predictor.pkl') # 10. Use in production predictor.add_input(x=new_house_data) prediction = predictor.predict() explanation = predictor.summarize() ``` -------------------------------- ### Install Shapash with Reporting Features Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install Shapash along with the necessary libraries to generate the Shapash Report. Use this if you need reporting functionality. ```bash pip install shapash[report] ``` -------------------------------- ### Install Shapash Source: https://github.com/maif/shapash/blob/master/docs/overview.md Use pip to install the Shapash library. ```ipython pip install shapash ``` -------------------------------- ### Initialize and compile SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/plots_and_charts/tuto-plot01-local_plot-and-to_pandas.ipynb Setup the SmartExplainer object for regression analysis. ```python from shapash import SmartExplainer ``` ```python xpl = SmartExplainer( model=regressor, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=house_dict, # Optional parameter, dict specifies label for features name ) ``` ```python xpl.compile( x=Xtest, y_pred=y_pred # Optional ) ``` -------------------------------- ### Install Node.js with Conda Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install Node.js using conda, which is a prerequisite for installing JupyterLab extensions. ```bash conda install -c conda-forge nodejs ``` -------------------------------- ### Example Output of to_pandas Source: https://github.com/maif/shapash/blob/master/tutorial/postprocess/tuto-postprocess01.ipynb This is an example of the output generated by the to_pandas() method, showing parameters used and the resulting DataFrame with feature contributions. ```text to_pandas params: {"features_to_hide": null, "threshold": null, "positive": null, "max_contrib": 20} Result: pred feature_1 value_1 contribution_1 feature_2 \ 863 1 Title of passenger Mrs 0.163479 Sex 224 0 Title of passenger Mr 0.094038 Sex 85 1 Title of passenger Miss 0.190529 Sex 681 1 Title of passenger Miss 0.237477 Port of embarkation 536 1 Title of passenger Miss 0.210166 Ticket class .. ... ... ... ... ... 507 1 Title of passenger Mrs 0.215332 Sex 468 0 Sex Man 0.100602 Passenger fare 741 0 Title of passenger Mr 0.131861 Sex 355 0 Title of passenger Mr 0.12679 Sex 450 0 Sex Man 0.13572 Title of passenger value_2 contribution_2 feature_3 value_3 \ 863 Woman 0.154309 Ticket class First 224 Man 0.0696282 Age 29.5 years old 85 Woman 0.135507 Ticket class Second 681 QUEENSTOWN 0.143451 Sex Woman 536 Second 0.168247 Sex Woman .. ... ... ... ... 507 Woman 0.194419 Ticket class Second 468 $26.55 -0.099794 Title of passenger Mr 741 Man 0.110845 Age 29.5 years old 355 Man 0.0933251 Age 29.5 years old 450 Major -0.0723023 Age 52.0 years old contribution_3 ... contribution_6 feature_7 \ 863 0.130221 ... 0.0406219 Name, First name 224 0.0658556 ... 0.0151605 Relatives such as brother or wife 85 0.0809714 ... -0.025286 Relatives like children or parents 681 0.127931 ... 0.0243567 Relatives like children or parents 536 0.0876445 ... 0.0147503 Relatives like children or parents .. ... ... ... ... 507 0.166437 ... -0.0079185 Relatives like children or parents 468 0.0967768 ... 0.0243706 Port of embarkation 741 0.104878 ... 0.0339308 Relatives such as brother or wife 355 0.0717939 ... -0.0271103 Name, First name 450 0.0690373 ... 0.027384 Relatives such as brother or wife value_7 contribution_7 \ 863 Swift Frederick Joel (Margaret Welles Barron) -0.0381955 224 0 -0.00855039 85 0 -0.0238222 681 0 0.0165205 536 2 0.0125069 .. ... ... 507 2 0.00407485 468 SOUTHAMPTON 0.0124424 741 0 -0.00715564 355 Yousif Wazli 0.0163174 450 0 -0.0134144 feature_8 value_8 \ 863 Port of embarkation SOUTHAMPTON 224 Relatives like children or parents 0 85 Relatives such as brother or wife 0 681 Passenger fare $8.14 536 Port of embarkation SOUTHAMPTON .. ... ... 507 Age 33.0 years old 468 Relatives such as brother or wife 0 741 Name, First name Hawksford Walter James 355 Relatives such as brother or wife 0 450 Relatives like children or parents 0 contribution_8 feature_9 \ 863 -0.0147327 Relatives like children or parents 224 0.00124433 Name, First name ``` -------------------------------- ### Launch Shapash Web App Source: https://github.com/maif/shapash/blob/master/docs/overview.md Start the interactive web application to visualize model explanations. ```ipython app = xpl.run_app() ``` -------------------------------- ### SmartExplainer Compilation Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Example of initializing and compiling the SmartExplainer object. ```APIDOC ## Initialization and Compilation ### Description How to initialize the SmartExplainer with a model and compile it using input data and target values. ### Request Example ```python >>> xpl = SmartExplainer(model, features_dict=featd, label_dict=labeld) >>> xpl.compile(x=x_encoded, y_target=y) >>> xpl.plot.features_importance() ``` ``` -------------------------------- ### Initialize classification SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/plots_and_charts/tuto-plot01-local_plot-and-to_pandas.ipynb Setup SmartExplainer for a classification problem with explicit labels. ```python xplclf = SmartExplainer( model=clf, preprocessing=encoder, features_dict=house_dict, label_dict=label_dict # Optional parameters: display explicit output ) ``` ```python xplclf.compile( x=Xtest, y_pred=y_pred_clf ) ``` -------------------------------- ### Install ipywidgets for JupyterLab Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Commands to install jupyterlab_widgets in the base environment and ipywidgets in a specific kernel environment. ```bash conda install -n base -c conda-forge jupyterlab_widgets conda install -n py36 -c conda-forge ipywidgets ``` -------------------------------- ### Launch and terminate the Shapash WebApp Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Starts the interactive web interface for model exploration and provides a method to stop the server. ```pycon >>> # Launch the WebApp in a Jupyter notebook >>> app = xpl.run_app(port=8050) >>> # Stop the app >>> app.kill() ``` -------------------------------- ### Verify ipywidgets Installation Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Python code to test if ipywidgets is correctly configured and functional. ```ipython3 from __future__ import print_function from ipywidgets import interact, interactive, fixed, interact_manual import ipywidgets as widgets def f(x): return x interact(f, x=10); ``` -------------------------------- ### Initialize and Compile SmartExplainer Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Initializes the SmartExplainer with a model and dictionaries, then compiles it with encoded data and target values. This setup is necessary before generating plots. ```python >>> xpl = SmartExplainer(model, features_dict=featd, label_dict=labeld) >>> xpl.compile(x=x_encoded, y_target=y) ``` -------------------------------- ### Verify Plotly Installation Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Python code to test if Plotly is correctly configured by rendering a basic figure. ```ipython3 import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Scatter(y=[2, 1, 4, 3])) fig.add_trace(go.Bar(y=[1, 4, 3, 2])) fig.update_layout(title = 'Hello Figure') fig.show("jupyterlab") ``` -------------------------------- ### Initialize Shapash SmartExplainer with Groups Source: https://github.com/maif/shapash/blob/master/tutorial/common/tuto-common01-groups_of_features.ipynb Instantiate the SmartExplainer object, providing the trained model, preprocessing steps, feature groups, and feature dictionaries. This setup enables Shapash to interpret and visualize model explanations with group-aware features. ```python from shapash import SmartExplainer # optional parameter, specifies label for features and groups name xpl = SmartExplainer( model=regressor, preprocessing=encoder, features_groups=features_groups, features_dict=house_dict ) ``` -------------------------------- ### Define postprocessing rules Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Example dictionary structure for applying postprocessing modifications to features in the x_init dataframe. ```python { ‘feature1’ : { ‘type’ : ‘prefix’, ‘rule’ : ‘age: ‘ }, ‘feature2’ : { ‘type’ : ‘suffix’, ‘rule’ : ‘$/week ‘ }, ‘feature3’ : { ‘type’ : ‘transcoding’, ‘rule‘: { ‘code1’ : ‘single’, ‘code2’ : ‘married’}}, ‘feature4’ : { ‘type’ : ‘regex’ , ‘rule‘: { ‘in’ : ‘AND’, ‘out’ : ‘ & ‘ }}, ‘feature5’ : { ‘type’ : ‘case’ , ‘rule‘: ‘lower’‘ } } ``` -------------------------------- ### Launch Shapash WebApp Source: https://context7.com/maif/shapash/llms.txt Start an interactive web interface for model exploration. The app runs in a separate thread and can be terminated using the kill method. ```python # Launch the webapp app = xpl.run_app( port=8050, host='0.0.0.0', title_story='My Model Analysis', settings={ 'rows': 100, # Number of rows to display 'points': 1000, # Points in scatter plots 'violin': 500, # Points in violin plots 'features': 15 # Features to show } ) # The app runs in a separate thread # Access at http://localhost:8050/ # Stop the webapp app.kill() ``` -------------------------------- ### Generate an interactive HTML report with Shapash Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md This example demonstrates how to invoke the generate_report method with custom project metadata, performance metrics, and interaction plot settings. ```pycon >>> xpl.generate_report( ... output_file="report.html", ... project_info_file="utils/project_info.yml", ... x_train=x_train, ... y_train=y_train, ... y_test=y_test, ... title_story="House Prices Project Report", ... title_description="Comprehensive interpretability analysis for the Kaggle house prices dataset.", ... metrics=[ ... {"path": "sklearn.metrics.mean_squared_error", "name": "Mean Squared Error"}, ... {"path": "sklearn.metrics.mean_absolute_error", "name": "Mean Absolute Error"}, ... ], ... display_interaction_plot=True, ... nb_top_interactions=5, ... ) ``` -------------------------------- ### Set Up Project Environment Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Initializes the project environment by setting up proxies and configuring Plotly for rendering. ```python from maif_datalab import utils utils.set_proxy() import warnings warnings.filterwarnings("ignore") import plotly plotly.io.renderers.default = 'png' ``` -------------------------------- ### Install ipywidgets with Conda Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install the ipywidgets package using conda. This is an alternative to pip for installing ipywidgets, especially in conda environments. ```bash conda install -c conda-forge ipywidgets ``` -------------------------------- ### Install Node.js for JupyterLab Error Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install Node.js using conda with a specific label (gcc7) to resolve a common error when installing the JupyterLab widgets manager. ```bash conda install -c conda-forge/label/gcc7 nodejs ``` -------------------------------- ### Initialize Project Variables Source: https://github.com/maif/shapash/blob/master/shapash/report/base_report.ipynb Initialize variables for directory path, project info file, and configuration dictionary. ```python dir_path = "" project_info_file = "" config = dict() ``` -------------------------------- ### Initialize and Run Custom Backend Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb Use the custom backend within the SmartExplainer workflow. ```python custom_backend = CustomShapBackend(model=model, data=X2_train.iloc[:20], preprocessing=encoder2) xpl = SmartExplainer( model=model, preprocessing=encoder2, backend=custom_backend, features_dict=house_dict ) ``` ```python xpl.compile(x=X2_test.iloc[:15]) ``` ```python xpl.plot.features_importance() ``` -------------------------------- ### Use LIME Backend Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb Select the LIME backend and provide necessary data for background estimation. ```python xpl = SmartExplainer( model=clf, preprocessing=encoder, backend='lime', data=Xtrain, # Specific arg used by our backend (here Lime) features_dict=titan_dict ) ``` ```python xpl.compile(x=Xtest.iloc[:20]) # Computations can take some time, we select only 20 observations ``` -------------------------------- ### Install ipywidgets with Pip Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install the ipywidgets package using pip. This is required for displaying inline graphs in Jupyter environments. ```bash pip install ipywidgets ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl01-Shapash-Viz-using-Shap-contributions.ipynb Import the SmartExplainer class to begin the explainability workflow. ```python from shapash import SmartExplainer ``` -------------------------------- ### Install Plotly and JupyterLab Plotly Extension Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install the plotly library and the jupyterlab-plotly extension within a JupyterLab environment using conda. ```bash conda install -c plotly plotly jupyter labextension install jupyterlab-plotly ``` -------------------------------- ### Install JupyterLab Widgets Manager Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install the JupyterLab extension manager for ipywidgets. This command is used for JupyterLab versions 1 or 2. ```bash jupyter labextension install @jupyter-widgets/jupyterlab-manager ``` -------------------------------- ### Initialize ProjectReport Source: https://github.com/maif/shapash/blob/master/tutorial/generate_report/utils/custom_report.ipynb Load the SmartExplainer and training data to instantiate the ProjectReport object. ```python import os import pandas as pd from shapash import SmartExplainer from shapash.report.project_report import ProjectReport from shapash.report.common import load_saved_df xpl = SmartExplainer.load(os.path.join(dir_path, 'smart_explainer.pickle')) x_train = load_saved_df(os.path.join(dir_path, 'x_train.csv')) y_train = load_saved_df(os.path.join(dir_path, 'y_train.csv')) y_test = load_saved_df(os.path.join(dir_path, 'y_test.csv')) report = ProjectReport( explainer=xpl, project_info_file=project_info_file, x_train=x_train, y_train=y_train, y_test=y_test, config=config ) ``` -------------------------------- ### Initialize SmartExplainer for Lime (num_samples=2000) Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Initializes a SmartExplainer object for Lime explanations, specifying the model and preprocessing steps. This is used to compile and analyze explanations. ```python xpl_lime2000 = SmartExplainer( model=rf, preprocessing=encoder ) ``` -------------------------------- ### Instantiate a LimeBackend Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb Use a specific backend class by instantiating it before passing it to the SmartExplainer. ```python from shapash.backend import LimeBackend lime_backend = LimeBackend(model=clf, data=Xtrain) xpl = SmartExplainer( model=clf, preprocessing=encoder, backend=lime_backend, features_dict=titan_dict ) ``` -------------------------------- ### Initialize SmartExplainer for Lime (num_samples=1000) Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Initializes a SmartExplainer object for Lime explanations with `num_samples=1000`, specifying the model and preprocessing. This is used for analyzing explanations with fewer samples. ```python xpl_lime1000 = SmartExplainer( model=rf, preprocessing=encoder ) ``` -------------------------------- ### Install Jupyter Widgets in Conda Environments Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Install the widgetsnbextension in the Jupyter Notebook server environment and ipywidgets in a specific kernel environment (e.g., py36) when they are in different conda environments. ```bash conda install -n base -c conda-forge widgetsnbextension conda install -n py36 -c conda-forge ipywidgets ``` -------------------------------- ### Load and Prepare Data Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl05-Shapash-using-Fasttreeshap.ipynb Load the house prices dataset and split it into features and target variables. ```python from shapash.data.data_loader import data_loading house_df, house_dict = data_loading('house_prices') ``` ```python y_df=house_df['SalePrice'].to_frame() X_df=house_df[house_df.columns.difference(['SalePrice'])] ``` -------------------------------- ### GET /detail_contributions Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.predictor.md Retrieves the detailed contributions associated with the predicted data. ```APIDOC ## GET /detail_contributions ### Description Associates the calculated contributions with the predicted data (ypred). ### Parameters - **contributions** (object) - Optional - Local contributions or list of contributions. - **use_groups** (bool) - Optional - Whether to compute groups of feature contributions. ### Response - **Returns** (pandas.DataFrame) - A dataset containing ypred and the associated contributions. ``` -------------------------------- ### Instantiate and Compile SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/explainability_quality/tuto-quality01-Builing-confidence-explainability.ipynb Sets up the SmartExplainer to evaluate model explanation compacity. ```python from shapash import SmartExplainer ``` ```python response_dict = {0: 'Death', 1:' Survival'} ``` ```python xpl = SmartExplainer( model=clf, preprocessing=encoder, features_dict=titanic_dict, # Optional parameters label_dict=response_dict # Optional parameters, dicts specify labels ) ``` ```python xpl.compile(x=Xtrain) ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/generate_report/tuto-shapash-report01.ipynb Configure the SmartExplainer instance with the model and optional preprocessing. ```python xpl = SmartExplainer( model=regressor, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=house_dict # optional parameter, specifies label for features name ) ``` -------------------------------- ### Load Sample Datasets Source: https://context7.com/maif/shapash/llms.txt Use the data_loading utility to retrieve built-in datasets for testing. ```python from shapash.data.data_loader import data_loading # Load house prices dataset with feature labels dictionary house_df, house_dict = data_loading('house_prices') # Load titanic dataset with feature labels titanic_df, titanic_dict = data_loading('titanic') # Load telco customer churn dataset (no dictionary available) telco_df = data_loading('telco_customer_churn') ``` -------------------------------- ### Compare Individual Contributions Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Example of using compare_plot to visualize contributions for specific rows. ```pycon >>> xpl.plot.compare_plot(row_num=[0, 1, 2]) ``` -------------------------------- ### Prepare and Add Input Data Source: https://github.com/maif/shapash/blob/master/tutorial/common/tuto-common01-groups_of_features.ipynb Create a sample input dataframe and register it with the SmartPredictor object. ```python sample_input = house_df.sample(4).drop('SalePrice', axis=1) sample_input ``` ```python predictor.add_input(sample_input) ``` -------------------------------- ### Update Shapash Dependencies Source: https://github.com/maif/shapash/blob/master/docs/installation-instructions/index.md Command to install specific library versions compatible with Shapash using extras. ```bash pip install shapash[xgboost] ``` -------------------------------- ### Initialize SmartPredictor Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.predictor.md Demonstrates instantiation of the SmartPredictor object using explicit parameters or by converting an existing SmartExplainer instance. ```pycon >>> predictor = SmartPredictor(features_dict=my_features_dict, >>> model=my_model, >>> backend=my_backend, >>> columns_dict=my_columns_dict, >>> features_types=my_features_type_dict, >>> label_dict=my_label_dict, >>> preprocessing=my_preprocess, >>> postprocessing=my_postprocess) ``` ```pycon >>> predictor = xpl.to_smartpredictor() ``` -------------------------------- ### Initialize and Compile SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb Configure the SmartExplainer with the model and encoders, then compile it. ```python xpl = SmartExplainer(model=clf, preprocessing=encoder, features_dict=titan_dict) ``` ```python xpl.compile(x=Xtest, y_target=ytest, # Optional: allows to display True Values vs Predicted Values ) ``` -------------------------------- ### Define feature groups Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Example dictionary structure for aggregating multiple features into named groups for analysis. ```python { … ‘feature_group_1’: [‘feature3’, ‘feature7’, ‘feature24’], … ‘feature_group_2’: [‘feature1’, ‘feature12’] … } ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/use_encoders/tuto-encoder03-using-dict.ipynb Create an instance of SmartExplainer with the model and preprocessing encoder. ```python xpl = SmartExplainer(model=clf, preprocessing=encoder) ``` -------------------------------- ### Initialize and compile Shapash SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/tutorial01-Shapash-Overview-Launch-WebApp.ipynb Initialize the SmartExplainer with the trained regressor, the encoder for preprocessing, and the features dictionary. Then, compile the explainer with the test data and target values. ```python from shapash import SmartExplainer xpl = SmartExplainer( model=regressor, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=house_dict # optional parameter, specifies label for features name ) xpl.compile(x=Xtest, y_target=ytest # Optional: allows to display True Values vs Predicted Values ) ``` -------------------------------- ### Initialize and Run Shapash SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl05-Shapash-using-Fasttreeshap.ipynb Compile the SmartExplainer with FastTreeSHAP contributions and launch the web application. ```python from shapash import SmartExplainer ``` ```python xpl = SmartExplainer( model=regressor, preprocessing=encoder, features_dict=house_dict ) ``` ```python xpl.compile( contributions=shap_contributions.reset_index(drop=True), # FastTreeSHAP Contribution pd.DataFrame y_target=ytest.reset_index(drop=True), # Optional: allows to display True Values vs Predicted Values x=Xtest.reset_index(drop=True) ) ``` ```python app = xpl.run_app(title_story='House Prices Fasttreeshap', port=8020) ``` ```python app.kill() ``` -------------------------------- ### Get detailed feature contributions Source: https://github.com/maif/shapash/blob/master/tutorial/predictor_to_production/tuto-smartpredictor-introduction-to-SmartPredictor.ipynb Retrieves the contribution of each feature for the current prediction, automatically mapped to the predicted label. ```python detailed_contributions = predictor_load.detail_contributions() ``` ```python detailed_contributions ``` -------------------------------- ### Define postprocessing modification types Source: https://github.com/maif/shapash/blob/master/tutorial/postprocess/tuto-postprocess01.ipynb Examples of the five available modification types: prefix, suffix, transcoding, regex, and case. ```python {'features_name': {'type': 'prefix', 'rule': 'Example : '} } ``` ```python {'features_name': {'type': 'suffix', 'rule': ' is an example'} } ``` ```python {'features_name': {'type': 'transcoding', 'rule': {'old_name1': 'new_name1', 'old_name2': 'new_name2', ... } } } ``` ```python {'features_name': {'type': 'regex', 'rule': {'in': '^M', 'out': 'm' } } } ``` ```python {'features_name': {'type': 'case', 'rule': 'upper'} ``` -------------------------------- ### Compile the Explainer Source: https://github.com/maif/shapash/blob/master/tutorial/postprocess/tuto-postprocess01.ipynb Initializes the explainer with test data and model predictions to prepare for analysis. ```python xpl.compile(x=Xtest, y_pred=y_pred) ``` -------------------------------- ### Compile SmartExplainer with Lime Contributions (num_samples=2000) Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Compiles the SmartExplainer with pre-computed Lime contributions and test data. This step prepares the explainer for plotting and analysis. ```python xpl_lime2000.compile( contributions=Lime2000, x=Xtest[0:100] ) ``` -------------------------------- ### Get Detailed Contributions Source: https://github.com/maif/shapash/blob/master/tutorial/tutorial03-Shapash-overview-model-in-production.ipynb Retrieves detailed contributions of each feature for each row of the dataset. For classification, it automatically associates contributions with the predicted label, or you can specify it. ```APIDOC ## GET /maif/shapash/detail_contributions ### Description Retrieves detailed contributions of each feature for each row of the dataset. For classification problems, it automatically associates contributions with the right predicted label. The predicted label can be computed automatically or specified. ### Method GET ### Endpoint /maif/shapash/detail_contributions ### Parameters #### Query Parameters - **ypred** (array) - Optional - The predicted labels to associate contributions with. ### Request Example ```python # Assuming predictor_load is an initialized Shapash predictor object # detailed_contributions = predictor_load.detail_contributions() ``` ### Response #### Success Response (200) - **detailed_contributions** (DataFrame) - A pandas DataFrame containing the detailed contributions of each feature for each row. #### Response Example ```json { "SalePrice": 208500, "1stFlrSF": -1104.994176, "2ndFlrSF": 1281.445856, "3SsnPorch": 0.0, "BedroomAbvGr": 375.679661, "BldgType": 12.259902, "BsmtCond": 157.224629, "BsmtExposure": -233.025420, "BsmtFinSF1": -738.445396, "BsmtFinSF2": -59.294761, "SaleType": -104.645827, "ScreenPorch": -351.621116, "Street": 0.0, "TotRmsAbvGrd": -498.228775, "TotalBsmtSF": -5165.503476, "Utilities": 0.0, "WoodDeckSF": -944.040092, "YearBuilt": 3870.961681, "YearRemodAdd": 2219.313761, "YrSold": 17.478037 } ``` ``` -------------------------------- ### Deploy with SmartPredictor Source: https://context7.com/maif/shapash/llms.txt Convert an explainer to a lightweight predictor for production, supporting input modification and summary generation. ```python # Convert SmartExplainer to SmartPredictor predictor = xpl.to_smartpredictor() # Add new data for prediction predictor.add_input(x=new_data_df) # Get predictions predictions = predictor.predict() # Get prediction probabilities (classification) probabilities = predictor.predict_proba() # Get detailed contributions contributions_df = predictor.detail_contributions() # Get summarized explanations summary_df = predictor.summarize() # Modify summary parameters predictor.modify_mask( features_to_hide=['Age'], threshold=0.05, positive=None, max_contrib=3 ) updated_summary = predictor.summarize() # Save predictor for production predictor.save('predictor.pkl') ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/tutorial02-Shapash-overview-in-Jupyter.ipynb Initialize the SmartExplainer with a trained model and optional preprocessing steps and feature dictionaries. ```python xpl = SmartExplainer( model=regressor, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=house_dict # Optional parameter, dict specifies label for features name ) ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/tutorial03-Shapash-overview-model-in-production.ipynb Import and configure the SmartExplainer object with the trained model and feature dictionary. ```python from shapash import SmartExplainer ``` ```python xpl = SmartExplainer( model=regressor, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=house_dict ) ``` -------------------------------- ### Check code quality with flake8 Source: https://github.com/maif/shapash/blob/master/CONTRIBUTING.md Use flake8 to check your code for style guide violations and potential issues. This command checks the current directory. ```bash flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics ``` -------------------------------- ### Initialize and Compile SmartExplainer Source: https://github.com/maif/shapash/blob/master/tests/integration_tests/backend_integration.ipynb Manually configure a SmartExplainer instance with model contributions and preprocessing steps. ```python from xplainable.explainer.smart_explainer import SmartExplainer ``` ```python xpl2 = SmartExplainer() ``` ```python contributions , bias = compute_contributions(x=X_test, explainer=explainer) ``` ```python X_encode = preprocessing.fit_transform(X) ``` ```python xpl2.compile(contributions=contributions, preprocessing=preprocessing, x_init=X_test, model=model, y_pred=pd.DataFrame(y_pred,columns=['target'],index=X_test.index)) ``` -------------------------------- ### Initialize Shapash SmartExplainer with Lime Backend Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Initializes the Shapash SmartExplainer using the trained Random Forest model and the Lime backend. It uses a subset of the test data for explanation. ```python xpl_lime = SmartExplainer( model=rf, backend='lime', data=Xtest[0:100], preprocessing=encoder, ) ``` -------------------------------- ### Initialize SmartExplainer Source: https://context7.com/maif/shapash/llms.txt Create a SmartExplainer instance by providing the trained model, preprocessing steps, and optional dictionaries for feature and label mapping. ```python from shapash import SmartExplainer from sklearn.ensemble import RandomForestRegressor from category_encoders import OrdinalEncoder import pandas as pd # Train a model encoder = OrdinalEncoder().fit(X_train) X_train_encoded = encoder.transform(X_train) model = RandomForestRegressor(n_estimators=100).fit(X_train_encoded, y_train) # Create SmartExplainer with preprocessing and feature labels xpl = SmartExplainer( model=model, backend='shap', # or 'lime' preprocessing=encoder, features_dict={ 'MSSubClass': 'Building Class', 'LotArea': 'Lot Size', 'OverallQual': 'Overall Quality' }, label_dict={0: 'Low Price', 1: 'High Price'}, # for classification title_story='House Price Prediction Model' ) ``` -------------------------------- ### Compute Local Lime Explanations (num_samples=1000) Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Computes local Lime explanations with a reduced `num_samples` parameter to assess the trade-off between computation time and explanation quality. This is a faster alternative to `num_samples=2000` or `5000`. ```python # Compute local Lime Explanation for each row in Test Sample contrib_1000=[] for ind in Xtest[0:100].index: exp = explainer.explain_instance(Xtest[0:100].loc[ind].values, rf.predict_proba, num_features=Xtest[0:100].shape[1],num_samples=1000) contrib_1000.append(dict([[features_check(elem[0]),elem[1]] for elem in exp.as_list()])) ``` -------------------------------- ### Initialize and compile SmartExplainer with postprocessing Source: https://github.com/maif/shapash/blob/master/tutorial/postprocess/tuto-postprocess01.ipynb Integrating the postprocessing dictionary into the SmartExplainer object and compiling it. ```python xpl_postprocess = SmartExplainer( model=classifier, postprocessing=postprocess, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=titanic_dict ) ``` ```python xpl_postprocess.compile( x=Xtest, y_pred=y_pred, # Optional ) ``` -------------------------------- ### Load and Prepare House Prices Data Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb Load dataset and prepare features for regression modeling. ```python house_df, house_dict = data_loading('house_prices') ``` ```python y2 = house_df['SalePrice'].to_frame() X2 = house_df.drop(columns='SalePrice') ``` ```python # Encode categorical features categorical_features = [col for col in X2.columns if X2[col].dtype == 'object'] encoder2 = OrdinalEncoder( cols=categorical_features, handle_unknown='ignore', return_df=True ).fit(X2) X2 = encoder2.transform(X2) ``` ```python X2_train, X2_test, y2_train, y2_test = train_test_split(X2, y2, train_size=0.75, random_state=1) ``` ```python # Train a model model = RandomForestRegressor().fit(X2_train, y2_train) ``` -------------------------------- ### Build Shapash wheel Source: https://github.com/maif/shapash/blob/master/CONTRIBUTING.md Attempt to build a wheel for Shapash to ensure the project can be packaged correctly. ```bash python setup.py bdist_wheel ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/use_encoders/tuto-encoder01-using-category_encoder.ipynb Imports and initializes the SmartExplainer with the trained model and preprocessing encoders. ```python from shapash import SmartExplainer ``` ```python xpl = SmartExplainer( model=clf, preprocessing=encoder, ) ``` -------------------------------- ### Initialize SmartExplainer with model and preprocessing details Source: https://github.com/maif/shapash/blob/master/tutorial/predictor_to_production/tuto-smartpredictor-introduction-to-SmartPredictor.ipynb Initializes the SmartExplainer object with the trained model, preprocessing steps, and custom dictionaries for feature and label renaming. ```python from shapash import SmartExplainer feature_dict = { 'Pclass': 'Ticket class', 'Sex': 'Sex', 'Age': 'Age', 'SibSp': 'Relatives such as brother or wife', 'Parch': 'Relatives like children or parents', 'Fare': 'Passenger fare', 'Embarked': 'Port of embarkation', 'Title': 'Title of passenger' } label_dict = {0: "Not Survived", 1: "Survived"} postprocessing = {"Pclass": {'type': 'transcoding', 'rule': { 'First class': '1st class', 'Second class': '2nd class', "Third class": "3rd class"}}} xpl = SmartExplainer( model=rf, preprocessing=categ_encoding, postprocessing=postprocessing, label_dict=label_dict, features_dict=feature_dict ) ``` -------------------------------- ### Initialize and Compile SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/use_encoders/tuto-encoder02-using-columntransformer.ipynb Initialize the Shapash SmartExplainer with the model and preprocessor, then compile it with test data. ```python from shapash import SmartExplainer ``` ```python xpl = SmartExplainer(model=clf, preprocessing=enc_columntransfo) ``` ```python xpl.compile(x=Xtest, y_target=ytest, # Optional: allows to display True Values vs Predicted Values ) ``` -------------------------------- ### Initialize SmartExplainer for Regression Source: https://github.com/maif/shapash/blob/master/tutorial/plots_and_charts/tuto-plot04-compare_plot.ipynb Instantiate SmartExplainer with a regression model, optional preprocessing, and a features dictionary for custom labels. ```python xpl = SmartExplainer( model=regressor, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=house_dict # Optional parameter, dict specifies label for features name ) ``` -------------------------------- ### Load and prepare dataset Source: https://github.com/maif/shapash/blob/master/tutorial/generate_report/tuto-shapash-report01.ipynb Load the house prices dataset and separate the target variable from features. ```python from shapash.data.data_loader import data_loading house_df, house_dict = data_loading('house_prices') y_df=house_df['SalePrice'] X_df=house_df[house_df.columns.difference(['SalePrice'])] ``` -------------------------------- ### Display explanation object Source: https://github.com/maif/shapash/blob/master/tutorial/predictor_to_production/tuto-smartpredictor-introduction-to-SmartPredictor.ipynb Outputs the explanation dictionary or dataframe. ```python explanation ``` -------------------------------- ### Import Consistency and SHAP Source: https://github.com/maif/shapash/blob/master/tutorial/explainability_quality/tuto-quality01-Builing-confidence-explainability.ipynb Import the Consistency class and the SHAP library for explainability analysis. ```python from shapash.explainer.consistency import Consistency import shap ``` -------------------------------- ### Define a Custom Backend Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl06-Shapash-custom-backend.ipynb Implement a custom backend by inheriting from BaseBackend and defining the run_explainer method. ```python import shap import logging from shapash.backend import BaseBackend logging.getLogger('shap').setLevel(logging.WARNING) # Disable info logs class CustomShapBackend(BaseBackend): def __init__(self, model, data, preprocessing): super(CustomShapBackend, self).__init__(model, preprocessing) self.explainer = shap.KernelExplainer(model=model.predict, data=data) def run_explainer(self, x): shap_contributions = self.explainer.shap_values(x) explain_data = dict(contributions=shap_contributions) return explain_data ``` -------------------------------- ### Compile Shapash Explainability Model Source: https://github.com/maif/shapash/blob/master/tutorial/plots_and_charts/tuto-plot07-additional_plots_visualizations.ipynb Initialize SmartExplainer with the trained model, preprocessing steps, and feature/label dictionaries. Compile the explainer with test data to prepare for model prediction analysis. ```python # Mapping encoded labels to original labels mappings = encoder_target.mapping[0] encoded_to_original = {v: k for k, v in mappings['mapping'].items()} encoded_to_original.pop(-2) # Remove invalid mappings # Initialize SmartExplainer xpl = SmartExplainer(model=clf, preprocessing=encoder, features_dict=titanic_dict, label_dict=encoded_to_original) # Compile explainability model xpl.compile(x=Xtest, y_target=ytest.iloc[:, 0]) ``` -------------------------------- ### Compile SmartExplainer with Lime Contributions (num_samples=1000) Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Compiles the SmartExplainer with Lime contributions computed using `num_samples=1000` and the test data. This prepares the explainer for stability analysis. ```python xpl_lime1000.compile( contributions=Lime1000, x=Xtest[0:100], ) ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tutorial/plots_and_charts/tuto-plot05-interactions-plot.ipynb Initialize the Shapash SmartExplainer with the trained model, preprocessing steps, and feature/label dictionaries for enhanced interpretability. ```python from shapash import SmartExplainer response_dict = {0: 'Death', 1:' Survival'} xpl = SmartExplainer( model=clf, preprocessing=encoder, # Optional: compile step can use inverse_transform method features_dict=titanic_dict, # Optional parameters label_dict=response_dict # Optional parameters, dicts specify labels ) ``` -------------------------------- ### Compile SmartExplainer Source: https://context7.com/maif/shapash/llms.txt Prepare the explainer for visualization by computing contributions using the test dataset. ```python # Basic compilation with test data X_test_encoded = encoder.transform(X_test) xpl.compile(x=X_test_encoded) # Full compilation with all optional parameters xpl.compile( x=X_test_encoded, y_pred=model.predict(X_test_encoded), # Custom predictions y_target=y_test, # True target values for comparison contributions=None, # Provide pre-computed contributions or let backend compute additional_data=X_test[['extra_feature']], # Extra features for webapp filtering additional_features_dict={'extra_feature': 'Extra Feature Label'}, columns_order=['OverallQual', 'LotArea', 'MSSubClass'] # Display order ) ``` -------------------------------- ### Import Libraries for Data Loading and Modeling Source: https://github.com/maif/shapash/blob/master/tutorial/plots_and_charts/tuto-plot04-compare_plot.ipynb Import necessary libraries for data manipulation, model training, and data splitting. These are foundational for the subsequent steps in the tutorial. ```python import pandas as pd from catboost import CatBoostRegressor from sklearn.model_selection import train_test_split ``` -------------------------------- ### Load and Prepare Data Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl04-Shapash-compute-Lime-faster.ipynb Loads the 'telco_customer_churn' dataset and performs initial data cleaning and preprocessing, including encoding categorical features. ```python from shapash.data.data_loader import data_loading ``` ```python df = data_loading('telco_customer_churn') ``` ```python df = df.reset_index().drop('customerID', axis=1) ``` ```python df['Churn'].replace('No', 0,inplace=True) df['Churn'].replace('Yes', 1,inplace=True) ``` ```python y_df = df['Churn'] X_df = df.drop('Churn', axis=1) ``` ```python categorical_features = [col for col in X_df.columns if X_df[col].dtype == 'object'] encoder = OrdinalEncoder( cols=categorical_features, handle_unknown='ignore', return_df=True).fit(X_df) X_df=encoder.transform(X_df) ``` -------------------------------- ### SmartExplainer.run_app Source: https://github.com/maif/shapash/blob/master/docs/autodocs/shapash.explainer.md Launches the Shapash interpretability WebApp associated with the SmartExplainer instance. ```APIDOC ## SmartExplainer.run_app ### Description Launch the Shapash interpretability WebApp associated with this SmartExplainer. This method starts the interactive WebApp that enables users to explore model predictions and explanations. ### Parameters #### Query Parameters - **port** (int) - Optional - Port number for the WebApp server. Defaults to 8050. - **host** (str) - Optional - Host address for the WebApp server. Defaults to 0.0.0.0. - **title_story** (str) - Optional - Custom title to display in the WebApp interface. - **settings** (dict) - Optional - Dictionary specifying default configuration values (e.g., 'rows', 'points', 'violin', 'features'). ### Response - **Returns** (CustomThread) - A thread instance running the WebApp server. ### Errors - **ValueError** - If the SmartExplainer has not been compiled before launching the app. ``` -------------------------------- ### Initialize SmartExplainer Source: https://github.com/maif/shapash/blob/master/tests/integration_tests/backend_integration.ipynb Import and instantiate `SmartExplainer` from `xplainable.explainer.smart_explainer`. This object is central to Shapash's model explanation capabilities. ```python import sys sys.path.append('../..') from xplainable.explainer.smart_explainer import SmartExplainer ``` ```python xpl = SmartExplainer() ``` -------------------------------- ### Import necessary libraries Source: https://github.com/maif/shapash/blob/master/tutorial/generate_webapp/tuto-webapp01-additional-data.ipynb Import pandas, train_test_split from sklearn, catboost, and OrdinalEncoder for data manipulation and model building. ```python import pandas as pd from sklearn.model_selection import train_test_split import catboost from category_encoders import OrdinalEncoder ``` -------------------------------- ### Import necessary libraries Source: https://github.com/maif/shapash/blob/master/tutorial/explainability_quality/tuto-quality01-Builing-confidence-explainability.ipynb Initial imports for data manipulation and model training. ```python import pandas as pd from sklearn.ensemble import ExtraTreesClassifier from sklearn.model_selection import train_test_split ``` -------------------------------- ### Initialize Lime Tabular Explainer Source: https://github.com/maif/shapash/blob/master/tutorial/explainer_and_backend/tuto-expl02-Shapash-Viz-using-Lime-contributions.ipynb Initializes a LimeTabularExplainer for classification tasks, using the training data, feature names, and class names. ```python #Training Tabular Explainer explainer = lime.lime_tabular.LimeTabularExplainer(Xtrain.values, mode='classification', feature_names=Xtrain.columns, class_names=ytrain) ``` -------------------------------- ### Initialize LightGBM Classifier Source: https://github.com/maif/shapash/blob/master/tests/integration_tests/backend_integration.ipynb Initializes a LightGBM classifier model. ```python model = LGBMClassifier() ``` -------------------------------- ### Import necessary libraries Source: https://github.com/maif/shapash/blob/master/tutorial/tutorial01-Shapash-Overview-Launch-WebApp.ipynb Import pandas, category_encoders, lightgbm, and sklearn modules for data manipulation, encoding, and model training. ```python import pandas as pd from category_encoders import OrdinalEncoder from lightgbm import LGBMRegressor from sklearn.model_selection import train_test_split from sklearn.ensemble import ExtraTreesRegressor ```