### Install and Develop with setup.py Arguments Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Demonstrates how to use standard setup.py arguments for installation and development. These commands allow for custom installation procedures, such as recording installed files or skipping dependency checks. ```bash python setup.py install --single-version-externally-managed --record=record.txt python setup.py develop --no-deps ``` -------------------------------- ### Install scikit-learn-intelex using Conda-Forge (Fresh Environment) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md Installs scikit-learn-intelex in a new conda environment using the conda-forge channel. This provides an alternative installation method for users who prefer or rely on conda-forge. ```default conda create -n sklex -c conda-forge --override-channels scikit-learn-intelex conda activate sklex ``` -------------------------------- ### Build scikit-learn-intelex using setup.py Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Demonstrates common commands for installing and developing the scikit-learn-intelex library using `setup.py`. This includes standard installation, development mode installation, and in-place building of extensions. ```bash python setup.py install ``` ```bash python setup.py develop ``` ```bash python setup.py build_ext --inplace --force python setup.py build ``` -------------------------------- ### Install scikit-learn-intelex using Intel Channel (Fresh Environment) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md Installs scikit-learn-intelex in a new conda environment using Intel's channel and conda-forge as a fallback. This is recommended for IntelĀ® Distribution for Python users. ```default conda create -n sklex -c https://software.repos.intel.com/python/conda/ -c conda-forge --override-channels scikit-learn-intelex conda activate sklex ``` -------------------------------- ### Install scikit-learn-intelex using Intel Channel (Existing Environment) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md Installs or updates scikit-learn-intelex in an existing conda environment using Intel's channel and conda-forge. This method ensures access to the latest versions and GPU builds. ```default conda install -c https://software.repos.intel.com/python/conda/ -c conda-forge --override-channels scikit-learn-intelex ``` -------------------------------- ### Install scikit-learn-intelex using Conda-Forge (Existing Environment) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md Installs or updates scikit-learn-intelex in an existing conda environment using the conda-forge channel. This is a straightforward method for users preferring conda-forge. ```default conda install -c conda-forge --override-channels scikit-learn-intelex ``` -------------------------------- ### Install scikit-learn-intelex via pip Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/README.md Installs the scikit-learn-intelex package using pip, enabling optimizations for scikit-learn workflows. This is a common method for setting up the extension. Other installation channels like conda-forge are also available. ```shell pip install scikit-learn-intelex ``` -------------------------------- ### Install Documentation Dependencies (Bash) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/contributor-reference.md Installs the necessary Python packages for building the project documentation locally. This command should be executed from the root of the repository. ```bash pip install -r requirements-doc.txt ``` -------------------------------- ### Install conda-build and conda-verify Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Command to install the necessary conda packages, `conda-build` and `conda-verify`, which are required for building the library using `conda-build`. ```bash conda install -c conda-forge conda-build conda-verify ``` -------------------------------- ### Install Pre-commit Hooks Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/contribute.md This snippet demonstrates how to install pre-commit hooks to automatically format your Python code according to project standards before committing changes. ```bash pip install pre-commit pre-commit install ``` -------------------------------- ### Install Optional Test Dependencies (GPU and Array API) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/tests.md Installs optional dependencies for GPU functionalities (dpctl, dpnp) and array API support (torch). These installations may require specific index URLs. Note potential compatibility issues between torch and dpctl/dpnp. ```bash pip install \ dpctl `# for GPU functionalities` \ dpnp `# for array API and GPU functionalities` pip install --index-url https://software.repos.intel.com/python/pypi \ torch `# for array API` ``` -------------------------------- ### Build Documentation Locally (Makefile) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/contributor-reference.md Builds the project documentation locally using the Makefile after the initial setup script has been run. This is useful for incremental builds or when no new notebooks/examples have been added. Execute from the 'doc' directory. ```bash cd doc make clean make html ``` -------------------------------- ### Install Python Dependencies for Scikit-learn-intelex (Pip) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Installs core Python dependencies required for building the scikit-learn-intelex extension using pip. This includes numpy, cython, jinja2, pybind11, and a specific version of setuptools. ```bash pip install numpy cython jinja2 pybind11 "setuptools<=79" ``` -------------------------------- ### Install impi-rt with conda-forge Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed-mode.md Installs the Intel MPI runtime library (impi_rt) from the conda-forge channel, explicitly setting the MPI backend to Intel's MPI (impi). This command is used when impi_rt needs to be installed separately. ```bash conda install -c conda-forge impi_rt mpi=*=impi ``` -------------------------------- ### Install Test Dependencies with Pip Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/tests.md Installs mandatory dependencies for running tests from the requirements-test.txt file. This is a prerequisite for executing the test suite. ```bash pip install -r requirements-test.txt ``` -------------------------------- ### Install DPC++ Runtime from PyPI Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/oneapi-gpu.md Installs the DPC++ C++ runtime library from the Python Package Index (PyPI). This is a dependency for enabling GPU computations with IntelĀ® Extension for Scikit-learn*. ```bash pip install dpcpp-cpp-rt ``` -------------------------------- ### Build and Run with ASan Enabled Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md A comprehensive example demonstrating the full build and execution process with ASan enabled. It includes sourcing an environment script, building the extension in-place, and running a Python script with ASan runtime preloading and Python memory allocation configured. ```bash source CC="ccache icx -fsanitize=address -g" CXX="ccache icpx -fsanitize=address -g" \ python setup.py build_ext --inplace --force --abs-rpath CC="icx -fsanitize=address -g" CXX="icpx -fsanitize=address -g" \ python setup.py build --abs-rpath LD_PRELOAD="$(clang -print-file-name=libclang_rt.asan-x86_64.so)" \ PYTHONMALLOC=malloc PYTHONPATH=$(pwd) \ python ``` -------------------------------- ### Patch Selected Scikit-learn Algorithms Globally Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md Use the `--algorithm` or `-a` flags to patch specific Scikit-learn estimators globally. This example patches only SVC and RandomForestClassifier. This command-line method requires write permissions for Scikit-learn files. ```bash python -m sklearnex.glob patch_sklearn -a svc random_forest_classifier ``` -------------------------------- ### Install impi-rt with pip Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed-mode.md Installs the Intel MPI runtime library (impi-rt) using pip. This command is used when the MPI runtime is needed separately from mpi4py and the environment primarily uses pip for package management. ```bash pip install impi-rt ``` -------------------------------- ### Install Optional Test Dependencies (Distributed Mode) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/tests.md Installs optional dependencies for distributed mode, including mpi4py and impi_rt from Intel's index, and pytest-mpi from the default index. Ensure mpi4py is installed from the Intel index for MPI backend compatibility. ```bash pip install --index-url https://software.repos.intel.com/python/pypi \ mpi4py impi_rt `# for distributed mode, be sure to install from Intel's index` \ && pip install pytest-mpi `# also required, but not from Intel's index` ``` -------------------------------- ### Build scikit-learn-intelex using conda-build Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Command to build the conda package for the scikit-learn-intelex library from the root of the repository after installing `conda-build`. ```bash conda build . ``` -------------------------------- ### Use daal4py QR Algorithm Example Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/about_daal4py.md A complete example showcasing the usage of the daal4py QR algorithm. It generates random data, instantiates the QR algorithm, computes the result, and verifies the output against NumPy's linalg.qr. This illustrates the compute method and result access pattern. ```python import daal4py import numpy as np rng = np.random.default_rng(seed=123) X = rng.standard_normal(size=(100,5)) qr_algo = daal4py.qr() qr_result = qr_algo.compute(X) np.testing.assert_almost_equal( np.abs( qr_result.matrixR ), np.abs( np.linalg.qr(X).R ), ) ``` -------------------------------- ### Install mpi4py and impi-rt with pip Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed-mode.md Installs the mpi4py package and the Intel MPI runtime library (impi-rt) using pip from Intel's Python Package Index. This is an alternative to conda installation for environments using pip. ```bash pip install --index-url https://software.repos.intel.com/python/pypi mpi4py impi-rt ``` -------------------------------- ### Install mpi4py with Intel MPI (conda-forge) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed-mode.md Installs the mpi4py package and sets the MPI backend to Intel's MPI (impi) from the conda-forge channel. This ensures compatibility with the Scikit-learn Extension when using Intel MPI. ```bash conda install -c conda-forge mpi4py mpi=*=impi ``` -------------------------------- ### Install DPC++ Runtime using Conda (Intel Repository) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/oneapi-gpu.md Installs the DPC++ C++ runtime library using Conda from Intel's official software repository. This is an alternative method for setting up the necessary runtime for GPU acceleration. ```bash conda install -c https://software.repos.intel.com/python/conda/ dpcpp_cpp_rt ``` -------------------------------- ### Install Python Dependencies for Scikit-learn-intelex (Conda) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Installs core Python dependencies required for building the scikit-learn-intelex extension using the conda package manager. This includes numpy, cython, jinja2, pybind11, and a specific version of setuptools. ```bash conda install -c conda-forge numpy cython jinja2 pybind11 "setuptools<=79" ``` -------------------------------- ### Install mpi4py with Intel MPI (Intel's conda channel) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed-mode.md Installs mpi4py and configures it to use Intel's MPI backend, prioritizing Intel's conda channel for compatibility. This command also overrides channels to ensure proper dependency resolution. ```bash conda install -c https://software.repos.intel.com/python/conda/ -c conda-forge --override-channels mpi4py mpi=*=impi ``` -------------------------------- ### Install impi-rt with Intel's conda channel Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed-mode.md Installs the Intel MPI runtime library (impi_rt) from Intel's conda channel, ensuring compatibility with other Intel-provided packages. This command also overrides channels to manage dependencies effectively. ```bash conda install -c https://software.repos.intel.com/python/conda/ -c conda-forge --override-channels impi_rt mpi=*=impi ``` -------------------------------- ### Global and Local Configuration Management with sklearnex Source: https://context7.com/uxlfoundation/scikit-learn-intelex/llms.txt Demonstrates how to get, set, and temporarily change configuration settings for scikit-learn acceleration using sklearnex. Requires sklearnex. ```python from sklearnex import get_config, set_config, config_context # Get current configuration config = get_config() print(f"Current config: {config}") # Set global configuration set_config( target_offload="auto", allow_fallback_to_host=True, allow_sklearn_after_onedal=True ) # Temporary configuration change with config_context(target_offload="gpu:0", allow_fallback_to_host=False): # GPU-only computations here config_inside = get_config() print(f"Config inside context: {config_inside['target_offload']}") # Configuration reverts outside context config_outside = get_config() print(f"Config outside context: {config_outside['target_offload']}") ``` -------------------------------- ### Install Non-Python Dependencies for Scikit-learn-intelex (Windows) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Installs essential non-Python dependencies for building scikit-learn-intelex on Windows using conda. This includes build tools like cmake, clang-format, cxx-compiler, and optional dependencies for GPU (dpcpp) and distributed (impi) modes. ```bash conda install -c conda-forge ^ cmake clang-format cxx-compiler ^ dpcpp-cpp-rt dpcpp_win-64 ^ impi-devel impi_rt ``` -------------------------------- ### Load NYC Bike Share Dataset Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/examples/notebooks/ridge_regression_bike_sharing.ipynb Loads the New York City Bike Share dataset using fetch_openml. The dataset is loaded as a pandas DataFrame for easier manipulation. ```python dataset = fetch_openml(data_id=43526, as_frame=True) ``` -------------------------------- ### Install Non-Python Dependencies for Scikit-learn-intelex (Linux) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Installs essential non-Python dependencies for building scikit-learn-intelex on Linux using conda. This includes build tools like cmake, clang-format, cxx-compiler, and optional dependencies for GPU (dpcpp) and distributed (impi) modes. ```bash conda install -c conda-forge \ cmake clang-format cxx-compiler `# mandatory dependencies` \ dpcpp-cpp-rt dpcpp_linux-64 `# required for GPU mode` \ impi-devel impi_rt `# required for distributed mode` ``` -------------------------------- ### Install DPC++ Runtime using Conda (conda-forge) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/oneapi-gpu.md Installs the DPC++ C++ runtime library using Conda from the conda-forge channel. This provides another option for managing the DPC++ runtime dependency. ```bash conda install -c conda-forge dpcpp_cpp_rt ``` -------------------------------- ### Verbose Output Example for DBSCAN Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/verbose.md These examples demonstrate the console output when verbose mode is enabled and the DBSCAN algorithm is called. The messages indicate whether the accelerated version provided by the Intel Extension for Scikit-learn* is running on CPU or if it falls back to the original Scikit-learn implementation. This output is specific to supported algorithms. ```text INFO:sklearnex: sklearn.cluster.DBSCAN.fit: running accelerated version on CPU ``` ```text INFO:sklearnex: sklearn.cluster.DBSCAN.fit: fallback to original Scikit-learn ``` -------------------------------- ### Import Libraries for Scikit-learn Intelex Example Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/examples/notebooks/nusvr_medical_charges.ipynb Imports necessary libraries for data manipulation, model training, and performance timing. Includes scikit-learn, scikit-learn-intelex, and IPython for display. ```python from timeit import default_timer as timer from sklearn.datasets import fetch_openml from sklearn.model_selection import train_test_split from IPython.display import HTML import warnings warnings.filterwarnings("ignore") ``` -------------------------------- ### Patching scikit-learn for CPU Acceleration Source: https://context7.com/uxlfoundation/scikit-learn-intelex/llms.txt Enable automatic acceleration of scikit-learn algorithms on CPU by patching the library at runtime. This example demonstrates patching for DBSCAN clustering. ```APIDOC ## Patching scikit-learn for CPU Acceleration ### Description Enable automatic acceleration of scikit-learn algorithms on CPU by patching the library at runtime. ### Method N/A (Runtime Patching) ### Endpoint N/A (Runtime Patching) ### Parameters N/A ### Request Example ```python import numpy as np from sklearnex import patch_sklearn patch_sklearn() from sklearn.cluster import DBSCAN # Data preparation X = np.array([[1., 2.], [2., 2.], [2., 3.], [8., 7.], [8., 8.], [25., 80.]], dtype=np.float32) # Train model - automatically uses accelerated implementation clustering = DBSCAN(eps=3, min_samples=2).fit(X) print(f"Labels: {clustering.labels_}") print(f"Core samples: {clustering.core_sample_indices_}") ``` ### Response #### Success Response (200) N/A (Execution output) #### Response Example ``` Labels: [-1 0 0 -1 -1 -1] Core samples: [1 2 4] ``` ``` -------------------------------- ### Enable Global Scikit-learn Patching via Code Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md Enable global Scikit-learn patching programmatically using the `patch_sklearn` function with `global_patch=True`. This enables patching for the current application and others in the same environment. Ensure `sklearnex` is imported. ```python from sklearnex import patch_sklearn patch_sklearn(global_patch=True) import sklearn ``` -------------------------------- ### Train TSNE with scikit-learn-intelex Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/examples/notebooks/tsne.ipynb Trains the TSNE algorithm using the patched Scikit-learn library for the generated dataset. It measures the execution time and prints the divergence. The `params` dictionary specifies the configuration for the TSNE model. ```python from sklearn.manifold import TSNE params = {"n_components": 2, "random_state": 42} start = timer() tsne = TSNE(**params) embedding_intelex = tsne.fit_transform(x) time_opt = timer() - start print(f"Extension for Scikit-learn time: {time_opt:.2f} s") print(f"Extension for Scikit-learn. Divergence: {tsne.kl_divergence_}") ``` -------------------------------- ### Run Scikit-learn Operations on GPU using target_offload with Device Name Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/oneapi-gpu.md Configures the scikit-learn* Extension to offload operations to a GPU using the 'target_offload' context manager. This example demonstrates passing the device name 'gpu' as a string. ```python from sklearnex import config_context from sklearnex.linear_model import LinearRegression from sklearn.datasets import make_regression X, y = make_regression() model = LinearRegression() with config_context(target_offload="gpu"): model.fit(X, y) pred = model.predict(X) ``` -------------------------------- ### Build with Custom Build Flags using setup.py Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Shows how to use environment variables like `NO_DIST` and `LDFLAGS` to customize the build process when using `setup.py`. This allows for disabling features like distributed mode or specifying linker flags. ```bash NO_DIST=1 python setup.py build_ext --inplace --force --abs-rpath ``` ```bash NO_DIST=1 LDFLAGS="-fuse-ld=lld" python setup.py build --using-lld ``` -------------------------------- ### Unpatch Scikit-learn and Re-import Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/quick-start.md To undo patching, use `sklearnex.unpatch_sklearn()`. After unpatching, it is essential to re-import the necessary Scikit-learn modules to use the original implementations. This example shows re-importing KMeans. ```python sklearnex.unpatch_sklearn() # Re-import scikit-learn algorithms after the unpatch from sklearn.cluster import KMeans ``` -------------------------------- ### Build Documentation Locally (Windows Batch) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/contributor-reference.md Provides instructions for building documentation on Windows using make.bat. Note that some elements might not render correctly if the file copying steps from build-doc.sh were not performed. ```batch make.bat ``` -------------------------------- ### Streaming Mode QR Decomposition with daal4py Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/about_daal4py.md Demonstrates how to perform QR decomposition in streaming mode using daal4py. This mode processes data in batches, which is useful for large datasets that do not fit into memory. The example initializes the QR algorithm with `streaming=True`, computes on data batches, and finalizes to get the result. It then compares the absolute values of the resulting R matrix with NumPy's QR decomposition. ```python import daal4py import numpy as np rng = np.random.default_rng(seed=123) X_full = rng.standard_normal(size=(100,5)) batches = np.split(np.arange(100), 5) qr_algo = daal4py.qr(streaming=True) for batch in batches: X_batch = X_full[batch] qr_algo.compute(X_batch) qr_result = qr_algo.finalize() np.testing.assert_almost_equal( np.abs( qr_result.matrixR ), np.abs( np.linalg.qr(X_full).R ), ) ``` -------------------------------- ### Install GPU OCL ICD Package for Conda Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/oneapi-gpu.md Installs the Intel GPU OCL ICD system package within a conda environment for GPU support. This is in addition to any system-wide installation. ```default conda install -c https://software.repos.intel.com/python/conda/ intel-gpu-ocl-icd-system ``` -------------------------------- ### Build Documentation Locally (Linux Bash) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/contributor-reference.md Builds the project documentation locally on Linux systems. This script copies necessary files and uses Sphinx to generate HTML documentation. It should be run from the root of the repository. ```bash ./doc/build-doc.sh ``` -------------------------------- ### Optimize Build by Omitting Components and Using LLD Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Illustrates how to optimize the build process for local development by disabling DPC and distribution components, using a lower optimization level, and employing the LLD linker. This can speed up compilation when these features are not actively being developed. ```bash NO_DPC=1 NO_DIST=1 CC="ccache icx -O0" CXX="ccache icpx -O0" LDFLAGS="-fuse-ld=lld" \ python setup.py build_ext --inplace --force --abs-rpath --using-lld NO_DPC=1 NO_DIST=1 CC="ccache icx -O0" CXX="ccache icpx -O0" LDFLAGS="-fuse-ld=lld" \ python setup.py build --abs-rpath --using-lld ``` -------------------------------- ### Install Intel OpenCL ICD on Debian Systems Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/oneapi-gpu.md Installs the Intel OpenCL Installable Client Driver (ICD) package on Debian-based Linux distributions. This includes necessary dependencies like intel-compute-runtime and intel-graphics-compiler, required for GPU support. ```bash sudo apt-get install intel-opencl-icd ``` -------------------------------- ### Globally patch scikit-learn installation Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/patching/patching-options.rst Enable patching for your entire scikit-learn installation for all subsequent runs using the global patching command. This method modifies the installation to apply optimizations universally. Execute `python -m sklearnex.glob patch_sklearn` to activate. ```bash python -m sklearnex.glob patch_sklearn ``` -------------------------------- ### Load Versions from Switcher URL Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/_templates/layout.html Fetches version information from a specified URL and loads it. This is typically used for dynamic version selection in documentation. ```javascript fetch("{{ switcher_url }}").then(response => response.text()).then(respText => load_versions(respText)); ``` -------------------------------- ### Use Local Configuration Context for GPU Offloading in scikit-learn-intelex Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/config-contexts.md This example demonstrates how to use a local configuration context manager (`config_context`) to direct computations to the GPU for a specific block of code. It imports necessary libraries, defines sample data, and then applies the `target_offload='gpu'` setting within a `with` statement to affect operations like DBSCAN clustering. ```python import numpy as np from sklearnex import config_context from sklearnex.cluster import DBSCAN X = np.array([[1., 2.], [2., 2.], [2., 3.], [8., 7.], [8., 8.], [25., 80.]], dtype=np.float32) with config_context(target_offload="gpu"): clustering = DBSCAN(eps=3, min_samples=2).fit(X) ``` -------------------------------- ### Import necessary libraries for Scikit-learn-Intelex example Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/examples/notebooks/tsne.ipynb Imports standard Python libraries for numerical operations, plotting, and machine learning, along with specific modules from scikit-learn and scikit-learn-intelex. ```python from timeit import default_timer as timer from sklearn import metrics from sklearn.datasets import make_blobs import matplotlib.pyplot as plt %matplotlib inline import warnings warnings.filterwarnings("ignore") ``` -------------------------------- ### Configure Git for Signed Commits Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/contribute.md This snippet shows how to configure your global Git settings for username and email, which is necessary for signing your patches when contributing code. ```bash git config --global user.name "Kate Developer" git config --global user.email "kate.developer@company.com" ``` -------------------------------- ### Set OneDAL Environment Variable (Offline Installer) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Sources the oneAPI setvars script to automatically set environment variables, including DALROOT, for the oneAPI Data Analytics Library when using an offline installer on Linux. ```bash source /opt/intel/oneapi/setvars.sh ``` -------------------------------- ### Import Optimized and Stock Classes Separately Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/README.md Demonstrates how to import both the optimized version of a scikit-learn class from `sklearnex` and the original stock version from `sklearn` into the same script. This allows for explicit comparison or selective use of optimized algorithms. ```python from sklearnex.cluster import DBSCAN as exDBSCAN from sklearn.cluster import DBSCAN as stockDBSCAN # ... ``` -------------------------------- ### Load and Prepare IEEE-CIS Fraud Detection Dataset Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/examples/notebooks/ridge_regression_fraud_detection.ipynb Loads the IEEE-CIS Fraud Detection dataset (data_id=45955) using fetch_openml and performs initial preprocessing. This involves loading the data into a pandas DataFrame, filling missing values with 0, and separating features (x) from the target variable (y). ```python dataset = fetch_openml(data_id=45955, as_frame=True) data = dataset.frame # Simple imputation for missing values data = data.fillna(0) # Set the target variable x = data.drop(columns=['fraud']) y = data['fraud'] ``` -------------------------------- ### Distributed QR Decomposition with daal4py Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/distributed_daal4py.md Demonstrates distributed QR decomposition using daal4py. This snippet shows how to initialize MPI, distribute data, perform QR decomposition in a distributed manner, and verify results. It relies on daal4py for MPI world size and process ID retrieval. ```python import daal4py import numpy as np NUM_NODES = daal4py.num_procs() # this the MPI world size THIS_NODE = daal4py.my_procid() # this is the MPI rank rng = np.random.RandomState(seed=123) X_full = rng.standard_normal(size=(100,5)) subsets = np.split(np.arange(100), NUM_NODES) X_node = X_full[ subsets[THIS_NODE] ] qr_algo = daal4py.qr(distributed=True)qr_result = qr_algo.compute(X_node) daal4py.daalfini() # call before accessing the results # Matrix R (shape=[ncols, ncols]) is common for all nodes np.testing.assert_almost_equal( np.abs( qr_result.matrixR ), np.abs( np.linalg.qr(X_full).R ), ) # Matrix Q (size=[nrows, ncols]) will be a subset of the full # result corresponding to the data from the node only np.testing.assert_almost_equal( np.abs( qr_result.matrixQ ), np.abs( np.linalg.qr(X_full).Q[ subsets[THIS_NODE] ] ), ) ``` -------------------------------- ### Install Additional Non-Python Dependencies via Pip Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Installs specific non-Python dependencies like clang-format and Intel's MPI (impi-devel, impi_rt) using pip. Note that using pip for Intel MPI requires manual setting of the $MPIROOT environment variable. ```bash pip install clang-format impi-devel impi_rt ``` -------------------------------- ### Model Builders API (GBT and LogReg serving) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/daal4py.md Documentation for model builders that enable fast predictions from GBT models generated by other libraries. ```APIDOC ## Model Builders (GBT and LogReg serving) ### Description Model builders for computing fast predictions from GBT (gradient-boosted decision tree) models produced by other libraries. See article [Serving GBT models from other libraries](model_builders.md#model-builders) for examples. ### Method N/A (Builder pattern) ### Endpoint N/A ### Parameters N/A ### Request Example N/A ### Response N/A ### Success Response (N/A) N/A ### Response Example N/A ``` -------------------------------- ### Set MPI Root Path for Building Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Specifies how to set the MPIROOT environment variable, which is required when building the library in distributed mode. It can point to the MPI library path or use Intel's MPI specific variable I_MPI_ROOT. Examples show manual setting and automatic setting via conda or oneAPI activation scripts. ```bash export MPIROOT="$CONDA_PREFIX" ``` ```bash source /opt/intel/oneapi/setvars.sh ``` -------------------------------- ### Enable scikit-learn Acceleration via Command-Line Patching Source: https://context7.com/uxlfoundation/scikit-learn-intelex/llms.txt Shows how to accelerate a Python scikit-learn application without modifying its source code by using the `sklearnex` module from the command line. Requires `sklearnex`. ```bash # Original application python my_sklearn_app.py # Accelerated execution with automatic patching python -m sklearnex my_sklearn_app.py # Application code (my_sklearn_app.py) remains unchanged: # from sklearn.ensemble import RandomForestClassifier # model = RandomForestClassifier().fit(X, y) ``` -------------------------------- ### Set Global Configuration Option for GPU Offloading in scikit-learn-intelex Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/config-contexts.md This example shows how to set a global configuration option using `set_config` to enable GPU offloading for all subsequent scikit-learn-intelex operations. It defines sample data, sets `target_offload` globally to 'gpu', performs DBSCAN clustering, and then restores the default 'auto' setting. ```python import numpy as np from sklearnex import set_config from sklearnex.cluster import DBSCAN X = np.array([[1., 2.], [2., 2.], [2., 3.], [8., 7.], [8., 8.], [25., 80.]], dtype=np.float32) set_config(target_offload="gpu") # set it globally clustering = DBSCAN(eps=3, min_samples=2).fit(X) set_config(target_offload="auto") # restore it back ``` -------------------------------- ### Running DBSCAN with Array API Strict Arrays using Scikit-learn* Extension Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/array_api.md This example illustrates how to use the `array-api-strict` library with patched Scikit-learn* estimators, specifically `sklearn.cluster.DBSCAN`. It involves patching Scikit-learn* with `sklearnex`, creating an `array_api_strict` array, fitting the DBSCAN model, and printing the resulting cluster labels. The `config_context(array_api_dispatch=True)` ensures compatibility, though it's not strictly required for this specific example if only using stock Scikit-learn* fallbacks. ```python # ============================================================================== # Copyright 2024 Intel Corporation # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ============================================================================== import array_api_strict from sklearnex import config_context, patch_sklearn patch_sklearn() from sklearn.cluster import DBSCAN X = array_api_strict.asarray( [[1.0, 2.0], [2.0, 2.0], [2.0, 3.0], [8.0, 7.0], [8.0, 8.0], [25.0, 80.0]], dtype=array_api_strict.float32, ) # Could be launched without `config_context(array_api_dispatch=True)`. This context # manager for sklearnex, only guarantee that in case of the fallback to stock # scikit-learn, fitted attributes to be from the same Array API namespace as # the training data. clustering = DBSCAN(eps=3, min_samples=2).fit(X) print(f"Fitted labels :\n", clustering.labels_) ``` -------------------------------- ### Import daal4py Module Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/about_daal4py.md Demonstrates how to import the daal4py module after installing the scikit-learn-intelex package. This is the standard way to access daal4py functionalities. ```python import daal4py ``` -------------------------------- ### Execute All Tests on Linux Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/tests.md Runs the entire test suite for the scikit-learn-intelex library on a Linux system. This script should be executed from the root of the repository. ```bash conda-recipe/run_test.sh ``` -------------------------------- ### GPU Acceleration with config_context Source: https://context7.com/uxlfoundation/scikit-learn-intelex/llms.txt Offload computations to GPU using the config_context manager for targeted acceleration. This example shows GPU acceleration for DBSCAN clustering. ```APIDOC ## GPU Acceleration with config_context ### Description Offload computations to GPU using the config_context manager for targeted acceleration. ### Method N/A (Runtime Configuration) ### Endpoint N/A (Runtime Configuration) ### Parameters - **target_offload** (string) - Required - Specifies the target device for offloading, e.g., "gpu:0". ### Request Example ```python import numpy as np from sklearnex import patch_sklearn, config_context patch_sklearn() from sklearn.cluster import DBSCAN X = np.array([[1., 2.], [2., 2.], [2., 3.], [8., 7.], [8., 8.], [25., 80.]], dtype=np.float32) # Execute on GPU device 0 with config_context(target_offload="gpu:0"): clustering = DBSCAN(eps=3, min_samples=2).fit(X) print(f"GPU-accelerated labels: {clustering.labels_}") # Code outside context runs on CPU clustering_cpu = DBSCAN(eps=3, min_samples=2).fit(X) ``` ### Response #### Success Response (200) N/A (Execution output) #### Response Example ``` GPU-accelerated labels: [-1 0 0 -1 -1 -1] ``` ``` -------------------------------- ### Import and Use Preview Functionality (Python) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/preview.md This Python code illustrates how to patch Scikit-learn to include preview functionality after enabling it via the environment variable. It then imports an example preview estimator, IncrementalPCA, and prints its module to confirm it's from the preview package. ```python from sklearnex import patch_sklearn patch_sklearn() from sklearn.decomposition import IncrementalPCA print(IncrementalPCA.__module__) ``` -------------------------------- ### Set OneDAL Environment Variable (Conda) Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/building-from-source.md Manually sets the DALROOT environment variable to the conda prefix path, which is necessary when installing the oneAPI Data Analytics Library via conda on Linux. ```bash export DALROOT="$CONDA_PREFIX" ``` -------------------------------- ### Configure TMS Settings and Load TMS Script Source: https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/doc/sources/_templates/layout.html Configures TMS (Tag Management System) settings, including local code and section, and dynamically loads the TMS script if the current URL matches the expected domain. This script is hosted on intel.com. ```javascript var wapLocalCode = 'us-en'; var wapSection = "scikit-learn"; if (document.location.href.includes("uxlfoundation.github.io/scikit-learn-intelex")) { (function () { var url = 'https://www.intel.com/content/dam/www/global/wap/tms-loader.js'; var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = url; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })(); } ```