### Install dtaianomaly from Source Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/installation.md Install dtaianomaly by building it from its source code. This method requires downloading the source files first and then running the installation command from the root directory. ```bash pip install . ``` -------------------------------- ### Install dtaianomaly with Specific Optional Dependencies Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/installation.md Install dtaianomaly with a selected subset of optional dependencies by replacing 'all' with the desired subset name (e.g., 'tqdm', 'docs'). Multiple subsets can be installed by separating their names with a comma. ```bash pip install dtaianomaly[subset_name] ``` -------------------------------- ### Install pre-commit hooks Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Install pre-commit to automatically format code and run checks before commits. ```bash pre-commit install ``` -------------------------------- ### Install dtaianomaly Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/README.md Install the dtaianomaly package using pip. This is the preferred method for installation. ```bash pip install dtaianomaly ``` -------------------------------- ### Install dtaianomaly from GitHub Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/installation.md Install the latest development version of dtaianomaly directly from its GitHub repository. This is useful for accessing unreleased features. ```bash pip install git+https://github.com/ML-KULeuven/dtaianomaly ``` -------------------------------- ### Run InTimeAD demonstrator programmatically Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/api/in_time_ad.md Start the InTimeAD demonstrator by importing and running the 'in_time_ad' module within a Python script. ```python from dtaianomaly import in_time_ad in_time_ad.run() ``` -------------------------------- ### Install dtaianomaly with All Optional Dependencies Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/installation.md Install dtaianomaly along with all its optional dependencies for full functionality. This includes dependencies for testing, documentation, notebooks, linting, and specific model integrations. ```bash pip install dtaianomaly[all] ``` -------------------------------- ### Install dtaianomaly with InTimeAD dependencies Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/api/in_time_ad.md Install the dtaianomaly library along with the optional 'demonstrator' and 'in_time_ad' dependencies using pip. ```bash pip install dtaianomaly[in_time_ad] ``` -------------------------------- ### Run documentation doctests Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Execute doctests within the documentation to ensure examples are correct and up-to-date. ```bash docs/make doctest ``` -------------------------------- ### Install editable dependencies Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Install dtaianomaly and all its optional dependencies in editable mode. This ensures your local code changes are reflected in the installed version. ```bash pip install --editable .[all] ``` -------------------------------- ### Install Specific Version of dtaianomaly from PyPI Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/installation.md Install a particular version (e.g., X.Y.Z) of dtaianomaly from PyPI. Replace X.Y.Z with the desired version number. ```bash pip install dtaianomaly==X.Y.Z ``` -------------------------------- ### Install Specific Version of dtaianomaly from GitHub Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/installation.md Install a specific version of dtaianomaly from GitHub using its tag (e.g., X.Y.Z). Replace X.Y.Z with the desired version tag. ```bash pip install git+https://github.com/ML-KULeuven/dtaianomaly@X.Y.Z ``` -------------------------------- ### Anomaly Detection Example Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/README.md Demonstrates loading data, preprocessing with a moving average, fitting a MatrixProfileDetector, and computing decision scores and anomaly probabilities. ```python from dtaianomaly.data import demonstration_time_series from dtaianomaly.preprocessing import MovingAverage from dtaianomaly.anomaly_detection import MatrixProfileDetector # Load the data X, y = demonstration_time_series() # Preprocess the data using a moving average preprocessor = MovingAverage(window_size=10) X_, _ = preprocessor.fit_transform(X) # Fit the matrix profile detector on the processed data detector = MatrixProfileDetector(window_size=100) detector.fit(X_) # Compute either the decision scores, specific to the detector, or the anomaly probabilities decision_scores = detector.decision_function(X_) anomaly_probabilities = detector.predict_proba(X_) ``` -------------------------------- ### Load Demonstration Time Series Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/data/README.rst Loads the demonstration time series used throughout the dtaianomaly documentation. This is useful for quick testing and examples. ```python from dtaianomaly.data import demonstration_time_series X, y = demonstration_time_series() ``` -------------------------------- ### Run InTimeAD demonstrator from command line Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/api/in_time_ad.md Execute the InTimeAD demonstrator directly from the command line. ```bash run-demonstrator ``` -------------------------------- ### Build documentation Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Generate the project documentation in HTML format. ```bash docs/make html ``` -------------------------------- ### Run Workflow from Configuration File Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md Use the `workflow_from_config` method to load a workflow defined in a JSON configuration file and then execute it using the `run` function. Ensure the configuration file path is correct. ```python from dtaianomaly.workflow import workflow_from_config workflow = workflow_from_config("Config.json") workflow.run() ``` -------------------------------- ### Instantiate PenmanshielDataLoader for Training Data Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Instantiates the PenmanshielDataLoader to load the training set. Ensure `return_train_data` is set to `True`. ```python train_loader = PenmanshielDataLoader( path='../data/Penmanshiel', do_caching=True, turbine_name='Penmanshiel 01', attribute='Lost Production Total (kWh)', train_size=0.1, return_train_data=True ) train_set = train_loader.load() ``` -------------------------------- ### Initialize Thresholds and Metrics Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Sets up a ContaminationRateThreshold for anomaly detection and initializes a list of performance metrics including Precision, Recall, FBeta, and AreaUnderROC. These are used to evaluate the effectiveness of the anomaly detectors. ```python thresholds = ContaminationRateThreshold(0.1) metrics = [Precision(), Recall(), FBeta(beta=1), AreaUnderROC()] ``` -------------------------------- ### Create and use a Pipeline for anomaly detection Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Automate the preprocessing and anomaly detection steps by initializing a Pipeline with a preprocessor and a detector. The fit and predict methods of the pipeline will handle both steps sequentially. ```python pipeline = Pipeline( preprocessor=preprocessor, detector=detector ) y_pred = pipeline.fit(X).predict_proba(X) ``` -------------------------------- ### Load and Plot Demonstration Time Series Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/anomaly_detection.md Loads and plots a demonstration time series for anomaly detection analysis. ```python from dtaianomaly.data import demonstration_time_series from dtaianomaly.visualize import plot_time_series_colored_by_score X, y = demonstration_time_series() plot_time_series_colored_by_score(X, y) ``` -------------------------------- ### Initialize Preprocessors for Workflow Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md Define a list of preprocessor objects to be applied to the time series data. This includes no preprocessing (Identity), standard scaling, and chained preprocessing steps like moving average followed by scaling. ```python preprocessors = [ Identity(), StandardScaler(), ChainedPreprocessor([MovingAverage(10), StandardScaler()]), ChainedPreprocessor([ExponentialMovingAverage(0.8), StandardScaler()]) ] ``` -------------------------------- ### Initialize and Run Workflow Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md Initialize the Workflow object with all defined components (dataloaders, metrics, thresholds, preprocessors, detectors) and set the number of parallel jobs. Execute the workflow using the `run()` method to obtain a results dataframe. ```python workflow = Workflow( dataloaders=dataloaders, metrics=metrics, thresholds=thresholds, preprocessors=preprocessors, detectors=detectors, n_jobs=4 ) results = workflow.run() ``` -------------------------------- ### Initialize MovingAverage preprocessor Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Initialize the MovingAverage preprocessor with a specified window size to reduce noise in the time series data before anomaly detection. ```python preprocessor = MovingAverage(window_size=10) ``` -------------------------------- ### Initialize DataLoaders for Workflow Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md Create a list of LazyDataLoader objects to specify the time series datasets for evaluation. Alternatively, use the `from_directory()` method to load all datasets from a specified directory. ```python dataloaders = [ UCRLoader('data/UCR-time-series-anomaly-archive/001_UCR_Anomaly_DISTORTED1sddb40_35000_52000_52620.txt'), UCRLoader('data/UCR-time-series-anomaly-archive/002_UCR_Anomaly_DISTORTED2sddb40_35000_56600_56900.txt') ] ``` -------------------------------- ### Run InTimeAD demonstrator with custom anomaly detectors Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/api/in_time_ad.md Integrate custom anomaly detector classes into the InTimeAD demonstrator by passing them to the run() method. ```python from dtaianomaly import in_time_ad from NbSigmaAnomalyDetector import NbSigmaAnomalyDetector in_time_ad.run(custom_anomaly_detectors=NbSigmaAnomalyDetector) ``` -------------------------------- ### Instantiate PenmanshielDataLoader for Test Data Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Instantiates the PenmanshielDataLoader to load the test set. Set `return_train_data` to `False` to retrieve the test data. ```python test_loader = PenmanshielDataLoader( path='../data/Penmanshiel', do_caching=True, turbine_name='Penmanshiel 01', attribute='Lost Production Total (kWh)', train_size=0.1, return_train_data=False # Make sure to load the test data! ) test_set = test_loader.load() ``` -------------------------------- ### Initialize Anomaly Detectors for Workflow Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md Initialize a list of anomaly detector objects. Each detector will be combined with each preprocessor and applied to each time series. ```python detectors = [LocalOutlierFactor(50), IsolationForest(50)] ``` -------------------------------- ### Initialize and Run Anomaly Detection Workflow Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Initializes a Workflow object with specified components and runs the anomaly detection process. Ensure all components (dataloaders, preprocessors, detectors, metrics, thresholds) are properly initialized before use. ```python workflow = Workflow( dataloaders=dataloaders, preprocessors=preprocessors, detectors=detectors, metrics=metrics, thresholds=thresholds ) workflow.run() ``` -------------------------------- ### Create a new branch Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Create a new branch for your specific issue or feature implementation. Replace with a descriptive name. ```bash git checkout -b ``` -------------------------------- ### Clone your fork of dtaianomaly Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Clone your forked repository to your local machine to begin making changes. ```bash git clone https://github.com/ ``` -------------------------------- ### Initialize Thresholds and Metrics for Workflow Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md Define thresholding strategies (e.g., TopN, FixedCutoff) for converting anomaly scores to binary labels, and evaluation metrics (e.g., Precision, AreaUnderPR, AreaUnderROC). Metrics are evaluated using predicted anomaly scores from `predict_proba()`. ```python thresholds = [TopN(20), FixedCutoff(0.1)] metrics = [Precision(), AreaUnderPR(), AreaUnderROC()] ``` -------------------------------- ### Create a Pipeline for Preprocessing and Detection Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Combines an Imputer and an IsolationForest detector into a Pipeline for streamlined preprocessing and anomaly detection. This pipeline can then be fitted to training data and used to predict anomaly probabilities on test data. ```python detector = Pipeline( preprocessor=Imputer(), detector=IsolationForest(window_size=15) ) y_pred = detector.fit(train_set.x).predict_proba(test_set.x) ``` -------------------------------- ### Workflow Configuration File Structure Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/quantitative_evaluation.md A configuration file is a JSON object where keys represent workflow parameters and values are either single entries or lists of entries, each defining a workflow component with its type and parameters. ```default { "type": , "optional-param": } ``` ```json { "dataloaders": [ { "type": "UCRLoader", "path":"../data/UCR-time-series-anomaly-archive/001_UCR_Anomaly_DISTORTED1sddb40_35000_52000_52620.txt" }, { "type": "UCRLoader", "path":"../data/UCR-time-series-anomaly-archive/002_UCR_Anomaly_DISTORTED2sddb40_35000_56600_56900.txt" } ], "metrics": [ {"type": "Precision"}, {"type": "AreaUnderPR"}, {"type": "AreaUnderROC"} ], "thresholds": [ {"type": "TopN", "n": 20}, {"type": "FixedCutoff", "cutoff": 0.5} ], "preprocessors": [ {"type": "Identity"}, {"type": "StandardScaler"}, {"type": "ChainedPreprocessor", "base_preprocessors": [ {"type": "MovingAverage", "window_size": 10}, {"type": "StandardScaler"} ]}, {"type": "ChainedPreprocessor", "base_preprocessors": [ {"type": "ExponentialMovingAverage", "alpha": 0.8}, {"type": "StandardScaler"} ]} ], "detectors": [ {"type": "LocalOutlierFactor", "window_size": 50}, {"type": "IsolationForest", "window_size": 50} ], "n_jobs": 4 } ``` -------------------------------- ### Preprocess data and detect anomalies Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Apply the initialized preprocessor to the time series data and then fit the detector to the processed data to predict anomaly probabilities. Note that fit_transform returns processed data and potentially processed ground truth. ```python X_, y_ = preprocessor.fit_transform(X) y_pred = detector.fit(X_).predict_proba(X_) ``` -------------------------------- ### Initialize Anomaly Detectors Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Initializes a list of anomaly detectors including Isolation Forest, Local Outlier Factor, and Matrix Profile Detector, each with a specified window size. These detectors are used to identify anomalies in time series data. ```python detectors = [ IsolationForest(window_size=15), LocalOutlierFactor(window_size=15), MatrixProfileDetector(window_size=15) ] ``` -------------------------------- ### Import Libraries Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/logo/dtaianomaly-logo.ipynb Imports essential Python libraries for numerical operations, scientific computing, and plotting. ```python import numpy as np import scipy import matplotlib.pyplot as plt ``` -------------------------------- ### Initialize MatrixProfileDetector Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Initialize the MatrixProfileDetector with a specified window size for detecting anomalies based on matrix profiles. This detector is suitable for identifying patterns that deviate from the norm. ```python detector = MatrixProfileDetector(window_size=100) ``` -------------------------------- ### Visualize and Save Logo Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/logo/dtaianomaly-logo.ipynb Creates a final visualization of the combined pattern with specific styling and saves it as an SVG file. The plot is displayed without axes and with a transparent background. ```python plt.figure(figsize=(6, 5)) plt.plot(bell_curve * wave, c='#00407A', linewidth=20) plt.axis('off') plt.tight_layout() plt.savefig('ts.svg', transparent=True) ``` -------------------------------- ### Commit Changes to Fork Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Use these commands to stage, commit, and push your changes to your remote fork after resolving an issue. ```bash git add . git commit -m git push ``` -------------------------------- ### Import necessary dtaianomaly modules Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Import all required modules from dtaianomaly for data handling, visualization, preprocessing, anomaly detection, pipeline creation, thresholding, and evaluation. ```python from dtaianomaly.data import demonstration_time_series from dtaianomaly.visualization import plot_time_series_colored_by_score, plot_time_series_anomalies, plot_anomaly_scores from dtaianomaly.preprocessing import MovingAverage from dtaianomaly.anomaly_detection import MatrixProfileDetector from dtaianomaly.pipeline import Pipeline from dtaianomaly.thresholding import FixedCutoffThreshold from dtaianomaly.evaluation import Precision, Recall, FBeta, AreaUnderROC, AreaUnderPR, ThresholdMetric ``` -------------------------------- ### Generate and Plot Data Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/logo/dtaianomaly-logo.ipynb Generates a bell curve and a sine wave, then combines them to create a composite pattern. Displays the individual components and the combined result. ```python nb_samples = 1000 x = np.linspace(0, 1, 1000) bell_curve = scipy.stats.norm.pdf((x-0.5) * 8, 0, 1) wave = np.sin(2 * np.pi * x * 5) fig, axs = plt.subplots(1, 3, figsize=(10, 3)) axs[0].plot(bell_curve) axs[1].plot(wave) axs[2].plot(bell_curve * wave); ``` -------------------------------- ### Run tests and check coverage Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/contributing.md Execute all tests and check code coverage using pytest. The --cov-report term-missing flag shows missing test coverage. ```bash pytest tests/ --cov=dtaianomaly --cov-report term-missing ``` -------------------------------- ### Custom PenmanshielDataLoader for Industrial Data Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Implements a data loader for the Penmanshiel dataset, inheriting from `PathDataLoader`. It handles reading SCADA and status data, aggregating it, and formatting it into numpy arrays for anomaly detection. Caching is managed by the parent class. ```python class PenmanshielDataLoader(PathDataLoader): def __init__(self, path: str, do_caching: bool, turbine_name: str, attribute: str, train_size: float, return_train_data: bool): super().__init__(path, do_caching) self.turbine_name: str = turbine_name self.attribute: str = attribute self.train_size: float = train_size self.return_train_data: bool = return_train_data def _load(self) -> DataSet: # Lists for the individual data frames status_data_frames = [] turbine_data_frames = [] # Iterate over all the directories in the path for sub_directory in os.listdir(self.path): # Skip non-SCADA data if not sub_directory.startswith('Penmanshiel_SCADA'): continue # Iterate over all the files in the directory for file_name in os.listdir(f'{self.path}/{sub_directory}'): # Skip files for other assets if self.turbine_name.replace(' ', '_') not in file_name: continue # Read the status data if file_name.startswith('Status'): status_data = pd.read_csv( f'{self.path}/{sub_directory}/{file_name}', skiprows=9, parse_dates=['Timestamp start'], date_format='%Y-%m-%d %H:%M:%S' ) status_data['Timestamp end'] = pd.to_datetime(status_data['Timestamp end'], errors='coerce', format='%Y-%m-%d %H:%M:%S') status_data['Duration'] = pd.to_timedelta(status_data['Duration'], errors='coerce') status_data_frames.append(status_data) # Read the turbine data if file_name.startswith('Turbine'): turbine_data = pd.read_csv( f'{self.path}/{sub_directory}/{file_name}', skiprows=9, parse_dates=['# Date and time'] ).rename(columns={'# Date and time': 'Date and time'}).set_index('Date and time') turbine_data_frames.append(turbine_data[self.attribute]) # Create one frame for the status and turbine data status_data = pd.concat(status_data_frames) turbine_data = pd.concat(turbine_data_frames).resample('10min').asfreq() # FIll in missing values with nan # Format the anomalous time steps anomalies = pd.Series(index=turbine_data.index, data=0.0, name='anomalies') for _, event in status_data[status_data['Status'] == 'Stop'].iterrows(): anomalies.loc[event['Timestamp start']:event['Timestamp end']] = 1 # Return either the train or test set ``` -------------------------------- ### Anomaly Detection with Pipeline Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/anomaly_detection.md Utilizes a Pipeline to automatically preprocess data and detect anomalies, simplifying the workflow. ```python from dtaianomaly.pipeline import Pipeline from dtaianomaly.preprocessing import MovingAverage from dtaianomaly.anomaly_detection import MatrixProfileDetector pipeline = Pipeline([ MovingAverage(n=5), MatrixProfileDetector() ]) scores = pipeline.fit_predict(X, y) plot_time_series_colored_by_score(X, scores) ``` -------------------------------- ### Anomaly Detection with MovingAverage and MatrixProfileDetector Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/anomaly_detection.md Applies MovingAverage preprocessing and MatrixProfileDetector for anomaly detection on a time series. ```python from dtaianomaly.preprocessing import MovingAverage from dtaianomaly.anomaly_detection import MatrixProfileDetector preprocessor = MovingAverage(n=5) detector = MatrixProfileDetector() X_, y_ = preprocessor.fit_transform(X, y) scores = detector.fit_predict(X_, y_) plot_time_series_colored_by_score(X_, scores) ``` -------------------------------- ### Quantitative Evaluation: FBeta Score Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/anomaly_detection.md Calculates the FBeta score using a ThresholdMetric with a specified threshold and beta value. ```python from dtaianomaly.evaluation import FBeta, ThresholdMetric, FixedCutoff fbeta = FBeta(beta=1) threshold_metric = ThresholdMetric(fbeta, FixedCutoff(threshold=0.85)) print(f"FBeta score: {threshold_metric.evaluate(scores, y_)}") ``` -------------------------------- ### Apply thresholding and plot anomalies Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Convert continuous anomaly scores into binary predictions using a fixed cutoff threshold and then visualize the detected anomalies on the time series, highlighting True Positives, False Positives, and False Negatives. ```python y_pred_binary = FixedCutoffThreshold(cutoff=0.85).threshold(y_pred) plot_time_series_anomalies(X, y, y_pred_binary, figsize=(10, 3)).tight_layout() ``` -------------------------------- ### Compute F1 Score with Integrated Thresholding Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Utilize ThresholdMetric to automatically apply a specified thresholding strategy before computing a binary evaluation metric like F-beta score. This simplifies the process of evaluating metrics that require binary inputs. ```python print('F1:', ThresholdMetric(thresholding, FBeta(1.0)).compute(y, y_pred)) ``` -------------------------------- ### Custom Imputer Preprocessor Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Defines a custom Imputer preprocessor that replaces missing values with the previous observed value. This is useful for handling missing data in time series before anomaly detection. ```python class Imputer(Preprocessor): def _fit(self, X, y=None) -> 'Preprocessor': return self def _transform(self, X, y=None) -> (np.ndarray, np.ndarray): for i in range(X.shape[0]): if np.isnan(X[i]): X[i] = X[i-1] if y is not None: if np.isnan(y[i]): y[i] = y[i-1] return X, y preprocessors = Imputer() ``` -------------------------------- ### Quantitative Evaluation: Precision and Recall Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/anomaly_detection.md Computes Precision and Recall metrics after applying FixedCutoff thresholding to anomaly scores. ```python from dtaianomaly.evaluation import Precision, Recall, FixedCutoff precision = Precision(FixedCutoff(threshold=0.85)) recall = Recall(FixedCutoff(threshold=0.85)) print(f"Precision: {precision.evaluate(scores, y_)}") print(f"Recall: {recall.evaluate(scores, y_)}") ``` -------------------------------- ### Compute Precision and Recall with Fixed Threshold Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Apply a fixed cutoff threshold to convert continuous anomaly scores to binary predictions, then compute precision and recall. This is useful when you need to assess detection accuracy at a specific operating point. ```python thresholding = FixedCutoffThreshold(0.85) y_pred_binary = thresholding.threshold(y_pred) print('Precision:', Precision().compute(y, y_pred_binary)) print('Recall:', Recall().compute(y, y_pred_binary)) ``` -------------------------------- ### Import Additional Python Libraries Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Imports common Python libraries used for data manipulation, numerical operations, and plotting, which are often used alongside dtaianomaly. ```python # Additional (non-dtaianomaly) dependencies for various utilities import os import numpy as np import pandas as pd import matplotlib.pyplot as plt ``` -------------------------------- ### Visualize Anomaly Detection Results Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Plots the time series data, ground truth anomaly scores, and predicted anomaly scores for visual analysis. This helps in understanding how well the detector is performing and identifying potential issues. ```python fig, (ax1, ax2, ax3) = plt.subplots(3, figsize=(20, 4), height_ratios=[1, 0.3, 0.3]) NB_OBSERVATIONS = 35000 ax1.plot(test_set.x[:NB_OBSERVATIONS]) ax1.set_title('Time series data') ax2.plot(test_set.y[:NB_OBSERVATIONS], color='red') ax2.set_title('Predicted anomaly scores') ax3.plot(y_pred[:NB_OBSERVATIONS], color='orange') ax3.set_title('Ground truth anomaly scores') fig.tight_layout() ``` -------------------------------- ### Quantitative Evaluation: Area Under ROC and PR Curves Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/getting_started/examples/anomaly_detection.md Computes Area Under ROC (AUC-ROC) and Area Under Precision-Recall (AUC-PR) curves using continuous anomaly scores. ```python from dtaianomaly.evaluation import AreaUnderROC, AreaUnderPR auc_roc = AreaUnderROC() auc_pr = AreaUnderPR() print(f"AUC-ROC: {auc_roc.evaluate(scores, y_)}") print(f"AUC-PR: {auc_pr.evaluate(scores, y_)}") ``` -------------------------------- ### Plot Time Series and Anomaly Scores Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Visualizes the time series data and its corresponding anomaly labels using matplotlib. This helps in visually inspecting the data. ```python fig, (ax1, ax2) = plt.subplots(2, figsize=(20, 3), sharex='all', height_ratios=[1, 0.3]) ax1.plot(train_set.x) ax1.set_title('Time series data') ax2.plot(train_set.y, color='red') ax2.set_title('Anomaly scores') fig.tight_layout() ``` -------------------------------- ### Load Data Loader Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Assigns the pre-defined train_loader to the dataloaders variable for use in the Workflow. ```python dataloaders = train_loader ``` -------------------------------- ### dtaianomaly Citation Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/docs/additional_information/references.md Use this BibTeX entry for citing the dtaianomaly library in academic work. ```bibtex @article{carpentier2025dtaianomaly, title={{dtaianomaly: A Python library for time series anomaly detection}}, author={Louis Carpentier and Nick Seeuws and Wannes Meert and Mathias Verbeke}, year={2025}, eprint={2502.14381}, archivePrefix={arXiv}, primaryClass={cs.LG}, journal={} } ``` -------------------------------- ### Plot anomaly scores Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Anomaly-detection.ipynb Visualize the original time series along with the predicted anomaly scores. This helps in understanding how well the detector identifies anomalous regions. The plot is saved to a file. ```python fig = plot_anomaly_scores(X, y, y_pred, method_to_plot=plot_time_series_colored_by_score, figsize=(10, 3)) fig.tight_layout() fig.axes[1].set_title('Anomaly scores of matrix profile detector') fig.savefig('Demonstration-time-series-detected-anomalies.svg') ``` -------------------------------- ### Compute Area Under ROC Curve Source: https://github.com/ml-kuleuven/dtaianomaly/blob/main/notebooks/Industrial-anomaly-detection.ipynb Calculates the Area Under the ROC curve (AUC-ROC) to evaluate the performance of the anomaly detector using the ground truth labels and predicted probabilities. This metric is useful when ground truth labels are available for the test set. ```python auc_roc = AreaUnderROC() f'AUC-ROC: {auc_roc.compute(test_set.y, y_pred)}' ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.