### Apply Trained Model to New Data Source: https://github.com/arundo/adtk/blob/develop/docs/quickstart.rst Loads new time series data for testing, validates it, and uses the previously trained SeasonalAD model to detect anomalies in this new data. The results are then visualized. ```python >>> s_test = pd.read_csv("./testing.csv", index_col="Datetime", parse_dates=True, squeeze=True) >>> s_test = validate_series(s_test) >>> print(s_test) >>> anomalies_pred = seasonal_ad.detect(s_test) >>> plot(s_test, anomaly=anomalies_pred, ... ts_linewidth=1, anomaly_color='red', anomaly_tag="marker") ``` -------------------------------- ### Install ADTK from Source Source: https://github.com/arundo/adtk/blob/develop/docs/install.rst Installs the latest, potentially unstable, version of ADTK from its source code repository. This involves cloning the repository, checking out the develop branch, and then installing locally. ```console git clone https://github.com/arundo/adtk.git cd adtk/ git checkout develop pip install ./ ``` -------------------------------- ### Cross-check Detection with Known Anomalies Source: https://github.com/arundo/adtk/blob/develop/docs/quickstart.rst Loads known anomaly data, converts it into an event format using ADTK's to_events, and visualizes both the known anomalies and the model-detected anomalies on the training data for comparison. ```python >>> known_anomalies = pd.read_csv("./known_anomalies.csv", index_col="Datetime", parse_dates=True, squeeze=True) >>> from adtk.data import to_events >>> known_anomalies = to_events(known_anomalies) >>> print(known_anomalies) >>> plot(s_train, ... anomaly={"Known": known_anomalies, "Model": anomalies}, ... anomaly_tag={"Known": "span", "Model": "marker"}, ... anomaly_color={"Known": "orange", "Model": "red"}) ``` -------------------------------- ### Install ADTK Documentation Dependencies Source: https://github.com/arundo/adtk/blob/develop/docs/README.md Installs the necessary Python packages for building the ADTK documentation from the requirements-docs.txt file. ```bash $ pip install -r requirements-docs.txt ``` -------------------------------- ### Install Documentation Dependencies Source: https://github.com/arundo/adtk/blob/develop/docs/developer.rst Installs ADTK with the 'doc' extra, which includes packages necessary for generating the project's documentation using Sphinx. ```console pip install adtk[doc] ``` -------------------------------- ### Install ADTK from Source Source: https://github.com/arundo/adtk/blob/develop/README.md Installs the latest, potentially unstable, version of ADTK directly from its source code repository. This method involves cloning the repository, checking out the develop branch, and then installing locally. ```shell git clone https://github.com/arundo/adtk.git cd adtk/ git checkout develop pip install ./ ``` -------------------------------- ### Install ADTK from PyPI Source: https://github.com/arundo/adtk/blob/develop/README.md Installs the latest stable release of the Anomaly Detection Toolkit (ADTK) from the Python Package Index (PyPI). This is the recommended installation method for most users. ```shell pip install adtk ``` -------------------------------- ### Detect Seasonal Pattern Violations Source: https://github.com/arundo/adtk/blob/develop/docs/quickstart.rst Initializes and applies the SeasonalAD detector to identify anomalies based on seasonal patterns. It then visualizes the detected anomalies on the time series data. ```python >>> from adtk.detector import SeasonalAD >>> seasonal_ad = SeasonalAD() >>> anomalies = seasonal_ad.fit_detect(s_train) >>> plot(s_train, anomaly=anomalies, anomaly_color="red", anomaly_tag="marker") ``` -------------------------------- ### Visualize Time Series Data Source: https://github.com/arundo/adtk/blob/develop/docs/quickstart.rst Visualizes the loaded time series data using ADTK's plot function. This helps in understanding the data's trends and patterns before applying anomaly detection. ```python >>> from adtk.visualization import plot >>> plot(s_train) ``` -------------------------------- ### Install ADTK from PyPI Source: https://github.com/arundo/adtk/blob/develop/docs/install.rst Installs the latest stable release of ADTK from the Python Package Index (PyPI). Requires Python 3.5 or later. ```console pip install adtk ``` -------------------------------- ### Arundo ADTK Project Setup and Tracking Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Describes package setup options for developers and fixes related to Travis and Coveralls status tracking. ```python # Added some package setup options for developers # Fixed an issue of tracking Travis and Coveralls status ``` -------------------------------- ### Load and Validate Time Series Data Source: https://github.com/arundo/adtk/blob/develop/docs/quickstart.rst Loads time series data from a CSV file using pandas, validates it using ADTK's validate_series function, and prints the resulting series. This step is crucial for ensuring data quality before anomaly detection. ```python >>> import pandas as pd >>> s_train = pd.read_csv("./training.csv", index_col="Datetime", parse_dates=True, squeeze=True) >>> from adtk.data import validate_series >>> s_train = validate_series(s_train) >>> print(s_train) ``` -------------------------------- ### Install Development Dependencies Source: https://github.com/arundo/adtk/blob/develop/docs/developer.rst Installs ADTK with the 'dev' extra, which includes the required formatter (Black) and linter (isort) for development. ```console pip install adtk[dev] ``` -------------------------------- ### Install Testing Dependencies Source: https://github.com/arundo/adtk/blob/develop/docs/developer.rst Installs ADTK with the 'test' extra, which includes testing frameworks like pytest, Coverage.py, and tox for automated testing across different Python environments. ```console pip install adtk[test] ``` -------------------------------- ### ADTK Component Overview Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst This section outlines the core components of ADTK: detectors for scanning time series, transformers for feature extraction, and aggregators for combining detection results. It also introduces Pipelines and Pipenets for composing these components. ```python import adtk.detector import adtk.transformer import adtk.aggregator import adtk.pipe # Detectors scan time series for anomalies. # Transformers transform time series for feature extraction. # Aggregators combine multiple detection results. # Pipelines connect components sequentially. # Pipenets connect components in more complex, non-sequential ways. # Example of creating a custom component from a function: # from adtk.detector import CustomizedDetector1D # def my_detector_function(time_series): # # ... anomaly detection logic ... # return anomaly_series # # customized_detector = CustomizedDetector1D(my_detector_function) ``` -------------------------------- ### Spike and Level Shift Detection with ADTK Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst Detectors for spikes and level shifts, which are time-dependent anomalies. These often utilize the DoubleRollingAggregate transformer. ```python from adtk.detector import PersistAD, LevelShiftAD from adtk.transformer import DoubleRollingAggregate # Example usage (conceptual): # double_rolling_transformer = DoubleRollingAggregate(agg_func='mean', window_size=10, agg_func_right='mean', window_size_right=5) # persist_detector = PersistAD(double_rolling_transformer) # level_shift_detector = LevelShiftAD(double_rolling_transformer) ``` -------------------------------- ### Outlier Detection Detectors in ADTK Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst Detectors for identifying outliers in time series data. These can use user-defined thresholds or learn normal ranges from historical data. ```python from adtk.detector import ThresholdAD, QuantileAD, InterQuartileRangeAD, GeneralizedESDTestAD # Example usage (conceptual): # threshold_detector = ThresholdAD(lower_threshold=-5, upper_threshold=5) # quantile_detector = QuantileAD(c=3) # iqr_detector = InterQuartileRangeAD(c=1.5) # esd_detector = GeneralizedESDTestAD(alpha=0.05, side='both') ``` -------------------------------- ### Rolling Aggregate for Temporal Pattern Detection Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst Utilize RollingAggregate to detect temporal changes by measuring statistics within sliding windows. Useful for identifying temporary anomalies like high request frequencies. ```python from adtk.transformer import RollingAggregate # Example tracking the number of non-zero values in a window rolling_non_zeros = RollingAggregate(agg=lambda x: (x != 0).sum()) ``` -------------------------------- ### Univariate and Multivariate Time Series Handling in ADTK Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst ADTK automatically applies univariate transformers and detectors to each series in a multivariate DataFrame. For anomalies requiring simultaneous analysis of multiple series (e.g., heat index), multivariate transformers and detectors should be considered. ```python # Applying a univariate transformer to a multivariate DataFrame # Example: df is a pandas DataFrame with multiple columns # univariate_transformer.fit_transform(df) # For multivariate-specific anomalies, use multivariate detectors/transformers # Example: multivariate_detector.fit_transform(df) ``` -------------------------------- ### Seasonal Decomposition with ClassicSeasonalDecomposition Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst Remove seasonal patterns from time series using ClassicSeasonalDecomposition to highlight deviations from normal seasonal behavior in the residual series. Suitable for data with fixed periods. ```python from adtk.transformer import ClassicSeasonalDecomposition # Example decomposing a time series with a weekly period seasonal_decomposition = ClassicSeasonalDecomposition(period=7) ``` -------------------------------- ### Rolling Aggregate Statistics with DoubleRollingAggregate Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst Detect shifts in time series patterns by applying various statistics like standard deviation over rolling windows. This transformer quantifies patterns of interest for anomaly detection. ```python from adtk.transformer import DoubleRollingAggregate # Example using standard deviation to detect volatility shifts double_rolling_std = DoubleRollingAggregate(agg=lambda x: x.std()) ``` -------------------------------- ### Autoregression for Cyclic Time Series Source: https://github.com/arundo/adtk/blob/develop/docs/userguide.rst Capture changes in autoregressive relationships for cyclic (but not seasonal) time series. Autoregression can be effective when seasonality decomposition is not appropriate due to variable cycle lengths. ```python from adtk.detector import AutoregressionAD # Initialize AutoregressionAD detectorautoregression_detector = AutoregressionAD() ``` -------------------------------- ### Build ADTK HTML Documentation Source: https://github.com/arundo/adtk/blob/develop/docs/README.md Builds the HTML version of the ADTK documentation using the make command. The output will be available in the _build/html directory. ```bash $ make html ``` -------------------------------- ### ADTk Pipenet Parameter and Algorithm Changes (Version 0.5.0) Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Highlights changes in version 0.5.0, including the modification of the `steps` parameter in pipenet from a list to a dictionary, the addition of a `summary` method to pipenet, and corrections to seasonal decomposition algorithms. It also mentions updates to the demo notebook and plotting functions. ```python Changed the parameter `steps` of pipenet from list to dict Added method `summary` to pipenet Corrected some major algorithmic issues on seasonal decomposition - Removed STL decomposition transformer, and hence the corresponding option in SeasonalAD detector - Recreated classic seasonal decomposition transformer Updated the demo notebook in the documentation Added an option to hide legend in the plotting function ``` -------------------------------- ### ADTK Versioning Scheme Source: https://github.com/arundo/adtk/blob/develop/docs/developer.rst Details the versioning strategy for ADTK, differentiating between stable releases and pre-release versions. Stable releases follow semantic versioning (MAJOR.MINOR.PATCH), while pre-release versions incorporate pull request information. ```APIDOC Stable Release Versions: - Format: MAJOR.MINOR.PATCH (e.g., 0.Y.Z) - Minor version increment (Y): Indicates API-breaking changes (new features, new models). - Patch version increment (Z): Indicates non-API-breaking changes (bug fixes, documentation updates). - Releases are published to PyPI and GitHub. - Stable documentation corresponds to the most recent release. Pre-release Versions: - Generated from pull requests merged into 'master' or 'develop' branches. - Format for API-changing PRs: 0.[Y+1].0-dev.N+pr.M - N: Monotonic increasing index. - M: Pull request index. - Example: Merging PR #37 (API change) to 'develop' after 0.1.2 release -> 0.2.0-dev.1+pr.37 - Format for non-API-changing PRs: 0.Y.[Z+1]-dev.N+pr.M - N: Monotonic increasing index. - M: Pull request index. - Example: Merging PR #38 (bug fix) to 'master' after 0.1.2 release -> 0.1.3-dev.1+pr.38 - Latest documentation corresponds to the most recent pre-release in the 'develop' branch. ``` -------------------------------- ### ADTK Metrics Module Source: https://github.com/arundo/adtk/blob/develop/docs/api/metrics.rst This section details the members of the adtk.metrics module, including any inherited members. It serves as a reference for the functionalities available within the metrics component of the ADTK library. ```python .. automodule:: adtk.metrics :members: :inherited-members: ``` -------------------------------- ### ADTK Transformers Module Source: https://github.com/arundo/adtk/blob/develop/docs/api/transformers.rst This section provides an overview of the classes within the `adtk.transformer` module. It includes details on inherited members and their functionalities for time series data transformation. ```python from adtk.transformer import * # Example usage of a transformer (e.g., ScalerTransformer) # from adtk.data import load_dataset # data = load_dataset("realKnownCause", # "A1Benchmark", # "realKnownCause-A1Benchmark.csv") # # scaler = ScalerTransformer() # scaled_data = scaler.fit_transform(data) # print(scaled_data.head()) ``` -------------------------------- ### adtk.pipe Module Members Source: https://github.com/arundo/adtk/blob/develop/docs/api/pipe.rst This section documents the members of the adtk.pipe module, including classes and functions for creating anomaly detection pipelines. It details the functionality and usage of these components within the ADTK library. ```python import adtk.pipe # Example usage (assuming other ADTK components are imported and configured) # from adtk.detector import ThresholdAD # from adtk.data import load_dataset # data = load_dataset("realKnownCause") # detector = ThresholdAD(high_value_only=True) # pipenet = adtk.pipe.Pipenet(data, detector) # pipenet.fit() # pipenet.detect() # pipeline = adtk.pipe.Pipeline() # pipeline.add(detector) # pipeline.fit(data) # pipeline.detect(data) ``` -------------------------------- ### Arundo ADTK Version 0.4.1 Changes Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Details changes in version 0.4.1, including fixes for tox environments and documentation. ```text Version 0.4.1 (Nov 21, 2019) =================================== - Fixed an issue of tox environments - Minor spelling/grammar fix in documentation ``` -------------------------------- ### ADTk Unit Tests, Dependency Changes, and Bug Fixes (Version 0.5.1) Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Details improvements in version 0.5.1, including the addition of new unit tests, removal of seaborn dependency, and fixes for bugs in the metric module, OutlierDetector, transformer pipeline, and pipenet summary. ```python Added many new unit tests, and modified some old unit test Removed seaborn from dependencies (use matplotlib built-in style now) Fixed a bug in the metric module of dict objects as input Fixed a bug in the detector OutlierDetector that output series has dtype object if NaN is present Fixed a bug in transformer pipeline that detect and transform methods are confused Fixed a bug in pipenet that an aggregator node may crash if its input is from a node where subset contains a single item Fixed a bug in pipenet summary that subset column are always "all" even if not Some minor optimization of code ``` -------------------------------- ### Model Class Inheritance Diagram Source: https://github.com/arundo/adtk/blob/develop/docs/inheritance.rst This diagram displays the hierarchical relationships between different model classes. It shows how base classes like _Model, _NonTrainableModel, and _TrainableModel are extended by more specific classes such as univariate/multivariate models, detectors, and transformers, including custom implementations. ```console _Model |-- _NonTrainableModel |-- _NonTrainableUnivariateModel | |-- _NonTrainableUnivariateDetector | | └-- ThresholdAD | | | └-- _NonTrainableUnivariateTransformer | |-- RollingAggregate | |-- DoubleRollingAggregate | |-- Retrospect | └-- StandardScale | | └-- _NonTrainableMultivariateModel | └-- _NonTrainableMultivariateTransformer | └-- SumAll | |-- _TrainableModel |-- _TrainableUnivariateModel | |-- _TrainableUnivariateDetector | | |-- QuantileAD | | |-- InterQuartileRangeAD | | |-- GeneralizedESDTestAD | | |-- PersistAD | | |-- LevelShiftAD | | |-- VolatilityShiftAD | | |-- SeasonalAD | | |-- AutoregressionAD | | └-- CustomizedDetector1D | | | └-- _TrainableUnivariateTransformer | |-- ClassicSeasonalDecomposition | └-- CustomizedTransformer1D | └-- _TrainableMultivariateModel |-- _TrainableMultivariateDetector | |-- MinClusterDetector | |-- OutlierDetector | |-- RegressionAD | |-- PcaAD | └-- CustomizedDetectorHD | └-- _TrainableMultivariateTransformer |-- RegressionResidual |-- PcaProjection |-- PcaReconstruction |-- PcaReconstructionError └-- CustomizedTransformerHD | └-- _Aggregator |-- AndAggregator |-- OrAggregator └-- CustomizedAggregator ``` -------------------------------- ### ADTK Detector Module Source: https://github.com/arundo/adtk/blob/develop/docs/api/detectors.rst This section details the detectors available in the `adtk.detector` module. It lists members and inherited members, providing insights into the functionalities and classes related to anomaly detection. ```python import adtk.detector # Accessing detector functionalities # Example: Using a specific detector class (assuming one exists like ThresholdADFDetector) # from adtk.detector import ThresholdADFDetector # detector = ThresholdADFDetector(c=2.5, side='both') # print(detector) ``` -------------------------------- ### Arundo ADTK Version 0.3.0 Changes Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Details the initial release of Arundo ADTK in version 0.3.0. ```text Version 0.3.0 (Sep 27, 2019) =================================== - Initial release ``` -------------------------------- ### ADTk Module Changes and API Redesign (Version 0.6.0) Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Details significant changes in version 0.6.0, including API redesigns for visualization, removal of the resample module, enhancements to data expansion functions, changes in output types, and modifications to required model parameters. It also notes the refactoring of inheritance structures and the addition of Python 3.8 support. ```python Re-designed the API of :py:mod:`adtk.visualization.plot` Removed :py:mod:`adtk.data.resample` because its functionality is highly overlapped with pandas resampler module Made :py:mod:`adtk.data.expand_events` accept events in the form of pandas Series/DataFrame Made :py:mod:`adtk.data.expand_events` accept time delta in the form of `str` or `int` Changed the output type of :py:mod:`adtk.data.split_train_test` from a 2-tuple of lists to a list of 2-tuples Turned the following model parameters required from optional - `window` in :py:mod:`adtk.detector.LevelShiftAD` - `window` in :py:mod:`adtk.detector.VolatilityShiftAD` - `window` in :py:mod:`adtk.transformer.RollingAggregate` - `window` in :py:mod:`adtk.transformer.DoubleRollingAggregate` - `model` in :py:mod:`adtk.detector.MinClusterDetector` - `model` in :py:mod:`adtk.detector.OutlierDetector` - `target` and `regressor` in :py:mod:`adtk.detector.RegressionAD` - `target` and `regressor` in :py:mod:`adtk.transformer.RegressionResidual` - `aggregate_func` in :py:mod:`adtk.aggregator.CustomizedAggregator` - `detect_func` in :py:mod:`adtk.detector.CustomizedDetector1D` - `detect_func` in :py:mod:`adtk.detector.CustomizedDetectorHD` - `transform_func` in :py:mod:`adtk.transformer.CustomizedTransformer1D` - `transform_func` in :py:mod:`adtk.detector.CustomizedTransformer1D` - `steps` in :py:mod:`adtk.pipe.Pipeline` Added consistency check between training and testing inputs in multivariate models Improved time index check in time-dependent models Turned all second-order sub-modules private, and a user now can only import from the following first-order modules - :py:mod:`adtk.detector` - :py:mod:`adtk.transformer` - :py:mod:`adtk.aggregator` - :py:mod:`adtk.pipe` - :py:mod:`adtk.data` - :py:mod:`adtk.metrics` - :py:mod:`adtk.visualization` Refactored the inheritance structure of model components (see :ref:`inheritance`) Added Python 3.8 support Fixed compatibility issues with statsmodels v0.11 Fixed compatibility issues with pandas v1.0 Created an interactive demo notebook in Binder Added type hints, and added type checking in CI/CD test Added `Black` and `isort` to developer requirement and CI/CD check Optimized release process by publishing package to PyPI through GitHub Actions Improved docstrings and API documentation Fixed many minor bugs and typos ``` -------------------------------- ### ADTk Workflow Optimization and Bug Fixes (Version 0.5.4) Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Focuses on optimizing the workflow for applying univariate models to pandas DataFrames in version 0.5.4. This includes adding more informative error messages, fixing bugs related to model-column matching, and clarifying the documentation. ```python Optimized the workflow of how a univariate model is applied to pandas DataFrame - Added more informative error messages - Fixed some bugs resulting in model-column matching error due to inconsistency between output Series names and DataFrame columns - Clarified the workflow in the documentation ``` -------------------------------- ### Arundo ADTK Version 0.4.0 Changes Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Details changes in version 0.4.0, including Python 3.5 support, improved tests, and README updates. ```text Version 0.4.0 (Nov 18, 2019) =================================== - Added support to Python 3.5 - Better unit tests on dependencies - Minor typo fix in documentation - Minor code optimization - Added download statistics to README - Added coverage test ``` -------------------------------- ### ADTk Dependency Management and Bug Fixes (Version 0.5.3) Source: https://github.com/arundo/adtk/blob/develop/docs/releasehistory.rst Addresses compatibility issues with statsmodels in version 0.5.3 by implementing a hotfix to avoid errors caused by statsmodels v0.11. This involves requiring statsmodels dependency to be less than version 0.11. ```python Quick hotfix to avoid errors caused by statsmodels v0.11 by requiring statsmodels dependency <0.11 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.