### Run PyPOTS with Docker Source: https://github.com/wenjiedu/pypots/blob/main/docs/install.rst Utilize Docker to run PyPOTS in a pre-configured environment. This command pulls the official PyPOTS Docker image, starts an instance, and allows access to a ready-to-use setup for running PyPOTS. ```bash docker run -it --name pypots wenjiedu/pypots # docker will auto pull our built image and run a instance for you # after things settled, you can run python in the container to access the well-configured environment for running pypots # if you'd like to detach from the container, press ctrl-P + ctrl-Q # run `docker attach pypots` to enter the container again. ``` -------------------------------- ### Install PyPOTS using Pip Source: https://github.com/wenjiedu/pypots/blob/main/docs/install.rst Install or upgrade PyPOTS using pip. This includes installing the latest stable version, updating to the newest release, or installing directly from the main branch of the GitHub repository for the latest features. ```bash pip install pypots # the first time installation pip install pypots --upgrade # update pypots to the latest version # install from the latest source code with the latest features but may be not officially released yet pip install https://github.com/WenjieDu/PyPOTS/archive/main.zip ``` -------------------------------- ### Configure CPU Acceleration for Intel CPUs Source: https://github.com/wenjiedu/pypots/blob/main/docs/install.rst Enhance performance on devices with Intel CPUs by installing the MKL distribution. This command installs optimized versions of numpy, scipy, and scikit-learn for multi-core Intel processors. ```bash conda install numpy scipy scikit-learn numexpr "libblas=*=*mkl" ``` -------------------------------- ### Install PyPOTS using pip Source: https://github.com/wenjiedu/pypots/blob/main/README_zh.md Installs or updates the PyPOTS library using pip. This method is suitable for the initial installation or upgrading to the latest stable version. It also shows how to install directly from the latest source code. ```bash pip install pypots # 首次安装 pip install pypots --upgrade # 更新为最新版本 pip install https://github.com/WenjieDu/PyPOTS/archive/main.zip ``` -------------------------------- ### Install PyPOTS using Conda Source: https://github.com/wenjiedu/pypots/blob/main/docs/install.rst Install or update PyPOTS using conda. This method ensures that PyPOTS and its dependencies are managed within the conda environment, providing a consistent setup. ```bash conda install conda-forge::pypots # the first time installation conda update conda-forge::pypots # update pypots to the latest version ``` -------------------------------- ### Configure CPU Acceleration for Mac with Apple Silicon Source: https://github.com/wenjiedu/pypots/blob/main/docs/install.rst Optimize processing speed on Macs with Apple Silicon by installing the 'accelerate' data-science packages. This command installs essential libraries with optimizations for Apple Silicon. ```bash conda install numpy scipy scikit-learn numexpr "libblas=*=*accelerate" ``` -------------------------------- ### Install and Run PyPOTS using Docker Source: https://github.com/wenjiedu/pypots/blob/main/README_zh.md Provides instructions for installing and running PyPOTS within a Docker container. This method is useful for users who want a pre-configured environment or to isolate PyPOTS from their system. It covers creating a container, running it, and re-attaching to it. ```bash # via docker docker run -it --name pypots wenjiedu/pypots # docker会自动拉取我们构建好的镜像并为你运行一个实例 # 运行结束后, 你可以在该容器中运行python即可使用我们为运行pypots配置好的环境 # 如果你想退出该容器, 先按ctrl-P然后按ctrl-Q即可退出 # 运行命令`docker attach pypots`可以重新进入该容器 ``` -------------------------------- ### Install PyPOTS using Pip, Conda, and Docker Source: https://github.com/wenjiedu/pypots/blob/main/README.md Instructions for installing PyPOTS using pip (including upgrade and from source), conda (including upgrade), and docker. These commands set up the necessary environment for using the PyPOTS library. ```bash # via pip pip install pypots # the first time installation pip install pypots --upgrade # update pypots to the latest version # install from the latest source code with the latest features but may be not officially released yet pip install https://github.com/WenjieDu/PyPOTS/archive/main.zip # via conda conda install conda-forge::pypots # the first time installation conda update conda-forge::pypots # update pypots to the latest version # via docker docker run -it --name pypots wenjiedu/pypots # docker will auto pull our built image and run a instance for you # after things settled, you can run python in the container to access the well-configured environment for running pypots # if you'd like to detach from the container, press ctrl-P + ctrl-Q # run `docker attach pypots` to enter the container again. ``` -------------------------------- ### Configure CPU Acceleration for AMD CPUs Source: https://github.com/wenjiedu/pypots/blob/main/docs/install.rst Install the OpenBLAS distribution to achieve faster processing speeds on devices with AMD CPUs. This command installs optimized versions of numpy, scipy, and scikit-learn using OpenBLAS. ```bash conda install -c conda-forge numpy scipy scikit-learn numexpr "libblas=*=*openblas" ``` -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Install PyPOTS using conda Source: https://github.com/wenjiedu/pypots/blob/main/README_zh.md Installs or updates the PyPOTS library using conda. This method is recommended for users who prefer the Anaconda distribution and ensures compatibility within the conda environment. It covers both initial installation and updating. ```bash conda install conda-forge::pypots # 首次安装 conda update conda-forge::pypots # 更新为最新版本 ``` -------------------------------- ### SAITS Model for Time Series Imputation on PhysioNet2012 Dataset Source: https://github.com/wenjiedu/pypots/blob/main/README_zh.md This example demonstrates how to use the SAITS model from PyPOTS for time series imputation on the PhysioNet2012 dataset. It covers data preprocessing, model training, imputation, evaluation using MAE, and model saving/loading. Dependencies include numpy, sklearn, pygrinder, and pypots. ```python import numpy as np from sklearn.preprocessing import StandardScaler from pygrinder import mcar, calc_missing_rate from benchpots.datasets import preprocess_physionet2012 # Load and preprocess the dataset data = preprocess_physionet2012(subset='set-a', rate=0.1) # Our tool library will automatically download and decompress the dataset train_X, val_X, test_X = data["train_X"], data["val_X"], data["test_X"] print(train_X.shape) # (n_samples, n_steps, n_features) print(val_X.shape) # The number of samples in the validation set is different from the training set (n_samples different), but the sample length (n_steps) and feature dimension (n_features) are consistent print(f"Missing rate in the training set train_X is {calc_missing_rate(train_X):.1%}") # Prepare datasets for the model train_set = {"X": train_X} # The training set only needs to contain incomplete time series val_set = { "X": val_X, "X_ori": data["val_X_ori"], # We need the true values in the validation set for evaluation and model selection } test_set = {"X": test_X} # The test set only provides incomplete time series to be imputed test_X_ori = data["test_X_ori"] # test_X_ori contains the true values for final evaluation indicating_mask = np.isnan(test_X) ^ np.isnan(test_X_ori) # Generate an indicating mask: mark the positions of artificially added missing values in the test set (positions where X has missing values but X_ori does not) from pypots.imputation import SAITS # Import the model you want to use from pypots.nn.functional import calc_mae # Initialize and train the SAITS model saits = SAITS(n_steps=train_X.shape[1], n_features=train_X.shape[2], n_layers=2, d_model=256, n_heads=4, d_k=64, d_v=64, d_ffn=128, dropout=0.1, epochs=5) saits.fit(train_set, val_set) # Train the model on the dataset # Impute and evaluate imputation = saits.impute(test_set) # Impute the original missing and artificially added missing values in the test set mae = calc_mae(imputation, np.nan_to_num(test_X_ori), indicating_mask) # Calculate MAE at the artificially added missing positions (compare imputed results with true values) # Save and load the model saits.save("save_it_here/saits_physionet2012.pypots") # Save the model for future use saits.load("save_it_here/saits_physionet2012.pypots") # Reload the model for subsequent imputation or continued training ``` -------------------------------- ### Data Preparation and SAITS Model Training with PyPOTS Source: https://github.com/wenjiedu/pypots/blob/main/docs/examples.rst This Python snippet demonstrates how to preprocess the PhysioNet 2012 dataset using `preprocess_physionet2012` and then train a SAITS model. It covers data organization for training, validation, and testing, including handling ground truth for evaluation. The model is trained using the `fit` method and imputation is performed with the `impute` method. Finally, it shows how to calculate Mean Absolute Error (MAE) and save/load the trained model. ```python import numpy as np from sklearn.preprocessing import StandardScaler from pygrinder import mcar, calc_missing_rate from benchpots.datasets import preprocess_physionet2012 from pypots.imputation import SAITS from pygrinder import calc_mae # prepare the dataset data = preprocess_physionet2012(subset='set-a',rate=0.1) # Our ecosystem libs will automatically download and extract it train_X, val_X, test_X = data["train_X"], data["val_X"], data["test_X"] print(train_X.shape) # (n_samples, n_steps, n_features) print(val_X.shape) # samples (n_samples) in train set and val set are different, but they have the same sequence len (n_steps) and feature dim (n_features) print(f"We have {calc_missing_rate(train_X):.1%} values missing in train_X") # organize the dataset for PyPOTS model input train_set = {"X": train_X} # in training set, simply put the incomplete time series into it val_set = { "X": val_X, "X_ori": data["val_X_ori"], # in validation set, we need ground truth for evaluation and picking the best model checkpoint } test_set = {"X": test_X} # in test set, only give the testing incomplete time series for model to impute # the test set for final evaluation test_X_ori = data["test_X_ori"] # test_X_ori bears ground truth for evaluation indicating_mask = np.isnan(test_X) ^ np.isnan(test_X_ori) # mask indicates the values that are missing in X but not in X_ori, i.e. where the gt values are # initialize the model _, n_steps, n_features = train_X.shape saits = SAITS( n_steps=n_steps, n_features=n_features, n_layers=2, d_model=256, d_ffn=128, n_heads=4, d_k=64, d_v=64, dropout=0.1, epochs=10, saving_path="examples/saits", # set the path for saving tensorboard logging file and model checkpoint model_saving_strategy="best", # only save the model with the best validation performance ) # train the model. You can also omit the val_set if you don't need to validate the model during training saits.fit(train_set, val_set) # impute the originally-missing values and artificially-missing values imputation = saits.impute(test_set) mae = calc_mae(imputation, np.nan_to_num(test_X_ori), indicating_mask) # calculate mean absolute error on the ground truth (artificially-missing values) # the best model has been already saved, but you can still manually save it with function save_model() as below saits.save(saving_path="examples/saits/manually_saved_saits_model") # you can load the saved model into a new initialized model saits.load("examples/saits/manually_saved_saits_model.pypots") ``` -------------------------------- ### Untitled No description -------------------------------- ### Integration with Ecosystem Libraries in Python Source: https://context7.com/wenjiedu/pypots/llms.txt Demonstrates integration of PyPOTS with TSDB for dataset loading, PyGrinder for missing data simulation, and BenchPOTS for preprocessing. Assumes necessary libraries are installed via pip. ```python # Install ecosystem # pip install pypots tsdb pygrinder benchpots from tsdb import list_available_datasets, load_dataset from pygrinder import mcar, mar, mnar from benchpots.datasets import preprocess_physionet2012 from pypots.imputation import SAITS # Load dataset from TSDB tsdb_data = load_dataset("physionet_2012") # Simulate missing values with PyGrinder data_with_mcar = mcar(tsdb_data['X'], rate=0.1) data_with_mar = mar(tsdb_data['X'], rate=0.1) # Use BenchPOTS preprocessing pipeline bench_data = preprocess_physionet2012(subset='set-a', rate=0.1) # Train PyPOTS model model = SAITS( n_steps=bench_data['train_X'].shape[1], n_features=bench_data['train_X'].shape[2], n_layers=2, d_model=256 ) model.fit({"X": bench_data['train_X']}, {"X": bench_data['val_X'], "X_ori": bench_data['val_X_ori']}) ``` -------------------------------- ### PyPOTS SAITS for Time Series Imputation Example Source: https://github.com/wenjiedu/pypots/blob/main/README.md A Python code example demonstrating the use of the SAITS model from PyPOTS for imputing missing values in time series data, using the PhysioNet2012 dataset. It covers data preprocessing, model training, imputation, evaluation, and model saving/loading. ```python import numpy as np from sklearn.preprocessing import StandardScaler from pygrinder import mcar, calc_missing_rate from benchpots.datasets import preprocess_physionet2012 data = preprocess_physionet2012(subset='set-a',rate=0.1) # Our ecosystem libs will automatically download and extract it train_X, val_X, test_X = data["train_X"], data["val_X"], data["test_X"] print(train_X.shape) # (n_samples, n_steps, n_features) print(val_X.shape) # samples (n_samples) in train set and val set are different, but they have the same sequence len (n_steps) and feature dim (n_features) print(f"We have {calc_missing_rate(train_X):.1%} values missing in train_X") train_set = {"X": train_X} # in training set, simply put the incomplete time series into it val_set = { "X": val_X, "X_ori": data["val_X_ori"], # in validation set, we need ground truth for evaluation and picking the best model checkpoint } test_set = {"X": test_X} # in test set, only give the testing incomplete time series for model to impute test_X_ori = data["test_X_ori"] # test_X_ori bears ground truth for evaluation indicating_mask = np.isnan(test_X) ^ np.isnan(test_X_ori) # mask indicates the values that are missing in X but not in X_ori, i.e. where the gt values are from pypots.imputation import SAITS # import the model you want to use from pypots.nn.functional import calc_mae saits = SAITS(n_steps=train_X.shape[1], n_features=train_X.shape[2], n_layers=2, d_model=256, n_heads=4, d_k=64, d_v=64, d_ffn=128, dropout=0.1, epochs=5) saits.fit(train_set, val_set) # train the model on the dataset imputation = saits.impute(test_set) # impute the originally-missing values and artificially-missing values mae = calc_mae(imputation, np.nan_to_num(test_X_ori), indicating_mask) # calculate mean absolute error on the ground truth (artificially-missing values) saits.save("save_it_here/saits_physionet2012.pypots") # save the model for future use saits.load("save_it_here/saits_physionet2012.pypots") # reload the serialized model file for following imputation or training ``` -------------------------------- ### Untitled No description -------------------------------- ### Hyperparameter Optimization with NNI in Python Source: https://context7.com/wenjiedu/pypots/llms.txt Configure hyperparameter search space and integrate PyPOTS models with Microsoft NNI for automatic tuning. Requires NNI installation and a training script to fetch parameters and report results. ```python # search_space.json configuration { "n_layers": {"_type": "choice", "_value": [1, 2, 3]}, "d_model": {"_type": "choice", "_value": [128, 256, 512]}, "n_heads": {"_type": "choice", "_value": [4, 8]}, "dropout": {"_type": "uniform", "_value": [0.0, 0.5]} } # Training script with NNI import nni from pypots.imputation import SAITS # Get hyperparameters from NNI params = nni.get_next_parameter() model = SAITS( n_steps=48, n_features=12, n_layers=params['n_layers'], d_model=params['d_model'], n_heads=params['n_heads'], dropout=params['dropout'], epochs=50 ) model.fit(train_set, val_set) val_loss = model.evaluate(val_set) # Report result to NNI nni.report_final_result(val_loss) ``` -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### PyPOTS CLI Utilities in Bash Source: https://context7.com/wenjiedu/pypots/llms.txt Utilize the PyPOTS Command Line Interface (CLI) for environment checks, documentation generation, code style analysis, running tests, and launching hyperparameter optimization (HPO). ```bash # Check PyPOTS environment pypots-cli env # Generate API documentation pypots-cli doc # Run development utilities pypots-cli dev --check-code-style pypots-cli dev --run-tests # Launch hyperparameter optimization pypots-cli hpo --config hpo_config.yml --port 8080 ``` -------------------------------- ### Untitled No description -------------------------------- ### Initialize and Train MOMENT for Time Series Forecasting Source: https://context7.com/wenjiedu/pypots/llms.txt Initializes the MOMENT foundation model for time series forecasting, supporting transfer learning. It requires parameters such as sequence length, number of features, forecast horizon, model dimensions, patch size, batch size, and training epochs. The model can be fine-tuned on domain-specific data. ```python from pypots.forecasting import MOMENT # MOMENT with pre-trained weights moment = MOMENT( n_steps=512, n_features=1, n_pred_steps=96, d_model=768, patch_size=16, batch_size=16, epochs=20 ) # Fine-tune on domain-specific data moment.fit(train_set, val_set) forecast = moment.predict(test_set) ``` -------------------------------- ### Initialize and Train TimeMixer for Forecasting Source: https://context7.com/wenjiedu/pypots/llms.txt Initializes the TimeMixer model for time series forecasting. It requires parameters like input sequence length, number of features, forecast horizon, model dimensions, and training epochs. The model is then trained using historical and validation data, and used to predict future values. ```python from pypots.forecasting import TimeMixer # Initialize TimeMixer for forecasting timemixer = TimeMixer( n_steps=96, # Input sequence length n_features=7, # Number of variables n_pred_steps=24, # Forecast horizon d_model=512, d_ffn=2048, n_layers=3, dropout=0.1, top_k=5, batch_size=32, epochs=50 ) # Prepare forecasting dataset forecast_train = {"X": train_data} # Historical data forecast_val = {"X": val_data, "X_pred": val_target} # Train and forecast timemixer.fit(forecast_train, forecast_val) predictions = timemixer.predict({"X": test_data}) print(f"Forecast shape: {predictions.shape}") # (n_samples, n_pred_steps, n_features) ``` -------------------------------- ### Custom Time Series Dataset Creation with BaseDataset Source: https://context7.com/wenjiedu/pypots/llms.txt Demonstrates how to create custom time series datasets using `BaseDataset`. This class handles multiple file formats and missing value patterns. Requires a dictionary containing data (e.g., 'X', 'y') for initialization. ```python from pypots.data import BaseDataset import numpy as np # Create custom dataset class MyTimeSeriesDataset(BaseDataset): def __init__(self, data_dict, return_labels=True): super().__init__( data=data_dict, return_X_ori=False, return_y=return_labels, file_type="hdf5" ) # Use custom dataset my_data = { "X": np.random.randn(1000, 48, 12), "y": np.random.randint(0, 3, 1000) } dataset = MyTimeSeriesDataset(my_data, return_labels=True) ``` -------------------------------- ### Initialize and Train TimeLLM for Forecasting Source: https://context7.com/wenjiedu/pypots/llms.txt Initializes the TimeLLM model, a large language model adapted for time series forecasting. It converts time series into text-like representations for LLM processing. Key parameters include sequence length, number of features, forecast horizon, LLM backbone choice, model dimensions, layers, batch size, and epochs. ```python from pypots.forecasting import TimeLLM # TimeLLM with LLM backbone timellm = TimeLLM( n_steps=96, n_features=21, n_pred_steps=48, llm_model="gpt2", d_model=256, n_layers=2, batch_size=8, epochs=30 ) timellm.fit(train_set, val_set) llm_forecast = timellm.predict(test_set) ``` -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description -------------------------------- ### Untitled No description