### Clone Repository Source: https://github.com/ibm-granite/granite-tsfm/blob/main/README.md Clone the repository to get started. Navigate into the cloned directory to proceed with setup. ```bash git clone "https://github.com/ibm-granite/granite-tsfm.git" cd granite-tsfm ``` -------------------------------- ### Install KServe Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/inference/README.md Downloads and executes the KServe quick install script. This script may need to be run multiple times if container pulls or starts are delayed. ```bash curl -s https://raw.githubusercontent.com/kserve/kserve/master/hack/quick_install.sh > quick_install.sh bash ./quick_install.sh -r ``` -------------------------------- ### Few-Shot Fine-tuning Setup Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/research_use/ttm-r2_freq_benchmarking_512_96.ipynb Initiates the few-shot fine-tuning process, indicating the percentage of few-shot data used and reporting the number of parameters before and after freezing the backbone. It also starts the learning rate finder algorithm. ```text -------------------- Running few-shot 5% -------------------- Number of params before freezing backbone 854972 Number of params after freezing the backbone 302162 LR Finder: Running learning rate (LR) finder algorithm. If the suggested LR is very low, we suggest setting the LR manually. LR Finder: Using GPU:0. ``` -------------------------------- ### Install and Run Demo Source: https://github.com/ibm-granite/granite-tsfm/blob/main/tsfmhfdemos/neurips/README.md Commands to install the required dependencies and launch the Streamlit application. ```bash pip install ".[demos]" cd tsfmhfdemos/neurips streamlit run app.py ``` -------------------------------- ### Install Notebooks Source: https://github.com/ibm-granite/granite-tsfm/blob/main/README.md Install the necessary packages to use the provided notebooks for model pre-training and finetuning. ```bash pip install ".[notebooks]" ``` -------------------------------- ### Install tsfm and dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patch_tsmixer_blog.ipynb Commands to clone the repository and install the required packages. ```bash git clone git@github.com:IBM/tsfm.git cd tsfm ``` ```bash pip install . ``` -------------------------------- ### Install and Import Core Libraries Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/LongHorizonForecast.ipynb Installs and imports essential Python libraries for data manipulation, numerical operations, and machine learning. Ensure these libraries are installed before proceeding. ```python %%capture try: import pandas # noqa: F401 except ImportError: !pip install pandas try: import numpy # noqa: F401 except ImportError: !pip install numpy try: import matplotlib # noqa: F401 except ImportError: !pip install matplotlib try: import sklearn # noqa: F401 except ImportError: !pip install scikit-learn try: import scipy # noqa: F401 except ImportError: !pip install scipy try: import torch # noqa: F401 except ImportError: !pip install torch try: import tqdm # noqa: F401 except ImportError: !pip install tqdm ``` -------------------------------- ### Install dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/inference/README.md Syncs the project environment using uv with development extras. ```sh uv sync --locked --extra dev --editable ``` -------------------------------- ### Few-Shot Training Setup Information Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/ttm-r1_benchmarking_1024_96.ipynb This output details the setup for few-shot training, including the percentage of data used, the number of parameters before and after backbone freezing, and the learning rate. ```text -------------------- Running few-shot 5% -------------------- Number of params before freezing backbone 946336 Number of params after freezing the backbone 389984 Using learning rate = 0.001 ``` -------------------------------- ### Install FAISS CPU Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tspulse_search_with_faiss.ipynb Install the FAISS library for CPU-based indexing. A Linux environment is recommended to avoid potential installation issues. ```python # Uncomment if not installed # Linux environment recommended to avoid installation issues # !pip install faiss-cpu ``` -------------------------------- ### Create Kind Cluster with Local Registry Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/inference/README.md Installs a Kind cluster and configures a local Docker registry. Ensure you have kubectl, kind, and helm installed. ```bash curl -s https://kind.sigs.k8s.io/examples/kind-with-registry.sh | bash ``` -------------------------------- ### Install Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/finetuning/README.md Installs project dependencies using uv, including development extras and editable mode. ```sh uv sync --locked --extra dev --editable ``` -------------------------------- ### Start Local Server for OpenAPI/Swagger Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/inference/README.md Starts a local server to view the OpenAPI 3.x specification and Swagger UI. Access it via http://127.0.0.1:8000 in your browser. ```bash make start_local_server ``` -------------------------------- ### Install Ray (Optional) Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/gift_leaderboard_ttm_r3_nc/README.md Installs the Ray library, which is an optional dependency for distributed execution. Use this if your workload benefits from parallel processing. ```bash pip install ray==2.54.0 ``` -------------------------------- ### Verify installation Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patch_tsmixer_blog.ipynb Python commands to verify that the necessary modules are importable. ```python from transformers import PatchTSMixerConfig from tsfm_public.toolkit.dataset import ForecastDFDataset ``` -------------------------------- ### Install Granite TSFM and Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tspulse/classification/README.md Install the library from the repository and required dependencies from the specified requirements file. ```bash pip install git+https://github.com/ibm-granite/granite-tsfm.git@v0.2.28 pip install -r ../tspulse_repro_libs.txt ``` -------------------------------- ### Start scheduler Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/research_use/ttm-r2_freq_benchmarking_512_96.ipynb Logs the addition of a job to the job store and indicates that the scheduler has started. ```python INFO:p-849332:t-23083607622400:base.py:_real_add_job:Added job "EmissionsTracker._measure_power" to job store "default" INFO:p-849332:t-23083607622400:base.py:start:Scheduler started ``` -------------------------------- ### Configure Optimizer and Trainer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial_with_ans.ipynb Setup the AdamW optimizer and OneCycleLR scheduler, then initialize the Trainer for model fine-tuning. ```python optimizer = AdamW(finetune_forecast_model.parameters(), lr=learning_rate) scheduler = OneCycleLR( optimizer, learning_rate, epochs=num_epochs, steps_per_epoch=math.ceil(len(train_dataset) / (batch_size)), ) finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback, tracking_callback], optimizers=(optimizer, scheduler), ) # Fine tune finetune_forecast_trainer.train() ``` -------------------------------- ### Install Demos Source: https://github.com/ibm-granite/granite-tsfm/blob/main/README.md Install the requirements for the demo presented at NeurIPS 2023. This requires pre-trained and finetuned models to be in place. ```bash pip install ".[demos]" ``` -------------------------------- ### Download and Extract Example Dataset Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/adaptive_conformal_tsad/README.md Downloads the TSB-AD-U dataset zip file and extracts its contents. It then copies a specific example dataset to the local `dataset/` folder for quick testing. ```bash # Download and extract TSB-AD-U dataset wget https://www.thedatum.org/datasets/TSB-AD-U.zip unzip TSB-AD-U.zip # Copy the example dataset to the dataset folder cp TSB-AD-U/672_YAHOO_id_122_WebService_tr_500_1st_857.csv dataset/ ``` -------------------------------- ### Install Kind Cluster with Local Registry Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/finetuning/README.md Installs a local Kubernetes cluster using kind and configures a local Docker registry. This is a prerequisite for deploying custom containerized workloads. ```zsh curl -s https://kind.sigs.k8s.io/examples/kind-with-registry.sh | bash ``` -------------------------------- ### Initialize and Train Forecasting Model Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb Initializes the Trainer with the model, arguments, datasets, callbacks, and optimizer/scheduler. Then, it starts the fine-tuning process. ```python finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback, tracking_callback], optimizers=(optimizer, scheduler), ) # Fine tune finetune_forecast_trainer.train() ``` -------------------------------- ### Run Local Unit Tests Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/finetuning/README.md Executes basic unit tests locally to verify the installation and setup. ```zsh make test_local ``` -------------------------------- ### Initialize and Train PatchTSTrainer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patch_tsmixer_transfer.ipynb Initializes the Trainer with the model, training arguments, datasets, and callbacks, then starts the pre-training process. ```python trainer = Trainer( model=model, args=training_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback], ) # pretrain trainer.train() ``` -------------------------------- ### Local Development Setup for Additional Handler Modules Source: https://github.com/ibm-granite/granite-tsfm/blob/main/SECURITY_PATCH-9mmf-qgpx-g4qx.ADDITIONAL_MODULES.md Set the TSFM_ADDITIONAL_HANDLER_MODULES environment variable in your shell or .env file for local development, then start the service. ```bash # In your shell or .env file export TSFM_ADDITIONAL_HANDLER_MODULES="mycompany.models.,custom.handlers." # Start the service python -m tsfminference.main ``` -------------------------------- ### Zero-Shot/Few-Shot TTM Model Execution Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/ttm-r2_benchmarking_1536_96.ipynb This snippet shows the setup and execution of zero-shot and few-shot fine-tuning for a TTM model. It includes model loading, dataset configuration, and the start of the fine-tuning process. ```python ==================================================================================================== Running zero-shot/few-shot for TTM-1536 on dataset = etth1, forecast_len = 96 Model will be loaded from ibm-granite/granite-timeseries-ttm-r2 ``` ```python -------------------- Running few-shot 5% -------------------- Number of params before freezing backbone 3081120 Number of params after freezing the backbone 1054560 ``` -------------------------------- ### Initialize Data Loaders, Model, and Optimizer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/LongHorizonForecast.ipynb Sets up the training data loader, neural network model, loss function, and optimizer. Ensure PyTorch and necessary data structures are imported. ```python HIDDEN_UNITS = 16 EPOCHS = 25 torch.manual_seed(42) tr_data = Data(X_train, y_train) tr_loader = DataLoader(dataset=tr_data, batch_size=64, shuffle=True) nn_model = CreateModel(CONTEXT_LENGTH, HIDDEN_UNITS, FORECAST_LENGTH) criterion = nn.MSELoss() optimizer = torch.optim.Adam(nn_model.parameters(), lr=0.001) ``` -------------------------------- ### Install granite-tsfm Package Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/adaptive_conformal_tsad/README.md Installs the core granite-tsfm library for Time Series Foundation Models. This is a mandatory installation. ```bash pip install granite-tsfm ``` -------------------------------- ### Install granite-tsfm Library Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/ttm_getting_started.ipynb Install the `granite-tsfm` library with notebook support. This is mandatory for Google Colab and optional for local runs if already installed. ```python # Install the tsfm library ! pip install "granite-tsfm[notebooks] @ git+https://github.com/ibm-granite/granite-tsfm.git@v0.2.22" ``` -------------------------------- ### Configure and Execute Training Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_with_exog_tutorial.ipynb Sets up TrainingArguments, callbacks, and the Trainer instance to perform the fine-tuning process. ```python print(f"Using learning rate = {learning_rate}") finetune_forecast_args = TrainingArguments( output_dir=os.path.join(OUT_DIR, "output"), overwrite_output_dir=True, learning_rate=learning_rate, num_train_epochs=num_epochs, do_eval=True, evaluation_strategy="epoch", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, dataloader_num_workers=8, report_to=None, save_strategy="epoch", logging_strategy="epoch", save_total_limit=1, logging_dir=os.path.join(OUT_DIR, "logs"), # Make sure to specify a logging directory load_best_model_at_end=True, # Load the best model when training ends metric_for_best_model="eval_loss", # Metric to monitor for early stopping greater_is_better=False, # For loss ) # Create the early stopping callback early_stopping_callback = EarlyStoppingCallback( early_stopping_patience=10, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.0, # Minimum improvement required to consider as improvement ) tracking_callback = TrackingCallback() # Optimizer and scheduler optimizer = AdamW(finetune_forecast_model.parameters(), lr=learning_rate) scheduler = OneCycleLR( optimizer, learning_rate, epochs=num_epochs, steps_per_epoch=math.ceil(len(train_dataset) / (batch_size)), ) finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback, tracking_callback], optimizers=(optimizer, scheduler), ) # Fine tune finetune_forecast_trainer.train() ``` -------------------------------- ### Instantiate and Run TimeSeriesForecastingPipeline Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/ttm_r3_getting_started.ipynb Create a `TimeSeriesForecastingPipeline` instance, configuring it with the loaded model, dataset column information, and forecasting parameters. Then, pass the prepared data to the pipeline to generate forecasts. ```python pipe = TimeSeriesForecastingPipeline( model=model, id_columns=[], timestamp_column=TIMESTAMP_COLUMN, target_columns=TARGET_COLUMNS, context_length=CONTEXT_LENGTH, prediction_length=PREDICTION_LENGTH, batch_size=BATCH_SIZE, impute_method=None, device=DEVICE, quantile_levels=[0.1, 0.5, 0.9], ) forecast_pipe = pipe(all_data) ``` -------------------------------- ### Install Project Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tspulse/anomaly_detection/README.md Installs all necessary Python packages for the project using a requirements file. Ensure Python 3.11 or higher is installed. ```bash pip install -r granite-tsfm/notebooks/hfdemo/tspulse/tspulse_repro_libs.txt ``` -------------------------------- ### Prepare Data and Preprocessor Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patch_tsmixer_blog.ipynb Load the dataset, split it into subsets, and initialize the TimeSeriesPreprocessor. ```python data = pd.read_csv( dataset_path, parse_dates=[timestamp_column], ) train_data = select_by_index( data, id_columns=id_columns, start_index=train_start_index, end_index=train_end_index, ) valid_data = select_by_index( data, id_columns=id_columns, start_index=valid_start_index, end_index=valid_end_index, ) test_data = select_by_index( data, id_columns=id_columns, start_index=test_start_index, end_index=test_end_index, ) tsp = TimeSeriesPreprocessor( timestamp_column=timestamp_column, id_columns=id_columns, target_columns=forecast_columns, scaling=True, ) tsp = tsp.train(train_data) ``` -------------------------------- ### Configure Trainer and Evaluate Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patch_tsmixer_blog.ipynb Set up training arguments, early stopping, and evaluate the model on the test dataset. ```python finetune_forecast_args = TrainingArguments( output_dir="./checkpoint/patchtsmixer/transfer/finetune/output/", overwrite_output_dir=True, learning_rate=0.0001, num_train_epochs=100, do_eval=True, evaluation_strategy="epoch", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, dataloader_num_workers=num_workers, report_to="tensorboard", save_strategy="epoch", logging_strategy="epoch", save_total_limit=3, logging_dir="./checkpoint/patchtsmixer/transfer/finetune/logs/", # Make sure to specify a logging directory load_best_model_at_end=True, # Load the best model when training ends metric_for_best_model="eval_loss", # Metric to monitor for early stopping greater_is_better=False, # For loss ) # Create a new early stopping callback with faster convergence properties early_stopping_callback = EarlyStoppingCallback( early_stopping_patience=5, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.001, # Minimum improvement required to consider as improvement ) finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback], ) print("\n\nDoing zero-shot forecasting on target data") result = finetune_forecast_trainer.evaluate(test_dataset) print("Target data zero-shot forecasting result:") print(result) ``` -------------------------------- ### Install tsfm library Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_with_exog_tutorial.ipynb Install the required tsfm library from the git repository. ```python # Install the tsfm library # ! pip install "tsfm_public[notebooks] @ git+ssh://git@github.com/ibm-granite/granite-tsfm.git" ``` -------------------------------- ### Set Up Training Arguments and Trainer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patch_tsmixer_getting_started.ipynb Configures training hyperparameters and initializes the Trainer with early stopping callbacks. ```python train_args = TrainingArguments( output_dir="./checkpoint/patchtsmixer/direct/train/output/", overwrite_output_dir=True, learning_rate=0.0001, num_train_epochs=100, do_eval=True, evaluation_strategy="epoch", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, dataloader_num_workers=num_workers, report_to="tensorboard", save_strategy="epoch", logging_strategy="epoch", save_total_limit=3, logging_dir="./checkpoint/patchtsmixer/direct/train/logs/", # Make sure to specify a logging directory load_best_model_at_end=True, # Load the best model when training ends metric_for_best_model="eval_loss", # Metric to monitor for early stopping greater_is_better=False, # For loss label_names=["future_values"], ) # Create a new early stopping callback with faster convergence properties early_stopping_callback = EarlyStoppingCallback( early_stopping_patience=5, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.001, # Minimum improvement required to consider as improvement ) trainer = Trainer( model=model, args=train_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback], ) print("\n\nDoing forecasting training on Etth1/train") trainer.train() ``` -------------------------------- ### Install tsfm library Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial.ipynb Install the required tsfm_public library from the GitHub repository. ```bash ! pip install "tsfm_public[notebooks] @ git+https://github.com/ibm-granite/granite-tsfm.git@v0.2.18" ``` -------------------------------- ### Environment Setup and Dataset Loading Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/gift_leaderboard_ttm_r3_nc/ttm_r3.ipynb Initializes the random seed, loads environment variables, and prepares dataset properties and names. It also identifies datasets to be processed. ```python # ------------------------- # Cell 5: Environment, FS mode, datasets # ------------------------- set_seed(SEED) load_dotenv() FS_MODE = None if getattr(args, "few_shot_data_limit_config", None): with open(args.few_shot_data_limit_config, "r") as f: FS_MODE = json.load(f) with open("resources/dataset_properties.json", "r") as f: dataset_properties_map = json.load(f) pretty_names = { "saugeenday": "saugeen", "temperature_rain_with_missing": "temperature_rain", "kdd_cup_2018_with_missing": "kdd_cup_2018", "car_parts_with_missing": "car_parts", } all_datasets = sorted(set(short_datasets.split() + med_long_datasets.split())) print("Datasets:", all_datasets) ``` -------------------------------- ### Configure Training Arguments and Trainer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial_with_ans.ipynb Set up training arguments, callbacks, and the trainer instance for the fine-tuning process. ```python print(f"Using learning rate = {learning_rate}") finetune_forecast_args = TrainingArguments( output_dir=os.path.join(OUT_DIR, "output"), overwrite_output_dir=True, learning_rate=learning_rate, num_train_epochs=num_epochs, do_eval=True, eval_strategy="epoch", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, dataloader_num_workers=8, report_to=None, save_strategy="epoch", logging_strategy="epoch", save_total_limit=1, logging_dir=os.path.join(OUT_DIR, "logs"), # Make sure to specify a logging directory load_best_model_at_end=True, # Load the best model when training ends metric_for_best_model="eval_loss", # Metric to monitor for early stopping greater_is_better=False, # For loss ) # Create the early stopping callback early_stopping_callback = EarlyStoppingCallback( early_stopping_patience=10, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.0, # Minimum improvement required to consider as improvement ) tracking_callback = TrackingCallback() # Optimizer and scheduler optimizer = AdamW(finetune_forecast_model.parameters(), lr=learning_rate) scheduler = OneCycleLR( optimizer, learning_rate, epochs=num_epochs, steps_per_epoch=math.ceil(len(train_dataset) / (batch_size)), ) finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback, tracking_callback], optimizers=(optimizer, scheduler), ) ``` -------------------------------- ### Few-Shot Forecasting Setup Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/ttm-r2_benchmarking_1536_96.ipynb Prepares datasets for few-shot learning by loading a specified fraction of training data. It also configures the model, potentially adjusting head dropout for specific datasets like 'ett'. ```python for fewshot_percent in [5]: # Set learning rate learning_rate = None # `None` value indicates that the optimal_lr_finder() will be used print("-" * 20, f"Running few-shot {fewshot_percent}%", "-" * 20) # Data prep: Get dataset dset_train, dset_val, dset_test = load_dataset( DATASET, context_length, forecast_length, fewshot_fraction=fewshot_percent / 100, dataset_root_path=DATA_ROOT_PATH, ) # change head dropout to 0.7 for ett datasets # change head dropout to 0.7 for ett datasets if "ett" in DATASET: finetune_forecast_model = get_model( hf_model_path, context_length=context_length, prediction_length=forecast_length, head_dropout=0.7 ) else: finetune_forecast_model = get_model( hf_model_path, context_length=context_length, prediction_length=forecast_length ) ``` -------------------------------- ### Install granite-tsfm repository Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/install_tsfm.ipynb Installs the tsfm_public package with notebook dependencies directly from the GitHub repository. ```python # Install ibm/tsfm ! pip install "tsfm_public[notebooks] @ git+https://github.com/ibm-granite/granite-tsfm.git@v0.2.12" ``` -------------------------------- ### Configuring Training and Finetuning Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tspulse_classification.ipynb Sets up training arguments, early stopping, optimizer, and scheduler for the Trainer API. ```python temp_dir = tempfile.mkdtemp() suggested_lr = None train_dict = {"per_device_train_batch_size": 32, "num_train_epochs": 200, "eval_accumulation_steps": None} EPOCHS = train_dict["num_train_epochs"] BATCH_SIZE = train_dict["per_device_train_batch_size"] eval_accumulation_steps = train_dict["eval_accumulation_steps"] NUM_WORKERS = 1 NUM_GPUS = 1 set_seed(42) if suggested_lr is None: lr, model = optimal_lr_finder( model, train_dataset, batch_size=BATCH_SIZE, ) suggested_lr = lr print("Suggested LR : ", suggested_lr) finetune_args = TrainingArguments( output_dir=temp_dir, overwrite_output_dir=True, learning_rate=suggested_lr, num_train_epochs=EPOCHS, do_eval=True, eval_strategy="epoch", per_device_train_batch_size=BATCH_SIZE, per_device_eval_batch_size=BATCH_SIZE, eval_accumulation_steps=eval_accumulation_steps, dataloader_num_workers=NUM_WORKERS, report_to="tensorboard", save_strategy="epoch", logging_strategy="epoch", save_total_limit=1, logging_dir=os.path.join(OUT_DIR, "output"), # Make sure to specify a logging directory load_best_model_at_end=True, # Load the best model when training ends metric_for_best_model="eval_loss", # Metric to monitor for early stopping greater_is_better=False, # For loss ) # Create the early stopping callback early_stopping_callback = EarlyStoppingCallback( early_stopping_patience=100, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.0001, # Minimum improvement required to consider as improvement ) # Optimizer and scheduler optimizer = AdamW(model.parameters(), lr=suggested_lr) scheduler = OneCycleLR( optimizer, suggested_lr, epochs=EPOCHS, steps_per_epoch=math.ceil(len(train_dataset) / (BATCH_SIZE * NUM_GPUS)), ) finetune_trainer = Trainer( model=model, args=finetune_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback], optimizers=(optimizer, scheduler), ) ``` -------------------------------- ### Install Required Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/DataLoadingAndStatistics.ipynb Installs necessary data science libraries if they are not already present in the environment. ```python %%capture try: import pandas # noqa: F401 except ImportError: !pip install pandas ``` ```python %%capture try: import numpy # noqa: F401 except ImportError: !pip install numpy ``` ```python %%capture try: import matplotlib # noqa: F401 except ImportError: !pip install matplotlib ``` ```python %%capture try: import sklearn # noqa: F401 except ImportError: !pip install scikit-learn ``` -------------------------------- ### Installing Faiss Dependency Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tspulse/search/README.md Installs the CPU version of the Faiss library required for search operations. ```bash pip install faiss-cpu ``` -------------------------------- ### Install granite-tsfm repository Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patchtst_fm/patchtst_fm.ipynb Clones the granite-tsfm repository and installs the required dependencies in the current environment. ```python import os import sys if not os.path.exists("granite-tsfm"): ! git clone git@github.com:ibm-granite/granite-tsfm.git %cd granite-tsfm ! pwd # Switch to the desired branch ! pip install ".[notebooks]" %cd .. else: print("Folder 'granite-tsfm' already exists. Skipping git clone.") sys.path.append("./granite-tsfm/notebooks/hfdemo/patchtst_fm/") ``` -------------------------------- ### Configure Training and Trainer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_channel_mix_finetuning.ipynb Sets up training arguments, callbacks, optimizer, and scheduler for the fine-tuning process. ```python print(f"Using learning rate = {learning_rate}") finetune_forecast_args = TrainingArguments( output_dir=os.path.join(OUT_DIR, "output"), overwrite_output_dir=True, learning_rate=learning_rate, num_train_epochs=num_epochs, do_eval=True, evaluation_strategy="epoch", per_device_train_batch_size=batch_size, per_device_eval_batch_size=batch_size, dataloader_num_workers=8, report_to=None, save_strategy="epoch", logging_strategy="epoch", save_total_limit=1, logging_dir=os.path.join(OUT_DIR, "logs"), # Make sure to specify a logging directory load_best_model_at_end=True, # Load the best model when training ends metric_for_best_model="eval_loss", # Metric to monitor for early stopping greater_is_better=False, # For loss ) # Create the early stopping callback early_stopping_callback = EarlyStoppingCallback( early_stopping_patience=10, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.0, # Minimum improvement required to consider as improvement ) tracking_callback = TrackingCallback() # Optimizer and scheduler optimizer = AdamW(finetune_forecast_model.parameters(), lr=learning_rate) scheduler = OneCycleLR( optimizer, learning_rate, epochs=num_epochs, steps_per_epoch=math.ceil(len(train_dataset) / (batch_size)), ) finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=train_dataset, eval_dataset=valid_dataset, callbacks=[early_stopping_callback, tracking_callback], optimizers=(optimizer, scheduler), ) ``` -------------------------------- ### Configure training parameters and paths Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/ttm-r2_benchmarking_512_96.ipynb Sets up random seed, model dimensions (context and forecast length), training epochs, number of workers, data root path, and output directory for benchmark results. ```python # Set seed SEED = 42 set_seed(SEED) # Specify model parameters context_length = 512 forecast_length = 96 freeze_backbone = True # Other args EPOCHS = 50 NUM_WORKERS = 16 # Make sure all the datasets in the following `list_datasets` are # saved in the `DATA_ROOT_PATH` folder. Or, change it accordingly. # Refer to the load_datasets() function # in notebooks/hfdemo/tinytimemixer/utils/ttm_utils.py # to see how it is used. DATA_ROOT_PATH = "/dccstor/tsfm23/datasets/" # This is where results will be saved OUT_DIR = f"ttm-r2_results_benchmark_{context_length}_{forecast_length}/" ``` -------------------------------- ### Confirm KServe Installation Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/inference/README.md Verifies that KServe has been successfully installed by checking for the creation of the default cluster storage container. ```bash # [snip]... clusterstoragecontainer.serving.kserve.io/default created 😀 Successfully installed KServe ``` -------------------------------- ### Configure model and data preprocessing Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/flowstate_getting_started_pipeline.ipynb Set up paths, model parameters, and data splitting configurations for the forecasting task. ```python # Path configurations out_dir = "./results" # Model configurations model_name = "ibm-granite/granite-timeseries-flowstate-r1" # Auxiliary configurations seed = 0 device = "cuda" if torch.cuda.is_available() else ("mps" if torch.mps.is_available() else "cpu") print(f"Device: {device}") dataset_path = "https://raw.githubusercontent.com/zhouhaoyi/ETDataset/main/ETT-small/ETTh1.csv" timestamp_column = "date" id_columns = [] # mention the ids that uniquely identify a time-series. target_columns = ["HUFL"] # , "HULL", "MUFL", "MULL", "LUFL", "LULL", "OT"] context_length = 512 prediction_length = 67 split_config = { "train": [0, 8640], "valid": [8640, 11520], "test": [ 11520, 14400, ], } data = pd.read_csv( dataset_path, parse_dates=[timestamp_column], ) train_df, valid_df, test_df = prepare_data_splits(data, context_length=context_length, split_config=split_config) tsp = TimeSeriesPreprocessor( target_columns=target_columns, timestamp_column=timestamp_column, id_columns=id_columns, context_length=context_length, prediction_length=prediction_length, scaling=True, encode_categorical=False, scaler_type="standard", ) tsp.train(train_df) batch_size = 16 ``` -------------------------------- ### Install and Import Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/StatisticalModels.ipynb Ensures required data science libraries are installed and imported for time series analysis. ```python %%capture try: import pandas # noqa: F401 except ImportError: !pip install pandas try: import numpy # noqa: F401 except ImportError: !pip install numpy try: import matplotlib # noqa: F401 except ImportError: !pip install matplotlib try: import sklearn # noqa: F401 except ImportError: !pip install scikit-learn try: import scipy # noqa: F401 except ImportError: !pip install scipy try: import statsmodels # noqa: F401 except ImportError: !pip install statsmodels ``` ```python import matplotlib.pyplot as plt import numpy as np import pandas as pd from sklearn.metrics import mean_squared_error as mse from sklearn.preprocessing import StandardScaler from statsmodels.tsa.arima.model import ARIMA ``` -------------------------------- ### Configure and Train Forecasting Model Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/ttm-r2_benchmarking_1024_96.ipynb Sets up the trainer with early stopping, optimizer, and scheduler, then executes the training process and evaluates the model. ```python early_stopping_patience=10, # Number of epochs with no improvement after which to stop early_stopping_threshold=0.0, # Minimum improvement required to consider as improvement ) tracking_callback = TrackingCallback() # Optimizer and scheduler optimizer = AdamW(finetune_forecast_model.parameters(), lr=learning_rate) scheduler = OneCycleLR( optimizer, learning_rate, epochs=EPOCHS, steps_per_epoch=math.ceil(len(dset_train) / (BATCH_SIZE)), ) finetune_forecast_trainer = Trainer( model=finetune_forecast_model, args=finetune_forecast_args, train_dataset=dset_train, eval_dataset=dset_val, callbacks=[early_stopping_callback, tracking_callback], optimizers=(optimizer, scheduler), ) finetune_forecast_trainer.remove_callback(INTEGRATION_TO_CALLBACK["codecarbon"]) # Fine tune finetune_forecast_trainer.train() # Evaluation print( "+" * 20, f"Test MSE after few-shot {fewshot_percent}% fine-tuning", "+" * 20, ) fewshot_output = finetune_forecast_trainer.evaluate(dset_test) print(fewshot_output) print("+" * 60) # Plot plot_predictions( model=finetune_forecast_trainer.model, dset=dset_test, plot_dir=SUBDIR, num_plots=10, plot_prefix=f"test_fewshot_{fewshot_percent}", channel=0, ) plt.close() # write results all_results[f"fs{fewshot_percent}_mse"].append(fewshot_output["eval_loss"]) all_results[f"fs{fewshot_percent}_mean_epoch_time"].append(tracking_callback.mean_epoch_time) all_results[f"fs{fewshot_percent}_total_train_time"].append(tracking_callback.total_train_time) all_results[f"fs{fewshot_percent}_best_val_metric"].append(tracking_callback.best_eval_metric) df_out = pd.DataFrame(all_results).round(3) print(df_out[["dataset", "zs_mse", "fs5_mse"]]) df_out.to_csv(f"{OUT_DIR}/results_zero_few.csv") df_out.to_csv(f"{OUT_DIR}/results_zero_few.csv") ``` -------------------------------- ### Scheduler Job Addition and Start Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/research_use/ttm-r2_freq_benchmarking_1024_96.ipynb This log indicates that a job has been added to the scheduler's job store and that the scheduler has started. ```text INFO:p-854016:t-23192246899456:base.py:_real_add_job:Added job "EmissionsTracker._measure_power" to job store "default" INFO:p-854016:t-23192246899456:base.py:start:Scheduler started ``` -------------------------------- ### Model Loading Logs Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/gift_leaderboard/ttm.ipynb Sample console output showing the model loading process and TTM configuration details. ```text INFO:p-1751135:t-22410562855680:get_model.py:get_model:Loading model from: ibm-granite/granite-timeseries-ttm-r2 WARNING:p-1751135:t-22410562855680:get_model.py:get_model:Requested `prediction_length` (12) is not exactly equal to any of the available TTM prediction lengths. Hence, TTM will forecast using the `prediction_filter_length` argument to provide the requested prediction length. Check the model card to know more about the supported context lengths and forecast/prediction lengths. INFO:p-1751135:t-22410562855680:get_model.py:get_model:Model loaded successfully from ibm-granite/granite-timeseries-ttm-r2, revision = 52-16-ft-l1-r2.1. INFO:p-1751135:t-22410562855680:get_model.py:get_model:[TTM] context_length = 52, prediction_length = 16 INFO:p-1751135:t-22410562855680:ttm_gluonts_predictor.py:_get_gift_model:The TTM has Prefix Tuning = True INFO:p-1751135:t-22410562855680:ttm_gluonts_predictor.py:train:Number of series: Train = 1, Valid = 1 ``` -------------------------------- ### Configure Environment and Paths Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/ttm_m4_hourly.ipynb Sets the random seed, defines data paths, and selects the compute device. ```python # Set seed for reproducibility SEED = 42 set_seed(SEED) # Save the M4 data files here M4_DATASET_PATH = "datasets/m4" # Model and results will be saved here M4_RESULTS_PATH = "ttm_finetuned_models" # M4 hourly official forecast horizon M4_FORECAST_HORIZON = 48 # Select device if torch.cuda.is_available(): device = torch.device("cuda") else: device = torch.device("cpu") ``` -------------------------------- ### Install tsfm Library Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_channel_mix_finetuning.ipynb Installs the tsfm library with notebook support from a Git repository. Ensure you have SSH access configured for the repository. ```python # Install the tsfm library # ! pip install "granite-tsfm[notebooks] @ git+ssh://git@github.com/ibm-granite/granite-tsfm.git" ``` -------------------------------- ### Configure benchmarking parameters and paths Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/sample_notebooks/ttm-r2_benchmarking_1024_96.ipynb Sets up random seed, model parameters like context and forecast lengths, training epochs, number of workers, and defines the root path for datasets and the output directory for results. ```python # Set seed SEED = 42 set_seed(SEED) # Specify model parameters context_length = 1024 forecast_length = 96 freeze_backbone = True # Other args EPOCHS = 50 NUM_WORKERS = 16 # Make sure all the datasets in the following `list_datasets` are # saved in the `DATA_ROOT_PATH` folder. Or, change it accordingly. # Refer to the load_datasets() function # in notebooks/hfdemo/tinytimemixer/utils/ttm_utils.py # to see how it is used. DATA_ROOT_PATH = "/dccstor/tsfm23/datasets/" # This is where results will be saved OUT_DIR = f"ttm-r2_results_benchmark_{context_length}_{forecast_length}/" ``` -------------------------------- ### Install TSFM Library and Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/ttm_forecasting_with_probabilistic_bounds.ipynb Installs the granite-tsfm library with notebook support and the wget utility. Ensure you have the correct version specified. ```bash ! pip install "granite-tsfm[notebooks]==0.3.2" ! pip install wget ``` -------------------------------- ### Clone and Install Evaluation Repo Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/gift_leaderboard_ttm_r3_nc/README.md Clones the 'gift-eval' repository and installs it in editable mode. This repository contains the necessary evaluation scripts and configurations. ```bash git clone https://github.com//gift-eval cd gift-eval pip install -e . cd .. ``` -------------------------------- ### Initialize Zeroshot Trainer Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial_with_ans.ipynb Sets up the Trainer for zeroshot evaluation using a specified model and training arguments. Requires a temporary directory for output. ```python temp_dir = tempfile.mkdtemp() # zeroshot_trainer zeroshot_trainer = Trainer( model=zeroshot_model, args=TrainingArguments( output_dir=temp_dir, per_device_eval_batch_size=64, ), ) ``` -------------------------------- ### Install Core Dependencies Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tinytimemixer/full_benchmarking/gift_leaderboard_ttm_r3_nc/README.md Installs the granite-tsfm package from its GitHub repository and other essential libraries like PyTorch and Transformers. Ensure your environment is activated. ```bash pip install git+https://github.com/ibm-granite/granite-tsfm.git@v0.2.28 pip install torch==2.4.0 "transformers>=4.44.0,<4.51.0" openpyxl ``` -------------------------------- ### Run Finetuning Experiments (Multivariate) Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/tspulse/anomaly_detection/README.md Execute finetuning experiments for multivariate anomaly detection. Navigate to the correct directory and specify dataset, evaluation file, mode, and output file. ```bash cd /notebooks/hfdemo/tspulse/anomaly_detection $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Tuning.csv" --mode "time" --out_file "benchmarks/TSB-AD-M-FT-Tuning-time.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Tuning.csv" --mode "fft" --out_file "benchmarks/TSB-AD-M-FT-Tuning-fft.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Tuning.csv" --mode "forecast" --out_file "benchmarks/TSB-AD-M-FT-Tuning-future.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Tuning.csv" --mode "time+fft+forecast" --out_file "benchmarks/TSB-AD-M-FT-Tuning-ebsemble.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Eva.csv" --mode "time" --out_file "benchmarks/TSB-AD-M-FT-Eva-time.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Eva.csv" --mode "fft" --out_file "benchmarks/TSB-AD-M-FT-Eva-fft.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Eva.csv" --mode "forecast" --out_file "benchmarks/TSB-AD-M-FT-Eva-future.csv" --finetune --epochs 10 --decoder common_channel ``` ```bash $ python run_experiment.py --data_direc "Datasets/TSB-AD-M" --eval_file "Datasets/File_List/TSB-AD-M-Eva.csv" --mode "time+fft+forecast" --out_file "benchmarks/TSB-AD-M-FT-Eva-ebsemble.csv" --finetune --epochs 10 --decoder common_channel ``` -------------------------------- ### Install tsfm Library Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/tutorial/ttm_tutorial_with_ans.ipynb Installs the tsfm library, including notebooks support, from a specific Git commit. Ensure you have the necessary permissions and environment. ```python # Install the tsfm library ! pip install "tsfm_public[notebooks] @ git+https://github.com/ibm-granite/granite-tsfm.git@v0.2.8" ``` -------------------------------- ### Install TSB-AD Evaluation Metrics Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/adaptive_conformal_tsad/README.md Installs the TSB-AD package, which is only required if you intend to use the advanced evaluation metrics provided in the `tsb_ad_evaluation.py` script. ```bash pip install TSB-AD ``` -------------------------------- ### Install TiRex Model Support Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/adaptive_conformal_tsad/README.md Installs optional dependencies required for TiRex model support. This is needed if you plan to use TiRex models. ```bash pip install tirex-ts ``` -------------------------------- ### Build Finetuning Image Source: https://github.com/ibm-granite/granite-tsfm/blob/main/services/finetuning/README.md Builds the TSFM Finetuning Docker image. Prefix with CONTAINER_BUILDER=podman if using podman. ```sh # If you are using podman, you must prefix the next command with # CONTAINER_BUILDER=podman # For finetuning, we only support building the GPU image CONTAINER_BUILDER=docker make image ``` -------------------------------- ### Prepare the preprocessor and model Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/ttm_forecasting_with_probabilistic_bounds.ipynb Initializes the TimeSeriesPreprocessor and loads a pre-trained TinyTimeMixer model. ```python tsp = TimeSeriesPreprocessor( timestamp_column=timestamp_column, target_columns=target_columns, context_length=context_length, prediction_length=prediction_length, scaling=True, encode_categorical=False, scaler_type="standard", ) ``` ```python TTM_MODEL_REVISION = "512-96-ft-r2.1" finetuned_model = TinyTimeMixerForPrediction.from_pretrained( "ibm-granite/granite-timeseries-ttm-r2", revision=TTM_MODEL_REVISION, num_input_channels=tsp.num_input_channels, ) ``` ```python tsp = tsp.train(train_df) ``` -------------------------------- ### Install Chronos Model Support Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/adaptive_conformal_tsad/README.md Installs optional dependencies required for Chronos model support. This is needed if you plan to use Chronos models. ```bash pip install chronos-forecasting ``` -------------------------------- ### Initialize Training Arguments Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/patchtsmixer_HF_blog.ipynb Set up the TrainingArguments for the model training process. ```python if PRETRAIN_AGAIN: training_args = TrainingArguments( output_dir="./checkpoint/patchtsmixer/electricity/pretrain/output/", ``` -------------------------------- ### Display Help for W1ACAS Source: https://github.com/ibm-granite/granite-tsfm/blob/main/notebooks/hfdemo/adaptive_conformal_tsad/README.md Use this command to view all available command-line options and their descriptions for the main_acas_w1.py script. ```bash python main_acas_w1.py --help ``` -------------------------------- ### Install Module or Add to Python Path Source: https://github.com/ibm-granite/granite-tsfm/blob/main/SECURITY_PATCH-9mmf-qgpx-g4qx.ADDITIONAL_MODULES.md Resolve 'Module not found' errors by ensuring the module is installed via pip or by adding its directory to the PYTHONPATH environment variable. ```bash pip install mycompany-models # or export PYTHONPATH=/path/to/mycompany:$PYTHONPATH ```