PIDSMaker

https://ubc-provenance.github.io/PIDSMaker/
Admin
PIDSMaker is an open-source framework designed to build, customize, and experiment with...
Tokens:30,591
Snippets:268
Trust Score:9.7
Update:1 week ago
Show doc for...
Tokens:
Raw
### Build and start the database container

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Builds and starts the PostgreSQL container using docker compose. Remember to source .env after modifying variables.

```bash
docker compose -p postgres -f compose-postgres.yml up -d --build
```

--------------------------------

### Start containers and enter pidsmaker shell

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Starts the postgres and pidsmaker containers and opens a shell inside the pidsmaker container to verify that the databases are built and filled.

```bash
docker compose -p postgres -f compose-postgres.yml up -d --build
docker compose -f compose-pidsmaker.yml up -d --build
docker compose exec pids bash

```

--------------------------------

### Run the visualizer server: pidsmaker.vizgen.web.viz_server

Source: https://ubc-provenance.github.io/PIDSMaker/features/visualization

Starts the Flask server inside the container. Open http://127.0.0.1:5000 afterwards. The last viewed run loads automatically; use the Run Browser to pick another. If localhost is refused but 127.0.0.1 works, bind to IPv4 only; start with --host :: to serve both.

```shell
python -m pidsmaker.vizgen.web.viz_server

```

--------------------------------

### Set up database with Makefile

Source: https://ubc-provenance.github.io/PIDSMaker/singularity_install

Run the full setup from the scripts/apptainer directory. This downloads and runs a PostgreSQL container via Singularity/Apptainer and loads database dumps using load_dumps.sh.

```bash
cd ./scripts/apptainer
make full-setup
```

--------------------------------

### Install NVIDIA Container Toolkit for Docker CUDA support

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Adds the NVIDIA package repository, installs the nvidia-container-toolkit, and restarts Docker services to enable GPU support.

```bash
# Add the NVIDIA package repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Update and install
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Restart services
sudo systemctl restart docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```

--------------------------------

### Install dependencies with conda

Source: https://ubc-provenance.github.io/PIDSMaker/singularity_install

Create a conda environment from the provided environment.yml file and activate it.

```bash
conda env create -f ./scripts/apptainer/environment.yml
conda activate pids
```

--------------------------------

### Manage container with make down and make up

Source: https://ubc-provenance.github.io/PIDSMaker/singularity_install

Stop the container with 'make down' and start it again with 'make up'.

```bash
make down
```

```bash
make up
```

--------------------------------

### Cross-dataset tuning with --tuning_file_path

Source: https://ubc-provenance.github.io/PIDSMaker/features/tuning

Commands to apply the same tuning configuration across all datasets. The first runs on all datasets, the second on a single dataset. The path passed to --tuning_file_path must start from systems/.

```bash
./run_all_datasets.py my_system \
    --tuning_mode=hyperparameters \
    --tuning_file_path=systems/default/tuning_my_system_all_datasets

```

```bash
./run.sh my_system CLEARSCOPE_E3 \
    --tuning_mode=hyperparameters \
    --tuning_file_path=systems/default/tuning_my_system_all_datasets

```

--------------------------------

### Install gdown for dataset download

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Installs gdown, the Google Drive download utility used to fetch the official public DARPA dataset files.

```bash
pip install gdown

```

--------------------------------

### Run the framework

Source: https://ubc-provenance.github.io/PIDSMaker/singularity_install

Execute the main script with the required arguments: SYSTEM, DATASET, and artifact directory, specifying the database host as localhost.

```bash
python pidsmaker/main.py SYSTEM DATASET --artifact_dir ./artifacts/ --database_host localhost
```

--------------------------------

### Start and enter PIDSMaker container manually

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Starts the PIDSMaker container in detached mode with build and opens a bash shell inside it. Use this manual approach instead of the VSCode Devcontainer extension.

```bash
docker compose -f compose-pidsmaker.yml up -d --build
docker compose exec pids bash

```

--------------------------------

### Launch hyperparameter tuning with --tuning_mode=hyperparameters

Source: https://ubc-provenance.github.io/PIDSMaker/features/tuning

Launches hyperparameter tuning for my_system on the CADETS_E3 dataset. Requires prior `wandb login` in the container's shell. Automatically loads the tuning configuration based on dataset and system names, falling back to the default tuning configuration if no dataset-specific file exists.

```bash
./run.sh my_system CADETS_E3 --tuning_mode=hyperparameters

```

--------------------------------

### Create database for OpTC datasets (create_database_optc.py)

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Fills the database for a DARPA OpTC dataset (optc_h051, optc_h201, optc_h501). Must be run inside the pidsmaker container shell.

```bash
python dataset_preprocessing/optc/create_database_optc.py orthrus DATASET

```

--------------------------------

### Get a shell into the postgres container

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Opens a bash shell inside the running postgres container.

```bash
docker compose -p postgres exec postgres bash
```

--------------------------------

### Configure run_n_times experiment in run_n_times.yml

Source: https://ubc-provenance.github.io/PIDSMaker/features/instability

Configuration for the `run_n_times` experiment, defined in `config/experiments/uncertainty/run_n_times.yml`. `iterations` sets the number of runs (default 5), and `restart_from` starts each iteration from the featurization stage, reusing earlier stages like construction.

```yaml
training_loop:
  run_evaluation: each_epoch

experiment:
  used_method: uncertainty
  uncertainty:
    deep_ensemble:
      iterations: 5        # number of runs
      restart_from: featurization  # task to restart from

```

--------------------------------

### Download datasets via CLI with download_datasets.sh

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Downloads pre-processed DARPA TC/OpTC database dumps from the command line. Requires a Google API access token obtained via the OAuth 2.0 Playground. Use lowercase dataset names, either as a comma-separated list or 'all' for every dataset.

```bash
# Example 1: Comma-separated datasets
./download_datasets.sh cadets_e3,optc_h201,clearscope_e5 YOUR_ACCESS_TOKEN

# Example 2: All datasets
./download_datasets.sh all YOUR_ACCESS_TOKEN

```

--------------------------------

### Run PIDSmaker on DARPA OpTC optc_h051

Source: https://ubc-provenance.github.io/PIDSMaker/datasets

Executes PIDSmaker on the optc_h051 dataset, which includes a malicious Notepad++ update installing Meterpreter, SYSTEM escalation, LSASS migration for Mimikatz, persistence, timestomping, and RDP admin account creation.

```bash
python pidsmaker/main.py SYSTEM optc_h051
```

--------------------------------

### Copy .env.local to .env

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Creates the .env file from the local template. Set INPUT_DIR to the data folder path and optionally ARTIFACTS_DIR for generated files.

```bash
cp .env.local .env
```

--------------------------------

### Run PIDSMaker with W&B monitoring

Source: https://ubc-provenance.github.io/PIDSMaker/introduction

Run the framework in the shell with W&B monitoring.

```bash
python pidsmaker/main.py SYSTEM DATASET --wandb

```

--------------------------------

### Define dataset configuration in pidsmaker/config/config.py

Source: https://ubc-provenance.github.io/PIDSMaker/datasets

Add a new dataset by defining its configuration in the DATASET_DEFAULT_CONFIG dictionary. The configuration includes raw directory, database names, number of node and edge types, start and end dates, and lists of dates for training, validation, testing, and unused dates. It also specifies the ground truth relative path and attack-to-time-window mappings.

```python
DATASET_DEFAULT_CONFIG = {
    "MY_DATASET": {
        "raw_dir": "",
        "database": "my_database_name",
        "database_all_file": "my_database_name",
        "num_node_types": 3, # Number of node types in the dataset __format__ (i.e., in pidsmaker/utils/dataset_utils.py)
        "num_edge_types": 10, # Number of edge types in the dataset __format__ (i.e., in pidsmaker/utils/dataset_utils.py)
        "start_date": "2018-04-02", # Start date (Inclusive)
        "end_date": "2018-04-14", # End date (Exclusive)
        "train_dates": [
            # Dates/graphs used for training (i.e., benign activity)
            "2018-04-02",
            "2018-04-03",
            "2018-04-04",
            "2018-04-05",
            "2018-04-07",
            "2018-04-08",
            "2018-04-09",
        ],
        "val_dates": [
            # Dates/graphs used for validation/threshold calibration (i.e., benign activity)
            "2018-04-10"
        ],
        "test_dates": [
            # Dates/graphs used for testing (i.e., contains both benign and attack activity)
            "2018-04-06",
            "2018-04-11",
            "2018-04-12",
            "2018-04-13"
        ],
        "unused_dates": [
            # Any unused dates/graphs that should be ignored
             "2018-04-14"
        ],
        "ground_truth_relative_path": ["MY_DATASET/labels.csv"],
        "attack_to_time_window": [
            ["MY_DATASET/labels.csv", "2018-04-11 10:00:00", "2018-04-12 12:00:00"],
        ],
    },
}

```

--------------------------------

### Format all files with pre-commit

Source: https://ubc-provenance.github.io/PIDSMaker/contributing

Run this command before opening a pull request to format all files with pre-commit.

```bash
pre-commit run --all-files

```

--------------------------------

### Configure custom_system run with batching, training, and encoder options

Source: https://ubc-provenance.github.io/PIDSMaker/tutorial

Three command-line invocations for configuring a custom system run. The first removes node type features to keep only the word2vec embedding. The second increases node embedding size. The third reduces the number of GNN layers.

```bash
./run.sh custom_system CADETS_E3 --project=test_custom_system \
    --batching.node_features=node_emb
```

```bash
./run.sh custom_system CADETS_E3 --project=test_custom_system \
    --training.node_hid_dim=256 \
    --training.node_out_dim=256
```

```bash
./run.sh custom_system CADETS_E3 --project=test_custom_system \
    --training.encoder.custom_encoder.num_layers=2
```

--------------------------------

### Restart pipeline from scratch in isolated folder (--restart_from_scratch flag)

Source: https://ubc-provenance.github.io/PIDSMaker/pipeline

Restarts the entire pipeline in an isolated environment by using a randomly generated hash for the output folder. All generated files are deleted at the end of the pipeline to avoid unnecessary disk usage.

```bash
python pidsmaker/main.py orthrus CADETS_E3 --restart_from_scratch

```

--------------------------------

### Load all database dumps at once

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Runs the load_dumps.sh script to load all datasets from the data folder if enough space is available.

```bash
./scripts/load_dumps.sh
```

--------------------------------

### Create database for TC E5 datasets (create_database_e5.py)

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Fills the database for a DARPA TC E5 dataset (CLEARSCOPE_E5, CADETS_E5, THEIA_E5). Must be run inside the pidsmaker container shell. Large storage capacity is needed; a single run can generate more than 15GB of artifact files on E3 datasets, and much more with E5 datasets.

```bash
python dataset_preprocessing/darpa_tc/create_database_e5.py orthrus DATASET

```

--------------------------------

### Run pretrained_supervised on THEIA_E3 with custom attack edges

Source: https://ubc-provenance.github.io/PIDSMaker/features/supervised

Run the supervised objective on THEIA_E3, pointing attack_edges_path to its YAML file. The backslash continues the command on the next line.

```bash
python pidsmaker/main.py pretrained_supervised THEIA_E3 --featurization.pretrained.weights_path=/home/pids/weights/spider \
    --training.decoder.predict_edge_supervised.attack_edges_path=config/attack_edges/theia_e3.yml
```

--------------------------------

### Run run_n_times with --training.stable_optim=True

Source: https://ubc-provenance.github.io/PIDSMaker/features/instability

Enables stable optimization (`training.stable_optim: True`) to reduce variance between runs, then executes the `run_n_times` experiment. Use this to measure the effect on the `*_std` metrics. `training.weight_decay` is ignored when this is enabled.

```bash
./run.sh orthrus CADETS_E3 --training.stable_optim=True --experiment=run_n_times

```

--------------------------------

### Log in to Weights & Biases CLI

Source: https://ubc-provenance.github.io/PIDSMaker/ten-minute-install

Logs into Weights & Biases from the CLI by pasting your API key. Required to use the W&B interface for visualizing and historizing experiments.

```bash
wandb login

```

--------------------------------

### Run pidsmaker/main.py to verify dataset

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Runs the PIDSMaker main tool inside the pidsmaker container to verify that the database is built and filled with data.

```bash
python pidsmaker/main.py orthrus DATASET

```

--------------------------------

### Run consistency with SPIDER weights (pretrained_velox)

Source: https://ubc-provenance.github.io/PIDSMaker/features/pretrained_encoders

Runs the pretrained_velox pipeline on CADETS_E3 with the SPIDER weights folder. Adjust the weights path to the actual container location, which is /home/pids/weights/spider after the download and extraction steps.

```bash
python pidsmaker/main.py pretrained_velox CADETS_E3 --featurization.pretrained.weights_path=/home/pids/weights/spider
```

--------------------------------

### Create DATA_FOLDER directory

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Creates the DATA_FOLDER directory that will hold all raw dataset files. Make sure the folder is empty before downloading and parsing raw data, and remove all old files before downloading a new dataset.

```bash
DATA_FOLDER=./data
mkdir ${DATA_FOLDER}

```

--------------------------------

### Create database for TC E3 datasets (create_database_e3.py)

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Fills the database for a DARPA TC E3 dataset (CLEARSCOPE_E3, CADETS_E3, THEIA_E3). Must be run inside the pidsmaker container shell. Large storage capacity is needed; a single run can generate more than 15GB of artifact files on E3 datasets.

```bash
python dataset_preprocessing/darpa_tc/create_database_e3.py orthrus DATASET

```

--------------------------------

### Run tuned system with --tuned

Source: https://ubc-provenance.github.io/PIDSMaker/features/tuning

Runs a system with its best hyperparameters on a particular dataset. Searches for config/tuned_baselines/cadets_e3/tuned_my_system.yml and overrides the default system config with the best hyperparameters.

```bash
./run.sh my_system CADETS_E3 --tuned

```

--------------------------------

### Kairos training command for optc_h201

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs Kairos on the optc_h201 dataset with dropout 0.3, learning rate 0.001, node hidden/out dim 64, 12 epochs, and embedding dim 32.

```bash
python pidsmaker/main.py kairos optc_h201 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=64 --training.node_out_dim=64 --training.num_epochs=12 --featurization.emb_dim=32
```

--------------------------------

### NodLink training command for optc_h201

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs NodLink on the optc_h201 dataset with dropout 0.3, learning rate 0.0001, node hidden/out dim 320, 12 epochs, and embedding dim 128.

```bash
python pidsmaker/main.py nodlink optc_h201 --training.encoder.dropout=0.3 --training.lr=0.0001 --training.node_hid_dim=320 --training.node_out_dim=320 --training.num_epochs=12 --featurization.emb_dim=128
```

--------------------------------

### NodLink training command for CLEARSCOPE_E5

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs NodLink on the CLEARSCOPE_E5 dataset with dropout 0.3, learning rate 0.0001, node hidden/out dim 128, 12 epochs, and embedding dim 256.

```bash
python pidsmaker/main.py nodlink CLEARSCOPE_E5 --training.encoder.dropout=0.3 --training.lr=0.0001 --training.node_hid_dim=128 --training.node_out_dim=128 --training.num_epochs=12 --featurization.emb_dim=256
```

--------------------------------

### Run PIDSMaker with arguments

Source: https://ubc-provenance.github.io/PIDSMaker/introduction

Basic command to run the PIDSMaker framework. SYSTEM must match a YML file in config/ and DATASET must be defined in DATASET_DEFAULT_CONFIG in config/config.py.

```bash
python pidsmaker/main.py SYSTEM DATASET --arg1=x --arg2=y

```

--------------------------------

### Kairos training command for CLEARSCOPE_E5

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs Kairos on the CLEARSCOPE_E5 dataset with dropout 0.3, learning rate 0.001, node hidden/out dim 64, 12 epochs, and embedding dim 32.

```bash
python pidsmaker/main.py kairos CLEARSCOPE_E5 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=64 --training.node_out_dim=64 --training.num_epochs=12 --featurization.emb_dim=32
```

--------------------------------

### Run Velox on CLEARSCOPE_E5 with tuned hyperparameters

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Command to run the Velox system on the CLEARSCOPE_E5 dataset using the best hyperparameters from grid search. Uses dropout 0.3, learning rate 0.001, hidden and output dimensions 128, 12 epochs, and embedding dimension 128.

```bash
python pidsmaker/main.py velox CLEARSCOPE_E5 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=128 --training.node_out_dim=128 --training.num_epochs=12 --featurization.emb_dim=128
```

--------------------------------

### NodLink training command for optc_h501

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs NodLink on the optc_h501 dataset with dropout 0.3, learning rate 0.0001, node hidden/out dim 128, 12 epochs, and embedding dim 256.

```bash
python pidsmaker/main.py nodlink optc_h501 --training.encoder.dropout=0.3 --training.lr=0.0001 --training.node_hid_dim=128 --training.node_out_dim=128 --training.num_epochs=12 --featurization.emb_dim=256
```

--------------------------------

### Run pipeline multiple times with --experiment=run_n_times

Source: https://ubc-provenance.github.io/PIDSMaker/features/instability

Runs the pipeline N times with the same configuration, each run using a different random seed. Requires the `--experiment=run_n_times` flag.

```bash
./run.sh orthrus CADETS_E3 --experiment=run_n_times

```

--------------------------------

### Run Velox on optc_h201 with tuned hyperparameters

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Command to run the Velox system on the optc_h201 dataset using the best hyperparameters from grid search. Uses dropout 0.3, learning rate 0.001, hidden and output dimensions 128, 12 epochs, and embedding dimension 256.

```bash
python pidsmaker/main.py velox optc_h201 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=128 --training.node_out_dim=128 --training.num_epochs=12 --featurization.emb_dim=256
```

--------------------------------

### NodLink training command for optc_h051

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs NodLink on the optc_h051 dataset with dropout 0.3, learning rate 0.0001, node hidden/out dim 256, 12 epochs, and embedding dim 256.

```bash
python pidsmaker/main.py nodlink optc_h051 --training.encoder.dropout=0.3 --training.lr=0.0001 --training.node_hid_dim=256 --training.node_out_dim=256 --training.num_epochs=12 --featurization.emb_dim=256
```

--------------------------------

### Kairos training command for optc_h051

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs Kairos on the optc_h051 dataset with dropout 0.3, learning rate 0.001, node hidden/out dim 128, 12 epochs, and embedding dim 32.

```bash
python pidsmaker/main.py kairos optc_h051 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=128 --training.node_out_dim=128 --training.num_epochs=12 --featurization.emb_dim=32
```

--------------------------------

### PIDSMaker project structure (config/ and pidsmaker/ layout)

Source: https://ubc-provenance.github.io/PIDSMaker/pipeline

Directory layout showing where configuration files and the main entry point reside. The pidsmaker/main.py file is the sole entry point, and each tasks/*.py module corresponds to one pipeline stage from construction through triage.

```plaintext
config/                         # existing systems are defined by their own YML file
├── orthrus.yml
├── kairos.yml
└── ...
pidsmaker/
├── main.py                     # only entry point of the framework
│── config/
│   ├── config.py               # available arguments to use in YML files
│   └── pipeline.py             # pipeline code
├── tasks/                      
│   ├── construction.py         # stage 1. parse raw provenance + graph construction
│   ├── transformation.py       # stage 2. graph transformations
│   ├── featurization.py        # stage 3. text embedding featurization (Word2Vec, Doc2Vec, ...)
│   ├── batching.py             # stage 4. batch construction, neighbor sampling, etc
│   ├── training.py             # stage 5. GNN training + inference loop
│   ├── evaluation.py           # stage 6. metrics calculation + plots
│   └── triage.py               # stage 7. optional post-processing attack tracing
```

--------------------------------

### Run Velox on CLEARSCOPE_E3 with tuned hyperparameters

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Command to run the Velox system on the CLEARSCOPE_E3 dataset using the best hyperparameters from grid search. Uses dropout 0.3, learning rate 0.001, hidden and output dimensions 64, 12 epochs, embedding dimension 128, and time window size 1.

```bash
python pidsmaker/main.py velox CLEARSCOPE_E3 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=64 --training.node_out_dim=64 --training.num_epochs=12 --featurization.emb_dim=128 --construction.time_window_size=1
```

--------------------------------

### Enable viz pre-saving during training: --save_for_viz

Source: https://ubc-provenance.github.io/PIDSMaker/features/visualization

Opt in to pre-saving extra artifacts during training for the fastest, most complete viz generation. These artifacts (per-epoch model checkpoints and a cached copy of test graphs) are large and off by default. Without it, the featurization space works but the GNN encoder space needs the checkpoints (skipped with a warning if absent), and test graphs are recomputed on demand.

```shell
python pidsmaker/main.py SYSTEM DATASET --save_for_viz

```

--------------------------------

### Kairos training command for optc_h501

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs Kairos on the optc_h501 dataset with dropout 0.3, learning rate 0.001, node hidden/out dim 64, 12 epochs, and embedding dim 16.

```bash
python pidsmaker/main.py kairos optc_h501 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=64 --training.node_out_dim=64 --training.num_epochs=12 --featurization.emb_dim=16
```

--------------------------------

### NodLink training command for CLEARSCOPE_E3

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Runs NodLink on the CLEARSCOPE_E3 dataset with dropout 0.3, learning rate 0.0001, node hidden/out dim 64, 12 epochs, embedding dim 128, and time window 1.

```bash
python pidsmaker/main.py nodlink CLEARSCOPE_E3 --training.encoder.dropout=0.3 --training.lr=0.0001 --training.node_hid_dim=64 --training.node_out_dim=64 --training.num_epochs=12 --featurization.emb_dim=128 --construction.time_window_size=1
```

--------------------------------

### Download DARPA OpTC dataset using download_dataset.py

Source: https://ubc-provenance.github.io/PIDSMaker/create-db-from-scratch

Downloads raw DARPA OpTC dataset files into the specified DATA_FOLDER. DATASET can be optc_h051, optc_h201, or optc_h501. DATA_FOLDER must be the absolute path and must be empty before downloading.

```bash
python ./dataset_preprocessing/optc/download_dataset.py DATASET ${DATA_FOLDER}

```

--------------------------------

### Assign sweep name and project with --exp and --project

Source: https://ubc-provenance.github.io/PIDSMaker/features/tuning

Recommended command for better experiment historization: assigns a name to the sweep with --exp and a dedicated project name with --project.

```bash
./run.sh my_system CADETS_E3 \
    --tuning_mode=hyperparameters \
    --exp=bench_fasttext_word2vec \
    --project=best_featurization_method

```

--------------------------------

### Run PIDSMaker in background with W&B

Source: https://ubc-provenance.github.io/PIDSMaker/introduction

Run the framework in the background with W&B monitoring, ideal for multiple parallel runs. Logs can be watched with `tail -f nohup.out`.

```bash
./run.sh SYSTEM DATASET

```

--------------------------------

### Run Velox on optc_h501 with tuned hyperparameters

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Command to run the Velox system on the optc_h501 dataset using the best hyperparameters from grid search. Uses dropout 0.3, learning rate 0.001, hidden and output dimensions 320, 12 epochs, and embedding dimension 256.

```bash
python pidsmaker/main.py velox optc_h501 --training.encoder.dropout=0.3 --training.lr=0.001 --training.node_hid_dim=320 --training.node_out_dim=320 --training.num_epochs=12 --featurization.emb_dim=256
```

--------------------------------

### Run Velox on THEIA_E5 with tuned hyperparameters

Source: https://ubc-provenance.github.io/PIDSMaker/tuned_systems

Command to run the Velox system on the THEIA_E5 dataset using the best hyperparameters from grid search. Uses dropout 0.3, learning rate 0.0001, hidden and output dimensions 128, 12 epochs, embedding dimension 128, and time window size 15.0.

```bash
python pidsmaker/main.py velox THEIA_E5 --training.encoder.dropout=0.3 --training.lr=0.0001 --training.node_hid_dim=128 --training.node_out_dim=128 --training.num_epochs=12 --featurization.emb_dim=128 --construction.time_window_size=15.0
```
Tokens:
Raw