### Install Kiso Core

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Installs the base Kiso package using pip. This is the fundamental installation required for using Kiso.

```shell
pip3 install kiso
```

--------------------------------

### Provision Resources and Install Software with Kiso

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Provisions the necessary resources (e.g., VMs) and installs specified software (e.g., Docker) as defined in the Kiso experiment configuration file.

```shell
$ kiso up experiment.yml
```

--------------------------------

### Install Kiso with FABRIC Support

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Installs Kiso with dependencies for interacting with FABRIC. This enables Kiso to provision and manage resources on the FABRIC testbed.

```shell
pip3 install kiso[fabric]
```

--------------------------------

### Kiso Experiment Configuration (YAML)

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Defines a Kiso experiment using YAML. This example configures a local Vagrant environment, installs Docker, and sets up a shell experiment to print 'Hello, world!' to a file.

```yaml
sites:
  - kind: vagrant
    backend: virtualbox
    box: bento/rockylinux-9
    user: vagrant
    resources:
      machines:
        - labels:
            - submit
          backend: virtualbox
          box: bento/rockylinux-9
          user: vagrant
          flavour: "large"
          number: 1
      networks:
        - labels:
            - net
          cidr: "172.16.42.0/16"

software:
  docker:
    labels:
      - submit

experiments:
  - kind: shell
    name: hello-world
    description: Print a message
    scripts:
      - labels:
          - submit
        script: |
          #!/bin/bash
          echo "Hello, world!" > hello.txt
    outputs:
      - labels:
          - submit
        src: hello.txt
        dst: ./

```

--------------------------------

### Install Kiso with Vagrant Support

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Installs Kiso along with the necessary dependencies to manage experiments using Vagrant and VirtualBox. This enables local VM provisioning.

```shell
pip3 install kiso[vagrant]
```

--------------------------------

### Development Setup Commands

Source: https://github.com/pegasus-isi/kiso/blob/main/CLAUDE.md

Installs Kiso and its development dependencies, and sets up pre-commit hooks for code quality checks.

```shell
pip install -e ".[all]"
pre-commit install
```

--------------------------------

### Setup Kiso Development Environment

Source: https://github.com/pegasus-isi/kiso/blob/main/README.md

Instructions for cloning the repository and installing the necessary dependencies for local development and contribution.

```shell
git clone https://github.com/pegasus-isi/kiso.git
cd kiso
pip install -e ".[all]"
pre-commit install
```

--------------------------------

### Install Kiso with Chameleon Cloud Support

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Installs Kiso with dependencies for interacting with Chameleon Cloud. This allows Kiso to provision and manage resources on Chameleon.

```shell
pip3 install kiso[chameleon]
```

--------------------------------

### Install Pre-commit Hooks

Source: https://github.com/pegasus-isi/kiso/blob/main/CONTRIBUTING.md

Installs the pre-commit framework and configures hooks to ensure code quality standards are met before commits are accepted. Requires Python 3 and pip installed on the system.

```sh
cd kiso

# Install pre-commit
pip3 install -U pre-commit

# Install pre-commit hook
pre-commit install
```

--------------------------------

### Run Kiso Experiment and View Output

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Executes the defined experiment on the provisioned resources. After execution, this command shows how to view the output file generated by the experiment.

```shell
$ kiso run experiment.yml

# View the output
cat hello.txt
```

--------------------------------

### Registering Custom Software Installer in pyproject.toml

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Shows how to register a custom software installer with Kiso by defining an entry point in the `pyproject.toml` file. This makes the custom installer discoverable by Kiso. The entry point maps a name (`mysoftware`) to the Python class that implements the installer.

```toml
[project.entry-points."kiso.software"]
mysoftware = "my_software.installer:MySoftwareInstaller"

```

--------------------------------

### Example Deployment Configuration (YAML)

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

This YAML snippet illustrates how a deployment entry point, defined in setup.cfg or setup.py, is referenced in the experiment configuration. The 'mydeployment' key under the 'deployment' section links to the specific deployment configuration.

```yaml
sites:
    ...

deployment:
    mydeployment:
        version: 5.1.0
        ...

experiments:
    ...
```

--------------------------------

### Run Unit Tests with Tox

Source: https://github.com/pegasus-isi/kiso/blob/main/CONTRIBUTING.md

Installs the tox automation tool and executes the test suite for the Python 3.9 environment. This ensures that the project dependencies and logic are functioning correctly.

```sh
pip3 install tox

# Unit tests
tox -e py39
```

--------------------------------

### Clone Kiso Repository

Source: https://github.com/pegasus-isi/kiso/blob/main/CONTRIBUTING.md

Initializes the development environment by cloning the Kiso source code from the official GitHub repository.

```sh
git clone https://github.com/pegasus-isi/kiso.git
```

--------------------------------

### Using Custom Software Installer in experiment.yml

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Demonstrates how to use a custom software installer within a Kiso experiment definition file (`experiment.yml`). It specifies the custom software by its registered name (`mysoftware`) and provides the necessary configuration parameters, including labels, version, and options.

```yaml
software:
  mysoftware:
    labels:
      - submit
    version: "2.0.0"
    options:
      enable_feature: true

```

--------------------------------

### Register Kiso Software Entry Point

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

These examples show how to register a custom Kiso software type using entry points in different project configuration files. This allows Kiso to discover and use the custom software installer. Supported formats include TOML, INI, and Python.

```toml
[project.entry-points."kiso.software"]

mysoftware = "my.software.module:MySoftwareInstaller"
```

```ini
[options.entry_points]

mysoftware = my.software.module:MySoftwareInstaller
```

```python
setup(
    entry_points = {
        "kiso.software": [
            "mysoftware = my.software.module:MySoftwareInstaller",
        ]
    }
)
```

--------------------------------

### Define Vagrant Local Resources

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Configuration example for provisioning local virtual machines using Vagrant and VirtualBox.

```yaml
sites:
  - kind: vagrant
    backend: virtualbox
    box: bento/rockylinux-9
    user: vagrant
    config_extra: 'config.vm.synced_folder ".", "/vagrant", disabled: true'
    resources:
      machines:
        - labels:
            - execute
          backend: virtualbox
          box: bento/rockylinux-9
          user: vagrant
          flavour: "large"
          number: 2
      networks:
        - labels:
            - r1
          cidr: "172.16.42.0/16"
```

--------------------------------

### Install Kiso using pip

Source: https://github.com/pegasus-isi/kiso/blob/main/web/index.html

This command installs the Kiso framework along with all its optional dependencies, enabling full functionality for various environments and integrations.

```bash
$ pip install kiso[all]
```

--------------------------------

### Configure Deployment Entry Points (Python)

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

This Python code demonstrates how to register a deployment entry point within a setup.py file. It uses the `setup` function to define entry points under the 'kiso.deployment' group, mapping 'mydeployment' to its corresponding module and class.

```python
setup(
    entry_points = {
        "kiso.deployment": [
            "mydeployment = my.deployment.module:MyDeploymentInstaller",
        ]
    }
)
```

--------------------------------

### Install Docker Software

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Configures the installation of Docker on specified nodes. This is typically used for containerization tasks on nodes labeled 'submit' and 'execute'.

```yaml
software:
  docker:
    labels:
      - submit
      - execute
```

--------------------------------

### Install rsync on macOS for FABRIC

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Installs rsync via Homebrew on macOS. This is a workaround for potential issues with the default rsync version when connecting to FABRIC sites with IPv6 management addresses.

```shell
brew install rsync
```

--------------------------------

### Define FABRIC Testbed Resources

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Configuration example for provisioning resources on the FABRIC testbed, including GPU and storage specifications.

```yaml
sites:
  - kind: fabric
    rc_file: secrets/fabric_rc
    walltime: "02:00:00"
    resources:
      machines:
        - labels:
            - submit
          site: FIU
          image: default_rocky_8
          flavour: big
          number: 1
          gpus:
            - model: TeslaT4
          storage:
            - kind: NVME
              model: P4510
              mount_point: /mnt/nvme
            - kind: Storage
              model: NAS
              name: kiso-fabric-integration
              auto_mount: true
      networks:
        - labels:
            - v4
          kind: FABNetv4
          site: FIU
          nic:
            kind: SharedNIC
            model: ConnectX-6
```

--------------------------------

### Install Apptainer Software

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Specifies the installation of Apptainer (formerly Singularity), a container runtime, on nodes labeled 'execute'.

```yaml
software:
  apptainer:
    labels:
      - execute
```

--------------------------------

### Implement Custom Deployment Installer in Python

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

Template for creating a custom deployment installer in Kiso. It requires a dataclass for configuration, a JSON schema for validation, and methods for checking and installing the deployment.

```python
from dataclasses import dataclass

@dataclass
class MyDeploymentConfiguration:
    """My Deployment configuration."""

    version: str = "0.0.1"

    ...


class MyDeploymentInstaller:
    """My deployment installer."""
    schema: dict = {
        "title": "My Deployment Configuration Schema",
        "type": "object",
        "properties": {
            "version": {
                "type": "string",
            },
            ...
        }
    }

    config_type: type = MyDeploymentConfiguration

    def __init__(
        self,
        config: MyDeploymentConfiguration,
    ) -> None:
        ...

    def check(self, label_to_machines: Roles) -> None:
        """Implement steps to check the configuration."""
        ...

    def __call__(self, env: Environment) -> None:
        """Implement steps to install your deployment."""
        ...
```

--------------------------------

### Define Shell Experiment in YAML

Source: https://context7.com/pegasus-isi/kiso/llms.txt

A complete example of a Kiso YAML configuration file defining a local Vagrant VM, Docker installation, and a shell script execution.

```yaml
name: hello-world-experiment

sites:
  - kind: vagrant
    backend: virtualbox
    box: bento/rockylinux-9
    user: vagrant
    resources:
      machines:
        - labels:
            - submit
          backend: virtualbox
          box: bento/rockylinux-9
          user: vagrant
          flavour: "large"
          number: 1
      networks:
        - labels:
            - net
          cidr: "172.16.42.0/16"

software:
  docker:
    labels:
      - submit

experiments:
  - kind: shell
    name: hello-world
    description: Print a message and save to file
    inputs:
      - labels:
          - submit
        src: data/input.txt
        dst: ~kiso/hello-world
    scripts:
      - labels:
          - submit
        executable: /bin/bash
        script: |
          #!/bin/bash
          echo "Hello, world!" > hello.txt
          echo "Current date: $(date)" >> hello.txt
          cat hello.txt
    outputs:
      - labels:
          - submit
        src: hello.txt
        dst: ./results
```

--------------------------------

### Configure Chameleon Cloud Site

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Example configuration for deploying resources on Chameleon Cloud, including lease details and machine specifications.

```yaml
sites:
  - kind: chameleon
    walltime: "04:00:00"
    lease_name: my-experiment-lease
    rc_file: secrets/chi-tacc-app-cred-openrc.sh
    key_name: my-ssh-key
    image: CC-Ubuntu22.04
    resources:
      machines:
        - labels:
            - central-manager
            - submit
          flavour: compute_zen3
          number: 1
          image: CC-Ubuntu22.04
        - labels:
            - execute
          flavour: compute_zen3
          number: 3
      networks:
        - sharednet1
```

--------------------------------

### Install Kiso via pip

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Commands to install the Kiso package, including optional dependencies for specific testbed support like Vagrant or Chameleon Cloud.

```bash
# Basic installation
pip install kiso

# With Vagrant support (for local development)
pip install kiso[vagrant]

# With Chameleon Cloud support
pip install kiso[chameleon]

# With FABRIC support
pip install kiso[fabric]

# Install all testbed dependencies
pip install kiso[all]
```

--------------------------------

### Define Custom Software Installer in Python

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

This Python code defines a custom software configuration using dataclasses and an installer class. The installer class includes a JSON schema for validation and methods for checking and installing the software. It requires the `dataclasses` module.

```python
from dataclasses import dataclass

@dataclass
class MySoftwareConfiguration:
    """My Software configuration."""

    version: str = "0.0.1"

    ...


class MySoftwareInstaller:
    """My software installer."""

    schema: dict = {
        "title": "My Software Configuration Schema",
        "type": "object",
        "properties": {
            "version": {
                "type": "string",
            },
            ...
        }
    }

    config_type: type = MySoftwareConfiguration

    def __init__(
        self,
        config: MySoftwareConfiguration,
    ) -> None:
        ...

    def check(self, label_to_machines: Roles) -> None:
        """Implement steps to check the configuration."""
        ...

    def __call__(self, env: Environment) -> None:
        """Implement steps to install your software."""
        ...
```

--------------------------------

### Custom Software Installer Implementation in Python

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Defines a custom software installer for Kiso by implementing a Python class with schema validation. The `MySoftwareInstaller` class uses `dataclasses` for configuration and defines a JSON schema for validation. It includes `check` and `__call__` methods for validation and installation logic, respectively. Dependencies include `enoslib` and `kiso.utils`.

```python
# my_software/installer.py
from dataclasses import dataclass
from pathlib import Path
from enoslib.objects import Roles
from enoslib.task import Environment

@dataclass
class MySoftwareConfiguration:
    """Configuration for my custom software."""
    labels: list[str]
    version: str = "1.0.0"
    options: dict = None

class MySoftwareInstaller:
    """Custom software installer."""

    schema: dict = {
        "title": "My Software Configuration",
        "type": "object",
        "properties": {
            "labels": {
                "type": "array",
                "items": {"type": "string"},
                "minItems": 1
            },
            "version": {"type": "string", "default": "1.0.0"},
            "options": {"type": "object"}
        },
        "required": ["labels"],
        "additionalProperties": False
    }

    config_type: type = MySoftwareConfiguration

    def __init__(self, config: MySoftwareConfiguration) -> None:
        self.config = config

    def check(self, label_to_machines: Roles) -> None:
        """Validate configuration against available machines."""
        if not self.config.labels:
            raise ValueError("At least one label must be specified")

    def __call__(self, env: Environment) -> None:
        """Install the software on target machines."""
        from kiso import utils
        labels = env["labels"]
        target_machines = utils.resolve_labels(labels, self.config.labels)
        # Installation logic here
        print(f"Installing version {self.config.version}")

```

--------------------------------

### Define Chameleon Cloud Resources

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Configuration example for provisioning cloud resources on Chameleon, including lease and authentication settings.

```yaml
sites:
  - kind: chameleon
    walltime: "04:00:00"
    lease_name: tacc-lease
    rc_file: secrets/chi-tacc-app-cred-openrc.sh
    key_name: mayani-mac-mini
    image: CC-Ubuntu18.04
    resources:
      machines:
        - labels:
            - submit
          flavour: compute_zen3
          number: 2
          image: CC-Ubuntu22.04
      networks:
        - sharednet1
```

--------------------------------

### Tear Down Kiso Resources

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Releases and cleans up all resources provisioned by Kiso for a specific experiment. This command reverts the system to its state before 'kiso up' was run.

```shell
$ kiso down experiment.yml
```

--------------------------------

### Validate Kiso Experiment Configuration

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/getting-started.md

Validates the syntax and structure of a Kiso experiment YAML file. This command checks for potential errors before provisioning resources.

```shell
$ kiso check experiment.yml
```

--------------------------------

### Provision Resources with Kiso

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Provisions infrastructure on configured testbeds, installs required software, and deploys workload management systems.

```bash
# Provision resources using experiment.yml
kiso up

# Provision with a specific config and output directory
kiso up --output my-env my-experiment.yml

# Force re-provisioning (destroys existing resources first)
kiso up --force experiment.yml
```

--------------------------------

### Define Chameleon Edge Resources

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Configuration example for provisioning edge computing resources on Chameleon, including containerized deployments.

```yaml
sites:
  - kind: chameleon-edge
    walltime: "04:00:00"
    lease_name: edge-lease
    rc_file: secrets/chi-edge-app-cred-openrc.sh
    resources:
      machines:
        - labels:
            - central-manager
          machine_name: raspberrypi4-64
          count: 1
          container:
            name: execute
            image: rockylinux:8
```

--------------------------------

### Register Custom Deployment via Entry Points

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

How to register a custom deployment installer in a project's configuration file using the kiso.deployment entry point group.

```toml
[project.entry-points."kiso.deployment"]

mydeployment = "my.deployment.module:MyDeploymentInstaller"
```

--------------------------------

### Advanced Multi-Site Kiso Plankifier Experiment

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

An advanced example demonstrating a multi-site Kiso plankifier experiment configuration, likely involving complex workflow orchestration across different nodes.

```yaml
apiVersion: kiso.ai/v1alpha1
kind: Experiment
metadata:
  name: plankifier-multi-site
spec:
  description: "A multi-site experiment using the plankifier"
  owner: "kiso-team"
  schedule: "@daily"
  workflow:
    name: "plankifier-workflow"
    engine: "pegasus"
    submit_node_labels:
      - "submit"
    script: "bin/run_plankifier.sh"
    inputs:
      - name: "config.yaml"
        source:
          type: "local"
          path: "./config/plankifier.yaml"
        destination:
          node_labels: ["submit"]
          path: "/home/user/plankifier/config.yaml"
      - name: "data.csv"
        source:
          type: "s3"
          bucket: "my-data-bucket"
          key: "input/data.csv"
        destination:
          node_labels: ["execute"]
          path: "/mnt/data/input/data.csv"
    outputs:
      - name: "results.json"
        source:
          node_labels: ["submit"]
          path: "/home/user/plankifier/results.json"
        destination:
          type: "local"
          path: "./output/plankifier_results.json"
    environment:
      variables:
        - name: "PLANKIFIER_LOG_LEVEL"
          value: "INFO"
        - name: "MAX_THREADS"
          value: "4"
    resources:
      requests:
        cpu: "2"
        memory: "4Gi"
      limits:
        cpu: "4"
        memory: "8Gi"
    parallelism:
      max_concurrent_runs: 5
    retry_policy:
      max_retries: 3
      backoff_factor: 2
    notifications:
      on_failure:
        - type: "email"
          address: "alerts@example.com"
      on_success:
        - type: "slack"
          channel: "#ci-cd"
    labels:
      - "multi-site"
      - "plankifier"
      - "production"
```

--------------------------------

### Install Ollama for LLMs

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Configures the installation of Ollama for running large language models, with custom settings for model queues, parallel processing, and context length based on node labels.

```yaml
software:
  ollama:
    - labels:
        - large-model-node
      models:
        - llama3:70b
      environment:
        OLLAMA_MAX_QUEUE: 512
        OLLAMA_NUM_PARALLEL: 4
    - labels:
        - small-model-node
      models:
        - llama3:8b
      environment:
        OLLAMA_CONTEXT_LENGTH: 8192
```

--------------------------------

### Define Kiso Software Configuration in YAML

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

Example structure for defining software versions and experiment configurations within the Kiso YAML configuration file.

```yaml
sites:
    ...

software:
    mysoftware:
        version: 5.1.0
        ...

experiments:
    ...
```

--------------------------------

### Define a Pegasus Experiment

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Defines a 'pegasus' experiment named 'process-experiment' that runs a Pegasus workflow. It includes configurations for input files, setup scripts, post-execution scripts, and output collection, targeting nodes with 'submit' and 'execute' labels.

```yaml
experiments:
  - kind: pegasus
    name: process-experiment
    description: A Pegasus workflow
    # Number of time to run the experiment
    count: 1
    # Script to run the Pegasus workflow
    main: bin/main.sh
    # The node from which the workflow will be submitted
    submit_node_labels:
      - submit

    # Optionally, specify input files and on which node to copy them on to setup the environment
    # By default, the directory containing the experiment.yml file will be copied to all provisioned nodes
    inputs:
      - labels:
          - execute
        src: README.md
        dst: ~kiso/kiso-process-experiment

    # Optionally, specify what scripts to run and on which node to run them on to setup the environment
    setup:
      - labels:
          - submit
        executable: /bin/bash
        script: |
          #!/bin/bash
          echo "Setup script here"

    # Optionally, specify what scripts to run and on which node to run them on after the environment
    post_scripts:
      - labels:
          - submit
        executable: /bin/bash
        script: |
          #!/bin/bash
          echo "Post script here"

    # Optionally, specify output files and on which node to copy them from after the experiment
    # By default, the Pegasus workflow submit directory will be copied to the local machine
    outputs:
      - labels:
          - submit
        src: ~kiso/kiso-process-experiment
        dst: local-machine
```

--------------------------------

### Deploy Full HTCondor Cluster

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Defines the deployment of a complete HTCondor cluster, including a central manager, submit nodes, and execute nodes. It also includes a configuration for a personal HTCondor setup.

```yaml
deployment:
  htcondor:
    - kind: central-manager
      labels:
        - central-manager
      # config_file: config/cm-condor_config  # Optional custom config

    - kind: submit
      labels:
        - submit
      # config_file: config/submit-condor_config

    - kind: execute
      labels:
        - execute
      # config_file: config/execute-condor_config

    # Personal HTCondor (single-node setup)
    - kind: personal
      labels:
        - edge-node
```

--------------------------------

### Hybrid Cloud-Edge Experiment with HTCondor

Source: https://context7.com/pegasus-isi/kiso/llms.txt

A complete Kiso experiment definition for a hybrid cloud-edge setup. It specifies resources on both cloud and edge sites, deploys HTCondor for orchestration, and defines a Pegasus experiment with data collection on the edge and processing in the cloud. Dependencies include Kiso, HTCondor, and Pegasus.

```yaml
name: plankifier-experiment

sites:
  # Edge devices for data collection
  - kind: chameleon-edge
    walltime: "04:00:00"
    lease_name: edge-lease
    rc_file: secrets/chi-edge-app-cred-openrc.sh
    resources:
      machines:
        - labels:
            - execute-edge
          machine_name: raspberrypi4-64
          count: 2
          container:
            name: edge-worker
            image: pegasus/plankifier

  # Cloud resources for heavy computation
  - kind: chameleon
    walltime: "04:00:00"
    lease_name: tacc-lease
    rc_file: secrets/chi-tacc-app-cred-openrc.sh
    key_name: my-key
    image: CC-Ubuntu22.04
    resources:
      machines:
        - labels:
            - central-manager
            - submit
            - execute-cloud
          flavour: compute_zen3
          number: 1
      networks:
        - sharednet1

deployment:
  htcondor:
    - kind: central-manager
      labels:
        - central-manager
    - kind: submit
      labels:
        - submit
    - kind: execute
      labels:
        - execute-cloud
    - kind: execute
      labels:
        - execute-edge
      config_file: config/edge-execute.conf

software:
  apptainer:
    labels:
      - execute-cloud

experiments:
  - kind: pegasus
    name: distributed-processing
    count: 1
    main: ./workflow.py
    submit_node_labels:
      - submit
    inputs:
      - labels:
          - execute-edge
        src: bin/collect.py
        dst: /srv/app/ 
    setup:
      - labels:
          - submit
        script: |
          chmod +x workflow.py
      - labels:
          - execute-edge
        script: |
          chmod +x /srv/app/collect.py
    outputs:
      - labels:
          - submit
        src: ~kiso/distributed-processing/results
        dst: ./output

```

--------------------------------

### Configure Deployment Entry Points (INI)

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

This snippet shows how to define a deployment entry point in a setup.cfg file using the INI format. It specifies the module and class to be used for the 'mydeployment' entry point.

```ini
[options.entry_points]

mydeployment = my.deployment.module:MyDeploymentInstaller
```

--------------------------------

### Configure Machine Labels for Resource Targeting

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Demonstrates how to assign labels to machines in the sites section and reference them in software, deployment, and experiment configurations to decouple infrastructure from logic.

```yaml
sites:
  - kind: vagrant
    resources:
      machines:
        - labels:
            - submit
          flavour: large
          number: 1
        - labels:
            - execute
          flavour: large
          number: 2

software:
  docker:
    labels:
      - submit

deployment:
  htcondor:
    - kind: submit
      labels:
        - submit
    - kind: execute
      labels:
        - execute

experiments:
  - kind: pegasus
    submit_node_labels:
      - submit
```

--------------------------------

### Define a Shell Experiment

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Defines a 'shell' experiment named 'shell-experiment' that prints 'Hello, world!' to a file. It specifies input and output files, and the script to be executed on nodes with 'submit' labels.

```yaml
experiments:
  - kind: shell
    name: shell-experiment
    description: An experiment to print a message
    # Optionally, specify output files and on which node to copy them from after the experiment
    inputs:
      - labels:
          - submit
        src: name.txt
        dst: ~kiso

    # Specify what scripts to run and on which node to run them on
    scripts:
      - labels:
          - submit
        script: |
          #!/bin/bash
          echo "Hello, world!" | tee hello.txt

    # Optionally, specify output files and on which node to copy them from after the experiment
    outputs:
      - labels:
          - submit
        src: hello.txt
        dst: output
```

--------------------------------

### Documentation Generation with Tox

Source: https://github.com/pegasus-isi/kiso/blob/main/CLAUDE.md

Commands to generate project documentation using tox, or manually by navigating to the docs directory and using make.

```shell
tox -e docs
# Or manually: cd docs && make html
```

--------------------------------

### Kiso CLI Commands

Source: https://github.com/pegasus-isi/kiso/blob/main/CLAUDE.md

Demonstrates the primary command-line interface commands for the Kiso experiment management platform.

```shell
# Validate experiment YAML against composed JSON schema
kiso check

# Provision infrastructure, install software/deployments
kiso up

# Execute experiments on provisioned resources
kiso run

# Tear down resources
kiso down
```

--------------------------------

### Linting and Formatting with Ruff

Source: https://github.com/pegasus-isi/kiso/blob/main/CLAUDE.md

Commands to run all pre-commit checks or specifically use Ruff for linting and formatting the source code and tests.

```shell
# Run all pre-commit checks
pre-commit run --all-files

# Ruff only
ruff check --fix src/ tests/
ruff format src/ tests/
```

--------------------------------

### Configure Ollama Software with Models and Environment

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Configures Ollama software, specifying different model configurations ('large-model', 'small-model') with associated model names and environment variables like OLLAMA_MAX_QUEUE and OLLAMA_CONTEXT_LENGTH.

```yaml
software:
  ollama:
    - labels:
        - large-model
      models:
        - gpt-oss:20b
      environment:
        OLLAMA_MAX_QUEUE: 512

    - labels:
        - small-model
      models:
        - qwen3.5:2b
      environment:
        OLLAMA_CONTEXT_LENGTH: 8192
```

--------------------------------

### Configure Docker Software

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Defines Docker software configuration with 'submit' labels. This allows specifying Docker configurations for submission-related operations.

```yaml
software:
  docker:
    labels:
      - submit
```

--------------------------------

### Run Shell Experiment

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Configures and executes a shell experiment, including staging input files, running bash scripts on specified nodes, and collecting output files. It demonstrates basic file operations and script execution.

```yaml
experiments:
  - kind: shell
    name: data-processing
    description: Process data files on remote nodes
    inputs:
      - labels:
          - submit
        src: data/input.csv
        dst: ~kiso/data-processing
    scripts:
      - labels:
          - submit
        executable: /bin/bash
        script: |
          #!/bin/bash
          set -e
          cd ~kiso/data-processing
          echo "Processing input.csv..."
          wc -l input.csv > line_count.txt
          head -100 input.csv > sample.csv
          echo "Processing complete"
      - labels:
          - execute
        script: |
          #!/bin/bash
          echo "Worker node ready"
          hostname >> /tmp/workers.txt
    outputs:
      - labels:
          - submit
        src: ~kiso/data-processing/line_count.txt
        dst: ./results
      - labels:
          - submit
        src: ~kiso/data-processing/sample.csv
        dst: ./results
```

--------------------------------

### Kiso Experiment Lifecycle Commands

Source: https://github.com/pegasus-isi/kiso/blob/main/web/index.html

A sequence of commands to manage the lifecycle of an experiment using Kiso. This includes defining the experiment in YAML, provisioning resources, running the experiment, and deprovisioning resources.

```bash
# Define your experiment in YAML
$ vim experiment.yml

# Provision resources across multiple testbeds
$ kiso up

# Run across multiple testbeds
$ kiso run

# Deprovision resources
$ kiso down
```

--------------------------------

### Execute Experiments with Kiso

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Runs defined experiments on provisioned resources, handles file transfers, and retrieves results.

```bash
# Run experiments
kiso run

# Run with specific output directory
kiso run --output my-env experiment.yml

# Force re-run (ignores previous state)
kiso run --force experiment.yml
```

--------------------------------

### Register Experiment Entry Point in Python Packaging

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

Configures a custom experiment runner by mapping a name to an entry point in the project's packaging configuration. This is required for the Kiso framework to discover and execute the experiment.

```toml
[project.entry-points."kiso.experiment"]

my-experiment = "my.experiment.module:MyExperimentRunner"
```

```ini
[options.entry_points]

my-experiment = my.experiment.module:MyExperimentRunner
```

```python
setup(
	entry_points = {
		"kiso.experiment": [
			"my-experiment = my.experiment.module:MyExperimentRunner",
		]
	}
)
```

--------------------------------

### Configure FABRIC Testbed Resources

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Defines resources for the FABRIC testbed, including machines with GPUs and NVME storage, and network configurations. It specifies machine images, flavours, and network types.

```yaml
sites:
  - kind: fabric
    rc_file: secrets/fabric_rc
    walltime: "02:00:00"
    resources:
      machines:
        - labels:
            - submit
            - execute
          site: FIU
          image: default_rocky_8
          flavour: big
          number: 1
          gpus:
            - model: TeslaT4
          storage:
            - kind: NVME
              model: P4510
              mount_point: /mnt/nvme
      networks:
        - labels:
            - v4
          kind: FABNetv4
          site: FIU
          nic:
            kind: SharedNIC
            model: ConnectX-6
```

--------------------------------

### Execute Kiso Experiment Lifecycle

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/example-experiments.md

Standard command-line sequence to initialize, provision, execute, and tear down an experiment environment using Kiso. Requires a valid experiment.yml configuration file in the current directory.

```shell
git clone <experiment-repo-url>
cd <experiment-directory>
kiso check experiment.yml
kiso up experiment.yml
kiso run experiment.yml
kiso down experiment.yml
```

--------------------------------

### Running Tests with Pytest and Tox

Source: https://github.com/pegasus-isi/kiso/blob/main/CLAUDE.md

Commands for executing the test suite using tox for comprehensive testing with coverage, or directly with pytest for more targeted testing.

```shell
# Full test suite with coverage (via tox, recommended)
tox -e py311

# Direct pytest
pytest --cov=src --no-cov-on-fail tests/

# Single test file
pytest tests/test__main__.py

# Single test function
pytest tests/test__main__.py::test_check
```

--------------------------------

### Configure Apptainer Software

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Defines Apptainer software configuration with 'submit' labels. This is used to specify how Apptainer should be configured for submission tasks.

```yaml
software:
  apptainer:
    labels:
      - submit
```

--------------------------------

### Manage Experiments with Kiso CLI

Source: https://github.com/pegasus-isi/kiso/blob/main/README.md

The Kiso CLI provides commands to validate, provision, execute, and tear down experiment environments defined in a YAML configuration file.

```shell
pip install kiso
kiso check experiment.yml
kiso up experiment.yml
kiso run experiment.yml
kiso down experiment.yml
```

--------------------------------

### Register Experiment Runner in pyproject.toml

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Registers a custom experiment runner with Kiso using Python entry points in the `pyproject.toml` file. This makes the `MyExperimentRunner` available to the Kiso framework under the 'my-experiment' kind.

```toml
[project.entry-points."kiso.experiment"]
my-experiment = "my_experiment.runner:MyExperimentRunner"
```

--------------------------------

### Configure Chameleon Edge Deployment

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Sets up resources on Chameleon edge devices, such as Raspberry Pi, for deploying containers. It specifies machine names, container images, and exposed ports.

```yaml
sites:
  - kind: chameleon-edge
    walltime: "04:00:00"
    lease_name: edge-lease
    rc_file: secrets/chi-edge-app-cred-openrc.sh
    resources:
      machines:
        - labels:
            - edge-node
          machine_name: raspberrypi4-64
          count: 2
          container:
            name: worker
            image: rockylinux:8
            exposed_ports:
              - "8080"
```

--------------------------------

### Validate Experiment Configuration with Kiso API (Python)

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Demonstrates programmatic validation of experiment configurations using the Kiso Python API. It shows how to validate both file paths and Python dictionaries against the defined schemas.

```python
from pathlib import Path
from kiso.task import check, up, run, down

# Validate a configuration file
check(Path("experiment.yml"))

# Or validate a dictionary directly
config = {
    "name": "test-experiment",
    "sites": [{
        "kind": "vagrant",
        "backend": "virtualbox",
        "box": "bento/rockylinux-9",
        "user": "vagrant",
        "resources": {
            "machines": [{
                "labels": ["test"],
                "flavour": "large",
                "number": 1
            }],
            "networks": [{
                "labels": ["net"],
                "cidr": "172.16.0.0/16"
            }]
        }
    }],
    "experiments": [{
        "kind": "shell",
        "name": "test",
        "scripts": [{
            "labels": ["test"],
            "script": "echo 'Hello'"
        }]
    }]
}
check(config)
```

--------------------------------

### Tear Down Kiso Resources

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Destroys all provisioned resources and cleans up the experiment environment state.

```bash
# Destroy resources
kiso down

# Destroy resources from specific environment
kiso down --output my-env experiment.yml
```

--------------------------------

### Define Custom Experiment Runner in Python

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Defines a custom experiment runner class `MyExperimentRunner` with configuration validation using dataclasses and a schema. It includes methods for initialization, checking configuration, and executing the experiment.

```python
from dataclasses import dataclass
from enoslib.objects import Roles
from enoslib.task import Environment

@dataclass
class MyExperimentConfiguration:
    """Configuration for custom experiment."""
    kind: str = "my-experiment"
    name: str = ""
    config_file: str = None

class MyExperimentRunner:
    """Custom experiment runner."""

    schema: dict = {
        "title": "My Experiment Configuration",
        "type": "object",
        "properties": {
            "kind": {"const": "my-experiment"},
            "name": {"type": "string"},
            "config_file": {"type": "string"}
        },
        "required": ["kind", "name"],
        "additionalProperties": False
    }

    config_type: type = MyExperimentConfiguration
    kind: str = "my-experiment"

    def __init__(
        self,
        experiment: MyExperimentConfiguration,
        index: int,
        variables: dict = None
    ) -> None:
        self.experiment = experiment
        self.index = index
        self.variables = variables or {}

    def check(self, config, label_to_machines: Roles) -> None:
        """Validate experiment configuration."""
        pass

    def __call__(
        self,
        wd: str,
        remote_wd: str,
        resultdir: str,
        labels: Roles,
        env: Environment
    ) -> None:
        """Execute the experiment."""
        print(f"Running experiment: {self.experiment.name}")
        # Experiment execution logic
```

--------------------------------

### Implement Custom Experiment Runner (Python)

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

This Python code outlines the structure for a custom experiment runner. It includes a schema for configuration validation, a reference to the configuration type, and methods for checking (`check`) and running (`__call__`) the experiment.

```python
class MyExperimentRunner:
    """My Experiment runner."""
    schema: dict = {
        "title": "My Experiment Configuration Schema",
        "type": "object",
        "properties": {
            "kind": {
                "const": "my-experiment"
            },
            ...
        }
    }

    config_type: type = MyExperimentConfiguration

    #: 
    kind: str = "my-experiment"

    def __init__(
        self,
        experiment: MyExperimentConfiguration,
        index: int,
        variables: dict[str, str | int | float] | None = None,
    ) -> None:
        ...

    def check(self, config: Kiso, label_to_machines: Roles) -> None:
        """Implement steps to check your experiment."""

    def __call__(self, wd: str, remote_wd: str, resultdir: str, labels: Roles, env: Environment) -> None:
        """Implement steps to run your experiment."""
        ...

```

--------------------------------

### Deploy HTCondor Cluster Components

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/concepts.rst

Defines the deployment configuration for an HTCondor cluster, including components like 'central-manager', 'execute', 'submit', and 'personal' nodes, each with specific labels and optional custom configuration file paths.

```yaml
deployment:
  htcondor:
    - kind: central-manager
      labels:
        - central-manager
      # Optionally, define a custom Condor configuration file
      # config_file: config/cm-condor_config

    - kind: execute
      labels:
        - execute
      # Optionally, define a custom Condor configuration file
      # config_file: config/exec-condor_config

    - kind: submit
      labels:
        - submit
      # Optionally, define a custom Condor configuration file
      # config_file: config/submit-condor_config

    - kind: personal
      labels:
        - edge-1
      #   Optionally, define a custom Condor configuration file
      #   config_file: config/personal-condor_config
```

--------------------------------

### Run Pegasus Workflow Experiment

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Sets up and runs a scientific workflow using the Pegasus Workflow Management System. This includes preparing the environment, submitting the workflow, and collecting results.

```yaml
experiments:
  - kind: pegasus
    name: scientific-workflow
    description: Execute a Pegasus DAX workflow
    count: 1  # Number of times to run the workflow
    main: bin/workflow.py  # Script that generates and submits the workflow
    submit_node_labels:
      - submit
    inputs:
      - labels:
          - execute
        src: bin/transform.py
        dst: /srv/workflow/
    setup:
      - labels:
          - submit
        executable: /bin/bash
        script: |
          #!/bin/bash
          chmod +x bin/workflow.py
          pip install pegasus-wms
      - labels:
          - execute
        script: |
          chmod +x /srv/workflow/transform.py
    post_scripts:
      - labels:
          - submit
        script: |
          echo "Workflow completed at $(date)"
    outputs:
      - labels:
          - submit
        src: ~kiso/scientific-workflow/output
        dst: ./workflow-results
```

--------------------------------

### Validate Kiso Configuration

Source: https://context7.com/pegasus-isi/kiso/llms.txt

Validates an experiment configuration file against the JSON schema to ensure all labels and software definitions are correct.

```bash
# Validate the default experiment.yml file
kiso check

# Validate a specific configuration file
kiso check my-experiment.yml
```

--------------------------------

### Type Checking with Mypy

Source: https://github.com/pegasus-isi/kiso/blob/main/CLAUDE.md

Executes Mypy for static type checking on the source code to catch type-related errors.

```shell
mypy src/
```

--------------------------------

### Define Custom Experiment Configuration (Python)

Source: https://github.com/pegasus-isi/kiso/blob/main/docs/extending.rst

This Python code defines a dataclass for a custom experiment configuration. It includes a 'kind' attribute, which serves as a unique identifier for the experiment type and is used for registration with Kiso.

```python
@dataclass
class MyExperimentConfiguration:
    """My Experiment configuration."""

    kind: str = "my-experiment"

    ...

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.