Parallel Works ACTIVATE

https://parallelworks.com/docs
Admin
Parallel Works ACTIVATE is a cloud HPC control plane that enables users to create, manage, and scale...
Tokens:22,626
Snippets:232
Trust Score:8.5
Update:7 months ago
Show doc for...
Tokens:
Raw
### Examples of Input Value and Schema Referencing

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Illustrates various ways to reference input values and schema properties using 'inputs', bracket notation, and relative paths ('../', '.').

```yaml
on:
  execute:
    inputs:
      field_1:
        type: number
      field_2:
        type: string
        default: "${{ inputs.field_1 }}" # The value of field 1
      field_3:
        type: string
        default: "${{ inputs[field_1] }}" # The field_1 schema object
      field_4:
        type: string
        default: "${{ inputs.field_1[type] }}" # The string "number"
      field_5:
        type: string
        default: "${{ ../field_1 }}" # The field_1 schema object
      field_6:
        type: string
        default: "${{ ../field_1.type }}" # The string "number"
      field_7:
        type: string
        default: "${{ .type }}" # The string "string"
```

--------------------------------

### Tooltip Usage Example

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Demonstrates how to use the 'tooltip' property to provide additional information for input fields. This assists users by offering more context or instructions when they hover over an input field.

```yaml
tooltip: 'Provide a detailed description of the job'
```

--------------------------------

### Serve Go Application on a Port

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

This snippet demonstrates how to serve a simple Go HTTP server. It defines a basic 'hello' handler and starts the server on a specified port. This is often used in conjunction with agent actions to expose services.

```go
package main

import (
    "fmt"
    "net/http"
)

func hello(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "Hello from slurm agent server!\n")
}

func main() {
    http.HandleFunc("/", hello)
    http.ListenAndServe(":${{ needs.main.outputs.sessionPort }}", nil)
}

```

--------------------------------

### Install Parallel Works Operator using Helm

Source: https://parallelworks.com/docs/self-hosting/kubernetes-deployment

This command adds the Parallel Works Helm repository and installs the operator. Ensure you replace `<repository-url>` with the actual URL of the Helm repository. The operator is installed into a dedicated namespace, which is created if it doesn't exist.

```bash
helm repo add corerepo <repository-url>
helm upgrade --install parallelworks-operator corerepo/operator \
  --namespace pw-operator-system \
  --create-namespace
```

--------------------------------

### Displaying Job Output in Bash Script

Source: https://parallelworks.com/docs/run/workflows-and-apps/running-workflows

This snippet shows an example of output from a simple bash script executed as part of a workflow. It displays the start time, input arguments, the hostname of the running resource, and a confirmation message. This output is typically found in the Job logs module.

```bash
Starting test workflow at Wed Jan 25 17:12:21 UTC 2023
INPUT ARGUMENTS: 

Running on the following computer: pw-user-demo
This is a NO-OP workflow - nothing got launched remotely.
Done! 

```

--------------------------------

### Scheduler Agent API

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

Provisions a node and starts an SSH server for remote command execution. Requires a cluster with a partition.

```APIDOC
## POST /websites/parallelworks/scheduler-agent

### Description
You can use the `scheduler-agent` action to provision a node and start an ssh server so that you can run commands on it through ssh. Note that you must use a cluster with a partition for this to properly work. See Building Sessions for example usage.

### Method
POST

### Endpoint
/websites/parallelworks/scheduler-agent

### Parameters
#### Inputs
- **wait** (boolean) - Optional - Whether to wait for the node to provision. Defaults to true.
- **scheduler-type** (string) - Optional - Use this scheduler. Defaults to slurm, options are slurm and pbs.
- **scheduler-flags** (object) - Optional - Use these flags for the sbatch command. Defaults to none. You can find a list of flags here under job submission for slurm and here for pbs.
- **script-headers** (string) - Optional - Additional lines to add to the top of the bash script being submitted via sbatch or qsub (depending on your scheduler). Useful if you want to allow users to add their own `#SBATCH/PBS` comment flags at will.

### Request Example
```json
{
  "wait": false,
  "scheduler-type": "slurm",
  "scheduler-flags": {
    "--time": "1:00:00"
  }
}
```

### Response
#### Success Response (200)
- **agentId** (string) - The ID of the provisioned agent.
- **remoteHost** (string) - The remote host address for the agent.
- **sshPort** (int) - The SSH port for the agent.

#### Response Example
```json
{
  "agentId": "agent-12345",
  "remoteHost": "192.168.1.100",
  "sshPort": 22
}
```
```

--------------------------------

### Provision Node and Start SSH Server with Scheduler Agent

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

Provisions a compute node and initiates an SSH server, enabling command execution via SSH. This action requires a cluster with a partition to function correctly. It offers options to wait for node provisioning and specify scheduler types and flags.

```yaml
uses: parallelworks/scheduler-agent
id: slurmstep
with:
  wait: false
```

--------------------------------

### Sample ParallelWorks Custom Resource YAML Configuration

Source: https://parallelworks.com/docs/self-hosting/kubernetes-deployment

This YAML configuration defines a custom resource for the ParallelWorks platform. It specifies settings for the platform's image, default scaling, ingress host, TLS certificate, and database configurations (PostgreSQL and MongoDB). This serves as a template for deploying the ParallelWorks application within a Kubernetes environment.

```yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: pw-system
---
apiVersion: platform.parallelworks.com/v1alpha1
kind: ParallelWorks
metadata:
  name: customer
  namespace: pw-system
spec:
  image:
    registry: ghcr.io
    repositoryPrefix: parallelworks
    tag: v5.203
    pullPolicy: IfNotPresent
    pullSecrets:
      - name: image-pull-secret
  
  # Default scale for all services that don't specify their own replicas
  # Services default to enabled=true, so you only need to specify services
  # if you want to customize them or disable them
  defaultScale: 1
 
  # Host is the ingress hostname for the platform
  host: customer.parallel.works # can be your own domain
 
  # TLSSecretName for TLS certificate
  tlsSecretName: pw-tls-secret
 
  # Database configuration
  database:
    # PostgreSQL configuration
    postgres:
      endpoint: 'pw-pgsql-service.postgres.svc.cluster.local:5432'
      database: 'pworks'
      username: 'postgres'
      urlSecretRef:
        name: 'postgres-secret'
        key: 'password'
 
    # MongoDB configuration (no authentication configured)
    mongodb:
      replicaSetName: 'rs0'
      endpoints:
        - 'mongodb-0-external.mongo.svc.cluster.local:27017'
        - 'mongodb-1-external.mongo.svc.cluster.local:27017'
        - 'mongodb-2-external.mongo.svc.cluster.local:27017'

```

--------------------------------

### Access Bucket Input Value in Expressions

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

This example demonstrates how to access specific fields of a 'bucket' input type using dot notation within Parallel Works expressions. The 'bucket' input provides an object with fields like 'uri' and 'csp' at runtime.

```expression
${{ inputs.mybucket.uri }}
${{ inputs.mybucket.csp }}
```

--------------------------------

### Fetch Bucket Credentials for AWS

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

When `generateCredentials: true` is set for a bucket input, temporary credentials are fetched from the vault. This example shows how to access and use AWS credentials (accessKeyId, secretAccessKey, sessionToken) and bucket details (bucketName, region) within a job to sync data using the AWS CLI.

```yaml
on:
  execute:
    inputs:
      data_bucket:
        type: bucket
        label: Data Bucket
        csp: aws
        generateCredentials: true

jobs:
  sync:
    steps:
      - name: Sync data from bucket
        run: |
          export AWS_ACCESS_KEY_ID=${{ inputs.data_bucket.credentials.accessKeyId }}
          export AWS_SECRET_ACCESS_KEY=${{ inputs.data_bucket.credentials.secretAccessKey }}
          export AWS_SESSION_TOKEN=${{ inputs.data_bucket.credentials.sessionToken }}
          aws s3 sync s3://${{ inputs.data_bucket.bucketName }}/data ./local-data --region ${{ inputs.data_bucket.region }}
```

--------------------------------

### Writing Key-Value Outputs to a File

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Shows how to create accessible outputs by writing key-value pairs to the file path specified by the `$OUTPUTS` environment variable. Each output must be on a new line in the `key=value` format.

```yaml
jobs:
  example:
    steps:
      - name: Create outputs
        run: |
          echo PORT=3001 >> $OUTPUTS
          echo DATABASE_URL=postgres://localhost:5432/mydb >> $OUTPUTS
          echo STATUS=success >> $OUTPUTS
```

--------------------------------

### Define Job Dependencies with 'needs' (YAML)

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

The 'needs' field specifies a list of jobs that must complete successfully before the current job can start. If a preceding job fails, dependent jobs will not execute.

```yaml
jobs:
  first:
    steps:
      - run: echo runs first
  second:
    needs:
      - first
    steps:
      - run: echo runs second
```

--------------------------------

### Verify Storage Mounts using df -h

Source: https://parallelworks.com/docs/storage/attaching-storage

This command verifies that storage resources are mounted correctly on a cluster. It lists all mounted filesystems and their usage. Ensure you have logged into your cluster to run this command.

```shell
df -h
```

--------------------------------

### Execute Commands Remotely via SSH

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

Configures the job or specific steps to execute commands on a remote host using SSH. This includes specifying the remote host, user, and an optional jump node for more complex network setups.

```yaml
jobs:
  echo_on_remote:
    ssh:
      remoteHost: "${{ inputs.resource1.ip }}"
      jumpNodeHost: "${{ inputs.resource2.ip }}"
    steps:
      - run: echo "This is executed on the remote host!"
      - run: echo "This is executed on the jump node!"
        ssh:
          remoteHost: "${{ inputs.resource2.ip }}"
      - run: echo "This is executed in the user workspace!"
        ssh: null
on:
  execute:
    inputs:
      resource1:
        label: Resource 1
        type: compute-clusters
        optional: false
      resource2:
        label: Resource 2
        type: compute-clusters
        optional: false
```

--------------------------------

### Job and Step Configuration

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

This section covers the configuration of individual steps within a job, including cleanup commands, conditional execution, retry logic, environment variables, error handling, working directories, and early cancellation.

```APIDOC
## `jobs.<job>.steps[*].cleanup`

### Description
Defines a cleanup command to be run after step execution finishes. This is used to delete temporary files, terminate connections, remove credentials, or perform any other cleanup necessary when a job is shutting down. Cleanup commands always run at the end of a job in reverse order of definition, regardless of the job's success, failure, or cancellation. If a step did not run, its cleanup step will not run.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  example_job:
    steps:
      - run: echo "Running main task"
        cleanup: echo "Cleaning up temporary files"
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A

## `jobs.<job>.steps[*].retry`

### Description
Configures retry logic for a step that has a chance of failure but is necessary for the workflow. It allows specifying the maximum number of retries, the interval between retries, and the timeout for each attempt.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  example_job:
    steps:
      - run: potentially_failing_command
        retry:
          max-retries: 5
          interval: 1m
          timeout: 10m
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A

## `jobs.<job>.steps[*].ignore-errors`

### Description
If set to `true`, non-zero exit codes from a step will not be counted as failures. By default, this attribute is set to `false`.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  example_job:
    steps:
      - run: command_that_might_fail_but_is_ok
        ignore-errors: true
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A

## `jobs.<job>.steps[*].working-directory`

### Description
Specifies the working directory for a step's execution.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  example_job:
    steps:
      - run: echo "Hello"
        working-directory: /app/src
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A

## `jobs.<job>.steps[*].early-cancel`

### Description
Defines conditions for early cancellation of a step. Currently, only `early-cancel: any-job-failed` is supported, which cancels the step if any job fails before the step finishes running.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  example_job:
    steps:
      - run: long_running_task
        early-cancel: any-job-failed
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A

## `jobs.<job>.if`

### Description
Prevents a job or step from running unless a specified condition evaluates to true. It accepts expressions like `${{ inputs.input-name }}` or `${{ always }}`. `${{ always }}` ensures execution even if dependencies fail.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  conditional_job:
    if: ${{ inputs.run-this-job }}
    steps:
      - run: echo "This step runs only if the condition is met."
        if: ${{ always }}
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A

## `jobs.<job>.env`

### Description
Sets environment variables for a job or step.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  example_job:
    env:
      MY_VARIABLE: my_value
    steps:
      - run: echo "$MY_VARIABLE"
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### Publishing Job Outputs Using the Outputs Map

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Illustrates how to explicitly define and publish specific job outputs using the `outputs` map. This allows other jobs to consume these outputs by referencing them via `needs.<job>.outputs.<output_name>`. This method ensures outputs are reliably available to dependent jobs.

```yaml
jobs:
  build:
    outputs:
      version: ${{ needs.build.steps.get_version.outputs.VERSION }}
      artifact_url: ${{ needs.build.steps.upload.outputs.URL }}
    steps:
      - name: Get version
        id: get_version
        run: echo VERSION=1.2.3 >> $OUTPUTS
      - name: Upload artifact
        id: upload
        run: echo URL=https://example.com/artifact.zip >> $OUTPUTS
```

--------------------------------

### Accessing Outputs from Named Steps

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Demonstrates how to define named steps using the `id` field and access their outputs in subsequent steps using the `needs.<job>.steps.<step_id>.outputs.<output_name>` syntax. Outputs from named steps are not automatically published to job outputs and require explicit mapping.

```yaml
jobs:
  main:
    steps:
      - name: Create step output
        id: test
        run: echo FOO=BAR >> $OUTPUTS
      - name: Access output from named step
        run: echo ${{ needs.main.steps.test.outputs.FOO }} # Prints BAR
```

--------------------------------

### Expression Syntax and Evaluation

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Illustrates the fundamental syntax for expressions in Parallelworks, which must be wrapped in `${{ }}`. It shows how expressions are evaluated and the importance of whitespace for correct parsing.

```text
`${{ true && 'truthy value' }}`
`${{ 1 + 1.5 }}`
`${{ 'a' + 'b' }}`
`${{ 1 - 1.5 }}`
`${{ 4 * 4 }}`
`${{ 9 / 4 }}`
`${{ 2 ** 4 }}`
`${{ 4 // 3 }}`
`${{ 'hello' ?? 'default' }}`
`${{ undefined ?? 'default' }}`
`${{ ['a', 'b'] get 1 }}`
`${{ {"a":"c", "b":"d"} get a }}`
`${{ 'ab' get 0 }}`
`${{ 'a' in ['a', 'b'] }}`
`${{ 'b' in {"a":"c", "b":"d"} }}`
`${{ 'd' in {"a":"c", "b":"d"} }}`
`${{ !false }}`
`${{ 'hello' !in ['hello', 'goodbye'] }}`
`${{ true && (true || true) && false }}`
`${{ 10 / (2 + 8) }}`
`${{ true && false ? 'a' : 'b' }}`
`${{ 'truthy value' ? 'a' : 'b' }}`
```

--------------------------------

### Wait For Agent API

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

Waits for an agent to become ready, typically used after initiating an agent with `wait: false`.

```APIDOC
## POST /websites/parallelworks/wait-for-agent

### Description
This action should only be used if you already used scheduler-agent with `wait: false` passed as an input to the `with` property, which might be a good idea if you want the workflow job to perform other operations while the scheduler-agent node is configuring.

### Method
POST

### Endpoint
/websites/parallelworks/wait-for-agent

### Parameters
#### Inputs
- **agentId** (string) - Required - The id of the agent to wait for. The output from the scheduler-agent action includes this field (see usage in example above).

### Request Example
```json
{
  "agentId": "agent-12345"
}
```

### Response
#### Success Response (200)
- **remoteHost** (string) - The remote host address for the agent.
- **sshPort** (int) - The SSH port for the agent.

#### Response Example
```json
{
  "remoteHost": "192.168.1.100",
  "sshPort": 22
}
```
```

--------------------------------

### Update Session with Information

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

Updates an existing session with specified details. This action is crucial for initializing session information, especially when sessions lack initial display data. It supports various session types like 'link' and 'tunnel' and allows configuration of target compute clusters and session status.

```yaml
uses: parallelworks/update-session
with:
  status: running
  name: ${{ sessions.session }}
  target: ${{ inputs.resource.id }}
  remoteHost: ${{ needs.main.steps.waitstep.outputs.remoteHost }}
  remotePort: ${{ needs.main.outputs.sessionPort }}
```

--------------------------------

### Project Configuration API

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

This section details the configuration parameters available for a ParallelWorks project, including job definitions, environment variables, and execution timeouts.

```APIDOC
## Project Configuration Details

### Description
This endpoint provides access to the configuration schema for a ParallelWorks project, outlining all possible inputs for job execution, session management, and general project settings.

### Method
GET

### Endpoint
/websites/parallelworks

### Parameters
#### Query Parameters
None

#### Request Body
None

### Response
#### Success Response (200)
- **jobs** (object) - Defines the jobs that can be executed within the project.
- **jobs.<job>.needs** (array) - Specifies dependencies for a job.
- **jobs.<job>.steps** (array) - A list of steps to be executed within a job.
- **jobs.<job>.steps[*].name** (string) - The name of a step.
- **jobs.<job>.steps[*].run** (string) - The command to run for a step.
- **jobs.<job>.steps[*].uses** (string) - A reusable action or script to use for a step.
- **jobs.<job>.steps[*].with** (object) - Input parameters for a reusable action.
- **jobs.<job>.steps[*].ssh** (boolean) - Enables SSH access for a step.
- **jobs.<job>.steps[*].cleanup** (boolean) - Enables cleanup after a step.
- **jobs.<job>.steps[*].if** (string) - Conditional execution for a step.
- **jobs.<job>.steps[*].retry** (integer) - Number of times to retry a step.
- **jobs.<job>.steps[*].env** (object) - Environment variables for a step.
- **jobs.<job>.steps[*].ignore-errors** (boolean) - Ignores errors for a step.
- **jobs.<job>.steps[*].working-directory** (string) - The working directory for a step.
- **jobs.<job>.steps[*].early-cancel** (boolean) - Enables early cancellation for a step.
- **jobs.<job>.steps[*].timeout** (integer) - Timeout for a step in seconds.
- **jobs.<job>.ssh** (boolean) - Enables SSH access for a job.
- **jobs.<job>.cleanup** (boolean) - Enables cleanup after a job.
- **jobs.<job>.if** (string) - Conditional execution for a job.
- **jobs.<job>.env** (object) - Environment variables for a job.
- **jobs.<job>.working-directory** (string) - The working directory for a job.
- **jobs.<job>.timeout** (integer) - Timeout for a job in seconds.
- **sessions** (object) - Defines session configurations.
- **sessions.<session>.type** (string) - The type of session.
- **sessions.<session>.openAI** (object) - OpenAI specific configurations for a session.
- **sessions.<session>.prompt-for-name** (boolean) - Prompts for session name.
- **sessions.<session>.redirect** (string) - Session redirect URL.
- **sessions.<session>.useTLS** (boolean) - Enables TLS for the session.
- **sessions.<session>.useCustomDomain** (boolean) - Uses a custom domain for the session.
- **permissions** (object) - Defines permissions for the project.
- **configurations** (object) - General project configurations.
- **env** (object) - Global environment variables for the project.
- **timeout** (integer) - Default timeout for the project in seconds.
- **on** (object) - Defines triggers for project execution.
- **on.execute.inputs** (object) - Input parameters for the execute trigger.

#### Response Example
```json
{
  "jobs": {},
  "sessions": {},
  "permissions": {},
  "configurations": {},
  "env": {},
  "timeout": 3600,
  "on": {
    "execute": {
      "inputs": {}
    }
  }
}
```
```

--------------------------------

### Checkout Git Repository using Parallel Works

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

The 'checkout' action clones a git repository. It supports specifying the repository URL, branch, and optionally a sparse checkout for specific files or directories. It can also clone to a cluster's file system using SSH.

```yaml
jobs:
  job-name:
    steps:
      - uses: parallelworks/checkout
        with:
          repo: https://github.com/parallelworks/interactive_session.git
          branch: main
```

```yaml
jobs:
  job-name:
    steps:
      - uses: parallelworks/checkout
        with:
          repo: https://github.com/parallelworks/interactive_session.git
          branch: main
          sparse_checkout:
            - utils
            - platforms
```

```yaml
jobs:
  job-name:
    steps:
      - uses: parallelworks/checkout
        with:
          repo: https://github.com/parallelworks/interactive_session.git
          branch: main
        ssh:
          remoteHost: ${{ inputs.cluster.ip }}
on:
  execute:
    inputs:
      cluster:
        type: compute-clusters
        optional: false
```

--------------------------------

### Access Input Schema Using Bracket Notation

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

An alternative method to conditionally hide an input using the 'inputs' keyword and bracket notation to access schema properties.

```yaml
on:
  execute:
    inputs:
      ...
      option_1_input:
        type: string
        hidden: "${{ some long and complicated boolean expression }}"
      option_2_input:
        type: string
        hidden: "${{ !inputs.option_1_input[hidden] }}"
```

--------------------------------

### Define Workflow Configurations

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

Save and reuse inputs for workflows using the `configurations` field. This allows for consistent and time-saving input management. Configurations can be defined in the workflow YAML or saved by users.

```yaml
configurations:
  config_1:
    variables:
      input_1: hello!
jobs:
  echo:
    steps:
      - run: echo ${{ inputs.input_1 }}
on:
  execute:
    inputs:
      input_1:
        type: string
```

--------------------------------

### Reference External Workflows or Actions with 'uses' (YAML)

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

The 'uses' field allows a step to execute another workflow from the current repository, a personal workflow, or a workflow from the Marketplace. It cannot be used with the 'run' field. Versions can be specified for Marketplace workflows.

```yaml
jobs:
  job-name:
    steps:
      - name: Sample Step
        uses: marketplace/default-local-workflow
        with:
          resource: sample-cluster
```

--------------------------------

### Execute Multi-line Commands in a Step (YAML)

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

The 'run' field within a step allows for the execution of shell commands. Multiple bash commands can be combined into a single step using the multi-line format.

```yaml
jobs:
  main:
    steps:
      - run: |
          echo part 1
          echo part 2
```

--------------------------------

### Wait for Agent to be Ready

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/actions

Waits for an agent to become ready, typically used after initiating the scheduler-agent with 'wait: false'. This allows the workflow to perform other tasks concurrently while the agent configures. It requires the agentId obtained from the scheduler-agent action.

```yaml
uses: parallelworks/wait-for-agent
id: waitstep
with:
  agentId: ${{ needs.main.steps.slurmstep.outputs.agentId }}
```

--------------------------------

### Dynamic Input Visibility Based on Boolean Toggle

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Configures conditional visibility for 'advanced_setting' and 'advanced_setting2' inputs. 'advanced_setting' is hidden based on the boolean value of 'show_advanced', while 'advanced_setting2' is hidden based on the 'hidden' attribute of 'advanced_setting'.

```yaml
on:
  execute:
    inputs:
      header:
        type: header
        text: Starter Local Workflow
        size: 20
      show_advanced:
        type: boolean
        label: Show Advanced Settings
        default: false
      advanced_setting:
        type: string
        label: Advanced Setting
        hidden: "${{ !inputs.show_advanced }}"
      advanced_setting2:
        type: string
        label: Advanced Setting 2
        hidden: "${{ inputs.advanced_setting[hidden] }}"
```

--------------------------------

### Accessing Job Outputs in Parallelworks

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Demonstrates how to access job outputs using the `needs` context. This allows jobs to share information, either with themselves or with dependent jobs, by referencing the outputs of previous steps or jobs.

```yaml
jobs:
  job1:
    steps:
      - name: fake output
        run: echo PORT=3001 >> $OUTPUTS
      - name: Echo output from self
        run: echo ${{ needs.job1.outputs.PORT }} # Prints 3001
  job2:
    needs:
      - job1
    steps:
      - name: Echo output from job 1
        run: echo ${{ needs.job1.outputs.PORT }} # Prints 3001
```

--------------------------------

### Define Cleanup Commands for Job Steps

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

Specifies a command to execute after a job step finishes, regardless of its success or failure. This is useful for removing temporary files or releasing resources. Cleanup commands run in reverse order of their definition.

```yaml
jobs:
  my_job:
    steps:
      - run: echo "Running main step"
        cleanup: rm -rf /tmp/myapp
```

--------------------------------

### Define Workflow Inputs in YAML

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

This snippet shows the YAML structure for defining workflow inputs. Inputs are configured under the 'on.execute.inputs' key. Each input has a name, a type, and potentially other type-specific attributes. Input names must consist of alphanumeric characters, dashes, and underscores.

```yaml
on:
  execute:
    inputs:
      name_of_input_1:
        type: type_of_input_1
        ... other fields (potentially type specific) ...
      name-of-input-2:
        type: type_of_input_2
        ... other fields (potentially type specific) ...
```

--------------------------------

### Implement Early Cancellation for Steps

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

Allows a step to be cancelled prematurely based on certain conditions. Currently, the only supported condition is 'any-job-failed', which cancels the step if any preceding job fails.

```yaml
jobs:
  my_job:
    steps:
      - run: potentially_long_running_task
        early_cancel: any_job_failed
```

--------------------------------

### Specify Working Directory for Steps

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

Sets the directory in which a job step will be executed. This is useful for navigating to specific project directories before running commands.

```yaml
jobs:
  my_job:
    steps:
      - run: echo "Running in specific directory"
        working_directory: /path/to/my/project
```

--------------------------------

### Configure Step Retries

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

Enables automatic retries for steps that have a chance of failure. You can configure the maximum number of retries, the interval between retries, and the timeout for each attempt.

```yaml
jobs:
  my_job:
    steps:
      - run: potentially_failing_command
        retry:
          max_retries: 5
          interval: 1m
          timeout: 1h
```

--------------------------------

### SSH Configuration for Remote Execution

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

This section details how to configure SSH access for executing commands on remote hosts, including specifying remote host details, usernames, and jump nodes.

```APIDOC
## `jobs.<job>.ssh`

### Description
Configures SSH access for executing commands on a remote host. It allows specifying the remote host's IP address, the username for SSH connection, and optionally, the IP address and username for a jump node.

### Method
N/A (Configuration Property)

### Endpoint
N/A

### Parameters
#### Path Parameters
N/A

#### Query Parameters
N/A

#### Request Body
N/A

### Request Example
```yaml
jobs:
  echo_on_remote:
    ssh:
      remoteHost: ${{ inputs.resource1.ip }}
      jumpNodeHost: ${{ inputs.resource2.ip }}
    steps:
      - run: echo "This is executed on the remote host!"
      - run: echo "This is executed on the jump node!"
        ssh:
          remoteHost: ${{ inputs.resource2.ip }}
      - run: echo "This is executed in the user workspace!"
        ssh: null

on:
  execute:
    inputs:
      resource1:
        label: Resource 1
        type: compute-clusters
        optional: false
      resource2:
        label: Resource 2
        type: compute-clusters
        optional: false
```

### Response
#### Success Response (200)
N/A

#### Response Example
N/A
```

--------------------------------

### Define Workflow Jobs and Steps (YAML)

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/yaml-fields

The 'jobs' field is essential for defining workflow execution. Each job must have 'steps', and by default, jobs run in parallel. Dependencies can be managed using the 'needs' field.

```yaml
jobs:
  job-name:
    steps:
      - name: Step 1
        run: echo step 1
      - name: Step 2
        run: echo step 2
```

--------------------------------

### Set Default Input Value Using Other Inputs

Source: https://parallelworks.com/docs/run/workflows-and-apps/building-workflows/inputs-and-expressions

Demonstrates setting a default value for the 'workdir' input based on the 'username' input. This uses string concatenation within the default value expression.

```yaml
on:
  execute:
    inputs:
      username:
        type: string
      workdir:
        type: string
        default: "${{ '~/path/to/directory/' + inputs.username }}"
```
Tokens:
Raw