### Example Spawn Command

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

A simple example of a spawn command for an engine.

```text
{engine_path}/gulp < INPUT > OUTPUT
```

--------------------------------

### Example Engine Platform Packages

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Specify a space or newline-separated list of packages to be installed by the system package manager for a given engine platform. Defaults to 'debian-10'.

```text
openmpi-bin wget
```

--------------------------------

### Install OpenSSH Server on Windows

Source: https://github.com/tilde-lab/yascheduler/blob/master/WINDOWS.md

Installs the OpenSSH server using a provided MSI package. It downloads the installer if not found locally and installs it silently. Ensure you have administrative privileges.

```powershell
function New-TemporaryDirectory {
    $parent = [System.IO.Path]::GetTempPath()
    [string] $name = [System.Guid]::NewGuid()
    New-Item -ItemType Directory -Path (Join-Path $parent $name)
}

$distroFilename = "OpenSSH-Win64-v8.9.1.0.msi"
$distroUrl = "https://github.com/PowerShell/Win32-OpenSSH/releases/download/v8.9.1.0p1-Beta/$distroFilename"

$tmpDir = New-TemporaryDirectory
$distroFilepath = Join-Path $tmpDir $distroFilename
if (Test-Path "$distroFilename") {
    Copy-Item "$distroFilename" "$distroFilepath"
} else {
    Invoke-WebRequest -Uri "$distroUrl" -OutFile "$distroFilepath"
}
Start-Process "$env:windir\System32\msiexec.exe" -ArgumentList "/i `"$distroFilepath`" /qn" -Wait
Remove-Item -Recurse $tmpDir
```

--------------------------------

### CLI Tool: yainit

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Initializes the PostgreSQL database schema and installs the system service (systemd or SysV). Must be run after configuring `/etc/yascheduler/yascheduler.conf`.

```APIDOC
## CLI Tool: yainit - Initialize Database and Service

The `yainit` command initializes the PostgreSQL database schema and installs the system service (systemd or SysV). Must be run after configuring `/etc/yascheduler/yascheduler.conf`.

### Usage
```bash
# Initialize yascheduler database and service
sudo yainit

# Output:
# Installing systemd service
# Database initialized!

# Or for SysV systems:
# Installing SysV service
# Database initialized!
```

### Post-Initialization Steps

After initialization, start the service:

```bash
# For systemd:
sudo systemctl start yascheduler

# For SysV:
sudo service yascheduler start
```

Check service status:

```bash
# For systemd:
sudo systemctl status yascheduler
```
```

--------------------------------

### Example Remote Archive Deployment

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Provide a URL to a .tar.gz archive that will be downloaded and unarchived on the remote machine. Conflicts with other deployment methods.

```text
https://example.org/dummyengine.tar.gz
```

--------------------------------

### Example Local File Deployment

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

List filenames separated by space or newline that will be copied from the local engine directory to the remote engine directory. Conflicts with archive deployments.

```text
dummyengine
```

--------------------------------

### Install Yascheduler with Cloud Connectors

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Install Yascheduler with specific cloud connectors using pip. For example, to install with Azure support, use 'pip install yascheduler[azure]'.

```sh
pip install yascheduler[azure]
```

```sh
pip install yascheduler[hetzner]
```

```sh
pip install yascheduler[upcloud]
```

--------------------------------

### AiiDA Integration Setup Commands

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Commands to set up AiiDA integration, including SSH connection, reentry scan, and Verdi computer/code setup and testing.

```sh
ssh aiidauser@localhost # important
reentry scan
verdi computer setup
verdi computer test $COMPUTER
verdi code setup

```

--------------------------------

### Enable and Start OpenSSH Service

Source: https://github.com/tilde-lab/yascheduler/blob/master/WINDOWS.md

Configures the OpenSSH service (sshd) to start automatically on system boot and then starts the service immediately. This ensures that the SSH server is running and accessible.

```powershell
Set-Service -Name sshd -StartupType 'Automatic'
Start-Service sshd
```

--------------------------------

### Install Yascheduler from Source

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Clone the repository and install Yascheduler locally. This method is useful for obtaining the latest updates and bugfixes.

```sh
git clone https://github.com/tilde-lab/yascheduler.git
pip install yascheduler/
```

--------------------------------

### Example Input Files for Task

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

A list of input filenames, separated by space or newline, to be copied to the remote task directory before execution. The first file is the main input.

```text
INPUT sibling.file
```

--------------------------------

### Example Output Files from Task

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

A list of output filenames, separated by space or newline, to be copied back from the remote task directory after completion.

```text
INPUT OUTPUT
```

--------------------------------

### Initialize Packer Configuration

Source: https://github.com/tilde-lab/yascheduler/blob/master/examples/own-vm-images/README.md

Run this command to download and install the required Packer plugins for your configuration file. Ensure the path to your .pkr.hcl file is correct.

```sh
packer init ./hcloud-debian-12-fleur.pkr.hcl
```

--------------------------------

### Example Local Archive Deployment

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Specify the name of a local .tar.gz archive to be copied and unarchived on the remote machine. Conflicts with other deployment methods.

```text
dummyengine.tar.gz
```

--------------------------------

### Supervisor Configuration for Yascheduler

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Example configuration for running Yascheduler as a supervisor service. This defines the command, user, logging, and restart policies.

```ini
[program:scheduler]
command=/usr/local/bin/yascheduler
user=root
autostart=true
autorestart=true
stderr_logfile=/data/yascheduler.log
stdout_logfile=/data/yascheduler.log
```

--------------------------------

### Start Yascheduler Daemon - Bash

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Starts the scheduler daemon, which processes the task queue, manages cloud nodes, and handles task distribution. Logging levels can be adjusted using the `-l` flag. Recommended for production use with a process manager like supervisor.

```bash
# Start scheduler in foreground with default log level (INFO)
yascheduler

# Start with debug logging for troubleshooting
yascheduler -l DEBUG

# Start with warning level only
yascheduler -l WARNING

# The daemon logs to /var/log/yascheduler.log by default
# Log format: timestamp - yascheduler - level - message
# Example output:
# 2024-01-15 10:30:00 - yascheduler - INFO - THREADS: 15 NODES: busy:2/enabled:5/total:8 TASKS: run:2/todo:10/done:150
# 2024-01-15 10:30:00 - yascheduler - INFO - QUEUES: conn_machine: 0/0 allocate: 2/3 deallocate: 0/0 consume: 0/1

# Using supervisor (recommended for production)
# /etc/supervisor/conf.d/yascheduler.conf:
# [program:scheduler]
# command=/usr/local/bin/yascheduler
# user=root
# autostart=true
# autorestart=true
# stderr_logfile=/data/yascheduler.log
# stdout_logfile=/data/yascheduler.log
```

--------------------------------

### Initialize Yascheduler Database and Service - Bash

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Initializes the PostgreSQL database schema and installs the system service (systemd or SysV). This command requires root privileges and must be run after configuring `/etc/yascheduler/yascheduler.conf`.

```bash
# Initialize yascheduler database and service
sudo yainit

# Output:
# Installing systemd service
# Database initialized!

# Or for SysV systems:
# Installing SysV service
# Database initialized!

# After initialization, start the service
sudo systemctl start yascheduler
# or
sudo service yascheduler start

# Check service status
sudo systemctl status yascheduler
```

--------------------------------

### Python: Complete Yascheduler Workflow

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

This script shows how to initialize the Yascheduler client, check available engines, prepare and submit a task with metadata, monitor its progress, and retrieve results. It also demonstrates how to get a summary of all tasks.

```python
#!/usr/bin/env python3
"""Complete yascheduler workflow example"""

import os
import time
from pathlib import Path
from yascheduler import Yascheduler

def main():
    # Initialize client
    yac = Yascheduler()

    # Check available engines
    print("Available engines:")
    for name, engine in yac.config.engines.items():
        print(f"  {name}: platforms={engine.platforms}, inputs={engine.input_files}")

    # Prepare calculation inputs
    work_dir = Path("/path/to/calculation")

    # Read input files
    with open(work_dir / "INPUT", "r") as f:
        main_input = f.read()
    with open(work_dir / "fort.34", "r") as f:
        structure_input = f.read()

    # Submit the task
    task_id = yac.queue_submit_task(
        label="Production calculation - NaCl optimization",
        metadata={
            "INPUT": main_input,
            "fort.34": structure_input,
            "local_folder": str(work_dir / "results"),
            "webhook_url": "https://myserver.com/api/task-complete",
            "webhook_custom_params": {
                "project": "materials-discovery",
                "calculation_type": "geometry_optimization"
            }
        },
        engine_name="pcrystal",
        webhook_onsubmit=True
    )
    print(f"Submitted task {task_id}")

    # Monitor progress
    while True:
        task = yac.queue_get_task(task_id)
        status_names = {0: "PENDING", 1: "RUNNING", 2: "DONE"}
        print(f"Task {task_id}: {status_names[task['status']]}")

        if task['status'] == yac.STATUS_RUNNING:
            print(f"  Running on: {task['ip']}")
            print(f"  Remote path: {task['metadata'].get('remote_folder')}")

        if task['status'] == yac.STATUS_DONE:
            meta = task['metadata']
            local_folder = meta.get('local_folder')

            if 'error' in meta:
                print(f"  ERROR: {meta['error']}")
            else:
                print(f"  Results saved to: {local_folder}")
                # List output files
                for f in Path(local_folder).iterdir():
                    print(f"    - {f.name} ({f.stat().st_size} bytes)")
            break

        time.sleep(60)

    # Get summary of all tasks
    print("\nTask Summary:")
    for status, name in [(0, "Pending"), (1, "Running"), (2, "Completed")]:
        tasks = yac.queue_get_tasks(status=[status])
        print(f"  {name}: {len(tasks)}")

if __name__ == "__main__":
    main()

```

--------------------------------

### CLI Tool: yascheduler

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Starts the scheduler daemon which processes the task queue, manages cloud nodes, and handles task distribution.

```APIDOC
## CLI Tool: yascheduler - Start the Daemon

The `yascheduler` command starts the scheduler daemon which processes the task queue, manages cloud nodes, and handles task distribution.

### Usage

Start the scheduler in the foreground with a specified log level:

```bash
# Start with default log level (INFO)
yascheduler

# Start with debug logging for troubleshooting
yascheduler -l DEBUG

# Start with warning level only
yascheduler -l WARNING
```

### Logging

The daemon logs to `/var/log/yascheduler.log` by default. The log format is:
`timestamp - yascheduler - level - message`

**Example Output:**
```
2024-01-15 10:30:00 - yascheduler - INFO - THREADS: 15 NODES: busy:2/enabled:5/total:8 TASKS: run:2/todo:10/done:150
2024-01-15 10:30:00 - yascheduler - INFO - QUEUES: conn_machine: 0/0 allocate: 2/3 deallocate: 0/0 consume: 0/1
```

### Production Deployment (using supervisor)

For production environments, it is recommended to use a process control system like `supervisor`.

**Example `/etc/supervisor/conf.d/yascheduler.conf`:**
```ini
[program:scheduler]
command=/usr/local/bin/yascheduler
user=root
autostart=true
autorestart=true
stderr_logfile=/data/yascheduler.log
stdout_logfile=/data/yascheduler.log
```
```

--------------------------------

### Example Command for Task Check

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

A command used to check if a task is still running. Conflicts with `check_pname`. The exit code is checked against `check_cmd_code`.

```sh
ps ax -ocomm= | grep -q dummyengine
```

--------------------------------

### Submit a New Task

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Submit a calculation task with a label, metadata including input files, and the target engine. Optionally configure webhook notifications for task submission, start, and completion.

```python
from yascheduler import Yascheduler

yac = Yascheduler()

# Prepare input data - must include all files required by the engine
label = "NaCl crystal optimization"
engine = "pcrystal"

# Read input files
with open("INPUT", "r") as f:
    setup_input = f.read()
with open("fort.34", "r") as f:
    struct_input = f.read()

# Submit task with required input files
metadata = {
    "INPUT": setup_input,        # Main setup file
    "fort.34": struct_input,     # Crystal structure file
    "local_folder": "/path/to/save/results",  # Optional: where to save results locally
}

task_id = yac.queue_submit_task(label, metadata, engine)
print(f"Task submitted with ID: {task_id}")

# Submit task with webhook notification
metadata_with_webhook = {
    "INPUT": setup_input,
    "fort.34": struct_input,
    "webhook_url": "https://my-server.com/webhook",
    "webhook_custom_params": {"job_type": "optimization", "user": "scientist1"}
}

task_id = yac.queue_submit_task(
    label="NaCl optimization with webhook",
    metadata=metadata_with_webhook,
    engine_name="pcrystal",
    webhook_onsubmit=True  # Send webhook when task is submitted
)
# Webhooks are also sent when task starts running and when it completes
```

--------------------------------

### Yascheduler Environment Variable Configuration

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Override default paths for configuration, log, and PID files using environment variables. Example shows running yascheduler with custom paths and debug logging.

```bash
# Override configuration file location
export YASCHEDULER_CONF_PATH=/custom/path/yascheduler.conf

# Override log file location
export YASCHEDULER_LOG_PATH=/var/log/custom-yascheduler.log

# Override PID file location
export YASCHEDULER_PID_PATH=/var/run/custom-yascheduler.pid

# Example: Run with custom paths
YASCHEDULER_CONF_PATH=/etc/yascheduler/production.conf \
YASCHEDULER_LOG_PATH=/data/logs/yascheduler.log \
yascheduler -l DEBUG
```

--------------------------------

### Hetzner Cloud Snapshot ID Output

Source: https://github.com/tilde-lab/yascheduler/blob/master/examples/own-vm-images/README.md

This is an example of the output you will see upon successful completion of the Packer build, indicating the ID of the newly created snapshot. Remember this ID for your yascheduler configuration.

```sh
--> hcloud.debian: A snapshot was created: 'fleur-debian-xxxxx' (ID: 123123123)
```

--------------------------------

### Get Single Task by ID - Python

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Retrieves a single task by its ID and polls its status until completion. Ensure the Yascheduler client is initialized.

```python
from yascheduler import Yascheduler
import time

yac = Yascheduler()

task_id = 42  # Previously submitted task

# Poll task status until completion
while True:
    task = yac.queue_get_task(task_id)

    if task is None:
        print(f"Task {task_id} not found")
        break

    print(f"Task {task['task_id']}: {task['label']}")
    print(f"  Status: {task['status']} ({['TO_DO', 'RUNNING', 'DONE'][task['status']]})")

    if task['status'] == yac.STATUS_DONE:
        metadata = task['metadata']
        print(f"  Local folder: {metadata.get('local_folder')}")
        print(f"  Remote folder: {metadata.get('remote_folder')}")
        if 'error' in metadata:
            print(f"  Error occurred: {metadata['error']}")
        break

    time.sleep(30)  # Check every 30 seconds
```

--------------------------------

### Add OpenSSH to System PATH

Source: https://github.com/tilde-lab/yascheduler/blob/master/WINDOWS.md

Appends the OpenSSH installation directory to the system's PATH environment variable, allowing SSH commands to be executed from any location. This change requires administrative privileges.

```powershell
# Append the Win32-OpenSSH install directory to the system path
[Environment]::SetEnvironmentVariable(
    "Path",
    $env:Path + ';' + ${Env:ProgramFiles} + '\OpenSSH',
    [System.EnvironmentVariableTarget]::Machine
)
```

--------------------------------

### queue_get_task - Get Single Task by ID

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Retrieves a single task by its unique ID. Returns the task dictionary or None if not found. This is a convenience method that wraps queue_get_tasks for single task lookup.

```APIDOC
## queue_get_task - Get Single Task by ID

### Description
Retrieves a single task by its unique ID. Returns the task dictionary or None if not found. This is a convenience method that wraps `queue_get_tasks` for single task lookup.

### Method
GET

### Endpoint
`/queue/task/{task_id}`

### Parameters
#### Path Parameters
- **task_id** (int) - Required - The unique identifier of the task to retrieve.

### Request Example
```python
from yascheduler import Yascheduler

yac = Yascheduler()
task_id = 42
task = yac.queue_get_task(task_id)
print(task)
```

### Response
#### Success Response (200)
- **task** (dict) - A dictionary containing the task details, or None if the task is not found.

#### Response Example
```json
{
  "task_id": 42,
  "label": "Example Task",
  "status": 2, // Corresponds to STATUS_DONE
  "metadata": {
    "INPUT": "...",
    "fort.34": "...",
    "local_folder": "/path/to/local/task/data",
    "remote_folder": "s3://bucket/path/to/task/data"
  }
}
```
```

--------------------------------

### Yascheduler Client Initialization and Configuration

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Demonstrates how to initialize the Yascheduler client, with options for default or custom configuration paths, and how to access task status constants and configured engines.

```APIDOC
## Yascheduler Client Initialization

### Description
Initializes the Yascheduler client, which serves as the primary interface for interacting with the scheduler. It can be configured with a default path or a custom configuration file.

### Method
`__init__`

### Parameters
- **config_path** (str) - Optional - Path to the custom configuration file.

### Request Example
```python
from yascheduler import Yascheduler

# Initialize with default config path
yac = Yascheduler()

# Initialize with a custom configuration file
yac = Yascheduler(config_path="/path/to/custom/yascheduler.conf")
```

### Response
- **Yascheduler instance** - An initialized Yascheduler client object.

### Accessing Configuration and Constants

#### Description
Access task status constants (e.g., `STATUS_TO_DO`, `STATUS_RUNNING`, `STATUS_DONE`) and configured engines from the initialized client.

#### Code Example
```python
print(f"TO_DO status: {yac.STATUS_TO_DO}")
print(f"RUNNING status: {yac.STATUS_RUNNING}")
print(f"DONE status: {yac.STATUS_DONE}")

for engine_name, engine in yac.config.engines.items():
    print(f"Engine: {engine_name}")
    print(f"  Platforms: {engine.platforms}")
    print(f"  Input files: {engine.input_files}")
    print(f"  Output files: {engine.output_files}")
```
```

--------------------------------

### Build OS Image with Packer

Source: https://github.com/tilde-lab/yascheduler/blob/master/examples/own-vm-images/README.md

Execute this command to build the OS image using your Packer configuration file. This process can take a significant amount of time. The output will include the ID of the created snapshot.

```sh
packer build ./hcloud-debian-12-fleur.pkr.hcl
```

--------------------------------

### View Convergence Analysis

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Use 'yastatus --view --convergence' to view convergence analysis. Requires pycrystal.

```bash
yastatus --view --convergence
```

--------------------------------

### Initialize Yascheduler Client

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Instantiate the Yascheduler client, optionally specifying a custom configuration file path. Access task status constants and configured engines.

```python
from yascheduler import Yascheduler

# Initialize the client with default config path (/etc/yascheduler/yascheduler.conf)
yac = Yascheduler()

# Or specify a custom configuration file
yac = Yascheduler(config_path="/path/to/custom/yascheduler.conf")

# Access task status constants
print(f"TO_DO status: {yac.STATUS_TO_DO}")      # 0
print(f"RUNNING status: {yac.STATUS_RUNNING}")  # 1
print(f"DONE status: {yac.STATUS_DONE}")        # 2

# Access configured engines
for engine_name, engine in yac.config.engines.items():
    print(f"Engine: {engine_name}")
    print(f"  Platforms: {engine.platforms}")
    print(f"  Input files: {engine.input_files}")
    print(f"  Output files: {engine.output_files}")
```

--------------------------------

### List Azure Subscriptions

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Use this command to list available Azure subscriptions and identify the `subscriptionId` needed for configuration.

```sh
az account subscription list
```

--------------------------------

### Create Azure Jump Host VM

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Creates a jump host virtual machine using a Debian 11 image. This VM is configured with a static public IP address and associated with the created VNet and NSG. It requires a private SSH key for access. The public IP address should be saved as `az_jump_host`.

```bash
az vm create \
    -g yascheduler-rg -l westeurope \
    --name yascheduler-jump-host \
    --image Debian11 \
    --size Standard_B1s \
    --nsg yascheduler-nsg \
    --public-ip-address yascheduler-jump-host-ip \
    --public-ip-address-allocation static \
    --public-ip-sku Standard \
    --vnet-name yascheduler-vnet \
    --subnet yascheduler-subnet \
    --admin-username yascheduler \
    --ssh-key-values "$(ssh-keygen -y -f path/to/private/key)"
```

--------------------------------

### Yascheduler Configuration for Engines and Clouds

Source: https://github.com/tilde-lab/yascheduler/blob/master/examples/own-vm-images/README.md

Configure the 'engine' sections for inpgen and fleur, specifying their spawn commands, check commands, and input/output files. The 'clouds' section defines cloud provider credentials and settings, including the Hetzner token, server type, location, and the OS image name (snapshot ID).

```ini
[engine.inpgen]
spawn = inpgen -explicit -inc +all -f aiida.in > shell.out 2> out.error
check_cmd = ps -eocomm= | grep -q inpgen
input_files = aiida.in
output_files = aiida.in inp.xml default.econfig shell.out out out.error scratch struct.xsf

[engine.fleur]
spawn = fleur -minimalOutput -wtime 360 > shell.out 2> out.error
check_cmd = ps -eocomm= | grep -q fleur
input_files = inp.xml
output_files = inp.xml kpts.xml sym.xml relax.xml shell.out out.error out out.xml FleurInputSchema.xsd FleurOutputSchema.xsd juDFT_times.json cdn1 usage.json

[clouds]
; Your API key
hetzner_token = xxx
; Your preffered server type.
; The disk should be at least as large as the snapshot.
hetzner_server_type = cpx21
; It's best to use the same location as the snapshot.
; This way servers will be created faster.
hetzner_location = fsn1
; OS image ID that you should have memorized earlier.
; You can always look under Snapshots in HCloud Dashboard.
hetzner_image_name = 237472643
```

--------------------------------

### Async Task Submission and Retrieval - Python

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Demonstrates submitting multiple tasks concurrently and retrieving their status asynchronously using the `_async` methods. Requires an asyncio event loop.

```python
import asyncio
from yascheduler import Yascheduler

async def submit_multiple_tasks():
    yac = Yascheduler()

    # Prepare multiple inputs
    calculations = [
        {"label": "Calc 1", "INPUT": "...", "fort.34": "..."},
        {"label": "Calc 2", "INPUT": "...", "fort.34": "..."},
        {"label": "Calc 3", "INPUT": "...", "fort.34": "..."},
    ]

    # Submit tasks concurrently
    tasks = []
    for calc in calculations:
        task = yac.queue_submit_task_async(
            label=calc["label"],
            metadata={"INPUT": calc["INPUT"], "fort.34": calc["fort.34"]},
            engine_name="pcrystal"
        )
        tasks.append(task)

    task_ids = await asyncio.gather(*tasks)
    print(f"Submitted tasks: {task_ids}")

    # Query tasks asynchronously
    all_tasks = await yac.queue_get_tasks_async(jobs=list(task_ids))
    for task in all_tasks:
        print(f"Task {task['task_id']}: {task['status']}")

    # Get single task async
    first_task = await yac.queue_get_task_async(task_ids[0])
    print(f"First task status: {first_task['status']}")

# Run the async function
asyncio.run(submit_multiple_tasks())
```

--------------------------------

### Yascheduler Configuration File

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

The main configuration file defines database credentials, storage paths, cloud providers, and compute engines. It uses an INI-like format with sections for different configurations.

```ini
# /etc/yascheduler/yascheduler.conf

[db]
user = yascheduler
password = secretpassword
database = yascheduler
host = localhost
port = 5432

[local]
data_dir = /srv/yascheduler
tasks_dir = %(data_dir)s/tasks
keys_dir = %(data_dir)s/keys
engines_dir = %(data_dir)s/engines
# Concurrency limits
webhook_reqs_limit = 5
conn_machine_limit = 10
allocate_limit = 20
consume_limit = 20
deallocate_limit = 5

[remote]
data_dir = ./data
tasks_dir = %(data_dir)s/tasks
engines_dir = %(data_dir)s/engines
user = root
# Optional jump host for SSH tunneling
# jump_host = bastion.example.com
# jump_user = tunnel

[clouds]
# Hetzner Cloud configuration
hetzner_token = your-api-token-here
hetzner_max_nodes = 10
hetzner_server_type = cx52
hetzner_location = fsn1
hetzner_image_name = debian-12
hetzner_idle_tolerance = 300
hetzner_priority = 100

# Azure Cloud configuration
az_tenant_id = your-tenant-id
az_client_id = your-client-id
az_client_secret = your-client-secret
az_subscription_id = your-subscription-id
az_max_nodes = 5
az_resource_group = yascheduler-rg
az_location = westeurope
az_vm_size = Standard_D4s_v3
az_user = azureuser
az_priority = 50

# UpCloud configuration
upcloud_login = your-username
upcloud_password = your-password
upcloud_max_nodes = 3
upcloud_priority = 25

[engine.pcrystal]
platforms = debian debian-11 debian-12
platform_packages = openmpi-bin wget
deploy_local_files = Pcrystal
spawn = cp {task_path}/INPUT OUTPUT && mpirun -np {ncpus} --allow-run-as-root -wd {task_path} {engine_path}/Pcrystal >> OUTPUT 2>&1
check_pname = Pcrystal
sleep_interval = 6
input_files =
    INPUT
    fort.34
output_files =
    INPUT
    fort.34
    OUTPUT
    fort.9
    fort.87

[engine.gulp]
platforms = debian
deploy_local_files = gulp
spawn = {engine_path}/gulp < INPUT > OUTPUT
check_pname = gulp
sleep_interval = 10
input_files = INPUT
output_files = INPUT OUTPUT

[engine.custom]
platforms = debian-12
deploy_remote_archive = https://example.com/myengine.tar.gz
spawn = {engine_path}/myengine --input {task_path}/input.dat --output {task_path}/output.dat
check_cmd = ps ax -ocomm= | grep -q myengine
check_cmd_code = 0
sleep_interval = 30
input_files = input.dat config.json
output_files = input.dat output.dat log.txt

```

--------------------------------

### Spawn Command for Calculations

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

This command is used by the scheduler to initiate calculations. It copies input to output, runs the mpirun command in the task path, and redirects output.

```sh
cp {task_path}/INPUT OUTPUT && mpirun -np {ncpus} --allow-run-as-root \
    -wd {task_path} {engine_path}/Pcrystal >> OUTPUT 2>&1

```

--------------------------------

### Add New Compute Node

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Use 'yasetnode' to add a new compute node. It supports custom SSH ports, users, and CPU limits. The '--skip-setup' flag can be used if the node is already configured.

```bash
# Add a new node (will setup engines automatically)
yasetnode 192.168.1.100
```

```bash
# Add node with custom user, port, and CPU limit
yasetnode admin@192.168.1.101:2222~4
```

```bash
# Add node without running setup (node already configured)
yasetnode 192.168.1.102 --skip-setup
```

--------------------------------

### Check Task Status - Bash

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Displays the current status of tasks. Options include showing all active tasks, specific tasks by ID, or detailed information including remote output. The `--view` option connects via SSH to show live output.

```bash
# Show all running and pending tasks (brief format)
yastatus
# Output:
# 42   RUNNING
# 43   TO_DO
# 44   TO_DO

# Show detailed info for all active tasks
yastatus --info
# Output:
# task_id=42	status=RUNNING	label=NaCl optimization	ip=192.168.1.100
task_id=43	status=TO_DO	label=MgO calculation	ip=-
task_id=44	status=TO_DO	label=CaF2 structure	ip=-

# Show specific tasks by ID
yastatus --jobs 42 43 100
# Output:
# 42   RUNNING
# 43   TO_DO
# 100  DONE

# View live output from running tasks (connects via SSH)
yastatus --view
# Output:
# ..................................................ID42 NaCl optimization at root@192.168.1.100:hetzner:/data/tasks/20240115_103000_42
# CRYSTAL - SCF iteration  15
# Total energy:  -1234.567890 Ha
# ...
```

--------------------------------

### Submit Task via AiiDA Script

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

The 'yasubmit' command submits tasks using an AiiDA-style script file. Ensure necessary input files exist in the current directory before submission.

```bash
# Create a submission script (submit.ya)
cat > submit.ya << EOF
ENGINE=pcrystal
LABEL=NaCl structure optimization
PARENT=abc123-def456-uuid
EOF
```

```bash
# Submit the task
yasubmit submit.ya
```

--------------------------------

### Create Azure Resource Group

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Creates a dedicated resource group for yascheduler resources in a specified location. Ensure the location and group name are appropriate for your deployment.

```bash
az group create -l westeurope -g yascheduler-rg
```

--------------------------------

### queue_submit_task - Submit a New Task

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Submits a new calculation task to the scheduler. This includes providing a label, metadata (input files, local folder, webhook details), and the target engine name. It returns a unique task ID.

```APIDOC
## queue_submit_task - Submit a New Task

### Description
Submits a new calculation task to the scheduler queue. The task includes a label for identification, metadata containing input files, and the target engine name. Returns the unique task ID for tracking.

### Method
`queue_submit_task(label: str, metadata: dict, engine_name: str, webhook_onsubmit: bool = False, webhook_onstart: bool = False, webhook_onfinish: bool = False)`

### Parameters
- **label** (str) - Required - A human-readable label for the task.
- **metadata** (dict) - Required - A dictionary containing task-specific data. This typically includes input file contents and can optionally include `local_folder` for results, `webhook_url`, and `webhook_custom_params`.
  - **INPUT** (str) - Content of the main setup file.
  - **[other_input_files]** (str) - Content of other required input files.
  - **local_folder** (str) - Optional - The local path where results should be saved.
  - **webhook_url** (str) - Optional - The URL to send webhook notifications to.
  - **webhook_custom_params** (dict) - Optional - Custom parameters to include in webhook notifications.
- **engine_name** (str) - Required - The name of the engine to use for the calculation.
- **webhook_onsubmit** (bool) - Optional - If True, sends a webhook notification when the task is submitted.
- **webhook_onstart** (bool) - Optional - If True, sends a webhook notification when the task starts running.
- **webhook_onfinish** (bool) - Optional - If True, sends a webhook notification when the task completes.

### Request Example
```python
from yascheduler import Yascheduler

yac = Yascheduler()

# Prepare input data
label = "NaCl crystal optimization"
engine = "pcrystal"

with open("INPUT", "r") as f:
    setup_input = f.read()
with open("fort.34", "r") as f:
    struct_input = f.read()

metadata = {
    "INPUT": setup_input,
    "fort.34": struct_input,
    "local_folder": "/path/to/save/results"
}

task_id = yac.queue_submit_task(label, metadata, engine)
print(f"Task submitted with ID: {task_id}")

# Submit task with webhook notification
metadata_with_webhook = {
    "INPUT": setup_input,
    "fort.34": struct_input,
    "webhook_url": "https://my-server.com/webhook",
    "webhook_custom_params": {"job_type": "optimization", "user": "scientist1"}
}

task_id = yac.queue_submit_task(
    label="NaCl optimization with webhook",
    metadata=metadata_with_webhook,
    engine_name="pcrystal",
    webhook_onsubmit=True
)
```

### Response
- **task_id** (int) - The unique identifier assigned to the submitted task.
```

--------------------------------

### Create Azure Enterprise Application

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Creates an Enterprise Application in Azure Active Directory for yascheduler. The `appId` generated is required for subsequent role assignments and configuration.

```bash
az ad app create --display-name yascheduler
```

--------------------------------

### queue_get_tasks - Query Tasks by Status or IDs

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Retrieves tasks from the scheduler database, allowing filtering by their current status or a list of specific task IDs. Returns a list of task dictionaries.

```APIDOC
## queue_get_tasks - Query Tasks by Status or IDs

### Description
Retrieves tasks from the database filtered by their IDs or current status. Returns a list of task dictionaries with all metadata. Cannot filter by both jobs and status simultaneously.

### Method
`queue_get_tasks(jobs: list = None, status: list = None)`

### Parameters
- **jobs** (list) - Optional - A list of task IDs to retrieve.
- **status** (list) - Optional - A list of status codes to filter tasks by. Use constants like `yac.STATUS_TO_DO`, `yac.STATUS_RUNNING`, `yac.STATUS_DONE`.

### Request Example
```python
from yascheduler import Yascheduler

yac = Yascheduler()

# Get all running and pending tasks
pending_tasks = yac.queue_get_tasks(status=[yac.STATUS_TO_DO])
running_tasks = yac.queue_get_tasks(status=[yac.STATUS_RUNNING])
completed_tasks = yac.queue_get_tasks(status=[yac.STATUS_DONE])

print(f"Pending: {len(pending_tasks)}, Running: {len(running_tasks)}, Done: {len(completed_tasks)}")

# Get multiple statuses at once
active_tasks = yac.queue_get_tasks(status=[yac.STATUS_TO_DO, yac.STATUS_RUNNING])
for task in active_tasks:
    print(f"Task {task['task_id']}: {task['label']} - Status: {task['status']}")
    print(f"  IP: {task['ip']}")
    print(f"  Engine: {task['metadata'].get('engine')}")
    print(f"  Remote folder: {task['metadata'].get('remote_folder')}")

# Get specific tasks by ID
specific_tasks = yac.queue_get_tasks(jobs=[1, 5, 10, 15])
for task in specific_tasks:
    print(f"Task {task['task_id']}: {task['label']}")
    if task['status'] == yac.STATUS_DONE:
        local_folder = task['metadata'].get('local_folder')
        print(f"  Results saved to: {local_folder}")
        if 'error' in task['metadata']:
            print(f"  Error: {task['metadata']['error']}")
```

### Response
- **list of task dictionaries** - Each dictionary represents a task and contains details such as `task_id`, `label`, `status`, `ip`, and `metadata`.
```

--------------------------------

### AiiDA Integration with Yascheduler

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Configure AiiDA to use yascheduler as a plugin for job submission and monitoring. Ensure reentry is scanned and the computer/code are set up.

```bash
# Configure AiiDA to use yascheduler

# 1. First, scan for the plugin
# $ reentry scan

# 2. Set up the computer in AiiDA
# $ verdi computer setup
# scheduler: yascheduler
# transport: local (yascheduler handles SSH)

# 3. Test the computer
# $ verdi computer test mycomputer

# 4. Set up the code
# $ verdi code setup
```

```python
# Example AiiDA workflow using yascheduler
from aiida import orm, engine
from aiida.plugins import CalculationFactory

PcrystalCalculation = CalculationFactory('crystal.main')

builder = PcrystalCalculation.get_builder()
builder.code = orm.load_code('pcrystal@yascheduler-cluster')
builder.structure = structure  # AiiDA StructureData
builder.parameters = parameters  # Calculation parameters

# Submit - yascheduler handles distribution to cloud nodes
result = engine.submit(builder)

# The yascheduler plugin maps:
# - Job submission -> yasubmit script
# - Job status query -> yastatus --jobs
# - Job states: TO_DO->QUEUED, RUNNING->RUNNING, DONE->DONE
```

--------------------------------

### Show All Registered Nodes

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

The 'yanodes' command displays information about all registered compute nodes, including their status and current tasks. Custom SSH ports can be specified.

```bash
# Show all registered nodes
yanodes
```

```bash
# Node with custom port
# ip=192.168.1.103:2222 ncpus=4 enabled=True occupied_by=- (task_id=-)
```

--------------------------------

### Create Azure Network Security Group and Rules

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Creates a Network Security Group (NSG) and adds a rule to allow SSH (port 22) and RDP (port 3389) traffic from any source. This is a prerequisite for creating virtual machines within the network.

```bash
az network nsg create \
    -g yascheduler-rg -l westeurope \
    -n yascheduler-nsg
```

```bash
az network nsg rule create \
    -g yascheduler-rg --nsg-name yascheduler-nsg \
    --name allow-ssh-rdp --priority 100 \
    --source-address-prefixes '*' \
    --destination-port-ranges 22 3389 \
    --protocol TCP --access Allow
```

--------------------------------

### Query Tasks by Status or IDs

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Retrieve tasks from the database filtered by their current status or specific IDs. Returns a list of task dictionaries. Cannot filter by both jobs and status simultaneously.

```python
from yascheduler import Yascheduler

yac = Yascheduler()

# Get all running and pending tasks
pending_tasks = yac.queue_get_tasks(status=[yac.STATUS_TO_DO])
running_tasks = yac.queue_get_tasks(status=[yac.STATUS_RUNNING])
completed_tasks = yac.queue_get_tasks(status=[yac.STATUS_DONE])

print(f"Pending: {len(pending_tasks)}, Running: {len(running_tasks)}, Done: {len(completed_tasks)}")

# Get multiple statuses at once
active_tasks = yac.queue_get_tasks(status=[yac.STATUS_TO_DO, yac.STATUS_RUNNING])
for task in active_tasks:
    print(f"Task {task['task_id']}: {task['label']} - Status: {task['status']}")
    print(f"  IP: {task['ip']}")
    print(f"  Engine: {task['metadata'].get('engine')}")
    print(f"  Remote folder: {task['metadata'].get('remote_folder')}")

# Get specific tasks by ID
specific_tasks = yac.queue_get_tasks(jobs=[1, 5, 10, 15])
for task in specific_tasks:
    print(f"Task {task['task_id']}: {task['label']}")
    if task['status'] == yac.STATUS_DONE:
        local_folder = task['metadata'].get('local_folder')
        print(f"  Results saved to: {local_folder}")
        if 'error' in task['metadata']:
            print(f"  Error: {task['metadata']['error']}")
```

--------------------------------

### Create Azure Virtual Network and Subnet

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Creates a virtual network (VNet) and a subnet within it, associated with the previously created NSG. This defines the internal network space for yascheduler resources.

```bash
az network vnet create \
    -g yascheduler-rg -l westeurope --nsg yascheduler-nsg \
    --name yascheduler-vnet --address-prefix 10.0.0.0/16 \
    --subnet-name yascheduler-subnet \
    --subnet-prefix 10.0.0.0/22
```

--------------------------------

### CLI Tool: yastatus

Source: https://context7.com/tilde-lab/yascheduler/llms.txt

Displays the current status of tasks. It can show all active tasks, specific tasks by ID, or detailed information including remote output.

```APIDOC
## CLI Tool: yastatus - Check Task Status

The `yastatus` command displays the current status of tasks. It can show all active tasks, specific tasks by ID, or detailed information including remote output.

### Usage

**Show all running and pending tasks (brief format):**
```bash
yastatus

# Output:
# 42   RUNNING
# 43   TO_DO
# 44   TO_DO
```

**Show detailed info for all active tasks:**
```bash
yastatus --info

# Output:
# task_id=42	status=RUNNING	label=NaCl optimization	ip=192.168.1.100
# task_id=43	status=TO_DO	label=MgO calculation	ip=-
# task_id=44	status=TO_DO	label=CaF2 structure	ip=-
```

**Show specific tasks by ID:**
```bash
yastatus --jobs 42 43 100

# Output:
# 42   RUNNING
# 43   TO_DO
# 100  DONE
```

**View live output from running tasks (connects via SSH):**
```bash
yastatus --view

# Output:
# ..................................................ID42 NaCl optimization at root@192.168.1.100:hetzner:/data/tasks/20240115_103000_42
# CRYSTAL - SCF iteration  15
# Total energy:  -1234.567890 Ha
# ...
```
```

--------------------------------

### Submit a Task with Yascheduler

Source: https://github.com/tilde-lab/yascheduler/blob/master/README.md

Use the Yascheduler Python API to submit a task. This involves creating a Yascheduler instance and calling the queue_submit_task method with task details and the engine name.

```python
from yascheduler import Yascheduler

yac = Yascheduler()
label = "test assignment"
engine = "pcrystal"
struct_input = str(...)  # simulation control file: crystal structure
setup_input = str(...)  # simulation control file: main setup, can include struct_input
result = yac.queue_submit_task(
    label, {"fort.34": struct_input, "INPUT": setup_input}, engine
)
print(result)
```

--------------------------------

### Clean Up SSH Keys for Cloud Images

Source: https://github.com/tilde-lab/yascheduler/blob/master/WINDOWS.md

Removes all SSH host keys and clears the authorized keys file. This is a critical step when preparing a Windows machine image for cloud deployment to prevent security issues related to pre-existing keys.

```powershell
Clear-Content "$env:PROGRAMDATA\ssh\administrators_authorized_keys"
Remove-Item "$env:PROGRAMDATA\ssh\ssh_host_ecdsa_key"
Remove-Item "$env:PROGRAMDATA\ssh\ssh_host_ec25519_key"
Remove-Item "$env:PROGRAMDATA\ssh\ssh_host_dsa_key"
Remove-Item "$env:PROGRAMDATA\ssh\ssh_host_rsa_key"
```

--------------------------------

### Reset Azure Application Credentials

Source: https://github.com/tilde-lab/yascheduler/blob/master/CLOUD.md

Resets the credentials for an Azure application registration, generating a new client secret. This command appends the new secret, which is then used as `az_client_secret` in the cloud configuration. Replace the placeholder `appId` with your actual application ID.

```bash
az ad app credential reset --id 00000000-0000-0000-0000-000000000000 --append
```