Tork

https://www.tork.run/
Admin
Tork is an open-source workflow engine designed for scale with simplicity in mind, supporting task...
Tokens:27,183
Snippets:195
Trust Score:6.2
Update:11 months ago
Show doc for...
Tokens:
Raw
### Run Tork in Standalone Mode

Source: https://www.tork.run/installation

This command starts Tork in 'standalone' mode. This mode is suitable for single-machine setups and simplifies initial testing and development by running all components locally without needing external services like RabbitMQ.

```shell
./tork run standalone
```

--------------------------------

### Run Tork Coordinator

Source: https://www.tork.run/installation

This command starts the Tork coordinator process. The coordinator is responsible for managing and distributing tasks to worker nodes in a distributed setup. It assumes Tork is installed and accessible in the current PATH or directory.

```shell
./tork run coordinator
```

--------------------------------

### Create Tork Directory and Navigate

Source: https://www.tork.run/installation

This snippet creates a directory named 'tork' in the user's home directory and then changes the current directory to the newly created 'tork' directory. This is a standard shell operation for organizing project files.

```shell
mkdir ~/tork
cd ~/tork
```

--------------------------------

### Start Minio Server (Docker)

Source: https://www.tork.run/video-transcoding

This command starts a Minio server as a Docker container. Minio provides a self-hosted S3-compatible object storage service, used here for intermediate and final video outputs. It maps ports 9000 and 9001 and binds the /data directory for persistent storage. The --name flag assigns a name to the container for easy management.

```bash
docker run -d \
  --name minio \
  -p 9000:9000 \
  -p 9001:9001 minio/minio \
  server /data \
  --console-address ":9001"

```

--------------------------------

### Start Postgres Database (Docker)

Source: https://www.tork.run/video-transcoding

This command starts a PostgreSQL database container using the official Docker image. It's configured with user 'tork', password 'tork', and database name 'tork'. The PGDATA environment variable sets the data directory, and a volume is mounted to persist data. Port 5432 is exposed for connections.

```bash
docker run -d \
  --name tork-postgres \
  -e POSTGRES_DB=tork \
  -e POSTGRES_PASSWORD=tork \
  -e POSTGRES_USER=tork \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -v $(pwd)/data:/var/lib/postgresql/data \
  -p 6432:5432 \
  postgres:15

```

--------------------------------

### Start RabbitMQ Management Container with Docker

Source: https://www.tork.run/installation

This Docker command starts a RabbitMQ container with the management plugin enabled, running in detached mode. It names the container 'tork-rabbit', maps ports 5672 (AMQP) and 15672 (Management UI) from the container to the host, and uses the 'rabbitmq:3-management' image.

```shell
docker run \
  -d \
  --name=tork-rabbit \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management
```

--------------------------------

### Start PostgreSQL Container with Docker

Source: https://www.tork.run/installation

This Docker command runs a PostgreSQL container in detached mode ('-d'). It names the container 'tork-postgres', maps port 5432 from the container to the host, sets the PostgreSQL user, password, and database name using environment variables, and specifies the data directory. It uses the 'postgres:15.3' image.

```shell
docker run -d \
  --name tork-postgres \
  -p 5432:5432 \
  -e POSTGRES_PASSWORD=tork \
  -e POSTGRES_USER=tork \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -e POSTGRES_DB=tork postgres:15.3
```

--------------------------------

### Start PostgreSQL Container

Source: https://www.tork.run/quick-start

Starts a PostgreSQL container named 'tork-postgres' accessible on the host's port 5432. It configures the user, password, and data directory for PostgreSQL. Suitable for local development and testing.

```docker
docker run -d \
  --name tork-postgres \
  -p 5432:5432 \
  -e POSTGRES_PASSWORD=tork \
  -e POSTGRES_USER=tork \
  -e PGDATA=/var/lib/postgresql/data/pgdata \
  -e POSTGRES_DB=tork postgres:15.3

```

--------------------------------

### Run Tork Executable

Source: https://www.tork.run/installation

This command executes the Tork binary. The '%' symbol suggests it might be run in a specific shell environment like zsh, but it's a standard way to invoke an executable file in the current directory.

```shell
% ./tork
```

--------------------------------

### Run Tork Worker

Source: https://www.tork.run/installation

This command starts a Tork worker process. Worker nodes are responsible for executing tasks assigned by the coordinator. Multiple workers can be started to increase processing capacity. This command assumes Tork is installed and accessible.

```shell
./tork run worker
```

--------------------------------

### Download Tork Binary (Linux)

Source: https://www.tork.run/video-transcoding

This snippet downloads the Tork binary for Linux AMD64 architecture using wget. It saves the downloaded archive as 'tork.tgz'. Ensure you have wget installed and sufficient permissions.

```bash
wget https://github.com/runabol/tork/releases/download/v0.1.66/tork_0.1.66_linux_amd64.tgz -O tork.tgz

```

--------------------------------

### Run Tork Database Migration

Source: https://www.tork.run/installation

This command runs the Tork database migration script. It sets the TORK_DATASTORE_TYPE environment variable to 'postgres' to indicate that Tork should connect to a PostgreSQL database for the migration process. This command creates the necessary database schema.

```shell
TORK_DATASTORE_TYPE=postgres ./tork migration
```

--------------------------------

### Start MinIO S3-Compatible Object Store

Source: https://www.tork.run/quick-start

Launches a MinIO container, an S3-compatible object storage service, for local use. It exposes ports 9000 and 9001 and sets the root user credentials to 'minioadmin'.

```docker
docker run --name=tork-minio \
  -d -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  minio/minio server /data \
  --console-address ":9001"

```

--------------------------------

### Start RabbitMQ Container

Source: https://www.tork.run/video-transcoding

This command starts a RabbitMQ container in detached mode with management UI enabled. It maps the broker port (5672) and management port (15672) to the host machine.

```docker
docker run -d \
  --name tork-rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management
```

--------------------------------

### Define and Submit a Tork Job

Source: https://www.tork.run/installation

This snippet first defines a simple Tork job configuration in a file named 'hello.yaml'. The job consists of two tasks that echo messages. Subsequently, it uses 'curl' to send this YAML file as a POST request to the Tork API endpoint '/jobs'. The output job ID is captured using 'jq' and stored in the JOB_ID variable for later use.

```shell
# hello.yaml
---
name: hello job
tasks:
  - name: say hello
    image: ubuntu:mantic #docker image
    run: |
      echo -n hello world
  - name: say goodbye
    image: ubuntu:mantic
    run: |
      echo -n bye world

JOB_ID=$(curl -s -X POST --data-binary @hello.yaml \
  -H "Content-type: text/yaml" http://localhost:8000/jobs | jq -r .id)
```

--------------------------------

### Initialize and Run Tork as a Go Library

Source: https://www.tork.run/extend

Demonstrates how to set up a new Go project, initialize it with Go modules, add Tork as a dependency, and run Tork with minimal boilerplate code. This is the starting point for using Tork programmatically.

```bash
mkdir tork-plus
cd tork-plus
go mod init github.com/example/tork-plus
go get github.com/runabol/tork
go mod tidy
go run main.go
```

```go
package main

import (
	"fmt"
	"os"

	"github.com/runabol/tork/cli"
	"github.com/runabol/tork/conf"
)

func main() {
	if err := conf.LoadConfig(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	if err := cli.New().Run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}
```

--------------------------------

### Tork Task Routing to Specific Queue

Source: https://www.tork.run/installation

Defines a Tork task and explicitly routes it to a specific queue, in this case, the 'video' queue. This is useful for directing tasks to specialized worker pools. The task definition includes properties like 'name', 'queue', 'image', and 'run' commands.

```yaml
name: transcode a video
queue: video
image: jrottenberg/ffmpeg:3.4-alpine
run: |
  ffmpeg \
    -i https://example.com/some/video.mov \
    output.mp4
```

--------------------------------

### Run Tork Coordinator

Source: https://www.tork.run/quick-start

Starts the Tork coordinator in distributed mode, configured to use PostgreSQL for datastore and RabbitMQ as the message broker. This process oversees task distribution.

```bash
TORK_DATASTORE_TYPE=postgres TORK_BROKER_TYPE=rabbitmq ./tork run coordinator

```

--------------------------------

### Unpack Tork Binary Archive

Source: https://www.tork.run/installation

This command unpacks a gzipped tar archive containing the Tork binary. It assumes the archive is located in the 'Downloads' directory and is named 'tork_0.1.66_darwin_arm64.tgz'. The 'x' flag extracts, 'z' indicates gzip compression, 'v' enables verbose output, and 'f' specifies the filename.

```shell
tar xzvf ~/Downloads/tork_0.1.66_darwin_arm64.tgz
```

--------------------------------

### Start Tork Web UI (Docker)

Source: https://www.tork.run/video-transcoding

This command launches the Tork Web UI as a Docker container. It exposes port 8100 for accessing the UI and configures the backend URL to communicate with the Tork coordinator running on the host machine. The '--add-host' flag is used to resolve 'host.docker.internal' to the host's gateway.

```bash
docker run \
  --name=tork-web \
  -d \
  -p 8100:8100 \
  --add-host=host.docker.internal:host-gateway \
  -e BACKEND_URL=http://host.docker.internal:8000 \
  runabol/tork-web

```

--------------------------------

### Run Minio Docker Container (Shell)

Source: https://www.tork.run/resizing-images

This command starts a Minio server instance in a Docker container. It maps ports 9000 and 9001, connects the container to the 'minio' network, and sets the data directory. The console address is also configured.

```shell
docker run -d \
  --name minio \
  --network minio \
  -p 9000:9000 \
  -p 9001:9001 minio/minio \
  server /data \
  --console-address ":9001"
```

--------------------------------

### Spawn a Sub-Job Task

Source: https://www.tork.run/tasks

Illustrates how a task can start another job using the `subjob` property. The parent task will wait for the sub-job to complete or fail. This example defines a simple sub-job with two tasks that print messages.

```yaml
- name: a task that starts a sub-job
  subjob:
    name: my sub job
    tasks:
      - name: hello sub task
        image: ubuntu:mantic
        run: echo start of sub-job
      - name: bye task
        image: ubuntu:mantic
        run: echo end of sub-job

```

--------------------------------

### Query Tork Job Status

Source: https://www.tork.run/installation

This command queries the status of a Tork job using its ID. It sends a GET request to the Tork API endpoint '/jobs/{JOB_ID}', where {JOB_ID} is a placeholder for the actual job identifier. The '-s' flag suppresses progress meters and error messages, making the output cleaner.

```shell
curl -s http://localhost:8000/jobs/$JOB_ID
```

--------------------------------

### Install Tork Web with Docker

Source: https://www.tork.run/web-ui

This snippet shows the Docker command to run the Tork Web UI. It maps port 3000 from the container to the host and sets the backend URL for Tork. Ensure the BACKEND_URL points to your Tork backend service.

```docker
docker run \
  -it \
  --rm \
  --name=tork-web \
  -p 3000:3000 \
  -e BACKEND_URL=http://my.tork.host:8000 \
  runabol/tork-web
```

--------------------------------

### Override Tork configuration using environment variables

Source: https://www.tork.run/config

This example shows how to override specific Tork configuration settings using environment variables. The format is TORK_ followed by the configuration property name, with dots replaced by underscores. This example sets the logging level to 'warn'.

```bash
TORK_LOGGING_LEVEL=warn

```

--------------------------------

### Tork Worker Multi-Queue Configuration

Source: https://www.tork.run/installation

Sets up a Tork worker to consume tasks from multiple queues, each with its own concurrency limit. This allows for specialized worker pools. It requires a `config.toml` file and assumes RabbitMQ is the broker. Tasks can be routed using the 'queue' property in the task definition.

```toml
# config.toml
[worker.queues]
default = 5
video = 2

[broker]
type = "rabbitmq"
```

--------------------------------

### Define a simple Tork Job with two tasks

Source: https://www.tork.run/jobs

This example demonstrates how to define a basic Tork Job with two sequential tasks. The first task 'say hello' writes 'hello world' to an output file, and the second task 'say goodbye' writes 'bye world'. It uses the 'ubuntu:mantic' image for both tasks and shows how to specify a variable to capture output.

```yaml
name: hello job
tasks:
  - name: say hello
    var: task1
    image: ubuntu:mantic
    run: |
      echo -n hello world > $TORK_OUTPUT
  - name: say goodbye
    image: ubuntu:mantic
    run: |
      echo -n bye world
```

--------------------------------

### Run Tork Worker

Source: https://www.tork.run/quick-start

Starts a Tork worker in distributed mode, configured to use RabbitMQ as the message broker. Workers pick up tasks from the broker and execute them. Multiple workers can be started to scale execution.

```bash
TORK_BROKER_TYPE=rabbitmq ./tork run worker

```

--------------------------------

### Start Tork in Standalone Mode

Source: https://www.tork.run/ci

Command to run Tork in standalone mode for local testing. This mode handles job scheduling and task execution in a single process.

```bash
./tork run standalone

```

--------------------------------

### Mount a Host Path using Bind Mount

Source: https://www.tork.run/tasks

Illustrates using a `bind` mount to make a host path (`/host/path`) accessible within the container at the specified `target` path (`/mnt/videos`). This allows tasks to interact with data residing on the host machine. The example shows ffmpeg processing a video file located on the host.

```yaml
- name: convert the first 5 seconds of a video
  image: jrottenberg/ffmpeg:3.4-alpine
  run: ffmpeg -i /mnt/videos/my_video.mov -t 5 /mnt/videos/output.mp4
  mounts:
    - type: bind
      target: /mnt/videos
      source: /host/path

```

--------------------------------

### Get Job Details

Source: https://www.tork.run/rest

Retrieves the details of a specific job.

```APIDOC
## GET /jobs/<JOB ID>

### Description
Retrieves the details of a specific job.

### Method
GET

### Endpoint
/jobs/<JOB ID>

### Parameters
#### Path Parameters
- **JOB ID** (string) - Required - The unique identifier of the job.

### Response
#### Success Response (200)
- **id** (string) - The unique identifier for the job.
- **name** (string) - The name of the job.
- **state** (string) - The current state of the job.
- **createdAt** (string) - The timestamp when the job was created.
- **startedAt** (string) - The timestamp when the job started.
- **completedAt** (string) - The timestamp when the job completed.
- **tasks** (array) - An array of task definitions within the job.
  - **name** (string) - The name of the task.
  - **run** (string) - The command to run for the task.
  - **image** (string) - The Docker image to use for the task.
- **execution** (array) - An array of execution details for each task.
  - **id** (string) - The unique identifier for the execution.
  - **jobId** (string) - The ID of the job this execution belongs to.
  - **position** (integer) - The position of the task in the job.
  - **name** (string) - The name of the task.
  - **state** (string) - The state of the execution.
  - **createdAt** (string) - The timestamp when the execution was created.
  - **scheduledAt** (string) - The timestamp when the execution was scheduled.
  - **startedAt** (string) - The timestamp when the execution started.
  - **completedAt** (string) - The timestamp when the execution completed.
  - **run** (string) - The command executed.
  - **image** (string) - The Docker image used.
  - **queue** (string) - The queue the task was processed on.
  - **nodeId** (string) - The ID of the node where the task was executed.
- **position** (integer) - The position of the job in the sequence.
- **taskCount** (integer) - The total number of tasks in the job.

### Response Example
```json
{
  "id": "9d7d7184b8f244249c5f5d8b4074f72e",
  "name": "my job",
  "state": "COMPLETED",
  "createdAt": "2023-09-05T01:33:08.413605Z",
  "startedAt": "2023-09-05T01:33:08.414339Z",
  "completedAt": "2023-09-05T01:33:14.167127Z",
  "tasks": [
    {
      "name": "my first task",
      "run": "echo hello world",
      "image": "alpine:3.18.3"
    }
  ],
  "execution": [
    {
      "id": "aa76535bc23649ca9403e636e377aca3",
      "jobId": "9d7d7184b8f244249c5f5d8b4074f72e",
      "position": 1,
      "name": "my first task",
      "state": "COMPLETED",
      "createdAt": "2023-09-05T01:33:08.414192Z",
      "scheduledAt": "2023-09-05T01:33:08.414378Z",
      "startedAt": "2023-09-05T01:33:08.414574Z",
      "completedAt": "2023-09-05T01:33:08.699749Z",
      "run": "echo hello world",
      "image": "alpine:3.18.3",
      "queue": "default",
      "nodeId": "93a1cd11800741e6863d1adef3bc3d18"
    }
  ],
  "position": 4,
  "taskCount": 3
}
```
```

--------------------------------

### Run Tork Worker Process

Source: https://www.tork.run/video-transcoding

This command starts the Tork worker in the background. It redirects standard output and standard error to a log file ('tork.log') and saves the process ID to 'tork.pid'.

```bash
nohup ./tork run worker > tork.log 2>&1 & echo $! > tork.pid
```

--------------------------------

### Tork Job: Stateful Example with MinIO using AWS CLI

Source: https://www.tork.run/quick-start

A Tork job definition in YAML format that creates a stateful workflow. It uses the `amazon/aws-cli:latest` Docker image for two tasks: writing a file to a MinIO bucket and then reading it back. The job requires MinIO endpoint details and credentials as inputs and secrets.

```yaml
name: stateful example
inputs:
  minio_endpoint: http://host.docker.internal:9000
secrets:
  minio_user: minioadmin
  minio_password: minioadmin
tasks:
  - name: write data to object store
    image: amazon/aws-cli:latest
    env:
      AWS_ACCESS_KEY_ID: "{{ secrets.minio_user }}"
      AWS_SECRET_ACCESS_KEY: "{{ secrets.minio_password }}"
      AWS_ENDPOINT_URL: "{{ inputs.minio_endpoint }}"
      AWS_DEFAULT_REGION: us-east-1
    run: |
      echo "Hello from Tork!" > /tmp/data.txt
      aws s3 mb s3://mybucket
      aws s3 cp /tmp/data.txt s3://mybucket/data.txt

  - name: read data from object store
    image: amazon/aws-cli:latest
    env:
      AWS_ACCESS_KEY_ID: "{{ secrets.minio_user }}"
      AWS_SECRET_ACCESS_KEY: "{{ secrets.minio_password }}"
      AWS_ENDPOINT_URL: "{{ inputs.minio_endpoint }}"
      AWS_DEFAULT_REGION: us-east-1
    run: |
      aws s3 cp s3://mybucket/data.txt /tmp/retrieved.txt
      echo "Contents of retrieved file:"
      cat /tmp/retrieved.txt

```

--------------------------------

### Run Tork with a specific config file using environment variable

Source: https://www.tork.run/config

This example demonstrates how to run Tork in standalone mode, specifying a custom configuration file path using the TORK_CONFIG environment variable. Ensure the Tork executable and the specified config file exist in the execution path.

```bash
TORK_CONFIG=myconfig.toml ./tork run standalone

```

--------------------------------

### Start RabbitMQ Container

Source: https://www.tork.run/quick-start

Launches a RabbitMQ container in detached mode, exposing management ports 5672 and 15672. This is used as a message broker for Tork's distributed mode.

```docker
docker run \
  -d -p 5672:5672 -p 15672:15672 \
  --name=tork-rabbitmq \
  rabbitmq:3-management

```

--------------------------------

### Print String Length using Built-in Function

Source: https://www.tork.run/tasks

Demonstrates using the built-in 'len' function to get the length of a string and store it in an environment variable. The length is then printed to the console. This requires the 'hello world' string to be provided as input to the 'len' function.

```yaml
- name: print the length of a string
  image: ubuntu:mantic
  env:
    LENGTH: '{{ len("hello world") }}'
  run: |
    echo "The length of the string is: $LENGTH"

```

--------------------------------

### Implement HTTP Middleware for Timing Requests in Tork

Source: https://www.tork.run/extend

Illustrates how to create and register HTTP middleware in Tork to intercept requests to the Coordinator API. This example defines a middleware function that measures and logs the processing time of incoming HTTP requests.

```go
package main

import (
	"time"
	"github.com/runabol/tork/middleware/web"
	"log"
	"github.com/runabol/tork/cli"
	"github.com/runabol/tork/conf"
	"github.com/runabol/tork/engine"
)

func main() {
	if err := conf.LoadConfig(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	mw := func(next web.HandlerFunc) web.HandlerFunc {
		return func(c web.Context) error {
			before := time.Now()
			// happens before the request is processed
			err := next(c)
			// happens after the request is processed
			log.Info().Msgf("The request to %s took %s to process",
				c.Request().URL.Path,
				time.Since(before),
			)
			return err
		}
	}

	engine.RegisterWebMiddleware(mw)

	if err := cli.New().Run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}
```

--------------------------------

### Docker config file for private registry authentication

Source: https://www.tork.run/tasks

Provides an example of a Docker configuration file used for authenticating with private registries. This JSON structure contains authentication details that can be passed to the Tork worker via an environment variable.

```json
{
  "auths": {
    "myregistry.com": {
      "auth": "base64encodedusername:base64encodedpassword"
    }
  }
}
```

--------------------------------

### Query Tork Job Status via HTTP GET

Source: https://www.tork.run/ci

Command to query the status of a submitted Tork job using its ID. It retrieves the job information and extracts the state using `jq`.

```bash
curl -s http://localhost:8000/jobs/$JOB_ID | jq -r .state

COMPLETED


```

--------------------------------

### Tork API Response for Scheduled Job Creation (JSON)

Source: https://www.tork.run/jobs

This is an example of the JSON response received from the Tork API after successfully scheduling a job. It includes the job's unique ID, its current state, the name provided, the creation timestamp, and the configured cron schedule.

```json
{
  "id": "c90188cce61244a1aabcdbedf31f51d6",
  "state": "ACTIVE",
  "name": "scheduled job test",
  "createdAt": "2024-12-21T14:53:05.709793Z",
  "cron": "0/5 * * * *"
}

```

--------------------------------

### Define a Tork Job with video conversion input

Source: https://www.tork.run/jobs

This example defines a Tork Job named 'mov to mp4' that utilizes job inputs. It specifies a 'source' input for the video file URL and uses this input within the task's environment variables to pass the video URL to the ffmpeg command for conversion.

```yaml
name: mov to mp4
inputs:
  source: https://example.com/path/to/video.mov
tasks:
  - name: convert the video to mp4
    image: jrottenberg/ffmpeg:3.4-alpine
    env:
      SOURCE_URL: '{{ inputs.source }}'
    run: |
      ffmpeg -i $SOURCE_URL /tmp/output.mp4
```

--------------------------------

### Run Tork in Standalone Mode

Source: https://www.tork.run/video-transcoding

This command starts the Tork process in 'standalone' mode, which combines the Coordinator and Worker functionalities. The output and errors are redirected to 'tork.log', and the process ID is saved to 'tork.pid'. The `nohup` command ensures the process continues running after the terminal session is closed.

```bash
nohup ./tork run standalone > tork.log 2>&1 & echo $!

```

--------------------------------

### Define a Tork Job with default task configurations

Source: https://www.tork.run/jobs

This example shows how to set default configurations for all tasks within a Tork Job using the 'defaults' block. It specifies default retry limits, resource limits (CPU and memory), timeouts, default queues, and priorities, which can be overridden at the individual task level.

```yaml
name: my job
defaults:
  retry:
    # a task will retry up to 2 times in case of a failure
    limit: 2
  # resource limits imposed on a task
  limits:
    # 1 CPU limit
    cpus: 1
    # 500MB of RAM limit
    memory: 500m
  # a task will automatically fail if not completed within 10 minutes
  timeout: 10m
  # tasks will be routed to the highcpu queue by default
  queue: highcpu
  # values between 0-9. Higher numbers mean higher priority
  priority: 3
tasks:
  - name: my task
    queue: default # override the job defaults
    image: alpine:latest
    run: |
      echo hello world
```

--------------------------------

### Iterate Tasks with Each Task (Sequence)

Source: https://www.tork.run/tasks

Demonstrates the `each` task for executing a task multiple times based on a list. This example uses `sequence(1,5)` to generate a list of numbers, and for each number, it sets an environment variable `ITEM` and `INDEX` and echoes a message. The `concurrency` property limits the number of parallel executions.

```yaml
- name: sample each task
  each:
    list: '{{ sequence(1,5) }}'
    concurrency: 3 # max 3 tasks would execute concurrently
    task:
      image: ubuntu:mantic
      env:
        ITEM: '{{ item.value }}'
        INDEX: '{{ item.index }}'
      run: echo -n HELLO $ITEM at $INDEX

```

--------------------------------

### Tork Configuration File (config.toml)

Source: https://www.tork.run/video-transcoding

This TOML configuration file sets up the Tork coordinator and datastore. It specifies the coordinator address and configures the datastore to use PostgreSQL with the provided connection details (host, user, password, database, port). Ensure this file is placed next to the tork binary.

```toml
[coordinator]
address = "0.0.0.0:8000"

[datastore]
type = "postgres"

[datastore.postgres]
dsn = "host=localhost user=tork password=tork dbname=tork port=6432 sslmode=disable"

```

--------------------------------

### Initialize Tork Database Migrations

Source: https://www.tork.run/video-transcoding

This command initializes the database schema for Tork by running its migration scripts. It assumes the Tork binary is executable in the current directory and that the PostgreSQL database is running and accessible via the 'config.toml' settings. Successful completion logs 'migration completed!'.

```bash
./tork migration

```

--------------------------------

### Extract Frame from Video using FFmpeg

Source: https://www.tork.run/index

This code snippet demonstrates how to extract a single frame from a video file using FFmpeg and upload it to a destination URL. It requires FFmpeg to be installed and accessible. The input is a video file path, and the output is a JPG image file uploaded to a specified destination.

```bash
name: extract a frame from a video
env:
  SRC: https://demo.com/input.mov
  DST: https://demo.com/upload/frame.jpg
image: jrottenberg/ffmpeg:3.4-alpine
run: |
  ffmpeg -i $SRC -vframes 1 \
    -an -s 400x222 -ss 30 frame.jpg
  wget --post-file=frame.jpg $DST
```

--------------------------------

### Mount a Docker Volume for Task Data

Source: https://www.tork.run/tasks

Shows how to use a `volume` mount to create a Docker volume accessible within the task's container at the specified `target` path (`/tmp`). This is useful for temporary storage or sharing data between tasks, as volumes are removed upon task termination. An example demonstrates using ffmpeg with a downloaded video.

```yaml
- name: convert the first 5 seconds of a video
  image: jrottenberg/ffmpeg:3.4-alpine
  run: ffmpeg -i /tmp/my_video.mov -t 5 /tmp/output.mp4
  mounts:
    - type: volume
      target: /tmp
  pre:
    - name: download the remote file
      image: alpine:3.18.3
      run: wget http://example.com/my_video.mov

```

--------------------------------

### Configure Tork to Use RabbitMQ Broker

Source: https://www.tork.run/installation

This TOML snippet configures Tork to use RabbitMQ as its message broker. It specifies the broker type as 'rabbitmq' and provides the connection URL for the RabbitMQ instance running on localhost at the default port 5672.

```toml
# config.toml
[broker]
type = "rabbitmq"

[broker.rabbitmq]
url = "amqp://guest:guest@localhost:5672/"
```

--------------------------------

### Resize Image to Multiple Resolutions and Upload

Source: https://www.tork.run/resizing-images

This task uses an 'each' loop to resize a source image to multiple specified resolutions and uploads each resized image to Minio. It iterates through a list of resolutions, performing the resizing and uploading steps for each. Dependencies include 'dpokidov/imagemagick' and 'amazon/aws-cli'.

```yaml
- name: Convert the image to various resolutions
  each:
    list: "{{ ['1920x1080','1366x768','1280x720','768x1024','100x100','200x200'] }}"
    task:
      name: 'Scale the image to {{ item.value }}'
      volumes:
        - /workdir
      networks:
        - minio
      image: dpokidov/imagemagick
      env:
        EXT: '{{ tasks.fileExt }}'
        SIZE: '{{ item.value }}'
      run: |
        mkdir /workdir/targets
        convert "/workdir/source.$EXT" -resize $SIZE "/workdir/targets/$SIZE.jpg"
      pre:
        - name: download the remote file
          image: alpine:3.18.3
          env:
            SOURCE: '{{ inputs.source }}'
            EXT: '{{ tasks.fileExt }}'
          run: |
            wget $SOURCE -O "/workdir/source.$EXT"
      post:
        - name: upload the converted image to minio
          run: aws --endpoint-url $ENDPOINT_URL s3 cp /workdir/targets/$SIZE.jpg $TARGET/$SIZE.jpg
          image: amazon/aws-cli:2.13.10
          env:
            AWS_ACCESS_KEY_ID: '{{inputs.accessKeyID}}'
            AWS_SECRET_ACCESS_KEY: '{{inputs.secretKeyID}}'
            TARGET: '{{inputs.target}}'
            ENDPOINT_URL: '{{inputs.endpointURL}}'
            SIZE: '{{ item.value }}'

```

--------------------------------

### Register a Custom HTTP Endpoint in Tork

Source: https://www.tork.run/extend

Shows how to extend Tork by registering a custom HTTP endpoint using the `RegisterEndpoint` hook. This allows you to add new API routes to Tork's coordinator. The example creates a GET endpoint at '/myendpoint' that returns 'Hello'.

```go
package main

import (
	"fmt"
	"net/http"
	"os"

	"github.com/runabol/tork/cli"
	"github.com/runabol/tork/conf"
	"github.com/runabol/tork/engine"
	"github.com/runabol/tork/middleware"
)

func main() {
	if err := conf.LoadConfig(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}

	handler := func(c middleware.Context) error {
		return c.String(http.StatusOK, "Hello")
	}

	engine.RegisterEndpoint(http.MethodGet, "/myendpoint", handler)

	if err := cli.New().Run(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}
```

--------------------------------

### Tork Worker Default Queue Configuration

Source: https://www.tork.run/installation

Configures a Tork worker to consume tasks from the default queue with a specified concurrency. This sets the maximum number of tasks the worker can process simultaneously from the 'default' queue. It requires a `config.toml` file and assumes RabbitMQ is the broker.

```toml
# config.toml
[worker.queues]
default = 5

[broker]
type = "rabbitmq"
```

--------------------------------

### Resize Image and Upload to Minio

Source: https://www.tork.run/resizing-images

This task resizes a source image to a specific resolution (100x100) using ImageMagick and then uploads the converted image to Minio. It involves a 'pre' task to download the source image and a 'post' task to upload the resized image. Dependencies include 'dpokidov/imagemagick' for resizing and 'amazon/aws-cli' for uploading.

```yaml
- name: 'Resize the source image to 100x100'
  image: dpokidov/imagemagick:7.1.1-15-ubuntu
  env:
    EXT: '{{ tasks.fileExt }}'
  mounts:
    - type: volume
      target: /workdir
  networks:
    - minio
  run: convert "/workdir/source.$EXT" -resize 100x100 /workdir/100x100.jpg
  pre:
    - name: download the remote file
      image: alpine:3.18.3
      env:
        SOURCE: '{{ inputs.source }}'
        EXT: '{{ tasks.fileExt }}'
      run: |
        wget $SOURCE -O "/workdir/source.$EXT"
  post:
    - name: upload the converted image to minio
      run: aws --endpoint-url $ENDPOINT_URL s3 cp /workdir/100x100.jpg $TARGET/100x100.jpg
      image: amazon/aws-cli:2.13.10
      env:
        AWS_ACCESS_KEY_ID: '{{inputs.accessKeyID}}'
        AWS_SECRET_ACCESS_KEY: '{{inputs.secretKeyID}}'
        TARGET: '{{inputs.target}}'
        ENDPOINT_URL: '{{inputs.endpointURL}}'

```

--------------------------------

### Run Tork Database Migrations

Source: https://www.tork.run/quick-start

Executes Tork database migrations using a PostgreSQL datastore. Ensure the TORK_DATASTORE_TYPE environment variable is set to 'postgres' and the Tork binary is accessible in the current path.

```bash
TORK_DATASTORE_TYPE=postgres ./tork migration

```

--------------------------------

### Tork Job Definition (hello.yaml)

Source: https://www.tork.run/quick-start

A Tork job definition file in YAML format. It specifies a job named 'hello job' with two sequential tasks: one that echoes 'hello world' using an Ubuntu image, and another that echoes 'bye world' using an Alpine image.

```yaml
# hello.yaml
---
name: hello job
tasks:
  - name: say hello
    image: ubuntu:mantic #docker image
    run: |
      echo -n hello world
  - name: say goodbye
    image: alpine:latest
    run: |
      echo -n bye world

```

--------------------------------

### Extract Tork Binary

Source: https://www.tork.run/video-transcoding

This command extracts the contents of the downloaded Tork binary archive (tork.tgz) into the current directory. It uses the tar utility with the x (extract), v (verbose), and z (gzip) flags. The extracted files will be placed in the specified directory, e.g., '~/tork/'.

```bash
tar xvzf tork.tgz

```

--------------------------------

### GET /jobs

Source: https://www.tork.run/rest-api

Retrieves a list of the most recent jobs. Supports pagination and full-text search.

```APIDOC
## List Jobs

### Description
Returns a list of the most recent jobs. Supports pagination and full-text search.

### Method
GET

### Endpoint
/jobs

### Query Parameters
- **page** (integer) - Optional - The page number to retrieve (default: 1).
- **size** (integer) - Optional - The number of items per page (default: 10, min: 1, max: 20).
- **q** (string) - Optional - A full-text search query to filter jobs.

### Response
#### Success Response (200)
- **items** (array) - A list of job objects.
  - **id** (string) - The unique identifier of the job.
  - **name** (string) - The name of the job.
  - **state** (string) - The current state of the job (e.g., "COMPLETED").
  - **createdAt** (string) - The timestamp when the job was created.
  - **startedAt** (string) - The timestamp when the job started execution.
  - **completedAt** (string) - The timestamp when the job completed execution.
- **number** (integer) - The current page number.
- **size** (integer) - The number of items per page.
- **totalPages** (integer) - The total number of pages available.

#### Response Example
```json
{
  "items": [
    {
      "id": "c5873550dad7439e85ac781168e6e124",
      "name": "sample job",
      "state": "COMPLETED",
      "createdAt": "2023-08-21T21:52:07.751041Z",
      "startedAt": "2023-08-22T01:52:07.765393Z",
      "completedAt": "2023-08-22T01:52:12.900569Z"
    }
  ],
  "number": 1,
  "size": 4,
  "totalPages": 1
}
```
```

--------------------------------

### Create Minio Docker Network (Shell)

Source: https://www.tork.run/resizing-images

This command creates a Docker network named 'minio' for container communication. It's a prerequisite for running the Minio Docker container.

```shell
docker network create minio
```

--------------------------------

### GET /queues

Source: https://www.tork.run/rest-api

Retrieves a list of all queues used by the broker, including their size and subscriber count.

```APIDOC
## List Queues

### Description
Returns a list of all queues used by the broker.

### Method
GET

### Endpoint
/queues

### Response
#### Success Response (200)
- **queues** (array) - A list of queue objects.
  - **name** (string) - The name of the queue.
  - **size** (integer) - The number of messages currently in the queue.
  - **subscribers** (integer) - The number of active subscribers to the queue.
  - **unacked** (integer) - The number of unacknowledged messages in the queue.

#### Response Example
```json
[
  {
    "name": "default",
    "size": 12,
    "subscribers": 3,
    "unacked": 3
  },
  {
    "name": "jobs",
    "size": 0,
    "subscribers": 1,
    "unacked": 0
  },
  {
    "name": "logs",
    "size": 0,
    "subscribers": 1,
    "unacked": 0
  }
]
```
```

--------------------------------

### GET /nodes

Source: https://www.tork.run/rest-api

Retrieves a list of all active nodes in the cluster, including their status, resource usage, and version.

```APIDOC
## List Nodes

### Description
Returns a list of all active nodes in the cluster.

### Method
GET

### Endpoint
/nodes

### Response
#### Success Response (200)
- **nodes** (array) - A list of node objects.
  - **id** (string) - The unique identifier of the node.
  - **name** (string) - The name of the node.
  - **startedAt** (string) - The timestamp when the node started.
  - **cpuPercent** (number) - The current CPU utilization percentage.
  - **lastHeartbeatAt** (string) - The timestamp of the last heartbeat from the node.
  - **status** (string) - The current status of the node (e.g., "UP").
  - **hostname** (string) - The hostname of the node.
  - **version** (string) - The version of the Tork Run agent running on the node.
  - **queue** (string) - Optional - The queue the node is associated with.

#### Response Example
```json
[
  {
    "id": "V2CjY4HYEaUCtTZziuJh6M",
    "name": "Coordinator",
    "startedAt": "2024-06-15T19:51:51.111662-04:00",
    "cpuPercent": 17.977528051675314,
    "lastHeartbeatAt": "2024-06-15T23:51:51.226284Z",
    "status": "UP",
    "hostname": "some-host-1",
    "version": "0.1.87"
  },
  {
    "id": "2C2LtZRhktaieM768LQpbP",
    "name": "Worker",
    "startedAt": "2024-06-15T23:51:50.995903Z",
    "cpuPercent": 9.952276537355232,
    "lastHeartbeatAt": "2024-06-15T23:51:51.135511Z",
    "queue": "x-2C2LtZRhktaieM768LQpbP",
    "status": "UP",
    "hostname": "some-host-2",
    "version": "0.1.87"
  }
]
```
```

--------------------------------

### GET /health

Source: https://www.tork.run/rest-api

Checks the health status of the Tork Run service. Returns 'UP' if the service is healthy.

```APIDOC
## Health Check

### Description
Checks the health status of the Tork Run service.

### Method
GET

### Endpoint
/health

### Response
#### Success Response (200)
- **status** (string) - The health status of the service (e.g., "UP").

#### Response Example
```json
{
  "status": "UP"
}
```
```

--------------------------------

### Resize Image using ImageMagick

Source: https://www.tork.run/resizing-images

A Tork task that uses the dpokidov/imagemagick Docker image to resize a source image to 100x100 pixels. It demonstrates using pre tasks for downloading remote files.

```yaml
- name: 'Resize the source image to 100x100'
  image: dpokidov/imagemagick:7.1.1-15-ubuntu
  env:
    EXT: '{{ tasks.fileExt }}'
  mounts:
    - type: volume
      target: /workdir
  run: convert "/workdir/source.$EXT" -resize 100x100 /workdir/100x100.jpg
  pre:
    - name: download the remote file
      image: alpine:3.18.3
      env:
        SOURCE: '{{ inputs.source }}'
        EXT: '{{ tasks.fileExt }}'
      run: |
        wget $SOURCE -O "/workdir/source.$EXT"
```
Tokens:
Raw