### Run Tork in Standalone Mode Source: https://www.tork.run/installation This command starts Tork in 'standalone' mode. This mode is suitable for single-machine setups and simplifies initial testing and development by running all components locally without needing external services like RabbitMQ. ```shell ./tork run standalone ``` -------------------------------- ### Run Tork Coordinator Source: https://www.tork.run/installation This command starts the Tork coordinator process. The coordinator is responsible for managing and distributing tasks to worker nodes in a distributed setup. It assumes Tork is installed and accessible in the current PATH or directory. ```shell ./tork run coordinator ``` -------------------------------- ### Create Tork Directory and Navigate Source: https://www.tork.run/installation This snippet creates a directory named 'tork' in the user's home directory and then changes the current directory to the newly created 'tork' directory. This is a standard shell operation for organizing project files. ```shell mkdir ~/tork cd ~/tork ``` -------------------------------- ### Start Minio Server (Docker) Source: https://www.tork.run/video-transcoding This command starts a Minio server as a Docker container. Minio provides a self-hosted S3-compatible object storage service, used here for intermediate and final video outputs. It maps ports 9000 and 9001 and binds the /data directory for persistent storage. The --name flag assigns a name to the container for easy management. ```bash docker run -d \ --name minio \ -p 9000:9000 \ -p 9001:9001 minio/minio \ server /data \ --console-address ":9001" ``` -------------------------------- ### Start Postgres Database (Docker) Source: https://www.tork.run/video-transcoding This command starts a PostgreSQL database container using the official Docker image. It's configured with user 'tork', password 'tork', and database name 'tork'. The PGDATA environment variable sets the data directory, and a volume is mounted to persist data. Port 5432 is exposed for connections. ```bash docker run -d \ --name tork-postgres \ -e POSTGRES_DB=tork \ -e POSTGRES_PASSWORD=tork \ -e POSTGRES_USER=tork \ -e PGDATA=/var/lib/postgresql/data/pgdata \ -v $(pwd)/data:/var/lib/postgresql/data \ -p 6432:5432 \ postgres:15 ``` -------------------------------- ### Start RabbitMQ Management Container with Docker Source: https://www.tork.run/installation This Docker command starts a RabbitMQ container with the management plugin enabled, running in detached mode. It names the container 'tork-rabbit', maps ports 5672 (AMQP) and 15672 (Management UI) from the container to the host, and uses the 'rabbitmq:3-management' image. ```shell docker run \ -d \ --name=tork-rabbit \ -p 5672:5672 \ -p 15672:15672 \ rabbitmq:3-management ``` -------------------------------- ### Start PostgreSQL Container with Docker Source: https://www.tork.run/installation This Docker command runs a PostgreSQL container in detached mode ('-d'). It names the container 'tork-postgres', maps port 5432 from the container to the host, sets the PostgreSQL user, password, and database name using environment variables, and specifies the data directory. It uses the 'postgres:15.3' image. ```shell docker run -d \ --name tork-postgres \ -p 5432:5432 \ -e POSTGRES_PASSWORD=tork \ -e POSTGRES_USER=tork \ -e PGDATA=/var/lib/postgresql/data/pgdata \ -e POSTGRES_DB=tork postgres:15.3 ``` -------------------------------- ### Start PostgreSQL Container Source: https://www.tork.run/quick-start Starts a PostgreSQL container named 'tork-postgres' accessible on the host's port 5432. It configures the user, password, and data directory for PostgreSQL. Suitable for local development and testing. ```docker docker run -d \ --name tork-postgres \ -p 5432:5432 \ -e POSTGRES_PASSWORD=tork \ -e POSTGRES_USER=tork \ -e PGDATA=/var/lib/postgresql/data/pgdata \ -e POSTGRES_DB=tork postgres:15.3 ``` -------------------------------- ### Run Tork Executable Source: https://www.tork.run/installation This command executes the Tork binary. The '%' symbol suggests it might be run in a specific shell environment like zsh, but it's a standard way to invoke an executable file in the current directory. ```shell % ./tork ``` -------------------------------- ### Run Tork Worker Source: https://www.tork.run/installation This command starts a Tork worker process. Worker nodes are responsible for executing tasks assigned by the coordinator. Multiple workers can be started to increase processing capacity. This command assumes Tork is installed and accessible. ```shell ./tork run worker ``` -------------------------------- ### Download Tork Binary (Linux) Source: https://www.tork.run/video-transcoding This snippet downloads the Tork binary for Linux AMD64 architecture using wget. It saves the downloaded archive as 'tork.tgz'. Ensure you have wget installed and sufficient permissions. ```bash wget https://github.com/runabol/tork/releases/download/v0.1.66/tork_0.1.66_linux_amd64.tgz -O tork.tgz ``` -------------------------------- ### Run Tork Database Migration Source: https://www.tork.run/installation This command runs the Tork database migration script. It sets the TORK_DATASTORE_TYPE environment variable to 'postgres' to indicate that Tork should connect to a PostgreSQL database for the migration process. This command creates the necessary database schema. ```shell TORK_DATASTORE_TYPE=postgres ./tork migration ``` -------------------------------- ### Start MinIO S3-Compatible Object Store Source: https://www.tork.run/quick-start Launches a MinIO container, an S3-compatible object storage service, for local use. It exposes ports 9000 and 9001 and sets the root user credentials to 'minioadmin'. ```docker docker run --name=tork-minio \ -d -p 9000:9000 -p 9001:9001 \ -e MINIO_ROOT_USER=minioadmin \ -e MINIO_ROOT_PASSWORD=minioadmin \ minio/minio server /data \ --console-address ":9001" ``` -------------------------------- ### Start RabbitMQ Container Source: https://www.tork.run/video-transcoding This command starts a RabbitMQ container in detached mode with management UI enabled. It maps the broker port (5672) and management port (15672) to the host machine. ```docker docker run -d \ --name tork-rabbitmq \ -p 5672:5672 \ -p 15672:15672 \ rabbitmq:3-management ``` -------------------------------- ### Define and Submit a Tork Job Source: https://www.tork.run/installation This snippet first defines a simple Tork job configuration in a file named 'hello.yaml'. The job consists of two tasks that echo messages. Subsequently, it uses 'curl' to send this YAML file as a POST request to the Tork API endpoint '/jobs'. The output job ID is captured using 'jq' and stored in the JOB_ID variable for later use. ```shell # hello.yaml --- name: hello job tasks: - name: say hello image: ubuntu:mantic #docker image run: | echo -n hello world - name: say goodbye image: ubuntu:mantic run: | echo -n bye world JOB_ID=$(curl -s -X POST --data-binary @hello.yaml \ -H "Content-type: text/yaml" http://localhost:8000/jobs | jq -r .id) ``` -------------------------------- ### Initialize and Run Tork as a Go Library Source: https://www.tork.run/extend Demonstrates how to set up a new Go project, initialize it with Go modules, add Tork as a dependency, and run Tork with minimal boilerplate code. This is the starting point for using Tork programmatically. ```bash mkdir tork-plus cd tork-plus go mod init github.com/example/tork-plus go get github.com/runabol/tork go mod tidy go run main.go ``` ```go package main import ( "fmt" "os" "github.com/runabol/tork/cli" "github.com/runabol/tork/conf" ) func main() { if err := conf.LoadConfig(); err != nil { fmt.Println(err) os.Exit(1) } if err := cli.New().Run(); err != nil { fmt.Println(err) os.Exit(1) } } ``` -------------------------------- ### Tork Task Routing to Specific Queue Source: https://www.tork.run/installation Defines a Tork task and explicitly routes it to a specific queue, in this case, the 'video' queue. This is useful for directing tasks to specialized worker pools. The task definition includes properties like 'name', 'queue', 'image', and 'run' commands. ```yaml name: transcode a video queue: video image: jrottenberg/ffmpeg:3.4-alpine run: | ffmpeg \ -i https://example.com/some/video.mov \ output.mp4 ``` -------------------------------- ### Run Tork Coordinator Source: https://www.tork.run/quick-start Starts the Tork coordinator in distributed mode, configured to use PostgreSQL for datastore and RabbitMQ as the message broker. This process oversees task distribution. ```bash TORK_DATASTORE_TYPE=postgres TORK_BROKER_TYPE=rabbitmq ./tork run coordinator ``` -------------------------------- ### Unpack Tork Binary Archive Source: https://www.tork.run/installation This command unpacks a gzipped tar archive containing the Tork binary. It assumes the archive is located in the 'Downloads' directory and is named 'tork_0.1.66_darwin_arm64.tgz'. The 'x' flag extracts, 'z' indicates gzip compression, 'v' enables verbose output, and 'f' specifies the filename. ```shell tar xzvf ~/Downloads/tork_0.1.66_darwin_arm64.tgz ``` -------------------------------- ### Start Tork Web UI (Docker) Source: https://www.tork.run/video-transcoding This command launches the Tork Web UI as a Docker container. It exposes port 8100 for accessing the UI and configures the backend URL to communicate with the Tork coordinator running on the host machine. The '--add-host' flag is used to resolve 'host.docker.internal' to the host's gateway. ```bash docker run \ --name=tork-web \ -d \ -p 8100:8100 \ --add-host=host.docker.internal:host-gateway \ -e BACKEND_URL=http://host.docker.internal:8000 \ runabol/tork-web ``` -------------------------------- ### Run Minio Docker Container (Shell) Source: https://www.tork.run/resizing-images This command starts a Minio server instance in a Docker container. It maps ports 9000 and 9001, connects the container to the 'minio' network, and sets the data directory. The console address is also configured. ```shell docker run -d \ --name minio \ --network minio \ -p 9000:9000 \ -p 9001:9001 minio/minio \ server /data \ --console-address ":9001" ``` -------------------------------- ### Spawn a Sub-Job Task Source: https://www.tork.run/tasks Illustrates how a task can start another job using the `subjob` property. The parent task will wait for the sub-job to complete or fail. This example defines a simple sub-job with two tasks that print messages. ```yaml - name: a task that starts a sub-job subjob: name: my sub job tasks: - name: hello sub task image: ubuntu:mantic run: echo start of sub-job - name: bye task image: ubuntu:mantic run: echo end of sub-job ``` -------------------------------- ### Query Tork Job Status Source: https://www.tork.run/installation This command queries the status of a Tork job using its ID. It sends a GET request to the Tork API endpoint '/jobs/{JOB_ID}', where {JOB_ID} is a placeholder for the actual job identifier. The '-s' flag suppresses progress meters and error messages, making the output cleaner. ```shell curl -s http://localhost:8000/jobs/$JOB_ID ``` -------------------------------- ### Install Tork Web with Docker Source: https://www.tork.run/web-ui This snippet shows the Docker command to run the Tork Web UI. It maps port 3000 from the container to the host and sets the backend URL for Tork. Ensure the BACKEND_URL points to your Tork backend service. ```docker docker run \ -it \ --rm \ --name=tork-web \ -p 3000:3000 \ -e BACKEND_URL=http://my.tork.host:8000 \ runabol/tork-web ``` -------------------------------- ### Override Tork configuration using environment variables Source: https://www.tork.run/config This example shows how to override specific Tork configuration settings using environment variables. The format is TORK_ followed by the configuration property name, with dots replaced by underscores. This example sets the logging level to 'warn'. ```bash TORK_LOGGING_LEVEL=warn ``` -------------------------------- ### Tork Worker Multi-Queue Configuration Source: https://www.tork.run/installation Sets up a Tork worker to consume tasks from multiple queues, each with its own concurrency limit. This allows for specialized worker pools. It requires a `config.toml` file and assumes RabbitMQ is the broker. Tasks can be routed using the 'queue' property in the task definition. ```toml # config.toml [worker.queues] default = 5 video = 2 [broker] type = "rabbitmq" ``` -------------------------------- ### Define a simple Tork Job with two tasks Source: https://www.tork.run/jobs This example demonstrates how to define a basic Tork Job with two sequential tasks. The first task 'say hello' writes 'hello world' to an output file, and the second task 'say goodbye' writes 'bye world'. It uses the 'ubuntu:mantic' image for both tasks and shows how to specify a variable to capture output. ```yaml name: hello job tasks: - name: say hello var: task1 image: ubuntu:mantic run: | echo -n hello world > $TORK_OUTPUT - name: say goodbye image: ubuntu:mantic run: | echo -n bye world ``` -------------------------------- ### Run Tork Worker Source: https://www.tork.run/quick-start Starts a Tork worker in distributed mode, configured to use RabbitMQ as the message broker. Workers pick up tasks from the broker and execute them. Multiple workers can be started to scale execution. ```bash TORK_BROKER_TYPE=rabbitmq ./tork run worker ``` -------------------------------- ### Start Tork in Standalone Mode Source: https://www.tork.run/ci Command to run Tork in standalone mode for local testing. This mode handles job scheduling and task execution in a single process. ```bash ./tork run standalone ``` -------------------------------- ### Mount a Host Path using Bind Mount Source: https://www.tork.run/tasks Illustrates using a `bind` mount to make a host path (`/host/path`) accessible within the container at the specified `target` path (`/mnt/videos`). This allows tasks to interact with data residing on the host machine. The example shows ffmpeg processing a video file located on the host. ```yaml - name: convert the first 5 seconds of a video image: jrottenberg/ffmpeg:3.4-alpine run: ffmpeg -i /mnt/videos/my_video.mov -t 5 /mnt/videos/output.mp4 mounts: - type: bind target: /mnt/videos source: /host/path ``` -------------------------------- ### Get Job Details Source: https://www.tork.run/rest Retrieves the details of a specific job. ```APIDOC ## GET /jobs/ ### Description Retrieves the details of a specific job. ### Method GET ### Endpoint /jobs/ ### Parameters #### Path Parameters - **JOB ID** (string) - Required - The unique identifier of the job. ### Response #### Success Response (200) - **id** (string) - The unique identifier for the job. - **name** (string) - The name of the job. - **state** (string) - The current state of the job. - **createdAt** (string) - The timestamp when the job was created. - **startedAt** (string) - The timestamp when the job started. - **completedAt** (string) - The timestamp when the job completed. - **tasks** (array) - An array of task definitions within the job. - **name** (string) - The name of the task. - **run** (string) - The command to run for the task. - **image** (string) - The Docker image to use for the task. - **execution** (array) - An array of execution details for each task. - **id** (string) - The unique identifier for the execution. - **jobId** (string) - The ID of the job this execution belongs to. - **position** (integer) - The position of the task in the job. - **name** (string) - The name of the task. - **state** (string) - The state of the execution. - **createdAt** (string) - The timestamp when the execution was created. - **scheduledAt** (string) - The timestamp when the execution was scheduled. - **startedAt** (string) - The timestamp when the execution started. - **completedAt** (string) - The timestamp when the execution completed. - **run** (string) - The command executed. - **image** (string) - The Docker image used. - **queue** (string) - The queue the task was processed on. - **nodeId** (string) - The ID of the node where the task was executed. - **position** (integer) - The position of the job in the sequence. - **taskCount** (integer) - The total number of tasks in the job. ### Response Example ```json { "id": "9d7d7184b8f244249c5f5d8b4074f72e", "name": "my job", "state": "COMPLETED", "createdAt": "2023-09-05T01:33:08.413605Z", "startedAt": "2023-09-05T01:33:08.414339Z", "completedAt": "2023-09-05T01:33:14.167127Z", "tasks": [ { "name": "my first task", "run": "echo hello world", "image": "alpine:3.18.3" } ], "execution": [ { "id": "aa76535bc23649ca9403e636e377aca3", "jobId": "9d7d7184b8f244249c5f5d8b4074f72e", "position": 1, "name": "my first task", "state": "COMPLETED", "createdAt": "2023-09-05T01:33:08.414192Z", "scheduledAt": "2023-09-05T01:33:08.414378Z", "startedAt": "2023-09-05T01:33:08.414574Z", "completedAt": "2023-09-05T01:33:08.699749Z", "run": "echo hello world", "image": "alpine:3.18.3", "queue": "default", "nodeId": "93a1cd11800741e6863d1adef3bc3d18" } ], "position": 4, "taskCount": 3 } ``` ``` -------------------------------- ### Run Tork Worker Process Source: https://www.tork.run/video-transcoding This command starts the Tork worker in the background. It redirects standard output and standard error to a log file ('tork.log') and saves the process ID to 'tork.pid'. ```bash nohup ./tork run worker > tork.log 2>&1 & echo $! > tork.pid ``` -------------------------------- ### Tork Job: Stateful Example with MinIO using AWS CLI Source: https://www.tork.run/quick-start A Tork job definition in YAML format that creates a stateful workflow. It uses the `amazon/aws-cli:latest` Docker image for two tasks: writing a file to a MinIO bucket and then reading it back. The job requires MinIO endpoint details and credentials as inputs and secrets. ```yaml name: stateful example inputs: minio_endpoint: http://host.docker.internal:9000 secrets: minio_user: minioadmin minio_password: minioadmin tasks: - name: write data to object store image: amazon/aws-cli:latest env: AWS_ACCESS_KEY_ID: "{{ secrets.minio_user }}" AWS_SECRET_ACCESS_KEY: "{{ secrets.minio_password }}" AWS_ENDPOINT_URL: "{{ inputs.minio_endpoint }}" AWS_DEFAULT_REGION: us-east-1 run: | echo "Hello from Tork!" > /tmp/data.txt aws s3 mb s3://mybucket aws s3 cp /tmp/data.txt s3://mybucket/data.txt - name: read data from object store image: amazon/aws-cli:latest env: AWS_ACCESS_KEY_ID: "{{ secrets.minio_user }}" AWS_SECRET_ACCESS_KEY: "{{ secrets.minio_password }}" AWS_ENDPOINT_URL: "{{ inputs.minio_endpoint }}" AWS_DEFAULT_REGION: us-east-1 run: | aws s3 cp s3://mybucket/data.txt /tmp/retrieved.txt echo "Contents of retrieved file:" cat /tmp/retrieved.txt ``` -------------------------------- ### Run Tork with a specific config file using environment variable Source: https://www.tork.run/config This example demonstrates how to run Tork in standalone mode, specifying a custom configuration file path using the TORK_CONFIG environment variable. Ensure the Tork executable and the specified config file exist in the execution path. ```bash TORK_CONFIG=myconfig.toml ./tork run standalone ``` -------------------------------- ### Start RabbitMQ Container Source: https://www.tork.run/quick-start Launches a RabbitMQ container in detached mode, exposing management ports 5672 and 15672. This is used as a message broker for Tork's distributed mode. ```docker docker run \ -d -p 5672:5672 -p 15672:15672 \ --name=tork-rabbitmq \ rabbitmq:3-management ``` -------------------------------- ### Print String Length using Built-in Function Source: https://www.tork.run/tasks Demonstrates using the built-in 'len' function to get the length of a string and store it in an environment variable. The length is then printed to the console. This requires the 'hello world' string to be provided as input to the 'len' function. ```yaml - name: print the length of a string image: ubuntu:mantic env: LENGTH: '{{ len("hello world") }}' run: | echo "The length of the string is: $LENGTH" ``` -------------------------------- ### Implement HTTP Middleware for Timing Requests in Tork Source: https://www.tork.run/extend Illustrates how to create and register HTTP middleware in Tork to intercept requests to the Coordinator API. This example defines a middleware function that measures and logs the processing time of incoming HTTP requests. ```go package main import ( "time" "github.com/runabol/tork/middleware/web" "log" "github.com/runabol/tork/cli" "github.com/runabol/tork/conf" "github.com/runabol/tork/engine" ) func main() { if err := conf.LoadConfig(); err != nil { fmt.Println(err) os.Exit(1) } mw := func(next web.HandlerFunc) web.HandlerFunc { return func(c web.Context) error { before := time.Now() // happens before the request is processed err := next(c) // happens after the request is processed log.Info().Msgf("The request to %s took %s to process", c.Request().URL.Path, time.Since(before), ) return err } } engine.RegisterWebMiddleware(mw) if err := cli.New().Run(); err != nil { fmt.Println(err) os.Exit(1) } } ``` -------------------------------- ### Docker config file for private registry authentication Source: https://www.tork.run/tasks Provides an example of a Docker configuration file used for authenticating with private registries. This JSON structure contains authentication details that can be passed to the Tork worker via an environment variable. ```json { "auths": { "myregistry.com": { "auth": "base64encodedusername:base64encodedpassword" } } } ``` -------------------------------- ### Query Tork Job Status via HTTP GET Source: https://www.tork.run/ci Command to query the status of a submitted Tork job using its ID. It retrieves the job information and extracts the state using `jq`. ```bash curl -s http://localhost:8000/jobs/$JOB_ID | jq -r .state COMPLETED ``` -------------------------------- ### Tork API Response for Scheduled Job Creation (JSON) Source: https://www.tork.run/jobs This is an example of the JSON response received from the Tork API after successfully scheduling a job. It includes the job's unique ID, its current state, the name provided, the creation timestamp, and the configured cron schedule. ```json { "id": "c90188cce61244a1aabcdbedf31f51d6", "state": "ACTIVE", "name": "scheduled job test", "createdAt": "2024-12-21T14:53:05.709793Z", "cron": "0/5 * * * *" } ``` -------------------------------- ### Define a Tork Job with video conversion input Source: https://www.tork.run/jobs This example defines a Tork Job named 'mov to mp4' that utilizes job inputs. It specifies a 'source' input for the video file URL and uses this input within the task's environment variables to pass the video URL to the ffmpeg command for conversion. ```yaml name: mov to mp4 inputs: source: https://example.com/path/to/video.mov tasks: - name: convert the video to mp4 image: jrottenberg/ffmpeg:3.4-alpine env: SOURCE_URL: '{{ inputs.source }}' run: | ffmpeg -i $SOURCE_URL /tmp/output.mp4 ``` -------------------------------- ### Run Tork in Standalone Mode Source: https://www.tork.run/video-transcoding This command starts the Tork process in 'standalone' mode, which combines the Coordinator and Worker functionalities. The output and errors are redirected to 'tork.log', and the process ID is saved to 'tork.pid'. The `nohup` command ensures the process continues running after the terminal session is closed. ```bash nohup ./tork run standalone > tork.log 2>&1 & echo $! ``` -------------------------------- ### Define a Tork Job with default task configurations Source: https://www.tork.run/jobs This example shows how to set default configurations for all tasks within a Tork Job using the 'defaults' block. It specifies default retry limits, resource limits (CPU and memory), timeouts, default queues, and priorities, which can be overridden at the individual task level. ```yaml name: my job defaults: retry: # a task will retry up to 2 times in case of a failure limit: 2 # resource limits imposed on a task limits: # 1 CPU limit cpus: 1 # 500MB of RAM limit memory: 500m # a task will automatically fail if not completed within 10 minutes timeout: 10m # tasks will be routed to the highcpu queue by default queue: highcpu # values between 0-9. Higher numbers mean higher priority priority: 3 tasks: - name: my task queue: default # override the job defaults image: alpine:latest run: | echo hello world ``` -------------------------------- ### Iterate Tasks with Each Task (Sequence) Source: https://www.tork.run/tasks Demonstrates the `each` task for executing a task multiple times based on a list. This example uses `sequence(1,5)` to generate a list of numbers, and for each number, it sets an environment variable `ITEM` and `INDEX` and echoes a message. The `concurrency` property limits the number of parallel executions. ```yaml - name: sample each task each: list: '{{ sequence(1,5) }}' concurrency: 3 # max 3 tasks would execute concurrently task: image: ubuntu:mantic env: ITEM: '{{ item.value }}' INDEX: '{{ item.index }}' run: echo -n HELLO $ITEM at $INDEX ``` -------------------------------- ### Tork Configuration File (config.toml) Source: https://www.tork.run/video-transcoding This TOML configuration file sets up the Tork coordinator and datastore. It specifies the coordinator address and configures the datastore to use PostgreSQL with the provided connection details (host, user, password, database, port). Ensure this file is placed next to the tork binary. ```toml [coordinator] address = "0.0.0.0:8000" [datastore] type = "postgres" [datastore.postgres] dsn = "host=localhost user=tork password=tork dbname=tork port=6432 sslmode=disable" ``` -------------------------------- ### Initialize Tork Database Migrations Source: https://www.tork.run/video-transcoding This command initializes the database schema for Tork by running its migration scripts. It assumes the Tork binary is executable in the current directory and that the PostgreSQL database is running and accessible via the 'config.toml' settings. Successful completion logs 'migration completed!'. ```bash ./tork migration ``` -------------------------------- ### Extract Frame from Video using FFmpeg Source: https://www.tork.run/index This code snippet demonstrates how to extract a single frame from a video file using FFmpeg and upload it to a destination URL. It requires FFmpeg to be installed and accessible. The input is a video file path, and the output is a JPG image file uploaded to a specified destination. ```bash name: extract a frame from a video env: SRC: https://demo.com/input.mov DST: https://demo.com/upload/frame.jpg image: jrottenberg/ffmpeg:3.4-alpine run: | ffmpeg -i $SRC -vframes 1 \ -an -s 400x222 -ss 30 frame.jpg wget --post-file=frame.jpg $DST ``` -------------------------------- ### Mount a Docker Volume for Task Data Source: https://www.tork.run/tasks Shows how to use a `volume` mount to create a Docker volume accessible within the task's container at the specified `target` path (`/tmp`). This is useful for temporary storage or sharing data between tasks, as volumes are removed upon task termination. An example demonstrates using ffmpeg with a downloaded video. ```yaml - name: convert the first 5 seconds of a video image: jrottenberg/ffmpeg:3.4-alpine run: ffmpeg -i /tmp/my_video.mov -t 5 /tmp/output.mp4 mounts: - type: volume target: /tmp pre: - name: download the remote file image: alpine:3.18.3 run: wget http://example.com/my_video.mov ``` -------------------------------- ### Configure Tork to Use RabbitMQ Broker Source: https://www.tork.run/installation This TOML snippet configures Tork to use RabbitMQ as its message broker. It specifies the broker type as 'rabbitmq' and provides the connection URL for the RabbitMQ instance running on localhost at the default port 5672. ```toml # config.toml [broker] type = "rabbitmq" [broker.rabbitmq] url = "amqp://guest:guest@localhost:5672/" ``` -------------------------------- ### Resize Image to Multiple Resolutions and Upload Source: https://www.tork.run/resizing-images This task uses an 'each' loop to resize a source image to multiple specified resolutions and uploads each resized image to Minio. It iterates through a list of resolutions, performing the resizing and uploading steps for each. Dependencies include 'dpokidov/imagemagick' and 'amazon/aws-cli'. ```yaml - name: Convert the image to various resolutions each: list: "{{ ['1920x1080','1366x768','1280x720','768x1024','100x100','200x200'] }}" task: name: 'Scale the image to {{ item.value }}' volumes: - /workdir networks: - minio image: dpokidov/imagemagick env: EXT: '{{ tasks.fileExt }}' SIZE: '{{ item.value }}' run: | mkdir /workdir/targets convert "/workdir/source.$EXT" -resize $SIZE "/workdir/targets/$SIZE.jpg" pre: - name: download the remote file image: alpine:3.18.3 env: SOURCE: '{{ inputs.source }}' EXT: '{{ tasks.fileExt }}' run: | wget $SOURCE -O "/workdir/source.$EXT" post: - name: upload the converted image to minio run: aws --endpoint-url $ENDPOINT_URL s3 cp /workdir/targets/$SIZE.jpg $TARGET/$SIZE.jpg image: amazon/aws-cli:2.13.10 env: AWS_ACCESS_KEY_ID: '{{inputs.accessKeyID}}' AWS_SECRET_ACCESS_KEY: '{{inputs.secretKeyID}}' TARGET: '{{inputs.target}}' ENDPOINT_URL: '{{inputs.endpointURL}}' SIZE: '{{ item.value }}' ``` -------------------------------- ### Register a Custom HTTP Endpoint in Tork Source: https://www.tork.run/extend Shows how to extend Tork by registering a custom HTTP endpoint using the `RegisterEndpoint` hook. This allows you to add new API routes to Tork's coordinator. The example creates a GET endpoint at '/myendpoint' that returns 'Hello'. ```go package main import ( "fmt" "net/http" "os" "github.com/runabol/tork/cli" "github.com/runabol/tork/conf" "github.com/runabol/tork/engine" "github.com/runabol/tork/middleware" ) func main() { if err := conf.LoadConfig(); err != nil { fmt.Println(err) os.Exit(1) } handler := func(c middleware.Context) error { return c.String(http.StatusOK, "Hello") } engine.RegisterEndpoint(http.MethodGet, "/myendpoint", handler) if err := cli.New().Run(); err != nil { fmt.Println(err) os.Exit(1) } } ``` -------------------------------- ### Tork Worker Default Queue Configuration Source: https://www.tork.run/installation Configures a Tork worker to consume tasks from the default queue with a specified concurrency. This sets the maximum number of tasks the worker can process simultaneously from the 'default' queue. It requires a `config.toml` file and assumes RabbitMQ is the broker. ```toml # config.toml [worker.queues] default = 5 [broker] type = "rabbitmq" ``` -------------------------------- ### Resize Image and Upload to Minio Source: https://www.tork.run/resizing-images This task resizes a source image to a specific resolution (100x100) using ImageMagick and then uploads the converted image to Minio. It involves a 'pre' task to download the source image and a 'post' task to upload the resized image. Dependencies include 'dpokidov/imagemagick' for resizing and 'amazon/aws-cli' for uploading. ```yaml - name: 'Resize the source image to 100x100' image: dpokidov/imagemagick:7.1.1-15-ubuntu env: EXT: '{{ tasks.fileExt }}' mounts: - type: volume target: /workdir networks: - minio run: convert "/workdir/source.$EXT" -resize 100x100 /workdir/100x100.jpg pre: - name: download the remote file image: alpine:3.18.3 env: SOURCE: '{{ inputs.source }}' EXT: '{{ tasks.fileExt }}' run: | wget $SOURCE -O "/workdir/source.$EXT" post: - name: upload the converted image to minio run: aws --endpoint-url $ENDPOINT_URL s3 cp /workdir/100x100.jpg $TARGET/100x100.jpg image: amazon/aws-cli:2.13.10 env: AWS_ACCESS_KEY_ID: '{{inputs.accessKeyID}}' AWS_SECRET_ACCESS_KEY: '{{inputs.secretKeyID}}' TARGET: '{{inputs.target}}' ENDPOINT_URL: '{{inputs.endpointURL}}' ``` -------------------------------- ### Run Tork Database Migrations Source: https://www.tork.run/quick-start Executes Tork database migrations using a PostgreSQL datastore. Ensure the TORK_DATASTORE_TYPE environment variable is set to 'postgres' and the Tork binary is accessible in the current path. ```bash TORK_DATASTORE_TYPE=postgres ./tork migration ``` -------------------------------- ### Tork Job Definition (hello.yaml) Source: https://www.tork.run/quick-start A Tork job definition file in YAML format. It specifies a job named 'hello job' with two sequential tasks: one that echoes 'hello world' using an Ubuntu image, and another that echoes 'bye world' using an Alpine image. ```yaml # hello.yaml --- name: hello job tasks: - name: say hello image: ubuntu:mantic #docker image run: | echo -n hello world - name: say goodbye image: alpine:latest run: | echo -n bye world ``` -------------------------------- ### Extract Tork Binary Source: https://www.tork.run/video-transcoding This command extracts the contents of the downloaded Tork binary archive (tork.tgz) into the current directory. It uses the tar utility with the x (extract), v (verbose), and z (gzip) flags. The extracted files will be placed in the specified directory, e.g., '~/tork/'. ```bash tar xvzf tork.tgz ``` -------------------------------- ### GET /jobs Source: https://www.tork.run/rest-api Retrieves a list of the most recent jobs. Supports pagination and full-text search. ```APIDOC ## List Jobs ### Description Returns a list of the most recent jobs. Supports pagination and full-text search. ### Method GET ### Endpoint /jobs ### Query Parameters - **page** (integer) - Optional - The page number to retrieve (default: 1). - **size** (integer) - Optional - The number of items per page (default: 10, min: 1, max: 20). - **q** (string) - Optional - A full-text search query to filter jobs. ### Response #### Success Response (200) - **items** (array) - A list of job objects. - **id** (string) - The unique identifier of the job. - **name** (string) - The name of the job. - **state** (string) - The current state of the job (e.g., "COMPLETED"). - **createdAt** (string) - The timestamp when the job was created. - **startedAt** (string) - The timestamp when the job started execution. - **completedAt** (string) - The timestamp when the job completed execution. - **number** (integer) - The current page number. - **size** (integer) - The number of items per page. - **totalPages** (integer) - The total number of pages available. #### Response Example ```json { "items": [ { "id": "c5873550dad7439e85ac781168e6e124", "name": "sample job", "state": "COMPLETED", "createdAt": "2023-08-21T21:52:07.751041Z", "startedAt": "2023-08-22T01:52:07.765393Z", "completedAt": "2023-08-22T01:52:12.900569Z" } ], "number": 1, "size": 4, "totalPages": 1 } ``` ``` -------------------------------- ### Create Minio Docker Network (Shell) Source: https://www.tork.run/resizing-images This command creates a Docker network named 'minio' for container communication. It's a prerequisite for running the Minio Docker container. ```shell docker network create minio ``` -------------------------------- ### GET /queues Source: https://www.tork.run/rest-api Retrieves a list of all queues used by the broker, including their size and subscriber count. ```APIDOC ## List Queues ### Description Returns a list of all queues used by the broker. ### Method GET ### Endpoint /queues ### Response #### Success Response (200) - **queues** (array) - A list of queue objects. - **name** (string) - The name of the queue. - **size** (integer) - The number of messages currently in the queue. - **subscribers** (integer) - The number of active subscribers to the queue. - **unacked** (integer) - The number of unacknowledged messages in the queue. #### Response Example ```json [ { "name": "default", "size": 12, "subscribers": 3, "unacked": 3 }, { "name": "jobs", "size": 0, "subscribers": 1, "unacked": 0 }, { "name": "logs", "size": 0, "subscribers": 1, "unacked": 0 } ] ``` ``` -------------------------------- ### GET /nodes Source: https://www.tork.run/rest-api Retrieves a list of all active nodes in the cluster, including their status, resource usage, and version. ```APIDOC ## List Nodes ### Description Returns a list of all active nodes in the cluster. ### Method GET ### Endpoint /nodes ### Response #### Success Response (200) - **nodes** (array) - A list of node objects. - **id** (string) - The unique identifier of the node. - **name** (string) - The name of the node. - **startedAt** (string) - The timestamp when the node started. - **cpuPercent** (number) - The current CPU utilization percentage. - **lastHeartbeatAt** (string) - The timestamp of the last heartbeat from the node. - **status** (string) - The current status of the node (e.g., "UP"). - **hostname** (string) - The hostname of the node. - **version** (string) - The version of the Tork Run agent running on the node. - **queue** (string) - Optional - The queue the node is associated with. #### Response Example ```json [ { "id": "V2CjY4HYEaUCtTZziuJh6M", "name": "Coordinator", "startedAt": "2024-06-15T19:51:51.111662-04:00", "cpuPercent": 17.977528051675314, "lastHeartbeatAt": "2024-06-15T23:51:51.226284Z", "status": "UP", "hostname": "some-host-1", "version": "0.1.87" }, { "id": "2C2LtZRhktaieM768LQpbP", "name": "Worker", "startedAt": "2024-06-15T23:51:50.995903Z", "cpuPercent": 9.952276537355232, "lastHeartbeatAt": "2024-06-15T23:51:51.135511Z", "queue": "x-2C2LtZRhktaieM768LQpbP", "status": "UP", "hostname": "some-host-2", "version": "0.1.87" } ] ``` ``` -------------------------------- ### GET /health Source: https://www.tork.run/rest-api Checks the health status of the Tork Run service. Returns 'UP' if the service is healthy. ```APIDOC ## Health Check ### Description Checks the health status of the Tork Run service. ### Method GET ### Endpoint /health ### Response #### Success Response (200) - **status** (string) - The health status of the service (e.g., "UP"). #### Response Example ```json { "status": "UP" } ``` ``` -------------------------------- ### Resize Image using ImageMagick Source: https://www.tork.run/resizing-images A Tork task that uses the dpokidov/imagemagick Docker image to resize a source image to 100x100 pixels. It demonstrates using pre tasks for downloading remote files. ```yaml - name: 'Resize the source image to 100x100' image: dpokidov/imagemagick:7.1.1-15-ubuntu env: EXT: '{{ tasks.fileExt }}' mounts: - type: volume target: /workdir run: convert "/workdir/source.$EXT" -resize 100x100 /workdir/100x100.jpg pre: - name: download the remote file image: alpine:3.18.3 env: SOURCE: '{{ inputs.source }}' EXT: '{{ tasks.fileExt }}' run: | wget $SOURCE -O "/workdir/source.$EXT" ```