### Install Hermes and Setup Virtual Environment Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/UpdateWDLParser.md Clone the Hermes repository, create and activate a Python 3.5 virtual environment, and install necessary dependencies. This ensures a consistent environment for parser generation. ```bash git clone git@github.com:scottfrazer/hermes.git cd hermes pyenv virtualenv 3.5.10 hermes-venv pyenv activate hermes-venv ``` ```bash pip install --index-url https://pypi.org/simple/ moody-templates xtermcolor nose pygments hermes_pygments python setup.py install hermes --help ``` -------------------------------- ### Install backpressure_report Source: https://github.com/broadinstitute/cromwell/blob/develop/scripts/backpressure_report/README.md Install the backpressure_report package. It is recommended to do this within a virtual environment. ```shell pip install . ``` -------------------------------- ### FTP Configuration Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/filesystems/FileTransferProtocol.md Example of mixing global and specific FTP configurations for engine and backend providers. Overrides default settings. ```hocon filesystems.ftp.global.config { # We only override what changes from the default max-connection-per-server-per-user = 20 } # Configure the default auth for the engine engine.filesystems.ftp { auth { username = "me" password = "my_password" } } # Configure the default auth for the local backend backend.providers.Local.config.filesystems.ftp { auth { username = "me" password = "my_password" } } ``` -------------------------------- ### Example qsub Command for SGE Job Submission Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/backends/SGE.md An example of the qsub command used by the SGE backend to submit a job, including various options. ```bash qsub \ -terse \ -V \ -b n \ -N ${job_name} \ -wd ${cwd} \ -o ${out}.qsub \ -e ${err}.qsub \ -pe smp ${cpu} \ ${'-l m_mem_free=' + memory_gb + 'gb'} \ ${'-q ' + sge_queue} \ ${'-P ' + sge_project} \ ${script} ``` -------------------------------- ### Start Cromwell Server Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/TimingDiagrams.md Start the Cromwell server in the background. Ensure the JAR file name matches your downloaded version. ```sh java -jar cromwell-29.jar server ``` -------------------------------- ### Combined HTTP and File Imports Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Imports.md A complete WDL workflow example demonstrating the use of both HTTP(S) and file-based imports. ```wdl import "https://raw.githubusercontent.com/broadinstitute/cromwell/master/engine/src/main/resources/3step.wdl" as http_import import "imports/imported.wdl" as provided_import workflow my_workflow { ... } ``` -------------------------------- ### Start MySQL Docker Container Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/PersistentServer.md Use this command to start a MySQL Docker container for Cromwell's data persistence. Ensure you replace placeholder values with your own. ```bash docker run -p 3306:3306 --name NameOfTheContainer -e MYSQL_ROOT_PASSWORD=YourPassword -e MYSQL_DATABASE=DatabaseName -e MYSQL_USER=ChooseAName -e MYSQL_PASSWORD=YourOtherPassword -d mysql:8.4 ``` -------------------------------- ### Set up PostgreSQL Database Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Configuring.md To use PostgreSQL, enable the Large Object extension. This example shows a typical configuration for the database connection. ```bash $ createdb cromwell $ psql -d cromwell -c "create extension lo;" ``` ```hocon database { profile = "slick.jdbc.PostgresProfile$" db { driver = "org.postgresql.Driver" url = "jdbc:postgresql://localhost:5432/cromwell" user = "user" password = "pass" port = 5432 connectionTimeout = 5000 } } ``` -------------------------------- ### Install SBT Build Tool Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/HPCSlurmWithLocalScratch.md Installs the SBT (Scala Build Tool) package on CentOS 8. ```hocon dnf -y install sbt ``` -------------------------------- ### WDL Runtime Docker Tag Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/Containers.md This is an example of how to specify a Docker image using a tag in a WDL runtime block. It is recommended to use image digests for better reproducibility. ```wdl runtime { docker: 'ubuntu:latest' } ``` -------------------------------- ### Starting Cromwell in `server` Mode Source: https://context7.com/broadinstitute/cromwell/llms.txt The `server` command starts Cromwell as a persistent REST API server, suitable for production use with parallel workflow submissions. ```APIDOC ## Starting Cromwell in `server` Mode The `server` command starts Cromwell as a persistent REST API server, suitable for production use with parallel workflow submissions. ```sh # Start with default settings (binds to http://localhost:8000) java -jar cromwell-93.jar server # Start with a custom configuration file java -Dconfig.file=/path/to/my.conf -jar cromwell-93.jar server # Start with inline config override (port 9000) java -Dwebservice.port=9000 -jar cromwell-93.jar server # Cromwell self-documents its API at http://localhost:8000 via Swagger UI ``` ``` -------------------------------- ### Logs Explorer Query Example Source: https://github.com/broadinstitute/cromwell/blob/develop/scripts/backpressure_report/README.md Example query for Google Logs Explorer to capture relevant backpressure messages. Multiple queries may be needed due to export limits. ```logql resource.labels.container_name="cromwell1-runner-app" (jsonPayload.message=~"IoActor backpressure off" OR jsonPayload.message=~"Beginning IoActor backpressure") ``` -------------------------------- ### Scattered Task Workflow Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/bitesize/workflowExecution/executionAndValueStoreExamples.md A WDL workflow featuring a scatter block, demonstrating how the Execution Store handles scatter expansion and task sharding. ```wdl version 1.0 workflow scattered_task_workflow { scatter (x in range(2)) { call scattered_task } output { Int results_count = length(scattered_task.string_out) } } task scattered_task { command { echo hello } output { String string_out = "hello" } } ``` -------------------------------- ### Adding Slurm Backend Provider Example Source: https://github.com/broadinstitute/cromwell/blob/develop/cromwell.example.backends/README.md Example of how to add the Slurm backend provider to the Cromwell configuration. Multiple providers can be added sequentially within the 'providers' block. ```hcl backend { # Override the default backend. #default = "LocalExample" # The list of providers. Copy paste the contents of a backend provider in this section providers { slurm { ... } } # Second backend provider would be copy pasted here! } } ``` -------------------------------- ### Run Cromwell with a Custom Configuration File Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/ConfigurationFiles.md Pass the path to your custom configuration file using the '-Dconfig.file' system property when starting Cromwell. ```bash java -Dconfig.file=/path/to/your.conf -jar cromwell-[VERSION].jar server ``` -------------------------------- ### Run Cromwell with Imports (Run Mode) Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Imports.md Command-line example for running a WDL workflow in Run mode, specifying the ZIP file containing imported resources. ```bash $ java -jar cromwell.jar run workflow.wdl --imports imports.zip ``` -------------------------------- ### Start Cromwell Server Source: https://context7.com/broadinstitute/cromwell/llms.txt Launch Cromwell as a persistent REST API server. Can be configured via a file or inline overrides. ```sh # Start with default settings (binds to http://localhost:8000) java -jar cromwell-93.jar server # Start with a custom configuration file java -Dconfig.file=/path/to/my.conf -jar cromwell-93.jar server # Start with inline config override (port 9000) java -Dwebservice.port=9000 -jar cromwell-93.jar server # Cromwell self-documents its API at http://localhost:8000 via Swagger UI ``` -------------------------------- ### Import WDLs from ZIP Root Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Imports.md Example of importing multiple WDL files that are located at the root of the provided ZIP bundle. ```wdl import "my_wdl_1.wdl" import "my_wdl_2.wdl" ``` -------------------------------- ### Single Task Workflow Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/bitesize/workflowExecution/executionAndValueStoreExamples.md A basic WDL workflow with a single task, demonstrating the initial state and updates of the Execution Store and Value Store. ```wdl version 1.0 workflow single_task_workflow { call single_task output { String string_out = single_task.string_out } } task single_task { command { echo hello } output { String string_out = "hello" } } ``` -------------------------------- ### Run Cromwell with GCP Batch Backend Source: https://context7.com/broadinstitute/cromwell/llms.txt After configuring the GCP Batch backend, authenticate your environment and start the Cromwell server. ```sh # Authenticate then run gcloud auth application-default login java -Dconfig.file=gcp.conf -jar cromwell-93.jar server ``` -------------------------------- ### Switch to Cromwell User Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/HPCSlurmWithLocalScratch.md Logs in as the 'cromwell' user to perform subsequent installation steps. ```hocon su - cromwell ``` -------------------------------- ### Run Cromwell Server with Persistent Data Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/PersistentServer.md Start the Cromwell server using your updated application.conf file to enable data persistence. ```bash java -Dconfig.file=/path/to/application.conf/ -jar cromwell-[version].jar ... ``` -------------------------------- ### Include Application Configuration in HOCON Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Configuring.md Start your custom HOCON configuration file by including the default application configuration. ```hocon # include the application.conf at the top include required(classpath("application")) ``` -------------------------------- ### Docker Runtime Attribute Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/RuntimeAttributes.md Specifies the Docker image to use for running the task. Mandatory for GCP and AWS Batch backends. ```wdl runtime { docker: "ubuntu:latest" } ``` -------------------------------- ### WDL Dependencies Listing Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/WOMtool.md This output shows a successful validation message followed by a list of all files (local, remote, or imported) that the WDL workflow depends on. ```hocon Success! List of Workflow dependencies are: /path/to/my/import/myImport.wdl /path/to/another/import/anotherImport.wdl https://path-to-http-import/httpImport.wdl ``` -------------------------------- ### TES Backend Configuration Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/backends/TES.md Configure the TES backend by providing the TES API endpoint and other settings like root directory and concurrent job limit. ```HOCON backend { default = "TES" providers { TES { actor-factory = "cromwell.backend.impl.tes.TesBackendLifecycleActorFactory" config { endpoint = "https:///v1/tasks" root = "cromwell-executions" dockerRoot = "/cromwell-executions" concurrent-job-limit = 1000 } } } } ``` -------------------------------- ### Centaur Cost Verification Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/Centaur.md Shows how to define an expected cost range for a workflow, applicable when `testFormat` is `WorkflowSuccessAndVerifyCost`. ```HOCON cost:[ 0.0000, 0.0001 ] ``` -------------------------------- ### Memory Runtime Attribute Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/RuntimeAttributes.md Allocates RAM for a task, specified with an amount and unit (e.g., '4G'). Backend interpretation may differ. ```wdl runtime { memory: "4G" } ``` -------------------------------- ### Create Cromwell Directory Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/HPCSlurmWithLocalScratch.md Creates a directory named 'cromwell' in the current location for installing the Cromwell workflow management system. ```hocon mkdir cromwell ``` -------------------------------- ### WDL Workflow Example with GCS Input Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/filesystems/Filesystems.md This WDL code demonstrates a workflow that uses a file input from Google Cloud Storage. The filesystem configuration for 'gs://' is determined by the backend. ```wdl version 1.0 task my_pipelines_task { input { File input_file } String content = read_string(input_file) command { echo ~{content} } runtime { docker: "ubuntu" } } workflow my_workflow { call my_pipelines_task { input: input_file = "gs://mybucket/my_file.txt" } } ``` -------------------------------- ### Basic Workflow Options Structure Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/wf_options/Overview.md This is a general example of the JSON structure for providing workflow options. Specific options are added as key-value pairs within this object. ```json { "option_name_1": "option value 1", "option_name_2": "option value 2" } ``` -------------------------------- ### Setting Specific Runtime Attributes and Backend Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/wf_options/Overview.md This example demonstrates how to globally override `continueOnReturnCode` for all tasks and set a default backend for the workflow. These settings can be overridden by task-specific runtime definitions. ```json { "continueOnReturnCode": false, "backend": "Local" } ``` -------------------------------- ### WDL Input Skeleton Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/WOMtool.md This JSON output represents a skeleton for WDL workflow inputs, where keys are input names and values indicate the expected data type. ```json { "three_step.cgrep.pattern": "String" } ``` -------------------------------- ### GCP Batch gsutil Error for Composite Files Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/backends/GCPBatch.md This is an example of an error message encountered when downloading a composite file on a system without a compiled crcmod. It highlights the need for crcmod to be installed for efficient integrity checking during downloads. ```bash / # gsutil -o GSUtil:parallel_composite_upload_threshold=150M cp gs://my-bucket/composite.bam . Copying gs://my-bucket/composite.bam... ==> NOTE: You are downloading one or more large file(s), which would run significantly faster if you enabled sliced object downloads. This feature is enabled by default but requires that compiled crcmod be installed (see "gsutil help crcmod"). CommandException: Downloading this composite object requires integrity checking with CRC32c, but your crcmod installation isn't using the module's C extension, so the hash computation will likely throttle download performance. For help installing the extension, please see "gsutil help crcmod". To download regardless of crcmod performance or to skip slow integrity checks, see the "check_hashes" option in your boto config file. NOTE: It is strongly recommended that you not disable integrity checks. Doing so could allow data corruption to go undetected during uploading/downloading. / # ``` -------------------------------- ### Start Cromwell Server Mode Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/CommandLine.md Run Cromwell as a web server using the 'server' command. This mode is suitable for production and accepts REST requests. The API is self-documented via Swagger. ```bash java -jar cromwell.jar server ``` -------------------------------- ### Download and Set Up Cromwell Directory Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/FiveMinuteIntro.md Use these commands to create a directory for Cromwell, copy the downloaded JAR file, and navigate into it. Adjust the JAR filename to match your downloaded version. ```sh cd ~ mkdir cromwell cp ~/Downloads/cromwell-XY.jar cromwell/ cd cromwell/ ``` -------------------------------- ### GCP Batch Custom Mount Points Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/backends/GCPBatch.md Specifies custom mount points for GCP Batch jobs. All custom mount points must be prefixed with /mnt/disks. ```wdl runtime { disks: "/mnt/disks/my_mnt 30 SSD, /mnt/disks/my_mnt2 500 HDD" } ``` -------------------------------- ### Install Cromwell using Homebrew Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Releases.md For Mac users with Homebrew, Cromwell can be installed using the brew command. ```bash brew install cromwell ``` -------------------------------- ### REST API: Get Workflow Outputs — `GET /api/workflows/v1/{id}/outputs` Source: https://context7.com/broadinstitute/cromwell/llms.txt Retrieve the final outputs of a completed workflow. ```APIDOC ## REST API: Get Workflow Outputs — `GET /api/workflows/v1/{id}/outputs` Retrieve the final outputs of a completed workflow. ```sh WF_ID="e442e52a-9de1-47f0-8b4f-e6e565008cf1" curl -s http://localhost:8000/api/workflows/v1/${WF_ID}/outputs # Expected response: # { # "id": "e442e52a-9de1-47f0-8b4f-e6e565008cf1", # "outputs": { # "myWorkflow.myTask.out": "hello world" # } # } ``` ``` -------------------------------- ### Install Cromwell using Conda Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Releases.md Install Cromwell in the base environment or a new environment using conda. Ensure the conda-forge channel is activated. ```bash conda install -c conda-forge cromwell ``` ```bash conda create -n cromwell cromwell ``` -------------------------------- ### Sub-Workflow Metadata Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/SubWorkflows.md Shows an example of a Cromwell metadata query response for a parent workflow that includes a sub-workflow, highlighting the 'subWorkflowId' attribute. ```json { "workflowName": "main_workflow", "submittedFiles": { "inputs": "{}", ``` -------------------------------- ### WDL Fork-Join Workflow Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/WOMtool.md This is a sample WDL workflow demonstrating a fork-join pattern with multiple tasks. It includes task definitions for file creation, pattern matching, counting lines, and calculation. ```wdl task mkFile { command { for i in `seq 1 1000` do echo $i done } output { File numbers = stdout() } runtime {docker: "ubuntu:latest"} } task grep { String pattern File in_file command { grep '${pattern}' ${in_file} | wc -l } output { Int count = read_int(stdout()) } runtime {docker: "ubuntu:latest"} } task wc { File in_file command { cat ${in_file} | wc -l } output { Int count = read_int(stdout()) } runtime {docker: "ubuntu:latest"} } task join { Int grepCount Int wcCount command { expr ${wcCount} / ${grepCount} } output { Int proportion = read_int(stdout()) } runtime {docker: "ubuntu:latest"} } workflow forkjoin { call mkFile call grep { input: in_file = mkFile.numbers } call wc { input: in_file=mkFile.numbers } call join { input: wcCount = wc.count, grepCount = grep.count } output { join.proportion } } ``` -------------------------------- ### Example WDL Workflow with EFS Source: https://github.com/broadinstitute/cromwell/blob/develop/supportedBackends/aws/src/main/scala/cromwell/backend/impl/aws/README.md This WDL workflow demonstrates taking input from S3, generating intermediate data on EFS, globbing files on S3, and delocalizing output from EFS to S3. It also shows reading files directly from EFS. ```wdl version 1.0 workflow TestEFS { input { # input file for WF is located on a public S3 File s3_file = 's3://cromwell-aws-cloudformation-templates/root-templates/aws-vpc.template.yaml' # set an input parameter holding the working dir on EFS String efs_wd = "/mnt/efs/MyTestProject" } # task one : create a file and a glob on efs. do not delocalize call task_one { input: infile = s3_file, wd = efs_wd } # read the outfile on EFS straight in a wdl structure Array[Array[String]] step1_info = read_tsv(task_one.efs_file) # task two : reuse the file on the wd and delocalize to s3 (via runtime setting) call task_two { input: wd = efs_wd, infile = task_one.efs_file } Array[Array[String]] step2_info = read_tsv(task_two.outfile) # run a task on the various files (get md5). call task_three { input: ``` -------------------------------- ### Markdown Code Block Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/Contribute.md Use three backticks to format a block of code in Markdown. This example shows a basic workflow definition. ```markdown ``` workflow myWorkflow { call myTask } ``` ``` -------------------------------- ### Import File-Based WDL Resources Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Imports.md Demonstrates importing WDL files from a ZIP bundle. Paths are relative to the root of the ZIP file. ```wdl import "my-wdl-in-the-root-directory.wdl" as file_import1 import "my/wdl/sub/directory/example.wdl" as file_import2 ``` -------------------------------- ### Backpressure Report CSV Output Example Source: https://github.com/broadinstitute/cromwell/blob/develop/scripts/backpressure_report/README.md Example of the CSV output generated by the backpressure_report script. It shows backpressure duration per interval and per pod. ```csv Interval (1 hour),All pods,Pod 47z68,Pod 4hgd4,Pod 7svrs,Pod 9l2ld,Pod 9p9j4,Pod bj4vh,Pod d85vc,Pod gdp8x,Pod gth4r,Pod jkpbj,Pod jrgsx,Pod ltmvs,Pod mkdjt,Pod qt2bq,Pod th2p8,Pod thwz9,Pod xvcrk,Pod z7jfk 2022-01-01 05:00:00+00:00,62,20,0,0,0,42,0,0,0,0,0,0,0,0,0,0,0,0,0 2022-01-01 06:00:00+00:00,40,0,0,0,0,0,0,0,40,0,0,0,0,0,0,0,0,0,0 2022-01-01 07:00:00+00:00,20,20,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 2022-01-01 08:00:00+00:00,40,20,0,0,0,20,0,0,0,0,0,0,0,0,0,0,0,0,0 2022-01-01 09:00:00+00:00,110,0,0,0,0,70,0,0,40,0,0,0,0,0,0,0,0,0,0 ... ``` -------------------------------- ### Markdown Syntax Highlighting Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/Contribute.md Include the language name after the opening three backticks for syntax highlighting in Markdown code blocks. This example demonstrates JSON formatting. ```markdown ```json { "MyWorkflow.MyTask.VariableTwo": "Variable2" } ``` ``` -------------------------------- ### Hello World WDL Workflow Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/GcpBatch101.md A simple WDL workflow that defines a task to echo a greeting and a workflow to call that task. It includes a runtime section specifying the Docker image. ```wdl task hello { String addressee command { echo "Hello ${addressee}! Welcome to Cromwell . . . on Google Cloud!" } output { String message = read_string(stdout()) } runtime { docker: "ubuntu:latest" } } workflow wf_hello { call hello output { hello.message } } ``` -------------------------------- ### Hello World WDL Inputs Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/GcpBatch101.md JSON file defining the input values for the 'wf_hello' workflow, specifically setting the 'addressee' for the 'hello' task. ```json { "wf_hello.hello.addressee": "World" } ``` -------------------------------- ### REST API: Get Workflow Metadata — `GET /api/workflows/v1/{id}/metadata` Source: https://context7.com/broadinstitute/cromwell/llms.txt Retrieve full metadata for a workflow including call-level details, timing, and execution events. ```APIDOC ## REST API: Get Workflow Metadata — `GET /api/workflows/v1/{id}/metadata` Retrieve full metadata for a workflow including call-level details, timing, and execution events. ```sh WF_ID="e442e52a-9de1-47f0-8b4f-e6e565008cf1" # Basic metadata curl -s "http://localhost:8000/api/workflows/v1/${WF_ID}/metadata" ``` ``` -------------------------------- ### Generate Java Client Library Source: https://github.com/broadinstitute/cromwell/blob/develop/RELEASE_PROCESS.md Run this script from the repository root to generate the client library and its tests. ```bash ./scripts/gen_java_client.sh ``` -------------------------------- ### Cromwell Workflow Output Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/GcpBatch101.md Example of a successful Cromwell workflow execution, showing the workflow ID and its outputs. ```json { "outputs": { "wf_hello.hello.message": "Hello World! Welcome to Cromwell . . . on Google Cloud!" }, "id": "08213b40-bcf5-470d-b8b7-1d1a9dccb10e" } ``` -------------------------------- ### Get Workflow Metadata Source: https://context7.com/broadinstitute/cromwell/llms.txt Retrieves detailed metadata for a specific workflow, with an option to expand sub-workflow information. ```APIDOC ## GET /api/workflows/v1/{WF_ID}/metadata ### Description Retrieves metadata for a specific workflow. Use `expandSubWorkflows=true` to include metadata for any sub-workflows. ### Method GET ### Endpoint /api/workflows/v1/{WF_ID}/metadata?expandSubWorkflows=true ### Parameters #### Query Parameters - **expandSubWorkflows** (boolean) - Optional - If true, includes metadata for sub-workflows. ### Response #### Success Response (200) - **workflowName** (string) - The name of the workflow. - **id** (string) - The unique identifier of the workflow. - **status** (string) - The current status of the workflow (e.g., Succeeded, Failed). - **submission** (string) - The timestamp when the workflow was submitted. - **start** (string) - The timestamp when the workflow started execution. - **end** (string) - The timestamp when the workflow finished execution. - **calls** (object) - Information about the calls made within the workflow. - **outputs** (object) - The outputs produced by the workflow. ``` -------------------------------- ### Start Cromwell Server Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/ServerMode.md Execute this command in your terminal to launch Cromwell in server mode. Replace '[version]' with your Cromwell JAR file's version. Access the server interface at http://localhost:8000. ```bash java -jar cromwell-[version].jar server ``` -------------------------------- ### Define a Simple Workflow in WDL Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/ServerMode.md This WDL script defines a basic task 'hello' that echoes a greeting and a workflow 'test' that calls this task. Save this content into a file named 'hello.wdl'. ```wdl task hello { String name command { echo 'Hello ${name}!' } output { File response = stdout() } } workflow test { call hello } ``` -------------------------------- ### Checkout Cromwell Version 52 Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/HPCSlurmWithLocalScratch.md Checks out version 52 of the Cromwell repository, as this guide was validated with this specific version. ```hocon cd cromwell git checkout 52 ``` -------------------------------- ### Display Cromwell Built-in Help Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/CommandLine.md Run the Cromwell JAR file with no arguments to display the built-in usage message. This provides an overview of available commands and options. ```bash java -jar cromwell-.jar ``` -------------------------------- ### Specify CPU Runtime Attribute Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/HPCIntro.md Example of defining the 'cpu' runtime attribute, which must be an integer and will be validated to be at least 1. ```hocon backend.providers.SGE.config { runtime-attributes = """ Int cpu = 1 # ... """ } ``` -------------------------------- ### Configure HTTP Filesystem in Backend Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/filesystems/HTTP.md Add a reference to the 'http' filesystem within a backend's 'filesystems' stanza to enable HTTP input support. No additional configuration is required for the HTTP filesystem itself. ```hcl backend { default = "Local" providers { Local { ... config { filesystems { local { ... } http { } } } ... } ... } } ``` -------------------------------- ### Cromwell Workflow Success Output Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/AwsBatch101.md Example of a successful Cromwell workflow execution log, indicating the workflow status and its outputs. ```json [info] SingleWorkflowRunnerActor workflow finished with status 'Succeeded'. { "outputs": { "wf_hello.hello.message": "Hello World! Welcome to Cromwell . . . on AWS!" }, "id": "08213b40-bcf5-470d-b8b7-1d1a9dccb10e" } ``` -------------------------------- ### CPU Runtime Attribute Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/RuntimeAttributes.md Specifies the number of CPU cores required for a task. The interpretation may vary by backend. ```wdl runtime { cpu: 2 } ``` -------------------------------- ### WDL Task with Runtime Attributes Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/RuntimeAttributes.md Example of a WDL task defining runtime attributes directly within the task definition. ```wdl task first { command { ... } } task second { command {...} runtime { docker: "my_docker_image" } } workflow w { call first call second } ``` -------------------------------- ### WOMtool General Command Structure Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/WOMtool.md The basic structure for running WOMtool commands involves specifying the JAR file, the action to perform, and any necessary parameters. ```bash java -jar /path/to/womtool.jar ``` -------------------------------- ### Sample DRS Filesystem Configuration Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/filesystems/DataRepositoryService.md A comprehensive example demonstrating the integration of DRS filesystem configuration within the Cromwell engine and a Papi backend. Ensure this is placed within the appropriate sections of your Cromwell configuration. ```hocon engine { filesystems { # ... other filesystems here, probably gcs, and then ... drs { auth = "application-default" } } } backend { # ... other global backend config here, probably just setting the default ... providers { # ... other providers here ... Papi { actor-factory = "cromwell.backend.google.pipelines.v2beta.PipelinesApiLifecycleActorFactory" config { # ... other config here ... filesystems { # ... other filesystems here, probably gcs, and then ... drs { auth = "application-default" } } } } } } drs { localization { docker-image = "broadinstitute/drs-localizer:latest" } } ``` -------------------------------- ### Query Workflows by Label Source: https://context7.com/broadinstitute/cromwell/llms.txt Filters workflows based on assigned labels. This example specifically searches for workflows tagged with 'project:my-analysis'. ```bash # Filter by label curl -s "http://localhost:8000/api/workflows/v1/query?label=project:my-analysis" ``` -------------------------------- ### WDL to Hermes AST Conversion Example Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/bitesize/workflowParsing/wdlToWdlom.md Shows a simple WDL declaration and how Hermes parses it into an Abstract Syntax Tree (AST). This AST is an intermediate representation used before WDLOM conversion. ```wdl Int foo = min(100, max(1,2)) ``` -------------------------------- ### Run Cromwell with Options File Source: https://context7.com/broadinstitute/cromwell/llms.txt Executes a WDL workflow using a specified options file. This allows for consistent configuration across multiple runs. ```bash # Use options file in run mode java -jar cromwell-93.jar run workflow.wdl --options options.json ``` -------------------------------- ### REST API: Check Workflow Status — `GET /api/workflows/v1/{id}/status` Source: https://context7.com/broadinstitute/cromwell/llms.txt Poll the execution status of a submitted workflow. ```APIDOC ## REST API: Check Workflow Status — `GET /api/workflows/v1/{id}/status` Poll the execution status of a submitted workflow. ```sh WF_ID="e442e52a-9de1-47f0-8b4f-e6e565008cf1" curl -s http://localhost:8000/api/workflows/v1/${WF_ID}/status # Possible status values: Submitted, Running, Aborting, Aborted, Failed, Succeeded # Expected response: # { # "id": "e442e52a-9de1-47f0-8b4f-e6e565008cf1", # "status": "Succeeded" # } ``` ``` -------------------------------- ### Generate and Host ScalaDoc Source: https://github.com/broadinstitute/cromwell/wiki/DevZone Generate Scala documentation using `sbt doc`, then commit the output to the `gh-pages` branch for hosting. This process involves moving the generated API to a specific directory and pushing to the remote repository. ```bash sbt doc git co gh-pages mv target/scala-2.12/api scaladoc git add scaladoc git commit -m "API Docs" git push origin gh-pages ``` -------------------------------- ### Define Memory Runtime Attribute in WDL Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/tutorials/HPCIntro.md Example of specifying memory requirements for a WDL task using a specific unit like MB. ```wdl task hello { command { echo hello } runtime { memory: "512 MB" } } ``` -------------------------------- ### Disable HTTP Filesystem Configuration Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/filesystems/Filesystems.md To disable a filesystem, explicitly set 'enabled: false' in your overriding configuration. This example disables the http filesystem. ```hocon engine { filesystems { http { enabled: false } } } ``` -------------------------------- ### Configure HSQLDB for File-Based Persistence Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/Configuring.md HSQLDB can be configured for file-based persistence, eliminating the need for a separate database server. This example details the connection URL and other HSQLDB-specific properties for optimal performance and persistence. ```hocon database { profile = "slick.jdbc.HsqldbProfile$" db { driver = "org.hsqldb.jdbcDriver" url = "" jdbc:hsqldb:file:cromwell-executions/cromwell-db/cromwell-db; shutdown=false; hsqldb.default_table_type=cached; hsqldb.tx=mvcc; hsqldb.result_max_memory_rows=10000; hsqldb.large_data=true; hsqldb.applog=1; hsqldb.lob_compressed=true; hsqldb.script_format=3 " connectionTimeout = 120000 numThreads = 1 } } ``` -------------------------------- ### Centaur Metadata Verification Examples Source: https://github.com/broadinstitute/cromwell/blob/develop/docs/developers/Centaur.md Illustrates how to specify metadata fields for verification after a workflow completes, including handling lists and nested structures. ```HOCON metadata { // Examples: // failures is a list, the first entry (0) might be the error you are looking for. If multiple errors are expected the entire list can be checked. // It has a "message" and a "causedBy" field. "failures.0.message": "Cromwell senses you did not use WomTool validate." "failures.0.causedBy": "BetweenKeyboardAndChairException" } ```