### Set Up Python Virtual Environment and Install Dependencies

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

Commands to create a Python virtual environment, activate it, upgrade pip, install development requirements, and install Cromshell in editable mode.

```shell
python3 -mvenv venv
. venv/bin/activate
pip install --upgrade pip
pip install -r dev-requirements.txt
pip install -e .
```

--------------------------------

### Cromshell Configuration File Example

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

An example JSON structure for the cromshell_config.json file, specifying the Cromwell server URL and request timeout.

```json
{
  "cromwell_server": "http://localhost:8000",
  "requests_timeout": 5
}
```

--------------------------------

### Install Cromshell from Source

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Installs Cromshell by cloning the repository and installing it using pip. This is useful for developers or users who want the latest unreleased changes.

```bash
git clone git@github.com:broadinstitute/cromshell.git
cd cromshell
pip install .
```

--------------------------------

### Cromshell Installation

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Instructions for installing cromshell using Homebrew or Conda, and an alternative method of downloading the script.

```bash
brew install broadinstitute/dsp/cromshell
conda install cromshell
```

--------------------------------

### Cromshell Usage Examples

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Common examples demonstrating how to use the cromshell script for submitting jobs, checking status, and retrieving metadata or logs.

```bash
cromshell submit workflow.wdl inputs.json dependencies.zip
cromshell status
cromshell -t 20 metadata
cromshell logs -2
```

--------------------------------

### Brew Formula for Cromshell

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/publish_release.md

Example of a Homebrew formula file for installing Cromshell. It specifies dependencies, resources (Python packages with their URLs and hashes), and installation steps.

```Ruby
class CromshellAT200Alpha1 < Formula
  include Language::Python::Virtualenv

  desc "Python tool to interact with a Cromwell server"
  homepage "https://github.com/broadinstitute/cromshell"
  url "https://github.com/broadinstitute/cromshell/archive/refs/tags/2.0.0.alpha.1.tar.gz"
  sha256 "0ac3604ff362fdf2107f91c886887856098f1731d2d1a9e9cff59278b7b292b9"
  license "BSD-3-Clause"
  depends_on "python@3.9"

  resource "termcolor" do
    url "https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz"
    sha256 "1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b"
  end
  resource "click>=8.0.0" do
    url "https://files.pythonhosted.org/packages/f4/09/ad003a2791c4c41111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
```

--------------------------------

### Cromshell CLI Usage Examples

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Demonstrates common commands for submitting workflows, checking status, and retrieving metadata using the Cromshell CLI.

```bash
cromshell submit workflow.wdl inputs.json options.json dependencies.zip
cromshell status
cromshell -t 20 metadata
cromshell logs -2
```

--------------------------------

### Unit Test Example for get_hello_world

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/addtests.md

An example of a unit test file for the `get_hello_world` function in Cromshell. It demonstrates importing the command, defining a test class, writing a basic test, and using pytest's parametrize feature for multiple test cases.

```python
import pytest

from cromshell.my_new_command import command as my_new_command


class TestMyNewCommand:
    
    """Tests for my_new_command """

    # First Test
    def test_get_hello_world(self):

        y = "hello world"

        assert my_new_command.get_hello_world() == y
    
    # Second Test
    @pytest.mark.parametrize(
        "y",
        [
            ("wombat"),
            ("foo"),
            ("Failed"),
        ],
    )
    def test_get_hello_world_using_parametrize(self, y: str):

        assert my_new_command.get_hello_world() != y
```

--------------------------------

### Install Testing Dependencies

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/addtests.md

Installs necessary Python packages for testing Cromshell using pip and a requirements file.

```shell
pip install -r test-requirements.txt
```

--------------------------------

### Initialize Cromshell Configuration

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

Command to run to trigger the creation of the default Cromshell configuration directory and files if they do not exist.

```shell
cromshell-alpha version
```

--------------------------------

### Install Cromshell via Pip

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Installs Cromshell using pip, the Python package installer. This method is suitable for users who prefer managing Python packages directly.

```bash
pip install cromshell
```

--------------------------------

### Install Cromshell via Homebrew

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Installs Cromshell using the Homebrew package manager. This is a convenient way to install and manage the tool on macOS.

```bash
brew tap broadinstitute/dsp
brew install cromshell
```

--------------------------------

### Cromshell Help Command

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

Displays the usage information for the Cromshell command-line interface, showing available options and commands.

```shell
cromshell --help
```

--------------------------------

### Clone and Checkout Cromshell Repository

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

Steps to clone the Cromshell Git repository and checkout the specific 'cromshell_2.0' branch, followed by creating a new feature branch.

```shell
git clone git@github.com:broadinstitute/cromshell.git
cd cromshell
git checkout cromshell_2.0
git checkout -b <your initial>_<name of your new branch>
```

--------------------------------

### Install Release Tools

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/publish_release.md

Installs or upgrades pip, twine, and build, which are essential tools for packaging and distributing Python projects.

```bash
python3 -m pip install --upgrade pip twine build
```

--------------------------------

### Integration Test Example for my_new_command

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/addtests.md

This Python code demonstrates an integration test for a hypothetical 'my-new-command' in cromshell. It utilizes the `run_cromshell_command` utility function to execute the command with a given workflow ID and asserts the expected output. It includes two test cases: one for the basic command execution and another for execution with a '-p' option.

```python
import pytest

from tests.integration import utility_test_functions

class TestMyNewCommand:

    """Tests for my_new_command """

    # First Test
    def test_my_new_command(self):
        workflow_id = "639b81f9-414e-4c65-8dcf-bf7c57ffdff7"
        # Run cromshell alias
        results = utility_test_functions.run_cromshell_command(
            command=[
                "my-new-command",
                workflow_id,
            ],
            exit_code=0,
        )
        assert results.stdout.rstrip() == "hello world"

    # Second Test
    def test_my_new_command_with_print_option(self):
        workflow_id = "639b81f9-414e-4c65-8dcf-bf7c57ffdff7"
        # Run cromshell alias
        results = utility_test_functions.run_cromshell_command(
            command=[
                "my-new-command",
                "-p",
                workflow_id,
            ],
            exit_code=0,
        )
        assert results.stdout.rstrip() == "hello world"+"\n"+workflow_id
```

--------------------------------

### Update Project Version with Bumpversion

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

Instructions on how to use the 'bumpversion' tool to update the project's semantic version (major, minor, or patch).

```shell
bumpversion PART
```

--------------------------------

### Cromshell CLI Help Output

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/addcommand.md

Example output of the `cromshell-beta --help` command after a new command has been added and integrated. This shows how the new command, 'my-new-command', appears in the list of available commands with its associated help text.

```bash
cromshell-beta --help
Usage: cromshell-beta [OPTIONS] COMMAND [ARGS]...
  Cromshell is a script for submitting workflows to a cromwell server and
  monitoring / querying their results.
  Notes:
      - A hidden folder will be created on initial run. The hidden folder
      (.../.cromshell) will be placed in users home directory but can be
      overridden by setting environment variable 'CROMSHELL_CONFIG'.
Options:
  --help                          Show this message and exit.
Commands:
  abort          Abort a running workflow.
  **my-new-command My-new-command does things.**
  alias          Label the given workflow ID or relative id with the...
  version        Print the version of cromshell
```

--------------------------------

### Workflow Status and Metadata

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Commands to check the status, retrieve metadata, and get job counts for workflows.

```APIDOC
status [workflow-id] [[workflow-id]...]
  Check the status of a workflow.

metadata [workflow-id] [[workflow-id]...]
  Get the full metadata of a workflow.

slim-metadata [workflow-id] [[workflow-id]...]
  Get a subset of the metadata from a workflow.

counts [-j] [-x] [workflow-id] [[workflow-id]...]
  Get the summarized status of all jobs in the workflow.
  -j: prints a JSON instead of a pretty summary of the execution status (compresses subworkflows).
  -x: compress sub-workflows for less detailed summarization.
```

```APIDOC
timing [workflow-id] [[workflow-id]...]
  Open the timing diagram in a browser.
```

--------------------------------

### Cromshell Cost Estimation Commands

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Commands for retrieving cost information for workflows. Costs are available for workflows completed more than 8 hours ago on a GCS backend. Requires specific configuration files and BigQuery setup.

```APIDOC
cost [workflow-id] [[workflow-id]...]
  Get the cost for one or more workflows.
  Requires GCS backend, completion > 8 hours ago, and configured BigQuery cost table.

cost-detailed [workflow-id] [[workflow-id]...]
  Get the cost for one or more workflows at the task level.
  Requires GCS backend, completion > 8 hours ago, and configured BigQuery cost table.
```

--------------------------------

### Run Cromwell Server with Docker

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

Docker command to run a Cromwell version 67 server in detached mode, exposing port 8000.

```shell
docker run -d -p 8000:8000 broadinstitute/cromwell:67 server
```

--------------------------------

### HelloWorldTask Outputs

Source: https://github.com/broadinstitute/cromshell/blob/main/tests/unit/mock_data/list_outputs/print_file_like_value_in_dict_no_indent_example.txt

Lists the output file paths for the HelloWorldTask, including BAM subset file index, BAM subset file, and timing information.

```text
HelloWorld.HelloWorldTask.bam_subset_file_index: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bai
HelloWorld.HelloWorldTask.bam_subset_file: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bam
HelloWorld.HelloWorldTask.timing_info: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.timingInformation.txt
```

--------------------------------

### HelloWorldTask File References

Source: https://github.com/broadinstitute/cromshell/blob/main/tests/unit/mock_data/list_outputs/helloworld_task_level_outputs.txt

This snippet details the file inputs and outputs for the HelloWorldTask. It includes references to BAM subset files, their indices, and timing information stored in Google Cloud Storage.

```cromshell
HelloWorld.HelloWorldTask:
	bam_subset_file_index: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bai
	bam_subset_file: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bam
	timing_info: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.timingInformation.txt
```

--------------------------------

### Cromshell Feature Explanations

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Explains key features of cromshell, including automatic job folder creation for reproducibility, referencing previous jobs using negative indices, overriding the default Cromwell server via environment variables, and passing custom headers for authentication.

```APIDOC
Features:
- Reproducibility: `submit` creates a job-specific folder in `~/.cromshell/${CROMWELL_URL}/` containing WDL and input files.
- Job Referencing: Use negative indices (e.g., -1, -2) to refer to recently submitted jobs.
- Server Override: Set the `CROMWELL_URL` environment variable to specify a different Cromwell server.
- Multiple IDs: Commands accept multiple workflow IDs, including relative and absolute references (e.g., `./cromwell status -1 -2 c2db2989-2e09-4f2c-8a7f-c3733ae5ba7b`).
- Custom Headers: Set the `CROMSHELL_HEADER` environment variable for custom HTTP headers, useful for authentication (e.g., `CROMSHELL_HEADER="Authorization: Bearer 3e2f34f2e..."`).
```

--------------------------------

### Uninstall Cromshell via Homebrew

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Uninstalls Cromshell when it was installed using Homebrew.

```bash
brew uninstall cromshell
```

--------------------------------

### Uninstall Cromshell via Pip

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Uninstalls Cromshell when it was installed using pip or from source.

```bash
pip uninstall cromshell
```

--------------------------------

### HelloWorld Task File Paths

Source: https://github.com/broadinstitute/cromshell/blob/main/tests/unit/mock_data/list_outputs/print_file_like_value_in_dict_example.txt

Lists the Google Cloud Storage (GCS) paths for various output files generated by the HelloWorld task.

```text
HelloWorld.HelloWorldTask.bam_subset_file_index: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bai
HelloWorld.HelloWorldTask.bam_subset_file: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bam
HelloWorld.HelloWorldTask.timing_info: gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.timingInformation.txt
```

--------------------------------

### Cromshell Supported Options

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Details the various command-line options available for configuring Cromshell's behavior, including server connection, timeouts, and output styling.

```APIDOC
Cromshell Options:

--no_turtle | --I_hate_turtles
  Description: Hide the turtle logo from the output.

--cromwell_url [TEXT]
  Description: Specifies the Cromwell server URL.
  Example: http://65.61.654.8:8000
  Note: The port may not be necessary depending on server configuration.

-t [TIMEOUT]
  Description: Sets the server connection timeout in seconds.
  Default: 5 seconds.
  Constraint: TIMEOUT must be a positive integer.

--gcloud_token_email [TEXT]
  Description: Uses the provided email to call `gcloud auth print-access-token` and includes the token in the auth header.

--referer_header_url [TEXT]
  Description: Provides a URL for the `Referer:` header, useful for servers requiring referer validation.

--machine_processable | --colorful_output
  Description: Overrides the automatic detection of output coloring. If neither is specified, output is colored if connected to an interactive terminal.
```

--------------------------------

### Cromshell Test Syntax

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Demonstrates the recommended syntax for writing tests in cromshell, using double brackets for test conditions.

```shell
[[ ... ]]
```

--------------------------------

### MiniWDL Dependency Option

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Explains the `--dependencies-zip` option for MiniWDL, used to specify local imports for workflows.

```bash
cromshell submit --dependencies-zip my_workflow_deps.zip my_workflow.wdl
```

--------------------------------

### Python Function for Alias Existence Check

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/README.md

A Python function stub that checks if an alias name already exists within a specified submission file. It includes type hints and a docstring.

```python
def alias_exists(alias_name: str, submission_file: str) -> bool:
   """
   Check if alias name already exists in submission file
   :param alias_name: Alternate string identifier for workflow submission
   :param submission_file: Path to cromshell submission file
   :return:
   """
   ...
```

--------------------------------

### Cromshell Command Line Arguments

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Demonstrates common command-line arguments for Cromshell, including overriding the Cromwell URL and configuration directory.

```bash
# Override Cromwell server URL
cromshell --cromwell_url http://localhost:8000 status

# Override Cromshell configuration folder
export CROMSHELL_CONFIG=/path/to/custom/config
cromshell submit my_workflow.wdl

# Use relative job IDs
cromshell status -1 -2

# Assign an alias to a job ID
cromshell alias -1 myAliasName
cromshell status myAliasName
```

--------------------------------

### Server and Cost Management

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Commands for updating the Cromwell server and retrieving workflow costs.

```APIDOC
update-server
  Change the Cromwell server that new jobs will be submitted to.

cost [-c] [-d] [workflow-id] [[workflow-id]...]
  Get the cost for a workflow.
  Only works for workflows that completed more than 24 hours ago on GCS. Requires billing export to BigQuery to be enabled.
  Requires the `bq_cost_table` key in the cromshell configuration file.
  Example `~/.cromshell/cromshell_config.json`:
  {
    "cromwell_server": "<cromwell_server_url>",
    "requests_timeout": 5,
    "bq_cost_table": "<table_name>"
  }
  -c/--color: Color outliers in task level cost results.
  -d/--detailed: Get the cost for a workflow at the task level.
```

--------------------------------

### Cromshell Local Cache and Server Update Commands

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Includes commands for managing the local cache of jobs and updating the Cromwell server configuration. `cleanup` removes completed jobs from the local list, optionally filtering by status. `update-server` changes the default Cromwell server for new submissions.

```APIDOC
cleanup [-s STATUS]
  Clean up local cached list of jobs.
  Removes jobs with completed states (Succeeded, Failed, Aborted).
  -s STATUS: If provided, only remove jobs with the specified STATUS.

update-server
  Change the Cromwell server that new jobs will be submitted to.
```

--------------------------------

### Local Job Listing and Cleanup

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Commands for listing submitted jobs and cleaning up the local cache.

```APIDOC
list [-c] [-u]
  Display a list of jobs submitted through cromshell.
  -c: Color the output by completion status.
  -u: Check completion status of all unfinished jobs.

cleanup [-s STATUS]
  [COMING SOON] Remove completed jobs from local list.
  Removes all jobs from the local list that are in a completed state: 'Succeeded', 'Failed', 'Aborted'.
  -s [STATUS]: If provided, will only remove jobs with the given [STATUS] from the local list.
```

--------------------------------

### Setting up Local Cromwell Server for Integration Testing

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/runtests.md

Command to run a local Cromwell server in a Docker container, which is necessary for certain integration tests that require a running Cromwell instance.

```shell
docker run -d -p 8000:8000 broadinstitute/cromwell:67 server
```

--------------------------------

### Cromshell Notification and Listing Commands

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Covers commands related to job completion notifications and listing submitted jobs. The `notify` command allows setting up email alerts, while `list` provides options for viewing job statuses with colorization and completion checks.

```APIDOC
notify [workflow-id] [daemon-server] email [cromwell-server]
  Get email notification on job completion.
  Parameters:
    daemon-server: Server to run the notification daemon on.

list [-c] [-u]
  Display a list of jobs submitted through cromshell.
  Options:
    -c: Color output by completion status.
    -u: Check completion status of unfinished jobs.
```

--------------------------------

### Cromshell Workflow Management Commands

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

This section details the core subcommands for managing workflows in cromshell. It covers submitting new workflows, aborting running ones, and aliasing workflows for easier referencing. Dependencies include WDL and JSON input files, and optionally womtool for validation.

```APIDOC
submit [-w] <wdl> <inputs_json> [options_json] [included_wdl_zip_file]
  Submit a new workflow. Optionally waits for status transition.
  Dependencies: womtool (for validation), WDL file, JSON input file.
  Options:
    -w: Wait for workflow to transition from 'Submitted' to another status.
    included_wdl_zip_file: Zip file containing included WDL files.

abort [workflow-id] [[workflow-id]...]
  Abort one or more running workflows.

alias <workflow-id> <alias_name>
  Label a workflow ID with an alias name for easier referencing.
```

--------------------------------

### Cromshell Log and Output Management Commands

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Commands for accessing logs and outputs generated by workflows. This includes listing log files, downloading all logs, listing output files, and downloading all output files. Supports referencing multiple workflows.

```APIDOC
logs [workflow-id] [[workflow-id]...]
  List log files produced by one or more workflows.

fetch-logs [workflow-id] [[workflow-id]...]
  Download all logs produced by one or more workflows.

list-outputs [workflow-id] [[workflow-id]...]
  List all output files produced by one or more workflows.

fetch-all [workflow-id] [[workflow-id]...]
  Download all output files produced by one or more workflows.
```

--------------------------------

### Log and Output Retrieval

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Commands for listing and fetching workflow logs and outputs.

```APIDOC
logs [workflow-id] [[workflow-id]...]
  List the log files produced by a workflow.

fetch-logs [workflow-id] [[workflow-id]...]
  [COMING SOON] Download all logs produced by a workflow.

list-outputs [workflow-id] [[workflow-id]...]
  List all output files produced by a workflow.

fetch-all [workflow-id] [[workflow-id]...]
  [COMING SOON] Download all output files produced by a workflow.
```

--------------------------------

### Linting Files

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/runtests.md

Commands to perform various linting tasks on the project files. This includes running all linting tools via Tox, reformatting code with Black, sorting imports with isort, checking PEP8 compliance with Flake8, and linting Python code with Pylint.

```shell
# run all linting commands
tox -e lint
```

```shell
# reformat all project files
black src tests
```

```shell
# sort imports in project files
isort -rc src tests
```

```shell
# check pep8 against all project files
flake8 src tests
```

```shell
# lint python code for common errors and codestyle issues
pylint src
```

--------------------------------

### Mock Data Directory Structure

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/mockdata.md

Illustrates the organization of mock data files and directories within the Cromshell project's tests directory. This includes workflow metadata, configuration files, and templates.

```text
> tests
    > integration
       > mock
           all.workflow.database.tsv
    > unit
       > mock
           > submit
               submission_file_template.text
           all.workflow.database.tsv
           cromshell_config.json           
    > workflows
        helloWorld.json
        helloWorld.wdl
    __init__.py
    conftest.py 
```

--------------------------------

### Workflow Submission and Management

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Commands for submitting new workflows, aborting running ones, and managing workflow aliases.

```APIDOC
submit [-w] <wdl> <inputs_json> [options_json] [included_wdl_zip_file]
  Submit a new workflow to the Cromwell server. Automatically validates the WDL and JSON file.
  -w [COMING SOON]: Wait for workflow to transition from 'Submitted' to some other status before SCRIPTNAME exits.
  included_wdl_zip_file: Zip file containing any WDL files included in the input WDL.

abort [workflow-id] [[workflow-id]...]
  Abort a running workflow.
```

```APIDOC
alias <workflow-id> <alias_name>
  Label the given workflow ID with the given alias_name. Aliases can be used in place of workflow IDs to reference jobs.
  Remove an alias by passing empty double quotes as alias_name (e.g. alias <workflow-id> "").
```

--------------------------------

### Upload to PyPI or TestPyPI

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/publish_release.md

Uploads the built distribution files from the 'dist' directory to either PyPI or testPyPI using twine. Requires a PYPI Token and specifies the repository to upload to.

```bash
python3 -m twine upload --username __token__ --password <PYPI_TOKEN> --repository <testpypi/pypi> dist/*
```

--------------------------------

### Build Cromshell Distribution

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/publish_release.md

Builds the Cromshell project, creating distribution files (e.g., .tar.gz, .whl) in the 'dist' directory. This command should be run from the root repository directory.

```bash
python3 -m build
```

--------------------------------

### Cromshell Workflow Information and Status Commands

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Provides commands to retrieve information and status about workflows. This includes checking the overall status, fetching detailed or slim metadata, counting execution statuses, and visualizing workflow timing. Multiple workflow IDs can be provided.

```APIDOC
status [workflow-id] [[workflow-id]...]
  Check the status of one or more workflows.

metadata [workflow-id] [[workflow-id]...]
  Get the full metadata for one or more workflows.

slim-metadata [workflow-id] [[workflow-id]...]
  Get a subset of metadata for one or more workflows.

execution-status-count, counts [-p] [-x] [workflow-id] [[workflow-id]...]
  Get summarized execution status of jobs within workflows.
  -p: Pretty print summary instead of JSON.
  -x: Expand sub-workflows for detailed summarization.

timing [workflow-id] [[workflow-id]...]
  Open the workflow timing diagram in a browser.
```

--------------------------------

### HelloWorld Workflow Analysis Data

Source: https://github.com/broadinstitute/cromshell/blob/main/tests/unit/mock_data/list_outputs/helloworld_workflow_level_outputs.txt

This JSON object represents the analysis results for the HelloWorld workflow. It contains key-value pairs detailing BAM file properties and their associated GCS paths. This data is crucial for understanding the scale and location of data used in the workflow.

```json
{
    "HelloWorld.analysis_ready_bam_size": 0,
    "HelloWorld.analysis_ready_bam_size_in_gb": 132,
    "HelloWorld.analysis_ready_bam_name": "NA12878.hg38.bam",
    "HelloWorld.HelloWorldTask.bam_subset_file_index": "gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bai",
	"HelloWorld.HelloWorldTask.bam_subset_file": "gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.chr2.bam",
	"HelloWorld.HelloWorldTask.timing_info": "gs://broad-dsp-lrma-cromwell-central/HelloWorld/9ee4aa2e-7ac5-4c61-88b2-88a4d10f168b/call-HelloWorldTask/shard-1/m64020_190210_035026.subreads.ccs.uncorrected.aligned.merged.timingInformation.txt"
}
```

--------------------------------

### Importing Cromshell Utility Modules

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/addcommand.md

Demonstrates how to import various utility modules from the cromshell.utilities package into a Python script. This allows developers to leverage pre-built functionalities for tasks such as handling HTTP requests, I/O operations, and workflow ID management.

```python
from cromshell.utilities import http_utils, io_utils, workflow_id_utils
```

--------------------------------

### Testing Workflow Status with Mock Data

Source: https://github.com/broadinstitute/cromshell/blob/main/developer_docs/mockdata.md

Demonstrates how to use a pytest fixture to access mock workflow metadata and test the `workflow_failed` function from the `cromshell.status.command` module. It shows how to load JSON metadata and assert the function's output.

```python
import json
import os

import pytest

from cromshell.status import command as status_command


class TestStatus:
    """Test the status command functions"""

    def test_workflow_failed_for_metadata_that_is_doomed(self, mock_data_path):
        workflow_metadata_path = os.path.join(
            mock_data_path, "doom_workflow_slim_metadata.json"
        )
        with open(workflow_metadata_path, "r") as f:
            workflow_metadata = json.load(f)

        assert status_command.workflow_failed(workflow_metadata) is True, (
            "A running doomed workflow metadata should have "
            "output 'True' to indicate workflow "
            "has failed."
        )

    @pytest.fixture
    def mock_data_path(self):
        return os.path.join(os.path.dirname(__file__), "mock_data/")
```

--------------------------------

### WDL Validation

Source: https://github.com/broadinstitute/cromshell/blob/main/README.md

Command to validate WDL files using miniwdl and womtool.

```APIDOC
validate [wdl] [input json] --dependencies-zip [wdl_zip_file]
  Validate a WDL file. Runs both miniwdl and womtool validation by default.
  Womtool validation via Cromwell server API does not support validation of imported files, however miniwdl does.
```

--------------------------------

### Cromshell Function Definition

Source: https://github.com/broadinstitute/cromshell/blob/main/legacy_cromshell/README.md

Shows the standard method for defining functions in cromshell, using the 'function' keyword.

```shell
function doThing()
```