### Verify Cromwell-tools Installation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Verifies the installation of the cromwell-tools CLI by checking its version. This command should be run after installation to ensure the tools are accessible.

```bash
cromwell-tools --version
```

--------------------------------

### Submit Workflow with Short Optional Arguments

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Demonstrates the usage of short optional arguments for the 'submit' command, which are equivalent to their long-form counterparts. This example uses short arguments for service account key, collection name, WDL, inputs, and dependencies.

```bash
cromwell-tools submit \
--url "https://cromwell.caas-prod.broadinstitute.org" \
--service-account-key "/path/to/your/service-account-json-key.json" \
-c "your-collection" \
-w "Examples/hello_world.wdl" \
-i "Examples/inputs.json" \
-d "Examples/helloworld.wdl"
```

--------------------------------

### Cromwell Query Dictionary Example

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

An example of a complex query dictionary used for filtering workflows in Cromwell, including labels, status, submission times, and names.

```json
{
    'label': {
        'cromwell-workflow-id': 'cromwell-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
        'project_shortname': 'Name of a project that trigerred this workflow'
    },
    'status': ['Running', 'Succeeded', 'Aborted', 'Submitted', 'On Hold', 'Failed'],
    'id': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
    'additionalQueryResultFields': 'labels',
    'submission': '2018-01-01T00:01:01.410150Z',
    'start': '2018-01-01T01:01:01.410150Z',
    'end': '2018-01-01T02:01:01.410150Z',
    'name': ['SmartSeq2SingleCell', '10xCount'],
    'additionalQueryResultFields': ['labels'],
    'includeSubworkflows': True
}
```

--------------------------------

### Submit Job to Cromwell

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Submits a workflow to Cromwell/Cromwell-as-a-Service with specified WDL file, input files, and dependencies.

```python
from cromwell_tools import api
from cromwell_tools import utilities

response = api.submit(auth=auth, 
                      wdl_file='Examples/hello_world.wdl', 
                      inputs_files=['Examples/inputs.json'], 
                      dependencies=['Examples/helloworld.wdl'])
```

```python
response.json()
```

--------------------------------

### Publish Cromwell-Tools to PyPI

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

A step-by-step guide to publishing a new version of Cromwell-Tools to the Python Package Index (PyPI). This involves ensuring a clean 'dist' folder, installing 'twine', building the package, and uploading it.

```bash
pip install twine
```

```bash
python setup.py sdist bdist_wheel
```

```bash
twine upload dist/* --verbose
```

--------------------------------

### Set Up Pre-commit for Code Formatting

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Steps to install and configure pre-commit for automatic code formatting with Black, ensuring adherence to PEP-8 standards. This includes installing pre-commit and installing the git hook.

```bash
pip install pre-commit
```

```bash
pre-commit install
```

--------------------------------

### Display Cromwell-tools Help Information

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Displays all available commands and general help information for the cromwell-tools CLI. This is useful for discovering the full range of functionalities.

```bash
cromwell-tools --help
```

--------------------------------

### Query Workflows in Cromwell

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Queries Cromwell/Cromwell-as-a-Service for workflows based on a provided query dictionary.

```python
from cromwell_tools import api
from cromwell_tools import utilities

custom_query_dict = {
    'name': 'HelloWorld',
    'id': response.json()['id']
}

response = api.query(query_dict=custom_query_dict, auth=auth)
```

```python
response.json()['results']
```

--------------------------------

### Build and Preview Documentation Locally

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Instructions to install Sphinx and its dependencies, then build the project's HTML documentation locally. It also provides the path to preview the built documentation.

```bash
pip install requirements-docs.txt
```

```bash
sphinx-build -b html docs/ docs/_build/
```

--------------------------------

### Display Help for 'submit' Command

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Retrieves detailed help information specifically for the 'submit' command, including its arguments and usage. This helps in understanding how to submit workflows.

```bash
cromwell-tools submit --help
```

--------------------------------

### No Authentication

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Connects to Cromwell without any authentication.

```python
from cromwell_tools.cromwell_auth import CromwellAuth

auth_no_auth = CromwellAuth.from_no_authentication(url='https://cromwell.caas-prod.broadinstitute.org')
```

--------------------------------

### Submit Cromwell Job

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Submits a job to Cromwell with specified WDL file, inputs, dependencies, and an option to place it on hold. Requires authentication and file paths.

```python
from cromwell_tools import api
from cromwell_tools import utilities

response = api.submit(auth=auth, 
                      wdl_file='Examples/hello_world.wdl', 
                      inputs_files=['Examples/inputs.json'], 
                      dependencies=['Examples/helloworld.wdl'], 
                      on_hold=True)
```

--------------------------------

### HTTPBasicAuth Authentication

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Authenticates with Cromwell using username and password via HTTPBasicAuth.

```python
from cromwell_tools.cromwell_auth import CromwellAuth

auth_httpbasic = CromwellAuth.from_user_password(username='username', 
                                                 password='password', 
                                                 url='https://cromwell.xxx.broadinstitute.org')
```

--------------------------------

### Verify cromwell-tools Installation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Checks if the cromwell-tools package has been installed correctly by displaying its version.

```bash
cromwell-tools --version
```

--------------------------------

### Submit Workflow with Local Files

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Submits a Cromwell workflow using local paths for WDL, inputs, dependencies, labels, and options files. Requires specifying the Cromwell server URL, service account key, and collection name.

```bash
cromwell-tools submit \
--url "https://cromwell.caas-prod.broadinstitute.org" \
--service-account-key "/path/to/your/service-account-json-key.json" \
--collection-name "your-collection" \
--wdl "Examples/hello_world.wdl" \
--inputs-files "Examples/inputs.json" \
--deps-file "Examples/helloworld.wdl" \
--label-file "Examples/options.json" \
--options-file "Examples/labels.json"
```

--------------------------------

### OAuth Authentication

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Authenticates with Cromwell using a service account JSON key file for OAuth.

```python
from cromwell_tools.cromwell_auth import CromwellAuth

auth_oauth = CromwellAuth.from_service_account_key_file(service_account_key='path/to/service/account/key.json', 
                                                        url='https://cromwell.caas-prod.broadinstitute.org')
```

--------------------------------

### Install and Upgrade cromwell-tools

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Installs or upgrades the cromwell-tools package from PyPI using pip. It is recommended to use a Python 3 virtual environment.

```bash
pip install -U cromwell-tools
```

--------------------------------

### Get Cromwell Job Status

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Retrieves the status of a Cromwell job using its unique identifier (UUID). Requires authentication and the job's UUID.

```python
response = api.status(auth=auth, 
                      uuid=response.json()['id'])
response.json()
```

--------------------------------

### Run Cromwell-tools Tests with Docker

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Provides instructions to run the tests for the cromwell-tools package using a Docker image, assuming docker-daemon is installed.

```bash
cd cromwell_tools/tests && bash test.sh
```

--------------------------------

### Run Cromwell-Tools Tests Locally

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Instructions to set up a virtual environment, install dependencies, and run tests for Cromwell-Tools using pytest. It also mentions the flexibility to choose the Python version for the virtual environment.

```bash
virtualenv test-env
source test-env/bin/activate
pip install -r requirements.txt -r requirements-test.txt
```

```bash
python -m pytest --cov=cromwell_tools cromwell_tools/tests
```

--------------------------------

### Standard Authentication Methods

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Demonstrates various ways to authenticate with Cromwell using the CromwellAuth class, including username/password, secrets file, service account key, and no authentication.

```python
from cromwell_tools.cromwell_auth import CromwellAuth

# Authenticate with Cromwell using HTTPBasicAuth (username and password)
auth = CromwellAuth.harmonize_credentials(username='username', 
                                          password='password', 
                                          url='https://cromwell.xxx.broadinstitute.org')

# Authenticate with Cromwell using HTTPBasicAuth (secret JSON file)
auth_2 = CromwellAuth.harmonize_credentials(secrets_file='path/to/secrets_file.json')

# Authenticate with Cromwell using OAuth (service account JSON key file)
auth_3 = CromwellAuth.harmonize_credentials(service_account_key='path/to/service/account/key.json', 
                                            url='https://cromwell.caas-prod.broadinstitute.org')

# Authenticate with Cromwell with no Auth
auth_4 = CromwellAuth.harmonize_credentials(url='https://cromwell.caas-prod.broadinstitute.org')
```

--------------------------------

### Submit Workflow with URL File Paths

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Submits a Cromwell workflow using HTTP/HTTPS URLs for WDL, inputs, and dependency files. Cromwell Tools will download and compose these files. Requires specifying the Cromwell server URL, service account key, and collection name.

```bash
cromwell-tools submit \
--url "https://cromwell.caas-prod.broadinstitute.org" \
--service-account-key "/path/to/your/service-account-json-key.json" \
--collection-name "your-collection" \
--wdl "https://raw.githubusercontent.com/broadinstitute/cromwell-tools/v2.0.0/notebooks/Quickstart/Examples/hello_world.wdl" \
--inputs-files "https://raw.githubusercontent.com/broadinstitute/cromwell-tools/v2.0.0/notebooks/Quickstart/Examples/inputs.json" \
--deps-file "https://raw.githubusercontent.com/broadinstitute/cromwell-tools/v2.0.0/notebooks/Quickstart/Examples/helloworld.wdl"
```

--------------------------------

### Submit Workflow with No Authentication

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Submits a workflow to a Cromwell instance that does not have an authentication layer. This is achieved by omitting all authentication-related arguments and only providing the Cromwell URL.

```bash
cromwell-tools submit \
--url "https://cromwell.xxx.broadinstitute.org" \
--wdl "Examples/hello_world.wdl" \
--inputs-files "Examples/inputs.json" \
--deps-file "Examples/helloworld.wdl"
```

--------------------------------

### Submit Workflow with HTTPBasicAuth (Username/Password)

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Submits a workflow to a Cromwell instance using HTTPBasicAuth with provided username and password. Requires the Cromwell URL, username, password, WDL file path, inputs file path, and dependencies file path.

```bash
cromwell-tools submit \
--url "https://cromwell.xxx.broadinstitute.org" \
--username "xxx" \
--password "xxx" \
--wdl "Examples/hello_world.wdl" \
--inputs-files "Examples/inputs.json" \
--deps-file "Examples/helloworld.wdl"
```

--------------------------------

### Submit Job with On Hold Status

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Demonstrates submitting a job with an 'On Hold' status, checking its status, and then releasing it.

```python
# Submit a job with "On Hold" status.
# Check the status.
# Release the job.
```

--------------------------------

### Secrets File Authentication

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Authenticates with Cromwell using credentials stored in a JSON secrets file.

```python
from cromwell_tools.cromwell_auth import CromwellAuth

auth_httpbasic_file = CromwellAuth.from_secrets_file(secrets_file='path/to/secrets_file.json')
```

--------------------------------

### Secrets File Format

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Illustrates the expected JSON format for a secrets file used for Cromwell authentication, containing URL, username, and password.

```json
{
    "url": "url",
    "username": "username",
    "password": "password"
}
```

--------------------------------

### Wait for Job Completion

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Waits for a specified workflow ID to complete with a given polling interval.

```python
api.wait(workflow_ids=[response.json()['id']], auth=auth, poll_interval_seconds=3)
```

--------------------------------

### Submit Workflow with OAuth (Service Account Key)

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Submits a workflow to an OAuth-enabled Cromwell instance using a Google Cloud service account JSON key file. This is common for Cromwell-as-a-Service (CaaS) and requires specifying a collection name for SAM permissions.

```bash
cromwell-tools submit \
--url "https://cromwell.caas-prod.broadinstitute.org" \
--service-account-key "/path/to/your/service-account-json-key.json" \
--collection-name "your-collection" \
--wdl "Examples/hello_world.wdl" \
--inputs-files "Examples/inputs.json" \
--deps-file "Examples/helloworld.wdl"
```

--------------------------------

### Submit Workflow with HTTPBasicAuth (Secrets File)

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Submits a workflow to Cromwell using HTTPBasicAuth credentials stored in a JSON secrets file. This method simplifies authentication by avoiding repeated credential input. The secrets file should contain 'url', 'username', and 'password'.

```bash
cromwell-tools submit \
--secrets-file "secrets.json" \
--wdl "Examples/hello_world.wdl" \
--inputs-files "Examples/inputs.json" \
--deps-file "Examples/helloworld.wdl"
```

--------------------------------

### Release Cromwell Job Hold

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/api_quickstart.ipynb

Releases a Cromwell job that was previously placed on hold. Requires authentication and the job's UUID.

```python
response = api.release_hold(auth=auth, 
                            uuid=response.json()['id'])
response.json()
```

--------------------------------

### Wait for Workflow Completion

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Monitors and polls the status of one or more Cromwell workflows using their IDs. Allows configuration of polling interval and timeout. If any workflow fails, the entire polling process terminates.

```bash
cromwell-tools wait \
--url "https://cromwell.caas-prod.broadinstitute.org" \
--service-account-key "/path/to/your/service-account-json-key.json" \
--poll-interval-seconds 10 \
a42e54ba-84e4-4305-be90-feeae33b7fc4 \
13f7577a-c496-4770-8f3e-0f085f4edac0 \
d9d8dc18-d462-46f6-a39d-b68b20dfb5ab
```

--------------------------------

### Secrets File Format for HTTPBasicAuth

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/Tutorials/Quickstart/cli_quickstart.ipynb

Specifies the JSON format for a secrets file used to store Cromwell authentication credentials (URL, username, and password). This file can be passed to the 'cromwell-tools submit' command using the '--secrets-file' argument.

```json
{
    "url": "url",
    "username": "username",
    "password": "password"
}
```

--------------------------------

### Configure Cromwell and Google Cloud Connection

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Sets up configuration for connecting to a local Cromwell server and a Google Cloud project. It initializes a Cromwell client and a Google Cloud Storage client, and checks if the Cromwell server is running.

```python
google_project = 'your_project'
cromwell_url = 'http://localhost:port_number'

local_config = {'cromwell_url': cromwell_url}
cromwell = cwm.Cromwell(**local_config)
cromwell.server_is_running()

client = storage.Client(project=google_project)
```

--------------------------------

### Cromwell-tools Command Line Interface Usage

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Shows the basic usage of the cromwell-tools CLI, including available sub-commands like submit, wait, status, abort, release_hold, query, and health.

```bash
$\!> cromwell-tools -h
usage: cromwell-tools [-h]
                  {submit,wait,status,abort,release_hold,query,health}
                  ...

positional arguments:
  {submit,wait,status,abort,release_hold,query,health}
                        sub-command help
    submit              submit help
    wait                wait help
    status              status help
    abort               abort help
    release_hold        release_hold help
    query               query help
    health              health help

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit
```

--------------------------------

### Load IPython Extensions and Libraries

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Loads necessary IPython extensions for automatic reloading and inline plotting, and imports essential Python libraries for interacting with Cromwell, Google Cloud Storage, data manipulation, and plotting.

```python
%load_ext autoreload
%autoreload 2
%matplotlib inline

import matplotlib.pyplot as plt
import cromwell_manager as cwm
from google.cloud import storage
import glob
from collections import defaultdict
from scsequtil.plot import grid
from operator import itemgetter
from sklearn.linear_model import LinearRegression
import numpy as np
import pandas as pd
```

--------------------------------

### Import and Use Cromwell-tools Python API

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Demonstrates how to import the cromwell_tools.api module in Python and use its functions, such as 'submit'.

```python
import cromwell_tools.api as cwt
cwt.submit(*args)
```

--------------------------------

### Manually Trigger Linters and Formatters

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/README.rst

Commands to manually run flake8 for linting and Black for code formatting on specified directories.

```bash
flake8 DIR1 DIR2
```

```bash
black DIR1 DIR2 --skip-string-normalization
```

--------------------------------

### Submit and Monitor Workflows

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Submits multiple WDL workflows with varying input sizes to Cromwell and monitors their completion status. It uses predefined WDL and input JSON files from GitHub repositories.

```python
input_files = [
    'https://raw.githubusercontent.com/ambrosejcarr/cromwell-manager/master/src/cromwell_manager/test/data/10x_count_inputs_1e4.json',
    'https://raw.githubusercontent.com/ambrosejcarr/cromwell-manager/master/src/cromwell_manager/test/data/10x_count_inputs_1e5.json',
    'https://raw.githubusercontent.com/ambrosejcarr/cromwell-manager/master/src/cromwell_manager/test/data/10x_count_inputs_1e6.json',
    'https://raw.githubusercontent.com/ambrosejcarr/cromwell-manager/master/src/cromwell_manager/test/data/10x_count_inputs_1e7.json'
]
wdl = 'https://raw.githubusercontent.com/HumanCellAtlas/skylab/master/10x/count/count.wdl'
options = 'https://raw.githubusercontent.com/ambrosejcarr/cromwell-manager/master/src/accessories/options.json'
memory_test_workflows = []
for i in input_files:
    memory_test_workflows.append(cwm.Workflow.from_submission(wdl, i, cromwell, client, options_json=options))
    print(memory_test_workflows[-1].status)

for workflow in memory_test_workflows:
    workflow.wait_until_complete()

for w in memory_test_workflows:
    print(w.status)
```

--------------------------------

### Cromwell-tools API Documentation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/index.rst

Provides comprehensive API documentation for Cromwell-tools, detailing available endpoints, methods, and parameters for interacting with Cromwell workflows.

```APIDOC
API/index:
  Description: Main entry point for Cromwell-tools API documentation.
  Content:
    - Tutorials/Quickstart/api_quickstart: Guide to getting started with the API.
    - API/index: Detailed API reference.
```

--------------------------------

### Cromwell API Documentation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/API/index.rst

Documentation for the cromwell_api module, detailing its public API and inherited members. This module likely handles interactions with the Cromwell API.

```APIDOC
cromwell_tools.cromwell_api:
  Exposes the public API for interacting with Cromwell.
  Includes methods for job submission, status retrieval, and workflow management.
  Inherits members from base classes, providing a rich set of functionalities.
```

--------------------------------

### Cromwell Utilities Documentation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/API/index.rst

Documentation for the utilities module, detailing its public API and inherited members. This module likely contains helper functions and utility classes for Cromwell-tools.

```APIDOC
cromwell_tools.utilities:
  Contains utility functions and classes for Cromwell-tools.
  May include helpers for data manipulation, file processing, or common tasks.
  Provides reusable components to streamline development.
```

--------------------------------

### Cromwell CLI Documentation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/API/index.rst

Documentation for the cli module, covering its public API and inherited members. This module likely defines the command-line interface for Cromwell-tools.

```APIDOC
cromwell_tools.cli:
  Defines the command-line interface for Cromwell-tools.
  Exposes commands for interacting with Cromwell workflows and jobs.
  Includes argument parsing and command execution logic.
```

--------------------------------

### Cromwell Authentication Documentation

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/docs/API/index.rst

Documentation for the cromwell_auth module, covering its public API and inherited members. This module is responsible for authentication mechanisms used by Cromwell-tools.

```APIDOC
cromwell_tools.cromwell_auth:
  Handles authentication for Cromwell-tools.
  Provides methods for generating and managing authentication credentials.
  Supports various authentication schemes and integrates with Cromwell's security features.
```

--------------------------------

### Calculate Input Size to Attribute Relationship

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Defines a function to calculate the linear relationship between input data size and a specified resource attribute (e.g., max_memory). It performs logarithmic transformations and linear regression to determine the intercept and slope, representing baseline usage and usage per input size.

```python
# calculate the memory usage equation (fit linear function)
def calculate_input_size_to_attribute_relationship(attribute, aggregated_tasks, name=None, *args, **kwargs):
    task_data = {}
    for task_name, results in sorted(aggregated_tasks.items(), key=itemgetter(0)):
        sizes, results = zip(*sorted(results, key=itemgetter(0)))
        usage = [getattr(r, attribute) for r in results]

        # regression inputs
        sizes = np.log(np.array(list(sizes))[:, None])
        usage = np.log(np.array(usage)[:, None])
        
        # run the regression
        lr = LinearRegression(*args, **kwargs)
        lr.fit(sizes, usage)
        b = np.ravel(np.exp(lr.intercept_))[0]  # intercept provides basic memory
        m = np.ravel(lr.coef_)[0]  # first coefficient provides slope of line
        task_data[task_name] = '{b:.2f} + {m:.2f}*x'.format(b=b, m=m)
    result = pd.Series(task_data)
    result.name = name
    return result
```

--------------------------------

### Calculate and Display Disk Usage Relationship

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Calculates and visualizes the linear relationship between input data size and disk usage for Cromwell tasks. It allows fitting the equation with or without an intercept, useful for understanding baseline disk utilization. The function highlights tasks with the highest disk usage.

```python
# Assuming 'calculate_input_size_to_attribute_relationship' is defined elsewhere
# and 'aggregated' is available.
# Example placeholder for calculate_input_size_to_attribute_relationship:
# def calculate_input_size_to_attribute_relationship(attribute, aggregated_tasks, name, fit_intercept=True):
#     print(f"Calculating relationship for {name} with fit_intercept={fit_intercept}")
#     # Placeholder for actual calculation and plotting logic
#     pass

# Calculate the linear equation for disk usage without intercept
# calculate_input_size_to_attribute_relationship(
#     attribute='max_disk',
#     aggregated_tasks=aggregated,
#     name='10x count disk usage',
#     fit_intercept=False
# )

# Display disk usage plots
# ag = plot_attribute('max_disk')
# plt.savefig('disk_utilization.png', dpi=150)

```

--------------------------------

### Retrieve and Display Task Resource Utilization

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Retrieves the resource utilization (memory, disk) for tasks within submitted workflows and displays the tasks for the first workflow. It then aggregates task data across all workflows.

```python
# get memory utilization
for t in memory_test_workflows:
    t.tasks(retrieve=True)

# display the run tasks
memory_test_workflows[0].tasks(retrieve=False)

aggregated = defaultdict(list)
for size, w in zip([1e4, 1e5, 1e6, 1e7], memory_test_workflows):
    for task_name, task in w.tasks(retrieve=False).items():
        aggregated[task_name].append([size, task.resource_utilization])
```

--------------------------------

### Analyze Memory Usage

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Applies the `calculate_input_size_to_attribute_relationship` function to analyze the 'max_memory' attribute of the aggregated task data, generating a series representing the linear relationship between input size and memory usage for the '10x count' process.

```python
calculate_input_size_to_attribute_relationship(
    attribute='max_memory', aggregated_tasks=aggregated, name='10x count memory usage')
```

--------------------------------

### Project Dependencies

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/requirements.txt

Specifies the Python packages and version requirements for the Cromwell Tools project. These dependencies ensure compatibility and proper functioning of the tools.

```python
requests>=2.20.0,<3
six>=1.11.0
google-auth>=1.6.1,<2
setuptools_scm>=3.1.0,<4
python-dateutil==2.8.0
google-api-python-client==1.7.11
```

--------------------------------

### Plot Task Attribute

Source: https://github.com/broadinstitute/cromwell-tools/blob/master/notebooks/estimate_resource_usage.ipynb

Generates scatter plots of a specified attribute (e.g., memory, disk) against input sizes for aggregated Cromwell task results. It uses matplotlib's AxesGrid for organized plotting and allows customization of plot appearance.

```python
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import AxesGrid
from operator import itemgetter

# Assuming 'aggregated' and 'grid' are defined elsewhere
# Example placeholder for aggregated and grid:
# aggregated = {'task1': [(10, {'max_memory': 100, 'max_disk': 50}), (20, {'max_memory': 150, 'max_disk': 70})], 'task2': [(15, {'max_memory': 120, 'max_disk': 60}), (30, {'max_memory': 200, 'max_disk': 90})]}
# class MockAxesGrid:
#     def __init__(self, nplots, figsize, sharex, sharey):
#         pass
#     def plot_all(self, args, plot_function):
#         for x, y, dependent_var, ax in args:
#             plot_function(x, y, dependent_var, ax)
# grid = MockAxesGrid

def plot_attribute(attribute):
    nplots = len(aggregated)
    # make some scatter plots with the amounts vs input sizes
    ag = grid.AxesGrid(nplots, figsize=(20, 20), sharex=True, sharey=True)


    def plot_function(x, y, dependent_var, ax):
        ax.loglog(
            x, y, c='royalblue', marker='o', markersize=10,
            markerfacecolor='indianred')
        ax.set_xlabel('input_size')
        ax.set_ylabel(dependent_var)

    # build argument groups
    args = []
    for task_name, results in sorted(aggregated.items(), key=itemgetter(0)):
        sizes, results = zip(*sorted(results, key=itemgetter(0)))
        usage = [getattr(r, attribute) for r in results]
        args.append((sizes, usage, task_name))

    ag.plot_all(args, plot_function)
    plt.tight_layout()
    return ag

# Example usage:
# ag = plot_attribute('max_memory')
# plt.savefig('memory_utilization.png', dpi=150)  # save the result

```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.