Nextflow

https://docs.seqera.io/nextflow
Admin
Nextflow is a workflow system for creating scalable, portable, and reproducible workflows using a...
Tokens:390,147
Snippets:4,136
Trust Score:9.7
Update:2 weeks ago
Show doc for...
Tokens:
Raw
### Install multiple plugins

Source: https://docs.seqera.io/nextflow/reference/cli/plugin

Installs multiple plugins as a comma-separated list.

```bash
$ nextflow plugin install my-plugin@0.5.0,other-plugin  

```

--------------------------------

### Install a plugin

Source: https://docs.seqera.io/nextflow/reference/cli/plugin

Installs a plugin by its name.

```bash
$ nextflow plugin install my-plugin  

```

--------------------------------

### Install Java with SDKMAN

Source: https://docs.seqera.io/nextflow/install

Install Java via SDKMAN: install SDKMAN, then install Java 17.0.10-tem, and confirm the installation.

```bash
curl -s https://get.sdkman.io | bash  

```

```bash
sdk install java 17.0.10-tem  

```

```bash
java -version  

```

--------------------------------

### Install a registry module with nextflow module install

Source: https://docs.seqera.io/nextflow/modules

Installs the nf-core/fastqc module from the Nextflow registry into the current project. This command is used to manage registry modules.

```bash
$ nextflow module install nf-core/fastqc  

```

--------------------------------

### Install a module with nextflow module install

Source: https://docs.seqera.io/nextflow/migrations/26-04

Use the `nextflow module install` command to install a remote module into the `modules` directory of your project. The example installs the nf-core/bwa/mem module.

```bash
nextflow module install nf-core/bwa/mem  
```

--------------------------------

### List installed modules

Source: https://docs.seqera.io/nextflow/reference/cli/module

Displays the installed modules as a table.

```bash
$ nextflow module list  
```

--------------------------------

### Example output of nextflow -version

Source: https://docs.seqera.io/nextflow/cli

Example output from nextflow -version showing detailed version information.

```bash
N E X T F L O W  
version 24.04.0 build 5917  
created 03-05-2024 15:07 UTC  
cite doi:10.1038/nbt.3820  
http://nextflow.io  

```

--------------------------------

### Example output of nextflow -v

Source: https://docs.seqera.io/nextflow/cli

Example output from nextflow -v showing version and build number.

```bash
nextflow version 24.04.0.5917  

```

--------------------------------

### Install a specific plugin version

Source: https://docs.seqera.io/nextflow/reference/cli/plugin

Installs a specific plugin version using the `@` syntax.

```bash
$ nextflow plugin install my-plugin@0.5.0  

```

--------------------------------

### Authenticate with an Enterprise installation

Source: https://docs.seqera.io/nextflow/reference/cli/auth

Logs in to a Seqera Enterprise installation by specifying the API endpoint with `-u`.

```bash
$ nextflow auth login -u https://tower.example.com/api  
```

--------------------------------

### Output of val input example

Source: https://docs.seqera.io/nextflow/process

Example output from the `val` input example. The order of the lines is not guaranteed because processes do not necessarily process items in the order they are received.

```text
process job 3  
process job 1  
process job 2  

```

--------------------------------

### Output of channel.value view example

Source: https://docs.seqera.io/nextflow/workflow

Expected output of the previous `channel.value(1).view` example.

```text
dataflow value is 1  

```

--------------------------------

### Output of COLLECT_FILE example

Source: https://docs.seqera.io/nextflow/tutorials/static-types-operators

Expected output of the `COLLECT_FILE` example, showing the contents of the generated file.

```text
alpha  
beta  
gamma  

```

--------------------------------

### Start interactive Flux instance

Source: https://docs.seqera.io/nextflow/tutorials/flux

Starts an interactive Flux instance with a test size of 4, from which you can submit jobs on the command line to test with Nextflow.

```bash
$ flux start --test-size=4  

```

--------------------------------

### Install latest module version

Source: https://docs.seqera.io/nextflow/reference/cli/module

Installs the latest version of a module from the Nextflow registry.

```bash
$ nextflow module install nf-core/fastqc  
```

--------------------------------

### Example output showing tagged tasks

Source: https://docs.seqera.io/nextflow/reference/process/directives/tag

This output shows the execution log entries for the example process, each tagged with the corresponding value from the channel. The tag appears in parentheses after the process name.

```text
[6e/28919b] Submitted process > hello (alpha)  
[d2/1c6175] Submitted process > hello (gamma)  
[1c/3ef220] Submitted process > hello (omega)  

```

--------------------------------

### Example fastqc.csv index file output

Source: https://docs.seqera.io/nextflow/tutorials/workflow-outputs

Example output of the generated fastqc.csv index file. It lists each sample id and the corresponding published file path, mirroring the structure of the published channel.

```bash
$ cat results/fastqc.csv  
id,fastqc  
lung,results/fastqc/lung  
spleen,results/fastqc/spleen  
gut,results/fastqc/gut  
liver,results/fastqc/liver  

```

--------------------------------

### Equivalent workflow declaration

Source: https://docs.seqera.io/nextflow/reference/syntax

This example shows the equivalent workflow declaration for the previous script-as-entry-workflow example. It wraps the statement in a workflow block.

```nextflow
workflow {  
    println 'Hello world!'  
}  
```

--------------------------------

### Output of channel.of view example

Source: https://docs.seqera.io/nextflow/workflow

Expected output of the previous `channel.of(1, 2, 3).view` example.

```text
channel emits 1  
channel emits 2  
channel emits 3  

```

--------------------------------

### Install Docker Desktop on Linux

Source: https://docs.seqera.io/nextflow/developer-env

Install Docker Desktop on Linux Debian/Ubuntu distributions using the downloaded .deb package. Requires Docker's package repository to be set up first.

```bash
sudo apt-get install ./docker-desktop-amd64.deb
```

--------------------------------

### combine(other) example

Source: https://docs.seqera.io/nextflow/reference/stdlib-types/channel

Emits every pairwise combination of the channel with another channel or dataflow value. The example shows the output tuples.

```groovy
numbers = channel.of(1, 2, 3)  
words = channel.of('hello', 'ciao')  
  
numbers  
    .combine(words)  
    .view()  

```

```text
[1, hello]  
[2, hello]  
[3, hello]  
[1, ciao]  
[2, ciao]  
[3, ciao]  

```

--------------------------------

### Filter example with nextflow log -filter

Source: https://docs.seqera.io/nextflow/reports

Example filter that matches task names starting with 'hello' and status 'FAILED'.

```bash
nextflow log goofy_kilby -filter 'name =~ /hello.*/ && status == "FAILED"'  

```

--------------------------------

### FastQC process in main.nf

Source: https://docs.seqera.io/nextflow/modules/developing-modules

Example main.nf process definition wrapping FastQC. Registry modules must contain a single main.nf and define a single process or named workflow.

```groovy
process FASTQC {  
    tag "$meta.id"  
    label 'process_medium'  
  
    conda 'bioconda::fastqc=0.12.1'  
    container 'biocontainers/fastqc:0.12.1--hdfd78af_0'  
  
    input:  
    tuple val(meta), path(reads)  
  
    output:  
    tuple val(meta), path("*.html"), emit: html  
    tuple val(meta), path("*.zip") , emit: zip  
  
    script:  
    """  
    fastqc $reads --threads $task.cpus  
    """  
}  

```

--------------------------------

### Block comment example

Source: https://docs.seqera.io/nextflow/reference/syntax

A block comment starts with /* and includes all subsequent characters up to the first */. The example shows a block comment followed by a println statement.

```nextflow
/*  
 * block comment  
 */  
println 'Hello again!'  
```

--------------------------------

### Show all profiles with -show-profiles

Source: https://docs.seqera.io/nextflow/reference/cli/config

Shows all configuration profiles defined in the project's configuration. The output includes the standard and cloud profiles with their process settings.

```bash
$ nextflow config -show-profiles  
  
docker {  
    enabled = true  
}  
  
profiles {  
    standard {  
        process {  
            executor = 'local'  
        }  
    }  
    cloud {  
        process {  
            executor = 'cirrus'  
            container = 'cbcrg/imagex'  
        }  
    }  
}  

```

--------------------------------

### Line comment example

Source: https://docs.seqera.io/nextflow/reference/syntax

A line comment starts with // and includes the rest of the line. The example shows a println statement followed by a line comment.

```nextflow
println 'Hello world!' // line comment  
```

--------------------------------

### Create a module

Source: https://docs.seqera.io/nextflow/reference/cli/module

Starts the interactive creation of a new module.

```bash
$ nextflow module create  
```

--------------------------------

### Launch a pipeline in Seqera Platform

Source: https://docs.seqera.io/nextflow/reference/cli/launch

Launches the nextflow-io/hello pipeline in Seqera Platform. This is the basic example from the documentation.

```bash
$ nextflow launch nextflow-io/hello  
```

--------------------------------

### Access list items and get length

Source: https://docs.seqera.io/nextflow/script

Access an item in a list with square-bracket notation (indexes start at 0) and get the length with the `size` method.

```groovy
println myList[0]  
println myList.size()  
```

--------------------------------

### nextflow list example

Source: https://docs.seqera.io/nextflow/reference/cli/list

Lists the downloaded projects in the global cache ~/.nextflow/assets. The output shows the project names.

```bash
$ nextflow list  
  
nextflow-io/hello  
nextflow-hub/fastqc  

```

--------------------------------

### Negate selectors with '!' operator

Source: https://docs.seqera.io/nextflow/config

Uses the '!' operator to negate selector expressions. Processes labeled 'hello' get 2 CPUs, all others get 4 CPUs, and processes whose name does not start with 'align' get the 'long' queue.

```nextflow
process {  
    withLabel: 'hello' { cpus = 2 }  
    withLabel: '!hello' { cpus = 4 }  
    withName: '!align.*' { queue = 'long' }  
}  

```

--------------------------------

### Module structure with resources directory

Source: https://docs.seqera.io/nextflow/modules/developing-modules

Example module structure with resources directory. When running with Wave, resources/ are mounted into the root directory of the task container.

```text
my-module/  
├── main.nf  
└── resources/  
    ├── data/  
    |   └── file.txt  
    └── usr/  
        └── bin/  
            └── hello.sh  

```

--------------------------------

### Install Docker on EC2

Source: https://docs.seqera.io/nextflow/aws

Installs Docker on an Amazon EC2 instance, starts the Docker service, and adds the ec2-user to the docker group. You must log out and log back in for the new permissions to take effect. These steps must be done before creating the AMI.

```bash
# install Docker  
sudo yum update -y  
sudo yum install docker  
  
# start the Docker service  
sudo service docker start  
  
# empower your user to run Docker without sudo  
sudo usermod -a -G docker ec2-user  

```

--------------------------------

### Example template script hello.sh

Source: https://docs.seqera.io/nextflow/process

A Bash template script that echoes a start message, the value of $STR, and a completion message. $STR is a Nextflow variable when executed by Nextflow, and a Bash variable when executed directly.

```bash
#!/bin/bash  
echo "process started at `date`"  
echo $STR  
echo "process completed"  

```

--------------------------------

### Module structure with templates directory

Source: https://docs.seqera.io/nextflow/modules/developing-modules

Example module structure with templates directory.

```text
my-module/  
|── main.nf  
└── templates/  
    └── hello.sh  

```

--------------------------------

### Create an interval channel with channel.interval

Source: https://docs.seqera.io/nextflow/reference/stdlib-namespaces/channel

Creates a channel that emits an incrementing index starting from zero at a periodic interval. The example emits 0, 1, 2, and so on, every second, forever. Use an operator such as take or until to close the channel based on a stopping condition.

```groovy
channel.interval('1s').view()  
```

--------------------------------

### Create a channel with channel.of and view output

Source: https://docs.seqera.io/nextflow/reference/stdlib-namespaces/channel

Creates a channel that emits each argument. The example emits 1, 3, 5, and 7, and the output is shown.

```groovy
ch = channel.of( 1, 3, 5, 7 ).view()  
```

```text
1  
3  
5  
7  
```

--------------------------------

### Override provisioning mode with seqera/machineRequirement.provisioning hint

Source: https://docs.seqera.io/nextflow/executor/seqera

Use this hint to set the provisioning mode to 'spotFirst' for a single process. The hint is specified with the 'seqera/' prefix to restrict it to the Seqera executor.

```groovy
process hello {  
    hints 'seqera/machineRequirement.provisioning': 'spotFirst'  
  
    script:  
    """  
    your_command --here  
    """  
}  
```

--------------------------------

### Install Nextflow plugin with nextflow plugin install

Source: https://docs.seqera.io/nextflow/cli

Installs a plugin to extend Nextflow functionality.

```bash
nextflow plugin install my-plugin  

```

--------------------------------

### Enable WSL with wsl --install

Source: https://docs.seqera.io/nextflow/developer-env

Run in an administrator PowerShell or Windows Command Prompt to enable WSL and install the Ubuntu distribution. Restart Windows when prompted, then create a Linux user name and password in the Ubuntu distribution.

```powershell
wsl --install
```

--------------------------------

### Install Git on macOS with Homebrew

Source: https://docs.seqera.io/nextflow/developer-env

Install Git on macOS with Homebrew. Requires Homebrew to be installed.

```bash
brew install git
```

--------------------------------

### Run a specific integration test with qrun.sh <FOLDER>

Source: https://docs.seqera.io/nextflow/developer

Run a specific integration test by providing the folder name as an argument to `qrun.sh`.

```bash
cd tests/checks  
./qrun.sh <FOLDER>  

```

--------------------------------

### Run integration tests with qrun.sh

Source: https://docs.seqera.io/nextflow/developer

Run the integration tests by navigating to the `tests/checks` directory and executing `qrun.sh`.

```bash
cd tests/checks  
./qrun.sh  

```

--------------------------------

### nextflow module install

Source: https://docs.seqera.io/nextflow/reference/cli/module

Installs a module from the registry into the project.

```APIDOC
## nextflow module install

### Description
Install a module from the registry into your project. Nextflow stores downloaded modules in the `modules/` directory and writes a `.module-info` file in each module directory to record the installed version.

### Usage
`nextflow module install [options] [namespace/name]`

### Options
- `-force` (flag) - Optional - Reinstall the module even if it is already installed. Without this flag, Nextflow reuses an installed module and does not overwrite a locally modified one.
- `-update-deps` (flag) - Optional - Re-vendor an installed module's dependencies from its `meta.yml`, leaving the module itself untouched. Nextflow installs each declared dependency at its pinned version, transitively, and removes any of the module's own direct dependencies that are no longer declared. If the module is not installed, Nextflow installs it as usual. Cannot be combined with `-force`.
- `-version` (string) - Optional - Module version to install, for example `1.0.0`. Defaults to the latest version.

```

--------------------------------

### Output of native execution example

Source: https://docs.seqera.io/nextflow/process

Example output from the native execution example. The order of the printed lines is not guaranteed because processes do not necessarily process items in the order they are received.

```text
Hello Mr. b  
Hello Mr. a  
Hello Mr. c  

```

--------------------------------

### Install AWS CLI with Miniconda

Source: https://docs.seqera.io/nextflow/aws

Installs the AWS CLI using Miniconda in the home folder. Requires bzip2 and wget, and uses the conda-forge channel. The AWS CLI is installed in $HOME/miniconda/bin/aws.

```bash
cd $HOME  
sudo yum install -y bzip2 wget  
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh  
bash Miniconda3-latest-Linux-x86_64.sh -b -f -p $HOME/miniconda  
$HOME/miniconda/bin/conda install -c conda-forge --override-channels -y awscli  
rm Miniconda3-latest-Linux-x86_64.sh  

```

--------------------------------

### Install VS Code on older Linux distributions

Source: https://docs.seqera.io/nextflow/developer-env

Install VS Code on older Linux distributions using dpkg, then install dependencies with apt-get. Replace <file> with the full file name.

```bash
sudo dpkg -i <file>.deb
```

```bash
sudo apt-get install -f
```

--------------------------------

### Example: nextflow help drop

Source: https://docs.seqera.io/nextflow/reference/cli/help

Prints the help for the `drop` command, showing its usage and options, including the `-f` flag and the `-h, -help` option.

```bash
$ nextflow help drop  
  
Delete the local copy of a project  
Usage: drop [options] name of the project to drop  
   Options:  
     -f  
          Delete the repository without taking care of local changes  
          Default: false  
     -h, -help  
          Print the command usage  
          Default: false  

```

--------------------------------

### Get a secret with nextflow secrets get

Source: https://docs.seqera.io/nextflow/reference/cli/secrets

Retrieves the value of the secret named FOO.

```bash
$ nextflow secrets get FOO  

```

--------------------------------

### Run Nextflow with Charliecloud

Source: https://docs.seqera.io/nextflow/container/charliecloud

Enable Charliecloud from the command line using the -with-charliecloud flag.

```bash
nextflow run main.nf -with-charliecloud  

```

--------------------------------

### Install Git on Debian/Ubuntu

Source: https://docs.seqera.io/nextflow/developer-env

Install the latest stable Git version on Linux Debian/Ubuntu distributions.

```bash
sudo apt-get install git-all
```

--------------------------------

### Configure cloud cache and resume with session ID

Source: https://docs.seqera.io/nextflow/cache-and-resume

Configure the cloud cache by setting NXF_CLOUDCACHE_PATH and NXF_IGNORE_RESUME_HISTORY, then run the pipeline with a run name and resume with the session ID from the initial run.

```bash
# configure cloud cache  
export NXF_CLOUDCACHE_PATH=s3://my-bucket/cache  
export NXF_IGNORE_RESUME_HISTORY=true  
  
# initial run  
nextflow run <pipeline> -name my-run-1  
  
# resumed run (use session ID from initial run)  
nextflow run <pipeline> -name my-run-2 -resume <session-id>  

```

--------------------------------

### Build Nextflow from source: git clone, make compile, launch.sh, make pack

Source: https://docs.seqera.io/nextflow/developer

Clones a Nextflow branch, compiles it, runs a local build with launch.sh, or packages a self-contained executable for testing PRs. Requires Java 11 or later; Gradle is not needed. Building from source downloads dependencies on the first run, which may take several minutes, and make pack creates a binary in build/releases. launch.sh does not support third-party plugins.

```bash
git clone -b <branch> git@github.com:nextflow-io/nextflow.git  
cd nextflow  
make compile  

```

```bash
./launch.sh run <script> ...  

```

```bash
make pack  

```

--------------------------------

### Authenticate and launch pipelines with Seqera Platform

Source: https://docs.seqera.io/nextflow/migrations/25-10

Use `nextflow auth` and `nextflow launch` to authenticate with Seqera Platform and launch pipelines from the command line, replacing the previous `nextflow run -with-tower` workflow.

```bash
# Authenticate with Seqera Cloud  
nextflow auth login  
  
# Check authentication status  
nextflow auth status  
  
# Configure Seqera Platform settings  
nextflow auth config  
  
# Launch a pipeline in Seqera Platform  
nextflow launch nextflow-io/hello  

```

--------------------------------

### filter example

Source: https://docs.seqera.io/nextflow/reference/stdlib-types/channel

Emits the values from the channel that satisfy a condition, discarding all other values. The example filters for odd numbers.

```groovy
channel.of( 1, 2, 3, 4, 5 )  
    .filter { v -> v % 2 == 1 }  
    .view()  

```

```text
1  
3  
5  

```

--------------------------------

### List installed modules with nextflow module list

Source: https://docs.seqera.io/nextflow/modules/using-modules

List all modules installed in your project with the `module list` command. The output shows each module's name, installed version, and whether it has been modified locally. Use `-output json` for machine-readable output.

```bash
$ nextflow module list  

```
Tokens:
Raw