SemiBin

https://semibin.readthedocs.io/en/latest/
Admin
SemiBin is a command-line tool for metagenomic binning using semi-supervised siamese neural...
Tokens:13,171
Snippets:219
Trust Score:9.6
Update:5 months ago
Show doc for...
Tokens:
Raw
### Install SemiBin from source

Source: https://semibin.readthedocs.io/en/latest/install

Install SemiBin after its dependencies have been installed by running pip install . in the source directory.

```bash
pip install .

```

--------------------------------

### Install SemiBin with pixi (CPU-only)

Source: https://semibin.readthedocs.io/en/latest/install

Use this configuration in your `pixi.toml` file to install SemiBin and its dependencies for CPU-only mode. Run `pixi install` in the same directory.

```toml
[project]
authors = ["Luis Pedro Coelho <luis@luispedro.org>"]
channels = ["conda-forge", "bioconda"]
name = "semibin_install"
platforms = ["linux-64", "osx-64"]
version = "0.1.0"

[tasks]

[dependencies]
semibin = ">=2.2.0,<3"

```

--------------------------------

### Install SemiBin with conda (Simple Mode)

Source: https://semibin.readthedocs.io/en/latest/install

Create and activate a new conda environment named 'SemiBin', then install SemiBin and its dependencies using conda channels.

```bash
conda create -n SemiBin
conda activate SemiBin
conda install -c conda-forge -c bioconda semibin

```

--------------------------------

### Install SemiBin with conda (GPU support)

Source: https://semibin.readthedocs.io/en/latest/install

Install SemiBin with GPU support by creating a conda environment with Python and pip, installing PyTorch with GPU support via pip, and then installing SemiBin via conda. Note: Pixi is the recommended method for GPU installations.

```bash
conda create -n SemiBin python pip
conda activate SemiBin
pip install torch torchvision torchaudio
conda install -c conda-forge -c bioconda semibin

```

--------------------------------

### Install SemiBin using Pixi

Source: https://semibin.readthedocs.io/en/latest

Installs SemiBin and its dependencies using the pixi package manager. Ensure pixi is initialized before adding the package.

```bash
pixi init
pixi add bioconda::semibin

```

--------------------------------

### Run SemiBin2 with Single Easy Bin

Source: https://semibin.readthedocs.io/en/latest/whatsnew

Example of how to programmatically call the SemiBin2 main function for single easy binning. Ensure necessary imports and arguments are provided.

```python
import SemiBin.main

...

SemiBin.main.main2(['single_easy_bin', '--input-fasta', ...])
```

--------------------------------

### Install SemiBin dependencies from source

Source: https://semibin.readthedocs.io/en/latest/install

Install required dependencies (Bedtools, Hmmer, Samtools) for building SemiBin from source using conda.

```bash
conda install -c bioconda bedtools hmmer samtools

```

--------------------------------

### Install SemiBin with pixi (GPU support)

Source: https://semibin.readthedocs.io/en/latest/install

Configure your `pixi.toml` to include `pytorch-gpu` and `cuda` for GPU-accelerated SemiBin installations. Ensure your system meets the CUDA version requirement.

```toml
[project]
authors = ["Luis Pedro Coelho <luis@luispedro.org>"]
channels = ["conda-forge", "bioconda"]
name = "semibin_install"
platforms = ["linux-64"]
version = "0.1.0"

[tasks]

[dependencies]
semibin = ">=2.2.0,<3"
pytorch-gpu = "*"

[system-requirements]
cuda = "12.0"

```

--------------------------------

### Example Concatenated FASTA Format

Source: https://semibin.readthedocs.io/en/latest/usage

This is an example of the format for a concatenated FASTA file, where contig names are prefixed with the sample name and a separator.

```fasta
>S1:Contig_1
AGATAATAAAGATAATAATA
>S1:Contig_2
CGAATTTATCTCAAGAACAAGAAAA
>S1:Contig_3
AAAAAGAGAAAATTCAGAATTAGCCAATAAAATA
>S2:Contig_1
AATGATATAATACTTAATA
>S2:Contig_2
AAAATATTAAAGAAATAATGAAAGAAA
>S3:Contig_1
ATAAAGACGATAAAATAATAAAAGCCAAATCCGACAAAGAAAGAACGG
>S3:Contig_2
AATATTTTAGAGAAAGACATAAACAATAAGAAAAGTATT
>S3:Contig_3
CAAAT

```

--------------------------------

### Generate Cannot-Link File (Multi-Sample Mode - Loop)

Source: https://semibin.readthedocs.io/en/latest/semi-supervised

For multi-sample mode, call `generate_cannot_links` independently for each input FASTA file. This example demonstrates using a bash loop for efficiency.

```bash
for sample in S1 S2 S3 S4 S5; do
    SemiBin2 generate_cannot_links -i ${sample}.fa -o ${sample}_output
done

```

--------------------------------

### Single-Sample Binning with Self-Supervised Learning

Source: https://semibin.readthedocs.io/en/latest/usage

Learn a new model directly from your data for potentially better results, though this option requires more time. Ensure your input files are correctly specified.

```bash
SemiBin2 single_easy_bin \
        --self-supervised \
        -i S1.fa \
        -b S1.sorted.bam \
        -o output

```

--------------------------------

### Single-Sample Binning with Pre-trained Model

Source: https://semibin.readthedocs.io/en/latest/usage

Use this command for fast single-sample binning with a pre-trained model. Specify the environment that best matches your data.

```bash
SemiBin2 single_easy_bin \
        --environment human_gut \
        -i S1.fa \
        -b S1.sorted.bam \
        -o output

```

```bash
SemiBin2 single_easy_bin \
        --environment human_gut \
        --sequencing-type long_read \
        -i S1.fa \
        -b S1.sorted.bam \
        -o output

```

--------------------------------

### Pre-train Model from Multiple Samples

Source: https://semibin.readthedocs.io/en/latest/usage

Trains a SemiBin2 model using data from multiple samples. This offers a balance between training time and result quality.

```bash
SemiBin2 train_self \
    --train-from-many \
    -i S1.fa S2.fa S3.fa \
    --data S1/data.csv S2/data.csv S3/data.csv \
    --data-split S1/data_split.csv S2/data_split.csv S3/data_split.csv \
    -o S1_output
```

--------------------------------

### Run SemiBin2 Single Easy Bin

Source: https://semibin.readthedocs.io/en/latest/aemb

Execute SemiBin2's single_easy_bin subcommand using the original contigs and the generated abundance files. The output bins will be located in the specified output directory.

```bash
SemiBin2 single_easy_bin -i contig.fa -a sample1_*.tsv -o aemb_output/sample1
```

--------------------------------

### Single-sample binning with pre-trained model

Source: https://semibin.readthedocs.io/en/latest

Performs single-sample binning using a pre-trained model. This is the fastest option and recommended for common metagenome environments. Specify the environment using the --environment flag.

```bash
SemiBin2 single_easy_bin \
        --environment human_gut \
        -i S1.fa \
        -b S1.sorted.bam \
        -o output

```

--------------------------------

### Map Reads with bowtie2 and samtools

Source: https://semibin.readthedocs.io/en/latest/generate

This sequence of commands demonstrates mapping reads to contigs using bowtie2 and then processing the output with samtools to generate sorted and indexed BAM files. Adjust the thread count for bowtie2 as needed.

```bash
bowtie2-build -f contig.fa contig.fa

bowtie2 -q --fr -x contig.fa -1 reads_1.fq.gz -2 reads_2.fq.gz -S contig.sam -p 4

samtools view -h -b -S contig.sam -o contig.bam
samtools view -b -F 4 contig.bam -o contig.mapped.bam
samtools sort contig.mapped.bam -o contig.mapped.sorted.bam

samtools index contig.mapped.sorted.bam

```

--------------------------------

### Easy Single-Sample Binning with Semi-Supervised Mode

Source: https://semibin.readthedocs.io/en/latest/semi-supervised

Use `single_easy_bin` with the `--semi-supervised` flag for a streamlined single-sample binning process. This command handles all necessary steps automatically.

```bash
SemiBin2 single_easy_bin \
        --semi-supervised \
        -i S1.fa \
        -b S1.sorted.bam \
        -o output

```

--------------------------------

### Assemble Contigs and Map Reads with NGLess

Source: https://semibin.readthedocs.io/en/latest/generate

Use this NGLess script to perform FastQ preprocessing, assembly into contigs, and mapping of reads to the contigs in a single workflow. Ensure quality filtering and minimum read length are set appropriately.

```python
ngless "1.5"
import "samtools" version "1.0"

input = paired('reads_1.fq.gz', 'reads_2.fq.gz')
input = preprocess(input) using |r|:
    r = substrim(r, min_quality=25)
    if len(r) < 45:
        discard

contigs = assemble(input)
write(contigs, ofile='contig.fa')

mapped = map(input, fafile=contigs)

write(samtools_sort(mapped),
    ofile='output.sorted.bam')

```

--------------------------------

### Train Binning Models for Each Sample

Source: https://semibin.readthedocs.io/en/latest/usage

This script trains a binning model for each sample independently. It uses the generated data CSV files and can be parallelized.

```bash
for sample in S1 S2 S3 S4 S5 ; do
    SemiBin2 train_self \
        --data multi_output/samples/${sample}/data.csv \
        --data-split multi_output/samples/${sample}/data_split.csv \
        --output ${sample}_output
done

```

--------------------------------

### Single-sample binning by learning a new model

Source: https://semibin.readthedocs.io/en/latest

Performs single-sample binning by learning a new model directly from the provided data. This approach may take longer but can be more accurate for specialized datasets.

```bash
SemiBin2 single_easy_bin \
        -i S1.fa \
        -b S1.sorted.bam \
        -o output

```

--------------------------------

### Train Self Model for Co-assembly

Source: https://semibin.readthedocs.io/en/latest/usage

Trains a SemiBin2 model for co-assembly using generated data and data-split files. GPU acceleration is recommended.

```bash
SemiBin2 train_self \
    --data contig_output/data.csv \
    --data-split contig_output/data_split.csv \
    -o contig_output
```

--------------------------------

### Easy Multi-Sample Binning (Short Reads)

Source: https://semibin.readthedocs.io/en/latest/usage

Perform easy multi-sample binning for short reads using a concatenated FASTA file and sorted BAM files.

```bash
SemiBin2 multi_easy_bin \
        -i concatenated.fa \
        -b S1.sorted.bam S2.sorted.bam S3.sorted.bam S4.sorted.bam S5.sorted.bam \
        -o multi_output

```

--------------------------------

### Train Self Model for Single Sample

Source: https://semibin.readthedocs.io/en/latest/usage

Trains a SemiBin2 model using data from a single sample. This step benefits from GPU acceleration and can be skipped if using a pre-trained model.

```bash
SemiBin2 train_self \
    --data S1_output/data.csv \
    --data-split S1_output/data_split.csv \
    -o S1_output
```

--------------------------------

### Automate All-by-All Abundance Estimates with Jug

Source: https://semibin.readthedocs.io/en/latest/aemb

This script automates the all-by-all abundance estimation process. It uses Jug for parallelization and expects specific input file structures. Remove @TaskGenerator decorators to run sequentially.

```python
from jug import TaskGenerator
from jug.utils import jug_execute
import subprocess
from os import makedirs, path
import yaml

samples = [
        'sample0',
        'sample1',
        'sample2',
        'sample3',
        'sample4',
        'sample5',
        'sample6',
        'sample7',
        ]

STROBEALIGN_THREADS = 8
SEMIBIN_THREADS = STROBEALIGN_THREADS

@TaskGenerator
def generate_inputs(s):
    contigs = f'samples/{s}_assembled.fna.gz'
    if not path.exists(contigs):
        raise IOError(f'Expected contig file {contigs} (for sample {s})')
    out = f'aemb_output/{s}'
    makedirs(out, exist_ok=True)
    subprocess.check_call(
            ['SemiBin2', 'split_contigs',
             '-i', contigs,
             '-o', out])
    return out

@TaskGenerator
def cross_map(ref_out, ref_s, s):
    f1 = f'clean-reads/{s}.pair.1.fq.gz'
    f2 = f'clean-reads/{s}.pair.2.fq.gz'
    if not path.exists(f1):
        raise IOError(f'Expected reads file {f1} (for sample {s})')
    if not path.exists(f2):
        raise IOError(f'Expected reads file {f2} (for sample {s}). Note that {f1} does exist!')
    ofile = f'aemb_output/{ref_s}/mapped_{s}.tsv'
    with open(ofile, 'wb') as out:
        subprocess.check_call(
            ['strobealign',
                '--aemb', f'{ref_out}/split_contigs.fna.gz',
                f1, f2,
                '-t', str(STROBEALIGN_THREADS),
                '-R', '6'],
            stdout=out)
    return ofile


for s in samples:
    out = generate_inputs(s)
    tsv = []
    for s2 in samples:
        tsv.append(cross_map(out, s, s2))

    sb = jug_execute(
        ['SemiBin2', 'single_easy_bin',
            '--threads', str(SEMIBIN_THREADS),
            '-i', f'samples/{s}_assembled.fna.gz',
            '-a'] + tsv + [
                '-o', f'aemb_output/{s}'])

```

--------------------------------

### Generate Cannot-Link File (Single-Sample Mode)

Source: https://semibin.readthedocs.io/en/latest/semi-supervised

This command generates a cannot-link file required for semi-supervised training in single-sample or co-assembly modes. It relies on `mmseqs2` and may take a significant amount of time. Consider using `--taxonomy-annotation-table` if `mmseqs2` has already been run for GTDB annotation.

```bash
SemiBin2 generate_cannot_links -i S1.fa -o S1_output

```

--------------------------------

### Bin Short Reads for Each Sample

Source: https://semibin.readthedocs.io/en/latest/usage

This command performs binning for short reads for each sample independently, using the trained model and generated data.

```bash
for sample in S1 S2 S3 S4 S5 ; do
    SemiBin2 bin_short \
        -i ${sample}.fa \
        --model ${sample}_output/model.pt \
        --data multi_output/samples/${sample}/data.csv \
        -o output
done

```

--------------------------------

### Bin Contigs for Co-assembly

Source: https://semibin.readthedocs.io/en/latest/usage

Bins contigs for a co-assembly project using a trained model. Requires input FASTA, the trained model, and data files.

```bash
SemiBin2 bin_short \
    -i contig.fa \
    --model contig_output/model.pt \
    --data contig_output/data.csv \
    -o output
```

--------------------------------

### Bin Contigs with Built-in Environment (Short Reads)

Source: https://semibin.readthedocs.io/en/latest/usage

Bins contigs using a pre-built environment model for short reads. Replaces the `--model` argument with `--environment`.

```bash
SemiBin2 bin_short \
    -i S1.fa \
    --environment human_gut \
    --data S1_output/data.csv \
    -o S1_output
```

--------------------------------

### Generate Cannot-Link File (Multi-Sample Mode - Individual Calls)

Source: https://semibin.readthedocs.io/en/latest/semi-supervised

This shows the equivalent of the bash loop for generating cannot-link files in multi-sample mode, where each sample's FASTA file is processed individually.

```bash
SemiBin2 generate_cannot_links -i S1.fa -o S1_output
SemiBin2 generate_cannot_links -i S2.fa -o S2_output
SemiBin2 generate_cannot_links -i S3.fa -o S3_output
SemiBin2 generate_cannot_links -i S4.fa -o S4_output
SemiBin2 generate_cannot_links -i S5.fa -o S5_output

```

--------------------------------

### Easy Multi-Sample Binning (Long Reads)

Source: https://semibin.readthedocs.io/en/latest/usage

Perform easy multi-sample binning for long reads using a concatenated FASTA file and sorted BAM files. Use the `--sequencing-type long_read` flag for long reads or hybrid assemblies.

```bash
SemiBin2 multi_easy_bin \
        --sequencing-type long_read \
        -i concatenated.fa \
        -b S1.sorted.bam S2.sorted.bam S3.sorted.bam S4.sorted.bam S5.sorted.bam \
        -o multi_output

```

--------------------------------

### Easy Co-assembly Binning Mode

Source: https://semibin.readthedocs.io/en/latest/usage

Performs co-assembly binning using an easy mode. This mode is similar to single-sample binning but uses multiple BAM files for feature generation.

```bash
SemiBin2 single_easy_bin \
    -i contig.fa \
    -b S1.sorted.bam S2.sorted.bam S3.sorted.bam \
    -o co-assembly_output
```

--------------------------------

### Bin Contigs with Short Read Model

Source: https://semibin.readthedocs.io/en/latest/usage

Bins contigs using a trained model for short reads. Requires input FASTA, the trained model, and data files.

```bash
SemiBin2 bin_short \
    -i S1.fa \
    --model S1_output/model.pt \
    --data S1_output/data.csv \
    -o S1_output
```

--------------------------------

### Generate Cannot-Link Constraints using Python Script

Source: https://semibin.readthedocs.io/en/latest/generate

Execute a Python script to generate cannot-link constraints, likely for use with SemiBin. This command requires specifying input files for CAT output, contigs, sample name, and an output directory.

```python
python script/generate_cannot_link.py -i CAT.out -c contig.fa -s sample-name -o output --CAT

```

--------------------------------

### Bin Long Reads for Each Sample

Source: https://semibin.readthedocs.io/en/latest/usage

This command performs binning for long reads for each sample independently, using the trained model and generated data.

```bash
for sample in S1 S2 S3 S4 S5 ; do
    SemiBin2 bin_long \
        -i ${sample}.fa \
        --model ${sample}_output/model.pt \
        --data multi_output/samples/${sample}/data.csv \
        -o output
done

```

--------------------------------

### Concatenate FASTA Files with SemiBin2

Source: https://semibin.readthedocs.io/en/latest/usage

Use this command to combine multiple FASTA files into a single file, renaming contigs to include sample names. This is a prerequisite for multi-sample binning.

```bash
SemiBin2 concatenate_fasta \
    --input-fasta S1.fa S2.fa S3.fa S4.fa S5.fa \
    --output output

```

--------------------------------

### Bin Contigs with Built-in Environment (Long Reads)

Source: https://semibin.readthedocs.io/en/latest/usage

Bins contigs using a pre-built environment model for long reads. Replaces the `--model` argument with `--environment`.

```bash
SemiBin2 bin_long \
    -i S1.fa \
    --environment human_gut \
    --data S1_output/data.csv \
    -o S1_output
```

--------------------------------

### Generate Sequence Features for Single Sample

Source: https://semibin.readthedocs.io/en/latest/usage

Generates feature files required for training and binning a single sample. Use this for single-sample or co-assembly modes.

```bash
SemiBin2 generate_sequence_features_single -i S1.fa -b S1.sorted.bam -o S1_output
```

--------------------------------

### Map Reads with strobealign-aemb

Source: https://semibin.readthedocs.io/en/latest/aemb

Map reads to split contigs using strobealign-aemb to generate abundance information. Requires strobealign version 0.13 or newer. This is repeated for each sample's read set against the split contigs of the current sample.

```bash
strobealign --aemb aemb_output/sample1/split_contigs.fna.gz read1.pair.1.fq.gz read1.pair.2.fq.gz -R 6 -t 8 > sample1_sample1.tsv
```

```bash
strobealign --aemb aemb_output/sample1/split_contigs.fna.gz read2.pair.1.fq.gz read2.pair.2.fq.gz -R 6 -t 8 > sample1_sample2.tsv
```

```bash
strobealign --aemb aemb_output/sample1/split_contigs.fna.gz read3.pair.1.fq.gz read3.pair.2.fq.gz -R 6 -t 8 > sample1_sample3.tsv
```

```bash
strobealign --aemb aemb_output/sample1/split_contigs.fna.gz read4.pair.1.fq.gz read4.pair.2.fq.gz -R 6 -t 8 > sample1_sample4.tsv
```

```bash
strobealign --aemb aemb_output/sample1/split_contigs.fna.gz read5.pair.1.fq.gz read5.pair.2.fq.gz -R 6 -t 8 > sample1_sample5.tsv
```

--------------------------------

### Bin Contigs with Long Read Model

Source: https://semibin.readthedocs.io/en/latest/usage

Bins contigs using a trained model for long reads. Requires input FASTA, the trained model, and data files.

```bash
SemiBin2 bin_long \
    -i S1.fa \
    --model S1_output/model.pt \
    --data S1_output/data.csv \
    -o S1_output
```

--------------------------------

### Generate Cannot-Link Constraints with CAT

Source: https://semibin.readthedocs.io/en/latest/generate

Utilize CAT to generate taxonomic classifications and cannot-link pairs for contigs. This advanced step requires specific database paths and may need adjustments to parameters like `--force` and `--top` based on your data. Ensure Prodigal and Diamond paths are correctly set.

```bash
CAT contigs \
        -c contig.fa \
        -d CAT_prepare_20200304/2020-03-04_CAT_database \
        --path_to_prodigal $path_to_prodigal \
        --path_to_diamond $path_to_diamond \
        -t CAT_prepare_20200304/2020-03-04_taxonomy \
        -o CAT_output/CAT \
        --force \
        -f 0.5 \
        --top 11 \
        --I_know_what_Im_doing \
        --index_chunks 1

CAT add_names \
    CAT_output/CAT.contig2classification.txt \
    -o CAT_output/CAT.out \
    -t CAT_prepare_20200304/2020-03-04_taxonomy \
    --force \
    --only_official

```

--------------------------------

### Generate Sequence Features for Multi-Sample Binning

Source: https://semibin.readthedocs.io/en/latest/usage

This command generates sequence features required for advanced multi-sample binning workflows. It takes a gzipped concatenated FASTA file and sorted BAM files as input.

```bash
SemiBin2 generate_sequence_features_multi \
    -i concatenated.fa.gz \
    -b S1.sorted.bam S2.sorted.bam S3.sorted.bam S4.sorted.bam S5.sorted.bam \
    -o output

```

--------------------------------

### Generate Sequence Features for Co-assembly

Source: https://semibin.readthedocs.io/en/latest/usage

Generates feature files for co-assembly binning. Uses the `generate_sequence_features_single` mode with multiple BAM files.

```bash
SemiBin2 generate_sequence_features_single \
    -i contig.fa \
    -b S1.sorted.bam S2.sorted.bam S3.sorted.bam \
    -o contig_output
```

--------------------------------

### Split Contigs for AEMB

Source: https://semibin.readthedocs.io/en/latest/aemb

Use this command to split contigs into halves for AEMB processing. Contigs shorter than the minimum length are excluded.

```bash
mkdir -p aemb_output/sample1
SemiBin2 split_contigs -i sample1_contigs.fna.gz -o aemb_output/sample1
```
Tokens:
Raw