### General Usage of Grenedalf Commands Source: https://github.com/lczech/grenedalf/wiki/Home Demonstrates the basic command-line structure for using grenedalf. Users can get an overview of all commands with `--help` and detailed options for specific commands by appending `--help` to the command name. ```bash grenedalf --help grenedalf --help ``` -------------------------------- ### Get Grenedalf Version Information Source: https://github.com/lczech/grenedalf/wiki/Home The `version` command provides extended version information about the grenedalf tool, including build details and dependencies. The `citation` command prints references for academic use. ```bash grenedalf version grenedalf citation ``` -------------------------------- ### Autoconf Dependency Check for HTSlib Source: https://github.com/lczech/grenedalf/blob/master/CMakeLists.txt Searches for the 'autoconf' executable, which is required for building htslib. If not found, it provides platform-specific installation instructions and terminates the build with a fatal error. ```cmake find_program(AUTOCONF_RUNTIME autoconf) if( AUTOCONF_RUNTIME STREQUAL "AUTOCONF_RUNTIME-NOTFOUND" ) message ( STATUS "${ColorRed}We could not find autotools/autoconf, " "which is needed to compile htslib.${ColorEnd}" ) IF(APPLE) message ( STATUS "On MacOS, try: `brew install autoconf automake libtool`" ) ELSE() message ( STATUS "On Linux, try: `sudo apt-get install autoconf autotools-dev`" ) ENDIF() message( FATAL_ERROR "Required autotools/autoconf for building htslib not found.") endif() ``` -------------------------------- ### Configure output settings Source: https://context7.com/lczech/grenedalf/llms.txt Allows customization of output directories, file naming conventions, and compression settings for result files. ```bash grenedalf diversity \ --out-dir ./results \ --file-prefix "analysis_" \ --file-suffix "_v1" \ --compress ``` -------------------------------- ### Input and Reference Settings Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-frequency Configuration for multi-file processing and reference genome integration. ```APIDOC ## CLI Configuration: Input Settings ### Description Settings to manage multi-file input logic and reference genome metadata for sequence analysis. ### Parameters - **--multi-file-locus-set** (TEXT) - Optional - Select 'union' or 'intersection' for processing multiple input files. Default is 'union'. - **--make-gapless** (FLAG) - Optional - If set, fills absent positions as missing data in output tables. - **--reference-genome-fasta** (TEXT:FILE) - Optional - Path to reference genome in .fasta format. Excludes --reference-genome-dict and --reference-genome-fai. - **--reference-genome-dict** (TEXT:FILE) - Optional - Path to sequence dictionary in .dict format. Excludes --reference-genome-fasta and --reference-genome-fai. - **--reference-genome-fai** (TEXT:FILE) - Optional - Path to sequence dictionary in .fai format. Excludes --reference-genome-fasta and --reference-genome-dict. ``` -------------------------------- ### Input File Handling Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-diversity Configure how multiple input files are processed and how missing data is handled. ```APIDOC ## --multi-file-locus-set ### Description When multiple input files are provided, select whether the union of all their loci is used (outer join), or their intersection (inner join). For their union, input files that do not have data at a particular locus are considered as missing at that locus. Note that we allow to use multiple input files even with different file types. ### Method (Not applicable, this is a configuration option) ### Endpoint (Not applicable) ### Parameters #### Query Parameters - **--multi-file-locus-set** (TEXT:{union,intersection}=union) - Optional - Specifies whether to use the union or intersection of loci from multiple input files. ### Request Example (Not applicable) ### Response #### Success Response (200) (Not applicable) #### Response Example (Not applicable) ## --make-gapless ### Description By default, we only operate on the positions for which there is data. In particular, positions that are absent in the input are completely ignored; they do not even show up in the `missing` column of output tables. This is because for the statistics, data being absent or (marked as) missing is merely a sementic distinction, but it does not change the results. However, it might make processing with downstream tools easier if the output contains all positions, for instance when using `single` windows. With this option, all absent positions are filled in as missing data, so that they show up in the `missing` column and as entries in single windows. If a referene genome or dictionary is given, this might also include positions beyond where there is input data, up until the length of each chromosome. Note that this can lead to large ouput tables when processing single positions. ### Method (Not applicable, this is a configuration option) ### Endpoint (Not applicable) ### Parameters #### Query Parameters - **--make-gapless** (FLAG) - Optional - Fill in absent positions with missing data to ensure all positions are represented in the output. ### Request Example (Not applicable) ### Response #### Success Response (200) (Not applicable) #### Response Example (Not applicable) ``` -------------------------------- ### Input VCF/BCF configuration Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-diversity Defines how to specify VCF/BCF files or directories and requirements for the input data. ```APIDOC ## [CLI] --vcf-path ### Description Specifies a list of VCF/BCF files or directories to process. The input must contain the per-sample VCF FORMAT field 'AD' (allelic depth) and only SNP positions are processed. ### Method CLI Argument ### Parameters #### Query Parameters - **--vcf-path** (TEXT:PATH) - Required - List of vcf/bcf files or directories to process. Expects 'AD' field and only processes SNP positions. ``` -------------------------------- ### Input Sync Configuration Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-sync Configuration for specifying sync files or directories for processing. ```APIDOC ## [CLI] --sync-path ### Description Specifies a list of sync files or directories to process, compatible with PoPoolation2 format. Directories are scanned for files with the .sync[.gz] extension. ### Method CLI Argument ### Parameters #### Query Parameters - **--sync-path** (TEXT:PATH) - Required - List of sync files or directories to process. Can be provided multiple times or space-separated. ``` -------------------------------- ### Configure windowing strategies Source: https://context7.com/lczech/grenedalf/llms.txt Defines how genomic data is partitioned for analysis, supporting sliding intervals, fixed SNP queues, or user-defined regions. ```bash # Sliding interval windows grenedalf fst --window-type interval --window-interval-width 50000 --window-interval-stride 10000 # Queue windows (fixed number of SNPs) grenedalf diversity --window-type queue --window-queue-count 100 # User-defined region windows grenedalf fst --window-type regions --window-region-list genes.bed ``` -------------------------------- ### Input VCF/BCF Configuration Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-sync Configuration for specifying VCF/BCF files or directories for processing, requiring the AD field. ```APIDOC ## [CLI] --vcf-path ### Description Specifies a list of VCF/BCF files or directories to process. Requires the per-sample VCF FORMAT field 'AD' (allelic depth) to interpret allele counts for pools of individuals. Only SNP positions are processed; indels are skipped. ### Method CLI Argument ### Parameters #### Query Parameters - **--vcf-path** (TEXT:PATH) - Required - List of vcf/bcf files or directories to process. Files must have .vcf[.gz] or .bcf extensions. ``` -------------------------------- ### Grenedalf Output API Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-fst Configuration for managing output directories and file naming. ```APIDOC ## Grenedalf Output ### Description Configuration for managing output directories and file naming. ### Parameters #### Query Parameters - **--out-dir** (TEXT) - Optional, defaults to '.' - Specifies the directory where output files will be written. - **--file-prefix** (TEXT) - Optional - Adds a prefix to the output file names, useful for distinguishing runs with different data. - **--file-suffix** (TEXT) - Optional - Adds a suffix to the output file names, useful for distinguishing runs with different data. - **--compress** (FLAG) - Optional - If set, output files will be compressed using gzip, and the file extension will be automatically appended with `.gz`. ``` -------------------------------- ### CLI: grenedalf simulate Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-simulate The simulate command creates a file with simulated random frequency data based on specified read depths and genome parameters. ```APIDOC ## CLI COMMAND: grenedalf simulate ### Description Creates a file with simulated random frequency data. This command is used to generate synthetic genomic data for testing and validation purposes. ### Method CLI Execution ### Endpoint grenedalf simulate [options] ### Parameters #### Required Options - **--read-depths** (TEXT) - Required - Read depths of samples as a comma- or tab-separated list. Can be a single number or min/max range. - **--length** (UINT) - Required - Total length of the chromosome to simulate. #### Settings Options - **--format** (TEXT) - Optional - Output format: 'pileup' or 'sync'. Default: 'pileup'. - **--random-seed** (UINT) - Optional - Seed for random number generation for reproducible results. #### Genome Options - **--chromosome** (TEXT) - Optional - Name of the chromosome. Default: 'A'. - **--mutation-rate** (FLOAT) - Optional - Mutation rate to simulate (0-1). Excludes --mutation-count. - **--mutation-count** (UINT) - Optional - Total number of mutations to simulate. Excludes --mutation-rate. - **--omit-invariant-positions** (FLAG) - Optional - If set, only write mutated positions. #### Pileup Options - **--with-quality-scores** (FLAG) - Optional - Write phred-scaled quality scores. - **--min-phred-score** (UINT) - Optional - Minimum phred score (0-90). Default: 10. - **--max-phred-score** (UINT) - Optional - Maximum phred score (0-90). Default: 40. ### Request Example `grenedalf simulate --read-depths 50,100 --length 1000 --format pileup` ### Response #### Success Response (Exit Code 0) - **file** (FILE) - A generated file containing simulated frequency data in the requested format. #### Response Example `Successfully generated simulated data in ./simulate_output.pileup` ``` -------------------------------- ### Simulate frequency data Source: https://context7.com/lczech/grenedalf/llms.txt Generates random frequency data for benchmarking and testing purposes. Users can define the number of samples, positions, and read depth constraints. ```bash grenedalf simulate \ --num-samples 5 \ --num-positions 10000 \ --min-read-depth 10 \ --max-read-depth 100 \ --out-dir ./simulated ``` -------------------------------- ### Grenedalf Settings API Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-fst Configuration options for computation methods, sample comparisons, and output details. ```APIDOC ## Grenedalf Settings ### Description Configuration options for computation methods, sample comparisons, and output details. ### Parameters #### Query Parameters - **--method** (TEXT: {unbiased-nei, unbiased-hudson, kofler, karlsson}) - Required - FST method to use for the computation. Options include unbiased pool-sequencing statistics (Nei, Hudson), Kofler et al. statistic, and Karlsson et al. estimator. Methods other than Karlsson require `--pool-sizes`. - **--pool-sizes** (TEXT) - Required for unbiased methods - Pool sizes for all samples. Can be a single value for all samples or a path to a file with sample names and pool sizes. - **--comparand** (TEXT) - Optional - Specifies a single sample name to compute pairwise statistics against all other samples. - **--second-comparand** (TEXT) - Optional - Requires `--comparand`. Specifies a second sample name to compute statistics only between the two specified samples. - **--comparand-list** (TEXT: FILE) - Optional - Excludes `--comparand` and `--second-comparand`. Specifies a file containing pairs of sample names for which to compute statistics. - **--write-pi-tables** (FLAG) - Optional - When using unbiased estimators (Nei or Hudson), this flag enables writing tables of involved pi values (within, between, and total). - **--no-extra-columns** (FLAG) - Optional - Suppresses extra columns containing counts for each position and sample pair, outputting only window coordinates and FST values. - **--no-nan-windows** (FLAG) - Optional - Prevents the output of windows where all values are NaN, reducing clutter for small window sizes or individual positions. ``` -------------------------------- ### Sample Subsampling Options Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-sync Options for subsampling nucleotide counts of each sample to a maximum read depth and the method used for subsampling. ```APIDOC ## --subsample-max-read-depth ### Description If provided, the nucleotide counts of each sample are subsampled so that they do not exceed this given maximum total read depth. If they are below this value anyway, they are not changed. This transformation is useful to limit the maximum read depth and avoid biases in diversity estimators. ### Method (Implicitly used in subsampling logic) ### Endpoint N/A (Configuration Option) ### Parameters #### Query Parameters - **--subsample-max-read-depth** (UINT) - Optional - Maximum total read depth to subsample to. ### Request Example (Not applicable for configuration options) ### Response #### Success Response (200) (Not applicable for configuration options) #### Response Example (Not applicable for configuration options) ## --subsample-method ### Description When using `--subsample-max-read-depth`, decide which method to use. Options include 'subscale' (re-scales base counts), 'subsample-with-replacement' (resamples from multinomial distribution), and 'subsample-without-replacement' (resamples from multivariate hypergeometric distribution). ### Method (Implicitly used in subsampling logic) ### Endpoint N/A (Configuration Option) ### Parameters #### Query Parameters - **--subsample-method** (TEXT) - Optional - Method to use for subsampling. Allowed values: {subscale, subsample-with-replacement, subsample-without-replacement}. Defaults to 'subscale'. Requires `--subsample-max-read-depth`. ### Request Example (Not applicable for configuration options) ### Response #### Success Response (200) (Not applicable for configuration options) #### Response Example (Not applicable for configuration options) ``` -------------------------------- ### Specify Genomic Regions for Analysis Source: https://github.com/lczech/grenedalf/wiki/Example Alternative configuration to compute diversity metrics for a specific genomic region instead of the whole genome. ```bash --window-type regions \ --window-region 5:2676415-3676415 ``` -------------------------------- ### POST /process-frequency-table Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-fst-cathedral Configures the input parameters for processing frequency table files, including column mapping and data transformation settings. ```APIDOC ## POST /process-frequency-table ### Description Configures the input settings for processing frequency table files (CSV/TSV). This endpoint defines how the system reads, interprets, and transforms allele frequency data into internal counts. ### Method POST ### Endpoint /process-frequency-table ### Parameters #### Request Body - **--frequency-table-path** (TEXT) - Required - List of frequency table files or directories to process. - **--frequency-table-separator-char** (TEXT) - Optional - Separator character (comma, tab, space, semicolon). Default: comma. - **--frequency-table-missing-value** (TEXT) - Optional - Marker for missing values. Default: ., nan, na. - **--frequency-table-depth-factor** (FLOAT) - Optional - Multiplier to convert frequencies to pseudo-counts. Default: 1000000. - **--frequency-table-freq-is-ref** (FLAG) - Optional - If set, interprets frequencies as reference allele frequencies instead of alternative. - **--frequency-table-chr-column** (TEXT) - Optional - Name of the chromosome column. - **--frequency-table-pos-column** (TEXT) - Optional - Name of the position column. - **--frequency-table-ref-base-column** (TEXT) - Optional - Name of the reference base column. - **--frequency-table-alt-base-column** (TEXT) - Optional - Name of the alternative base column. - **--frequency-table-sample-ref-count-column** (TEXT) - Optional - Prefix/suffix for reference count columns. - **--frequency-table-sample-alt-count-column** (TEXT) - Optional - Prefix/suffix for alternative count columns. ### Request Example { "--frequency-table-path": "/data/freqs.tsv", "--frequency-table-separator-char": "tab", "--frequency-table-depth-factor": 1000000 } ### Response #### Success Response (200) - **status** (string) - Processing status message. #### Response Example { "status": "success", "message": "Frequency table processed successfully." } ``` -------------------------------- ### Frequency Table Columns Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-diversity Configure how per-sample frequency and depth columns are identified in input files. ```APIDOC ## --frequency-table-sample-freq-column ### Description Specify the exact prefix or suffix of the per-sample frequency columns in the header, case sensitive. By default, we look for column names having "frequency", "freq", "maf", "af", or "allelefrequency", case insensitive, and ignoring any extra punctuation marks, as a prefix or suffix, with the remainder of the column name used as the sample name. For example, "S1.freq" indicates the frequency column for sample "S1". Note that when the input data contains frequencies, but no reference or alternative base columns, such as HAF-pipe output tables, we cannot know the bases, and will hence guess. To properly set the reference bases, consider providing the `--reference-genome-fasta` option. ### Method (Not applicable, this is a configuration option) ### Endpoint (Not applicable) ### Parameters #### Query Parameters - **--frequency-table-sample-freq-column** (TEXT) - Required (if not using default) - Specify the exact prefix or suffix of the per-sample frequency columns. ### Request Example (Not applicable) ### Response #### Success Response (200) (Not applicable) #### Response Example (Not applicable) ## --frequency-table-sample-depth-column ### Description Specify the exact prefix or suffix of the per-sample read depth columns in the header, case sensitive. By default, we look for column names having "readdepth", "depth", "coverage", "cov", or "ad", case insensitive, and ignoring any extra punctuation marks, as a prefix or suffix, with the remainder of the column name used as the sample name. For example, "S1.read-depth" indicates the read depth column for sample "S1". ### Method (Not applicable, this is a configuration option) ### Endpoint (Not applicable) ### Parameters #### Query Parameters - **--frequency-table-sample-depth-column** (TEXT) - Required (if not using default) - Specify the exact prefix or suffix of the per-sample read depth columns. ### Request Example (Not applicable) ### Response #### Success Response (200) (Not applicable) #### Response Example (Not applicable) ``` -------------------------------- ### Frequency Table Configuration Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-frequency Parameters for defining how per-sample frequency and depth columns are parsed from input tables. ```APIDOC ## CLI Configuration: Frequency Table Parameters ### Description Configures how the tool identifies sample-specific frequency and read-depth columns within input data files. ### Parameters - **--frequency-table-sample-freq-column** (TEXT) - Optional - Specify the exact prefix or suffix of the per-sample frequency columns. Defaults to case-insensitive matching of 'frequency', 'freq', 'maf', 'af', or 'allelefrequency'. Requires --frequency-table-path. - **--frequency-table-sample-depth-column** (TEXT) - Optional - Specify the exact prefix or suffix of the per-sample read depth columns. Defaults to case-insensitive matching of 'readdepth', 'depth', 'coverage', 'cov', or 'ad'. Requires --frequency-table-path. ``` -------------------------------- ### Sample Filtering Options Source: https://github.com/lczech/grenedalf/wiki/Subcommand:-sync This section covers filters applied on a per-sample basis to refine variant calls. ```APIDOC ## Sample Filtering Options ### Description These options allow for filtering variants based on counts and read depth within individual samples. ### Method N/A (These are command-line arguments or configuration parameters) ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Min Count ### Description Minimum base count for a nucleotide (in ACGT) to be considered as an allele. Counts below that are set to zero, and hence ignored as an allele/variant. For example, singleton read sequencing errors can be filtered out this way. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Max Count ### Description Maximum base count for a nucleotide (in ACGT) to be considered as an allele. Counts above that are set to zero, and hence ignored as an allele/variant. For example, spuriously high read counts can be filtered out this way. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Deletions Limit ### Description Maximum number of deletions at a position before being filtered out. If this is set to a value greater than 0, and the number of deletions at the position is equal to or greater than this value, the sample is filtered out. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Min Read Depth ### Description Minimum read depth expected for a position in a sample to be considered covered. If the sum of nucleotide counts (in ACGT) at a given position in a sample is less than the provided value, the sample is ignored at this position. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Max Read Depth ### Description Maximum read depth expected for a position in a sample to be considered covered. If the sum of nucleotide counts (in ACGT) at a given position in a sample is greater than the provided value, the sample is ignored at this position. This can for example be used to filter spuriously high read depth positions. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Only SNPs ### Description Filter out any positions in a sample that do not have two or more alleles (i.e., that are invariant). That is, after applying --filter-sample-min-count and --filter-sample-max-count, if less than two counts (in ACGT) are non-zero, the position is not considered a SNP for the sample, and ignored. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ## Filter Sample Only Biallelic SNPs ### Description Filter out any positions in a sample that do not have exactly two alleles. That is, after applying --filter-sample-min-count and --filter-sample-max-count, if not exactly two counts (in ACGT) are non-zero, the position is not considered a biallelic SNP for the sample, and ignored. ### Method N/A ### Endpoint N/A ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body None ### Request Example None ### Response #### Success Response (N/A) None #### Response Example None ``` -------------------------------- ### Download Project Dependencies (CMake) Source: https://github.com/lczech/grenedalf/blob/master/CMakeLists.txt Includes a CMake script to download project dependencies from GitHub. It defines commit hashes for CLI11 and genesis, and uses a helper function `DOWNLOAD_GITHUB_DEPENDENCY` to fetch and store these libraries in the specified local path. ```cmake # Downloads dependencies if they are not there. We use a special script for this. include( "${CMAKE_CURRENT_LIST_DIR}/tools/cmake/DownloadDependency.cmake" ) # Store the commit hashes of our dependencies. # These are replaced by tools/cmake/update_dependencies.sh to the hashes that are currently checked out. # Thus, do not replace the hashes manually! SET( CLI11_COMMIT_HASH "5cb3efabce007c3a0230e4cc2e27da491c646b6c" ) #CLI11_COMMIT_HASH# SET( genesis_COMMIT_HASH "f1a5fb82d4b59706e8dbf60d41d094b88022fc94" ) #genesis_COMMIT_HASH# # Call the github download function, which takes four arguments: # - LIBPATH : Path to the libracy dir where dependencies are stored. # - LIBNAME : Name of the dependency, i.e., the name of its main directory within the ${LIBPATH}. # - TESTFILE : A testfile to check if the dependency is already there. # - REPOSITORY : Github repository (user/reponame) # - COMMITHASH : Hash of the commit to check out DOWNLOAD_GITHUB_DEPENDENCY( "${PROJECT_SOURCE_DIR}/libs" "CLI11" "CMakeLists.txt" "CLIUtils/CLI11" ${CLI11_COMMIT_HASH} ) DOWNLOAD_GITHUB_DEPENDENCY( "${PROJECT_SOURCE_DIR}/libs" "genesis" "CMakeLists.txt" "lczech/genesis" ${genesis_COMMIT_HASH} ) ``` -------------------------------- ### Configure input data formats Source: https://context7.com/lczech/grenedalf/llms.txt Grenedalf supports various input formats including SAM/BAM/CRAM, VCF, Sync files, and pre-computed frequency tables. Pool sizes can be defined inline or via an external CSV file. ```bash # SAM/BAM/CRAM input grenedalf diversity --sam-path ./sample.bam --pool-sizes "sample:50" ... # Multiple SAM files grenedalf diversity --sam-path ./samples/*.bam --pool-sizes pool-sizes.csv ... # Sync file input grenedalf diversity --sync-path ./data.sync.gz --pool-sizes pool-sizes.csv ... # VCF input grenedalf diversity --vcf-path ./variants.vcf.gz --pool-sizes pool-sizes.csv ... # Frequency table input grenedalf fst \ --frequency-table-path ./frequencies.csv \ --frequency-table-depth-factor 1200 \ --pool-sizes pool-sizes.csv ... ```