NCBI Datasets

https://www.ncbi.nlm.nih.gov/datasets/docs/v2/
Admin
NCBI Datasets provides command-line tools and API access to download genomic, gene, taxonomy, and...
Tokens:177,558
Snippets:1,509
Trust Score:10
Update:6 days ago
Show doc for...
Tokens:
Raw
### Run the Python program using the generated library

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/api/languages

Commands to run the example Python program. Creates a virtual environment, activates it, installs the generated library from python_lib, and runs gene_get_info.py to retrieve a zip file with gene information and print the protein.faa file.

```bash
# Create a virtual env and install the newly built library
virtualenv venv && source venv/bin/activate
pip install python_lib
# Run program to retrieve a zip file with gene information, and print the protein.faa file
python gene_get_info.py
```

--------------------------------

### datasets summary genome taxon examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary/genome/datasets_summary_genome_taxon

Shows three example invocations of the command using a common name, a quoted scientific name, and a taxonomy ID.

```bash
datasets summary genome taxon human
  datasets summary genome taxon "mus musculus"
  datasets summary genome taxon 10116
```

--------------------------------

### datasets summary genome examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary/genome

Example invocations of the datasets summary genome command: by accession, by taxon, with assembly-level filter, and with search strings.

```bash
datasets summary genome accession GCF_000001405.40
  datasets summary genome taxon "mus musculus"
  datasets summary genome taxon human --assembly-level chromosome,complete
  datasets summary genome taxon "mus musculus" --search C57BL/6J --search "Broad Institute"
```

--------------------------------

### datasets download gene examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/download/gene

Examples of downloading gene data packages by gene ID, symbol, accession, and with a fasta filter.

```bash
datasets download gene gene-id 672
```

```bash
datasets download gene symbol brca1 --taxon "mus musculus"
```

```bash
datasets download gene accession NP_000483.3
```

```bash
datasets download gene gene-id 2778 --fasta-filter NC_000020.11,NM_001077490.3,NP_001070958.1
```

--------------------------------

### datasets summary examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary

Examples of using 'datasets summary' to retrieve metadata for genomes, genes, taxonomy, and viruses. Each command specifies a subcommand and query parameters.

```bash
datasets summary genome accession GCF_000001405.40
```

```bash
datasets summary genome taxon "mus musculus"
```

```bash
datasets summary gene gene-id 672
```

```bash
datasets summary gene symbol brca1 --taxon "mus musculus"
```

```bash
datasets summary gene accession NP_000483.3
```

```bash
datasets summary taxonomy taxon "mus musculus"
```

```bash
datasets summary virus genome accession NC_045512.2
```

```bash
datasets summary virus genome taxon sars-cov-2 --host dog
```

--------------------------------

### datasets download virus genome - Examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/download/virus/genome

Two example commands: one downloads SARS-CoV-2 genomes from dog hosts including protein sequences, the other downloads coronaviridae genomes from Manis javanica hosts.

```bash
  datasets download virus genome taxon sars-cov-2 --host dog --include protein
  datasets download virus genome taxon coronaviridae --host "manis javanica"

```

--------------------------------

### dataformat excel genome examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/dataformat/excel/dataformat_excel_genome

Two example invocations: one using --inputfile with a JSONL report, and one using --package with a zip archive and field selection with wildcards.

```bash
dataformat excel genome --inputfile human/ncbi_dataset/data/assembly_data_report.jsonl --outputfile human-genomes.xlsx
```

```bash
dataformat excel genome --package human.zip --outputfile human-genomes.xlsx --fields accession,'organism*','*busco*'
```

--------------------------------

### dataformat tsv virus-genome examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/dataformat/tsv/dataformat_tsv_virus-genome

Two example invocations: one using --inputfile with a JSONL report and field selection, and one using --package with a zip archive.

```bash
dataformat tsv virus-genome --inputfile sars2_package/ncbi_dataset/data/data_report.jsonl --fields accession,'*name','isolate*'
```

```bash
dataformat tsv virus-genome --package virus-sars2-refseq.zip
```

--------------------------------

### dataformat tsv prok-gene examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/dataformat/tsv/dataformat_tsv_prok-gene

Two example invocations: one using --inputfile with a JSONL report and --fields, and one using --package with a zip archive.

```bash
dataformat tsv prok-gene --inputfile prokaryote_package/ncbi_dataset/data/data_report.jsonl --fields accession,'name*'
```

```bash
dataformat tsv prok-gene --package prokaryote-package.zip
```

--------------------------------

### dataformat tsv gene examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/dataformat/tsv/dataformat_tsv_gene

Two example invocations: one using --inputfile with --fields to select specific fields, and one using --package to read from a zip archive.

```bash
dataformat tsv gene --inputfile gene_package/ncbi_dataset/data/data_report.jsonl --fields annotation-assembly-accession,'*date','summary*'
```

```bash
dataformat tsv gene --package genes.zip
```

--------------------------------

### datasets summary genome accession examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary/genome/datasets_summary_genome_accession

Examples of using the datasets summary genome accession command with various accessions and flags. The first example retrieves a genome report for a single Assembly accession. The second shows multiple accessions. The third uses the sequence report with JSON lines output. The fourth uses a BioProject accession.

```bash
datasets summary genome accession GCF_000001405.40
```

```bash
datasets summary genome accession GCA_003774525.2 GCA_000001635
```

```bash
datasets summary genome accession GCF_000001405.40 --report sequence --as-json-lines
```

```bash
datasets summary genome accession PRJNA31257
```

--------------------------------

### Generate and install zsh completion script

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/completion/datasets_completion_zsh

Generates the zsh completion script and saves it to the first directory in fpath as _ncbi_datasets. Run once, then start a new shell for the setup to take effect.

```bash
datasets completion zsh > "${fpath[1]}/_ncbi_datasets"

```

--------------------------------

### dataformat excel microbigge examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/dataformat/excel/dataformat_excel_microbigge

Two example invocations: one using --inputfile to specify a JSONL data report and --outputfile to name the resulting Excel file, and another using --package to read from a zip archive.

```bash
dataformat excel microbigge --inputfile pathogen_package/ncbi_dataset/data/data_report.jsonl --outputfile pathogen.xlsx
  dataformat excel microbigge --package van.zip --outputfile van.xlsx
```

--------------------------------

### Get virus primary metadata by accession

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/how-tos/virus/influenza-metadata

For virus genomes, use GenBank nucleotide accessions instead of genome assembly accessions (which start with GCA or GCF). Example uses accession PP756108.1.

```bash
datasets summary virus genome accession PP756108.1
```

--------------------------------

### Get virus primary metadata by accession

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/how-tos/virus/virus-metadata

Use the `datasets summary virus genome accession` command with a GenBank nucleotide accession (not a genome assembly accession starting with GCA or GCF) to retrieve primary metadata for a specific virus genome.

```bash
datasets summary virus genome accession NC_063383.1

```

--------------------------------

### datasets download genome examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/download/genome

Four example invocations of the datasets download genome command: by accession with chromosome and include filters, by taxon with dehydrated flag, by taxon with assembly-level filter, and by taxon with multiple search terms.

```bash
datasets download genome accession GCF_000001405.40 --chromosomes X,Y --include genome,gff3,rna
```

```bash
datasets download genome taxon "bos taurus" --dehydrated
```

```bash
datasets download genome taxon human --assembly-level chromosome,complete --dehydrated
```

```bash
datasets download genome taxon "house mouse" --search C57BL/6J --search "Broad Institute" --dehydrated
```

--------------------------------

### datasets download examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/download

Examples of using the `datasets download` command for genome, gene, taxonomy, and virus datasets. Each command shows a different use case, such as downloading by accession, taxon, gene ID, symbol, or with specific options like --chromosomes and --include.

```bash
datasets download genome accession GCF_000001405.40 --chromosomes X,Y --include cds,genome,protein,seq-report,gbff,gff3
```

```bash
datasets download genome taxon "bos taurus" --reference
```

```bash
datasets download gene gene-id 672
```

```bash
datasets download gene symbol brca1 --taxon "mus musculus"
```

```bash
datasets download gene accession NP_000483.3
```

```bash
datasets download taxonomy taxon human,sars-cov-2
```

```bash
datasets download virus genome taxon sars-cov-2 --host dog
```

```bash
datasets download virus protein S --host dog --filename SARS2-spike-dog.zip
```

--------------------------------

### Install NCBI Datasets CLI via conda

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/command-line-tools/download-and-install

Install the NCBI Datasets CLI tools (datasets and dataformat) via conda. First create a conda environment, then activate it, and finally install the ncbi-datasets-cli package from conda-forge.

```bash
conda create -n ncbi_datasets
```

```bash
conda activate ncbi_datasets
```

```bash
conda install -c conda-forge ncbi-datasets-cli
```

--------------------------------

### datasets summary taxonomy taxon examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary/taxonomy/datasets_summary_taxonomy_taxon

Shows multiple example invocations of the 'datasets summary taxonomy taxon' command, demonstrating queries by common name, scientific name, multiple taxa, children, parents, rank filtering, and ids_only report.

```bash
datasets summary taxonomy taxon human
```

```bash
datasets summary taxonomy taxon "mus musculus" "human" --report names
```

```bash
datasets summary taxonomy taxon "human" --children
```

```bash
datasets summary taxonomy taxon "mus musculus" --rank genus
```

```bash
datasets summary taxonomy taxon "human" --parents --report names
```

```bash
datasets summary taxonomy taxon 10116 --report ids_only
```

--------------------------------

### datasets summary gene symbol examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary/gene/datasets_summary_gene_symbol

Shows basic usage of the datasets summary gene symbol command. The first example retrieves a summary for the human TP53 gene. The second example specifies a taxon using the --taxon flag with the scientific name for mouse.

```bash
datasets summary gene symbol tp53
```

```bash
datasets summary gene symbol brca1 --taxon "mus musculus"
```

--------------------------------

### Example genomic FASTA defline

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/data-packages/virus-genome

Example FASTA defline for a genomic nucleotide sequence in the genomic.fna file. Shows the format with accession, description, and organism information.

```fasta
>MW583405.1 Severe acute respiratory syndrome coronavirus 2 isolate SARS-CoV-2/human/USA/TX-CDC-9N37-8996/2021, complete genome

```

--------------------------------

### dataformat tsv genome-seq examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/dataformat/tsv/dataformat_tsv_genome-seq

Two example invocations: the first uses --inputfile with --fields to select specific fields (including wildcards), and the second uses --package to specify a zip archive and an inputfile relative to the archive root.

```bash
dataformat tsv genome-seq --inputfile human_package/ncbi_dataset/data/GCF_000001405.40/sequence_report.jsonl --fields accession,'gc*','*count'
  dataformat tsv genome-seq --package human.zip --inputfile GCF_000001405.40/sequence_report.jsonl
```

--------------------------------

### Example protein FASTA defline

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/data-packages/virus-genome

Example FASTA defline for a protein sequence in the protein.faa file. Shows the format with accession, coordinates, protein name, and organism details.

```fasta
>QMT27626.1:1-180 leader protein [polyprotein=ORF1ab polyprotein] [organism=Severe acute respiratory syndrome coronavirus 2] [isolate=SARS-CoV-2/human/USA/WA-S1488/2020]

```

--------------------------------

### Download taxonomy package by taxon

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/download/taxonomy/datasets_download_taxonomy_taxon

Examples of downloading a taxonomy data package by taxon name or ID, with options to include names report and include parents/children.

```bash
datasets download taxonomy taxon "bos taurus"
```

```bash
datasets download taxonomy taxon human,"drosophila melanogaster" --include names
```

```bash
datasets download taxonomy taxon 10116 --parents --children
```

--------------------------------

### Get genome metadata by Assembly accession

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/how-tos/genomes/get-genome-metadata

Use `datasets summary genome accession GCF_000001405.40` to get metadata for the human reference genome GRCh38 in JSON format.

```bash
datasets summary genome accession GCF_000001405.40

```

--------------------------------

### Download virus data package examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/download/virus

Two examples using the datasets download virus command. The first downloads a virus genome by taxon, specifying a host. The second downloads a SARS-CoV-2 protein by name, specifying a host and a custom output filename.

```bash
datasets download virus genome taxon sars-cov-2 --host dog
```

```bash
datasets download virus protein S --host dog --filename SARS2-spike-dog.zip
```

--------------------------------

### Get genome metadata by BioProject accession

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/how-tos/genomes/get-genome-metadata

Use `datasets summary genome accession PRJEB33226` to get metadata for genomes belonging to the Sanger 25 Genomes Project.

```bash
datasets summary genome accession PRJEB33226

```

--------------------------------

### Example protein FASTA defline

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/data-packages/genome

Example FASTA defline for a protein sequence in the genome data package. The defline includes the protein accession, protein name, isoform, and organism.

```fasta
>NP_000005.3 alpha-2-macroglobulin isoform a precursor [Homo sapiens]

```

--------------------------------

### datasets summary virus genome examples

Source: https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/command-line/datasets/summary/virus/genome

Two example invocations: one by accession (NC_045512.2) and one by taxon (sars-cov-2) with a host filter. The output is a JSON data report.

```bash
datasets summary virus genome accession NC_045512.2
	datasets summary virus genome taxon sars-cov-2 --host dog
```
Tokens:
Raw