FetchGenomes

Script: fetchgenomes.sh Package: prok Class: FetchGenomes.java

Selects a phylogenetically diverse set of RefSeq genomes from NCBI assembly_summary.txt files and writes a hardened download script (curl with retries, gzip integrity checks, fna+gff pairing, _tid_ header renaming). Replacement for fetchproks.sh; no FTP crawling — selection happens locally from the summary table.

Basic Usage

fetchgenomes.sh summary=<file[,file2]> out=<script.sh> [options]

First download the NCBI assembly summary table(s), for example:

curl -O https://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt
curl -o archaea_summary.txt https://ftp.ncbi.nlm.nih.gov/genomes/refseq/archaea/assembly_summary.txt

FetchGenomes reads the summary table(s), selects a diverse subset of assemblies under per-species/genus/family caps, and writes out a self-contained download script — it does not download anything itself.

Parameters

Parameters

summary=<file>
Comma-list of assembly_summary.txt files. Required.
out=<file>
Output download script. Required.
tree=f
Load a TaxTree for lineage-aware quotas. Default is f (disabled). Set tree=t, tree=auto, or tree=<path> to enable it; treefile=<path> selects the source without changing this flag.
usetree=f
Alias for tree=f.
maxperspecies=1
Maximum assemblies kept per species, best-ranked first.
maxpergenus=2
Maximum species kept per genus. 0 means unlimited.
maxperfamily=0
Maximum species kept per family. 0 means unlimited.
skip=<file>
One taxid per line to exclude, e.g. an existing collection you don't want to re-select.
allowexcluded=f
Keep assemblies flagged excluded_from_refseq.
rename=t
Add a _tid_<taxid> suffix to sequence headers in the generated download script.
minsize=0
Minimum genome_size, applied when that column is present in the summary table.

Examples

Basic Diverse Selection

fetchgenomes.sh summary=bacteria_summary.txt out=download_bacteria.sh

Selects one assembly per species (up to 2 species per genus) from the bacteria summary table and writes a hardened download script.

Combining Multiple Summary Tables

fetchgenomes.sh summary=bacteria_summary.txt,archaea_summary.txt out=download_proks.sh maxpergenus=3

Combines two domains' summary tables and allows up to 3 species per genus.

Lineage-Aware Quotas, Excluding a Prior Collection

fetchgenomes.sh summary=bacteria_summary.txt out=download_new.sh tree=auto skip=existing_taxids.txt maxperfamily=10

Loads a TaxTree for lineage-aware quotas, excludes taxids already present in an existing collection, and caps selection at 10 species per family.

Support

For questions and support: