FuseByTaxa - BBTools Documentation

Basic Usage

fusebytaxa.sh in=<input> out=<output> floor=1m pad=10 maxlen=50m

FuseByTaxa processes taxonomically-sorted input sequences (FASTA or FASTQ, gzip supported) and outputs fused sequences for each taxonomy ID. The tool streams data without loading the entire file into memory, making it suitable for large genome databases.

Input Requirements

Input sequences must be sorted by taxonomy ID, with contigs of the same taxid adjacent to each other. If your input is not sorted, pre-process it with sortbyname.sh taxa=t. The tool extracts the taxonomy ID from sequence headers in the conventional format tid|TAXID|... — specifically, the text between the first and second pipe characters. Headers containing no pipe character are recorded under the ID "NA".

Important: Taxonomic Sorting Requirement

Input MUST be sorted by taxonomy ID (same taxid contigs adjacent)
Use sortbyname.sh taxa=t input.fa output.fa to sort if needed
Tool streams input and cannot buffer, so it processes by looking at adjacent sequences
The sort order is not checked. A taxid that reappears after a different taxid starts a new taxon with its chunk counter reset to 0, so the same tid can appear in more than one header (for example two tid|X|fused0 records), and it is counted again in the taxa kept/dropped totals.

Output Format

Output sequences are FASTA format with headers: >tid|TAXID|fusedCHUNK len=LENGTH. Each taxonomy ID may generate multiple sequences if the fused length exceeds maxlen, creating numbered chunks (fused0, fused1, etc.).

Common Use Cases

Processing RefSeq Clade Files

Create assembly-attempt sequences from a combined RefSeq clade file containing rRNA markers and partial assemblies:

sortbyname.sh taxa=t refseq_clade.fna.gz sorted.fna.gz
fusebytaxa.sh in=sorted.fna.gz out=genomes.fna.gz floor=1m pad=10 maxlen=50m

This produces genome-scale sequences suitable for contig-classification benchmarking, with marker-only taxa automatically dropped.

Processing Viral Sequences

Viruses are typically small, so set floor=0 to emit all fused sequences:

fusebytaxa.sh in=sorted_viruses.fna.gz out=viral_genomes.fna.gz floor=0 pad=10 maxlen=50m

This preserves viral sequences even if they don't meet a minimum size threshold.

Adjusting Padding and Maximum Chunk Size

When fusing contigs with minimal spacing and limiting genome size:

fusebytaxa.sh in=sorted.fna.gz out=genomes.fna.gz floor=1m pad=5 maxlen=100m

This uses 5 Ns between contigs (instead of 10) and allows fused sequences up to 100 Mb before splitting into chunks.

Parameters

FuseByTaxa parameters control input/output, sequence filtering, and chunking behavior. The tool uses a streaming architecture with minimal memory footprint regardless of input size.

Input/Output Parameters

in=<file>: Input sequences (FASTA or FASTQ, may be gzip-compressed). Required. Must be sorted by taxonomy ID with adjacent contigs sharing the same taxid.
out=<file>: Output fused sequences in FASTA format. Required. Output sequences have headers: >tid|TAXID|fusedCHUNK len=LENGTH
overwrite=true: Overwrite the output file if it already exists. Default: true. The shared Parser default is true and FuseByTaxa uses it, so an existing out= file is replaced without a prompt. Set overwrite=f (or ow=f) to refuse overwriting; the run then fails with "File ... exists and overwrite=false" as an AssertionError (the launcher passes -ea by default).

Sequence Filtering Parameters

floor=1m (alias: minlen): Minimum length (bases) of each fused sequence to emit. Fused sequences shorter than this threshold are not written. The check is applied to each output sequence, so a taxon is counted as dropped only if none of its sequences reaches the floor. Accepts integer values and K/M/G suffixes (e.g., 1m = 1,000,000 bp). Set to 0 to emit every non-empty fused sequence (e.g., for viral datasets where small sequences are meaningful). Default: 1m (1,000,000 bp).

Padding and Chunking Parameters

pad=10 (alias: npad): Number of 'N' characters inserted between consecutive contigs when fusing sequences from the same taxonomy ID. This padding helps distinguish original contig boundaries in the fused sequence. No padding is added before the first contig of a sequence or between chunks, so chunk boundaries contain no Ns. Default: 10 Ns.
maxlen=50m: Maximum length (bases) for a fused sequence before a new chunk is started. A chunk is closed when adding the next contig and its padding would exceed this length, and the new chunk takes the next number (fused0, fused1, etc.). Individual contigs are never split, so a single contig longer than maxlen is written as its own sequence that exceeds maxlen (for example, a 270 bp contig with maxlen=250 is emitted with len=270). Accepts integer values and K/M/G suffixes (e.g., 50m = 50,000,000 bp). Must be >= floor. Default: 50m (50,000,000 bp).

Other Parameters

verbose=false: Verbose flag. The help text describes it as printing verbose status messages, but in the current FuseByTaxa.java source the flag is parsed and stored only, and no additional output is produced. Default: false.
-Xmx<mem>: Set maximum Java heap memory (e.g., -Xmx2g for 2 GB). Default: 2g. FuseByTaxa is streaming and requires minimal memory regardless of input size.

Output Statistics

FuseByTaxa prints statistics to standard error after processing:

Time and Processing Rate: Total time, reads per second, bases per second
Taxa Kept: Number of taxonomy IDs with fused sequences >= floor
Taxa Dropped: Number of taxonomy IDs for which no fused sequence reached the floor (printed as "Taxa dropped: N (below floor=F)")
Sequences Out: Total output sequences and total bases

Algorithm Details

Streaming Architecture

FuseByTaxa processes sequences in a single streaming pass, reading from input and writing to output sequentially. It maintains only the current fused chunk in memory via a ByteBuilder, making it suitable for processing gigabyte-scale datasets with minimal heap usage. Input must remain taxonomically sorted for this streaming approach to work correctly.

Taxonomy ID Extraction

Taxonomy IDs are extracted from sequence headers by finding the first pipe character (|) and taking the content up to the next pipe or the end of the string. This matches the conventional tid|TAXID|... header format, but note the tid prefix itself is not validated — any pipe-delimited header yields the field after its first pipe. Headers with no pipe character at all are recorded under the ID "NA".

Fusing and Chunking Logic

When the current fused sequence length plus padding plus the next contig length would exceed maxlen, the current chunk is closed and a new chunk begins. Padding is added only when the chunk is non-empty, so a new chunk starts with the contig itself. Contigs are never split, so a single contig longer than maxlen becomes a chunk on its own and exceeds maxlen. A closed chunk is written only if it reaches floor, but the chunk counter advances whether or not it was written, so a dropped chunk leaves a gap in the numbering (the first written sequence for a taxon can be fused1, for example). When a new taxonomy ID is encountered, the open chunk for the previous taxid is closed the same way, and the sequence counter resets to 0 for the new taxid.

Filtering

Sequences are emitted only if their length (in bases, after padding) is >= floor. Taxa producing no output sequences are counted as "dropped". The tool automatically distinguishes between taxa that produced output and those below the threshold.

Examples

Example 1: Basic Genome Fusion from RefSeq

Processing an archaeal RefSeq file with mixed rRNA and assembly data:

sortbyname.sh taxa=t refseq_archaea.fna.gz sorted_archaea.fna.gz
fusebytaxa.sh in=sorted_archaea.fna.gz out=archaea_genomes.fna.gz floor=2m pad=10 maxlen=50m

This sorts archaea sequences by taxonomy ID, then fuses contigs for each species into genome-scale sequences. Only species with >= 2 Mb of sequence data are retained.

Example 2: Strict Filtering for High-Quality Genomes

Creating a high-quality dataset with only well-assembled taxa:

fusebytaxa.sh in=sorted_bacteria.fna.gz out=quality_genomes.fna.gz floor=5m pad=20 maxlen=100m

Uses a 5 Mb floor to retain only taxa with substantial sequence content, 20 Ns spacing for clear contig distinction, and allows genomes up to 100 Mb.

Example 3: Viral Dataset Processing

Processing sorted viral sequences where size filtering is inappropriate:

fusebytaxa.sh in=sorted_viruses.fna floor=0 pad=5 maxlen=20m out=viral_genomes.fna

Emits all viral fused sequences regardless of size (floor=0), uses minimal padding (5 Ns), and limits chunk size to 20 Mb for manageability.

Example 4: Custom Memory Settings

For systems with limited memory or large datasets:

fusebytaxa.sh -Xmx500m in=large_dataset.fna.gz out=fused.fna.gz floor=1m

Explicitly sets Java heap to 500 MB. Because FuseByTaxa is streaming, even large input files process with minimal memory.

Workflow Integration

Part of Contig-Classification Benchmarking

FuseByTaxa is designed as a preprocessing step for contig-classification benchmarks. The typical workflow is:

Download RefSeq clade data (mixed rRNA markers and full assemblies)
Sort by taxonomy ID: sortbyname.sh taxa=t
Fuse into genome-scale sequences: fusebytaxa.sh
Generate synthetic reads or use for alignment benchmarking

Complementary Tools

FuseByTaxa works with other BBTools:

sortbyname.sh: Pre-sort input by taxonomy ID (required)
fakeReads.sh: Generate synthetic reads from fused genomes
bbmap.sh: Align reads to fused genomes for benchmarking
filterbyname.sh: Filter sequences by name/ID after fusing

Support

For questions and support: