FuseByTaxa

Basic Usage

fusebytaxa.sh in=<input> out=<output> floor=1m pad=10 maxlen=50m

FuseByTaxa processes taxonomically-sorted input sequences (FASTA or FASTQ, gzip supported) and outputs fused sequences for each taxonomy ID. The tool streams data without loading the entire file into memory, making it suitable for large genome databases.

Input Requirements

Input sequences must be sorted by taxonomy ID, with contigs of the same taxid adjacent to each other. If your input is not sorted, pre-process it with sortbyname.sh taxa=t. The tool extracts the taxonomy ID from sequence headers in the conventional format tid|TAXID|... — specifically, the text between the first and second pipe characters. Headers containing no pipe character are recorded under the ID "NA".

Output Format

Output sequences are FASTA format with headers: >tid|TAXID|fusedCHUNK len=LENGTH. Each taxonomy ID may generate multiple sequences if the fused length exceeds maxlen, creating numbered chunks (fused0, fused1, etc.).

Common Use Cases

Processing RefSeq Clade Files

Create assembly-attempt sequences from a combined RefSeq clade file containing rRNA markers and partial assemblies:

sortbyname.sh taxa=t refseq_clade.fna.gz sorted.fna.gz
fusebytaxa.sh in=sorted.fna.gz out=genomes.fna.gz floor=1m pad=10 maxlen=50m

This produces genome-scale sequences suitable for contig-classification benchmarking, with marker-only taxa automatically dropped.

Processing Viral Sequences

Viruses are typically small, so set floor=0 to emit all fused sequences:

fusebytaxa.sh in=sorted_viruses.fna.gz out=viral_genomes.fna.gz floor=0 pad=10 maxlen=50m

This preserves viral sequences even if they don't meet a minimum size threshold.

Adjusting Padding and Maximum Chunk Size

When fusing contigs with minimal spacing and limiting genome size:

fusebytaxa.sh in=sorted.fna.gz out=genomes.fna.gz floor=1m pad=5 maxlen=100m

This uses 5 Ns between contigs (instead of 10) and allows fused sequences up to 100 Mb before splitting into chunks.

Parameters

FuseByTaxa parameters control input/output, sequence filtering, and chunking behavior. The tool uses a streaming architecture with minimal memory footprint regardless of input size.

Output Statistics

FuseByTaxa prints statistics to standard error after processing:

• Time and Processing Rate: Total time, reads per second, bases per second
• Taxa Kept: Number of taxonomy IDs with fused sequences >= floor
• Taxa Dropped: Number of taxonomy IDs below the floor threshold
• Sequences Out: Total output sequences and total bases

Algorithm Details

Streaming Architecture

FuseByTaxa processes sequences in a single streaming pass, reading from input and writing to output sequentially. It maintains only the current fused chunk in memory via a ByteBuilder, making it suitable for processing gigabyte-scale datasets with minimal heap usage. Input must remain taxonomically sorted for this streaming approach to work correctly.

Taxonomy ID Extraction

Taxonomy IDs are extracted from sequence headers by finding the first pipe character (|) and taking the content up to the next pipe or the end of the string. This matches the conventional tid|TAXID|... header format, but note the tid prefix itself is not validated — any pipe-delimited header yields the field after its first pipe. Headers with no pipe character at all are recorded under the ID "NA".

Fusing and Chunking Logic

When the current fused sequence length plus padding plus the next contig length would exceed maxlen, the current chunk is flushed (if >= floor) and a new chunk begins. Taxa exceeding maxlen thus produce multiple output sequences with numbered suffixes. When a new taxonomy ID is encountered, any accumulated sequences for the previous taxid are flushed, and the sequence counter resets to 0 for the new taxid.

Filtering

Sequences are emitted only if their length (in bases, after padding) is >= floor. Taxa producing no output sequences are counted as "dropped". The tool automatically distinguishes between taxa that produced output and those below the threshold.

Examples

Example 1: Basic Genome Fusion from RefSeq

Processing a RefSeq file with ~5000 species and mixed rRNA/assembly data:

sortbyname.sh taxa=t refseq_archaea.fna.gz sorted_archaea.fna.gz
fusebytaxa.sh in=sorted_archaea.fna.gz out=archaea_genomes.fna.gz floor=2m pad=10 maxlen=50m

This sorts archaea sequences by taxonomy ID, then fuses contigs for each species into genome-scale sequences. Only species with >= 2 Mb of sequence data are retained.

Example 2: Strict Filtering for High-Quality Genomes

Creating a high-quality dataset with only well-assembled taxa:

fusebytaxa.sh in=sorted_bacteria.fna.gz out=quality_genomes.fna.gz floor=5m pad=20 maxlen=100m

Uses a 5 Mb floor to retain only taxa with substantial sequence content, 20 Ns spacing for clear contig distinction, and allows genomes up to 100 Mb.

Example 3: Viral Dataset Processing

Processing sorted viral sequences where size filtering is inappropriate:

fusebytaxa.sh in=sorted_viruses.fna floor=0 pad=5 maxlen=20m out=viral_genomes.fna

Emits all viral fused sequences regardless of size (floor=0), uses minimal padding (5 Ns), and limits chunk size to 20 Mb for manageability.

Example 4: Custom Memory Settings

For systems with limited memory or large datasets:

fusebytaxa.sh -Xmx500m in=large_dataset.fna.gz out=fused.fna.gz floor=1m

Explicitly sets Java heap to 500 MB. Because FuseByTaxa is streaming, even large input files process with minimal memory.

Workflow Integration

Part of Contig-Classification Benchmarking

FuseByTaxa is designed as a preprocessing step for contig-classification benchmarks. The typical workflow is:

1. Download RefSeq clade data (mixed rRNA markers and full assemblies)
2. Sort by taxonomy ID: sortbyname.sh taxa=t
3. Fuse into genome-scale sequences: fusebytaxa.sh
4. Generate synthetic reads or use for alignment benchmarking

Complementary Tools

FuseByTaxa works with other BBTools:

• sortbyname.sh: Pre-sort input by taxonomy ID (required)
• fakeReads.sh: Generate synthetic reads from fused genomes
• bbmap.sh: Align reads to fused genomes for benchmarking
• filterbyname.sh: Filter sequences by name/ID after fusing

Support

For questions and support:

• Email: bbushnell@lbl.gov
• Documentation: bbmap.org
• GitHub: github.com/bbushnell/BBTools