SplitNextera

Important Usage Notes

Recommended Workflow

Brian Bushnell recommends this two-step workflow for processing Nextera LMP libraries:

bbduk.sh in=reads.fq out=trimmed.fq ref=adapters.fa ktrim=r k=23 mink=11 hdist=1 tpe tbo

splitnextera.sh in=trimmed.fq out=lmp.fq outf=fragments.fq outu=unknown.fq outs=singletons.fq mask

This produces four essential output files:

• lmp.fq: Long-mate pairs with proper LMP orientation
• fragments.fq: Fragment pairs (short pairs) with standard orientation
• unknown.fq: Pairs where linkers were not found (may be long-mate pairs)
• singletons.fq: Individual reads that lost their pairs during processing

Basic Usage

splitnextera.sh in=<file> out=<file> outf=<file> outu=<file> outs=<file>

For pairs in two files, use in1, in2, out1, out2, etc.

Processing Approaches

SplitNextera offers two approaches for junction detection, with different performance characteristics:

splitnextera.sh in=trimmed.fq out=lmp.fq outf=fragments.fq outu=unknown.fq outs=singletons.fq mask=t

# Step 1: Pre-mask junctions with BBDuk
bbduk.sh in=trimmed.fq out=stdout.fq ktmask=J k=19 hdist=1 mink=11 hdist2=0 literal=CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG | \
# Step 2: Split using pre-masked junctions
splitnextera.sh in=stdin.fq out=lmp.fq outf=fragments.fq outu=unknown.fq outs=singletons.fq

Parameters

Parameters are organized by their function in the splitting process, matching the exact organization from the shell script.

Examples

# Step 1: Remove standard adapters
bbduk.sh in=nextera_lmp_raw.fq out=adapter_trimmed.fq ref=adapters.fa ktrim=r k=23 mink=11 hdist=1 tpe tbo

# Step 2: Split library by junction orientation
splitnextera.sh in=adapter_trimmed.fq out=lmp.fq outf=fragments.fq outu=unknown.fq outs=singletons.fq mask=t

# Method 1: Pre-mask adapters with BBDuk for maximum speed
bbduk.sh in=adapter_trimmed.fq out=junction_masked.fq ktmask=J k=19 hdist=1 mink=11 hdist2=0 literal=CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG
splitnextera.sh in=junction_masked.fq out=lmp.fq outf=frag.fq outu=unknown.fq outs=singleton.fq

# Method 2: Pipeline approach (equivalent, slightly faster)
bbduk.sh in=adapter_trimmed.fq out=stdout.fq ktmask=J k=19 hdist=1 mink=11 hdist2=0 literal=CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG | \
splitnextera.sh in=stdin.fq out=lmp.fq outf=frag.fq outu=unknown.fq outs=singleton.fq

# Process paired-end Nextera LMP data with comprehensive output
splitnextera.sh in1=trimmed_R1.fq in2=trimmed_R2.fq \
                out1=lmp_R1.fq out2=lmp_R2.fq \
                outf1=frag_R1.fq outf2=frag_R2.fq \
                outu1=unk_R1.fq outu2=unk_R2.fq \
                outs=singleton.fq \
                mask=t innerlmp=t merge=t testmerge=0.1

# Apply stricter length filtering and generate processing statistics
splitnextera.sh in=library.fq out=lmp.fq outf=frag.fq outu=unk.fq outs=single.fq \
                mask=t minlength=60 stats=split_stats.txt

Algorithm Details

Nextera LMP Library Architecture

Nextera LMP libraries are created through a specialized process that circularizes DNA fragments, incorporates junction sequences, and then linearizes them to create long-distance mate pairs. The junction sequence CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG serves as the critical marker that distinguishes different read pair orientations and enables proper library demultiplexing.

Junction Detection Strategy

SplitNextera implements a dual-mode junction detection system optimized for different performance requirements:

• Symbol-based mode (default): Searches for pre-masked junction symbols (default 'J') using Tools.indexOf() and Tools.lastIndexOf() to identify adapter boundaries. This mode assumes junctions have been previously marked by BBDuk or similar tools.
• Integrated masking mode (mask=t): Constructs kmer hash tables using AbstractKmerTable.ARRAY1D with TableLoaderLockFree and 12 hash functions at 0.9 load factor. The standard Nextera junction sequence is processed into overlapping kmers (k=19 default, mink=11) with configurable distance tolerances (hdist=1, hdist2=0, edist=0). TableReader.kMask() performs real-time kmer lookup with reverse-complement matching when rcomp=t.

Read Pair Classification Logic

Each read pair undergoes systematic classification based on junction positions stored in Read.start and Read.stop coordinates:

• Long Mate Pairs (LMP): Generated when outer segments meet length requirements (minlength=40 default). Primary LMP pairs use r1left (positions 0 to r1.start) paired with r2right (positions 0 to r2.start). When innerlmp=t and both reads contain junctions, inner LMP pairs are created from r1right and r2left segments.
• Fragment Pairs: Created from inner segments between junctions (r1right from r1.stop+1 to r1.length, r2left from r2.stop+1 to r2.length) or complementary left/right fragments when segments exceed minimum length thresholds.
• Unknown Pairs: Read pairs where Tools.indexOf() returns -1 for both reads, indicating absence of detectable junction sequences. These may represent valid LMP pairs with undetectable junctions due to sequencing errors or fragment pairs without junction incorporation.
• Singletons: Individual reads generated via Read.setPairnum(0) when paired segments fail length filtering or when asymmetric junction detection prevents proper pairing.

Overlap Merging Integration

The optional merging functionality provides enhanced junction detection through sequence overlap analysis. When merge=t, BBMerge.findOverlapStrict() identifies exact overlaps between paired reads before junction processing. The testmerge parameter enables adaptive behavior: if not reading from stdin, BBMerge.mergeableFraction() samples up to 1 million read pairs to measure overlap rate, enabling merging only when the rate exceeds the hardcoded 10% threshold. Successfully merged reads use Read.joinRead() to create single sequences, with Read.reverseComplement() operations maintaining proper orientation. Junction detection then proceeds on the merged sequence, potentially improving detection accuracy for short insert libraries.

Memory Architecture and Performance

SplitNextera uses concurrent I/O architecture with ConcurrentReadInputStream and ConcurrentReadOutputStream systems employing buffer size 4 for parallel processing. Read processing occurs through ListNum batch structures, enabling simultaneous read/write operations across four output streams (LMP, fragment, unknown, singleton). The base memory requirement is approximately 200MB (default -Xmx200m), scaling linearly when mask=t to accommodate kmer hash tables at roughly 20 bytes per unique kmer. For optimal performance, Brian recommends pre-masking with BBDuk using identical parameters (k=19, mink=11, hdist=1, ktmask=J) rather than built-in masking, as this eliminates redundant TableLoaderLockFree overhead and table construction during runtime.

Statistical Reporting and Quality Control

Comprehensive processing statistics track junction detection efficiency and read fate distribution. The system maintains counters for junctionsSought (total read pairs processed), junctionsDetected (pairs with identified junctions), and category-specific read/base counts (readsLmp, readsFrag, readsUnk, readsSingle). Statistical output uses TextStreamWriter to generate detailed reports including junction detection rate (junctionsDetected/junctionsSought × 100%), read distribution percentages calculated with pairedInput multipliers (100.0 for paired, 50.0 for single-ended), and base recovery metrics. When merging is enabled, additional statistics track mergedReadCount and mergedBaseCount with percentage calculations. All metrics are formatted using Tools.format() for consistent decimal precision and written to configurable output streams (default: stderr).

Output Categories

Long Mate Pairs (LMP)

These represent the primary product of successful Nextera LMP library preparation. LMP pairs have the expected long-distance orientation where the outer segments of junction-containing reads are paired together, creating mate pairs with insert distances much larger than the original DNA fragments. These pairs are essential for applications requiring long-range connectivity information such as scaffolding, structural variant detection, and complex genome assembly.

Fragment Pairs

Standard paired-end fragments that result from incomplete adapter ligation, failed circularization, or other library preparation artifacts. These pairs have normal fragment insert sizes (typically 300-800bp) rather than the long mate pair distances (2-40kb+). While not the intended product of LMP preparation, fragment pairs retain standard paired-end utility for mapping and variant calling applications.

Unknown Pairs

Read pairs where junction sequences were not detected despite the presence of potential LMP structure. This category captures several scenarios: LMP pairs where junction detection failed due to sequencing errors or base quality issues, fragments that lack incorporated junction sequences, or cases where the junction sequence has been modified during library preparation. Unknown pairs typically represent approximately 15-30% of LMP libraries and require careful evaluation to determine their ultimate classification.

Singletons

Individual reads that lost their mate during the splitting process. Common causes include asymmetric junction detection (junction found in only one read of a pair), size filtering that removes segments below the minimum length threshold, or quality issues that prevent proper junction identification. Singletons can sometimes be rescued through secondary analysis or merged with other singleton populations for single-end applications.

Troubleshooting

Low Junction Detection Rate

If junction detection rates are below 60%, consider:

• Verifying that the input library is indeed Nextera LMP (not standard Nextera)
• Checking adapter trimming quality - excessive trimming may remove junction sequences
• Adjusting junction detection parameters (k, mink, hdist values) for your specific library
• Examining raw reads for the presence of the expected junction sequence

High Unknown Percentage

Unknown pairs exceeding 40% may indicate:

• Library preparation issues affecting junction incorporation
• Sequencing quality problems in junction regions
• Need for more permissive matching parameters (increased hdist or edist values)
• Presence of non-LMP fragments in the library preparation

Performance Optimization

For maximum throughput with large datasets:

• Use the two-step BBDuk pre-masking approach rather than integrated masking
• Process libraries in streaming mode using stdin/stdout to minimize I/O overhead
• Allocate sufficient memory (typically 4-8GB) when processing very large kmer sets with mask=t
• Consider parallel processing of multiple libraries rather than single-threaded sequential processing

Support

For questions and support:

• Email: bbushnell@lbl.gov
• Documentation: bbmap.org
• Guide: Read bbtools/docs/guides/SplitNexteraGuide.txt for more information