MergeSam2

Script: mergesam2.sh Package: jgi Class: MergeSam2.java

Merges multiple SAM or BAM files into one, keeping only the header from the first file. Unlike the original MergeSam, which operated on plain-text SAM by concatenating bytes, MergeSam2 reads each input through a native BGZF/BAM decoder and writes through a proper BAM writer — so both BAM inputs and BAM outputs are handled correctly. This is a concatenation, not a coordinate-sorted merge; the output header is marked SO:unsorted regardless of what the inputs claimed.

Basic Usage

mergesam2.sh <files> out=<file>
   or:  mergesam2.sh in=a.bam,b.bam,c.bam out=merged.bam

Input files may be specified positionally on the command line (space-separated) or via the in= flag as a comma-delimited list. Output format is inferred from the file extension: use .bam for BAM output, .sam for plain SAM, or stdout.sam / stdout.bam to stream to standard output.

MergeSam2 vs MergeSam

  • MergeSam2 (this tool): Uses native BGZF decoding and encoding. Handles BAM input and BAM output correctly.
  • MergeSam (legacy): Concatenates raw bytes from text SAM files. Does not handle BAM format.
  • Use MergeSam2 whenever any input or the desired output is BAM. MergeSam is only appropriate for plain-text SAM files.

Memory Requirements

MergeSam2 is a streaming tool and uses very little memory — the default heap size is 400 MB, which is sufficient for virtually any input. Compression and decompression of BGZF blocks runs in Streamer and Writer thread pools, so throughput scales with available CPU cores.

Header Handling

The output header is taken verbatim from the first input file, with one modification: the SO: field on the @HD line is forced to unsorted. If the @HD line is missing entirely, a minimal @HD VN:1.6 SO:unsorted line is synthesized. All inputs are assumed to share the same reference dictionary (@SQ lines).

Parameters

MergeSam2 is intentionally minimal. The heavy lifting (BGZF I/O, threading) is handled transparently by the Streamer and Writer infrastructure.

Input parameters

in=<file,file,...>
Comma-delimited list of input SAM or BAM files. May be specified multiple times (each in= flag can carry a comma-delimited list). Positional arguments (files that exist on disk without an = sign) are also accepted, matching the legacy mergesam.sh usage pattern.

Output parameters

out=stdout.sam
Output file. Extension determines format: .sam for plain SAM, .bam for BAM. Default is stdout.sam (stream SAM to standard output).
overwrite=t
(ow) Allow overwriting an existing output file. Set to f to abort if the output file already exists.
append=f
Append to an existing output file instead of overwriting it.

Processing parameters

verbose=f
Print a diagnostic message to stderr each time a new input Streamer is started. Useful for monitoring progress when merging many files.

Java Parameters

-Xmx
Override Java heap size. The default (400 MB) is sufficient for MergeSam2 in nearly all cases, since records are streamed rather than loaded into memory. Example: -Xmx1g.
-eoom
Exit if an out-of-memory exception occurs. Requires Java 8u92+.
-da
Disable assertions.

Examples

Merge BAM files from multiple lanes

# Positional syntax — files separated by spaces
mergesam2.sh lane1.bam lane2.bam lane3.bam out=merged.bam

# Equivalent in= syntax
mergesam2.sh in=lane1.bam,lane2.bam,lane3.bam out=merged.bam

Both forms are equivalent. Use positional syntax for convenience when listing files manually; use in= when constructing the command programmatically or passing comma-delimited shell variables.

Merge SAM files and write plain SAM output

mergesam2.sh in=sample_A.sam,sample_B.sam out=combined.sam

Output format is determined by the out= extension. SAM input can be freely mixed with BAM input — the Streamer handles both transparently.

Stream merged BAM to standard output for piping

# Pipe merged BAM into samtools sort
mergesam2.sh in=part1.bam,part2.bam out=stdout.bam | samtools sort -o sorted.bam

# Or stream as SAM for text processing
mergesam2.sh in=part1.bam,part2.bam out=stdout.sam | grep -v "^@" | wc -l

Use stdout.bam to stream compressed BAM or stdout.sam to stream plain SAM text. This allows MergeSam2 to act as a source in a shell pipeline without writing an intermediate file.

Algorithm Details

Concatenation, Not Coordinate Sort

MergeSam2 performs a simple ordered concatenation: records from the first input file are written first, then all records from the second, and so on. No sorting or re-ordering takes place. If the inputs are individually coordinate-sorted, the output will not be coordinate-sorted unless the reference regions covered by each file happen to be disjoint and in order. The SO:unsorted header tag correctly communicates this to downstream tools.

Threading Architecture

Each input file is processed in series by a single drain thread that pulls batches of SamLine objects from the file's Streamer. The Streamer itself uses a thread pool for BGZF block decompression in parallel with reading. A single shared Writer handles BAM block compression for the output. Batch IDs are assigned globally and monotonically increasing across all input files, satisfying the ordering contract of the Writer even when inputs are processed sequentially.

BAM Passthrough

Records are passed through as raw SamLine objects without reconstructing reads from CIGAR or MD tags. No reference sequence is needed. Base quality scores, flags, tags, and all other fields are preserved exactly as they appear in the input.

Related Tools

Support

For questions and support: