BBMerge-Auto

Script: bbmerge-auto.sh Package: jgi Class: BBMerge.java Type: Memory-adaptive wrapper

Memory-adaptive wrapper for BBMerge that automatically detects and uses all available memory instead of a fixed amount. Designed for Tadpole error-correction (ecct) and extension operations that require substantial memory.

Overview

bbmerge-auto.sh is a simple wrapper script that automatically calculates and allocates all available system memory for BBMerge operations. This is specifically designed for memory-intensive Tadpole features like error-correction and extension.

When to Use Each Tool

Use bbmerge.sh: For overlap-only merging with fixed memory allocation (1GB default)
Use bbmerge-auto.sh: For Tadpole operations (ecct, extend, extend2) that require more memory
Equivalent behavior: If you manually set memory with -Xmx flag, both scripts are identical

Memory Detection Algorithm

The script uses calcmem.sh to automatically detect available system memory:

Default allocation: 14GB minimum if detection fails
Detection method: Reads /proc/meminfo (MemFree, Cached, Buffers) and calculates available memory
Formula: ((available_memory - 500MB) × 84%) for heap allocation
SLURM support: Respects SLURM_MEM_PER_NODE environment variable
Manual override: If -Xmx is specified in arguments, automatic detection is bypassed

Parameters

bbmerge-auto.sh accepts all the same parameters as bbmerge.sh.

For complete parameter documentation, algorithm details, and usage examples, please see:

Complete Documentation

BBMerge Documentation: bbmerge.html - Complete parameter reference and algorithm details
BBMerge Guide: BBTools/docs/guides/BBMergeGuide.txt - Detailed usage guide with examples
Command help: Run bbmerge.sh without arguments for parameter list

Examples

Basic Auto-Memory Merging

bbmerge-auto.sh in=reads.fq out=merged.fq outu=unmerged.fq

Basic merging with automatic memory detection for better performance.

Error Correction with Tadpole

bbmerge-auto.sh in=reads.fq out=merged.fq ecct k=31

Error-correct reads with Tadpole before merging - requires substantial memory, automatically allocated.

Extension for Non-overlapping Reads

bbmerge-auto.sh in=reads.fq out=merged.fq extend2=20 iterations=5

Extend reads up to 20bp per iteration for up to 5 iterations - memory-intensive operation handled automatically.

Manual Memory Override

bbmerge-auto.sh -Xmx32g in=reads.fq out=merged.fq ecct

Override automatic detection to use exactly 32GB. Equivalent to running bbmerge.sh with same -Xmx setting.

Technical Implementation

The wrapper script performs these steps:

Check arguments: Scans command line for manual -Xmx flags
Memory calculation: If no manual override, calls calcmem.sh for automatic detection
BBMerge execution: Runs java -Xmx[detected]m -Xms[detected]m jgi.BBMerge [parameters]

The underlying algorithm, parameter handling, and all functionality is identical to BBMerge - only the memory allocation strategy differs.

Support

For questions about BBMerge parameters and usage:

Email: bbushnell@lbl.gov
Complete documentation: BBMerge documentation
Usage guide: BBTools/docs/guides/BBMergeGuide.txt