XSEDE12

Tech: A Distributed Memory Out-of-Core Method and Its Application to Quantum Chemistry Applications

Abstract: Out-of-core methods, which repeatedly offload data to disk in order toovercome local on-node memory constraints are encountered in a rangeof scientific computing disciplines, including quantumchemistry. Unfortunately, these methods do not often map nicely ontoglobal parallel file systems employed on modern HPC clusters and canoverwhelm even the most capable of file systems causing unacceptablylow application performance (while also degrading I/O performance forall system users). To address this bottleneck and explore moreefficient use of HPC clusters for a quantum chemistry application,CFOUR, a new MPI-based utility has been developed to supportout-of-core methods on distributed memory systems. This MPI Ocore utility leverages the high-speed interconnect available onHPC clusters to offload and retrieve out-of-core records to one ormore remote memory storage pools, avoiding excessive I/O transactionson local or global file systems. In this paper, we present an overviewof the Ocore implementation, it's direct application within a largequantum chemistry application, and micro-benchmark and applicationperformance results from an HPC cluster interconnected withquad-data-rate InfiniBand.