Title: [YVO07] Performance Characterization and Optimization of Parallel I/O on the Cray XT
Year: 2007
Location: 22nd IEEE International Parallel and Distributed Processing Symposium (IPDPS08), 2008. Miami, FL.
Author(s): Weikuan Yu, Jeffrey S. Vetter, H. Sarp Oral

This paper presents an extensive characterization, tuning, and optimization of parallel I/O on the Cray XT supercomputer, named Jaguar, at Oak Ridge National Laboratory. We have characterized the performance and scalability for different levels of storage hierarchy including a single Lustre object storage target, a single S2A storage couplet, and the entire system. Our analysis covers both data- and metadata-intensive I/O patterns. In particular, for small, non-contiguous data intensive I/O on Jaguar, we have evaluated several parallel I/O techniques, such as data sieving and two phase collective I/O, and shed light on their effectiveness. Based on our characterization, we have demonstrated that it is possible, and often prudent, to improve the I/O performance of scientific benchmarks and applications by tuning and optimizing I/O. For example, we demonstrate that the I/O performance of the S3D combustion application can be improved at large scale by tuning the I/O system to avoid a bandwidth degradation of 49% with 8192 processes when compared to 4096 processes. We have also shown that the performance of Flash I/O can be improved by 34% by tuning the collective I/O parameters carefully.


You are here:   SDMCenter/Publications/YVO07
Last Updated r1 on 2009-04-07 15:51:19 by Asim