Total Knowledge of I/O Through Holistic I/O Performance Analysis

Investigators: Suren Byna (LBNL), P. Carns (ANL)

Program Manager: Lucy Nowell

Scientific Accomplishment

We developed the Total Knowledge of I/O (TOKIO) framework to synthesize performance data from HPC I/O components into a holistic view of I/O behavior. The Unified Monitoring and Metrics Interface (UMAMI) to TOKIO, shown below, leverages this technology to explore application performance in its broader historical and platform context. UMAMI illustrates performance trends, quantifies variability, and reveals previously hidden interactions that can adversely affect the performance of data-intensive scientific applications.

Unified Monitoring and Metrics Interface (UMAMI) is a visualization technique for displaying performance data from many components of the I/O system in a normalized and coherent way. By plotting historic performance data of similar I/O motifs over time, we establish a statistical approach to identifying "bad" performance throughout the I/O subsystem.

Significance and Impact

TOKIO and UMAMI provide a holistic view of the entire I/O subsystem and reduce users' reliance on institutional I/O experts to understand and improve I/O performance and mitigate variability. These tools and methods provide context for the performance data generated on complex I/O subsystems in a generalized, portable fashion.

Research Details

• Ran four benchmark applications daily on both NERSC Edison and ALCF Mira to establish I/O variability on each system
• Analyzed variability using UMAMI diagrams to identify poor performance caused by bandwidth contention, metadata contention, and file system health issues
• Determined that architecture, application I/O pattern, and overall file system climate all can affect variation, but none represents a surefire way to predict performance.