People Publications Projects

Total Knowledge of I/O Through Holistic I/O Performance Analysis

Investigators: Suren Byna (LBNL), P. Carns (ANL)

Program Manager: Lucy Nowell

Scientific Accomplishment

We developed the Total Knowledge of I/O (TOKIO) framework to synthesize performance data from HPC I/O components into a holistic view of I/O behavior. The Unified Monitoring and Metrics Interface (UMAMI) to TOKIO, shown below, leverages this technology to explore application performance in its broader historical and platform context. UMAMI illustrates performance trends, quantifies variability, and reveals previously hidden interactions that can adversely affect the performance of data-intensive scientific applications.

Unified Monitoring and Metrics Interface (UMAMI) is a visualization technique for displaying performance data from many components of the I/O system in a normalized and coherent way. By plotting historic performance data of similar I/O motifs over time, we establish a statistical approach to identifying "bad" performance throughout the I/O subsystem.

Significance and Impact

TOKIO and UMAMI provide a holistic view of the entire I/O subsystem and reduce users' reliance on institutional I/O experts to understand and improve I/O performance and mitigate variability. These tools and methods provide context for the performance data generated on complex I/O subsystems in a generalized, portable fashion.

Research Details