Data Elevator: Low-contention Data Movement in Hierarchical Storage System

Investigators: Suren Byna, Quincey Koziol, Bin Dong, John Wu and Prabhat

Program Manager: Lucy Nowell

Challenge

Efficient data movement in HPC storage with multiple levels of hierarchical and heterogeneous hardware is challenging, and software solutions to facilitate data movement in a deep storage hierarchy are still in infancy and are inefficient.

Our Solution

We developed Data Elevator to support low-contention data movement in hierarchical storage systems by offloading data movements to idle CPU cores.

Data Elevator supports efficient, transparent, and asynchronous data movement using HDF5 VOL without the need to modify application source code.

Outcome

• tested with scientific codes at scale
• Climate modeling (CAMR)
• Plasma physics (VPIC)
• 4X performance advantage over Cray DataWarp in moving data using the burst buffer on Cori
• HiPC 2016 paper

"Data Elevator abstraction is the right approach for optimizing our I/O on novel storage architectures, like Cori's burst buffers. It involved no code changes to our MPI + HDF5 approach on other platforms, and provides great performance without hurting our simulation throughput. We're looking forward to using this capability to improve our data pre-processing and indexing as well"
- Hans Johansen (Developer of CAMR - 1km high-res climate modeling code)