[View PDF]

Chris Sewell, Jim Ahrens, LANL
Berk Geveci, Patrick O’Leary, Kitware Inc.

Objectives Disk I/O has become a significant bottleneck for large scale simulations and data analysis of simulation output In situ analysis aims to embed data analysis and visualization into the simulation to reduce the need for disk I/O Our objective is to demonstrate the feasibility of in situ analysis for large scale simulations Impact In situ analysis promises to reduce time to insight for many DOE applications As the gap between computational power and I/O capability increases, certain types of high fidelity analysis will be achievable only through in situ analysis Accomplishments - FY13 Released ParaView Catalyst, a general purpose in situ library that utilizes the computational engine from ParaView Coupled Catalyst with a number of DOE Office of Science codes including MPAS, VPIC and Albany as well as ASC and DoD codes Demonstrated the feasibility of general purpose in situ analysis through scalability studies

Notes:

From the Houston “Scientific Discovery at Exascale” report: Post-processing is currently the dominant processing paradigm for visualization and analysis on ASCR supercomputers (and other supercomputers): simulations write out files, and applications dedicated to visualization and analysis read these files and calculate results. However, supercomputers that have come on line recently are increasing memory and FLOPs more quickly than I/O bandwidth and capacity. In other words, the I/O capability is decreasing relative to the rest of the computer. It will be slow to write data to disk, there will not be enough space to store data, and it will be very slow to read the data back in. This trend hampers the traditional post-processing paradigm; it will force simulations to reduce the amount of data they write to disk and force visualization and analysis algorithms to reduce the amount of data they read from disk.”

The goal of in situ processing is to tightly couple analysis and visualization codes with simulation codes and run the together on supercomputers. This has the benefit that disk I/O is no longer necessary and data can be transferred in-memory from one code to another. There are some concerns about this approach however:

• Scalability of analysis and visualization codes, which will be essential to continued scalability of simulation codes due to the tight coupling,
• Memory usage of analysis and visualization codes given the limited total memory on compute nodes,
• Stability of analysis and visualization codes at scale

Kitware, Sandia and LANL has developed an in situ library based on DOE’s flagship visualization application ParaView. This library, called ParaView Catalyst, is a general purpose tool that can be embedded in many types of simulation codes for in situ analysis and visualization.

Under SciDAC SDAV, we have started integrating Catalyst with various codes important to ASCR. We have been investigating the issues described above and addressing them as needed. The figure on the slides shows how Catalyst, coupled with VPIC, scales well to 5K processors and has significantly less overhead than writing data to disk. In this case, we used ParaView Catalyst to generate iso-surfaces and render the result all in situ.