As simulations approach petascale and beyond, there is a clear need to analyze and reduce the data produced before writing it to storage. Automatic analysis enables such data triage and reduction to prepare it for subsequent I/O and in-depth analysis and visualization. Built on existing data mining and analysis techniques, we research, develop, and deploy application-driven, architecture-aware techniques for performing in situ data analysis, filtering, and reduction to help minimize and optimize downstream I/O tasks while enabling a flexible post-processing analysis. This results in a collection of scalable libraries to be adopted by the community at large. In our previous SciDAC work, we successfully transitioned data mining and analysis techniques into tools like VisIt and ParaView while parallelizing several key operations. In SciDAC-3 we continue to deploy new capabilities using a flexible software infrastructure consisting of several lightweight libraries. Our design and implementation exploits both advanced multi-core and many-core architectures. Overall, this constitutes an in situ data analysis framework that is adaptable to a large variety of data representations, such as regular grids, block-regular structures, AMR models, or unstructured meshes and ultimately respond directly to the needs of the users.

Leaders:

This email address is being protected from spambots. You need JavaScript enabled to view it. (Utah), This email address is being protected from spambots. You need JavaScript enabled to view it. (UCD)

Team Members:

This email address is being protected from spambots. You need JavaScript enabled to view it., LBNL

This email address is being protected from spambots. You need JavaScript enabled to view it., U. Utah

This email address is being protected from spambots. You need JavaScript enabled to view it., LBNL

This email address is being protected from spambots. You need JavaScript enabled to view it., NWU

This email address is being protected from spambots. You need JavaScript enabled to view it., U. Utah

This email address is being protected from spambots. You need JavaScript enabled to view it., UC Davis

This email address is being protected from spambots. You need JavaScript enabled to view it., ANL

This email address is being protected from spambots. You need JavaScript enabled to view it., NCSU

This email address is being protected from spambots. You need JavaScript enabled to view it., ORNL

This email address is being protected from spambots. You need JavaScript enabled to view it., LBNL

Projects:

Statistical and Data Mining Techniques

Importance-Driven Analysis Techniques

Topological Methods

Vector Field Analysis

Feature-Driven Data Exploration