International Collaboration Framework for Extreme Scale Experiments (ICEE)

Large-scale scientific exploration in high-energy physics, fusion, climate and so on are based on international collaborations. As these collaborations produce more and more data at faster rates, the existing workflow management systems are hard pressed to keep pace. A necessary solution is to process, analyze, summarize and reduce the data before it reaches the relatively slow disk storage system. This approach is generally known as in-transit processing or in-flight analysis. The ICEE project aims to introduce this in-transit analysis capability into a collaborative workflow system by leveraging the in-transit capability of ADIOS and selective data access capability of FastBit. Additionally, ICEE seeks to provide an effective data flow management for distributed workflows and enable large international projects to make near real-time collaborative decisions.

• Project duration: from Oct. 1, 2012 to Sep. 30, 2015
• Funding source: DOE ASCR under 2012 Scientific Collaborations at Extreme-Scale
• Program manager: Rich Carlson
• Introduction
• Project Reports
• Documents, Publications and Presentations

Contacts

• John Wu: john.wulnersc.gov
• Alex Sim: asimllbl.gov
• CS Chang: cschanglpppl.gov
• Scott Klasky: klaskylornl.gov