Understanding I/O Behavior in Scientific and Data-Intensive Computing

Summary

Dagstuhl Seminar 21332, "Understanding I/O behavior in scientific and data-intensive computing," brought together computer scientists from around the world to survey how I/O workloads are measured and analyzed on high-performance computing (HPC) systems, identify gaps in methodologies, and debate how to best apply this technology to advance HPC productivity. The hybrid, week-long event attracted 10 physical and 25 virtual attendees. They included representatives from seven countries spanning a variety of career levels in academia, industry, and government. The diversity of perspectives, combined with an intense week-long seminar format, offered an unprecedented opportunity for researchers to share ideas and spark new collaborative opportunities.

The seminar agenda was structured as a combination of full-group plenary sessions and subgroup breakout sessions. The plenary sessions were used to discuss high-level issues, vote on subtopics to investigate, relay results from breakout sessions, and present ``lightning'' talks that highlighted key issues in the community. The breakout sessions employed small groups (roughly five people each) to follow up in ``deep dive'' discussions on specific subtopics. This format enabled attendees from numerous time zones to remain productively engaged throughout the week. We also found it to be successful in facilitating discussion despite the COVID-19 safety considerations that prevented us from assembling at a single venue. The final day of the seminar was devoted to recording seminar findings in a timely manner while subject matter experts were still available for consultation.

Over the course of the seminar, the attendees converged on six high-level topics for deep dive discussions that are covered in this report.

• Tools: Cross-Cutting Issues (Section 4.1) explored common challenges in development of tools for understanding HPC I/O.
• Data Sources and Acquisition (Section 4.2) addressed how to acquire various forms of raw I/O instrumentation from production systems.
• Analysis (Section 4.3) focused on how to interpret I/O instrumentation once acquired.
• Enacting Actionable Responses (Section 4.4) investigated how to best utilize the outcomes from I/O analysis.
• Data Center Support (Section 4.5) focused on strategies for facility operators to facilitate better understanding of I/O behavior.
• Community Support (Section 4.6) explored the unique characteristics of the I/O analysis community and how to foster its growth.

This report presents a separate summary for each deep dive topic, including a survey of the state of the art, gaps, challenges, and recommendations. The report concludes in Section 5 with a summary of cross-cutting themes and recommendations produced by the seminar as a whole. We found that understanding I/O behavior in scientific and data-intensive computing is increasingly important in an era of evolving workloads and increasingly complex HPC systems and that several cross-cutting challenges must be addressed in order to maximize its potential.