https://www.dagstuhl.de/21332

August 15 – 20 , 2021, Dagstuhl Seminar 21332

Understanding I/O Behavior in Scientific and Data-Intensive Computing

Organizers

Philip Carns (Argonne National Laboratory, US)
Julian Kunkel (Gesellschaft f. wissenschaftl. Datenverarbeitung, DE)
Kathryn Mohror (LLNL – Livermore, US)
Martin Schulz (TU München, DE)

For support, please contact

Dagstuhl Service Team

Documents

Dagstuhl Report, Volume 11, Issue 7 Dagstuhl Report
Aims & Scope
List of Participants
Shared Documents
Dagstuhl Seminar Schedule [pdf]

Summary

Dagstuhl Seminar 21332, "Understanding I/O behavior in scientific and data-intensive computing," brought together computer scientists from around the world to survey how I/O workloads are measured and analyzed on high-performance computing (HPC) systems, identify gaps in methodologies, and debate how to best apply this technology to advance HPC productivity. The hybrid, week-long event attracted 10 physical and 25 virtual attendees. They included representatives from seven countries spanning a variety of career levels in academia, industry, and government. The diversity of perspectives, combined with an intense week-long seminar format, offered an unprecedented opportunity for researchers to share ideas and spark new collaborative opportunities.

The seminar agenda was structured as a combination of full-group plenary sessions and subgroup breakout sessions. The plenary sessions were used to discuss high-level issues, vote on subtopics to investigate, relay results from breakout sessions, and present ``lightning'' talks that highlighted key issues in the community. The breakout sessions employed small groups (roughly five people each) to follow up in ``deep dive'' discussions on specific subtopics. This format enabled attendees from numerous time zones to remain productively engaged throughout the week. We also found it to be successful in facilitating discussion despite the COVID-19 safety considerations that prevented us from assembling at a single venue. The final day of the seminar was devoted to recording seminar findings in a timely manner while subject matter experts were still available for consultation.

Over the course of the seminar, the attendees converged on six high-level topics for deep dive discussions that are covered in this report.

  • Tools: Cross-Cutting Issues (Section 4.1) explored common challenges in development of tools for understanding HPC I/O.
  • Data Sources and Acquisition (Section 4.2) addressed how to acquire various forms of raw I/O instrumentation from production systems.
  • Analysis (Section 4.3) focused on how to interpret I/O instrumentation once acquired.
  • Enacting Actionable Responses (Section 4.4) investigated how to best utilize the outcomes from I/O analysis.
  • Data Center Support (Section 4.5) focused on strategies for facility operators to facilitate better understanding of I/O behavior.
  • Community Support (Section 4.6) explored the unique characteristics of the I/O analysis community and how to foster its growth.

This report presents a separate summary for each deep dive topic, including a survey of the state of the art, gaps, challenges, and recommendations. The report concludes in Section 5 with a summary of cross-cutting themes and recommendations produced by the seminar as a whole. We found that understanding I/O behavior in scientific and data-intensive computing is increasingly important in an era of evolving workloads and increasingly complex HPC systems and that several cross-cutting challenges must be addressed in order to maximize its potential.

Summary text license
  Creative Commons BY 4.0
  Philip Carns, Julian Kunkel, Kathryn Mohror, and Martin Schulz

Related Dagstuhl Seminar

Classification

  • Distributed / Parallel / And Cluster Computing
  • Information Retrieval
  • Performance

Keywords

  • I/O performance measurement
  • Understanding user I/O patterns
  • HPC I/O
  • I/O Characterization

Documentation

In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.

 

Download overview leaflet (PDF).

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.

Publications

Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.