http://www.dagstuhl.de/17202

May 14 – 19 , 2017, Dagstuhl Seminar 17202

Challenges and Opportunities of User-Level File Systems for HPC

Organizers

André Brinkmann (Universität Mainz, DE)
Kathryn Mohror (LLNL – Livermore, US)
Weikuan Yu (Florida State University – Tallahassee, US)

For support, please contact

Susanne Bach-Bernhard for administrative matters

Roswitha Bardohl for scientific matters

Documents

List of Participants
Shared Documents

Motivation

Although the benefits of hierarchical storage have been adequately demonstrated to the point that the newest leadership class HPC systems will employ burst buffers, critical questions remain for supporting hierarchical storage systems, including: How should we present hierarchical storage systems to user applications, such that they are easy to use and that application code is portable across systems? How should we manage data movement through a storage hierarchy for best performance and resilience of data? How do the particular I/O use cases mandate the way we manage data? There have been many efforts to explore this space in the form of file systems, with increasingly more implemented at the user level. This is because it is relatively easy to swap in new, specialized user-level file systems for use by applications on a case-by-case basis, as opposed to the current mainstream approach of using general-purpose, system-level file systems which may not be optimized for HPC workloads and must be installed by administrators. In contrast, file systems at the user level can be tailored for specific HPC workloads for high performance and can be used by applications without administrator intervention.

Many such user-level file system developers have found themselves “having to reinvent the wheel” to implement various optimizations in their file systems. Thus, it would benefit the larger community if we could develop a common framework for HPC file system development. The framework we envision would be based on solid software engineering practices and would enable “plug and play” interchange of basic file system components. Having this framework would facilitate both the operation needs of current production systems and the research efforts for novel features in future systems. On the production side, the framework would provide a trusted backbone upon which users and developers could rely for basic services. For research, the backbone of the framework would enable rapid prototyping of file system components so that students and other researchers did not have to implement the entire framework simply to experiment with a specific optimization.

The goal of this Dagstuhl Seminar is to bring together experts in I/O performance, file systems, and storage, and collectively explore the space of current and future problems and solutions for I/O on hierarchical storage systems in order to begin a community effort in designing a user-level file system framework for HPC systems. We expect a lively week of learning about each other’s approaches as well as unique I/O use cases that can influence the design of a community-driven file system framework. Our hope is that the effort initiated during this seminar will result in a long-term collaboration that will benefit the HPC community as a whole.

License
  Creative Commons BY 3.0 DE
  André Brinkmann and Kathryn Mohror and Weikuan Yu

Classification

  • Operating Systems

Keywords

  • I/O performance
  • File systems
  • High performance computing workflows

Book exhibition

Books from the participants of the current Seminar 

Book exhibition in the library, ground floor, during the seminar week.

Documentation

In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.

 

Download overview leaflet (PDF).

Publications

Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.