https://www.dagstuhl.de/19111

March 10 – 15 , 2019, Dagstuhl Seminar 19111

Theoretical Foundations of Storage Systems

Organizers

Martin Farach-Colton (Rutgers University – Piscataway, US)
Inge Li Grtz (Technical University of Denmark – Lyngby, DK)
Rob Johnson (VMware – Palo Alto, US)
Donald E. Porter (University of North Carolina at Chapel Hill, US)

For support, please contact

Annette Beyer for administrative matters

Shida Kunz for scientific matters

Documents

List of Participants
Shared Documents

Motivation

Storage systems, including databases and file systems, are at the heart of all large data applications. Recently, some storage systems have incorporated theoretical advances in data organization techniques, with substantial improvements in performance. However, there is little contact between the systems designers who build storage systems and theoreticians who design new ways to organize data. This Seminar aims to bridge that gap, to the benefit of both communities and to improve the design of all storage systems.

External-memory algorithms are those where the data is too large to fit in memory, and hence needs to be stored on disk and accessed using I/O. Algorithmic analysis of such algorithms therefore focuses on the number of I/Os needed to complete a computation, rather than the number of instructions. This is because an I/O can be many orders of magnitude slower than a machine instruction and therefore I/Os can be the bottleneck in such computations.

The theoretical analysis of such external-memory algorithms has produced many exciting results in the last two decades. Many of these are directly relevant to practical applications, but only a few have made the leap to deployment. This Seminar aims to bring together theoreticians, who have an extensive understanding of the state of the art in external memory data structures, and storage systems researchers and practitioners, who understand the details of the problems that need to be solved.

Specific questions that would be addressed:

  • How can we use the huge improvements in string data structures to improve storage systems that manipulate strings? Many data structures, such as LSMs and Bε-trees, rely heavily on the assumption that keys are indivisible and small.
  • How can we use new multi-dimensional data indexes in working systems?
  • Many indexes suffer from fragmentation. Are there data structure improvements that would allow efficient storage on disks that are nearly full? Currently, disks are kept only a fraction full because the performance of existing data structures decays dramatically as the disk fills. This suggests another problem:
  • How can theoretical models be improved to capture such issues as:
    • full disk: The external memory model, the disk is of unbounded size.
    • sequential access: Both hard disks and SSDs require very large blocks of sequential I/O to capture a large fraction of bandwidth. The external-memory model assumes that disks are random access.
  • Concurrency: Can data structures be designed to exploit memory locality on disk while maintaining concurrency in RAM?

The storage system world is in a ferment as new hardware becomes available. Now is the time to establish deep partnerships across disciplines in computer science to solve some of the most pressing big data infrastructure problems.

License
  Creative Commons BY 3.0 DE
  Martin Farach-Colton, Inge Li Grtz, Rob Johnson, and Donald E. Porter

Classification

  • Data Structures / Algorithms / Complexity
  • Hardware
  • Operating Systems

Keywords

  • Storage Systems
  • External Memory Algorithms

Documentation

In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.

 

Download overview leaflet (PDF).

Publications

Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.

NSF young researcher support