Dagstuhl Seminar 23441
Ensuring the Reliability and Robustness of Database Management Systems
( Oct 29 – Nov 03, 2023 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/23441
Organizers
- Alexander Böhm (Google Germany - Schwetzingen, DE)
- Hannes Mühleisen (CWI - Amsterdam, NL)
- Danica Porobic (Oracle Switzerland - Zürich, CH)
- Manuel Rigger (National University of Singapore, SG)
Contact
- Andreas Dolzmann (for scientific matters)
- Jutka Gasiorowski (for administrative matters)
Impacts
- Surprise Benchmarking : The Why, What, and How : article in DBTest '24: Proceedings of the Tenth International Workshop on Testing Database System - Benson, Lawrence; Binnig, Carsten; Bodensohn, Jan-Micha; Lorenzi, Federico; Luo, Jigao; Sanghi, Anupam; Sears, Russell; Ziegler, Tobias; Tözün, Pinar; Rabl, Tilmann; Porobic, Danica - New York : ACM, 2024. - Pages 1–8.
- Ensuring the Reliability and Robustness of Database Management Systems - Mühleisen, Hannes; Porobic, Danica; Rigger, Manuel - Wadern : LZI, 2024 - (Dagstuhl Reports : 13. 2024, 10 : article).
Schedule
Database systems are an essential component of most software systems. It is crucial that they function correctly and are efficient. Achieving this is difficult, given that growing demands require increasingly sophisticated systems and adapting them to new hardware platforms. Building on the success of the last seminar, the goal of this Dagstuhl Seminar was to advance database systems reliability by bringing together both practitioners as well as researchers working in this domain. We discussed current practices, approaches, and open challenges in manual and automated correctness and performance testing, isolation-level testing, benchmarking, database and query generation, query languages, as well as debugging.
Goals and Outcomes
As a concrete tangible outcome of the seminar, we aimed to write a manuscript on (1) best practices in ensuring database systems' reliability, (2) the state of the art on this topic in research, as well as (3) open challenges, which might also serve as the cornerstone of writing a book on this topic. We believe we have achieved this goal, with the reports of the four working groups presented in the subsequent chapters. The majority of the contents were composed during the seminar and reflect the experiences and opinions of various attendees. Thus, some of the contents might be incomplete, contradictory, or redundant.
Attendee Mix
The seminar’s attendees were 36 internationally renowned researchers as well as high-profile practitioners whose work is closely related to the topic of this seminar. Given that the last years have seen an increased focus on research related to database system reliability, a larger fraction of researchers had significant expertise in this topic as compared to the first edition of the seminar, whose attendees mix had a broader, but less specific expertise. Overall, 25% of the second seminar’s attendees joined also the first seminar – note that the first edition was a small seminar with a hybrid format that had only 12 in-person attendees, of which 67% joined the second seminar. A large number of attendees from Europe accepted the seminar invitation, constituting 56% of the attendees. The remaining attendees joined from the US (27%), Asia (14%), and one attendee from South Africa. 36% of the attendees were from industry, which is higher compared to the last seminar, which had only 23% attendees from industry. By increasing their percentage, we aimed to gain more insights into the current practices in the industry. Unfortunately, one of the co-organizers, Alexander Böhm was unable to attend the seminar; we want to thank him for his contributions in organizing it.
Seminar Structure
The seminar lasted for five days and accepted only in-person attendees. We started the first day with a round of introductions, where each attendee could introduce themselves and their interests in a five-minute presentation. The majority of the remaining days were filled with discussions by the individual working groups – we deliberately avoided having presentations on past work to keep the discussion centered on open problems and future directions. While we identified the main topics of interest that helped form the working groups on the evening of the first day, attendees could freely move between these groups and regroup. We reconvened in the larger group once or twice per day, to present and discuss the results of the working groups. The four working groups listed in this report reflect the working groups at the end of the seminar; for example, the first working group on automated testing split into multiple subgroups that discussed various topics during the course of the seminar. In addition to the working groups, we had two tutorials after dinner, as well as an excursion on Wednesday afternoon.
Future Plans
he individual working groups' reports, contained in the full report, are a concrete outcome of this edition's seminar. First, working groups indicated their plans to refine their reports as well as follow up on the work to present the results to a broader audience aiming to encourage future work on their topics. Second, we expect individual research groups to take on the challenges identified in the seminar and propose solutions to them. Third, we plan to propose a third edition of the seminar to discuss the progress on tackling the identified problems and expand the scope in terms of reliability. In this seminar, we also want to fill gaps in terms of topics and attendee mix; for example, in this edition of the seminar, we focused our discussions on relational database systems, omitting systems built on other models. As another example, we had no major discussions on the reliability of learned components of database systems.
Creative Commons BY 4.0
Manuel Rigger, Hannes Mühleisen, and Danica Porobic
Database systems are an essential component of most software systems. It is crucial that they function correctly and operate efficiently. Achieving this is difficult, given that growing demands require increasingly sophisticated systems and adapting them to new hardware platforms.
Building on the success of the last seminar in this series, Dagstuhl Seminar 21442, the goal of this Dagstuhl Seminar is to advance the reliability of database systems by bringing together both practitioners as well as researchers working in this domain. We expect to discuss practices, approaches, and open challenges in making database systems and other data-centric systems more reliable (e.g., in terms of correctness, performance, or fault tolerance).
We plan to focus on the following sub-topics:
- Manual testing and test suites
- Automated testing for correctness and performance
- Isolation-level testing
- Benchmarking
- Database and query generation
- Debugging
- Query languages
- Formal methods and verification
While the first seminar in the series, Dagstuhl Seminar 21442, was open-ended, we plan for a more focused follow-up seminar with Dagstuhl Seminar 23441. Specifically, as a concrete tangible outcome of the seminar, we aim to brainstorm, write, and publish a manuscript on (1) best practices in ensuring database systems' reliability, (2) the state of the art on this topic in research, as well as (3) open challenges, which might also serve as the cornerstone of writing a book on this topic.
Alexander Böhm, Hannes Mühleisen, Danica Porobic, and Manuel Rigger
- Jinsheng Ba (National University of Singapore, SG) [dblp]
- Lawrence Benson (Hasso-Plattner-Institut, Universität Potsdam, DE) [dblp]
- Carsten Binnig (TU Darmstadt, DE) [dblp]
- Ankush Desai (Amazon - Cupertino, US) [dblp]
- Adam Dickinson (Snowflake Computing Inc. - Seattle, US) [dblp]
- Wensheng Dou (Chinese Academy of Sciences - Beijing, CN) [dblp]
- Stefania Dumbrava (ENSIIE - Paris, FR) [dblp]
- Moritz Eyssen (Snowflake - Berlin, DE)
- Tim Fischer (Universität Tübingen, DE) [dblp]
- Florian Gerlinghoff (MotherDuck - Amsterdam, NL)
- Torsten Grust (Universität Tübingen, DE) [dblp]
- Muhammad Ali Gulzar (Virginia Polytechnic Institute - Blacksburg, US) [dblp]
- Denis Hirn (Universität Tübingen, DE) [dblp]
- Hong Hu (Pennsylvania State University - University Park, US) [dblp]
- Zu-Ming Jiang (ETH Zürich, CH) [dblp]
- Marcel Kost (Salesforce - München, DE) [dblp]
- Burcu Kulahcioglu Ozkan (TU Delft, NL) [dblp]
- Federico Lorenzi (TigerBeetle - Cape Town, ZA)
- Umang Mathur (National University of Singapore, SG) [dblp]
- Everett Maus (Google - Seattle, US)
- Hannes Mühleisen (CWI - Amsterdam, NL) [dblp]
- Thomas Neumann (TU München - Garching, DE) [dblp]
- Danica Porobic (Oracle Switzerland - Zürich, CH) [dblp]
- Mark Raasveldt (DuckDB Labs - Amsterdam, NL) [dblp]
- Tilmann Rabl (Hasso-Plattner-Institut, Universität Potsdam, DE) [dblp]
- Manuel Rigger (National University of Singapore, SG) [dblp]
- Stan Rosenberg (Cockroach Labs - New York, US) [dblp]
- Anupam Sanghi (IBM India - Bangalore, IN) [dblp]
- Gambhir Sankalp (EPFL - Lausanne, CH) [dblp]
- Andrei Satarin (Google - Mountain View, US)
- Russell Sears (Crystal DB - San Francisco, US)
- Thodoris Sotiropoulos (ETH Zürich, CH) [dblp]
- Caleb Stanford (University of California - Davis, US) [dblp]
- Cheng Tan (Northeastern University - Boston, US) [dblp]
- Pinar Tözün (IT University of Copenhagen, DK) [dblp]
- Chengyu Zhang (ETH Zürich, CH) [dblp]
Related Seminars
- Dagstuhl Seminar 21442: Ensuring the Reliability and Robustness of Database Management Systems (2021-11-01 - 2021-11-04) (Details)
Classification
- Databases
- Performance
- Software Engineering
Keywords
- database testing
- database reliability
- database benchmarking
