18. – 21. März 2018, Dagstuhl Seminar 18122

Automatic Quality Assurance and Release


Bram Adams (Polytechnique Montreal, CA)
Benoit Baudry (KTH Royal Institute of Technology – Stockholm, SE)
Sigrid Eldh (Ericsson AB – Stockholm, SE)
Andy Zaidman (TU Delft, NL)

Auskunft zu diesem Dagstuhl Seminar erteilt

Dagstuhl Service Team


Gemeinsame Dokumente
Programm des Dagstuhl Seminars [pdf] (Aktualisieren)

(Zum Einloggen bitte Seminarnummer und Zugangscode verwenden)


This seminar brings together researchers and practitioners who share an interest in the quality assurance perspective of release engineering and DevOps. The overarching goal is twofold: (1) sharing views on the state of the art and state of the practice regarding techniques to ensure quality within time-constrained release cycles; (2) identifying challenges and best practices that are relevant both for the software industry and as scientific investigations.

The bridge between quality assurance and release engineering is part of a wider movement to "shift right" the testing activities. For example, companies are increasingly applying A/B tests on subsets of their actual users, i.e., in production, to decide about design questions, or are using canary releases to test (and possibly roll back) a new version in the field. As such, “shift right” test strategies are becoming a key approach to reconcile the fast delivery of increasingly complex software systems with a level of quality that is higher than ever before.

Seminar topics to be included are:

Reconciling industry and academic perspectives

Both academia and industry (either open or closed source) participate in building techniques to ensure high quality within release engineering and DevOps. Yet, their understanding of these techniques varies in terms of implementation and, more importantly, with respect to the issues that need to be resolved to improve the applicability and efficiency of DevOps. The seminar aims to align academic perspectives with the industry’s needs, while sharing recent academic breakthroughs and techniques with industry.

Shift left versus shift right

Early testing (shift left) has been advocated as a good practice for decades. Meanwhile, DevOps and associated methodologies push for late (shift right) approaches to testing. The seminar wants to discuss both approaches and how one should balance both views to achieve effective quality assurance for release engineering processes.

Best practices

Can we define a set of best practices for implementing effective release engineering and DevOps pipelines? Aspects of these go beyond mere automation of the "build to release" pipeline, and also consider managing multiple system variants in parallel, limiting the impact of a feature change, having fast and automatic support for fault localization and debugging, or dealing with automatic installation and rollback in complex environments and configurations. Other practices advocate the creation of abstraction layers and complex architectures to limit fault propagation, yet this appears to stand in conflict with system testing to release fast.

Technical and organizational DevOps issues

What are both the technical and organizational issues in building effective QA into a release engineering and/or DevOps process? Here, we aim at identifying software quality risks that arise as a consequence of DevOps adoption. These risks can be related to the lack of tools or techniques, to the gap between existing techniques and the requirements for DevOps, but they can also arise because of organizational decisions taken to implement a DevOps process. The exchange between academic and industrial participants will be essential to identify these risks and crystallize challenges for future research.

  Creative Commons BY 3.0 DE
  Bram Adams, Benoit Baudry, Sigrid Eldh, and Andy Zaidman


  • Software Engineering


  • Software testing
  • Software release
  • Software deployment
  • Quality assurance
  • Continuous integration


Bücher der Teilnehmer 

Buchausstellung im Erdgeschoss der Bibliothek

(nur in der Veranstaltungswoche).


In der Reihe Dagstuhl Reports werden alle Dagstuhl-Seminare und Dagstuhl-Perspektiven-Workshops dokumentiert. Die Organisatoren stellen zusammen mit dem Collector des Seminars einen Bericht zusammen, der die Beiträge der Autoren zusammenfasst und um eine Zusammenfassung ergänzt.


Download Übersichtsflyer (PDF).


Es besteht weiterhin die Möglichkeit, eine umfassende Kollektion begutachteter Arbeiten in der Reihe Dagstuhl Follow-Ups zu publizieren.

Dagstuhl's Impact

Bitte informieren Sie uns, wenn eine Veröffentlichung ausgehend von
Ihrem Seminar entsteht. Derartige Veröffentlichungen werden von uns in der Rubrik Dagstuhl's Impact separat aufgelistet  und im Erdgeschoss der Bibliothek präsentiert.