24. Februar – 01. März 2013, Dagstuhl-Seminar 13091

Analysis, Test and Verification in The Presence of Variability


Paulo Borba (Federal University of Pernambuco – Recife, BR)
Myra B. Cohen (University of Nebraska – Lincoln, US)
Axel Legay (INRIA Rennes – Bretagne Atlantique, FR)
Andrzej Wasowski (IT University of Copenhagen, DK)

Auskunft zu diesem Dagstuhl-Seminar erteilt

Dagstuhl Service Team


Dagstuhl Report, Volume 3, Issue 2 Dagstuhl Report
Dagstuhl's Impact: Dokumente verfügbar


The seminar "Analysis, Test and Verification in The Presence of Variability" that took place at Schloss Dagstuhl from February 24 to March 1, 2013, had the goal of consolidating and stimulating research on analysis of software models with variability, enabling the design of variability-aware tool chains. We brought together 46 key researchers from three continents, working on quality assurance challenges that arise from introducing variability, and some who do not work with variability, but that are experts in their respective areas in the broader domain of software analysis or testing research. The participants ranged from those in senior academic positions to successful graduate students. We also enjoyed the presence of several relevant experts from the software development industry.

The seminar included:

1. Invited presentations on state of the art research in SPL testing and verification

The presentations were delivered by experts in variability research. The topics included classifying and unifying product-line analyses, combinatorial interaction testing, model-based testing, analysis of programs with variability and model checking with variability.

Material relevant to the topic of this Dagstuhl was organized in a recent classification by Thüm and coauthors [4]. The Dagstuhl seminar opened with a presentation of this classification, which created a common ontology for later presentations and discussions. This was very helpful for participants who had different areas of expertise.

2. A keynote presentation on the Challenges and Science of Variability

We organized a special keynote shared with the German FOSD meeting, that took place in parallel at the Schloss Dagstuhl facilities. The keynote speaker, Professor Don Batory, called for creating a simple meta-theory identifying and relating the core concepts and properties of variability science, i.e. the body of knowledge created by the community of researchers studying engineering of highly configurable systems. During the workshop, several candidates for the starting point of such theory were mentioned, such as using simple models in constructive logic [2], choice calculus [3] or Clafer [1].

3. A series of presentations on recent results in Variability Analysis

The bulk of the programme was filled with a mixture of research presentations about recent research advances in verification, analysis and test of software with variability. This function of the seminar was particularly important, as the usual dissemination outlets for these contributions are often disjoint -- much of the work is normally presented in domain specific publication channels devoted to only test, verification or programming languages. For many participants the seminar created an opportunity to learn about advances at addressing similar problems in the neighboring research communities -- an experience that is rarely possible outside of Dagstuhl.

4. A session of student presentations

In order to enrich the presentations by senior researchers with a stream of fresh ideas, we organized a special session devoted to short student presentations. The presenters were selected from the participants of the German FOSD meeting. For many of the students it was a rare opportunity to share their ideas with international authorities in their work area. The topics of these lightning presentations were closely related to the seminar goals and included among others, discussions of experimental evaluation of product line analysis strategies, static analysis, type checking for variability, and performance prediction for configurable systems. The session enabled closer integration between the participants of the two events. Many discussions between the two groups continued throughout the week.

5. Dynamically planned sessions on how to address the challenges, how to transfer knowledge, tools, and benchmarks between research areas

The first session (run by Professor Krzysztof Czarnecki) was devoted to extracting challenges for variability analysis out of industrial requirements. Participants from industry and participants from academia involved in industrial projects provided background on requirements known from projects in avionics, automotive and risk assessment domains. These were further discussed to identify research challenges for future work. The discussions were continued in a breakout session on product lines of safety critical systems. Other breakout sessions included dynamic product lines, generic representation of variability, and testing and modeling variability.

Overall, a core set of techniques were discussed at this seminar which include program analysis, model checking, type checking, and testing. We believe that the seminar fruitfully mixed computer science and software engineering researchers from several research sub-domains, allowing them to derive interesting basic research problems stemming from practical needs all related to how variability impacts their respective domains, with the sub-goal of inspiring the use of the latest research advances in software analysis technology to advance variability management tools.


The different kinds of interactions offered by the seminar helped the participants to relate work covering different aspects in a number of dimensions such as:

  1. An overall approach to thinking about variability, as defined by Thüm's classification [4] of analysis into product based, family based, feature based and hybrids;
  2. Core techniques: testing, verification, refactoring, model checking, static analysis;
  3. Mechanisms for representing variability: if-defs, deltas, generic representation, etc.;
  4. Application domains;
  5. The nature of variability: static product lines, dynamic product lines, configurable systems.

The seminar also produced a bibliography of core readings on the topic, that can enable new graduate students to engage more quickly in this area of research.

Trying to classify approaches with respect to these dimensions helped to identify similarities and differences among different techniques (static analysis, model checking, testing, and verification). This, in turn, might trigger new collaborations and research results. The presentations and the ad-hoc discussion sessions helped people to clarify differences and similarities among configurable systems and dynamic and static product lines, with similar consequences to the ones described above. More generally, of course, the Dagstuhl provided the benefit of mixing young and experienced researchers, from different countries and research areas.

An informal survey among a handful of participants has shown that each of them have started 2-3 new collaborations as a result of the seminar. These collaborations took the form of initiated research papers, mutual research visits, or student exchanges. In one anecdotal case, a researcher started a collaboration with a colleague sitting in the same corridor at his home university--- but apparently one had to meet in Dagstuhl to enable the exchange of ideas. We can thus expect a new wave of research results in this area to flourish about a year from the seminar time. Because of this success, we intend to organize a follow up event in several years, be it under the Schloss Dagstuhl programme or under some other appropriate venue.


  1. Kacper Bak, Krzysztof Czarnecki, and Andrzej Wasowski. Feature and Meta-Models in Clafer: Mixed, Specialized, and Coupled. In Proc. of the 3rd Int'l Conf. on Software Language Engineering (SLE'10), LNCS, Vol.6563, pp.102-122, Springer, 2011.}{DOI: 10.1007/978-3-642-19440-5\_7.
  2. Benjamin Delaware, William R. Cook, and Don S. Batory. Product lines of theorems. In Proc. of the 2011 ACM Int'l Conf. on Object-oriented Programming Systems, Languages, and Applications (OOPSLA'11), pp. 595-608, ACM, 2011. DOI:10.1145/2048066.2048113.
  3. Martin Erwig and Eric Walkingshaw. The Choice Calculus: A Representation for Software Variation. ACM Trans. Softw. Eng. Methodol., Vol. 21, Issue 1, pp. 6:1-6:27, 2011. DOI:10.1145/2063239.2063245.
  4. Thomas Thüm, Sven Apel, Christian Kästner, Martin Kuhlemann, Ina Schaefer, and Gunter Saake. Analysis Strategies for Software Product Lines. Technical Report FIN-004-2012, School of Computer Science, University of Magdeburg, April 2012.
Summary text license
  Creative Commons BY 3.0 Unported license
  Paulo Borba, Myra B. Cohen, Axel Legay, and Andrzej Wasowski


  • Programming Languages / Compiler
  • Semantics / Formal Methods
  • Software Engineering


  • Verification
  • Program Analysis
  • Testing
  • Semantics of Programming Languages
  • Software Engineering


In der Reihe Dagstuhl Reports werden alle Dagstuhl-Seminare und Dagstuhl-Perspektiven-Workshops dokumentiert. Die Organisatoren stellen zusammen mit dem Collector des Seminars einen Bericht zusammen, der die Beiträge der Autoren zusammenfasst und um eine Zusammenfassung ergänzt.


Download Übersichtsflyer (PDF).

Dagstuhl's Impact

Bitte informieren Sie uns, wenn eine Veröffentlichung ausgehend von Ihrem Seminar entsteht. Derartige Veröffentlichungen werden von uns in der Rubrik Dagstuhl's Impact separat aufgelistet  und im Erdgeschoss der Bibliothek präsentiert.


Es besteht weiterhin die Möglichkeit, eine umfassende Kollektion begutachteter Arbeiten in der Reihe Dagstuhl Follow-Ups zu publizieren.