https://www.dagstuhl.de/98361

06. – 11. September 1998, Dagstuhl-Seminar 98361

Test Automation for Reactive Systems - Theorie and Practice

Organisator

E. Brinksma (Twente), J. Peleska (Bremen), M. Siegel (Kiel)

Auskunft zu diesem Dagstuhl-Seminar erteilt

Dagstuhl Service Team

Dokumente

Dagstuhl-Seminar-Report 223

The design and implementation of correct reactive systems is one of the major challenges of information technology. More than ever before modern society has become critically dependent upon applications of such systems. Important application areas include telematic systems, such as communication protocols and services, process control systems, and embedded-software systems, ranging from aircraft control to consumer products like television sets. Most applications are critical in one or more aspects, e.g. in the context of safety, economy, ecology etc.

Testing is an indispensable technique for the validation of reactive systems. Although other tool supported methods, such as simulation, model checking and verification by theorem proving, are increasingly being applied to assess the correctness of system designs, testing remains the practically most widely used validation technique. Although testing is generally too weak to guarantee correctness, its advantages are that it is more readily applicable to large industrial systems, and that it is in fact the only validation technique that can link the functionality of a (physical) realisation and its formal specification.

On the one hand, the great growth of reactive system applications and with that the growing need to assess their correctness, has brought about a considerable practical interest in the improvement of their testing procedures. On the other hand, academic researchers, who formerly regarded testing as an inferior and superfluous validation method, have begun to investigate the use of formal methods and tools to specify, generate and implement tests. This is of great practical interest, as most test suites for reactive systems are still being produced on the basis of ill-understood ad hoc procedures. As a result the production and maintenance of test suites have become very expensive activities, presenting a considerable practical bottleneck. Even minor improvements on existing working procedures are economically attractive. Major improvements are expected from algorithms for the (semi-)automated derivation and selection of test suites from formal specifications of the implementations under test, as well as the machine-assisted evaluation of test results.

In spite of these bright prospects for the application of testing theory and related tools, still much work is needed to turn the existing theories and tools into an effective testing workbench fit for industrial application.

The Dagstuhl seminar on "Test Automation for Reactive Systems" was the first of its kind, and was very well attended (more than 40 participants). It brought together people from different backgrounds with a strong interest in testing, such as protocol testing, real-time testing, performance testing, statistical testing, etc., to present and discuss the state-of-the-art in testing and test automation. There was a strong representation from the formal methods community, but here again a wide range of models and formalisms was represented: Mealy automata, labelled transition systems, abstract data types, I/O automata, timed automata, Z etc.

The presentations and discussions made clear that although there are quite a few research groups active in the area now, we are only at the beginning of things and major contributions to the (semi-)automation of practical software testing are still to come. A bottleneck is the limited availability of formal system specifications that can be used to derive tests from. In certain areas, however, such as e.g. safety-critical and embedded systems, the willingness of industry to go more formal is growing, which is among others motivated by the potential gains for testing implementations.

A discussion session that was held to talk about future research themes in test automation suggested at least the following topics: relating test methods to common design and implementation methodologies (e.g. objected oriented design), the issue of design for testability, which has been very successful in the area of hardware testing, and measures to quantify the coverage of tests. The general opinion was that the seminar was a success and it is planned to organise another Dagstuhl meeting on testing in the future.

The support of the TMR Program of the European Community is gratefully acknowledged.

Dokumentation

In der Reihe Dagstuhl Reports werden alle Dagstuhl-Seminare und Dagstuhl-Perspektiven-Workshops dokumentiert. Die Organisatoren stellen zusammen mit dem Collector des Seminars einen Bericht zusammen, der die Beiträge der Autoren zusammenfasst und um eine Zusammenfassung ergänzt.

 

Download Übersichtsflyer (PDF).

Publikationen

Es besteht weiterhin die Möglichkeit, eine umfassende Kollektion begutachteter Arbeiten in der Reihe Dagstuhl Follow-Ups zu publizieren.

Dagstuhl's Impact

Bitte informieren Sie uns, wenn eine Veröffentlichung ausgehend von
Ihrem Seminar entsteht. Derartige Veröffentlichungen werden von uns in der Rubrik Dagstuhl's Impact separat aufgelistet  und im Erdgeschoss der Bibliothek präsentiert.