March 14 – 19 , 2010, Dagstuhl Seminar 10111

Practical Software Testing: Tool Automation and Human Factors


Mark Harman (University College London, GB)
Henry Muccini (University of L'Aquila, IT)
Wolfram Schulte (Microsoft Corporation – Redmond, US)
Tao Xie (North Carolina State University – Raleigh, US)

The Dagstuhl Foundation gratefully acknowledges the donation from:

  •   Microsoft Research Redmond, Software Engineering Group, US

For support, please contact

Dagstuhl Service Team


Dagstuhl Seminar Proceedings DROPS
List of Participants
Dagstuhl Seminar Schedule [pdf]


The main goal of the seminar “Practical Software Testing: Tool Automation and Human Factors” was to bring together academics working on algorithms, methods, and techniques for practical software testing, with practitioners, interested in developing more soundly-based and well-understood testing processes and practices. The seminar’s purpose was to make researchers aware of industry’s problems, and practitioners aware of research approaches. The seminar focused in particular on testing automation and human factors:

Tool automation. Automation of testing is a crucial concern in industry. It is only with automation that testing becomes practical and scalable to the size of a typical system with which the industry has to deal. Test automation or tool support spans the spectrum from test planning, generation, minimization, execution, oracle checking, to management. Test automation can exploit not only knowledge from the code under test but also from available models or specifications.

Human factors. Human factors play important roles in software testing. Given the code under test, tools can try to automate the generation of test inputs as much as possible, but test oracles still need to come from testers, who specify them in the form of specifications, properties, or test assertions, or directly inspect the actual test outputs for correctness. In addition, tools are not always perfect to deal with software complexity; testers need to cooperate with tools to effectively carry out testing tasks, by giving guidance to tools and interpreting results produced by tools. Thus testers need to be well trained.

In the week of March 14-19, 2010, 40 researchers from 11 countries (Canada, France, Germany, Italy, Luxembourg, the Netherlands, Sweden, Switzerland, South Africa, United Kingdom, United States) discussed their recent work, and recent and future trends in software testing. The seminar consisted of five main types of presentations or activities:topic-oriented presentations, researchoriented presentations, short self-introduction presentations, tool demos, and working group meetings and presentations.

In summary, the seminar accomplished all the expected goals, generating a great deal of forward momentum. The discussion and working groups allowed participants to form better understanding of open challenges and future directions in software testing. During the seminar, academic researchers and industrial researchers fully exchanged ideas for attempting to bridge the gap between research and practice. A number of participants exploited the substantial interactions at the seminar to foster future collaborations. After the seminar, the seminar organizers and participants compiled a bibliography by collecting a list of papers discussed or mentioned during the seminar. Several of the groups indicated that they intended to continue the discussion process after the seminar. We hope that the ideas and collaborations initiated at this Dagstuhl seminar in March 2010 will find fruition in papers, funded research projects, and technical innovations in the years to come.


  • Sw-engineering
  • Society / HCI
  • Verification / Logic
  • Semantics / Formal Methods


  • Software testing
  • Test generation
  • Test automation
  • Test oracles
  • Testing tools
  • Humancomputer interaction
  • Code-based testing
  • Specification-based testing
  • Model-based testing


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.