Practical Software Testing : Tool Automation and Human Factors

Summary

The main goal of the seminar “Practical Software Testing: Tool Automation and Human Factors” was to bring together academics working on algorithms, methods, and techniques for practical software testing, with practitioners, interested in developing more soundly-based and well-understood testing processes and practices. The seminar’s purpose was to make researchers aware of industry’s problems, and practitioners aware of research approaches. The seminar focused in particular on testing automation and human factors:

Tool automation. Automation of testing is a crucial concern in industry. It is only with automation that testing becomes practical and scalable to the size of a typical system with which the industry has to deal. Test automation or tool support spans the spectrum from test planning, generation, minimization, execution, oracle checking, to management. Test automation can exploit not only knowledge from the code under test but also from available models or specifications.

Human factors. Human factors play important roles in software testing. Given the code under test, tools can try to automate the generation of test inputs as much as possible, but test oracles still need to come from testers, who specify them in the form of specifications, properties, or test assertions, or directly inspect the actual test outputs for correctness. In addition, tools are not always perfect to deal with software complexity; testers need to cooperate with tools to effectively carry out testing tasks, by giving guidance to tools and interpreting results produced by tools. Thus testers need to be well trained.

In the week of March 14-19, 2010, 40 researchers from 11 countries (Canada, France, Germany, Italy, Luxembourg, the Netherlands, Sweden, Switzerland, South Africa, United Kingdom, United States) discussed their recent work, and recent and future trends in software testing. The seminar consisted of five main types of presentations or activities:topic-oriented presentations, researchoriented presentations, short self-introduction presentations, tool demos, and working group meetings and presentations.

In summary, the seminar accomplished all the expected goals, generating a great deal of forward momentum. The discussion and working groups allowed participants to form better understanding of open challenges and future directions in software testing. During the seminar, academic researchers and industrial researchers fully exchanged ideas for attempting to bridge the gap between research and practice. A number of participants exploited the substantial interactions at the seminar to foster future collaborations. After the seminar, the seminar organizers and participants compiled a bibliography by collecting a list of papers discussed or mentioned during the seminar. Several of the groups indicated that they intended to continue the discussion process after the seminar. We hope that the ideas and collaborations initiated at this Dagstuhl seminar in March 2010 will find fruition in papers, funded research projects, and technical innovations in the years to come.