Practical Software Testing : Tool Automation and Human Factors

Software testing consumes approximately 50% of the costs of software development, making techniques for improved quality and efficacy of software testing a pressing concern, both for academic research and industrial practice. Software testing is thus an ideal topic for bringing together academics, practitioners, as well as tool vendors.

Software testing combines challenging research problems with real practical importance for the software development industry, and the wider society that it serves. It presents an excellent and wide-ranging set of open research questions, amongst other things, algorithm design, complexity, decidability, source code analysis, transformation, psychological and social factors, empirical software engineering, measurement, process improvement, modeling, simulation, specification, and formal methods.

The issues thrown up by testing problems touch almost every aspect of computer science as it has developed over the past half a century. Clearly therefore, testing earns the right to take up a place at the heart of academic research in computer science and software engineering. However, software testing is also of critical practical significance to almost every organization involved in the production and use of software. Answers to the currently open research questions in software testing can have a major impact upon industrial practice, with far-reaching implications for the development of the global economy. This combination of academic challenge and industrial relevance makes software testing a natural topic for the seminar.

There are two essential topics in research and practice of software testing:

Tool automation. Automation of testing is a crucial concern in industry. It is only with automation that testing becomes practical and scalable to the size of system with which the industry has to deal. Test automation or tool support spans from test planning, generation, minimization, execution, oracle checking, to management. Test automation can exploit not only knowledge from the code under test but also from available models or specifications.

Human factors. Human factors play important roles in software testing. Given the code under test, tools can try to automate the generation of test inputs as much as possible but test oracles still need to come from testers, who specify them in the form of specifications, properties, or test assertions, or directly inspect the actual test outputs for correctness. In addition, tools are not always perfect to deal with software complexity; testers need to cooperate with tools to effectively carry out testing tasks, by giving guidance to tools and interpreting results produced by tools. Thus testers need to be well trained.

The seminar will bring together academics working on algorithms, methods, and techniques for practical software testing, with practitioners, interested in developing more soundly-based and well-understood testing processes and practices. The seminar’s purpose is to make researchers aware of industry’s problems, and practitioners aware of research approaches. The seminar will focus in particular on testing automation and human factors, and should play a significant role in developing, extending, and cementing collaboration between academics and industry.