The goal of this Dagstuhl Seminar is to bring together members of three communities – academic researchers in cybersecurity, industrial practitioners and software engineering researchers – to tackle the problem of empirically evaluating secure development processes.

In the past decades, the cybersecurity community has created many principles and practices for developing secure software. However, this knowledge has generally been assembled by the application of common sense and experience, and while individual measures and techniques are often based on real-world data and root-cause analysis, broader strategies and processes for creating secure software and assessing software security are usually not subjected to rigorous evaluation. This is a serious shortcoming: common sense can be mistaken and experiences over-generalized. Evaluation techniques are necessary to provide a firm scientific basis that can support progress in the field.

Some such techniques do exist for the later software development stages – implementation and testing. Here one enjoys good automation, and the mapping between technique and end-product is relatively clear-cut. It is also in these stages where security teams succeed at least partially in providing software developers with concrete prescriptive guidance. Nonetheless, the earlier developmental stages – requirements elicitation, threat modeling, architecture – are at least as critical to the security of the final product if not more so, yet pose a much greater experimental challenge because of the gap between the process and product. Experience has shown only limited success at incorporating security into these crucial initial stages.

This seminar wants to enable conversations between these three traditionally-separate communities and drive empirical research to help the developers of secure software. Through these conversations, we aim to bring empirical research practices already established in the software engineering community to the security research community, to make software engineering researchers aware of the important issues in software security, and to make academic researchers aware of the practical constraints and challenges faced by industry.

As a concrete eventual outcome of this Dagstuhl Seminar, we plan to produce an edited open-access book collecting essays of what we know and do not know about secure software practices and processes from a sound scientific empirical perspective. This volume will merge empirical software engineering and security research to assist the involved communities, including industry and academia, in focusing their research efforts, and help newcomers to our field find fertile research areas.

In order to do this, this seminar will engage participants in multiple breakout sessions, both free-form and more formally organized, to foster discussions and generate ideas. By seminar end, we intend to have produced the book outline and multi-disciplinary teams that will work together to complete it.

Creative Commons BY 4.0

Eric Bodden, Brendan Murphy, Sam Weber, Laurie Williams, and Steve Lipner