May 1 – 5 , 2023, Dagstuhl Seminar 23181

Empirical Evaluation of Secure Development Processes


Eric Bodden (Universität Paderborn, DE)
Brendan Murphy (Microsoft Research – Cambridge, GB)
Sam Weber (Carnegie Mellon University – Pittsburgh, US)
Laurie Williams (North Carolina State University – Raleigh, US)

For support, please contact

Simone Schilke for administrative matters

Marsha Kleinbauer for scientific matters


Dagstuhl Seminar Schedule (Upload here)

(Use personal credentials as created in DOOR to log in)


The goal of this Dagstuhl Seminar is to bring together members of three communities – academic researchers in cybersecurity, industrial practitioners and software engineering researchers – to tackle the problem of empirically evaluating secure development processes.

In the past decades, the cybersecurity community has created many principles and practices for developing secure software. However, this knowledge has generally been assembled by the application of common sense and experience, and while individual measures and techniques are often based on real-world data and root-cause analysis, broader strategies and processes for creating secure software and assessing software security are usually not subjected to rigorous evaluation. This is a serious shortcoming: common sense can be mistaken and experiences over-generalized. Evaluation techniques are necessary to provide a firm scientific basis that can support progress in the field.

Some such techniques do exist for the later software development stages – implementation and testing. Here one enjoys good automation, and the mapping between technique and end-product is relatively clear-cut. It is also in these stages where security teams succeed at least partially in providing software developers with concrete prescriptive guidance. Nonetheless, the earlier developmental stages – requirements elicitation, threat modeling, architecture – are at least as critical to the security of the final product if not more so, yet pose a much greater experimental challenge because of the gap between the process and product. Experience has shown only limited success at incorporating security into these crucial initial stages.

This seminar wants to enable conversations between these three traditionally-separate communities and drive empirical research to help the developers of secure software. Through these conversations, we aim to bring empirical research practices already established in the software engineering community to the security research community, to make software engineering researchers aware of the important issues in software security, and to make academic researchers aware of the practical constraints and challenges faced by industry.

As a concrete eventual outcome of this Dagstuhl Seminar, we plan to produce an edited open-access book collecting essays of what we know and do not know about secure software practices and processes from a sound scientific empirical perspective. This volume will merge empirical software engineering and security research to assist the involved communities, including industry and academia, in focusing their research efforts, and help newcomers to our field find fertile research areas.

In order to do this, this seminar will engage participants in multiple breakout sessions, both free-form and more formally organized, to foster discussions and generate ideas. By seminar end, we intend to have produced the book outline and multi-disciplinary teams that will work together to complete it.

Motivation text license
  Creative Commons BY 4.0
  Eric Bodden, Brendan Murphy, Sam Weber, Laurie Williams, and Steve Lipner

Related Dagstuhl Seminar


  • Cryptography And Security
  • Software Engineering


  • Empirical software engineering
  • Secure system design


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.