TOP
Search the Dagstuhl Website
Looking for information on the websites of the individual seminars? - Then please:
Not found what you are looking for? - Some of our services have separate websites, each with its own search option. Please check the following list:
Schloss Dagstuhl - LZI - Logo
Schloss Dagstuhl Services
Seminars
Within this website:
External resources:
  • DOOR (for registering your stay at Dagstuhl)
  • DOSA (for proposing future Dagstuhl Seminars or Dagstuhl Perspectives Workshops)
Publishing
Within this website:
External resources:
dblp
Within this website:
External resources:
  • the dblp Computer Science Bibliography


Dagstuhl Seminar 23181

Empirical Evaluation of Secure Development Processes

( May 01 – May 05, 2023 )

(Click in the middle of the image to enlarge)

Permalink
Please use the following short url to reference this page: https://www.dagstuhl.de/23181

Organizers

Contact

Shared Documents


Schedule

Summary

In the past decades, the cybersecurity community has created many principles and practices for developing secure software. However, this knowledge has generally been assembled by the application of common sense and experience, and while individual measures and techniques are often based on real-world data, broader strategies and processes for creating secure software are usually not subjected to rigorous evaluation. This is a serious shortcoming: common sense can be mistaken and experiences over-generalized. Evaluation techniques are necessary to provide a firm scientific basis that can support progress in the field.

Some such techniques do exist for the later software development stages - implementation and testing. Here one enjoys good automation and the mapping between technique and end-product is relatively clear-cut. It is also in these stages where security teams succeed at least partially in providing software developers with concrete prescriptive guidance. Unfortunately, the earlier developmental stages - requirements elicitation, threat modeling, architecture - are just as critical to the security of the final product, yet pose a much greater experimental challenge because of the gap between the process and the product. Experience has shown only limited success at turning software engineers into security experts, particularly so for these crucial initial stages.

Our previous Dagstuhl Seminar 19231 formed a community interested in empirical investigation of secure development practices. This Dagstuhl Seminar now sought to compile a volume merging empirical software engineering and security research to assist the involved communities, including industry and academia, in focusing their research efforts, and to help newcomers to our field find fertile research areas.

The seminar was designed to be highly interactive, with only three introductory presentations on how security researchers, software engineering researchers, and practitioners think about secure software engineering, and which challenges they perceive, particularly with respect to empirical assessment and evidence. Participants then regularly regrouped in altogether one dozen interactive breakout sessions on various topics covering all activities of a prototypical secure development lifecycle, with the intention of eventually gaining the ability to formulate chapters in a to-be-written textbook on the subject.

A special highlight of the seminar was the remote talk by Steve Lipner, former security executive at Microsoft and now executive director of SAFECode, who recapped the most interesting recollections about his introduction of the first secure development lifecycle at Microsoft some 25 years ago, known as the Window Security Push, details about which can be found in the full report.

Copyright Eric Bodden, Sam Weber, and Laurie Williams

Motivation

The goal of this Dagstuhl Seminar is to bring together members of three communities – academic researchers in cybersecurity, industrial practitioners and software engineering researchers – to tackle the problem of empirically evaluating secure development processes.

In the past decades, the cybersecurity community has created many principles and practices for developing secure software. However, this knowledge has generally been assembled by the application of common sense and experience, and while individual measures and techniques are often based on real-world data and root-cause analysis, broader strategies and processes for creating secure software and assessing software security are usually not subjected to rigorous evaluation. This is a serious shortcoming: common sense can be mistaken and experiences over-generalized. Evaluation techniques are necessary to provide a firm scientific basis that can support progress in the field.

Some such techniques do exist for the later software development stages – implementation and testing. Here one enjoys good automation, and the mapping between technique and end-product is relatively clear-cut. It is also in these stages where security teams succeed at least partially in providing software developers with concrete prescriptive guidance. Nonetheless, the earlier developmental stages – requirements elicitation, threat modeling, architecture – are at least as critical to the security of the final product if not more so, yet pose a much greater experimental challenge because of the gap between the process and product. Experience has shown only limited success at incorporating security into these crucial initial stages.

This seminar wants to enable conversations between these three traditionally-separate communities and drive empirical research to help the developers of secure software. Through these conversations, we aim to bring empirical research practices already established in the software engineering community to the security research community, to make software engineering researchers aware of the important issues in software security, and to make academic researchers aware of the practical constraints and challenges faced by industry.

As a concrete eventual outcome of this Dagstuhl Seminar, we plan to produce an edited open-access book collecting essays of what we know and do not know about secure software practices and processes from a sound scientific empirical perspective. This volume will merge empirical software engineering and security research to assist the involved communities, including industry and academia, in focusing their research efforts, and help newcomers to our field find fertile research areas.

In order to do this, this seminar will engage participants in multiple breakout sessions, both free-form and more formally organized, to foster discussions and generate ideas. By seminar end, we intend to have produced the book outline and multi-disciplinary teams that will work together to complete it.

Copyright Eric Bodden, Brendan Murphy, Sam Weber, Laurie Williams, and Steve Lipner

Participants

Related Seminars
  • Dagstuhl Seminar 19231: Empirical Evaluation of Secure Development Processes (2019-06-02 - 2019-06-07) (Details)

Classification
  • Cryptography and Security
  • Software Engineering

Keywords
  • empirical software engineering
  • secure system design