October 30 – November 4 , 2022, Dagstuhl Seminar 22442

Toward Scientific Evidence Standards in Empirical Computer Science


Brett A. Becker (University College Dublin, IE)
Christopher D. Hundhausen (Oregon State University – Corvallis, US)
Ciera Jaspan (Google – Mountain View, US)
Andreas Stefik (University of Nevada – Las Vegas, US)
Thomas Zimmermann (Microsoft Corporation – Redmond, US)

For support, please contact

Simone Schilke for administrative matters

Michael Gerke for scientific matters

Dagstuhl Reports

As part of the mandatory documentation, participants are asked to submit their talk abstracts, working group results, etc. for publication in our series Dagstuhl Reports via the Dagstuhl Reports Submission System.


List of Participants
Shared Documents
Dagstuhl Seminar Wiki
Dagstuhl Seminar Schedule [pdf]

(Use personal credentials as created in DOOR to log in)


Many scientific fields of study use formally established evidence standards during the peer review and evaluation process, such as CONSORT ( in medicine, the What Works Clearinghouse ( in education, or the APA Journal Article Reporting Standards (JARS) in psychology ( The basis for these standards is community agreement on what to report in empirical studies. Such standards achieve two key goals. First, they make it easier to compare studies, facilitating replications which can provide confidence that multiple research teams can obtain the same results. Second, they establish community agreement on how to report on and evaluate studies using different methodologies.

The discipline of computer science does not have formalized evidence standards, even for major conferences or journals. This Dagstuhl Seminar has three primary objectives:

  1. To establish a process for creating or adopting an existing evidence standard for empirical research in computer science.
  2. To build a community of scholars that can discuss what a general standard should include.
  3. To kickstart the discussion with scholars from software engineering, human computer interaction, and computer science education.

In order to better discuss and understand the implications of such standards across several empirical subfields of computer science and to facilitate adoption, our plan for the seminar includes having representatives from prominent journals in attendance.

Motivation text license
  Creative Commons BY 4.0
  Brett A. Becker, Christopher D. Hundhausen, Ciera Jaspan, Andreas Stefik, and Thomas Zimmermann


  • Computers And Society
  • Human-Computer Interaction
  • Software Engineering


  • Community Evidence Standards
  • Human Factors


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.