Dagstuhl Seminar 22392
Transparent Quantitative Research as a User Interface Problem
( Sep 25 – Sep 30, 2022 )
Permalink
Organizers
- Kasper Hornbaek (University of Copenhagen, DK)
- Yvonne Jansen (CNRS - Talence, FR)
- Amelia A. McNamara (University of St. Thomas - St. Paul, US)
- Judy Robertson (University of Edinburgh, GB)
- Chat Wacharamanotham (Swansea University, GB)
Contact
- Andreas Dolzmann (for scientific matters)
- Jutka Gasiorowski (for administrative matters)
Shared Documents
- Dagstuhl Materials Page (Use personal credentials as created in DOOR to log in)
Schedule
Many research fields are currently rethinking their research methods towards more transparent practices. The most rapid progress towards transparency can be observed in fields that were heavily affected by a replication crisis, like psychology, while changes are slower and receive more resistance in interdisciplinary fields, such as human-computer interaction (HCI) and visualization (VIS). In this Dagstuhl Seminar, we want to address the issue of hesitant adoption of transparent research methods by framing it as a user interface problem: the 'interface' of using transparent methods is ill-adapted and needs to better respond to the needs and concerns of researchers and other stakeholders in research (e.g., study participants, journal editors, institutional review board).
This Dagstuhl Seminar will bring together a diverse group of junior and senior researchers from HCI and VIS as well as neighboring fields concerned with transparent quantitative research approaches who together are in a unique position: they can (1) conduct inquiries into problems and barriers in the human aspect of transparent research and (2) contribute to designing and developing potential solutions to support transparent research practices — both within their fields and beyond. The seminar will be guided by survey data collected prior to the seminar to enable a data-driven approach and organized around hackathon activities, guest speakers, and comments from policymakers. The seminar's intended outcome is a synthesis of the collected data, a research agenda, a manifesto of transparent quantitative research practices, and the development of future research collaborations.
We expect these activities to stimulate discussion on questions such as the following: How can we lower technical barriers for authors to share their research materials and preregister data analysis plans? How to encourage teachers of research methods and statistics courses to change their material to prioritize clear statistical communication and associated transparent research practices? How can we quantify the "transparency" of statistical practices in a given field? How could policies balance data privacy while also providing for more transparency?
Expected outcome
- Define a transdisciplinary research agenda for improving transparent quantitative research, including the development of evaluation methods, empirical studies, and new technologies.
- Produce a manifesto for transparent quantitative research practices to be promoted by seminar attendees through their networks, and shared with journal editors and conference chairs by the organizers to increase adoption of practices in transparent quantitative research.
- Developsynergies and networks on this research theme.

- Lonni Besancon (Linköping University, SE) [dblp]
- Sophia Crüwell (University of Cambridge, GB)
- Pierre Dragicevic (INRIA - Bordeaux, FR) [dblp]
- Julien Gori (Sorbonne University - Paris, FR)
- Lahari Goswami (University of Lausanne, CH)
- Lynda Hardman (CWI - Amsterdam, NL & Utrecht University, NL) [dblp]
- Olga Iarygina (University of Copenhagen, DK)
- Yvonne Jansen (CNRS - Talence, FR) [dblp]
- Eunice Jun (University of Washington - Seattle, US)
- Ulrik Lyngs (University of Oxford, GB)
- Amelia A. McNamara (University of St. Thomas - St. Paul, US) [dblp]
- Duong Nhu (Monash University - Clayton, AU)
- Viktorija Paneva (Universität Bayreuth, DE)
- Michael Sedlmair (Universität Stuttgart, DE) [dblp]
- Kavous Selahzadeh Niksirat (University of Lausanne, CH)
- Theophanis Tsandilas (Université Paris-Saclay, Orsay, FR & Inria, Orsay, FR)
- Jan Benjamin Vornhagen (Aalto University, FI)
- Chat Wacharamanotham (Swansea University, GB)
- Erich Weichselgartner (Universität Graz, AT) [dblp]
- Wesley J. Willett (University of Calgary, CA) [dblp]
Classification
- Human-Computer Interaction
Keywords
- Open Science
- Transparency
- Reproducibility
- Statistics