Dagstuhl Seminar 24161
Research Software Engineering: Bridging Knowledge Gaps
( Apr 14 – Apr 19, 2024 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/24161
Organizers
- Stephan Druskat (German Aerospace Center (DLR), Berlin, DE)
- Lars Grunske (HU Berlin, DE)
- Caroline Jay (University of Manchester, GB)
- Daniel S. Katz (University of Illinois Urbana-Champaign, US)
Contact
- Marsha Kleinbauer (for scientific matters)
- Jutka Gasiorowski (for administrative matters)
Schedule
Research Software Engineering (RSEng) is the practice of applying knowledge, methods and tools from software engineering in research. Software Engineering Research (SER) develops methods to support software engineering work in different domains. The practitioners of research software engineering working in academia – Research Software Engineers (RSEs) – are often not trained software engineers. Nevertheless, RSEs are the software experts in academic research. They translate research to software, enable new and improved research, and create software as an important output of research [1].
Hypothetically, the RSEng community and the SER community could benefit from each other. RSEs could leverage software engineering research knowledge to adopt state-of-the-art methods and tools, thereby improving RSEng practice towards better research software. Vice versa, software engineering research could adopt RSEng more comprehensively as a research object, to investigate the methods and tools required for the application of state-of-the-art software engineering in research contexts [2].
There are currently both unknown and known unknowns that make it hard for either community to attain the benefits mentioned above. We call these unknowns gaps, and we call methods to discover the unknown unknowns and to clarify or resolve the (subsequently) known unknowns bridges.
To find the gaps between research software engineering and software engineering research, and start building bridges between the two communities with the aim to create mutual benefit through reciprocal collaboration, we organized and held a five-day seminar in April 2024 at Schloss Dagstuhl – Leibniz Center for Informatics, as Dagstuhl Seminar 24161 “Research Software Engineering: Bridging Knowledge Gaps”. Here, we report and document the seminar’s program, outputs, and potential outcomes.
Seminar participants
In the past, there has been little focused direct communication between the RSEng and SER communities. Anecdotally, while the German RSEng and SER conferences co-located in 2023 in Paderborn and shared a break room, there was little, and only informal, exchange in session attendance by participants of either conference, and hence little knowledge exchange.
We specifically organized our Dagstuhl Seminar to serve as a bridge across this type of communication gap between the communities, by inviting international experts from both communities as well as individuals who have a track record of working at the intersection between the communities. We also invited experts in adjacent fields such as education and training, and research software funding.
In a pre-seminar survey, we asked participants if they could contribute experience in any of the following areas:
- Research Software Engineering: practicing software engineering in a research context
- Software Engineering: practicing software engineering
- Research on Software Engineering: conducting SER
- Research on Research Software Engineering: conducting research on RSEng
- Teaching, training, instruction
- Research software funding
- Community building
Answers were provided on a 5-point Likert scale: “None” – “A little” – “Some” – “Much” – “A lot”. The results are shown in Figure 1. They suggest that we were mostly successful in bringing together participants with at least some relevant expertise, with the possible exception of research software funding, where more than half of participants claimed little or no expertise. This was partly due to invitees with a known background in research software funding being unavailable to attend the seminar.
Figure 1 Survey responses to a question on experience in a given area. A total
of 35 participants replied to the survey. Each person answered one of five levels of experience in the respective area.
Seminar program
The seminar program was prepared to mainly enable direct collaboration of participants from both the SER and the RSEng communities. Sessions were run either in the plenary, or in breakout groups.
After an introductory session where participants introduced themselves and their work, short presentations were given on ideas for collaborative groups to form at the seminar. From the original set of ideas, participants eventually formed a total of seven breakout groups to work on specific topics. Their work is summarized in the respective working group abstracts in this report. In addition, three additional ideas were brought forward in the course of the seminar in a more informal manner outside of sessions. For these additional ideas, work would either be done in parallel to the sessions, or was planned to be done after the seminar. An overview of all topics is given in Table 1.
Table 1 Topics of working groups and breakout initiatives worked on at the seminar.
The introductory session was rounded off by a primer presentation of research software engineering, and a preliminary discussion of options for the dissemination of seminar outcomes. Based on early feedback from participants, we decided to give up on our original idea of collecting seminar outcomes as chapters of an edited volume - a field manual for research software engineering. Instead, we discussed options for collecting some of the outputs in a journal special issue, while leaving it generally up to groups to determine the best way for their outputs to be disseminated.
For the remainder of the seminar, work was originally planned to take place in breakout groups, with regular reporting and feedback sessions in the plenary. The third day was planned as an exception, with two fishbowl sessions scheduled for the morning sessions, and a group excursion in the afternoon.
Based on feedback from participants on the first day of the seminar, we realized that there was a wider perception gap between RSEs and software engineering researchers than originally anticipated: software engineering researchers specifically did not feel entirely confident in their understanding of the scope and practice of RSEng, and of what research software projects looked like. Vice versa, RSEs did not feel entirely confident in their understanding of the aims and scope of software engineering research. We aimed to address this gap on the second day of the seminar with an ad-hoc "Ask us anything" plenary session where software engineering researchers asked members of the RSEng community questions about their work and experience, and with a subsequent ad-hoc session where RSEs presented the contexts they work in, setup of RSE groups, and particular research software projects.
In light of the need to address this kind of feedback quickly, we moved to adapt the original schedule for each day the night before, incorporating feedback we have had during the day.
In the same spirit as the "Ask us anything" sessions, we ran two sessions of "mythbusting fishbowls," where members of the SER, and then the RSEng, community formed a dynamic panel who discussed topics suggested by the respective other community: preconceptions that one community had about the other's work were fielded via sli.do, voted for by the audience, and the most popular ones then discussed by the panel. Panel members were replaced whenever another community member wanted to contribute to the discussion.
Outputs of the seminar activities are presented in brief in the next section.
Outputs
A central outcome of the seminar is that community members from the SER and RSEng communities started collaborations to identify and bridge gaps between them. The outputs of the working groups will be invited to be submitted as articles to a special issue “Research Software Engineering: Discovering and Bridging Knowledge Gaps” in IEEE Computing in Science & Engineering, to be edited by the seminar organizers and published in 2025.
Beyond collecting outputs in this way, one of the working groups developed and published a website that aims to collect outputs from the seminar in a way accessible to a wider public and invites contributions from the community: ser-rse-bridge.github.io. The website also includes a mapping of terms between SER and RSEng, based on the Guide to the Software Engineering Body of Knowledge [3], which is currently under development (see Section 3.1 of the full report).
Additionally, a series of short videos was produced during the seminar. In these videos, participants introduce central SER and RSEng knowledge concepts in under a minute. These can be used on social media platforms to create interest in these topics with, e.g., students looking to choose their courses.
Conclusion and future work
In conclusion, Dagstuhl Seminar "Research Software Engineering: Bridging Knowledge Gaps" (24161) was successful in bringing together members of two communities that have a vested interest in research software engineering: research software practitioners and software engineering researchers. Together with software education and community experts, we learned about each others' work and started conversations and collaborations.
Creating conversations between separate communities and their cultures and codes proved to be challenging at times, e.g., where incentives differed. Where we observed antagonism, or where it was brought to our attention by participants, we tried to defuse it and steer conversations into a constructive direction. We are confident that by and large, this worked well due to flexibility on the side of the participants and a general will to collaborate and progress.
We found that adapting the program to the needs of participants where possible while maintaining the general direction, and arguably intensive workload, helped make the seminar very productive and engaging. Participants have continued to engage after the seminar to identify gaps and potential bridges between the communities.
Future work should focus on the continuation of the efforts started at the seminar, and continued communication and collaboration between the communities. We believe that the seminar marked a starting point for collaboration that can realize future reciprocal benefit for research software engineering and software engineering research in equal measure. Interested parties can refer to the seminar website at https://ser-rse-bridge.github.io/ for related resources and activities.
References
- J. Cohen, D. S. Katz, M. Barker, N. Chue Hong, R. Haines, and C. Jay, “The Four Pillars of Research Software Engineering,” IEEE Software, vol. 38, no. 1, pp. 97–105, Jan. 2021, doi: 10.1109/MS.2020.2973362.
- M. Felderer, M. Goedicke, L. Grunske, W. Hasselbring, A.-L. Lamprecht, and B. Rumpe, “Toward Research Software Engineering Research,” Zenodo, Jun. 2023. doi: 10.5281/zenodo. 8020525.
- P. Bourque and R. E. Fairley, Eds., Guide to the Software Engineering Body of Knowledge, Version 3.0. IEEE, 2014.
Creative Commons BY 4.0
Stephan Druskat, Lars Grunske, Caroline Jay, and Daniel S. Katz
Research software is crucial to all computational research across many academic disciplines. Multiple communities are invested in research software, including computer scientists, and first and foremost among them, software engineering researchers, who investigate and develop methods to improve the quality of software and its lifecycle processes. The software engineering for science community does this specifically for research software. Research software engineers (RSEs) in turn select, customize, and apply software engineering methods from computer science within the domain of academic research to create research software.
There is currently an insufficient transfer of state-of-the-art research knowledge from computer science to research software engineering, and vice versa, in part leading to an incomplete understanding in computer science of the domain-specific and general challenges in research software engineering.
This interactive seminar therefore brings the computer science and software engineering research community and the research software engineering community together to define a common language, and apply it to improve reciprocal knowledge transfer, beginning with five key topics:
- Effective specification of the requirements of research software in different application and research disciplines.
- Design, modeling, implementation, operation, reproducibility and maintenance of research software based on the current state of the art in computer science, and specifically (research) software engineering.
- Formal and informal education and training of RSEs in state-of-the-art software engineering research and community building, and vice versa, the provision of better access for the software engineering research community to the problems and challenges that research software engineers face in practice.
- Tackling the challenges of increasingly complex domain problems, data, and software while operating under the resource constraints in current research ecosystems.
- Generalizing the diversity of opinions, experiences, and practices of research software engineering to successfully inform and influence computer science research.
Initialized by select input talks, participants from the different communities will work interdisciplinarily in dynamic breakout groups to discuss the issues at hand, identify potential solutions, and initiate common research programmes. A goal of this Dagstuhl Seminar is that the outcomes be drafted as chapters for a research software engineering field manual.
Stephan Druskat, Lars Grunske, Caroline Jay, and Daniel S. Katz
- Stuart Allen (Cardiff University, GB)
- David E. Bernholdt (Oak Ridge National Laboratory, US)
- Jeffrey Carver (University of Alabama, US) [dblp]
- Mikaela Cashman McDevitt (Lawrence Berkeley National Laboratory, US)
- Neil Chue Hong (University of Edinburgh, GB) [dblp]
- Myra B. Cohen (Iowa State University - Ames, US) [dblp]
- Hannah Cohoon (University of Utah - Salt Lake City, US)
- Ian Cosden (Princeton University, US)
- Stephan Druskat (German Aerospace Center (DLR), Berlin, DE)
- Nasir Eisty (Boise State University, US)
- Michael Felderer (DLR - Köln, DE) [dblp]
- Carole Goble (University of Manchester, GB) [dblp]
- Michael Goedicke (Universität Duisburg - Essen, DE) [dblp]
- Samuel Grayson (University of Illinois - Urbana-Champaign, US)
- Lars Grunske (HU Berlin, DE) [dblp]
- Robert Haines (University of Manchester, GB) [dblp]
- Wilhelm Hasselbring (Universität Kiel, DE) [dblp]
- Toby Hodges (The Carpentries - Oakland, US)
- Caroline Jay (University of Manchester, GB) [dblp]
- Guido Juckeland (Helmholtz-Zentrum Dresden-Rossendorf, DE)
- Daniel S. Katz (University of Illinois Urbana-Champaign, US) [dblp]
- Timo Kehrer (Universität Bern, CH) [dblp]
- Anna-Lena Lamprecht (Universität Potsdam, DE)
- Christopher Klaus Lazik (HU Berlin, DE) [dblp]
- Jan Linxweiler (TU Braunschweig, DE)
- Frank Löffler (Friedrich-Schiller-Universität Jena, DE)
- Michael Meinel (DLR - Berlin, DE)
- Sebastian Müller (HU Berlin, DE) [dblp]
- Lata Nautiyal (University of Bristol, GB)
- Bernhard Rumpe (RWTH Aachen, DE) [dblp]
- Heidi Seibold (München, DE) [dblp]
- Jan Philipp Thiele (Weierstraß Institut - Berlin, DE)
- Colin Venters (University of Huddersfield, GB) [dblp]
- Samantha Wittke (CSC Ltd. - Espoo, FI)
- Yo Yehudi (Open Life Science - London, GB)
- Shurui Zhou (University of Toronto, CA) [dblp]
Related Seminars
- Dagstuhl Perspectives Workshop 16252: Engineering Academic Software (2016-06-19 - 2016-06-24) (Details)
Classification
- Computers and Society
- Software Engineering
Keywords
- Research Software Engineering
- Software Engineering
- Data Science
