June 19 – 24 , 2016, Dagstuhl Perspectives Workshop 16252

Engineering Academic Software


Carole Goble (University of Manchester, GB)
James Howison (University of Texas – Austin, US)
Claude Kirchner (INRIA – Le Chesnay, FR)
Oscar M. Nierstrasz (Universität Bern, CH)


Jurgen J. Vinju (CWI – Amsterdam, NL)

1 / 4 >

For support, please contact

Dagstuhl Service Team


Dagstuhl Report, Volume 6, Issue 6 Dagstuhl Report
Aims & Scope
List of Participants
Shared Documents
Dagstuhl's Impact: Documents available


This Dagstuhl Perspectives Workshop brought together activists, experts and stakeholders on the subject of high quality software produced in an academic context. Our current dependence on software across the sciences is already significant, yet there are still more opportunities to be explored and risks to be overcome. The academic context is unique in terms of its personnel, its goals of exploring the unknown and its demands on quality assurance and reproducibility.

We refer to the IEEE Internet Computing article "Better Software, Better Research" [1] which motivated the topic. In this workshop we took the following perspective of a research team which is in either or both of the following situations:

  • consuming or producing software as an output of the academic process;
  • consuming or producing software as emph{a component} of the research methods.

Society is now in the tricky situation where several deeply established academic fields (eg physics, biology, mathematics) are shifting towards dependence on software, programming technology and software engineering methodology which are backed only by young and rapidly evolving fields of research (computer science and software engineering). Full accountability and even validity of software-based research results are now duly being challenged.

With the outputs of this interactive and productive perspectives workshop, we strive to contribute in a positive manner to the above challenges. We formulated taxonomies with definitions to clarify the domain, we co-authored concrete policy and process documents to improve the status and recognition of academic software development and academic software engineers, and finally we formulated a list of 18 concrete declarations of intent ("I wil" pledges). This list was presented to the WSSSPE community [2] in September 2016 to acquire feedback and it will be the backbone of the Dagstuhl Manifesto document we are editing. It serves to motivate change by proposing policy changes with concrete actions and instilling positive attitudes towards academic software.

Participants.The participants of the workshop came from three major groups. The first group consists of active and visible members of the global academic software engineering community. They represent (formal) institutions such as the Software Sustainability Institute, the Software Carpentry Foundation, and eScience and data science centers from across the globe. The second group contributed researchers in empirical software engineering, with a specific eye on studying the principles and practices of academic software engineering. The final group contributed researchers as an audience: software engineering researchers with a long experience in engineering software for software itself or software for specific academic research fields.

We found that without exception the participants were strongly motivated and able to actively contribute to the proceedings of the workshop; the mix of people proved to be well-balanced. This balance is an accomplishment, given that invitees from computer science were far more likely to know of Dagstuhl workshops than other groups. To attest to our outcomes we've selectively listed three (paraphrased) verbal statements here:

  • "The workshop was a transformational experience for me; I've learned an entire new perspective on my field and I intend to apply the insights in my daily practice."
  • "I had an epiphany yesterday after dinner; now I understand how to connect the data science research at my university to the computer science department."
  • "Before the workshop I had no idea so many initiatives were already underway in [improving] academic software engineering; this has given my understanding of the challenges a real boost and I know what the some of the next steps to take are."

Schedule. The schedule of the workshop was designed to maximize both interactive discussion and work towards tangible outputs. Key points were: to start the day with inspiring presentations to set the stage, then to have at least 40\% of the day time allocated to free discussion time, and to explicitly share successes (output) of each day's breakout groups in a plenary session.

The workshop started on Monday with a quick and tightly timed round of 2 minute personal introductions. Otherwise on Monday, Tuesday and Thursday the program was structured equally: in the morning we would have plenary presentations which included exploratory discussions. These sessions were meant to bring everybody up-to-speed with ongoing and past initiatives. During and after lunch we used a board with sticky notes to define break-out groups. Each break-out group was centred around a specific discussion topic and (usually) a specific idea for an output document was associated with it. After coffee we would go back to the same break-out group to collaboratively record the notes and lessons from each group (stored in a shared online document). Between 17:00 and 18:00 we reconvened and harvested the results of each breakout group with the others. People could and did freely switch between breakout groups but this was not a common thing.

On Wednesday we had an "open-mic" session with 8 presentations of around 10 minutes, sharing experiences and results, before we had a long walk in the surroundings. The organizers also designed an initial skeleton structure and ideas for the manifesto that day.

On Thursday afternoon and Friday morning we all worked together on our Dagstuhl Manifesto by first reworking our notes into the ideas around the manifesto, specifically a list of "I will" pledges with references and motivation. Finally, Friday afternoon a small remaining group re-ordered the group's manifesto notes into a well-structured list of 18 pledges. Two of the organizers remained to continue to edit the current report and the manifesto document.

Output. Output documents of the workshop are organized under the "DagstuhlEAS" organisation on GitHub. This currently features 6 draft documents, including the current report and (a) the manifesto, (b) the Research Software Engineering Handbook, (c) a Literature Survey, (d) a Taxonomy on Software Credit Roles, and (e) a Software Award Proposal. Next to these documents, an R&D project proposal was produced on measuring the impact of academic software.

The remainder of this document summarizes the morning sessions by listing the abstracts of each talk, the afternoon breakouts by describing each topic and its results, and finally the research questions on the topic of engineering academic software we have collected.


  1. Carole Goble. Better software, better research. IEEE Internet Computing, 18(5):4–8, Sep 2014.
  2. Alice Allen, Cecilia Aragon, Christophe Becker, Jeffrey C. Carver, Andrei Chis, Benoit Combemale, Mike Croucher, Kevin Crowston, Daniel Garijo, Ashish Gehani, Carole Goble, Robert Haines, Robert Hirschfeld, James Howison, Kathryn Huff, Caroline Jay, Daniel S. Katz, Claude Kirchner, Kateryna Kuksenok, Ralf Lämmel, Oscar Nierstrasz, Matthew Turk, Rob van Nieuwpoort, Matthew Vaughn, and Jurgen Vinju. Lightning talk: “I solemnly pledge” – a manifesto for personal responsibility in the engineering of academic software. In Proceedings of the Fourth Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE4).
  Creative Commons BY 3.0 Unported license
  Carole Goble and James Howison and Claude Kirchner and Oscar M. Nierstrasz and Jurgen J. Vinju


  • Bioinformatics
  • Data Bases / Information Retrieval
  • Software Engineering


  • Software
  • Software Eng
  • Research Methods
  • Sustainable Software

Book exhibition

Books from the participants of the current Seminar 

Book exhibition in the library, ground floor, during the seminar week.


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.