Compared to a decade ago, publishing research in software engineering has gotten objectively more complex. Expectations for research articles toward artifact sharing, open science, and replicability have increased. For instance, major conference venues require a statement on data/artifact availability on submission. A mandatory data availability section is included in the published version of each article, making the reasons for potentially not sharing research artifacts public.

Open Science and reproducibility initiatives have made rapid developments in the past five years. Likewise, the processes involved in this have been the subject of studies inspecting various aspects of their design and implementation (e.g. the availability of shared research artifacts). However, the different communities in software engineering research do not coordinate efforts and, thus, the processes differ in their understanding of open science. Researchers new to the field (e.g., junior PhD students) struggle to understand the community values and processes.

Our goal is to provide a means to inform the community about research artifact sharing and open science practices. To this end, we want to bring together people involved in the advancement and study of open science and research artifact sharing in software engineering and other disciplines of computer science. We will discuss the current state of the art of the different areas of open science, including replication, artifact sharing, registered reports, as well as incentives in the process. Furthermore, we will identify other relevant aspects and inspect selected other communities within computer science and outside. Our collective outcome will be practical guidelines for research work in software engineering. We envision a joint book (preferably in a living document style) and/or the establishment of a special issue in a journal or even the establishment of a new journal to foster the development of standards and encourage regular updates to standards. We also want to inspire policy change and studies accompanying these changes.

Creative Commons BY 4.0

Kelly Blincoe, Neil Ernst, Ben Hermann, and Janet Siegmund