Dagstuhl Seminar 15162
Software and Systems Traceability for Safety-Critical Projects
( Apr 12 – Apr 17, 2015 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/15162
Organizers
- Jane Cleland-Huang (DePaul University - Chicago, US)
- Patrick Mäder (TU Ilmenau, DE)
- Sanjai Rayadurgam (University of Minnesota - Minneapolis, US)
- Wilhelm Schäfer (Universität Paderborn, DE)
Contact
- Susanne Bach-Bernhard (for administrative matters)
This seminar on Software Traceability for Safety-Critical Systems will bring together researchers and industrial practitioners working in the field of dependable systems to explore the needs, challenges, and solutions for Software and Systems Traceability in this domain.
Current guidelines in many safety-critical industries prescribe traceability for two reasons. First, the presence of traceability is as an indirect measure that good practice has been followed. Second, traceability is used more directly to show that specific hazards have been explored, faults identified, and that the delivered product fully addresses them. However, the gap between traceability prescribed by guidelines and that delivered by manufacturers is significant. Our recent analysis of five representative technical guidelines (DO-178B, ISO 26262-6, ECSS-E-40, FDA Guide for Submissions) relevant to four industrial domains: automotive, aviation, medical, and space showed that while all of the projects made some attempt to satisfy the traceability guidelines, serious deficiencies were found. These included missing traceability paths, missing and redundant links, and problems in trace granularity which made it very difficult to analyze the effectiveness of the links and to use them to evaluate the safety of the product.
The seminar is designed to explore the gap between what is prescribed and what is actually delivered. Starting from a clean slate, we will clearly articulate traceability needs for safety-critical software systems. We will then identify challenges, explore practical solutions, and outline a joint industry-academic research agenda for future research and technology transfer.
This Dagstuhl Seminar is expected to produce a deep understanding of traceability needs in safety-critical systems, an analysis of why organizations fail to deliver traceability as prescribed in current guidelines, and a clear vision for solving the identified problems.
We are inviting several groups of participants to the seminar. Some will be practitioners who have first-hand knowledge of building dependable systems, while others will be experienced certifiers or approvers. We are also inviting practitioners and academics, who have experience developing tools to support safety analysis and/or traceability of dependable systems. Finally, we are inviting a representative group of researchers working on dependable systems, traceability, safety-critical product lines, or other topics related to constructing, analyzing, and validating safety-critical systems.
Safety-critical systems, defined as systems whose "failure could result in loss of life, significant property damage, or damage to the environment" pervade our society. Developing software is a challenging process. Not only must the software deliver the required features, but it must do so in a way that ensures that the system is safe and secure for its intended use. To this end safety-critical systems must meet stringent guidelines before they can be approved or certified for use. For example, software developed for the aerospace industry must comply to the ISO12207 and/or the DO-178B/C guidelines, while software developed for European railway communication, signaling, and processing systems, must comply to EN50128. Most guidelines prescribe a set of steps and deliverable documents that focus around planning, analysis and design, implementation, verification and validation, configuration management, and quality assurance activities. In addition they often provide specific guidelines for the creation and use of traceability in the project. For example, depending upon the criticality level of a requirement, the US Federal Aviation Authority guideline DO-178B requires traceability from requirements to design, and from requirements to source code and executable object code.
In practice, traceability is achieved through the creation and use of trace links, defined by the Center of Excellence for Software and Systems Traceability as "specified associations between pair of artifacts, one comprising the source artifact and one comprising the target artifact". Software traceability serves an important role in demonstrating that a delivered software system satisfies its software design constraints and mitigates all identified hazards. When correct, traceability demonstrates that a rigorous software development process has been established and systematically followed. Current guidelines, in many safety-critical industries, prescribe traceability for two reasons. First, as an indirect measure that good practice has been followed, the general idea being that traceability information serves as an indicator that design and production practices were conducted in a sound fashion; and second, as a more direct measure, to show that specific hazards have been explored, potential failure modes identified, and that the system is designed and implemented in a "demonstrably rational way".
Unfortunately, there is a significant gap between prescribed and actual traceability. An analysis of the traceability information submitted by various organizations to the US Food and Drug Administration (FDA) as part of the medical device approval process, showed a significant traceability gap between the traceability expectations as laid out in the FDA's "Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices", and the traceability data documented in the submissions. While all of the submissions made some attempt to satisfy the FDA's traceability guidelines, serious deficiencies were found in almost all the submissions in terms of missing traceability paths, missing and redundant links, and problems in trace granularity. These deficiencies made it very difficult to understand the rationale for individual links. A more recent systematic analysis of seven software projects that originated from four different domains (automotive, aviation, medical, and space) revealed similar problems. The provided software development artifacts were analyzed with respect to four technical guideline documents (ISO 26262-6, DO-178B, FDA Guide for Submissions, ECSS-E-40), where each document is a representative guideline of one of the four domains.
Problems are exacerbated in the systems engineering domain in which core concepts and designs are often documented across multiple models, each of which might depict a single viewpoint or perspective of the system. For example, the system might include separate models for functional and behavioral requirements, software components, electrical components, thermodynamics, and mechanical components. Furthermore, although each of these perspectives is modeled separately in isolation from one another, they interact to produce the final behavior of the system. Traceability solutions must extend across these heterogeneous models. Deficiencies in traceability are certainly not new. As far back as 1995, Gotel et al. identified several different traceability problems and attributed them to poor coordination, lack of perceived benefits, time to market pressures, and lack of sufficient tooling. These problems observed almost 20 years ago, continue to plague the traceability landscape today, meaning that the traceability gap between what is prescribed and what is practiced is still very real.
Given that the software and systems engineering communities have been unable to solve this problem in over 20 years, it seems prudent to reexamine traceability needs and their prescribed solutions. Within this Dagstuhl seminar, we engaged software and systems engineering researchers and practitioners from the safety-critical domain alongside traceability experts, in highly focused discussions. The aim was to gain a deeper understanding of exactly what traceability is needed for safety-critical systems, and to identify practical and achievable solutions. To the best of our knowledge this was the first time researchers from the safety-critical and traceability domains came together in a dedicated forum to tackle this problem.
We started the week with a number of more general presentations and discussions from experts in the respective areas to form a common understanding for later discussions. Subsequently, the seminar continued with shorter talks focusing on a variety of specific aspects of open challenges and potential solutions accompanied by intensive and highly interactive discussions. In parallel, we parted for about one third of the time into four focus groups working on what had been identified as the most relevant and urgent challenges for closing the traceability gap. The four areas of focus were: tracing qualities, traceability in the context of models and tools, cost-benefit and stakeholder perspectives, and traceability in the context of evolution and change. In result, we intend to publish a white-paper that systematically analyzes the existing traceability gap based on the outcome of the four focus groups. Furthermore, the workshop has initiated collaborations and potential research projects between previously separate areas with the potential of significant impact.
Creative Commons BY 3.0 Unported license
Jane Cleland-Huang, Patrick Mäder, Sanjai Rayadurgam, and Wilhelm Schäfer
- Markus Borg (Lund University, SE) [dblp]
- Jane Cleland-Huang (DePaul University - Chicago, US) [dblp]
- Krzysztof Czarnecki (University of Waterloo, CA) [dblp]
- Christopher Gerking (Universität Paderborn, DE) [dblp]
- Paul Grünbacher (Universität Linz, AT) [dblp]
- Lars Grunske (Universität Stuttgart, DE) [dblp]
- Kai Höfig (Siemens AG - München, DE) [dblp]
- Patrick Mäder (TU Ilmenau, DE) [dblp]
- Shiva Nejati (University of Luxembourg, LU) [dblp]
- Leon J. Osterweil (University of Massachusetts - Amherst, US) [dblp]
- Mona Rahimi (DePaul University - Chicago, US) [dblp]
- Sanjai Rayadurgam (University of Minnesota - Minneapolis, US) [dblp]
- Gilbert Regan (Dundalk Institute of Technology, IE) [dblp]
- Patrick Rempel (TU Illmenau, DE) [dblp]
- Mehrdad Sabetzadeh (University of Luxembourg, LU) [dblp]
- Nicolas Sannier (University of Luxembourg, LU) [dblp]
- Wilhelm Schäfer (Universität Paderborn, DE) [dblp]
- Andres Toom (IB Krates OÜ - Tallinn, EE) [dblp]
- Marc Zeller (Siemens AG - München, DE) [dblp]
Classification
- semantics / formal methods
- software engineering
Keywords
- safety-critical software development
- assurance cases
- software and systems traceability
