Generative AI is reshaping the technological foundations of Connected, Cooperative and Automated Mobility (CCAM). At the core of this shift are foundation models, large pre-trained architectures that learn directly from data and generalize across tasks, increasingly spanning the full driving stack. Yet the very properties that make these models attractive for representing open-world, collaboratively perceived environments in real time also make them difficult to assure in terms of trustworthiness. They are probabilistic and prone to hallucination; they underrepresent the rare situations that matter most for vulnerable road users; and they are susceptible to adversarial manipulation, from training data to V2X communication. Conventional assurance frameworks assume systems that can be decomposed into functions with characterizable failure modes, each covered by evidence. An end-to-end foundation model offers no such decomposition: it can be confidently wrong without any internal signal of failure.

The problem reaches into the verification and validation (V&V) itself. When a learned world model generates the scenarios used to validate a system trained on overlapping data, the generator and system under test share the same prior. The evidence becomes circular exactly at the safety-critical edge cases. What generator-independent evidence can break this circle? Formal validity constraints, physically grounded sensor models, real-world crash and behavior data, and human driving behavior models are all candidates. Which of them suffice, and how they combine, remain open questions.

This Dagstuhl Seminar aims to bring together researchers from machine learning and uncertainty quantification, formal methods and knowledge representation, automotive safety engineering and empirical traffic safety, human behavior modeling, vehicular security, and standardization and regulation. These communities rarely share a venue. None of them can resolve this tension alone. The seminar will be organized around three topics that follow the safety lifecycle: monitor the system in operation, verify the reasoning behind its outputs, and trust the cooperation between systems.

The first topic is formal grounding and V&V – both runtime and offline. Confidence alone does not show that a model reasons over the right concepts. How can the implicit knowledge in its parameters be connected – faithfully – to symbolic representations of traffic rules and operational design domains, and how can their safety assessment be operationalized? Neuro-symbolic architectures offer a natural path; can they carry a certification argument that reduces test load and proves acceptable safety compared to human drivers? And can the same formal representations serve both offline V&V, and runtime monitoring, so that what we verify/validate and what we monitor stay consistent?

Directly connected is the topic of uncertainty quantification. How can a foundation model pipeline recognize that it is operating outside its competence? How does uncertainty, also from formal plausibilisation, propagate and combine across camera, LiDAR, radar, V2X, and language inputs into one safety-relevant confidence signal, and how can uncertainty be operationalized in terms of safety? And how can that signal stay reliable under adversarial manipulation, and remain actionable for planners and runtime safety monitors?

The third topic is multi-agent trust. Vehicles, roadside units, and edge infrastructure will fuse the views of independently trained models into a shared situational picture. How should such an epistemic coalition be formed? How can an agent assess a collective state it did not construct? And how can a compromised contribution be detected once fusion has hidden its provenance?

The guiding question for the seminar's roadmap is how a system can continuously assess (both on-line and off-line) and demonstrate the level of trustworthiness on which it operates, in service as well as in offline assessment. Participants are expected to develop a structured catalogue of open problems, an analysis of the gaps in current safety and AI standards, and a prioritised research roadmap. Beyond the Dagstuhl Report, we aim for a joint publication and for dissemination of the roadmap to the CCAM community and to relevant standardization bodies.

Creative Commons BY 4.0

Jonas Bärgman, Thanassis Giannetsos, Ioannis Krontiris, and Gesina Schwalbe