Dagstuhl Seminar 26051
User-Aligned Assessment of AI Systems
( Jan 25 – Jan 30, 2026 )
(Click in the middle of the image to enlarge)
Permalink
Please use the following short url to reference this page:
https://www.dagstuhl.de/26051
Organizers
- YooJung Choi (Arizona State University - Tempe, US)
- Georgios Fainekos (Toyota Motor North America, R&D - Ann Arbor, US)
- Siddharth Srivastava (Arizona State University - Tempe, US)
- Hazem Torfah (Chalmers University of Technology - Göteborg, SE)
Contact
- Andreas Dolzmann (for scientific matters)
- Susanne Bach-Bernhard (for administrative matters)
Dagstuhl Reports
As part of the mandatory documentation, participants are asked to submit their talk abstracts, working group results, etc. for publication in our series Dagstuhl Reports via the Dagstuhl Reports Submission System.
- Upload (Use personal credentials as created in DOOR to log in)
Dagstuhl Seminar Wiki
- Dagstuhl Seminar Wiki (Use personal credentials as created in DOOR to log in)
Shared Documents
- Dagstuhl Materials Page (Use personal credentials as created in DOOR to log in)
This Dagstuhl Seminar addresses research gaps in the continual assessment of AI systems amid post-deployment changes in requirements, user-specific objectives, deployment environments, and the AI systems themselves.
With non-expert users increasingly encountering AI systems, the operational domain of AI has expanded from single purpose to more generalized applications. This shift raises broad questions about post-deployment assessment of these systems' limits and capabilities, especially as they tackle user-specific tasks and environments not envisioned during design. The seminar will focus on processes and technical approaches for conceptualizing, managing, and enforcing continual, user-driven assessment of AI systems. Emphasis will be on systems that adapt and learn with evolving user requirements and deployment environments.
The seminar will bring together ideas across two highly active fields of research: AI and formal methods. Some of the key research questions motivating this seminar include:
- Can we design well-founded algorithmic approaches for identifying the limits and capabilities of a learning-enabled AI system?
- Can we design systems that enable users to specify their task objectives and fairness and safety considerations, while avoiding the pitfalls of mis-specified or under-specified preferences and objectives?
- How might we evaluate compliance with such properties efficiently and on-the-fly for new tasks and environments?
- What may be the use cases where a non-expert user needs to assess the performance, safety, and reliability of an AI system?
- What ethical considerations must be considered when developing user-aligned assessment methods for AI systems?
- How might changes in regulatory frameworks affect the development and deployment of user-aligned assessment strategies for embodied AI systems?
- What kinds of assessment protocols and interfaces should new AI systems provide to support such post-deployment assessment?
- Finally, how would assessment approaches differ for embodied vs. purely computational AI agents?
These problems extend beyond classical verification and validation, where operational requirements and system specifications are available a priori. In contrast, adaptive AI systems, such as household robots, may change their control paradigms due to system updates and/or learning, as well as due to adaptation to day-to-day changes in the requirements (which can be user-provided) and in the dynamic environments they operate in. All these factors make assessment of adaptive AI systems an emerging and pressing problem that has received relatively little research attention.
We will explore what it means for users to assess the safety and performance of AI systems that continuously evolve and adapt. Discussions will focus on specifying and managing properties from the user's perspective, and methods for verifying, monitoring, and enforcing safety and alignment.
Creative Commons BY 4.0
YooJung Choi, Georgios Fainekos, Siddharth Srivastava, and Hazem Torfah
Please log in to DOOR to see more details.
- Houssam Abbas (Oregon State University - Corvallis, US) [dblp]
- Chih-Hong Cheng (Chalmers University of Technology - Göteborg, SE) [dblp]
- YooJung Choi (Arizona State University - Tempe, US) [dblp]
- Jyotirmoy Deshmukh (USC - Los Angeles, US) [dblp]
- Bishwamittra Ghosh (MPI-SWS - Kaiserslautern, DE) [dblp]
- Nakul Gopalan (Arizona State University - Tempe, US) [dblp]
- Malte Helmert (Universität Basel, CH) [dblp]
- Till Hofmann (RWTH Aachen, DE) [dblp]
- Nicholas Judd (UL Research Institutes, US) [dblp]
- Sebastian Junges (Radboud University Nijmegen, NL) [dblp]
- Sarah Keren (Technion - Haifa, IL) [dblp]
- Jan Kretinsky (Masaryk University - Brno, CZ) [dblp]
- Hanna Kurniawati (Australian National University - Canberra, AU) [dblp]
- Sterre Lutz (TU Delft, NL)
- Bernhard Nebel (Universität Freiburg, DE) [dblp]
- Dejan Nickovic (AIT - Austrian Institute of Technology - Wien, AT) [dblp]
- Yash Vardhan Pant (University of Waterloo, CA)
- Giulia Pedrielli (Arizona State University - Tempe, US) [dblp]
- Ron Petrick (Heriot-Watt University - Edinburgh, GB) [dblp]
- Ruzica Piskac (Yale University - New Haven, US) [dblp]
- Mark Roberts (Iconium Labs - Washington, US) [dblp]
- Gabriele Röger (Universität Basel, CH) [dblp]
- Oliver Schön (ETH Zürich, CH) [dblp]
- Sanjit A. Seshia (University of California - Berkeley, US) [dblp]
- Scott Shapiro (Yale University - New Haven, US) [dblp]
- Xujie Si (University of Toronto, CA) [dblp]
- Reid Simmons (Carnegie Mellon University - Pittsburgh, US) [dblp]
- Alberto Speranzon (Lockheed Martin - Eagan, US) [dblp]
- Siddharth Srivastava (Arizona State University - Tempe, US) [dblp]
- Hazem Torfah (Chalmers University of Technology - Göteborg, SE) [dblp]
- Martin Törngren (KTH Royal Institute of Technology - Stockholm, SE) [dblp]
- Jana Tumova (KTH Royal Institute of Technology - Stockholm, SE) [dblp]
- Marcell Vazquez-Chanlatte (Nissan North America - Santa Clara, US) [dblp]
- Pulkit Verma (Indian Institute of Techology Madras, IN)
Classification
- Artificial Intelligence
- Logic in Computer Science
- Machine Learning
Keywords
- Continuous assessment
- safe AI
- requirements
- cyber-physical-systems
- robotics
