https://www.dagstuhl.de/22202

May 15 – 20 , 2022, Dagstuhl Seminar 22202

Anticipatory Human-Machine Interaction

Organizers

Jelmer Borst (University of Groningen, NL)
Andreas Bulling (Universität Stuttgart, DE)
Cleotilde Gonzalez (Carnegie Mellon University – Pittsburgh, US)
Nele Rußwinkel (TU Berlin, DE)

For support, please contact

Simone Schilke for administrative matters

Marsha Kleinbauer for scientific matters

Dagstuhl Reports

As part of the mandatory documentation, participants are asked to submit their talk abstracts, working group results, etc. for publication in our series Dagstuhl Reports via the Dagstuhl Reports Submission System.

Documents

List of Participants
Shared Documents
Dagstuhl Seminar Wiki

(Use personal credentials as created in DOOR to log in)

Motivation

Even after three decades of research on human-machine interaction (HMI), current systems still lack the ability to predict mental states of their users, i.e., they fail to understand users' intentions, goals, and needs and therefore cannot anticipate their actions. This lack of anticipation drastically restricts their capabilities to interact and collaborate effectively with humans. The goal of this Dagstuhl Seminar is to discuss the scientific foundations of a new generation of human-machine systems that anticipate, and proactively adapt to, human actions by monitoring their attention, behavior, and predicting their mental states. Anticipation might be realized by using mental models of tasks, specific situations and systems to build up expectations about intentions, goals, and mental states that gathered evidence can be tested against.

The seminar will provide an inter-disciplinary forum to discuss this emerging topic by bringing together researchers from a range of fields, including human-computer interaction, cognitive-inspired AI, machine learning, computational cognitive science, and social and decision sciences. We will discuss theoretical foundations, key research challenges and opportunities, new computational methods, and future applications of anticipatory human-machine interaction from three perspectives:

1) Anticipating the intentions, beliefs, mental states and next actions of the user

This perspective focuses on the challenges, as well as possible solutions, for machine anticipation of user intentions, goals, motivations, and behaviors. This includes the underlying question of which mental states need to be modeled and anticipated in the first place. This might be different depending on the purpose of the collaboration or on the specific circumstances.

2) Anticipating the outcome of the machine’s own actions

The ability of a machine to anticipate its own actions and outcomes is important, especially for interaction in teams and for learning purposes. Artificial agents need to have a concept of the outcome of their own actions, as that will naturally affect the collaboration with the human or other agents. In addition, in case the expected outcome of an action is not reached, they have to learn how they can achieve the expected outcome in an alternative manner.

3) Anticipating the outcome of collaborative work and human-machine teaming

The third perspective builds on the previous two. In human-machine teaming, a machine needs to anticipate the human partners’ actions, identify problems, and develop ideas on how the partners in a team can be supported. We will discuss how artificial agents can engage in effective teamwork with humans and which specific abilities are required. We will also discuss whether we can construct artificial agents that behave indistinguishable from human agents – and whether this is even desirable.

As one key outcome of the seminar we plan to prepare a report that we aim to become a key reference for researchers interested in a multi-disciplinary approach to anticipatory HMI and, as such, to provide both an overview of the state of the art as well as a starting point into this emerging area of research.

Motivation text license
  Creative Commons BY 4.0
  Jelmer Borst, Andreas Bulling, Cleotilde Gonzalez, and Nele Rußwinkel

Classification

  • Artificial Intelligence
  • Human-Computer Interaction
  • Machine Learning

Keywords

  • Computational Cognitive Modelling
  • Computational Theory of Mind
  • Human-Machine Teaming
  • Cognition-aware Computing
  • Goal and Plan Recognition

Documentation

In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.

 

Download overview leaflet (PDF).

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.

Publications

Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.