Anticipatory Human-Machine Interaction

Motivation

Even after three decades of research on human-machine interaction (HMI), current systems still lack the ability to predict mental states of their users, i.e., they fail to understand users' intentions, goals, and needs and therefore cannot anticipate their actions. This lack of anticipation drastically restricts their capabilities to interact and collaborate effectively with humans. The goal of this Dagstuhl Seminar is to discuss the scientific foundations of a new generation of human-machine systems that anticipate, and proactively adapt to, human actions by monitoring their attention, behavior, and predicting their mental states. Anticipation might be realized by using mental models of tasks, specific situations and systems to build up expectations about intentions, goals, and mental states that gathered evidence can be tested against.

The seminar will provide an inter-disciplinary forum to discuss this emerging topic by bringing together researchers from a range of fields, including human-computer interaction, cognitive-inspired AI, machine learning, computational cognitive science, and social and decision sciences. We will discuss theoretical foundations, key research challenges and opportunities, new computational methods, and future applications of anticipatory human-machine interaction from three perspectives:

1) Anticipating the intentions, beliefs, mental states and next actions of the user

This perspective focuses on the challenges, as well as possible solutions, for machine anticipation of user intentions, goals, motivations, and behaviors. This includes the underlying question of which mental states need to be modeled and anticipated in the first place. This might be different depending on the purpose of the collaboration or on the specific circumstances.

2) Anticipating the outcome of the machine’s own actions

The ability of a machine to anticipate its own actions and outcomes is important, especially for interaction in teams and for learning purposes. Artificial agents need to have a concept of the outcome of their own actions, as that will naturally affect the collaboration with the human or other agents. In addition, in case the expected outcome of an action is not reached, they have to learn how they can achieve the expected outcome in an alternative manner.

3) Anticipating the outcome of collaborative work and human-machine teaming

The third perspective builds on the previous two. In human-machine teaming, a machine needs to anticipate the human partners’ actions, identify problems, and develop ideas on how the partners in a team can be supported. We will discuss how artificial agents can engage in effective teamwork with humans and which specific abilities are required. We will also discuss whether we can construct artificial agents that behave indistinguishable from human agents – and whether this is even desirable.

As one key outcome of the seminar we plan to prepare a report that we aim to become a key reference for researchers interested in a multi-disciplinary approach to anticipatory HMI and, as such, to provide both an overview of the state of the art as well as a starting point into this emerging area of research.