January 5 – 10 , 2020, Dagstuhl Seminar 20021

Spoken Language Interaction with Virtual Agents and Robots (SLIVAR): Towards Effective and Ethical Interaction


Laurence Devillers (LIMSI – Orsay, FR)
Tatsuya Kawahara (Kyoto University, JP)
Roger K. Moore (University of Sheffield, GB)
Matthias Scheutz (Tufts University – Medford, US)

For support, please contact

Susanne Bach-Bernhard for administrative matters

Shida Kunz for scientific matters


Recent times have seen growing interest in spoken language-based interaction between human beings and so-called ‘intelligent’ machines. Presaged by the release of Apple’s Siri in 2011, speech-enabled devices - such as Amazon Echo, Google Home, and Apple HomePod - are now becoming a familiar feature in people’s homes. Coming years are likely to see the appearance of more embodied social agents (such as robots), but, as yet, there is no clear theoretical basis, nor even practical guidelines, for the optimal integration of spoken language interaction with such entities.

One possible reason for this situation is that the spoken language processing (SLP) and human-robot interaction (HRI) communities are fairly distinct, with only modest overlap. This means that spoken language technologists are often working with arbitrary robots (or limit themselves to conversational agents), and roboticists are typically using off-the-shelf spoken language components without too much regard for their appropriateness. As a consequence, an artefact’s visual, vocal, and behavioural affordances are often not aligned (such as providing non-human robots with inappropriate humanlike voices), and usability suffers – the human-machine interface is not ‘habitable’.

These usability issues can only be resolved by the establishment of a meaningful dialogue between the SLP and HRI communities. Both would benefit from a deeper understanding of each other's methodologies and research perspectives through an open and flexible discussion. The aim of this Dagstuhl Seminar is thus to bring together a critical mass of researchers from the SLP and HRI communities in order to (i) provide a timely opportunity to review the critical open research questions, (ii) propose appropriate evaluation protocols for speech-based human-robot interaction, (iii) investigate opportunities to collect and share relevant corpora, and (iv) consider the ethical and societal issues associated with such machines.

Examples of issues to be addressed at the seminar are:

  • How real is the ‘habitability’ issue, how can it be measured, and what steps could/should be taken to mitigate its effects?
  • What are the dimensions of multimodal affordances in human-robot interaction involving spoken language?
  • What role do emotions play in speech-based HRI?
  • What is the relation between human ‘natural’ language and the language deployed in speechbased human-robot interaction?
  • What can be learnt about the process of ‘understanding’ language that is situated and grounded in real-world action and interaction by using robots as an investigative platform?
  • To what extent do robots have to understand gestures, eye-gaze, and non-linguistic sounds as part of natural language interactions?
  • To what extent do robots have to understand indirect speech acts?
  • How important is the ability to acquire/learn/integrate novel linguistic concepts in real-time?
  • What issues arise from environments containing multiple agents (human or non-human)?
  • What standards and resources already exist, and what is missing?
  • What ethical issues arise from the development of spoken language enabled artefacts?

  Creative Commons BY 3.0 DE
  Laurence Devillers, Tatsuya Kawahara, Roger K. Moore, and Matthias Scheutz


  • Artificial Intelligence / Robotics
  • Society / Human-computer Interaction


  • Spoken language technology
  • Human-robot interaction
  • Embodied conversational agents


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.

NSF young researcher support