March 14 – 19 , 2004, Dagstuhl Seminar 04121

Evaluating Embodied Conversational Agents


Elisabeth André (Universität Augsburg, DE)
Kristina Höök (Swedish Institute of CS – Kista, SE)
W. Lewis Johnson (USC – Marina del Rey, US)
Catherine Pelachaud (University of Paris, FR)
Zsofia Ruttkay (University of Twente, NL)

For support, please contact

Dagstuhl Service Team


Dagstuhl Seminar Proceedings DROPS
External Homepage
List of Participants


In the past years we have seen an increasing number of ECAs (Embodied Conversational Agents), as results (and sometimes also as medium) of academic research as well as in commercial applications in fields like information providing (news reader, info kiosk), user interfaces for deaf (lip-reading, signing avatars), education, commerce and entertainment. Due to the novelty of the field and the complexity of the problems, different paradigms and tools have been used to produce the individual ECAs, often tailored to the special needs of an application or of a research issue.

The comparison and re-use of ECAs has not been dealt with to a sufficient degree, in spite of the high costs of developing yet new ECAs. There have been studies that indicate added value of particular features of ECAs, but it remains to be determined in what situations fully functional ECAs are advantageous, and for what purpose. The sporadic evaluation experiments concentrated on special issues and under restricted circumstances. All the same, several of these partial evaluations have already produced surprising and for the future development and application of ECAs essential results, e.g. on the different preferences of different users.

For the evaluation and design guidelines of ECAs one should rely on results and methodologies of related fields like HCI, psychology, cultural anthropology and even arts. The different language and methodologies of these disciplines and lack of common forums have hindered the cross-fertilization. Feedback from the industry on commercial applications would also very useful.

Motivated by the above observations, the main objectives of the seminar were:

  • to review the results, tools and resources;
  • identify key problems and future research directions;
  • initiate further activities to trigger and dissimilate work on ECA design and evaluation.


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.