Coordination and Fusion in Multimodal Interaction

Motivation

More effective, efficient, and natural interfaces to support access to information, applications, and people are increasingly relevant in our information society which is plagued by information overload from corporate, national, and global information webs, increasing system complexity, and shrinking task time lines. Multimodal interfaces combine speech, gesture, and mimics input and output. They include a number of new facilities that promise to mitigate these modern challenges including:

1. Understanding of possibly imprecise, ambiguous, or partial multimodal input
2. Generation of coordinated, cohesive, and coherent multimodal presentations
3. Management of the interaction (e.g., task completion, tailoring interaction styles, adapting the interface) by representing, reasoning, and exploiting models of the user, domain, task, context and the media (e.g., text, audio, video) itself
4. Assistance in the design and development of multimodal user interfaces that decreases the time, expense, and level of expertise necessary to construct successful user interfaces

The purpose of this seminar is to bring together the global research leadership of this area in a focused setting to assess current advances, identify fundamental research challenges, and chart the course of collaborative research in this strategic area. As multimodal interaction is by its nature multidisciplinary, an added objective will be to enhance the awareness by the leadership of subcommunities (e.g. in speech and language processing, image processing, interface design, human factors) of each other's research and methods.