December 2 – 7 , 2012, Dagstuhl Seminar 12492

Human Activity Recognition in Smart Environments


James Bo Begole (Samsung Research – San Jose, US)
James L. Crowley (INRIA – Grenoble, FR)
Paul Lukowicz (DFKI – Kaiserslautern, DE)
Albrecht Schmidt (Universität Stuttgart, DE)


Kai Kunze (Osaka Prefecture University, JP)

For support, please contact

Dagstuhl Service Team


Dagstuhl Report, Volume 2, Issue 12 Dagstuhl Report
List of Participants
Dagstuhl Seminar Schedule [pdf]

Press Room


Today, commercial systems have become popular that utilize a broad range of sensors to facilitate gesture and motion-based interaction. Examples range from multi-touch surfaces, through tilt control common in mobile phone applications, and complex motion-based games controllers, e.g. Nintendo WII and Microsoft Kinect. While these systems are mainstream, the next basic research challenge is activity-driven, implicit interaction. Two key differences to existing systems are:

  1. the interpretation of complex human activities, and
  2. the extension of interaction from periods where a user consciously performs control gestures to permanent monitoring of user activity.

Conceptually, activity-driven interaction builds on the vision of context awareness developed since the 1990. Applications range from sports, through mobile gaming, information retrieval, personal healthcare to industrial work support. For example, monitoring certain activities can support therapy in areas ranging from cardiovascular diseases to psychiatric disorders and cognitive disabilities. Activity based support (automatically showing correct manual pages, pointing out mistakes) can speed up industrial maintenance tasks by up to 30%.

Despite demonstrated potential, currently only very simple activity based applications such as physical activity monitoring have managed to go beyond early stage lab demonstrations. From the scientific point of view the question is how to map information available from unreliable, often simple sources onto complex human activities. The main challenges stem from the combination of three factors:

  • In every day situations sensor choice, placement and configuration is often dictated by practicability, usability, and user acceptance constraints rather by the requirements of the recognition system. In addition, the system configuration may dynamically change.
  • The diversity of human behavior. Even the simplest activities can be performed in a multitude of ways differing not only between people, but also between individual execution instances of a single person. (e.g. using different arms, different hand positions, or even the hip to close a drawer)
  • The complexity of human behavior. Relevant human actions are seldom atomic and independent. Instead, a complex hierarchy of actions that may be executed in parallel, overlap and interleave is to be considered by the recognition system.

The field lacks also definitions for many commonly used terms including "action," "sensor," "evidence," and even "activity" itself, leading to ambiguity in scientific discourse. The conceptual grounding provided by Nardi and Kaptelinin's definition of Activity Theory are perfectly understandable to a human. Yet, they are not easily codified into machine programmable constructs. The theory recognizes that elements of "activity" such as goal and motive are socially constructed, depending on the perspectives of the actors in the system. Despite the complexities of "activity" at the human cognitive level, researchers demonstrated that some notions of activity can be utilized in computer systems, but meanings of terms differ among the various research groups. Currently, many different communities are involved in research related to activity recognition, including the core ubicomp community, human computer interaction, computer vision, cognitive science and artificial intelligence.

Privacy concerns are a critical barrier to adoption of activity-based technologies. These concerns range from risk of criminal activities( e.g., stalking and identity theft), to social issues of managing personal relationships. Technological approaches to addressing the concerns must also be based on deep understanding of the psychological, sociological and political constraints under which people will operate activity-based systems.

The top level objective of the workshop was to define and establish the scientific community and associated research questions/methodologies related to the broad area of activity recognition. The major tangible outcomes are the start of the creation of an activity recognition repository accessible under and the plan of writing a standard book about activity recognition.


  • Ubiquitous Computing
  • Mobile Computing
  • Artificial Intelligence
  • HCI


  • Context Awareness
  • Activity Recognition
  • Pattern Analysis
  • Sensor Networks
  • Wearable computing
  • Computer Vision


In the series Dagstuhl Reports each Dagstuhl Seminar and Dagstuhl Perspectives Workshop is documented. The seminar organizers, in cooperation with the collector, prepare a report that includes contributions from the participants' talks together with a summary of the seminar.


Download overview leaflet (PDF).

Dagstuhl's Impact

Please inform us when a publication was published as a result from your seminar. These publications are listed in the category Dagstuhl's Impact and are presented on a special shelf on the ground floor of the library.


Furthermore, a comprehensive peer-reviewed collection of research papers can be published in the series Dagstuhl Follow-Ups.