Dagstuhl Seminar 19132
Users and Automated Driving Systems: How Will We Interact with Tomorrow's Vehicles?
( Mar 24 – Mar 29, 2019 )
- Susanne Boll (Universität Oldenburg, DE)
- Andrew Kun (University of New Hampshire - Durham, US)
- Andreas Riener (TH Ingolstadt, DE)
- C. Y. David Yang (AAA Foundation for Traffic Safety - Washington, US)
- Andreas Dolzmann (for scientific matters)
- Annette Beyer (for administrative matters)
- Exploring the concept of the (future) mobile office : article in AutomotiveUI '19 Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings - Janssen, Christian P.; Kun, Andrew L.; Brewster, Stephen; Boyle, Linda Ng; Brumby, Duncan P.; Chuang, Lewis L. - New York : ACM, 2019. - Pages 465-467.
In today’s vehicles, the driving task is increasingly often shared between the driver and the vehicle. The success of automation will, however, be highly dependent on how well we can design human-machine interaction (HMI) for automated vehicles. This is the motivation for this Dagstuhl seminar, which will address the following inter-related research questions:
- Communicating automation mode: With automated vehicles, automation and humans will share control responsibilities. How can in-vehicle HMI help communicate the automation mode of operation to the driver and other traffic participants?
- Transfer of control: How can automation safely transfer control to the driver? This is a critical question, especially in the near term, because in the near term transfer of control will occur frequently.
- Trust: Drivers must have trust in the automation features in order to take advantage of them. How does the design of HMI in automated vehicles affect trust in automation?
- Creating a place for work and play: One important benefit of automation is that drivers can become passengers, and thus use the time in the vehicle to work or play. How can HMI help people in vehicles take advantage of their newfound freedom from driving?
- Communication between traffic participants: In the foreseeable future the transportation environment will include automated and manually driven vehicles, pedestrians, bicyclists, and others. How can HMI support communication between all transportation users?
- Advanced technologies for in-vehicle HMI: How can we best utilize advanced technologies, such as natural language processing and augmented reality, for HMI in automated vehicles?
The goals for the seminar are as follows:
- Overview of state-of-the-art technologies, methods, and models. We will create an overview of successes and challenges in research, development, and implementation for HMI for automated vehicles.
- List of research problems and hypotheses. We will create a list of key research problems and accompanying hypotheses. We expect that in the coming 3 to 10 years these problems and hypotheses will serve as inspiration for the research of the seminar attendees, and more broadly the communities involved in designing HMI for automated vehicles.
- Roadmap for research. We will aim to create a roadmap for addressing the problems and hypotheses. The roadmap would include proposed methods of dissemination, proposed research collaborations, as well as current, and recommended new, funding mechanisms.
This seminar is a follow-up to the 2016 Dagstuhl seminar 16262 “Automotive User Interfaces in the Age of Automation.”
For much of the time since the invention of the automobile, human-machine interaction (HMI) in vehicles was reasonably clear: drivers controlled the vehicle by manipulating the steering wheel, pedals, and a few levers, buttons, or similar mechanical input devices . They received information about the state of the vehicle through dials, warning lights, and sounds. And, they interacted with a relatively simple in-vehicle entertainment device: the radio, or perhaps the cassette- or CD player.
It is true that the number of input and output devices increased dramatically over the years - for example in the late 1950s, the Ford Edsel was described as a “devilish assemblage of gadgets” . The Edsel was soon out of production, but the number of gadgets kept climbing. It is also true that drivers sometimes operated the vehicle when they were tired, and fell asleep at the wheel. Other times they consumed too much alcohol, and were not able to safely control their vehicle. Yet, the basic concepts of human-machine interactions in the vehicle were well-defined for research and development purposes. The driver’s primary task was to drive: keep the vehicle on the road, avoid crashes, maneuver through traffic, and ultimately reach a destination. The driver also engaged in secondary tasks, such as manipulating the radio, as well as other non-driving-related tasks, such as talking to passengers, and eating. Creating good human-machine interfaces meant supporting the driver in these primary and secondary tasks, while assuring safety for everyone on the road.
Then, with the introduction of mobile computing devices, engagement in secondary tasks, such as talking to remote conversants, as well as sending text messages, and manipulating the interfaces of various mobile applications, became a significant issue in cars. In a sense these distractions were the same as those that drivers faced with the myriad buttons in the Ford Edsel. But, there were differences too: the Edsel did not allow the driver to communicate to remote conversants, nor did it have a touch-screen with ever-changing content.
Today, we again find ourselves at a crossroads. Our cars have myriad buttons, as well as different mobile technologies, both for drivers and for passengers. But, additionally, the primary task of driving is often shared between the driver and the vehicle . Most studies in distracted driving tend to focus on how non-driving activities serve as a distraction from the primary task of vehicle control. In the context of highly automated vehicles (HAV), driving will be the distraction from non-driving activities . Sometimes, the vehicle can effectively take over the driving task, and we can expect that this will become the norm rather than the exception in the foreseeable future: the driving task will be automated, at least on some road segments, and the driver will become a passenger. Yet, in this same foreseeable future we can also expect that the vehicle will sometimes hand the driving task back to the driver, who will have to transition back from the role of passenger to the role of the driver , . This is the new landscape of in-vehicle human-machine interaction, and it presents a number of research questions that we addressed in this Dagstuhl seminar. In the rest of this report, we introduce pre-workshop tasks and summarize the activities and outcome of the seminar. Automated traffic is a challenge not limited to the interaction between a human driver and an automated vehicle. Automated vehicles will be part of a mixed traffic with other traffic participants of less or no automation. Also further traffic participants such as pedestrian and bicycles are part of this and requires a certain level of communication and recognition of the vehicles intention and actions among vehicles and the surrounding traffic participants.
Research questions tackled in Dagstuhl seminar 19132
- Handover: One of the key questions in designing in-vehicle human-machine interaction for partially automated vehicles is, how can the vehicle safely hand back the task of controlling the vehicle to the driver. In the short term this is one of the most important questions for those designing vehicles with automation, because in the short term such vehicles will have to hand control back to the driver relatively often , . We need to understand how the modality, conveyed information, and reliability of take-over requests (TORs), engagement in non-driving tasks, and motion perception can influence drivers performance in task switching in highly automated driving context .
- Trust: Drivers must trust the automation features in order to take advantage of them . We need to individually understand the trust in the individual actions of the vehicle starting out from assistance systems  to more automated functions , .
- Creating a place for work and play: One important benefit of automation would be that drivers can become passengers, and thus use the time in the vehicle to either be productive (work), or relax (play). How can human-machine interaction for automated vehicles be designed, such that drivers can take advantage of their newfound freedom from driving , ? How can we do this, taking into account the physical and computational characteristics of the vehicle, as well as the potential for motion sickness?
- Communication between all traffic participants: With the advent of automation, the transportation environment will include partially and fully automated vehicles. Yet, manually driven vehicles will remain for the foreseeable future, as will pedestrians, bicyclists, and other transportation users. For safe driving, all of these transportation users will have to communicate, but it is not yet clear how this can best be accomplished .
- Advanced technologies for in-vehicle HMI: The technologies that are available for human-machine interaction are continuously improving. Two exciting technologies that will be worth examining in the context of automated vehicles are speech interaction (e. g.), and augmented reality e. g.  and .
- Legal aspects of in-vehicle interfaces: Automation, as well as the user interfaces built for partially and fully automated vehicles, will have to fit into the legal structures of the countries where the vehicles are used . What are these structures, what do designers need to know about them, and how can they help develop the future legal structures?
- Ingrid Pettersson, Florian Lachner, Anna-Katharina Frison, Andreas Riener, and Andreas Butz. A bermuda triangle?: A review of method application and triangulation in user experience evaluation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, pages 461:1–461:16, New York, NY, USA, 2018. ACM.
- Akamatsu, M., Green, P., & Bengler, K. Automotive technology and human factors research: Past, present, and future. In International journal of vehicular technology, 2013.
- Borojeni, S.S., Weber, L., Heuten, W., & Boll, S. From reading to driving: priming mobile users for take-over situations in highly automated driving. Paper presented at the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2018.
- Borojeni, S. S., Chuang, L., Heuten, W., & Boll, S. Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving. Paper presented at the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2016.
- Brooks, J. Business adventures: Twelve classic tales from the world of wall street: Open Road Media. 2014.
- Hancock, P. A. Driven to distraction and back again. In In J. Lee (Ed.), Driver distraction and inattention: Advances in research and countermeasures, (pp. 9-25): CRC Press., 2013.
- Inners, M., & Kun, A. L. Beyond Liability: Legal Issues of Human-Machine Interaction for Automated Vehicles. In 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, (pp. 245-253), Oldenburg, Germany, 2017. ACM.
- Kun, A., Paek, T., & Medenica, Z. The effect of speech interface accuracy on driving performance. Paper presented at the Eighth Annual Conference of the International, 2007.
- Kun, A. L., Boll, S., & Schmidt, A. Shifting Gears: User Interfaces in the Age of Autonomous Driving. In IEEE Pervasive Computing, 15(1), 32-38., 2016.
- Kun, A. L., Tscheligi, M., Riener, A., & Meulen, H. v. d. ARV 2017: Workshop on Augmented Reality for Intelligent Vehicles. Paper presented at the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct, Oldenburg, Germany, 2017. ACM.
- Kun, A. L., van der Meulen, H., & Janssen, C.P. Calling while driving: An initial experiment with HoloLens. Paper presented at the Driving Assessment, 2017.
- Kun, A. L., Wachtel, J., Miller, W. T., Son, P., & Lavallière, M. ARV 2017: Workshop on Augmented Reality for Intelligent Vehicles. Paper presented at the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2015.
- Löcken, A., Heuten, W., & Boll, S. AutoAmbiCar: Using Ambient Light to Inform Drivers About Intentions of Their Automated Cars. Paper presented at the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications Adjunct 2016.
- Merat, N., Jamson, A. H., Lai, F. C., Daly, M., & Carsten, O. M. User interfaces for first responder vehicles: views from practitioners, industry, and academia. In Transportation research part F: traffic psychology and behaviour, 27, 274-282., 2014.
- Mok, B., Johns, M., Miller, D., & Ju, W. Tunneled In: Drivers with Active Secondary Tasks Need More Time to Transition from Automation. Paper presented at the 2017 CHI Conference on Human Factors in Computing Systems, CHI , 2017.
- Rasouli, A., Kotseruba, I., & Tsotsos, J. K. Agreeing to Cross: How Drivers and Pedestrians Communicate. In arXiv preprint, arXiv:1702.03555., 2017.
- Riener, A., Boll, S., & Kun, A. L. Automotive User Interfaces in the Age of Automation (Dagstuhl Seminar 16262). Paper presented at the Dagstuhl Reports., 2016. 1
- van der Meulen, H., Kun, A. L., & Janssen, C.P. Switching Back to Manual Driving: How Does it Compare to Simply Driving Away After Parking? Paper presented at the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Ann Arbor, MI., 2016.
- Wintersberger, P., Riener, A. Trust in Technology as a Safety Aspect in Highly Automated Driving. In i-com 15(3), 2016.
- Wintersberger, P., Frison, A.-K., Riener, A., & Boyle, L. N. Towards a Personalized Trust Model for Highly Automated Driving. Paper presented at the Mensch und Computer 2016–Workshopband, 2016.
- Yan, F., Eilers, M., Lüdtke, A., & Baumann, M. Developing a model of driver’s uncertainty in lane change situations for trustworthy lane change decision aid systems. Paper presented at the Intelligent Vehicles Symposium (IV), IEEE, 2016.
- Ignacio J. Alvarez (Intel - Hillsboro, US) [dblp]
- Martin Baumann (Universität Ulm, DE) [dblp]
- Susanne Boll (Universität Oldenburg, DE) [dblp]
- Linda Ng Boyle (University of Washington - Seattle, US) [dblp]
- Stephen Brewster (University of Glasgow, GB) [dblp]
- Duncan Brumby (University College London, GB) [dblp]
- Gary Burnett (University of Nottingham, GB) [dblp]
- Lewis Chuang (LMU München, DE) [dblp]
- Birsen Donmez (University of Toronto, CA) [dblp]
- Frank Flemisch (Fraunhofer FKIE - Wachtberg, DE) [dblp]
- Joanne Harbluk (Transport Canada - Ottawa, CA) [dblp]
- Christian P. Janssen (Utrecht University, NL) [dblp]
- Wendy Ju (Cornell Tech - New York, US) [dblp]
- Josef Krems (TU Chemnitz, DE) [dblp]
- Andrew Kun (University of New Hampshire - Durham, US) [dblp]
- Sabine Langlois (IRT SystemX, FR) [dblp]
- Roderick McCall (Luxembourg Inst. of Science & Technology, LU) [dblp]
- Alexander Meschtscherjakov (Universität Salzburg, AT) [dblp]
- Ingrid Pettersson (Volvo - Göteborg, SE) [dblp]
- Bastian Pfleging (TU Eindhoven, NL) [dblp]
- Andreas Riener (TH Ingolstadt, DE) [dblp]
- Maria Rimini-Doering (Robert Bosch GmbH - Stuttgart, DE) [dblp]
- Nele Rußwinkel (TU Berlin, DE) [dblp]
- Albrecht Schmidt (LMU München, DE) [dblp]
- Ronald Schroeter (Queensland University of Technology - Brisbane, AU) [dblp]
- Steven E. Shladover (University of California at Berkeley, US) [dblp]
- Jaka Sodnik (University of Ljubljana, SI) [dblp]
- Christine Sutter (Deutsche Hochschule der Polizei - Münster-Hiltrup, DE) [dblp]
- Bruce N. Walker (Georgia Institute of Technology, US) [dblp]
- C. Y. David Yang (AAA Foundation for Traffic Safety - Washington, US) [dblp]
- Jürgen Ziegler (Universität Duisburg-Essen, DE) [dblp]
- Dagstuhl Seminar 16262: Automotive User Interfaces in the Age of Automation (2016-06-26 - 2016-07-01) (Details)
- Dagstuhl Seminar 21232: Human-Computer Interaction to Support Work and Wellbeing in Mobile Environments (2021-06-06 - 2021-06-11) (Details)
- Dagstuhl Seminar 22222: Radical Innovation and Design for Connected and Automated Vehicles (2022-05-29 - 2022-06-03) (Details)
- mobile computing
- society / human-computer interaction
- human-computer interaction
- automated driving