How can humans understand their automated cars? HMI principles, problems and solutions
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
As long as vehicles do not provide full automation, the design and function of the Human Machine Interface (HMI) is crucial for ensuring that the human “driver” and the vehicle-based automated systems collaborate in a safe manner. When the driver is decoupled from active control, the design of the HMI becomes even more critical. Without mutual understanding, the two agents (human and vehicle) will fail to accurately comprehend each other’s intentions and actions. This paper proposes a set of design principles for in-vehicle HMI and reviews some current HMI designs in the light of those principles. We argue that in many respects, the current designs fall short of best practice and have the potential to confuse the driver. This can lead to a mismatch between the operation of the automation in the light of the current external situation and the driver’s awareness of how well the automation is currently handling that situation. A model to illustrate how the various principles are interrelated is proposed. Finally, recommendations are made on how, building on each principle, HMI design solutions can be adopted to address these challenges.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 1×
“Verberne adjusted his definition for automated vehicles based on Mayer et al. (1995) who used vulnerability to the action of another party.”
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“Lee and See (2004) specified calibration of trust as one of three components of appropriate trust.”
Lewis, J. D., & Weigert, A. (1985) · Social Forces · doi:10.2307/2578601 · Cited by 4,163 · cited 1×
“Trust has been referred to as having many different meanings (Williamson 1993), definitions being a confusing potpourri (Shapiro 1987) and leading to conceptual confusion (Lewis and Weigert 1985).”
Rasmussen, J. (1983) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1983.6313160 · Cited by 3,434 · cited 1×
“…to maintain the human at the skills and, where needed, the rules levels, and thus to avoid the slower and less effective (and thus potentially more unsafe) knowledge level (for more information about the skill-based, rule-based and knowledge-based behaviour hierarchy, see Rasmussen 1983, 1987).”
Calculativeness, Trust, and Economic Organization
Williamson, O. E. (1993) · The Journal of Law and Economics · doi:10.1086/467284 · Cited by 2,998 · cited 1×
“Trust has been referred to as having many different meanings (Williamson 1993), definitions being a confusing potpourri (Shapiro 1987) and leading to conceptual confusion (Lewis and Weigert 1985).”
Bainbridge, L. (1983) · Automatica · doi:10.1016/0005-1098(83)90046-8 · Cited by 2,205 · cited 1×
“Here the reflections of Bainbridge (1983) are highly apposite: We know from many ‘vigilance’ studies … that it is impossible for even a highly motivated human being to maintain effective visual attention towards a source of information on which very little happens, for more than about half an hour.”
The social control of impersonal trust
Shapiro, S. P. (1987) · American Journal of Sociology · doi:10.1086/228791 · Cited by 1,725 · cited 1×
“Trust has been referred to as having many different meanings (Williamson 1993), definitions being a confusing potpourri (Shapiro 1987) and leading to conceptual confusion (Lewis and Weigert 1985).”
A Critical View of Driver Behavior Models: What Do We Know, What Should We Do?
Michon, J. A. (1985) · Human Behavior and Traffic Safety · doi:10.1007/978-1-4613-2173-6_19 · Cited by 1,230 · cited 1×
“If properly designed, the passenger then requires only journey-related (strategic) information as defined by Michon (1985) together with information about costs, ride sharing or emergency situations.”
Ecological interface design: theoretical foundations
Vicente, K. J., & Rasmussen, J. (1992) · IEEE Transactions on Systems, Man and Cybernetics · doi:10.1109/21.156574 · Cited by 1,084
MUIR, B. M., & MORAY, N. (1996) · Ergonomics · doi:10.1080/00140139608964474 · Cited by 993 · cited 1×
“A third explanation is that in questionnaires there is no direct risk perceived in case of automation failure (Lee 1991; Muir and Moray 1996).”
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 1×
“Trust, once lost, is hard to regain (Muir 1994).”
de Winter, J. C. F., Happee, R., Martens, M. H., & Stanton, N. A. (2014) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2014.06.016 · Cited by 739 · cited 2×
“In case of system failure just before a curve, participants were not able to keep the vehicle on the road (Flemisch et al. 2008) (for an overview of more of these examples, see de Winter et al. 2014).”
Self-citationNew technology and human error
Kirwan, B. (1988) · Applied Ergonomics · doi:10.1016/0003-6870(88)90052-x · Cited by 645
Ten challenges for making automation a ‘team player’ in joint human-agent activity
Klein, G., Woods, D. D., Bradshaw, J. M., Hoffman, R. R., & Feltovich, P. J. (2004) · IEEE Intelligent Systems · doi:10.1109/mis.2004.74 · Cited by 590 · cited 2×
“As indicated above, a similar process was adopted by Klein et al. (2004) for automation in general.”
Behind Human Error: Cognitive Systems, Computers and Hindsight
Woods, D. D., Johannesen, L. J., Cook, R. I., & Sarter, N. B. (1994) · Defense Technical Information Center (DTIC) · Cited by 393
Coping with human errors through system design: implications for ecological interface design
Rasmussen, J., & Vicente, K. J. (1989) · International Journal of Man-Machine Studies · doi:10.1016/0020-7373(89)90014-x · Cited by 388 · cited 1×
“In designing an HMI, it is pertinent to apply ecological interface design (EID) principles (Rasmussen and Vicente 1989; Vicente and Rasmussen 1992).”
Control Task Substitution in Semiautomated Driving: Does It Matter What Aspects Are Automated?
Carsten, O., Lai, F. C. H., Barnard, Y., Jamson, A. H., & Merat, N. (2012) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812460246 · Cited by 335 · cited 1×
“That effect was observed by Carsten et al. (2012) in a driving simulator experiment where participants were given the freedom to engage in a range of secondary when they so choose.”
Self-citationDiels, C., & Bos, J. E. (2016) · Applied Ergonomics · doi:10.1016/j.apergo.2015.09.009 · Cited by 331 · cited 4×
“These recommendations are all described in Diels et al. (2016).”
InfluentialO'Hanlon, J. F., & McCauley, M. E. (1973) · doi:10.21236/ad0768215 · Cited by 318 · cited 1×
“Although there is some variation between people and circumstances, the onset time of signals for motion sickness for passengers in normal vehicles is normally between 10 and 20 min (O’Hanlon and McCauley 1974), indicating it is something that needs some time to build up.”
Coactive Design: Designing Support for Interdependence in Joint Activity
Johnson, M., Bradshaw, J. M., Feltovich, P. J., Jonker, C. M., Van Riemsdijk, M. B., & Sierhuis, M. (2014) · Journal of Human-Robot Interaction · doi:10.5898/jhri.3.1.johnson · Cited by 310 · cited 1×
“As Johnson et al. (2014) have described, the best interface will fail for the user if system operation is not observable, predictable and directable at an appropriate level of granularity and timeliness.”
Crampton, G. H. (1990) · Medical Entomology and Zoology · Cited by 273
Sarter, N. B., & Woods, D. D. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778667997 · Cited by 208 · cited 2×
“However, in civil aviation, there are three major airframe manufacturers worldwide, and there is a common design philosophy within each manufacturer such as Boeing and Airbus (Sarter and Woods 1997).”
Summala, H. (2007) · Modelling Driver Behaviour in Automotive Environments · doi:10.1007/978-1-84628-618-6_11 · Cited by 186
(2008) · Driver Distraction · doi:10.1201/9781420007497-10 · Cited by 175
Motion sickness in public road transport: passenger behaviour and susceptibility
TURNER, M. (1999) · Ergonomics · doi:10.1080/001401399185586 · Cited by 175 · cited 1×
“Motion sickness is primarily related to horizontal accelerations caused by accelerating, braking and cornering (Guignard and McCauley 1990; Turner and Griffin 1999a, b).”
Liang, Y., Lee, J. D., & Yekhshatyan, L. (2012) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812446965 · Cited by 165 · cited 1×
“While several studies have shown that long glances away from the front view have a direct effect on accident risk (e.g. Klauer et al. 2010; Liang et al. 2012), this will be slightly different with automated functions.”
Cognitive Control and Human Error Mechanisms
Rasmussen, J. (1987) · Cited by 130
(2010) · PsycEXTRA Dataset · doi:10.1037/e729232011-001 · Cited by 127 · cited 1×
“While several studies have shown that long glances away from the front view have a direct effect on accident risk (e.g. Klauer et al. 2010; Liang et al. 2012), this will be slightly different with automated functions.”
Situation awareness: operationally necessary and scientifically grounded
Endsley, M. R. (2015) · Cognition Technology & Work · doi:10.1007/s10111-015-0323-5 · Cited by 119 · cited 1×
“Here, it noteworthy that Endsley (2015) specifically advocates the improvement of user interfaces rather than the application of decision tools or remedial skills training as the appropriate countermeas-ure to problems in operator situation awareness.”
Turner, M., & Griffin, M. J. (1999) · British Journal of Psychology · doi:10.1348/000712699161594 · Cited by 101 · cited 1×
“Motion sickness is primarily related to horizontal accelerations caused by accelerating, braking and cornering (Guignard and McCauley 1990; Turner and Griffin 1999a, b).”
Trusting a Virtual Driver That Looks, Acts, and Thinks Like You
Verberne, F. M. F., Ham, J., & Midden, C. J. H. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815580749 · Cited by 101 · cited 1×
“This also links with the definition used by Verberne et al. (2015), describing trust in automated systems as “…an antecedent to reliant behaviour, a willingness to accept vulnerability in expectation of a positive outcome”.”
Individual differences in stress and fatigue in two field studies of driving
Desmond, P. A., & Matthews, G. (2009) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2008.12.006 · Cited by 83 · cited 1×
“There is also extensive literature on stress from the driving situation, e.g. Gstalter (1985) and Desmond and Matthews (2009).”
Cooperative Control and Active Interfaces for Vehicle Assitsance and Automation
Flemisch, F. O., Kelsch, J., Löper, C., Schieben, A., Schindler, J., & Heesen, M. (2008) · elib (German Aerospace Center) · Cited by 82
Drivers' decision-making at signalised intersections: an optimisation of the yellow timing
Horst, R. V. D., & Wilmink, A. (1986) · Traffic engineering & control · Cited by 72
Interaction Principles for Cooperative Human-Machine Systems
Bengler, K. J., Zimmermann, M., Bortot, D., Kienle, M., & Damböck, D. (2012) · it - Information Technology · doi:10.1524/itit.2012.0680 · Cited by 72 · cited 1×
“Bengler et al. (2012) proposed that multi-modal interfaces including haptic should be used in human–machine cooperation, for example with driver assistance systems, but did not provide any further specifics.”
Behavioral Measurement of Trust in Automation: The Trust Fall
Miller, D., Johns, M., Mok, B., Gowda, N., Sirkin, D., Lee, K., & Ju, W. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601422 · Cited by 71 · cited 2×
“This difference between opinions as measured via questionnaires and signs of trust when experiencing automated driving has also been described as the Trust Fall (Miller et al. 2016).”
Motion Sickness in Automated Vehicles: The Elephant in the Room
Diels, C., Bos, J. E., Hottelart, K., & Reilhac, P. (2016) · Lecture notes in mobility · doi:10.1007/978-3-319-40503-2_10 · Cited by 70 · cited 2×
“These recommendations are all described in Diels et al. (2016).”
'Running the red' at signals on high-speed roads
Baguley, C. J. (1988) · Traffic engineering & control · Cited by 53
Design of Human Computer Interfaces for Highly Automated Vehicles in the EU-Project HAVEit
Flemisch, F. O., Schieben, A., Schömig, N., Strauss, M., Lueke, S., & Heyden, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21666-4_30 · Cited by 41 · cited 2×
“…project carried out a driving simulator study to evaluate a number of proposed designs to handle driver-initiated transitions of automated driving (Schieben et al. 2011) and also initiated an HMI design process to arrive at prototype designs for the project’s demonstrator automated vehicles…”
Behavioral Aspects of Automatic Vehicle Guidance: Relationship Between Headway and Driver Comfort
Vos, A. P. D., Theeuwes, J., Hoekstra, W., & Coemet, M. (1997) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/1573-03 · Cited by 27 · cited 1×
“De Vos et al. (1997) already found that comfort levels decreased with decreasing headway in automated vehicle platooning.”
de Gelder, E., Cara, I., Uittenbogaard, J., Kroon, L., van Iersel, S., & Hogema, J. H. (2016) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2016.7535544 · Cited by 27 · cited 1×
“De Gelder et al. (2016) showed a clear diversity of preference between drivers in real-life driving situations with ACC.”
Measurement of Driver's Reaction Time to Failure of Steering Controller during Automatic Driving
Omae, M., Hashimoto, N., Sugamoto, T., & Shimizu, H. (2005) · Review of automotive engineering · Cited by 24
The Dynamics of Trust in a Supervisory Control Simulation
Lee, J. D. (1991) · Proceedings of the Human Factors Society Annual Meeting · doi:10.1177/154193129103501712 · Cited by 15 · cited 1×
“A third explanation is that in questionnaires there is no direct risk perceived in case of automation failure (Lee 1991; Muir and Moray 1996).”
Human performance at sea assessed by dynamic visual acuity
Bos, J. E., Hogervorst, Munnoch, K., & Perrault, D. (2008) · TNO Repository · Cited by 10
SIMULATION AND DYNAMICS OF SIGNAL CONTROLLED JUNCTIONS
Niittymäki, J., & Pursula, M. (1994) · Cited by 1
ANALYSIS OF RIDE COMFORT CONSIDERING DRIVER ASSESSMENT.
Kudritzki, D. K. (1999) · Cited by 1
Research works printed but not linked: 10
- 4. Blanco M, Atwood J, Vasquez HM, Trimble TE, Fitchett VL, Radlbeck J, Fitch GM, Russell SM, Green CA, Cullinane B, Morgan JF (2015) Human factors evaluation of level 2 and level 3 automated driving concepts. Report No DOT HS 812 182. National Highway Traffic Safety Administration, Washington DC Bos JE, Hogervorst MA, Munnoch K, Perrault D (2008) Human perfor- mance at sea assessed by dynamic visual acuity. In: Proceedings of ABCS symposium human performance in the maritime envi- ronment. Pacific 2008 International Maritime Conference, Sydney
- 13. EASA (2017) Type rating & license endorsement list-all aircraft, excluding helicopters. European Aviation Safety Agency, Cologne
- 15. Fitch G (2015) The HMI for the automated vehicle: enabling safe inter- action with automated vehicles. TU Automotive, London
- 18. Gstalter H (1985) Workload and stress in driving. In: Ergonomics Inter- national 85: proceedings of the ninth congress of the international ergonomics association. Taylor & Francis, London, pp 994-996
- 20. Hurts K, de Boer RJ (2015) Automation surprise looked at from a demands-resources perspective. In: Proceedings of the human fac- tors and ergonomics society Europe chapter annual conference, pp 221-233
- 27. Lee JD, Young KL, Regan MA (2008) Defining driver distraction. In: Regan MA, Lee JD, Young K (eds) Driver distraction: theory, effects and mitigation. CRC Press, Boca Raton, pp 32-40
- 30. Malta L, Aust ML, Faber F, Metz B, Saint Pierre G, Benmimoun M, Schäfer R (2012) Final results: impacts on traffic safety. EuroFOT Deliverable D6.4. Ford Research Centre GmbH, Aachen
- 46. Sarter NB, Woods DD, Billings CE (1997) Automation surprise. In: Salvendy G (ed) Handbook of human factors & ergonomics, sec- ond edition, 1926-1943. Wiley, New York
- 47. Schieben A, Flemisch F, Temme G, Köster F (2011) What happens when I push the accelerator pedal? Exploration and proposal for the design of driver-initiated transitions between different automa- tion modes in a highly automated vehicle. In: Der Fahrer im 21. Jahrhundert. VDI Berichte, vol 2134, pp 1-21
- 56. White A (2009) From comfort zone to performance management: understanding development and performance. White & MacLean Publishing, Belgium
Cited by
A Survey of Autonomous Driving: Common Practices and Emerging Technologies
Yurtsever, E., Lambert, J., Carballo, A., & Takeda, K. (2020) · IEEE Access · doi:10.1109/access.2020.2983149 · Cited by 1,889
“The user and the ADS need to have a mutual understanding; otherwise, they will not be able to grasp the intentions of each other [266].”
BackgroundThe “Out-of-the-Loop” concept in automated driving: proposed definition, measures and implications
Merat, N., Seppelt, B., Louw, T., Engström, J., Lee, J. D., Johansson, E., Green, C. A., Katazaki, S., Monk, C., Itoh, M., et al. (2019) · Cognition, Technology & Work · doi:10.1007/s10111-018-0525-8 · Cited by 282
An Introduction to Ethics in Robotics and AI
Bartneck, C., Lütge, C., Wagner, A., & Welsh, S. (2021) · SpringerBriefs in Ethics · doi:10.1007/978-3-030-51110-4 · Cited by 223
Xing, Y., Lv, C., Cao, D., & Hang, P. (2021) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2021.103199 · Cited by 209
From HMI to HMIs: Towards an HMI Framework for Automated Driving
Bengler, K., Rettenmaier, M., Fritz, N., & Feierle, A. (2020) · Information · doi:10.3390/info11020061 · Cited by 193
“2020 , 11 , 61 7 of 17 Carsten et al. [40] has provided recommendations that should be considered in the design of aHMIs. according to SAE and the role of the passenger can be met.”
Background
Metadata
- Authors
- Carsten, O., & Martens, M. H.
- Affiliations
- University of Leeds (GB)
- Netherlands Organisation for Applied Scientific Research (NL)
- Corresponding author
- Carsten, Oliver
- Venue
- Cognition, Technology & Work
- Published
- 2018-05-12
- Volume
- 21
- Issue
- 1
- Pages
- 3-20
- ISSN
- 1435-5558 (ISSN-L)
- 1435-5566
- OpenAlex
- W2799403119
- Cited by
- 284
- References
- 46
- Citations by year
- 2026: 43 · 2025: 36 · 2024: 38 · 2023: 37 · 2022: 44 · 2021: 47 · 2020: 30 · 2019: 8 · 2018: 1
- FWCI
- 29.11
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- Locations
- Cognition Technology & Work (publishedVersion, cc-by)
- University of Twente Research Information (publishedVersion, other-oa)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- TU/e Research Portal (submittedVersion)
- TNO Repository (submittedVersion)
- Repository hosted by TU Delft Library (TU Delft) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- University of Twente Research Information (submittedVersion, other-oa)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- human-machine interaction, autonomous driving, driver-vehicle interaction, driver awareness, in-vehicle displays, human-automation collaboration, driver confusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.