Toward Measurement of Situation Awareness in Autonomous Vehicles
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Until vehicles are fully autonomous, safety, legal and ethical obligations require that drivers remain aware of the driving situation. Key decisions about whether a driver can take over when the vehicle is confused, or its capabilities are degraded, depend on understanding whether he or she is responsive and aware of external conditions. The leading techniques for measuring situation awareness in simulated environments are ill-suited to autonomous driving scenarios, and particularly to on-road testing. We have developed a technique, named Daze, to measure situation awareness through real-time, in-situ event alerts. The technique is ecologically valid: it resembles applications people use in actual driving. It is also flexible: it can be used in both simulator and on-road research settings. We performed simulator-based and on-road test deployments to (a) check that Daze could characterize drivers' awareness of their immediate environment and (b) understand practical aspects of the technique's use. Our contributions include the Daze technique, examples of collected data, and ways to analyze such data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Results show that 58% of respondents could correctly differentiate between different events in case of visible alert, 83% in case of audible alert, and 95% in the case of visible and audible alerts. Additionally, respondents need an average response time of 5–9 s in order to take control back depending on the alert method [193].”
From Public acceptance and perception of autonomous vehicles: a comprehensive review (2021) · cites this paper as [193]
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
Toward a Theory of Situation Awareness in Dynamic Systems
Endsley, M. R. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049543 · Cited by 8,875
Cognitive Load Measurement as a Means to Advance Cognitive Load Theory
Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. M. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_8 · Cited by 2,527
Measurement of Situation Awareness in Dynamic Systems
Endsley, M. R. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049499 · Cited by 2,039 · cited 4×
“These methods include eye-tracking, the simulation freeze technique [9], subjective ratings, question probes (during and post activity), and task performance measures.”
MethodInfluentialLee, K. M. (2004) · Communication Theory · doi:10.1111/j.1468-2885.2004.tb00302.x · Cited by 1,631
ENDSLEY, M. R., & KABER, D. B. (1999) · Ergonomics · doi:10.1080/001401399185595 · Cited by 1,088
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2008) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534308x284417 · Cited by 695
Situational Awareness Rating Technique (Sart): The Development of a Tool for Aircrew Systems Design
Taylor, R. M. (2017) · Situational Awareness · doi:10.4324/9781315087924-8 · Cited by 525
Models of driving behavior: A review of their evolution
Ranney, T. A. (1994) · Accident Analysis & Prevention · doi:10.1016/0001-4575(94)90051-5 · Cited by 512
The Invisible Gorilla Strikes Again: Sustained Inattentional Blindness in Expert Observers
Drew, T., Võ, M. L.-H., & Wolfe, J. M. (2013) · Psychological Science · doi:10.1177/0956797613479386 · Cited by 483
Situation awareness measurement: A review of applicability for C4i environments
Salmon, P., Stanton, N., Walker, G., & Green, D. (2006) · Applied Ergonomics · doi:10.1016/j.apergo.2005.02.001 · Cited by 386
Behavioural impacts of Advanced Driver Assistance Systems–an overview
Brookhuis, K. A., De Waard, D., & Janssen, W. H. (2001) · European Journal of Transport and Infrastructure Research · doi:10.18757/ejtir.2001.1.3.3667 · Cited by 245 · cited 3×
“Driving research has shown that rapid transitions from a low alertness state to active vehicle control can be quite dangerous [5, 19], as a lack of alertness and a lack of situation awareness can, when combined, lead to poor performance in a safety critical situation.”
BackgroundAttention and automation: New perspectives on mental underload and performance
Young, M. S., & Stanton, N. A. (2002) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220210123789 · Cited by 194
Borghini, G., Vecchiato, G., Toppi, J., Astolfi, L., Maglione, A., Isabella, R., Caltagirone, C., Kong, W., Wei, D., Zhou, Z., et al. (2012) · 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2012.6347469 · Cited by 180
Jahn, G., Oehme, A., Krems, J. F., & Gelau, C. (2005) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2005.04.009 · Cited by 180
Goal-specific influences on the representation of spatial perspective
Taylor, H. A., Naylor, S. J., & Chechile, N. A. (1999) · Memory & Cognition · doi:10.3758/bf03211414 · Cited by 158
Automation: how much is too much?
Hancock, P. A. (2014) · Ergonomics · doi:10.1080/00140139.2013.816375 · Cited by 129
Human factors and driving performance
(1983) · Applied Ergonomics · doi:10.1016/0003-6870(83)90097-2 · Cited by 117
Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines
Solovey, E. T., Girouard, A., Chauncey, K., Hirshfield, L. M., Sassaroli, A., Zheng, F., Fantini, S., & Jacob, R. J. K. (2009) · Proceedings of the 22nd annual ACM symposium on User interface software and technology · doi:10.1145/1622176.1622207 · Cited by 115 · cited 1×
“These measures can be difficult for researchers to implement and interpret [33], especially in ecological settings such as in on-road studies, as they are highly susceptible to noise from muscle activity and from interference from automotive systems.”
BackgroundSituation awareness and driving performance in a simulated navigation task
Ma, R., & KABER, D. B. (2007) · Ergonomics · doi:10.1080/00140130701318913 · Cited by 102
Traffic Condition Is More Than Colored Lines on a Map: Characterization of Waze Alerts
Silva, T. H., de Melo, P. O. S. V., Viana, A. C., Almeida, J. M., Salles, J., & Loureiro, A. A. F. (2013) · Lecture notes in computer science · doi:10.1007/978-3-319-03260-3_27 · Cited by 100
Timing of unstructured transitions of control in automated driving
Mok, B. K.-J., Johns, M., Lee, K. J., Ive, H. P., Miller, D., & Ju, W. G. (2015) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2015.7225841 · Cited by 91
Self-citationThe RRADS platform: a real road autonomous driving simulator
Baltodano, S., Sibi, S., Martelaro, N., Gowda, N., & Ju, W. (2015) · Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/2799250.2799288 · Cited by 82
Self-citationThe Department of Transportation: Concept and Structure
Barsness, R. W. (1970) · The Western Political Quarterly · doi:10.1177/106591297002300305 · Cited by 79
SASSAROLI, A., ZHENG, F., HIRSHFIELD, L. M., GIROUARD, A., SOLOVEY, E. T., JACOB, R. J. K., & FANTINI, S. (2008) · Journal of Innovative Optical Health Sciences · doi:10.1142/s1793545808000224 · Cited by 65
Measuring the Effects of Driver Distraction: Direct Driving Performance Methods and Measures
Regan, M., Lee, J. D., & Young, K. L. (2008) · CRC Press eBooks · doi:10.1201/9781420007497-15 · Cited by 48
Understanding Driver-Automated Vehicle Interactions Through Wizard of Oz Design Improvisation
Mok, B. K.-J., Sirkin, D., Sibi, S., Miller, D. B., & Ju, W. (2015) · Driving Assessment Conference 2015 · doi:10.17077/drivingassessment.1598 · Cited by 44
Self-citationSPAM: The Real-Time Assessment of SA
Banbury, S. P. (2017) · doi:10.4324/9781315263977-16 · Cited by 39
Bacon, L. P., & Strybel, T. Z. (2012) · Safety Science · doi:10.1016/j.ssci.2012.06.019 · Cited by 37
Partially intelligent automobiles and driving experience at the moment of system transition
Lee, K. J., Joo, Y. K., & Nass, C. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557370 · Cited by 30
The Relationship between SPAM, Workload, and Task Performance on a Simulated ATC Task
Pierce, R. S., Vu, K. L., Nguyen, J. H., & Strybel, T. Z. (2008) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120805200109 · Cited by 16
Measuring the Effects of Driver Distraction: Direct Driving Performance Methods and Measures
Young, K., Regan, M., & Lee, J. (2008) · Driver Distraction · doi:10.1201/9781420007497.ch7 · Cited by 16
Martelaro, N., Sirkin, D. M., & Ju, W. G. (2015) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/2809730.2809753 · Cited by 12
Self-citationUnderstanding intuition and how the mind works
Jennings, C., & Hogan, M. (2012) · The Journal of Positive Psychology · doi:10.1080/17439760.2012.753466 · Cited by 1
The Relationship Between SPAM, Workload, and Task Performance on a Simulated ATC Task
Pierce, R. S., Vu, K. L., Nguyen, J. H., & Strybel, T. Z. (2008) · PsycEXTRA Dataset · doi:10.1037/e578082012-009 · Cited by 1
(2000) · TNO Repository
Research works printed but not linked: 5
- Cheryl A. Bolstad , Haydee M. Cuevas , Jingjing Wang-Costello , Mica R. Endsley , and Linda S . Angell . 2009 . Measurement of situation awareness for automobile technologies of the future. In Performance Metrics for Assessing Driver Distraction. SAE , 195--213. Cheryl A. Bolstad, Haydee M. Cuevas, Jingjing Wang-Costello, Mica R. Endsley, and Linda S. Angell. 2009. Measurement of situation awareness for automobile technologies of the future. In Performance Metrics for Assessing Driver Distraction. SAE, 195--213.
- National Highway Traffic Safety Administration and U.S. Department of Transportation . 2016 . Federal Automated Vehicles Policy: Accelerating the Next Revolution in Roadway Safety. Washington, DC. National Highway Traffic Safety Administration and U.S. Department of Transportation. 2016. Federal Automated Vehicles Policy: Accelerating the Next Revolution in Roadway Safety. Washington, DC.
- National Highway Traffic Safety Administration and others. 2013. Preliminary Statement of Policy Concerning Automated Vehicles . Washington, DC . National Highway Traffic Safety Administration and others. 2013. Preliminary Statement of Policy Concerning Automated Vehicles. Washington, DC.
- Thomas B. Sheridan . 2006. Supervisory control. Handbook of Human Factors and Ergonomics ( 2006 ), 1025--1052. http://onlinelibrary.wiley.com/doi/10. 1002/0470048204.ch38/summary Thomas B. Sheridan. 2006. Supervisory control. Handbook of Human Factors and Ergonomics (2006), 1025--1052. http://onlinelibrary.wiley.com/doi/10. 1002/0470048204.ch38/summary
- M. W. Smolensky . 1993 . Toward the physiological measurement of situation awareness: The case for eye movement measurements . In Proc. of the Human Factors and Ergonomics Society , Vol. 41 . M. W. Smolensky. 1993. Toward the physiological measurement of situation awareness: The case for eye movement measurements. In Proc. of the Human Factors and Ergonomics Society, Vol. 41.
Cited by
Explanations in Autonomous Driving: A Survey
Omeiza, D., Webb, H., Jirotka, M., & Kunze, L. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3122865 · Cited by 355
Public acceptance and perception of autonomous vehicles: a comprehensive review
Othman, K. (2021) · AI and Ethics · doi:10.1007/s43681-021-00041-8 · Cited by 347
“Additionally, respondents need an average response time of 5–9 s in order to take control back depending on the alert method [193].”
BackgroundXing, Y., Lv, C., Cao, D., & Hang, P. (2021) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2021.103199 · Cited by 209
Zhou, F., Yang, X. J., & de Winter, J. C. F. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3069776 · Cited by 204
Assessing Distraction Potential of Augmented Reality Head-Up Displays for Vehicle Drivers
Kim, H., & Gabbard, J. L. (2019) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720819844845 · Cited by 142
Metadata
- Authors
- Sirkin, D., Martelaro, N., Johns, M., & Ju, W.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2017-05-02
- Pages
- 405-415
- OpenAlex
- W2611536443
- Cited by
- 61
- References
- 35
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 6 · 2023: 8 · 2022: 11 · 2021: 8 · 2020: 8 · 2019: 6 · 2018: 4 · 2017: 1
- FWCI
- 8.41
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Center for Design Research Affiliates Program
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Traffic and Road Safety
- Keywords
- situational awareness, autonomous vehicles, driver monitoring, driver takeover, road testing, driving simulator, ecological validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.