The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
The “Out-of-the-Loop” concept in automated driving: proposed definition, measures and implications
Open accessAbstract
Despite an abundant use of the term “Out of the loop” (OOTL) in the context of automated driving and human factors research, there is currently a lack of consensus on its precise definition, how it can be measured, and the practical implications of being in or out of the loop during automated driving. The main objective of this paper is to consider the above issues, with the goal of achieving a shared understanding of the OOTL concept between academics and practitioners. To this end, the paper reviews existing definitions of OOTL and outlines a set of concepts, which, based on the human factors and driver behaviour literature, could serve as the basis for a commonly-agreed definition. Following a series of working group meetings between representatives from academia, research institutions and industrial partners across Europe, North America, and Japan, we suggest a precise definition of being in, out, and on the loop in the driving context. These definitions are linked directly to whether or not the driver is in physical control of the vehicle, and also the degree of situation monitoring required and afforded by the driver. A consideration of how this definition can be operationalized and measured in empirical studies is then provided, and the paper concludes with a short overview of the implications of this definition for the development of automated driving functions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Toward a Theory of Situation Awareness in Dynamic Systems
Endsley, M. R. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049543 · Cited by 8,875
Egeth, H., & Kahneman, D. (1975) · The American Journal of Psychology · doi:10.2307/1421603 · Cited by 6,972
Controlled and automatic human information processing: I. Detection, search, and attention.
Schneider, W., & Shiffrin, R. M. (1977) · Psychological Review · doi:10.1037/0033-295x.84.1.1 · Cited by 5,902
Engineering Psychology and Human Performance
Wickens, C. D., Hollands, J. G., Banbury, S., & Parasuraman, R. (2015) · Psychology Press eBooks · doi:10.4324/9781315665177 · Cited by 4,982
(2016) · Interpersonal Communication · doi:10.4324/9781315506173-8 · Cited by 4,693
Broadbent, D. E. (1958) · doi:10.1037/10037-000 · Cited by 4,337
Multiple resources and performance prediction
Wickens, C. D. (2002) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220210123806 · Cited by 2,621 · cited 1×
“However, the resource concept here is used in a broader sense to denote any sensory, actuator, perceptual, motor or cognitive mechanisms that are used in performing activities (Wickens 2002).”
BackgroundComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
In the Age of the Smart Machine
Zuboff, S. (1988) · Cited by 1,516
Proceedings of the Human Factors and Ergonomics Society 2019 Annual Meeting
(2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631171 · Cited by 1,432
The Out-of-the-Loop Performance Problem and Level of Control in Automation
Endsley, M. R., & Kiris, E. O. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779064555 · Cited by 1,406
A Critical View of Driver Behavior Models: What Do We Know, What Should We Do?
Michon, J. A. (1985) · Human Behavior and Traffic Safety · doi:10.1007/978-1-4613-2173-6_19 · Cited by 1,230
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932
Self-citationHow in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control
Sarter, N. B., & Woods, D. D. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049516 · Cited by 849
Joint Cognitive Systems: Foundations of Cognitive Systems Engineering
Hollnagel, E., & Woods, D. D. (2005) · Thammasat University Digital Collections · Cited by 787
“Take over!” How long does it take to get the driver back into the loop?
Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571433 · Cited by 754
Takeover Time in Highly Automated Vehicles: Noncritical Transitions to and From Manual Control
Eriksson, A., & Stanton, N. A. (2017) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816685832 · Cited by 690
Transition to manual: Driver behaviour when resuming control from a highly automated vehicle
Merat, N., Jamson, A. H., Lai, F. C. H., Daly, M., & Carsten, O. M. J. (2014) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2014.09.005 · Cited by 642
Self-citationHarbluk, J. L., Noy, Y. I., Trbovich, P. L., & Eizenman, M. (2007) · Accident Analysis & Prevention · doi:10.1016/j.aap.2006.08.013 · Cited by 526 · cited 1×
“1999; Samuel and Fisher 2015), to perform tactical maneuvers such as lane changes, turns at intersections (Harbluk et al. 2007), and to identify relevant signs and landmarks for route orientation and navigational tasks.”
BackgroundModels of driving behavior: A review of their evolution
Ranney, T. A. (1994) · Accident Analysis & Prevention · doi:10.1016/0001-4575(94)90051-5 · Cited by 512
The ‘problem ’ with automation: inappropriate feedback and interaction, not ‘over-automation’
Norman, D. A. (1990) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1990.0101 · Cited by 510
Zeeb, K., Buchner, A., & Schrauf, M. (2015) · Accident Analysis & Prevention · doi:10.1016/j.aap.2015.02.023 · Cited by 486
Radlmayr, J., Gold, C., Lorenz, L., Farid, M., & Bengler, K. (2014) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931214581434 · Cited by 448
Driver Distraction: Theory, Effects, and Mitigation
(2008) · CRC Press eBooks · doi:10.1201/9781420007497 · Cited by 439
Human Detection and Diagnosis of System Failures
(1981) · Springer eBooks · doi:10.1007/978-1-4615-9230-3 · Cited by 439
A Two-Point Visual Control Model of Steering
Salvucci, D. D., & Gray, R. (2004) · Perception · doi:10.1068/p5343 · Cited by 427
Which parts of the road guide steering?
Land, M., & Horwood, J. (1995) · Nature · doi:10.1038/377339a0 · Cited by 411
Engineering Psychology and Human Performance
Van Vuuren, L. J. (1987) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v13i1.457 · Cited by 386
Hergeth, S., Lorenz, L., Vilimek, R., & Krems, J. F. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815625744 · Cited by 355
Control Task Substitution in Semiautomated Driving: Does It Matter What Aspects Are Automated?
Carsten, O., Lai, F. C. H., Barnard, Y., Jamson, A. H., & Merat, N. (2012) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812460246 · Cited by 335
Self-citationSituation awareness and workload in driving while using adaptive cruise control and a cell phone
Ma, R., & Kaber, D. B. (2005) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2005.04.002 · Cited by 292
How can humans understand their automated cars? HMI principles, problems and solutions
Carsten, O., & Martens, M. H. (2019) · Cognition, Technology & Work · doi:10.1007/s10111-018-0484-0 · Cited by 284
Modeling take-over performance in level 3 conditionally automated vehicles
Gold, C., Happee, R., & Bengler, K. (2018) · Accident Analysis & Prevention · doi:10.1016/j.aap.2017.11.009 · Cited by 260
THE ATTENTIONAL DEMAND OF AUTOMOBILE DRIVING
Senders, J. W., Kristofferson, A. B., Levison, W. H., Dietrich, C. W., & Ward, J. L. (1967) · Highway Research Record · Cited by 256
Kaber, D. B., & Endsley, M. R. (1997) · Process Safety Progress · doi:10.1002/prs.680160304 · Cited by 254
The challenges of partially automated driving
Casner, S. M., Hutchins, E. L., & Norman, D. (2016) · Communications of the ACM · doi:10.1145/2830565 · Cited by 251
Behavioural impacts of Advanced Driver Assistance Systems–an overview
Brookhuis, K. A., De Waard, D., & Janssen, W. H. (2001) · European Journal of Transport and Infrastructure Research · doi:10.18757/ejtir.2001.1.3.3667 · Cited by 245
Louw, T., & Merat, N. (2017) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2017.01.001 · Cited by 216
Self-citationSemi-automated versus highly automated driving in critical situations caused by automation failures
Strand, N., Nilsson, J., Karlsson, I. C. M., & Nilsson, L. (2014) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2014.04.005 · Cited by 194
Take-Over Time in Highly Automated Vehicles
Eriksson, A., & Stanton, N. A. (2018) · Driver Reactions to Automated Vehicles · doi:10.1201/9781351207515-4 · Cited by 189
(2008) · Driver Distraction · doi:10.1201/9781420007497-10 · Cited by 175
Perceptual Factors in Car-Following
Michaels, R. (1963) · Medical Entomology and Zoology · Cited by 175
Automated Driving: Human-Factors Issues and Design Solutions
Saffarian, M., de Winter, J., & Happee, R. (2012) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181312561483 · Cited by 162
Motor learning affects car-to-driver handover in automated vehicles
Russell, H. E. B., Harbott, L. K., Nisky, I., Pan, S., Okamura, A. M., & Gerdes, J. C. (2016) · Science Robotics · doi:10.1126/scirobotics.aah5682 · Cited by 152
Louw, T., Markkula, G. M., Boer, E. R., Madigan, R., Carsten, O., & Merat, N. (2017) · Accident Analysis & Prevention · doi:10.1016/j.aap.2017.08.011 · Cited by 149
Self-citationEngaging with Highly Automated Driving: To be or Not to be in the Loop?
Louw, T., Merat, N., & Jamson, H. (2015) · International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (Driving Assessment Conference) · doi:10.17077/drivingassessment.1570 · Cited by 144
Self-citationZeeb, K., Härtel, M., Buchner, A., & Schrauf, M. (2017) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2017.07.008 · Cited by 144
Driving context and visual-manual phone tasks influence glance behavior in naturalistic driving
Tivesten, E., & Dozza, M. (2014) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2014.08.004 · Cited by 128
Visual Demand of Driving and the Execution of Display-Intensive in-Vehicle Tasks
Tsimhoni, O., & Green, P. (2001) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120104502305 · Cited by 116
Were they in the loop during automated driving? Links between visual attention and crash potential
Louw, T., Madigan, R., Carsten, O., & Merat, N. (2016) · Injury Prevention · doi:10.1136/injuryprev-2016-042155 · Cited by 116
Self-citationFailure Detection in Dynamic Systems
Wickens, C. D., & Kessel, C. (1981) · Springer eBooks · doi:10.1007/978-1-4615-9230-3_11 · Cited by 109
Technology and Values: Essential Readings
Brett‐Crowther, M. (2010) · International Journal of Environmental Studies · doi:10.1080/00207231003750009 · Cited by 96
Potential Solutions to Human Factors Challenges in Road Vehicle Automation
Seppelt, B., & Victor, T. (2016) · Lecture notes in mobility · doi:10.1007/978-3-319-40503-2_11 · Cited by 94
Self-citationThe Effects of Participatory Mode and Task Workload on the Detection of Dynamic System Failures
Wickens, C. D., & Kessel, C. (1979) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1979.4310070 · Cited by 92
Glass half-full: On-road glance metrics differentiate crashes from near-crashes in the 100-Car data
Seppelt, B., Seaman, S. R., Lee, J., Angell, L., Mehler, B. L., & Reimer, B. (2017) · Accident Analysis & Prevention · doi:10.1016/j.aap.2017.07.021 · Cited by 87
Self-citationSebok, A., & Wickens, C. D. (2017) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816665201 · Cited by 75
Presence, workload and performance effects of synthetic environment design factors
Ma, R., & KABER, D. B. (2006) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2005.12.003 · Cited by 75
Meyer, G., & Beiker, S. A. (2016) · Lecture notes in mobility · doi:10.1007/978-3-319-40503-2 · Cited by 62
Driving performance at lateral system limits during partially automated driving
Naujoks, F., Purucker, C., Wiedemann, K., Neukum, A., Wolter, S. C., & Steiger, R. (2017) · Accident Analysis & Prevention · doi:10.1016/j.aap.2017.08.027 · Cited by 59
The out-of-the-loop Brain: A neuroergonomic approach of the human automation interaction
Berberian, B., Somon, B., Sahaï, A., & Gouraud, J. (2017) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2017.09.010 · Cited by 59
Distraction Assessment Methods Based on Visual Behavior and Event Detection
Victor, T., Engstr√∂m, J., & Harbluk, J. L. (2008) · CRC Press eBooks · doi:10.1201/9781420007497.ch10 · Cited by 54
Self-citationNorman, D. A. (2015) · Lecture notes in mobility · doi:10.1007/978-3-319-19078-5_7 · Cited by 43
Cited by
Kettwich, C., Schrank, A., & Oehl, M. (2021) · Multimodal Technologies and Interaction · doi:10.3390/mti5050026 · Cited by 86
Understanding artificial agency
Dung, L. (2025) · The Philosophical Quarterly · doi:10.1093/pq/pqae010 · Cited by 69
Metadata
- Authors
- Merat, N., Seppelt, B., Louw, T., Engström, J., Lee, J. D., Johansson, E., Green, C. A., Katazaki, S., Monk, C., Itoh, M., McGehee, D., Sunda, T., Unoura, K., Victor, T., Schieben, A., & Keinath, A.
- Venue
- Cognition, Technology & Work
- Published
- 2018-09-15
- Volume
- 21
- Issue
- 1
- Pages
- 87-98
- ISSN
- 1435-5558 (ISSN-L)
- 1435-5566
- OpenAlex
- W2892156502
- Cited by
- 282
- References
- 62
- FWCI
- 21.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.