Meaningful human control of partially automated driving systems: Insights from interviews with Tesla users
Open accessAbstract
Partially automated driving systems are designed to perform specific driving tasks—such as steering, accelerating, and braking—while still requiring human drivers to monitor the environment and intervene when necessary. This shift of driving responsibilities from human drivers to automated systems raises concerns about accountability, particularly in scenarios involving unexpected events. To address these concerns, the concept of meaningful human control (MHC) has been proposed. MHC emphasises the importance of humans retaining oversight and responsibility for decisions made by automated systems. Despite extensive theoretical discussion of MHC in driving automation, there is limited empirical research on how real-world partially automated systems align with MHC principles. This study offers two main contributions: (1) an empirical evaluation of MHC in partially automated driving, based on 103 semi-structured interviews with users of Tesla's Autopilot and Full Self-Driving (FSD) Beta systems; and (2) a methodological framework for assessing MHC through qualitative interview data. We operationalise the previously proposed tracking and tracing conditions of MHC using a set of evaluation criteria to determine whether these systems support meaningful human control in practice. Our findings indicate that several factors influence the degree to which MHC is achieved. Failures in tracking—where drivers' expectations regarding system safety are not adequately met—arise from technological limitations, susceptibility to environmental conditions (e.g., adverse weather or inadequate infrastructure), and discrepancies between technical performance and user satisfaction. Tracing performance—the ability to clearly assign responsibility—is affected by inconsistent adherence to safety protocols, varying levels of driver confidence, and the specific driving mode in use (e.g., Autopilot versus FSD Beta). These findings contribute to ongoing efforts to design partially automated driving systems that more effectively support meaningful human control and promote more appropriate use of automation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · see citing works
Bainbridge, L. (1983) · Automatica · doi:10.1016/0005-1098(83)90046-8 · Cited by 2,205 · see citing works
Investigating the Importance of Trust on Adopting an Autonomous Vehicle
Choi, J. K., & Ji, Y. G. (2015) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2015.1070549 · Cited by 1,200 · see citing works
The responsibility gap: Ascribing responsibility for the actions of learning automata
Matthias, A. (2004) · Ethics and Information Technology · doi:10.1007/s10676-004-3422-1 · Cited by 1,101 · see citing works
Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems
On-Road Automated Driving (ORAD) Committee (2017) · doi:10.4271/j3016_201401 · Cited by 675 · see citing works
What drives people to accept automated vehicles? Findings from a field experiment
Xu, Z.-G., Zhang, K.-F., Min, H.-G., Wang, Z., Zhao, X.-M., & Liu, P. (2018) · Transportation Research Part C: Emerging Technologies · doi:10.1016/j.trc.2018.07.024 · Cited by 595 · see citing works
Meaningful Human Control over Autonomous Systems: A Philosophical Account
Santoni de Sio, F., & van den Hoven, J. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00015 · Cited by 566 · see citing works
Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them
Santoni de Sio, F., & Mecacci, G. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00450-x · Cited by 490 · see citing works
Cicchino, J. B. (2016) · Accident Analysis & Prevention · doi:10.1016/j.aap.2016.11.009 · Cited by 481 · see citing works
Young, M. S., & Stanton, N. A. (2002) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024497709 · Cited by 406 · see citing works
Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations
Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P. (2020) · Organizational Research Methods · doi:10.1177/1094428120971683 · Cited by 400 · see citing works
Autonomous Driving Systems: A Preliminary Naturalistic Study of the Tesla Model S
Endsley, M. R. (2017) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343417695197 · Cited by 286 · see citing works
Is partially automated driving a bad idea? Observations from an on-road study
Banks, V. A., Eriksson, A., O'Donoghue, J., & Stanton, N. A. (2017) · Applied Ergonomics · doi:10.1016/j.apergo.2017.11.010 · Cited by 244 · see citing works
A Review of Best Practice Recommendations for Text Analysis in R (and a User-Friendly App)
Banks, G. C., Woznyj, H. M., Wesslen, R. S., & Ross, R. L. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-017-9528-3 · Cited by 194 · see citing works
Driver Vigilance in Automated Vehicles: Hazard Detection Failures Are a Matter of Time
Greenlee, E. T., DeLucia, P. R., & Newton, D. C. (2018) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720818761711 · Cited by 162 · see citing works
Ethical issues in focus by the autonomous vehicles industry
Martinho, A. M., Herber, N., Kroesen, M., & Chorus, C. G. (2021) · Transport Reviews · doi:10.1080/01441647.2020.1862355 · Cited by 160 · see citing works
Meaningful human control: actionable properties for AI system development
Cavalcante Siebert, L., Lupetti, M. L., Aizenberg, E., Beckers, N., Zgonnikov, A., Veluwenkamp, H., Abbink, D., Giaccardi, E., Houben, G.-J., Jonker, C. M., et al. (2023) · AI and Ethics · doi:10.1007/s43681-022-00167-3 · Cited by 138 · see citing works
Self-citationTheory-Driven Analysis of Large Corpora: Semisupervised Topic Classification of the UN Speeches
Watanabe, K., & Zhou, Y. (2020) · Social Science Computer Review · doi:10.1177/0894439320907027 · Cited by 130 · see citing works
Drivers are blamed more than their automated cars when both make mistakes
Awad, E., Levine, S., Kleiman-Weiner, M., Dsouza, S., Tenenbaum, J., Shariff, A. F., Bonnefon, J.-F., & Rahwan, I. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0762-8 · Cited by 116 · see citing works
Poulter, D. R., Chapman, P., Bibby, P. A., Clarke, D. D., & Crundall, D. E. (2008) · Accident Analysis & Prevention · doi:10.1016/j.aap.2008.09.002 · Cited by 109 · see citing works
2023 IEEE Intelligent Vehicles Symposium (IV)
(2023) · doi:10.1109/iv55152.2023 · Cited by 95 · see citing works
Li, J. J., Zhao, X., Cho, M.-J., Ju, W. G., & Malle, B. F. (2016) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2016-01-0164 · Cited by 94 · see citing works
Self-Driving Vehicles—an Ethical Overview
Hansson, S. O., Belin, M.-Å., & Lundgren, B. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00464-5 · Cited by 91 · see citing works
Driver trust & mode confusion in an on-road study of level-2 automated vehicle technology
Wilson, K., Yang, S., Roady, T., Kuo, J., & Lenné, M. G. (2020) · Safety Science · doi:10.1016/j.ssci.2020.104845 · Cited by 83 · see citing works
Meaningful human control as reason-responsiveness: the case of dual-mode vehicles
Mecacci, G., & Santoni de Sio, F. (2020) · Ethics and Information Technology · doi:10.1007/s10676-019-09519-w · Cited by 82 · see citing works
Nordhoff, S., Stapel, J., He, X.-L., Gentner, A., & Happee, R. (2021) · PLoS ONE · doi:10.1371/journal.pone.0260953 · Cited by 77 · see citing works
Self-citationAnalysis, Design and Evaluation of Man–Machine Systems
(1983) · Elsevier eBooks · doi:10.1016/c2013-0-06057-4 · Cited by 74 · see citing works
Nordhoff, S., Lee, J. D., Calvert, S. C., Berge, S. H., Hagenzieker, M., & Happee, R. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1101520 · Cited by 65 · see citing works
Self-citationNaturalistic observations of driver hand positions
Walton, D. K., & Thomas, J. A. (2005) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2005.04.010 · Cited by 52 · see citing works
Wirthmüller, F., Schlechtriemen, J., Hipp, J., & Reichert, M. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3002070 · Cited by 48 · see citing works
Drivers’ and cyclists’ safety perceptions in overtaking maneuvers
Rasch, A., Moll, S., López, G., García, A., & Dozza, M. (2021) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2021.11.014 · Cited by 47 · see citing works
Cabrall, C. D. D., Eriksson, A., Dreger, F. A., Happee, R., & de Winter, J. (2019) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2018.1528484 · Cited by 45 · see citing works
The many meanings of meaningful human control
Robbins, S. (2023) · AI and Ethics · doi:10.1007/s43681-023-00320-6 · Cited by 45 · see citing works
Koglbauer, I. V., Holzinger, J., Eichberger, A., & Lex, C. (2017) · Traffic Injury Prevention · doi:10.1080/15389588.2017.1407411 · Cited by 43 · see citing works
Gaps in the Control of Automated Vehicles on Roads
Calvert, S. C., van Arem, B., Heikoop, D. D., Hagenzieker, M., Mecacci, G., & de Sio, F. S. (2020) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2019.2926278 · Cited by 32 · see citing works
Self-citationReasons for Meaningful Human Control
Veluwenkamp, H. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09673-8 · Cited by 30 · see citing works
Behavioral Adaptations to Lane Keeping Systems: Effects of Exposure and Withdrawal
Miller, E. E., & Boyle, L. N. (2018) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720818800538 · Cited by 28 · see citing works
Kim, S., He, X., van Egmond, R., & Happee, R. (2024) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2024.02.009 · Cited by 26 · see citing works
Steen, M., van Diggelen, J., Timan, T., & van der Stap, N. (2022) · AI and Ethics · doi:10.1007/s43681-022-00168-2 · Cited by 23 · see citing works
A Practicable Operationalisation of Meaningful Human Control
Kwik, J. (2022) · Laws · doi:10.3390/laws11030043 · Cited by 22 · see citing works
Research Handbook on Meaningful Human Control of Artificial Intelligence Systems
Mecacci, G., Amoroso, D., Cavalcante Siebert, L., et al. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802204131 · Cited by 22 · see citing works
Misuse or abuse of automation? Exploring drivers’ intentions to nap during automated driving
Wörle, J., & Metz, B. (2023) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2023.10.023 · Cited by 21 · see citing works
Drivers of partially automated vehicles are blamed for crashes that they cannot reasonably avoid
Beckers, N., Siebert, L. C., Bruijnes, M., Jonker, C. M., & Abbink, D. A. (2022) · Scientific Reports · doi:10.1038/s41598-022-19876-0 · Cited by 19 · see citing works
A conceptual framework for automation disengagements
Nordhoff, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-57882-6 · Cited by 18 · see citing works
Self-citationCalvert, S. C., & Mecacci, G. (2020) · IEEE Open Journal of Intelligent Transportation Systems · doi:10.1109/ojits.2020.3021461 · Cited by 17 · see citing works
Self-citationOskina, M., Farah, H., Morsink, P., et al. (2022) · Transportation Research Record Journal of the Transportation Research Board · doi:10.1177/03611981221112423 · Cited by 12 · see citing works
Designing automated vehicle and traffic systems towards meaningful human control
Calvert, S. C., Johnsen, S. O., & George, A. (2024) · Edward Elgar Publishing eBooks · doi:10.4337/9781802204131.00018 · Cited by 12 · see citing works
Self-citationNordhoff, S., & Hagenzieker, M. (2024) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2024.09.021 · Cited by 10 · see citing works
Self-citationHow to ensure control of cooperative vehicle and truck platoons using Meaningful Human Control
Calvert, S. C., Mecacci, G., Heikoop, D. D., & Janssen, R. (2021) · European journal of transport and infrastructure research · doi:10.18757/ejtir.2021.21.2.5354 · Cited by 4 · see citing works
Self-citationSuryana, L. E., Nordhoff, S., Calvert, S. C., Zgonnikov, A., & van Arem, B. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588431 · Cited by 4 · see citing works
Self-citationGeorge, A., Siebert, L. C., Abbink, D. A., & Zgonnikov, A. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230349 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Still being checked.
- Authors
- Suryana, L. E., Nordhoff, S., Calvert, S., Zgonnikov, A., & van Arem, B.
- Affiliations
- Delft University of Technology (NL)
- University of California, Davis (US)
- Corresponding author
- Suryana, Lucas Elbert
- Venue
- Transportation Research Part F: Traffic Psychology and Behaviour
- Publisher
- Elsevier BV
- Published
- 2025-08
- Volume
- 113
- Pages
- 213-236
- ISSN
- 1369-8478 (ISSN-L)
- OpenAlex
- W4410218573
- Semantic Scholar
- 77c26fb42d66b53d89c775cf654cae936f648649
- Cited by
- 6
- References
- 51
- Citations by year
- 2026: 3 · 2025: 3
- FWCI
- 6.56
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Transportation Research Part F Traffic Psychology and Behaviour (publishedVersion, cc-by)
- SSRN Electronic Journal (acceptedVersion)
- Funders
- Lembaga Pengelola Dana Pendidikan (0006552/TRA/D/19/lpdp2021)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- meaningful human control, partially automated driving, Tesla Autopilot, driver responsibility, accountability, driver expectations, driver confidence, safety procedures, adverse weather, user satisfaction, driver-automation interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review