Challenges in autonomous vehicle testing and validation
Abstract
Software testing is all too often simply a bug hunt rather than a well-considered exercise in ensuring quality. A more methodical approach than a simple cycle of system-level test-fail-patch-test will be required to deploy safe autonomous vehicles at scale. The ISO 26262 development V process sets up a framework that ties each type of testing to a corresponding design or requirement document, but presents challenges when adapted to deal with the sorts of novel testing problems that face autonomous vehicles. This paper identifies five major challenge areas in testing according to the V model for autonomous vehicles: driver out of the loop, complex requirements, non-deterministic algorithms, inductive learning algorithms, and fail-operational systems. General solution approaches that seem promising across these different challenge areas include: phased deployment using successively relaxed operational scenarios, use of a monitor/actuator pair architecture to separate the most complex autonomy functions from simpler safety functions, and fault injection as a way to perform more efficient edge case testing. While significant challenges remain in safety-certifying the type of algorithms that provide high-level autonomy themselves, it seems within reach to instead architect the system and its accompanying design process to be able to employ existing software safety approaches.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Design issues on data collection, pre-processing, cleaning, labeling and augmentation, including big data challenges, exist for ML applications.
“-Data Design: Design issues on data collection, pre-processing, cleaning, labeling and augmentation, including big data challenges [17,23,25-27, 31,33,41,44,74-76,105,114-116].”
From Software engineering challenges for machine learning applications: A literature review (2020) · cites this paper as [23]
Data management for ML applications, including tools for data dependencies, automatic data validation and cleaning during runtime, and concept drift adaptation, is a maintenance challenge.
“-Data Management: Tools for data dependencies, automatic data validation and cleaning during runtime, and concept drift adaptation [17,23,60,69].”
From Software engineering challenges for machine learning applications: A literature review (2020) · cites this paper as [23]
Training data, ML models and ML platforms are key components of ML applications, and challenges on the quality of each component have been discussed, including data anomaly, imbalanced or biased data, encrypted data, data evaluation/cleaning, and novelty detection.
“-Component Quality: Training data, ML models and ML platforms (e.g.scikit-learn [125], Tensor flow [126], Weka [127]) is the key components of ML applications.The challenges on the quality of each components have been discussed.The challenge issues on data quality for ML applications includes data anomaly [23], imbalanced or biased data, encrypted data [60], data evaluation/cleaning [39][40][41]74”
From Software engineering challenges for machine learning applications: A literature review (2020) · cites this paper as [23]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · see citing works
The Byzantine generals problem
Lamport, L., Shostak, R., & Pease, M. (1982) · ACM Transactions on Programming Languages and Systems · doi:10.1145/357172.357176 · Cited by 6,023 · see citing works
The black swan: the impact of the highly improbable
(2007) · Choice Reviews Online · doi:10.5860/choice.45-1430 · Cited by 3,456 · see citing works
Pedestrian Detection: An Evaluation of the State of the Art
Dollár, P., Wojek, C., Schiele, B., & Perona, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.155 · Cited by 3,412 · see citing works
An Inquiry concerning human understanding
Hume, D. (1955) · Cited by 1,774 · see citing works
System structure for software fault tolerance
Randell, B. (1975) · ACM SIGPLAN Notices · doi:10.1145/390016.808467 · Cited by 1,523 · see citing works
System structure for software fault tolerance
Randell, B. (1975) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1975.6312842 · Cited by 1,096 · see citing works
The Byzantine generals problem
International, S., Lamport, L., Shostak, R. E., International, S., Pease, M. C., & International, S. (2019) · ACM eBooks · doi:10.1145/3335772.3335936 · Cited by 1,034 · see citing works
The infeasibility of quantifying the reliability of life-critical real-time software
Butler, R. W., & Finelli, G. B. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.210303 · Cited by 399 · see citing works
A Comparative Study of Probabilistic Roadmap Planners
Gerærts, R., & Overmars, M. H. (2004) · Springer tracts in advanced robotics · doi:10.1007/978-3-540-45058-0_4 · Cited by 265 · see citing works
On Fault Representativeness of Software Fault Injection
Natella, R., Cotroneo, D., Duraes, J. A., & Madeira, H. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.124 · Cited by 199 · see citing works
MARTIN, M. C., & Moravec, H. (1996) · Cited by 154 · see citing works
The infeasibility of experimental quantification of life-critical software reliability
Butler, R. W., & Finelli, G. B. (1991) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/123041.123054 · Cited by 135 · see citing works
Dependability Benchmarking for Computer Systems
(2008) · Wiley eBooks · doi:10.1002/9780470370506 · Cited by 104 · see citing works
Design considerations in Boeing 777 fly-by-wire computers
Yeh, Y. C. (2002) · doi:10.1109/hase.1998.731596 · Cited by 103 · see citing works
Inverting Convolutional Networks with Convolutional Networks.Preprint
Dosovitskiy, A., & Brox, T. (2015) · arXiv (Cornell University) · Cited by 75 · see citing works
A Case Study on Runtime Monitoring of an Autonomous Research Vehicle (ARV) System
Kane, A., Chowdhury, O., Datta, A., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23820-3_7 · Cited by 61 · see citing works
A Philosophy for Developing Trust in Self-driving Cars
Wagner, M., & Koopman, P. J. (2015) · Lecture notes in mobility · doi:10.1007/978-3-319-19078-5_14 · Cited by 57 · see citing works
Self-citationActive learning from demonstration for robust autonomous navigation
Silver, D., Bagnell, J. A., & Stentz, A. (2012) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2012.6224757 · Cited by 49 · see citing works
System safety as an emergent property in composite systems
Black, J., & Koopman, P. J. (2009) · IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) · doi:10.1109/dsn.2009.5270316 · Cited by 32 · see citing works
Self-citationShladover, S. E., VanderWerf, J., Miller, M. A., Kourjanskaia, N., & Krishnan, H. (2001) · RePEc: Research Papers in Economics · Cited by 31 · see citing works
Design by extrapolation: an evaluation of fault-tolerant avionics
Hammett, R. (2002) · AIAA/IEEE Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc.2001.963314 · Cited by 31 · see citing works
Design by extrapolation: an evaluation of fault tolerant avionics
Hammett, R. (2002) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/62.995184 · Cited by 26 · see citing works
Functional Evolution of Automated Highway System for Incremental Deployment
Bayouth, M., & Koopman, P. J. (1998) · Transportation Research Record Journal of the Transportation Research Board · doi:10.3141/1651-12 · Cited by 8 · see citing works
Self-citationLearning product set models of fault triggers in high-dimensional software interfaces
Vernaza, P., Guttendorf, D., Wagner, M., et al. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353866 · Cited by 6 · see citing works
Self-citationFault injection and automotive development process
Kanoun, K., Pintard, L., Fabre, J.-C., et al. (2014) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Autonomous Vehicle Safety: An Interdisciplinary Challenge
Koopman, P., & Wagner, M. (2017) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2016.2583491 · Cited by 688 · see citing works
Self-citationA Systematic Review of Perception System and Simulators for Autonomous Vehicles Research
Rosique, F., Navarro, P. J., Fernández, C., & Padilla, A. (2019) · Sensors · doi:10.3390/s19030648 · Cited by 469 · see citing works
Safety and Security in Cyber-Physical Systems and Internet-of-Things Systems
Wolf, M., & Serpanos, D. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2781198 · Cited by 250 · see citing works
On the safety of machine learning: Cyber-physical systems, decision sciences, and data products
Varshney, K. R., & Alemzadeh, H. (2017) · Big Data · doi:10.1089/big.2016.0051 · Cited by 211 · see citing works
“r less confident predictions, stop the autonomous control software and switch to a backup system or a degraded level of autonomy (e.g., full control by the driver) are considered for self-driving cars [48]. B. Decision Sciences In decision sciences applications, people are in the loop in a different way than in cyberph…”
BackgroundSystematic literature review of validation methods for AI systems
Myllyaho, L., Raatikainen, M., Männistö, T., Mikkonen, T., & Nurminen, J. K. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111050 · Cited by 145 · see citing works
Algorithmic Decision-Making in AVs: Understanding Ethical and Technical Concerns for Smart Cities
Lim, H. S. M., & Taeihagh, A. (2019) · Sustainability · doi:10.3390/su11205791 · Cited by 136 · see citing works
Requirements-driven test generation for autonomous vehicles with machine learning components
Tuncali, C. E., Fainekos, G., Prokhorov, D., Ito, H., & Kapinski, J. (2020) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2019.2955903 · Cited by 129 · see citing works
An Analysis of ISO 26262: Machine Learning and Safety in Automotive Software
Salay, R., Queiroz, R., & Czarnecki, K. (2018) · SAE Technical Paper Series · doi:10.4271/2018-01-1075 · Cited by 109 · see citing works
Software engineering challenges for machine learning applications: A literature review
Kumeno, F. (2020) · Intelligent Decision Technologies · doi:10.3233/idt-190160 · Cited by 81 · see citing works
Xu, Z., Wang, X., Wang, X., & Zheng, N. (2025) · Accident Analysis & Prevention · doi:10.1016/j.aap.2025.108011 · Cited by 29 · see citing works
Disruptive Innovations and Disruptive Assurance: Assuring Machine Learning and Autonomy
Bloomfield, R., Khlaaf, H., Ryan Conmy, P., & Fletcher, G. (2019) · Computer · doi:10.1109/mc.2019.2914775 · Cited by 23 · see citing works
An overview of key trustworthiness attributes and KPIs for trusted ML-based systems engineering
Mattioli, J., Sohier, H., Delaborde, A., Amokrane-Ferka, K., Awadid, A., Chihani, Z., Khalfaoui, S., & Pedroza, G. (2024) · AI and Ethics · doi:10.1007/s43681-023-00394-2 · Cited by 22 · see citing works
- Authors
- Koopman, P., & Wagner, M.
- Affiliations
- Carnegie Mellon University (US)
- Edge Case Research (United States) (US)
- ORCID
- Koopman, Philip
- Venue
- SAE International Journal of Transportation Safety
- Publisher
- SAE International
- Published
- 2016-04-05
- Volume
- 04
- Issue
- 1
- Pages
- 15-24
- ISSN
- 2327-5626 (ISSN-L)
- 2327-5634
- OpenAlex
- W2328067583
- Cited by
- 675
- References
- 29
- Citations by year
- 2026: 67 · 2025: 83 · 2024: 68 · 2023: 73 · 2022: 71 · 2021: 84 · 2020: 89 · 2019: 66 · 2018: 47 · 2017: 26 · 2016: 1
- FWCI
- 88.44
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- SAE International Journal of Transportation Safety (publishedVersion)
- Topics
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- autonomous vehicles, software testing, ISO 26262, V-model, safety certification, non-deterministic algorithms, inductive learning algorithms, fail-operational systems, phased deployment, fault injection, software safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
