Machine Learning Testing: Survey, Landscapes and Horizons
Open accessThe DOI matches its Crossref record; 258 of the 258 references it cites were found in a registry or an index.
Abstract
This paper provides a comprehensive survey of techniques for testing machine learning systems; Machine Learning Testing (ML testing) research. It covers 144 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation). The paper also analyses trends concerning datasets, research trends, and research focus, concluding with research challenges and promising research directions in ML testing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Verification and validation of the behavior of artificial intelligence-powered systems is still an open issue, despite the many recent results [241].”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [241]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–258 of 258
Albarghouthi, A., & Vinitsky, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287588 · Cited by 51
Automated Test Generation to Detect Individual Discrimination in AI ModelsPreprint
Agarwal, A., Lohia, P., Nagar, S., Dey, K., & Saha, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03260 · Cited by 50
Detecting Overfitting via Adversarial ExamplesPreprint
Werpachowski, R., György, A., & Szepesvári, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02380 · Cited by 49
Assessing the Local Interpretability of Machine Learning Models
Slack, D., Friedler, S. A., Scheidegger, C., & Roy, C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.03501 · Cited by 46
Cruz-Benito, J., Vázquez‐Ingelmo, A., Sánchez‐Prieto, J. C., Therón, R., García‐Peñalvo, F. J., & Martín-González, M. (2017) · IEEE Access · doi:10.1109/access.2017.2782678 · Cited by 45
Multiple-Implementation Testing of Supervised Learning Software
Srisakaokul, S., Wu, Z., Astorga, A., Alebiosu, O., & Xie, T. (2016) · National Conference on Artificial Intelligence · Cited by 44
The Promise and Peril of Human Evaluation for Model Interpretability
Herman, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07414 · Cited by 42
Dataset Coverage for Testing Machine Learning Computer Programs
Nakajima, S., & Bui, H. N. (2016) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2016.049 · Cited by 41
Wang, H., Sun, D. L., & Xing, E. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017136 · Cited by 41
A monte carlo method for metamorphic testing of machine translation services
Pesu, D., Zhou, Z. Q., Zhen, J., & Towey, D. (2018) · IEEE/ACM International Workshop on Metamorphic Testing (MET) · doi:10.1145/3193977.3193980 · Cited by 41
Manifesting Bugs in Machine Learning Code: An Explorative Study with Mutation Testing
Cheng, D., Cao, C., Xu, C., & Ma, X. (2018) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs.2018.00044 · Cited by 40
Safety and Trustworthiness of Deep Neural Networks: A SurveyPreprint
Huang, X., Kroening, D., Kwiatkowska, M., Ruan, W., Sun, Y., Thamo, E., Wu, M., & Yi, X. (2018) · arXiv (Cornell University) · Cited by 40
Ariadne: analysis for machine learning programs
Dolby, J., Shinnar, A., Allain, A., & Reinen, J. (2018) · ACM SIGPLAN International Workshop on Machine Learning and Programming Languages (MAPL) · doi:10.1145/3211346.3211349 · Cited by 39
Towards Dependability Metrics for Neural Networks
Cheng, C., Nührenberg, G., Huang, C.-H., Rueß, H., & Yasuoka, H. (2018) · ACM/IEEE International Conference on Formal Methods and Models for Co-Design (MEMOCODE) · doi:10.1109/memcod.2018.8556962 · Cited by 39
Squeeziness: An information theoretic measure for avoiding fault masking
Clark, D., & Hierons, R. M. (2012) · Information Processing Letters · doi:10.1016/j.ipl.2012.01.004 · Cited by 36
Secure Deep Learning Engineering: A Software Quality Assurance PerspectivePreprint
Ma, L., Juefei-Xu, F., Xue, M., Hu, Q., Chen, S., Li, B., Liu, Y., Zhao, J., Yin, J., & See, S. C. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.04538 · Cited by 36
Self-citationStorm: program reduction for testing and debugging probabilistic programming systems
Dutta, S., Zhang, W., Huang, Z., & Misailovíc, S. (2019) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3338906.3338972 · Cited by 35
A Noise-Sensitivity-Analysis-Based Test Prioritization Technique for Deep Neural NetworksPreprint
Zhang, L., Sun, X., Li, Y., & Zhang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.00054 · Cited by 35
Testing Machine Learning Algorithms for Balanced Data Usage
Sharma, A., & Wehrheim, H. (2019) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2019.00022 · Cited by 33
Testing Untestable Neural Machine Translation: An Industrial Case
Zheng, W., Wang, W., Liu, D., Zhang, C., Zeng, Q., Deng, Y., Yang, W., He, P., & Xie, T. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion.2019.00131 · Cited by 33
Coverage-Guided Fuzzing for Deep Neural Networks.Preprint
Xie, X., Ma, L., Juefei-Xu, F., Chen, H., Xue, M., Li, B., Liu, Y., Zhao, J., Yin, J., & See, S. C. W. (2018) · arXiv (Cornell University) · Cited by 33
Self-citationOn the assessment of the added value of new predictive biomarkers
Chen, W., Samuelson, F. W., Gallas, B. D., Kang, L., Sahiner, B., & Petrick, N. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-98 · Cited by 32
Calibration of medical diagnostic classifier scores to the probability of disease
Chen, W., Sahiner, B., Samuelson, F., Pezeshk, A., & Petrick, N. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280216661371 · Cited by 29
Offline Contextual Bandits with High Probability Fairness Guarantees
Metevier, B., Giguere, S., Brockman, S., Kobren, A., Brun, Y., Brunskill, E., & Thomas, P. S. (2019) · neural information processing systems · Cited by 29
Ramanathan, A., Pullum, L., Hussain, F., Chakrabarty, D., & Jha, S. K. (2016) · Design, Automation, and Test in Europe · doi:10.3850/9783981537079_0954 · Cited by 29
Hains, G., Jakobsson, A., & Khmelevsky, Y. (2018) · 2018 Annual IEEE International Systems Conference (SysCon) · doi:10.1109/syscon.2018.8369576 · Cited by 28
[Invited] Quality Assurance of Machine Learning Software
Nakajima, S. (2018) · IEEE Global Conference on Consumer Electronics (GCCE) · doi:10.1109/gcce.2018.8574766 · Cited by 27
Fairness-aware machine learning: a perspectivePreprint
Žliobaitė, I. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.00754 · Cited by 27
Classifier variability: Accounting for training and testing
Chen, W.-J., Gallas, B. D., & Yousef, W. A. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.12.024 · Cited by 26
A Framework for Ensuring the Quality of a Big Data Service
Ding, J., Zhang, D., & Hu, X. (2016) · IEEE International Conference on Services Computing · doi:10.1109/scc.2016.18 · Cited by 22
Repairing Decision-Making Programs Under Uncertainty
Albarghouthi, A., D’Antoni, L., & Drews, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-63387-9_9 · Cited by 22
A Test Architecture for Machine Learning Product
Nishi, Y., Masuda, S., Ogawa, H., & Uetsuki, K. (2018) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2018.00060 · Cited by 21
DeepBase: Deep Inspection of Neural Networks
Sellam, T., Lin, K., Huang, I., Yang, M. Y., Vondrick, C., & Wu, E. (2019) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3299869.3300073 · Cited by 20
Metamorphic Relation Based Adversarial Attacks on Differentiable Neural ComputerPreprint
Chan, A., Ma, L., Juefei-Xu, F., Xie, X., Liu, Y., & Ong, Y.-S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02444 · Cited by 19
Self-citationConcepts in Quality Assessment for Machine Learning - From Test Data to Arguments
Ishikawa, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00847-5_39 · Cited by 19
Testing Neural Program AnalyzersPreprint
Rabin, M. R. I., Wang, K., & Alipour, M. A. (2019) · arXiv (Cornell University) · doi:10.13140/rg.2.2.11576.70405 · Cited by 19
Parameterizing random test data according to equivalence classes
Murphy, C., Kaiser, G. E., & Arias, M. (2007) · doi:10.1145/1292414.1292425 · Cited by 18
Grammar Based Directed Testing of Machine Learning Systems
Udeshi, S., & Chattopadhyay, S. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2953066 · Cited by 18
Quality-efficiency trade-offs in machine learning for text processing
Baeza-Yates, R. A., & Liaghat, Z. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258006 · Cited by 18
Dataset Diversity for Metamorphic Testing of Machine Learning Software
Nakajima, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-13651-2_2 · Cited by 17
Gossmann, A., Pezeshk, A. X., & Sahiner, B. (2018) · SPIE Medical Imaging: Image Perception, Observer Performance, and Technology Assessment · doi:10.1117/12.2293818 · Cited by 13
Adaptation of General Concepts of Software Testing to Neural Networks
Karpov, Y. L., Карпов, Л. Е., & Smetanin, Y. G. (2018) · Programming and Computer Software · doi:10.1134/s0361768818050031 · Cited by 11
Towards a Bayesian Approach for Assessing Fault Tolerance of Deep Neural Networks
Banerjee, S. S., James, C., Jha, S., Kalbarczyk, Z., & Iyer, R. K. (2019) · IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S) · doi:10.1109/dsn-s.2019.00018 · Cited by 11
Testing Neural Program AnalyzersPreprint
Rabin, M. R. I., Wang, K., & Alipour, M. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10711 · Cited by 11
Al-Azani, S., & Hassine, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-69456-6_7 · Cited by 10
Generalized Oracle for Testing Machine Learning Computer Programs
Nakajima, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-74781-1_13 · Cited by 10
SynEva: Evaluating ML Programs by Mirror Program Synthesis
Qin, Y., Wang, H., Xu, C., Ma, X., & Lü, J. (2018) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs.2018.00031 · Cited by 10
Detecting Failures of Neural Machine Translation in the Absence of Reference Translations
Wang, W., Zheng, W., Liu, D., Zhang, C., Zeng, Q., Deng, Y., Yang, W. M., He, P., & Xie, T. (2019) · doi:10.1109/dsn-industry.2019.00007 · Cited by 10
(2016) · Cited by 10
An Orchestrated Empirical Study on Deep Learning Frameworks and PlatformsPreprint
Guo, Q., Xie, X., Ma, L., Hu, Q., Feng, R., Li, L., Liu, Y., Zhao, J., & Li, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05187 · Cited by 9
Self-citationOpen Questions in Testing of Learned Computer Vision Functions for Automated Driving
Woehrle, M., Gladisch, C., & Heinzemann, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26250-1_27 · Cited by 9
Do Pseudo Test Suites Lead to Inflated Correlation in Measuring Test Effectiveness?
Zhang, J. M., Zhang, L., Hao, D., Wang, M., & Zhang, L. (2019) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2019.00033 · Cited by 8
Self-citationTowards Testing of Deep Learning Systems with Training Set ReductionPreprint
Spieker, H., & Arnaud Gotlieb, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.04169 · Cited by 7
Perturbed Model Validation: A New Framework to Validate Model RelevancePreprint
Zhang, J. M., Barr, E. T., Guedj, B., Harman, M., & Shawe-Taylor, J. S. (2019) · HAL · Cited by 7
Self-citationLifelong Machine Learning Test
Li, L., & Yang, Q. (2015) · Cited by 7
Telemade: A Testing Framework for Learning-Based Malware Detection Systems
Yang, W. M., & Xie, T. (2018) · National Conference on Artificial Intelligence · Cited by 6
“Feedback Loop”: The Civil Rights Act of 1964 and its Progeny
Days, & Drew, S. (2005) · bepress Legal Repository · Cited by 4
“Feedback Loop”: The Civil Rights Act of 1964 and Its Progeny
Days, D. S. (2005) · bepress Legal Repository · Cited by 1
Research works printed but not linked: 16
- Neter, 1996, Applied Linear Statistical Models, 4
- 10.1038/naturei4539 · doi:10.1038/naturei4539
- 2010, IEEE Standard Classification for Software Anomalies
- Kirk, 2014, Thoughtful Machine Learning: A Test-Driven Approach
- 1990, IEEE Standard Glossary of Software Engineering Terminology
- Biran, Explanation and justification in machine learning: A survey
- Goodfellow, Generative adversarial nets, 2672
- Kang, Model assertions for debugging machine learning
- 10.1145/3238147.3238172 · DOI · Paper page
- 10.1109/issre.2018.00021 · DOI · Paper page
- Japkowicz, Why question machine learning evaluation methods, 6
- Bichsel, DD-Finder: Finding differential privacy violations by sampling and optimization, 508
- McClure, 2017, TensorFlow Machine Learning Cookbook
- 10.1109/icse-companion.2019.00131 · DOI
- Belitskaya, Handwritten letters
- 2007, Report to the congress on credit scoring and its effects on the availability and affordability of credit
Cited by
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
Trustworthy AI: From Principles to Practices
Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., & Zhou, B. (2023) · ACM Computing Surveys · doi:10.1145/3555803 · Cited by 612
“Development Functional Testing [164, 235, 381] Performance Benchmarking [40, 56, 88, 93, 123, 127, 153, 238, 248, 307, 327]”
BackgroundSoftware Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
Dawn of the Transformer Era in Speech Emotion Recognition: Closing the Valence Gap
Wagner, J., Triantafyllopoulos, A., Wierstorf, H., Schmitt, M., Burkhardt, F., Eyben, F., & Schuller, B. W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3263585 · Cited by 395
Metadata
- Authors
- Zhang, J. M., Harman, M., Ma, L., & Liu, Y.
- Affiliations
- University College London (GB)
- Meta (United Kingdom) (GB)
- Kyushu University (JP)
- Nanyang Technological University (SG)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2022-01-01
- Volume
- 48
- Issue
- 1
- Pages
- 1-36
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W3007157104
- Cited by
- 897
- References
- 256
- Citations by year
- 2026: 92 · 2025: 115 · 2024: 153 · 2023: 150 · 2022: 141
- FWCI
- 240.63
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Funders
- ERC (741278)
- JSPS KAKENHI (19K24348, 19H04086)
- Qdai-jump Research Program (01277)
- Nvidia
- National Research Foundation Singapore (NRF2018NCR-NCR005-0001)
- National Satellite of Excellence in Trustworthy Software System (NRF2018NCR-NSOE003-0001)
- NTU (NGF-2019-06-024)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- machine learning testing, correctness testing, robustness testing, fairness testing, test case generation, test evaluation, data testing, autonomous driving, machine translation, research trends
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.