Testing and quality validation for AI software–perspectives, issues, and practices
Open accessAbstract
With the fast growth of artificial intelligence and big data computing technologies, more and more software service systems have been developed using diverse machine learning models and technologies to make business and intelligent decisions based on their multimedia input to achieve intelligent features, such as image recognition, recommendation, decision making, prediction, etc. Nevertheless, there are increasing quality problems resulting in erroneous testing costs in enterprises and businesses. Existing work seldom discusses how to perform testing and quality validation for AI software. This paper focuses on quality validation for AI software function features. The paper provides our understanding of AI software testing for new features and requirements. In addition, current AI software testing categories are presented and different testing approaches are discussed. Moreover, test quality assessment and criteria analysis are illustrated. Furthermore, a practical study on quality validation for an image recognition system is performed through a metamorphic testing method. Study results show the feasibility and effectiveness of the approach.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It is necessary to deploy and integrate AI models into production systems and assure quality of continuously evolving and self-adapting systems to harness the power of AI.
“To harness the power of AI, it is necessary to deploy and integrate AI models into production systems and to assure the quality of the resulting continuously evolving and self-adapting systems [F24, 46, 47, 38].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [47]
Quality assurance requires thorough attention to guarantee safe and reliable behaviour and increase accountability and responsibility of involved AI systems.
“However, quality assurance requires thorough attention to guarantee safe and reliable behaviour and increase the accountability and responsibility of the involved AI systems [38, 47, 13, 23].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [47]
AI characteristics such as inherent non-determinism lead to a certain degree of uncertainty.
“AI characteristics such as the inherent non-determinism lead to a certain degree of uncertainty [46, 47, 38].”
From The pipeline for the continuous development of artificial intelligence models—Current state of research and practice (2023) · cites this paper as [47]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290 · cited 1× · see citing works
“Turing test was introduced by Turing as the imitation game in 1950 [5], aiming to test a machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.”
MethodThe category-partition method for specifying and generating functional tests
Ostrand, T. J., & Balcer, M. (1988) · Communications of the ACM · doi:10.1145/62959.62964 · Cited by 790 · see citing works
DeepGauge: multi-granularity testing criteria for deep learning systems
Ma, L., Juefei-Xu, F., Zhang, F., Sun, J., Xue, M., Li, B., Chen, C., Su, T., Li, L., Liu, Y., et al. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3238202 · Cited by 710 · see citing works
NAS-Unet: Neural Architecture Search for Medical Image Segmentation
Weng, Y., Zhou, T., Li, Y., & Qiu, X. (2019) · IEEE Access · doi:10.1109/access.2019.2908991 · Cited by 487 · see citing works
Testing and validating machine learning classifiers by metamorphic testing
Xie, X., Ho, J. W. K., Murphy, C., Kaiser, G. E., Xu, B., & Chen, T. Y. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.11.920 · Cited by 394 · see citing works
Pinar Saygin, A., Cicekli, I., & Akman, V. (2000) · Minds and Machines · doi:10.1023/a:1011288000451 · Cited by 383 · see citing works
Automated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., & Minch, M. (2003) · Cited by 314 · see citing works
Intelligence Testing for Autonomous Vehicles: A New Approach
Li, L., Huang, W., Liu, Y., Zheng, N., & Wang, F. (2016) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2016.2608003 · Cited by 267 · see citing works
Automated support for classifying software failure reports
Podgurski, A., Leon, D., Francis, P., Masri, W., Minch, M., Sun, J., & Wang, B. (2003) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2003.1201224 · Cited by 226 · see citing works
A Practitioner's Guide to Software Test Design
Copeland, L. (2003) · Medical Entomology and Zoology · Cited by 210 · see citing works
Qi, L., Zhang, X., Dou, W., & Ni, Q. (2017) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2017.2760458 · Cited by 204 · see citing works
Active learning for automatic classification of software behavior
Bowring, J. F., Rehg, J. M., & Harrold, M. J. (2004) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/1007512.1007539 · Cited by 179 · see citing works
Parallel learning: a perspective and a framework
Li, L., Lin, Y., Zheng, N., & Wang, F. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510493 · Cited by 173 · see citing works
An innovative approach for testing bioinformatics programs using metamorphic testing
Chen, T. Y., Ho, J. W. K., Liu, H., & Xie, X. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-24 · Cited by 159 · see citing works
Properties of Machine Learning Applications for Use in Metamorphic Testing
Murphy, C., Kaiser, G. E., & Hu, L. (2008) · doi:10.7916/d8xk8pfd · Cited by 155 · see citing works
Big Data's Big Unintended Consequences
Wigan, M. R., & Clarke, R. (2013) · Computer · doi:10.1109/mc.2013.195 · Cited by 155 · see citing works
Artificial intelligence test: a case study of intelligent vehicles
Li, L., Lin, Y.-L., Zheng, N., Wang, F., Liu, Y., Cao, D., Wang, K., & Huang, W. (2018) · Artificial Intelligence Review · doi:10.1007/s10462-018-9631-5 · Cited by 152 · see citing works
A study on key technologies of unmanned driving
Zhang, X., Gao, H., Guo, M. Y., Li, G., Liu, Y., & Li, D. (2016) · CAAI Transactions on Intelligence Technology · doi:10.1016/j.trit.2016.03.003 · Cited by 143 · see citing works
Location-Aware Service Recommendation With Enhanced Probabilistic Matrix Factorization
Yin, Y., Chen, L., Xu, Y., & Wan, J. (2018) · IEEE Access · doi:10.1109/access.2018.2877137 · Cited by 123 · see citing works
Using a neural network in the software testing process
Vanmali, M., Last, M., & Kandel, A. (2002) · International Journal of Intelligent Systems · doi:10.1002/int.1002 · Cited by 117 · see citing works
Metamorphic testing of programs on partial differential equations: a case study
Chen, T. Y., Feng, J., & Tse, T. H. (2003) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/cmpsac.2002.1045022 · Cited by 113 · see citing works
Toward service selection for workflow reconfiguration:An interface-based computing solution
Gao, H., Huang, W., Yang, X., Duan, Y., & Yin, Y. (2018) · Future Generation Computer Systems · doi:10.1016/j.future.2018.04.064 · Cited by 112 · see citing works
PROUD—Public Road Urban Driverless-Car Test
Broggi, A., Cerri, P., Debattisti, S., Laghi, M. C., Medici, P., Molinari, D., Panciroli, M., & Prioletti, A. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2477556 · Cited by 96 · see citing works
Dynamic Mobile Crowdsourcing Selection for Electricity Load Forecasting
Qi, L., Dou, W., Wang, W., Li, G., Yu, H., & Wan, S. (2018) · IEEE Access · doi:10.1109/access.2018.2866641 · Cited by 91 · see citing works
Tree-Based Methods for Classifying Software Failures
Francis, P., Leon, D., Minch, M., & Podgurski, A. (2005) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2004.43 · Cited by 81 · see citing works
QoS Prediction for Web Service Recommendation with Network Location-Aware Neighbor Selection
Yin, Y., Aihua, S., Min, G., Xu, Y., & Wang, S. (2016) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194016400040 · Cited by 75 · see citing works
An Empirical Study on Real Bugs for Machine Learning Programs
Sun, X., Zhou, T., Li, G., Hu, J., Yang, H. M., & Li, B. (2017) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2017.41 · Cited by 72 · see citing works
Naphade, M., Anastasiu, D. C., Sharma, A., Jagrlamudi, V., Jeon, H., Liu, K., Chang, M., Lyu, S., & Gao, Z. (2017) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397673 · Cited by 70 · see citing works
Gao, H., Mao, S., Huang, W., & Yang, X. (2018) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2018.2865217 · Cited by 62 · see citing works
Gao, H., Zhang, K., Yang, J., Wu, F., & Liu, H. (2018) · International Journal of Distributed Sensor Networks · doi:10.1177/1550147718761583 · Cited by 61 · see citing works
The data mining approach to automated software testing
Last, M., Friedman, M., & Kandel, A. (2003) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/956750.956795 · Cited by 59 · see citing works
An Approach to Data Consistency Checking for the Dynamic Replacement of Service Process
Gao, H., Duan, Y., Miao, H., & Yin, Y. (2017) · IEEE Access · doi:10.1109/access.2017.2715322 · Cited by 58 · see citing works
Collaborative QoS Prediction for Mobile Service with Data Filtering and SlopeOne Model
Yin, Y., Xu, W., Xu, Y., He, L., & Yu, L. (2017) · Mobile Information Systems · doi:10.1155/2017/7356213 · Cited by 53 · see citing works
A rule-based software test data generator
Deason, W. H., Brown, D. B., Chang, K.-H., & Cross, J. H. (1991) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.75894 · Cited by 51 · see citing works
Automatic Convolutional Neural Architecture Search for Image Classification Under Different Scenes
Weng, Y., Zhou, T., Liu, L., & Xia, C. (2019) · IEEE Access · doi:10.1109/access.2019.2906369 · Cited by 51 · see citing works
On Random Testing of Image Processing Applications
Mayer, J., & Guderlei, R. (2006) · Proceedings of the ... International Conference on Quality Software/Proceedings · doi:10.1109/qsic.2006.45 · Cited by 49 · see citing works
Enhancing developer recommendation with supplementary information via mining historical commits
Sun, X.-B., Yang, H., Xia, X., & Li, B. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2017.09.021 · Cited by 44 · see citing works
A Learning-Based Approach to Unit Testing of Numerical Software
Meinke, K., & Niu, F. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16573-3_16 · Cited by 36 · see citing works
Parallel Systems for Traffic Control: A Rethinking
Li, L., & Wen, D. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2015.2494625 · Cited by 32 · see citing works
How security bugs are fixed and what can be improved: an empirical study with Mozilla
Sun, X., Peng, X., Zhang, K., Liu, Y., & Cai, Y. (2018) · Science China Information Sciences · doi:10.1007/s11432-017-9459-5 · Cited by 31 · see citing works
Inferring Automatic Test Oracles
Langdon, W. B., Yoo, S., & Harman, M. (2017) · doi:10.1109/sbst.2017.1 · Cited by 28 · see citing works
Straub, J., & Huber, J. (2013) · Computers · doi:10.3390/computers2020067 · Cited by 26 · see citing works
Modeling Input Space for Testing Scientific Computational Software: A Case Study
Vilkomir, S., Swain, W. T., Poore, J. H., & Clarno, K. T. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69389-5_34 · Cited by 20 · see citing works
Enhanced Extraction Clinical Data Technique to Improve Data Quality in Clinical Data Warehouse
Mohammed, A. O., & Talab, S. A. (2015) · International Journal of Database Theory and Application · doi:10.14257/ijdta.2015.8.3.29 · Cited by 9 · see citing works
ComboRT: A New Approach for Generating Regression Test Cases for Evolving Programs
Sun, X., Peng, X., LEUNG, H. K. N., & Li, B. (2016) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194016500340 · Cited by 6 · see citing works
A Rule-Based Software Testing Method for VHDL Models.
Andrews, A. A., O'Fallon, A., & Chen, T. (2003) · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- J. Gao, C. L. Xie, and C. Q. Tao, ''Quality assurance for big data-issuss, challenges, and needs,'' in Proc. IEEE 10th Int. Symp. Service Oriented Syst. Eng., Apr. 2016, pp. 433-441.
- A. Souri, M. E. Akbari, and A. Salehpour, ''Reduction and modification of test cases in Web applications by using multi objective genetic algorithm,'' J. Amer. Sci., vol. 8, no. 4, pp. 757-762, 2012.
Cited by
Sorted by citations
Giray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187 · see citing works
Steidl, M., Felderer, M., & Ramler, R. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111615 · Cited by 126 · see citing works
A survey on artificial intelligence assurance
Batarseh, F. A., Freeman, L., & Huang, C. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00445-7 · Cited by 117 · see citing works
Sustainable MLOps: Trends and Challenges
Tamburri, D. A. (2020) · 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC) · doi:10.1109/synasc51798.2020.00015 · Cited by 124 · see citing works
Early Detection of Earthquakes Using IoT and Cloud Infrastructure: A Survey
Abdalzaher, M. S., Krichen, M., Yiltas-Kaplan, D., Dhaou, I. B., & Adoni, W. Y. H. (2023) · Sustainability · doi:10.3390/su151511713 · Cited by 95 · see citing works
Diana, L., & Dini, P. (2024) · Remote Sensing · doi:10.3390/rs16213957 · Cited by 50 · see citing works
Systematic literature review on software quality for AI-based software
Gezici, B., & Tarhan, A. (2022) · Empirical Software Engineering · doi:10.1007/s10664-021-10105-2 · Cited by 47 · see citing works
ISO/IEC quality standards for AI engineering
Oviedo, J., Rodríguez, M., Trenta, A., Cannas, D., Natale, D., & Piattini, M. G. (2024) · Computer Science Review · doi:10.1016/j.cosrev.2024.100681 · Cited by 42 · see citing works
AI Regulation Is (not) All You Need
Lucaj, L., van der Smagt, P., & Benbouzid, D. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594079 · Cited by 40 · see citing works
Addressing Accuracy Paradox Using Enhanched Weighted Performance Metric in Machine Learning
Uddin, M. F. (2019) · HCT Information Technology Trends (ITT) · doi:10.1109/itt48889.2019.9075071 · Cited by 38 · see citing works
A Systematic Mapping of Quality Models for AI Systems, Software and Components
Ali, M. A., Yap, N. K., Ghani, A., Zulzalil, H. B., Admodisastro, N. I., & Najafabadi, A. A. (2022) · Applied Sciences · doi:10.3390/app12178700 · Cited by 24 · see citing works
Trust Management for Artificial Intelligence: A Standardization Perspective
Um, T., Kim, J., Lim, S., & Lee, G. M. (2022) · Applied Sciences · doi:10.3390/app12126022 · Cited by 17 · see citing works
Megaro, A., Carrubbo, L., Polese, F., & Sirianni, C. A. (2022) · The TQM Journal · doi:10.1108/tqm-02-2022-0072 · Cited by 18 · see citing works
Artificial Intelligence in Software Testing
Banala, S., Panyaram, S., & Selvakumar, P. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9356-7.ch009 · Cited by 14 · see citing works
IoT-based Battery Management System by Deploying an Machine Learning Model
Prabha, A., Saravanan, S., Balaji, S., Malini, K. V., Thopate, K., Sivakami, T., & Ramkumar, M. S. (2022) · International Conference on Smart Electronics and Communication (ICOSEC) · doi:10.1109/icosec54921.2022.9952044 · Cited by 12 · see citing works
Indykov, V., Strüber, D., & Wohlrab, R. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112373 · Cited by 11 · see citing works
Development of a Maturity Model for Software Quality Assurance Practices
MohamadSaleh, A. A., & Alzahrani, S. (2023) · Systems · doi:10.3390/systems11090464 · Cited by 10 · see citing works
Khan, S. A., Oshin, N. T., Nizam, M., Ahmed, I., Musfique, M. M., & Hasan, M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-8346-9_28 · Cited by 8 · see citing works
AI Test Modeling and Analysis for Intelligent Chatbot Mobile App - A Case Study on Wysa
Gao, J. Z., Garsole, P., Agarwal, R., & Liu, S. (2024) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest62860.2024.00024 · Cited by 8 · see citing works
Model-Based Test Modeling and Automation Tool for Intelligent Mobile Apps
Gao, J. Z., Patil, P. H., Lu, S., Cao, D., & Tao, C. (2021) · IEEE International Symposium on Service-Oriented System Engineering (SOSE) · doi:10.1109/sose52839.2021.00028 · Cited by 8 · see citing works
Gao, J. Z., Wu, W., & Aktouf, O.-E.-K. (2022) · IEEE International Symposium on Service-Oriented System Engineering (SOSE) · doi:10.1109/sose55356.2022.00025 · Cited by 8 · see citing works
Metin, B., Wynn, M. G., Tunalı, A., & Kepir, Y. (2025) · Information · doi:10.3390/info16070585 · Cited by 7 · see citing works
Artificial Intelligence For Software Testing-Perspectives And Practices
Jha, N., & Popli, R. (2021) · International Conference on Computational Intelligence and Communication Technologies (CCICT) · doi:10.1109/ccict53244.2021.00075 · Cited by 7 · see citing works
Transparency in AI Systems for Value Co-creation in Healthcare
Megaro, A. (2023) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-80382-551-920231007 · Cited by 8 · see citing works
Towards a Comprehensive Ontology for Requirements Engineering for AI-Powered Systems
Sadovski, E., Aviv, I., & Hadar, I. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-57327-9_14 · Cited by 7 · see citing works
AI Testing for Intelligent Chatbots—A Case Study
Gao, J. Z., Agarwal, R., & Garsole, P. (2025) · Software · doi:10.3390/software4020012 · Cited by 7 · see citing works
State-of-the-Art Review of Taxonomies for Quality Assessment of Intelligent Software Systems
Kharlamova, A., Jabborov, A., Kruglov, A. V., & Succi, G. (2022) · International Informatics and Software Engineering Conference (IISEC) · doi:10.1109/iisec56263.2022.9998232 · Cited by 6 · see citing works
Artificial Intelligence Applied to Software Testing
Singh, A. K., & Al-Azzam, O. (2023) · doi:10.5121/csit.2023.132001 · Cited by 5 · see citing works
Malin, C. D., Fleiß, J., & Thalmann, S. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1561322 · Cited by 9 · see citing works
The Role of AI in Software Test Automation
Ranapana, R. M. S., & Wijayanayake, J. I. (2025) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc64760.2025.10962814 · Cited by 3 · see citing works
Tablada-Dominguez, A., Muñoz, M., Ocharán-Hernández, J. O., & Sánchez-García, Á. J. (2023) · Mexican International Conference on Computer Science · doi:10.1109/enc60556.2023.10508685 · Cited by 3 · see citing works
Computer Vision Intelligence Test Modeling and Generation: A Case Study on Smart OCR
Shu, J., Miu, B.-J., Chang, E., Gao, J. Z., & Liu, J. (2024) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest62860.2024.00011 · Cited by 3 · see citing works
Machine Learning and Supply Chain Management
Quayson, M., Bai, C. A., Effah, D., & Ofori, K. S. (2023) · Springer eBooks · doi:10.1007/978-3-030-89822-9_92-1 · Cited by 2 · see citing works
Thammastitkul, A., & Petsuwan, J. (2024) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v13.i2.pp1294-1303 · Cited by 2 · see citing works
Machine Learning and Supply Chain Management
Quayson, M., Bai, C. A., Effah, D., & Ofori, K. S. (2024) · Springer eBooks · doi:10.1007/978-3-031-19884-7_92 · Cited by 1 · see citing works
Quality of Big Data Systems: a Systematic Review of Practices Methods and Tools
de Oliveira, I. S., Alves, J. M., Alcântara, S., Santos, I. D. S., & Andrade, R. M. C. (2024) · Brazilian Symposium on Software Quality (SBQS) · doi:10.1145/3701625.3701642 · Cited by 1 · see citing works
Software Fairness Testing in Practice
Santos, R. D. S., Leça, M. D. M., Santos, R., & de Magalhães, C. V. C. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00012 · Cited by 1 · see citing works
Enhancing Software Testing Through Artificial Intelligence
Erol, E., & Senan, S. (2024) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/979-8-3693-3502-4.ch012 · Cited by 1 · see citing works
Metamorphic Testing for Plant Identification Mobile Applications Based on Test Contexts
Guo, H., Tao, C., & Huang, Z. (2020) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-64214-3_15 · Cited by 1 · see citing works
A Case Study of Testing an Image Recognition Application (S)
Tao, C. (2021) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2021-194 · Cited by 1 · see citing works
Myklebust, T., & Stålhane, T. (2021) · Springer eBooks · doi:10.1007/978-3-030-86152-0_9 · Cited by 1 · see citing works
Decision support system choosing software testing strategy
Чернов, В. Г., Dorokhova, L., Dorokhov, O., & Egorovskaya, G. (2020) · Bulletin of the "Transilvania" University of Braşov. Series III, Mathematics and Computer Science · doi:10.31926/but.mif.2019.12.61.2.23 · Cited by 1 · see citing works
A Case Study of Testing an Image Recognition
Tao, C., & Tao, C. (2021) · doi:10.18293/jvlc2021-n2-194
Zhong, Y., & Luo, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-35930-9_13
Alphonce, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4607901
Sustainability-centric integration software requirements validation
Kandukuri, P., Kumar, D. N. V. S., Ramesh, G., & Kativarapu, P. (2024) · MATEC Web of Conferences · doi:10.1051/matecconf/202439201160
An Ethical Trio—AI, Cybersecurity, and Coding at Scale
Hurlburt, G. F. (2024) · IT Professional · doi:10.1109/mitp.2024.3386050
Marwala, T. (2024) · Springer eBooks · doi:10.1007/978-981-97-9251-1_6
Evaluating AI-Based Unit Testing Tools: A Focus on Usability, Reliability, and Integration
Aarifeen, H., & Wickramaarachchi, D. (2025) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc64760.2025.10963099
AI and Knowledge Management: Navigating the Misinformation Maze
Kheng, L., & Schriml, M. D. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-80592-391-620261007
EP-BAST: Ensemble Predictive Time Budget Allocation for Software Testing
Marçal, I., & Garcia, R. E. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-12876-8_43
Artificial intelligence as a low-precision technology
Marwala, T. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-36322-1.00030-4
Marwala, T. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-36322-1.00009-2
Morriss, J. M., Edsberg, L. E., Cox, J., & Capasso, V. A. (2026) · Journal of Wound Ostomy and Continence Nursing · doi:10.1097/won.0000000000001259
Lisaru, C.-V., & Kifor, C. V. (2026) · Acta Informatica Pragensia · doi:10.18267/j.aip.303
Big Data Testing: A Multivocal Literature Review Bridging Academic Research and Industry PracticePreprint
Icaro, Santos, I. D. S., Alves, J. M. D. L., da Silva, N. N. A., & Andrade, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6880211
AI-Based Software Verification and Validation System for Software Vendors
Njoku, D. O., Onukwugha, C. G., Nwanga, M. E., Ewunonu, T. C., Ajunwa, I. H. H., & Nwoke, E. C. (2025) · IEEE Connecting the Unconnected Europe, Middle East, and Africa Regional Summit (CTUS-EMEA) · doi:10.1109/ctus-emea68989.2025.11592270
Xia, M., & Li, Y. (2026) · International Symposium on AI-Driven Engineering Systems (ISADES) · doi:10.1109/isades69945.2026.11608231
- Authors
- Tao, C., Gao, J., & Wang, T.
- Affiliations
- Nanjing University (CN)
- San Jose State University (US)
- Nanjing University of Aeronautics and Astronautics (CN)
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 120164-120175
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2969466556
- Cited by
- 60
- References
- 46
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 13 · 2023: 12 · 2022: 7 · 2021: 7 · 2020: 3 · 2019: 1
- FWCI
- 10.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Key Research and Development Program of China (2018YFB1003900)
- National Natural Science Foundation of China (61402229, 61602267)
- Collaborative Innovation Center of Novel Software Technology and Industrialization
- Fundamental Research Funds for the Central Universities (NS2019058)
- State Key Laboratory of Novel Software Technology (KFKT2018B19)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Networks and Communications › Software System Performance and Reliability
- Keywords
- metamorphic testing, image recognition system, test quality evaluation, test criteria
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
