The DOI matches its Crossref record; 115 of the 115 references it cites were found in a registry or an index.
An orchestrated survey of methodologies for automated software test case generation
Open accessAbstract
Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software testing for several decades, resulting in many different approaches and tools. This paper presents an orchestrated survey of the most prominent techniques for automatic generation of software test cases, reviewed in self-standing sections. The techniques presented include: (a) structural testing using symbolic execution, (b) model-based testing, (c) combinatorial testing, (d) random testing and its variant of adaptive random testing, and (e) search-based testing. Each section is contributed by world-renowned active researchers on the technique, and briefly covers the basic ideas underlying the method, the current state of the art, a discussion of the open research problems, and a perspective of the future development of the approach. As a whole, the paper aims at giving an introductory, up-to-date and (relatively) short overview of research in automatic test case generation, while ensuring a comprehensive and authoritative treatment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
de Moura, L., & Bjørner, N. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78800-3_24 · Cited by 6,595 · cited 1×
“During the last decade, many powerful constraint solvers (e.g., Z3 (de Moura and Bjørner, 2008), Yices (Dutertre and de Moura, 2006), STP (Ganesh and Dill, 2007)) have been developed.”
BackgroundFisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Software Product Lines: Practices and Patterns
Clements, P., & Northrop, L. (2001) · Cited by 3,232
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
Cadar, C., Dunbar, D., & Engler, D. (2008) · doi:10.5555/1855741.1855756 · Cited by 2,620
The B-book: assigning programs to meanings
(1997) · Choice Reviews Online · doi:10.5860/choice.34-3906 · Cited by 2,106
DART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065036 · Cited by 2,067
Cochran, W. G., & Cox, G. M. (1957) · Cited by 1,729
CUTE: a concolic unit testing engine for C
Sen, K., Marinov, D., & Agha, G. (2005) · Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/1081706.1081750 · Cited by 1,651
DART: directed automated random testing
Godefroid, P., Klarlund, N., & Sen, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065036 · Cited by 1,570
Search‐based software test data generation: a survey
McMinn, P. (2004) · Software Testing, Verification and Reliability · doi:10.1002/stvr.294 · Cited by 1,368
Self-citationSoftware unit test coverage and adequacy
Zhu, H., Hall, P. A. V., & May, J. H. R. (1997) · ACM Computing Surveys · doi:10.1145/267580.267590 · Cited by 1,247
Self-citationPrinciples and methods of testing finite state machines-a survey
Lee, D., & Yannakakis, M. (1996) · Proceedings of the IEEE · doi:10.1109/5.533956 · Cited by 1,228
Practical Model-Based Testing: A Tools Approach
Utting, M., & Legeard, B. (2006) · Cited by 1,192
de Alfaro, L., & Henzinger, T. A. (2001) · Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering · doi:10.1145/503209.503226 · Cited by 1,171
Automated Whitebox Fuzz Testing.
Godefroid, P., Levin, M. Y., & Molnar, D. A. (2008) · Cited by 1,095
QuickCheck: a lightweight tool for random testing of Haskell programs
Claessen, K., & Hughes, J. (2011) · ACM SIGPLAN Notices · doi:10.1145/1988042.1988046 · Cited by 1,011
Search-based software engineering
Harman, M., & Jones, B. F. (2001) · Information and Software Technology · doi:10.1016/s0950-5849(01)00189-6 · Cited by 914
Self-citationSoftware Testing Research: Achievements, Challenges, Dreams
Bertolino, A. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.25 · Cited by 840
Self-citationThe category-partition method for specifying and generating functional tests
Ostrand, T. J., & Balcer, M. J. (1988) · Communications of the ACM · doi:10.1145/62959.62964 · Cited by 791
Pex–White Box Test Generation for .NET
Tillmann, N., & de Halleux, J. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79124-9_10 · Cited by 791
Software fault interactions and implications for software testing
Kuhn, D. R., Wallace, D. R., & Gallo, A. M. (2004) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2004.24 · Cited by 789
(2000) · doi:10.1145/351240.351266 · Cited by 753
A survey of combinatorial testing
Nie, C., & Leung, H. (2011) · ACM Computing Surveys · doi:10.1145/1883612.1883618 · Cited by 714
A Fast Linear-Arithmetic Solver for DPLL(T)
Dutertre, B., & de Moura, L. (2006) · Lecture notes in computer science · doi:10.1007/11817963_11 · Cited by 679
Börger, E., & Stärk, R. F. (2003) · Springer eBooks · doi:10.1007/978-3-642-18216-7 · Cited by 671
Protein-to-protein interactions: Technologies, databases, and algorithms
Cannataro, M., Guzzi, P. H., & Veltri, P. (2010) · ACM Computing Surveys · doi:10.1145/1824795.1824796 · Cited by 620
A Decision Procedure for Bit-Vectors and Arrays
Ganesh, V., & Dill, D. L. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-73368-3_52 · Cited by 585
Test Generation with Inputs, Outputs and Repetitive Quiescence
Tretmans, J. (1996) · University of Twente Research Information · Cited by 579
Generalized Symbolic Execution for Model Checking and Testing
Khurshid, S., PĂsĂreanu, C. S., & Visser, W. (2003) · Lecture Notes in Computer Science · doi:10.1007/3-540-36577-x_40 · Cited by 568
The Art of Prolog: Advanced Programming Techniques
STERLING, L. S., & Shapiro, E. Y. (1994) · Cited by 532
S2E: a platform for in-vivo multi-path analysis of software systems
Chipounov, V., Kuznetsov, V., & Candea, G. (2011) · Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems · doi:10.1145/1950365.1950396 · Cited by 514
Generating software test data by evolution
Michael, C. C., McGraw, G., & Schatz, M. A. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.988709 · Cited by 478
Grammar-based whitebox fuzzing
Godefroid, P., Kiezun, A., & Levin, M. Y. (2008) · Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1375581.1375607 · Cited by 462
Ali, S., Briand, L. C., Hemmati, H., & Panesar-Walawege, R. K. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.52 · Cited by 452
A systematic review of search-based testing for non-functional system properties
Afzal, W., Torkar, R., & Feldt, R. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.12.005 · Cited by 447
Majumdar, R., & Sen, K. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.41 · Cited by 426
Evolutionary testing of classes
Tonella, P. (2004) · Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/1007512.1007528 · Cited by 410
Chen, T. Y., Leung, H., & Mak, I. K. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30502-6_23 · Cited by 409
Self-citationCompositional dynamic test generation
Godefroid, P. (2007) · ACM SIGPLAN Notices · doi:10.1145/1190215.1190226 · Cited by 408
A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search
Harman, M., & McMinn, P. (2010) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2009.71 · Cited by 400
Self-citationCUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools
Sen, K., & Agha, G. (2006) · Lecture Notes in Computer Science · doi:10.1007/11817963_38 · Cited by 389
A Domain Strategy for Computer Program Testing
White, L. J., & Cohen, E. I. (1980) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1980.234486 · Cited by 378
Adaptive Random Testing: The ART of test case diversity
Chen, T. Y., Kuo, F.-C., Merkel, R. G., & Tse, T. H. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2009.02.022 · Cited by 353
Self-citationSoftware testing and analysis: process, principles, and techniques
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0935 · Cited by 340
TimeAware test suite prioritization
Walcott, K. R., Soffa, M. L., Kapfhammer, G. M., & Roos, R. S. (2006) · Proceedings of the 2006 international symposium on Software testing and analysis · doi:10.1145/1146238.1146240 · Cited by 340
Symbolic execution for software testing in practice: preliminary assessment
Cadar, C., Godefroid, P., Khurshid, S., Păsăreanu, C. S., Sen, K., Tillmann, N., & Visser, W. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985995 · Cited by 325
Generating Tests from UML Specifications
Offutt, J., & Abdurazik, A. (1999) · Cited by 323
Search-Based Software Testing: Past, Present and Future
McMinn, P. (2011) · 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops · doi:10.1109/icstw.2011.100 · Cited by 322
Self-citationHarman, M., Mansouri, S., & Zhang, Y. (2009) · UCL Discovery (University College London) · Cited by 313
Self-citationConstructing test suites for interaction testing
Cohen, M. B., Gibbons, P. B., Mugridge, W. B., & Colbourn, C. J. (2003) · doi:10.5555/776816.776822 · Cited by 305
Self-citationThe b-book: Assigning programs to meanings
(1997) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(97)90049-3 · Cited by 289
Automatic Patch-Based Exploit Generation is Possible: Techniques and Implications
Brumley, D., Poosankam, P., Song, D., & Zheng, J. (2008) · 2008 IEEE Symposium on Security and Privacy (sp 2008) · doi:10.1109/sp.2008.17 · Cited by 281
Symbolic PathFinder: symbolic execution of Java bytecode
Păsăreanu, C. S., & Rungta, N. (2010) · Proceedings of the IEEE/ACM international conference on Automated software engineering · doi:10.1145/1858996.1859035 · Cited by 259
A novel co-evolutionary approach to automatic software bug fixing
Arcuri, A., & Yao, X. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4630793 · Cited by 250
Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer
Veanes, M., Campbell, C., Grieskamp, W., Schulte, W., Tillmann, N., & Nachmanson, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78917-8_2 · Cited by 236
Self-citationRWset: Attacking Path Explosion in Constraint-Based Test Generation
Boonstoppel, P., Cadar, C., & Engler, D. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78800-3_27 · Cited by 234
A survey of new trends in symbolic execution for software testing and analysis
Păsăreanu, C. S., & Visser, W. (2009) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-009-0118-1 · Cited by 231
Generating Tests from UML Specifications
Offutt, J., & Abdurazik, A. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-46852-8_30 · Cited by 229
Demand-Driven Compositional Symbolic Execution
Anand, S., Godefroid, P., & Tillmann, N. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78800-3_28 · Cited by 217
Self-citationAutomated and Scalable T-wise Test Case Generation Strategies for Software Product Lines
Perrouin, G., Sen, S., Klein, J., Baudry, B., & Traon, Y. L. (2010) · 2010 Third International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2010.43 · Cited by 217
Ma, K.-K., Yit Phang, K., Foster, J. S., & Hicks, M. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23702-7_11 · Cited by 209
Combinatorial Aspects of Covering Arrays
Colbourn, C. J. (2004) · Le Matematiche · Cited by 203
ARTOO: adaptive random testing for object-oriented software
Ciupa, I., Leitner, A., Oriol, M., & Meyer, B. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368099 · Cited by 198
A new approach to program testing
King, J. C. (1975) · ACM SIGPLAN Notices · doi:10.1145/390016.808444 · Cited by 194
Generating finite state machines from abstract state machines
Grieskamp, W., Gurevich, Y. G., Schulte, W., et al. (2002) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/566171.566190 · Cited by 184
Self-citationLoop-extended symbolic execution on binary programs
Saxena, P., Poosankam, P., McCamant, S., & Song, D. (2009) · Proceedings of the eighteenth international symposium on Software testing and analysis · doi:10.1145/1572272.1572299 · Cited by 163
Test-Suite Augmentation for Evolving Software
Santelices, R., Chittimalli, P. K., Apiwattanapong, T., Orso, A., & Harrold, M. J. (2008) · 2008 23rd IEEE/ACM International Conference on Automated Software Engineering · doi:10.1109/ase.2008.32 · Cited by 160
Self-citationMutation-driven generation of unit tests and oracles
Fraser, G., & Zeller, A. (2010) · International Symposium on Software Testing and Analysis · doi:10.1145/1831708.1831728 · Cited by 159
A multi-objective approach to search-based test data generation
Lakhotia, K., Harman, M., & McMinn, P. (2007) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1276958.1277175 · Cited by 158
Self-citationRex: Symbolic Regular Expression Explorer
Veanes, M., de Halleux, P., & Tillmann, N. (2010) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2010.15 · Cited by 152
Strong higher order mutation-based test data generation
Harman, M., Jia, Y., & Langdon, W. B. (2011) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2025113.2025144 · Cited by 142
Self-citationArcuri, A., & Briand, L. (2011) · International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001452 · Cited by 130
Automatic Mutation Test Case Generation via Dynamic Symbolic Execution
Papadakis, M., & Malevris, N. (2010) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2010.38 · Cited by 127
\nTest Generation Based on Symbolic Specifications
Frantzen, L., Tretmans, J., & Willemse, T. A. C. (2005) · Radboud Repository (Radboud University) · doi:10.1007/978-3-540-31848-4_1 · Cited by 125
On the automation of fixing software bugs
Arcuri, A. (2008) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1370175.1370223 · Cited by 125
Search Based Requirements Optimisation: Existing Work and Challenges
Zhang, Y., Finkelstein, A., & Harman, M. (2008) · Springer eBooks · doi:10.1007/978-3-540-69062-7_8 · Cited by 123
Self-citationAutomatic partial loop summarization in dynamic test generation
Godefroid, P., & Luchaup, D. (2011) · International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001424 · Cited by 120
A deterministic density algorithm for pairwise interaction coverage.
Colbourn, C. J., Cohen, M. B., & Turban, R. (2004) · Cited by 118
Self-citationAutomated web application testing using search based software engineering
Alshahwan, N., & Harman, M. (2011) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2011.6100082 · Cited by 116
Self-citationBetter bug reporting with better privacy
Castro, M., Costa, M., & Martin, J.-P. (2008) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/1346281.1346322 · Cited by 111
Directed test generation using symbolic grammars
Majumdar, R., & Xu, R.-G. (2007) · doi:10.1145/1321631.1321653 · Cited by 104
Giacobazzi, R., & Cousot, R. (2013) · HAL · Cited by 101
An upper bound on software testing effectiveness
Chen, T. Y., & Merkel, R. (2008) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/1363102.1363107 · Cited by 99
Self-citationA subset of precise UML for model-based testing
Bouquet, F., Grandpierre, C., Legeard, B., Peureux, F., Vacelet, N., & Utting, M. (2007) · doi:10.1145/1291535.1291545 · Cited by 99
Symbolic execution with mixed concrete-symbolic solving
Păsăreanu, C. S., Rungta, N., & Visser, W. (2011) · International Symposium on Software Testing and Analysis · doi:10.1145/2001420.2001425 · Cited by 96
Adaptive random testing through dynamic partitioning
Chen, T. Y., Merkel, R., Wong, P. K.-Y., et al. (2004) · doi:10.1109/qsic.2004.6 · Cited by 95
Self-citationBaars, A. I., Harman, M., Hassoun, Y., Lakhotia, K., McMinn, P., Tonella, P., & Vos, T. E. J. (2011) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2011.6100119 · Cited by 93
Self-citationHarman, M., Kim, S. G., Lakhotia, K., McMinn, P., & Yoo, S. (2010) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2010.31 · Cited by 85
Self-citationFloPSy - Search-Based Floating Point Constraint Solving for Symbolic Execution
Lakhotia, K., Tillmann, N., Harman, M., & de Halleux, J. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-16573-3_11 · Cited by 82
Self-citationAn empirical investigation into branch coverage for C programs using CUTE and AUSTIN
Lakhotia, K., McMinn, P., & Harman, M. (2010) · Journal of Systems and Software · doi:10.1016/j.jss.2010.07.026 · Cited by 77
Self-citationjFuzz: A Concolic Whitebox Fuzzer for Java
Jayaraman, K., Harvison, D., Ganesh, V., & Kieżun, A. (2009) · Cited by 74
Optimal strategies for testing nondeterministic systems
Nachmanson, L., Veanes, M., Schulte, W., Tillmann, N., & Grieskamp, W. (2004) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/1007512.1007520 · Cited by 73
Self-citationSoftware Engineering Meets Evolutionary Computation
Harman, M. (2011) · Computer · doi:10.1109/mc.2011.263 · Cited by 72
Self-citationCORAL: Solving Complex Constraints for Symbolic PathFinder
Souza, M., Borges, M., d’Amorim, M., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-20398-5_26 · Cited by 70
Symbolic execution with abstraction
Anand, S., Păsăreanu, C. S., & Visser, W. (2008) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-008-0090-1 · Cited by 69
Self-citationAutomating test case generation from Z specifications with Isabelle
Helke, S., Neustupny, T., & Santen, T. (1997) · Lecture notes in computer science · doi:10.1007/bfb0027283 · Cited by 67
Automated synthesis of symbolic instruction encodings from I/O samples
Godefroid, P., & Taly, A. (2012) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/2254064.2254116 · Cited by 65
Reducing qualitative human oracle costs associated with automatically generated test data
McMinn, P., Stevenson, M., & Harman, M. (2010) · doi:10.1145/1868048.1868049 · Cited by 64
Self-citationDarwin: an approach for debugging evolving programs
Qi, D., Roychoudhury, A., Liang, Z., & Vaswani, K. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595704 · Cited by 63
Wegener, J., & Bühler, O. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24855-2_160 · Cited by 61
Variably interprocedural program analysis for runtime error detection
Tomb, A., Brat, G. P., & Visser, W. (2007) · International Symposium on Software Testing and Analysis · doi:10.1145/1273463.1273478 · Cited by 59
Search-Based Test Input Generation for String Data Types Using the Results of Web Queries
McMinn, P., Shahbaz, M., & Stevenson, M. (2012) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2012.94 · Cited by 59
Self-citationEmpirical evaluation of a nesting testability transformation for evolutionary testing
McMinn, P., Binkley, D., & Harman, M. (2009) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/1525880.1525884 · Cited by 58
Self-citationExploiting Common Object Usage in Test Case Generation
Fraser, G., & Zeller, A. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.53 · Cited by 56
A Divergence-Oriented Approach to Adaptive Random Testing of Java Programs
Lin, Y., Tang, X., Chen, Y., & Zhao, J. (2009) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2009.13 · Cited by 50
Hemmati, H., Arcuri, A., & Briand, L. (2011) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2011.12 · Cited by 50
Reducing Test Inputs Using Information Partitions
Majumdar, R., & Xu, R.-G. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02658-4_41 · Cited by 45
An Approach to Test Data Generation for Killing Multiple Mutants
Liu, M., Gao, Y.-F., Shan, J.-H., Liu, J., Zhang, L., & Sun, J. (2006) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsm.2006.13 · Cited by 33
Test generation for graphical user interfaces based on symbolic execution
Ganov, S., Killmar, C., Sarfraz Khurshid, D., & Perry, D. E. (2008) · International Workshop on Automation of Software Test (AST) · doi:10.1145/1370042.1370050 · Cited by 31
Type-Dependence Analysis and Program Transformation for Symbolic Execution
Anand, S., Orso, A., & Harrold, M. J. (2007) · Springer eBooks · doi:10.1007/978-3-540-71209-1_11 · Cited by 28
Self-citationWallace, I., & Rovatsos, M. (2009) · Adaptive Agents and Multi-Agents Systems · Cited by 6
Cited by
DeepTest: automated testing of deep-neural-network-driven autonomous cars
Tian, Y., Pei, K., Jana, S., & Ray, B. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180220 · Cited by 1,217
Large language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
“Clearly,automated testing techniques [3]–[5] will have a central role to play in ensuring correctness, just as they already do for human-engineered artefacts.”
BackgroundSelf-citation(2021) · doi:10.1109/tse.2019.2946563 · Cited by 574
A Survey on Metamorphic Testing
Segura, S., Fraser, G., Sanchez, A. B., & Ruiz-Cortes, A. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2532875 · Cited by 563
Large Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
Self-citation
Metadata
- Authors
- Anand, S., Burke, E. K., Chen, T. Y., Clark, J., Cohen, M. B., Grieskamp, W., Harman, M., Harrold, M. J., McMinn, P., Bertolino, A., Jenny Li, J., & Zhu, H.
- Venue
- Journal of Systems and Software
- Publisher
- Elsevier BV
- Published
- 2013-08
- Volume
- 86
- Issue
- 8
- Pages
- 1978-2001
- ISSN
- 0164-1212 (ISSN-L)
- OpenAlex
- W1979345446
- Cited by
- 668
- References
- 115
- FWCI
- 158.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.