The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Software unit test coverage and adequacy
Abstract
Objective measurement of test quality is one of the key issues in software testing. It has been a major research focus for the last two decades. Many test criteria have been proposed and studied for this purpose. Various kinds of rationales have been presented in support of one criterion or another. We survey the research work in this area. The notion of adequacy criteria is examined together with its role in software dynamic testing. A review of criteria classification is followed by a summary of the methods for comparison and assessment of criteria.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
McCabe, T. J. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233837 · Cited by 6,059
Language identification in the limit
Gold, E. M. (1967) · Information and Control · doi:10.1016/s0019-9958(67)91165-5 · Cited by 3,572
Valiant, L. G. (1984) · Communications of the ACM · doi:10.1145/1968.1972 · Cited by 3,279
Hints on Test Data Selection: Help for the Practicing Programmer
DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978) · Computer · doi:10.1109/c-m.1978.218136 · Cited by 2,081
(1990) · Elsevier eBooks · doi:10.1016/c2009-0-21635-7 · Cited by 1,953
(1998) · PDCA/Test · doi:10.1201/9781420048131.axh · Cited by 1,900
Selecting Software Test Data Using Data Flow Information
Rapps, S., & Weyuker, E. J. (1985) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1985.232226 · Cited by 1,075
Evaluating software complexity measures
Weyuker, E. J. (1988) · IEEE Transactions on Software Engineering · doi:10.1109/32.6178 · Cited by 847
Software Metrics. A Rigorous Approach
(1991) · Journal of Information Technology · doi:10.1177/0268396291006003-421 · Cited by 843
Notes on Structured Programming
Dijkstra, E. W. (1970) · TU/e Research Portal · doi:10.26153/tsw/53177 · Cited by 824
Constraint-based automatic test data generation
DeMilli, R. A., & Offutt, A. J. (1991) · IEEE Transactions on Software Engineering · doi:10.1109/32.92910 · Cited by 804
Myers, G. J., Sandler, C., & Badgett, T. (1979) · Cited by 793
The category-partition method for specifying and generating functional tests
Ostrand, T. J., & Balcer, M. J. (1988) · Communications of the ACM · doi:10.1145/62959.62964 · Cited by 791
Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria
Hutchins, M., Foster, H. D., Goradia, T., & Ostrand, T. J. (1994) · doi:10.5555/257734.257766 · Cited by 774
Foundations of Measurement, Vol. I: Additive and Polynomial Representations
Krantz, D. H., Luce, D., Suppes, P., & Tversky, A. (1971) · Cited by 764
Software testing techniques (2nd ed.)
Beizer, B. (1990) · Cited by 744
An Evaluation of Random Testing
Duran, J. W., & Ntafos, S. C. (1984) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1984.5010257 · Cited by 684
Testing Programs with the Aid of a Compiler
Hamlet, R. G. (1977) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1977.231145 · Cited by 612
On testing non-testable programs
Weyuker, E. J. (1982) · The Computer Journal · doi:10.1093/comjnl/25.4.465 · Cited by 606
Current trends in programming methodology
Yeh, R. T., & Chandy, K. M. (1977) · CERN Document Server (European Organization for Nuclear Research) · Cited by 545
Toward a theory of test data selection
Goodenough, J. B., & Gerhart, S. L. (1975) · ACM SIGPLAN Notices · doi:10.1145/390016.808473 · Cited by 514
Comparing the Effectiveness of Software Testing Strategies
Basili, V. R., & Selby, R. W. (1987) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1987.232881 · Cited by 483
Weak Mutation Testing and Completeness of Test Sets
Howden, W. E. (1982) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1982.235571 · Cited by 477
Validation, Verification, and Testing of Computer Software
Adrion, W. R., Branstad, M. A., & Cherniavsky, J. C. (1982) · ACM Computing Surveys · doi:10.1145/356876.356879 · Cited by 426
Partition testing does not inspire confidence (program testing)
Hamlet, D., & Taylor, R. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.62448 · Cited by 400
Reliability of the Path Analysis Testing Strategy
Howden, W. E. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233816 · Cited by 400
A fortran language system for mutation‐based software testing
King, K. N., & Offutt, A. J. (1991) · Software: Practice and Experience · doi:10.1002/spe.4380210704 · Cited by 379
A Domain Strategy for Computer Program Testing
White, L. J., & Cohen, E. I. (1980) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1980.234486 · Cited by 378
Investigations of the software testing coupling effect
Offutt, A. J. (1992) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/125489.125473 · Cited by 371
Software testing based on formal specifications: a theory and a tool
Bernot, G., Claude Gaudel, M., & Marre, B. (1991) · Software Engineering Journal · doi:10.1049/sej.1991.0040 · Cited by 323
Software System Testing and Quality Assurance
Beizer, B. (1984) · Medical Entomology and Zoology · Cited by 315
An experimental comparison of the effectiveness of branch testing and data flow testing
Frankl, P. G., & Weiss, S. N. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.238581 · Cited by 295
(1980) · IEEE Transactions on Computers · doi:10.1109/tc.1980.1675485 · Cited by 286
A theory of fault-based testing
Morell, L. J. (1990) · IEEE Transactions on Software Engineering · doi:10.1109/32.57623 · Cited by 285
Foundations of Measurement, Vol. II: Geometrical, Threshold, and Probabilistic Representations
Suppes, P., Krantz, D. H., Luce, D., & Tversky, A. (1989) · Cited by 281
Estimating the probability of failure when testing reveals no failures
Miller, K. W., Morell, L. J., Noonan, R. E., Park, S. K., Nicol, D. M., Murrill, B. W., & Voas, M. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.120314 · Cited by 281
Two notions of correctness and their relation to testing
Budd, T. A., & Angluin, D. (1982) · Acta Informatica · doi:10.1007/bf00625279 · Cited by 260
An extended overview of the Mothra software testing environment
DeMillo, R. A., Guindi, D. S., McCracken, W. M., Offutt, A. J., & King, K. N. (2003) · [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis · doi:10.1109/wst.1988.5369 · Cited by 257
Additive and Polynomial Representations
(1971) · Elsevier eBooks · doi:10.1016/c2013-0-11004-5 · Cited by 256
Symbolic Testing and the DISSECT Symbolic Evaluation System
Howden, W. E. (1977) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1977.231144 · Cited by 251
An experimental evaluation of selective mutation
Offutt, A. J., Rothermel, G., & Zapf, C. (1993) · doi:10.5555/257572.257597 · Cited by 244
Specification-based test oracles for reactive systems
Richardson, D. J., Aha, S. L., & O'Malley, T. O. (1992) · Proceedings of the 14th international conference on Software engineering - ICSE '92 · doi:10.1145/143062.143100 · Cited by 234
Theories of Program Testing and the Application of Revealing Subdomains
Weyuker, E. J., & Ostrand, T. J. (1980) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1980.234485 · Cited by 223
Automatic generation of random self-checking test cases
Bird, D. L., & Munoz, C. U. (1983) · IBM Systems Journal · doi:10.1147/sj.223.0229 · Cited by 222
A formal evaluation of data flow path selection criteria
Clarke, L. A., Podgurski, A., Richardson, D. J., & Zeil, S. J. (1989) · IEEE Transactions on Software Engineering · doi:10.1109/32.41326 · Cited by 222
Functional program testing and analysis
Howden, W. E. (1986) · McGraw-Hill, Inc. eBooks · Cited by 208
The growth of software testing
Gelperin, D., & Hetzel, B. (1988) · Communications of the ACM · doi:10.1145/62959.62965 · Cited by 202
Methodology for the Generation of Program Test Data
Howden, W. E. (1975) · IEEE Transactions on Computers · doi:10.1109/t-c.1975.224259 · Cited by 196
Proceedings of the 37th Annual Hawaii International Conference on System Sciences
(2004) · 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the · doi:10.1109/hicss.2004.1265029 · Cited by 178
Certifying the reliability of software
Currit, P. A., Dyer, M., & Mills, H. D. (1986) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1986.6312914 · Cited by 176
Performance, effectiveness, and reliability issues in software testing
Mathur, A. P. (2002) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/cmpsac.1991.170248 · Cited by 168
An extension to the cyclomatic measure of program complexity
Myers, G. J. (1977) · ACM SIGPLAN Notices · doi:10.1145/954627.954633 · Cited by 151
Theoretical and Empirical Studies of Program Testing
Howden, W. E. (1978) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1978.231514 · Cited by 144
From weak to strong, dead or alive? an analysis of some mutation testing issues
Woodward, M. R., & Halewood, K. (2003) · doi:10.1109/wst.1988.5370 · Cited by 138
Experimental results from an automatic test case generator
DeMillo, R. A., & Offutt, A. J. (1993) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/151257.151258 · Cited by 132
Using formal methods to derive test frames in category-partition testing
Ammann, P., & Offutt, J. (2002) · Annual Conference on Computer Assurance, Systems Integrity, Software Safety and Process Security (COMPASS) · doi:10.1109/cmpass.1994.318466 · Cited by 120
Partition Analysis: A Method Combining Testing and Verification
Richardson, D. J., & Clarke, L. A. (1985) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1985.231892 · Cited by 114
Coverage measurement experience during function test
Piwowarski, P., Ohba, M., & Caruso, J. (1993) · doi:10.5555/257572.257635 · Cited by 110
The coupling effect: fact or fiction
Offutt, A. J. (1989) · doi:10.1145/75308.75324 · Cited by 110
Assessing Test Data Adequacy through Program Inference
Weyuker, E. J. (1983) · ACM Transactions on Programming Languages and Systems · doi:10.1145/69575.357231 · Cited by 102
A functional approach to program testing and analysis
Howden, W. E. (1986) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1986.6313016 · Cited by 101
The design of a prototype mutation system for program testing
Budd, T. A., Lipton, R. J., DeMillo, R. A., & Sayward, F. G. (1899) · AFIPS conference proceedings · doi:10.1109/afips.1978.195 · Cited by 92
Combining static concurrency analysis with symbolic execution
Young, M., & Taylor, R. N. (1988) · IEEE Transactions on Software Engineering · doi:10.1109/32.6195 · Cited by 91
Analysis of structured programs
Kosaraju, S. R. (1974) · Journal of Computer and System Sciences · doi:10.1016/s0022-0000(74)80043-7 · Cited by 87
An analysis of test data selection criteria using the RELAY model of fault detection
Richardson, D. J., & Thompson, M. C. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.232020 · Cited by 87
(2000) · Auerbach Publications eBooks · doi:10.1201/9781420048124.axh · Cited by 87
The RELAY model of error detection and its application
Richardson, D. J., & Thompson, M. C. (2003) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1109/wst.1988.5378 · Cited by 85
In-process evaluation for software inspection and test
Chaar, J. K., Halliday, M. J., Bhandari, I. S., & Chillarege, R. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.256853 · Cited by 71
The applicability of program schema results to programs
Weyuker, E. J. (1979) · International Journal of Parallel Programming · doi:10.1007/bf00995175 · Cited by 61
Data flow-based test adequacy analysis for languages with pointers
Ostrand, T. J., & Weyuker, E. J. (1991) · doi:10.1145/120807.120814 · Cited by 60
Geometrical, Threshold, and Probabilistic Representations
Suppes, P. (1989) · Medical Entomology and Zoology · Cited by 56
Generalized algorithmic debugging and testing
Fritzson, P., Gyimóthy, T., Kamkar, M., & Shahmehri, N. (1991) · Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/113445.113472 · Cited by 55
Offutt, A. J., & Lee, S. D. (1991) · doi:10.1145/120807.120826 · Cited by 48
Marick, B. (1991) · doi:10.1145/120807.120825 · Cited by 41
Representation, axiomatization, and invariance
Luce, R. D. (1990) · Dover Publications eBooks · Cited by 39
Predicate-based test generation for computer programs
Tai, K. (1993) · doi:10.5555/257572.257631 · Cited by 36
An empirical study of testing and integration strategies using artificial software systems
Solheim, J. A., & Rowland, J. H. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.245736 · Cited by 35
Test data adequacy measurement
Zhu, H. Y., & Hall, P. A. V. (1993) · Software Engineering Journal · doi:10.1049/sej.1993.0004 · Cited by 33
Self-citationA Recursion Theoretic Approach to Program Testing
Cherniavsky, J. C., & Smith, C. H. (1987) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1987.233489 · Cited by 30
Relationship between specifications and testing
Hall, P. (1991) · Information and Software Technology · doi:10.1016/0950-5849(91)90023-5 · Cited by 20
Self-citationClarifying some fundamental concepts in software testing
Parrish, A. S., & Zweben, S. H. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.238573 · Cited by 18
A formal notion of program-based test data adequacy
Davis, M. D., & Weyuker, E. J. (1983) · Information and Control · doi:10.1016/s0019-9958(83)80050-3 · Cited by 15
Some new models of software testing with performance comparisons
Downs, T., & Garrone, P. (1991) · IEEE Transactions on Reliability · doi:10.1109/24.85452 · Cited by 13
Valiant, L. G. (2026) · Foundations of Computation and Machine Learning · doi:10.1145/3749740.3749746
Cited by
Introduction to Software Testing
(2021) · Essentials of Software Testing · doi:10.1017/9781108974073.004 · Cited by 1,056
QuickCheck: a lightweight tool for random testing of Haskell programs
Claessen, K., & Hughes, J. (2011) · ACM SIGPLAN Notices · doi:10.1145/1988042.1988046 · Cited by 1,011
Software Testing Research: Achievements, Challenges, Dreams
Bertolino, A. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.25 · Cited by 840
(2000) · doi:10.1145/351240.351266 · Cited by 753
An orchestrated survey of methodologies for automated software test case generation
Anand, S., Burke, E. K., Chen, T. Y., Clark, J., Cohen, M. B., Grieskamp, W., Harman, M., Harrold, M. J., McMinn, P., Bertolino, A., et al. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2013.02.061 · Cited by 668
Self-citation
Metadata
- Authors
- Zhu, H., Hall, P. A. V., & May, J. H. R.
- Venue
- ACM Computing Surveys
- Published
- 1997-12
- Volume
- 29
- Issue
- 4
- Pages
- 366-427
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2154897437
- Cited by
- 1,247
- References
- 84
- FWCI
- 33.51
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.