The DOI matches its Crossref record.
Code coverage for suite evaluation by developers
Abstract
One of the key challenges of developers testing code is determining a test suite's quality -- its ability to find faults. The most common approach is to use code coverage as a measure for test suite quality, and diminishing returns in coverage or high absolute coverage as a stopping rule. In testing research, suite quality is often evaluated by a suite's ability to kill mutants (artificially seeded potential faults). Determining which criteria best predict mutation kills is critical to practical estimation of test suite quality. Previous work has only used small sets of programs, and usually compares multiple suites for a single program. Practitioners, however, seldom compare suites --- they evaluate one suite. Using suites (both manual and automatically generated) from a large set of real-world open-source projects shows that evaluation results differ from those for suite-comparison: statement (not block, branch, or path) coverage predicts mutation kills best.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Klees, G., Ruef, A., Cooper, B., Wei, S., & Hicks, M. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243804 · Cited by 633
Mutation Testing Advances: An Analysis and Survey
(2019) · Advances in Computers · doi:10.1016/bs.adcom.2018.03.015 · Cited by 425
Coverage is not strongly correlated with test suite effectiveness
Inozemtseva, L., & Holmes, R. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568271 · Cited by 407
Effective test generation using pre-trained Large Language Models and mutation testing
Dakhel, A. M., Nikanjam, A., Majdinasab, V., Khomh, F., & Desmarais, M. C. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107468 · Cited by 157
On the reliability of coverage-based fuzzer benchmarking
Böhme, M., Szekeres, L., & Metzman, J. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510230 · Cited by 104
Metadata
- Authors
- Gopinath, R., Jensen, C., & Groce, A.
- Venue
- Proceedings of the 36th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2014-05-31
- Pages
- 72-82
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W2063068372
- Cited by
- 171
- References
- 0
- FWCI
- 9.07
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Code coverage for suite evaluation by developers. https://app.cochat.ai/papers/10.1145/2568225.2568278
- The paper
- Gopinath, R., Jensen, C., & Groce, A. (2014). Code coverage for suite evaluation by developers. Proceedings of the 36th International Conference on Software Engineering. https://doi.org/10.1145/2568225.2568278
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.