The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Quantitative analysis of faults and failures in a complex software system
Abstract
The authors describe a number of results from a quantitative study of faults and failures in two releases of a major commercial software system. They tested a range of basic software engineering hypotheses relating to: the Pareto principle of distribution of faults and failures; the use of early fault data to predict later fault and failure data; metrics for fault prediction; and benchmarking fault data. For example, we found strong evidence that a small number of modules contain most of the faults discovered in prerelease testing and that a very small number of modules contain most of the faults discovered in operation. We found no evidence to support previous claims relating module size to fault density nor did we find evidence that popular complexity metrics are good predictors of either fault-prone or failure-prone modules. We confirmed that the number of faults discovered in prerelease testing is an order of magnitude greater than the number discovered in 12 months of operational use. The most important result was strong evidence of a counter-intuitive relationship between pre- and postrelease faults; those modules which are the most fault-prone prerelease are among the least fault-prone postrelease, while conversely, the modules which are most fault-prone postrelease are among the least fault-prone prerelease. This observation has serious ramifications for the commonly used fault density measure. Our results provide data-points in building up an empirical picture of the software development process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
McCabe, T. J. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233837 · Cited by 6,059
Ireson, W. G., & Juran, J. M. (1952) · Journal of the American Statistical Association · doi:10.2307/2280757 · Cited by 2,849
Software Metrics: A Rigorous and Practical Approach
Fenton, N., & Pfleeger, S. L. (2013) · Cited by 2,509
Self-citationA critique of software defect prediction models
Fenton, N. E., & Neil, M. (1999) · IEEE Transactions on Software Engineering · doi:10.1109/32.815326 · Cited by 1,040
Self-citationSoftware Structure Metrics Based on Information Flow
Henry, S., & Kafura, D. (1981) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1981.231113 · Cited by 782
Operational profiles in software-reliability engineering
Musa, J. D. (1993) · IEEE Software · doi:10.1109/52.199724 · Cited by 765
Software errors and complexity: an empirical investigation0
Basili, V. R., & Perricone, B. T. (1984) · Communications of the ACM · doi:10.1145/69605.2085 · Cited by 634
Software measurement: a necessary scientific basis
Fenton, N. (1994) · IEEE Transactions on Software Engineering · doi:10.1109/32.268921 · Cited by 501
Self-citationPractical Software Metrics for Project Management and Process Improvement
Grady, R. B. (1992) · Cited by 442
The detection of fault-prone programs
Munson, J. C., & Khoshgoftaar, T. M. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.135775 · Cited by 442
Cited by
Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
Data Mining Static Code Attributes to Learn Defect Predictors
Menzies, T., Greenwald, J., & Frank, A. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256941 · Cited by 1,215
A Systematic Literature Review on Fault Prediction Performance in Software Engineering
Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.103 · Cited by 1,154
A critique of software defect prediction models
Fenton, N. E., & Neil, M. (1999) · IEEE Transactions on Software Engineering · doi:10.1109/32.815326 · Cited by 1,040
Self-citationUse of relative code churn measures to predict system defect density
Nagappan, N., & Ball, T. (2005) · Proceedings of the 27th international conference on Software engineering - ICSE '05 · doi:10.1145/1062455.1062514 · Cited by 775
“Towards this end we intend that our case study contributes towards strengthening the existing empirical body of knowle dg in this field [7, 9, 13, 15, 17, 19, 20].”
Predicting the location and number of faults in large software systems
Ostrand, T. J., Weyuker, E. J., & Bell, R. M. (2005) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2005.49 · Cited by 720
An empirical study of operating systems errors
Chou, A., Yang, J., Chelf, B., Hallem, S., & Engler, D. (2001) · Proceedings of the eighteenth ACM symposium on Operating systems principles · doi:10.1145/502034.502042 · Cited by 713
Moser, R., Pedrycz, W., & Succi, G. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368114 · Cited by 701
Cross-project defect prediction: a large scale experiment on data vs. domain vs. process
Zimmermann, T., Nagappan, N., Gall, H. C., Giger, E., & Murphy, B. (2009) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-25785 · Cited by 694
An extensive comparison of bug prediction approaches
D'Ambros, M., Lanza, M., & Robbes, R. (2010) · 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010) · doi:10.1109/msr.2010.5463279 · Cited by 637
Evaluating defect prediction approaches: a benchmark and an extensive comparison
D’Ambros, M., Lanza, M., & Robbes, R. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9173-9 · Cited by 626
Predicting defects using network analysis on dependency graphs
Zimmermann, T., & Nagappan, N. (2008) · Proceedings of the 13th international conference on Software engineering - ICSE '08 · doi:10.1145/1368088.1368161 · Cited by 580
Software fault prediction metrics: A systematic literature review
Radjenović, D., Heričko, M., Torkar, R., & Živkovič, A. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.02.009 · Cited by 538
Predicting defect-prone software modules using support vector machines
Elish, K. O., & Elish, M. O. (2008) · Journal of Systems and Software · doi:10.1016/j.jss.2007.07.040 · Cited by 525
Zhou, J., Zhang, H., & Lo, D. (2012) · 2012 34th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2012.6227210 · Cited by 507
The confounding effect of class size on the validity of object-oriented metrics
El Emam, K., Benlarbi, S., Goel, N., & Rai, S. N. (2001) · IEEE Transactions on Software Engineering · doi:10.1109/32.935855 · Cited by 422
Coverage is not strongly correlated with test suite effectiveness
Inozemtseva, L., & Holmes, R. (2014) · Proceedings of the 36th International Conference on Software Engineering · doi:10.1145/2568225.2568271 · Cited by 407
Zhou, J. H., Zhang, H., & Lo, D. (2012) · doi:10.5555/2337223.2337226 · Cited by 372
The prediction of faulty classes using object-oriented design metrics
El Emam, K., Melo, W., & Machado, J. C. (2001) · Journal of Systems and Software · doi:10.1016/s0164-1212(00)00086-8 · Cited by 359
An investigation on the feasibility of cross-project defect prediction
He, Z., Shu, F., Yang, Y., Li, M., & Wang, Q. (2012) · Automated Software Engineering · doi:10.1007/s10515-011-0090-3 · Cited by 333
Fenton, N. E., & Neil, M. (2000) · Proceedings of the Conference on The Future of Software Engineering · doi:10.1145/336512.336588 · Cited by 317
Self-citationAn empirical study on software defect prediction with a simplified metric set
He, P., Li, B., Liu, X., Chen, J., & Ma, Y. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2014.11.006 · Cited by 309
Bennin, K. E., Keung, J., Phannachitta, P., Monden, A., & Mensah, S. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2731766 · Cited by 298
Have things changed now?: an empirical study of bug characteristics in modern open source software
Li, Z., Tan, L., Wang, X., Lu, S., Zhou, Y., & Zhai, C. (2006) · Proceedings of the 1st workshop on Architectural and system support for improving software dependability · doi:10.1145/1181309.1181314 · Cited by 291
Ostrand, T. J., Weyuker, E. J., & Bell, R. M. (2004) · Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis · doi:10.1145/1007512.1007524 · Cited by 269
Log-based anomaly detection with deep learning: how far are we?
Le, V.-H., & Zhang, H. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510155 · Cited by 257
Can developer-module networks predict failures?
Pinzger, M., Nagappan, N., & Murphy, B. (2008) · Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering · doi:10.1145/1453101.1453105 · Cited by 256
An empirical study of software reuse vs. defect-density and stability
Mohagheghi, P., Conradi, R., Killi, O. M., & Schwarz, H. (2004) · Proceedings. 26th International Conference on Software Engineering · doi:10.1109/icse.2004.1317450 · Cited by 84
Social engineering in social networking sites: Affect-based model
Algarni, A., Xu, Y., Chan, T., & Tian, Y.-C. (2013) · 8th International Conference for Internet Technology and Secured Transactions (ICITST-2013) · doi:10.1109/icitst.2013.6750253 · Cited by 41
Landman, D., Serebrenik, A., Bouwers, E., & Vinju, J. J. (2016) · Journal of Software: Evolution and Process · doi:10.1002/smr.1760 · Cited by 29
(2023) · doi:10.1109/ase56229.2023.00157 · Cited by 14
A Case Study of Measuring Degeneration of Software Architectures from a Defect Perspective
Li, Z., & Long, J. (2011) · 2011 18th Asia-Pacific Software Engineering Conference · doi:10.1109/apsec.2011.51 · Cited by 12
Metadata
- Authors
- Fenton, N. E., & Ohlsson, N.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2000
- Volume
- 26
- Issue
- 8
- Pages
- 797-814
- ISSN
- 0098-5589 (ISSN-L)
- OpenAlex
- W2141051748
- Cited by
- 699
- References
- 24
- FWCI
- 57.93
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.