The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities
Abstract
Security inspection and testing require experts in security who think like an attacker. Security experts need to know code locations on which to focus their testing and inspection efforts. Since vulnerabilities are rare occurrences, locating vulnerable code locations can be a challenging task. We investigated whether software metrics obtained from source code and development history are discriminative and predictive of vulnerable code locations. If so, security experts can use this prediction to prioritize security inspection and testing efforts. The metrics we investigated fall into three categories: complexity, code churn, and developer activity metrics. We performed two empirical case studies on large, widely used open-source projects: the Mozilla Firefox web browser and the Red Hat Enterprise Linux kernel. The results indicate that 24 of the 28 metrics collected are discriminative of vulnerabilities for both projects. The models using all three types of metrics together predicted over 80 percent of the known vulnerable files with less than 25 percent false positives for both projects. Compared to a random selection of files for inspection and testing, these models would have reduced the number of files and the number of lines of code to inspect or test by over 71 and 28 percent, respectively, for both projects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Data Mining: Practical Machine Learning Tools and Techniques
(2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,437
Community structure in social and biological networks
Girvan, M., & Newman, M. E. J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.122653799 · Cited by 15,502 · cited 1×
“To collect complexity metrics, we used Understand C++ 8 , a commercial metrics collection tool.”
MethodOn a Test of Whether one of Two Random Variables is Stochastically Larger than the Other
Mann, H. B., & Whitney, D. R. (1947) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730491 · Cited by 14,320
Applied Multivariate Statistical Analysis.
Johnson, A. R., & Wichern, D. W. (1988) · Biometrics · doi:10.2307/2531616 · Cited by 11,353
Witten, I. H. (2008) · Cited by 9,143
An Introduction to Variable and Feature Selection.
Guyon, I., & Elisseeff, A. (2003) · Cited by 6,576
McCabe, T. J. (1976) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1976.233837 · Cited by 6,059
(2024) · Physics Subject Headings (PhySH) · doi:10.29172/7c2a6982-6d72-4cd8-bba6-2fccb06a7011 · Cited by 3,857
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322753616 · Cited by 3,502
Cited by
VulDeePecker: A Deep Learning-Based System for Vulnerability Detection
Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., & Zhong, Y. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23158 · Cited by 898
SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities
Li, Z., Zou, D., Xu, S., Jin, H., Zhu, Y., & Chen, Z. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3051525 · Cited by 639
Software Vulnerability Detection Using Deep Neural Networks: A Survey
Lin, G., Wen, S., Han, Q.-L., Zhang, J., & Xiang, Y. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.2993293 · Cited by 457
Nam, J., Pan, S. J., & Kim, S. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606584 · Cited by 449
Ghaffarian, S. M., & Shahriari, H. R. (2018) · ACM Computing Surveys · doi:10.1145/3092566 · Cited by 353
Predicting Vulnerable Software Components via Text Mining
Scandariato, R., Walden, J., Hovsepyan, A., & Joosen, W. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2340398 · Cited by 337
DeepWukong: Statically Detecting Software Vulnerabilities Using Deep Graph Neural Network
Cheng, X., Wang, H., Hua, J., Xu, G., & Sui, Y. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3436877 · Cited by 335
An empirical study on software defect prediction with a simplified metric set
He, P., Li, B., Liu, X., Chen, J., & Ma, Y. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2014.11.006 · Cited by 309
A Comprehensive Investigation of the Role of Imbalanced Learning for Software Defect Prediction
Song, Q., Guo, Y., & Shepperd, M. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2836442 · Cited by 295
Respite for SMEs: A Systematic Review of Socio-Technical Cybersecurity Metrics
van Haastrecht, M., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M. (2021) · Applied Sciences · doi:10.3390/app11156909 · Cited by 45
“In papers covering the defender, it is quite common to ignore threats entirely [78] or to use a proxy such as the prevalence of vulnerabilities to represent threats [79].”
BackgroundNo Single Metric Captures Productivity
Jaspan, C., & Sadowski, C. (2019) · Rethinking Productivity in Software Engineering · doi:10.1007/978-1-4842-4221-6_2 · Cited by 20
The role of firm AI capabilities in generative AI-pair coding
Bughin, J. (2024) · Journal of Decision Systems · doi:10.1080/12460125.2024.2428187 · Cited by 6
Metadata
- Authors
- Shin, Y., Meneely, A., Williams, L., & Osborne, J. A.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2011-11
- Volume
- 37
- Issue
- 6
- Pages
- 772-787
- ISSN
- 0098-5589 (ISSN-L)
- OpenAlex
- W2043837581
- Cited by
- 607
- References
- 55
- FWCI
- 35.24
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.