Neural network models for software development effort estimation: a comparative study
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Software development effort estimation (SDEE) is one of the main tasks in software project management. It is crucial for a project manager to efficiently predict the effort or cost of a software project in a bidding process, since overestimation will lead to bidding loss and underestimation will cause the company to lose money. Several SDEE models exist; machine learning models, especially neural network models, are among the most prominent in the field. In this study, four different neural network models—multilayer perceptron, general regression neural network, radial basis function neural network, and cascade correlation neural network—are compared with each other based on: (1) predictive accuracy centred on the mean absolute error criterion, (2) whether such a model tends to overestimate or underestimate, and (3) how each model classifies the importance of its inputs. Industrial datasets from the International Software Benchmarking Standards Group (ISBSG) are used to train and validate the four models. The main ISBSG dataset was filtered and then divided into five datasets based on the productivity value of each project. Results show that the four models tend to overestimate in 80 % of the datasets, and the significance of the model inputs varies based on the selected model. Furthermore, the cascade correlation neural network outperforms the other three models in the majority of the datasets constructed on the mean absolute residual criterion.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“While there are some examples of using ML algorithms for project management (two examples in ML for SE by Nassif et al. (2016) and Pospieszny et al. (2018)), our primary study pool does not include any practices for managing ML systems development projects.”
From A software engineering perspective on engineering machine learning systems: State of the art and challenges (2021) · cites this paper as Nassif et al., 2016
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Although open source, Godot 18 is not yet as mature as its proprietary counterparts [18].”
From Game industry problems: An extensive analysis of the gray literature (2021) · cites this paper as [18]
Experimental — accuracy not yet measuredIn the citing paper
“Game studios are secretive and have a closed-source culture.”
From Game industry problems: An extensive analysis of the gray literature (2021) · cites this paper as [18]
In the citing paper
“GitHub has few popular game-engine projects while its counterparts, software frameworks, are numerous [18].”
From Game industry problems: An extensive analysis of the gray literature (2021) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An introduction to computing with neural nets
Lippmann, R. (1987) · IEEE ASSP Magazine · doi:10.1109/massp.1987.1165576 · Cited by 7,825
Software Engineering Economics
Boehm, B. W. (2001) · Pioneers and Their Contributions to Software Engineering · doi:10.1007/978-3-642-48354-7_5 · Cited by 5,981
An introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775
A general regression neural network
Specht, D. F. (1991) · IEEE Transactions on Neural Networks · doi:10.1109/72.97934 · Cited by 4,492
A scaled conjugate gradient algorithm for fast supervised learning
Møller, M. F. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80056-5 · Cited by 4,052
Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks
Broomhead, D. S., & Lowe, D. (1988) · OpenGrey (Institut de l'Information Scientifique et Technique) · Cited by 3,443
The Cascade-Correlation learning architecture
Fahlman, S. E., & Lebiere, C. J. (2018) · doi:10.1184/r1/6610403 · Cited by 2,372
A General Empirical Solution to the Macro Software Sizing and Estimating Problem
Putnam, L. H. (1978) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1978.231521 · Cited by 800
A systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 742
A Systematic Review of Software Development Cost Estimation Studies
Jorgensen, M., & Shepperd, M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256943 · Cited by 729
Software development cost estimation approaches — A survey
Boehm, B., Abts, C., & Chulani, S. (2000) · Annals of Software Engineering · doi:10.1023/a:1018991717352 · Cited by 588
Systematic literature review of machine learning based software development effort estimation models
Wen, J., Li, S., Lin, Z., Hu, Y., & Huang, C. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2011.09.002 · Cited by 526
Cross versus Within-Company Cost Estimation Studies: A Systematic Review
Kitchenham, B. A., Mendes, E., & Travassos, G. H. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.1001 · Cited by 424
Evaluating prediction systems in software project estimation
Shepperd, M., & MacDonell, S. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2011.12.008 · Cited by 405
Towards an early software estimation using log-linear regression and a multilayer perceptron model
Nassif, A. B., Ho, D., & Capretz, L. F. (2013) · Journal of Systems and Software · doi:10.1016/j.jss.2012.07.050 · Cited by 217
Self-citationForecasting of software development work effort: Evidence on expert judgement and formal models
Jørgensen, M. (2007) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2007.05.008 · Cited by 158
Why comparative effort prediction studies may be invalid
Kitchenham, B., & Mendes, E. (2009) · Proceedings of the 5th International Conference on Predictor Models in Software Engineering · doi:10.1145/1540438.1540444 · Cited by 144
Improving the COCOMO model using a neuro-fuzzy approach
Huang, X., Ho, D., Ren, J., & Capretz, L. F. (2005) · Applied Soft Computing · doi:10.1016/j.asoc.2005.06.007 · Cited by 139
Self-citationAn empirical validation of a neural network model for software effort estimation
Park, H., & Baek, S. H. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.08.001 · Cited by 136
Special issue on repeatable results in software engineering prediction
Menzies, T., & Shepperd, M. (2012) · Empirical Software Engineering · doi:10.1007/s10664-011-9193-5 · Cited by 122
Software Sizing, Estimation, and Risk Management: When Performance is Measured Performance Improves
Galorath, D. D., & Evans, M. W. (2006) · Auerbach Publications eBooks · doi:10.1201/9781420013122 · Cited by 108
COCOMO cost model using fuzzy logic
Idri, A., Abran, A., & Kjiri, L. (2000) · Cited by 107
López‐Martín, C. (2014) · Applied Soft Computing · doi:10.1016/j.asoc.2014.10.033 · Cited by 78
Estimating Software Effort Based on Use Case Point Model Using Sugeno Fuzzy Inference System
Nassif, A. B., Capretz, L. F., & Ho, D. (2011) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2011.64 · Cited by 73
Self-citationEstimating Software Effort Using an ANN Model Based on Use Case Points
Nassif, A. B., Capretz, L. F., & Ho, D. (2012) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2012.138 · Cited by 66
Self-citationNassif, A. B., Capretz, L. F., & Ho, D. (2012) · ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD) · doi:10.1109/snpd.2012.40 · Cited by 61
Self-citationLópez‐Martín, C., Isaza, C., & Chavoya, A. (2011) · Empirical Software Engineering · doi:10.1007/s10664-011-9192-6 · Cited by 57
A comparison of case-based reasoning approaches
Mendes, E., Mosley, N., & Watson, I. D. (2002) · International World Wide Web Conference (WWW) · doi:10.1145/511446.511482 · Cited by 48
López‐Martín, C. (2010) · Applied Soft Computing · doi:10.1016/j.asoc.2009.12.034 · Cited by 43
Nassif, A. B. (2012) · Scholarship@Western (Western University) · Cited by 36
Self-citationOrthogonal Forward Selection for Constructing the Radial Basis Function Network with Tunable Nodes
Chen, S., Hong, X., & Harris, C. J. (2005) · Lecture notes in computer science · doi:10.1007/11538059_81 · Cited by 34
Non-Functional Requirements Size Measurement Method (NFSM) with COSMIC-FFP
Kassab, M., Ormandjieva, O., Daneva, M., & Abran, A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85553-8_14 · Cited by 22
López‐Martín, C. (2010) · Neural Computing and Applications · doi:10.1007/s00521-010-0405-5 · Cited by 22
Non-Functional Requirements: Modeling and Assessment
Kassab, M. (2009) · Medical Entomology and Zoology · Cited by 19
A HYBRID INTELLIGENT MODEL FOR SOFTWARE COST ESTIMATION
Du (2013) · Journal of Computer Science · doi:10.3844/jcssp.2013.1506.1513 · Cited by 15
A comparison of case-based reasoning approaches
Mendes, E., Mosley, N., & Watson, I. D. (2002) · International World Wide Web Conference (WWW) · doi:10.1145/511479.511482 · Cited by 4
Research works printed but not linked: 3
- Nassif AB, Capretz LF, Ho D (2011) A regression model with mamdani fuzzy inference system for early software effort estimation based on use case diagrams. In: Third international conference on intelligent computing and intelligent systems, Guangzhou, Guangdong, pp 615–620
- Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355
- Lopez-Martin C, Isaza C, Chavoya A (2011) Software development effort prediction of industrial projects applying a general regression neural network. Empir Softw Eng 17:1–19
Cited by
Samareh Moosavi, S. H., & Khatibi Bardsiri, V. (2017) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2017.01.006 · Cited by 309
Artificial Intelligence Enabled Project Management: A Systematic Literature Review
Taboada, I., Daneshpajouh, A., Toledo, N., & de Vass, T. (2023) · Applied Sciences · doi:10.3390/app13085014 · Cited by 216
Software Development Effort Estimation Using Regression Fuzzy Models
Nassif, A. B., Azzeh, M., Idri, A., & Abran, A. (2019) · Computational Intelligence and Neuroscience · doi:10.1155/2019/8367214 · Cited by 194
Self-citationGiray, G. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111031 · Cited by 187
Šilhavý, R., Šilhavý, P., & Prokopová, Z. (2016) · Journal of Systems and Software · doi:10.1016/j.jss.2016.11.029 · Cited by 159
Metadata
- Authors
- Nassif, A. B., Azzeh, M., Capretz, L. F., & Ho, D.
- Affiliations
- University of Sharjah (AE)
- Western University (CA)
- Richmond Hospital (CA)
- Corresponding author
- Nassif, Ali Bou
- Venue
- Neural Computing and Applications
- Published
- 2015-11-19
- Volume
- 27
- Issue
- 8
- Pages
- 2369-2381
- ISSN
- 0941-0643 (ISSN-L)
- 1433-3058
- OpenAlex
- W2178751732
- Cited by
- 122
- References
- 36
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 6 · 2023: 11 · 2022: 14 · 2021: 25 · 2020: 19 · 2019: 15 · 2018: 6 · 2017: 10 · 2016: 6
- FWCI
- 19.79
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Neural Computing and Applications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Scholarship@Western (Western University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- University of Sharjah
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Reliability and Analysis Research
- Keywords
- software effort estimation, neural network model, multilayer perceptron, generalized regression neural network, radial basis function neural network, cascade correlation neural network, overestimation and underestimation, software project management, bidding process
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.