The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Software cost estimation with incomplete data
Open accessAbstract
The construction of software cost estimation models remains an active topic of research. The basic premise of cost modeling is that a historical database of software project cost data can be used to develop a quantitative model to predict the cost of future projects. One of the difficulties faced by workers in this area is that many of these historical databases contain substantial amounts of missing data. Thus far, the common practice has been to ignore observations with missing data. In principle, such a practice can lead to gross biases and may be detrimental to the accuracy of cost estimation models. We describe an extensive simulation where we evaluate different techniques for dealing with missing data in the context of software cost modeling. Three techniques are evaluated: listwise deletion, mean imputation, and eight different types of hot-deck imputation. Our results indicate that all the missing data techniques perform well with small biases and high precision. This suggests that the simplest technique, listwise deletion, is a reasonable choice. However, this will not necessarily provide the best performance. Consistent best performance (minimal bias and highest precision) can be obtained by using hot-deck imputation with Euclidean distance and a z-score standardization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Nonparametric Statistics for the Behavioral Sciences
Guitton, H., & Siegel, S. (1958) · Revue économique · doi:10.2307/3498751 · Cited by 35,245
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
A test of missing completely at random for multivariate data with missing values
Little, R. J. A. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478722 · Cited by 8,963
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473
Finding Groups in Data: An Introduction to Cluster Analysis
Kaufman, L., & Rousseeuw, P. J. (1990) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316801 · Cited by 7,344
Regression Diagnostics: Identifying Influential Data and Sources of Collinearity
Belsley, D. A., Kuh, E., & Welsch, R. E. (1980) · Wiley Series in Probability and Statistics · doi:10.1002/0471725153 · Cited by 7,297
Mail and Telephone Surveys: The Total Design Method.
Babbie, E., & Dillman, D. A. (1982) · Contemporary Sociology · doi:10.2307/2068433 · Cited by 6,852
Regression Diagnostics -- Identifying Influential Data and Sources of Collinearity
Beaumont, C. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · The Journal of the Operational Research Society · doi:10.2307/2581267 · Cited by 6,359
Software Engineering Economics
Boehm, B. W. (2001) · Pioneers and Their Contributions to Software Engineering · doi:10.1007/978-3-642-48354-7_5 · Cited by 5,981
Statistical Analysis With Missing Data
Ghosh, S. (1988) · Technometrics · doi:10.1080/00401706.1988.10488446 · Cited by 4,576
Wiley Series in Probability and Statistics
(2008) · Wiley series in probability and statistics · doi:10.1002/series1345 · Cited by 4,217
Brooks, F. P. (1975) · ACM SIGPLAN Notices · doi:10.1145/390016.808439 · Cited by 3,224
The measurement of user information satisfaction
Ives, B., Olson, M. H., & Baroudi, J. J. (1983) · Communications of the ACM · doi:10.1145/358413.358430 · Cited by 1,820
Designing and Conducting Survey Research: A Comprehensive Guide
Rea, L. M., & Parker, R. (1992) · Cited by 1,761
Albrecht, A. J., & Gaffney, J. E. (1983) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1983.235271 · Cited by 1,558
Nonparametric Statistics for the Behavioral Sciences
(2010) · Encyclopedia of Research Design · doi:10.4135/9781412961288.n273 · Cited by 1,414
Measuring application development productivity
Albrecht, A. (1979) · Medical Entomology and Zoology · Cited by 1,258
Missing-Data Adjustments in Large Surveys
Little, R. J. A. (1988) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1988.10509663 · Cited by 1,101
Software engineering metrics and models
Conte, S. D., Dunsmore, H. E., & Shen, V. Y. (1986) · Benjamin-Cummings Publishing Co., Inc. eBooks · Cited by 1,062
Heberlein, T. A., & Baumgartner, R. (1978) · American Sociological Review · doi:10.2307/2094771 · Cited by 999
Mathematics, measurement, and psychophysics.
Stevens, S. S. (1951) · Cited by 975
Regression with Missing X 's: A Review
Little, R. J. A. (1992) · Journal of the American Statistical Association · doi:10.1080/01621459.1992.10476282 · Cited by 968
Estimating software project effort using analogies
Shepperd, M., & Schofield, C. (1997) · IEEE Transactions on Software Engineering · doi:10.1109/32.637387 · Cited by 954
MISSING DATA: A CONCEPTUAL REVIEW FOR APPLIED PSYCHOLOGISTS
ROTH, P. L. (1994) · Personnel Psychology · doi:10.1111/j.1744-6570.1994.tb01736.x · Cited by 901
An empirical validation of software cost estimation models
Kemerer, C. F. (1987) · Communications of the ACM · doi:10.1145/22899.22906 · Cited by 896
A study of standardization of variables in cluster analysis
Milligan, G. W., & Cooper, M. C. (1988) · Journal of Classification · doi:10.1007/bf01897163 · Cited by 877
Software engineering economics: background, current practices, and future directions
Erdogmus, H., Boehm, B. W., Harrison, W., Reifer, D. J., & Sullivan, K. J. (2002) · Proceedings of the 24th international conference on Software engineering - ICSE '02 · doi:10.1145/581441.581444 · Cited by 711
Regressions by Leaps and Bounds
Furnival, G. M., & Wilson, R. W. (1974) · Technometrics · doi:10.1080/00401706.1974.10489231 · Cited by 625
Software engineering metrics and models
Vanamstel, J. J. (1987) · European Journal of Operational Research · doi:10.1016/0377-2217(87)90230-x · Cited by 520
The Assignment of Numbers to Rank Order Categories
Labovitz, S. (2017) · Scaling · doi:10.4324/9781315128948-36 · Cited by 508
Machine learning approaches to estimating software development effort
Srinivasan, K., & Fisher, D. (1995) · IEEE Transactions on Software Engineering · doi:10.1109/32.345828 · Cited by 483
A method of programming measurement and estimation
Walston, C. E., & Felix, C. P. (1977) · IBM Systems Journal · doi:10.1147/sj.161.0054 · Cited by 462
The Mythical Man-Month: Essays on Softw
Brooks, J. F. P. (1978) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 448
The Assignment of Numbers to Rank Order Categories
Labovitz, S. (1970) · American Sociological Review · doi:10.2307/2092993 · Cited by 445
Nominal, Ordinal, Interval, and Ratio Typologies are Misleading
Velleman, P. F., & Wilkinson, L. (1993) · The American Statistician · doi:10.1080/00031305.1993.10475938 · Cited by 385
Some Observations on Measurement and Statistics
Labovitz, S. (1967) · Social Forces · doi:10.2307/2574595 · Cited by 356
Regressions by Leaps and Bounds
Furnival, G. M., & Wilson, R. W. (2000) · Technometrics · doi:10.2307/1271435 · Cited by 355
Mail and Other Self-Administered Questionnaires
Dillman, D. A. (1983) · Handbook of Survey Research · doi:10.1016/b978-0-12-598226-9.50016-1 · Cited by 295
Software development cost estimation using function points
Matson, J. E., Barrett, B. E., & Mellichamp, J. M. (1994) · IEEE Transactions on Software Engineering · doi:10.1109/32.277575 · Cited by 291
Nominal, Ordinal, Interval, and Ratio Typologies are Misleading
Velleman, P., & Wilkinson, L. (1994) · Trends and Perspectives in Empirical Social Research · doi:10.1515/9783110887617.161 · Cited by 290
Finnie, G. R., Wittig, G. E., & Desharnais, J.-M. (1997) · Journal of Systems and Software · doi:10.1016/s0164-1212(97)00055-1 · Cited by 286
The Treatment of Missing Data in Multivariate Analysis
Kim, J.-O., & Curry, J. (1977) · Sociological Methods & Research · doi:10.1177/004912417700600206 · Cited by 285
An assessment and comparison of common software cost estimation modeling techniques
Briand, L. C., El Emam, K., Surmann, D., Wieczorek, I., & Maxwell, K. D. (1999) · Proceedings of the 21st international conference on Software engineering · doi:10.1145/302405.302647 · Cited by 277
Self-citationSoftware effort models for early estimation of process control applications
Mukhopadhyay, T., & Kekre, S. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.163607 · Cited by 272
Robustness in Regression Analysis
Bohrnstedt, G. W., & Carter, T. M. (1971) · Sociological Methodology · doi:10.2307/270820 · Cited by 266
A meta-model for software development resource expenditures
Bailey, J. W., & Basili, V. R. (1981) · doi:10.5555/800078.802522 · Cited by 265
A Model to Evaluate Variables Impacting the Productivity of Software Maintenance Projects
Banker, R. D., Datar, S. M., & Kemerer, C. F. (1991) · Management Science · doi:10.1287/mnsc.37.1.1 · Cited by 243
A pattern recognition approach for software engineering data analysis
Briand, L. C., Basili, V. R., & Thomas, W. M. (1992) · IEEE Transactions on Software Engineering · doi:10.1109/32.177363 · Cited by 239
A replicated assessment and comparison of common software cost modeling techniques
Briand, L. C., Langley, T., & Wieczorek, I. (2000) · Proceedings of the 22nd international conference on Software engineering - ICSE '00 · doi:10.1145/337180.337223 · Cited by 220
Baker, B. O., Hardyck, C. D., & Petrinovich, L. F. (1966) · Educational and Psychological Measurement · doi:10.1177/001316446602600204 · Cited by 216
Scale Economies in New Software Development
Banker, R. D., & Kemerer, C. F. (1989) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1989.559768 · Cited by 210
On the application of measurement theory in software engineering
Briand, L., Emam, K. E., & Morasca, S. (1996) · Empirical Software Engineering · doi:10.1007/bf00125812 · Cited by 199
A comparison of techniques for developing predictive models of software metrics
Gray, A. R., & MacDonell, S. G. (1997) · Information and Software Technology · doi:10.1016/s0950-5849(96)00006-7 · Cited by 188
Missing Data in Regression Analysis
Haitovsky, Y. (1968) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1968.tb01507.x · Cited by 181
Robust regression for developing software estimation models
Miyazaki, Y., Terakado, M., Ozaki, K., & Nozaki, H. (1994) · Journal of Systems and Software · doi:10.1016/0164-1212(94)90110-4 · Cited by 178
A Comparison of Methods for Treating Incomplete Data in Selection Research
Raymond, M. R., & Roberts, D. M. (1987) · Educational and Psychological Measurement · doi:10.1177/0013164487471002 · Cited by 172
A comprehensive evaluation of capture-recapture models for estimating software defect content
Briand, L. C., El Emam, K., Freimut, B. G., & Laitenberger, O. (2000) · IEEE Transactions on Software Engineering · doi:10.1109/32.852741 · Cited by 168
Self-citationMissing Data in Evaluation Research
Raymond, M. R. (1986) · Evaluation & the Health Professions · doi:10.1177/016327878600900401 · Cited by 166
Jeffery, R., Ruhe, M., & Wieczorek, I. (2000) · Information and Software Technology · doi:10.1016/s0950-5849(00)00153-1 · Cited by 163
Gardner, P. L. (1975) · Review of Educational Research · doi:10.3102/00346543045001043 · Cited by 156
Nominal, Ordinal, Interval, and Ratio Typologies Are Misleading
Velleman, P. F., & Wilkinson, L. (1993) · The American Statistician · doi:10.2307/2684788 · Cited by 152
Evidence on economies of scale in software development
Banker, R. D., Chang, H., & Kemerer, C. F. (1994) · Information and Software Technology · doi:10.1016/0950-5849(94)90083-3 · Cited by 135
A Monte Carlo Analysis of Missing Data Techniques in a HRM Setting
Roth, P. L., & Switzer, F. S. (1995) · Journal of Management · doi:10.1177/014920639502100511 · Cited by 115
Analyzing Marketing Research Data with Incomplete Information on the Dependent Variable
Malhotra, N. K. (1987) · Journal of Marketing Research · doi:10.1177/002224378702400107 · Cited by 110
Annotation: what can be done about missing data? Approaches to imputation.
Heitjan, D. F. (1997) · American Journal of Public Health · doi:10.2105/ajph.87.4.548 · Cited by 99
Calibrating the COCOMO II post-architecture model
Clark, B., Chulani, S., & BOEHM, B. W. (1998) · doi:10.5555/302163.302218 · Cited by 84
Some Simple Procedures for Handling Missing Data in Multivariate Analysis
Frane, J. W. (1976) · Psychometrika · doi:10.1007/bf02293565 · Cited by 82
Maxwell, K. D., Wassenhove, L. N. V., & Dutta, S. (1999) · Management Science · doi:10.1287/mnsc.45.6.787 · Cited by 75
Method to estimate parameter values in software prediction models
Miyazaki, Y., Takanou, A., Nozaki, H., et al. (1991) · Information and Software Technology · doi:10.1016/0950-5849(91)90139-3 · Cited by 62
A replicated assessment and comparison of common software cost modeling techniques
Briand, L. C., Langley, T., & Wieczorek, I. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2000.870428 · Cited by 45
Calibrating the COCOMO II Post-Architecture model
Clark, B., Devnani-Chulani, S., & BOEHM, B. W. (2002) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.1998.671610 · Cited by 43
Data Fusion: An Appraisal and Experimental Evaluation
O'Brien, J. P., Harris, P., & Baker, K. (1997) · Market Research Society Journal · doi:10.1177/147078539703900101 · Cited by 39
Ratings of Equal and Unequal Response Choice Intervals
Spector, P. E. (1980) · The Journal of Social Psychology · doi:10.1080/00224545.1980.9924303 · Cited by 29
A two-stage imputation procedure for item nonresponse in surveys
Gilley, O. W., & Leone, R. P. (1991) · Journal of Business Research · doi:10.1016/0148-2963(91)90035-v · Cited by 23
Strategies for the Analysis of Imputed Data From a Sample Survey
Lepkowski, J. M., Landis, J. R., & Stehouwer, S. A. (1987) · Medical Care · doi:10.1097/00005650-198708000-00004 · Cited by 18
Analysis of multivariate polychoric correlation models with incomplete data
Lee, S., & Chiu, Y. (1990) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1990.tb00931.x · Cited by 15
Cited by
A systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 742
A Systematic Review of Software Development Cost Estimation Studies
Jorgensen, M., & Shepperd, M. (2007) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2007.256943 · Cited by 729
Missing value imputation: a review and analysis of the literature (2006–2017)
Lin, W.-C., & Tsai, C.-F. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09709-4 · Cited by 612
A simulation study of the model evaluation criterion mmre
Foss, T., Stensrud, E., Kitchenham, B., & Myrtveit, I. (2003) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2003.1245300 · Cited by 480
ReLink: recovering links between bugs and changes
Wu, R., Zhang, H., Kim, S., & Cheung, S.-C. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025120 · Cited by 412
Metadata
- Authors
- Strike, K., El Emam, K., & Madhavji, N.
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2001
- Volume
- 27
- Issue
- 10
- Pages
- 890-908
- ISSN
- 0098-5589 (ISSN-L)
- OpenAlex
- W2136691316
- Cited by
- 252
- References
- 77
- FWCI
- 19.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.