What We Know About Spreadsheet Errors
The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Abstract
Although some spreadsheets are small “scratch pad” applications, many are large and complex, and many missioncritical decisions depend on spreadsheet analyses. In recent years, we have learned a good deal about the errors that people make when they develop spreadsheets. In general, errors seem to occur in a few percent of all cells, meaning that for large spreadsheets, the issue is how many errors there are, not whether an error exists. These error rates, although troubling, are in line with error rates in programming and other human cognitive domains. In programming, we have learned to follow strict development disciplines to eliminate most errors. Surveys of spreadsheet developers indicate that spreadsheet creation, in contrast, is informal, and that few organizations have comprehensive policies for spreadsheet development. Although prescriptive articles have focused on such disciplines as modularization and having assumptions sections, these may be far less important than postdevelopment testing.Request access from your librarian to read this article's full text.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Research shows that errors are surprisingly common in the business setting [4]”
From Gene name errors: Lessons not learned (2021) · cites this paper as [4]
Citations
Cites
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
(2013) · Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship · doi:10.1007/978-1-4614-3858-8_100431 · Cited by 4,799
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
(1998) · PDCA/Test · doi:10.1201/9781420048131.axh · Cited by 1,900
Identifying the organization of writing processes
Haves, J. (1980) · Medical Entomology and Zoology · Cited by 1,553
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Animal Intelligence: An Experimental Study of the Associative Processes in Animals
Kline, L. W., & Thorndike, E. L. (1898) · The American Journal of Psychology · doi:10.2307/1412682 · Cited by 1,092
Thinking, Problem Solving, Cognition
D., C. P., & Mayer, R. E. (1984) · The American Journal of Psychology · doi:10.2307/1422612 · Cited by 992
How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life
Gilovich, T. D. (1991) · Medical Entomology and Zoology · Cited by 990
Edmondson, A. C. (1996) · The Journal of Applied Behavioral Science · doi:10.1177/0021886396321001 · Cited by 937
Cited by
Items 401–500 of 589
Ferson, S. A., Mickley, J., & McGill, W. L. (2011) · Second International Conference on Vulnerability and Risk Analysis and Management (ICVRAM) and the Sixth International Symposium on Uncertainty, Modeling, and Analysis (ISUMA) · doi:10.1061/41170(400)9 · Cited by 1
Oktatáskutatás határon innen és túl
Juhász, E., & Kozma, T. (2014) · REAL-EOD (Library of the Hungarian Academy of Sciences and the Information Center Oriental Collection) · Cited by 1
A Blueprint for Reliable Business Valuation
Mello-e-Souza, C. A. D., & Bee, S. (2006) · Business Valuation Review · doi:10.5791/0882-2875-25.1.4 · Cited by 1
WebSheets: Web Applications for Non-Programmers
Pelizzi, R., & Sekar, R. (2015) · New Security Paradigms Workshop · doi:10.1145/2841113.2841124 · Cited by 1
Williams, J. A. S., Reid, R. C., Gallamore, K., & Rankin, M. (2017) · INFORMS Transactions on Education · doi:10.1287/ited.2017.0179 · Cited by 1
McKay, S. K. (2009) · Cited by 1
Roy, S., van Deursen, A., & Hermans, F. (2018) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c.2018.00046 · Cited by 1
Empirical study on spreadsheet quality: Case of Serbian SMEs
Raković, L., Sakal, M., Matković, P., Vuković, V., & Pavlićević, V. (2019) · Strategic Management · doi:10.5937/straman1902056r · Cited by 1
Expert and Novice End-User Spreadsheet Debugging: A Comparative Study of Performance and Behaviour
Bishop, B., & McDaid, K. (2012) · IGI Global eBooks · doi:10.4018/978-1-4666-2059-9.ch005 · Cited by 1
WYSIWYT testing in the spreadsheet paradigm : an empirical study of end users
Cook, C. R., Stevenson, B., Schonfeld, J., Rothermel, K. J., & Burnett, M. (2000) · Cited by 1
Egy természettudományos projekt hatásának ön- és peerértékelése a középiskolában
Markóczi, I. R., Huszti, A., & Futóné Monori, E. (2013) · University of Debrecen Electronic Archive (University of Debrecen) · Cited by 1
Risks of User-Developed Application In Small Business
Dahalin, Z. (2005) · International Journal of Management Studies · doi:10.32890/ijms2005.12.2.3 · Cited by 1
Managing the operational risks of user-developed software
Pryor, L. (2002) · Cited by 1
Murphy, S. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0801.3853 · Cited by 1
Why Task-Based Training is Superior to Traditional Training MethodsPreprint
McGuire, K. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0802.3480 · Cited by 1
Accuracy in Spreadsheet Modelling SystemsPreprint
Thomas, A. (2003) · arXiv (Cornell University) · Cited by 1
Efficient Methods for Checking Integrity: An Integrated Spreadsheet Engineering Methodology (ISEM)
Rajalingham, K., Chadwick, D. R., Knight, B., & Edwards, D. (2000) · IFIP advances in information and communication technology · doi:10.1007/978-0-387-35501-6_4 · Cited by 1
Interactive fault localization techniques to empower the debugging efforts of end-user programmers
Ruthruff, J. R., & Burnett, M. (2004) · Cited by 1
Wu, N. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.5007 · Cited by 1
Concerning the Feasibility of Example-driven Modelling TechniquesPreprint
Thorne, S., Ball, D., & Lawson, Z. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0802.3477 · Cited by 1
Detecting and refactoring code smells in spreadsheet formula
Hermans, F., Pinzger, M., & van Deursen, A. (2013) · Research Repository (Delft University of Technology) · Cited by 1
Mining Qualitative Behavioral Data from Quantitative Data: A Case Study from the Gender HCI Project.
Beckwith, L., Chintakovid, T., Wiedenbeck, S., & Burnett, M. (2005) · PPIG · Cited by 1
Implementing Stick-Ons for Spreadsheets
Chan, S. S. M., & Pino, J. A. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30112-7_17 · Cited by 1
The Potential of Example Driven Modelling for Decision Support Spreadsheets
Thorne, S., & Ball, D. (2009) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2009.445 · Cited by 1
Mireault, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.06453 · Cited by 1
Detekcija i analiza grešaka u implementaciji dinamičkih diskretnih modela upravljanja zalihama
Đorđević, L. (2016) · Repository of the University of Belgrade Faculty of Organizational Sciences (RFOS) · Cited by 1
ScratchTalk and social computation: towards a natural language scripting model
Eslick, I. S. (2008) · Cited by 1
Proceedings of the European Data and Computational Journalism Conference
Petříček, T., Heravi, B. R., & Stark, J. A. (2017) · Arrow@dit (Dublin Institute of Technology) · Cited by 1
Mitigating Spreadsheet Risk in Complex Multi-Dimensional Models in ExcelPreprint
Litt, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01640 · Cited by 1
Use of Excel Spreadsheet Calculators in Handling Data Generated From UV- Spectrometer
Okezue, M. A., Clase, K. L., & Byrn, S. (2019) · Purdue e-Pubs (Purdue University System) · doi:10.5703/1288284317098 · Cited by 1
Intelligent Voice Navigation of Spreadsheets: An Empirical Evaluation
Flood, D., McDaid, K., McCaffery, F., & Bishop, B. (2008) · STOR (Dundalk Institute of Technology) · Cited by 1
Inclusion AnalysisPreprint
Colver, D. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1001.4104 · Cited by 1
Data Analysis in Excel and R: A Comparative EvaluationPreprint
Magar, R. (2025) · Research Square · doi:10.21203/rs.3.rs-8299990/v1 · Cited by 1
Impact of End User Computing on is Development: A Study of it Usage in an Engineering Organization
Jennex, M. E. (2002) · Springer eBooks · doi:10.1007/978-1-4615-0595-2_20 · Cited by 1
Pause, J. (2017) · mediaTUM (Technical University of Munich) · Cited by 1
Rompiendo barreras en el análisis de datos: R4XCL un puente de integración entre R y Excel
Gómez, M. B. (2026) · Investigative News in Education (Universidad de Costa Rica) · Cited by 1
Erzegovesi, L. (2026) · Institutional Research Information System (Università degli Studi di Trento) · doi:10.5281/zenodo.23064177 · Cited by 1
Erzegovesi, L. (2026) · Iris (University of Trento) · doi:10.5281/zenodo.23078959 · Cited by 1
Supply Chain Forecasting & Planning: Move On from Microsoft Excel?Preprint
Singh, S. K. (2013) · RePEc: Research Papers in Economics · Cited by 1
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_9
Evaluation Methodology and Test Corpora
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_4
Basic Lexical Concepts and Measurements
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_2
Applications to Political Speeches
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_10
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_11
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_3
Reproducible Data Analysis With R in Laboratory Hematology
Obstfeld, A. E. (2026) · International Journal of Laboratory Hematology · doi:10.1111/ijlh.70147
Altman, M., Gill, J., & McDonald, M. P. (2003) · Wiley series in probability and statistics · doi:10.1002/0471475769.biblio
THE SENSITIVITY OF SENSITIVITY: HAZARDS OF SOFTWARE-BASED FINANCIAL DECISION MAKING
Das, A. K., Chua, C. E. H., & Seal, K. C. (2007) · AIS Electronic Library (AISeL)
A Study of Error Detection Methods for Tabular Data with a Small Number of Labelled Instances
Tian, W., & Shao, J. (2026) · Journal of Computer and Communications · doi:10.4236/jcc.2026.147011
Energy and Carbon Management Systems: Organizational Implementation and ApplicationPreprint
Melville, N. P., & Whisnant, R. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2411879
The HERMES Framework for Experimental Reproducibility
Oga, E., Yolandia, J. N. E., & Mughe, H. C. (2026) · Laboratories · doi:10.3390/laboratories3030017
Malaga, R. A., & Zejnullahi, R. (2026) · Journal of Education for Business · doi:10.1080/08832323.2026.2704963
Dewey, K. T. (2011) · RIT Scholar Works (Rochester Institute of Technology)
An Empirical Study of MCL-based Spreadsheet Visualization
Ayalew, Y., & Tshukudu, E. (2016) · International Journal of Software Engineering and Its Applications · doi:10.14257/ijseia.2016.10.3.10
Amoruso, P. E., Hossain, M., & Amoruso, E. L. (2026) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3787109.3816049
Monitoring interface and automated testing for Seaweed, a web-based economic game system
Yuan, J. (2010) · DSpace@MIT (Massachusetts Institute of Technology)
Chabert, C., Collado, A., Cheval, B., & Hüe, O. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.03.12.435109
A Preliminary Fundamental Financial Analysis Framework Using Structured LLM Prompting - A Case Study
Gupta, I., Sharma, N., Kaushal, A., & Rajeswara Rao, K. V. S. (2025) · International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS) · doi:10.1109/csitss67709.2025.11294475
Rapid Exploratory Data Visualization for Cancer Genomics
Regunath, K. (2026) · Methods in molecular biology · doi:10.1007/978-1-0716-4901-5_31
Un environnement dynamique de développement (EDD) pour le prototypage rapide d'interfaces graphiques
Jodoin, A. (2010) · PolyPublie (École Polytechnique de Montréal)
Timeline: Adding the Time Dimension to Spreadsheets
Petříček, T., & Boďa, T. (2026) · Proceedings of the ACM on Programming Languages · doi:10.1145/3839502
(2026) · Pesci Egyetemi Archivum (University of Pécs) · doi:10.15170/xviippn-tk-ktk-2026
Houghton, K. A. (2024) · UEA Digital Repository (University of East Anglia)
A prudenciális adatminőség ontológiája és diagnosztikája a magyar bankrendszerben
Lovász, Z. (2025) · Magyar Minőség · doi:10.65513/mami.2025.10.56
Whose turn is it? Context-guided model routing for stateful agentsPreprint
Parashar, A., Dutta, A., Chauhan, V., Gulwani, S., Gupta, P., Hasanbeig, H., Le, V., Tiwari, A., & Williams, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.19292
Vaz, P. N. B. N. D. S. (2023) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
End-User Programming In Time: Implementation and Empirical Studies
Arredondo-Castro, M. A. (2001)
The role of business intelligence systems in organizations
Egger, J. (2018) · Digital Library of the University of Innsbruck (University of Innsbruck)
Anatomy of a Spreadsheet Failure, Analysing the EuSpRIG Horror Story CorpusPreprint
Thorne, S., & Collins, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.31815
Deficiencies in spreadsheets : a mental model perspective
Kankuzi, B. (2015) · UEF eRepo (University of Eastern Finland)
Lambret, E. (2025) · University Library of Skövde (University of Skövde)
Zeng, D. (2026) · Academic Journal of Applied Sciences · doi:10.54097/3sftyr05
Exploring AI Assistance in Visual Dataflow Programming
Manesh, D., Rabbah, R., & Krosnick, R. (2026) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc71707.2026.11772829
Johannessen, T. V., & Fuglseth, A. M. (2026) · Magma · doi:10.23865/magma.v29.1607
An Adaptive E-Learning System based on Student’s Learning Styles and Knowledge Level
Hariyanto, D. (2019) · Qucosa (Saxon State and University Library Dresden)
Preparation for Strategy Execution
Heesen, B. (2011) · Management for professionals · doi:10.1007/978-3-642-19205-0_3
Gaeta, G. (2012) · Wiley eBooks · doi:10.1002/9781119199090.biblio
Understanding the spreadsheets use in budgeting.
Paulsson, W. (2012) · Lund University Publications (Lund University)
Commercial Credit Intelligence: A Framework for Enterprise Debt Portfolio ManagementPreprint
Toncheff, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7345779
Bustamante, G., Villouta, C., Corrêa, C. M. C., Núñez, E., Mata, R., Sanhueza, F., & Behrens, E. (2026) · Buildings · doi:10.3390/buildings16214440
Ravulapally, S. P. (2026) · International Journal of Innovative Research in Engineering & Multidisciplinary Physical Sciences · doi:10.37082/ijirmps.v14.i1.233026
ALUCINAÇÃO NUMÉRICA NO SOLVER: RUPTURA DE INTEGRIDADE SOB CONFIGURAÇÃO REGIONAL NO MICROSOFT EXCEL
Gonçalves, R. A. (2026) · RECIMA21 - Revista Científica Multidisciplinar · doi:10.47820/recima21.v7i5.7892
Can we enhance performance via learning theory?
Kissinger, C. (2005) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2005.25
Error Estimation in Large Spreadsheets using Bayesian StatisticsPreprint
Bradley, L., & McDaid, K. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0908.1190
A Social Information Foraging Approach to Improving End-User Developers’ Productivity
Jin, X. (2017) · OhioLink ETD Center (Ohio Library and Information Network)
Cellery: Gradual, Composable Structure for Spreadsheets
Yim, J., Peechara, S., Hempel, B., & Coblenz, M. J. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830625
Szociális szakemberképzés a Partiumi Keresztény Egyetemen. Egy képzés története.
Enikő, S., Katalin, T. T., & (szociológus), T. T. K. (.-). (2018) · University of Debrecen Electronic Archive (University of Debrecen)
Pedagógusok folyamatos szakmai fejlődése: Nemzetközi tendenciák, hazai vonatkozások
Zoller, K. (2013) · University of Debrecen Electronic Archive (University of Debrecen)
Data Organization in Spreadsheets
Antony, M., & Pankajakshan, P. M. (2025) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-23215
Towards unifying spreadsheets with databases for ad-hoc interactive data management at scale
Bendre, M. (2018) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Erzegovesi, L. (2026) · Institutional Research Information System (Università degli Studi di Trento) · doi:10.5281/zenodo.23056099
Preventing data loss in PFMEA; a digital solution
singh, M. (2021) · Osuva (University of Vaasa)
(2024) · The MIT Press eBooks · doi:10.7551/mitpress/14872.003.0018
Moraes, A. P. S. (2025) · Lume (Universidade Federal do Rio Grande do Sul)
Implementing flexible software techniques in a 4GL environment
O'Connor, S. (1999) · Research Online (Edith Cowan University)
Managing Multi-Valued Attributes in Spreadsheet Applications.
Churcher, C., McLennan, T., & Spray, W. (2005) · AIS Electronic Library (AISeL)
Automated Support for Model Selection Using Analytic Hierarchy Process
Missakian, M. S. (2011) · NSUWorks (Nova Southeastern University)
Cervesato, I. (2013) · Cognitive technologies · doi:10.1007/978-3-642-37747-1_3
Identifying Structure in Spreadsheets
Abraham, R. (2005) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2004.30
Tait, A., & Richardson, K. A. (2013) · OR Insight · doi:10.1057/ori.2013.8
Metadata
- Authors
- Panko, R. R.
- Affiliations
- University of Hawaiʻi at Mānoa (US)
- Corresponding author
- Panko, Raymond R.
- Venue
- Journal of Organizational and End User Computing
- Publisher
- IGI Global Scientific Publishing
- Published
- 1998-04
- Volume
- 10
- Issue
- 2
- Pages
- 15-21
- ISSN
- 1546-2234 (ISSN-L)
- 1546-5012
- OpenAlex
- W2112338634
- Cited by
- 464
- References
- 64
- Citations by year
- 2026: 12 · 2025: 9 · 2024: 6 · 2023: 7 · 2022: 7 · 2021: 8 · 2020: 17 · 2019: 14 · 2018: 16 · 2017: 27 · 2016: 21 · 2015: 22 · 2014: 17 · 2013: 19 · 2012: 25
- FWCI
- 33.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Organizational and End User Computing (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Spreadsheets and End-User Computing
- Social Sciences › Business, Management and Accounting › Management Information Systems › Decision Support System Applications
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Keywords
- spreadsheet errors, error rate, spreadsheet development, spreadsheet testing, human error, modularization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.