What We Know About Spreadsheet Errors
The DOI matches its Crossref record; 63 of the 64 references it cites were found in a registry or an index.
Abstract
Although some spreadsheets are small “scratch pad” applications, many are large and complex, and many missioncritical decisions depend on spreadsheet analyses. In recent years, we have learned a good deal about the errors that people make when they develop spreadsheets. In general, errors seem to occur in a few percent of all cells, meaning that for large spreadsheets, the issue is how many errors there are, not whether an error exists. These error rates, although troubling, are in line with error rates in programming and other human cognitive domains. In programming, we have learned to follow strict development disciplines to eliminate most errors. Surveys of spreadsheet developers indicate that spreadsheet creation, in contrast, is informal, and that few organizations have comprehensive policies for spreadsheet development. Although prescriptive articles have focused on such disciplines as modularization and having assumptions sections, these may be far less important than postdevelopment testing.Request access from your librarian to read this article's full text.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Research shows that errors are surprisingly common in the business setting [4]”
From Gene name errors: Lessons not learned (2021) · cites this paper as [4]
Citations
Cites
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
(2013) · Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship · doi:10.1007/978-1-4614-3858-8_100431 · Cited by 4,799
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
(1998) · PDCA/Test · doi:10.1201/9781420048131.axh · Cited by 1,900
Identifying the organization of writing processes
Haves, J. (1980) · Medical Entomology and Zoology · Cited by 1,553
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Animal Intelligence: An Experimental Study of the Associative Processes in Animals
Kline, L. W., & Thorndike, E. L. (1898) · The American Journal of Psychology · doi:10.2307/1412682 · Cited by 1,092
Thinking, Problem Solving, Cognition
D., C. P., & Mayer, R. E. (1984) · The American Journal of Psychology · doi:10.2307/1422612 · Cited by 992
How We Know What Isn't So: The Fallibility of Human Reason in Everyday Life
Gilovich, T. D. (1991) · Medical Entomology and Zoology · Cited by 990
Edmondson, A. C. (1996) · The Journal of Applied Behavioral Science · doi:10.1177/0021886396321001 · Cited by 937
Cited by
Items 201–300 of 589
Introducing a Taxonomy for Classifying Qualitative Spreadsheet Errors
Leon, L. A., Przasnyski, Z. H., & Seal, K. C. (2015) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2015010102 · Cited by 10
Data Exploration through Dot-driven Development
Petříček, T. (2017) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.ecoop.2017.21 · Cited by 10
The development of data collection tool on spreadsheet format
Nurdiantoro, F., Asnar, Y. D. W., & Widagdo, T. E. (2017) · International Conference on Data and Software Engineering · doi:10.1109/icodse.2017.8285889 · Cited by 10
Experiences in using a contingency factor-based validation methodology for spreadsheet DSS
Anastasakis, L., Olphert, C. W., & Wilson, J. (2008) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2602391 · Cited by 10
An Integrated Software Engineering Approach for End-User Programmers
Burnett, M., Rothermel, G., & Cook, C. R. (2006) · Human-computer interaction series · doi:10.1007/1-4020-5386-x_5 · Cited by 10
Influencing the success of spreadsheet development by novice users
Babbitt, T. G., Galletta, D. F., & Lopes, A. B. (1998) · AIS Electronic Library (AISeL) · doi:10.5555/353053.353087 · Cited by 10
Exploring Human Factors in Spreadsheet DevelopmentPreprint
Thorne, S., & Ball, D. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0803.1862 · Cited by 10
Maximizing Correctness with Minimal User Effort to Learn Data Transformations
Wu, B., & Knoblock, C. A. (2016) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2856767.2856791 · Cited by 9
Wallace, S., Papoutsaki, A., Tan, N. H., Guo, H., & Huang, J. J. S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479592 · Cited by 9
Roy, S., Hermans, F., Aivaloglou, E., Winter, J., & van Deursen, A. (2016) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2016.98 · Cited by 9
Data Collection in Zooarchaeology: Incorporating Touch-Screen, Speech-Recognition, Barcodes, and GIS
Dibble, F. (2015) · Ethnobiology Letters · doi:10.14237/ebl.6.2.2015.393 · Cited by 9
Aurigemma, S., & Panko, R. (2014) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2014010103 · Cited by 9
Self-citationProgram visualisation for visual programs
Noble, J., & Biddle, R. (2002) · Australasian User Interface Conference · doi:10.1145/563997.563990 · Cited by 9
McDaid, K., MacRuairi, R., Clynch, N., Logue, K., Clancy, C., & Hayes, S. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.6866 · Cited by 9
An Empirical Study of End-User Behaviour in Spreadsheet Debugging
Bishop, B., & McDaid, K. (2008) · Cited by 8
Too Simple? Notions of Task Complexity used in Maintenance-based Studies of Programming Tools
Rein, P., Beckmann, T., Krebs, E., Mattis, T., & Hirschfeld, R. (2023) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc58990.2023.00040 · Cited by 8
Karlsson, F., & Hedström, K. (2013) · Scandinavian Journal of Information Systems · Cited by 8
PlanAlyzer: assessing threats to the validity of online experiments
Tosch, E., Bakshy, E., Berger, E. D., Jensen, D. D., & Moss, J. E. B. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360608 · Cited by 8
Cunha, J., Fernandes, J. P., Mendes, J., & Saraiva, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-15940-9_6 · Cited by 8
A SYSTEMATIC EVALUATION OF MOBILE SPREADSHEET APPS
Flood, D., Harrison, R., Martin, C. E., & McDaid, K. (2011) · Cited by 8
Thorne, S. (2012) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2012.579 · Cited by 8
An Activity-Based Costing Assessment Task: Using an Excel Spreadsheet
Ringelstein, D. (2009) · Cited by 8
Thorne, S. (2013) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2013070102 · Cited by 8
A Toolkit for Scalable Spreadsheet VisualizationPreprint
Clermont, M. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0802.3924 · Cited by 8
Returning the “socio” to decision support research: Expanding beyond a purely technical mindset
Chua, C. E. H., & Niederman, F. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114352 · Cited by 7
Knowledge Management through End User Developed Expert Systems: Potential and Limitations
Wagner, C. (2003) · Advances in end user computing series/Advances in end user computing (AEUC) book series · doi:10.4018/978-1-59140-065-3.ch008 · Cited by 7
Improvement of Spreadsheet Quality through Reduction of End-User Overconfidence: Case Study
Raković, L., Sakal, M., & Vuković, V. (2019) · Periodica Polytechnica Social and Management Sciences · doi:10.3311/ppso.12392 · Cited by 7
The first workshop on end-user software engineering
Rothermel, G., & Elbaum, S. G. (2005) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1062455.1062623 · Cited by 7
Fehlererkennung in Spreadsheets
Zitzelsberger, S. (2012) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-2804 · Cited by 7
THE DICTIONARY STRUCTURE FOR EFFECTIVE WORD SEARCH
Karwowski, W. (2017) · Information System in Management · doi:10.22630/isim.2017.6.4.3 · Cited by 7
Challenges for Students of Mechanical Engineering Using UML - Typical Questions and Faults
Vogel‐Heuser, B., Land, K., & Bi, F. (2020) · IEEE Congress on Information Science and Technology (CiSt) · doi:10.1109/cist49399.2021.9357186 · Cited by 7
Spreadsheet Risk Management in OrganisationsPreprint
Rittweger, B. G., & Langan, E. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1009.2775 · Cited by 7
Thorne, S., Ball, D., & Lawson, Z. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0803.1754 · Cited by 7
Controlling the Information Flow in SpreadsheetsPreprint
Samar, V., & Patni, S. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0803.2527 · Cited by 7
Using Layout Information for Spreadsheet VisualizationPreprint
Hipfl, S. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0802.3939 · Cited by 7
From Good Practices to Effective Policies for Preventing Errors in SpreadsheetsPreprint
Kulesz, D. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.6878 · Cited by 7
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_6 · Cited by 6
Borthick, A. F., & Schneider, G. P. (2023) · Issues in Accounting Education · doi:10.2308/issues-2021-010 · Cited by 6
Ochs, A. R., Mehrabi, M., Becker, D., Asad, M. N., Zhao, J. J., Zaragoza, M. V., & Grosberg, A. (2019) · Journal of Visualized Experiments · doi:10.3791/60038 · Cited by 6
Visualizing the problem domain for spreadsheet users: A mental model perspective
Kankuzi, B., & Sajaniemi, J. (2014) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2014.6883040 · Cited by 6
Spreadsheets on the Move: An Evaluation of Mobile SpreadsheetsPreprint
Flood, D., Harrison, R., & McDaid, K. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1112.4191 · Cited by 6
Explaining software fault predictions to spreadsheet users
Mukhtar, A., Hofer, B., Jannach, D., & Wotawa, F. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111676 · Cited by 6
Kim, D. J., Locke, S., Chen, T.-H. P., Toma, A., Sporea, S., Weinkam, L., & Sajedi, S. (2023) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip58684.2023.00014 · Cited by 6
Topes: enabling end-user programmers to validate and reformat data
Shaw, M., & Scaffidi, C. P. (2009) · Cited by 6
Dynamische Prüfung von Spreadsheets
Lemcke, M. (2013) · OPUS Publication Server of the University of Stuttgart (University of Stuttgart) · doi:10.18419/opus-3128 · Cited by 6
A domain terms visualization tool for spreadsheets
Kankuzi, B., & Sajaniemi, J. (2014) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2014.6883059 · Cited by 6
Do You Speak, Write and Think in Informatics?
Csernoch, M., & Dani, E. (2022) · Acta Polytechnica Hungarica · doi:10.12700/aph.19.1.2022.19.8 · Cited by 6
Revising the Panko-Halverson Taxonomy of Spreadsheet Risks
Panko, R. R. (2009) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2009.373 · Cited by 6
Self-citationDinmore, M. D. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0908.1192 · Cited by 6
Karlsson, F. (2008) · International Workshop on End-User Software Engineering (WEUSE) · doi:10.1145/1370847.1370857 · Cited by 6
CLOUD COMPUTING AND ITS BENEFITS
Parlińska, M. (2017) · Information System in Management · doi:10.22630/isim.2017.6.4.5 · Cited by 6
AMCIS 2002 Panels and Workshops II: Spreadsheet-Based DSS Curriculum Issues
Ragsdale, C. T., Power, D., & Bergey, P. K. (2002) · Communications of the Association for Information Systems · doi:10.17705/1cais.00921 · Cited by 6
REDUCING SPREADSHEET ERROR RATES
Benham, H., & Giullian, M. (2005) · Issues in Information Systems · doi:10.48009/1_iis_2005_28-34 · Cited by 6
A Literature Review of Spreadsheet Technology
Bock, A. A. (2016) · IT University Of Copenhagen (IT University of Copenhagen) · Cited by 6
Voice-controlled Debugging of SpreadsheetsPreprint
Flood, D., & McDaid, K. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0802.3483 · Cited by 6
Executive Perspectives on Strategic Error Management
Giolito, V. J., & Verdin, P. J. (2018) · Springer eBooks · doi:10.1007/978-3-319-76403-0_4 · Cited by 5
Rothengatter, D. (2012) · doi:10.3990/1.9789036533805 · Cited by 5
Impact of high-intensity negotiated-style interruptions on end-user debugging
Robertson, T. J., Lawrance, J., & Burnett, M. (2005) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2005.09.002 · Cited by 5
Sarkar, A., Xiaotong, Xu, Toronto, N., Drosos, I., & Poelitz, C. (2024) · arXiv · doi:10.48550/arxiv.2412.15030 · Cited by 5
Data Accuracy in Medical Record Abstraction
Nahm, M. (2010) · Digital Commons-TMC (Texas Medical Center) · Cited by 5
Extending the Single Case Analysis and Review Framework ( SCARF ‐ UI ): A review and discussion
Gilroy, S. P., Ledford, J. R., Elliott, T. C., Ayres, K. M., & McGill, F. (2025) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.70033 · Cited by 5
Expert and Novice End-User Spreadsheet Debugging: A Comparative Study of Performance and Behaviour
Bishop, B., & McDaid, K. (2011) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2011040104 · Cited by 5
Spreadsheet Design: An Optimal Checklist For Accountants
Barnes, J. N., Tufte, D. R., & Christensen, D. S. (2009) · American Journal of Business Education (AJBE) · doi:10.19030/ajbe.v2i9.4617 · Cited by 5
Leitão, R., & Roast, C. (2014) · SHURA (Sheffield Hallam University Research Archive) (Sheffield Hallam University) · Cited by 5
Error Trapping and Metamorphic Testing for Spreadsheet Failure Detection
Poon, P., Liu, H., & Chen, T. Y. (2017) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2017040102 · Cited by 5
Test-Driven Development for Spreadsheet Risk Management
McDaid, K., & Rust, A. (2009) · IEEE Software · doi:10.1109/ms.2009.143 · Cited by 5
Implementing Nested FOR Loops as Spreadsheet Formulas
Mireault, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-50230-4_31 · Cited by 5
A user-centered approach for testing spreadsheets
Ayalew, Y. (2007) · Fountain Publishers eBooks · Cited by 5
Integrating Critical Spreadsheet Competencies into the Accounting Curriculum.
Walters, L. M., & Pergola, T. M. (2012) · Journal of instructional pedagogies · Cited by 5
Kritisk lys på utarbeidelse av prognoser ved verdsettelser
Kaldestad, Y., & Møller, B. (2011) · Praktisk økonomi & finans · doi:10.18261/issn1504-2871-2011-02-06 · Cited by 5
Test-Driven Development: Can It Work for Spreadsheet Engineering?
Rust, A., Bishop, B., & McDaid, K. (2006) · Lecture notes in computer science · doi:10.1007/11774129_28 · Cited by 5
Reducing Error in Spreadsheets: Example Driven Modeling Versus Traditional Programming
Thorne, S., Ball, D., & Lawson, Z. (2012) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2012.677744 · Cited by 5
The Production Benefits of Effective Well Integrity Management
Smith, C. J. B., Smith, L., & Spowage, A. C. (2016) · Offshore Technology Conference Asia · doi:10.4043/26698-ms · Cited by 5
The Use of Influence Diagrams in the Design of Spreadsheet Models: an experimental study
O’Donnell, P. (2001) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v9i1.230 · Cited by 5
Visualising 1,051 visual programs module choice and layout in the Nord Modular patch language
Noble, J., & Biddle, R. (2001) · Cited by 5
Reducing Spreadsheet Risk with FormulaDataSleuthPreprint
Bekenn, B., & Hooper, R. (2008) · arXiv (Cornell University) · doi:10.48550/arxiv.0807.2997 · Cited by 5
Teaching methods are erroneous: approaches which lead to erroneous end-user computingPreprint
Csernoch, M., & Biró, P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01130 · Cited by 5
Multidisciplinary Engineering Models: Methodology and Case Study in Spreadsheet AnalyticsPreprint
Birch, D., Liang, H., Kelly, P. H. J., Mullineux, G., Field, T., Ko, J., & Simondetti, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4582 · Cited by 5
Kaparthi, S., & Power, D. (2003) · Elsevier eBooks · doi:10.1016/b0-12-227240-4/00165-9 · Cited by 5
Enron versus EUSES: A Comparison of Two Spreadsheet CorporaPreprint
Jansen, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.04055 · Cited by 5
Elena Ferrante: A Case Study in Authorship Attribution
Savoy, J. (2020) · Springer eBooks · doi:10.1007/978-3-030-53360-1_8 · Cited by 4
A Study of Help Requested Online by Spreadsheet Users
Chambers, C., Sommers, Z., & Scaffidi, C. P. (2012) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.2012100103 · Cited by 4
Auditing spreadsheets: With or without a tool?Preprint
Schalkwijk, S., Hermans, F., van der Ven, M., & Duits, H. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.02261 · Cited by 4
Pedagógusjelöltek szabadidő-eltöltési szokásai és egészségi állapota
Kovács, K. (2015) · University of Debrecen Electronic Archive (University of Debrecen) · Cited by 4
Automatic Data Repair without Format Specifications
Luo, Z., Kirschner, L., Soremekun, E., & Gopinath, R. (2025) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre66568.2025.00049 · Cited by 4
Krawiec, M. (2019) · Studia Linguistica · doi:10.19195/0137-1169.37.4 · Cited by 4
A survey of the use and purpose of spreadsheetsin SMEs in Serbia
Sakal, M., Raković, L., & Vuković, V. (2017) · Ekonomika preduzeca · doi:10.5937/ekopre1704294s · Cited by 4
Spreadsheets and the Violence of Forms: Tracking Organisational and Domestic Use
Milne, E. (2015) · M/C Journal · doi:10.5204/mcj.1023 · Cited by 4
Visualising Formula Structures to Support Exploratory Modelling
Roast, C., Leitão, R., & Gunning, M. (2016) · International Conference on Computer Supported Education · doi:10.5220/0005812303830390 · Cited by 4
SS-BDD: Automated Acceptance Testing for Spreadsheets
Almeida, L., Cirilo, E. J. R., & Adachi, E. (2016) · Brazilian Symposium on Systematic and Automated Software Testing (SAST) · doi:10.1145/2993288.2993296 · Cited by 4
Tools and Technologies for the Implementation of Big Data
Self, R. J., & Voorhis, D. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-801967-2.00010-0 · Cited by 4
A Pragmatic Approach to Spreadsheet Training Based Upon the “Projection–Screen” Model
Hodnigg, K. (2005) · Lecture notes in computer science · doi:10.1007/978-3-540-31958-0_15 · Cited by 4
A visualization-based approach for improving spreadsheet quality
Ayalew, Y. (2009) · doi:10.1145/1527033.1527038 · Cited by 4
A Spreadsheet for Everyday Symbolic Reasoning
Cervesato, I. (2005) · KiltHub Repository · doi:10.1184/r1/6591392 · Cited by 4
Design implications for end-user debugging tools : a strategy-based view
Grigoreanu, V., Burnett, M., & Robertson, G. G. (2009) · Cited by 4
Unobtrusive data acquisition for spreadsheet research
Bishop, B., & McDaid, K. (2008) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2008.4639074 · Cited by 4
Towards Requirements Engineering Practice for Professional End User Developers: A Case Study
Faily, S. (2008) · doi:10.1109/reet.2008.8 · Cited by 4
End-user assertions : propagating their implications
Summet, J. W. (2001) · Cited by 4
Tool support for data validation by end-user programmers
Scaffidi, C. P., Myers, B. A., & Shaw, M. (2008) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1368088.1368226 · Cited by 4
Konzeption des persönlichen InformationsmanagementsPreprint
Klotz, M. (2011) · RePEc: Research Papers in Economics · Cited by 4
Metadata
- Authors
- Panko, R. R.
- Affiliations
- University of Hawaiʻi at Mānoa (US)
- Corresponding author
- Panko, Raymond R.
- Venue
- Journal of Organizational and End User Computing
- Publisher
- IGI Global Scientific Publishing
- Published
- 1998-04
- Volume
- 10
- Issue
- 2
- Pages
- 15-21
- ISSN
- 1546-2234 (ISSN-L)
- 1546-5012
- OpenAlex
- W2112338634
- Cited by
- 464
- References
- 64
- Citations by year
- 2026: 12 · 2025: 9 · 2024: 6 · 2023: 7 · 2022: 7 · 2021: 8 · 2020: 17 · 2019: 14 · 2018: 16 · 2017: 27 · 2016: 21 · 2015: 22 · 2014: 17 · 2013: 19 · 2012: 25
- FWCI
- 33.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Organizational and End User Computing (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Spreadsheets and End-User Computing
- Social Sciences › Business, Management and Accounting › Management Information Systems › Decision Support System Applications
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Keywords
- spreadsheet errors, error rate, spreadsheet development, spreadsheet testing, human error, modularization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.