Application of Benford’s law: a valuable tool for detecting scientific papers with fabricated data?: A case study using proven falsified articles against a comparison group
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Background In naturally occurring numbers the frequencies of digits 1-9 in the leading position are counterintuitively distributed because the frequencies of occurrence are unequal. Benford-Newcomb's law describes the expected distribution of these frequencies. It was previously shown that known fraudulent articles consistently violated this law.Objective To compare the features of 12 known fraudulent articles from a single Japanese author to the features of 13 articles in the same research field from other Japanese authors, published during the same time period and identified with a Medline database search.Results All 25 articles were assessed to determine whether the data violated the law. Formulas provided by the law were used to determine the frequencies of occurrence for the first two leading digits in manually extracted numbers. It was found that all the known fraudulent papers violated the law and 6 of the 13 articles used for comparison followed the law. Assuming that the articles in the comparison group were not falsified or fabricated, the sensitivity of assessing articles with Benford-Newcomb's law was 100% (95% confidence interval CI: 73.54-100%) but the specificity was only 46.15% (95% CI: 19.22-74.87%) and the positive predictive value was 63.16% (95% CI: 38.36-83.71%).Conclusion All 12 of the known falsified articles violated Benford-Newcomb's law, which indicated that this analysis had a high sensitivity. The low specificity of the assessment may be explained by the assumptions made about the articles identified for comparison. Violations of Benford-Newcomb's law about the frequencies of the leading digits cannot serve as proof of falsification but they may provide a basis for deeper discussions between the editor and author about a submitted work.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although sharing of individual participant data has been encouraged for several years, 16 data sharing is still uncommon for published RCTs.17”
From Assessing Research Misconduct in Randomized Controlled Trials (2021) · cites this paper as [17]
Citations
Cites
Statistical Fraud Detection: A Review
Bolton, R. J., & Hand, D. J. (2002) · Statistical Science · doi:10.1214/ss/1042727940 · Cited by 1,450
A Statistical Derivation of the Significant-Digit Law
Hill, T. P. (1995) · Statistical Science · doi:10.1214/ss/1177009869 · Cited by 670
The effective use of Benford's Law to assist in detecting fraud in accounting data
Durtschi, C., Hillison, W., & Pacini, C. (2004) · Digital Commons - University of South Florida (University of South Florida) · Cited by 466
The analysis of 168 randomised controlled trials to test data integrity
Carlisle, J. B. (2012) · Anaesthesia · doi:10.1111/j.1365-2044.2012.07128.x · Cited by 255 · cited 5×
“In 2012, John Carlisle demonstrated conclusively using statistics that this author had faked data in 168 investigated studies.”
BackgroundMethodInfluentialThe Significant-Digit Phenomenon
Hill, T. P. (1995) · The American Mathematical Monthly · doi:10.1080/00029890.1995.11990578 · Cited by 183
Kranke, P., Apfel, C. C., & Roewer, N. (2000) · Anesthesia & Analgesia · doi:10.1213/00000539-200004000-00053 · Cited by 156
Fujii, Y. (2000) · Anesthesia & Analgesia · doi:10.1213/00000539-200004000-00054 · Cited by 107
The Significant-Digit Phenomenon
Hill, T. P. (1995) · American Mathematical Monthly · doi:10.2307/2974952 · Cited by 90
Carlisle, J. B., Dexter, F., Pandit, J. J., Shafer, S. L., & Yentis, S. M. (2015) · Anaesthesia · doi:10.1111/anae.13126 · Cited by 87 · cited 3×
“In future, itmaybeof value to assess a single submitted article for fraud by applying the combination of Benford-Newcomb’s law and aMontoCarlo simulation of data [2, 23, 24].”
MethodEvidence for non‐random sampling in randomised, controlled trials by Yuhji Saitoh
Carlisle, J. B., & Loadsman, J. A. (2017) · Anaesthesia · doi:10.1111/anae.13650 · Cited by 64
Research works printed but not linked: 4
- Benford F (1938) The law of anomalous numbers. Proc Am Philos Soc 78(4):551–572
- Kossovsky AE (2014) Benford‘s law – forensic digital analysis & fraud detection. http://www.forensicbenford.com/macros.html . Zugegriffen: 25.10.2016
- Nigrini MJ (1996) A taxpayer compliance application of Benford’s law. J Am Tax Assoc 18:72–91
- Nigrini MJ (2000) Digital analysis using Benford’s law, 2nd edn., Global Audit Publications, Vancouver
Cited by
Data integrity of 35 randomised controlled trials in women' health
Bordewijk, E. M., Wang, R., Askie, L. M., Gurrin, L. C., Thornton, J. G., van Wely, M., Li, W., & Mol, B. W. (2020) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2020.04.016 · Cited by 66
Methods to assess research misconduct in health-related research: A scoping review
Bordewijk, E. M., Li, W., van Eekelen, R., Wang, R., Showell, M., Mol, B. W., & van Wely, M. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.012 · Cited by 68
Moore, A., Fisher, E., & Eccleston, C. (2022) · British Journal of Anaesthesia · doi:10.1016/j.bja.2022.09.030 · Cited by 57
Assessing Research Misconduct in Randomized Controlled Trials
Li, W., Bordewijk, E. M., & Mol, B. W. (2021) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000004513 · Cited by 37
Detection of data fabrication using statistical tools
Hartgerink, C. H. J., Voelkel, J. G., Wicherts, J. M., & van Assen, M. A. L. M. (2019) · doi:10.31234/osf.io/jkws4 · Cited by 16
Metadata
- Authors
- Hüllemann, S., Schüpfer, G., & Mauch, J.
- Affiliations
- Luzerner Kantonsspital (CH)
- Corresponding author
- Schüpfer, G.
- Venue
- Der Anaesthesist
- Published
- 2017-06-26
- Volume
- 66
- Issue
- 10
- Pages
- 795-802
- ISSN
- 0003-2417 (ISSN-L)
- 1432-055X
- OpenAlex
- W2739044138
- Cited by
- 25
- References
- 34
- Citations by year
- 2025: 5 · 2024: 2 · 2023: 4 · 2022: 2 · 2021: 5 · 2020: 3 · 2019: 2 · 2018: 2
- FWCI
- 0.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Der Anaesthesist (publishedVersion)
- PubMed (publishedVersion)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- Benford's law, research misconduct, leading digit distribution, scientific fraud, sensitivity and specificity, peer review
- MeSH terms
- Algorithms, Postoperative Pain, Peer Review, Propofol, Scientific Misconduct, Databases, Factual, MEDLINE, Anesthetics, Intravenous, Professional Misconduct
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Application of Benford’s law: a valuable tool for detecting scientific papers with fabricated data?: A case study using proven falsified articles against a comparison group. https://app.cochat.ai/papers/10.1007/s00101-017-0333-1
- The paper
- Hüllemann, S., Schüpfer, G., & Mauch, J. (2017). Application of Benford’s law: a valuable tool for detecting scientific papers with fabricated data?: A case study using proven falsified articles against a comparison group. Der Anaesthesist. https://doi.org/10.1007/s00101-017-0333-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.