Statistical heartburn: an attempt to digest four pizza publications from the Cornell Food and Brand Lab
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
We present the results of a reanalysis of four articles from the Cornell Food and Brand Lab based on data collected from diners at an Italian restaurant buffet. We calculated whether the means, standard deviations, and test statistics were compatible with the sample size. Test statistics and p values were recalculated. We also applied deductive logic to see whether the claims made in each article were compatible with the claims made in the others. We have so far been unable to obtain the data from the authors of the four articles. A thorough reading of the articles and careful reanalysis of the results revealed a wide range of problems. The sample sizes for the number of diners in each condition are incongruous both within and between the four articles. In some cases, the degrees of freedom of between-participant test statistics are larger than the sample size, which is impossible. Many of the computed F and t statistics are inconsistent with the reported means and standard deviations. In some cases, the number of possible inconsistencies for a single statistic was such that we were unable to determine which of the components of that statistic were incorrect. Our Appendix reports approximately 150 inconsistencies in these four articles, which we were able to identify from the reported statistics alone. We hope that our analysis will encourage readers, using and extending the simple methods that we describe, to undertake their own efforts to verify published results, and that such initiatives will improve the accuracy and reproducibility of the scientific literature. We also anticipate that the editors of the journals that published these four articles may wish to consider whether any corrective action is required.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239
The rules of the game called psychological science
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612459060 · Cited by 800
The prevalence of statistical reporting errors in psychology (1985–2013)
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0664-2 · Cited by 510
The (mis)reporting of statistical results in psychology journals
Bakker, M., & Wicherts, J. M. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0089-5 · Cited by 356 · cited 1×
“The prevalence of errors in the p values associated with reported test statistics has been estimated to be anywhere from around 18% to almost 50% [7, 8].”
BackgroundBrown, N. J. L., & Heathers, J. A. J. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550616673876 · Cited by 127
Self-citationThe GRIMMER test: A method for testing the validity of reported measures of variability
Anaya, J. (2016) · PeerJ Preprints · doi:10.7287/peerj.preprints.2400v1 · Cited by 31 · cited 17×
“A further discrepancy in the reported sample sizes across these articles is apparent when we examine Article 1, although Table 3 of this article provides neither sample sizes nor SDs to go with the reported means and F statistics.”
BackgroundMethodInfluentialSelf-citation
Cited by
Methods to assess research misconduct in health-related research: A scoping review
Bordewijk, E. M., Li, W., van Eekelen, R., Wang, R., Showell, M., Mol, B. W., & van Wely, M. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.012 · Cited by 68
Publish and Who Should Perish: You or Science?
Kun, Á. (2018) · Publications · doi:10.3390/publications6020018 · Cited by 44
Heathers, J. A., Anaya, J., van der Zee, T., & Brown, N. J. (2018) · PeerJ Preprints · doi:10.7287/peerj.preprints.26968v1 · Cited by 50
“Recently, both tests were central to the identification of problems in a series of papers in food psychology (van der Zee et al., 2017; Anaya et al., 2017).”
BackgroundResultSelf-citation
Metadata
- Authors
- van der Zee, T., Anaya, J., & Brown, N. J. L.
- Venue
- BMC Nutrition
- Published
- 2017-07-10
- Volume
- 3
- Issue
- 1
- Pages
- 54-54
- ISSN
- 2055-0928 (ISSN-L)
- OpenAlex
- W2584315339
- Cited by
- 61
- References
- 7
- FWCI
- 5.78
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Statistical heartburn: an attempt to digest four pizza publications from the Cornell Food and Brand Lab. https://app.cochat.ai/papers/10.1186/s40795-017-0167-x
- The paper
- van der Zee, T., Anaya, J., & Brown, N. J. L. (2017). Statistical heartburn: an attempt to digest four pizza publications from the Cornell Food and Brand Lab. BMC Nutrition. https://doi.org/10.1186/s40795-017-0167-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.