The DOI matches its Crossref record; 57 of the 60 references it cites were found in a registry or an index.
Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests
Abstract
W e investigated the way experienced users interpret Null Hypothesis Significance Testing (NHST) outcomes. An empirical study was designed to compare the reactions of two populations of NHST users, psychological researchers and professional applied statisticians, when faced with contradictory situations. The subjects were presented with the results of an experiment designed to test the efficacy of a drug by comparing two groups (treatment/placebo). Four situations were constructed by combining the outcome of the t test (significant vs. nonsignificant) and the observed difference between the two means D (large vs. small). Two of these situations appeared as conflicting (t significant/D small and t nonsignificant/D large). Three fundamental aspects of statistical inference were investigated by means of open questions: drawing inductive conclusions about the magnitude of the true difference from the data in hand, making predictions for future data, and making decisions about stopping the experiment. The subjects were 25 statisticians from pharmaceutical companies in France, subjects well versed in statistics, and 20 psychological researchers from various laboratories in France, all with experience in processing and analyzing experimental data. On the whole, statisticians and psychologists reacted in a similar way and were very impressed by significant results. It must be outlined that professional applied statisticians were not immune to misinterpretations, especially in the case of nonsignificance. However, the interpretations that accustomed users attach to the outcome of NHST can vary from one individual to another, and it is hard to conceive that there could be a consensus in the face of seemingly conflicting situations. In fact, beyond the superficial report of “erroneous” interpretations, it can be seen in the misuses of NHST intuitive judgmental “adjustments” that try to overcome its inherent shortcomings. These findings encourage the many recent attempts to improve the habitual ways of analyzing and reporting experimental data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This data is consistent with previous studies (Haller and Krauss, 2002; Lecoutre et al., 2003; Monterde-i-Bort et al., 2010) and with the original research from Spain (Badenes-Ribera et al., 2015a).”
From Misconceptions of the p-value among Chilean and Italian Academic Psychologists (2016) · cites this paper as Lecoutre et al., 2003
Citations
Cites
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Judgment under Uncertainty: Heuristics and Biases
(1982) · doi:10.1017/cbo9780511809477 · Cited by 8,240
Judgment Under Uncertainty: Heuristics and Biases.
Shafer, G. R., Kahnerman, D., Slovic, P., & Tversky, A. (1984) · Journal of the American Statistical Association · doi:10.2307/2288362 · Cited by 5,657
Subjective probability: A judgment of representativeness
Kahneman, D., & Tversky, A. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90016-3 · Cited by 4,851
Judgment under uncertainty: List of contributors
Kahneman, D. J., Slovic, P., & Tversky, A. (1982) · Cited by 4,670
Judgment under uncertainty: Causality and attribution
Kahneman, D. J., Slovic, P., & Tversky, A. (1982) · Cited by 4,670
Publication Manual of the American Psychological Association
DeCleene, K. E., & Fogo, J. (2012) · Occupational Therapy In Health Care · doi:10.3109/07380577.2011.629024 · Cited by 3,167
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Using confidence intervals in within-subject designs
Loftus, G. R., & Masson, M. E. J. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03210951 · Cited by 2,448
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,185
Research works printed but not linked: 9
- Cox D. R., 1977, The role of significance tests (With discussion), Scandinavian Journal of Statistics, 4, 49
- 10.1037/1082-989X.5.4.411 · doi:10.1037/1082-989x.5.4.411 · Paper page
- Lecoutre B., 1985, How to derive Bayes‐fiducial conclusions from usual significance tests, Cahiers de Psychologie Cognitive, 5, 553
- Lecoutre B., 2001, Bayesian predictive procedures for designing and monitoring experiments. In Bayesian Methods with Applications to Science, Policy and Official Statistics, 301
- Lecoutre B., 2001, Bayesian methods with applications to science, policy and official statistics, 311
- Lecoutre M.‐P., 1985, La démarche du chercheur en psychologie dans des situations d'analyse statistique de données expérimentales, Journal de Psychologie Normale et Pathologique, 3, 275
- 10.1037/0033-2909.113.3.553 · doi:10.1037/0033-2909.113.3.553 · Paper page
- Rouanet H., 2000, New ways in statistical methodology: From significance tests to Bayesian inference
- Thompson B., 1994, Guidelines for authors, Educational and Psychological Measurement, 54, 837
Cited by
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890
“As I mentioned before, in contrast, I believe that some degree of illusion is necessary to keep the null ritual alive, and the empirical evidence supports this conjecture (e.g., Lecoutre et al., 2003; Tversky and Kahneman, 1971).”
BackgroundRationality for mortals: how people cope with uncertainty
(2009) · Choice Reviews Online · doi:10.5860/choice.46-4133 · Cited by 469
Robust misinterpretation of confidence intervals
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0572-3 · Cited by 425
“Falk and Greenbaum (1995) found similar results in a replication of Oakes’s study, and Haller and Krauss (2002) showed that even professors and lecturers teaching statistics often endorse false statements about the results from NHST. Lecoutre, Poitevineau, and Lecoutre (2003) found the same for statisticians working f…”
ResultInfluentialKühberger, A., Fritz, A., & Scherndl, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105825 · Cited by 399
“[11] showed that researchers collect more data if the results are nearly, but not quite, statistically significant, and John et al.”
Background
Metadata
- Authors
- Lecoutre, M., Poitevineau, J., & Lecoutre, B.
- Affiliations
- Université de Rouen Normandie (FR)
- Centre National de la Recherche Scientifique (FR)
- Corresponding author
- Lecoutre, Marie‐Paule
- Venue
- International Journal of Psychology
- Publisher
- Wiley
- Published
- 2003-02
- Volume
- 38
- Issue
- 1
- Pages
- 37-45
- ISSN
- 0020-7594 (ISSN-L)
- 1464-066X
- OpenAlex
- W2021666929
- Cited by
- 86
- References
- 60
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 3 · 2023: 3 · 2022: 3 · 2021: 5 · 2020: 5 · 2019: 1 · 2018: 7 · 2017: 8 · 2016: 8 · 2015: 3 · 2014: 5 · 2013: 1 · 2012: 3
- FWCI
- 2.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Psychology (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- null hypothesis significance testing, misinterpretation of statistical results, statistical inference, non-significant results, effect size, inductive reasoning, prediction, stopping decisions, drug efficacy trials
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests. https://app.cochat.ai/papers/10.1080/00207590244000250
- The paper
- Lecoutre, M., Poitevineau, J., & Lecoutre, B. (2003). Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests. International Journal of Psychology. https://doi.org/10.1080/00207590244000250
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.