Must psychologists change the way they analyze their data?

Bem, Daryl J.; Utts, Jessica; Johnson, Wesley O.

doi:10.1037/a0024777

Cited by 106Statistical Methods and Bayesian InferenceMeta-analysis and systematic reviewsParanormal Experiences and Beliefs

Must psychologists change the way they analyze their data?

Bem, Daryl J., Utts, Jessica, Johnson, Wesley O.·2011·Journal of Personality and Social Psychology

DOI 10.1037/a0024777

Open this paper in CoChat — free account

Abstract

Wagenmakers, Wetzels, Borsboom, and van der Maas (2011) argued that psychologists should replace the familiar "frequentist" statistical analyses of their data with bayesian analyses. To illustrate their argument, they reanalyzed a set of psi experiments published recently in this journal by Bem (2011), maintaining that, contrary to his conclusion, his data do not yield evidence in favor of the psi hypothesis. We argue that they have incorrectly selected an unrealistic prior distribution for their analysis and that a bayesian analysis using a more reasonable distribution yields strong evidence in favor of the psi hypothesis. More generally, we argue that there are advantages to bayesian analyses that merit their increased use in the future. However, as Wagenmakers et al.'s analysis inadvertently revealed, they contain hidden traps that must be better understood before being more widely substituted for the familiar frequentist analyses currently employed by most research psychologists.

Integrity signals

Registry

Crossref · 10.1037/a0024777

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Effect sizes in psychology typically fall in the range of 0.2 to 0.3.
“Bem et al. (2011) and Richard et al. (2003) stated that effect sizes in psychology typically fall in the range of 0.2 to 0.3.”
From Using the Student's t-test with extremely small sample sizes (2013) · cites this paper as Bem et al. (2011)
Large effect sizes are uncommon in the behavioral/psychological sciences.
“However, large effect sizes are uncommon in the behavioral/psychological sciences.”
From Using the Student's t-test with extremely small sample sizes (2013) · cites this paper as Bem et al. (2011)

Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,385 · see citing works
The Theory of Probability
Jeffreys, H., Moran, P. A. P., & Chatfield, C. (1922) · Nature · doi:10.1038/109132a0 · Cited by 7,837 · see citing works
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. J. (2000) · Statistics and Computing · doi:10.1023/a:1008929526011 · Cited by 5,372 · see citing works
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688 · see citing works
Exposure and affect: Overview and meta-analysis of research, 1968–1987.
Bornstein, R. F. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.2.265 · Cited by 1,970 · see citing works
One Hundred Years of Social Psychology Quantitatively Described
Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003) · Review of General Psychology · doi:10.1037/1089-2680.7.4.331 · Cited by 1,558 · see citing works
Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011).
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0022790 · Cited by 977 · see citing works
Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect.
Bem, D. J. (2011) · Journal of Personality and Social Psychology · doi:10.1037/a0021524 · Cited by 893 · see citing works
Self-citation
Why Isn't Everyone a Bayesian?
Efron, B. (1986) · The American Statistician · doi:10.1080/00031305.1986.10475342 · Cited by 379 · see citing works
Bayesian Ideas and Data Analysis
Christensen, R., Johnson, W. O., Branscum, A. J., & Hanson, T. (2010) · CRC Press eBooks · doi:10.1201/9781439894798 · Cited by 294 · see citing works
Self-citation
A Bayes factor meta-analysis of Bem’s ESP claim
Rouder, J. N., & Morey, R. D. (2011) · Psychonomic Bulletin & Review · doi:10.3758/s13423-011-0088-7 · Cited by 194 · see citing works
Predictive Physiological Anticipation Preceding Seemingly Unpredictable Stimuli: A Meta-Analysis
Mossbridge, J., Tressoldi, P. E., & Utts, J. M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00390 · Cited by 148 · see citing works
Self-citation
The Bayesian Two-SampletTest
Gönen, M., Johnson, W. O., Lu, Y., & Westfall, P. H. (2005) · The American Statistician · doi:10.1198/000313005x55233 · Cited by 100 · see citing works
Self-citation
George Box: A Model Statistician
Wasserstein, R. (2010) · Significance · doi:10.1111/j.1740-9713.2010.00442.x · Cited by 17 · see citing works
THE STRENGTH OF EVIDENCE VERSUS THE POWER OF BELIEF: ARE WE ALL BAYESIANS?
Utts, J. M., Norris, M., Suess, E. A., & Johnson, W. O. (2010) · Cited by 8 · see citing works
Self-citation

Flagged first, then by citations

Sorted by citations

Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications
Wagenmakers, E., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., et al. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1343-3 · Cited by 1,652 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
Using the Student's t-test with extremely small sample sizes
de Winter, J. C. F. (2013) · University of Massachusetts Amherst · doi:10.7275/e4r6-dj05 · Cited by 992 · see citing works
The Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963 · see citing works
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750 · see citing works
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376 · see citing works
A systematic review of Bayesian articles in psychology: The last 25 years.
van de Schoot, R., Winter, S. D., Ryan, O., Zondervan-Zwijnenburg, M. A. J., & Depaoli, S. A. (2017) · Psychological Methods · doi:10.1037/met0000100 · Cited by 376 · see citing works
Bayesian tests to quantify the result of a replication attempt.
Verhagen, J., & Wagenmakers, E. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0036731 · Cited by 287 · see citing works
Too good to be true: Publication bias in two prominent studies from experimental psychology
Francis, G. (2012) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0227-9 · Cited by 210 · see citing works
Failing the Future: Three Unsuccessful Attempts to Replicate Bem's ‘Retroactive Facilitation of Recall’ Effect
Ritchie, S. J., Wiseman, R. J., & French, C. C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033423 · Cited by 184 · see citing works
Statistical inference as severe testing how to get beyond the statistics wars
Mayo, D. G. (2018) · Cited by 152 · see citing works
Robust Bayesian meta‐analysis: Model‐averaging across complementary publication bias adjustment methods
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 138 · see citing works
Rethinking Reproducibility as a Criterion for Research Quality
Leonelli, S. (2018) · Research in the history of economic thought and methodology · doi:10.1108/s0743-41542018000036b009 · Cited by 136 · see citing works
How to quantify the evidence for the absence of a correlation
Wagenmakers, E., Verhagen, J., & Ly, A. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0593-0 · Cited by 129 · see citing works
Oxford Studies in Philosophy of Religion
(2015) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198722335.001.0001 · Cited by 108 · see citing works
Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events
Bem, D., Tressoldi, P. E., Rabeyron, T., & Duggan, M. (2016) · F1000Research · doi:10.12688/f1000research.7177.2 · Cited by 76 · see citing works
Self-citation
Towards understanding causality between work engagement and psychological capital
de Waal, J. J., & Pienaar, J. (2013) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v39i2.1113 · Cited by 69 · see citing works
Is the call to abandon p-values the red herring of the replicability crisis?
Savalei, V., & Dunn, E. W. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00245 · Cited by 62 · see citing works
Novelty versus Replicability: Virtues and Vices in the Reward System of Science
Romero, F. (2017) · Philosophy of Science · doi:10.1086/694005 · Cited by 61 · see citing works
Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events
Bem, D., Tressoldi, P. E., Rabeyron, T., & Duggan, M. (2015) · F1000Research · doi:10.12688/f1000research.7177.1 · Cited by 52 · see citing works
Self-citation
A Bayes factor meta-analysis of recent extrasensory perception experiments: Comment on Storm, Tressoldi, and Di Risio (2010).
Rouder, J. N., Morey, R. D., & Province, J. M. (2013) · Psychological Bulletin · doi:10.1037/a0029008 · Cited by 48 · see citing works
Why Most Research Findings About Psi Are False: The Replicability Crisis, the Psi Paradox and the Myth of Sisyphus
Rabeyron, T. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.562992 · Cited by 42 · see citing works
The objectivity of Subjective Bayesianism
Sprenger, J. (2018) · European Journal for Philosophy of Science · doi:10.1007/s13194-018-0200-1 · Cited by 46 · see citing works
Decision qualities of Bayes factor and p value-based hypothesis testing.
Jeon, M., & de Boeck, P. A. L. (2017) · Psychological Methods · doi:10.1037/met0000140 · Cited by 40 · see citing works
贝叶斯因子及其在JASP中的实现
Chuan-Peng, H., Kong, X., Er, I.-J., Al, E., & Peng, K. (2018) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2018.00951 · Cited by 39 · see citing works
The Sense of Agency as Tracking Control
Caspar, É. A., Desantis, A., Dienes, Z. P., Cleeremans, A., & Haggard, P. (2016) · PLoS ONE · doi:10.1371/journal.pone.0163892 · Cited by 38 · see citing works
Raising the value of research studies in psychological science by increasing the credibility of research reports: the transparent Psi project
Kekecs, Z., Pálfi, B., Szászi, B., Szécsi, P., Zrubka, M., Kovács, M., Bakos, B. E., Cousineau, D., Tressoldi, P. E., Schmidt, K., et al. (2023) · Royal Society Open Science · doi:10.1098/rsos.191375 · Cited by 58 · see citing works
On the correspondence between dream content and target material under laboratory conditions: A meta-analysis of dream-ESP studies, 1966-2016
Storm, L., Sherwood, S. J., Roe, C. A., Tressoldi, P. E., Rock, A. J., & Di Risio, L. (2017) · International Journal of Dream Research · doi:10.11588/ijodr.2017.2.34888 · Cited by 29 · see citing works
Bayesian Hypothesis Testing: An Alternative to Null Hypothesis Significance Testing (NHST) in Psychology and Social Sciences
Ortega, A., & Navarrete, G. (2017) · Bayesian Inference · doi:10.5772/intechopen.70230 · Cited by 31 · see citing works
Explicit Anomalous Cognition: A Review of the Best Evidence in Ganzfeld, Forced-choice, Remote Viewing and Dream Studies
Baptista, J., Derakhshani, M., Tressoldi, P. E., Cardeña, E., Palmer, J. A., & Eds, D. M.-C. (2014) · Cited by 29 · see citing works
From Discovery to Justification: Outline of an Ideal Research Program in Empirical Psychology
Witte, E. H., & Zenker, F. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01847 · Cited by 30 · see citing works
Why Bayesian Psychologists Should Change the Way They Use the Bayes Factor
Hoijtink, H. J. A., van Kooten, P., & Hulsker, K. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2014.969364 · Cited by 28 · see citing works
RoBMA: Robust Bayesian Meta-Analyses
Bartos̆, F., & Maier, M. (2020) · doi:10.32614/cran.package.robma · Cited by 28 · see citing works
REFLECTIONS ON BIASED ASSIMILATION AND BELIEF POLARIZATION
Ross, L. D. (2012) · Critical Review · doi:10.1080/08913811.2012.711025 · Cited by 26 · see citing works
Feeling the past: The absence of experimental evidence for anomalous retroactive influences on text processing
Traxler, M. J., Foss, D. J., Podali, R., & Zirnstein, M. (2012) · Memory & Cognition · doi:10.3758/s13421-012-0232-2 · Cited by 22 · see citing works
Testing the Storm et al. (2010) meta-analysis using Bayesian and frequentist approaches: reply to Rouder et al. (2013).
Storm, L., Tressoldi, P. E., & Utts, J. (2013) · Psychological Bulletin · doi:10.1037/a0029506 · Cited by 12 · see citing works
Self-citation
We should have seen this coming
Schwarzkopf, D. S. (2014) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2014.00332 · Cited by 19 · see citing works
Intentional Observer Effects on Quantum Randomness: A Bayesian Analysis Reveals Evidence Against Micro-Psychokinesis
Maier, M., Dechamps, M. C., & Pflitsch, M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00379 · Cited by 18 · see citing works
Enhancing Data Science Ethics Through Statistical Education and Practice
Utts, J. M. (2021) · International Statistical Review · doi:10.1111/insr.12446 · Cited by 17 · see citing works
Self-citation
Raising the value of research studies in psychological science by increasing the credibility of research reports: The Transparent Psi Project - PreprintPreprint
Kekecs, Z., Pálfi, B., Szászi, B., Szécsi, P., Zrubka, M., Kovács, M., Bakos, B. E., Cousineau, D., Tressoldi, P. E., Schmidt, K., et al. (2019) · doi:10.31234/osf.io/uwk7y · Cited by 14 · see citing works
Observation-Oriented Modeling: Going Beyond “Is It All a Matter of Chance”?
Grice, J. W., Yepez, M., Wilson, N. L., & Shoda, Y. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164416667985 · Cited by 12 · see citing works
Comparing Objective and Subjective Bayes Factors for the Two-Sample Comparison: The Classification Theorem in Action
Gönen, M., Johnson, W. O., Lu, Y., & Westfall, P. H. (2017) · The American Statistician · doi:10.1080/00031305.2017.1322142 · Cited by 12 · see citing works
Self-citation
How Smokers Change Their World and How the World Responds: Testing the Oscillatory Nature of Micro-Psychokinetic Observer Effects on Addiction-Related Stimuli
Dechamps, M. C., & Maier, M. (2019) · Journal of Scientific Exploration · doi:10.31275/2019/1513 · Cited by 10 · see citing works
A Skeptical Eye on PsiPreprint
Wagenmakers, E., Wetzels, R., Borsboom, D., Kievit, R., & van der Maas, H. L. J. (2020) · doi:10.31234/osf.io/syq2h · Cited by 10 · see citing works
A preregistered multi-lab replication of Maier et al. (2014, Exp. 4) testing retroactive avoidance
Maier, M., Buechner, V. L., Dechamps, M. C., Pflitsch, M., Kurzrock, W., Tressoldi, P. E., Rabeyron, T., Cardeña, E., Marcusson‐Clavertz, D., & Martsinkovskaja, T. (2020) · PLoS ONE · doi:10.1371/journal.pone.0238373 · Cited by 11 · see citing works
A Need for Individual Data Analyses for Assessments of Temporal Control: Invertebrate Fixed Interval Performance
Craig, D. P. A., & Abramson, C. I. (2015) · International Journal of Comparative Psychology · doi:10.46867/ijcp.2015.28.02.10 · Cited by 9 · see citing works
Bayes factor in one-sample tests of means with a sensitivity analysis: A discussion of separate prior distributions
Du, H., Edwards, M. C., & Zhang, Z. J. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01262-w · Cited by 9 · see citing works
Investigating expectation effects using multiple physiological measures
Siller, A., Ambach, W., & Vaitl, D. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01553 · Cited by 9 · see citing works
Plausibility, statistical interpretations, physical mechanisms and a new outlook: Response to commentaries on a precognition review.
Mossbridge, J., & Radin, D. I. (2018) · Psychology of Consciousness Theory Research and Practice · doi:10.1037/cns0000152 · Cited by 8 · see citing works
Beyond Schools: Reply to Marsman, Ly, and Wagenmakers
Witte, E. H., & Zenker, F. (2016) · Basic and Applied Social Psychology · doi:10.1080/01973533.2016.1227710 · Cited by 7 · see citing works
What’s in a Name: A Bayesian Hierarchical Analysis of the Name-Letter Effect
Dyjas, O., Grasman, R. P. P. P., Wetzels, R., van der Maas, H. L. J., & Wagenmakers, E. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00334 · Cited by 6 · see citing works
Significance Tests: Vitiated or Vindicated by the Replication Crisis in Psychology?
Mayo, D. G. (2020) · Review of Philosophy and Psychology · doi:10.1007/s13164-020-00501-w · Cited by 7 · see citing works
Contracts as reference points: A replication
Hippel, S., & Hoeppner, S. (2020) · International Review of Law and Economics · doi:10.1016/j.irle.2020.105973 · Cited by 6 · see citing works
Observer Dependent Biases of Quantum Randomness
Dechamps, M. C., Maier, M., Pflitsch, M., & Duggan, M. (2021) · Journal of Anomalous Experience and Cognition · doi:10.31156/jaex.23205 · Cited by 6 · see citing works
A note on replication analysis
Hoeppner, S. (2019) · International Review of Law and Economics · doi:10.1016/j.irle.2019.05.004 · Cited by 6 · see citing works
Non-dichotomous inference using bootstrapped evidencePreprint
Schwarzkopf, D. S. (2015) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/017327 · Cited by 6 · see citing works
Cobblers, Let’s Stick to Our Lasts! A Song of Sorrow (and of Hope) About the State of Personnel and Human Resource Management Science
Götz, M., & O’Boyle, E. H. (2023) · Emerald Group Publishing Limited eBooks · doi:10.1108/s0742-730120230000041004 · Cited by 6 · see citing works
Comment: Bayesian Statistics in the Twenty First Century
Johnson, W. O. (2012) · The American Statistician · doi:10.1080/00031305.2012.751880 · Cited by 5 · see citing works
Self-citation
How Smokers Change Their World and How the World Responds: Testing the Oscillatory Nature of Micro-psychokinetic Observer Effects on Addiction-related StimuliPreprint
Dechamps, M. C., & Maier, M. (2019) · doi:10.31234/osf.io/bwsj5 · Cited by 5 · see citing works
Istotność statystyczna II. Pułapki interpretacyjne
Wolski, P. (2017) · Rocznik Kognitywistyczny · doi:10.4467/20843895rk.16.006.6412 · Cited by 4 · see citing works
The Objectivity of Subjective Bayesian InferencePreprint
Sprenger, J. (2015) · PhilSci-Archive (University of Pittsburgh) · Cited by 4 · see citing works
Beyond Kotler and Böhme, towards postmaterialism? Tracing neglected theorisations of atmospheres and atmospherics
Tadajewski, M. (2025) · Journal of Marketing Management · doi:10.1080/0267257x.2025.2492874 · Cited by 3 · see citing works
Bayesian Model Averaging to Estimate Economic Optimum Nitrogen Rates in Agricultural Crops
Palmero, F., Ciampitti, I. A., & Hefley, T. J. (2026) · Field Crops Research · doi:10.1016/j.fcr.2026.110397 · Cited by 2 · see citing works
False Equivalencies and the Mediocrity of Nonlocal Consciousness Research Criticism
Schwartz, S. A. (2013) · EXPLORE · doi:10.1016/j.explore.2013.03.008 · Cited by 2 · see citing works
The Application of Comparative Psychology to Parapsychology: A Recommendation
Somers, A., Byrd, M. D., Curliss, B., Lay, V., Jones, I. T., & Abramson, C. I. (2020) · Natural Science · doi:10.4236/ns.2020.126029 · Cited by 2 · see citing works
An investigation into entrepreneurial intent amongst final year commerce students at the Durban University of Technology
Ndovela, S. M. (2017) · doi:10.51415/10321/2527 · Cited by 2 · see citing works
The t-test, resource pooling, and psi: why the minimum sample size counts towards theoretical progress in behavioral sciencePreprint
Witte, E. H., & Zenker, F. (2021) · doi:10.31219/osf.io/hx4fk · Cited by 3 · see citing works
Does Rural Household Electrification Improve Education in the Global South? Evidence from a Robust Bayesian Meta-Analysis
Ekeocha, D. O., & Garikipati, S. (2025) · The Energy Journal · doi:10.1177/01956574251340007 · Cited by 1 · see citing works
How Not to Corrupt Power
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.014 · Cited by 1 · see citing works
Empirical Bayes factors for common hypothesis tests
Dudbridge, F. (2022) · PLoS ONE · doi:10.1371/journal.pone.0297874 · Cited by 1 · see citing works
How Strong is the Confirmation of a Hypothesis by Significant Data?
Bartelborth, T. (2016) · Journal for General Philosophy of Science · doi:10.1007/s10838-016-9341-0 · Cited by 1 · see citing works
BAYESOV FAKTOR: OPIS I RAZLOZI ZA UPOTREBU U PSIHOLOŠKIM ISTRAŽIVANJIMA
Lakić, S. (2019) · ГОДИШЊАК ЗА ПСИХОЛОГИЈУ · doi:10.46630/gpsi.18.2019.03 · Cited by 1 · see citing works
A Case for Statistical Thinking, Not Just Statistical Technique: Using Non-Parametric Techniques Appropriately
Fox, C. R. (2018) · SAGE Publications Ltd eBooks · doi:10.4135/9781526439369 · Cited by 1 · see citing works
False-positive Effect in the Radin Double-slit Experiment: HARKing is used by Radin et al. to Misrepresent the Advanced Meta-experimental Protocol used in Walleczek and von Stillfried (2019)Preprint
Walleczek, J., & von Stillfried, N. (2020) · doi:10.31234/osf.io/a2vkn · Cited by 1 · see citing works
La falta de reproducibilitat de la investigació: L'estadística com a legitimació del resultat
Goddard, S. D., & Johnson, V. E. (2014) · Mètode: Revista de difusió de la investigació de la Universitat de Valencia · doi:10.7203/metode.0.3913
La falta de reproducibilitat de la investigació: L'estadística com a legitimació del resultat
Goddard, S. D., & Johnson, V. E. (2014) · Mètode: Revista de difusió de la investigació de la Universitat de Valencia · doi:10.7203/metode.83.3913
Results from a Conrmatory Replication Study of Bem (2011): Precognitive Detection of Erotic Stimuli?
Wagenmakers, E., Wetzels, R., Borsboom, D., & Kievit, R. (2011)
Am I Luckier Than You? Individual Differences in Precognition Performance
Zdrenka, M. (2018) · doi:10.26686/wgtn.19314362 · Cited by 1 · see citing works
The Myth of “The Myth of Objectivity”
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.009
Decision letter: Addressing selective reporting of experiments through predefined exclusion criteria
(2020) · eLife · doi:10.7554/elife.56626.sa1
Author response: Addressing selective reporting of experiments through predefined exclusion criteria
Neves, K., & Amaral, O. B. (2020) · eLife · doi:10.7554/elife.56626.sa2
Falsification, Pseudoscience, Induction
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.005
Beyond Probabilism and Performance
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.002 · Cited by 1 · see citing works
Statistical Inference in Psychological and Medical Research
Acree, M. C. (2021) · Springer eBooks · doi:10.1007/978-3-030-73257-8_9
Nonlocal consciousness: Distinguishing deniers from skeptics
Schwartz, S. A. (2021) · EXPLORE · doi:10.1016/j.explore.2021.08.010
Induction and Confirmation
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.004
Pragmatic and Error Statistical Bayesians
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.017
What Ever Happened to Bayesian Foundations?
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.016
Power: Pre-data and Post-data
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.013
Rejection Fallacies: Who’s Exaggerating What?
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.010
More Auditing: Objectivity and Model Checking
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.012
Souvenirs
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.018
Ingenious and Severe Tests
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.006
Error Probing Tools versus Logics of Evidence
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.003
Auditing: Biasing Selection Effects and Randomization
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.011
Capability and Severity: Deeper Concepts
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.008
It’s The Methods, Stupid
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.007
Preface
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.001
Deconstructing the N-P versus Fisher Debates
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.015
Index
Mayo, D. G. (2018) · Cambridge University Press eBooks · doi:10.1017/9781107286184.019

Authors: Bem, D. J., Utts, J., & Johnson, W. O.
Affiliations: Cornell University (US)
University of California, Irvine (US)
ORCID: Utts, Jessica
Johnson, Wesley O.
Venue: Journal of Personality and Social Psychology
Publisher: American Psychological Association (APA)
Published: 2011-10
Volume: 101
Issue: 4
Pages: 716-719
ISSN: 0022-3514 (ISSN-L)
1939-1315
DOI: 10.1037/a0024777
OpenAlex: W2042930386
Cited by: 105
References: 16
Citations by year: 2026: 2 · 2025: 5 · 2024: 1 · 2023: 3 · 2022: 1 · 2021: 5 · 2020: 10 · 2019: 6 · 2018: 28 · 2017: 10 · 2016: 6 · 2015: 9 · 2014: 6 · 2013: 4 · 2012: 8
FWCI: 5.33
Type: journal article
Language: en
Open access: closed
Locations: Journal of Personality and Social Psychology (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Psychology › Social Psychology › Paranormal Experiences and Beliefs
Keywords: Bayesian analysis, frequentist statistics, prior selection, reanalysis, evidence evaluation
MeSH terms: Data Interpretation, Statistical, Humans, Psychology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Must psychologists change the way they analyze their data?

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks