Cited by 1,042Meta-analysis and systematic reviewsPsychology Research and BibliometricsStatistical Methods and Bayesian Inference

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

An Agenda for Purely Confirmatory Research

Wagenmakers, Eric-Jan, Wetzels, Ruud, Borsboom, Denny, van der Maas, Han L. J., Kievit, Rogier A.·2012·Perspectives on Psychological Science

DOI 10.1177/1745691612463078

Open this paper in CoChat — free account

Abstract

The veracity of substantive research claims hinges on the way experimental data are collected and analyzed. In this article, we discuss an uncomfortable fact that threatens the core of psychology's academic enterprise: almost without exception, psychologists do not commit themselves to a method of data analysis before they see the actual data. It then becomes tempting to fine tune the analysis to the data in order to obtain a desired result-a procedure that invalidates the interpretation of the common statistical tests. The extent of the fine tuning varies widely across experiments and experimenters but is almost impossible for reviewers and readers to gauge. To remedy the situation, we propose that researchers preregister their studies and indicate in advance the analyses they intend to conduct. Only these analyses deserve the label "confirmatory," and only for these analyses are the common statistical tests valid. Other analyses can be carried out but these should be labeled "exploratory." We illustrate our proposal with a confirmatory replication attempt of a study on extrasensory perception.

Integrity signals

Registry

Crossref · 10.1177/1745691612463078

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Bayes factors
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Journal of personality and social psychology
Armstrong, J. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90126-4 · Cited by 10,544
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Wildlife Society Bulletin
Wildlife Society (2011) · Wildlife Society Bulletin · doi:10.1002/(issn)1938-5463a · Cited by 5,571
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“confirmatory experiments, wonky statistics, ESP, Bayesian hypothesis test Several years ago, Ioannidis (2005) famously argued that “most published research findings are false.””
Journal of the American Statistical Association
Fildes, R. A. (1989) · International Journal of Forecasting · doi:10.1016/0169-2070(89)90106-4 · Cited by 3,956
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 1×
“And indeed, recent results from biomedical and cancer research suggest that replication rates are lower than 50%, with some as low as 11% (Begley & Ellis, 2012; Osherovich, 2011; Prinz, Schlange, & Asadullah, 2011).”
Result

Flagged first, then by citations

Show all 48 references

Research works printed but not linked: 4

Feynman R. P., 1974, Engineering & Science, 37, 10
Goldacre B., 2009, Bad science
Jeffreys H., 1961, Theory of probability
Roediger H. L., 2012, APS Observer, 25, 9

Sorted by citations

Items 101–200 of 998

The daily association between affect and alcohol use: A meta-analysis of individual participant data.
Dora, J., Piccirillo, M. L., Foster, K. T., Arbeau, K. J., Armeli, S., Auriacombe, M., Bartholow, B. D., Beltz, A. M., Blumenstock, S. M., Bold, K. W., et al. (2023) · Psychological Bulletin · doi:10.1037/bul0000387 · Cited by 175
Answers to 18 Questions About Open Science Practices
Banks, G. C., Field, J. G., Oswald, F. L., O’Boyle, E. H., Landis, R. S., Rupp, D. E., & Rogelberg, S. G. (2018) · Journal of Business and Psychology · doi:10.1007/s10869-018-9547-8 · Cited by 167
Mapping the universe of registered reports
Hardwicke, T. E., & Ioannidis, J. P. A. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0444-y · Cited by 155
Researchers’ Intuitions About Power in Psychological Research
Bakker, M., Hartgerink, C. H. J., Wicherts, J. M., & van der Maas, H. L. J. (2016) · Psychological Science · doi:10.1177/0956797616647519 · Cited by 158
Robust Bayesian meta‐analysis: Model‐averaging across complementary publication bias adjustment methods
Bartos̆, F., Maier, M., Wagenmakers, E., Doucouliagos, H., & Stanley, T. D. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1594 · Cited by 156
Self-citation
Does a single session of reading literary fiction prime enhanced mentalising performance? Four replication experiments of Kidd and Castano (2013)
Samur, D., Tops, M., & Koole, S. L. (2018) · Cognition and Emotion · doi:10.1080/02699931.2017.1279591 · Cited by 142
Development of Indirect Measures of Conscientiousness: Combining a Facets Approach and Network Analysis
Costantini, G., Richetin, J., Borsboom, D., Fried, E. I., Rhemtulla, M., & Perugini, M. (2015) · European Journal of Personality · doi:10.1002/per.2014 · Cited by 148
Self-citation
Justifications Shape Ethical Blind Spots
Pittarello, A., Leib, M., Gordon‐Hecker, T., & Shalvi, S. (2015) · Psychological Science · doi:10.1177/0956797615571018 · Cited by 141
Choosing the negative: A behavioral demonstration of morbid curiosity
Oosterwijk, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178399 · Cited by 136
Ensuring the quality and specificity of preregistrations
Bakker, M., Veldkamp, C. L. S., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000937 · Cited by 138
Robust Modeling in Cognitive Science
Lee, M. D., Criss, A. H., Devezer, B., Donkin, C., Etz, A. J., Leite, F. P., Matzke, D., Rouder, J. N., Trueblood, J. S., White, C. N., et al. (2019) · Computational Brain & Behavior · doi:10.1007/s42113-019-00029-y · Cited by 148
The Production of Knowledge
(2020) · Cambridge University Press eBooks · doi:10.1017/9781108762519 · Cited by 154
Meta-analysis of ERP investigations of pain empathy underlines methodological issues in ERP research
Coll, M. (2018) · Social Cognitive and Affective Neuroscience · doi:10.1093/scan/nsy072 · Cited by 142
Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis
Schweinsberg, M., Feldman, M., Staub, N., van den Akker, O. R., van Aert, R. C. M., van Assen, M. A. L. M., Liu, Y., Althoff, T., Heer, J., Kale, A. M., et al. (2021) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2021.02.003 · Cited by 145
Making the Black Box Transparent: A Template and Tutorial for Registration of Studies Using Experience-Sampling Methods
Kirtley, O. J., Lafit, G., Achterhof, R., Hiekkaranta, A. P., & Myin‐Germeys, I. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920924686 · Cited by 146
How to quantify the evidence for the absence of a correlation
Wagenmakers, E., Verhagen, J., & Ly, A. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0593-0 · Cited by 133
Self-citation
Research Practices That Can Prevent an Inflation of False-Positive Rates
Murayama, K., Pekrun, R., & Fiedler, K. (2014) · Personality and Social Psychology Review · doi:10.1177/1088868313496330 · Cited by 124
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education
Adelson, J. L., Barton, E., Bradshaw, C., Nosek, B. A., Bryant, D., Cook, B. G., Coyne, M., DeBettencourt, L., DeHaven, A. C., Dymond, S., et al. (2018) · Exceptional Children · doi:10.1177/0014402918793138 · Cited by 136
The frequency of excess success for articles in Psychological Science
Francis, G. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0601-x · Cited by 130
Preregistration of secondary data analysis: A template and tutorial
van den Akker, O. R., Weston, S. J., Campbell, L., Chopik, B., Damian, R. I., Davis-Kean, P. E., Hall, A., Kosie, J. E., Kruse, E. T., Olsen, J., et al. (2021) · Meta-Psychology · doi:10.15626/mp.2020.2625 · Cited by 137
How Trustworthy Is the Scientific Literature in Industrial and Organizational Psychology?
Kepes, S., & McDaniel, M. A. (2013) · Industrial and Organizational Psychology · doi:10.1111/iops.12045 · Cited by 136
Standardized protocols for characterizing women’s fertility: A data-driven approach
Blake, K. R., Dixson, B. J. W., O'Dean, S. M., & Denson, T. F. (2016) · Hormones and Behavior · doi:10.1016/j.yhbeh.2016.03.004 · Cited by 123
The search for human pheromones: The lost decades and the necessity of returning to first principles
Wyatt, T. D. (2015) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2014.2994 · Cited by 123
A Tale of Two Systems
Keren, G. B. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691613483474 · Cited by 122
Moral elevation reduces prejudice against gay men
Lai, C. K., Haidt, J., & Nosek, B. A. (2013) · Cognition & Emotion · doi:10.1080/02699931.2013.861342 · Cited by 123
The Need for Bayesian Hypothesis Testing in Psychological Science
Wagenmakers, E., Verhagen, J., Ly, A., Matzke, D., Steingroever, H., Rouder, J. N., & Morey, R. D. (2017) · Wiley eBooks · doi:10.1002/9781119095910.ch8 · Cited by 124
Self-citation
Big little lies: a compendium and simulation of p -hacking strategies
Stefan, A. M., & Schönbrodt, F. D. (2023) · Royal Society Open Science · doi:10.1098/rsos.220346 · Cited by 177
Underreporting in Psychology Experiments
Franco, A., Malhotra, N. A., & Simonovits, G. (2015) · Social Psychological and Personality Science · doi:10.1177/1948550615598377 · Cited by 119
Aspiring to greater intellectual humility in science
Hoekstra, R., & Vazire, S. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01203-8 · Cited by 117
The effect of horizontal eye movements on free recall: a preregistered adversarial collaboration.
Matzke, D., Nieuwenhuis, S., van Rijn, H., Slagter, H., van der Molen, M. W., & Wagenmakers, E. (2015) · Journal of experimental psychology. General · doi:10.1037/xge0000038 · Cited by 148
Self-citation
Untitled work
Peterson, D. (2016) · doi:10.1177/2378023115625071 · Cited by 111
A Guide to Robust Statistical Methods in Neuroscience
Wilcox, R. R., & Rousselet, G. A. (2018) · Current Protocols in Neuroscience · doi:10.1002/cpns.41 · Cited by 107
The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline
Schweinsberg, M., Madan, N., Vianello, M., Sommer, S. A., Jordan, J., Tierney, W., Awtrey, E. C., Zhu, L. L., Diermeier, D. A., Heinze, J. E., et al. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.10.001 · Cited by 110
Self-citation
The psychology of experimental psychologists: Overcoming cognitive constraints to improve research: The 47th Sir Frederic Bartlett Lecture
Bishop, D. (2019) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819886519 · Cited by 115
Feminism and psychology: Critiques of methods and epistemology.
Eagly, A. H., & Riger, S. (2014) · American Psychologist · doi:10.1037/a0037372 · Cited by 126
Suicide and self-injury-related implicit cognition: A large-scale examination and replication.
Glenn, J. J., Werntz, A. J., Slama, S. J. K., Steinman, S. A., Teachman, B. A., & Nock, M. K. (2016) · Journal of Abnormal Psychology · doi:10.1037/abn0000230 · Cited by 110
A new reality: Fan perceptions of augmented reality readiness in sport marketing
Goebert, C., & Greenhalgh, G. P. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.106231 · Cited by 111
Calibrating the Scientific Ecosystem Through Meta-Research
Hardwicke, T. E., Serghiou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (2020) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031219-041104 · Cited by 98
PsychDisclosure.org
LeBel, E. P., Borsboom, D., Giner‐Sorolla, R., Hasselman, F., Peters, K. R., Ratliff, K. A., & Smith, C. T. (2013) · Perspectives on Psychological Science · doi:10.1177/1745691613491437 · Cited by 106
Self-citation
Digital Screen Time and Pediatric Sleep: Evidence from a Preregistered Cohort Study
Przybylski, A. K. (2019) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2018.09.054 · Cited by 102
Outlier Removal and the Relation with Reporting Errors and Quality of Psychological Research
Bakker, M., & Wicherts, J. M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0103360 · Cited by 101
A methodological synthesis and meta-analysis of judgment tasks in second language research
Plonsky, L., Marsden, E. J., Crowther, D., Gass, S. M., & Spinner, P. (2019) · Second language Research · doi:10.1177/0267658319828413 · Cited by 111
Kinds of Replication: Examining the Meanings of “Conceptual Replication” and “Direct Replication”
Derksen, M., & Morawski, J. G. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916211041116 · Cited by 104
A Call for Examining Replication and Bias in Special Education Research
Cook, B. G. (2014) · Remedial and Special Education · doi:10.1177/0741932514528995 · Cited by 100
Exploratory research in sport and exercise science: Perceptions, challenges, and recommendations
Ditroilo, M., Mesquida, C., Abt, G., & Lakens, D. (2025) · Journal of Sports Sciences · doi:10.1080/02640414.2025.2486871 · Cited by 127
The influence of gender stereotype threat on mathematics test scores of Dutch high school students: a registered report
Flore, P. C., Mulder, J., & Wicherts, J. M. (2018) · Comprehensive Results in Social Psychology · doi:10.1080/23743603.2018.1559647 · Cited by 105
Arrested Theory Development: The Misguided Distinction Between Exploratory and Confirmatory Research
Szollosi, A., & Donkin, C. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620966796 · Cited by 107
Action video games do not improve the speed of information processing in simple perceptual tasks.
van Ravenzwaaij, D., Boekel, W., Forstmann, B. U., Ratcliff, R., & Wagenmakers, E. (2014) · Journal of Experimental Psychology General · doi:10.1037/a0036923 · Cited by 98
Self-citation
Improving transparency and reproducibility through registration: The status of intervention trials published in clinical psychology journals.
Cybulski, Ł., Mayo‐Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 94
Does Taking a Short Break from Social Media Have a Positive Effect on Well-being? Evidence from Three Preregistered Field Experiments
Przybylski, A. K., Nguyen, T.-V. T., Law, W., & Weinstein, N. (2021) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00189-w · Cited by 91
Distributions of p -values smaller than .05 in psychology: what is going on?
Hartgerink, C. H. J., van Aert, R. C. M., Nuijten, M. B., Wicherts, J. M., & van Assen, M. A. L. M. (2016) · PeerJ · doi:10.7717/peerj.1935 · Cited by 90
The registration continuum in clinical science: A guide toward transparent practices
Benning, S. D., Bachrach, R. L., Smith, E. A., Freeman, A. J., & Wright, A. G. C. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000451 · Cited by 77
The stimulated social brain: effects of transcranial direct current stimulation on social cognition
Sellaro, R., Nitsche, M. A., & Colzato, L. S. (2016) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13098 · Cited by 95
Single-case intervention research design standards: Additional proposed upgrades and future directions
Kratochwill, T. R., Horner, R. H., Levin, J. R., Machalicek, W., Ferron, J. M., & Johnson, A. H. (2023) · Journal of School Psychology · doi:10.1016/j.jsp.2022.12.002 · Cited by 100
Cluster failure or power failure? Evaluating sensitivity in cluster-level inference
Noble, S., Scheinost, D., & Constable, R. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116468 · Cited by 88
On the trajectory of discrimination: A meta-analysis and forecasting survey capturing 44 years of field experiments on gender and hiring decisions
Schaerer, M., du Plessis, C., Nguyen, M. H. B., van Aert, R. C. M., Tiokhin, L., Lakens, D., Giulia Clemente, E., Pfeiffer, T., Dreber, A., Johannesson, M., et al. (2023) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2023.104280 · Cited by 87
Can infants learn phonology in the lab? A meta-analytic answer
Cristià, A. (2017) · Cognition · doi:10.1016/j.cognition.2017.09.016 · Cited by 86
Power Analysis and Exploratory Research
Haile, Z. T. (2023) · Journal of Human Lactation · doi:10.1177/08903344231195625 · Cited by 91
A comprehensive meta-analysis of money priming.
Lodder, P., Ong, H. H., Grasman, R. P. P. P., & Wicherts, J. M. (2019) · Journal of Experimental Psychology General · doi:10.1037/xge0000570 · Cited by 86
Mega‐analysis methods in ENIGMA : The experience of the generalized anxiety disorder working group
Zugman, A., Harrewijn, A., Cardinale, E. M., Zwiebel, H., Freitag, G. F., Werwath, K. E., Bas‐Hoogendam, J. M., Groenewold, N. A., Aghajani, M., Hilbert, K., et al. (2020) · Human Brain Mapping · doi:10.1002/hbm.25096 · Cited by 83
Preregistration Is Neither Sufficient nor Necessary for Good Science
Pham, M. T., & Oh, T. T. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1209 · Cited by 91
Actionable recommendations for narrowing the science-practice gap in open science
Aguinis, H., Banks, G. C., Rogelberg, S. G., & Cascio, W. F. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.02.007 · Cited by 90
Journal Data Sharing Policies and Statistical Reporting Inconsistencies in Psychology
Nuijten, M. B., Borghuis, J., Veldkamp, C. L. S., Dominguez-Alvarez, L., van Assen, M. A. L. M., & Wicherts, J. M. (2017) · Collabra: Psychology · doi:10.1525/collabra.102 · Cited by 79
Playing with Data—Or How to Discourage Questionable Research Practices and Stimulate Researchers to Do Things Right
Sijtsma, K. (2015) · Psychometrika · doi:10.1007/s11336-015-9446-0 · Cited by 88
Exploratory hypothesis tests can be more compelling than confirmatory hypothesis tests
Rubin, M., & Donkin, C. (2022) · Philosophical Psychology · doi:10.1080/09515089.2022.2113771 · Cited by 98
Virtual reality check: Statistical power, reported results, and the validity of research on the psychology of virtual reality and immersive environments
Lanier, M., Waddell, T. F., Elson, M., Tamul, D. J., Ivory, J., & Przybylski, A. K. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.06.015 · Cited by 79
Judgments of learning (JOLs) selectively improve memory depending on the type of test
Myers, S. J., Rhodes, M. G., & Hausman, H. E. (2020) · Memory & Cognition · doi:10.3758/s13421-020-01025-5 · Cited by 76
Finite mixture (or latent class) modeling in transportation: Trends, usage, potential, and future directions
Kim, S. H., & Mokhtarian, P. L. (2023) · Transportation Research Part B Methodological · doi:10.1016/j.trb.2023.03.001 · Cited by 81
Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events
Bem, D., Tressoldi, P. E., Rabeyron, T., & Duggan, M. (2016) · F1000Research · doi:10.12688/f1000research.7177.2 · Cited by 84
Many analysts, one dataset: Making transparent how variations in analytical choices affect resultsPreprint
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E. C., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., et al. (2017) · doi:10.31234/osf.io/qkwst · Cited by 78
Self-citation
Study preregistration: An evaluation of a method for transparent reporting
Toth, A. A., Banks, G. C., Mellor, D., O’Boyle, E. H., Dickson, A., Davis, D. J., DeHaven, A., Bochantin, J., & Borns, J. (2021) · Journal of Business and Psychology · doi:10.1007/s10869-020-09695-3 · Cited by 73
Development and Validation of the Certainty About Mental States Questionnaire (CAMSQ): A Self-Report Measure of Mentalizing Oneself and Others
Müller, S., Wendt, L. P., & Zimmermann, J. (2021) · Assessment · doi:10.1177/10731911211061280 · Cited by 95
A survey on how preregistration affects the research workflow: better science but more work
Sarafoglou, A., Kovács, M., Bakos, B. E., Wagenmakers, E., & Aczél, B. (2022) · Royal Society Open Science · doi:10.1098/rsos.211997 · Cited by 83
Self-citation
Heterogeneity in effect size estimates
Holzmeister, F., Johannesson, M., BOHM, R. M., Dreber, A., Huber, J., & Kirchler, M. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2403490121 · Cited by 89
Reproducibility of science: Fraud, impact factors and carelessness
Eisner, D. A. (2017) · Journal of Molecular and Cellular Cardiology · doi:10.1016/j.yjmcc.2017.10.009 · Cited by 80
Lack of preregistered analysis plans allows unacceptable data mining for and selective reporting of consensus in Delphi studies
Grant, S., Booth, M. S., & Khodyakov, D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.03.007 · Cited by 79
Exploring Small, Confirming Big: An alternative system to The New Statistics for advancing cumulative and replicable psychological research
Sakaluk, J. K. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.013 · Cited by 76
Open Science Is Liberating and Can Foster Creativity
Frankenhuis, W. E., & Nettle, D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691618767878 · Cited by 74
Dataset decay and the problem of sequential analyses on open datasets
Thompson, W. H., Wright, J., Bissett, P. G., & Poldrack, R. A. (2020) · eLife · doi:10.7554/elife.53498 · Cited by 70
Confidence in masked orientation judgments is informed by both evidence and visibility
Rausch, M., Hellmann, S., & Zehetleitner, M. (2017) · Attention Perception & Psychophysics · doi:10.3758/s13414-017-1431-5 · Cited by 74
The Replication Paradox: Combining Studies can Decrease Accuracy of Effect Size Estimates
Nuijten, M. B., van Assen, M. A. L. M., Veldkamp, C. L. S., & Wicherts, J. M. (2015) · Review of General Psychology · doi:10.1037/gpr0000034 · Cited by 73
Psychologists psychologizing scientific psychology: An epistemological reading of the replication crisis
Flis, I. (2019) · Theory & Psychology · doi:10.1177/0959354319835322 · Cited by 74
Replication and Reproducibility in Cross-Cultural Psychology
Milfont, T. L., & Klein, R. A. (2018) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022117744892 · Cited by 69
The Assessment and Modeling of Perceptual Control: A Transformation in Research Methodology to Address the Replication Crisis
Mansell, W., & Huddy, V. C. (2018) · Review of General Psychology · doi:10.1037/gpr0000147 · Cited by 67
Using mouse cursor tracking to investigate online cognition: Preserving methodological ingenuity while moving toward reproducible science
Schoemann, M., O’Hora, D. P., Dale, R. A. C., & Scherbaum, S. (2020) · Psychonomic Bulletin & Review · doi:10.3758/s13423-020-01851-3 · Cited by 70
Der Glaube an Big Data
Lünich, M. (2022) · Springer eBooks · doi:10.1007/978-3-658-36368-0 · Cited by 68
Excess Success for Psychology Articles in the Journal Science
Francis, G., Tanzman, J. S., & Matthews, W. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114255 · Cited by 69
Validating a Standardized Approach to the Taylor Aggression Paradigm
Chester, D. S., & Lasko, E. N. (2018) · Social Psychological and Personality Science · doi:10.1177/1948550618775408 · Cited by 67
Creative destruction in science
Tierney, W., Hardy, J. H., Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., et al. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.07.002 · Cited by 67
Questionable Research Practices in Experimental Communication Research: A Systematic Analysis From 1980 to 2013
Matthes, J., Marquart, F., Naderer, B., Arendt, F., Schmuck, D., & Adam, K. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096334 · Cited by 74
A prospective study of the motivational and health dynamics of Internet Gaming Disorder
Weinstein, N., Przybylski, A. K., & Murayama, K. (2017) · PeerJ · doi:10.7717/peerj.3838 · Cited by 78
Selection bias, vote counting, and money-priming effects: A comment on Rohrer, Pashler, and Harris (2015) and Vohs (2015).
Vadillo, M. A., Hardwicke, T. E., & Shanks, D. R. (2016) · Journal of Experimental Psychology General · doi:10.1037/xge0000157 · Cited by 69
Introducing Registered Reports at Language Learning: Promoting Transparency, Replication, and a Synthetic Ethic in the Language Sciences
Marsden, E. J., Morgan‐Short, K., Trofimovich, P., & Ellis, N. C. (2018) · Language Learning · doi:10.1111/lang.12284 · Cited by 71
The relationship between parental behavior and infant regulation: A systematic review
Samdan, G., Kiel, N., Petermann, F., Rothenfußer, S., Zierul, C., & Reinelt, T. (2020) · Developmental Review · doi:10.1016/j.dr.2020.100923 · Cited by 83
Variable control and conceptual change: A large-scale quantitative study in elementary school
Edelsbrunner, P. A., Schalk, L., Schumacher, R., & Stern, E. (2018) · Learning and Individual Differences · doi:10.1016/j.lindif.2018.02.003 · Cited by 72
To veg or not to veg? The impact of framing on vegetarian food choice
Krpan, D., & Houtsma, N. (2020) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2020.101391 · Cited by 78
Bugs as features (part 1): concepts and foundations for the compositional data analysis of the microbiome–gut–brain axis
Bastiaanssen, T. F. S., Quinn, T. P., & Loughman, A. (2023) · Nature Mental Health · doi:10.1038/s44220-023-00148-3 · Cited by 71
Encourage Playing with Data and Discourage Questionable Reporting Practices
Wigboldus, D. H. J., & Dotsch, R. (2015) · Psychometrika · doi:10.1007/s11336-015-9445-1 · Cited by 63
The benefits of preregistration and Registered Reports
Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024) · Evidence-Based Toxicology · doi:10.1080/2833373x.2024.2376046 · Cited by 93
Replication Study: Fusobacterium nucleatum infection is prevalent in human colorectal carcinoma
Repass, J. F., Reproducibility Project: Cancer Biology, Iorns, E. J., Denis, A., Williams, S. R., Perfito, N., & Errington, T. M. (2018) · eLife · doi:10.7554/elife.25801 · Cited by 63

Authors: Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A.
Affiliations: University of Amsterdam (NL)
ORCID: Wagenmakers, Eric-Jan
Borsboom, Denny
van der Maas, Han L. J.
Kievit, Rogier A.
Venue: Perspectives on Psychological Science
Publisher: SAGE Publications
Published: 2012-11
Volume: 7
Issue: 6
Pages: 632-638
ISSN: 1745-6916 (ISSN-L)
1745-6924
DOI: 10.1177/1745691612463078
OpenAlex: W2123263696
Cited by: 1,042
References: 48
Citations by year: 2026: 57 · 2025: 41 · 2024: 65 · 2023: 85 · 2022: 102 · 2021: 108 · 2020: 114 · 2019: 98 · 2018: 88 · 2017: 86 · 2016: 75 · 2015: 51 · 2014: 43 · 2013: 28 · 2012: 1
FWCI: 44.75
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Perspectives on Psychological Science (publishedVersion)
PubMed (publishedVersion)
UvA-DARE (University of Amsterdam) (publishedVersion)
UvA-DARE (University of Amsterdam) (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords: preregistration, confirmatory analysis, exploratory analysis, data snooping, statistical validity, research integrity, extrasensory perception, replication, hypothesis testing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

An Agenda for Purely Confirmatory Research

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks