Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond
Open accessAbstract
The last ten years have witnessed increasing awareness of questionable research practices (QRPs) in the life sciences [1,2], including p-hacking [3], HARKing [4], lack of replication [5], publication bias [6], low statistical power [7] and lack of data sharing ([8]; see Figure 1). Concerns about such behaviours have been raised repeatedly for over half a century [9–11] but the incentive structure of academia has not changed to address them. \nDespite the complex motivations that drive academia, many QRPs stem from the simple fact that the incentives which offer success to individual scientists conflict with what is best for science [12]. On the one hand are a set of gold standards that centuries of the scientific method have proven to be crucial for discovery: rigour, reproducibility, and transparency. On the other hand are a set of opposing principles born out of the academic career model: the drive to produce novel and striking results, the importance of confirming prior expectations, and the need to protect research interests from competitors. Within a culture that pressures scientists to produce rather than discover, the outcome is a biased and impoverished science in which most published results are either unconfirmed genuine discoveries or unchallenged fallacies [13]. This observation implies no moral judgement of scientists, who are as much victims of this system as they are perpetrators.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Protocol registration has emerged as a mechanism that is likely to improve the transparency of animal research.
“Akin to the approach taken for clinical trials, protocol registration has emerged as a mechanism that is likely to improve the transparency of animal research [232,237,238].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [237]
The Conclusions (page unavailable) describe preregistration as growing in basic science and as an incentive for transparent practices. The discussion of clinical trial registration (page unavailable) also supports the analogy to clinical trials. The paper does not specifically establish that protocol registration improves transparency in animal research. (LLM, conf=0.83)
Experimental — accuracy not yet measuredIn the Registered Report format, protocols undergo peer review, and if accepted, the journal commits to publishing the completed research regardless of the results obtained.
“For example, in the Registered Report format offered by an increasing number of journals, protocols undergo peer review, and if accepted, the journal commits to publishing the completed research regardless of the results obtained [237].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [237]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cook, R. J., Dickens, B. M., & Fathalla, M. F. (2003) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199241323.003.0025 · Cited by 10,813 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688 · see citing works
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490 · see citing works
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824 · see citing works
Nosek (2012) · doi:10.1177/1745691612459058 · Cited by 1,301 · see citing works
Sterling, T. D. (1959) · Journal of the American Statistical Association · doi:10.1080/01621459.1959.10501497 · Cited by 964 · see citing works
Bayesian Versus Orthodox Statistics: Which Side Are You On?
Dienes, Z. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691611406920 · Cited by 938 · see citing works
Replications in Psychology Research: How Often Do They Really Occur?
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612460688 · Cited by 750 · see citing works
“Positive” results increase down the Hierarchy of the Sciences
Fanelli, D. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010068 · Cited by 734 · see citing works
Registered Reports: A Method to Increase the Credibility of Published Results
Nosek, B. A., & Lakens, D. (2014) · Social Psychology · doi:10.1027/1864-9335/a000192 · Cited by 716 · see citing works
Registered Reports: A new publishing initiative at Cortex
Chambers, C. (2012) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 574 · see citing works
Why Science Is Not Necessarily Self-Correcting
Ioannidis, J. P. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612464056 · Cited by 563 · see citing works
Deep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441 · see citing works
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011) · PLoS ONE · doi:10.1371/journal.pone.0026828 · Cited by 424 · see citing works
Adolescent impulsivity phenotypes characterized by distinct brain networks
the IMAGEN Consortium, Whelan, R., Conrod, P. J., Poline, J.-B., Lourdusamy, A., Banaschewski, T., Barker, G. J., Bellgrove, M. A., Büchel, C., Byrne, M., et al. (2012) · Nature Neuroscience · doi:10.1038/nn.3092 · Cited by 420 · see citing works
The Long Way From α-Error Control to Validity Proper
Fiedler, K., Kutzner, F., & Krueger, J. I. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612462587 · Cited by 229 · see citing works
Groot, A. D. D. (2014) · Acta Psychologica · doi:10.1016/j.actpsy.2014.02.001 · Cited by 190 · see citing works
Use of Trial Register Information during the Peer Review Process
Mathieu, S., Chan, A., & Ravaud, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059910 · Cited by 71 · see citing works
Stahl, C. (2014) · Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) · doi:10.1027/1618-3169/a000257 · Cited by 45 · see citing works
Strube, M. (2006) · Behavior Research Methods · doi:10.3758/bf03192746 · Cited by 41 · see citing works
Registered Reports and Replications in Attention, Perception, & Psychophysics
(2013) · Attention Perception & Psychophysics · doi:10.3758/s13414-013-0502-5 · Cited by 20 · see citing works
Registered Reports: A new submission format at Drug and Alcohol Dependence
Munafo, M. R., & Strain, E. C. (2014) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2014.02.699 · Cited by 7 · see citing works
Storkel, H. L., & Gallun, F. J. (2022) · Journal of Speech Language and Hearing Research · doi:10.1044/2021_jslhr-21-00513 · Cited by 7 · see citing works
Hill, C. R., Samendinger, S., & Rymal, A. M. (2020) · Annals of Behavioral Medicine · doi:10.1093/abm/kaaa080 · Cited by 3 · see citing works
Registered Reports: A new chapter at Ecology & Evolution
Gya, R., Birkeli, K., Dahle, I. J., Foote, C. G., Geange, S. R., Lynn, J. S., Töpper, J. P., Vandvik, V., Zernichow, C., & Jenkins, G. B. (2023) · Ecology and Evolution · doi:10.1002/ece3.10023 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Research on Application and Impact of IFRS 9 Financial Instruments , 2022, 32, 1035-6908, 409, 10.1111/auar.12391 123. Holly L. Storkel, Frederick J. Gallun, · DOI
Cited by
Sorted by citations
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0
Percie du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000411 · Cited by 2,677 · see citing works
Genome-wide association studies
Uffelmann, E., Huang, Q. Q., Munung, N. S., de Vries, J., Okada, Y., Martin, A. R., Martin, H. C., Lappalainen, T., & Posthuma, D. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-021-00056-9 · Cited by 1,849 · see citing works
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · see citing works
Scanning the horizon: towards transparent and reproducible neuroimaging research
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., Nichols, T. E., Poline, J.-B., Vul, E., & Yarkoni, T. (2017) · Nature Reviews Neuroscience · doi:10.1038/nrn.2016.167 · Cited by 1,410 · see citing works
Indices of Effect Existence and Significance in the Bayesian Framework
Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02767 · Cited by 983 · see citing works
Itay Goldstein, B., Jiang, W., & Karolyi, G. A. (2019) · Review of Financial Studies · doi:10.1093/rfs/hhz025 · Cited by 709 · see citing works
Reporting animal research: Explanation and Elaboration for the ARRIVE guidelines 2019Preprint
du Sert, N. P., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/703355 · Cited by 557 · see citing works
Hong, H. G., Karolyi, G. A., & Scheinkman, J. A. (2019) · Review of Financial Studies · doi:10.1093/rfs/hhz146 · Cited by 550 · see citing works
Transparency, Reproducibility, and the Credibility of Economics Research
Christensen, G., & Miguel, E. A. (2018) · Journal of Economic Literature · doi:10.1257/jel.20171350 · Cited by 448 · see citing works
Pre-registration in social psychology—A discussion and suggested template
Veer, A. V. '., & Giner‐Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 436 · see citing works
How Bayes factors change scientific practice
Dienes, Z. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.10.003 · Cited by 364 · see citing works
The psycho Package: an Efficient and Publishing-Oriented Workflow for Psychological Science
Makowski, D. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00470 · Cited by 380 · see citing works
Improving inferences about null effects with Bayes factors and equivalence tests.
Lakens, D., McLatchie, N., Isager, P. M., Scheel, A. M., & Dienes, Z. (2020) · The Journals of Gerontology: Series B · doi:10.1093/geronb/gby065 · Cited by 279 · see citing works
A multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J., Dugan, J. M., Graziotin, D., Jacques, D., Waldner, F., Mietchen, D., Elkhatib, Y., Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.1 · Cited by 289 · see citing works
Open science saves lives: lessons from the COVID-19 pandemic
Besançon, L., Peiffer‐Smadja, N., Ségalas, C., Jiang, H., Masuzzo, P., Smout, C., Billy, E., Déforet, M., & Leyrat, C. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01304-y · Cited by 288 · see citing works
Mindfulness and acceptance approaches to sporting performance enhancement: a systematic review.
Noetel, M., Ciarrochi, J., Van Zanden, B., & Lonsdale, C. (2019) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2017.1387803 · Cited by 243 · see citing works
An open investigation of the reproducibility of cancer biology research
Errington, T. M., Iorns, E., Gunn, W., Tan, F. E., Lomax, J., & Nosek, B. A. (2014) · eLife · doi:10.7554/elife.04333 · Cited by 312 · see citing works
On the reproducibility of meta-analyses: six practical recommendations
Lakens, D., Hilgard, J., & Staaks, J. P. C. (2016) · BMC Psychology · doi:10.1186/s40359-016-0126-3 · Cited by 231 · see citing works
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 190 · see citing works
The changing forms and expectations of peer review
Horbach, S. P. J. M., & Halffman, W. (. W. (2018) · Research Integrity and Peer Review · doi:10.1186/s41073-018-0051-5 · Cited by 179 · see citing works
The Practical Alternative to the p Value Is the Correctly Used p Value
Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620958012 · Cited by 181 · see citing works
A Practical Guide for Improving Transparency and Reproducibility in Neuroimaging Research
Gorgolewski, K. J., & Poldrack, R. A. (2016) · PLoS Biology · doi:10.1371/journal.pbio.1002506 · Cited by 158 · see citing works
Seven Easy Steps to Open Science
Crüwell, S., van Doorn, J. B., Etz, A. J., Makel, M. C., Moshontz, H., Niebaum, J. C., Orben, A. C., Parsons, S. D., & Schulte‐Mecklenbeck, M. (2019) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000387 · Cited by 153 · see citing works
Mapping the universe of registered reports
Hardwicke, T. E., & Ioannidis, J. P. A. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0444-y · Cited by 147 · see citing works
Fields, E. C., & Kuperberg, G. R. (2019) · Psychophysiology · doi:10.1111/psyp.13468 · Cited by 143 · see citing works
Strengthening the Practice of Exercise and Sport-Science Research
Halperin, I., Vigotsky, A. D., Foster, C., & Pyne, D. B. (2017) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.2017-0322 · Cited by 132 · see citing works
Promoting Open Science to Increase the Trustworthiness of Evidence in Special Education
Cook, B. G., Lloyd, J., Mellor, D., Nosek, B. A., & Therrien, W. J. (2018) · The Exceptional Child · doi:10.1177/0014402918793138 · Cited by 115 · see citing works
Analysis of Open Data and Computational Reproducibility in Registered Reports in Psychology
Obels, P., Lakens, D., Coles, N. A., Gottfried, J., & Green, S. A. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920918872 · Cited by 127 · see citing works
Rejection odds and rejection ratios: A proposal for statistical practice in testing hypotheses
Bayarri, M. J., Benjamin, D. J., Berger, J. O., & Sellke, T. M. (2016) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2015.12.007 · Cited by 126 · see citing works
Current Research and Statistical Practices in Sport Science and a Need for Change
Bernards, J., Sato, K., Haff, G., & Bazyler, C. (2017) · Sports · doi:10.3390/sports5040087 · Cited by 120 · see citing works
Both Questionable and Open Research Practices Are Prevalent in Education Research
Makel, M. C., Hodges, J., Cook, B. G., & Plucker, J. A. (2021) · Educational Researcher · doi:10.3102/0013189x211001356 · Cited by 92 · see citing works
Methods for Heart Rate Variability Biofeedback (HRVB): A Systematic Review and Guidelines
Lalanza, J. F., Lorente, S., Bullich, R., García, C., Losilla, J.-M., & Capdevila, L. (2023) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-023-09582-6 · Cited by 102 · see citing works
Improving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2019) · Journal of Neuroscience Research · doi:10.1002/jnr.24340 · Cited by 83 · see citing works
Mega‐analysis methods in ENIGMA: The experience of the generalized anxiety disorder working group
Zugman, A., Harrewijn, A., Cardinale, E. M., Zwiebel, H., Freitag, G. F., Werwath, K., Bas-Hoogendam, J., Groenewold, N., Aghajani, M., Hilbert, K., et al. (2020) · Human Brain Mapping · doi:10.1002/hbm.25096 · Cited by 83 · see citing works
Preregistration Is Neither Sufficient nor Necessary for Good Science
Pham, M. T., & Oh, T. T. (2020) · Journal of Consumer Psychology · doi:10.1002/jcpy.1209 · Cited by 85 · see citing works
Actionable recommendations for narrowing the science-practice gap in open science
Aguinis, H., Banks, G. C., Rogelberg, S. G., & Cascio, W. F. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.02.007 · Cited by 81 · see citing works
Mitigating Illusory Results through Preregistration in Education
Gehlbach, H., & Robinson, C. D. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2017.1387950 · Cited by 79 · see citing works
The ability of different peer review procedures to flag problematic publications
Horbach, S. P. J. M., & Halffman, W. (2019) · Scientometrics · doi:10.1007/s11192-018-2969-2 · Cited by 73 · see citing works
The aura of charisma: A review on the embodiment perspective as signaling
Reh, S., Quaquebeke, N. V., & Giessner, S. R. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.001 · Cited by 73 · see citing works
Turner, E. H., Cipriani, A., Furukawa, T. A., Salanti, G., & de Vries, Y. A. (2022) · PLoS Medicine · doi:10.1371/journal.pmed.1003886 · Cited by 65 · see citing works
Matthes, J., Marquart, F., Naderer, B., Arendt, F., Schmuck, D., & Adam, K. (2015) · Communication Methods and Measures · doi:10.1080/19312458.2015.1096334 · Cited by 66 · see citing works
Marsden, E. J., Morgan‐Short, K., Trofimovich, P., & Ellis, N. C. (2018) · Language Learning · doi:10.1111/lang.12284 · Cited by 67 · see citing works
Increasing propensity to mind‐wander by transcranial direct current stimulation? A registered report
Boayue, N. M., Csifcsák, G., Aslaksen, P. M., Turi, Z., Antal, A., Groot, J. M., Hawkins, G. E., Forstmann, B. U., Opitz, A., Thielscher, A., et al. (2019) · European Journal of Neuroscience · doi:10.1111/ejn.14347 · Cited by 61 · see citing works
Improving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2018) · Brain and Behavior · doi:10.1002/brb3.1141 · Cited by 61 · see citing works
Open Communication Science: A Primer on Why and Some Recommendations for How
Lewis, N. A. (2019) · Communication Methods and Measures · doi:10.1080/19312458.2019.1685660 · Cited by 59 · see citing works
Flake, J. K. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1898962 · Cited by 58 · see citing works
Köhler, T., González‐Morales, M. G., Banks, G. C., O’Boyle, E. H., Allen, J. A., Sinha, R., Woo, S. E., & Gulick, L. M. V. (2020) · Industrial and Organizational Psychology · doi:10.1017/iop.2019.121 · Cited by 56 · see citing works
Open Science Saves Lives: Lessons from the COVID-19 PandemicPreprint
Besançon, L., Peiffer‐Smadja, N., Ségalas, C., Jiang, H., Masuzzo, P., Smout, C., Billy, É., Déforet, M., & Leyrat, C. (2020) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2020.08.13.249847 · Cited by 59 · see citing works
Stahl, C. (2015) · Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) · doi:10.1027/1618-3169/a000289 · Cited by 54 · see citing works
How to build up big team science: a practical guide for large-scale collaborations
Baumgartner, H. A., Alessandroni, N., Byers‐Heinlein, K., Frank, M. C., Hamlin, J. K., Söderström, M., Voelkel, J. G., Willer, R., Yuen, F. L., & Coles, N. A. (2023) · Royal Society Open Science · doi:10.1098/rsos.230235 · Cited by 51 · see citing works
Transparency, Reproducibility, and the Credibility of Economics ResearchPreprint
Christensen, G., & Miguel, E. A. (2017) · doi:10.31222/osf.io/9a3rw · Cited by 57 · see citing works
Preregistration of Analyses of Preexisting Data
Mertens, G., & Krypotos, A. (2019) · Psychologica Belgica · doi:10.5334/pb.493 · Cited by 49 · see citing works
Bloomfield, R. J., Rennekamp, K. M., & Steenhoven, B. A. (2018) · Journal of Accounting Research · doi:10.1111/1475-679x.12208 · Cited by 48 · see citing works
Byrne, J. A., Park, Y., Richardson, R. A. K., Pathmendra, P., Sun, M., & Stoeger, T. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkac1139 · Cited by 46 · see citing works
Practical Considerations for Navigating Registered Reports
Kiyonaga, A., & Scimeca, J. M. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2019.07.003 · Cited by 44 · see citing works
Increasing the Credibility of Political Science Research: A Proposal for Journal Reforms
Nyhan, B. J. (2015) · PS Political Science & Politics · doi:10.1017/s1049096515000463 · Cited by 43 · see citing works
Field, S. M., Wagenmakers, E.-J., Kiers, H. A. L., Hoekstra, R., Ernst, A. F., & van Ravenzwaaij, D. (2020) · Royal Society Open Science · doi:10.1098/rsos.181351 · Cited by 43 · see citing works
A Case for Multisite Second Language Acquisition Research: Challenges, Risks, and Rewards
Moranski, K., & Ziegler, N. (2020) · Language Learning · doi:10.1111/lang.12434 · Cited by 42 · see citing works
Grand, J. A., Rogelberg, S. G., Banks, G. C., Landis, R. S., & Tonidandel, S. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691618767883 · Cited by 45 · see citing works
Estimating the Replicability of Sports and Exercise Science Research
Murphy, J., Caldwell, A. R., Mesquida, C., Ladell, A. J. M., Martínez, A. E., Tual, A., Denys, A. T., Cameron, B., Hooren, B. V., Parr, B., et al. (2025) · Sports Medicine · doi:10.1007/s40279-025-02201-w · Cited by 48 · see citing works
Stracke, C. M. (2020) · Radical Solutions and Open Science · doi:10.1007/978-981-15-4276-3_2 · Cited by 24 · see citing works
Conditional equivalence testing: An alternative remedy for publication bias
Campbell, H., & Gustafson, P. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195145 · Cited by 40 · see citing works
A multi-disciplinary perspective on emergent and future innovations in peer review
Tennant, J., Dugan, J. M., Graziotin, D., Jacques, D., Waldner, F., Mietchen, D., Elkhatib, Y., Collister, L., Pikas, C. K., Crick, T., et al. (2017) · F1000Research · doi:10.12688/f1000research.12037.2 · Cited by 42 · see citing works
Standard Operating Procedures: A Safety Net for Pre-Analysis Plans
Lin, W., & Green, D. P. (2016) · Political Science and Politics · doi:10.1017/s1049096516000810 · Cited by 71 · see citing works
Rubin, M. (2017) · Review of General Psychology · doi:10.1037/gpr0000135 · Cited by 32 · see citing works
A practical guide for transparency in psychological sciencePreprint
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018) · doi:10.31234/osf.io/rtygm · Cited by 46 · see citing works
Robust Standards in Cognitive Science
Crüwell, S., Stefan, A. M., & Evans, N. J. (2019) · Computational Brain & Behavior · doi:10.1007/s42113-019-00049-8 · Cited by 34 · see citing works
Improving norms in research culture to incentivize transparency and rigor
Mellor, D. T. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1902329 · Cited by 36 · see citing works
Radical Solutions and Open Science
Burgos, D. (2020) · Lecture notes in educational technology · doi:10.1007/978-981-15-4276-3 · Cited by 43 · see citing works
Implications of the open science era for educational psychology research syntheses
Patall, E. A. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1897009 · Cited by 34 · see citing works
Promoting reproducibility with registered reports
(2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0034 · Cited by 32 · see citing works
Transparency, Reproducibility, and the Credibility of Economics Research
Christensen, G., & Miguel, E. A. (2016) · National Bureau of Economic Research · doi:10.3386/w22989 · Cited by 30 · see citing works
Gorman, D. M. (2017) · Children and Youth Services Review · doi:10.1016/j.childyouth.2017.07.009 · Cited by 29 · see citing works
Preregistration and registered reports
Reich, J. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1900851 · Cited by 30 · see citing works
Evaluating Registered Reports: A Naturalistic Comparative Study of Article Impact
Hopun, L. T., Singleton Thorn, F., Nosek, B. A., & Errington, T. M. (2017) · doi:10.31219/osf.io/5y8w7 · Cited by 29 · see citing works
Improving energy research practices: guidance for transparency, reproducibility and quality
Huebner, G. M., Fell, M. J., & Watson, N. E. (2021) · Buildings and Cities · doi:10.5334/bc.67 · Cited by 28 · see citing works
Putze, F., Putze, S., Sagehorn, M., Micek, C., & Solovey, E. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3490554 · Cited by 28 · see citing works
Beyond single paradigms, pipelines, and outcomes: Embracing multiverse analyses in psychophysiology
Clayson, P. E. (2024) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2024.112311 · Cited by 29 · see citing works
The Scientific Method in the Science of Machine LearningPreprint
Forde, J. Z., & Paganini, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.10922 · Cited by 33 · see citing works
Wyndham, J. M., Weisenberg, N., McCarty, C. A., Goldman, G. T., Brown, M., & Borenstein, J. (2022) · doi:10.1126/aaas.ade8203 · Cited by 29 · see citing works
Briker, R., & Gerpott, F. H. (2023) · Organizational Research Methods · doi:10.1177/10944281231210309 · Cited by 27 · see citing works
Crüwell, S., & Evans, N. (2021) · Royal Society Open Science · doi:10.1098/rsos.210155 · Cited by 13 · see citing works
Which Findings Should Be Published?
Frankel, A. P., & Kasy, M. (2022) · American Economic Journal Microeconomics · doi:10.1257/mic.20190133 · Cited by 26 · see citing works
Spellman, B. A., Gilbert, E. A., & Corker, K. S. (2018) · Wiley eBooks · doi:10.1002/9781119170174.epcn519 · Cited by 84 · see citing works
Pre-Registration in Social Psychology—a Discussion and Suggested TemplatePreprint
Veer, A. V. '., & Giner‐Sorolla, R. (2016) · doi:10.31234/osf.io/4frms · Cited by 31 · see citing works
How our approaches to assessing benefits and harms can be improved
Sena, E., & Currie, G. (2019) · Animal Welfare · doi:10.7120/09627286.28.1.107 · Cited by 23 · see citing works
A Practical Guide to Registered Reports for Economists
Arpinon, T., & Espinosa, R. (2022) · Journal of the Economic Science Association · doi:10.1007/s40881-022-00123-1 · Cited by 23 · see citing works
Combating the scientific decline effect with confidence (intervals)
Groppe, D. M. (2016) · Psychophysiology · doi:10.1111/psyp.12616 · Cited by 23 · see citing works
Embracing Open Science and Transparency in Health Psychology
Hagger, M. S. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1605614 · Cited by 23 · see citing works
Opening the Door to Registered Reports: Census of Journals Publishing Registered Reports (2013–2020)
Montoya, A. K., Krenzer, W. L. D., & Fossum, J. L. (2021) · Collabra Psychology · doi:10.1525/collabra.24404 · Cited by 22 · see citing works
Improving Human Interaction Research through Ecological Grounding
Albert, S., & de Ruiter, J. P. (2018) · Collabra Psychology · doi:10.1525/collabra.132 · Cited by 21 · see citing works
Mapping the Universe of Registered ReportsPreprint
Hardwicke, T. E., & Ioannidis, J. P. A. (2018) · doi:10.31222/osf.io/fzpcy · Cited by 21 · see citing works
Doing better leadership science via replications and registered reports
Carsten, M. K., Clapp‐Smith, R., Haslam, S. A., Bastardoz, N., Gooty, J., Connelly, S., & Spain, S. M. (2023) · The Leadership Quarterly · doi:10.1016/j.leaqua.2023.101712 · Cited by 22 · see citing works
Efendić, E., & van Zyl, L. E. (2019) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v45i0.1607 · Cited by 20 · see citing works
Results-blind review: a masked crusader for science
Woznyj, H. M., Grenier, K., Ross, R. L., Banks, G. C., & Rogelberg, S. G. (2018) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2018.1496081 · Cited by 20 · see citing works
Eben, C., Bőthe, B., Brevers, D., Clark, L., Grubbs, J. B., Heirene, R. M., Kräplin, A., Lewczuk, K., Palmer, L., Perales, J. C., et al. (2023) · Journal of Behavioral Addictions · doi:10.1556/2006.2023.00052 · Cited by 19 · see citing works
Researcher Degrees of Freedom in the Psychology of Religion
Charles, S. J., Bartlett, J. E., Messick, K. J., Coleman, T. J., III, & Uzdavines, A. (2019) · International Journal for the Psychology of Religion · doi:10.1080/10508619.2019.1660573 · Cited by 18 · see citing works
A Meta-analysis of the relationship between growth mindset and grit
Lam, K. K. L., & Zhou, M. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.104872 · Cited by 25 · see citing works
EJPA Introduces Registered Reports as New Submission Format
Greiff, S., & Allen, M. S. (2018) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000492 · Cited by 18 · see citing works
- Authors
- D. Chambers, C., 1 Cardiff University Brain Research Imaging Centre, S. O. P., Feredoes, E., D. Muthukumaraswamy, S., & J. Etchells, P.
- Affiliations
- University of Reading (GB)
- ORCID
- Feredoes, Eva
- Venue
- AIMS Neuroscience
- Published
- 2014
- Volume
- 1
- Issue
- 1
- Pages
- 4-17
- ISSN
- 2373-7972 (ISSN-L)
- OpenAlex
- W2072600048
- Cited by
- 213
- References
- 29
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 9 · 2023: 19 · 2022: 19 · 2021: 25 · 2020: 23 · 2019: 25 · 2018: 35 · 2017: 17 · 2016: 16 · 2015: 8 · 2014: 2
- FWCI
- 27.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- AIMS neuroscience (publishedVersion, cc-by)
- ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Biotechnology and Biological Sciences Research Council (BB/K008277/1, BB/C519854/2)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- questionable research practices, p-hacking, HARKing, replication, publication bias, statistical power, data sharing, research incentives, reproducibility, transparency, brain research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
