Protecting against researcher bias in secondary data analysis: challenges and potential solutions
Open accessAbstract
Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There is clear evidence from both MR and the natural experiment of an increase in the school leaving age that an increase in the number of years in education has a causal protective effect on health behaviours such as smoking.
“For example, there is clear evidence from both MR and the natural experiment of an increase in the school leaving age that an increase in the number of years in education has a causal protective effect on health behaviours such as smoking.156–160”
From Mendelian randomization (2022) · cites this paper as [158]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable
Protzko, J., Krosnick, J. A., Nelson, L. D., Nosek, B. A., Axt, J. R., Berent, M., Buttrick, N. R., DeBell, M., Ebersole, C. R., Lundmark, S., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01749-9 · Cited by 90 · see citing works
RetractedConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397 · see citing works
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 2,161 · see citing works
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · see citing works
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989 · see citing works
Equivalence Testing for Psychological Research: A Tutorial
Lakens, D., Scheel, A. M., & Isager, P. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918770963 · Cited by 1,560 · cited 1× · see citing works
“, Bayesian methods [58] or equivalence tests, which determine whether an observed effect is surprisingly small [52, 59, 60].”
MethodPublication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542 · see citing works
The association between adolescent well-being and digital technology use
Orben, A., & Przybylski, A. K. (2019) · Nature Human Behaviour · doi:10.1038/s41562-018-0506-1 · Cited by 1,500 · cited 4× · see citing works
“First, researchers now have increasing access to high-dimensional datasets that offer a multitude of ways to analyse the same data [6].”
BackgroundInfluentialIncreasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,388 · see citing works
Negative results are disappearing from most disciplines and countries
Fanelli, D. (2012) · Scientometrics · doi:10.1007/s11192-011-0494-7 · Cited by 1,248 · see citing works
Problems of reporting genetic associations with complex outcomes
Colhoun, H. M., McKeigue, P., & Smith, G. D. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12715-8 · Cited by 1,203 · see citing works
Rubin, D. B. (2007) · Statistics in Medicine · doi:10.1002/sim.2739 · Cited by 1,047 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
Duncan, L. E., & Keller, M. C. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11020191 · Cited by 1,032 · see citing works
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 2× · see citing works
“While this is a positive outcome for research integrity, null results are less likely to be published [13, 41, 42] and cited [11], which could disadvantage researchers’ careers.”
BackgroundResultAccounting for missing data in statistical analyses: multiple imputation is not always the answer
Hughes, R. A., Heron, J., Sterne, J. A. C., & Tilling, K. (2019) · International Journal of Epidemiology · doi:10.1093/ije/dyz032 · Cited by 802 · see citing works
Schäfer, T., & Schwarz, M. A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00813 · Cited by 774 · see citing works
Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E. C., Bahník, Š., Bai, F., Bannard, C., Bonnier, E., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245917747646 · Cited by 773 · see citing works
Bayes factor design analysis: Planning for compelling evidence
Schönbrodt, F. D., & Wagenmakers, E. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1230-y · Cited by 732 · see citing works
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730 · see citing works
Border, R., Johnson, E. C., Evans, L. M., Smolen, A., Berley, N., Sullivan, P. F., & Keller, M. C. (2019) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.18070881 · Cited by 629 · see citing works
Open science challenges, benefits and tips in early career and beyond
Allen, C., & Mehler, D. M. A. (2019) · PLOS Biology · doi:10.1371/journal.pbio.3000246 · Cited by 608 · see citing works
Registered Reports: A new publishing initiative at Cortex
Chambers, C. (2012) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 574 · see citing works
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499 · see citing works
Using genetic data to strengthen causal inference in observational research
Pingault, J.-B., O’Reilly, P. F., Schoeler, T., Ploubidis, G. B., Rijsdijk, F., & Dudbridge, F. (2018) · Nature Reviews Genetics · doi:10.1038/s41576-018-0020-3 · Cited by 444 · see citing works
Self-citationWhy Hypothesis Testers Should Spend Less Time Testing Hypotheses
Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021) · Perspectives on Psychological Science · doi:10.1177/1745691620966795 · Cited by 382 · cited 1× · see citing works
“Furthermore, without a strong theoretical background, hypotheses will be arbitrary and could lead to unhelpful inferences [32, 33], and so should be avoided in novel areas of research.”
BackgroundAn Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
Scheel, A. M., Schijen, M., & Lakens, D. (2020) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211007467 · Cited by 359 · see citing works
Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time
Kaplan, R. M., & Irvin, V. L. (2015) · PLOS ONE · doi:10.1371/journal.pone.0132382 · Cited by 356 · see citing works
Preregistration Is Hard, And Worthwhile
Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., Veer, A. V. '., & Vazire, S. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.07.009 · Cited by 325 · see citing works
How scientists fool themselves – and how they can stop
Nuzzo, R. (2015) · Nature · doi:10.1038/526182a · Cited by 296 · see citing works
Gopalakrishna, G., ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022) · PLOS ONE · doi:10.1371/journal.pone.0263023 · Cited by 296 · see citing works
Improving inferences about null effects with Bayes factors and equivalence tests.
Lakens, D., McLatchie, N., Isager, P. M., Scheel, A. M., & Dienes, Z. (2020) · The Journals of Gerontology: Series B · doi:10.1093/geronb/gby065 · Cited by 279 · see citing works
Blind analysis: Hide results to seek the truth
MacCoun, R., & Perlmutter, S. (2015) · Nature · doi:10.1038/526187a · Cited by 218 · see citing works
Recommendations for Increasing the Transparency of Analysis of Preexisting Data Sets
Weston, S. J., Ritchie, S. J., Rohrer, J. M., & Przybylski, A. K. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919848684 · Cited by 206 · see citing works
Specification Curve: Descriptive and Inferential Statistics on All Reasonable SpecificationsPreprint
Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2694998 · Cited by 199 · see citing works
Claesen, A., Gomes, S., Tuerlinckx, F., & Vanpaemel, W. (2021) · Royal Society Open Science · doi:10.1098/rsos.211037 · Cited by 197 · see citing works
Invited Commentary: The Need for Cognitive Science in Methodology
Greenland, S. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx259 · Cited by 163 · see citing works
Probing Birth-Order Effects on Narrow Traits Using Specification-Curve Analysis
Rohrer, J. M., Egloff, B., & Schmukle, S. C. (2017) · Psychological Science · doi:10.1177/0956797617723726 · Cited by 146 · see citing works
Is Preregistration Worthwhile?
Szollosi, A., Kellen, D., Navarro, D., Shiffrin, R. M., van Rooij, I., Zandt, T. V., & Donkin, C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.11.009 · Cited by 146 · see citing works
Quintana, D. S. (2020) · eLife · doi:10.7554/elife.53275 · Cited by 142 · see citing works
Giudice, M. D., & Gangestad, S. W. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920954925 · Cited by 140 · see citing works
Preregistration of secondary data analysis: A template and tutorial
van den Akker, O. R., Weston, S. J., Campbell, L., Chopik, B., Damian, R. I., Davis-Kean, P. E., Hall, A., Kosie, J. E., Kruse, E. T., Olsen, J., et al. (2021) · Meta-Psychology · doi:10.15626/mp.2020.2625 · Cited by 131 · see citing works
Bishop, D. (2019) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021819886519 · Cited by 109 · see citing works
de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P. J., Munafo, M. R., & Bastiaansen, J. A. (2018) · Psychological Medicine · doi:10.1017/s0033291718001873 · Cited by 100 · see citing works
Exploratory reports: A new article type for Cortex
McIntosh, R. D. (2017) · Cortex · doi:10.1016/j.cortex.2017.07.014 · Cited by 87 · see citing works
Dataset decay and the problem of sequential analyses on open datasets
Thompson, W. H., Wright, J., Bissett, P. G., & Poldrack, R. A. (2020) · eLife · doi:10.7554/elife.53498 · Cited by 70 · see citing works
Split-Sample Strategies for Avoiding False Discoveries
Anderson, M. L., & Magruder, J. R. (2017) · National Bureau of Economic Research · doi:10.3386/w23544 · Cited by 65 · see citing works
The past, present, and future of Registered ReportsPreprint
Chambers, C., & Tzavella, L. (2020) · doi:10.31222/osf.io/43298 · Cited by 63 · see citing works
Does preregistration improve the credibility of research findings?
Rubin, M. (2020) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.16.4.p376 · Cited by 61 · see citing works
specr: Conducting and Visualizing Specification Curve Analyses
Masur, P. K., & Scharkow, M. (2020) · doi:10.32614/cran.package.specr · Cited by 58 · see citing works
High Replicability of Newly-Discovered Social-behavioral Findings is AchievablePreprint
Protzko, J., Krosnick, J. A., Nelson, L. D., Nosek, B. A., Axt, J. R., Berent, M. K., Buttrick, N., DeBell, M., Ebersole, C. R., Lundmark, S., et al. (2020) · doi:10.31234/osf.io/n2a9x · Cited by 48 · see citing works
Preregistration of secondary data analysis: A template and tutorialPreprint
van den Akker, O. R., Weston, S. J., Campbell, L., et al. (2019) · doi:10.31234/osf.io/hvfmr · Cited by 46 · see citing works
Practical Considerations for Navigating Registered Reports
Kiyonaga, A., & Scimeca, J. M. (2019) · Trends in Neurosciences · doi:10.1016/j.tins.2019.07.003 · Cited by 44 · see citing works
A computational reproducibility study of PLOS ONE articles featuring longitudinal data analyses
Seibold, H., Czerny, S., Decke, S., Dieterle, R., Eder, T., Fohr, S., Hahn, N., Hartmann, R., Heindl, C., Kopper, P., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251194 · Cited by 37 · see citing works
Gopalakrishna, G., ter Riet, G., Vink, G., et al. (2021) · doi:10.31222/osf.io/vk9yt · Cited by 37 · see citing works
Preregistration of Epidemiologic Studies
Vandenbroucke, J. P. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181e942b8 · Cited by 32 · see citing works
Blind Analysis as a Correction for Confirmatory Bias in Physics and in Psychology
MacCoun, R. J., & Perlmutter, S. (2017) · doi:10.1002/9781119095910.ch15 · Cited by 30 · see citing works
Lash, T. L. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181e9bba6 · Cited by 29 · see citing works
Greenland, S. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym174 · Cited by 18 · see citing works
What should a preregistration contain?Preprint
McPhetres, J. (2020) · doi:10.31234/osf.io/cj5mh · Cited by 16 · see citing works
Goldstein, B. A. (2020) · Clinical Trials · doi:10.1177/1740774520931211 · Cited by 13 · see citing works
Lawlor, D. A. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym168 · Cited by 13 · see citing works
Enabling Confirmatory Secondary Data Analysis by Logging Data Checkout
Scott, K. M., & Struhl, M. K. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918815849 · Cited by 9 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [26]Small DS, Firth D, Keele L, et al. Protocol for a study of the effect of surface mining in central appalachia on adverse birth outcomes. arXiv.org. 2020
- [27]Deshpande SK, Hasegawa RB, Weiss J, Small DS. Protocol for an observational study on the effects of playing football in adolescence on mental health in early adulthood. arXiv preprint 2018
Cited by
Sorted by citations
Sanderson, E., Glymour, M. M., Holmes, M. V., Kang, H., Morrison, J., Munafò, M. R., Palmer, T., Schooling, C. M., Wallace, C., Zhao, Q., et al. (2022) · Nature Reviews Methods Primers · doi:10.1038/s43586-021-00092-5 · Cited by 2,229 · see citing works
Skrivankova, V. W., Richmond, R. C., Woolf, B. A. R., Davies, N. M., Swanson, S. A., VanderWeele, T. J., Timpson, N. J., Higgins, J. P. T., Dimou, N., Langenberg, C., et al. (2021) · BMJ · doi:10.1136/bmj.n2233 · Cited by 1,648 · see citing works
Guidelines for performing Mendelian randomization investigations
Burgess, S., Davey Smith, G., Davies, N. M., Dudbridge, F., Gill, D., Glymour, M. M., Hartwig, F. P., Holmes, M. V., Minelli, C., Relton, C. L., et al. (2020) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15555.2 · Cited by 1,273 · see citing works
Guidelines for performing Mendelian randomization investigations: update for summer 2023
Burgess, S., Smith, G. D., Davies, N. M., Dudbridge, F., Gill, D., Glymour, M. M., Hartwig, F. P., Kutalik, Z., Holmes, M. V., Minelli, C., et al. (2023) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15555.3 · Cited by 737 · see citing works
Reducing bias, increasing transparency and calibrating confidence with preregistration
Hardwicke, T. E., & Wagenmakers, E. (2023) · Nature Human Behaviour · doi:10.1038/s41562-022-01497-2 · Cited by 190 · see citing works
Cardiac sympathetic-vagal activity initiates a functional brain–body response to emotional arousal
Candia‐Rivera, D., Catrambone, V., Thayer, J. F., Gentili, C., & Valenza, G. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2119599119 · Cited by 163 · see citing works
Secondary Qualitative Research Methodology Using Online Data within the Context of Social Sciences
Cheong, H.-I., Lyons, A., Houghton, R., & Majumdar, A. (2023) · International Journal of Qualitative Methods · doi:10.1177/16094069231180160 · Cited by 228 · see citing works
The Aging of Polymers under Electromagnetic Radiation
Maraveas, C., Kyrtopoulos, I.-V., Arvanitis, K. G., & Bartzanas, T. (2024) · Polymers · doi:10.3390/polym16050689 · Cited by 81 · see citing works
IMPACT: Studies in Language, Culture and Society
(2013) · Impact · doi:10.1075/impact · Cited by 75 · see citing works
A genetically informed Registered Report on adverse childhood experiences and mental health
Baldwin, J. R., Sallis, H., Schoeler, T., Taylor, M. J., Kwong, A. S. F., Tielbeek, J. J., Barkhuizen, W., Warrier, V., Howe, L. D., Danese, A., et al. (2022) · Nature Human Behaviour · doi:10.1038/s41562-022-01482-9 · Cited by 72 · see citing works
Self-citationCourtney, D. B., Watson, P., Krause, K. R., Chan, B. W. C., Bennett, K., Gunlicks-Stoessel, M., Rodak, T., Neprily, K., Zentner, T., & Szatmari, P. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.46331 · Cited by 63 · see citing works
Cullen, N. C., Novák, P., Tosun, D., Kováčech, B., Hanes, J., Kontseková, E., Fresser, M., Ropele, S., Feldman, H., Schmidt, R., et al. (2023) · EBioMedicine · doi:10.1016/j.ebiom.2023.104923 · Cited by 55 · see citing works
Filho, W. L., Wall, T., Williams, K. A., Dinis, M. A. P., Fernández, R. M. G., Mazhar, M. U., & Gatto, A. (2025) · Journal of Environmental Management · doi:10.1016/j.jenvman.2025.125008 · Cited by 55 · see citing works
Knowing What We're Talking About
Peters, G.-J., & Crutzen, R. (2024) · Meta-Psychology · doi:10.15626/mp.2022.3638 · Cited by 50 · see citing works
Dhiman, P., Ma, J., Andaur Navarro, C. L., Speich, B., Bullock, G., Damen, J. A. A., Hooft, L., Kirtley, S., Riley, R. D., Van Calster, B., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.03.012 · Cited by 41 · see citing works
Toward Open and Reproducible Epidemiology
Mathur, M. B., & Fox, M. P. (2023) · American Journal of Epidemiology · doi:10.1093/aje/kwad007 · Cited by 46 · see citing works
Jones, L. B., Jadhakhan, F., & Falla, D. (2024) · Applied Ergonomics · doi:10.1016/j.apergo.2023.104216 · Cited by 34 · see citing works
Using preregistration as a tool for transparent fNIRS study design
Schroeder, P. A., Artemenko, C., Kosie, J. E., Cockx, H., Stute, K., Pereira, J., Klein, F., & Mehler, D. M. A. (2023) · Neurophotonics · doi:10.1117/1.nph.10.2.023515 · Cited by 35 · see citing works
Comprehensive review on single-cell RNA sequencing: A new frontier in Alzheimer's disease research
Jin, W., Pei, J., Roy, J. R., Jayaraman, S., Ahalliya, R. M., Kanniappan, G. V., Mironescu, M., & Palanisamy, C. P. (2024) · Ageing Research Reviews · doi:10.1016/j.arr.2024.102454 · Cited by 30 · see citing works
Treur, J. L., Lukas, E., Sallis, H., & Wootton, R. E. (2024) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796024000623 · Cited by 21 · see citing works
Self-citationQuality Indicators of Secondary Data Analyses in Special Education Research: A Preregistration Guide
Lombardi, A. R., Rifenbark, G. G., & Taconet, A. V. (2022) · Exceptional Children · doi:10.1177/00144029221141029 · Cited by 20 · see citing works
The evolution of computational research in a data-centric world
Deshpande, D., Chhugani, K., Ramesh, T., Pellegrini, M. A., Shifman, S., Abedalthagafi, M., Alqahtani, S. A., Ye, C., Liu, X. S., Leek, J. T., et al. (2024) · Cell · doi:10.1016/j.cell.2024.07.045 · Cited by 18 · see citing works
Balafoutas, L., Celse, J., Karakostas, A., & Umashev, N. (2024) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2024.102327 · Cited by 18 · see citing works
Black, L., Humphrey, N., & Marquez, J. (2023) · Royal Society Open Science · doi:10.1098/rsos.221230 · Cited by 16 · see citing works
Relationship between grammar and schizophrenia: a systematic review and meta-analysis
Elleuch, D., Chen, Y., Luo, Q., & Palaniyappan, L. K. (2025) · Communications Medicine · doi:10.1038/s43856-025-00944-1 · Cited by 21 · see citing works
Automatic synthesis of insects bioacoustics using machine learning: a systematic review
Kyalo, H., Tonnang, H. E. Z., Egonyu, J. P., Olukuru, J., Tanga, C. M., & Senagi, K. (2025) · International Journal of Tropical Insect Science · doi:10.1007/s42690-024-01406-2 · Cited by 16 · see citing works
Ten (not so) simple rules for clinical trial data-sharing
Pellen, C., Louarn, A. L., Spurrier‐Bernard, G., Decullier, É., Chrétien, J., Rosenthal, É., Goff, G. L., Moher, D., Ioannidis, J. P. A., & Naudet, F. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010879 · Cited by 15 · see citing works
Gergov, V., Prevendar, T., Vousoura, E., Ulberg, R., Dahl, H., Feller, C., Jacobsen, C. F., Karain, A., Milic, B., Poznyak, E., et al. (2023) · Adolescent Research Review · doi:10.1007/s40894-023-00228-6 · Cited by 15 · see citing works
Fields, N., Leonard, A., Mutembei, M., Nganga, A., Martindale, L., Bergman, M., Kaoma, M., Howells, M., & Brown, E. D. (2025) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2025.115917 · Cited by 12 · see citing works
Gokhale, S., Taylor, D., Gill, J., Hu, Y., Zeps, N., Lequertier, V., Teede, H. J., & Enticott, J. (2023) · Digital Health · doi:10.1177/20552076231177497 · Cited by 12 · see citing works
Leducq, S., Zaki, F., Hollestein, L. M., Apfelbacher, C., Ponna, N. P., Mazmudar, R. S., & Gran, S. (2024) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2024.111341 · Cited by 15 · see citing works
Sjukriana, J., Hanafiah, M. H., Asyraff, M. A., & Kusumah, A. H. G. (2024) · International Journal of Event and Festival Management · doi:10.1108/ijefm-06-2024-0077 · Cited by 12 · see citing works
Abba, A., Sankarannair, S., Gautam, R., & Kaparaju, P. (2025) · Scientific Reports · doi:10.1038/s41598-025-10507-y · Cited by 12 · see citing works
Decisions, Decisions, Decisions: An Ethnographic Study of Researcher Discretion in Practice
van Drimmelen, T., Slagboom, M. N., Reis, R., Bouter, L. M., & van der Steen, J. T. (2024) · Science and Engineering Ethics · doi:10.1007/s11948-024-00481-5 · Cited by 11 · see citing works
Abbott, I., & Whitley, G. A. (2023) · International Emergency Nursing · doi:10.1016/j.ienj.2023.101376 · Cited by 11 · see citing works
Amit, S., Levermore, R., Noor, S. S., & Kafy, A. ‐. A. (2024) · Business Strategy & Development · doi:10.1002/bsd2.70005 · Cited by 10 · see citing works
Organic food research: Key contributors, research hotspots, and emerging trends
Azizan, A., Awal, A., Zain, W. Z. W. M., Hairoman, N., Endrini, S., & Hassan, F. (2025) · Applied Food Research · doi:10.1016/j.afres.2025.101109 · Cited by 10 · see citing works
Yoon, C. Y., Joseph, T., Moussa, G., Voss, T., Ledoux, T. A., & Johnston, C. A. (2023) · Appetite · doi:10.1016/j.appet.2023.107073 · Cited by 9 · see citing works
Николаевич, К. М., & Khasanova, D. R. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1288718 · Cited by 8 · see citing works
What is Beyond Measurement for Social Cohesion?
Tok, T. Q. H., Woods, O., & Kong, L. (2024) · Social Indicators Research · doi:10.1007/s11205-024-03430-8 · Cited by 8 · see citing works
Open Research: Examples of good practice, and resources across disciplines (2025 edition)Preprint
Farran, E. K., Silverstein, P., Ameen, A. A., Misheva, I., & Gilmore, C. K. (2020) · doi:10.31219/osf.io/3r8hb · Cited by 7 · see citing works
Busenhart, D. M., Schätzle, M. A., Eliades, T., & Papageorgiou, S. N. (2024) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12756 · Cited by 8 · see citing works
Edbais, A., & Hossain, M. (2025) · Sustainability · doi:10.3390/su17020477 · Cited by 7 · see citing works
Property tax reassessment among local authorities: the implementation and its key challenges
Senawi, A., & Osmadi, A. (2023) · Journal of Financial Management of Property and Construction · doi:10.1108/jfmpc-04-2022-0022 · Cited by 7 · see citing works
Seasonal Association With Hypothermia in Combat Trauma
Eisenhauer, I. F., April, M. D., Rizzo, J. A., Fisher, A. D., Maddry, J. K., Bebarta, V. S., & Schauer, S. G. (2023) · Military Medicine · doi:10.1093/milmed/usad451 · Cited by 6 · see citing works
Sauter, E. R., Butera, G., & Agurs‐Collins, T. (2024) · Cancers · doi:10.3390/cancers16193296 · Cited by 6 · see citing works
Expressive pragmatic language in mood and psychotic disorders: a systematic review and meta-analysis
Meister, F., Silva, M. S., Melshin, G., Mouslih, C. E., Zaher, F., Sattari, R., Wei, H. T., Mekideche, N., Bambini, V., Voppel, A., et al. (2026) · Schizophrenia · doi:10.1038/s41537-026-00733-2 · Cited by 6 · see citing works
The impact of investigator bias in nutrition research
Shyam, S., & Salas‐Salvadó, J. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1513343 · Cited by 6 · see citing works
Tang, X., Fan, C., Wang, C., Wang, W., Chen, Z., Xu, C., & Ling, L. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1032217 · Cited by 6 · see citing works
UPAYA PARTAI POLITIK DALAM MEMANFAATKAN DEMOKRASI VIRTUAL DI ERA DIGITAL
Boestam, A. B., Prakosa, A., & Avianto, B. N. (2023) · Jurnal Pustaka Komunikasi · doi:10.32509/pustakom.v6i1.2281 · Cited by 6 · see citing works
Syntax and Schizophrenia: A Meta-Analysis of Comprehension and ProductionPreprint
Elleuch, D., Chen, Y., Luo, Q., & Palaniyappan, L. K. (2024) · medRxiv · doi:10.1101/2024.10.26.24316171 · Cited by 6 · see citing works
Muriithi, F. G., Easter, C., Osoti, A. O., Qureshi, Z. P., Devall, A., & Coomarasamy, A. (2025) · Frontiers in Global Women s Health · doi:10.3389/fgwh.2025.1481495 · Cited by 5 · see citing works
Craig, P., Campbell, M., Deidda, M., Dundas, R., Green, J., Katikireddi, S. V., Lewsey, J., Ogilvie, D., de Vocht, F., & White, M. J. (2025) · Public Health Research · doi:10.3310/jtyw6582 · Cited by 5 · see citing works
Николаевич, К. М., & Khasanova, D. R. (2024) · S S Korsakov Journal of Neurology and Psychiatry · doi:10.17116/jnevro202412403255 · Cited by 5 · see citing works
Fellnhofer, K., Wennberg, K. J., Allison, T. H., Arenius, P. M., Lévesque, M., Gish, J. J., & Pollack, J. M. (2026) · Entrepreneurship Theory and Practice · doi:10.1177/10422587251401175 · Cited by 5 · see citing works
Αντωνιάδου, Μ., Sokratous, S., Dimitriou, E. C., & Tzoutzas, I. G. (2025) · Hygiene · doi:10.3390/hygiene5020024 · Cited by 4 · see citing works
Nhongo, D., Holt, A., Bail, K. S., & Flenady, T. (2024) · Collegian Journal of the Royal College of Nursing Australia · doi:10.1016/j.colegn.2024.04.004 · Cited by 4 · see citing works
Giannakopoulou, T. T., Papadopoulou, A. K., Busenhart, D. M., Eliades, T., & Papageorgiou, S. N. (2025) · Journal of the World Federation of Orthodontists · doi:10.1016/j.ejwf.2025.05.001 · Cited by 5 · see citing works
Necessary conditions for valid causal inference from observational data
Cox, L. A. (2025) · Critical Reviews in Toxicology · doi:10.1080/10408444.2025.2611827 · Cited by 4 · see citing works
Meier, E. M., Rubin, N., Dermody, S. S., Tessier, K., Hecht, S. S., Murphy, S. E., Jensen, J. A., Donny, E. C., Al’Absi, M., Drobes, D. J., et al. (2022) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntac231 · Cited by 4 · see citing works
Using preregistration as a tool for transparent fNIRS study designPreprint
Schroeder, P. A., Artemenko, C., Kosie, J. E., Cockx, H., Stute, K., Pereira, J., Klein, F., & Mehler, D. M. A. (2022) · doi:10.31222/osf.io/zfawx · Cited by 4 · see citing works
Huertas‐Zurriaga, A., Giménez‐Díez, D., & Leyva‐Moral, J. M. (2025) · Healthcare · doi:10.3390/healthcare13020168 · Cited by 3 · see citing works
Dental Care Utilization Among Middle Eastern and North African Children in the United States
Brown, K. K., Dallo, F. J., & Kindratt, T. B. (2025) · Journal of Public Health Dentistry · doi:10.1111/jphd.12667 · Cited by 3 · see citing works
Nguemeleu, E. T., Karemere, H. B., Sia, D., & Kapiteni, W. (2023) · Journal of Epidemiology and Global Health · doi:10.1007/s44197-023-00147-8 · Cited by 3 · see citing works
Hamsar, M. N., Sazili, A. Q., & Tohid, S. F. M. (2024) · Journal of Agriculture and Food Research · doi:10.1016/j.jafr.2024.101026 · Cited by 3 · see citing works
Beyond Boundaries: The Role of Learning Types in Shaping MOOC Learner Engagement and Progression
John, H., Kerr, J., & Andrieux, G. (2024) · Journal of Interactive Media in Education · doi:10.5334/jime.890 · Cited by 3 · see citing works
Makhanya, M. T. B. (2024) · International Journal of Research in Business and Social Science (2147-4478) · doi:10.20525/ijrbs.v13i7.3617 · Cited by 3 · see citing works
van den Akker, O. R., Thibault, R. T., Ioannidis, J. P. A., Schorr, S. G., & Strech, D. (2025) · Royal Society Open Science · doi:10.1098/rsos.241364 · Cited by 3 · see citing works
Eichhorn, S., Mitze, F. N., Wagner, F., Seuthe, I. M. C., Brinks, R., Zimmermann, A., Dastur, M., Ladenbauer, J., & Park, J. J.-H. (2026) · PLOS One · doi:10.1371/journal.pone.0339647 · Cited by 2 · see citing works
McGuire, A., Smith, D. W., & Kilpatrick, D. G. (2024) · Development and Psychopathology · doi:10.1017/s0954579424001354 · Cited by 2 · see citing works
Karadağ, E., & Ciftci, S. K. (2025) · Higher Education · doi:10.1007/s10734-025-01404-4 · Cited by 2 · see citing works
Grégoire, C., Cousson‐Gélie, F., & Baussard, L. (2025) · International Journal of Clinical and Health Psychology · doi:10.1016/j.ijchp.2025.100590 · Cited by 2 · see citing works
Gower, H., Doran, N., Hamza, N., & Smith, S. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-096087 · Cited by 2 · see citing works
Navigating Ambiguity and Uncertainty in End-of-Life Care Research Practice
van Drimmelen, T., Slagboom, M. N., Reis, R., Bouter, L. M., & van der Steen, J. T. (2025) · Journal of Empirical Research on Human Research Ethics · doi:10.1177/15562646251377121 · Cited by 2 · see citing works
Hamdallah, F. A. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.102740 · Cited by 2 · see citing works
Lüchinger, R., Blondon, K., Perron, N. J., & Audétat, M. (2026) · Journal of Healthcare Leadership · doi:10.2147/jhl.s584769 · Cited by 2 · see citing works
What Makes Parents of Young Children Stressed? A Systematic Review of Quantitative Studies
d’Orsi, D., Garraio, C., Santos, R., Matias, M., Veríssimo, M., & Diniz, E. (2026) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-026-00573-7 · Cited by 2 · see citing works
Mahmud, A. (2021) · Improving Schools · doi:10.1177/13654802211031089 · Cited by 3 · see citing works
Pedraza, C. A. A., & Park, C. J. (2022) · Genealogy · doi:10.3390/genealogy6040091 · Cited by 2 · see citing works
Fitria, N., Ikramah, N., Sukma, A., & Nasif, H. (2024) · Journal of Obstetrics Gynecology and Cancer Research · doi:10.30699/jogcr.9.3.276 · Cited by 2 · see citing works
Sawyer, J. M., Coles, D. C., & Perkins, N. H. (2025) · Journal of Public Child Welfare · doi:10.1080/15548732.2025.2487913 · Cited by 2 · see citing works
Mastromarchi, P., May, S., Ali, N., McLean, S. M., & Peat, G. (2025) · Physiotherapy · doi:10.1016/j.physio.2025.101815 · Cited by 2 · see citing works
Roettger, T. B., Askelund, A. D., Birkenæs, V., Bjørndal, L. D., Bochyńska, A., Glaser, B. D., Kalandadze, T., Korbmacher, M., Malovic, I., Mayor, J., et al. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02601-6 · Cited by 2 · see citing works
Varah, A., Purvis, A., & Palma, A. D. (2025) · Ecological Solutions and Evidence · doi:10.1002/2688-8319.70070 · Cited by 2 · see citing works
Dope stereotypes: When perception runs south and prevalence points north
Pitsch, W., & Christiansen, A. V. (2025) · Performance Enhancement & Health · doi:10.1016/j.peh.2025.100398 · Cited by 2 · see citing works
The Use of Peer Assessment as an Inclusive Learning Strategy in Higher Education Institutions
Meletiadou, E. (2021) · Advances in educational marketing, administration, and leadership book series · doi:10.4018/978-1-7998-8579-5.ch001 · Cited by 2 · see citing works
The cross‐over of statistical thinking and practices: A pandemic catalyst
Garrett, A. (2022) · Pharmaceutical Statistics · doi:10.1002/pst.2221 · Cited by 2 · see citing works
Strategies for integrating martial arts-based mindfulness into educational settings
Kwok, W. W.-Y. (2024) · doi:10.17760/d20649864 · Cited by 2 · see citing works
Open Science Practices in Psychiatric Genetics: A Primer
Kępińska, A. P., Johnson, J., & Huckins, L. M. (2023) · Biological Psychiatry Global Open Science · doi:10.1016/j.bpsgos.2023.08.007 · Cited by 2 · see citing works
Ghali, A. N., Shaikh, Y., Elaydi, A., Irfan, B., Qasim, T. S., & Saleh, K. (2025) · Bioethics · doi:10.1111/bioe.13425 · Cited by 1 · see citing works
Australian neonatal nurses’ professional quality of life: A descriptive cross-sectional study
Lowe, P. L., Jakimowicz, S., Levett-Jones, T. L., & Demant, D. (2025) · Journal of Child Health Care · doi:10.1177/13674935251339351 · Cited by 1 · see citing works
Garg, P., & Jeet, G. (2025) · Current Diabetes Reports · doi:10.1007/s11892-025-01596-0 · Cited by 1 · see citing works
Abdela, U. (2025) · Journal of Agriculture and Environment for International Development (JAEID) · doi:10.36253/jaeid-16741 · Cited by 1 · see citing works
Weckstein, A. R., Frajzyngier, V., Vititoe, S. E., Baglivo, A., Beebe, E., Govil, P., Bradley, M. C., Pérez-Vilar, S., Liu, W., Rivera, D. R. R., et al. (2025) · Epidemiology · doi:10.1097/ede.0000000000001901 · Cited by 1 · see citing works
Reusing clinical trial data to consolidate and advance medical knowledge
Varvara, G., Burgwinkel, C., Locher, C., Ross, J. S., Mansmann, U., & Naudet, F. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111984 · Cited by 2 · see citing works
Estimating Discrimination in Sentencing: Distinguishing between Good and Bad Controls
Pina-Sánchez, J., Hamilton, M., & Tennant, P. (2025) · European journal of empirical legal studies · doi:10.62355/ejels.53977 · Cited by 1 · see citing works
Schippling, M. (2026) · Impact · doi:10.1075/impact.56 · Cited by 1 · see citing works
Towards Community-Based Participatory Statistics Research Using Secondary Data
Morton, C. M., & Kelling, C. (2026) · The American Statistician · doi:10.1080/00031305.2026.2634877 · Cited by 1 · see citing works
Christodoulou, A. M., Zakopoulou, V., Tsamis, K. I., & Voniati, L. (2026) · Applied Neuropsychology Child · doi:10.1080/21622965.2026.2643348 · Cited by 1 · see citing works
Challenges and future directions for Mendelian randomization
Sanderson, E. C. M., Levin, M. G., Walker, V. M., Yuan, S., Badini, I., Dolce, J., Mahida, K. J., Nho, J.-W., Pingault, J., Damrauer, S. M., et al. (2026) · Nature Genetics · doi:10.1038/s41588-026-02546-6 · Cited by 2 · see citing works
Self-citation
- Authors
- Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R.
- Affiliations
- King's College London (GB)
- University College London (GB)
- University of Bristol (GB)
- Corresponding author
- Baldwin, Jessie R.
- Venue
- European Journal of Epidemiology
- Published
- 2022-01
- Volume
- 37
- Issue
- 1
- Pages
- 1-10
- ISSN
- 0393-2990 (ISSN-L)
- 1573-7284
- PMC
- PMC8791887
- OpenAlex
- W4205384179
- Cited by
- 178
- References
- 63
- Citations by year
- 2026: 55 · 2025: 50 · 2024: 31 · 2023: 27 · 2022: 10 · 2021: 3 · 2020: 1 · 2013: 1
- FWCI
- 17.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- European Journal of Epidemiology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- Explore Bristol Research (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- Wellcome Trust (215917/Z/19/Z)
- Medical Research Foundation (MRF-160-0002-ELP-PINGA)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- researcher bias, secondary analysis, preregistration, questionable research practices, research robustness
- MeSH terms
- Humans, Surveys and Questionnaires, Cohort Studies, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
