The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
Protecting against researcher bias in secondary data analysis: challenges and potential solutions
Open accessAbstract
Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, there is clear evidence from both MR and the natural experiment of an increase in the school leaving age that an increase in the number of years in education has a causal protective effect on health behaviours such as smoking.156–160”
From Mendelian randomization (2022) · cites this paper as [158]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
RETRACTED ARTICLE: High replicability of newly discovered social-behavioural findings is achievable
Protzko, J., Krosnick, J. A., Nelson, L. D., Nosek, B. A., Axt, J. R., Berent, M., Buttrick, N. R., DeBell, M., Ebersole, C. R., Lundmark, S., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01749-9 · Cited by 97
RetractedConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Lakens, D. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617697177 · Cited by 2,161
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Equivalence Testing for Psychological Research: A Tutorial
Lakens, D., Scheel, A. M., & Isager, P. M. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918770963 · Cited by 1,560 · cited 1×
“, Bayesian methods [58] or equivalence tests, which determine whether an observed effect is surprisingly small [52, 59, 60].”
MethodPublication bias in the social sciences: Unlocking the file drawer
Franco, A., Malhotra, N., & Simonovits, G. (2014) · Science · doi:10.1126/science.1255484 · Cited by 1,542
Increasing Transparency Through a Multiverse Analysis
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616658637 · Cited by 1,512
Flagged first, then by citations
Research works printed but not linked: 2
- Small DS, Firth D, Keele L, et al. Protocol for a study of the effect of surface mining in central appalachia on adverse birth outcomes. arXiv.org. 2020
- Deshpande SK, Hasegawa RB, Weiss J, Small DS. Protocol for an observational study on the effects of playing football in adolescence on mental health in early adulthood. arXiv preprint 2018
Cited by
Sorted by citations
Items 101–179 of 179
Harrington, C. J., Khan, U., Stennett, C. A., O’Hara, N. N., Sprague, S., Marchand, L. S., Patterson, J. T., Beltran, M. J., Obremskey, W. T., O’Toole, R. V., et al. (2026) · Injury · doi:10.1016/j.injury.2026.113289 · Cited by 1
AI-Based Science Assessment: Potential Pitfalls and Responsible and Ethical Practices
Lee, G., Zhaı, X., & Lee, O. (2026) · Springer eBooks · doi:10.1007/978-3-032-16871-9_10 · Cited by 1
Ridoy, M. N., Supto, S. T. J., Saha, G., & Hossain, M. S. (2026) · doi:10.3390/engproc2026138007 · Cited by 2
Hanifah, A. K., Astari, R. V., Muktamiroh, H., & Saleh, A. Y. (2022) · Folia Medica Indonesiana · doi:10.20473/fmi.v58i3.33228 · Cited by 1
Wynes, S., & Matthews, H. D. (2023) · Climate Policy · doi:10.1080/14693062.2023.2190870 · Cited by 5
Modelling Customised Student Support Framework to Enhance University at-risk Students’ Study Skills
Ndobe, T. V., Seeletse, S. M., & Hungwe, T. (2024) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.23.1.10 · Cited by 1
Young, A., & Stallings, B. (2023) · Journal of Student Research · doi:10.47611/jsrhs.v12i4.5273 · Cited by 1
Triwiyanto, T. (2025) · International Urogynecology Journal · doi:10.1007/s00192-025-06164-4 · Cited by 1
Trotman, H., Jelley, B., & Umla‐Runge, K. (2025) · Clinical Parkinsonism & Related Disorders · doi:10.1016/j.prdoa.2025.100355 · Cited by 1
Brown, M. J., & Marsh, L. (2023) · Evidence-Based Nursing · doi:10.1136/ebnurs-2022-103562
Pelatihan Penggunaan Potentiostat sebagai Transfer Pengetahuan Ilmu Kimia untuk Guru – Guru SM
Syarif, N., & Sya’baniah, N. F. (2023) · ABDINE Jurnal Pengabdian Masyarakat · doi:10.52072/abdine.v3i1.565
Machaku, D. P., Kafuye, M. Y., Joseph, R. S., Kazyoba, P. E., & Aboud, M. (2023) · African Journal of Bioethics · doi:10.58177/ajb230003
Arachchige, A. S. P. M., & Verma, Y. (2023) · AIMS neuroscience · doi:10.3934/neuroscience.2023019
Belcher, S., Larmer, P. J., Christopherson, R. M., & Sampath, K. K. (2024) · New Zealand Journal of Physiotherapy · doi:10.15619/nzjp.v52i1.343
Aquatic and Maritime Drowning Deaths Among Undocumented Migrants—Reply
Lussier, A., & Lindholm, P. (2024) · JAMA · doi:10.1001/jama.2024.10310
Fundamentals on transparency, reproducibility, and validation
Camarasu-Pop, S., Vila, G., Tristan Glatard, B., Bonnet, A., Frindel, C., & Ratiney, H. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00017-1 · Cited by 1
The Role of Predictive Analytics in Enhancing Employee Retention in India’s Start-Up Ecosystem
Kundu, B., Shyam, R., & Mondal, B. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem39218
Vazquez, E., Ross, J. S., Gross, C. P., Childers, K., Bamford, S., Ritchie, J. D., Waldstreicher, J., Krumholz, H. M., & Wallach, J. D. (2024) · Clinical Trials · doi:10.1177/17407745241304355
Tangkamolsuk, B., & La, Q. D. (2025) · Academic Emergency Medicine · doi:10.1111/acem.70005
Sulistyani, D. C., Krobthong, T., & Mochama, O. E. (2025) · Schrödinger Journal of Physics Education · doi:10.37251/sjpe.v6i1.1065 · Cited by 2
Marketing Recommendation Algorithm for New Media Platform Based on Data Analysis
Ma, S. (2025) · International Conference on Integrated Circuits and Communication Systems (ICICACS) · doi:10.1109/icicacs65178.2025.10968742
Ashour, H. K., Alramadan, N. S., Neamah, A. H., & Khaleel, A. A. (2025) · European Journal of Accounting Auditing and Finance Research · doi:10.37745/ejaafr.2013/vol13n56472
Akter, R., Rahman, M. A., Bijly, R. A., Akhter, M. S., Khatun, M. C. S., Ferdiousi, N., Khandaker, G., & Rafiquzzaman, M. (2025) · American Journal of Analytical Chemistry · doi:10.4236/ajac.2025.165005
Pragmatic language production in mood and psychotic disorders a systematic review and meta-analysisPreprint
Meister, F., Silva, M. S., Melshin, G., Mouslih, E. C., Zaher, F., Sattari, R., Wei, H., Mekideche, N., Bambini, V., Voppel, A., et al. (2025) · Research Square · doi:10.21203/rs.3.rs-6839307/v1
Nor, N. A. M., Anuwar, A. H. K., & Yusof, Z. Y. M. (2025) · Springer eBooks · doi:10.1007/978-981-96-5154-2_5 · Cited by 2
Krishnan, A., & Mukherjee, D. (2025) · World Journal of Hepatology · doi:10.4254/wjh.v17.i6.106573
Comparing ChatGPT with human perceptions of illusory faces
Kramer, R. S. S. (2025) · Visual Cognition · doi:10.1080/13506285.2025.2529875
Karageorgos, S. A., & Koutroulis, I. (2025) · Pediatric Research · doi:10.1038/s41390-025-04471-6
Litchfield, H. D., Gill, B., Irwin, J. D., Tucker, P., Fenesi, B., & Burke, S. M. (2025) · Review of Educational Research · doi:10.3102/00346543251367772
Nguyen, P., Turner, S., Korevaar, E., Page, M. J., & McKenzie, J. E. (2025) · medRxiv · doi:10.1101/2025.11.06.25339702
Kumar, A., Chen, Y., Cifarelli, J., Anand, A., Xu, A., Gurney, J., Marcus, D. S., & Dhar, R. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01761-7
Five dollars and a dress: gendered dynamics and the politics of funding Zimbabwe's ‘Mighty Warriors’
Chiweshe, M. K. (2025) · Soccer and Society · doi:10.1080/14660970.2025.2603776
How can we overcome the barriers to researchers producing high quality, open research?
Tukanova, M., Hsing, P.-Y., Freeman, A., Fellows, T., Thompson, J., & Munafo, M. R. (2023) · PhilPapers (PhilPapers Foundation) · doi:10.57874/axc8-vs07
Os, M. V., Acar, S., & Hilmi, N. (2025) · Erciyes Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi · doi:10.18070/erciyesiibd.1709960
Appendix B. Supplementary material to study 2
(2026) · Impact · doi:10.1075/impact.56.app2
(2026) · Impact · doi:10.1075/impact.56.los
(2026) · Impact · doi:10.1075/impact.56.c5
(2026) · Impact · doi:10.1075/impact.56.refs
Appendix A. Supplementary material to study 1
(2026) · Impact · doi:10.1075/impact.56.app1
(2026) · Impact · doi:10.1075/impact.56.c7
(2026) · Impact · doi:10.1075/impact.56.index
(2026) · Impact · doi:10.1075/impact.56.abb
(2026) · Impact · doi:10.1075/impact.56.c6
Points of departure and positioning of this work within academic discourses
Schippling, M. (2026) · Impact · doi:10.1075/impact.56.c2
(2026) · Impact · doi:10.1075/impact.56.ack
(2026) · Impact · doi:10.1075/impact.56.c1
(2026) · Impact · doi:10.1075/impact.56.toc
(2026) · Impact · doi:10.1075/impact.56.con
(2026) · Impact · doi:10.1075/impact.56.prelim
French, K. A., Storey, R., Egdom, D. V., & Spitzmueller, C. (2026) · Journal of Business and Psychology · doi:10.1007/s10869-026-10109-z
Design of an ergonomic lounge chair using water hyacinth material after going through a tensile test
Gozali, L., Setiawan, R., Daywin, F. J., Rafael, G., Gunadi, A., Ismail, S., Simatupang, T. M., Garza‐Reyes, J. A., Kumar, V., Cheng, W. H., et al. (2026) · Sustainable Futures · doi:10.1016/j.sftr.2026.101788
An update on community water fluoridation, part 2
Kumar, J. V., Levy, S. M., & Warren, J. J. (2026) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2026.02.010
Han, W. (2026) · Journal of dementia and Alzheimer's disease · doi:10.3390/jdad3020021
Fernandez-Agudo, A., Lester, H., Tarazona, J. V., & Rivero‐Pino, F. (2026) · Frontiers in Toxicology · doi:10.3389/ftox.2026.1829326
Pre-registration for confirmatory and exploratory research
Dunleavy, P., & Monteath, T. (2026) · LSE Press eBooks · doi:10.31389/lsepress.dos.d
Nguyen, P.-Y., Turner, S., Korevaar, E., Page, M. J., & McKenzie, J. E. (2026) · BMC Public Health · doi:10.1186/s12889-026-27475-1
Júnior, D. S. T., Gonçalves, A. L., Cruz, A. C., Veloso, H. D. H., de Oliveira, F. X., de Sousa, M. B., de Oliveira, R. C., & de Assis, E. M. (2026) · Acta Parasitologica · doi:10.1007/s11686-026-01312-2
Petróczi, A., Sagoe, D., Kiss, A., Soós, S., Chegeni, R., Veltmaat, A., Cruyff, M., van der Heijden, P. G. M., & de Hon, O. M. (2026) · Sports · doi:10.3390/sports14060229
Okonkwo, U. K. C., Umeonwuka, O. O., & Paul, B. S. (2026) · World Development Sustainability · doi:10.1016/j.wds.2026.100332
Li, B., Hsu, P., Khatab, Z., Zai, C. C. H., Zai, G. C., Das, S., & Mah, L. (2026) · Equality Diversity and Inclusion An International Journal · doi:10.1108/edi-10-2025-0737
How to Utilize New Technologies to Improve Productivity
Liu, Y., Guo, Y., & Gao, Z. (2026) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.413104
Dorji, T., Choden, K., Bidha, N., Pelden, T., Dema, T., & Rai, S. (2026) · American Journal of STEM Education · doi:10.32674/f4y6aq26
Mamba, J. T., & Mamba, S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-4893-3.ch001
The humanizing power of nostalgia: a pathway to psychological richness
Wei, X., Sedikides, C., Wildschut, T., & Yu, F. (2026) · The Journal of Positive Psychology · doi:10.1080/17439760.2026.2697168
The functional use of theory in accounting research: evidence from leading journals
Alshamari, A., Ridley, G., & Wills, D. (2026) · Journal of Accounting & Organizational Change · doi:10.1108/jaoc-09-2025-0394
Sustainable Eco-Tourism for Cultural Preservation and Community Development
Mtshali, L., Mamba, S., & Beharry-Ramraj, A. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-7976-0.ch019
Developing high-quality research questions
Covvey, J. R., Gionfriddo, M. R., Kalarchian, M. A., Nolfi, D., & McClendon, C. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-44607-8.00008-x
Moher, D., Cobey, K. D., Brown, A. N., Armond, A. C. V., Cenci, M. S., Chan, L., Fox, S. H., Gambelli, L., Halas, M., Hoffman, M., et al. (2026) · eLife · doi:10.7554/elife.112683
Moher, D., Cobey, K. D., Brown, A. N., Armond, A. C. V., Cenci, M. S., Chan, L., Fox, S. H., Gambelli, L., Halas, M., Hoffman, M., et al. (2026) · eLife · doi:10.7554/elife.112683.1
Arianpoor, A., Webster, C. S., Taylor, S., Shulruf, B., & Huang, P. (2026) · Applied Psychological Measurement · doi:10.1177/01466216261492558
Sentia, P. D., & Sari, C. M. A. (2026) · Progress in Disaster Science · doi:10.1016/j.pdisas.2026.100746 · Cited by 1
El ecosistema de la movilidad digital en Galicia: redes y vínculos regionales
Lampón, J. F., Crespo-Velando, M. Á., Pérez-Moure, H., & García-Álvarez, Ó. (2026) · Revista Galega de Economía · doi:10.15304/rge.35.3.11358
Soil Microbiome Analysis: Comparative Evaluation of Direct and Indirect Techniques
Sharma, V., Sheershwal, A., Mansi, & Trivedi, B. (2026) · Springer protocols handbooks/Springer protocols · doi:10.1007/978-1-0716-5595-5_12
Metadata
- Authors
- Baldwin, J. R., Pingault, J.-B., Schoeler, T., Sallis, H. M., & Munafò, M. R.
- Affiliations
- King's College London (GB)
- University College London (GB)
- University of Bristol (GB)
- Corresponding author
- Baldwin, Jessie R.
- Venue
- European Journal of Epidemiology
- Published
- 2022-01
- Volume
- 37
- Issue
- 1
- Pages
- 1-10
- ISSN
- 0393-2990 (ISSN-L)
- 1573-7284
- PMC
- PMC8791887
- OpenAlex
- W4205384179
- Cited by
- 178
- References
- 63
- Citations by year
- 2026: 55 · 2025: 50 · 2024: 31 · 2023: 27 · 2022: 10
- FWCI
- 17.81
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- European Journal of Epidemiology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- Explore Bristol Research (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Funders
- Wellcome Trust (215917/Z/19/Z)
- Medical Research Foundation (MRF-160-0002-ELP-PINGA)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- researcher bias, secondary analysis, preregistration, questionable research practices, research robustness
- MeSH terms
- Humans, Surveys and Questionnaires, Cohort Studies, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review