Distinguishing between exploratory and confirmatory preclinical research will improve translation
Open accessAbstract
Preclinical researchers confront two overarching agendas related to drug development: selecting interventions amid a vast field of candidates, and producing rigorous evidence of clinical promise for a small number of interventions. We suggest that each challenge is best met by two different, complementary modes of investigation. In the first (exploratory investigation), researchers should aim at generating robust pathophysiological theories of disease. In the second (confirmatory investigation), researchers should aim at demonstrating strong and reproducible treatment effects in relevant animal models. Each mode entails different study designs, confronts different validity threats, and supports different kinds of inferences. Research policies should seek to disentangle the two modes and leverage their complementarity. In particular, policies should discourage the common use of exploratory studies to support confirmatory inferences, promote a greater volume of confirmatory investigation, and customize design and reporting guidelines for each mode.
Verifying this paper
7 of 9 steps done · this page updates itself — you can leave and come back
- Registry recordVerified
- Full textRead the full text
- Reference listQueued — the list appears here once it is fetched
- Reference checkChecking each cited work
- StatisticsNeeds the full text
- ClaimsNeeds the full text
- Cited sourcesNeeds claims to check
- Self-checkNeeds the full text
- VerificationFindings verified
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Testing a specific hypothesis has implications for both the study design and the data analysis.
“Testing a specific hypothesis has implications for both the study design and the data analysis [16,167].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [167]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1× · see citing works
“This dynamic, known as the ‘‘winner’s curse’’ [7], reflects the fact that research in the exploratory mode will often test many different strategies in parallel, and this is only feasible if small sample sizes are used.”
BackgroundScience as a Process: An Evolutionary Account of the Social and Conceptual Development of Science
Hull, D. L. (1988) · DigitalGeorgetown (Georgetown University Library) · doi:10.7208/chicago/9780226360492.001.0001 · Cited by 2,066 · see citing works
Can Animal Models of Disease Reliably Inform Human Studies?
van der Worp, H. B., Howells, D. W., Sena, E. S., Porritt, M. J., Rewell, S., O'Collins, V., & Macleod, M. R. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000245 · Cited by 1,354 · see citing works
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196 · see citing works
Heterogeneity in Cancer: Cancer Stem Cells versus Clonal Evolution
Shackleton, M. J., Quintana, E., Fearon, E. R., & Morrison, S. J. (2009) · Cell · doi:10.1016/j.cell.2009.08.017 · Cited by 1,152 · see citing works
Shall we Really do it Again? The Powerful Concept of Replication is Neglected in the Social Sciences
Schmidt, S. (2009) · Review of General Psychology · doi:10.1037/a0015108 · Cited by 1,125 · see citing works
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042 · see citing works
Publication bias in reports of animal stroke studies leads to major overstatement of efficacy
Sena, E. S., van der Worp, H. B., Bath, P. M. W., Howells, D. W., & Macleod, M. R. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000344 · Cited by 600 · cited 1× · see citing works
“Because many preclinical studies poorly address internal validity threats [1], fail attempts at replication [2], are not published [3], or provide exaggerated estimates of clinical utility, numerous stakeholders are urging reforms in the way preclinical research is performed [4].”
BackgroundRegistered Reports: A new publishing initiative at Cortex
Chambers, C. (2012) · Cortex · doi:10.1016/j.cortex.2012.12.016 · Cited by 574 · see citing works
Henderson, V. C., Kimmelman, J., Fergusson, D. A., Grimshaw, J., & Hackam, D. G. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001489 · Cited by 300 · see citing works
Self-citationReplication and reproducibility in spinal cord injury research
Steward, O., Popovich, P. G., Dietrich, W. D., & Kleitman, N. (2012) · Experimental Neurology · doi:10.1016/j.expneurol.2011.06.017 · Cited by 186 · see citing works
Bringing rigour to translational medicine
Howells, D. W., Sena, E. S., & Macleod, M. R. (2014) · Nature Reviews Neurology · doi:10.1038/nrneurol.2013.232 · Cited by 130 · cited 2× · see citing works
“Once intervention strategies are discovered in exploration, those wishing to launch clinical development should be expected to run, or at least reference, stand-alone confirmatory studies before launching trials [10,13].”
BackgroundMethodA Concerted Appeal for International Cooperation in Preclinical Stroke Research
Dirnagl, U., Hakim, A. M., Macleod, M. R., Fisher, M., Howells, D. W., Allan, S. M., Steinberg, G. K., Planas, A. M., Boltze, J., Savitz, S. I., et al. (2013) · Stroke · doi:10.1161/strokeaha.113.000734 · Cited by 102 · see citing works
Self-citationShould preclinical studies be registered?
Kimmelman, J., & Anderson, J. A. (2012) · Nature Biotechnology · doi:10.1038/nbt.2261 · Cited by 91 · see citing works
Self-citationEnhancing translation: Guidelines for standard pre-clinical experiments in mdx mice
Willmann, R., Luca, A. D., Benatar, M. G., Grounds, M. D., Dubach, J., Raymackers, J., & Nagaraju, K. (2011) · Neuromuscular Disorders · doi:10.1016/j.nmd.2011.04.012 · Cited by 76 · see citing works
Mogil, J. S., Simmonds, K. P., & Simmonds, M. J. (2008) · Pain · doi:10.1016/j.pain.2008.11.012 · Cited by 56 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0
Percie du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000411 · Cited by 2,677 · see citing works
Self-citationThe economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055 · see citing works
Investigating the replicability of preclinical cancer biology
Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.71601 · Cited by 393 · see citing works
“Theoretical analyses point to a system of incentives that prioritizes innovation over verification, leading to infrequent efforts to replicate findings and to behaviors that could reduce the replicability of published findings such as selective reporting, presenting exploratory findings as confirmatory tests, and fail…”
BackgroundHardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. A. (2022) · Perspectives on Psychological Science · doi:10.1177/1745691620979806 · Cited by 187 · see citing works
Ioannidis, J. P. A., Kim, B. Y. S., & Trounson, A. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0314-y · Cited by 175 · see citing works
Ramirez, F. D., Motazedian, P., Jung, R. G., Di Santo, P., MacDonald, Z. D., Moreland, R., Simard, T., Clancy, A. A., Russo, J. J., Welch, V. A., et al. (2017) · Circulation Research · doi:10.1161/circresaha.117.310628 · Cited by 138 · see citing works
“These criteria are consistent with previously proposed definitions of preclinical (“confirmatory” or “proof-of-concept”) experiments.(7,30,31) CVDs of interest included atherosclerosis or vascular homeostasis, arterial aneurysms or dissections, myocardial infarction, valvular disease, cardiomyopathy or heart failure,…”
Holman, C., Piper, S. K., Grittner, U., Diamantaras, A. A., Kimmelman, J., Siegerink, B., & Dirnagl, U. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002331 · Cited by 114 · see citing works
Self-citationPreregistration of exploratory research: Learning from the golden age of discovery
Dirnagl, U. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000690 · Cited by 81 · see citing works
Self-citationImproving transparency and scientific rigor in academic publishing
Prager, E. M., Chambers, K. E., Plotkin, J. L., McArthur, D. L., Bandrowski, A. E., Bansal, N., Martone, M. E., Bergstrom, H. C., Bespalov, A., & Graf, C. (2019) · Journal of Neuroscience Research · doi:10.1002/jnr.24340 · Cited by 83 · see citing works
Introduction to the EQIPD quality system
Bespalov, A., Bernard, R., Gilis, A., Gerlach, B., Guillén, J., Castagné, V., Lefevre, I. A., Ducrey, F., Monk, L., Bongiovanni, S., et al. (2021) · eLife · doi:10.7554/elife.63294 · Cited by 76 · see citing works
Self-citationvan der Naald, M., Chamuleau, S. A. J., Menon, J. M. L., de Leeuw, W., de Haan, J., Duncker, D. J., & Wever, K. E. (2022) · BMJ Open Science · doi:10.1136/bmjos-2021-100259 · Cited by 26 · see citing works
How our approaches to assessing benefits and harms can be improved
Sena, E., & Currie, G. (2019) · Animal Welfare · doi:10.7120/09627286.28.1.107 · Cited by 23 · see citing works
Addressing selective reporting of experiments through predefined exclusion criteria
Neves, K., & Amaral, O. B. (2020) · eLife · doi:10.7554/elife.56626 · Cited by 15 · see citing works
- Authors
- Kimmelman, J., Mogil, J. S., & Dirnagl, U.
- Affiliations
- McGill University (CA)
- German Center for Neurodegenerative Diseases (DE)
- Corresponding author
- Kimmelman, Jonathan
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-05-20
- Volume
- 12
- Issue
- 5
- Pages
- e1001863
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC4028181
- OpenAlex
- W2026025998
- Cited by
- 236
- References
- 16
- Citations by year
- 2026: 18 · 2025: 14 · 2024: 10 · 2023: 17 · 2022: 20 · 2021: 29 · 2020: 24 · 2019: 23 · 2018: 25 · 2017: 20 · 2016: 18 · 2015: 14 · 2014: 4
- FWCI
- 27.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- DZNE Pub (submittedVersion, other-oa)
- Funders
- Canadian Institutes of Health Research (111391_1)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Health Professions › Health Information Management › Methodologies in Health Research and Practice
- Keywords
- exploratory research, confirmatory research, preclinical research, drug development, translational research, animal models, research design, reproducibility, science policy, reporting guidelines
- MeSH terms
- Animals, Drug Evaluation, Preclinical, Humans, Research Design, Practice Guidelines as Topic, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
