The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments
Open accessAbstract
BACKGROUND: The vast majority of medical interventions introduced into clinical development prove unsafe or ineffective. One prominent explanation for the dismal success rate is flawed preclinical research. We conducted a systematic review of preclinical research guidelines and organized recommendations according to the type of validity threat (internal, construct, or external) or programmatic research activity they primarily address. METHODS AND FINDINGS: We searched MEDLINE, Google Scholar, Google, and the EQUATOR Network website for all preclinical guideline documents published up to April 9, 2013 that addressed the design and conduct of in vivo animal experiments aimed at supporting clinical translation. To be eligible, documents had to provide guidance on the design or execution of preclinical animal experiments and represent the aggregated consensus of four or more investigators. Data from included guidelines were independently extracted by two individuals for discrete recommendations on the design and implementation of preclinical efficacy studies. These recommendations were then organized according to the type of validity threat they addressed. A total of 2,029 citations were identified through our search strategy. From these, we identified 26 guidelines that met our eligibility criteria--most of which were directed at neurological or cerebrovascular drug development. Together, these guidelines offered 55 different recommendations. Some of the most common recommendations included performance of a power calculation to determine sample size, randomized treatment allocation, and characterization of disease phenotype in the animal model prior to experimentation. CONCLUSIONS: By identifying the most recurrent recommendations among preclinical guidelines, we provide a starting point for developing preclinical guidelines in other disease domains. We also provide a basis for the study and evaluation of preclinical research practice. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2002) · Cited by 13,211
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
Bellomo, R., Ronco, C., Kellum, J. A., Mehta, R. L., Palevsky, P., & workgroup, T. A. (2004) · Critical Care · doi:10.1186/cc2872 · Cited by 6,781
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
Can the pharmaceutical industry reduce attrition rates?
Kola, I., & Landis, J. (2004) · Nature Reviews Drug Discovery · doi:10.1038/nrd1470 · Cited by 4,167
AGREE II: advancing guideline development, reporting and evaluation in health care
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., et al. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.090449 · Cited by 3,790
Self-citationAnimal research: Reporting in vivo experiments: The ARRIVE guidelines
Kilkenny, C., Browne, W., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · British Journal of Pharmacology · doi:10.1111/j.1476-5381.2010.00872.x · Cited by 3,625
Clinical Practice Guidelines We Can Trust
Committee on Standards for Developing Trustworthy Clinical Practice Guidelines, Board on Health Care Services, & Institute of Medicine (2011) · National Academies Press eBooks · doi:10.17226/13058 · Cited by 3,081
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 2×
“Various commentators have reported problems such as difficulty in replicating preclinical studies [5,6], publication bias [7], and the prevalence of methodological practices that result in threats to validity [8].”
BackgroundThe OBO Foundry: coordinated evolution of ontologies to support biomedical data integration
The OBI Consortium, Smith, B., Ashburner, M., Rosse, C., Bard, J., Bug, W., Ceusters, W., Goldberg, L. J., Eilbeck, K., Ireland, A., et al. (2007) · Nature Biotechnology · doi:10.1038/nbt1346 · Cited by 2,560
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · cited 1×
“Certainly there is evidence that many practices captured in Table 2 are relevant in clinical trials [27,28], and recommendations like those concerning justification of sample size or selection of models have an irrefutable logic.”
BackgroundAnimal Research: Reporting in vivo Experiments—The ARRIVE Guidelines
Kilkenny, C., Browne, W., Cuthill, I. C., Emerson, M., & Altman, D. G. (2011) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/jcbfm.2010.220 · Cited by 2,012
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Various commentators have reported problems such as difficulty in replicating preclinical studies [5,6], publication bias [7], and the prevalence of methodological practices that result in threats to validity [8].”
Background1,026 Experimental treatments in acute stroke
O'Collins, V. E., Macleod, M. R., Donnan, G. A., Horky, L. L., van der Worp, B. H., & Howells, D. W. (2006) · Annals of Neurology · doi:10.1002/ana.20741 · Cited by 1,460
Can Animal Models of Disease Reliably Inform Human Studies?
van der Worp, H. B., Howells, D. W., Sena, E. S., Porritt, M. J., Rewell, S., O'Collins, V., & Macleod, M. R. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000245 · Cited by 1,354
Update of the Stroke Therapy Academic Industry Roundtable Preclinical Recommendations
Fisher, M., Feuerstein, G., Howells, D. W., Hurn, P. D., Kent, T. A., Savitz, S. I., & Lo, E. H. (2009) · Stroke · doi:10.1161/strokeaha.108.541128 · Cited by 1,352 · cited 4×
“, guidelines for studies of traumatic brain injury [36], amyotrophic lateral sclerosis [37], and stroke [10,12]), others offer none (e.”
BackgroundMethodInfluentialRecommendations for Standards Regarding Preclinical Neuroprotective and Restorative Drug Development
(1999) · Stroke · doi:10.1161/01.str.30.12.2752 · Cited by 1,251
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196
WALKER, M. J. A., CURTIS, M. J., HEARSE, D. J., CAMPBELL, R. W. F., JANSE, M. J., YELLON, D. M., COBBE, S. M., COKER, S. J., HARNESS, J. B., HARRON, D. W. G., et al. (1988) · Cardiovascular Research · doi:10.1093/cvr/22.7.447 · Cited by 1,105
Designing Evaluations of Educational and Social Programs.
Hendrickson, L., & Cronbach, L. J. (1983) · Contemporary Sociology · doi:10.2307/2068975 · Cited by 1,042
Kilkenny, C., Parsons, N., Kadyszewski, E., Festing, M. F. W., Cuthill, I. C., Fry, D., Hutton, J., & Altman, D. G. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007824 · Cited by 860
AGREE II: advancing guideline development, reporting, and evaluation in health care
Brouwers, M. C., Kho, M. E., Browman, G. P., Burgers, J. S., Cluzeau, F., Feder, G., Fervers, B., Graham, I. D., Grimshaw, J., Hanna, S. E., et al. (2010) · Preventive Medicine · doi:10.1016/j.ypmed.2010.08.005 · Cited by 758
Self-citationTranslation of Research Evidence From Animals to Humans
Hackam, D. G., & Redelmeier, D. A. (2006) · JAMA · doi:10.1001/jama.296.14.1731 · Cited by 733 · cited 1×
“Though evidence that implementation of recommendations leads to better translational outcomes is very limited [32], we think there is a plausible case insofar as such practices have been shown to be relevant in the clinical realm [33].”
BackgroundSelf-citationPublication bias in reports of animal stroke studies leads to major overstatement of efficacy
Sena, E. S., van der Worp, H. B., Bath, P. M. W., Howells, D. W., & Macleod, M. R. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000344 · Cited by 600 · cited 2×
“Various commentators have reported problems such as difficulty in replicating preclinical studies [5,6], publication bias [7], and the prevalence of methodological practices that result in threats to validity [8].”
BackgroundTaylor, C. F., Field, D., Sansone, S.-A., Aerts, J., Apweiler, R., Ashburner, M., Ball, C. A., Binz, P.-A., Bogue, M., Booth, T., et al. (2008) · Nature Biotechnology · doi:10.1038/nbt.1411 · Cited by 558 · cited 2×
“These include the establishment of preclinical data repositories [20], minimum reporting checklists for biomedical investigations [21], biomedical data ontologies [22], and reporting standards for animal studies [15].”
BackgroundDesign, power, and interpretation of studies in the standard murine model of ALS
Scott, S., Kranz, J. E., Cole, J., Lincecum, J. M., Thompson, K., Kelly, N., Bostrom, A., Theodoss, J., Al‐Nakhala, B. M., Vieira, F. G., et al. (2008) · Amyotrophic Lateral Sclerosis · doi:10.1080/17482960701856300 · Cited by 504 · cited 1×
“Similarly, most preclinical efficacy studies using the SOD1 murine model for amyotrophic lateral sclerosis do not measure disease response directly, but instead measure random biologic variability, in part because of a lack of disease phenotype characterization (via quantitative genotyping of copy number) prior to the…”
BackgroundRandomisation to protect against selection bias in healthcare trials
Odgaard-Jensen, J., Vist, G. E., Timmer, A., Kunz, R., Akl, E. A., Schünemann, H., Briel, M., Nordmann, A. J., Pregno, S., & Oxman, A. D. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000012.pub3 · Cited by 439
The FDA Critical Path Initiative and Its Influence on New Drug Development
Woodcock, J., & Woosley, R. (2008) · Annual Review of Medicine · doi:10.1146/annurev.med.59.090506.155819 · Cited by 434
Myocardial Protection at a Crossroads: The Need for Translation Into Clinical Therapy
Bolli, R., Becker, L., Gross, G., Mentzer, R., Balshaw, D., & Lathrop, D. A. (2004) · Circulation Research · doi:10.1161/01.res.0000137171.97172.d7 · Cited by 428
Grounds, M. D., Radley, H. G., Lynch, G. S., Nagaraju, K., & De Luca, A. (2008) · Neurobiology of Disease · doi:10.1016/j.nbd.2008.03.008 · Cited by 344
Noseworthy, J. H., Ebers, G. C., Vandervoort, M. K., Farquhar, R. E., Yetisir, E., & Roberts, R. (1994) · Neurology · doi:10.1212/wnl.44.1.16 · Cited by 341
Good Laboratory Practice: Preventing Introduction of Bias at the Bench
Macleod, M. R., Fisher, M., O'Collins, V., Sena, E. S., Dirnagl, U., Bath, P. M. W., Buchan, A., van der Worp, H. B., Traystman, R., Minematsu, K., et al. (2009) · Stroke · doi:10.1161/strokeaha.108.525386 · Cited by 331
EQUATOR: reporting guidelines for health research
Altman, D. G., Simera, I., Hoey, J., Moher, D., & Schulz, K. (2008) · The Lancet · doi:10.1016/s0140-6736(08)60505-x · Cited by 325
Translation of highly promising basic science research into clinical applications
Contopoulos-Ioannidis, D. G., Ntzani, E. E., & Ioannidis, J. P. A. (2003) · The American Journal of Medicine · doi:10.1016/s0002-9343(03)00013-5 · Cited by 323
Guidelines for preclinical animal research in ALS/MND: A consensus meeting
Ludolph, A. C., Bendotti, C., Blaugrund, E., Chio, A., Greensmith, L., Loeffler, J.-P., Mead, R., Niessen, H. G., Petri, S., Pradat, P.-F., et al. (2010) · Amyotrophic Lateral Sclerosis · doi:10.3109/17482960903545334 · Cited by 315
Rice, A. S. C., Cimino-Brown, D., Eisenach, J. C., Kontinen, V. K., Lacroix-Fralish, M. L., Machin, I., Mogil, J. S., & Stöhr, T. (2008) · Pain · doi:10.1016/j.pain.2008.08.017 · Cited by 290
Curtis, M. J., Hancox, J. C., Farkas, A., Wainwright, C. L., Stables, C. L., Saint, D. A., Clements-Jewery, H., Lambiase, P. D., Billman, G. E., Janse, M. J., et al. (2013) · Pharmacology & Therapeutics · doi:10.1016/j.pharmthera.2013.04.008 · Cited by 286
Stem Cell Therapy as an Emerging Paradigm for Stroke (STEPS) II
Savitz, S. I., Chopp, M., Deans, R., Carmichael, S. T., Phinney, D., & Wechsler, L. (2011) · Stroke · doi:10.1161/strokeaha.110.601914 · Cited by 263
Announcement: Reducing our irreproducibility
(2013) · Nature · doi:10.1038/496398a · Cited by 243
Participants, T. S. (2009) · Stroke · doi:10.1161/strokeaha.108.526863 · Cited by 237
EQUATOR: reporting guidelines for health research.
Altman, D. G., Simera, I., Hoey, J., Moher, D., & Schulz, K. (2008) · PubMed Central · Cited by 236
Crossley, N. A., Sena, E., Goehler, J., Horn, J., van der Worp, B., Bath, P. M. W., Macleod, M., & Dirnagl, U. (2008) · Stroke · doi:10.1161/strokeaha.107.498725 · Cited by 234
North of England evidence based guidelines development project: methods of guideline development
Eccles, M., Clapp, Z., Grimshaw, J., Adams, P. C., Higgins, B., Purves, I., & Russell, I. (1996) · BMJ · doi:10.1136/bmj.312.7033.760 · Cited by 216
Self-citationSchwartz, R. S., Edelman, E., Virmani, R., Carter, A., Granada, J. F., Kaluza, G. L., Chronos, N. A. F., Robinson, K. A., Waksman, R., Weinberger, J., et al. (2008) · Circulation: Cardiovascular Interventions · doi:10.1161/circinterventions.108.789974 · Cited by 215
Preclinical research in Rett syndrome: setting the foundation for translational success
Katz, D. M., Berger-Sweeney, J. E., Eubanks, J. H., Justice, M. J., Neul, J. L., Pozzo-Miller, L., Blue, M. E., Christian, D., Crawley, J. N., Giustetto, M., et al. (2012) · Disease Models & Mechanisms · doi:10.1242/dmm.011007 · Cited by 206
RODENT STROKE MODEL GUIDELINES FOR PRECLINICAL STROKE TRIALS (1ST EDITION)
Liu, S., Zhen, G., Meloni, B. P., Campbell, K., & Winn, H. R. (2009) · Journal of Experimental Stroke and Translational Medicine · doi:10.6030/1939-067x-2.2.2 · Cited by 164
Accelerating drug discovery for Alzheimer's disease: best practices for preclinical animal studies
Shineman, D. W., Basi, G. S., Bizon, J. L., Colton, C. A., Greenberg, B., Hollister, B. A., Lincecum, J. M., Leblanc, G. G., Lee, L. H., Luo, F. R., et al. (2011) · Alzheimer s Research & Therapy · doi:10.1186/alzrt90 · Cited by 149
Effects of NXY‐059 in experimental stroke: an individual animal meta‐analysis
Bath, P. M., Gray, L. J., Bath, A. J. G., Buchan, A. M., Miyata, T., & Green, A. R. (2009) · British Journal of Pharmacology · doi:10.1111/j.1476-5381.2009.00196.x · Cited by 135
Bolon, B. N., Stolina, M., King, C., Middleton, S. C., Gasser, J., Zack, D. J., & Feige, U. (2010) · BioMed Research International · doi:10.1155/2011/569068 · Cited by 135
Introducing Critical Appraisal to studies of animal models investigating novel therapies in sepsis
Piper, R. D., Cook, D. J., Bone, R. C., & Sibbald, W. J. (1996) · Critical Care Medicine · doi:10.1097/00003246-199612000-00021 · Cited by 132
Kamath, A. T., Fruth, U., Brennan, M. J., Dobbelaer, R., Hubrechts, P., Ho, M. M., Mayner, R. E., Thole, J. E. R., Walker, K. B., Liu, M., et al. (2005) · Vaccine · doi:10.1016/j.vaccine.2005.03.001 · Cited by 127
Peters, J., Sutton, A. J., Jones, D. R. H., Rushton, L., & Abrams, K. R. (2006) · Journal of Environmental Science and Health Part B · doi:10.1080/03601230600857130 · Cited by 105
Beyond Access vs. Protection in Trials of Innovative Therapies
London, A. J., Kimmelman, J., & Emborg, M. E. (2010) · Science · doi:10.1126/science.1189369 · Cited by 93
Self-citationShould preclinical studies be registered?
Kimmelman, J., & Anderson, J. A. (2012) · Nature Biotechnology · doi:10.1038/nbt.2261 · Cited by 92
Self-citationEnhancing translation: Guidelines for standard pre-clinical experiments in mdx mice
Willmann, R., Luca, A. D., Benatar, M. G., Grounds, M. D., Dubach, J., Raymackers, J., & Nagaraju, K. (2011) · Neuromuscular Disorders · doi:10.1016/j.nmd.2011.04.012 · Cited by 76
Dopamine agonists in animal models of Parkinson’s disease: A systematic review and meta-analysis
Rooke, E. D., Vesterinen, H. M., Sena, E. S., Egan, K. J., & Macleod, M. R. (2011) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2011.02.010 · Cited by 75
Verhagen, H., Aruoma, O. I., van Delft, J. H. M., Dragsted, L. O., Ferguson, L. R., Knasmüller, S., Pool-Zobel, B. L., Poulsen, H. E., Williamson, G., & Yannai, S. (2003) · Food and Chemical Toxicology · doi:10.1016/s0278-6915(03)00025-5 · Cited by 63
Drug administration in animal studies of cardiac arrest does not reflect human clinical experience
Reynolds, J. C., Rittenberger, J. C., & Menegazzi, J. J. (2007) · Resuscitation · doi:10.1016/j.resuscitation.2006.10.032 · Cited by 59
Inroads to Predict in Vivo Toxicology—An Introduction to the eTOX Project
Briggs, K. A., Cases, M., Heard, D. J., Pastor, M., Pognan, F., Sanz, F., Schwab, C. H., Steger‐Hartmann, T., Sutter, A. P., Watson, D. K., et al. (2012) · International Journal of Molecular Sciences · doi:10.3390/ijms13033820 · Cited by 57
Lamontagne, F., Briel, M., Duffett, M., Fox‐Robichaud, A., Cook, D. J., Guyatt, G. H., Lesur, O., & Meade, M. O. (2010) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181fa0468 · Cited by 48
Pullen, N., Birch, C. L., Douglas, G. J., Hussain, Q., Pruimboom-Brees, I., & Walley, R. J. (2011) · Human Reproduction Update · doi:10.1093/humupd/dmr030 · Cited by 42
Guía de uso adecuado de modelos animales para el desarrollo de terapias en esclerosis múltiple
Bruna, B. M., Espejo, C., Nieto, L. M., García, M. S., López, D. C., Soubriet, F. D. C., Fernández, Ó., Montalbán, X., Villoslada, P., & Rodríguez, C. G. (2012) · Revista de Neurología · doi:10.33588/rn.5402.2011424 · Cited by 24
Combination therapies for traumatic brain injury: prospective considerations
Margulies, S. S., & Hicks, R. (2009) · Journal of Neurotrauma · doi:10.1089/neu.2008-0794 · Cited by 22
Guía de recomendaciones en la aplicación de modelos animales para el estudio del ictus
García‐Bonilla, L., Rosell, A., Torregrosa, G., Salom, J. B., Alborch, E., Gutiérrez, M., Díez‐Tejedor, E., Martínez‐Murillo, R., Agulla, J., Ramos‐Cabrer, P., et al. (2010) · Neurología · doi:10.1016/j.nrl.2010.09.001 · Cited by 12
Recommendations guide for experimental animal models in stroke research
García‐Bonilla, L., Rosell, A., Torregrosa, G., Salom, J. B., Alborch, E., Gutiérrez, M., Díez‐Tejedor, E., Martínez‐Murillo, R., Agulla, J., Ramos‐Cabrer, P., et al. (2011) · Neurología (English Edition) · doi:10.1016/s2173-5808(11)70021-9 · Cited by 12
Cited by
Organs-on-chips at the frontiers of drug discovery
Esch, E. W., Bahinski, A., & Huh, D. (2015) · Nature Reviews Drug Discovery · doi:10.1038/nrd4539 · Cited by 1,185
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035
Animal models of human disease: challenges in enabling translation
McGonigle, P., & Ruggeri, B. (2014) · Biochemical Pharmacology · doi:10.1016/j.bcp.2013.08.006 · Cited by 610
Meta-analysis of data from animal studies: A practical guide
Vesterinen, H. M., Sena, E. S., Egan, K. J., Hirst, T. C., Churolov, L., Currie, G. L., Antonic, A., Howells, D. W., & Macleod, M. R. (2014) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2013.09.010 · Cited by 488
Pound, P., & Ritskes-Hoitinga, M. (2018) · Journal of Translational Medicine · doi:10.1186/s12967-018-1678-1 · Cited by 419
Metadata
- Authors
- Henderson, V. C., Kimmelman, J., Fergusson, D., Grimshaw, J. M., & Hackam, D. G.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-23
- Volume
- 10
- Issue
- 7
- Pages
- e1001489
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- OpenAlex
- W1977680690
- Cited by
- 302
- References
- 65
- FWCI
- 6.59
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.