Methodological Rigor in Preclinical Cardiovascular Studies: Targets to Enhance Reproducibility and Promote Research Translation
Open accessAbstract
Rationale: Methodological sources of bias and suboptimal reporting contribute to irreproducibility in preclinical science and may negatively affect research translation. Randomization, blinding, sample size estimation, and considering sex as a biological variable are deemed crucial study design elements to maximize the quality and predictive value of preclinical experiments. Objective: To examine the prevalence and temporal patterns of recommended study design element implementation in preclinical cardiovascular research. Methods and Results: All articles published over a 10-year period in 5 leading cardiovascular journals were reviewed. Reports of in vivo experiments in nonhuman mammals describing pathophysiology, genetics, or therapeutic interventions relevant to specific cardiovascular disorders were identified. Data on study design and animal model use were collected. Citations at 60 months were additionally examined as a surrogate measure of research impact in a prespecified subset of studies, stratified by individual and cumulative study design elements. Of 28 636 articles screened, 3396 met inclusion criteria. Randomization was reported in 21.8%, blinding in 32.7%, and sample size estimation in 2.3%. Temporal and disease-specific analyses show that the implementation of these study design elements has overall not appreciably increased over the past decade, except in preclinical stroke research, which has uniquely demonstrated significant improvements in methodological rigor. In a subset of 1681 preclinical studies, randomization, blinding, sample size estimation, and inclusion of both sexes were not associated with increased citations at 60 months. Conclusions: Methodological shortcomings are prevalent in preclinical cardiovascular research, have not substantially improved over the past 10 years, and may be overlooked when basing subsequent studies. Resultant risks of bias and threats to study validity have the potential to hinder progress in cardiovascular medicine as preclinical research often precedes and informs clinical trials. Stroke research quality has uniquely improved in recent years, warranting a closer examination for interventions to model in other cardiovascular fields.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Other studies using shorter checklists with more editorial follow-up have shown a marked improvement in the nature and detail of the information included in publications.
“In contrast, other studies using shorter checklists (primarily focused on experimental design) with more editorial follow-up have shown a marked improvement in the nature and detail of the information included in publications [34–36].”
From The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research (2020) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · see citing works
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300 · see citing works
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1× · see citing works
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
SYRCLE’s risk of bias tool for animal studies
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-43 · Cited by 4,415 · see citing works
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · see citing works
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 5× · see citing works
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
InfluentialClinical development success rates for investigational drugs
Hay, M., Thomas, D. W., Craighead, J. L., Economides, C., & Rosenthal, J. (2014) · Nature Biotechnology · doi:10.1038/nbt.2786 · Cited by 2,536 · cited 1× · see citing works
“Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among the lowest success rates and likelihood of approval at all phases of clinical trials.(24) Moreover, research and…”
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · see citing works
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 2× · see citing works
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861 · see citing works
Policy: NIH to balance sex in cell and animal studies
Clayton, J. A., & Collins, F. S. (2014) · Nature · doi:10.1038/509282a · Cited by 1,816 · cited 2× · see citing works
“logical variable in applications for preclinical research funding.(26) These principles and guidelines have been endorsed by prominent academic societies, associations, and journals, including all American Heart Association (AHA) journals,(25) with evidence of editorial commitment to complying.”
Sex bias in neuroscience and biomedical research
Beery, A. K., & Zucker, I. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.07.002 · Cited by 1,763 · see citing works
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · see citing works
1,026 Experimental treatments in acute stroke
O’Collins, V. E., Macleod, M. R., Donnan, G. A., Horky, L. L., van der Worp, H. B., & Howells, D. W. (2006) · Annals of Neurology · doi:10.1002/ana.20741 · Cited by 1,453 · see citing works
Can Animal Models of Disease Reliably Inform Human Studies?
van der Worp, H. B., Howells, D. W., Sena, E. S., Porritt, M. J., Rewell, S., O'Collins, V., & Macleod, M. R. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000245 · Cited by 1,354 · cited 2× · see citing works
“The high degree of experimental control that is possible in preclinical research (eg, via genetic and environmental homogeneity) can reduce variation and the sample sizes required relative to clinical trials; however, variations in injury or disease induction and the potential for persistent and unrecognized confounde…”
Update of the Stroke Therapy Academic Industry Roundtable Preclinical Recommendations
Fisher, M., Feuerstein, G., Howells, D. W., Hurn, P. D., Kent, T. A., Savitz, S. I., & Lo, E. H. (2009) · Stroke · doi:10.1161/strokeaha.108.541128 · Cited by 1,352 · see citing works
Reducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319 · cited 1× · see citing works
“Several guidelines and checklists have been developed to improve preclinical methodology(45,58,59) and reporting, yet there is little indication that they are effecting change despite being widely endorsed.(43,58,63) Systematic reviews and meta-analyses have been advocated as safeguards to expose bias in preclinical r…”
Roundtable, S. T. A. I. (1999) · Stroke · doi:10.1161/01.str.30.12.2752 · Cited by 1,258 · see citing works
Policy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · see citing works
A call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196 · cited 5× · see citing works
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
InfluentialThe economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055 · cited 1× · see citing works
“The suggestion that methodologically robust studies are given equal consideration as studies at greater risk of bias by cardiovascular researchers is troubling as it undermines the reputed “self-correcting” tenet of science and increases the risk of pursuing unfruitful avenues of research.(2,5,44,48) Cogent analyses a…”
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049 · see citing works
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
Begley, C. G., & Ioannidis, J. P. A. (2015) · Circulation Research · doi:10.1161/circresaha.114.303819 · Cited by 1,035 · cited 5× · see citing works
“However, evidence of changes in research practice and data on the impact of corrective actions are scarce.(13) Several guidelines and checklists have been developed to improve preclinical methodology(45,58,59) and reporting, yet there is little indication that they are effecting change despite being widely endorsed.”
InfluentialThe productivity crisis in pharmaceutical R&D
Pammolli, F., Magazzini, L., & Riccaboni, M. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3405 · Cited by 918 · cited 3× · see citing works
“opment productivity has declined with decreasing numbers of medications approved, increasing attrition rates at all stages of research, and lengthening development times despite rising expenditures.(4) This decline has been attributed, in part, to flawed preclinical research methodology, highlighting the importance of…”
Kilkenny, C., Parsons, N., Kadyszewski, E., Festing, M. F. W., Cuthill, I. C., Fry, D., Hutton, J., & Altman, D. G. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007824 · Cited by 860 · see citing works
Comparison of Treatment Effects between Animal Experiments and Clinical Trials: Systematic Review
Perel, P., Roberts, I., Sena, E., Wheble, P., Briscoe, C., Sandercock, P., Macleod, M., Mignini, L. E., Jayaram, P., & Khan, K. S. (2007) · BMJ · doi:10.1136/bmj.39048.407928.be · Cited by 827 · see citing works
Kulkarni, A. V. (2009) · JAMA · doi:10.1001/jama.2009.1307 · Cited by 821 · cited 1× · see citing works
“Scopus was selected because it is the largest database available for citation analysis and retrieves a greater number of citations when compared with others.(38) Prespecified analyses of citations at 60 months after the index publication were performed, stratified by individual and cumulative study design elements and…”
Revised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808 · see citing works
Concato, J., Peduzzi, P. N., Holford, T. R., & Feinstein, A. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00510-2 · Cited by 797 · see citing works
How to Make More Published Research True
Ioannidis, J. P. A. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001747 · Cited by 760 · see citing works
Translation of Research Evidence From Animals to Humans
Hackam, D. G., & Redelmeier, D. A. (2006) · JAMA · doi:10.1001/jama.296.14.1731 · Cited by 733 · cited 1× · see citing works
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
Pooling of Animal Experimental Data Reveals Influence of Study Design and Publication Bias
Macleod, M., O'Collins, T., Howells, D., & Donnan, G. (2004) · Stroke · doi:10.1161/01.str.0000125719.25853.20 · Cited by 692 · see citing works
Preclinical research: Make mouse studies work
Perrin, S. (2014) · Nature · doi:10.1038/507423a · Cited by 668 · see citing works
Evidence of bias and variation in diagnostic accuracy studies
Rutjes, A. W., Reitsma, J. B., Nisio, M. D., Smidt, N., van Rijn, J. C., & Bossuyt, P. M. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050090 · Cited by 613 · see citing works
Where is the evidence that animal research benefits humans?
Pound, P., Ebrahim, S. B. J., Sandercock, P. A., Bracken, M. B., & Roberts, I. (2004) · BMJ · doi:10.1136/bmj.328.7438.514 · Cited by 507 · see citing works
Design, power, and interpretation of studies in the standard murine model of ALS
Scott, S., Kranz, J. E., Cole, J., Lincecum, J. M., Thompson, K., Kelly, N., Bostrom, A., Theodoss, J., Al‐Nakhala, B. M., Vieira, F. G., et al. (2008) · Amyotrophic Lateral Sclerosis · doi:10.1080/17482960701856300 · Cited by 504 · see citing works
Why has the number of scientific retractions increased?
Steen, R. G., Casadevall, A., & Fang, F. C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068397 · Cited by 415 · cited 1× · see citing works
“Given the possibility that certain publications may have garnered attention in the short-term but could have ultimately been disproven or deemed irreproducible, a sensitivity analysis of citation counts at 36 months was also undertaken as this time point approximates contemporary mean time-to-retraction.(39) Categoric…”
How can we improve the pre-clinical development of drugs for stroke?
Sena, E. S., van der Worp, H. B., Howells, D. W., & Macleod, M. R. (2007) · Trends in Neurosciences · doi:10.1016/j.tins.2007.06.009 · Cited by 365 · see citing works
Risk of bias in reports of in vivo research: a focus for improvement
Macleod, M. R., Lawson McLean, A., Kyriakopoulou, A., Serghiou, S., de Wilde, A., Sherratt, N., Hirst, T., Hemblade, R., Bahor, Z., Nunes-Fonseca, C., et al. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002273 · Cited by 358 · cited 1× · see citing works
“deemed crucial to improving the quality of preclinical research by addressing selection bias (randomization); minimizing performance, detection, and attrition bias (blinding); ensuring adequate statistical power and the ethical use of animals (sample size estimation); and promoting research relevance for both men and…”
Studying both sexes: a guiding principle for biomedicine
Clayton, J. A. (2016) · The FASEB Journal · doi:10.1096/fj.15-279554 · Cited by 358 · cited 2× · see citing works
“The suggestion that methodologically robust studies are given equal consideration as studies at greater risk of bias by cardiovascular researchers is troubling as it undermines the reputed “self-correcting” tenet of science and increases the risk of pursuing unfruitful avenues of research.(2,5,44,48) Cogent analyses a…”
Hooijmans, C., Leenaars, M., & Ritskes-Hoitinga, M. (2010) · Alternatives to laboratory animals : ATLA · doi:10.1177/026119291003800208 · Cited by 335 · see citing works
Macleod, M. R., van der Worp, H. B., Sena, E. S., Howells, D. W., Dirnagl, U., & Donnan, G. A. (2008) · Stroke · doi:10.1161/strokeaha.108.515957 · Cited by 319 · cited 1× · see citing works
“described candidate therapies that have advanced to testing in humans despite irreproducible or inconsistent results in animals.(8,20,21) Since the mid-1990s, pharmaceutical research and devel-”
Baker, D., Lidster, K., Sottomayor, A., & Amor, S. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001756 · Cited by 305 · see citing works
Henderson, V. C., Kimmelman, J., Fergusson, D. A., Grimshaw, J., & Hackam, D. G. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001489 · Cited by 300 · see citing works
The need for randomization in animal trials: an overview of systematic reviews
Hirst, J. A., Howick, J., Aronson, J. K., Roberts, N., Perera, R., Koshiaris, C., & Heneghan, C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098856 · Cited by 294 · see citing works
Sex bias exists in basic science and translational surgical research
Yoon, D. Y., Mansukhani, N. A., Stubbs, V. C., Helenowski, I. B., Woodruff, T. K., & Kibbe, M. R. (2014) · Surgery · doi:10.1016/j.surg.2014.07.001 · Cited by 270 · see citing works
Getting pharmaceutical R&D back on target
Bunnage, M. E. (2011) · Nature Chemical Biology · doi:10.1038/nchembio.581 · Cited by 258 · cited 3× · see citing works
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
Announcement: Reducing our irreproducibility
(2013) · Nature · doi:10.1038/496398a · Cited by 243 · see citing works
Distinguishing between exploratory and confirmatory preclinical research will improve translation
Kimmelman, J., Mogil, J. S., & Dirnagl, U. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001863 · Cited by 236 · cited 2× · see citing works
“These criteria are consistent with previously proposed definitions of preclinical (“confirmatory” or “proof-of-concept”) experiments.(7,30,31) CVDs of interest included atherosclerosis or vascular homeostasis, arterial aneurysms or dissections, myocardial infarction, valvular disease, cardiomyopathy or heart failure,…”
Empirical Evidence of Bias in the Design of Experimental Stroke Studies
Crossley, N. A., Sena, E. S., Göhler, J., Horn, J., van der Worp, H. B., Bath, P. M., Macleod, M. R., & Dirnagl, U. (2008) · Stroke · doi:10.1161/strokeaha.107.498725 · Cited by 231 · see citing works
Nimodipine in Animal Model Experiments of Focal Cerebral Ischemia
Horn, J., de Haan, R. J., Vermeulen, M., Luiten, P. G. M., & Limburg, M. (2001) · Stroke · doi:10.1161/hs1001.096009 · Cited by 231 · cited 2× · see citing works
“but also may result in erroneous conclusions regarding the treatment effects,(9,21,23,45,46) which can ultimately spur or deter clinical trials in humans with consequent risk of harm.(12,20,47)”
Six red flags for suspect work
Begley, C. G. (2013) · Nature · doi:10.1038/497433a · Cited by 230 · see citing works
Are Peer Reviewers Encouraged to Use Reporting Guidelines? A Survey of 116 Health Research Journals
Hirst, A., & Altman, D. G. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035621 · Cited by 200 · cited 1× · see citing works
“Journal editors and reviewers directly influence what is published thereby often serving as ultimate gatekeepers of research findings; yet, few journals effectively encourage the use of reporting guidelines.(67) At the time of writing, there exists considerable variation in author guidelines for reporting animal studi…”
Systematic Review and Metaanalysis of the Efficacy of FK506 in Experimental Stroke
Macleod, M., O'Collins, T., Horky, L. L., Howells, D., & Donnan, G. (2005) · Journal of Cerebral Blood Flow and Metabolism · doi:10.1038/sj.jcbfm.9600064 · Cited by 196 · see citing works
Incongruence between test statistics and P values in medical papers
García-Berthou, E., & Alcaraz, C. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-13 · Cited by 181 · see citing works
Systematic reviews of animal experiments
Sandercock, P. A., & Roberts, I. (2002) · The Lancet · doi:10.1016/s0140-6736(02)09812-4 · Cited by 157 · see citing works
How to make more published research true
Ioannidis, J. P. A. (2015) · Acimed · Cited by 115 · see citing works
Whole animal experiments should be more like human randomized controlled trials
Muhlhausler, B. S., Bloomfield, F. H., & Gillman, M. W. (2013) · PLoS Biology · doi:10.1371/journal.pbio.1001481 · Cited by 105 · cited 1× · see citing works
“deemed crucial to improving the quality of preclinical research by addressing selection bias (randomization); minimizing performance, detection, and attrition bias (blinding); ensuring adequate statistical power and the ethical use of animals (sample size estimation); and promoting research relevance for both men and…”
Systematic Reviews of Animal Studies; Missing Link in Translational Research?
van Luijk, J. A. K. R., Bakker, B., Rovers, M. M., Ritskes-Hoitinga, M. J., de Vries, R. B. M., & Leenaars, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089981 · Cited by 89 · see citing works
Danthi, N. S., Wu, C. O., Shi, P., & Lauer, M. S. (2014) · Circulation Research · doi:10.1161/circresaha.114.302656 · Cited by 82 · cited 1× · see citing works
“Although article citation is the most commonly used measure of research impact, it is an imperfect indicator.(33,34) Finally, study design elements may have been implemented but not reported, which could result in underestimates of their prevalence.”
Tzoulaki, I., Siontis, K. C., & Ioannidis, J. P. A. (2011) · BMJ · doi:10.1136/bmj.d6829 · Cited by 79 · see citing works
Ramirez, F. D., Motazedian, P., Jung, R. G., Santo, P. D., MacDonald, Z., Simard, T., Clancy, A. A., Russo, J. J., Welch, V. A., Wells, G. A., et al. (2017) · Circulation · doi:10.1161/circulationaha.116.026668 · Cited by 77 · see citing works
Self-citationA mouse’s house may ruin experiments
Reardon, S. (2016) · Nature · doi:10.1038/nature.2016.19335 · Cited by 63 · see citing works
Predicting Productivity Returns on Investment
Lauer, M. S., Danthi, N. S., Kaltman, J. R., & Wu, C. O. (2015) · Circulation Research · doi:10.1161/circresaha.115.306830 · Cited by 54 · see citing works
Kaltman, J. R., Evans, F. J., Danthi, N. S., Wu, C. O., DiMichele, D., & Lauer, M. S. (2014) · Circulation Research · doi:10.1161/circresaha.115.304766 · Cited by 51 · see citing works
Osherovich, L. Z. (2011) · Science-Business eXchange · doi:10.1038/scibx.2011.416 · Cited by 43 · cited 1× · see citing works
“increasingly so among the lay public and the biopharmaceutical industry.(13,18) In 2011, Bayer HealthCare published an analysis of 67 in-house target identification and validation projects in the”
Reporting standards for preclinical studies of stroke therapy
Vahidy, F., Schäbitz, W.-R., Fisher, M., & Aronowski, J. (2016) · Stroke · doi:10.1161/strokeaha.116.013643 · Cited by 39 · cited 1× · see citing works
“These principles and guidelines have been endorsed by prominent academic societies, associations, and journals, including all American Heart Association (AHA) journals,(25) with evidence of editorial commitment to complying.(27,28) However, there are limited data on the extent of suboptimal methodological rigor or inc…”
Poorly Cited Articles in Peer-Reviewed Cardiovascular Journals from 1997 to 2007
Ranasinghe, I., Shojaee, A., Bikdeli, B., Gupta, A., Chen, R., Ross, J. S., Masoudi, F. A., Spertus, J. A., Nallamothu, B. K., & Krumholz, H. M. (2015) · Circulation · doi:10.1161/circulationaha.114.015080 · Cited by 37 · cited 1× · see citing works
“Although article citation is the most commonly used measure of research impact, it is an imperfect indicator.(33,34) Finally, study design elements may have been implemented but not reported, which could result in underestimates of their prevalence.”
Methodological Quality of Experimental Stroke Studies Published in the Stroke Journal
Minnerup, J., Zentsch, V., Schmidt, A., Fisher, M., & Schäbitz, W. (2015) · Stroke · doi:10.1161/strokeaha.115.011695 · Cited by 36 · see citing works
All Is Not Well in the World of Translational Research
Unger, E. F. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.04.067 · Cited by 30 · cited 3× · see citing works
“These criteria are consistent with previously proposed definitions of preclinical (“confirmatory” or “proof-of-concept”) experiments.(7,30,31) CVDs of interest included atherosclerosis or vascular homeostasis, arterial aneurysms or dissections, myocardial infarction, valvular disease, cardiomyopathy or heart failure,…”
Daugherty, A., Hegele, R. A., Mackman, N., Rader, D. J., Schmidt, A. M., & Weber, C. (2016) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.116.307906 · Cited by 14 · cited 1× · see citing works
“These principles and guidelines have been endorsed by prominent academic societies, associations, and journals, including all American Heart Association (AHA) journals,(25) with evidence of editorial commitment to complying.(27,28) However, there are limited data on the extent of suboptimal methodological rigor or inc…”
Fabsitz, R. R., Papanicolaou, G., Sholinsky, P. D., Coady, S., Jaquish, C. E., Nelson, C. R., Olson, J. L., Puggal, M. A., Purkiser, K. L., Srinivas, P. R., et al. (2015) · Circulation · doi:10.1161/circulationaha.114.014147 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000410 · Cited by 5,939 · see citing works
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Experimental Physiology · doi:10.1113/ep088870 · Cited by 3,538 · see citing works
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research*
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x20943823 · Cited by 1,443 · see citing works
“34– 36 It is likely that the level of resource required from journals and editors currently prohibits the implementation of all the items of the ARRIVE guidelines.”
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · British Journal of Pharmacology · doi:10.1111/bph.15193 · Cited by 1,119 · see citing works
The ARRIVE guidelines 2019: updated guidelines for reporting animal researchPreprint
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/703181 · Cited by 1,042 · see citing works
The ARRIVE guidelines 2.0: updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · The Journal of Physiology · doi:10.1113/jp280389 · Cited by 623 · see citing works
“…contrast, other studies using shorter checklists (primarily focused on experimental design) with more editorial follow-up have shown a marked improvement in the nature and detail of the information included in publications (Han et al. 2017, Ramirez et al. 2017; The NPQIP Collaborative group, 2019).”
ResultGuidelines for experimental models of myocardial ischemia and infarction
Lindsey, M. L., Bolli, R., Canty, J. M., Du, X.-J., Frangogiannis, N. G., Frantz, S., Gourdie, R. G., Holmes, J. W., Jones, S. P., Kloner, R. A., et al. (2018) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00335.2017 · Cited by 526 · see citing works
“Much of the lack of reproducibility has been ascribed to limited or lacking scientific rigor (29, 250).”
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · BMC Veterinary Research · doi:10.1186/s12917-020-02451-y · Cited by 509 · see citing works
Bøtker, H. E., Hausenloy, D. J., Andreadou, I., Antonucci, S., Boengler, K., Davidson, S. M., Deshwal, S., Devaux, Y., Lisa, F. D., Sante, M. D., et al. (2018) · Basic Research in Cardiology · doi:10.1007/s00395-018-0696-8 · Cited by 426 · see citing works
Challenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 323 · see citing works
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · BMJ Open Science · doi:10.1136/bmjos-2020-100115 · Cited by 288 · see citing works
Methodological quality of COVID-19 clinical research
Jung, R. G., Di Santo, P., Clifford, C., Prosperi-Porta, G., Skanes, S., Hung, A., Parlow, S., Visintini, S., Ramirez, F. D., Simard, T., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21220-5 · Cited by 189 · see citing works
Self-citationLecour, S., Andreadou, I., Bøtker, H. E., Davidson, S. M., Heusch, G., Ruiz‐Meana, M., Schulz, R., Zuurbier, C. J., Ferdinandy, P., Hausenloy, D. J., et al. (2021) · Basic Research in Cardiology · doi:10.1007/s00395-021-00893-5 · Cited by 142 · see citing works
Provencher, S., Archer, S. L., Ramirez, F. D., Hibbert, B. M., Paulin, R., Boucherat, O., Lacasse, Y., & Bonnet, S. (2018) · Circulation Research · doi:10.1161/circresaha.117.312579 · Cited by 137 · see citing works
“However, few journals enforce methodological and reporting guidelines for confirmatory studies, despite being associated with significant improvements in methodological quality in other fields of preclinical research.(28,29,100)”
Self-citationHan, S., Olonisakin, T. F., Pribis, J. P., Zupetic, J., Yoon, J. H., Holleran, K. M., Jeong, K., Shaikh, N., Rubio, D. M., & Lee, J. S. (2017) · PLOS ONE · doi:10.1371/journal.pone.0183591 · Cited by 131 · see citing works
An Analysis of Mesenchymal Stem Cell-Derived Extracellular Vesicles for Preclinical Use
Tieu, A., Lalu, M. M., Slobodian, M., Gnyra, C., Fergusson, D. A., Montroy, J. R., Burger, D. E., Stewart, D. J., & Allan, D. (2020) · ACS Nano · doi:10.1021/acsnano.0c01363 · Cited by 136 · see citing works
Reproducibility and Rigor in Animal-Based Research
Macleod, M., & Mohan, S. (2019) · ILAR Journal · doi:10.1093/ilar/ilz015 · Cited by 104 · see citing works
“of animal research in 4 other journals from the same publisher even when adjusted for the disease under study and the species of animal used [44].”
BackgroundDid a change in Nature journals’ editorial policy for life sciences research improve reporting?
The NPQIP Collaborative group, Study steering committee, Macleod, M., Sena, E., Howells, D., Macleod, M., Sena, E., Howells, D., Kiermer, V., Swaminathan, S., et al. (2019) · BMJ Open Science · doi:10.1136/bmjos-2017-000035 · Cited by 102 · see citing works
Self-citationZingarelli, B., Coopersmith, C. M., Drechsler, S., Efron, P. A., Marshall, J. C., Moldawer, L. L., Wiersinga, W. J., Xiao, X., Osuchowski, M. F., & Thiemermann, C. (2018) · Shock · doi:10.1097/shk.0000000000001243 · Cited by 86 · see citing works
“lowest use of randomization (17%) was reported in mousebased experiments (49)—the species whose size enables repet-”
BackgroundThe gut microbiome of laboratory mice: considerations and best practices for translational research
Ericsson, A. C., & Franklin, C. L. (2021) · Mammalian Genome · doi:10.1007/s00335-021-09863-7 · Cited by 85 · see citing works
Translational Research in Cardiovascular Repair: A Call for a Paradigm Shift
Chamuleau, S., van der Naald, M., Climent, A., Kraaijeveld, A., Wever, K., Duncker, D., Fernández-Avilés, F., & Bolli, R. (2018) · Circulation Research · doi:10.1161/circresaha.117.311565 · Cited by 49 · see citing works
“3,7,9,10,45 A detailed set of guidelines aimed at increasing rigor and reproducibility in preclinical research has recently been published.”
Cui, L., Gołubczyk, D., Tolppanen, A., Boltze, J., & Jolkkonen, J. (2019) · Annals of Neurology · doi:10.1002/ana.25493 · Cited by 66 · see citing works
“stroke therapies into the clinic, but the quality of preclinical stroke studies is clearly improving.(9) However, significant”
BackgroundMelleby, A. O., Romaine, A., Aronsen, J. M., Veras, I., Zhang, L., Sjaastad, I., Lunde, I. G., & Christensen, G. (2018) · Cardiovascular Research · doi:10.1093/cvr/cvy141 · Cited by 65 · see citing works
Shin, H. S., Shin, H., & Shudo, Y. (2021) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2021.673683 · Cited by 63 · see citing works
“…of original papers published in cardiovascular journals in the same period were poorly cited (with < = 5 citations in the 5 years following publication); however, 44% of cardiovascular journals had more than threefourths of the journal’s content poorly cited at 5 years (Ranasinghe et al., 2015).”
Hypertrophy Regression With N-Acetylcysteine in Hypertrophic Cardiomyopathy (HALT-HCM)
Marian, A. J., Tan, Y., Li, L., Chang, J., Syrris, P., Hessabi, M., Rahbar, M. H., Willerson, J. T., Cheong, B., Liu, C., et al. (2018) · Circulation Research · doi:10.1161/circresaha.117.312647 · Cited by 61 · see citing works
Benefits of a factorial design focusing on inclusion of female and male animals in one experiment
Buch, T., Moos, K., Ferreira, F. M., Fröhlich, H., Gebhard, C., & Tresch, A. (2019) · Journal of Molecular Medicine · doi:10.1007/s00109-019-01774-0 · Cited by 61 · see citing works
Be positive about negatives–recommendations for the publication of negative (or null) results
Bespalov, A., Steckler, T., & Skolnick, P. (2019) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2019.10.007 · Cited by 59 · see citing works
Premise for Standardized Sepsis Models
Remick, D. G., Ayala, A., Chaudry, I. H., Coopersmith, C. M., Deutschman, C. S., Hellman, J., Moldawer, L. L., & Osuchowski, M. F. (2018) · Shock · doi:10.1097/shk.0000000000001164 · Cited by 57 · see citing works
“A 2017 paper (23) analyzed the level of rigor in preclinical cardiovascular studies by examining nearly 30,000 published studies.”
BackgroundPalmer, D. D., Dumont, J. R., Dexter, T. D., Prado, M. A. M., Finger, E. C., Bussey, T. J., & Saksida, L. M. (2021) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2021.107443 · Cited by 52 · see citing works
Donner, D. G., Kiriazis, H., Du, X., Marwick, T. H., & McMullen, J. R. (2018) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00157.2018 · Cited by 48 · see citing works
Optimizing functional outcome endpoints for stroke recovery studies
Balkaya, M., & Cho, S. (2019) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x19875212 · Cited by 41 · see citing works
“While certain crucial recommendations of these initiatives are yet to be adequately implemented,(7) the scientific rigor in stroke studies has been ameliorated drastically.(6) Despite improvements, preclinical research is still affected by quality issues, including publication bias, false positives, lack of randomizat…”
Gender Differences in Cardiac Chronotropic Control: Implications for Heart Rate Variability Research
Williams, D. P., Joseph, N. C., Gerardo, G. M., Hill, L. K., Koenig, J., & Thayer, J. F. (2021) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-021-09528-w · Cited by 41 · see citing works
van der Naald, M., Wenker, S., Doevendans, P. A., Wever, K. E., & Chamuleau, S. A. J. (2020) · BMJ Open Science · doi:10.1136/bmjos-2019-100051 · Cited by 40 · see citing works
Fergusson, D. A., Avey, M. T., Barron, C. C., Bocock, M., Biefer, K., Boet, S., Bourque, S. L., Conic, I., Chen, K., Dong, Y., et al. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215221 · Cited by 38 · see citing works
“Widely cited factors influencing this irreproducibility are poor preclinical study design, incomplete reporting, and a lack of transparency of results [4, 6, 7].”
BackgroundEffect of Intracoronary Metformin on Myocardial Infarct Size in Swine
Techiryan, G., Weil, B. R., Palka, B., & Canty, J. M. (2018) · Circulation Research · doi:10.1161/circresaha.118.313341 · Cited by 37 · see citing works
Zheng, Q., Zhu, J., Bao, X., Zhu, P.-C., Tong, Q., Huang, Y.-Y., Zhang, Q., Zhang, K., Zheng, G., & Wang, Y. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00795 · Cited by 34 · see citing works
Clinical and Preclinical Systematic Review of Panax ginseng C. A. Mey and Its Compounds for Fatigue
Jin, T.-Y., Rong, P.-Q., Liang, H., Zhang, P., Zheng, G., & Lin, Y. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.01031 · Cited by 34 · see citing works
Bolli, R. (2017) · Circulation Research · doi:10.1161/circresaha.117.311678 · Cited by 34 · see citing works
Buglak, N. E., Lucitti, J. L., Ariel, P., Maiocchi, S. L., Miller, F. J., & Bahnson, E. M. (2020) · Cardiovascular Research · doi:10.1093/cvr/cvaa037 · Cited by 33 · see citing works
Risk of bias reporting in the recent animal focal cerebral ischaemia literature
Bahor, Z., Liao, J., Macleod, M. R., Bannach‐Brown, A., McCann, S., Wever, K. E., Thomas, J., Ottavi, T., Howells, D. W., Rice, A., et al. (2017) · Clinical Science · doi:10.1042/cs20160722 · Cited by 29 · see citing works
Jin, X., Li, P., Michalski, D., Li, S., Zhang, Y., Jolkkonen, J., Cui, L., Didwischus, N., Xuan, W., & Boltze, J. (2022) · CNS Neuroscience & Therapeutics · doi:10.1111/cns.13816 · Cited by 31 · see citing works
McCann, S., & Lawrence, C. B. (2020) · BMJ Open Science · doi:10.1136/bmjos-2019-100013 · Cited by 31 · see citing works
Systematic Interrogation of Angiogenesis in the Ischemic Mouse Hind Limb
Lee, J. J., Arpino, J., Yin, H. R., Nong, Z., Szpakowski, A., Hashi, A. A., Chevalier, J. M., O’Neil, C. H., & Pickering, J. G. (2020) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.120.315028 · Cited by 31 · see citing works
Female sex-specific considerations to improve rigor and reproducibility in cardiovascular research
Chang, D. H., Dumanski, S. M., & Ahmed, S. B. (2022) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00462.2022 · Cited by 31 · see citing works
Ameri, P., Mercurio, V., Pollesello, P., Anker, M. S., Backs, J., Bayés‐Genís, A., Borlaug, B. A., Burkhoff, D., Caravita, S., Chan, S. Y., et al. (2024) · European Journal of Heart Failure · doi:10.1002/ejhf.3236 · Cited by 28 · see citing works
Western diet increases cardiac ceramide content in healthy and hypertrophied hearts
Butler, T., Ashford, D. A., & Seymour, A.-M. (2017) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2017.08.007 · Cited by 27 · see citing works
van der Naald, M., Chamuleau, S. A. J., Menon, J. M. L., de Leeuw, W., de Haan, J., Duncker, D. J., & Wever, K. E. (2022) · BMJ Open Science · doi:10.1136/bmjos-2021-100259 · Cited by 26 · see citing works
Coronary artery disease in post-menopausal women: are there appropriate means of assessment?
Chiu, M. H., Heydari, B., Batulan, Z., Maarouf, N., Subramanya, V., Schenck-Gustafsson, K. G., & O’Brien, E. R. (2018) · Clinical Science · doi:10.1042/cs20180067 · Cited by 26 · see citing works
Felizardo, A. A., Marques, D. V. B., Caldas, I. S., Gonçalves, R. V., & Novaes, R. D. (2018) · Experimental Gerontology · doi:10.1016/j.exger.2018.01.022 · Cited by 25 · see citing works
Bonafiglia, J. T., Islam, H., Preobrazenski, N., & Gurd, B. J. (2021) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2021.03.005 · Cited by 25 · see citing works
Boltze, J., & Pérez‐Pinzón, M. A. (2022) · Stroke · doi:10.1161/strokeaha.122.039005 · Cited by 25 · see citing works
From methods to monographs: fostering a culture of research quality
Crawford, D. C., Hoye, M. L., & Silberberg, S. D. (2023) · eneuro · doi:10.1523/eneuro.0247-23.2023 · Cited by 25 · see citing works
“Similar findings were found in an evaluation of preclinical cardiovascular studies (Ramirez et al., 2017) and in an evaluation of papers published by Nature Publishing Group (NPQIP Collaborative group, 2019).”
Macleod, M. R., & The NPQIP Collaborative group (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/187245 · Cited by 25 · see citing works
Boltze, J., Aronowski, J. A., Badaut, J., Buckwalter, M. S., Caleo, M., Chopp, M., Dave, K. R., Didwischus, N., Dijkhuizen, R. M., Doeppner, T. R., et al. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.623751 · Cited by 22 · see citing works
“Moreover, with the refinement of our research tools, some of which have been discussed herein, and increasing study quality (Ramirez et al., 2017), it seems that the translational roadblock is slowly overcome as new therapeutic concepts start to arise (Liebeskind et al., 2018).”
BackgroundShiomi, M. (2019) · Journal of Atherosclerosis and Thrombosis · doi:10.5551/jat.rv17038-2 · Cited by 21 · see citing works
King, D. R., Hardin, K. M., Hoeker, G. S., & Poelzing, S. (2022) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00164.2022 · Cited by 21 · see citing works
Ramirez, F. D., Jung, R. G., Motazedian, P., Perry‐Nguyen, D., Santo, P. D., MacDonald, Z., Clancy, A. A., Labinaz, A., Promislow, S., Simard, T., et al. (2019) · Stroke · doi:10.1161/strokeaha.119.026564 · Cited by 23 · see citing works
Self-citationYang, J., Wang, J., & Saeed, Y. (2022) · Stem Cells International · doi:10.1155/2022/5930244 · Cited by 19 · see citing works
“Since the inadequate quality of preclinical tests has been previously considered a major reason for the unsuccessful translation of experimental stroke therapies into the clinic [88, 89], therefore, it is necessary to evaluate and perform preclinical and clinical trials under Good Manufacturing Practice (GMP) before p…”
Martella, A. M., Martella, R. C., Yatcilla, J., Newson, A., Shannon, E. N., & Voorhis, C. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09826-1 · Cited by 25 · see citing works
“There have been many approaches taken to assess the methodological rigor in a number of disciplines (e.g., Bratt & Moons, 2015; Garavan et al., 2019; Pennington et al., 2021; Ramirez et al., 2017; Yang et al., 2012).”
Bolon, B. N., Baze, W. B., Shilling, C. J., Keatley, K. L., Patrick, D. J., & Schafer, K. A. (2018) · ILAR Journal · doi:10.1093/ilar/ily008 · Cited by 20 · see citing works
Hietamies, T. M., Ostrowski, C., Pei, Z., Feng, L., McCabe, C., Work, L. M., & Quinn, T. (2018) · Journal of Cerebral Blood Flow and Metabolism · doi:10.1177/0271678x18799858 · Cited by 20 · see citing works
Assessing the Completeness of Reporting in Preclinical Oncolytic Virus Therapy Studies
Fergusson, D. A., Wesch, N. L., Leung, G. J., MacNeil, J. L., Conic, I., Presseau, J., Cobey, K. D., Diallo, J., Auer, R. C., Kimmelman, J., et al. (2019) · Molecular Therapy — Oncolytics · doi:10.1016/j.omto.2019.05.004 · Cited by 18 · see citing works
“ca INTRODUCTION Incomplete and non-transparent reporting of animal experiments(1) contributes to the irreproducibility of basic science studies.(2,3) Such irreproducibility may help explain high attrition rates in drug development.”
BackgroundFemale Authorship in Preclinical Cardiovascular Research
Labinaz, A., Marbach, J. A., Jung, R. G., Moreland, R., Motazedian, P., Santo, P. D., Clancy, A. A., MacDonald, Z., Simard, T., Hibbert, B. M., et al. (2019) · JACC Basic to Translational Science · doi:10.1016/j.jacbts.2019.04.004 · Cited by 19 · see citing works
“The journals examined were selected based on their prominence in cardiovascular research, their collective focus on a wide range of cardiovascular disorders, and their endorsement of NIH guidelines on rigor and reproducibility (10).”
BackgroundMethodInfluentialSelf-citationSimard, T., Alqahtani, F., Hibbert, B. M., Mamas, M. A., El‐Hajj, S., Harris, A. H., Hohmann, S. F., & Alkhouli, M. A. (2021) · Catheterization and Cardiovascular Interventions · doi:10.1002/ccd.29499 · Cited by 19 · see citing works
Self-citationRisk of bias in exercise science: A systematic review of 340 studies
Preobrazenski, N., McCaig, A., Turner, A., Kushner, M., Pacitti, L. J., Mendolia, P., MacDonald, B., Storoschuk, K. L., Bouck, T., Zaza, Y., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109010 · Cited by 17 · see citing works
“79 However, we selected the 1.0 Tool to facilitate comparisons with other studies that have performed bias assessments in the preclinical 27 and clinical 26,35 sciences.”
BackgroundMethodInfluentialTraumatic brain injury: Symptoms to systems in the 21st century
Dobson, G. P., Morris, J. L., & Letson, H. L. (2024) · Brain Research · doi:10.1016/j.brainres.2024.149271 · Cited by 19 · see citing works
Mechanisms of Sex Disparities in Cardiovascular Function and Remodeling
Chaudhari, S. K., Cushen, S. C., Osikoya, O., Jaini, P. A., Posey, R., Mathis, K. W., & Goulopoulou, S. (2018) · Comprehensive physiology · doi:10.1002/cphy.c180003 · Cited by 16 · see citing works
Kiernan, M., & Baiocchi, M. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac019 · Cited by 15 · see citing works
Circadian effects on stroke outcome – Did we not wake up in time for neuroprotection?
Boltze, J., Didwischus, N., Merrow, M. W., Dallmann, R., & Plesnila, N. (2020) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x20978711 · Cited by 14 · see citing works
“Major improvements in methodological quality were achieved in the stroke field, which may now have outpaced other areas of cardiovascular research in this regard.(1) Moreover, an increasing body of evidence suggests that animal models adequately reflect human stroke pathophysiology and outcome when used appropriately.”
Modifiable Risk Factors and Residual Risk Following Coronary Revascularization
Simard, T., Jung, R. G., Santo, P. D., Harnett, D. T., Abdel‐Razek, O., Ramirez, F. D., Motazedian, P., Parlow, S., Labinaz, A., Moreland, R., et al. (2021) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2021.09.001 · Cited by 14 · see citing works
Self-citationChallenges of the heterogeneous nutrition response: interpreting the group mean
Drew, J. E. (2019) · Proceedings of The Nutrition Society · doi:10.1017/s002966511900096x · Cited by 15 · see citing works
Polypathologies and Animal Models of Traumatic Brain Injury
Freeman-Jones, E., Miller, W. H., Work, L. M., & Fullerton, J. L. (2023) · Brain Sciences · doi:10.3390/brainsci13121709 · Cited by 13 · see citing works
“However, poor bench-to-bedside translation is not specific to TBI itself and we see that other domains of research, like cardiovascular studies, encounter similar challenges [18].”
Backgroundvan der Naald, M., Chamuleau, S. A. J., Menon, J. M. L., Leeuw, W. D., de Haan, J. J., Duncker, D. J., & Wever, K. E. (2021) · PLoS Biology · doi:10.1371/journal.pbio.3001397 · Cited by 13 · see citing works
“Reporting of animal studies is poor, including incomplete reporting of study details, measures to reduce bias, and statistical analyses [1].”
BackgroundLalu, M. M., Presseau, J., Foster, M., Hunniford, V. T., Cobey, K. D., Brehaut, J., Ilkow, C. S., Montroy, J. R., Cardenas, A., Sharif, A., et al. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3001932 · Cited by 11 · see citing works
Divergent effects of tumor necrosis factor (TNF) in sepsis: a meta-analysis of experimental studies
Kassasseya, C., Torsin, L. I., Musset, C., Benhamou, M., Chaudry, I. H., Cavaillon, J., Grall, N., Monteiro, R. C., de Chaisemartin, L., Longrois, D., et al. (2024) · Critical Care · doi:10.1186/s13054-024-05057-0 · Cited by 12 · see citing works
“However, a more recent meta-analysis focusing on animal studies in cardiology from 2006 to 2016 found better rates (78% and 67%), suggesting some improvement over time [85].”
Jung, R. G., Stotts, C., Makwana, D., Motazedian, P., Santo, P. D., Goh, C. Y., Verreault‐Julien, L., Simard, T., Ramirez, F. D., & Hibbert, B. M. (2021) · Circulation Research · doi:10.1161/circresaha.121.319921 · Cited by 12 · see citing works
Self-citationDemographic and Methodological Heterogeneity in Electrocardiogram Signals From Guinea Pigs
Haq, K. T., Cooper, B. L., Berk, F., Roberts, A., Swift, L., & Posnack, N. G. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.925042 · Cited by 12 · see citing works
“The lack of studies focused on the heterogeneity of recording methodologies can contribute to the ‘reproducibility crisis’, influence ECG recording interpretation, and limit the broad utility of experimental data sets (Ramirez et al., 2017).”
BackgroundCardiology researchers’ practices and perceived barriers to open science: an international survey
Cobey, K. D., Alayche, M., Saba, S., Barnes, N. Y., Ebrahimzadeh, S., Alarcon, E. I., Hibbert, B. M., & Moher, D. (2024) · Open Heart · doi:10.1136/openhrt-2023-002433 · Cited by 10 · see citing works
Self-citationGonadal sex and animal experimentation: Perfection vs. 3R principle?
Bie, P., & Debrabant, B. (2020) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/bcpt.13411 · Cited by 10 · see citing works
“Inadequacy of the information provided in scientific articles is not a new phenomenon (4-6), but in recent years, considerable concern has mounted among researchers, funding agencies, editors and publishers (7-18) leading to a drive towards improvements of the value of experimental results, e.”
BackgroundO'Connor, A., Totton, S., Cullen, J. N., Ramezani, M., Kalivarapu, V., Yuan, C., & Gilbert, S. B. (2018) · PLoS ONE · doi:10.1371/journal.pone.0199441 · Cited by 8 · see citing works
Hankenson, F. C., Prager, E. M., & Berridge, B. R. (2024) · Annual Review of Animal Biosciences · doi:10.1146/annurev-animal-021022-043531 · Cited by 10 · see citing works
Adenosine as a Marker and Mediator of Cardiovascular Homeostasis: A Translational Perspective
Simard, T., Jung, R. G., Labinaz, A., Faraz, M. A., Ramirez, F. D., Santo, P. D., Pitcher, I., Motazedian, P., Gaudet, C., Rochman, R., et al. (2018) · Cardiovascular & Haematological Disorders - Drug Targets · doi:10.2174/1871529x18666181011103719 · Cited by 9 · see citing works
Self-citationRamirez, F. D., & Hibbert, B. M. (2018) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.118.310942 · Cited by 9 · see citing works
Self-citationJanakiram, N. B., Motherwell, J. M., Goldman, S. M., & Dearth, C. L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274132 · Cited by 8 · see citing works
“As discussed earlier, high-quality animal studies are crucial for the translation of animal data into clinical studies [52].”
BackgroundPreclinical Research Reporting in Shock: Room for Improvement
Reynolds, P. S., & GARVAN, C. W. (2020) · Shock · doi:10.1097/shk.0000000000001544 · Cited by 8 · see citing works
“Transparent reporting also improves communication of research methods and findings, facilitates extraction of key information for systematic reviews (1), and increases potential for translation (2, 3).”
BackgroundMethodJasińska-Stroschein, M., & Orszulak–Michalak, D. (2021) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2021.173952 · Cited by 7 · see citing works
Age-Related Differential Structural and Transcriptomic Responses in the Hypertensive Heart
Marques, F. Z., Chu, P., Ziemann, M., Kaspi, A., Kiriazis, H., Du, X.-J., El‐Osta, A., & Kaye, D. M. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00817 · Cited by 7 · see citing works
“The current crisis in lack of reproducibility of findings from preclinical to clinical studies and, thus, low translation of findings to the clinic, has unleashed a call for improved methodological rigor (Ramirez et al., 2017).”
BackgroundSandinge, A.-S., Janefeldt, A., Pehrsson, S., & Nylander, S. (2018) · PLoS ONE · doi:10.1371/journal.pone.0201202 · Cited by 7 · see citing works
I’ll Have the Rigor, but Hold the Mortis
Jones, S. P. (2017) · Circulation Research · doi:10.1161/circresaha.117.311114 · Cited by 7 · see citing works
“At another time, we can discuss the (ab)use of statistics in preclinical studies, which was also highlighted by Ramirez et al.2 Rather than list all of the oft-cited preclinical issues that potentially limit translation to the clinic, consider the share of ownership of this failure from the clinical side.”
InfluentialPaul Simpson and Scientific Rigor
Bolli, R. (2019) · Circulation Research · doi:10.1161/circresaha.118.314621 · Cited by 6 · see citing works
Editorial: In Musculoskeletal Research, Too Many Animals are Being Harmed for Too Small a Return
Leopold, S. S. (2021) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001661 · Cited by 6 · see citing works
Ignacio, A. C., Oliveira, N. L., da Silva, L. X. N., Feter, J. S., Nardi, A. T. D., Helal, L. C. A. S., dos Santos, M. R., Soares, D. D. S., Galliano, L. M., Alano, T. S., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309087 · Cited by 5 · see citing works
“The identified weaknesses can reduce reproducibility, reliability, and result in diminishing research waste to some extent [12].”
Sansbury, B. E., Nystoriak, M. A., Uchida, S., Wysoczynski, M., & Moore, J. B. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.913612 · Cited by 6 · see citing works
Animal Models and Cardiac Extracellular Matrix Research
Hacker, T. A. (2018) · Advances in experimental medicine and biology · doi:10.1007/978-3-319-97421-7_3 · Cited by 5 · see citing works
Biomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation
Defensor, E. B., Lim, M. A., & Schaevitz, L. R. (2021) · ILAR Journal · doi:10.1093/ilar/ilab018 · Cited by 4 · see citing works
Park, S. H., Lee, S. B., Park, S., Kim, E. Y., Pizzol, D., Trott, M., Barnett, Y., Koyanagi, A., Jacob, L., Soysal, P., et al. (2023) · British Journal of Urology · doi:10.1111/bju.16171 · Cited by 4 · see citing works
“…ureteric obstruction, infertility, kidney stone, micturition, and urinary retention), and animal model used (mouse, rat, rabbit, dog, pig, primate, sheep, others [guinea-pig, cats, alpaca, lamb, goat, bull, koala, miniature pig, hamster, and swine], and combination [multiple animals used]) [42,43].”
Leopold, S. S. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001420 · Cited by 4 · see citing works
Nong, Y., Guo, Y., Gumpert, A. M., Li, Q., Tomlin, A. A., Zhu, X., & Bolli, R. (2021) · Molecular and Cellular Biochemistry · doi:10.1007/s11010-021-04227-w · Cited by 4 · see citing works
Considerations on Preclinical Neuromuscular Disease Gene Therapy Studies
Duan, D. (2019) · Springer eBooks · doi:10.1007/978-3-030-03095-7_17 · Cited by 4 · see citing works
Connecting the dots in neuroscience research: The future of evidence synthesis
Hair, K., Arroyo-Araujo, M., Vojvodić, S., Economou, M., Wong, C., Tinsdeall, F., Smith, S., Rackoll, T., Sena, E. S., & McCann, S. (2024) · Experimental Neurology · doi:10.1016/j.expneurol.2024.115047 · Cited by 4 · see citing works
“Subsequent to these interventions, there is evidence for improved validity of stroke experiments, including increased reporting of measures to reduce risks of bias and other quality indicators (Minnerup et al., 2016), changes that are not evident in related fields (Ramirez et al., 2017).”
- Authors
- Ramirez, F. D., Motazedian, P., Jung, R. G., Di Santo, P., MacDonald, Z. D., Moreland, R., Simard, T., Clancy, A. A., Russo, J. J., Welch, V. A., Wells, G. A., & Hibbert, B.
- Affiliations
- From the Division of Cardiology (F.D.R., P.M., R.G.J., P.D.S., T.S., J.J.R., B.H.), CAPITAL Research Group (F.D.R., P.M., R.G.J., P.D.S., Z.D.M.D., R.M., T.S., J.J.R., B.H.), Vascular Biology and Experimental Medicine Laboratory (R.G.J., T.S., B.H.), and Cardiovascular Research Methods Centre (G.A.W.), University of Ottawa Heart Institute, Ontario, Canada; and School of Epidemiology, Public Health and Preventive Medicine (F.D.R., V.A.W., G.A.W.), Department of Cellular and Molecular Medicine (R.G.J....
- Corresponding author
- Hibbert, Benjamin
- ORCID
- Venue
- Circulation Research
- Published
- 2017-06-09
- Volume
- 120
- Issue
- 12
- Pages
- 1916-1926
- ISSN
- 0009-7330 (ISSN-L)
- 1524-4571
- PMC
- PMC5466021
- OpenAlex
- W2604920686
- Cited by
- 138
- References
- 74
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 11 · 2023: 12 · 2022: 16 · 2021: 17 · 2020: 22 · 2019: 18 · 2018: 25 · 2017: 8
- FWCI
- 31.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Circulation Research (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- methodological quality, randomization, blinding, sample size determination, sex as a biological variable, research bias, irreproducibility, citation impact, animal models, stroke research, research translation
- MeSH terms
- Animals, Cardiovascular Diseases, Disease Models, Animal, Drug Evaluation, Preclinical, Humans, Research Design, Reproducibility of Results, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
