The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Methodological Rigor in Preclinical Cardiovascular Studies: Targets to Enhance Reproducibility and Promote Research Translation
Open accessAbstract
Rationale: Methodological sources of bias and suboptimal reporting contribute to irreproducibility in preclinical science and may negatively affect research translation. Randomization, blinding, sample size estimation, and considering sex as a biological variable are deemed crucial study design elements to maximize the quality and predictive value of preclinical experiments. Objective: To examine the prevalence and temporal patterns of recommended study design element implementation in preclinical cardiovascular research. Methods and Results: All articles published over a 10-year period in 5 leading cardiovascular journals were reviewed. Reports of in vivo experiments in nonhuman mammals describing pathophysiology, genetics, or therapeutic interventions relevant to specific cardiovascular disorders were identified. Data on study design and animal model use were collected. Citations at 60 months were additionally examined as a surrogate measure of research impact in a prespecified subset of studies, stratified by individual and cumulative study design elements. Of 28 636 articles screened, 3396 met inclusion criteria. Randomization was reported in 21.8%, blinding in 32.7%, and sample size estimation in 2.3%. Temporal and disease-specific analyses show that the implementation of these study design elements has overall not appreciably increased over the past decade, except in preclinical stroke research, which has uniquely demonstrated significant improvements in methodological rigor. In a subset of 1681 preclinical studies, randomization, blinding, sample size estimation, and inclusion of both sexes were not associated with increased citations at 60 months. Conclusions: Methodological shortcomings are prevalent in preclinical cardiovascular research, have not substantially improved over the past 10 years, and may be overlooked when basing subsequent studies. Resultant risks of bias and threats to study validity have the potential to hinder progress in cardiovascular medicine as preclinical research often precedes and informs clinical trials. Stroke research quality has uniquely improved in recent years, warranting a closer examination for interventions to model in other cardiovascular fields.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In contrast, other studies using shorter checklists (primarily focused on experimental design) with more editorial follow-up have shown a marked improvement in the nature and detail of the information included in publications [34–36].”
From The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research (2020) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
SYRCLE’s risk of bias tool for animal studies
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-43 · Cited by 4,415
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 5×
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
InfluentialClinical development success rates for investigational drugs
Hay, M., Thomas, D. W., Craighead, J. L., Economides, C., & Rosenthal, J. (2014) · Nature Biotechnology · doi:10.1038/nbt.2786 · Cited by 2,536 · cited 1×
“Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among the lowest success rates and likelihood of approval at all phases of clinical trials.(24) Moreover, research and…”
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154
Research works printed but not linked: 3
- Lehrer J. The truth wears off: Is there something wrong with the scientific method? The New Yorker. 2010.
- Freedman DH. Lies, damned lies, and medical science. The Atlantic. 2010.
- 10.1038/scibx.2011.1416 · DOI
Cited by
Sorted by citations
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000410 · Cited by 5,939
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Experimental Physiology · doi:10.1113/ep088870 · Cited by 3,547
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research*
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x20943823 · Cited by 1,443
“34– 36 It is likely that the level of resource required from journals and editors currently prohibits the implementation of all the items of the ARRIVE guidelines.”
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · British Journal of Pharmacology · doi:10.1111/bph.15193 · Cited by 1,152
The ARRIVE guidelines 2019: updated guidelines for reporting animal researchPreprint
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/703181 · Cited by 1,044
The ARRIVE guidelines 2.0: updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · The Journal of Physiology · doi:10.1113/jp280389 · Cited by 633
“…contrast, other studies using shorter checklists (primarily focused on experimental design) with more editorial follow-up have shown a marked improvement in the nature and detail of the information included in publications (Han et al. 2017, Ramirez et al. 2017; The NPQIP Collaborative group, 2019).”
ResultGuidelines for experimental models of myocardial ischemia and infarction
Lindsey, M. L., Bolli, R., Canty, J. M., Du, X.-J., Frangogiannis, N. G., Frantz, S., Gourdie, R. G., Holmes, J. W., Jones, S. P., Kloner, R. A., et al. (2018) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00335.2017 · Cited by 526
“Much of the lack of reproducibility has been ascribed to limited or lacking scientific rigor (29, 250).”
The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · BMC Veterinary Research · doi:10.1186/s12917-020-02451-y · Cited by 531
Bøtker, H. E., Hausenloy, D. J., Andreadou, I., Antonucci, S., Boengler, K., Davidson, S. M., Deshwal, S., Devaux, Y., Lisa, F. D., Sante, M. D., et al. (2018) · Basic Research in Cardiology · doi:10.1007/s00395-018-0696-8 · Cited by 436
Challenges for assessing replicability in preclinical cancer biology
Errington, T. M., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021) · eLife · doi:10.7554/elife.67995 · Cited by 345
du Sert, N. P., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · BMJ Open Science · doi:10.1136/bmjos-2020-100115 · Cited by 309
Methodological quality of COVID-19 clinical research
Jung, R. G., Di Santo, P., Clifford, C., Prosperi-Porta, G., Skanes, S., Hung, A., Parlow, S., Visintini, S., Ramirez, F. D., Simard, T., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21220-5 · Cited by 189
Self-citationLecour, S., Andreadou, I., Bøtker, H. E., Davidson, S. M., Heusch, G., Ruiz‐Meana, M., Schulz, R., Zuurbier, C. J., Ferdinandy, P., Hausenloy, D. J., et al. (2021) · Basic Research in Cardiology · doi:10.1007/s00395-021-00893-5 · Cited by 142
Provencher, S., Archer, S. L., Ramirez, F. D., Hibbert, B. M., Paulin, R., Boucherat, O., Lacasse, Y., & Bonnet, S. (2018) · Circulation Research · doi:10.1161/circresaha.117.312579 · Cited by 139
“However, few journals enforce methodological and reporting guidelines for confirmatory studies, despite being associated with significant improvements in methodological quality in other fields of preclinical research.(28,29,100)”
Self-citationHan, S., Olonisakin, T. F., Pribis, J. P., Zupetic, J., Yoon, J. H., Holleran, K. M., Jeong, K., Shaikh, N., Rubio, D. M., & Lee, J. S. (2017) · PLOS ONE · doi:10.1371/journal.pone.0183591 · Cited by 131
An Analysis of Mesenchymal Stem Cell-Derived Extracellular Vesicles for Preclinical Use
Tieu, A., Lalu, M. M., Slobodian, M., Gnyra, C., Fergusson, D. A., Montroy, J. R., Burger, D. E., Stewart, D. J., & Allan, D. (2020) · ACS Nano · doi:10.1021/acsnano.0c01363 · Cited by 139
Reproducibility and Rigor in Animal-Based Research
Macleod, M., & Mohan, S. (2019) · ILAR Journal · doi:10.1093/ilar/ilz015 · Cited by 104
“of animal research in 4 other journals from the same publisher even when adjusted for the disease under study and the species of animal used [44].”
BackgroundDid a change in Nature journals’ editorial policy for life sciences research improve reporting?
The NPQIP Collaborative group, Study steering committee, Macleod, M., Sena, E., Howells, D., Macleod, M., Sena, E., Howells, D., Kiermer, V., Swaminathan, S., et al. (2019) · BMJ Open Science · doi:10.1136/bmjos-2017-000035 · Cited by 102
Self-citationZingarelli, B., Coopersmith, C. M., Drechsler, S., Efron, P. A., Marshall, J. C., Moldawer, L. L., Wiersinga, W. J., Xiao, X., Osuchowski, M. F., & Thiemermann, C. (2018) · Shock · doi:10.1097/shk.0000000000001243 · Cited by 91
“lowest use of randomization (17%) was reported in mousebased experiments (49)—the species whose size enables repet-”
BackgroundThe gut microbiome of laboratory mice: considerations and best practices for translational research
Ericsson, A. C., & Franklin, C. L. (2021) · Mammalian Genome · doi:10.1007/s00335-021-09863-7 · Cited by 90
Translational Research in Cardiovascular Repair
Chamuleau, S. A. J., van der Naald, M., Climent, A. M., Kraaijeveld, A. O., Wever, K., Duncker, D. J., Fernández‐Avilés, F., & Bolli, R. (2018) · Circulation Research · doi:10.1161/circresaha.117.311565 · Cited by 78
“3,7,9,10,45 A detailed set of guidelines aimed at increasing rigor and reproducibility in preclinical research has recently been published.”
Cui, L., Gołubczyk, D., Tolppanen, A., Boltze, J., & Jolkkonen, J. (2019) · Annals of Neurology · doi:10.1002/ana.25493 · Cited by 66
“stroke therapies into the clinic, but the quality of preclinical stroke studies is clearly improving.(9) However, significant”
BackgroundMelleby, A. O., Romaine, A., Aronsen, J. M., Veras, I., Zhang, L., Sjaastad, I., Lunde, I. G., & Christensen, G. (2018) · Cardiovascular Research · doi:10.1093/cvr/cvy141 · Cited by 64
Shin, H. S., Shin, H., & Shudo, Y. (2021) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2021.673683 · Cited by 63
“…of original papers published in cardiovascular journals in the same period were poorly cited (with < = 5 citations in the 5 years following publication); however, 44% of cardiovascular journals had more than threefourths of the journal’s content poorly cited at 5 years (Ranasinghe et al., 2015).”
Hypertrophy Regression With N-Acetylcysteine in Hypertrophic Cardiomyopathy (HALT-HCM)
Marian, A. J., Tan, Y., Li, L., Chang, J., Syrris, P., Hessabi, M., Rahbar, M. H., Willerson, J. T., Cheong, B., Liu, C., et al. (2018) · Circulation Research · doi:10.1161/circresaha.117.312647 · Cited by 61
Benefits of a factorial design focusing on inclusion of female and male animals in one experiment
Buch, T., Moos, K., Ferreira, F. M., Fröhlich, H., Gebhard, C., & Tresch, A. (2019) · Journal of Molecular Medicine · doi:10.1007/s00109-019-01774-0 · Cited by 61
Be positive about negatives–recommendations for the publication of negative (or null) results
Bespalov, A., Steckler, T., & Skolnick, P. (2019) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2019.10.007 · Cited by 59
Premise for Standardized Sepsis Models
Remick, D. G., Ayala, A., Chaudry, I. H., Coopersmith, C. M., Deutschman, C. S., Hellman, J., Moldawer, L. L., & Osuchowski, M. F. (2018) · Shock · doi:10.1097/shk.0000000000001164 · Cited by 61
“A 2017 paper (23) analyzed the level of rigor in preclinical cardiovascular studies by examining nearly 30,000 published studies.”
BackgroundPalmer, D. D., Dumont, J. R., Dexter, T. D., Prado, M. A. M., Finger, E. C., Bussey, T. J., & Saksida, L. M. (2021) · Neurobiology of Learning and Memory · doi:10.1016/j.nlm.2021.107443 · Cited by 54
Donner, D. G., Kiriazis, H., Du, X., Marwick, T. H., & McMullen, J. R. (2018) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00157.2018 · Cited by 48
Optimizing functional outcome endpoints for stroke recovery studies
Balkaya, M., & Cho, S. (2019) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x19875212 · Cited by 41
“While certain crucial recommendations of these initiatives are yet to be adequately implemented,(7) the scientific rigor in stroke studies has been ameliorated drastically.(6) Despite improvements, preclinical research is still affected by quality issues, including publication bias, false positives, lack of randomizat…”
Gender Differences in Cardiac Chronotropic Control: Implications for Heart Rate Variability Research
Williams, D. P., Joseph, N. C., Gerardo, G. M., Hill, L. K., Koenig, J., & Thayer, J. F. (2021) · Applied Psychophysiology and Biofeedback · doi:10.1007/s10484-021-09528-w · Cited by 41
van der Naald, M., Wenker, S., Doevendans, P. A., Wever, K. E., & Chamuleau, S. A. J. (2020) · BMJ Open Science · doi:10.1136/bmjos-2019-100051 · Cited by 40
Fergusson, D. A., Avey, M. T., Barron, C. C., Bocock, M., Biefer, K., Boet, S., Bourque, S. L., Conic, I., Chen, K., Dong, Y., et al. (2019) · PLoS ONE · doi:10.1371/journal.pone.0215221 · Cited by 38
“Widely cited factors influencing this irreproducibility are poor preclinical study design, incomplete reporting, and a lack of transparency of results [4, 6, 7].”
BackgroundEffect of Intracoronary Metformin on Myocardial Infarct Size in Swine
Techiryan, G., Weil, B. R., Palka, B., & Canty, J. M. (2018) · Circulation Research · doi:10.1161/circresaha.118.313341 · Cited by 36
Zheng, Q., Zhu, J., Bao, X., Zhu, P.-C., Tong, Q., Huang, Y.-Y., Zhang, Q., Zhang, K., Zheng, G., & Wang, Y. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00795 · Cited by 35
Clinical and Preclinical Systematic Review of Panax ginseng C. A. Mey and Its Compounds for Fatigue
Jin, T.-Y., Rong, P.-Q., Liang, H., Zhang, P., Zheng, G., & Lin, Y. (2020) · Frontiers in Pharmacology · doi:10.3389/fphar.2020.01031 · Cited by 34
Bolli, R. (2017) · Circulation Research · doi:10.1161/circresaha.117.311678 · Cited by 34
Buglak, N. E., Lucitti, J. L., Ariel, P., Maiocchi, S. L., Miller, F. J., & Bahnson, E. M. (2020) · Cardiovascular Research · doi:10.1093/cvr/cvaa037 · Cited by 33
Risk of bias reporting in the recent animal focal cerebral ischaemia literature
Bahor, Z., Liao, J., Macleod, M. R., Bannach‐Brown, A., McCann, S., Wever, K. E., Thomas, J., Ottavi, T., Howells, D. W., Rice, A., et al. (2017) · Clinical Science · doi:10.1042/cs20160722 · Cited by 31
Jin, X., Li, P., Michalski, D., Li, S., Zhang, Y., Jolkkonen, J., Cui, L., Didwischus, N., Xuan, W., & Boltze, J. (2022) · CNS Neuroscience & Therapeutics · doi:10.1111/cns.13816 · Cited by 31
McCann, S., & Lawrence, C. B. (2020) · BMJ Open Science · doi:10.1136/bmjos-2019-100013 · Cited by 31
Systematic Interrogation of Angiogenesis in the Ischemic Mouse Hind Limb
Lee, J. J., Arpino, J., Yin, H. R., Nong, Z., Szpakowski, A., Hashi, A. A., Chevalier, J. M., O’Neil, C. H., & Pickering, J. G. (2020) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.120.315028 · Cited by 31
Female sex-specific considerations to improve rigor and reproducibility in cardiovascular research
Chang, D. H., Dumanski, S. M., & Ahmed, S. B. (2022) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00462.2022 · Cited by 34
Ameri, P., Mercurio, V., Pollesello, P., Anker, M. S., Backs, J., Bayés‐Genís, A., Borlaug, B. A., Burkhoff, D., Caravita, S., Chan, S. Y., et al. (2024) · European Journal of Heart Failure · doi:10.1002/ejhf.3236 · Cited by 28
Western diet increases cardiac ceramide content in healthy and hypertrophied hearts
Butler, T., Ashford, D. A., & Seymour, A.-M. (2017) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2017.08.007 · Cited by 27
van der Naald, M., Chamuleau, S. A. J., Menon, J. M. L., de Leeuw, W., de Haan, J., Duncker, D. J., & Wever, K. E. (2022) · BMJ Open Science · doi:10.1136/bmjos-2021-100259 · Cited by 26
Coronary artery disease in post-menopausal women: are there appropriate means of assessment?
Chiu, M. H., Heydari, B., Batulan, Z., Maarouf, N., Subramanya, V., Schenck-Gustafsson, K. G., & O’Brien, E. R. (2018) · Clinical Science · doi:10.1042/cs20180067 · Cited by 26
Felizardo, A. A., Marques, D. V. B., Caldas, I. S., Gonçalves, R. V., & Novaes, R. D. (2018) · Experimental Gerontology · doi:10.1016/j.exger.2018.01.022 · Cited by 26
Bonafiglia, J. T., Islam, H., Preobrazenski, N., & Gurd, B. J. (2021) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2021.03.005 · Cited by 30
Boltze, J., & Pérez‐Pinzón, M. A. (2022) · Stroke · doi:10.1161/strokeaha.122.039005 · Cited by 25
From methods to monographs: fostering a culture of research quality
Crawford, D. C., Hoye, M. L., & Silberberg, S. D. (2023) · eneuro · doi:10.1523/eneuro.0247-23.2023 · Cited by 25
“Similar findings were found in an evaluation of preclinical cardiovascular studies (Ramirez et al., 2017) and in an evaluation of papers published by Nature Publishing Group (NPQIP Collaborative group, 2019).”
Macleod, M. R., & The NPQIP Collaborative group (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/187245 · Cited by 27
Boltze, J., Aronowski, J. A., Badaut, J., Buckwalter, M. S., Caleo, M., Chopp, M., Dave, K. R., Didwischus, N., Dijkhuizen, R. M., Doeppner, T. R., et al. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.623751 · Cited by 22
“Moreover, with the refinement of our research tools, some of which have been discussed herein, and increasing study quality (Ramirez et al., 2017), it seems that the translational roadblock is slowly overcome as new therapeutic concepts start to arise (Liebeskind et al., 2018).”
BackgroundShiomi, M. (2019) · Journal of Atherosclerosis and Thrombosis · doi:10.5551/jat.rv17038-2 · Cited by 21
King, D. R., Hardin, K. M., Hoeker, G. S., & Poelzing, S. (2022) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00164.2022 · Cited by 21
Ramirez, F. D., Jung, R. G., Motazedian, P., Perry‐Nguyen, D., Santo, P. D., MacDonald, Z., Clancy, A. A., Labinaz, A., Promislow, S., Simard, T., et al. (2019) · Stroke · doi:10.1161/strokeaha.119.026564 · Cited by 23
Self-citationYang, J., Wang, J., & Saeed, Y. (2022) · Stem Cells International · doi:10.1155/2022/5930244 · Cited by 19
“Since the inadequate quality of preclinical tests has been previously considered a major reason for the unsuccessful translation of experimental stroke therapies into the clinic [88, 89], therefore, it is necessary to evaluate and perform preclinical and clinical trials under Good Manufacturing Practice (GMP) before p…”
Martella, A. M., Martella, R. C., Yatcilla, J., Newson, A., Shannon, E. N., & Voorhis, C. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09826-1 · Cited by 25
“There have been many approaches taken to assess the methodological rigor in a number of disciplines (e.g., Bratt & Moons, 2015; Garavan et al., 2019; Pennington et al., 2021; Ramirez et al., 2017; Yang et al., 2012).”
Bolon, B. N., Baze, W. B., Shilling, C. J., Keatley, K. L., Patrick, D. J., & Schafer, K. A. (2018) · ILAR Journal · doi:10.1093/ilar/ily008 · Cited by 20
Hietamies, T. M., Ostrowski, C., Pei, Z., Feng, L., McCabe, C., Work, L. M., & Quinn, T. J. (2018) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x18799858 · Cited by 19
Assessing the Completeness of Reporting in Preclinical Oncolytic Virus Therapy Studies
Fergusson, D. A., Wesch, N. L., Leung, G. J., MacNeil, J. L., Conic, I., Presseau, J., Cobey, K. D., Diallo, J., Auer, R. C., Kimmelman, J., et al. (2019) · Molecular Therapy — Oncolytics · doi:10.1016/j.omto.2019.05.004 · Cited by 18
“ca INTRODUCTION Incomplete and non-transparent reporting of animal experiments(1) contributes to the irreproducibility of basic science studies.(2,3) Such irreproducibility may help explain high attrition rates in drug development.”
BackgroundFemale Authorship in Preclinical Cardiovascular Research
Labinaz, A., Marbach, J. A., Jung, R. G., Moreland, R., Motazedian, P., Santo, P. D., Clancy, A. A., MacDonald, Z., Simard, T., Hibbert, B. M., et al. (2019) · JACC Basic to Translational Science · doi:10.1016/j.jacbts.2019.04.004 · Cited by 19
“The journals examined were selected based on their prominence in cardiovascular research, their collective focus on a wide range of cardiovascular disorders, and their endorsement of NIH guidelines on rigor and reproducibility (10).”
BackgroundMethodInfluentialSelf-citationSimard, T., Alqahtani, F., Hibbert, B. M., Mamas, M. A., El‐Hajj, S., Harris, A. H., Hohmann, S. F., & Alkhouli, M. A. (2021) · Catheterization and Cardiovascular Interventions · doi:10.1002/ccd.29499 · Cited by 19
Self-citationRisk of bias in exercise science: A systematic review of 340 studies
Preobrazenski, N., McCaig, A., Turner, A., Kushner, M., Pacitti, L. J., Mendolia, P., MacDonald, B., Storoschuk, K. L., Bouck, T., Zaza, Y., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109010 · Cited by 23
“79 However, we selected the 1.0 Tool to facilitate comparisons with other studies that have performed bias assessments in the preclinical 27 and clinical 26,35 sciences.”
BackgroundMethodInfluentialTraumatic brain injury: Symptoms to systems in the 21st century
Dobson, G. P., Morris, J. L., & Letson, H. L. (2024) · Brain Research · doi:10.1016/j.brainres.2024.149271 · Cited by 20
Mechanisms of Sex Disparities in Cardiovascular Function and Remodeling
Chaudhari, S. K., Cushen, S. C., Osikoya, O., Jaini, P. A., Posey, R., Mathis, K. W., & Goulopoulou, S. (2018) · Comprehensive physiology · doi:10.1002/cphy.c180003 · Cited by 16
Kiernan, M., & Baiocchi, M. (2022) · American Journal of Epidemiology · doi:10.1093/aje/kwac019 · Cited by 21
Circadian effects on stroke outcome – Did we not wake up in time for neuroprotection?
Boltze, J., Didwischus, N., Merrow, M. W., Dallmann, R., & Plesnila, N. (2020) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x20978711 · Cited by 14
“Major improvements in methodological quality were achieved in the stroke field, which may now have outpaced other areas of cardiovascular research in this regard.(1) Moreover, an increasing body of evidence suggests that animal models adequately reflect human stroke pathophysiology and outcome when used appropriately.”
Modifiable Risk Factors and Residual Risk Following Coronary Revascularization
Simard, T., Jung, R. G., Santo, P. D., Harnett, D. T., Abdel‐Razek, O., Ramirez, F. D., Motazedian, P., Parlow, S., Labinaz, A., Moreland, R., et al. (2021) · Mayo Clinic Proceedings Innovations Quality & Outcomes · doi:10.1016/j.mayocpiqo.2021.09.001 · Cited by 14
Self-citationChallenges of the heterogeneous nutrition response: interpreting the group mean
Drew, J. E. (2019) · Proceedings of The Nutrition Society · doi:10.1017/s002966511900096x · Cited by 15
Polypathologies and Animal Models of Traumatic Brain Injury
Freeman-Jones, E., Miller, W. H., Work, L. M., & Fullerton, J. L. (2023) · Brain Sciences · doi:10.3390/brainsci13121709 · Cited by 15
“However, poor bench-to-bedside translation is not specific to TBI itself and we see that other domains of research, like cardiovascular studies, encounter similar challenges [18].”
Backgroundvan der Naald, M., Chamuleau, S. A. J., Menon, J. M. L., Leeuw, W. D., de Haan, J. J., Duncker, D. J., & Wever, K. E. (2021) · PLoS Biology · doi:10.1371/journal.pbio.3001397 · Cited by 13
“Reporting of animal studies is poor, including incomplete reporting of study details, measures to reduce bias, and statistical analyses [1].”
BackgroundLalu, M. M., Presseau, J., Foster, M., Hunniford, V. T., Cobey, K. D., Brehaut, J., Ilkow, C. S., Montroy, J. R., Cardenas, A., Sharif, A., et al. (2023) · PLoS Biology · doi:10.1371/journal.pbio.3001932 · Cited by 11
Divergent effects of tumor necrosis factor (TNF) in sepsis: a meta-analysis of experimental studies
Kassasseya, C., Torsin, L. I., Musset, C., Benhamou, M., Chaudry, I. H., Cavaillon, J., Grall, N., Monteiro, R. C., de Chaisemartin, L., Longrois, D., et al. (2024) · Critical Care · doi:10.1186/s13054-024-05057-0 · Cited by 13
“However, a more recent meta-analysis focusing on animal studies in cardiology from 2006 to 2016 found better rates (78% and 67%), suggesting some improvement over time [85].”
Jung, R. G., Stotts, C., Makwana, D., Motazedian, P., Santo, P. D., Goh, C. Y., Verreault‐Julien, L., Simard, T., Ramirez, F. D., & Hibbert, B. M. (2021) · Circulation Research · doi:10.1161/circresaha.121.319921 · Cited by 12
Self-citationDemographic and Methodological Heterogeneity in Electrocardiogram Signals From Guinea Pigs
Haq, K. T., Cooper, B. L., Berk, F., Roberts, A., Swift, L., & Posnack, N. G. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.925042 · Cited by 12
“The lack of studies focused on the heterogeneity of recording methodologies can contribute to the ‘reproducibility crisis’, influence ECG recording interpretation, and limit the broad utility of experimental data sets (Ramirez et al., 2017).”
BackgroundCardiology researchers’ practices and perceived barriers to open science: an international survey
Cobey, K. D., Alayche, M., Saba, S., Barnes, N. Y., Ebrahimzadeh, S., Alarcon, E. I., Hibbert, B. M., & Moher, D. (2024) · Open Heart · doi:10.1136/openhrt-2023-002433 · Cited by 11
Self-citationGonadal sex and animal experimentation: Perfection vs. 3R principle?
Bie, P., & Debrabant, B. (2020) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/bcpt.13411 · Cited by 12
“Inadequacy of the information provided in scientific articles is not a new phenomenon (4-6), but in recent years, considerable concern has mounted among researchers, funding agencies, editors and publishers (7-18) leading to a drive towards improvements of the value of experimental results, e.”
BackgroundO'Connor, A., Totton, S., Cullen, J. N., Ramezani, M., Kalivarapu, V., Yuan, C., & Gilbert, S. B. (2018) · PLoS ONE · doi:10.1371/journal.pone.0199441 · Cited by 8
Hankenson, F. C., Prager, E. M., & Berridge, B. R. (2024) · Annual Review of Animal Biosciences · doi:10.1146/annurev-animal-021022-043531 · Cited by 10
Adenosine as a Marker and Mediator of Cardiovascular Homeostasis: A Translational Perspective
Simard, T., Jung, R. G., Labinaz, A., Faraz, M. A., Ramirez, F. D., Santo, P. D., Pitcher, I., Motazedian, P., Gaudet, C., Rochman, R., et al. (2018) · Cardiovascular & Haematological Disorders - Drug Targets · doi:10.2174/1871529x18666181011103719 · Cited by 9
Self-citationRamirez, F. D., & Hibbert, B. M. (2018) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.118.310942 · Cited by 11
Self-citationJanakiram, N. B., Motherwell, J. M., Goldman, S. M., & Dearth, C. L. (2022) · PLoS ONE · doi:10.1371/journal.pone.0274132 · Cited by 8
“As discussed earlier, high-quality animal studies are crucial for the translation of animal data into clinical studies [52].”
BackgroundPreclinical Research Reporting in Shock: Room for Improvement
Reynolds, P. S., & GARVAN, C. W. (2020) · Shock · doi:10.1097/shk.0000000000001544 · Cited by 8
“Transparent reporting also improves communication of research methods and findings, facilitates extraction of key information for systematic reviews (1), and increases potential for translation (2, 3).”
BackgroundMethodJasińska-Stroschein, M., & Orszulak–Michalak, D. (2021) · European Journal of Pharmacology · doi:10.1016/j.ejphar.2021.173952 · Cited by 7
Age-Related Differential Structural and Transcriptomic Responses in the Hypertensive Heart
Marques, F. Z., Chu, P., Ziemann, M., Kaspi, A., Kiriazis, H., Du, X.-J., El‐Osta, A., & Kaye, D. M. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00817 · Cited by 7
“The current crisis in lack of reproducibility of findings from preclinical to clinical studies and, thus, low translation of findings to the clinic, has unleashed a call for improved methodological rigor (Ramirez et al., 2017).”
BackgroundSandinge, A.-S., Janefeldt, A., Pehrsson, S., & Nylander, S. (2018) · PLoS ONE · doi:10.1371/journal.pone.0201202 · Cited by 7
I’ll Have the Rigor, but Hold the Mortis
Jones, S. P. (2017) · Circulation Research · doi:10.1161/circresaha.117.311114 · Cited by 7
“At another time, we can discuss the (ab)use of statistics in preclinical studies, which was also highlighted by Ramirez et al.2 Rather than list all of the oft-cited preclinical issues that potentially limit translation to the clinic, consider the share of ownership of this failure from the clinical side.”
InfluentialPaul Simpson and Scientific Rigor
Bolli, R. (2019) · Circulation Research · doi:10.1161/circresaha.118.314621 · Cited by 6
Editorial: In Musculoskeletal Research, Too Many Animals are Being Harmed for Too Small a Return
Leopold, S. S. (2021) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001661 · Cited by 6
Ignacio, A. C., Oliveira, N. L., da Silva, L. X. N., Feter, J. S., Nardi, A. T. D., Helal, L. C. A. S., dos Santos, M. R., Soares, D. D. S., Galliano, L. M., Alano, T. S., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0309087 · Cited by 5
“The identified weaknesses can reduce reproducibility, reliability, and result in diminishing research waste to some extent [12].”
Sansbury, B. E., Nystoriak, M. A., Uchida, S., Wysoczynski, M., & Moore, J. B. (2022) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2022.913612 · Cited by 7
Animal Models and Cardiac Extracellular Matrix Research
Hacker, T. A. (2018) · Advances in experimental medicine and biology · doi:10.1007/978-3-319-97421-7_3 · Cited by 5
Biomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation
Defensor, E. B., Lim, M. A., & Schaevitz, L. R. (2021) · ILAR Journal · doi:10.1093/ilar/ilab018 · Cited by 4
Park, S. H., Lee, S. B., Park, S., Kim, E. Y., Pizzol, D., Trott, M., Barnett, Y., Koyanagi, A., Jacob, L., Soysal, P., et al. (2023) · British Journal of Urology · doi:10.1111/bju.16171 · Cited by 4
“…ureteric obstruction, infertility, kidney stone, micturition, and urinary retention), and animal model used (mouse, rat, rabbit, dog, pig, primate, sheep, others [guinea-pig, cats, alpaca, lamb, goat, bull, koala, miniature pig, hamster, and swine], and combination [multiple animals used]) [42,43].”
Leopold, S. S. (2020) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000001420 · Cited by 4
Nong, Y., Guo, Y., Gumpert, A. M., Li, Q., Tomlin, A. A., Zhu, X., & Bolli, R. (2021) · Molecular and Cellular Biochemistry · doi:10.1007/s11010-021-04227-w · Cited by 4
Considerations on Preclinical Neuromuscular Disease Gene Therapy Studies
Duan, D. (2019) · Springer eBooks · doi:10.1007/978-3-030-03095-7_17 · Cited by 5
Connecting the dots in neuroscience research: The future of evidence synthesis
Hair, K., Arroyo-Araujo, M., Vojvodić, S., Economou, M., Wong, C., Tinsdeall, F., Smith, S., Rackoll, T., Sena, E. S., & McCann, S. (2024) · Experimental Neurology · doi:10.1016/j.expneurol.2024.115047 · Cited by 5
“Subsequent to these interventions, there is evidence for improved validity of stroke experiments, including increased reporting of measures to reduce risks of bias and other quality indicators (Minnerup et al., 2016), changes that are not evident in related fields (Ramirez et al., 2017).”
Metadata
- Authors
- Ramirez, F. D., Motazedian, P., Jung, R. G., Di Santo, P., MacDonald, Z. D., Moreland, R., Simard, T., Clancy, A. A., Russo, J. J., Welch, V. A., Wells, G. A., & Hibbert, B.
- Affiliations
- From the Division of Cardiology (F.D.R., P.M., R.G.J., P.D.S., T.S., J.J.R., B.H.), CAPITAL Research Group (F.D.R., P.M., R.G.J., P.D.S., Z.D.M.D., R.M., T.S., J.J.R., B.H.), Vascular Biology and Experimental Medicine Laboratory (R.G.J., T.S., B.H.), and Cardiovascular Research Methods Centre (G.A.W.), University of Ottawa Heart Institute, Ontario, Canada; and School of Epidemiology, Public Health and Preventive Medicine (F.D.R., V.A.W., G.A.W.), Department of Cellular and Molecular Medicine (R.G.J....
- Corresponding author
- Hibbert, Benjamin
- ORCID
- Venue
- Circulation Research
- Published
- 2017-06-09
- Volume
- 120
- Issue
- 12
- Pages
- 1916-1926
- ISSN
- 0009-7330 (ISSN-L)
- 1524-4571
- PMC
- PMC5466021
- OpenAlex
- W2604920686
- Cited by
- 138
- References
- 74
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 11 · 2023: 12 · 2022: 16 · 2021: 17 · 2020: 22 · 2019: 18 · 2018: 25 · 2017: 8
- FWCI
- 31.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Circulation Research (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- methodological quality, randomization, blinding, sample size determination, sex as a biological variable, research bias, irreproducibility, citation impact, animal models, stroke research, research translation
- MeSH terms
- Animals, Cardiovascular Diseases, Disease Models, Animal, Drug Evaluation, Preclinical, Humans, Research Design, Reproducibility of Results, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
