The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Methodological Rigor in Preclinical Cardiovascular Studies: Targets to Enhance Reproducibility and Promote Research Translation
Open accessAbstract
Rationale: Methodological sources of bias and suboptimal reporting contribute to irreproducibility in preclinical science and may negatively affect research translation. Randomization, blinding, sample size estimation, and considering sex as a biological variable are deemed crucial study design elements to maximize the quality and predictive value of preclinical experiments. Objective: To examine the prevalence and temporal patterns of recommended study design element implementation in preclinical cardiovascular research. Methods and Results: All articles published over a 10-year period in 5 leading cardiovascular journals were reviewed. Reports of in vivo experiments in nonhuman mammals describing pathophysiology, genetics, or therapeutic interventions relevant to specific cardiovascular disorders were identified. Data on study design and animal model use were collected. Citations at 60 months were additionally examined as a surrogate measure of research impact in a prespecified subset of studies, stratified by individual and cumulative study design elements. Of 28 636 articles screened, 3396 met inclusion criteria. Randomization was reported in 21.8%, blinding in 32.7%, and sample size estimation in 2.3%. Temporal and disease-specific analyses show that the implementation of these study design elements has overall not appreciably increased over the past decade, except in preclinical stroke research, which has uniquely demonstrated significant improvements in methodological rigor. In a subset of 1681 preclinical studies, randomization, blinding, sample size estimation, and inclusion of both sexes were not associated with increased citations at 60 months. Conclusions: Methodological shortcomings are prevalent in preclinical cardiovascular research, have not substantially improved over the past 10 years, and may be overlooked when basing subsequent studies. Resultant risks of bias and threats to study validity have the potential to hinder progress in cardiovascular medicine as preclinical research often precedes and informs clinical trials. Stroke research quality has uniquely improved in recent years, warranting a closer examination for interventions to model in other cardiovascular fields.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In contrast, other studies using shorter checklists (primarily focused on experimental design) with more editorial follow-up have shown a marked improvement in the nature and detail of the information included in publications [34–36].”
From The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research (2020) · cites this paper as [36]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
SYRCLE’s risk of bias tool for animal studies
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-43 · Cited by 4,415
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
Raise standards for preclinical cancer research
Begley, C. G., & Ellis, L. M. (2012) · Nature · doi:10.1038/483531a · Cited by 2,979 · cited 5×
“portending a waning confidence in animal model data to identify promising therapeutic targets.(5,7,11,12,22,23) Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among t…”
InfluentialClinical development success rates for investigational drugs
Hay, M., Thomas, D. W., Craighead, J. L., Economides, C., & Rosenthal, J. (2014) · Nature Biotechnology · doi:10.1038/nbt.2786 · Cited by 2,536 · cited 1×
“Cardiovascular research and development, in particular, has fared poorly in recent years relative to other disease categories, exhibiting the greatest decline as a percentage of total projects(4) and among the lowest success rates and likelihood of approval at all phases of clinical trials.(24) Moreover, research and…”
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154
Research works printed but not linked: 3
- Lehrer J. The truth wears off: Is there something wrong with the scientific method? The New Yorker. 2010.
- Freedman DH. Lies, damned lies, and medical science. The Atlantic. 2010.
- 10.1038/scibx.2011.1416 · DOI
Cited by
Sorted by citations
Items 101–138 of 138
Rasmussen, C. L. M., Burkhart, A., Moos, T., & Thomsen, L. B. (2025) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-024-03479-1 · Cited by 3
The Next Best Thing in Cardiovascular Research
Boudreau, R. L. (2017) · Journal of the American Heart Association · doi:10.1161/jaha.117.007245 · Cited by 3
“A recent study published in Circulation Research revealed that a significant lack of methodological rigor (eg, not randomizing participants, blinding analyses, or adequately determining group sizes) is likely at the root of the field’s plague of irreproducible data.(3) Considering the detrimental effects this has on s…”
Motazedian, P., Coutinho, T. A., & Ramirez, F. D. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2022.01.023 · Cited by 3
Self-citationEvaluation of a Rabbit Model of Vascular Stent Healing: Application of Optical Coherence Tomography
Simard, T., Jung, R. G., Santo, P. D., Sarathy, K., Majeed, K., Motazedian, P., Short, S., Dhaliwal, S., Labinaz, A., Sarma, D., et al. (2023) · Journal of Cardiovascular Translational Research · doi:10.1007/s12265-023-10399-1 · Cited by 3
Self-citationCarleton, R. N., Teckchandani, T. A., Neary, J. P., Samayoa, J., Khoury, J. M. B., Maguire, K. Q., Krätzig, G. P., & Asmundson, G. J. G. (2025) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2025.10.044 · Cited by 2
Fujinami, Y., Liu, S., Bassi, G. L., Osuchowski, M. F., Yao, Y., Fraser, J. F., & Inoue, S. (2026) · Intensive Care Medicine Experimental · doi:10.1186/s40635-026-00874-9 · Cited by 2
Collazo, A., Kuhn, H. G., Kurth, T., Piccininni, M., & Rohmann, J. L. (2024) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x241275760 · Cited by 2
“44 We want to emphasize that our findings likely also extend to other in vivo disease models as well as in vitro research.”
Wilson, E., Currie, G., Macleod, M. R., Kind, P. C., & Sena, E. S. (2024) · Brain and Neuroscience Advances · doi:10.1177/23982128241287279 · Cited by 2
“Over the last two decades, researchers have adopted systematic review methods to summarise and appraise evidence from laboratory animal studies (de Vries et al., 2014) and have achieved similar success in research improvement (McCann et al., 2016; Ramirez et al., 2017).”
BackgroundThe lack of evidence-based management in electrical storm: a scoping review
Motazedian, P., Prosperi‐Porta, G., Mathieu, M.-E., Beauregard, N., Knoll, W., Quinn, N., Parlow, S., Santo, P. D., Abdel‐Razek, O., Jung, R., et al. (2025) · BMC Cardiovascular Disorders · doi:10.1186/s12872-025-05024-9 · Cited by 3
“In the published RCTs, three had overlapping patients and the fourth study did not include a sample size calculation, raising concerns of potential publication and ascertainment biases, and therefore overestimation of benefits and underestimation of harm [58, 59].”
Self-citationNovelli, D., Fumagalli, F., Staszewsky, L. I., Ristagno, G., Olivari, D., Masson, S., Giorgio, D. D., Ceriani, S., Massafra, R., Logu, F. D., et al. (2019) · MethodsX · doi:10.1016/j.mex.2019.100771 · Cited by 2
The Next Generation of Physician-Scientists: Adapting to Academic Cardiology in the 21st Century
Marbach, J. A., Almufleh, A. S., Froeschl, M. P. V., & Hibbert, B. M. (2018) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2018.05.011 · Cited by 2
Self-citationFisher, M., & Seastrong, R. (2020) · Stroke · doi:10.1161/strokeaha.120.029113 · Cited by 1
“Another evaluation compared the quality of preclinical research published by 5 leading cardiovascular journals, including Stroke, over a 10-year time period from 2006 to 2016.(13) The study concluded that only”
From Hype to High-Quality Research
Lauer, M. S. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.28683 · Cited by 1
Circulation Research “In This Issue” Anthology
Williams, R. R. E. (2018) · Circulation Research · doi:10.1161/res.0000000000000211 · Cited by 1
Ji, X., Walczak, P., van Beusekom, H. M. M., Casas, A. I., Clarkson, A. N., Farr, T. D., Jolkkonen, J., Liang, Y., Modo, M., Rosado-de-Castro, P. H., et al. (2022) · Neuroprotection/Neuroprotection (Chichester, England. Print) · doi:10.1002/nep3.8 · Cited by 1
“Whereas previous preclinical research and the translation of results obtained therein into clinical treatments were often impaired by shortcomings in experimental design, research quality in the field has improved substantially and methodological rigor applied today is outstanding across many fields.(1) Moreover, the…”
Evaluating Study Design Rigor in Preclinical Cardiovascular Research: A Replication StudyPreprint
Jimenez, I. C., Montenegro, G. C., Zahiri, K., Patel, D. N., & Mueller, A. L. (2023) · doi:10.7554/elife.91498.1 · Cited by 1
Wilson, E., Currie, G. L., Macleod, M. R., Kind, P., & Sena, E. S. (2023) · doi:10.31222/osf.io/s5xd4 · Cited by 1
Smith, A. J. (2024) · Clinical Orthopaedics and Related Research · doi:10.1097/corr.0000000000003013 · Cited by 1
Evaluating Study Design Rigor in Preclinical Cardiovascular Research: A Replication StudyPreprint
Jimenez, I. C., Montenegro, G., Zahiri, K., Patel, D. N., & Mueller, A. L. (2025) · eLife · doi:10.7554/elife.91498.2 · Cited by 1
High Appraisal of Methodological Quality of Basic Science Articles Published in Stroke
Schäbitz, W.-R., Minnerup, J., Fisher, M., & Aronowski, J. A. (2017) · Stroke · doi:10.1161/strokeaha.117.017717
“A post hoc analysis by Ramirez et al(4) came to the conclusion that it is likely a combination of both: Stroke uniquely showed improvements in all study design elements even after adjusting for cardiovascular disorder studied and animal model used, but stroke as the cardiovascular disorder studied was identified as an…”
Schäbitz, W., Fisher, M., & Aronowski, J. A. (2018) · Stroke · doi:10.1161/strokeaha.118.021784
Ramirez, F. D. (2019) · uO Research (University of Ottawa) · doi:10.20381/ruor-23670
Self-citationAllometric Scaling of Dietary Bioactives in Metabolic Research: The Present and Future
Whelan, J. (2020) · Royal Society of Chemistry eBooks · doi:10.1039/9781839160608-00295
Park, S. H., Lee, S. B., Park, S., Kim, E. Y., Pizzol, D., Trott, M., Barnett, Y., Koyanagi, A., Jacob, L., Soysal, P., et al. (2022) · Research Square · doi:10.21203/rs.3.rs-2183732/v1
Starting off strong: Rigor and reproducibility in the early career
Vagnozzi, R. J. (2022) · Journal of Molecular and Cellular Cardiology Plus · doi:10.1016/j.jmccpl.2022.100022
“The “ reproducibilitycrisis ” remainsacriticalprobleminbiomedicalsci-ence including in cardiovascular research [1].”
Evaluating Study Design Rigor in Preclinical Cardiovascular Research: A Replication StudyPreprint
Jimenez, I. C., Montenegro, G. C., Zahiri, K., Patel, D. N., & Mueller, A. L. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.06.27.546731
“This study is a replication and extension of a study performed by Ramirez et al. (2017), which investigated the prevalence of these four SDEs in preclinical studies published in five leading cardiovascular journals of the American Heart Association (Circulation; Circulation Research; Hypertension;…”
BackgroundMethodResultInfluentialREFLEXÕES EDUCATIVAS SOBRE A CRISE DE REPRODUTIBILIDADE DAS PESQUISAS BIOMÉDICAS
Ferrari, C. K. B. (2023) · LEOPOLDIANUM · doi:10.58422/releo2023.e1399
Evaluating Study Design Rigor in Preclinical Cardiovascular Research: A Replication StudyPreprint
Jimenez, I. C., Montenegro, G. C., Zahiri, K., Patel, D. N., & Mueller, A. L. (2023) · eLife · doi:10.7554/elife.91498
Scrutton, A. M., Ollis, F., & Boltze, J. (2023) · Neuroprotection/Neuroprotection (Chichester, England. Print) · doi:10.1002/nep3.29
“46 Of note, the fi eld of stroke is meanwhile leading (but not yet completely suf fi cient) regarding methodological quality according to Ramirez et al., potentially due to the impact of the 2009 STAIR guidelines.”
Overview of preclinical research in cardiovascular diseases
Nouthé, B., Kamani, C. K., Nso, N., Ngonge, A. L., & Kengne, A. P. (2025) · Elsevier eBooks · doi:10.1016/b978-0-323-91790-2.00065-4
Mechanisms of Sex Disparities in Cardiovascular Function and Remodeling
Chaudhari, S. K., Cushen, S. C., Osikoya, O., Jaini, P. A., Posey, R., Mathis, K. W., & Goulopoulou, S. (2019) · Comprehensive physiology · doi:10.1002/j.2040-4603.2019.tb00059.x
“The majority of previously published experimental studies investigating molecular mechanisms and efficacy of certain therapeutics in CVD have primarily used male animals (311).”
Additional file 1 of The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650470
Additional file 4 of The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650479
Additional file 2 of The ARRIVE guidelines 2.0: Updated guidelines for reporting animal research
Percie du Sert, N., Hurst, V., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., et al. (2020) · Figshare · doi:10.6084/m9.figshare.12650473
Yi, S., Ruan, J., Wang, Z., Ding, F., Liu, Y., Ao, Q., Ren, Z., Wu, C., Luo, Y., Niu, M., et al. (2026) · BMC Veterinary Research · doi:10.1186/s12917-026-05468-x
“A growing number of studies have shown that random assignment, concealed grouping, and blinding are equally important measures to reduce the risk of bias inherent in an interventional animal experiment [24].”
An Integrated Framework for Publishable Sport Science Research
Plakias, S. (2026) · Publications · doi:10.3390/publications14020026 · Cited by 2
“Ensuring methodological rigor is a fundamental requirement for producing reliable and meaningful scientific evidence in various fields (Ramirez et al., 2017; Sells et al., 2018).”
Gaburro, S., Berridge, B. R., Kelley, N. W., von Keutz, E., Ferrer, M., Roper, C. S., Fossler, M. J., Bajramovic, J. J., Valentin, J., McGovern, T., et al. (2026) · Regulatory Toxicology and Pharmacology · doi:10.1016/j.yrtph.2026.106237
Metadata
- Authors
- Ramirez, F. D., Motazedian, P., Jung, R. G., Di Santo, P., MacDonald, Z. D., Moreland, R., Simard, T., Clancy, A. A., Russo, J. J., Welch, V. A., Wells, G. A., & Hibbert, B.
- Affiliations
- From the Division of Cardiology (F.D.R., P.M., R.G.J., P.D.S., T.S., J.J.R., B.H.), CAPITAL Research Group (F.D.R., P.M., R.G.J., P.D.S., Z.D.M.D., R.M., T.S., J.J.R., B.H.), Vascular Biology and Experimental Medicine Laboratory (R.G.J., T.S., B.H.), and Cardiovascular Research Methods Centre (G.A.W.), University of Ottawa Heart Institute, Ontario, Canada; and School of Epidemiology, Public Health and Preventive Medicine (F.D.R., V.A.W., G.A.W.), Department of Cellular and Molecular Medicine (R.G.J....
- Corresponding author
- Hibbert, Benjamin
- ORCID
- Venue
- Circulation Research
- Published
- 2017-06-09
- Volume
- 120
- Issue
- 12
- Pages
- 1916-1926
- ISSN
- 0009-7330 (ISSN-L)
- 1524-4571
- PMC
- PMC5466021
- OpenAlex
- W2604920686
- Cited by
- 138
- References
- 74
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 11 · 2023: 12 · 2022: 16 · 2021: 17 · 2020: 22 · 2019: 18 · 2018: 25 · 2017: 8
- FWCI
- 31.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Circulation Research (publishedVersion, cc-by-nc-nd)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- methodological quality, randomization, blinding, sample size determination, sex as a biological variable, research bias, irreproducibility, citation impact, animal models, stroke research, research translation
- MeSH terms
- Animals, Cardiovascular Diseases, Disease Models, Animal, Drug Evaluation, Preclinical, Humans, Research Design, Reproducibility of Results, Translational Research, Biomedical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
