Systematic Survey of the Design, Statistical Analysis, and Reporting of Studies Published in the 2008 Volume of the Journal of Cerebral Blood Flow and Metabolism
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Translating experimental findings into clinically effective therapies is one of the major bottlenecks of modern medicine. As this has been particularly true for cerebrovascular research, attention has turned to the quality and validity of experimental cerebrovascular studies. We set out to assess the study design, statistical analyses, and reporting of cerebrovascular research. We assessed all original articles published in the Journal of Cerebral Blood Flow and Metabolism during the year 2008 against a checklist designed to capture the key attributes relating to study design, statistical analyses, and reporting. A total of 156 original publications were included (animal, in vitro, human). Few studies reported a primary research hypothesis, statement of purpose, or measures to safeguard internal validity (such as randomization, blinding, exclusion or inclusion criteria). Many studies lacked sufficient information regarding methods and results to form a reasonable judgment about their validity. In nearly 20% of studies, statistical tests were either not appropriate or information to allow assessment of appropriateness was lacking. This study identifies a number of factors that should be addressed if the quality of research in basic and translational biomedicine is to be improved. We support the widespread implementation of the ARRIVE (Animal Research Reporting In Vivo Experiments) statement for the reporting of experimental studies in biomedicine, for improving training in proper study design and analysis, and that reviewers and editors adopt a more constructively critical approach in the assessment of manuscripts for publication.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Improving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277 · cited 1×
“Indeed, Kilkenny et al (2010) have recently proposed such reporting guidelines (Animal Research Reporting In Vivo Experiments, ARRIVE; http:// www.nc3rs.org/ARRIVE) using the CONSORT statement as a foundation.”
MethodHigh agreement but low Kappa: I. the problems of two paradoxes
Feinstein, A. R., & Cicchetti, D. V. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90158-l · Cited by 3,034 · cited 1×
“As k is highly affected by the prevalence of tpositive scores, we also calculated separate indices of the proportionate agreement in the observers’ positive (‘yes’) and negative (‘no’) decisions (Feinstein and Cicchetti, 1990; Cicchetti and Feinstein, 1990).”
MethodMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Investigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
High agreement but low kappa: II. Resolving the paradoxes
Cicchetti, D. V., & Feinstein, A. R. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90159-m · Cited by 1,838 · cited 1×
“As k is highly affected by the prevalence of tpositive scores, we also calculated separate indices of the proportionate agreement in the observers’ positive (‘yes’) and negative (‘no’) decisions (Feinstein and Cicchetti, 1990; Cicchetti and Feinstein, 1990).”
MethodUpdate of the Stroke Therapy Academic Industry Roundtable Preclinical Recommendations
Fisher, M., Feuerstein, G., Howells, D. W., Hurn, P. D., Kent, T. A., Savitz, S. I., & Lo, E. H. (2009) · Stroke · doi:10.1161/strokeaha.108.541128 · Cited by 1,352 · cited 2×
“…and reporting problems are prevalent, and that this might be an important contributor to the ‘translational roadblock’ that exists in the field (Dirnagl, 2006; Sena et al, 2007; Phillips et al, 2008; Fisher et al, 2009; Crossley et al, 2008; Jerndal et al, 2010; Dirnagl and Macleod, 2009).”
BackgroundMoher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027 · cited 3×
“These measures have vastly improved the validity of clinical trials and ultimately their impact on patients (Moher et al, 2001a).”
BackgroundPlint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00557.x · Cited by 872
Kilkenny, C., Parsons, N., Kadyszewski, E., Festing, M. F. W., Cuthill, I. C., Fry, D., Hutton, J., & Altman, D. G. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007824 · Cited by 860
Cited by
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
Woodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427
“…of environmental health science (Goldman and Silbergeld 2013) and the importance to environmental health of nascent efforts in the pre clinical arena to develop improved experi mental animal study design and reporting (Landis et al. 2012; van der Worp and Macleod 2011; Vesterinen et al. 2011).”
BackgroundKoustas, E., Lam, J., Sutton, P., Johnson, P. I., Atchley, D. S., Sen, S., Robinson, K. A., Axelrad, D. A., & Woodruff, T. J. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307177 · Cited by 349
Baker, D., Lidster, K., Sottomayor, A., & Amor, S. (2014) · PLoS Biology · doi:10.1371/journal.pbio.1001756 · Cited by 305
The need for randomization in animal trials: an overview of systematic reviews
Hirst, J. A., Howick, J., Aronson, J. K., Roberts, N., Perera, R., Koshiaris, C., & Heneghan, C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098856 · Cited by 294
Metadata
- Authors
- Vesterinen, H. V., Egan, K., Deister, A., Schlattmann, P., Macleod, M. R., & Dirnagl, U.
- Venue
- Journal of Cerebral Blood Flow & Metabolism
- Publisher
- SAGE Publications
- Published
- 2010-12-15
- Volume
- 31
- Issue
- 4
- Pages
- 1064-1072
- ISSN
- 0271-678X (ISSN-L)
- 1559-7016
- OpenAlex
- W2158641962
- Cited by
- 58
- References
- 16
- FWCI
- 3.64
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Systematic Survey of the Design, Statistical Analysis, and Reporting of Studies Published in the 2008 Volume of the Journal of Cerebral Blood Flow and Metabolism. https://app.cochat.ai/papers/10.1038/jcbfm.2010.217
- The paper
- Vesterinen, H. V., Egan, K., Deister, A., Schlattmann, P., Macleod, M. R., & Dirnagl, U. (2011). Systematic Survey of the Design, Statistical Analysis, and Reporting of Studies Published in the 2008 Volume of the Journal of Cerebral Blood Flow and Metabolism. Journal of Cerebral Blood Flow & Metabolism. https://doi.org/10.1038/jcbfm.2010.217
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.