Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Background: In January 2003, STAndards for the Reporting of Diagnostic accuracy studies (STARD) were published in a number of journals, to improve the quality of reporting in diagnostic accuracy studies. We designed a study to investigate the inter-assessment reproducibility, and intra- and inter-observer reproducibility of the items in the STARD statement. Methods: Thirty-two diagnostic accuracy studies published in 2000 in medical journals with an impact factor of at least 4 were included. Two reviewers independently evaluated the quality of reporting of these studies using the 25 items of the STARD statement. A consensus evaluation was obtained by discussing and resolving disagreements between reviewers. Almost two years later, the same studies were evaluated by the same reviewers. For each item, percentages agreement and Cohen's kappa between first and second consensus assessments (inter-assessment) were calculated. Intraclass Correlation coefficients (ICC) were calculated to evaluate its reliability. Results: The overall inter-assessment agreement for all items of the STARD statement was 85% (Cohen's kappa 0.70) and varied from 63% to 100% for individual items. The largest differences between the two assessments were found for the reporting of the rationale of the reference standard (kappa 0.37), number of included participants that underwent tests (kappa 0.28), distribution of the severity of the disease (kappa 0.23), a cross tabulation of the results of the index test by the results of the reference standard (kappa 0.33) and how indeterminate results, missing data and outliers were handled (kappa 0.25). Within and between reviewers, also large differences were observed for these items. The inter-assessment reliability of the STARD checklist was satisfactory (ICC = 0.79 [95% CI: 0.62 to 0.89]). Conclusion: Although the overall reproducibility of the quality of reporting on diagnostic accuracy studies using the STARD statement was found to be good, substantial disagreements were found for specific items. These disagreements were not so much caused by differences in interpretation of the items by the reviewers but rather by difficulties in assessing the reporting of these items due to lack of clarity within the articles. Including a flow diagram in all reports on diagnostic accuracy studies would be very helpful in reducing confusion between readers and among reviewers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Self-citationTowards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · Radiology · doi:10.1148/radiol.2261021292 · Cited by 3,215
Self-citationMoher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861
Moher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124
Self-citationMoher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027
Epidemiology and reporting of randomised trials published in PubMed journals
Chan, A.-W., & Altman, D. G. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71879-1 · Cited by 474
Pai, M., Flores, L. L., Pai, N., Hubbard, A., Riley, L. W., & Colford, J. M. (2003) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(03)00772-2 · Cited by 384
Cited by
Nonalcoholic Fatty Liver Disease: A Systematic Review
Rinella, M. E. (2015) · JAMA · doi:10.1001/jama.2015.5370 · Cited by 2,267
Teshima, C. W., Kuipers, E. J., van Zanten, S. V., & Mensink, P. B. F. (2011) · Journal of Gastroenterology and Hepatology · doi:10.1111/j.1440-1746.2010.06530.x · Cited by 295
Mokkink, L. B., Terwee, C. B., Gibbons, E., Stratford, P. W., Alonso, J., Patrick, D. L., Knol, D. L., Bouter, L. M., & de Vet, H. C. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-82 · Cited by 285
Self-citationReporting Diagnostic Accuracy Studies: Some Improvements after 10 Years of STARD
Korevaar, D. A., Wang, J., van Enst, W. A., Leeflang, M. M., Hooft, L., Smidt, N., & Bossuyt, P. M. M. (2015) · Radiology · doi:10.1148/radiol.14141160 · Cited by 110
Self-citationDiagnostic Accuracy of Peak Nasal Inspiratory Flow and Rhinomanometry in Functional Rhinosurgery
Bermüller, C., Kirsche, H., Rettinger, G., & Riechelmann, H. (2008) · The Laryngoscope · doi:10.1097/mlg.0b013e318161e56b · Cited by 96
Metadata
- Authors
- Smidt, N., Rutjes, A. W., van der Windt, D. A., Ostelo, R. W., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C.
- Venue
- BMC Medical Research Methodology
- Published
- 2006-03-15
- Volume
- 6
- Issue
- 1
- Pages
- 12-12
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2170446768
- Cited by
- 66
- References
- 14
- FWCI
- 7.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies. https://app.cochat.ai/papers/10.1186/1471-2288-6-12
- The paper
- Smidt, N., Rutjes, A. W., van der Windt, D. A., Ostelo, R. W., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C. (2006). Reproducibility of the STARD checklist: an instrument to assess the quality of reporting of diagnostic accuracy studies. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-6-12
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.