The DOI matches its Crossref record; 125 of the 125 references it cites were found in a registry or an index.
Development and validation of methods for assessing the quality of diagnostic accuracy studies
Open accessAbstract
OBJECTIVES: To develop a quality assessment tool which will be used in systematic reviews to assess the quality of primary studies of diagnostic accuracy. DATA SOURCES: Electronic databases including MEDLINE, EMBASE, BIOSIS and the methodological databases of both CRD and the Cochrane Collaboration. REVIEW METHODS: Three systematic reviews were conducted to provide an evidence base for the development of the quality assessment tool. A Delphi procedure was used to develop the quality assessment tool and the information provided by the reviews was incorporated into this. A panel of nine experts in the area of diagnostic accuracy studies took part in the Delphi procedure to agree on the items to be included in the tool. Panel members were also asked to provide feedback on various other items and whether they would like to see the development of additional topic and design specific items. The Delphi procedure produced the quality assessment tool, named the QUADAS tool, which consisted of 14 items. A background document was produced describing each item included in the tool and how each of the items should be scored. RESULTS: The reviews produced 28 possible items for inclusion in the quality assessment tool. It was found that the sources of bias supported by the most empirical evidence were variation by clinical and demographic subgroups, disease prevalence/severity, partial verification bias, clinical review bias and observer/instrument variation. There was also some evidence of bias for the effects of distorted selection of participants, absent or inappropriate reference standard, differential verification bias and review bias. The evidence for the effects of other sources of bias was insufficient to draw conclusions. The third review found that only one item, the avoidance of review bias, was included in more than 75% of tools. Spectrum composition, population recruitment, absent or inappropriate reference standard and verification bias were each included in 50-75% of tools. Other items were included in less than 50% of tools. The second review found that the quality assessment tool should have the potential to be discussed narratively, reported in a tabular summary, used as recommendations for future research, used to conduct sensitivity or regression analyses and used as criteria for inclusion in the review or a primary analysis. This suggested that some distinction is needed between high- and low-quality studies. Component analysis was considered the best approach to incorporate quality into systematic reviews of diagnostic studies and this was taken into consideration when developing the tool. CONCLUSIONS: This project produced an evidence-based quality assessment tool to be used in systematic reviews of diagnostic accuracy studies. Through the various stages of the project the current lack of such a tool and the need for a systematically developed validated tool were demonstrated. Further work to validate the tool continues beyond the scope of this project. The further development of the tool by the addition of design- and topic-specific criteria is proposed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
Zweig, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,569
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Two-sided confidence intervals for the single proportion: comparison of seven methods
Newcombe, R. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8<857::aid-sim777>3.0.co;2-e · Cited by 5,107
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861
Problems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic Tests
Ransohoff, D. F., & Feinstein, A. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197810262991705 · Cited by 1,734
Systematic reviews of evaluations of diagnostic and screening tests
Deeks, J. J. (2001) · BMJ · doi:10.1136/bmj.323.7305.157 · Cited by 1,340
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124
Self-citation
Cited by
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592
Self-citationOlsson, B., Lautner, R., Andreasson, U., Öhrfelt, A., Portelius, E., Bjerke, M., Hölttä, M., Rosén, C., Olsson, C., Strobel, G., et al. (2016) · The Lancet Neurology · doi:10.1016/s1474-4422(16)00070-3 · Cited by 2,049
Zeng, X., Zhang, Y., Kwong, J. S. W., Zhang, C., Li, S., Sun, F., Niu, Y., & Du, L. (2015) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12141 · Cited by 2,082
Ma, L.-L., Wang, Y.-Y., Yang, Z.-H., Huang, D., Weng, H., & Zeng, X.-T. (2020) · Military Medical Research · doi:10.1186/s40779-020-00238-8 · Cited by 1,648
Gilbody, S., Richards, D., Brealey, S., & Hewitt, C. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0333-y · Cited by 1,290
The reliability of three-dimensional kinematic gait measurements: A systematic review
McGinley, J. L., Baker, R., Wolfe, R., & Morris, M. E. (2009) · Gait & Posture · doi:10.1016/j.gaitpost.2008.09.003 · Cited by 1,144
Tang, B. M., Eslick, G. D., Craig, J. C., & McLean, A. S. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70052-x · Cited by 851
Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review
Whiting, P., Rutjes, A. W. S., Reitsma, J. B., Glas, A. S., Bossuyt, P. M. M., & Kleijnen, J. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-3-200402030-00010 · Cited by 798
Self-citationEvaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Weswood, M. E., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. N., & Kleijnen, J. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-9 · Cited by 769
Self-citationEPA guidance on the early intervention in clinical high risk states of psychoses
Schmidt, S. J., Schultze-Lutter, F., Schimmelmann, B. G., Maric, N. P., Salokangas, R. K. R., Riecher-Rössler, A., van der Gaag, M., Meneghelli, A., Nordentoft, M., Marshall, M., et al. (2015) · European Psychiatry · doi:10.1016/j.eurpsy.2015.01.013 · Cited by 585
Hegedus, E. J., Goode, A. P., Cook, C. E., Michener, L., Myer, C. A., Myer, D. M., & Wright, A. A. (2012) · British Journal of Sports Medicine · doi:10.1136/bjsports-2012-091066 · Cited by 538
Hegedus, E. J., Goode, A., Campbell, S., Morin, A., Tamaddoni, M., Moorman, C. T., & Cook, C. (2008) · British Journal of Sports Medicine · doi:10.1136/bjsm.2007.038406 · Cited by 536
Eichler, K., Hempel, S., Wilby, J., Myers, L., Bachmann, L. M., & Kleijnen, J. (2006) · Journal of Urology · doi:10.1016/s0022-5347(05)00957-2 · Cited by 518
Self-citationTHE CELLULAR, METABOLIC, AND SYSTEMIC CONSEQUENCES OF AGGRESSIVE FLUID RESUSCITATION STRATEGIES
Cotton, B. A., Guy, J. S., Morris, J. A., & Abumrad, N. N. (2006) · Shock · doi:10.1097/01.shk.0000209564.84822.f2 · Cited by 517
Mant, J., Doust, J., Roalfe, A., Barton, P., Cowie, M., Glasziou, P., Mant, D., McManus, R., Holder, R., Deeks, J., et al. (2009) · Health Technology Assessment · doi:10.3310/hta13320 · Cited by 498
Mitchell, A. J. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.10.0438 · Cited by 468
Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.005 · Cited by 405
Self-citationEPA guidance on the early detection of clinical high risk states of psychoses
Schultze-Lutter, F., Michel, C., Schmidt, S. J., Schimmelmann, B. G., Maric, N. P., Salokangas, R. K. R., Riecher-Rössler, A., van der Gaag, M., Nordentoft, M., Raballo, A., et al. (2015) · European Psychiatry · doi:10.1016/j.eurpsy.2015.01.010 · Cited by 405
Shaheen, A. A. M., & Myers, R. P. (2007) · Hepatology · doi:10.1002/hep.21835 · Cited by 397
The development of a quality appraisal tool for studies of diagnostic reliability (QAREL)
Lucas, N. P., Macaskill, P., Irwig, L., & Bogduk, N. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.10.002 · Cited by 374
Shaheen, A. A. M., Wan, A. F., & Myers, R. P. (2007) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2007.01466.x · Cited by 360
Use of PCR for diagnosis of invasive aspergillosis: systematic review and meta-analysis
Mengoli, C., Cruciani, M., Barnes, R. A., Loeffler, J., & Donnelly, J. P. (2009) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(09)70019-2 · Cited by 354
No role for quality scores in systematic reviews of diagnostic accuracy studies
Whiting, P., Harbord, R., & Kleijnen, J. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-19 · Cited by 337
Self-citationPhysical examination for lumbar radiculopathy due to disc herniation in patients with low-back pain
van der Windt, D. A., Simons, E., Riphagen, I. I., Ammendolia, C., Verhagen, A. P., Laslett, M., Devillé, W., Deyo, R. A., Bouter, L. M., de Vet, H. C., et al. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007431.pub2 · Cited by 334
Diagnostic Validity of Criteria for Sacroiliac Joint Pain: A Systematic Review
Szadek, K. M., van der Wurff, P., van Tulder, M. W., Zuurmond, W. W., & Perez, R. S. G. M. (2009) · The Journal of Pain · doi:10.1016/j.jpain.2008.09.014 · Cited by 334
Lamoth, F., Cruciani, M., Mengoli, C., Castagnola, E., Lortholary, O., Richardson, M., Marchetti, O., & (ECIL-3), O. B. O. T. T. E. C. O. I. I. L. (2012) · Clinical Infectious Diseases · doi:10.1093/cid/cir897 · Cited by 302
Recommendations for the management of autoinflammatory diseases
ter Haar, N. M., Oswald, M., Jeyaratnam, J., Anton, J., Barron, K. S., Brogan, P. A., Cantarini, L., Galeotti, C., Grateau, G., Hentgen, V., et al. (2015) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2015-207546 · Cited by 278
Chang, K., Lu, W., Wang, J., Zhang, K., Jia, S., Li, F., Deng, S., & Chen, M. (2012) · Journal of Infection · doi:10.1016/j.jinf.2012.02.012 · Cited by 267
Diagnostic criteria for atopic dermatitis: a systematic review
Brenninkmeijer, E. E. A., Schram, M. E., Leeflang, M. M. G., Bos, J. D., & Spuls, P. I. (2008) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2007.08412.x · Cited by 256
Empirical evidence of the importance of comparative studies of diagnostic test accuracy
Takwoingi, Y., Leeflang, M. M. G., & Deeks, J. J. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-7-201304020-00006 · Cited by 187
Alghaithy, R. A., & Qualtrough, A. J. E. (2017) · International Endodontic Journal · doi:10.1111/iej.12611 · Cited by 178
Wassenaar, M., van Rijn, R. M., van Tulder, M. W., Verhagen, A. P., van der Windt, D. A. W. M., Koes, B. W., de Boer, M. R., Ginai, A. Z., & Ostelo, R. W. J. G. (2012) · European Spine Journal · doi:10.1007/s00586-011-2019-8 · Cited by 161
Systematic review and meta-analysis of middle cerebral artery Doppler to predict perinatal wellbeing
Morris, R. K., Say, R., Robson, S. C., Kleijnen, J., & Khan, K. S. (2012) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2012.07.027 · Cited by 70
Self-citationToddler Screening for Autism Spectrum Disorder: A Meta-Analysis of Diagnostic Accuracy
Sánchez-García, A. B., Galindo-Villardón, P., Nieto-Librero, A. B., Martín-Rodero, H., & Robins, D. L. (2019) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-018-03865-2 · Cited by 66
Strategies to optimize the validity of disease models in the drug discovery process
Sams-Dodd, F. (2006) · Drug Discovery Today · doi:10.1016/j.drudis.2006.02.005 · Cited by 60
Lefebvre, C., Glanville, J., Beale, S., Boachie, C., Duffy, S., Fraser, C., Harbour, J., McCool, R., & Smith, L. (2017) · Health Technology Assessment · doi:10.3310/hta21690 · Cited by 49
Weatherly, H. L. A., Griffin, S. C., Mc Daid, C., Durée, K. H., Davies, R. J. O., Stradling, J. R., Westwood, M. E., & Sculpher, M. J. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309090047 · Cited by 48
Zartaloudi, E., Laws, K. R., & Bramon, E. (2019) · Psychiatric Genetics · doi:10.1097/ypg.0000000000000241 · Cited by 25
Metadata
- Authors
- Whiting, P., Rutjes, A., Dinnes, J., Reitsma, J., Bossuyt, P., & Kleijnen, J.
- Venue
- Health Technology Assessment
- Published
- 2004-06
- Volume
- 8
- Issue
- 25
- Pages
- iii, 1-234
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- OpenAlex
- W2004084637
- Cited by
- 372
- References
- 125
- FWCI
- 24.03
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.