CoChat

A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools

Open access
Whiting, Penny, Rutjes, Anne W.S., Dinnes, Jacqueline, Reitsma, Johannes B., Bossuyt, Patrick M.M., Kleijnen, Jos · 2005 · Journal of Clinical Epidemiology
Cited by 79

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

BACKGROUND AND OBJECTIVE To review existing quality assessment tools for diagnostic accuracy studies and to examine to what extent quality was assessed and incorporated in diagnostic systematic reviews. METHODS Electronic databases were searched for tools to assess the quality of studies of diagnostic accuracy or guides for conducting, reporting or interpreting such studies. The Database of Abstracts of Reviews of Effects (DARE; 1995-2001) was used to identify systematic reviews of diagnostic studies to examine the practice of quality assessment of primary studies. RESULTS Ninety-one quality assessment tools were identified. Only two provided details of tool development, and only a small proportion provided any indication of the aspects of quality they aimed to assess. None of the tools had been systematically evaluated. We identified 114 systematic reviews, of which 58 (51%) had performed an explicit quality assessment and were further examined. The majority of reviews used more than one method of incorporating quality. CONCLUSION Most tools to assess the quality of diagnostic accuracy studies do not start from a well-defined definition of quality. None has been systematically evaluated. The majority of existing systematic reviews fail to take differences in quality into account. Reviewers should consider quality as a possible source of heterogeneity.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2004.04.008

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Research works printed but not linked: 100

  1. Jaeschke, 1994, Users' guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid?, JAMA, 271, 389, 10.1001/jama.271.5.389 · doi:10.1001/jama.271.5.389 · Paper page
  2. Jaeschke, 1994, Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients?, JAMA, 271, 703, 10.1001/jama.271.9.703 · doi:10.1001/jama.271.9.703 · Paper page
  3. Irwig, 1994, Guidelines for meta-analyses evaluating diagnostic tests, Ann Intern Med, 120, 667, 10.7326/0003-4819-120-8-199404150-00008 · DOI · Paper page
  4. Whiting, 2004, Sources of variation and bias in studies of diagnostic accuracy: a systematic review, Ann Intern Med, 140, 189, 10.7326/0003-4819-140-3-200402030-00010 · DOI · Paper page
  5. Whiting, 2004, Development and validation of methods for assessing the quality of diagnostic accuracy studies, Health Technol Assess, 8, 1, 10.3310/hta8250 · DOI · Paper page
  6. Reid, 1995, Use of methodological standards in diagnostic test research: getting better but still not good, JAMA, 274, 645, 10.1001/jama.274.8.645 · doi:10.1001/jama.274.8.645 · Paper page
  7. Sheps, 1984, The assessment of diagnostic tests: a survey of current medical research, JAMA, 252, 2418, 10.1001/jama.252.17.2418 · doi:10.1001/jama.252.17.2418 · Paper page
  8. Cooper, 1988, The poor quality of early evaluations of magnetic resonance imaging, JAMA, 259, 3277, 10.1001/jama.259.22.3277 · doi:10.1001/jama.259.22.3277
  9. Beam, 1991, Status of clinical MR evaluations 1985–1988: baseline and design for further assessments, Radiology, 180, 265, 10.1148/radiology.180.1.2052708 · doi:10.1148/radiology.180.1.2052708
  10. Kent, 1992, Disease, level of impact, and quality of research methods: three dimensions of clinical efficacy assessment applied to magnetic resonance imaging, Invest Radiol, 27, 245, 10.1097/00004424-199203000-00014 · DOI · Paper page
  11. Chien, 1997, The diagnostic accuracy of cervico-vaginal fetal fibronectin in predicting preterm delivery: an overview, Br J Obstet Gynaecol, 104, 436, 10.1111/j.1471-0528.1997.tb11495.x · DOI · Paper page
  12. de Vries, 1996, Summary receiver operating characteristic curves as a technique for meta-analysis of the diagnostic performance of duplex ultrasonography in peripheral arterial disease, Acad Radiol, 3, 361, 10.1016/S1076-6332(96)80257-1 · DOI
  13. Fahey, 1995, Meta-analysis of Pap test accuracy, Am J Epidemiol, 141, 680, 10.1093/oxfordjournals.aje.a117485 · DOI · Paper page
  14. Heffner, 1995, Pleural fluid chemical analysis in parapneumonic effusions: a meta-analysis, Am J Respir Crit Care Med, 151, 1700, 10.1164/ajrccm.151.6.7767510 · doi:10.1164/ajrccm.151.6.7767510
  15. Heffner, 1998, Methodologic standards for diagnostic test research in pulmonary medicine, Chest, 114, 877, 10.1378/chest.114.3.877 · DOI
  16. Hobbs, 1997, A review of near patient testing in primary care, Health Technol Assess, 1, 1, 10.3310/hta1050 · DOI
  17. Lensing, 1993, 125I-fibrinogen leg scanning: reassessment of its role for the diagnosis of venous thrombosis in post-operative patients, Thromb Haemost, 69, 2, 10.1055/s-0038-1651177 · DOI
  18. Radack, 1993, Is there a valid association between skin tags and colonic polyps: insights from a quantitative and methodologic analysis of the literature, J Gen Intern Med, 8, 413, 10.1007/BF02599617 · DOI
  19. Devous, 1998, SPECT brain imaging in epilepsy: a meta-analysis, J Nucl Med, 39, 285
  20. Mol, 1997, The accuracy of serum chlamydial antibodies in the diagnosis of tubal pathology: a meta-analysis, Fertil Steril, 67, 1031, 10.1016/S0015-0282(97)81435-5 · DOI · Paper page
  21. Owens, 1996, Polymerase chain reaction for the diagnosis of HIV infection in adults: a meta-analysis with recommendations for clinical practice and study design, Ann Intern Med, 124, 803, 10.7326/0003-4819-124-9-199605010-00004 · DOI
  22. Rao, 1995, The role of antineutrophil cytoplasmic antibody (c-ANCA) testing in the diagnosis of Wegener granulomatosis: a literature review and meta-analysis, Ann Intern Med, 123, 925, 10.7326/0003-4819-123-12-199512150-00005 · DOI · Paper page
  23. Reed, 1996, Sputum Gram's stain in community-acquired pneumococcal pneumonia: a meta-analysis, West J Med, 165, 197
  24. Swart, 1995, The accuracy of hysterosalpingography in the diagnosis of tubal pathology: a meta-analysis, Fertil Steril, 64, 486, 10.1016/S0015-0282(16)57781-4 · doi:10.1016/s0015-0282(16)57781-4
  25. Wells, 1995, Accuracy of ultrasound for the diagnosis of deep venous thrombosis in asymptomatic patients after orthopedic surgery: a meta-analysis, Ann Intern Med, 122, 47, 10.7326/0003-4819-122-1-199501010-00008 · DOI · Paper page
  26. Attia, 1999, Diagnosis of thyroid disease in hospitalized patients: a systematic review, Arch Intern Med, 159, 658, 10.1001/archinte.159.7.658 · DOI
  27. Badgett, 1996, How well can the chest radiograph diagnose left ventricular dysfunction?, J Gen Intern Med, 11, 625, 10.1007/BF02599031 · DOI
  28. Becker, 1996, D-dimer testing and acute venous thromboembolism, Arch Intern Med, 156, 939, 10.1001/archinte.156.9.939 · doi:10.1001/archinte.156.9.939 · Paper page
  29. Bradley, 1998, Alcohol screening questionnaires in women: a critical review, JAMA, 280, 166, 10.1001/jama.280.2.166 · DOI · Paper page
  30. Buntinx, 1997, The diagnostic value of macroscopic haematuria in diagnosing urological cancers: a meta-analysis, Fam Pract, 14, 63, 10.1093/fampra/14.1.63 · doi:10.1093/fampra/14.1.63
  31. Conde-Agudelo, 1998, Triple-marker test as screening for Down syndrome: a meta-analysis, Obstet Gynecol Surv, 53, 369, 10.1097/00006254-199806000-00022 · DOI
  32. Da Silva, 1995, Accuracy of leukocyte indices and C-reactive protein for diagnosis of neonatal sepsis: a critical review, Pediatr Infect Dis J, 14, 362, 10.1097/00006454-199505000-00005 · DOI · Paper page
  33. De Bernardinis, 1999, Discriminant power and information content of Ranson's prognostic signs in acute pancreatitis: a meta-analytic study, Crit Care Med, 27, 2272, 10.1097/00003246-199910000-00035 · DOI
  34. Hallan, 1997, The accuracy of C-reactive protein in diagnosing acute appendicitis, Scand J Clin Lab Invest, 57, 373, 10.3109/00365519709084584 · DOI
  35. Heffner, 1997, Diagnostic value of tests that discriminate between exudative and transudative pleural effusions, Chest, 111, 970, 10.1378/chest.111.4.970 · doi:10.1378/chest.111.4.970
  36. Kearon, 1998, Noninvasive diagnosis of deep venous thrombosis, Ann Intern Med, 128, 663, 10.7326/0003-4819-128-8-199804150-00011 · DOI · Paper page
  37. Koelemay, 1996, Diagnosis of arterial disease of the lower extremities with duplex ultrasonography, Br J Surg, 83, 404, 10.1002/bjs.1800830336 · DOI · Paper page
  38. Koumans, 1998, Laboratory testing for Neisseria gonorrhoeae by recently introduced nonculture tests: a performance review with clinical and public health considerations, Clin Infect Dis, 27, 1171, 10.1086/514994 · DOI
  39. Lacasse, 1999, Transthoracic needle aspiration biopsy for the diagnosis of localised pulmonary lesions: a meta-analysis, Thorax, 54, 884, 10.1136/thx.54.10.884 · DOI
  40. Littenberg, 1995, Clinical efficacy of SPECT bone imaging for low back pain, J Nucl Med, 36, 1707
  41. Metlay, 1997, Does this patient have community-acquired pneumonia? Diagnosing pneumonia by history and physical examination, JAMA, 278, 1440, 10.1001/jama.278.17.1440 · doi:10.1001/jama.278.17.1440 · Paper page
  42. Mol, 1998, The performance of CA-125 measurement in the detection of endometriosis: a meta-analysis, Fertil Steril, 70, 1101, 10.1016/S0015-0282(98)00355-0 · DOI · Paper page
  43. Mol, 1998, The accuracy of single serum progesterone measurement in the diagnosis of ectopic pregnancy: a meta-analysis, Hum Reprod, 13, 3220, 10.1093/humrep/13.11.3220 · DOI · Paper page
  44. Nuovo, 1997, Is cervicography a useful diagnostic test? a systematic overview of the literature, J Am Board Fam Pract, 10, 390
  45. Pollitt, 1997, Neonatal screening for inborn errors of metabolism: cost, yield and outcome, Health Technol Assess, 1, 1, 10.3310/hta1070 · DOI · Paper page
  46. Rappeport, 1996, MR imaging before arthroscopy in knee joint disorders?, Acta Radiol, 37, 602, 10.3109/02841859609177683 · doi:10.3109/02841859609177683
  47. van den Hoogen, 1995, On the accuracy of history, physical examination, and erythrocyte sedimentation rate in diagnosing low back pain in general practice, Spine, 20, 318, 10.1097/00007632-199502000-00010 · DOI · Paper page
  48. van Tulder, 1997, Spinal radiographic findings and nonspecific low back pain: a systematic review of observational studies, Spine, 22, 427, 10.1097/00007632-199702150-00015 · DOI · Paper page
  49. Panzer, 1986, A methodologic assessment of studies comparing magnetic resonance imaging and computed tomography of the brain, Acta Radiol Suppl, 369, 269
  50. Hrung, 1999, Accuracy of MR imagining in the work-up of suspicious breast lesions: a diagnostic meta-analysis, Acad Radiol, 6, 387, 10.1016/S1076-6332(99)80189-5 · doi:10.1016/s1076-6332(99)80189-5
  51. Rothwell, 2000, Critical appraisal of the design and reporting of studies of imaging and measurement of carotid stenosis, Stroke, 31, 1444, 10.1161/01.STR.31.6.1444 · DOI
  52. van der Wurff, 2000, Clinical tests of the sacroiliac joint: a systematic methodological review. Part 1: Reliability, Man Ther, 5, 30, 10.1016/S1356-689X(99)90228-X · doi:10.1016/s1356-689x(99)90228-x
  53. Windeler, 1988, Description and evaluation of diagnostic-tests in German-language medical journals, Schweiz Med Wochenschr, 118, 1437
  54. Mullins, 2000, The role of spiral volumetric computed tomography in the diagnosis of pulmonary embolism, Arch Intern Med, 160, 293, 10.1001/archinte.160.3.293 · doi:10.1001/archinte.160.3.293
  55. Fiellin, 2000, Screening for alcohol problems in primary care, Arch Intern Med, 160, 1977, 10.1001/archinte.160.13.1977 · DOI · Paper page
  56. Berry, 2002, The cost-effectiveness of magnetic resonance angiography for carotid artery stenosis and peripheral vascular disease: a systematic review, Health Technol Assess, 6, 1, 10.3310/hta6070 · DOI · Paper page
  57. Becker, 1989, Real-time ultrasonography for the diagnosis of lower extremity deep venous thrombosis: the wave of the future?, Arch Intern Med, 149, 1731, 10.1001/archinte.149.8.1731 · doi:10.1001/archinte.149.8.1731
  58. Guyatt, 1992, Laboratory diagnosis of iron-deficiency anemia: an overview, J Gen Intern Med, 7, 145, 10.1007/BF02598003 · DOI · Paper page
  59. Hoffman, 1991, Diagnostic accuracy and clinical utility of thermography for lumbar radiculopathy: a meta-analysis, Spine, 16, 623, 10.1097/00007632-199106000-00005 · doi:10.1097/00007632-199106000-00005
  60. Holleman, 1995, Does the clinical examination predict airflow limitations?, JAMA, 273, 313, 10.1001/jama.273.4.313 · doi:10.1001/jama.273.4.313
  61. Kent, 1992, Diagnosis of lumbar spinal stenosis in adults: a metaanalysis of the accuracy of CT, MR, and myelography, AJR Am J Roentgenol, 158, 1135, 10.2214/ajr.158.5.1533084 · DOI · Paper page
  62. Philbrick, 1980, Methodologic problems of exercise testing for coronary artery disease: groups, analysis and bias, Am J Cardiol, 46, 807, 10.1016/0002-9149(80)90432-4 · DOI · Paper page
  63. Arrive, 2000, A scale of methodological quality for clinical studies of radiologic examinations, Radiology, 217, 69, 10.1148/radiology.217.1.r00oc0669 · DOI · Paper page
  64. Khan, 2001, Systematic reviews to evaluate diagnostic tests, Eur J Obstet Gynecol Reprod Biol, 95, 6, 10.1016/S0301-2115(00)00463-2 · DOI
  65. Mulrow, 1989, Assessing quality of a diagnostic test evaluation, J Gen Intern Med, 4, 288, 10.1007/BF02597398 · DOI
  66. Black, 1990, How to evaluate the radiology literature, AJR Am J Roentgenol, 154, 17, 10.2214/ajr.154.1.2104704 · doi:10.2214/ajr.154.1.2104704
  67. Jensen, 1999, Methodik diagnostischer Validierungsstudien: Fehler in der Studienplanung und Auswertung, Med Klin, 94, 522, 10.1007/BF03044946 · DOI
  68. Greiner, 2000, Epidemiologic issues in the validation of veterinary diagnostic tests, Prev Vet Med, 45, 3, 10.1016/S0167-5877(00)00114-8 · DOI · Paper page
  69. Kobberling, 1990, Memorandum for the evaluation of diagnostic measures, J Clin Chem Clin Biochem, 28, 873
  70. Greenhalgh, 1997, How to read a paper: papers that report diagnostic or screening tests, BMJ, 315, 540, 10.1136/bmj.315.7107.540 · DOI · Paper page
  71. Heffner, 1998, Evaluating diagnostic tests in the pleural space: differentiating transudates from exudates as a model, Clin Chest Med, 19, 277, 10.1016/S0272-5231(05)70077-0 · DOI
  72. 1981, How to read clinical journals: II. To learn about a diagnostic test, Can Med Assoc J, 124, 703
  73. Deeks, 1999, Using evaluations of diagnostic tests: understanding their limitations and making the most of available evidence, Ann Oncol, 10, 761, 10.1023/A:1008359805260 · DOI
  74. Freedman, 1987, Evaluating and comparing imaging techniques: a review and classification of study designs, Br J Radiol, 60, 1071, 10.1259/0007-1285-60-719-1071 · DOI
  75. Gifford, 1999, Evaluating dementia screening tests: methodologic standards to rate their performance, Neurology, 52, 224, 10.1212/WNL.52.2.224 · DOI · Paper page
  76. Thornbury, 1991, Increasing the scientific quality of clinical efficacy studies of magnetic resonance imaging, Invest Radiol, 26, 829, 10.1097/00004424-199109000-00011 · doi:10.1097/00004424-199109000-00011
  77. Deyo, 1994, Designing studies of diagnostic tests for low back pain or radiculopathy, Spine, 19, S2057, 10.1097/00007632-199409151-00007 · doi:10.1097/00007632-199409151-00007
  78. Bruns, 2000, Toward a checklist for reporting of studies of diagnostic accuracy of medical tests, Clin Chem, 46, 893, 10.1093/clinchem/46.7.893 · DOI · Paper page
  79. Haynes, 1995, Purpose and procedure (abbreviated), Evid Based Med, 1, 2
  80. Mower, 1999, Evaluating bias and variability in diagnostic test reports, Ann Emerg Med, 33, 85, 10.1016/S0196-0644(99)70422-1 · DOI
  81. Deeks, 2001, Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests, BMJ, 323, 157, 10.1136/bmj.323.7305.157 · DOI · Paper page
  82. Badgett, 1997, Can the clinical examination diagnose left-sided heart failure in adults?, JAMA, 277, 1712, 10.1001/jama.277.21.1712 · DOI
  83. Bastian, 1997, Is this patient pregnant? Can you reliably rule in or rule out early pregnancy by clinical examination?, JAMA, 278, 586, 10.1001/jama.278.7.586 · DOI
  84. Selker, 1997, An evaluation of technologies for identifying acute cardiac ischemia in the emergency department: a report from a National Heart Attack Alert Program working group, Emerg Med, 29, 13
  85. Anand, 1998, Does this patient have deep vein thrombosis?, JAMA, 279, 1094, 10.1001/jama.279.14.1094 · DOI
  86. Tugwell, 1997, Laboratory evaluation in the diagnosis of Lyme disease: clinical guideline, part 2, Ann Intern Med, 127, 1109, 10.7326/0003-4819-127-12-199712150-00011 · DOI · Paper page
  87. Bachmann, 1998, Impact of diabetic retinopathy screening on a British district population: case detection and blindness prevention in an evidence-based model, J Epidemiol Community Health, 52, 45, 10.1136/jech.52.1.45 · DOI · Paper page
  88. Spencer-Green, 1997, Test performance in systemic sclerosis: anti-centromere and Anti-Scl-70 antibodies, Am J Med, 103, 242, 10.1016/S0002-9343(97)00023-5 · DOI
  89. Whited, 1998, Does this patient have a mole or a melanoma?, JAMA, 279, 696, 10.1001/jama.279.9.696 · DOI
  90. Bell, 1998, Screening for ovarian cancer: a systematic review, Health Technol Assess, 2, 1, 10.3310/hta2020 · doi:10.3310/hta2020
  91. Bonis, 1997, Correlation of biochemical response to interferon alfa with histological improvement in hepatitis C: a meta-analysis of diagnostic test characteristics, Hepatology, 26, 1035, 10.1002/hep.510260436 · DOI
  92. Mayer, 1997, Systematic review of the diagnostic accuracy of dermatoscopy in detecting malignant melanoma, Med J Aust, 167, 206, 10.5694/j.1326-5377.1997.tb138847.x · DOI · Paper page
  93. Huicho, 1996, Fecal screening tests in the approach to acute infectious diarrhea: a scientific overview, Pediatr Infect Dis J, 15, 486, 10.1097/00006454-199606000-00004 · DOI · Paper page
  94. Pearl, 1996, Ultrasonography for the initial evaluation of blunt abdominal trauma: a review of prospective trials, Ann Emerg Med, 27, 353, 10.1016/S0196-0644(96)70273-1 · DOI
  95. Bastian, 1998, Diagnostic efficiency of home pregnancy test kits: a meta-analysis, Arch Fam Med, 7, 465, 10.1001/archfami.7.5.465 · DOI
  96. Owens, 1996, A meta-analytic evaluation of the polymerase chain reaction for the diagnosis of HIV infection in infants, JAMA, 275, 1342, 10.1001/jama.275.17.1342 · doi:10.1001/jama.275.17.1342
  97. Liedberg, 1996, Evidence-based evaluation of three imaging methods for the temporomandibular disc, Dentomaxillofac Radiol, 25, 234, 10.1259/dmfr.25.5.9161176 · DOI
  98. Loy, 1996, Do commercial serological kits for Helicobacter pylori infection differ in accuracy? A meta-analysis, Am J Gastroenterol, 91, 1138
  99. Whiting, 2003, The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews, BMC Med Res Methodol, 3, 25, 10.1186/1471-2288-3-25 · DOI · Paper page
  100. de Vet, 2001, Systematic reviews of diagnostic research: considerations about assessment and incorporation of methodological quality, Eur J Epidemiol, 17, 301, 10.1023/A:1012751326462 · DOI

Cited by

See all 79

Metadata

Authors
Whiting, P., Rutjes, A. W. S., Dinnes, J., Reitsma, J. B., Bossuyt, P. M. M., & Kleijnen, J.
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2005-01
Volume
58
Issue
1
Pages
1-12
ISSN
0895-4356 (ISSN-L)
OpenAlex
W2030551462
Cited by
79
References
5
FWCI
1.94
Type
journal article
Language
en
Open access
green
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools. https://app.cochat.ai/papers/10.1016/j.jclinepi.2004.04.008
The paper
Whiting, P., Rutjes, A. W. S., Dinnes, J., Reitsma, J. B., Bossuyt, P. M. M., & Kleijnen, J. (2005). A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2004.04.008

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.