Cited by 1,861Meta-analysis and systematic reviewsStatistical Methods in EpidemiologyClinical Reasoning and Diagnostic Skills

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Empirical evidence of design-related bias in studies of diagnostic tests.

Lijmer, Jeroen G.·1999·JAMA

DOI 10.1001/jama.282.11.1061

Open this paper in CoChat — free account

Abstract

CONTEXT: The literature contains a large number of potential biases in the evaluation of diagnostic tests. Strict application of appropriate methodological criteria would invalidate the clinical application of most study results. OBJECTIVE: To empirically determine the quantitative effect of study design shortcomings on estimates of diagnostic accuracy. DESIGN AND SETTING: Observational study of the methodological features of 184 original studies evaluating 218 diagnostic tests. Meta-analyses on diagnostic tests were identified through a systematic search of the literature using MEDLINE, EMBASE, and DARE databases and the Cochrane Library (1996-1997). Associations between study characteristics and estimates of diagnostic accuracy were evaluated with a regression model. MAIN OUTCOME MEASURES: Relative diagnostic odds ratio (RDOR), which compared the diagnostic odds ratios of studies of a given test that lacked a particular methodological feature with those without the corresponding shortcomings in design. RESULTS: Fifteen (6.8%) of 218 evaluations met all 8 criteria; 64 (30%) met 6 or more. Studies evaluating tests in a diseased population and a separate control group overestimated the diagnostic performance compared with studies that used a clinical population (RDOR, 3.0; 95% confidence interval [CI], 2.0-4.5). Studies in which different reference tests were used for positive and negative results of the test under study overestimated the diagnostic performance compared with studies using a single reference test for all patients (RDOR, 2.2; 95% CI, 1.5-3.3). Diagnostic performance was also overestimated when the reference test was interpreted with knowledge of the test result (RDOR, 1.3; 95% CI, 1.0-1.9), when no criteria for the test were described (RDOR, 1.7; 95% CI, 1.1-2.5), and when no description of the population under study was provided (RDOR, 1.4; 95% CI, 1.1-1.7). CONCLUSION: These data provide empirical evidence that diagnostic studies with methodological shortcomings may overestimate the accuracy of a diagnostic test, particularly those including nonrepresentative patients or applying different reference standards.

Integrity signals

Registry

Crossref · 10.1001/jama.282.11.1061

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Untitled work
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Users' Guides to the Medical Literature
Jaeschke, R. (1994) · JAMA · doi:10.1001/jama.271.9.703 · Cited by 2,688
Problems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic Tests
Ransohoff, D. F., & Feinstein, A. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197810262991705 · Cited by 1,739
Combining independent studies of a diagnostic test into a summary roc curve: Data‐analytic approaches and some additional considerations
Moses, L. E., Shapiro, D. E., & Littenberg, B. (1993) · Statistics in Medicine · doi:10.1002/sim.4780121403 · Cited by 1,405
Users' Guides to the Medical Literature
Jaeschke, R. (1994) · JAMA · doi:10.1001/jama.271.5.389 · Cited by 1,174
THE ESTIMATION AND SIGNIFICANCE OF THE LOGARITHM OF A RATIO OF FREQUENCIES
HALDANE, B. J. B. S. (1956) · Annals of Human Genetics · doi:10.1111/j.1469-1809.1955.tb01285.x · Cited by 1,083
Selection and interpretation of diagnostic tests and procedures. Principles and applications.
Griner, P. F., Mayewski, R. J., Mushlin, A. I., & Greenland, P. (1981) · PubMed · Cited by 765
Use of Methodological Standards in Diagnostic Test Research
Reid, M. C. (1995) · JAMA · doi:10.1001/jama.274.8.645 · Cited by 710
Assessment of diagnostic tests when disease verification is subject to selection bias
Begg, C. B., & Greenes, R. A. (1983) · Biometrics · doi:10.2307/2530820 · Cited by 708
Meta-analysis of Pap Test Accuracy
Fahey, M. T., Irwig, L., & Macaskill, P. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117485 · Cited by 690

Show all 38 references

Sorted by citations

QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · BMJ · doi:10.1136/bmj.h5527 · Cited by 3,660
“Despite their apparent simplicity, such studies are at risk of bias [6, 7].”
BackgroundSelf-citation
Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369
“Again, the effect of these differences in design and conduct will be estimated the sROC approach as changes on the diagnostic odds scale[8,29].”
Method
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,270
Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219
Self-citation
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Self-citation
Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: The PRISMA-DTA Statement
McInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Bossuyt, P. M., and the PRISMA-DTA Group, Clifford, T., Cohen, J. F., Deeks, J. J., Gatsonis, C., et al. (2018) · JAMA · doi:10.1001/jama.2017.19163 · Cited by 3,210
The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed
Deeks, J. J., Macaskill, P., & Irwig, L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.016 · Cited by 2,910
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907
The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration
Cohen, J. F., Korevaar, D. A., Altman, D. G., Bruns, D. E., Gatsonis, C. A., Hooft, L., Irwig, L., Levine, D., Reitsma, J. B., de Vet, H. C. W., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-012799 · Cited by 2,570
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
The diagnostic odds ratio: a single indicator of test performance
Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00177-x · Cited by 2,261
Self-citation
Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Optimal cut-off score for diagnosing depression with the Patient Health Questionnaire (PHQ-9): a meta-analysis
Manea, L., Gilbody, S., & McMillan, D. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.110829 · Cited by 1,966
Screening for anxiety disorders with the GAD-7 and GAD-2: a systematic review and diagnostic metaanalysis
Plummer, F., Manea, L., Trepel, D., & McMillan, D. (2016) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.11.005 · Cited by 1,882
Serum Procalcitonin and C-Reactive Protein Levels as Markers of Bacterial Infection: A Systematic Review and Meta-analysis
Simon, L., Gauvin, F., Amre, D. K., Saint-Louis, P., & Lacroix, J. (2004) · Clinical Infectious Diseases · doi:10.1086/421997 · Cited by 1,802
Grading quality of evidence and strength of recommendations for diagnostic tests and strategies
Schünemann, H. J., Oxman, A. D., Brozek, J., Glasziou, P., Jaeschke, R., Vist, G. E., Williams, J. W., Kunz, R., Craig, J., Montori, V. M., et al. (2008) · BMJ · doi:10.1136/bmj.39500.677199.ae · Cited by 1,782
Association between CSF biomarkers and incipient Alzheimer's disease in patients with mild cognitive impairment: a follow-up study
Hansson, O., Zetterberg, H., Buchhave, P., Londos, E., Blennow, K., & Minthon, L. (2006) · The Lancet Neurology · doi:10.1016/s1474-4422(06)70355-6 · Cited by 1,700
PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Multidetector Computed Tomography for Acute Pulmonary Embolism
Stein, P. D., Fowler, S. E., Goodman, L. R., GOTTSCHALK, A. R., Hales, C. A., Hull, R. D., Leeper, K. V., Popovich, J., Quinn, D. A., Sos, T. A., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa052367 · Cited by 1,647
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
Psychological testing and psychological assessment: A review of evidence and issues.
Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2001) · American Psychologist · doi:10.1037//0003-066x.56.2.128 · Cited by 1,469
A prospective study of the prevalence of primary aldosteronism in 1,125 hypertensive patients
Rossi, G. P., Bernini, G., Caliumi, C., Desideri, G., Fabris, B., Ferri, C., Ganzaroli, C., Giacchetti, G., Letizia, C., Maccario, M., et al. (2006) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2006.07.059 · Cited by 1,467
STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C., Glasziou, P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · Radiology · doi:10.1148/radiol.2015151516 · Cited by 1,360
Self-citation
Systematic reviews of evaluations of diagnostic and screening tests
Deeks, J. J. (2001) · BMJ · doi:10.1136/bmj.323.7305.157 · Cited by 1,345
Magnetic resonance imaging and computed tomography in emergency assessment of patients with suspected acute stroke: a prospective comparison
Chalela, J. A., Kidwell, C. S., Nentwich, L. M., Luby, M. L., Butman, J. A., Demchuk, A. M., Hill, M. D., Patronas, N. J., Latour, L. L., & Warach, S. J. (2007) · The Lancet · doi:10.1016/s0140-6736(07)60151-2 · Cited by 1,302
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267
Gamma Glutamyl Transferase
Whitfield, J. B. (2001) · Critical Reviews in Clinical Laboratory Sciences · doi:10.1080/20014091084227 · Cited by 1,158
The reliability of three-dimensional kinematic gait measurements: A systematic review
McGinley, J. L., Baker, R. J., Wolfe, R. S., & Morris, M. E. (2008) · Gait & Posture · doi:10.1016/j.gaitpost.2008.09.003 · Cited by 1,155
Systematic Reviews of Diagnostic Test Accuracy
Leeflang, M. M. G., Deeks, J. J., Gatsonis, C., & Bossuyt, P. M. M. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-12-200812160-00008 · Cited by 1,124
The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Moher, D., Rennie, D., de Vet, H. C. W., & Lijmer, J. G. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00012-w1 · Cited by 1,124
Self-citation
Screening for Thyroid Cancer: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force
Lin, J. S., Bowles, E. J. A., Williams, S. B., & Morrison, C. C. (2017) · JAMA · doi:10.1001/jama.2017.0562 · Cited by 1,082
Rectal Cancer: Local Staging and Assessment of Lymph Node Involvement with Endoluminal US, CT, and MR Imaging—A Meta-Analysis
Bipat, S., Glas, A. S., Slors, F. J. M., Zwinderman, A. H., Bossuyt, P. M., & Stoker, J. (2004) · Radiology · doi:10.1148/radiol.2323031368 · Cited by 1,009
Screening for Depression in the General Population with the Center for Epidemiologic Studies Depression (CES-D): A Systematic Review with Meta-Analysis
Vilagut, G., Forero, C. G., Barbaglia, G., & Alonso, J. (2016) · PLoS ONE · doi:10.1371/journal.pone.0155431 · Cited by 1,053
STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · Clinical Chemistry · doi:10.1373/clinchem.2015.246280 · Cited by 984
Self-citation
Conducting systematic reviews of diagnostic studies: didactic guidelines
Devillé, W. L. J. M., Buntinx, F., Bouter, L. M., Montori, V. M., de Vet, H. C. W., van der Windt, D., & Bezemer, P. D. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-9 · Cited by 958
The diagnostic accuracy of CT and MRI in the staging of pelvic lymph nodes in patients with prostate cancer: a meta-analysis
Hövels, A. M., Heesakkers, R. A. M., Adang, E., Jager, G. J., Strum, S. B., Hoogeveen, Y. L., Severens, J. L., & Barentsz, J. O. (2008) · Clinical Radiology · doi:10.1016/j.crad.2007.05.022 · Cited by 956
Interferon-γ assays in the immunodiagnosis of tuberculosis: a systematic review
Pai, M., Riley, L. W., & Colford, J. M. (2004) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(04)01206-x · Cited by 913
Diagnostic Value of Procalcitonin, Interleukin-6, and Interleukin-8 in Critically Ill Patients Admitted with Suspected Sepsis
Harbarth, S., Holecková, K., Froidevaux, C., Pittet, D., Ricou, B., Grau, G. E., Vadas, L., & Pugin, J. (2001) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/ajrccm.164.3.2009052 · Cited by 897
A readers' guide to the interpretation of diagnostic test properties: clinical example of sepsis
Fischer, J. E., Bachmann, L. M., & Jaeschke, R. (2003) · Intensive Care Medicine · doi:10.1007/s00134-003-1761-8 · Cited by 890
Systems to rate the strength of scientific evidence.
West, S. L., King, V., Carey, T. S., Lohr, K. N., Mckoy, N. A., Sutton, S. F., & Lux, L. J. (2002) · PubMed Central · Cited by 873
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C., Glasziou, P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · PubMed · doi:10.1373/49.1.1 · Cited by 867
Self-citation
Effectiveness of positron emission tomography in the preoperative assessment of patients with suspected non-small-cell lung cancer: the PLUS multicentre randomised trial
van Tinteren, H., Hoekstra, O. S., Smit, E. F., van den Bergh, J. H. A. M., Schreurs, A. J. M., Stallaert, R. A. L. M., Velthoven, P. C. M. V., Comans, E. F. I., Diepenhorst, F. W., Verboom, P., et al. (2002) · The Lancet · doi:10.1016/s0140-6736(02)08352-6 · Cited by 851
Accuracy of procalcitonin for sepsis diagnosis in critically ill patients: systematic review and meta-analysis
Tang, B. M. P., Eslick, G. D., Craig, J. C., & McLean, A. S. (2007) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(07)70052-x · Cited by 854
Screening for Colorectal Cancer: A Targeted, Updated Systematic Review for the U.S. Preventive Services Task Force
Whitlock, E. P., Lin, J. S., Liles, E. G., Beil, T. L., & Fu, R. F. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-9-200811040-00245 · Cited by 844
Lymphatic mapping and sentinel lymph node biopsy in early‐stage breast carcinoma
Kim, T., Giuliano, A. E., & Lyman, G. H. (2005) · Cancer · doi:10.1002/cncr.21568 · Cited by 841
Sources of Variation and Bias in Studies of Diagnostic Accuracy
Whiting, P. F., Rutjes, A. W., Reitsma, J. B., Glas, A. S., Bossuyt, P. M., & Kleijnen, J. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-3-200402030-00010 · Cited by 804
d -Dimer for the Exclusion of Acute Venous Thrombosis and Pulmonary Embolism
Stein, P. D., Hull, R. D., Patel, K. C., Olson, R. E., Ghali, W. A., Brant, R., Biel, R. K., Bharadia, V., & Kalra, N. K. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-8-200404200-00005 · Cited by 797
Systematic reviews of evaluations of prognostic variables
Altman, D. G. (2001) · BMJ · doi:10.1136/bmj.323.7306.224 · Cited by 756
A unification of models for meta-analysis of diagnostic accuracy studies
Harbord, R. M., Deeks, J. J., Egger, M., Whiting, P. F., & Sterne, J. A. C. (2006) · Biostatistics · doi:10.1093/biostatistics/kxl004 · Cited by 753
Procalcitonin as a diagnostic test for sepsis in critically ill adults and after surgery or trauma: A systematic review and meta-analysis
Uzzan, B., Cohen, R., Nicolas, P., Cucherat, M., & Perret, G. Y. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000226413.54364.36 · Cited by 738
US or CT for Diagnosis of Appendicitis in Children and Adults? A Meta-Analysis
Doria, A. S., Moineddin, R., Kellenberger, C. J., Epelman, M. S., Beyene, J., Schuh, S., Babyn, P., & Dick, P. T. (2006) · Radiology · doi:10.1148/radiol.2411050913 · Cited by 742
Preferred reporting items for systematic review and meta-analysis of diagnostic test accuracy studies (PRISMA-DTA): explanation, elaboration, and checklist
Salameh, J.-P., Bossuyt, P. M., McGrath, T. A., Thombs, B. D., Hyde, C. J., Macaskill, P., Deeks, J. J., Leeflang, M., Korevaar, D. A., Whiting, P., et al. (2020) · BMJ · doi:10.1136/bmj.m2632 · Cited by 719
Use of uterine artery Doppler ultrasonography to predict pre-eclampsia and intrauterine growth restriction: a systematic review and bivariable meta-analysis
Cnossen, J. S., Morris, R. K., ter Riet, G., Mol, B. W. J., van der Post, J. A. M., Coomarasamy, A., Zwinderman, A. H., Robson, S. C., Bindels, P. J. E., Kleijnen, J., et al. (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.070430 · Cited by 719
Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses
Ioannidis, J. P. A. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081086 · Cited by 716
Assessing the clinical significance of change scores recorded on subjective outcome measures
Hurst, H., & Bolton, J. E. (2004) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2003.11.003 · Cited by 729
Screening for Colorectal Cancer
Lin, J. S., Piper, M., Perdue, L. A., Rutter, C. M., Webber, E. M., O’Connor, E., Smith, N. X., & Whitlock, E. P. (2016) · JAMA · doi:10.1001/jama.2016.3332 · Cited by 704
The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group
Teutsch, S. M., Bradley, L., Palomaki, G. E., Haddow, J. E., Piper, M., Calonge, N., Dotson, W. D., Douglas, M. P., & Berg, A. O. (2009) · Genetics in Medicine · doi:10.1097/gim.0b013e318184137c · Cited by 684
Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data
Walter, S. D. (2002) · Statistics in Medicine · doi:10.1002/sim.1099 · Cited by 662
A systematic review of rapid diagnostic tests for the detection of tuberculosis infection
Dinnes, J., Deeks, J., Kunst, H., Gibson, A., Cummins, E., Waugh, N., Drobniewski, F., & Lalvani, A. (2007) · Health Technology Assessment · doi:10.3310/hta11030 · Cited by 666
Screening for Colorectal Cancer
Lin, J. S., Perdue, L. A., Henrikson, N. B., Bean, S., & Blasi, P. R. (2021) · JAMA · doi:10.1001/jama.2021.4417 · Cited by 661
Diagnostic accuracy of the Berlin questionnaire, STOP-BANG, STOP, and Epworth sleepiness scale in detecting obstructive sleep apnea: A bivariate meta-analysis
Chiu, H.-Y., Chen, P.-Y., Chuang, L.-P., Chen, N.-H., Tu, Y.-K., Hsieh, Y.-J., Wang, Y.-C., & Guilleminault, C. (2017) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2016.10.004 · Cited by 650
Inflammatory Bowel Disease Diagnosed with US, MR, Scintigraphy, and CT: Meta-analysis of Prospective Studies
Horsthuis, K., Bipat, S., Bennink, R. J., & Stoker, J. (2008) · Radiology · doi:10.1148/radiol.2471070611 · Cited by 629
European Consensus Conference on Diagnosis and Treatment of Germ Cell Cancer: A Report of the Second Meeting of the European Germ Cell Cancer Consensus group (EGCCCG): Part I
Krege, S., Beyer, J., Souchon, R., Albers, P., Albrecht, W., Ferran Algaba, B., Bamberg, M., Bodrogi, I., Bokemeyer, C., Cavallin-Ståhl, E. K., et al. (2008) · European Urology · doi:10.1016/j.eururo.2007.12.024 · Cited by 619
Evidence of bias and variation in diagnostic accuracy studies
Rutjes, A. W., Reitsma, J. B., Nisio, M. D., Smidt, N., van Rijn, J. C., & Bossuyt, P. M. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050090 · Cited by 623
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600
Sputum processing methods to improve the sensitivity of smear microscopy for tuberculosis: a systematic review
Steingart, K. R., Ng, V., Henry, M., Hopewell, P. C., Ramsay, A., Cunningham, J. A., Urbanczik, R., Perkins, M. D., Aziz, M. A., & Pai, M. (2006) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(06)70602-8 · Cited by 580
Home Diagnosis of Sleep Apnea: A Systematic Review of the Literature
Flemons, W. W., Littner, M. R., Rowley, J. A., Gay, P. C., Anderson, W. M., Hudgel, D. W., McEvoy, R. D., & Loube, D. I. (2003) · CHEST Journal · doi:10.1378/chest.124.4.1543 · Cited by 568
Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests.
Florkowski, C. M. (2008) · PubMed Central · Cited by 562
Assessing the Diagnostic Accuracy of Pulse Pressure Variations for the Prediction of Fluid Responsiveness
Cannesson, M. P., Manach, Y. L., Hofer, C. K., Goarin, J., Lehot, J.-J., Vallet, B., & Tavernier, B. (2011) · Anesthesiology · doi:10.1097/aln.0b013e318225b80a · Cited by 549
Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 549
Colorectal Liver Metastases: CT, MR Imaging, and PET for Diagnosis—Meta-analysis
Bipat, S., van Leeuwen, M. S., Comans, E. F. I., Pijl, M. E. J., Bossuyt, P. M., Zwinderman, A. H., & Stoker, J. (2005) · Radiology · doi:10.1148/radiol.2371042060 · Cited by 537
Accuracy of Ultrasonography, Spiral CT, Magnetic Resonance, and Alpha-Fetoprotein in Diagnosing Hepatocellular Carcinoma: A Systematic Review. CME
Colli, A., Fraquelli, M., Casazza, G., Massironi, S., Colucci, A., Conte, D., & Duca, P. (2006) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2006.00467.x · Cited by 538
Physical examination tests of the shoulder: a systematic review with meta-analysis of individual tests
Hegedus, E. J., Goode, A. P., Campbell, S., Morin, A., Tamaddoni, M., Moorman, C. T., & Cook, C. E. (2007) · British Journal of Sports Medicine · doi:10.1136/bjsm.2007.038406 · Cited by 533
Meta-Analysis of MR Imaging in the Diagnosis of Breast Lesions
Peters, N. H. G. M., Rinkes, I. H. M. B., Zuithoff, N. P. A., Mali, W. P. T. M., Moons, K. G. M., & Peeters, P. H. M. (2007) · Radiology · doi:10.1148/radiol.2461061298 · Cited by 529
Xpert MTB/RIF assay for the diagnosis of extrapulmonary tuberculosis: a systematic review and meta-analysis
Denkinger, C. M., Schumacher, S. G., Boehme, C. C., Dendukuri, N., Pai, M., & Steingart, K. R. (2014) · European Respiratory Journal · doi:10.1183/09031936.00007814 · Cited by 527
Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review
Bachmann, L. M., Kolb, E., Koller, M. T., Steurer, J., & ter Riet, G. (2003) · BMJ · doi:10.1136/bmj.326.7386.417 · Cited by 526
Evaluation of diagnostic tests when there is no gold standard. A review of methods
Rutjes, A., Reitsma, J., Coomarasamy, A., Khan, K., & Bossuyt, P. (2007) · Health Technology Assessment · doi:10.3310/hta11500 · Cited by 524
Ultrasound-Based Transient Elastography for the Detection of Hepatic Fibrosis: Systematic Review and Meta-analysis
Talwalkar, J. A., Kurtz, D. M., Schoenleber, S. J., West, C. P., & Montori, V. M. (2007) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2007.07.020 · Cited by 523
The urine dipstick test useful to rule out infections. A meta-analysis of the accuracy
Devillé, W. L., Yzermans, J. C., van Duijn, N. P., Bezemer, P. D., van der Windt, D. A., & Bouter, L. M. (2004) · BMC Urology · doi:10.1186/1471-2490-4-4 · Cited by 519
Screening for circulating galactomannan as a noninvasive diagnostic tool for invasive aspergillosis in prolonged neutropenic patients and stem cell transplantation recipients: a prospective validation
Maertens, J. A., Verhaegen, J., Lagrou, K., Eldere, J. V., & Boogaerts, M. (2001) · Blood · doi:10.1182/blood.v97.6.1604 · Cited by 513
European consensus on diagnosis and treatment of germ cell cancer: a report of the European Germ Cell Cancer Consensus Group (EGCCCG)
Schmoll, H., Souchon, R., Krege, S., Albers, P., Beyer, J., Kollmannsberger, C., Fosså, S. D., Skakkebæk, N. E., de Wit, R., Fizazi, K. S., et al. (2004) · Annals of Oncology · doi:10.1093/annonc/mdh301 · Cited by 510
Statistical Evaluation of a Biomarker
Ray, P., Manach, Y. L., Riou, B., Houle, T. T., & Warner, D. S. (2010) · Anesthesiology · doi:10.1097/aln.0b013e3181d47604 · Cited by 499
Case–Control and Two-Gate Designs in Diagnostic Accuracy Studies
Rutjes, A. W., Reitsma, J. B., Vandenbroucke, J. P., Glas, A. S., & Bossuyt, P. M. (2005) · Clinical Chemistry · doi:10.1373/clinchem.2005.048595 · Cited by 516
Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy
Song, F., Khan, K. S., Dinnes, J., & Sutton, A. J. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.88 · Cited by 485
Statistics Notes: Blinding in clinical trials and other studies
Day, S. J. (2000) · BMJ · doi:10.1136/bmj.321.7259.504 · Cited by 487
Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis.
Leeflang, M., Bossuyt, P., & Irwig, L. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.007 · Cited by 481
Diagnosing and Managing Common Food Allergies: A Systematic Review
Chafen, J. J. S., Newberry, S. J., Riedl, M. A., Bravata, D. M., Maglione, M., Suttorp, M. J., Sundaram, V., Paige, N. M., Towfigh, A., Hulley, B. J., et al. (2010) · JAMA · doi:10.1001/jama.2010.582 · Cited by 482
How to Use an Article About a Diagnostic Test
Scales, C. D., Dahm, P., Sultan, S., Campbell-Scherer, D. L., & Devereaux, P. J. (2008) · The Journal of Urology · doi:10.1016/j.juro.2008.04.026 · Cited by 457
Pleural ultrasonography versus chest radiography for the diagnosis of pneumothorax: review of the literature and meta-analysis
Alrajab, S., Youssef, A. M., Akkuş, N. İ., & Caldito, G. C. (2013) · Critical Care · doi:10.1186/cc13016 · Cited by 461
Prevalence of and screening for serious spinal pathology in patients presenting to primary care settings with acute low back pain
Henschke, N., Maher, C. G., Refshauge, K. M., Herbert, R., Cumming, R. G., Bleasel, J. F., York, J. R., Das, A., & McAuley, J. H. (2009) · Arthritis & Rheumatism · doi:10.1002/art.24853 · Cited by 477
Exploring sources of heterogeneity in systematic reviews of diagnostic tests
Lijmer, J. G., Bossuyt, P. M., & Heisterkamp, S. H. (2002) · Statistics in Medicine · doi:10.1002/sim.1185 · Cited by 446
Self-citation
Systematic Review: Computed Tomography and Ultrasonography To Detect Acute Appendicitis in Adults and Adolescents
Terasawa, T., Blackmore, C. C., Bent, S., & Kohlwes, R. J. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-7-200410050-00011 · Cited by 442
A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy
Dinnes, J., Deeks, J. J., Kirby, J., & Roderick, P. J. (2005) · Health Technology Assessment · doi:10.3310/hta9120 · Cited by 440
Extending an evidence hierarchy to include topics other than treatment: revising the Australian 'levels of evidence'
Merlin, T., Weston, A., & Tooher, R. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-34 · Cited by 435
Test Characteristics of α-Fetoprotein for Detecting Hepatocellular Carcinoma in Patients with Hepatitis C
Gupta, S. N., Bent, S., & Kohlwes, J. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-139-1-200307010-00012 · Cited by 423
FDG PET in the differential diagnosis of parkinsonian disorders
Eckert, T., Barnes, A., DHAWAN, V. M., Frucht, S. J., Gordon, M. F., Feigin, A., & Eidelberg, D. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.03.012 · Cited by 418
GRADE guidelines: 21 part 1. Study design, risk of bias, and indirectness in rating the certainty across a body of evidence for test accuracy
Schünemann, H. J., Mustafa, R. A., Brożek, J., Steingart, K. R., Leeflang, M., Murad, M. H., Bossuyt, P. M., Glasziou, P., Jaeschke, R., Lange, S., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.12.020 · Cited by 433

Authors: Lijmer, J. G.
Affiliations: Amsterdam UMC Location University of Amsterdam (NL)
Corresponding author: Lijmer, Jeroen G.
ORCID: Lijmer, Jeroen G.
Venue: JAMA
Publisher: American Medical Association (AMA)
Published: 1999-09-15
Volume: 282
Issue: 11
Pages: 1061
ISSN: 0002-9955 (ISSN-L)
0098-7484
DOI: 10.1001/jama.282.11.1061
OpenAlex: W1971348077
Cited by: 1,861
References: 38
Citations by year: 2026: 56 · 2025: 38 · 2024: 38 · 2023: 45 · 2022: 45 · 2021: 48 · 2020: 74 · 2019: 50 · 2018: 73 · 2017: 66 · 2016: 79 · 2015: 82 · 2014: 73 · 2013: 78 · 2012: 79
FWCI: 92.03
Type: journal article
Language: en
Open access: closed
Locations: JAMA (publishedVersion)
PubMed (publishedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion)
Pure Amsterdam UMC (submittedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords: diagnostic accuracy, study design bias, methodological quality, diagnostic odds ratio, reference materials
MeSH terms: Diagnostic Tests, Routine, Evaluation Studies as Topic, Quality Control, Reference Standards, Regression Analysis, Research Design, Sensitivity and Specificity, Bias, Odds Ratio

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Empirical evidence of design-related bias in studies of diagnostic tests.

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks