Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 901–1,000 of 1,197

Artificial Intelligence for Medical Image Analysis: A Guide for Authors and Reviewers
England, J. R., & Cheng, P. M. (2018) · American Journal of Roentgenology · doi:10.2214/ajr.18.20490 · Cited by 180
AB ‐Bind: Antibody binding mutational database for computational affinity predictions
Sirin, S., Apgar, J. R., Bennett, E. M., & Keating, A. E. (2016) · Protein Science · doi:10.1002/pro.2829 · Cited by 179
Evaluation of new acute kidney injury biomarkers in a mixed intensive care unit*
Doi, K., Negishi, K., Ishizu, T., Katagiri, D., Fujita, T., Matsubara, T., Yahagi, N., Sugaya, T., & Noiri, E. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e318225761a · Cited by 179
Pre-resuscitation factors associated with mortality in 49,130 cases of in-hospital cardiac arrest: a report from the National Registry for Cardiopulmonary Resuscitation.
Larkin, G., Copes, W., Nathanson, B., & Kaye, W. (2010) · Resuscitation · doi:10.1016/j.resuscitation.2009.11.021 · Cited by 174
Comparison of the Unstructured Clinician Gestalt, the Wells Score, and the Revised Geneva Score to Estimate Pretest Probability for Suspected Pulmonary Embolism
Penaloza, A., Verschuren, F., Meyer, G., Quentin-Georget, S., Soulié, C., Thys, F., & Roy, P. (2013) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2012.11.002 · Cited by 179
Prediction of survival odds in COVID-19 by zinc, age and selenoprotein P as composite biomarker
Heller, R. A., Sun, Q., Hackler, J., Seelig, J., Seibert, L., Cherkezov, A., Minich, W. B., Seemann, P., Diegmann, J., Pilz, M., et al. (2020) · Redox Biology · doi:10.1016/j.redox.2020.101764 · Cited by 185
Validation of Endoscopic Activity Scores in Patients With Crohn's Disease Based on a Post Hoc Analysis of Data From SONIC
Ferrante, M., Colombel, J., Sandborn, W. J., Reinisch, W., Mantzaris, G. J., Kornbluth, A. A., Rachmilewitz, D., Lichtiger, S., D’Haens, G., van der Woude, C. J., et al. (2013) · Gastroenterology · doi:10.1053/j.gastro.2013.08.010 · Cited by 178
Predicting Mortality in Non-Variceal Upper Gastrointestinal Bleeders: Validation of the Italian PNED Score and Prospective Comparison With the Rockall Score
Marmo, R., Koch, M., Cipolletta, L., Capurso, L., Grossi, E., Cestari, R., Bianco, M. A., Pandolfo, N., Dezi, A., Casetti, T., et al. (2010) · The American Journal of Gastroenterology · doi:10.1038/ajg.2009.687 · Cited by 178
Antimüllerian hormone in gonadotropin releasing-hormone antagonist cycles: prediction of ovarian response and cumulative treatment outcome in good-prognosis patients
Arce, J., Marca, A. L., Klein, B. M., Andersen, A. N., & Fleming, R. (2013) · Fertility and Sterility · doi:10.1016/j.fertnstert.2012.12.048 · Cited by 178
Dynapenic-Obesity and Physical Function in Older Adults
B̀ouchard, D. R., & Janssen, I. (2009) · The Journals of Gerontology Series A · doi:10.1093/gerona/glp159 · Cited by 180
Noninvasive Saliva-based EGFR Gene Mutation Detection in Patients with Lung Cancer
Wei, F., Lin, C., Joon, A. Y., Feng, Z., Troche, G. E., Lira, M. E., Chia, D., Mao, M., Ho, C., Su, W., et al. (2014) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201406-1003oc · Cited by 180
Fully Automated Diagnosis of Anterior Cruciate Ligament Tears on Knee MR Images by Using Deep Learning
Liu, F., Guan, B., Zhou, Z., Samsonov, A. A., Rosas, H. G., Lian, K., Sharma, R., Kanarek, A. A., Kim, J., Guermazi, A., et al. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180091 · Cited by 178
ACPred-Fuse: fusing multi-view information improves the prediction of anticancer peptides
Rao, B., Zhou, C., Zhang, G., Su, R., & Wei, L. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz088 · Cited by 178
Screening Healthy Infants for Iron Deficiency Using Reticulocyte Hemoglobin Content
Ullrich, C. (2005) · JAMA · doi:10.1001/jama.294.8.924 · Cited by 180
Text Categorization Models for High-Quality Article Retrieval in Internal Medicine
Aphinyanaphongs, Y. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1641 · Cited by 177
Integration of Early Physiological Responses Predicts Later Illness Severity in Preterm Infants
Saria, S., Rajani, A. K., Gould, J. B., Koller, D., & Penn, A. A. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3001304 · Cited by 177
Comparison of Cardiovascular Magnetic Resonance and Single-Photon Emission Computed Tomography in Women With Suspected Coronary Artery Disease From the Clinical Evaluation of Magnetic Resonance Imaging in Coronary Heart Disease (CE-MARC) Trial
Greenwood, J. P., Motwani, M., Maredia, N., Brown, J. M., Everett, C., Nixon, J., Bijsterveld, P., Dickinson, C. J., Ball, S. G., & Plein, S. (2013) · Circulation · doi:10.1161/circulationaha.112.000071 · Cited by 177
Clinical and Epidemiologic Phenotypes of Childhood Asthma
Depner, M., Fuchs, O., Genuneit, J., Karvonen, A. M., Hyvärinen, A., Kaulek, V., Roduit, C., Weber, J., Schaub, B., Lauener, R. P., et al. (2013) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201307-1198oc · Cited by 177
Can Serum β-Hydroxybutyrate Be Used to Diagnose Diabetic Ketoacidosis?
Sheikh-Ali, M., Karon, B. S., Basu, A., Kudva, Y. C., Muller, L. A., Xu, J., Schwenk, W. F., & Miles, J. M. (2008) · Diabetes Care · doi:10.2337/dc07-1683 · Cited by 159
Predictive validity of the SVR-20 and Static-99 in a Dutch sample of treated sex offenders.
de Vogel, V., de Ruiter, C., van Beek, D., & Mead, G. (2004) · Law and Human Behavior · doi:10.1023/b:lahu.0000029137.41974.eb · Cited by 177
Predicting Intensive Care Unit Readmission with Machine Learning Using Electronic Health Record Data
Rojas, J. C., Carey, K. A., Edelson, D. P., Venable, L. R., Howell, M. D., & Churpek, M. M. (2018) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201710-787oc · Cited by 186
Early Diagnosis of Myocardial Infarction With Point-of-Care High-Sensitivity Cardiac Troponin I
Boeddinghaus, J., Nestelberger, T., Koechlin, L., Wussler, D. N., López‐Ayala, P., Walter, J. E., Troester, V., Ratmann, P. D., Seidel, F., Zimmermann, T., et al. (2020) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2019.12.065 · Cited by 177
Risk Prediction Models for Post‐Operative Mortality in Patients With Cirrhosis
Mahmud, N., Fricker, Z. P., Hubbard, R. A., Ioannou, G. N., Lewis, J. D., Taddei, T. H., Rothstein, K. D., Serper, M., Goldberg, D. S., & Kaplan, D. E. (2020) · Hepatology · doi:10.1002/hep.31558 · Cited by 177
The LAS VEGAS risk score for prediction of postoperative pulmonary complications: An observational study
Neto, A. S., da Costa, L. G. V., Hemmes, S. N. T., Canet, J., Hedenstierna, G., Jaber, S., Hiesmayr, M., Hollmann, M. W., Mills, G. H., Vidal Melo, M. F., et al. (2018) · European Journal of Anaesthesiology · doi:10.1097/eja.0000000000000845 · Cited by 160
Growth Differentiation Factor 15 in Heart Failure with Preserved vs. Reduced Ejection Fraction
Chan, M. M. Y., Santhanakrishnan, R., Chong, J. P. C., Chen, Z., Tai, B. C., Liew, O. W., Ng, T. P., Ling, L. H., Sim, D., Leong, K. T. G., et al. (2015) · European Journal of Heart Failure · doi:10.1002/ejhf.431 · Cited by 176
SAFE biopsy
Sebastiani, G., Halfon, P., Castéra, L., Pol, S., Thomas, D. L., Mangia, A., Marco, V. D., Pirisi, M., Voiculescu, M., Guido, M., et al. (2009) · Hepatology · doi:10.1002/hep.22859 · Cited by 176
Using sensation seeking to target adolescents for substance use interventions
Sargent, J. D., Tanski, S. E., Stoolmiller, M., & Hanewinkel, R. (2010) · Addiction · doi:10.1111/j.1360-0443.2009.02782.x · Cited by 176
Hepatic Fibrosis: Prospective Comparison of MR Elastography and US Shear-Wave Elastography for Evaluation
Yoon, J. H., Lee, J. M., Joo, I., Lee, E. S., Sohn, J. Y., Jang, S. K., Lee, K. B., Han, J. K., & Choi, B. I. (2014) · Radiology · doi:10.1148/radiol.14132000 · Cited by 176
ALBI and PALBI grade predict survival for HCC across treatment modalities and BCLC stages in the MELD Era
Liu, P., Hsu, C., Hsia, C., Lee, Y.-H., Chiou, Y., Huang, Y., Lee, F., Lin, H., Hou, M., & Huo, T. (2016) · Journal of Gastroenterology and Hepatology · doi:10.1111/jgh.13608 · Cited by 176
Maximization of the usage of coronary CTA derived plaque information using a machine learning based algorithm to improve risk stratification; insights from the CONFIRM registry.
van Rosendael, A. V., Maliakal, G., Kolli, K., Beecy, A., Al'aref, S., Dwivedi, A., Singh, G., Panday, M., Kumar, A., Ma, X.-Y., et al. (2018) · Journal of cardiovascular computed tomography · doi:10.1016/j.jcct.2018.04.011 · Cited by 166
Longitudinal MRI-based fusion novel model predicts pathological complete response in breast cancer treated with neoadjuvant chemotherapy: a multicenter, retrospective study
Huang, Y., Zhu, T., Zhang, X., Li, W., Zheng, X., Cheng, M., Ji, F., Zhang, L., Yang, C., Wu, Z., et al. (2023) · eClinicalMedicine · doi:10.1016/j.eclinm.2023.101899 · Cited by 175
Metabolomic Derangements Are Associated with Mortality in Critically Ill Adult Patients
Rogers, A. J., McGeachie, M. J., Baron, R. M., Gazourian, L., Haspel, J. A., Nakahira, K., Fredenburgh, L. E., Hunninghake, G. M., Raby, B. A., Matthay, M. A., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0087538 · Cited by 179
Left Atrial Strain Provides Incremental Value for Embolism Risk Stratification over CHA2DS2-VASc Score and Indicates Prognostic Impact in Patients with Atrial Fibrillation
Obokata, M., Negishi, K., Kurosawa, K., Tateno, R., Tange, S., Arai, M., Amano, M., & Kurabayashi, M. (2014) · Journal of the American Society of Echocardiography · doi:10.1016/j.echo.2014.03.010 · Cited by 177
Protective Factors for Violence Risk in Forensic Psychiatric Patients: A Retrospective Validation Study of the SAPROF
Robbé, M. D. V., de Vogel, V., & de Spa, E. (2011) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2011.600232 · Cited by 163
New Metrics for Assessing Diagnostic Potential of Candidate Biomarkers
Pickering, J. W., & Endre, Z. H. (2012) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.09590911 · Cited by 175
Breast Lesions: Evaluation with US Strain Imaging–Clinical Experience of Multiple Observers
Regner, D. M., Hesley, G. K., Hangiandreou, N. J., Morton, M. J., Nordland, M. R., Meixner, D. D., Hall, T. J., Farrell, M. A., Mandrekar, J. N., Harmsen, W. S., et al. (2006) · Radiology · doi:10.1148/radiol.2381041336 · Cited by 175
Validity of the 12-item version of the General Health Questionnaire in detecting depression in the general population
Lundin, A., Hallgren, M. Å., Theobald, H., Hellgren, C., & Torgén, M. (2016) · Public Health · doi:10.1016/j.puhe.2016.03.005 · Cited by 175
Use of brain diffusion tensor imaging for the prediction of long-term neurological outcomes in patients after cardiac arrest: a multicentre, international, prospective, observational, cohort study
Velly, L. J., Perlbarg, V., Boulier, T., Adam, N., Delphine, S., Luyt, C. E., Battisti, V., Torkomian, G., Arbelot, C., Chabanne, R., et al. (2018) · The Lancet Neurology · doi:10.1016/s1474-4422(18)30027-9 · Cited by 175
MicroRNA Expression Levels Are Altered in the Cerebrospinal Fluid of Patients with Young-Onset Alzheimer’s Disease
McKeever, P. M., Schneider, R., Taghdiri, F., Weichert, A., Multani, N., Brown, R. A., Boxer, A. L., Karydas, A., Miller, B., Robertson, J., et al. (2018) · Molecular Neurobiology · doi:10.1007/s12035-018-1032-x · Cited by 166
Can peritumoral radiomics increase the efficiency of the prediction for lymph node metastasis in clinical stage T1 lung adenocarcinoma on CT?
Wang, X., Zhao, X., Li, Q., Xia, W., Peng, Z., Zhang, R., Li, Q., Jian, J., Wang, W., Tang, Y., et al. (2019) · European Radiology · doi:10.1007/s00330-019-06084-0 · Cited by 176
Shock index: an effective predictor of outcome in postpartum haemorrhage?
Nathan, H., El Ayadi, A., Hezelgrave, N., Seed, P., Butrick, E., Miller, S., Briley, A., Bewley, S., & Shennan, A. (2015) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.13206 · Cited by 161
Screening Sarcopenia in Community-Dwelling Older Adults: SARC-F vs SARC-F Combined With Calf Circumference (SARC-CalF)
Yang, M., Hu, X., Xie, L., Zhang, L., Zhou, J., Lin, J., Wang, Y., Li, Y., Han, Z., Zhang, D., et al. (2018) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2017.12.016 · Cited by 174
Extracellular matrix proteomics identifies molecular signature of symptomatic carotid plaques
Langley, S. R., Willeit, K., Didangelos, A., Matic, L. P., Skroblin, P., Barallobre-Barreiro, J., Lengquist, M., Rungger, G., Kapustin, A., Kedenko, L., et al. (2017) · Journal of Clinical Investigation · doi:10.1172/jci86924 · Cited by 174
Comparison of Thoracic Auscultation, Clinical Score, and Ultrasonography as Indicators of Bovine Respiratory Disease in Preweaned Dairy Calves
Buczinski, S., Forté, G., Francoz, D., & Bélanger, A.-M. (2014) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.12251 · Cited by 174
Predicting in vivo binding sites of RNA-binding proteins using mRNA secondary structure
Li, X., Quon, G., Lipshitz, H. D., & Morris, Q. (2010) · RNA · doi:10.1261/rna.2017210 · Cited by 181
Changes in Discard Rate After the Introduction of the Kidney Donor Profile Index (KDPI)
Bae, S., Massie, A. B., Luo, X., Anjum, S. K., Desai, N. M., & Segev, D. L. (2016) · American Journal of Transplantation · doi:10.1111/ajt.13769 · Cited by 174
PHASES Score for the Management of Intracranial Aneurysm
Bijlenga, P., Gondar, R., Schilling, S., Morel, S., Hirsch, S., Cuony, J., Corniola, M.-V., Perren, F., Rüfenacht, D. A., & Schaller, K. (2017) · Stroke · doi:10.1161/strokeaha.117.017391 · Cited by 175
Integrated prediction of lesion-specific ischaemia from quantitative coronary CT angiography using machine learning: a multicentre study
Dey, D., Gaur, S., Øvrehus, K. A., Slomka, P. J., Betancur, J., Goeller, M., Hell, M. M., Gransar, H., Berman, D. S., Achenbach, S. S., et al. (2018) · European Radiology · doi:10.1007/s00330-017-5223-z · Cited by 179
The Antenatal Risk Questionnaire (ANRQ): Acceptability and use for psychosocial risk assessment in the maternity setting
Austin, M., Colton, J., Priest, S. R., Reilly, N. M., & Hadži-Pavlović, D. (2011) · Women and Birth · doi:10.1016/j.wombi.2011.06.002 · Cited by 173
Comparison of Logistic Regression and Artificial Neural Network Models in Breast Cancer Risk Estimation
Ayer, T., Chhatwal, J., Alagöz, O., Kahn, C. E., Woods, R. W., & Burnside, E. S. (2009) · Radiographics · doi:10.1148/rg.301095057 · Cited by 173
Breast Lesions: Evaluation with Shear Wave Elastography, with Special Emphasis on the “Stiff Rim” Sign
Zhou, J., Zhan, W., Chang, C., Zhang, X.-X., Jia, Y., Dong, Y., Zhou, C. J., Sun, J., & Grant, E. G. (2014) · Radiology · doi:10.1148/radiol.14130818 · Cited by 173
Heart Rate Variability Can Be Used to Estimate Sleepiness-related Decrements in Psychomotor Vigilance during Total Sleep Deprivation
Chua, E. C.-P., Tan, W. Q., Yeo, S. C., Lau, P., Lee, I., Mien, I. H., Puvanendran, K., & Gooley, J. J. (2012) · SLEEP · doi:10.5665/sleep.1688 · Cited by 173
Angiogenic factor imbalance early in pregnancy predicts adverse outcomes in patients with lupus and antiphospholipid antibodies: results of the PROMISSE study
Kim, M., Buyon, J. P., Guerra, M. M., Rana, S., Zhang, D., Laskin, C. A., Petri, M., Lockshin, M. D., Sammaritano, L. R., Branch, D. W., et al. (2015) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2015.09.066 · Cited by 176
Mechanical power normalized to predicted body weight as a predictor of mortality in patients with acute respiratory distress syndrome
Zhang, Z., Zheng, B., Liu, N., Ge, H., & Hong, Y. (2019) · Intensive Care Medicine · doi:10.1007/s00134-019-05627-9 · Cited by 179
Plasma amyloid β 40/42 ratio predicts cerebral amyloidosis in cognitively normal individuals at risk for Alzheimer's disease
Vergallo, A., Mégret, L., Lista, S., Cavedo, E., Zetterberg, H., Blennow, K., Vanmechelen, E., Vos, A. D., Habert, M., Potier, M., et al. (2019) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2019.03.009 · Cited by 178
Assessment of Intratumoral and Peritumoral Computed Tomography Radiomics for Predicting Pathological Complete Response to Neoadjuvant Chemoradiation in Patients With Esophageal Squamous Cell Carcinoma
Hu, Y., Xie, C., Yang, H., Ho, J. W. K., Wen, J., Han, L., Chiu, K. W.-H., Fu, J., & Vardhanabhuti, V. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.15927 · Cited by 174
Identifying early glaucoma with optical coherence tomography
Nouri‐Mahdavi, K., Hoffman, D., Tannenbaum, D. P., Law, S. K., & Caprioli, J. (2004) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2003.09.004 · Cited by 173
The suitability of the Pediatric Index of Mortality (PIM), PIM2, the Pediatric Risk of Mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand
Slater, A., & Shann, F. A. (2004) · Pediatric Critical Care Medicine · doi:10.1097/01.pcc.0000138557.31831.65 · Cited by 173
Heterogeneous Network Edge Prediction: A Data Integration Approach to Prioritize Disease-Associated Genes
Himmelstein, D. S., & Baranzini, S. E. (2015) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1004259 · Cited by 172
Depressive, anxiety, and somatoform disorders in primary care: prevalence and recognition
Mergl, R., Seidscheck, I., Allgaier, A., Möller, H., Hegerl, U., & Henkel, V. (2006) · Depression and Anxiety · doi:10.1002/da.20192 · Cited by 172
Discrimination between Glaucomatous and Nonglaucomatous Eyes Using Quantitative Imaging Devices and Subjective Optic Nerve Head Assessment
́n-Ortega, J. E. D., Arthur, S. N., McGwin, G., Xie, A., Monheit, B. E., & Girkin, C. A. (2006) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.05-1239 · Cited by 172
An Artificial Intelligence–based Mammography Screening Protocol for Breast Cancer: Outcome and Radiologist Workload
Lauritzen, A. D., Rodríguez‐Ruiz, A., von Euler‐Chelpin, M., Lynge, E., Vejborg, I., Nielsen, M., Karssemeijer, N., & Lillholm, M. (2022) · Radiology · doi:10.1148/radiol.210948 · Cited by 187
Pericoronary Adipose Tissue Attenuation, Low-Attenuation Plaque Burden, and 5-Year Risk of Myocardial Infarction
Tzolos, E., Williams, M. C., McElhinney, P. A., Lin, A., Grodecki, K., Tomasino, G. F., Cadet, S. J., Kwieciński, J., Doris, M. K., Adamson, P. D., et al. (2022) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2022.02.004 · Cited by 172
Pulmonary Arterial Capacitance Is an Important Predictor of Mortality in Heart Failure With a Preserved Ejection Fraction
Al‐Naamani, N., Preston, I. R., Paulus, J. K., Hill, N. S., & Roberts, K. E. (2015) · JACC Heart Failure · doi:10.1016/j.jchf.2015.01.013 · Cited by 176
Validation of the Cepstral Spectral Index of Dysphonia (CSID) as a Screening Tool for Voice Disorders: Development of Clinical Cutoff Scores
Awan, S. N., Roy, N., Zhang, D., & Cohen, S. M. (2015) · Journal of Voice · doi:10.1016/j.jvoice.2015.04.009 · Cited by 174
Comparison of Positron Emission Tomography Measurement of Adenosine-Stimulated Absolute Myocardial Blood Flow Versus Relative Myocardial Tracer Content for Physiological Assessment of Coronary Artery Stenosis Severity and Location
Hajjiri, M., Leavitt, M. B., Zheng, H., Spooner, A. E., Fischman, A. J., & Gewirtz, H. (2009) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2009.04.004 · Cited by 171
Projecting future changes in distributions of pelagic fish species of Northeast Pacific shelf seas
Cheung, W. W. L., Brodeur, R. D., Okey, T. A., & Pauly, D. F. (2014) · Progress In Oceanography · doi:10.1016/j.pocean.2014.09.003 · Cited by 175
Fecal Calprotectin Levels Predict Colorectal Inflammation Among Patients With Chronic Diarrhea Referred for Colonoscopy
Limburg, P. J., Ahlquist, D. A., Sandborn, W. J., Mahoney, D. W., Devens, M. E., Harrington, J. J., & Zinsmeister, A. R. (2000) · The American Journal of Gastroenterology · doi:10.1016/s0002-9270(00)01987-0 · Cited by 171
Sensitivity and Specificity of Anthropometrics for the Prediction of Diabetes in a Biracial Cohort
Stevens, J., Couper, D. J., Pankow, J. S., Folsom, A. R., Duncan, B. B., Nieto, F., Jones, D. W., & Tyroler, H. A. (2001) · Obesity Research · doi:10.1038/oby.2001.94 · Cited by 175
Psychometric Testing Confirms That the Brazilian-Portuguese Adaptations, the Original Versions of the Fear-Avoidance Beliefs Questionnaire, and the Tampa Scale of Kinesiophobia Have Similar Measurement Properties
de Souza, F. S., Marinho, C. D. S., Siqueira, F. B., Maher, C. G., & Costa, L. O. P. (2008) · Spine · doi:10.1097/brs.0b013e31816c8329 · Cited by 173
Early Invasive Cervical Cancer: CT and MR Imaging in Preoperative Evaluation—ACRIN/GOG Comparative Study of Diagnostic Performance and Interobserver Variability
Hricak, H., Gatsonis, C. A., Coakley, F. V., Snyder, B. S., Reinhold, C., Schwartz, L. H., Woodward, P. J., Pannu, H. K., Amendola, M. A., & Mitchell, D. G. (2007) · Radiology · doi:10.1148/radiol.2452061983 · Cited by 171
Technical and Diagnostic Performance of 6 Assays for the Measurement of Citrullinated Protein/Peptide Antibodies in the Diagnosis of Rheumatoid Arthritis
Coenen, D., Verschueren, P., Westhovens, R., & Bossuyt, X. (2007) · Clinical Chemistry · doi:10.1373/clinchem.2006.078063 · Cited by 173
Integrating CT Myocardial Perfusion and CT-FFR in the Work-Up of Coronary Artery Disease
Coenen, A., Rossi, A., Lubbers, M. M., Kurata, A., Kono, A. K., Chelu, R. G., Segreto, S., Dijkshoorn, M. L., Wragg, A., van Geuns, R., et al. (2017) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2016.09.028 · Cited by 171
Identification of Alzheimer's disease using a convolutional neural network model based on T1-weighted magnetic resonance imaging
Bae, J. B., Lee, S., Jung, W., Park, S., Kim, W., Oh, H., Han, J. W., Kim, G. E., Kim, J. S., Kim, J. H., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-79243-9 · Cited by 171
Potential Usefulness of Inflammatory Markers to Monitor Respiratory Functional Impairment in Sarcoidosis
Kos, S., van Dieijen‐Visser, M. P., Mulder, P. G. H., & Drent, M. (2003) · Clinical Chemistry · doi:10.1373/49.9.1510 · Cited by 171
A Permutation Test to Compare Receiver Operating Characteristic Curves
Venkatraman, E. S. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.01134.x · Cited by 170
Effects of early corticosteroid treatment on magnetic resonance imaging and ultrasonography findings in giant cell arteritis
Hauenstein, C., Reinhard, M., Geiger, J., Markl, M., Hetzel, A., Treszl, A., Vaith, P., & Bley, T. A. (2012) · Rheumatology · doi:10.1093/rheumatology/kes153 · Cited by 170
Radiologist Performance in the Diagnosis of Internal Hernia by Using Specific CT Findings with Emphasis on Transmesenteric Hernia
Blachar, A., Federle, M. P., Brancatelli, G., Peterson, M. S., Oliver, J. H., & Li, W. (2001) · Radiology · doi:10.1148/radiol.2212010126 · Cited by 170
Non-Invasive Assessment of Hepatic Steatosis in Patients with NAFLD Using Controlled Attenuation Parameter and 1H-MR Spectroscopy
Karlas, T., Petroff, D., Garnov, N., Böhm, S. K., Tenckhoff, H., Wittekind, C., Wiese, M., Schiefke, I., Linder, N., Schaudinn, A., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0091987 · Cited by 169
A Comparison of Prediction Models for Fractures in Older Women
Ensrud, K. E. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.404 · Cited by 169
Osteocalcin Expression by Circulating Endothelial Progenitor Cells in Patients With Coronary Atherosclerosis
Gössl, M., Mödder, U. I., Atkinson, E. J., Lerman, A., & Khosla, S. (2008) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2008.07.019 · Cited by 169
Improving customer complaint management by automatic email classification using linguistic style features as predictors
Coussement, K., & den Poel, D. V. (2007) · Decision Support Systems · doi:10.1016/j.dss.2007.10.010 · Cited by 169
The Mayo Prosthetic Joint Infection Risk Score: Implication for Surgical Site Infection Reporting and Risk Stratification
Berbari, E. F., Osmon, D. R., Lahr, B. D., Eckel‐Passow, J. E., Tsaras, G., Hanssen, A. D., Mabry, T. M., Steckelberg, J. M., & Thompson, R. L. (2012) · Infection Control and Hospital Epidemiology · doi:10.1086/666641 · Cited by 169
Monocyte Distribution Width: A Novel Indicator of Sepsis-2 and Sepsis-3 in High-Risk Emergency Department Patients*
Crouser, E. D., Parrillo, J. E., Seymour, C. W., Angus, D. C., Bicking, K., Esguerra, V. G., Peck‐Palmer, O. M., Magari, R. T., JULIAN, M. W., Kleven, J. M., et al. (2019) · Critical Care Medicine · doi:10.1097/ccm.0000000000003799 · Cited by 169
Comparing Rapid Scoring Systems in Mortality Prediction of Critically Ill Patients With Novel Coronavirus Disease
Hu, H., Yao, N., & Qiu, Y. (2020) · Academic Emergency Medicine · doi:10.1111/acem.13992 · Cited by 169
The CASE adherence index: A novel method for measuring adherence to antiretroviral therapy
Mannheimer, S. B., Mukherjee, R., Hirschhorn, L. R., Dougherty, J., Celano, S. A., Ciccarone, D., Graham, K. K., Mantell, J. E., Mundy, L. M., Eldred, L., et al. (2006) · AIDS Care · doi:10.1080/09540120500465160 · Cited by 166
Prediction of Outcome in Patients With Acute Ischemic Stroke Based on Initial Severity and Improvement in the First 24 h
Wouters, A., Nysten, C., Thijs, V., & Lemmens, R. (2018) · Frontiers in Neurology · doi:10.3389/fneur.2018.00308 · Cited by 162
Hepatic Dysfunction in Ambulatory Patients With Heart Failure
Kim, M. S., Kato, T., FARR, M. A., Wu, C. C. N., Givens, R. C., Collado, E., MANCINI, D. M., & Schulze, P. C. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.12.056 · Cited by 168
Trabecular Bone Score (TBS) Predicts Vertebral Fractures in Japanese Women Over 10 Years Independently of Bone Density and Prevalent Vertebral Deformity: The Japanese Population-Based Osteoporosis (JPOS) Cohort Study
Iki, M., Tamaki, J., Kadowaki, E., Sato, Y., Dongmei, N., Winzenrieth, R., Kagamimori, S., Kagawa, Y., & Yoneshima, H. (2013) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.2048 · Cited by 173
Diagnostic and prognostic gene expression signatures in 177 soft tissue sarcomas: hypoxia-induced transcription profile signifies metastatic potential
Francis, P., Namløs, H. M., Müller, C., Edén, P., Fernebro, J., Berner, J., Bjerkehagen, B., Åkerman, M., Bendahl, P., Isinger-Ekstrand, A., et al. (2007) · BMC Genomics · doi:10.1186/1471-2164-8-73 · Cited by 169
Prediction of non-alcoholic fatty-liver disease and liver fat content by serum molecular lipids
Orešič, M., Hyötyläinen, T., Kotronen, A., Gopalacharyulu, P. V., Nygrén, H., Arola, J. T., Castillo, S., Mattila, I. M., Hakkarainen, A., Borra, R., et al. (2013) · Diabetologia · doi:10.1007/s00125-013-2981-2 · Cited by 173
Aortic Annular Sizing for Transcatheter Aortic Valve Replacement Using Cross-Sectional 3-Dimensional Transesophageal Echocardiography
Jilaihawi, H., Doctor, N., Kashif, M., Chakravarty, T., Rafique, A. M., Makar, M. M., Furugen, A., Nakamura, M., Mirocha, J., Gheorghiu, M., et al. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.11.055 · Cited by 168
Quantification of Coronary Atherosclerosis and Inflammation to Predict Coronary Events and All-Cause Mortality
Möhlenkamp, S., Lehmann, N., Moebus, S., Schmermund, A., Dragano, N., Stang, A., Siegrist, J., Mann, K., Jöckel, K., & Erbel, R. M. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.10.043 · Cited by 171
Concussion Symptom Inventory: An Empirically Derived Scale for Monitoring Resolution of Symptoms Following Sport-Related Concussion
Randolph, C., Millis, S., Barr, W., McCrea, M., Guskiewicz, K. M., Hammeke, T. A., & Kelly, J. P. (2009) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acp025 · Cited by 168
Derivation and Validation of Diagnostic Thresholds for Central Blood Pressure Measurements Based on Long-Term Cardiovascular Risks
Cheng, H.-M., Chuang, S.-Y., Sung, S.-H., Yu, W.-C., Pearson, A., Lakatta, E. G., Pan, W.-H., & Chen, C.-H. (2013) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.06.029 · Cited by 148
The development and validation of a CT-based radiomics signature for the preoperative discrimination of stage I-II and stage III-IV colorectal cancer
Liang, C., Huang, Y., Chen, X., Chen, X., Ma, Z., Dong, D., Tian, J., Liang, C., & Liu, Z. (2016) · Oncotarget · doi:10.18632/oncotarget.8919 · Cited by 168
Validation of deep-learning image reconstruction for coronary computed tomography angiography: Impact on noise, image quality and diagnostic accuracy
Benz, D. C., Benetos, G., Rampidis, G. P., von Felten, E., Bakula, A., Šustar, A., Kudura, K., Messerli, M., Fuchs, T. A., Gebhard, C., et al. (2020) · Journal of cardiovascular computed tomography · doi:10.1016/j.jcct.2020.01.002 · Cited by 169
T-cell CX3CR1 expression as a dynamic blood-based biomarker of response to immune checkpoint inhibitors
Yamauchi, T., Hoki, T., Oba, T., Jain, V., Chen, H., Attwood, K. M., Battaglia, S., George, S., Chatta, G. S., Puzanov, I. J., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21619-0 · Cited by 174
Diagnostic accuracy of the GDx VCC for glaucoma
Reus, N. J., & Lemij, H. G. (2004) · Ophthalmology · doi:10.1016/j.ophtha.2004.04.024 · Cited by 168
Point-of-Care Screening for Heart Failure with Reduced Ejection Fraction Using Artificial Intelligence during ECG-Enabled Stethoscope Examination in London, UK: A Prospective, Observational, Multicentre Study
Bachtiger, P., Petri, C. F., Scott, F. E., Ri Park, S., Kelshiker, M. A., Sahemey, H. K., Dumea, B., Alquero, R., Padam, P. S., Hatrick, I. R., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00256-9 · Cited by 167

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks