Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 301–400 of 1,197

Detection of High-grade Prostate Cancer Using a Urinary Molecular Biomarker–Based Risk Score
Neste, L. V., Hendriks, R. J., Dijkstra, S., Trooskens, G., Cornel, E. B., Jannink, S. A., de Jong, H., Hessels, D., Smit, F. P., Melchers, W. J. G., et al. (2016) · European Urology · doi:10.1016/j.eururo.2016.04.012 · Cited by 374
Amino-Terminal Pro-B-Type Natriuretic Peptide and B-Type Natriuretic Peptide in the General Community
Costello‐Boerrigter, L. C., Boerrigter, G., Redfield, M. M., Rodeheffer, R. J., Urban, L. H., Mahoney, D. W., Jacobsen, S. J., Heublein, D. M., & Burnett, J. C. J. (2006) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2005.09.025 · Cited by 371
Identifying co-morbidity in surgical patients using administrative data with the Royal College of Surgeons Charlson Score
Armitage, J. N., & van der Meulen, J. (2010) · British journal of surgery · doi:10.1002/bjs.6930 · Cited by 370
Urinary Biomarkers in the Clinical Prognosis and Early Detection of Acute Kidney Injury
Koyner, J. L., Vaidya, V. S., BENNETT, M. R., Ma, Q., Worcester, E. M., Akhter, S. A., Raman, J., Jeevanandam, V., O’Connor, M. F., Devarajan, P., et al. (2010) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.00740110 · Cited by 370
Early detection of type 2 diabetes mellitus using machine learning-based prediction models
Kopitar, L., Kocbek, P., Cilar, L., Sheikh, A., & Stiglic, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-68771-z · Cited by 369
Diastolic Pulmonary Vascular Pressure Gradient
Gerges, C., Gerges, M., Lang, M., Zhang, Y., Jakowitsch, J., Probst, P., Maurer, G., & Lang, I. M. (2013) · CHEST Journal · doi:10.1378/chest.12-1653 · Cited by 369
Detecting early glaucoma by assessment of retinal nerve fiber layer thickness and visual function.
Bowd, C., Zangwill, L. M., Berry, C. C., Blumenthal, E. Z., Vasile, C., Sánchez-Galeana, C. A., Bosworth, C. F., Sample, P. A., & Weinreb, R. N. (2001) · PubMed · Cited by 369
Use of a Self-Report-Generated Charlson Comorbidity Index for Predicting Mortality
Chaudhry, S. I., Jin, L., & Meltzer, D. (2005) · Medical Care · doi:10.1097/01.mlr.0000163658.65008.ec · Cited by 368
Systemic Complement Activation in Age-Related Macular Degeneration
Scholl, H. P. N., Issa, P. C., Walier, M., Janzer, S., Pollok‐Kopp, B., Börncke, F., Fritsche, L. G., Chong, V. N., Fimmers, R., Wienker, T. F., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0002593 · Cited by 374
The diagnostic value of biomarkers (SteatoTest) for the prediction of liver steatosis
Poynard, T., Ratziu, V., Naveau, S., Thabut, D., Charlotte, F., Messous, D., Capron, D., Abella, A., Massard, J., Ngo, Y., et al. (2005) · Comparative Hepatology · doi:10.1186/1476-5926-4-10 · Cited by 375
Glaucoma risk index:Automated glaucoma detection from color fundus images
Bock, R., Meier, J., Nyúl, L. G., Joachim Hornegger, A., & Michelson, G. (2010) · Medical Image Analysis · doi:10.1016/j.media.2009.12.006 · Cited by 360
Multicenter Evaluation of a 0-Hour/1-Hour Algorithm in the Diagnosis of Myocardial Infarction With High-Sensitivity Cardiac Troponin T
Mueller, C. E., Giannitsis, E., Christ, M., Ordóñez‐Llanos, J., deFilippi, C. R., McCord, J., Body, R. M., Panteghini, M., Jernberg, T., Plebani, M., et al. (2016) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2015.11.013 · Cited by 360
Deep Learning for Prediction of Obstructive Disease From Fast Myocardial Perfusion SPECT
Betancur, J., Commandeur, F., Motlagh, M. E., Sharir, T., Einstein, A. J., Bokhari, S., Fish, M. B., Ruddy, T. D., Kaufmann, P. A., Sinusas, A. J., et al. (2018) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2018.01.020 · Cited by 363
ROC Curves in Clinical Chemistry: Uses, Misuses, and Possible Solutions
Obuchowski, N. A., Lieber, M. L., & Wians, F. H. (2004) · Clinical Chemistry · doi:10.1373/clinchem.2004.031823 · Cited by 358
Mapping of Macular Substructures with Optical Coherence Tomography for Glaucoma Diagnosis
Tan, O., Li, G., Lu, A. T.-H., Varma, R., & Huang, D. (2007) · Ophthalmology · doi:10.1016/j.ophtha.2007.08.011 · Cited by 358
Cerebrospinal Fluid Tau and β-Amyloid: How Well Do These Biomarkers Reflect Autopsy-Confirmed Dementia Diagnoses?
Clark, C. M., Xie, S., Chittams, J., Ewbank, D., Peskind, E., Galasko, D., Morris, J. C., McKeel, D. W., Farlow, M., Weitlauf, S. L., et al. (2003) · Archives of Neurology · doi:10.1001/archneur.60.12.1696 · Cited by 357
What is the best predictor of future type 2 diabetes
Abdul-Ghani, M. A., Williams, K., DeFronzo, R. A., & Stern, M. (2007) · Diabetes Care · doi:10.2337/dc06-1331 · Cited by 355
Microbiota-based model improves the sensitivity of fecal immunochemical test for detecting colonic lesions
Baxter, N. T., Ruffin, M. T., Rogers, M. A. M., & Schloss, P. D. (2016) · Genome Medicine · doi:10.1186/s13073-016-0290-3 · Cited by 354
A Family of Nonparametric Statistics for Comparing Diagnostic Markers with Paired or Unpaired Data
Wieand, S., Gail, M. H., James, B. R., & James, K. L. (1989) · Biometrika · doi:10.1093/biomet/76.3.585 · Cited by 354
Deep Learning in Mammography: Diagnostic Accuracy of a Multipurpose Image Analysis Software in the Detection of Breast Cancer
Becker, A. S., Marcon, M., Ghafoor, S., Wurnig, M. C., Frauenfelder, T., & Boss, A. (2017) · Investigative Radiology · doi:10.1097/rli.0000000000000358 · Cited by 352
Noninvasive elastography-based assessment of liver fibrosis progression and prognosis in primary biliary cirrhosis
Corpechot, C., Carrat, F., Poujol-Robert, A., Gaouar, F., Wendum, D., Chazouillères, O., & Poupon, R. (2012) · Hepatology · doi:10.1002/hep.25599 · Cited by 351
HE4 and CA125 as a diagnostic test in ovarian cancer: prospective validation of the Risk of Ovarian Malignancy Algorithm
Gorp, T. V., Cadron, I., Despierre, E., Daemen, A., Leunen, K., Amant, F. C., Timmerman, D., Moor, B. D., & Vergote, I. B. (2011) · British Journal of Cancer · doi:10.1038/sj.bjc.6606092 · Cited by 351
Serum Inhibin B as a Marker of Spermatogenesis
Pierik, F. H., Vreeburg, J. T. M., Stijnen, T., de Jong, F. H., & Weber, R. F. A. (1998) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jcem.83.9.5121 · Cited by 351
Habitual sleep duration and insomnia and the risk of cardiovascular events and all-cause death: report from a community-based cohort
Chien, K.-L., Chen, P.-C., Hsu, H.-C., Su, T.-C., Sung, F.-C., Chen, M.-F., & Lee, Y.-T. (2010) · SLEEP · doi:10.1093/sleep/33.2.177 · Cited by 350
Quantitative scoring of differential drug sensitivity for individually optimized anticancer therapies
Yadav, B., Pemovska, T., Szwajda, A., Kulesskiy, E., Kontro, M., Karjalainen, R., Majumder, M. M., Malani, D., Murumägi, A., Knowles, J., et al. (2014) · Scientific Reports · doi:10.1038/srep05193 · Cited by 358
Urine TMPRSS2:ERG Fusion Transcript Stratifies Prostate Cancer Risk in Men with Elevated Serum PSA
Tomlins, S. A., Aubin, S. M. J., Siddiqui, J., Lonigro, R. J., Sefton-Miller, L., Miick, S., Williamsen, S., Hodge, P., Meinke, J., Blase, A. B., et al. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3001970 · Cited by 350
Development and validation of the Osteoporosis Risk Assessment Instrument to facilitate selection of women for bone densitometry.
Cadarette, S. M., Jaglal, S. B., Kreiger, N., McIsaac, W. J., Darlington, G., & Tu, J. V. (2000) · Europe PMC (PubMed Central) · Cited by 350
Optical Coherence Tomography to Optimize Results of Percutaneous Coronary Intervention in Patients with Non–ST-Elevation Acute Coronary Syndrome
Meneveau, N. F., Souteyrand, G., Motreff, P., Caussin, C., Amabile, N., Ohlmann, P., Morel, O., Lefrançois, Y., Descotes‐Genon, V., Silvain, J., et al. (2016) · Circulation · doi:10.1161/circulationaha.116.024393 · Cited by 349
ROC and AUC with a Binary Predictor: a Potentially Misleading Metric
Muschelli, J. (2019) · Journal of Classification · doi:10.1007/s00357-019-09345-1 · Cited by 349
The influence of non-steroidal anti-inflammatory drugs on the gut microbiome
Rogers, M. A. M., & Aronoff, D. M. (2016) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2015.10.003 · Cited by 346
Novel Pachymetric Parameters Based on Corneal Tomography for Diagnosing Keratoconus
Ambrósio, R., Canedo, A. L., Guerra, F., Louzada, R. N., Roy, A. S., da Luz, A. C., Dupps, W. J., & Belin, M. W. (2011) · Journal of Refractive Surgery · doi:10.3928/1081597x-20110721-01 · Cited by 346
A clinical prediction rule for ambulation outcomes after traumatic spinal cord injury: a longitudinal cohort study
van Middendorp, J. J., Hosman, A. J. F., Donders, A. R. T., Pouw, M. H., Ditunno, J. F., Curt, A. E. P., Geurts, A. C. H., & van de Meent, H. (2011) · The Lancet · doi:10.1016/s0140-6736(10)62276-3 · Cited by 351
Controlled Attenuation Parameter (CAP): a noninvasive method for the detection of hepatic steatosis based on transient elastography
Myers, R. P., Pollett, A. F., Kirsch, R., Pomier–Layrargues, G., Beaton, M. D., Levstik, M. A., Duarte‐Rojo, A., Wong, D., Crotty, P., & Elkashab, M. (2012) · Liver International · doi:10.1111/j.1478-3231.2012.02781.x · Cited by 350
Discriminating Between Normal and Glaucomatous Eyes Using the Heidelberg Retina Tomograph, GDx Nerve Fiber Analyzer, and Optical Coherence Tomograph
Zangwill, L. M. (2001) · Archives of Ophthalmology · doi:10.1001/archopht.119.7.985 · Cited by 344
A novel faecal Lachnoclostridium marker for the non-invasive diagnosis of colorectal adenoma and cancer
Liang, Q., Li, T., Nakatsu, G., Chen, Y., Yau, T. O., Chu, E. S., Wong, S. H., Szeto, C. H., Ng, S. C., Chan, F. K. L., et al. (2019) · Gut · doi:10.1136/gutjnl-2019-318532 · Cited by 349
A review in radiomics: Making personalized medicine a reality via routine imaging
Guiot, J., Vaidyanathan, A., Deprez, L., Zerka, F., Danthine, D., Frix, A.-N., Lambin, P., Bottari, F., Tsoutzidis, N., Miraglio, B., et al. (2021) · Medicinal Research Reviews · doi:10.1002/med.21846 · Cited by 356
Radimagenet: an open Radiologic deep learning research Dataset for effective transfer learning
Mei, X., Liu, Z., Robson, P. M., Marinelli, B., Huang, M., Doshi, A., Jacobi, A., Cao, C., Link, K. E., Yang, T., et al. (2022) · Radiology Artificial Intelligence · doi:10.1148/ryai.210315 · Cited by 341
Performance of Pharmacovigilance Signal-Detection Algorithms for the FDA Adverse Event Reporting System
Harpaz, R., DuMouchel, W., LePendu, P., Bauer‐Mehren, A., Ryan, P. B., & Shah, N. H. (2013) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2013.24 · Cited by 341
External Validation of the Fatty Liver Index for Identifying Nonalcoholic Fatty Liver Disease in a Population-based Study
Koehler, E. M., Schouten, J. N. L., Hansen, B. E., Albert Hofman, D., Stricker, B. H. C., & Janssen, H. L. A. (2013) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2012.12.031 · Cited by 340
MICE Models: Superior to the HERG Model in Predicting Torsade de Pointes
Kramer, J., Obejero‐Paz, C. A., Myatt, G. J., Kuryshev, Y. A., Bruening‐Wright, A., Verducci, J. S., & Brown, A. M. (2013) · Scientific Reports · doi:10.1038/srep02100 · Cited by 340
Partial AUC Estimation and Regression
Dodd, L. E., & PEPE, M. S. (2003) · Biometrics · doi:10.1111/1541-0420.00071 · Cited by 339
Urinary clusterin, cystatin C, β2-microglobulin and total protein as markers to detect drug-induced kidney injury
Dieterle, F., Perentes, E., Cordier, A., Roth, D. R., Verdes, P. S., Grenet, O., Pantano, S., Moulin, P., Wahl, D. R., Mahl, A., et al. (2010) · Nature Biotechnology · doi:10.1038/nbt.1622 · Cited by 338
Effectiveness of the AUDIT-C as a Screening Test for Alcohol Misuse in Three Race/Ethnic Groups
Frank, D., DeBenedetti, A. F., Volk, R. J., Williams, E. C., Kivlahan, D. R., & Bradley, K. A. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-008-0594-0 · Cited by 336
Statistical Comparison of Two ROC-curve Estimates Obtained from Partially-paired Datasets
Metz, C. E., Herman, B. A., & Roe, C. A. (1998) · Medical Decision Making · doi:10.1177/0272989x9801800118 · Cited by 336
Temporal Relationship and Predictive Value of Urinary Acute Kidney Injury Biomarkers After Pediatric Cardiopulmonary Bypass
Krawczeski, C. D., Goldstein, S. L., Woo, J. G., Wang, Y., Piyaphanee, N., Ma, Q., BENNETT, M. R., & Devarajan, P. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.08.017 · Cited by 336
Evaluation of the glaucomatous damage on retinal nerve fiber layer thickness measured by optical coherence tomography
Kanamori, A., Nakamura, M., Escaño, M. F., Seya, R., Maeda, H., & Negi, A. (2003) · American Journal of Ophthalmology · doi:10.1016/s0002-9394(02)02003-2 · Cited by 336
Vulnerability of South African animal taxa to climate change
Erasmus, B. F. N., van Jaarsveld, A. S., Chown, S. L., Kshatriya, M., & Wessels, K. (2002) · Global Change Biology · doi:10.1046/j.1365-2486.2002.00502.x · Cited by 336
Diagnostic usefulness of bone mineral density and biochemical markers of bone turnover in predicting fracture in CKD stage 5D patients--a single-center cohort study
Iimori, S., Mori, Y., Akita, W., Kuyama, T., Takada, S., Asai, T., Kuwahara, M., Sasaki, S., & Tsukamoto, Y. (2011) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfr317 · Cited by 336
Copeptin Improves Early Diagnosis of Acute Myocardial Infarction
Keller, T., Tzikas, S., Zeller, T., Czyz, E., Lillpopp, L., Ojeda, F., Roth, A., Bickel, C., Baldus, S. E., Sinning, C., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.01.029 · Cited by 334
Measurement of tubular enzymuria facilitates early detection of acute renal impairment in the intensive care unit
Westhuyzen, J. (2003) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/18.3.543 · Cited by 334
Fecal bacteria act as novel biomarkers for noninvasive diagnosis of colorectal cancer
Liang, Q., Chiu, J., Chen, Y., Huang, Y., Higashimori, A., Fang, J., Brim, H., Ashktorab, H., Ng, S. C., Ng, S. S. M., et al. (2017) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-16-1599 · Cited by 332
The impact of site-specific digital histology signatures on deep learning model accuracy and bias
Howard, F. M., Dolezal, J., Kochanny, S., Schulte, J., Chen, H., Heij, L., Huo, D., Nanda, R., Olopade, O. I., Kather, J. N., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-24698-1 · Cited by 330
Validation of the 10-item Centre for Epidemiological Studies Depression Scale (CES-D-10) in Zulu, Xhosa and Afrikaans populations in South Africa
Garman, E. C., Davies, T., & Lund, C. (2017) · BMC Psychiatry · doi:10.1186/s12888-016-1178-x · Cited by 330
Quantitative Assessment of Myocardial Perfusion in the Detection of Significant Coronary Artery Disease
Danad, I., Uusitalo, V., Kero, T., Saraste, A., Raijmakers, P. G. H. M., Lammertsma, A. A., Heymans, M. W., Kajander, S. A., Pietilä, M., James, S. K., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2014.05.069 · Cited by 338
Fast, scalable prediction of deleterious noncoding variants from functional and population genomic data
Huang, Y.-F., Gulko, B., & Siepel, A. C. (2017) · Nature Genetics · doi:10.1038/ng.3810 · Cited by 337
The bootstrap: A technique for data-driven statistics. Using computer-intensive analyses to explore experimental data
Henderson, A. R. (2005) · Clinica Chimica Acta · doi:10.1016/j.cccn.2005.04.002 · Cited by 328
Receiver operating characteristic (ROC) analysis: Basic principles and applications in radiology
van Erkel, A. R., & Pattynama, P. M. T. (1998) · European Journal of Radiology · doi:10.1016/s0720-048x(97)00157-5 · Cited by 327
Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Tschandl, P., Rosendahl, C., Akay, B., Argenziano, G., Blum, A., Braun, R., Cabo, H., Gourhant, J., Kreusch, J., Lallas, A., et al. (2019) · JAMA dermatology · doi:10.1001/jamadermatol.2018.4378 · Cited by 313
Association of Peritumoral Radiomics With Tumor Biology and Pathologic Response to Preoperative Targeted Therapy for HER2 (ERBB2) –Positive Breast Cancer
Braman, N. M., Prasanna, P., Whitney, J. R., Singh, S., Beig, N. G., Etesami, M., Bates, ‬. D. B., Gallagher, K. M., Bloch, B., Vulchi, M., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.2561 · Cited by 327
Predicting hospital admission at emergency department triage using machine learning
Hong, W. S., Haimovich, A. D., & Taylor, R. A. (2018) · PLOS ONE · doi:10.1371/journal.pone.0201016 · Cited by 326
Yellow Fever in Africa: Estimating the Burden of Disease and Impact of Mass Vaccination from Outbreak and Serological Data
Garske, T., Kerkhove, M. D. V., Yactayo, S., Ronveaux, O., Lewis, R. F., Staples, J. E., Perea, W., & Ferguson, N. M. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001638 · Cited by 326
Gradient modeling of conifer species using random forests
Evans, J. S., & Cushman, S. A. (2009) · Landscape Ecology · doi:10.1007/s10980-009-9341-0 · Cited by 342
Quantitative Methods for Modeling Species Habitat: Comparative Performance and an Application to Australian Plants
Elith, J. (2006) · Springer eBooks · doi:10.1007/0-387-22648-6_4 · Cited by 326
Role of the GALAD and BALAD-2 Serologic Models in Diagnosis of Hepatocellular Carcinoma and Prediction of Survival in Patients
Berhane, S., Toyoda, H., Tada, T., Kumada, T., Kagebayashi, C., Satomura, S., Schweitzer, N., Vogel, A., Manns, M. P., Benckert, J., et al. (2016) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2015.12.042 · Cited by 324
The development of a machine learning inpatient acute kidney injury prediction model.
Koyner, J. L., Carey, K. A., Edelson, D. P., & Churpek, M. M. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003123 · Cited by 323
A checklist for maximizing reproducibility of ecological niche models
Feng, X., Park, D. S., Walker, C., Peterson, A. T., Merow, C., & Papeş, M. (2019) · Nature Ecology & Evolution · doi:10.1038/s41559-019-0972-5 · Cited by 323
Left Atrial Strain and Compliance in the Diagnostic Evaluation of Heart Failure with Preserved Ejection Fraction
Reddy, Y. N. V., Obokata, M., Egbe, A. C., Yang, J. H., Pislaru, S. V., Lin, G., Carter, R. E., & Borlaug, B. A. (2019) · European Journal of Heart Failure · doi:10.1002/ejhf.1464 · Cited by 323
Validation techniques for logistic regression models
Miller, M. E., Hui, S. L., & Tierney, W. M. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100805 · Cited by 322
Untitled work
Obuchowski (2005) · doi:10.2214/ajr.184.2.01840364 · Cited by 321
Serial Measurement of Growth-Differentiation Factor-15 in Heart Failure
Anand, I. S., Kempf, T., Rector, T. S., Tapken, H., Allhoff, T., Jantzen, F., Kuskowski, M. A., Cohn, J. N., Drexler, H., & Wollert, K. C. (2010) · Circulation · doi:10.1161/circulationaha.109.928846 · Cited by 321
A comparison of different diagnostic criteria of acute kidney injury in critically ill patients
Luo, X.-Y., Jiang, L., Du, B., Wen, Y., Wang, M., Xi, X., & workgroup, T. B. A. K. I. T. (. (2014) · Critical Care · doi:10.1186/cc13977 · Cited by 321
Effect of Finasteride on the Sensitivity of PSA for Detecting Prostate Cancer
Thompson, I. M., Chen, C., Ankerst, D. P., Goodman, P. J., Tangen, C. M., Lippman, S. M., Lucia, M. S., Parnes, H. L., & Coltman, C. A. (2006) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djj307 · Cited by 319
Attributable risk of Alzheimer's dementia attributed to age-related neuropathologies
Boyle, P. A., Yu, L., Leurgans, S. E., Wilson, R. S., Brookmeyer, R., Schneider, J. A., & Bennett, D. A. (2019) · Annals of Neurology · doi:10.1002/ana.25380 · Cited by 318
Accuracy of cone-beam computed tomography imaging of the temporomandibular joint: Comparisons with panoramic radiology and linear tomography
Honey, O. B., Scarfe, W. C., Hilgers, M. J., Klueber, K. M., Silveira, A. M., Haskell, B. S., & Farman, A. G. (2007) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/j.ajodo.2005.10.032 · Cited by 318
The NoSAS score for screening of sleep-disordered breathing: a derivation and validation study
Marti-Soler, H., Hirotsu, C., Marques-Vidal, P., Vollenweider, P., Waeber, G., Preisig, M., Tafti, M., Tufik, S. B., Bittencourt, L., Tufik, S., et al. (2016) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(16)30075-3 · Cited by 317
Vasoactive-inotropic score and the prediction of morbidity and mortality after cardiac surgery
Koponen, T., Karttunen, J., Musialowicz, T., Pietiläinen, L., Uusaro, A. V., & Lahtinen, P. (2019) · British Journal of Anaesthesia · doi:10.1016/j.bja.2018.12.019 · Cited by 317
Predicting Outcome After Acute and Subacute Stroke
Counsell, C. E., Dennis, M. S., McDowall, M., & Warlow, C. P. (2002) · Stroke · doi:10.1161/hs0402.105909 · Cited by 319
Predicting Survival in Patients Receiving Continuous Flow Left Ventricular Assist Devices
Cowger, J. A., Sundareswaran, K. S., Rogers, J. G., Park, S. J., Pagani, F. D., Bhat, G., Jaski, B. E., Farrar, D. J., & Slaughter, M. S. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.09.055 · Cited by 316
Complementary Roles for Biomarkers of Biomechanical Strain ST2 and N-Terminal Prohormone B-Type Natriuretic Peptide in Patients With ST-Elevation Myocardial Infarction
Sabatine, M. S., Morrow, D. A., Higgins, L., MacGillivray, C., Guo, W., Bode, C., Rifai, N., Cannon, C. P., Gerszten, R. E., & Lee, R. T. (2008) · Circulation · doi:10.1161/circulationaha.107.728022 · Cited by 315
Quantitative DNA Methylation Analysis of FOXP3 as a New Method for Counting Regulatory T Cells in Peripheral Blood and Solid Tissue
Wieczorek, G., Asemissen, A. M., Model, F., Türbachova, I., Floess, S., Liebenberg, V., Baron, U., Stauch, D., Kotsch, K., Pratschke, J., et al. (2009) · Cancer Research · doi:10.1158/0008-5472.can-08-2361 · Cited by 321
Gliomas: Histogram Analysis of Apparent Diffusion Coefficient Maps with Standard- or High- b -Value Diffusion-weighted MR Imaging—Correlation with Tumor Grade
Kang, Y., Choi, S. H., Kim, Y., Kim, K. G., Sohn, C., Kim, J., Yun, T. J., & Chang, K. H. (2011) · Radiology · doi:10.1148/radiol.11110686 · Cited by 314
Prognosis of Negative Adenosine Stress Magnetic Resonance in Patients Presenting to an Emergency Department With Chest Pain
Ingkanisorn, W. P., Kwong, R. Y. K., Bohme, N. S., Geller, N. L., Rhoads, K. L., Dyke, C. K., Paterson, D. I., Syed, M., Aletras, A. H., & Arai, A. E. (2006) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2005.11.059 · Cited by 313
Misuse of DeLong test to compare AUCs for nested models
Demler, O., Pencina, M. J., & D’Agostino, R. B. (2012) · Statistics in Medicine · doi:10.1002/sim.5328 · Cited by 313
Next-Generation Stool DNA Test Accurately Detects Colorectal Cancer and Large Adenomas
Ahlquist, D. A., Zou, H., Domanico, M., Mahoney, D. W., Yab, T. C., Taylor, W. R., Butz, M. L., Thibodeau, S. N., Rabeneck, L., Paszat, L. F., et al. (2012) · Gastroenterology · doi:10.1053/j.gastro.2011.10.031 · Cited by 315
Predicting customer retention and profitability by using random forests and regression forests techniques
Larivière, B., & den Poel, D. V. (2005) · Expert Systems with Applications · doi:10.1016/j.eswa.2005.04.043 · Cited by 313
Comparison of Echocardiographic Indices Used to Predict Fluid Responsiveness in Ventilated Patients
Vignon, P., Repessé, X., Bégot, E., Léger, J., Jacob, C., Bouferrache, K., Slama, M., Prat, G., & Vieillard‐Baron, A. (2016) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201604-0844oc · Cited by 322
A machine learning approach to radiogenomics of breast cancer: a study of 922 subjects and 529 DCE-MRI features
Saha, A., Harowicz, M. R., Grimm, L. J., Kim, C. E., Ghate, S. V., Walsh, R., & Mazurowski, M. A. (2018) · British Journal of Cancer · doi:10.1038/s41416-018-0185-8 · Cited by 313
The Effect of Image Resolution on Deep Learning in Radiography
Sabottke, C. F., & Spieler, B. M. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2019190015 · Cited by 312
Meta-analyses of FibroTest diagnostic value in chronic liver disease
Poynard, T., Morra, R., Halfon, P., Castéra, L., Ratziu, V., Imbert‐Bismut, F., Naveau, S., Thabut, D., Lebrec, D., Zoulim, F., et al. (2007) · BMC Gastroenterology · doi:10.1186/1471-230x-7-40 · Cited by 312
A strategy for predicting the chemosensitivity of human cancers and its application to drug discovery
Lee, J. K., Havaleshko, D. M., Cho, H., Weinstein, J. N., Kaldjian, E. P., Karpovich, J. F., Grimshaw, A. S., & Theodorescu, D. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610292104 · Cited by 311
Selecting from correlated climate variables: a major source of uncertainty for predicting species distributions under climate change
Braunisch, V., Coppes, J., Arlettaz, R., Suchant, R., Schmid, H., & Bollmann, K. (2013) · Ecography · doi:10.1111/j.1600-0587.2013.00138.x · Cited by 332
Plasma Fetuin-A Levels and the Risk of Myocardial Infarction and Ischemic Stroke
Weikert, C., Stefan, N., Schulze, M. B., Pischon, T., Berger, K., Joost, H., Häring, H., Boeing, H., & Fritsche, A. (2008) · Circulation · doi:10.1161/circulationaha.108.814418 · Cited by 315
Echo Parameters for Differential Diagnosis in Cardiac Amyloidosis
Pagourelias, E. D., Mirea, O., Duchenne, J., Cleemput, J. V., Delforge, M., Bogaert, J. G., Kuznetsova, T., & Voigt, J. (2017) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.116.005588 · Cited by 310
Estimation of time‐dependent area under the ROC curve for long‐term risk prediction
Chambless, L. E., & Diao, G. (2005) · Statistics in Medicine · doi:10.1002/sim.2299 · Cited by 320
Comparative analysis of multiple-casualty incident triage algorithms
Garner, A. A., Lee, A. S. Y., Harrison, K., & Schultz, C. H. (2001) · Annals of Emergency Medicine · doi:10.1067/mem.2001.119053 · Cited by 309
Microbiome Data Distinguish Patients with Clostridium difficile Infection and Non-C. difficile-Associated Diarrhea from Healthy Controls
Schubert, A. M., Rogers, M. A. M., Ring, C., Mogle, J. A., Petrosino, J., Young, V. B., Aronoff, D. M., & Schloss, P. D. (2014) · mBio · doi:10.1128/mbio.01021-14 · Cited by 308
Predicting Live Birth, Preterm Delivery, and Low Birth Weight in Infants Born from In Vitro Fertilisation: A Prospective Study of 144,018 Treatment Cycles
Nelson, S. M., & Lawlor, D. A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000386 · Cited by 308
Bioprosthetic Valve Thrombosis Versus Structural Failure
Egbe, A. C., Pislaru, S. V., Pellikka, P. A., Poterucha, J. T., Schaff, H. V., Maleszewski, J. J., & Connolly, H. M. (2015) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.09.022 · Cited by 308
Ovarian-Adnexal Reporting Data System Magnetic Resonance Imaging (O-RADS MRI) Score for Risk Stratification of Sonographically Indeterminate Adnexal Masses
Thomassin‐Naggara, I., Poncelet, É., Jalaguier-Coudray, A., Guerra, A., Fournier, L. S., Stojanović, S., Millet, I., Bharwani, N., Juhan, V., Cunha, T. M., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.19896 · Cited by 308
Clinical and Laboratory Predictors of In-Hospital Mortality in 305 Patients with COVID-19: A Cohort Study in Wuhan, China
Wang, K., Zuo, P., Liu, Y., Zhang, M., Zhao, X., Xie, S., Zhang, H., Chen, X., & Liu, C. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa538 · Cited by 308

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks