Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 201–300 of 1,197

Body mass index, waist circumference and waist–hip ratio: which is the better discriminator of cardiovascular disease mortality risk? Evidence from an individual‐participant meta‐analysis of 82 864 participants from nine cohort studies
Czernichow, S., Kengne, A. P., Stamatakis, E., Hamer, M., & Batty, G. D. (2011) · Obesity Reviews · doi:10.1111/j.1467-789x.2011.00879.x · Cited by 470
Association of a Peripheral Blood Metabolic Profile With Coronary Artery Disease and Risk of Subsequent Cardiovascular Events
Shah, S. H., Bain, J. R., Muehlbauer, M. J., Stevens, R. D., Crosslin, D. R., Haynes, C. S., Dungan, J., Newby, L. K., Hauser, E. R., Ginsburg, G. S., et al. (2010) · Circulation Cardiovascular Genetics · doi:10.1161/circgenetics.109.852814 · Cited by 468
Diagnostic Accuracy of Digital versus Film Mammography: Exploratory Analysis of Selected Population Subgroups in DMIST
Pisano, E. D., Hendrick, R. E., Yaffe, M. J., Baum, J. K., Acharyya, S., Cormack, J. B., Hanna, L. A., Conant, E. F., Fajardo, L. L., Bassett, L. W., et al. (2008) · Radiology · doi:10.1148/radiol.2461070200 · Cited by 464
A Prospective Evaluation of the Clinical Utility of the Lower-Extremity Injury-Severity Scores
Bosse, M. J., MacKenzie, E. J., Kellam, J. F., Burgess, A. R., Webb, L. X., Swiontkowski, M. F., SANDERS, R. W., Jones, A. L., McAndrew, M. P., Patterson, B. M., et al. (2001) · Journal of Bone and Joint Surgery · doi:10.2106/00004623-200101000-00002 · Cited by 464
Comparison of triglyceride glucose index, and related parameters to predict insulin resistance in Korean adults: An analysis of the 2007-2010 Korean National Health and Nutrition Examination Survey
Lim, J., Kim, J., Koo, S. H., & Kwon, G. C. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212963 · Cited by 463
New insights into churn prediction in the telecommunication sector: a profit driven data mining approach
Verbeke, W., Dejaeger, K., Martens, D., Hur, J., & Baesens, B. (2012) · European Journal of Operational Research · doi:10.1016/j.ejor.2011.09.031 · Cited by 462
Utility of Absolute and Relative Changes in Cardiac Troponin Concentrations in the Early Diagnosis of Acute Myocardial Infarction
Reichlin, T., Irfan, A. B., Twerenbold, R., Reiter, M., Hochholzer, W., Burkhalter, H., Bassetti, S., Steuer, S., Winkler, K., Peter, F., et al. (2011) · Circulation · doi:10.1161/circulationaha.111.023937 · Cited by 459
Biologically informed deep neural network for prostate cancer discovery
Elmarakeby, H. A., Hwang, J., Arafeh, R., Crowdis, J., Gang, S., Liu, D., AlDubayan, S. H., Salari, K., Kregel, S., Richter, C., et al. (2021) · Nature · doi:10.1038/s41586-021-03922-4 · Cited by 456
Well-aerated Lung on Admitting Chest CT to Predict Adverse Outcome in COVID-19 Pneumonia
Colombi, D., Bodini, F. C., Petrini, M., Maffi, G., Morelli, N., Milanese, G., Silva, M., Sverzellati, N., & Michieletti, E. (2020) · Radiology · doi:10.1148/radiol.2020201433 · Cited by 456
High-resolution magnetic resonance imaging reveals nuclei of the human amygdala: manual segmentation to automatic atlas
Saygin, Z. M., Kliemann, D., Iglesias, J. E., van der Kouwe, A. J. W., Boyd, E. C., Reuter, M., Stevens, A. A., Leemput, K. V., McKee, A. C., Frosch, M. P., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.046 · Cited by 469
Measuring and comparing the accuracy of species distribution models with presence–absence data
Liu, C.-R., White, M., & Newell, G. (2011) · Ecography · doi:10.1111/j.1600-0587.2010.06354.x · Cited by 454
Non-contrast T1-mapping detects acute myocardial edema with high diagnostic accuracy: a comparison to T2-weighted cardiovascular magnetic resonance
Ferreira, V. M., Piechnik, S. K., Dall’Armellina, E., Karamitsos, T. D., Francis, J. M., Choudhury, R. P., Friedrich, M. G., Robson, M. D., & Neubauer, S. (2012) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-14-42 · Cited by 454
Comparison of the GDx VCC Scanning Laser Polarimeter, HRT II ConfocalScanning Laser Ophthalmoscope, and Stratus OCT Optical Coherence Tomographfor the Detection of Glaucoma
Medeiros, F. A. (2004) · Archives of Ophthalmology · doi:10.1001/archopht.122.6.827 · Cited by 453
Comparison of Coronary Computed Tomography Angiography, Fractional Flow Reserve, and Perfusion Imaging for Ischemia Diagnosis
Driessen, R. S., Danad, I., Stuijfzand, W. J., Raijmakers, P. G. H. M., Schumacher, S. P., van Diemen, P. A., Leipsic, J. A., Knuuti, J. M., Underwood, S. R., van de Ven, P. M., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.10.056 · Cited by 456
The Predictive Value of Different Measures of Obesity for Incident Cardiovascular Events and Mortality
Schneider, H. J., Friedrich, N., Klotsche, J., Pieper, L., Nauck, M. A., John, U., Dörr, M., Felix, S. B., Lehnert, H., Pittrow, D. B., et al. (2010) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2009-1584 · Cited by 451
PCA3: A Molecular Urine Assay for Predicting Prostate Biopsy Outcome
Deras, I. L., Aubin, S. M. J., Blase, A. B., Day, J. R., Koo, S., Partin, A. W., Ellis, W. J., Marks, L. S., Fradet, Y., Rittenhouse, H. G., et al. (2008) · The Journal of Urology · doi:10.1016/j.juro.2007.11.038 · Cited by 450
The C-Reactive Protein/Albumin Ratio, a Novel Inflammation-Based Prognostic Score, Predicts Outcomes in Patients with Hepatocellular Carcinoma
Kinoshita, A., Onoda, H., Imai, N., Iwaku, A., Oishi, M., Tanaka, K., Fushiya, N., Koike, K., Nishino, H., & Matsushima, M. (2014) · Annals of Surgical Oncology · doi:10.1245/s10434-014-4048-0 · Cited by 448
Functional Status Scale: New Pediatric Outcome Measure
Pollack, M. M., Holubkov, R., Glass, P., Dean, J. M., Meert, K. L., Zimmerman, J. J., Anand, K. J. S., Carcillo, J. A., Newth, C. J. L., Harrison, R. E., et al. (2009) · PEDIATRICS · doi:10.1542/peds.2008-1987 · Cited by 445
Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration
Arbabshirani, M. R., Fornwalt, B. K., Mongelluzzo, G. J., Suever, J. D., Geise, B. D., Patel, A. A., & Moore, G. J. (2018) · npj Digital Medicine · doi:10.1038/s41746-017-0015-z · Cited by 443
Common Mental Disorders among HIV-Infected Individuals in South Africa: Prevalence, Predictors, and Validation of Brief Psychiatric Rating Scales
Myer, L., Smit, J., Roux, L. L., Parker, S., Stein, D. J., & Seedat, S. (2008) · AIDS Patient Care and STDs · doi:10.1089/apc.2007.0102 · Cited by 442
C-Reactive Protein Modulates Risk Prediction Based on the Framingham Score
Köenig, W., Löwel, H., Baumert, J. J., & Meisinger, C. (2004) · Circulation · doi:10.1161/01.cir.0000120707.98922.e3 · Cited by 440
Comparison of Reliability, Validity, and Responsiveness of the Mini-BESTest and Berg Balance Scale in Patients With Balance Disorders
Godi, M., Franchignoni, F. P., Caligari, M., Giordano, A., Turcato, A. M., & Nardone, A. (2012) · Physical Therapy · doi:10.2522/ptj.20120171 · Cited by 440
The Revised Child Anxiety and Depression Scale-Short Version: Scale reduction via exploratory bifactor modeling of the broad anxiety factor.
Ebesutani, C., Reise, S. P., Chorpita, B. F., Ale, C., Regan, J., Young, J., Higa-McMillan, C., & Weisz, J. R. (2012) · Psychological Assessment · doi:10.1037/a0027283 · Cited by 439
Early diagnosis of Alzheimer's disease using cortical thickness: impact of cognitive reserve
Querbes, O., Aubry, F., Pariente, J., Lotterie, J.-A., Démonet, J., Duret, V., Puel, M., Berry, I., Fort, J., Celsis, P., et al. (2009) · Brain · doi:10.1093/brain/awp105 · Cited by 467
Development of diagnostic criteria and a prognostic score for hepatitis B virus-related acute-on-chronic liver failure
Wu, T., Li, J., Shao, L., Xin, J., Jiang, L., Zhou, Q., Shi, D., Jiang, J., Sun, S., Jin, L., et al. (2018) · Gut · doi:10.1136/gutjnl-2017-314641 · Cited by 437
Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: international multicentre prospective study
Stanley, A. J., Laine, L., Dalton, H. R., Ngu, J. H., Schultz, M., Abazi, R., Zakko, L., Thornton, S., Wilkinson, K., Khor, C. J. L., et al. (2017) · BMJ · doi:10.1136/bmj.i6432 · Cited by 436
Performance of Fully Automated Plasma Assays as Screening Tests for Alzheimer Disease–Related β-Amyloid Status
Palmqvist, S., Janelidze, S., Stomrud, E., Zetterberg, H., Karl, J., Zink, K., Bittner, T., Mattsson, N., Eichenlaub, U., Blennow, K., et al. (2019) · JAMA Neurology · doi:10.1001/jamaneurol.2019.1632 · Cited by 435
Nonparametric Analysis of Clustered ROC Curve Data
Obuchowski, N. A. (1997) · Biometrics · doi:10.2307/2533958 · Cited by 435
Incremental Value of Copeptin for Rapid Rule Out of Acute Myocardial Infarction
Reichlin, T., Hochholzer, W., Stelzig, C., Laule, K., Freidank, H., Morgenthaler, N. G., Bergmann, A., Potocki, M., Noveanu, M., Breidthardt, T., et al. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.01.076 · Cited by 434
Plasma oxidized low-density lipoprotein, a strong predictor for acute coronary heart disease events in apparently healthy, middle-aged men from the general population
Meisinger, C., Baumert, J., Khuseyinova, N., Loewel, H., & Koenig, W. (2005) · Circulation · doi:10.1161/circulationaha.104.529297 · Cited by 434
Churn prediction in subscription services: An application of support vector machines while comparing two parameter-selection techniques
Coussement, K., & den Poel, D. V. (2006) · Expert Systems with Applications · doi:10.1016/j.eswa.2006.09.038 · Cited by 453
Diagnostic accuracy of FibroScan and comparison to liver fibrosis biomarkers in chronic viral hepatitis: A multicenter prospective study (the FIBROSTIC study)
Degos, F., Perez, P., Roche, B., Mahmoudi, A., Asselineau, J., Voitot, H., & Bédossa, P. (2010) · Journal of Hepatology · doi:10.1016/j.jhep.2010.05.035 · Cited by 429
Glucose variability and mortality in patients with sepsis*
Ali, N. A., O’Brien, J. M., Dungan, K. M., Phillips, G. S., Marsh, C. B., Lemeshow, S. A., Connors, A. F., & Preiser, J. (2008) · Critical Care Medicine · doi:10.1097/ccm.0b013e3181810378 · Cited by 439
An automated method for the determination of the sFlt-1/PIGF ratio in the assessment of preeclampsia
Verlohren, S., Galindo, A., Schlembach, D., Zeisler, H., Herraı̀z, I., Moertl, M. G., Pape, J., Dudenhausen, J. W., Denk, B., & Stepan, H. (2009) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2009.09.016 · Cited by 433
Comparing two correlated C indices with right‐censored survival outcome: a one‐shot nonparametric approach
Kang, L., Chen, W., Petrick, N., & Gallas, B. D. (2014) · Statistics in Medicine · doi:10.1002/sim.6370 · Cited by 430
The 10-Item Positive and Negative Affect Schedule for Children, Child and Parent Shortened Versions: Application of Item Response Theory for More Efficient Assessment
Ebesutani, C. K., Regan, J., Smith, A. M., Reise, S. P., Higa-McMillan, C. K., & Chorpita, B. F. (2012) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-011-9273-2 · Cited by 470
Robust breast cancer detection in mammography and digital breast tomosynthesis using an annotation-efficient deep learning approach
Lotter, W., Diab, A. R., Haslam, B., Kim, J. G., Grisot, G., Wu, E., Wu, K., Onieva, J. O., Boyer, Y., Boxerman, J. L., et al. (2021) · Nature Medicine · doi:10.1038/s41591-020-01174-9 · Cited by 423
Global Registry of Acute Coronary Events (GRACE) hospital discharge risk score accurately predicts long-term mortality post acute coronary syndrome
Tang, E. W., Wong, C., & Herbison, P. (2006) · American Heart Journal · doi:10.1016/j.ahj.2006.10.004 · Cited by 423
The Prognostic and Pathophysiologic Role of Pro‐ and Antiinflammatory Cytokines in Severe Malaria
Day, N., Hien, T. T., Schollaardt, T., Loc, P. P., Chuong, L. V., Chau, T. T. H., Mai, N. T. H., Phu, N. H., Sinh, D. X., White, N. J., et al. (1999) · The Journal of Infectious Diseases · doi:10.1086/315016 · Cited by 422
A large-scale comparison of cortical thickness and volume methods for measuring Alzheimer's disease severity
Schwarz, C. G., Gunter, J. L., Wiste, H. J., Przybelski, S. A., Weigand, S. D., Ward, C. P., Senjem, M. L., Vemuri, P., Murray, M. E., Dickson, D. W., et al. (2016) · NeuroImage Clinical · doi:10.1016/j.nicl.2016.05.017 · Cited by 422
Magnetic Resonance vs Transient Elastography Analysis of Patients With Nonalcoholic Fatty Liver Disease: A Systematic Review and Pooled Analysis of Individual Participants
Hsu, C. L., Caussy, C., Imajo, K., Chen, J., Singh, S., Kaulback, K., Le, M., Hooker, J. C., Tu, X. M., Bettencourt, R., et al. (2018) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2018.05.059 · Cited by 422
Prediction of childhood obesity by infancy weight gain: an individual‐level meta‐analysis
Druet, C., Stettler, N., Sharp, S. J., Simmons, R. K., Cooper, C. C., Smith, G. D., Ekelund, U., Lévy‐Marchal, C., Järvelin, M., Kuh, D. J., et al. (2011) · Paediatric and Perinatal Epidemiology · doi:10.1111/j.1365-3016.2011.01213.x · Cited by 420
Shear Wave Elastography: A New Ultrasound Imaging Mode for the Differential Diagnosis of Benign and Malignant Thyroid Nodules
Sebag, F. A., Vaillant-Lombard, J., Berbis, J., Griset, V., Henry, J.-P., Petit, P., & Oliver, C. (2010) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2010-0766 · Cited by 416
Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients
Lauridsen, H. H., Hartvigsen, J., Manniche, C., Korsholm, L., & Grunnet-Nilsson, N. (2006) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-7-82 · Cited by 425
Magnetic resonance imaging improves 3‐month outcome prediction in mild traumatic brain injury
Yuh, E. L., Mukherjee, P., Lingsma, H. F., Yue, J. K., Ferguson, A. R., Gordon, W. A., Valadka, A. B., Schnyer, D. M., Okonkwo, D. O., Maas, A. I. R., et al. (2012) · Annals of Neurology · doi:10.1002/ana.23783 · Cited by 415
Alternative Fistula Risk Score for Pancreatoduodenectomy (a-FRS)
Mungroop, T. H., van Rijssen, L. B., Klaveren, D. V., Smits, F. J., van Woerden, V., Linnemann, R., Pastena, M. D., Klompmaker, S., Marchegiani, G., Ecker, B. L., et al. (2017) · Annals of Surgery · doi:10.1097/sla.0000000000002620 · Cited by 416
Noninvasive In Vivo Liver Fibrosis Evaluation Using Supersonic Shear Imaging: A Clinical Study on 113 Hepatitis C Virus Patients
Bavu, É., Gennisson, J., Couade, M., Bercoff, J., Mallet, V. O., Fink, M., Badel, A., Vallet‐Pichard, A., Nalpas, B., Tanter, M., et al. (2011) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2011.05.016 · Cited by 422
Lung Ultrasound-Implemented Diagnosis of Acute Decompensated Heart Failure in the ED
Pivetta, E. E., Goffi, A., Lupia, E., Tizzani, M., Porrino, G., Ferreri, E., Volpicelli, G., Balzaretti, P., Banderali, A., Iacobucci, A., et al. (2015) · CHEST Journal · doi:10.1378/chest.14-2608 · Cited by 412
Classifying Esophageal Motility by Pressure Topography Characteristics: A Study of 400 Patients and 75 Controls
Pandolfino, J. E., Ghosh, S. K., Rice, J. P., Clarke, J. O., Kwiatek, M. A., & Kahrilas, P. J. (2008) · The American Journal of Gastroenterology · doi:10.1111/j.1572-0241.2007.01532.x · Cited by 412
Noninvasive Tests for Fibrosis and Liver Stiffness Predict 5-Year Outcomes of Patients With Chronic Hepatitis C
Vergniol, J., Foucher, J., Terrebonne, É., Bernard, P., Bail, B. L., Merrouche, W., Couzigou, P., & de Lédinghen, V. (2011) · Gastroenterology · doi:10.1053/j.gastro.2011.02.058 · Cited by 411
Renal biomarker qualification submission: a dialog between the FDA-EMEA and Predictive Safety Testing Consortium
Dieterle, F., Sistare, F. D., Goodsaid, F. M., Papaluca, M., Ozer, J. S., Webb, C. P., Baer, W., Senagore, A. J., Schipper, M. J., Vonderscher, J., et al. (2010) · Nature Biotechnology · doi:10.1038/nbt.1625 · Cited by 410
Glial and neuronal proteins in serum predict outcome after severe traumatic brain injury
Vos, P. E., Lamers, K. J. B., Hendriks, J. C. M., van Haaren, M., Beems, T., Zimmerman, C., Geel, W. V., de Reus, H., Biert, J., & Verbeek, M. M. (2004) · Neurology · doi:10.1212/01.wnl.0000120550.00643.dc · Cited by 409
Comprehensive Prognosis Assessment by CMR Imaging After ST-Segment Elevation Myocardial Infarction
Eitel, I., de Waha, S., Wöhrle, J., Fuernau, G. F., Lurz, P. C., Pauschinger, M., Desch, S., Schuler, G. C., & Thiele, H. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2014.06.1194 · Cited by 408
A First-Generation Multiplex Biomarker Analysis of Urine for the Early Detection of Prostate Cancer
Laxman, B., Morris, D. S., Yu, J., Siddiqui, J., Cao, J., Mehra, R., Lonigro, R. J., Tsodikov, A., Wei, J. T., Tomlins, S. A., et al. (2008) · Cancer Research · doi:10.1158/0008-5472.can-07-3224 · Cited by 407
Learning When Data Sets are Imbalanced and When Costs are Unequal and Unknown
Maloof, M. A. (2003) · Cited by 406
Quantitative Assessment of Structural Image Quality
Rosen, A. F. G., Roalf, D., Ruparel, K., Blake, J., Seelaus, K., Villa, L., Ciric, R., Cook, P. A., Davatzikos, C., Elliott, M., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.12.059 · Cited by 443
Quantification of Incomplete Revascularization and its Association With Five-Year Mortality in the Synergy Between Percutaneous Coronary Intervention With Taxus and Cardiac Surgery (SYNTAX) Trial Validation of the Residual SYNTAX Score
Farooq, V., Serruys, P. W. J. C., Bourantas, C. V., Zhang, Y., Muramatsu, T., Feldman, T. E., Holmes, D. R., Mack, M. J., Morice, M., Ståhle, E., et al. (2013) · Circulation · doi:10.1161/circulationaha.113.001803 · Cited by 405
Diagnostic Accuracy of a Machine-Learning Approach to Coronary Computed Tomographic Angiography–Based Fractional Flow Reserve
Coenen, A., Kim, Y., Kruk, M. B. P., Tesche, C., Geer, J. D., Kurata, A., Lubbers, M. M., Daemen, J., Lucian Mihai, I. T. U., Rapaka, S., et al. (2018) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.117.007217 · Cited by 409
Analytical performance of the ThyroSeq v3 genomic classifier for cancer diagnosis in thyroid nodules
Nikiforova, M. N., Mercurio, S., Wald, A. I., de Moura, M. B., Callenberg, K. M., Santana‐Santos, L., Gooding, W. E., Yip, L., Ferris, R. L., & Nikiforov, Y. E. (2018) · Cancer · doi:10.1002/cncr.31245 · Cited by 409
Deep Learning Models for Histopathological Classification of Gastric and Colonic Epithelial Tumours
Iizuka, O., Kanavati, F., Kato, K., Rambeau, M., Arihiro, K., & Tsuneki, M. (2020) · Scientific Reports · doi:10.1038/s41598-020-58467-9 · Cited by 412
Cardiac Biomarkers Are Associated With an Increased Risk of Stroke and Death in Patients With Atrial Fibrillation: A Randomized Evaluation of Long-Term Anticoagulation Therapy (RE-LY) Substudy
Hijazi, Z., Oldgren, J., Andersson, U., Connolly, S. J., Ezekowitz, M. D., Hohnloser, S. H., Reilly, P. A., Vinereanu, D., Siegbahn, A., Yusuf, S., et al. (2012) · Circulation · doi:10.1161/circulationaha.111.038729 · Cited by 402
CatBoost model and artificial intelligence techniques for corporate failure prediction
Jabeur, S. B., Gharib, C., Mefteh‐Wali, S., & Arfi, W. B. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120658 · Cited by 420
An Observational Study of Deep Learning and Automated Evaluation of Cervical Images for Cancer Screening
Hu, L., Bell, D. A., Antani, S., Xue, Z., Yu, K., Horning, M. P., Gachuhi, N., Wilson, B., Jaiswal, M. S., Befano, B., et al. (2018) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djy225 · Cited by 412
Quadratus Lumborum Block Versus Transversus Abdominis Plane Block for Postoperative Pain After Cesarean Delivery
Blanco, R., Ansari, T., Riad, W., & Shetty, N. (2016) · Regional Anesthesia & Pain Medicine · doi:10.1097/aap.0000000000000495 · Cited by 400
Six-Minute-Walk Distance Predicts Waiting List Survival in Idiopathic Pulmonary Fibrosis
Lederer, D. J., Arcasoy, S. M., Wilt, J. S., D’Ovidio, F., Sonett, J. R., & Kawut, S. M. (2006) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200604-520oc · Cited by 404
Prevalence and Severity of Coronary Artery Disease and Adverse Events Among Symptomatic Patients With Coronary Artery Calcification Scores of Zero Undergoing Coronary Computed Tomography Angiography
Villines, T. C., Hulten, E. A., Shaw, L. J., Goyal, M., Dunning, A. M., Achenbach, S. S., Al-Mallah, M. H., Berman, D. S., Budoff, M. J., Cademartiri, F., et al. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.10.851 · Cited by 404
Alanine aminotransferase‐based algorithms of liver stiffness measurement by transient elastography (Fibroscan) for liver fibrosis in chronic hepatitis B
Chan, H. L., Wong, G. L., Choi, P. C., Chan, A. W. H., Chim, A. M.-L., Yiu, K. K. L., Chan, F. K. L., Sung, J. J. �., & Wong, V. W. (2008) · Journal of Viral Hepatitis · doi:10.1111/j.1365-2893.2008.01037.x · Cited by 398
Cognitive tests that best discriminate between presymptomatic AD and those who remain nondemented
Chen, P., Ratcliff, G., Belle, S. H., Cauley, J. A., DeKosky, S. T., & Ganguli, M. (2000) · Neurology · doi:10.1212/wnl.55.12.1847 · Cited by 405
Parameters behind “Nonparametric” Statistics: Kendall's tau, Somers’ D and Median Differences
Newson, R. (2002) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0200200103 · Cited by 394
Clinimetric Testing of Three Self-report Outcome Measures for Low Back Pain Patients in Brazil
Costa, L. O. P., Maher, C. G., Latimer, J., Ferreira, P. H., Ferreira, M. L., Pozzi, G. C., & Freitas, L. M. A. (2008) · Spine · doi:10.1097/brs.0b013e3181849dbe · Cited by 394
The DECAF Score: predicting hospital mortality in exacerbations of chronic obstructive pulmonary disease
Steer, J., Gibson, J., & Bourke, S. C. (2012) · Thorax · doi:10.1136/thoraxjnl-2012-202103 · Cited by 396
A Primer on Receiver Operating Characteristic Analysis and Diagnostic Efficiency Statistics for Pediatric Psychology: We Are Ready to ROC
Youngstrom, E. A. (2013) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jst062 · Cited by 410
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
Serum sodium predicts mortality in patients listed for liver transplantation
Biggins, S. W., Rodriguez, H. J., Bacchetti, P., Bass, N. M., Roberts, J. P., & Terrault, N. A. (2004) · Hepatology · doi:10.1002/hep.20517 · Cited by 392
Dense genotyping of immune-related disease regions identifies nine new risk loci for primary sclerosing cholangitis
The UK-PSCSC Consortium, Liu, J. Z., The International IBD Genetics Consortium, The International PSC Study Group, Hov, J. R., Folseraas, T., Ellinghaus, E., Rushbrook, S. M., Doncheva, N. T., Andreassen, O. A., et al. (2013) · Nature Genetics · doi:10.1038/ng.2616 · Cited by 391
The “cardiometabolic index” as a new marker determined by adiposity and blood lipids for discrimination of diabetes mellitus
Wakabayashi, I., & Daimon, T. (2014) · Clinica Chimica Acta · doi:10.1016/j.cca.2014.08.042 · Cited by 391
The reactive oxygen species—total antioxidant capacity score is a new measure of oxidative stress to predict male infertility*
Sharma, R. K., Pasqualotto, F. F., Nelson, D. R., Thomas, A. J., Jr., & Agarwal, A. K. (1999) · Human Reproduction · doi:10.1093/humrep/14.11.2801 · Cited by 391
A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets
Antropova, N., Huynh, B. Q., & Giger, M. L. (2017) · Medical Physics · doi:10.1002/mp.12453 · Cited by 389
“In order to assess statistical significance, DeLong tests were performed.(27) Bonferroni-Holm corrections were used to account for multiple comparisons.”
Diffusion-Weighted MRI for Selection of Complete Responders After Chemoradiation for Locally Advanced Rectal Cancer: A Multicenter Study
Lambregts, D. M. J., Vandecaveye, V., Barbaro, B., Bakers, F. C. H., Lambrecht, M. L. F., Maas, M., Haustermans, K. M., Valentini, V., Beets, G. L., & Beets‐Tan, R. G. H. (2011) · Annals of Surgical Oncology · doi:10.1245/s10434-011-1607-5 · Cited by 390
Serum anti-Mullerian hormone and FSH: prediction of live birth and extremes of response in stimulated cycles implications for individualization of therapy
Nelson, S. M., Yates, R. W. S., & Fleming, R. (2007) · Human Reproduction · doi:10.1093/humrep/dem204 · Cited by 389
A novel method for accurate operon predictions in all sequenced prokaryotes
Price, M. N. (2005) · Nucleic Acids Research · doi:10.1093/nar/gki232 · Cited by 388
Clinical Prediction Model To Characterize Pulmonary Nodules
Herder, G. J. M., van Tinteren, H., Golding, R. P., Kostense, P. J., Comans, E. F. I., Smit, E. F., & Hoekstra, O. S. (2005) · CHEST Journal · doi:10.1378/chest.128.4.2490 · Cited by 387
Plasma p‐tau181, p‐tau217, and other blood‐based Alzheimer's disease biomarkers in a multi‐ethnic, community study
Brickman, A. M., Manly, J. J., Honig, L. S., Sanchez, D., Reyes‐Dumeyer, D., Lantigua, R. A., Lao, P. J., Stern, Y., Vonsattel, J. P., Teich, A. F., et al. (2021) · Alzheimer's & Dementia · doi:10.1002/alz.12301 · Cited by 387
Elevations in Markers of Liver Injury and Risk of Type 2 Diabetes
Hanley, A. J., Williams, K., Festa, A., Wagenknecht, L. E., D’Agostino, R. B., Kempf, J., Zinman, B., & Haffner, S. M. (2004) · Diabetes · doi:10.2337/diabetes.53.10.2623 · Cited by 386
SVMTriP: A Method to Predict Antigenic Epitopes Using Support Vector Machine to Integrate Tri-Peptide Similarity and Propensity
Yao, B., Zhang, L., Liang, S., & Zhang, C. (2012) · PLoS ONE · doi:10.1371/journal.pone.0045152 · Cited by 386
Ticagrelor for Prevention of Ischemic Events After Myocardial Infarction in Patients With Peripheral Artery Disease
Bonaca, M. P., Bhatt, D. L., Storey, R. F., Steg, P. G., Cohen, M. C., Kuder, J., Goodrich, E. L., Nicolau, J. C., Parkhomenko, A. N., Lopez-Sendon, J. L., et al. (2016) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2016.03.524 · Cited by 385
Lymph Node Metastasis Prediction from Primary Breast Cancer US Images Using Deep Learning
Zhou, L.-Q., Wu, X.-L., Huang, S.-Y., Wu, G.-G., Ye, H.-R., Wei, Q., Bao, L.-Y., Deng, Y.-B., Li, X.-R., Cui, X.-W., et al. (2020) · Radiology · doi:10.1148/radiol.2019190372 · Cited by 384
The Framingham Predictive Instrument in Chronic Kidney Disease
Weiner, D. E., Tighiouart, H., Elsayed, E. F., Griffith, J. L., Salem, D. N., Levey, A. S., & Sarnak, M. J. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.03.037 · Cited by 385
Predicting survival with good neurological recovery at hospital admission after successful resuscitation of out-of-hospital cardiac arrest: the OHCA score
Adrie, C., Cariou, A., Mourvillier, B., Laurent, I., Dabbane, H., Hantala, F., Rhaoui, A., Thuong, M., & Monchi, M. (2006) · European Heart Journal · doi:10.1093/eurheartj/ehl335 · Cited by 383
Neonatal Candidiasis: Epidemiology, Risk Factors, and Clinical Judgment
Benjamin, D. K., Stoll, B. J., Gantz, M. G., Walsh, M. C., Sánchez, P. J., Das, A., Shankaran, S., Higgins, R. D., Auten, K. J., Miller, N. A., et al. (2010) · PEDIATRICS · doi:10.1542/peds.2009-3412 · Cited by 380
Comprehensive prediction of secondary metabolite structure and biological activity from microbial genome sequences
Skinnider, M. A., Johnston, C. W., Gunabalasingam, M., Merwin, N. J., Kieliszek, A., MacLellan, R. J., Li, H., Ranieri, M. R. M., Webster, A., Cao, M. P. T., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19986-1 · Cited by 380
T1 Mapping for the Diagnosis of Acute Myocarditis Using CMR
Ferreira, V. M., Piechnik, S. K., Dall’Armellina, E., Karamitsos, T. D., Francis, J. M., Ntusi, N., Holloway, C. J., Choudhury, R. P., Kardos, A., Robson, M. D., et al. (2013) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2013.03.008 · Cited by 380
Systematic Evaluation and Comparison of Statistical Tests for Publication Bias
Hayashino, Y., Noguchi, Y., & Fukui, T. (2005) · Journal of Epidemiology · doi:10.2188/jea.15.235 · Cited by 379
Quantification and Impact of Untreated Coronary Artery Disease After Percutaneous Coronary Intervention
Généreux, P., Palmerini, T., Caixeta, A. M., Rosner, G. F., Green, P., Dressler, O., Xu, K., Parise, H., Mehran, R., Serruys, P. W. J. C., et al. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2012.03.010 · Cited by 378
Concordance between the assessment of Aβ42, T‐tau, and P‐T181‐tau in peripheral blood neuronal‐derived exosomes and cerebrospinal fluid
Jia, L., Qiu, Q., Zhang, H., Chu, L., Du, Y., Zhang, J., Zhou, C., Liang, F., Shi, S., Wang, S., et al. (2019) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2019.05.002 · Cited by 383
Urinary cystatin C as an early biomarker of acute kidney injury following adult cardiothoracic surgery
Koyner, J. L., BENNETT, M. R., Worcester, E. M., Ma, Q., Raman, J., Jeevanandam, V., Kasza, K., O’Connor, M. F., Konczal, D., Trevino, S., et al. (2008) · Kidney International · doi:10.1038/ki.2008.341 · Cited by 375
The interpretation of diagnostic tests
Shapiro, D. E. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800203 · Cited by 375
easyROC: An Interactive Web-tool for ROC Curve Analysis Using R Language Environment
Göksülük, D., Korkmaz, S., Zararsız, G., & Karaağaoğlu, A. E. (2016) · The R Journal · doi:10.32614/rj-2016-042 · Cited by 375
Child Sexual Behavior Inventory: Normative and clinical comparisons.
Friedrich, W. N., Grambsch, P., Damon, L., Hewitt, S. K., Koverola, C., Lang, R. A., Wolfe, V., & Broughton, D. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.3.303 · Cited by 374
GFAP and S100B are biomarkers of traumatic brain injury
Vos, P. E., Jacobs, B., Andriessen, T. M. J. C., Lamers, K. J. B., Borm, G. F., Beems, T., Edwards, M., Rosmalen, C. F., & Vissers, J. L. M. (2010) · Neurology · doi:10.1212/wnl.0b013e3181fd62d2 · Cited by 374

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks