Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 801–900 of 1,197

Repeat FDG-PET After Neoadjuvant Therapy is a Predictor of Pathologic Response in Patients With Non-Small Cell Lung Cancer
Cerfolio, R. J., Bryant, A. S., Winokur, T. S., Ohja, B., & Bartolucci, A. A. (2004) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2004.06.102 · Cited by 194
A score for predicting salvage and outcome in Gustilo type-IIIA and type-IIIB open tibial fractures
Rajasekaran, S., Babu, J. N., Dheenadhayalan, J., Shetty, A. P., Sundararajan, S. R., Kumar, M., & Sabapathy, S. R. (2006) · Journal of Bone and Joint Surgery - British Volume · doi:10.1302/0301-620x.88b10.17631 · Cited by 194
APOE genotype and extent of bleeding and outcome in lobar intracerebral haemorrhage: a genetic association study
Biffi, A., Anderson, C. D., Jagiella, J. M., Schmidt, H., D, B. M. K. M., Hansen, B. M., Jiménez-Conde, J., Pires, C. R., Ayres, A. M., Schwab, K. M., et al. (2011) · The Lancet Neurology · doi:10.1016/s1474-4422(11)70148-x · Cited by 197
Calcified Atherosclerosis in Different Vascular Beds and the Risk of Mortality
Allison, M., Hsi, S., Wassel, C. L., Morgan, C. G., Ix, J. H., Criqui, M. H., & Criqui, M. H. (2011) · Arteriosclerosis Thrombosis and Vascular Biology · doi:10.1161/atvbaha.111.235234 · Cited by 202
Machine Learning for Early Lung Cancer Identification Using Routine Clinical and Laboratory Data
Gould, M. K., Huang, B. Z., Tammemägi, M. C., Kinar, Y., & Shiff, R. (2021) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202007-2791oc · Cited by 194
Bone Density Variation and Its Effects on Risk of Vertebral Deformity in Men and Women Studied in Thirteen European Centers: The EVOS Study
Lunt, M., Felsenberg, D., REEVE, J., Benevolenskaya, L. I., Cannata-Andía, J. B., Dequeker, J., Dodenhof, C., Ja, F., Masaryk, P., Pols, H. A. P., et al. (1997) · Journal of Bone and Mineral Research · doi:10.1359/jbmr.1997.12.11.1883 · Cited by 194
The importance of spatial autocorrelation, extent and resolution in predicting forest bird occurrence
Betts, M. G., Diamond, A. W., Forbes, G. J., Villard, M., & Gunn, J. (2005) · Ecological Modelling · doi:10.1016/j.ecolmodel.2005.04.027 · Cited by 193
Prostate-Specific Antigen (PSA) Isoform p2PSA Significantly Improves the Prediction of Prostate Cancer at Initial Extended Prostate Biopsies in Patients with Total PSA Between 2.0 and 10 ng/ml: Results of a Prospective Study in a Clinical Setting
Guazzoni, G. F., Nava, L. D., Lazzeri, M., Scattoni, V., Giovanni Lughezzani, A., Maccagnano, C., Dorigatti, F., Ceriotti, F., Pontillo, M., Bini, V., et al. (2011) · European Urology · doi:10.1016/j.eururo.2011.03.052 · Cited by 193
A combined model based on spleen stiffness measurement and Baveno VI criteria to rule out high-risk varices in advanced chronic liver disease
Colecchia, A., Ravaioli, F., Marasco, G., Colli, A., Dajti, E., Biase, A. R. D., Reggiani, M. L. B., Berzigotti, A., Pinzani, M., & Festi, D. (2018) · Journal of Hepatology · doi:10.1016/j.jhep.2018.04.023 · Cited by 193
Random Forests for multiclass classification: Random MultiNomial Logit
Prinzie, A. A., & den Poel, D. V. (2007) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.01.029 · Cited by 193
The aspect ratio (dome/neck) of ruptured and unruptured aneurysms
Weir, B., Amidei, C., Kongable, G., Findlay, J. M., Kassell, N. F., Kelly, J. J., Dai, L., & Karrison, T. G. (2003) · Journal of neurosurgery · doi:10.3171/jns.2003.99.3.0447 · Cited by 193
The Association of Pediatric Low- and High-Density Lipoprotein Cholesterol Dyslipidemia Classifications and Change in Dyslipidemia Status With Carotid Intima-Media Thickness in Adulthood
Magnussen, C. G., Venn, A., Thomson, R. J., Juonala, M., Srinivasan, S. R., Viikari, J. S. A., Berenson, G. S., Dwyer, T., & Raitakari, O. T. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2008.09.061 · Cited by 192
Domain Transfer Learning for MCI Conversion Prediction
Cheng, B., Liu, M., Zhang, D., Munsell, B. C., & Shen, D. G. (2015) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2015.2404809 · Cited by 192
Health Care Expenditure Prediction With a Single Item, Self-Rated Health Measure
DeSalvo, K. B., Jones, T. M., Peabody, J. W., McDonald, J., Fihn, S. D., Fan, V. S., He, J., & Muntner, P. (2009) · Medical Care · doi:10.1097/mlr.0b013e318190b716 · Cited by 192
Risk Prediction for Late-Stage Ovarian Cancer by Meta-analysis of 1525 Patient Samples
Riester, M., Wei, W., Waldron, L., Culhane, A. C., Trippa, L., Oliva, E., Kim, S., Franziska Michor, A., Huttenhower, C., Parmigiani, G., et al. (2014) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dju048 · Cited by 192
Validation of the Fatty Liver Index for Nonalcoholic Fatty Liver Disease in Middle-Aged and Elderly Chinese
Huang, X., Xu, M., Chen, Y., Peng, K., Huang, Y. F., Wang, P., Ding, L., Lin, L., Xu, Y., Chen, Y., et al. (2015) · Medicine · doi:10.1097/md.0000000000001682 · Cited by 192
Genome-Wide lncRNA Microarray Profiling Identifies Novel Circulating lncRNAs for Detection of Gastric Cancer
Zhang, K., Shi, H., Xi, H., Wu, X., Cui, J., Gao, Y., Liang, W., Hu, C., Liu, Y., Li, J., et al. (2016) · Theranostics · doi:10.7150/thno.16044 · Cited by 192
Evaluation of mutant huntingtin and neurofilament proteins as potential markers in Huntington’s disease
Byrne, L. M., Rodrigues, F. B., Johnson, E. B., Wijeratne, P. A., Vita, E. D., Alexander, D. C., Palermo, G., Czech, C., Schobel, S., Scahill, R. I., et al. (2018) · Science Translational Medicine · doi:10.1126/scitranslmed.aat7108 · Cited by 192
Magnetic resonance elastography is superior to acoustic radiation force impulse for the Diagnosis of fibrosis in patients with biopsy‐proven nonalcoholic fatty liver disease: A prospective study
Cui, J. Y., Heba, E. R., Hernandez, C., Haufe, W. M., Hooker, J. C., André, M. P., Valasek, M. A., Aryafar, H., Sirlin, C. B., & Loomba, R. S. (2015) · Hepatology · doi:10.1002/hep.28337 · Cited by 191
Discriminative Accuracy of Physician and Nurse Predictions for Survival and Functional Outcomes 6 Months After an ICU Admission
Detsky, M. E., Harhay, M. O., Bayard, D. F., Delman, A. M., Buehler, A. E., Kent, S. A., Ciuffetelli, I. V., Cooney, E. L., Gabler, N. B., Ratcliffe, S. J., et al. (2017) · JAMA · doi:10.1001/jama.2017.4078 · Cited by 191
B7-H4 Is a Novel Membrane-Bound Protein and a Candidate Serum and Tissue Biomarker for Ovarian Cancer
Simón, I., Zhuo, S., Corral, L. G., Diamandis, E. P., Sarno, M., Wolfert, R. L., & Kim, N. W. (2006) · Cancer Research · doi:10.1158/0008-5472.can-04-3550 · Cited by 191
DeepCOVID-XR: An artificial intelligence algorithm to detect COVID-19 on chest radiographs trained and tested on a large U.S. clinical data set
Wehbe, R. M., Sheng, J., Dutta, S., Chai, S., Dravid, A., Barutcu, S., Wu, Y., Cantrell, D. R., Xiao, N., Allen, B. D., et al. (2021) · Radiology · doi:10.1148/radiol.2020203511 · Cited by 190
Influence of Coronary Calcification on the Diagnostic Performance of CT Angiography Derived FFR in Coronary Artery Disease
Nørgaard, B. L., Gaur, S., Leipsic, J. A., Ito, H., Miyoshi, T., Park, S., Zvaigzne, L., Tzemos, N., Jensen, J. M., Hansson, N. C., et al. (2015) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2015.06.003 · Cited by 190
Contrast letter acuity as a visual component for the Multiple Sclerosis Functional Composite
Balcer, L. J., Baier, M. L., Cohen, J. A., Kooijmans, M., Sandrock, A., Nano-Schiavi, M. L., Pfohl, D., Mills, M., Bowen, J. D., Ford, C. C., et al. (2003) · Neurology · doi:10.1212/01.wnl.0000094315.19931.90 · Cited by 190
Infarct architecture and characteristics on delayed enhanced magnetic resonance imaging and electroanatomic mapping in patients with postinfarction ventricular arrhythmia
Desjardins, B., Crawford, T., Good, E. D., Oral, H., Chugh, A., Pelosi, F., Morady, F., & Bogun, F. M. (2009) · Heart Rhythm · doi:10.1016/j.hrthm.2009.02.018 · Cited by 190
Extraction of dynamic functional connectivity from brain grey matter and white matter for MCI classification
Chen, X., Zhang, H., Zhang, L., Shen, C., Lee, S., & Shen, D. G. (2017) · Human Brain Mapping · doi:10.1002/hbm.23711 · Cited by 190
Association of Frailty With 30-Day Outcomes for Acute Myocardial Infarction, Heart Failure, and Pneumonia Among Elderly Adults
Kundi, H., Wadhera, R. K., Strom, J. B., Valsdottir, L. R., Shen, C., Kazi, D. S., & Yeh, R. W. (2019) · JAMA Cardiology · doi:10.1001/jamacardio.2019.3511 · Cited by 190
Total Aβ 42 /Aβ 40 ratio in plasma predicts amyloid-PET status, independent of clinical AD diagnosis
Doecke, J. D., Pérez‐Grijalba, V., Fandos, N., Fowler, C. J., Villemagne, V. L., Masters, C. L., Pesini, P., & Sarasa, M. (2020) · Neurology · doi:10.1212/wnl.0000000000009240 · Cited by 202
Improved Risk Stratification for Ventricular Arrhythmias and Sudden Death in Patients With Nonischemic Dilated Cardiomyopathy
Marco, A. D., Brown, P. F., Bradley, J., Nucifora, G., Claver, E., de Frutos, F., Dallaglio, P., Colet, J. C., Anguera, I., Miller, C., et al. (2021) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2021.04.030 · Cited by 190
Detection of structural damage from glaucoma with confocal laser image analysis.
Uchida, H., Brigatti, L., & Caprioli, J. (1996) · PubMed · Cited by 190
Combining CA 125 and SMR serum markers for diagnosis and early detection of ovarian carcinoma
McIntosh, M. W., Drescher, C. W., Karlan, B. Y., Scholler, N., Urban, N. D., Hellström, K. E., & Hellström, I. (2004) · Gynecologic Oncology · doi:10.1016/j.ygyno.2004.07.039 · Cited by 190
A multi-centre randomised trial comparing ultrasound vs mammography for screening breast cancer in high-risk Chinese women
Shen, S., Zhou, Y., Xu, Y., Zhang, B.-Y., Duan, X., Huang, R., Li, B., Shi, Y. X., Shao, Z., Liao, H., et al. (2015) · British Journal of Cancer · doi:10.1038/bjc.2015.33 · Cited by 189
A revised methodology for aortic-valvar complex calcium quantification for transcatheter aortic valve implantation
Jilaihawi, H., Makkar, R., Kashif, M., Okuyama, K., Chakravarty, T., Shiota, T., Friede, G., Nakamura, M., Doctor, N., Rafique, A. M., et al. (2014) · European Heart Journal - Cardiovascular Imaging · doi:10.1093/ehjci/jeu162 · Cited by 189
Level of HER2 Gene Amplification Predicts Response and Overall Survival in HER2-Positive Advanced Gastric Cancer Treated With Trastuzumab
Gómez-Martín, C., Plaza, C., Pazo-Cid, R., Salud, A., Pons, F., Fonseca, P., León, A., Alsina, M., Visa, L., Rivera, F., et al. (2013) · Journal of Clinical Oncology · doi:10.1200/jco.2013.48.9070 · Cited by 189
Laboratory parameter‐based machine learning model for excluding non‐alcoholic fatty liver disease ( NAFLD ) in the general population
Yip, T. C., Ma, A. J., Wong, V. W., Tse, Y. K., Chan, H. L., Yuen, P. C., & Wong, G. L. (2017) · Alimentary Pharmacology & Therapeutics · doi:10.1111/apt.14172 · Cited by 189
The Pathologic Basis of Q-Wave and Non-Q-Wave Myocardial Infarction
Moon, J., de Arenaza, D. P., Elkington, A. G., Taneja, A. K., John, A. S., Wang, D., Janardhanan, R., Senior, R., Lahiri, A., Poole-Wilson, P. A., et al. (2004) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2004.03.076 · Cited by 188
A Framework for Evaluating Biomarkers for Early Detection: Validation of Biomarker Panels for Ovarian Cancer
Zhu, C. S., Pinsky, P. F., Cramer, D. W., Ransohoff, D. F., Hartge, P. A., Pfeiffer, R. M., Urban, N. D., Mor, G., Bast, R. C., Moore, L. E., et al. (2011) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-10-0193 · Cited by 188
Integrative random forest for gene regulatory network inference
Petralia, F., Wang, P., Yang, J., & Tu, Z. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv268 · Cited by 194
Novel Risk Score to Predict Pneumonia After Acute Ischemic Stroke
Ji, R., Shen, H., Pan, Y., Wang, P., Liu, G., Wang, Y., Li, H., & Wang, Y. (2013) · Stroke · doi:10.1161/strokeaha.111.000598 · Cited by 188
Classification performance of instantaneous wave-free ratio (iFR) and fractional flow reserve in a clinical population of intermediate coronary stenoses: results of the ADVISE registry
Petraco, R., Escaned, J., Sen, S., Nijjer, S. S., Asrress, K., Pinto, M. E., Lockie, T., Khawaja, M. Z., Cuevas, C., Foin, N., et al. (2013) · EuroIntervention · doi:10.4244/eijv9i1a14 · Cited by 188
A Comparison of a Machine Learning Model with EuroSCORE II in Predicting Mortality after Elective Cardiac Surgery: A Decision Curve Analysis
Allyn, J., Allou, N., Augustin, P., Philip, I., Martinet, O., Belghiti, M., Provenchère, S., Montravers, P., & Ferdynus, C. (2017) · PLoS ONE · doi:10.1371/journal.pone.0169772 · Cited by 188
Clinical and analytical comparison of six Simoa assays for plasma P-tau isoforms P-tau181, P-tau217, and P-tau231
Bayoumy, S., Verberk, I. M. W., den Dulk, B., Hussainali, Z., Zwan, M. D., van der Flier, W. M., Ashton, N. J., Zetterberg, H., Blennow, K., Vanbrabant, J., et al. (2021) · Alzheimer s Research & Therapy · doi:10.1186/s13195-021-00939-9 · Cited by 196
Folstein vs Modified Mini-Mental State Examination in Geriatric Stroke
Grace, J. (1995) · Archives of Neurology · doi:10.1001/archneur.1995.00540290067019 · Cited by 188
Radiomics-based Machine-learning Models Can Detect Pancreatic Cancer on Prediagnostic Computed Tomography Scans at a Substantial Lead Time Before Clinical Diagnosis
Mukherjee, S., Patra, A., Khasawneh, H., Korfiatis, P., Rajamohan, N., Suman, G., Majumder, S., Panda, A., Johnson, M. P., Larson, N. B., et al. (2022) · Gastroenterology · doi:10.1053/j.gastro.2022.06.066 · Cited by 187
Amyloid-β 42/40 Cerebrospinal Fluid Concentration Ratio in the Diagnostics of Alzheimer's Disease: Validation of Two Novel Assays
Lewczuk, P., Lelental, N., Spitzer, P., Maler, J. M., & Kornhuber, J. (2014) · Journal of Alzheimer s Disease · doi:10.3233/jad-140771 · Cited by 192
Predictive Validity Performance Indicators in Violence Risk Assessment: A Methodological Primer
Singh, J. P. (2013) · Behavioral Sciences & the Law · doi:10.1002/bsl.2052 · Cited by 205
A comparison of clinically utilized human papillomavirus detection methods in head and neck cancer
Schlecht, N. F., Brandwein‐Gensler, M. S., Nuovo, G. J., Li, M., Dunne, A., Kawachi, N., Smith, R. V., Burk, R. D., & Prystowsky, M. B. (2011) · Modern Pathology · doi:10.1038/modpathol.2011.91 · Cited by 187
An investigation of bankruptcy prediction in imbalanced datasets
Veganzones, D., & Séverin, É. (2018) · Decision Support Systems · doi:10.1016/j.dss.2018.06.011 · Cited by 197
Diagnosis of obstructive coronary artery disease using computed tomography angiography in patients with stable chest pain depending on clinical probability and in clinically important subgroups: meta-analysis of individual patient data
Haase, R., Schlattmann, P., Guéret, P., Andreini, D., Pontone, G., Alkadhi, H., Hausleiter, J., Garcia, M. J., Leschka, S., Meijboom, W. B., et al. (2019) · BMJ · doi:10.1136/bmj.l1945 · Cited by 194
Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients
Lassau, N., Ammari, S., Chouzenoux, É., Gortais, H., Hérent, P., Devilder, M., Soliman, S., Meyrignac, O., Talabard, M.-P., Lamarque, J.-P., et al. (2021) · Nature Communications · doi:10.1038/s41467-020-20657-4 · Cited by 187
Automatic detection of oral cancer in smartphone-based images using deep learning for early diagnosis
Lin, H., Chen, H., Weng, L., Shao, J., & Lin, J. T. (2021) · Journal of Biomedical Optics · doi:10.1117/1.jbo.26.8.086007 · Cited by 196
Towards the prediction of essential genes by integration of network topology, cellular localization and biological process information
Acencio, M. L., & Lemke, N. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-290 · Cited by 186
Urinary biomarkers trefoil factor 3 and albumin enable early detection of kidney tubular injury
Yu, Y. P., Jin, H. L., Holder, D., Ozer, J. S., Villarreal, S. A., Shughrue, P. J., Shi, S., Figueroa, D. J., Clouse, H., Su, M., et al. (2010) · Nature Biotechnology · doi:10.1038/nbt.1624 · Cited by 187
Predicting High-Grade Cancer at Ten-Core Prostate Biopsy Using Four Kallikrein Markers Measured in Blood in the ProtecT Study
Bryant, R. J., Sjoberg, D. D., Vickers, A. J., Robinson, M., Kumar, R., Marsden, L., Davis, M., Scardino, P. T., Donovan, J., Neal, D. E., et al. (2015) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djv095 · Cited by 186
Validating the Injury Severity Score (ISS) in Different Populations: ISS Predicts Mortality Better Among Hispanics and Females
Bolorunduro, O. B., Villegas, C. V., Oyetunji, T. A., Haut, E. R., Stevens, K. A., Chang, D. C., Cornwell, E. E., Efron, D. T., & Haider, A. H. (2010) · Journal of Surgical Research · doi:10.1016/j.jss.2010.04.012 · Cited by 186
A deep learning methodology for improved breast cancer diagnosis using multiparametric MRI
Hu, Q., Whitney, H. M., & Giger, M. L. (2020) · Scientific Reports · doi:10.1038/s41598-020-67441-4 · Cited by 190
Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology
Laleh, N. G., Muti, H. S., Loeffler, C. M. L., Echle, A., Saldanha, O. L., Mahmood, F., Lu, M. Y., Trautwein, C., Langer, R., Dislich, B., et al. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102474 · Cited by 193
GH-releasing hormone and GH-releasing peptide-6 for diagnostic testing in GH-deficient adults
Popovic, V., Leal, A., Micic, D., Koppeschaar, H. P., Torres, E., Paramo, C., Obradovic, S., Dieguez, C., & Casanueva, F. F. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02755-0 · Cited by 185
The Generalized Anxiety Disorder Screener (GAD‐7 ) and the anxiety module of the Hospital and Depression Scale (HADS‐A ) as screening tools for generalized anxiety disorder among cancer patients
Esser, P., Hartung, T. J., Friedrich, M., Johansen, C., Wittchen, H., Faller, H., Koch, U., Härter, M., Keller, M., Schulz, H., et al. (2018) · Psycho-Oncology · doi:10.1002/pon.4681 · Cited by 185
Bayesian kernel based classification for financial distress detection
Gestel, T. V., Baesens, B., Suykens, J. A. K., den Poel, D. V., Baestaens, D.-E., & Willekens, M. (2005) · European Journal of Operational Research · doi:10.1016/j.ejor.2004.11.009 · Cited by 187
Performance of pneumonia severity index and CURB-65 in predicting 30-day mortality in patients with COVID-19
Satıcı, C., Demirkol, M. A., Altunok, E. S., Gürsoy, B., Alkan, M., Kamat, S., Demirok, B., Sürmeli, C. D., Çalık, M., Çavuş, Z., et al. (2020) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2020.06.038 · Cited by 185
Biomarkers of inflammation and repair in kidney disease progression
Puthumana, J., Thiessen‐Philbrook, H., Xu, L., Coca, S. G., Garg, A. X., Himmelfarb, J., Bhatraju, P. K., Ikizler, T. A., Siew, E. D., Ware, L. B., et al. (2020) · Journal of Clinical Investigation · doi:10.1172/jci139927 · Cited by 185
Diagnostic validity of three swab techniques for identifying chronic wound infection
Gardner, S. E., Frantz, R. A., Saltzman, C. L., Hillis, S. L., Park, H., & Scherubel, M. (2006) · Wound Repair and Regeneration · doi:10.1111/j.1743-6109.2006.00162.x · Cited by 185
Validation of the REMA score for predicting mast cell clonality and systemic mastocytosis in patients with systemic mast cell activation symptoms
Alvarez-Twose, I., González-de-Olano, D., Sánchez-Muñoz, L., Matito, A., Jara-Acevedo, M., Teodosio, C., García-Montero, A., Morgado, J. M., Orfao, A., & Escribano, L. (2012) · International Archives of Allergy and Immunology · doi:10.1159/000329856 · Cited by 184
The False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184
Is the Children's Depression Inventory Short version a valid screening tool in pediatric care? A comparison to its full-length version
Allgaier, A., Frühe, B., Pietsch, K., Saravo, B., Baethmann, M., & Schulte‐Körne, G. (2012) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2012.08.016 · Cited by 184
Comparing the Prognostic Accuracy for All-Cause Mortality of Frailty Instruments: A Multicentre 1-Year Follow-Up in Hospitalized Older Patients
Pilotto, A., Rengo, F., Marchionni, N., Sancarlo, D., Fontana, A., Panza, F., Ferrucci, L., & on behalf of the FIRI-SIGG Study Group (2012) · PLoS ONE · doi:10.1371/journal.pone.0029090 · Cited by 192
A Randomized Trial of Iron-Biofortified Pearl Millet in School Children in India ,
Finkelstein, J. L., Mehta, S., Udipi, S. A., Ghugre, P. S., Luna, S., Wenger, M. J., Murray‐Kolb, L. E., Przybyszewski, E. M., & Haas, J. D. (2015) · Journal of Nutrition · doi:10.3945/jn.114.208009 · Cited by 190
The “Laboratory” Effect: Comparing Radiologists' Performance and Variability during Prospective Clinical and Laboratory Mammography Interpretations
Gur, D., Bandos, A. I., Cohen, C. S., Hakim, C. M., Hardesty, L. A., Ganott, M. A., Perrin, R. L., Poller, W. R., Shah, R., Sumkin, J. H., et al. (2008) · Radiology · doi:10.1148/radiol.2491072025 · Cited by 184
Fundamentals of clinical research for radiologists
Joseph, L., & Reinhold, C. (2005) · American Journal of Roentgenology · Cited by 184
Comparison of five systems of classification of diabetic foot ulcers and predictive factors for amputation
Jeon, B., Choi, H. J., Kang, J. S., Tak, M. S., & Park, E. S. (2016) · International Wound Journal · doi:10.1111/iwj.12642 · Cited by 184
Predicting mortgage default using convolutional neural networks
Kvamme, H., Sellereite, N., Aas, K., & Sjursen, S. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.02.029 · Cited by 193
Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury
Gravesteijn, B. Y., Nieboer, D., Ercole, A., Lingsma, H. F., Nelson, D., CALSTER, B. V., Steyerberg, E. W., Åkerlund, C. A. I., Amrein, K., Anđelić, N., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.03.005 · Cited by 191
Relationship of deoxycytidine kinase and cytoplasmic 5'-nucleotidase to the chemotherapeutic efficacy of 2-chlorodeoxyadenosine
Kawasaki, H., Carrera, C., Piro, L. D., Saven, A., Kipps, T. J., & Carson, D. (1993) · Blood · doi:10.1182/blood.v81.3.597.597 · Cited by 184
Automated macular pathology diagnosis in retinal OCT images using multi-scale spatial pyramid and local binary patterns in texture and shape encoding
Liu, Y.-Y., Chen, M., Ishikawa, H., Wollstein, G., Schuman, J. S., & Rehg, J. M. (2011) · Medical Image Analysis · doi:10.1016/j.media.2011.06.005 · Cited by 183
Macular Ganglion Cell Imaging Study: Glaucoma Diagnostic Accuracy of Spectral-Domain Optical Coherence Tomography
Jeoung, J. W., Choi, Y. J., Park, K. H., & Kim, D. M. (2013) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.12-11273 · Cited by 183
An apoptosis panel for nonalcoholic steatohepatitis diagnosis
Tamimi, T., Elgouhari, H. M., Alkhouri, N., Yerian, L. M., Berk, M. P., López, R., Schauer, P. R., Zein, N. N., & Feldstein, A. E. (2010) · Journal of Hepatology · doi:10.1016/j.jhep.2010.08.023 · Cited by 187
Paleontological baselines for evaluating extinction risk in the modern oceans
Finnegan, S., Anderson, S. C., Harnik, P. G., Simpson, C., Tittensor, D. P., Byrnes, J. E. K., Finkel, Z. V., Lindberg, D. R., Liow, L. H., Lockwood, R., et al. (2015) · Science · doi:10.1126/science.aaa6635 · Cited by 187
Risk estimation and risk prediction using machine-learning methods
Kruppa, J., Ziegler, A., & König, I. R. (2012) · Human Genetics · doi:10.1007/s00439-012-1194-y · Cited by 183
The Area under the ROC Curve and Its Competitors
Hilden, J. (1991) · Medical Decision Making · doi:10.1177/0272989x9101100204 · Cited by 183
A Multicenter Experience With Infliximab for Ulcerative Colitis: Outcomes and Predictors of Response, Optimization, Colectomy, and Hospitalization
Oussalah, A., Evesque, L., Laharie, D., Roblin, X., Boschetti, G., Nancey, S., Filippi, J., Flourié, B., Hébuterne, X., Bigard, M., et al. (2010) · The American Journal of Gastroenterology · doi:10.1038/ajg.2010.345 · Cited by 182
Utility of the Chronic Disease Score and Charlson Comorbidity Index as Comorbidity Measures for Use in Epidemiologic Studies of Antibiotic-resistant Organisms
McGregor, J. C., Kim, P. W., Perencevich, E. N., Bradham, D. D., Furuno, J. P., Kaye, K. S., Fink, J. C., LANGENBERG, P. W., Roghmann, M., & Harris, A. D. (2005) · American Journal of Epidemiology · doi:10.1093/aje/kwi068 · Cited by 182
Hybrid approach improves success of chronic total occlusion angioplasty
WILSON, W. M., Walsh, S. J., Yan, A. T.-K., Hanratty, C. G., Bagnall, A. J., Egred, M., Smith, E. J., Oldroyd, K. G., McEntegart, M. B., Irving, J., et al. (2016) · Heart · doi:10.1136/heartjnl-2015-308891 · Cited by 182
SAFE Artificial Intelligence in finance
Giudici, P., & Raffinetti, E. (2023) · Finance research letters · doi:10.1016/j.frl.2023.104088 · Cited by 182
Artificial Intelligence risk measurement
Giudici, P., Centurelli, M., & Turchetta, S. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121220 · Cited by 181
Cortical Porosity Identifies Women With Osteopenia at Increased Risk for Forearm Fractures
Bala, Y., Zebaze, R. M. D., Ghasemzadeh, A., Atkinson, E. J., Iuliano, S., Peterson, J. M., Amin, S., Bjørnerem, Å., Melton, L. J., Johansson, H., et al. (2014) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.2167 · Cited by 181
Coronary Computed Tomographic Prediction Rule for Time-Efficient Guidewire Crossing Through Chronic Total Occlusion
Opolski, M. P., Achenbach, S. S., Schuhbäck, A., Rolf, A., Möllmann, H., NEF, H. M., Rixe, J., Renker, M., Witkowski, A., Kępka, C., et al. (2015) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2014.07.031 · Cited by 181
Sampling variability of nonparametric estimates of the areas under receiver operating characteristic curves: An update
Hanley, J. A., & Hajian‐Tilaki, K. (1997) · Academic Radiology · doi:10.1016/s1076-6332(97)80161-4 · Cited by 181
ABC score: a new risk score that accurately predicts mortality in acute upper and lower gastrointestinal bleeding: an international multicentre study
Laursen, S. B., Oakland, K., Laine, L., Bieber, V., Marmo, R., Redondo‐Cerezo, E., Dalton, H. R., Ngu, J., Schultz, M., Soncini, M., et al. (2020) · Gut · doi:10.1136/gutjnl-2019-320002 · Cited by 185
Association between abdominal obesity indices and risk of cardiovascular events in Chinese populations with type 2 diabetes: a prospective cohort study
Qiao, T., Luo, T., Pei, H., Yimingniyazi, B., Aili, D., Aimudula, A., Zhao, H., Zhang, H., Dai, J., & Wang, D. (2022) · Cardiovascular Diabetology · doi:10.1186/s12933-022-01670-x · Cited by 183
Development and Validation of the Sequential Organ Failure Assessment (SOFA)-2 Score
Ranzani, O., Singer, M., Salluh, J. I., Shankar‐Hari, M., Pilcher, D., Berger‐Estilita, J., Coopersmith, C. M., Juffermans, N. P., Laffey, J. G., Reinikainen, M., et al. (2025) · JAMA · doi:10.1001/jama.2025.20516 · Cited by 187
Comparison of MELD, Child-Pugh, and Emory model for the prediction of survival in patients undergoing transjugular intrahepatic portosystemic shunting
Schepke, M., Roth, F., Fimmers, R., Brensing, K.-A., Sudhop, T., Schild, H. H., & Sauerbruch, T. (2003) · The American Journal of Gastroenterology · doi:10.1016/s0002-9270(03)00295-8 · Cited by 181
New Vitals After Injury: Shock Index for the Young and Age × Shock Index for the Old
Zarzaur, B. L., Croce, M. A., Fischer, P. E., Magnotti, L. J., & Fabian, T. C. (2008) · Journal of Surgical Research · doi:10.1016/j.jss.2008.03.025 · Cited by 180
A novel serum marker, glycosylated Wisteria floribunda agglutinin-positive Mac-2 binding protein (WFA+-M2BP), for assessing liver fibrosis
Toshima, T., Shirabe, K., Ikegami, T., Yoshizumi, T., Kuno, A., Togayachi, A., Gotoh, M., Narimatsu, H., Korenaga, M., Mizokami, M., et al. (2014) · Journal of Gastroenterology · doi:10.1007/s00535-014-0946-y · Cited by 180
Lost in spatial translation – A novel tool to objectively assess spatial disorientation in Alzheimer's disease and frontotemporal dementia
Tu, S., Wong, S., Hodges, J. R., Irish, M., Piguet, O., & Hornberger, M. (2015) · Cortex · doi:10.1016/j.cortex.2015.03.016 · Cited by 180
Direct comparison of fluorodeoxyglucose positron emission tomography and arterial spin labeling magnetic resonance imaging in Alzheimer's disease
Musiek, E. S., Chen, Y., Korczykowski, M., Saboury, B., Martinez, P. M., Reddin, J. S., Alavi, A. A., Kimberg, D. Y., Wolk, D. A., Julin, P., et al. (2011) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2011.06.003 · Cited by 180
Accuracy of Digital Breast Tomosynthesis for Depicting Breast Cancer Subgroups in a UK Retrospective Reading Study (TOMMY Trial)
Gilbert, F. J., Tucker, L., Gillan, M. G., Willsher, P., Cooke, J., Duncan, K. A., Michell, M. J., Dobson, H. M., Lim, Y. Y., Suaris, T. D., et al. (2015) · Radiology · doi:10.1148/radiol.2015142566 · Cited by 180
Distributed learning: Developing a predictive model based on data from multiple hospitals without data leaving the hospital - A real life proof of concept.
Jochems, A., Deist, T., van Soest, J., Eble, M., Bulens, P., Coucke, P., Dries, W., Lambin, P., & Dekker, A. (2016) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2016.10.002 · Cited by 189
Refining diagnosis of Parkinson's disease with deep learning-based interpretation of dopamine transporter imaging
Choi, H., Ha, S., Im, H. J., Paek, S. H., & Lee, D. S. (2017) · NeuroImage Clinical · doi:10.1016/j.nicl.2017.09.010 · Cited by 186
An Evaluation of Early Warning Models for Systemic Banking Crises: Does Machine Learning Improve Predictions?
Beutel, J., List, S., & von Schweinitz, G. (2018) · Journal of Financial Stability · doi:10.1016/j.jfs.2019.100693 · Cited by 188

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks