Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 1,001–1,100 of 1,197

Ability of Serum Glial Fibrillary Acidic Protein, Ubiquitin C-Terminal Hydrolase-L1, and S100B To Differentiate Normal and Abnormal Head Computed Tomography Findings in Patients with Suspected Mild or Moderate Traumatic Brain Injury
Welch, R. D., Ayaz, S. I., Lewis, L. M., Undén, J., Chen, J. Y., Mika, V. H., Saville, B., Tyndall, J. A., Nash, M., Büki, A., et al. (2015) · Journal of Neurotrauma · doi:10.1089/neu.2015.4149 · Cited by 167
Does Simplicity Compromise Accuracy in ACS Risk Prediction? A Retrospective Analysis of the TIMI and GRACE Risk Scores
Aragam, K. G., Tamhane, U. U., Kline-Rogers, E. M., Li, J., Fox, K. A. A., Goodman, S. G., Eagle, K. A., & Gurm, H. S. (2009) · PLoS ONE · doi:10.1371/journal.pone.0007947 · Cited by 168
Accuracy and predictive value of classification schemes for ketosis-prone diabetes
Balasubramanyam, A., Garza, G., Rodriguez, L., Hampe, C. S., Gaur, L., Lernmark, A., & Maldonado, M. R. (2006) · Diabetes Care · doi:10.2337/dc06-0749 · Cited by 166
Clinical prediction from structural brain MRI scans: A large-scale empirical study
Initiative, F. T. A. D. N., Sabuncu, M. R., & Konukoglu, E. (2015) · Neuroinformatics · doi:10.1007/s12021-014-9238-1 · Cited by 161
Levels of Soluble Apolipoprotein E/Amyloid-β (Aβ) Complex Are Reduced and Oligomeric Aβ Increased with APOE4 and Alzheimer Disease in a Transgenic Mouse Model and Human Samples*
Tai, L. M., Bilousova, T., Jungbauer, L., Roeske, S. K., Youmans, K. L., Yu, C., Poon, W. W., Cornwell, L. B., Miller, C. A., Vinters, H. V., et al. (2013) · Journal of Biological Chemistry · doi:10.1074/jbc.m112.442103 · Cited by 165
Deep Learning for Chest Radiograph Diagnosis in the Emergency Department
Hwang, E. J., Nam, J. G., Lim, W. H., Park, S. J., Jeong, Y. S., Kang, J. H., Hong, E. K., Kim, T. M., Goo, J. M., Park, S., et al. (2019) · Radiology · doi:10.1148/radiol.2019191225 · Cited by 164
The Effect of Combined Out-of-Hospital Hypotension and Hypoxia on Mortality in Major Traumatic Brain Injury
Spaite, D. W., Hu, C., Bobrow, B. J., Chikani, V., Barnhart, B., Gaither, J. B., Denninghoff, K. R., Adelson, P. D., Keim, S. M., Viscusi, C., et al. (2017) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2016.08.007 · Cited by 187
Use of HbA1c in Predicting Progression to Diabetes in French Men and Women
Droumaguet, C., Balkau, B., Simon, D., Caces, E., Tichet, J., Charles, M. A., Eschwege, E., & the DESIR Study Group (2006) · Diabetes Care · doi:10.2337/dc05-2525 · Cited by 159
A comparative study of CSF neurofilament light and heavy chain protein in MS
Kuhle, J., Plattner, K., Bestwick, J. P., Lindberg, R. L., Ramagopalan, S. V., Norgren, N., Nissim, A., Malaspina, A., Leppert, D., Giovannoni, G., et al. (2013) · Multiple Sclerosis Journal · doi:10.1177/1352458513482374 · Cited by 155
A review of statistical updating methods for clinical prediction models
Su, T.-L., Jaki, T., Hickey, G. L., Buchan, I., & Sperrin, M. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280215626466 · Cited by 147
Clinically Meaningful Scores on Pain Catastrophizing Before and After Multidisciplinary Rehabilitation: A Prospective Study of Individuals With Subacute Pain After Whiplash Injury
Scott, W., Wideman, T. H., & Sullivan, M. J. L. (2014) · The Clinical Journal of Pain · doi:10.1097/ajp.0b013e31828eee6c · Cited by 155
Dynamic trends in cardiac surgery: why the logistic EuroSCORE is no longer suitable for contemporary cardiac surgery and implications for future risk models
Hickey, G. L., Grant, S. W., Murphy, G. J., Bhabra, M., Pagano, D., McAllister, K., Buchan, I., & Bridgewater, B. (2013) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezs584 · Cited by 154
Smart watch accelerometry for analysis and diagnosis of tremor
Wile, D. J., Ranawaya, R., & Kiss, Z. H. T. (2014) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2014.04.021 · Cited by 148
CSF tau phosphorylation occupancies at T217 and T205 represent improved biomarkers of amyloid and tau pathology in Alzheimer’s disease
Barthélemy, N. R., Saef, B., Li, Y., Gordon, B. A., He, Y., Horie, K., Stomrud, E., Salvadó, G., Janelidze, S., Sato, C., et al. (2023) · Nature Aging · doi:10.1038/s43587-023-00380-7 · Cited by 153
Synaptic biomarkers in CSF aid in diagnosis, correlate with cognition and predict progression in MCI and Alzheimer's disease
Galasko, D., Xiao, M., Xu, D., Smirnov, D., Salmon, D. P., Dewit, N., Vanbrabant, J., Jacobs, D., Vanderstichele, H., Vanmechelen, E., et al. (2019) · Alzheimer's & Dementia: Translational Research & Clinical Interventions · doi:10.1016/j.trci.2019.11.002 · Cited by 133
Detection of Left Ventricular Systolic Dysfunction From Electrocardiographic Images
Sangha, V., Nargesi, A. A., Dhingra, L. S., Khunte, A., Mortazavi, B. J., Ribeiro, A. H., Banina, E., Adeola, O., Garg, N., Brandt, C. A., et al. (2023) · Circulation · doi:10.1161/circulationaha.122.062646 · Cited by 150
Hepatic lipidomic remodeling in severe obesity manifests with steatosis and does not evolve with non-alcoholic steatohepatitis
Ooi, G. J., Meikle, P. J., Huynh, K., Earnest, A., Roberts, S. K., Kemp, W., Parker, B. L., Brown, W., Burton, P., & Watt, M. J. (2021) · Journal of Hepatology · doi:10.1016/j.jhep.2021.04.013 · Cited by 137
An Absolute Risk Model to Identify Individuals at Elevated Risk for Pancreatic Cancer in the General Population
Klein, A. P., Lindström, S., Mendelsohn, J. B., Steplowski, E., Arslan, A. A., Bueno-de-Mesquita, H. B., Fuchs, C. S., Gallinger, S., Gross, M., Helzlsouer, K., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0072311 · Cited by 148
Development and Validation of a Biomarker for Diarrhea-Predominant Irritable Bowel Syndrome in Human Subjects
Pimentel, M., Morales, W., Rezaie, A., Marsh, E., Lembo, A., Mirocha, J., Leffler, D. A., Marsh, Z., Weitsman, S., Chua, K. S., et al. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126438 · Cited by 148
Strong association of waist circumference (WC), body mass index (BMI), waist-to-height ratio (WHtR), and waist-to-hip ratio (WHR) with diabetes: A population-based cross-sectional study in jilin province, China
Zhang, F.-L., Ren, J.-X., Zhang, P., Jin, H., Qu, Y., Yu, Y., Guo, Z.-N., & Yang, Y. (2021) · Journal of Diabetes Research · doi:10.1155/2021/8812431 · Cited by 144
Prediction model of in-hospital mortality in intensive care unit patients with heart failure: machine learning-based, retrospective analysis of the MIMIC-III database
Li, F., Xin, H., Zhang, J., Fu, M., Zhou, J., & Lian, Z. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044779 · Cited by 128
Clinical utility of magnetic resonance imaging biomarkers for identifying nonalcoholic steatohepatitis patients at high risk of progression: a multicenter pooled data and meta-analysis
Andersson, A., Kelly, M., Imajo, K., Nakajima, A., Fallowfield, J. A., Hirschfield, G., Pavlides, M., Sanyal, A. J., Noureddin, M., Banerjee, R., et al. (2022) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2021.09.041 · Cited by 148
Performance of the PRO-C3 collagen neo-epitope biomarker in non-alcoholic fatty liver disease
Boyle, M., Tiniakos, D., Schattenberg, J. M., Ratziu, V., Bugianessi, E., Petta, S., Oliveira, C. P., Govaere, O., Younes, R., McPherson, S., et al. (2019) · JHEP Reports · doi:10.1016/j.jhepr.2019.06.004 · Cited by 140
Use of HbA1c for diagnoses of diabetes and prediabetes: comparison with diagnoses based on fasting and 2-hr glucose values and effects of gender, race, and age
Guo, F., Moellering, D. R., & Garvey, W. T. (2014) · Metabolic Syndrome and Related Disorders · doi:10.1089/met.2013.0128 · Cited by 139
Enhanced liver fibrosis score predicts transplant-free survival in primary sclerosing cholangitis
Vesterhus, M., Hov, J. R., Holm, A., Schrumpf, E., Nygård, S., Godang, K., Andersen, I. M., Næss, S., Thorburn, D., Saffioti, F., et al. (2015) · Hepatology · doi:10.1002/hep.27825 · Cited by 138
ROC curves for clinical prediction models part 1. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models
Verbakel, J. Y., Steyerberg, E. W., Uno, H., De Cock, B., Wynants, L., Collins, G. S., & Van Calster, B. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.028 · Cited by 138
Direct comparison of US and MR elastography for staging liver fibrosis in patients with nonalcoholic fatty liver disease
Imajo, K., Honda, Y., Kobayashi, T., Nagai, K., Ozaki, A., Iwaki, M., Kessoku, T., Ogawa, Y., Takahashi, H., Saigusa, Y., et al. (2022) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2020.12.016 · Cited by 140
Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: a pilot usability study.
Brennan, M., Puri, S., Ozrazgat-Baslanti, T., Feng, Z., Ruppert, M., Hashemighouchani, H., Momcilovic, P., Li, X., Wang, D. Z., & Bihorac, A. (2019) · Surgery · doi:10.1016/j.surg.2019.01.002 · Cited by 131
Cerebrospinal fluid macrophage biomarkers in amyotrophic lateral sclerosis
Thompson, A. G., Gray, E., Thézénas, M., Charles, P. D., Evetts, S., Hu, M. T., Talbot, K., Fischer, R., Kessler, B. M., & Turner, M. R. (2018) · Annals of Neurology · doi:10.1002/ana.25143 · Cited by 130
Comparing the Areas Under Two Correlated ROC Curves: Parametric and Non‐Parametric Approaches
Molodianovitch, K., Faraggi, D., & Reiser, B. (2006) · Biometrical Journal · doi:10.1002/bimj.200610223 · Cited by 131
Early diagnosis of mild cognitive impairment and mild dementia through basic and instrumental activities of daily living: Development of a new evaluation tool
Cornelis, E., Gorus, E., Beyer, I., Bautmans, I., & De Vriendt, P. (2017) · PLoS Medicine · doi:10.1371/journal.pmed.1002250 · Cited by 133
Predictive accuracy of risk scales following self-harm: multicentre, prospective cohort study
Quinlivan, L., Cooper, J., Meehan, D., Longson, D., Potokar, J., Hulme, T., Marsden, J., Brand, F., Lange, K., Riseborough, E., et al. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.189993 · Cited by 130
Validation of probabilistic predictions
Miller, M. E., Langefeld, C. D., Tierney, W. M., Hui, S. L., & McDonald, C. J. (1993) · Medical Decision Making · doi:10.1177/0272989x9301300107 · Cited by 129
Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin
Sinz, C., Tschandl, P., Rosendahl, C., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J., et al. (2017) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2017.07.022 · Cited by 127
Serum wisteria floribunda agglutinin-positive Mac-2-binding protein level predicts liver fibrosis and prognosis in primary biliary cirrhosis
Umemura, T., Joshita, S., Sekiguchi, T., Usami, Y., Shibata, S., Kimura, T., Komatsu, M., Matsumoto, A., Ota, M., & Tanaka, E. (2015) · American Journal of Gastroenterology · doi:10.1038/ajg.2015.118 · Cited by 127
Deep learning radiomics of ultrasonography can predict response to neoadjuvant chemotherapy in breast cancer at an early stage of treatment: a prospective study
Gu, J., Tong, T., He, C., Xu, M., Yang, X., Tian, J., Jiang, T., & Wang, K. (2022) · European Radiology · doi:10.1007/s00330-021-08293-y · Cited by 126
Combining radiomic features with a miRNA classifier may improve prediction of malignant pathology for pancreatic intraductal papillary mucinous neoplasms
Permuth, J. B., Collaborative, T. F. P., Choi, J., Balarunathan, Y., Kim, J., Chen, D.-T., Chen, L., Orcutt, S., Doepker, M. P., Gage, K., et al. (2016) · Oncotarget · doi:10.18632/oncotarget.11768 · Cited by 127
Effects of Weight Loss, Weight Cycling, and Weight Loss Maintenance on Diabetes Incidence and Change in Cardiometabolic Traits in the Diabetes Prevention Program
Delahanty, L. M., Pan, Q., Jablonski, K. A., Aroda, V. R., Watson, K. E., Bray, G. A., Kahn, S. E., Florez, J. C., Perreault, L., Franks, P. W., et al. (2014) · Diabetes Care · doi:10.2337/dc14-0018 · Cited by 125
AUDIT and AUDIT-C as screening instruments for alcohol problem use in adolescents
Liskola, J., Haravuori, H., Lindberg, N., Niemelä, S., Karlsson, L., Kiviruusu, O., & Marttunen, M. (2018) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2018.04.015 · Cited by 120
Low‐Grade Inflammation in Symptomatic Knee Osteoarthritis: Prognostic Value of Inflammatory Plasma Lipids and Peripheral Blood Leukocyte Biomarkers
Attur, M., Krasnokutsky, S., Statnikov, A., Samuels, J., Li, Z., Friese, O., Hellio Le Graverand‐Gastineau, M., Rybak, L., Kraus, V. B., Jordan, J. M., et al. (2015) · Arthritis & Rheumatology · doi:10.1002/art.39279 · Cited by 120
Neurite orientation dispersion and density imaging (NODDI) and free‐water imaging in Parkinsonism
Mitchell, T., Archer, D. B., Chu, W. T., Coombes, S. A., Lai, S., Wilkes, B. J., McFarland, N. R., Okun, M. S., Black, M. L., Herschel, E., et al. (2019) · Human Brain Mapping · doi:10.1002/hbm.24760 · Cited by 118
Improved marketing decision making in a customer churn prediction context using generalized additive models
Coussement, K., Benoit, D. F., & Van den Poel, D. (2010) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.07.029 · Cited by 117
Depression Screening in Stroke: A Comparison of Alternative Measures With the Structured Diagnostic Interview for the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (Major Depressive Episode) as Criterion Standard
Turner, A., Hambridge, J., White, J., Carter, G., Clover, K., Nelson, L., & Hackett, M. (2012) · Stroke · doi:10.1161/strokeaha.111.643296 · Cited by 114
Classification of Single Normal and Alzheimer’s Disease Individuals from Cortical Sources of Resting State EEG Rhythms
Babiloni, C., Triggiani, A. I., Lizio, R., Cordone, S., Tattoli, G., Bevilacqua, V., Soricelli, A., Ferri, R., Nobili, F., Gesualdo, L., et al. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00047 · Cited by 106
“As a novelty, the present ROC curve analysis indicated that some EEG markers of intra- and inter-hemispherical source connectivity at low-frequency alpha had an AUROC higher than 70%, which is the threshold of a moderate classification rate (DeLong et al., 1988).”
BackgroundMethodInfluential
Learning the natural history of human disease with generative transformers
Shmatko, A., Jung, A. W., Gaurav, K., Brunak, S., Mortensen, L. H., Birney, E., Fitzgerald, T., & Gerstung, M. (2025) · Nature · doi:10.1038/s41586-025-09529-3 · Cited by 110
Biophysical prediction of protein–peptide interactions and signaling networks using machine learning
Cunningham, J. M., Rogers, J. R., Koytiger, G., Sorger, P. K., & AlQuraishi, M. (2020) · Nature Methods · doi:10.1038/s41592-019-0687-1 · Cited by 108
Association of C-Reactive Protein With Cardiovascular Disease Mortality According to Diabetes Status
Kengne, A. P., Batty, G. D., Hamer, M., Stamatakis, E., & Czernichow, S. (2012) · Diabetes Care · doi:10.2337/dc11-1588 · Cited by 105
Multimodal Deep Learning for Integrating Chest Radiographs and Clinical Parameters: A Case for Transformers
Khader, F., Müller-Franzes, G., Wang, T., Han, T., Tayebi Arasteh, S., Haarburger, C., Stegmaier, J., Bressem, K., Kuhl, C., Nebelung, S., et al. (2023) · Radiology · doi:10.1148/radiol.230806 · Cited by 107
Effect of a Deep Learning Framework-Based Computer-Aided Diagnosis System on the Diagnostic Performance of Radiologists in Differentiating between Malignant and Benign Masses on Breast Ultrasonography
Choi, J. S., Han, B.-K., Ko, E. S., Bae, J. M., Ko, E. Y., Song, S. H., Kwon, M.-R., Shin, J. H., & Hahn, S. Y. (2019) · Korean Journal of Radiology · doi:10.3348/kjr.2018.0530 · Cited by 104
PNPLA3 genotype and diabetes identify patients with nonalcoholic fatty liver disease at high risk of incident cirrhosis
Chen, V. L., Oliveri, A., Miller, M. J., Wijarnpreecha, K., Du, X., Chen, Y., Cushing, K. C., Lok, A. S., & Speliotes, E. K. (2023) · Gastroenterology · doi:10.1053/j.gastro.2023.01.040 · Cited by 108
Workplace mental health promotion online to enhance well-being of nurses and allied health professionals: A cluster-randomized controlled trial
Bolier, L., Ketelaar, S. M., Nieuwenhuijsen, K., Smeets, O., Gärtner, F. R., & Sluiter, J. K. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.10.002 · Cited by 98
Association of the novel serologic anti-glycan antibodies anti-laminarin and anti-chitin with complicated Crohnʼs disease behavior
Rieder, F., Schleder, S., Wolf, A., Dirmeier, A., Strauch, U., Obermeier, F., Lopez, R., Spector, L., Fire, E., Yarden, J., et al. (2010) · Inflammatory Bowel Diseases · doi:10.1002/ibd.21046 · Cited by 101
Identifying women who are afraid of giving birth: A comparison of the fear of birth scale with the WDEQ-A in a large Australian cohort
Haines, H. M., Pallant, J. F., Fenwick, J., Gamble, J., Creedy, D. K., Toohill, J., & Hildingsson, I. (2015) · Sexual & Reproductive Healthcare · doi:10.1016/j.srhc.2015.05.002 · Cited by 102
Machine learning algorithms based on signals from a single wearable inertial sensor can detect surface- and age-related differences in walking
Hu, B., Dixon, P. C., Jacobs, J. V., Dennerlein, J. T., & Schiffman, J. M. (2018) · Journal of Biomechanics · doi:10.1016/j.jbiomech.2018.01.005 · Cited by 99
Needle-based confocal laser endomicroscopy of pancreatic cystic lesions: a prospective multicenter validation study in patients with definite diagnosis
Napoleon, B., Palazzo, M., Lemaistre, A.-I., Caillol, F., Palazzo, L., Aubert, A., Buscail, L., Maire, F., Morellon, B. M., Pujol, B., et al. (2019) · Endoscopy · doi:10.1055/a-0732-5356 · Cited by 98
Imbalance-Aware Machine Learning for Predicting Rare and Common Disease-Associated Non-Coding Variants
Schubach, M., Re, M., Robinson, P. N., & Valentini, G. (2017) · Scientific Reports · doi:10.1038/s41598-017-03011-5 · Cited by 97
Network analysis reveals rare disease signatures across multiple levels of biological organization
Buphamalai, P., Kokotovic, T., Nagy, V., & Menche, J. (2021) · Nature Communications · doi:10.1038/s41467-021-26674-1 · Cited by 95
Detection of left ventricular systolic dysfunction from single-lead electrocardiography adapted for portable and wearable devices
Khunte, A., Sangha, V., Oikonomou, E. K., Dhingra, L. S., Aminorroaya, A., Mortazavi, B. J., Coppi, A., Brandt, C. A., Krumholz, H. M., & Khera, R. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00869-w · Cited by 95
Prediction of survival in patients with esophageal carcinoma using artificial neural networks
Sato, F., Shimada, Y., Selaru, F. M., Shibata, D., Maeda, M., Watanabe, G., Mori, Y., Stass, S. A., Imamura, M., & Meltzer, S. J. (2005) · Cancer · doi:10.1002/cncr.20938 · Cited by 94
External validation of prognostic models for critically ill patients required substantial sample sizes
Peek, N., Arts, D. G. T., Bosman, R. J., van der Voort, P. H. J., & de Keizer, N. F. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.08.011 · Cited by 92
Statistical Shape Modeling of the Left Ventricle: Myocardial Infarct Classification Challenge
Suinesiaputra, A., Dhooge, J., Duchateau, N., Ehrhardt, J., Frangi, A. F., Gooya, A., Grau, V., Lekadir, K., Lu, A., Mukhopadhyay, A., et al. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2652449 · Cited by 95
Diagnostic criteria for disorders of arousal: A video-polysomnographic assessment
Lopez, R., Shen, Y., Chenini, S., Rassu, A. L., Evangelista, E., Barateau, L., Jaussent, I., & Dauvilliers, Y. (2018) · Annals of Neurology · doi:10.1002/ana.25153 · Cited by 92
Relation of Direct and Surrogate Measures of Insulin Resistance to Cardiovascular Risk Factors in Nondiabetic Finnish Offspring of Type 2 Diabetic Individuals
Lorenzo, C., Haffner, S. M., Stančáková, A., & Laakso, M. (2010) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2010-1144 · Cited by 92
Relation between umbilical cord blood pH, base deficit, lactate, 5-minute Apgar score and development of hypoxic ischemic encephalopathy
Wiberg, N., Källén, K., Herbst, A., & Olofsson, P. (2010) · Acta Obstetricia Et Gynecologica Scandinavica · doi:10.3109/00016349.2010.513426 · Cited by 92
Multi-Modal Neuroimaging Feature Fusion for Diagnosis of Alzheimer’s Disease
Zhang, T., & Shi, M. (2020) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2020.108795 · Cited by 91
Comparison of machine learning classifiers for influenza detection from emergency department free-text reports
López Pineda, A., Ye, Y., Visweswaran, S., Cooper, G. F., Wagner, M. M., & Tsui, F. (. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.08.019 · Cited by 88
Blockchain for Privacy Preserving and Trustworthy Distributed Machine Learning in Multicentric Medical Imaging (C-DistriM)
Zerka, F., Urovi, V., Vaidyanathan, A., Barakat, S., Leijenaar, R. T. H., Walsh, S., Gabrani-Juma, H., Miraglio, B., Woodruff, H. C., Dumontier, M., et al. (2020) · IEEE Access · doi:10.1109/access.2020.3029445 · Cited by 89
Glycemic variability determined with a continuous glucose monitoring system can predict prognosis after acute coronary syndrome
Takahashi, H., Iwahashi, N., Kirigaya, J., Kataoka, S., Minamimoto, Y., Gohbara, M., Abe, T., Okada, K., Matsuzawa, Y., Konishi, M., et al. (2018) · Cardiovascular Diabetology · doi:10.1186/s12933-018-0761-5 · Cited by 86
Clinical neuropathy scales in neuropathy associated with impaired glucose tolerance
Zilliox, L. A., Ruby, S. K., Singh, S., Zhan, M., & Russell, J. W. (2015) · Journal of Diabetes and its Complications · doi:10.1016/j.jdiacomp.2015.01.011 · Cited by 85
The combined role of serotonin and interleukin-6 as biomarker for autism
Yang, C.-J., Liu, C.-L., Sang, B., Zhu, X.-M., & Du, Y.-J. (2015) · Neuroscience · doi:10.1016/j.neuroscience.2014.10.011 · Cited by 80
First-trimester placental ultrasound and maternal serum markers as predictors of small-for-gestational-age infants
Schwartz, N., Sammel, M. D., Leite, R., & Parry, S. (2014) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2014.02.033 · Cited by 80
Radiomics analysis for the differentiation of autoimmune pancreatitis and pancreatic ductal adenocarcinoma in 18 F‐FDG PET/CT
Zhang, Y., Cheng, C., Liu, Z., Wang, L., Pan, G., Sun, G., Chang, Y., Zuo, C., & Yang, X. (2019) · Medical Physics · doi:10.1002/mp.13733 · Cited by 80
Development and Validation of a Mortality Prediction Model for Patients Receiving 14 Days of Mechanical Ventilation
Hough, C. L., Caldwell, E. S., Cox, C. E., Douglas, I. S., Kahn, J. M., White, D. B., Seeley, E. J., Bangdiwala, S. I., Rubenfeld, G. D., Angus, D. C., et al. (2015) · Critical Care Medicine · doi:10.1097/ccm.0000000000001205 · Cited by 79
Transcription factor induction of human oligodendrocyte progenitor fate and differentiation
Wang, J., Pol, S. U., Haberman, A. K., Wang, C., O’Bara, M. A., & Sim, F. J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1408295111 · Cited by 79
A virtual chromoendoscopy artificial intelligence system to detect endoscopic and histologic activity/remission and predict clinical outcomes in ulcerative colitis
Cannatelli, R., Parigi, T. L., Iacucci, M., Nardone, O. M., Tontini, G. E., Labarile, N., Buda, A., Rimondi, A., Bazarova, A., Bisschops, R., et al. (2023) · Endoscopy · doi:10.1055/a-1960-3645 · Cited by 80
Concomitant medication use can confound interpretation of the combined dexamethasone-corticotropin releasing hormone test in Cushing's syndrome.
Valassi, E., Swearingen, B., Lee, H., Nachtigall, L. B., Donoho, D. A., Klibanski, A., & Biller, B. M. K. (2009) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2009-1500 · Cited by 78
Comparison of Three Screening Methods for Sarcopenia in Community-Dwelling Older Persons
Mo, Y.-H., Zhong, J., Dong, X., Su, Y.-D., Deng, W.-Y., Yao, X.-M., Liu, B.-B., Wang, X.-Q., & Wang, X.-H. (2021) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2020.05.041 · Cited by 73
Prediction models for incident Type 2 diabetes mellitus in the older population: KORA S4/F4 cohort study
Rathmann, W., Kowall, B., Heier, M., Herder, C., Holle, R., Thorand, B., Strassburger, K., Peters, A., Wichmann, H. ‐., Giani, G., et al. (2010) · Diabetic Medicine · doi:10.1111/j.1464-5491.2010.03065.x · Cited by 74
Tumour gene expression signature in primary melanoma predicts long-term outcomes
Garg, M., Couturier, D.-L., Nsengimana, J., Fonseca, N. A., Wongchenko, M., Yan, Y., Lauss, M., Jönsson, G. B., Newton-Bishop, J., Parkinson, C., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-21207-2 · Cited by 70
Base deficit estimation in umbilical cord blood is influenced by gestational age, choice of fetal fluid compartment, and algorithm for calculation
Wiberg, N., Källén, K., & Olofsson, P. (2006) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2006.05.043 · Cited by 69
Identifying pulmonary nodules or masses on chest radiography using deep learning: external validation and strategies to improve clinical practice
Liang, C.-H., Liu, Y.-C., Wu, M.-T., Garcia-Castro, F., Alberich-Bayarri, A., & Wu, F.-Z. (2020) · Clinical Radiology · doi:10.1016/j.crad.2019.08.005 · Cited by 69
Role of early second-trimester uterine artery Doppler screening to predict small-for-gestational-age babies in nulliparous women
Parry, S., Sciscione, A., Haas, D. M., Grobman, W. A., Iams, J. D., Mercer, B. M., Silver, R. M., Simhan, H. N., Wapner, R. J., Wing, D. A., et al. (2017) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2017.06.013 · Cited by 66
Development of PancRISK, a urine biomarker-based risk score for stratified screening of pancreatic cancer patients
Blyuss, O., Zaikin, A., Cherepanova, V., Munblit, D., Kiseleva, E. M., Prytomanova, O. M., Duffy, S. W., & Crnogorac-Jurcevic, T. (2020) · British Journal of Cancer · doi:10.1038/s41416-019-0694-0 · Cited by 67
Intestinal ischemia after cardiac surgery: analysis of a large registry
Nilsson, J., Hansson, E., & Andersson, B. (2013) · Journal of Cardiothoracic Surgery · doi:10.1186/1749-8090-8-156 · Cited by 60
Haemoglobin A1c even within non-diabetic level is a predictor of cardiovascular disease in a general Japanese population: the Hisayama Study
Ikeda, F., Doi, Y., Ninomiya, T., Hirakawa, Y., Mukai, N., Hata, J., Shikata, K., Yoshida, D., Matsumoto, T., Kitazono, T., et al. (2013) · Cardiovascular Diabetology · doi:10.1186/1475-2840-12-164 · Cited by 67
Weak correlation between antibody titers and neutralizing activity in sera from SARS-CoV-2 infected subjects
Criscuolo, E., Diotti, R. A., Strollo, M., Rolla, S., Ambrosi, A., Locatelli, M., Burioni, R., Mancini, N., Clementi, M., & Clementi, N. (2021) · Journal of Medical Virology · doi:10.1002/jmv.26605 · Cited by 62
Regional correlations between [11C]PIB PET and post-mortem burden of amyloid-beta pathology in a diverse neuropathological cohort
Seo, S. W., Ayakta, N., Grinberg, L. T., Villeneuve, S., Lehmann, M., Reed, B., DeCarli, C., Miller, B. L., Rosen, H. J., Boxer, A. L., et al. (2017) · NeuroImage: Clinical · doi:10.1016/j.nicl.2016.11.008 · Cited by 63
Use of alternative thresholds defining insulin resistance to predict incident type 2 diabetes mellitus and cardiovascular disease
Rutter, M. K., Wilson, P. W. F., Sullivan, L. M., Fox, C. S., D’Agostino, R. B., & Meigs, J. B. (2008) · Circulation · doi:10.1161/circulationaha.107.727727 · Cited by 64
Generating and evaluating cross‐sectional synthetic electronic healthcare data: Preserving data utility and patient privacy
Wang, Z., Myles, P., & Tucker, A. (2021) · Computational Intelligence · doi:10.1111/coin.12427 · Cited by 65
Machine-learning-based prediction models for high-need high-cost patients using nationwide clinical and claims data
Osawa, I., Goto, T., Yamamoto, Y., & Tsugawa, Y. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00354-8 · Cited by 66
Effectiveness of arterial, venous, and capillary blood lactate as a sepsis triage tool in ED patients
Contenti, J., Corraze, H., Lemoël, F., & Levraut, J. (2015) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2014.11.003 · Cited by 60
Maternal and paternal depression symptoms during NICU stay and transition home
Garfield, C. F., Lee, Y. S., Warner-Shifflett, L., Christie, R., Jackson, K. L., & Miller, E. (2021) · Pediatrics · doi:10.1542/peds.2020-042747 · Cited by 63
Comparative analytical performance of multiple plasma Aβ42 and Aβ40 assays and their ability to predict positron emission tomography amyloid positivity
Zicha, S., Bateman, R. J., Shaw, L. M., Zetterberg, H., Bannon, A. W., Horton, W. A., Baratta, M., Kolb, H. C., Dobler, I., Mordashova, Y., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.12697 · Cited by 58
Evaluating the reliability of neurocognitive biomarkers of neurodegenerative diseases across countries: A machine learning approach
Bachli, M. B., Sedeño, L., Ochab, J. K., Piguet, O., Kumfor, F., Reyes, P., Torralva, T., Roca, M., Cardona, J. F., Campo, C. G., et al. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116456 · Cited by 57
Evaluation of a Deep Learning System For Identifying Glaucomatous Optic Neuropathy Based on Color Fundus Photographs
Al-Aswad, L. A., Kapoor, R., Chu, C. K., Walters, S., Gong, D., Garg, A., Gopal, K., Patel, V., Sameer, T., Rogers, T. W., et al. (2019) · Journal of Glaucoma · doi:10.1097/ijg.0000000000001319 · Cited by 59
Deep-Learning Approaches to Identify Critically Ill Patients at Emergency Department Triage Using Limited Information
Joseph, J. W., Leventhal, E. L., Grossestreuer, A. V., Wong, M. L., Joseph, L. J., Nathanson, L. A., Donnino, M. W., Elhadad, N., & Sanchez, L. D. (2020) · doi:10.1101/2020.05.02.20089052 · Cited by 57
Circulating fatty acids and risk of gestational diabetes mellitus: prospective analyses in China
Pan, X.-F., Huang, Y., Li, X., Wang, Y., Ye, Y., Chen, H., Marklund, M., Wen, Y., Liu, Y., Zeng, H., et al. (2021) · European Journal of Endocrinology · doi:10.1530/eje-21-0118 · Cited by 57
Machine learning methods to improve bedside fluid responsiveness prediction in severe sepsis or septic shock: An observational study
Bataille, B., de Selle, J., Moussot, P.-E., Marty, P., Silva, S., & Cocquet, P. (2021) · British Journal of Anaesthesia · doi:10.1016/j.bja.2020.11.039 · Cited by 56
Comparing the Hadlock fetal growth standard to the Eunice Kennedy Shriver National Institute of Child Health and Human Development racial/ethnic standard for the prediction of neonatal morbidity and small for gestational age
Blue, N. R., Beddow, M. E., Savabi, M., Katukuri, V. R., & Chao, C. R. (2018) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2018.08.011 · Cited by 54
Early acetaminophen-protein adducts predict hepatotoxicity following overdose (ATOM-5)
Chiew, A. L., James, L. P., Isbister, G. K., Pickering, J. W., McArdle, K., Chan, B. S. H., & Buckley, N. A. (2020) · Journal of Hepatology · doi:10.1016/j.jhep.2019.10.030 · Cited by 53

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks