Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 401–500 of 1,197

Predicting global habitat suitability for stony corals on seamounts
Tittensor, D. P., Baco, A. R., Brewin, P. E., Clark, M. R., Consalvey, M., Hall-Spencer, J. M., Rowden, A. A., Schlacher, T. A., Stocks, K. I., & Rogers, A. D. (2009) · Journal of Biogeography · doi:10.1111/j.1365-2699.2008.02062.x · Cited by 307
Physiological Risk Factors for Severe High-Altitude Illness: A Prospective Cohort Study
Richalet, J., Larmignat, P., Poitrine, E., Letournel, M., & Canouï‐Poitrine, F. (2011) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201108-1396oc · Cited by 307
Estimation and Comparison of Receiver Operating Characteristic Curves
PEPE, M. S., Longton, G., & Janes, H. (2009) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900101 · Cited by 306
Validity of the mental health component scale of the 12-item Short-Form Health Survey (MCS-12) as measure of common mental disorders in the general population
Olesen, S. C., Butterworth, P., Rodgers, B., & Mackinnon, A. J. (2007) · Psychiatry Research · doi:10.1016/j.psychres.2006.11.005 · Cited by 314
Impact of the Clinical Frailty Scale on Outcomes After Transcatheter Aortic Valve Replacement
Shimura, T., Yamamoto, M., Kano, S., Kagase, A., Kodama, A., Koyama, Y., Tsuchikane, E., Suzuki, T., Otsuka, T., Kohsaka, S., et al. (2017) · Circulation · doi:10.1161/circulationaha.116.025630 · Cited by 308
Phenomapping of Patients with Heart Failure with Preserved Ejection Fraction Using Machine Learning-Based Unsupervised Cluster Analysis
Segar, M. W., Patel, K. V., Ayers, C. R., Basit, M. A., Tang, W. H. W., Willett, D. L., Berry, J. D., Grodin, J. L., & Pandey, A. (2019) · European Journal of Heart Failure · doi:10.1002/ejhf.1621 · Cited by 308
Concurrent Validity of the Child Behavior Checklist DSM-Oriented Scales: Correspondence with DSM Diagnoses and Comparison to Syndrome Scales
The Research Network on Youth Mental Health, Ebesutani, C., Bernstein, A., Nakamura, B. J., Chorpita, B. F., Higa-McMillan, C. K., & Weisz, J. R. (2010) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-009-9174-9 · Cited by 303
Early Invasive Cervical Cancer: Tumor Delineation by Magnetic Resonance Imaging, Computed Tomography, and Clinical Examination, Verified by Pathologic Results, in the ACRIN 6651/GOG 183 Intergroup Study
Mitchell, D. G., Snyder, B. S., Coakley, F. V., Reinhold, C., Thomas, G., Amendola, M. A., Schwartz, L. H., Woodward, P. J., Pannu, H. K., & Hricak, H. (2006) · Journal of Clinical Oncology · doi:10.1200/jco.2006.07.4799 · Cited by 303
The Prostate Health Index Selectively Identifies Clinically Significant Prostate Cancer
Loeb, S., Sanda, M. G., Broyles, D. L., Shin, S. S., Bangma, C. H., Wei, J. T., Partin, A. W., Klee, G. G., Slawin, K. M., Marks, L. S., et al. (2014) · The Journal of Urology · doi:10.1016/j.juro.2014.10.121 · Cited by 302
Use of the nicotine metabolite ratio as a genetically informed biomarker of response to nicotine patch or varenicline for smoking cessation: a randomised, double-blind placebo-controlled trial
Lerman, C. E., Schnoll, R. A., Hawk, L. W., Cinciripini, P. M., George, T. P., Wileyto, E. P., Swan, G. E., Benowitz, N. L., Heitjan, D. F., & Tyndale, R. F. (2015) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(14)70294-2 · Cited by 302
Transfer learning with deep convolutional neural network for liver steatosis assessment in ultrasound images
Byra, M., Styczyński, G., Szmigielski, C. A., Kalinowski, P., Michałowski, Ł., Paluszkiewicz, R., Ziarkiewicz‐Wróblewska, B., Zieniewicz, K., Sobieraj, P., & Nowicki, A. (2018) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-018-1843-2 · Cited by 314
Diagnosing ICD-10 Depressive Episodes: Superior Criterion Validity of the Patient Health Questionnaire
Löwe, B., Gräfe, K., Zipfel, S., Witte, S., Loerch, B., & Herzog, W. (2004) · Psychotherapy and Psychosomatics · doi:10.1159/000080393 · Cited by 301
Kidney Function as a Predictor of Noncardiovascular Mortality
Fried, L. F., Katz, R., Sarnak, M. J., Shlipak, M. G., Chaves, P. H. M., Jenny, N. S., Stehman‐Breen, C. O., Gillen, D., Bleyer, A. J., Hirsch, C. H., et al. (2005) · Journal of the American Society of Nephrology · doi:10.1681/asn.2005040384 · Cited by 301
Machine Learning to Predict the Risk of Incident Heart Failure Hospitalization Among Patients With Diabetes: The WATCH-DM Risk Score
Segar, M. W., Vaduganathan, M., Patel, K. V., McGuire, D. K., Butler, J., Fonarow, G. C., Basit, M. A., Kannan, V., Grodin, J. L., Everett, B. M., et al. (2019) · Diabetes Care · doi:10.2337/dc19-0587 · Cited by 307
A 7 gene signature identifies the risk of developing cirrhosis in patients with chronic hepatitis C†
Huang, H., Shiffman, M. L., Friedman, S. L., Venkatesh, R., Bzowej, N. H., Abar, O. T., Rowland, C. M., Catanese, J. J., Leong, D. U., Sninsky, J. J., et al. (2007) · Hepatology · doi:10.1002/hep.21695 · Cited by 300
Plasma proteomic profiles predict future dementia in healthy adults
Guo, Y., You, J., Zhang, Y., Liu, W., Huang, Y., Zhang, Y.-R., Zhang, W., Dong, Q., Feng, J., Cheng, W., et al. (2024) · Nature Aging · doi:10.1038/s43587-023-00565-0 · Cited by 327
Risk stratification of smoldering multiple myeloma incorporating revised IMWG diagnostic criteria
Lakshman, A., Rajkumar, S. V., Buadi, F. K., Binder, M., Gertz, M. A., Lacy, M. Q., Dispenzieri, A. A., Dingli, D., Fonder, A. L., Hayman, S. R., et al. (2018) · Blood Cancer Journal · doi:10.1038/s41408-018-0077-4 · Cited by 299
Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection.
Liu, F., Zhou, Z., Samsonov, A., Blankenbaker, D., Larison, W., Kanarek, A., Lian, K., Kambhampati, S., & Kijowski, R. (2018) · Radiology · doi:10.1148/radiol.2018172986 · Cited by 297
“For the ROC analysis, the area under the ROC curve (AUC) was calculated for each evaluation of the cartilage lesion detection system, with the AUCs obtained during the two evaluations compared by using a nonparametric approach (36).”
Diagnostic value of biochemical markers (NashTest) for the prediction of non alcoholo steato hepatitis in patients with non-alcoholic fatty liver disease
the LIDO Study Group and the CYTOL study group, Poynard, T., Ratziu, V., Charlotte, F., Messous, D., Munteanu, M., Imbert‐Bismut, F., Massard, J., Bonyhay, L., Tahiri, M., et al. (2006) · BMC Gastroenterology · doi:10.1186/1471-230x-6-34 · Cited by 302
Predictive models for diabetes mellitus using machine learning techniques
Lai, H., Huang, H., Keshavjee, K., Guergachi, A., & Gao, X. (2019) · BMC Endocrine Disorders · doi:10.1186/s12902-019-0436-6 · Cited by 297
Mass spectrometric profiling of oxidized lipid products in human nonalcoholic fatty liver disease and nonalcoholic steatohepatitis
Feldstein, A. E., López, R., Tamimi, T., Yerian, L. M., Chung, Y.-M., Berk, M. P., Zhang, R., McIntyre, T. M., & Hazen, S. L. (2010) · Journal of Lipid Research · doi:10.1194/jlr.m007096 · Cited by 301
Strain Echocardiography Improves Risk Prediction of Ventricular Arrhythmias After Myocardial Infarction
Haugaa, K. H., Grenne, B. L., Eek, C. H., Ersbøll, M. K., Valeur, N., Svendsen, J. H., Florian, A. R., Sjøli, B., Brunvand, H., Køber, L., et al. (2013) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2013.03.005 · Cited by 295
Blood‐brain barrier breakdown, neuroinflammation, and cognitive decline in older adults
Bowman, G. L., Dayon, L., Kirkland, R. E., Wojcik, J., Peyratout, G., Severin, I. C., Henry, H., Oikonomidi, A., Migliavacca, E., Bacher, M., et al. (2018) · Alzheimer s & Dementia · doi:10.1016/j.jalz.2018.06.2857 · Cited by 307
Accuracy of Anthropometric Indicators of Obesity to Predict Cardiovascular Risk
Schneider, H. J., Glaesmer, H., Klotsche, J., Böhler, S., Lehnert, H., Zeiher, A. M., März, W., Pittrow, D. B., Stalla, G. K., Wïttchen, H., et al. (2006) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2006-0254 · Cited by 294
Comparison of Machine Learning Methods With Traditional Models for Use of Administrative Claims With Electronic Medical Records to Predict Heart Failure Outcomes
Desai, R. J., Wang, S. V., Vaduganathan, M., Evers, T., & Schneeweiß, S. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.18962 · Cited by 315
Growth differentiation factor 15 as a useful biomarker for mitochondrial disorders
Yatsuga, S., Fujita, Y., Ishii, A., Fukumoto, Y., Arahata, H., Kakuma, T., Kojima, T., Ito, M., Tanaka, M., Saiki, R., et al. (2015) · Annals of Neurology · doi:10.1002/ana.24506 · Cited by 293
A New Severity of Illness Scale Using a Subset of Acute Physiology and Chronic Health Evaluation Data Elements Shows Comparable Predictive Accuracy*
Johnson, A. E. W., Kramer, A. A., & Clifford, G. D. (2013) · Critical Care Medicine · doi:10.1097/ccm.0b013e31828a24fe · Cited by 292
Predicting the Presence and Side of Extracapsular Extension: A Nomogram for Staging Prostate Cancer
Ohori, M., Kattan, M. W., Koh, H., Maru, N., Slawin, K. M., Shariat, S. F., Muramoto, M., Reuter, V. E., Wheeler, T. M., & Scardino, P. T. (2004) · The Journal of Urology · doi:10.1097/01.ju.0000121693.05077.3d · Cited by 292
Diagnostic and prognostic values of noninvasive biomarkers of fibrosis in patients with alcoholic liver disease†
Naveau, S., Gaudé, G., Asnacios, A., Agostini, H., Abella, A., Barri‐Ova, N., Dauvois, B., Prevot, S., Ngo, Y., Munteanu, M., et al. (2008) · Hepatology · doi:10.1002/hep.22576 · Cited by 292
GALAD Score Detects Early Hepatocellular Carcinoma in an International Cohort of Patients With Nonalcoholic Steatohepatitis
Best, J., Bechmann, L. P., Sowa, J., Sydor, S., Dechêne, A., Pflanz, K., Bedreli, S., Schotten, C., Geier, A., Berg, T., et al. (2019) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2019.11.012 · Cited by 296
Child-Pugh versus MELD score in predicting survival in patients undergoing transjugular intrahepatic portosystemic shunt
Angermayr, B., Cejna, M., Karnel, F., Gschwantler, M., König, F., Pidlich, J., Mendel, H., Pichler, L., Wichlas, M., Kreil, A., et al. (2003) · Gut · doi:10.1136/gut.52.6.879 · Cited by 291
Pediatric Metabolic Syndrome Predicts Adulthood Metabolic Syndrome, Subclinical Atherosclerosis, and Type 2 Diabetes Mellitus – But Is No Better Than Body Mass Index Alone: The Bogalusa Heart Study and the Cardiovascular Risk in Young Finns Study
Magnussen, C., Koskinen, J., Chen, W., Thomson, R., Schmidt, M., Srinivasan, S., Kivimäki, M., Mattsson, N., Kähönen, M., Laitinen, T., et al. (2010) · Circulation · doi:10.1161/circulationaha.110.940809 · Cited by 304
Biomarkers Predict Progression of Acute Kidney Injury after Cardiac Surgery
Koyner, J. L., Garg, A. X., Coca, S. G., Sint, K., Thiessen‐Philbrook, H., Patel, U. D., Shlipak, M. G., & Parikh, C. R. (2012) · Journal of the American Society of Nephrology · doi:10.1681/asn.2011090907 · Cited by 290
Early Adult Risk Factor Levels and Subsequent Coronary Artery Calcification
Loria, C. M., Liu, K., Lewis, C. E., Hulley, S. B., Sidney, S. S., Schreiner, P. J., Williams, O. D., Bild, D. E., & Detrano, R. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.03.009 · Cited by 288
Significance of sperm characteristics in the evaluation of male infertility
Nallella, K. P., Sharma, R. K., Aziz, N., & Agarwal, A. (2006) · Fertility and Sterility · doi:10.1016/j.fertnstert.2005.08.024 · Cited by 288
The triglyceride and glucose index (TyG) is an effective biomarker to identify nonalcoholic fatty liver disease
Zhang, S., Du, T., Zhang, J., Lu, H., Lin, X., Xie, J., Yang, Y., & Yu, X. (2017) · Lipids in Health and Disease · doi:10.1186/s12944-017-0409-6 · Cited by 288
Projection-Resolved Optical Coherence Tomography Angiography of Macular Retinal Circulation in Glaucoma
Takusagawa, H. L., Liu, L., Ma, K. N., Jia, Y., Gao, S. S., Zhang, M., Edmunds, B., Parikh, M., Tehrani, S., Morrison, J. C., et al. (2017) · Ophthalmology · doi:10.1016/j.ophtha.2017.06.002 · Cited by 288
Toward robust mammography-based models for breast cancer risk
Yala, A., Mikhael, P. G., Strand, F., Lin, G., Smith, K., Wan, Y.-L., Lamb, L., Hughes, K., Lehman, C., & Barzilay, R. (2021) · Science Translational Medicine · doi:10.1126/scitranslmed.aba4373 · Cited by 287
Early diagnosis of acute myocardial infarction in the elderly using more sensitive cardiac troponin assays
Reiter, M., Twerenbold, R., Reichlin, T., Haaf, P., Peter, F., Meissner, J., Hochholzer, W., Stelzig, C., Freese, M., Heinisch, C., et al. (2011) · European Heart Journal · doi:10.1093/eurheartj/ehr033 · Cited by 287
Serodiagnosis of Lyme Disease by Kinetic Enzyme‐Linked Immunosorbent Assay Using Recombinant VlsE1 or Peptide Antigens ofBorrelia burgdorferiCompared with 2‐Tiered Testing Using Whole‐Cell Lysates
Bacon, R. M., Biggerstaff, B. J., Schriefer, M. E., Gilmore, R. D., Philipp, M. T., Steere, A. C., Wormser, G. P., Marques, A. R., & Johnson, B. J. B. (2003) · The Journal of Infectious Diseases · doi:10.1086/374395 · Cited by 287
Diffusion-weighted MRI Findings Predict Pathologic Response in Neoadjuvant Treatment of Breast Cancer: The ACRIN 6698 Multicenter Trial
Partridge, S. C., Zhang, Z., Newitt, D. C., Gibbs, J., Chenevert, T. L., Rosen, M. A., Bolan, P. J., Marques, H. S., Romanoff, J., Cimino, L., et al. (2018) · Radiology · doi:10.1148/radiol.2018180273 · Cited by 287
Discrimination and Net Reclassification of Cardiovascular Risk With Lipoprotein(a)
Willeit, P., Kiechl, S., Kronenberg, F., Witztum, J. L., Santer, P., Mayr, M., Xu, Q., Mayr, A., Willeit, J., & Tsimikas, S. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2014.03.061 · Cited by 286
A Predictive Model Identifies Patients Most Likely to Have Inadequate Bowel Preparation for Colonoscopy
Hassan, C., Fuccio, L., Bruno, M., Pagano, N., Spada, C., Carrara, S., Giordanino, C., Rondonotti, E., Curcio, G., Dulbecco, P., et al. (2012) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2011.12.037 · Cited by 286
Pulmonary capillary wedge pressure during exercise and long-term mortality in patients with suspected heart failure with preserved ejection fraction
Dorfs, S., Zeh, W., Hochholzer, W., Jander, N., Kienzle, R.-P., Pieske, B. M., & Neumann, F. (2014) · European Heart Journal · doi:10.1093/eurheartj/ehu315 · Cited by 286
Cardiovascular magnetic resonance by non contrast T1-mapping allows assessment of severity of injury in acute myocardial infarction
Dall’Armellina, E., Piechnik, S. K., Ferreira, V. M., Le, S. Q., Robson, M. D., Francis, J. M., Cuculi, F., Kharbanda, R. K., BANNING, A. P., Choudhury, R. P., et al. (2012) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-14-15 · Cited by 286
Validation of a Claims-Based Frailty Index Against Physical Performance and Adverse Health Outcomes in the Health and Retirement Study
Kim, D. H., Glynn, R. J., Avorn, J., Lipsitz, L. A., Rockwood, K., Pawar, A., & Schneeweiss, S. (2019) · The Journals of Gerontology: Series A · doi:10.1093/gerona/gly197 · Cited by 285
Machine Learning–Based Prediction of Clinical Outcomes for Children During Emergency Department Triage
Goto, T., Camargo, C. A., Faridi, M. K., Freishtat, R. J., & Hasegawa, K. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2018.6937 · Cited by 285
Frailty as a Predictor of Death or New Disability After Surgery
McIsaac, D. I., Taljaard, M., Bryson, G. L., Beaulé, P. E., Gagné, S., Hamilton, G. M., Hladkowicz, E., Huang, A. R., Joanisse, J. A., Lavallée, L. T., et al. (2018) · Annals of Surgery · doi:10.1097/sla.0000000000002967 · Cited by 293
Furosemide Stress Test and Biomarkers for the Prediction of AKI Severity
Koyner, J. L., Davison, D. L., Brasha-Mitchell, E., Chalikonda, D. M., Arthur, J. M., Shaw, A., Tumlin, J. A., Trevino, S. A., BENNETT, M. R., Kimmel, P. L., et al. (2015) · Journal of the American Society of Nephrology · doi:10.1681/asn.2014060535 · Cited by 283
Genome-wide mapping of 5-hydroxymethylcytosines in circulating cell-free DNA as a non-invasive approach for early detection of hepatocellular carcinoma
Cai, J., Chen, L., Zhang, Z., Zhang, X., Lu, X., Liu, W., Shi, G., Ge, Y., Gao, P., Yang, Y., et al. (2019) · Gut · doi:10.1136/gutjnl-2019-318882 · Cited by 283
Liver stiffness-based optimization of hepatocellular carcinoma risk score in patients with chronic hepatitis B
Wong, G. L., Chan, H. L., Wong, C. K., Leung, C. K.-L., Chan, C. Y., Ho, P. P., Chung, V. C., Chan, Z. C., Tse, Y., Chim, A. M., et al. (2013) · Journal of Hepatology · doi:10.1016/j.jhep.2013.09.029 · Cited by 281
CDT, GGT, and AST As Markers of Alcohol Use: The WHO/ISBRA Collaborative Project
Conigrave, K. M., Degenhardt, L. J., Whitfield, J. B., Saunders, J. B., Helander, A., Tabakoff, B., & on behalf of the WHO/ISBRA Study Group (2002) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2002.tb02542.x · Cited by 281
Prediction of Heart Failure Mortality in Emergent Care
Lee, D. S., Stitt, A., Austin, P. C., Stukel, T. A., Schull, M. J., Chong, A. S., Newton, G. E., Lee, J. S., & Tu, J. V. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-11-201206050-00003 · Cited by 286
Definition of iron deficiency based on the gold standard of bone marrow iron staining in heart failure patients
Grote Beverborg, N., Klip, I. T., Meijers, W. C., Voors, A. A., Vegter, E. L., van der Wal, H. H., Swinkels, D. W., van Pelt, J., Mulder, A. B., Bulstra, S. K., et al. (2018) · Circulation Heart Failure · doi:10.1161/circheartfailure.117.004519 · Cited by 278
Federated deep learning for detecting COVID-19 lung abnormalities in CT: a privacy-preserving multinational validation study
Dou, Q., So, T. Y., Jiang, M., Liu, Q., Vardhanabhuti, V., Kaissis, G., Li, Z., Si, W., Lee, H. H. C., Yu, K., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00431-6 · Cited by 278
Comparison of 19 pre-operative risk stratification models in open-heart surgery
Nilsson, J., Algotsson, L., Höglund, P., Lührs, C., & Brandt, J. (2006) · European Heart Journal · doi:10.1093/eurheartj/ehi720 · Cited by 278
Detection of Hemodynamically Significant Coronary Artery Stenosis: Incremental Diagnostic Value of Dynamic CT-based Myocardial Perfusion Imaging
Bamberg, F., Becker, A., Schwarz, F., Marcus, R. P., Greif, M., von Ziegler, F., Blankstein, R., Hoffmann, U., Sommer, W. H., Hoffmann, V. S., et al. (2011) · Radiology · doi:10.1148/radiol.11110638 · Cited by 277
Derivation and validation of cutoffs for clinical use of cell cycle arrest biomarkers
Hoste, E. A. J., McCullough, P. A., KASHANI, K. B., Chawla, L. S., Joannidis, M., Shaw, A., Feldkamp, T., Uettwiller-Geiger, D. L., McCarthy, P., Shi, J., et al. (2014) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfu292 · Cited by 277
High Prevalence of Liver Fibrosis Among European Adults With Unknown Liver Disease: A Population-Based Study
Caballería, L., Pera, G., Arteaga, Í., Rodríguez, L., Alumà, A., Morillas, R. M., de la Ossa, N., Díaz, A., Expósito, C., Miranda, D., et al. (2018) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2017.12.048 · Cited by 276
Linear Combinations of Multiple Diagnostic Markers
Su, J. Q., & Liu, J. S. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476417 · Cited by 275
Improved performance of urinary biomarkers of acute kidney injury in the critically ill by stratification for injury duration and baseline renal function
Endre, Z. H., Pickering, J. W., Walker, R. J., Devarajan, P., Edelstein, C. L., Bonventre, J. V., FRAMPTON, C. M. A., BENNETT, M. R., Ma, Q., Sabbisetti, V. S., et al. (2011) · Kidney International · doi:10.1038/ki.2010.555 · Cited by 275
Multiparametric plasma EV profiling facilitates diagnosis of pancreatic malignancy
Yang, K. S., Im, H., Hong, S., Pergolini, I., del Castillo, A. F., Wang, R., Clardy, S. M., Huang, C.-H., Pille, C., Ferrone, S., et al. (2017) · Science Translational Medicine · doi:10.1126/scitranslmed.aal3226 · Cited by 280
Defining ADHD symptom persistence in adulthood: optimizing sensitivity and specificity
Sibley, M. H., Swanson, J. M., Arnold, L. E., Hechtman, L. T., Owens, E. B., Stehli, A., Abikoff, H., Hinshaw, S. P., Molina, B. S. G., Mitchell, J. T., et al. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12620 · Cited by 274
Can Urinary PCA3 Supplement PSA in the Early Detection of Prostate Cancer?
Wei, J. T., Feng, Z., Partin, A. W., Brown, E. C., Thompson, I. M., Sokoll, L. J., Chan, D. W., Lotan, Y., Kibel, A. S., Busby, J. E., et al. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2013.52.8505 · Cited by 274
Serum Macrophage Inhibitory Cytokine 1 as a Marker of Pancreatic and Other Periampullary Cancers
Koopmann, J., Buckhaults, P., Brown, D. A., ZAHURAK, M. L., Sato, N., Fukushima, N., Sokoll, L. J., Chan, D. W., Yeo, C. J., Hruban, R. H., et al. (2004) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-03-0165 · Cited by 274
Risk Factors for Perioperative Acute Kidney Injury After Adult Cardiac Surgery: Role of Perioperative Management
Parolari, A., Pesce, L. L., Pacini, D., Mazzanti, V., Salis, S., Sciacovelli, C., Rossi, F., & Alamanni, F. (2012) · The Annals of Thoracic Surgery · doi:10.1016/j.athoracsur.2011.09.073 · Cited by 272
Noninvasive Fractional Flow Reserve Derived From Computed Tomography Angiography for Coronary Lesions of Intermediate Stenosis Severity
Nakazato, R., Park, H., Berman, D. S., Gransar, H., Koo, B., Ērglis, A., Lin, F. Y.-H., Dunning, A. M., Budoff, M. J., Malpeso, J. M., et al. (2013) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.113.000297 · Cited by 272
Using the concept of Chou's pseudo amino acid composition for risk type prediction of human papillomaviruses
Esmaeili, M., Mohabatkar, H., & Mohsenzadeh, S. (2009) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2009.11.016 · Cited by 271
Predicting tree species presence and basal area in Utah: A comparison of stochastic gradient boosting, generalized additive models, and tree-based methods
Moisen, G. G., Freeman, E. A., Blackard, J. A., Frescino, T. S., Zimmermann, N. E., & Edwards, T. C. (2006) · Ecological Modelling · doi:10.1016/j.ecolmodel.2006.05.021 · Cited by 275
Prognostic Value of 64-Slice Cardiac Computed Tomography
Chow, B. J. W., Wells, G. A., Chen, L., Yam, Y., Galiwango, P. J., Abraham, A., Sheth, T., Dennie, C. J., Beanlands, R. S., & Ruddy, T. D. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.10.039 · Cited by 273
Predictors of Complicated Grief Among Dementia Caregivers: A Prospective Study of Bereavement
Schulz, R., Boerner, K., Shear, K. M., Zhang, S., & Gitlin, L. N. (2006) · American Journal of Geriatric Psychiatry · doi:10.1097/01.jgp.0000203178.44894.db · Cited by 269
Prediction of Alzheimer's disease in subjects with mild cognitive impairment from the ADNI cohort using patterns of cortical thinning
Eskildsen, S. F., Coupé, P., García-Lorenzo, D., Fonov, V., Pruessner, J. C., & Collins, D. L. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.09.058 · Cited by 269
Modified PAGE-B score predicts the risk of hepatocellular carcinoma in Asians with chronic hepatitis B on antiviral therapy
Kim, J. H., Kim, Y. D., Lee, M., Jun, B. G., Kim, T. S., Suk, K. T., Kang, S. H., Kim, M. Y., Cheon, G. J., Kim, D. J., et al. (2018) · Journal of Hepatology · doi:10.1016/j.jhep.2018.07.018 · Cited by 269
Histopathological validation of a three-dimensional magnetic resonance spectroscopy index as a predictor of tumor presence
McKnight, T. R., Bussche, M. V. D., Vigneron, D. B., Lü, Y., Berger, M. S., McDermott, M. W., Dillon, W. P., Graves, E. E., Pirzkall, A., & Nelson, S. J. (2002) · Journal of neurosurgery · doi:10.3171/jns.2002.97.4.0794 · Cited by 268
A Comparison of Machine Learning Methods for the Diagnosis of Pigmented Skin Lesions
Dreiseitl, S., Lucila Ohno-machado, A., Kittler, H., Vinterbo, S. A., Billhardt, H., & Binder, M. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1004 · Cited by 268
The association of endothelial cell signaling, severity of illness, and organ dysfunction in sepsis
Shapiro, N. I., Schüetz, P., Yano, K., Sorasaki, M., Parikh, S. M., Jones, A. E., Trzeciak, S., Ngo, L. H., & Aird, W. C. (2010) · Critical Care · doi:10.1186/cc9290 · Cited by 268
Prediction of Type 2 Diabetes Mellitus With Alternative Definitions of the Metabolic Syndrome
Hanley, A. J., Karter, A. J., Williams, K., Festa, A., Agostino, R. B. D., Wagenknecht, L. E., & Haffner, S. M. (2005) · Circulation · doi:10.1161/circulationaha.105.559633 · Cited by 267
A 15-Year Warranty Period for Asymptomatic Individuals Without Coronary Artery Calcium
Valenti, V., Hartaigh, B. Ó., Heo, R., Cho, I., Schulman‐Marcus, J., Gransar, H., Truong, Q. A., Shaw, L. J., Knapper, J. T., Kelkar, A. A., et al. (2015) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2015.01.025 · Cited by 267
Diagnosis and Staging of Ovarian Cancer: Comparative Values of Doppler and Conventional US, CT, and MR Imaging Correlated with Surgery and Histopathologic Analysis—Report of the Radiology Diagnostic Oncology Group
Kurtz, A. B., Tsimikas, J., Tempany, C. M., Hamper, U. M., Arger, P. H., Bree, R. L., Wechsler, R. J., Francis, I. R., Kuhlman, J. E., Siegelman, E. S., et al. (1999) · Radiology · doi:10.1148/radiology.212.1.r99jl3619 · Cited by 267
Circulating microRNAs in plasma as early detection markers for breast cancer
Ćuk, K., Zucknick, M., Heil, J., Madhavan, D., Schott, S., Turchinovich, A. A., Arlt, D., Rath, M., Sohn, C., Benner, A., et al. (2012) · International Journal of Cancer · doi:10.1002/ijc.27799 · Cited by 268
Characteristic repartition of monocyte subsets as a diagnostic signature of chronic myelomonocytic leukemia
Selimoglu‐Buet, D., Wagner‐Ballon, O., Saada, V., Bardet, V., Itzykson, R., Bencheikh, L., Morabito, M., Met, E., Debord, C., Benayoun, E., et al. (2015) · Blood · doi:10.1182/blood-2015-01-620781 · Cited by 277
Nomograms Provide Improved Accuracy for Predicting Survival after Radical Cystectomy
Shariat, S. F., Karakiewicz, P. I., Palapattu, G. S., Amiel, G. E., Lotan, Y., ROGERS, C. G., Vazina, A., Bastian, P. J., Gupta, A., Sagalowsky, A. I., et al. (2006) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-06-0372 · Cited by 265
Neutrophil-to-lymphocyte ratio predicts acute appendicitis and distinguishes between complicated and uncomplicated appendicitis: A systematic review and meta-analysis
Hajibandeh, S., Hajibandeh, S., Hobbs, N., & Mansour, M. (2019) · The American Journal of Surgery · doi:10.1016/j.amjsurg.2019.04.018 · Cited by 265
Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature
Cowley, L., Farewell, D. M., Maguire, S., & Kemp, A. (2019) · Diagnostic and Prognostic Research · doi:10.1186/s41512-019-0060-y · Cited by 271
Image-based consensus molecular subtype (imCMS) classification of colorectal cancer using deep learning
Sirinukunwattana, K., Domingo, E., Richman, S. D., Redmond, K. L., Blake, A., Verrill, C., Leedham, S. J., Chatzipli, A., Hardy, C., Whalley, C. M., et al. (2021) · Gut · doi:10.1136/gutjnl-2019-319866 · Cited by 264
Predicting online-purchasing behaviour
den Poel, D. V., & Buckinx, W. (2004) · European Journal of Operational Research · doi:10.1016/j.ejor.2004.04.022 · Cited by 264
Fractional Flow Reserve Computed from Noninvasive CT Angiography Data: Diagnostic Performance of an On-Site Clinician-operated Computational Fluid Dynamics Algorithm
Coenen, A., Lubbers, M. M., Kurata, A., Kono, A. K., Dedic, A., Chelu, R. G., Dijkshoorn, M. L., Gijsen, F. J. H., Ouhlous, M., van Geuns, R., et al. (2014) · Radiology · doi:10.1148/radiol.14140992 · Cited by 264
Deep Learning vs. Radiomics for Predicting Axillary Lymph Node Metastasis of Breast Cancer Using Ultrasound Images: Don't Forget the Peritumoral Region
Sun, Q., Lin, X., Zhao, Y., Li, L., Yan, K., Liang, D., Sun, D., & Li, Z. (2020) · Frontiers in Oncology · doi:10.3389/fonc.2020.00053 · Cited by 264
Dynamic Bilateral Contrast-enhanced MR Imaging of the Breast: Trade-off between Spatial and Temporal Resolution
Kuhl, C. K., Schild, H. H., & Morakkabati, N. (2005) · Radiology · doi:10.1148/radiol.2363040811 · Cited by 263
Cerebrospinal Fluid Aβ 42/40 Corresponds Better than Aβ 42 to Amyloid PET in Alzheimer’s Disease
Lewczuk, P., Matzen, A., Blennow, K., Parnetti, L., Molinuevo, J. L., Eusebi, P., Kornhuber, J., Morris, J. C., & Fagan, A. M. (2016) · Journal of Alzheimer’s Disease · doi:10.3233/jad-160722 · Cited by 253
An integrated model for the prediction of pre-eclampsia using maternal factors and uterine artery Doppler velocimetry in unselected low-risk women
Yu, C.-S., Smith, G. C. S., Papageorghiou, A. T., Cacho, A. M., & Nicolaides, K. H. (2006) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2006.06.010 · Cited by 263
Quantitative versus standard pupillary light reflex for early prognostication in comatose cardiac arrest patients: an international prospective multicenter double-blinded study
Oddo, M., Sandroni, C., Citerio, G., Miroz, J., Horn, J., Rundgren, M., Cariou, A., Payen, J., Storm, C., Stammet, P., et al. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5448-6 · Cited by 262
Can Quantitative CT Texture Analysis be Used to Differentiate Fat-poor Renal Angiomyolipoma from Renal Cell Carcinoma on Unenhanced CT Images?
Hodgdon, T., McInnes, M. D. F., Schieda, N., Flood, T. A., Lamb, L. R., & Thornhill, R. E. (2015) · Radiology · doi:10.1148/radiol.2015142215 · Cited by 261
Integration of genetic risk factors into a clinical algorithm for multiple sclerosis susceptibility: a weighted genetic risk score
Jager, P. L. D., Chibnik, L. B., Cui, J., Reischl, J., Lehr, S., Simon, K. C., Aubin, C., Bauer, D., Heubach, J. F., Sandbrink, R., et al. (2009) · The Lancet Neurology · doi:10.1016/s1474-4422(09)70275-3 · Cited by 261
Risk scores for risk stratification in acute coronary syndromes: useful but simpler is not necessarily better
Yan, A. T.-K., Yan, R. T., Tan, M., Casanova, A., Labinaz, M. X., Sridhar, K., FITCHETT, D. H., Langer, A., & Goodman, S. G. (2007) · European Heart Journal · doi:10.1093/eurheartj/ehm004 · Cited by 261
Rectal Cancer: Assessment of Complete Response to Preoperative Combined Radiation Therapy with Chemotherapy—Conventional MR Volumetry versus Diffusion-weighted MR Imaging
Curvo‐Semedo, L., Lambregts, D. M. J., Maas, M., Thywissen, T., Mehsen, R. T., Lammering, G., Beets, G. L., Caseiro‐Alves, F., & Beets‐Tan, R. G. H. (2011) · Radiology · doi:10.1148/radiol.11102467 · Cited by 261
Combined Longitudinal and Radial Dyssynchrony Predicts Ventricular Response After Resynchronization Therapy
Gorcsan, J., Tanabe, M., Bleeker, G. B., Suffoletto, M. S., Thomas, N. C., Saba, S., Tops, L. F., Schalij, M. J., & Bax, J. J. (2007) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.06.043 · Cited by 262
A Novel Tool for the Assessment of Pain: Validation in Low Back Pain
Scholz, J., Mannion, R., Hord, E. D., Griffin, R. S., Rawal, B., Zheng, H., Scoffings, D. J., Phillips, A., Guo, J., Laing, R. J. C., et al. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000047 · Cited by 263
Plasma Brain Natriuretic Peptide to Detect Preclinical Ventricular Systolic or Diastolic Dysfunction
Redfield, M. M., Rodeheffer, R. J., Jacobsen, S. J., Mahoney, D. W., Bailey, K. R., & Burnett, J. C. J. (2004) · Circulation · doi:10.1161/01.cir.0000130845.38133.8f · Cited by 260
Diffusion-weighted magnetic resonance without bowel preparation for detecting colonic inflammation in inflammatory bowel disease
Oussalah, A., Laurent, V., Bruot, O., Bressenot, A., Bigard, M., Régent, D., & Peyrin‐Biroulet, L. (2010) · Gut · doi:10.1136/gut.2009.197665 · Cited by 259

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks