Cited by 22,649Imbalanced Data Classification TechniquesStatistical Methods in EpidemiologyStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

DeLong, Elizabeth R., DeLong, David M., Clarke-Pearson, Daniel L.·1988·Biometrics

DOI 10.2307/2531595

Open this paper in CoChat — free account

Abstract

Methods of evaluating and comparing the performance of diagnostic tests are of increasing importance as new tests are developed and marketed. When a test is based on an observed variable that lies on a continuous or graded scale, an assessment of the overall value of the test can be made through the use of a receiver operating characteristic (ROC) curve. The curve is constructed by varying the cutpoint used to determine which values of the observed variable will be considered abnormal and then plotting the resulting sensitivities against the corresponding false positive rates. When two or more empirical curves are constructed based on tests performed on the same individuals, statistical analysis on differences between curves must take into account the correlated nature of the data. This paper presents a nonparametric approach to the analysis of areas under correlated ROC curves, by using the theory on generalized U-statistics to generate an estimated covariance matrix.

Integrity signals

Registry

Crossref · 10.2307/2531595

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,122
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,898
The area above the ordinal dominance graph and the area below the receiver operating characteristic graph
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,452
Untitled work
(1982) · Cited by 1,288
Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data
Dorfman, D. D., & Alf, E. F. (1969) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(69)90019-4 · Cited by 878
A New Approach for Testing the Significance of Differences Between ROC Curves Measured from Correlated Data
Metz, C. E., Wang, P.-L., & Kronman, H. B. (1984) · Information Processing in Medical Imaging · doi:10.1007/978-94-009-6045-9_25 · Cited by 359
Jackknifing $U$-Statistics
Arvesen, J. N. (1969) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177697287 · Cited by 286
On Some Convergence Properties of UStatistics
Sen, P. K. (1960) · Calcutta Statistical Association Bulletin · doi:10.1177/0008068319600101 · Cited by 182
Surgical management of bowel obstruction in advanced ovarian carcinoma.
Krebs, H. B., & Goplerud, D. R. (1983) · PubMed · Cited by 132

Flagged first, then by citations

Show all 12 references

Sorted by citations

Items 101–200 of 1,197

Magnetic Resonance Elastography for the Noninvasive Staging of Liver Fibrosis
Huwart, L., Sempoux, C. L., Vicaut, É., Salameh, N., Annet, L., Danse, É., Peeters, F., ter Beek, L. C., Rahier, J., Sinkus, R., et al. (2008) · Gastroenterology · doi:10.1053/j.gastro.2008.03.076 · Cited by 726
An empirically based tool for analyzing mortality associated with congenital heart surgery
O’Brien, S. M., Clarke, D. R., Jacobs, J. P., Jacobs, M. L., Lacour-Gayet, F. G., Pizarro, C., Welke, K. F., Maruszewski, B. J., Tobota, Z., Miller, W. J., et al. (2009) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2009.03.071 · Cited by 724
Magnetic Resonance Elastography vs Transient Elastography in Detection of Fibrosis and Noninvasive Measurement of Steatosis in Patients With Biopsy-Proven Nonalcoholic Fatty Liver Disease
Park, C. C., Nguyen, P., Hernandez, C., Bettencourt, R., Ramirez, K., Fortney, L., Hooker, J. C., Sy, E. Z., Savides, M. T., Alquiraish, M. H., et al. (2016) · Gastroenterology · doi:10.1053/j.gastro.2016.10.026 · Cited by 715
On determining the most appropriate test cut-off value: the case of tests with continuous results
Habibzadeh, F., Habibzadeh, P., & Yadollahie, M. (2016) · Biochemia Medica · doi:10.11613/bm.2016.034 · Cited by 696
The use of multiple novel tumor biomarkers for the detection of ovarian carcinoma in patients with a pelvic mass
Moore, R. G., Brown, A., Miller, M. C., Skates, S. J., Allard, W. J., Verch, T., Steinhoff, M. M., Lambert-Messerlian, G. M., DiSilvestro, P., Granai, C. O., et al. (2007) · Gynecologic Oncology · doi:10.1016/j.ygyno.2007.10.017 · Cited by 695
Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis
Motwani, M., Dey, D., Berman, D. S., Germano, G., Achenbach, S., Al-Mallah, M. H., Andreini, D., Budoff, M. J., Cademartiri, F., Callister, T. Q., et al. (2016) · European Heart Journal · doi:10.1093/eurheartj/ehw188 · Cited by 692
Fast, Systematic, and Continuous Delirium Assessment in Hospitalized Patients: The Nursing Delirium Screening Scale
Gaudreau, J., Gagnon, P. R., Harel, F., Tremblay, A., & Roy, M. (2005) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2004.07.009 · Cited by 691
Sensitivity and Specificity of a Single Emergency Department Measurement of Urinary Neutrophil Gelatinase–Associated Lipocalin for Diagnosing Acute Kidney Injury
Nickolas, T. L., O’Rourke, M., Yang, J., Sise, M. E., Canetta, P. A., Barasch, N., Buchen, C., Khan, F., Mori, K., Giglio, J. F., et al. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-11-200806030-00003 · Cited by 683
Validity of real time ultrasound in the diagnosis of hepatic steatosis: A prospective study
Dasarathy, S., Dasarathy, J., Khiyami, A., Joseph, R., López, R., & McCullough, A. J. (2009) · Journal of Hepatology · doi:10.1016/j.jhep.2009.09.001 · Cited by 683
A Novel Urine Exosome Gene Expression Assay to Predict High-grade Prostate Cancer at Initial Biopsy
McKiernan, J., Donovan, M. J., O’Neill, V., Bentink, S., Noerholm, M., Belzer, S. K., Skog, J., Kattan, M. W., Partin, A. W., Andriole, G. L., et al. (2016) · JAMA Oncology · doi:10.1001/jamaoncol.2016.0097 · Cited by 691
Adaptation and Validation of a Pediatric Sequential Organ Failure Assessment Score and Evaluation of the Sepsis-3 Definitions in Critically Ill Children
Matics, T. J., & Sanchez-Pinto, L. N. (2017) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2017.2352 · Cited by 676
Angiogenic factors and the risk of adverse outcomes in women with suspected preeclampsia
Rana, S., Powe, C. E., Salahuddin, S., Verlohren, S., Perschel, F. H., Levine, R. J., Lim, K.-H., Wenger, J. B., Thadhani, R., & Karumanchi, S. A. (2012) · Circulation · doi:10.1161/circulationaha.111.054361 · Cited by 668
Detection of Macular Ganglion Cell Loss in Glaucoma by Fourier-Domain Optical Coherence Tomography
Tan, O., Chopra, V., Lu, A. T.-H., Schuman, J. S., Ishikawa, H., Wollstein, G., Varma, R., & Huang, D. (2009) · Ophthalmology · doi:10.1016/j.ophtha.2009.05.025 · Cited by 668
Metabolic Syndrome vs Framingham Risk Score for Prediction of Coronary Heart Disease, Stroke, and Type 2 Diabetes Mellitus
Wannamethee, S. G., Shaper, A. G., LENNON, L. T., & Morris, R. W. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.22.2644 · Cited by 667
Accuracy of real-time shear wave elastography for assessing liver fibrosis in chronic hepatitis C: A pilot study
Ferraioli, G., Tinelli, C., Bello, B. D., Zicchetti, M., Filice, G., Fìlice, C., & on behalf of the Liver Fibrosis Study Group (2012) · Hepatology · doi:10.1002/hep.25936 · Cited by 665
Determination of reliability criteria for liver stiffness evaluation by transient elastography
Boursier, J., Zarski, J.-P., de Ledinghen, V., Rousselet, M.-C., Sturm, N., Lebail, B., Fouchard-Hubert, I., Gallois, Y., Oberti, F., Bertrais, S., et al. (2013) · Hepatology · doi:10.1002/hep.25993 · Cited by 662
Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data
Walter, S. D. (2002) · Statistics in Medicine · doi:10.1002/sim.1099 · Cited by 662
The National Cholesterol Education Program–Adult Treatment Panel III, International Diabetes Federation, and World Health Organization Definitions of the Metabolic Syndrome as Predictors of Incident Cardiovascular Disease and Diabetes
Lorenzo, C., Williams, K., Hunt, K. J., & Haffner, S. M. (2006) · Diabetes Care · doi:10.2337/dc06-1414 · Cited by 669
Evaluation of retinal nerve fiber layer, optic nerve head, and macular thickness measurements for glaucoma detection using optical coherence tomography
Medeiros, F. A., Zangwill, L. M., Bowd, C., Vessani, R. M., Susanna, R., & Weinreb, R. N. (2005) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2004.08.069 · Cited by 654
Elevation of circulating branched-chain amino acids is an early event in human pancreatic adenocarcinoma development
Mayers, J. R., Wu, C. H., Clish, C. B., Kraft, P. L., Torrence, M. E., Fiske, B. P., Yuan, C., Bao, Y., Townsend, M. K., Tworoger, S. S., et al. (2014) · Nature Medicine · doi:10.1038/nm.3686 · Cited by 672
Bone microarchitecture assessed by TBS predicts osteoporotic fractures independent of bone density: The manitoba study
Hans, D. B., Goertzen, A. L., Krieg, M.-A., & Leslie, W. D. (2011) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.499 · Cited by 643
Fast Implementation of DeLong’s Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves
Sun, X., & Xu, W. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2337313 · Cited by 638
Plasma MicroRNA Panel to Diagnose Hepatitis B Virus–Related Hepatocellular Carcinoma
Zhou, J., Yu, L., Gao, X. J., Hu, J., Wang, J., Dai, Z., Wang, J.-F., Zhang, Z., Lu, S., Huang, X., et al. (2011) · Journal of Clinical Oncology · doi:10.1200/jco.2011.38.2697 · Cited by 636
Significance of a Fragmented QRS Complex Versus a Q Wave in Patients With Coronary Artery Disease
Das, M. K., Khan, B. R., Jacob, S., Kumar, A., & Mahenthiran, J. (2006) · Circulation · doi:10.1161/circulationaha.105.595892 · Cited by 632
Circulating tumor DNA as an early marker of therapeutic response in patients with metastatic colorectal cancer
Tie, J., Kinde, I., Wang, Y., Wong, H., Roebert, J. K., Christie, M., TACEY, M. A., Wong, R., Singh, M., Karapetis, C. S., et al. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv177 · Cited by 632
Bootstrap Methods for Developing Predictive Models
Austin, P. C., & Tu, J. V. (2004) · The American Statistician · doi:10.1198/0003130043277 · Cited by 622
Plasma phospho‐tau181 increases with Alzheimer's disease clinical severity and is associated with tau‐ and amyloid‐positron emission tomography
Mielke, M. M., Hagen, C. E., Xu, J., Chai, X., Vemuri, P., Lowe, V. J., Airey, D. C., Knopman, D. S., Roberts, R. O., Machulda, M. M., et al. (2018) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2018.02.013 · Cited by 621
Early prediction of severity in acute pancreatitis by urinary trypsinogen activation peptide: a multicentre study
Neoptolemos, J. P., KEMPPAINEN, E., Mayer, J., Fitzpatrick, J., Raraty, M. G. T., Slavin, J. P., Beger, H.-G., Hietaranta, A. J., & Pa, P. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02327-8 · Cited by 621
Validation of the Tricuspid Annular Plane Systolic Excursion/Systolic Pulmonary Artery Pressure Ratio for the Assessment of Right Ventricular-Arterial Coupling in Severe Pulmonary Hypertension
Tello, K., Wan, J., Dalmer, A., Vanderpool, R. R., Ghofrani, H. A., Naeije, R., Roller, F. C., Mohajerani, E., Seeger, W., Herberg, U., et al. (2019) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.119.009047 · Cited by 613
Plasma phosphorylated tau 217 and phosphorylated tau 181 as biomarkers in Alzheimer's disease and frontotemporal lobar degeneration: a retrospective diagnostic performance study
Thijssen, E. H., La Joie, R., Strom, A., Fonseca, C., Iaccarino, L., Wolf, A., Spina, S., Allen, I. E., Cobigo, Y., Heuer, H., et al. (2021) · The Lancet Neurology · doi:10.1016/s1474-4422(21)00214-3 · Cited by 607
Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer
Zheng, X., Yao, Z., Huang, Y., Yu, Y., Wang, Y., Liu, Y., Mao, R., Li, F., Xiao, Y., Wang, Y., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15027-z · Cited by 607
Validation of a prediction tool for chemotherapy toxicity in older adults with cancer
Hurria, A., Mohile, S., Gajra, A., Klepin, H., Muss, H., Chapman, A., Feng, T., Smith, D., Sun, C.-L., De Glas, N., et al. (2016) · Journal of Clinical Oncology · doi:10.1200/jco.2015.65.4327 · Cited by 598
Better prediction of functional effects for sequence variants
Hecht, M., Bromberg, Y., & Rost, B. (2015) · BMC Genomics · doi:10.1186/1471-2164-16-s8-s1 · Cited by 569
Comparison of BISAP, Ranson's, APACHE-II, and CTSI Scores in Predicting Organ Failure, Complications, and Mortality in Acute Pancreatitis
Papachristou, G. I., Muddana, V. N., Yadav, D. K., O’Connell, M. R., Sanders, M. K., Slivka, A., & Whitcomb, D. C. (2009) · The American Journal of Gastroenterology · doi:10.1038/ajg.2009.622 · Cited by 598
Screening Peers Softly: Inferring the Quality of Small Borrowers
Iyer, R., Khwaja, A. I., Luttmer, E. F. P., & Shue, K. (2015) · Management Science · doi:10.1287/mnsc.2015.2181 · Cited by 613
Superpixel Classification Based Optic Disc and Optic Cup Segmentation for Glaucoma Screening
Cheng, J. T., Liu, J., Xu, Y., Yin, F., Wong, D. W. K., Tan, N.-M., Tao, D., Cheng, C., Aung, T., & Wong, T. Y. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2247770 · Cited by 594
Cerebral amyloid-β PET with florbetaben (18F) in patients with Alzheimer's disease and healthy controls: a multicentre phase 2 diagnostic study
Barthel, H., Gertz, H., Dresel, S. H. J., Peters, O., Bartenstein, P., Büerger, K., Hiemeyer, F. J., Wittemer‐Rump, S., Seibyl, J. P., Reininger, C. B., et al. (2011) · The Lancet Neurology · doi:10.1016/s1474-4422(11)70077-1 · Cited by 600
Automated volumetry and regional thickness analysis of hippocampal subfields and medial temporal cortical structures in mild cognitive impairment
Yushkevich, P. A., Pluta, J. B., Wang, H., Xie, L., Ding, S., Gertje, E. C., Mancuso, L., Kliot, D., Das, S. R., & Wolk, D. A. (2014) · Human Brain Mapping · doi:10.1002/hbm.22627 · Cited by 587
Left atrial volume as an index ofleft atrial size: a population-based study
Pritchett, A. M., Jacobsen, S. J., Mahoney, D. W., Rodeheffer, R. J., Bailey, K. R., & Redfield, M. M. (2003) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(02)02981-9 · Cited by 583
A foundation model for clinical-grade computational pathology and rare cancers detection
Vorontsov, E., Bozkurt, A., Casson, A., Shaikovski, G., Zelechowski, M., Severson, K., Zimmermann, E., Hall, J., Tenenholtz, N., Fusi, N., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03141-0 · Cited by 572
Logistic regression was as good as machine learning for predicting major chronic diseases
Nusinovici, S., Tham, Y. C., Yan, M. Y. C., Ting, D. S. W., Li, J., Sabanayagam, C., Wong, T. Y., & Cheng, C. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.03.002 · Cited by 597
Derivation and external validation of the PLASMIC score for rapid assessment of adults with thrombotic microangiopathies: a cohort study.
Bendapudi, P., Hurwitz, S., Fry, A., Marques, M., Waldo, S., Li, A., Sun, L., Upadhyay, V., Hamdan, A., Brunner, A., et al. (2017) · The Lancet Haematology · doi:10.1016/s2352-3026(17)30026-1 · Cited by 470
High-Risk Plaque Detected on Coronary CT Angiography Predicts Acute Coronary Syndromes Independent of Significant Stenosis in Acute Chest Pain
Puchner, S. B., Liu, T., Mayrhofer, T., Truong, Q. A., Lee, H., Fleg, J. L., Nagurney, J. T., Udelson, J. E., Hoffmann, U., & Ferencik, M. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2014.05.039 · Cited by 565
Laboratory diagnosis of iron-deficiency anemia: An overview
Guyatt, G. H., Oxman, A. D., Ali, M., Willan, A., McIlroy, W., & Patterson, C. (1992) · Journal of General Internal Medicine · doi:10.1007/bf02598003 · Cited by 564
Monitoring clinical changes in patients with heart failure: A comparison of methods
Spertus, J. A., Peterson, E. D., Conard, M. W., Heidenreich, P. A., Krumholz, H. M., Jones, P. G., McCullough, P. A., Piña, I. L., Tooley, J. F., Weintraub, W. S., et al. (2005) · American Heart Journal · doi:10.1016/j.ahj.2004.12.010 · Cited by 564
Handling class imbalance in customer churn prediction
Burez, J., & den Poel, D. V. (2008) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.05.027 · Cited by 590
A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology
Iglesias, J. E., Insausti, R., Lerma‐Usabiaga, G., Bocchetta, M., Leemput, K. V., Greve, D. N., van der Kouwe, A. J. W., Fischl, B. R., Caballero‐Gaudes, C., & Paz‐Alonso, P. M. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.08.012 · Cited by 575
Pregnancy-Associated Plasma Protein A as a Marker of Acute Coronary Syndromes
Bayés‐Genís, A., Conover, C. A., Overgaard, M. T., Bailey, K. R., Christiansen, M., Holmes, D. R., Virmani, R., Oxvig, C., & Schwartz, R. S. (2001) · New England Journal of Medicine · doi:10.1056/nejmoa003147 · Cited by 558
OptimalCutpoints : An R Package for Selecting Optimal Cutpoints in Diagnostic Tests
López‐Ratón, M., Rodríguez‐Álvarez, M. X., Cadarso-Suárez, C., & Gudé, F. (2014) · Journal of Statistical Software · doi:10.18637/jss.v061.i08 · Cited by 556
Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study
Tschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R. P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., et al. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30333-x · Cited by 555
Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine
Obuchowski, N. A., & Bullen, J. A. (2018) · Physics in Medicine and Biology · doi:10.1088/1361-6560/aab4b1 · Cited by 554
The human gut microbiome as a screening tool for colorectal cancer
Zackular, J. P., Rogers, M. A. M., Ruffin, M. T., & Schloss, P. D. (2014) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-14-0129 · Cited by 551
Synovial fluid leukocyte count and differential for the diagnosis of prosthetic knee infection
Trampuž, A., Hanssen, A. D., Osmon, D. R., Mandrekar, J. N., Steckelberg, J. M., & Patel, R. (2004) · The American Journal of Medicine · doi:10.1016/j.amjmed.2004.06.022 · Cited by 550
Assessing the Diagnostic Accuracy of Pulse Pressure Variations for the Prediction of Fluid Responsiveness
Cannesson, M. P., Manach, Y. L., Hofer, C. K., Goarin, J., Lehot, J.-J., Vallet, B., & Tavernier, B. (2011) · Anesthesiology · doi:10.1097/aln.0b013e318225b80a · Cited by 549
Utility of the Apolipoprotein E Genotype in the Diagnosis of Alzheimer's Disease
Mayeux, R., Saunders, A. M., Shea, S., Mirra, S., Evans, D., Roses, A. D., Hyman, B. T., Crain, B., Tang, M.-X., & Phelps, C. H. (1998) · New England Journal of Medicine · doi:10.1056/nejm199802193380804 · Cited by 547
Coronary Computed Tomography Angiography for Early Triage of Patients With Acute Chest Pain
Hoffmann, U., Bamberg, F., Chae, C. U., Nichols, J. H., Rogers, I. S., SENEVIRATNE, S. K., Truong, Q. A., Cury, R. C., Abbara, S., Shapiro, M. D., et al. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.01.052 · Cited by 547
Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer
Vallières, M., Kay‐Rivest, E., Perrin, L. J., Liem, X., Furstoss, C., Aerts, H. J. W. L., Khaouam, N., Nguyen‐Tan, P. F., Wang, C.-S., Sultanem, K., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-10371-5 · Cited by 569
Brachial-Ankle Pulse Wave Velocity and the Risk Prediction of Cardiovascular Disease: An Individual Participant Data Meta-Analysis
Ohkuma, T., Ninomiya, T., Tomiyama, H., Kario, K., Hoshide, S., Kita, Y., Inoguchi, T., Maeda, Y., Kohara, K., Tabara, Y., et al. (2017) · Hypertension · doi:10.1161/hypertensionaha.117.09097 · Cited by 541
Development and Validation of Deep Learning–based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs
Nam, J. G., Park, S., Hwang, E. J., Lee, J. H., Jin, K.-N., Lim, K. Y., Vu, T. H., Sohn, J. H., Hwang, S., Goo, J. M., et al. (2019) · Radiology · doi:10.1148/radiol.2018180237 · Cited by 539
Coronary Calcification Improves Cardiovascular Risk Prediction in the Elderly
Vliegenthart, R., Oudkerk, M., Albert Hofman, D., Oei, H.-H. S., van Dijck, W., van Rooij, F. J. A., & Witteman, J. C. M. (2005) · Circulation · doi:10.1161/circulationaha.104.488916 · Cited by 536
Measurement of Spleen Stiffness to Evaluate Portal Hypertension and the Presence of Esophageal Varices in Patients With HCV-Related Cirrhosis
Colecchia, A., Montrone, L., Scaioli, E., Reggiani, M. L. B., Colli, A., Casazza, G., Schiumerini, R., Turco, L., Biase, A. R. D., Mazzella, G., et al. (2012) · Gastroenterology · doi:10.1053/j.gastro.2012.05.035 · Cited by 531
Emergency department triage prediction of clinical outcomes using machine learning models
Raita, Y., Goto, T., Faridi, M. K., Brown, D., Camargo, C. A., & Hasegawa, K. (2019) · Critical Care · doi:10.1186/s13054-019-2351-7 · Cited by 556
Prediction of Mortality After Primary Percutaneous Coronary Intervention for Acute Myocardial Infarction
Halkin, A., Singh, M., Nikolsky, E., Grines, C. L., Tcheng, J. E., Garcı́a, E., Cox, D. A., Turco, M. A., Stuckey, T., Na, Y., et al. (2005) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2005.01.041 · Cited by 529
Using Clinical Factors and Mammographic Breast Density to Estimate Breast Cancer Risk: Development and Validation of a New Predictive Model
Tice, J. A., Cummings, S. R., Smith‐Bindman, R., Ichikawa, L., Barlow, W. E., & Kerlikowske, K. M. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-5-200803040-00004 · Cited by 527
MELD accurately predicts mortality in patients with alcoholic hepatitis†
Dunn, W., Jamil, L. H., Brown, L. S., Wiesner, R. H., Kim, W. R., Menon, K. V., Malinchoc, M., Kamath, P. S., & Shah, V. H. (2005) · Hepatology · doi:10.1002/hep.20503 · Cited by 525
Deep image mining for diabetic retinopathy screening
Quellec, G., Charrière, K., Boudi, Y., Cochener, B., & Lamard, M. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.04.012 · Cited by 537
PresenceAbsence : An R Package for Presence Absence Analysis
Freeman, E. A., & Moisen, G. (2008) · Journal of Statistical Software · doi:10.18637/jss.v023.i11 · Cited by 461
Ultrasound assessment of lung aeration loss during a successful weaning trial predicts postextubation distress*
Soummer, A., Perbet, S., Brisson, H., Arbelot, C., Constantin, J., Lü, Q., & Rouby, J. (2012) · Critical Care Medicine · doi:10.1097/ccm.0b013e31824e68ae · Cited by 524
Superiority of Brain Natriuretic Peptide as a Hormonal Marker of Ventricular Systolic and Diastolic Dysfunction and Ventricular Hypertrophy
Yamamoto, K., Burnett, J. C. J., Jougasaki, M., Nishimura, R. A., Bailey, K. R., Saito, Y., Nakao, K., & Redfield, M. M. (1996) · Hypertension · doi:10.1161/01.hyp.28.6.988 · Cited by 518
Application of support vector machine modeling for prediction of common diseases: the case of diabetes and pre-diabetes
Yu, W., Liu, T., Valdez, R. A., GWINN, M. L., & Khoury, M. J. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-16 · Cited by 529
Urinary N-Acetyl-β-(D)-Glucosaminidase Activity and Kidney Injury Molecule-1 Level Are Associated with Adverse Outcomes in Acute Renal Failure
Liangos, O., Perianayagam, M. C., Vaidya, V. S., Han, W. K., Wald, R., Tighiouart, H., MacKinnon, R. W., Li, L., Balakrishnan, V. S., Pereira, B. J. G., et al. (2007) · Journal of the American Society of Nephrology · doi:10.1681/asn.2006030221 · Cited by 511
Systemic endothelial dysfunction is related to the extent and severity of coronary artery disease
Neunteufl, T., Katzenschlager, R., Hassan, A. H. M., Klaar, U., Schwarzacher, S. P., Glogar, D., Bauer, P., & Weidinger, F. F. (1997) · Atherosclerosis · doi:10.1016/s0021-9150(96)06018-2 · Cited by 510
Magnetic Resonance Imaging of the Breast Prior to Biopsy
Bluemke, D. A. (2004) · JAMA · doi:10.1001/jama.292.22.2735 · Cited by 509
Mid-Region Pro-Hormone Markers for Diagnosis and Prognosis in Acute Dyspnea
Maisel, A. S., Mueller, C. E., Nowak, R. M., Peacock, W. F., Landsberg, J. W., Ponikowski, P. P., Möckel, M. M., Hogan, C., Wu, A. H. B., Richards, M., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.02.025 · Cited by 508
Serial Changes in Highly Sensitive Troponin I Assay and Early Diagnosis of Myocardial Infarction
Keller, T., Zeller, T., Ojeda, F., Tzikas, S., Lillpopp, L., Sinning, C., Wild, P. S., Genth‐Zotz, S., Warnholtz, A., Giannitsis, E., et al. (2011) · JAMA · doi:10.1001/jama.2011.1896 · Cited by 508
Identification of Persons at High Risk for Type 2 Diabetes Mellitus: Do We Need the Oral Glucose Tolerance Test?
D, M. P. S. M., Williams, K., & Haffner, S. M. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-8-200204160-00006 · Cited by 506
A Multicenter Study of [-2]Pro-Prostate Specific Antigen Combined With Prostate Specific Antigen and Free Prostate Specific Antigen for Prostate Cancer Detection in the 2.0 to 10.0 ng/ml Prostate Specific Antigen Range
Catàlona, W. J., Partin, A. W., Sanda, M. G., Wei, J. T., Klee, G. G., Bangma, C. H., Slawin, K. M., Marks, L. S., Loeb, S., Broyles, D. L., et al. (2011) · The Journal of Urology · doi:10.1016/j.juro.2010.12.032 · Cited by 503
Feasibility and Diagnostic Performance of the Fibroscan XL Probe for Liver Stiffness Measurement in Overweight and Obese Patients
Myers, R. P., Pomier–Layrargues, G., Kirsch, R., Pollett, A. F., Duarte‐Rojo, A., Wong, D., Beaton, M. D., Levstik, M. A., Crotty, P., & Elkashab, M. (2011) · Hepatology · doi:10.1002/hep.24624 · Cited by 511
Meaningful Gait Speed Improvement During the First 60 Days Poststroke: Minimal Clinically Important Difference
Tilson, J. K., Sullivan, K. J., Cen, S., Rose, D. K., Koradia, C. H., Azen, S. P., Duncan, P. W., & for the Locomotor Experience Applied Post Stroke (LEAPS) Investigative Team (2009) · Physical Therapy · doi:10.2522/ptj.20090079 · Cited by 501
Locally Advanced Breast Cancer: MR Imaging for Prediction of Response to Neoadjuvant Chemotherapy—Results from ACRIN 6657/I-SPY TRIAL
Hylton, N. M., Blume, J. D., Bernreuter, W. K., Pisano, E. D., Rosen, M. A., Morris, E. A., Weatherall, P. T., Lehman, C. D., Newstead, G. M., Polin, S. M., et al. (2012) · Radiology · doi:10.1148/radiol.12110748 · Cited by 499
Statistical Evaluation of a Biomarker
Ray, P., Manach, Y. L., Riou, B., Houle, T. T., & Warner, D. S. (2010) · Anesthesiology · doi:10.1097/aln.0b013e3181d47604 · Cited by 499
The GPCOG: A New Screening Test for Dementia Designed for General Practice
Brodaty, H., Pond, D., Kemp, N. M., Luscombe, G. M., Harding, L., Berman, K., & Huppert, F. A. (2002) · Journal of the American Geriatrics Society · doi:10.1046/j.1532-5415.2002.50122.x · Cited by 496
Prediction of In‐hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data–Driven, Machine Learning Approach
Taylor, R. A., Pare, J. R., Venkatesh, A. K., Mowafi, H. O., Melnick, E. R., Fleischman, W. M., & Hall, M. K. (2015) · Academic Emergency Medicine · doi:10.1111/acem.12876 · Cited by 513
Radiomics of Multiparametric MRI for Pretreatment Prediction of Pathologic Complete Response to Neoadjuvant Chemotherapy in Breast Cancer: A Multicenter Study
Liu, Z., Li, Z., Qu, J., Zhang, R., Zhou, X., Li, L., Sun, K., Tang, Z., Jiang, H., Li, H., et al. (2019) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-3190 · Cited by 494
Development and validation of a short version of the Kansas City Cardiomyopathy Questionnaire
Spertus, J. A., & Jones, P. G. (2015) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.115.001958 · Cited by 491
Diagnostic value of biochemical markers (FibroTest-FibroSURE) for the prediction of liver fibrosis in patients with non-alcoholic fatty liver disease
Ratziu, V., Massard, J., Charlotte, F., Messous, D., Imbert‐Bismut, F., Bonyhay, L., Tahiri, M., Munteanu, M., Thabut, D., Cadranel, J. F. D., et al. (2006) · BMC Gastroenterology · doi:10.1186/1471-230x-6-6 · Cited by 504
Prognostic Value of Noninvasive Cardiovascular Testing in Patients With Stable Chest Pain
Hoffmann, U., Ferencik, M., Udelson, J. E., Picard, M. H., Truong, Q. A., Patel, M. R., Huang, M., Pencina, M. J., Mark, D. B., Heitner, J. F., et al. (2017) · Circulation · doi:10.1161/circulationaha.116.024360 · Cited by 485
Integration of Scheimpflug-Based Corneal Tomography and Biomechanical Assessments for Enhancing Ectasia Detection
Ambrósio, R., Lopes, B. T., Faria-Correia, F., Salomão, M. Q., Bühren, J., Roberts, C. J., Elsheikh, A., Vinciguerra, R., & Vinciguerra, P. (2017) · Journal of Refractive Surgery · doi:10.3928/1081597x-20170426-02 · Cited by 484
Local Causal and Markov Blanket Induction for Causal Discovery and Feature Selection for Classification Part I: Algorithms and Empirical Evaluation
Aliferis, C., Statnikov, A., Tsamardinos, I., Mani, S., & Koutsoukos, X. (2010) · doi:10.5555/1756006.1756013 · Cited by 483
Discovery of a structural class of antibiotics with explainable deep learning
Wong, F. J., Zheng, E. J., Valeri, J. A., Donghia, N. M., Anahtar, M. N., Omori, S., Li, A., Cubillos-Ruiz, A., Krishnan, A., Jin, W., et al. (2023) · Nature · doi:10.1038/s41586-023-06887-8 · Cited by 530
Health Status Predicts Long-Term Outcome in Outpatients With Coronary Disease
Spertus, J., Jones, P. G., McDonell, M., Fan, V., & Fihn, S. (2002) · Circulation · doi:10.1161/01.cir.0000020688.24874.90 · Cited by 528
Logistic Regression Model to Distinguish Between the Benign and Malignant Adnexal Mass Before Surgery: A Multicenter Study by the International Ovarian Tumor Analysis Group
Timmerman, D., Testa, A. C., Bourne, T., Ferrazzi, E. M., Ameye, L., Konstantinovic, M. L., CALSTER, B. V., Collins, W. P., Vergote, I. B., Huffel, S. V., et al. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.01.7632 · Cited by 480
Different regional patterns of cortical thinning in Alzheimer's disease and frontotemporal dementia
Du, A., Schuff, N. W., Kramer, J. H., Rosen, H. J., Gorno‐Tempini, M. L., Rankin, K. P., Miller, B. L., & Weiner, M. W. (2006) · Brain · doi:10.1093/brain/awm016 · Cited by 491
Cross-Sectional Computed Tomographic Assessment Improves Accuracy of Aortic Annular Sizing for Transcatheter Aortic Valve Replacement and Reduces the Incidence of Paravalvular Aortic Regurgitation
Jilaihawi, H., Kashif, M., Fontana, G. P., Furugen, A., Shiota, T., Friede, G., Makhija, R. R., Doctor, N., Leon, M. B., & Makkar, R. (2012) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.11.045 · Cited by 478
Performance of Acoustic Radiation Force Impulse imaging for the staging of liver fibrosis: a pooled meta‐analysis
Friedrich-Rust, M., Nierhoff, J., Lupșor‐Platon, M., Sporea, I., Fierbinteanu-Braticevici, C. G., Strobel, D. D., Takahashi, K., Yoneda, M., Suda, T., Zeuzem, S., et al. (2011) · Journal of Viral Hepatitis · doi:10.1111/j.1365-2893.2011.01537.x · Cited by 476
Psychometric properties of the Depression Anxiety and Stress Scale-21 in older primary care patients
Gloster, A. T., Rhoades, H. M., Novy, D., Klotsche, J., Senior, A., Kunik, M., Wilson, N., & Stanley, M. A. (2008) · Journal of Affective Disorders · doi:10.1016/j.jad.2008.01.023 · Cited by 474
The cyclophosphamide equivalent dose as an approach for quantifying alkylating agent exposure: A report from the childhood cancer survivor study
Green, D. M., Nolan, V. G., Goodman, P. J., Whitton, J. A., Srivastava, D. K., Leisenring, W. M., Neglia, J. P., Sklar, C. A., Kaste, S. C., Hudson, M. M., et al. (2013) · Pediatric Blood & Cancer · doi:10.1002/pbc.24679 · Cited by 482
Financial distress and bankruptcy prediction among listed companies using accounting, market and macroeconomic variables
Tinoco, M. H., & Wilson, N. (2013) · International Review of Financial Analysis · doi:10.1016/j.irfa.2013.02.013 · Cited by 474
CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII
Kocak, B., Baessler, B., Bakas, S., Cuocolo, R., Fedorov, A., Maier-Hein, L., Mercaldo, N., Müller, H., Orlhac, F., Pinto dos Santos, D., et al. (2023) · Insights into Imaging · doi:10.1186/s13244-023-01415-8 · Cited by 471
Assessment of biopsy‐proven liver fibrosis by two‐dimensional shear wave elastography: An individual patient data‐based meta‐analysis
Herrmann, E. M., de Lédinghen, V., Cassinotto, C., Chu, C. W., Leung, V. Y. F., Ferraioli, G., Fìlice, C., Castéra, L., Vilgrain, V., Ronot, M., et al. (2017) · Hepatology · doi:10.1002/hep.29179 · Cited by 471

Authors: DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L.
Affiliations: IQVIA (United States) (US)
IQVIA (United Kingdom) (GB)
ORCID: DeLong, Elizabeth R.
Venue: Biometrics
Publisher: JSTOR
Published: 1988-09
Volume: 44
Issue: 3
Pages: 837
ISSN: 0006-341X (ISSN-L)
DOI: 10.2307/2531595
OpenAlex: W2328176404
Cited by: 22,649
References: 12
Citations by year: 2026: 1897 · 2025: 1546 · 2024: 1644 · 2023: 1628 · 2022: 1861 · 2021: 1946 · 2020: 1651 · 2019: 1364 · 2018: 1193 · 2017: 1076 · 2016: 1001 · 2015: 924 · 2014: 775 · 2013: 745 · 2012: 621
FWCI: 4.52
Type: journal article
Language: en
Open access: closed
Locations: Biometrics (publishedVersion)
PubMed (publishedVersion)
Funders: Health Services Research and Development
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: receiver operating characteristic, area under the ROC curve, diagnostic test comparison, nonparametric covariance estimation, sensitivity and false positive rate
MeSH terms: Algorithms, Analysis of Variance, Female, Humans, Intestinal Obstruction, Ovarian Neoplasms, Predictive Value of Tests, ROC Curve, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks