Receiver Operating Characteristic Curves and Their Use in Radiology
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Sensitivity and specificity are the basic measures of accuracy of a diagnostic test; however, they depend on the cut point used to define "positive" and "negative" test results. As the cut point shifts, sensitivity and specificity shift. The receiver operating characteristic (ROC) curve is a plot of the sensitivity of a test versus its false-positive rate for all possible cut points. The advantages of the ROC curve as a means of defining the accuracy of a test, construction of the ROC, and identification of the optimal cut point on the ROC curve are discussed. Several summary measures of the accuracy of a test, including the commonly used percentage of correct diagnoses and area under the ROC curve, are described and compared. Two examples of ROC curve application in radiologic research are presented.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
Zweig, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,569
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
ROC Methodology in Radiologic Imaging
METZ, C. E. (1986) · Investigative Radiology · doi:10.1097/00004424-198609000-00009 · Cited by 1,642
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,439
Some Practical Issues of Experimental Design and Data Analysis in Radiological ROC Studies
METZ, C. E. (1989) · Investigative Radiology · doi:10.1097/00004424-198903000-00012 · Cited by 859
Receiver operating characteristic (ROC) methodology: the state of the art.
Hanley, J. A. (1989) · PubMed · Cited by 807
Analyzing a Portion of the ROC Curve
McClish, D. K. (1989) · Medical Decision Making · doi:10.1177/0272989x8900900307 · Cited by 707
Lusted, L. B. (1971) · Science · doi:10.1126/science.171.3977.1217 · Cited by 304
Cited by
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Self-citationAhuja, C. S., Wilson, J. R., Nori, S., Kotter, M. R. N., Druschel, C., Curt, A., & Fehlings, M. G. (2017) · Nature Reviews Disease Primers · doi:10.1038/nrdp.2017.18 · Cited by 2,441
Use and Misuse of the Receiver Operating Characteristic Curve in Risk Prediction
Cook, N. R. (2007) · Circulation · doi:10.1161/circulationaha.106.672402 · Cited by 2,112
(2013) · Europe PMC (PubMed Central) · Cited by 2,089
Lakhani, P., & Sundaram, B. (2017) · Radiology · doi:10.1148/radiol.2017162326 · Cited by 1,775
Receiver operating characteristic curve: overview and practical use for clinicians
Nahm, F. S. (2022) · Korean Journal of Anesthesiology · doi:10.4097/kja.21209 · Cited by 1,484
Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models
Zou, K. H., O’Malley, A. J., & Mauri, L. (2007) · Circulation · doi:10.1161/circulationaha.105.594929 · Cited by 1,473
Receiver Operating Characteristic (ROC) Curve: Practical Review for Radiologists
Park, S. H., Goo, J. M., & Jo, C.-H. (2004) · Korean Journal of Radiology · doi:10.3348/kjr.2004.5.1.11 · Cited by 907
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894
The use of receiver operating characteristic curves in biomedical informatics
Lasko, T. A., Bhagwat, J. G., Zou, K. H., & Ohno-Machado, L. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.02.008 · Cited by 823
Mazurowski, M. A., Habas, P. A., Zurada, J. M., Lo, J. Y., Baker, J. A., & Tourassi, G. D. (2008) · Neural Networks · doi:10.1016/j.neunet.2007.12.031 · Cited by 737
Natural language processing in radiology: a systematic review
Pons, E., Braun, L. M. M., Hunink, M. G. M., & Kors, J. A. (2016) · Radiology · doi:10.1148/radiol.16142770 · Cited by 573
What's under the ROC? An Introduction to Receiver Operating Characteristics Curves
Streiner, D. L., & Cairney, J. (2007) · The Canadian Journal of Psychiatry · doi:10.1177/070674370705200210 · Cited by 527
Ren, F., Ward, L., Williams, T., Laws, K. J., Wolverton, C., Hattrick-Simpers, J., & Mehta, A. (2018) · Science Advances · doi:10.1126/sciadv.aaq1566 · Cited by 524
Critchley, L. A., Lee, A., & Ho, A. M.-H. (2010) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e3181f08a5b · Cited by 512
Honda, H., Qureshi, A. R., Heimbürger, O., Barany, P., Wang, K., Pecoits-Filho, R., Stenvinkel, P., & Lindholm, B. (2006) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2005.09.014 · Cited by 510
Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders
Subasi, A. (2013) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2013.01.020 · Cited by 478
Receiver operating characteristics curves and related decision measures: A tutorial
Brown, C. D., & Davis, H. T. (2006) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2005.05.004 · Cited by 456
Kim, S., Loevner, L., Quon, H., Sherman, E., Weinstein, G., Kilger, A., & Poptani, H. (2009) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-08-1287 · Cited by 392
Dheeba, J., Albert Singh, N., & Tamil Selvi, S. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.01.010 · Cited by 380
“These networks are a new powerful tool for approximation and deals effectively with the problems of high dimensional model [31].”
MethodWakabayashi, I., & Daimon, T. (2015) · Clinica Chimica Acta · doi:10.1016/j.cca.2014.08.042 · Cited by 377
Very early phonological and language skills: estimating individual risk of reading disability
Puolakanaho, A., Ahonen, T., Aro, M., Eklund, K., Leppänen, P. H. T., Poikkeus, A., Tolvanen, A., Torppa, M., & Lyytinen, H. (2007) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01763.x · Cited by 316
Breast cancer diagnosis using GA feature selection and Rotation Forest
Aličković, E., & Subasi, A. (2017) · Neural Computing and Applications · doi:10.1007/s00521-015-2103-9 · Cited by 306
Lin, D., Feng, S., Pan, J., Chen, Y., Lin, J., Chen, G., Xie, S., Zeng, H., & Chen, R. (2011) · Optics Express · doi:10.1364/oe.19.013565 · Cited by 298
The Performance of Risk Prediction Models
Gerds, T. A., Cai, T., & Schumacher, M. (2008) · Biometrical Journal · doi:10.1002/bimj.200810443 · Cited by 288
Honda, H., Qureshi, A. R., Axelsson, J., Heimburger, O., Suliman, M. E., Barany, P., Stenvinkel, P., & Lindholm, B. (2007) · The American Journal of Clinical Nutrition · doi:10.1093/ajcn/86.3.633 · Cited by 282
General Retinal Vessel Segmentation Using Regularization-Based Multiconcavity Modeling
Lam, B. S. Y., Gao, Y., & Liew, A. W.-C. (2010) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2043259 · Cited by 275
A novel feature selection approach for biomedical data classification
Peng, Y., Wu, Z., & Jiang, J. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.008 · Cited by 257
El Khouli, R. H., Macura, K. J., Barker, P. B., Habba, M. R., Jacobs, M. A., & Bluemke, D. A. (2009) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.21947 · Cited by 213
Artificial Intelligence Techniques for Automated Diagnosis of Neurological Disorders
Raghavendra, U., Acharya, U. R., & Adeli, H. (2019) · European Neurology · doi:10.1159/000504292 · Cited by 174
Methods for clinical evaluation of artificial intelligence algorithms for medical diagnosis
Park, S. H., Han, K., Jang, H. Y., Park, J. E., Lee, J.-G., Kim, D. W., & Choi, J. (2023) · Radiology · doi:10.1148/radiol.220182 · Cited by 163
Behzadi-khormouji, H., Rostami, H., Salehi, S., Derakhshande-Rishehri, T., Masoumi, M., Salemi, S., Keshavarz, A., Gholamrezanezhad, A., Assadi, M., & Batouli, A. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2019.105162 · Cited by 112
O’Caoimh, R., Cornally, N., Weathers, E., O'Sullivan, R., Fitzgerald, C., Orfila, F., Clarnette, R., Paúl, C., & Molloy, D. W. (2015) · Maturitas · doi:10.1016/j.maturitas.2015.03.009 · Cited by 89
“8, the value considered for good to excellent accuracy [51].”
BackgroundCerebrospinal fluid α‐synuclein contributes to the differential diagnosis of Alzheimer's disease
Shi, M., Tang, L., Toledo, J. B., Ginghina, C., Wang, H., Aro, P., Jensen, P. H., Weintraub, D., Chen‐Plotkin, A. S., Irwin, D. J., et al. (2018) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2018.02.015 · Cited by 42
Predicting patient demographics from chest Radiographs with deep learning
Adleberg, J., Wardeh, A., Doo, F. X., Marinelli, B., Cook, T. S., Mendelson, D. S., & Kagen, A. (2022) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2022.06.008 · Cited by 32
Functional brain abnormalities in major depressive disorder using the Hilbert-Huang transform
Yu, H., Li, F., Wu, T., Li, R., Yao, L., Wang, C., & Wu, X. (2018) · Brain Imaging and Behavior · doi:10.1007/s11682-017-9816-6 · Cited by 17
Metadata
- Authors
- Obuchowski, N. A.
- Venue
- Radiology
- Published
- 2003-10
- Volume
- 229
- Issue
- 1
- Pages
- 3-8
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W2124537635
- Cited by
- 843
- References
- 15
- FWCI
- 14.74
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.