Receiver Operating Characteristic Curve in Diagnostic Test Assessment
Open accessAbstract
The performance of a diagnostic test in the case of a binary predictor can be evaluated using the measures of sensitivity and specificity. However, in many instances, we encounter predictors that are measured on a continuous or ordinal scale. In such cases, it is desirable to assess performance of a diagnostic test over the range of possible cutpoints for the predictor variable. This is achieved by a receiver operating characteristic (ROC) curve that includes all the possible decision thresholds from a diagnostic test result. In this brief report, we discuss the salient features of the ROC curve, as well as discuss and interpret the area under the ROC curve, and its utility in comparing two different tests or predictor variables of interest.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The receiver operating characteristic (ROC) curve is particularly useful for continuous or ordinal results when a single positive or negative threshold for a test is unclear.
“When a single positive or negative threshold for a test is unclear, the receiver operating characteristic (ROC) curve, a plot of sensitivity against specificity across all possible cutpoints, is particularly useful for continuous or ordinal results [197].”
From Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors (2026) · cites this paper as [197]
SNORD115 and SNORD116 have high discriminatory power to differentiate AD samples from NC.
“We applied ddPCR technique and identified SNORD115 and SNORD116 with a high discriminatory power to differentiate AD samples from NC.”
From Small nucleolar RNAs in plasma extracellular vesicles and their discriminatory power as diagnostic biomarkers of Alzheimer's disease (2021) · cites this paper as Mandrekar, 2010
The AUC values for SNORD115 (0.861), SNORD116 (0.886), and combined (0.947) exceeded the threshold of 0.8.
“The values of AUC for SNORD115–0.861, as well as for SNORD116–0.886 exceeded the threshold, thus according to widely accepted classification within the range of “outstanding” (Konstantinou et al., 2012; Mandrekar, 2010).”
From Small nucleolar RNAs in plasma extracellular vesicles and their discriminatory power as diagnostic biomarkers of Alzheimer's disease (2021) · cites this paper as Mandrekar, 2010
Cites
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305 · see citing works
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Shared computational principles for language processing in humans and deep language models
Goldstein, A., Zada, Z., Buchnik, E., Schain, M., Price, A., Aubrey, B., Nastase, S. A., Feder, A., Emanuel, D., Cohen, A., et al. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01026-4 · Cited by 447 · see citing works
Cabitza, F., Campagner, A., Soares, F., García de Guadiana-Romualdo, L., Challa, F., Sulejmani, A., Seghezzi, M., & Carobene, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106288 · Cited by 284 · see citing works
Brand, A. L., Lawler, P. E., Bollinger, J. G., Li, Y., Schindler, S. E., Li, M., Lopez, S., Ovod, V., Nakamura, A., Shaw, L. M., et al. (2022) · Alzheimer's Research & Therapy · doi:10.1186/s13195-022-01117-1 · Cited by 151 · see citing works
Lin, Y., Lau, H. C.-H., Liu, Y., Kang, X., Wang, Y., Ting, N. L.-N., Kwong, T. N.-Y., Han, J., Liu, W., Liu, C., et al. (2022) · Gastroenterology · doi:10.1053/j.gastro.2022.06.038 · Cited by 142 · see citing works
Huang, A. Y. Q., Lu, O. H. T., Huang, J. C. H., Yin, C. J., & Yang, S. J. H. (2020) · Interactive Learning Environments · doi:10.1080/10494820.2019.1636086 · Cited by 124 · see citing works
Practical early prediction of students’ performance using machine learning and eXplainable AI
Jang, Y., Choi, S., Jung, H., & Kim, H. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11120-6 · Cited by 114 · see citing works
Equitably Allocating Resources during Crises: Racial Differences in Mortality Prediction Models
Ashana, D. C., Anesi, G. L., Liu, V. X., Escobar, G. J., Chesley, C., Eneanya, N. D., Weissman, G. E., Miller, W. D., Harhay, M. O., & Halpern, S. D. (2021) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202012-4383oc · Cited by 98 · see citing works
Savage, T., Wang, J., Gallo, R., Boukil, A., Patel, V., Safavi-Naini, S. A. A., Soroush, A., & Chen, J. H. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae254 · Cited by 92 · see citing works
Screening for perinatal anxiety disorders: Room to grow
Fairbrother, N., Corbyn, B., Thordarson, D. S., Ma, A., & Surm, D. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.03.052 · Cited by 82 · see citing works
Prospective Development and Validation of the Computerized Adaptive Screen for Suicidal Youth
King, C. A., Brent, D., Grupp-Phelan, J., Casper, T. C., Dean, J. M., Chernick, L. S., Fein, J. A., Mahabee-Gittens, E. M., Patel, S. J., Mistry, R. D., et al. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4576 · Cited by 71 · see citing works
Adoption Factors of FinTech: Evidence from an Emerging Economy Country-Wide Representative Sample
Mahmud, K., Joarder, M. M. A., & Muheymin-Us-Sakib, K. (2022) · International Journal of Financial Studies · doi:10.3390/ijfs11010009 · Cited by 68 · see citing works
Blue, N. R., Beddow, M. E., Savabi, M., Katukuri, V. R., & Chao, C. R. (2018) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2018.08.011 · Cited by 54 · see citing works
Machine learning based analysis of speech dimensions in functional oropharyngeal dysphagia
Roldan-Vasco, S., Orozco-Duque, A., Suarez-Escudero, J. C., & Orozco-Arroyave, J. R. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106248 · Cited by 49 · see citing works
Fitz, N. F., Wang, J., Kamboh, M. I., Koldamova, R., & Lefterov, I. (2021) · Neurobiology of Disease · doi:10.1016/j.nbd.2021.105481 · Cited by 49 · see citing works
“The values of AUC for SNORD115–0.861, as well as for SNORD116–0.886 exceeded the threshold, thus according to widely accepted classification within the range of “outstanding” (Konstantinou et al., 2012; Mandrekar, 2010).”
InfluentialBurghaus, S., Drazic, P., Wölfler, M., Mechsner, S., Zeppernick, M., Meinhold‐Heerlein, I., Mueller, M. D., Rothmund, R., Vigano, P., Becker, C. M., et al. (2024) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.15062 · Cited by 38 · see citing works
Newman, M. G., Rackoff, G. N., Zhu, Y., & Kim, H. (2023) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2022.102662 · Cited by 42 · see citing works
Artificial Intelligence for Dermatopathology: Current Trends and the Road Ahead
Chen, S. B., & Novoa, R. A. (2022) · Seminars in Diagnostic Pathology · doi:10.1053/j.semdp.2022.01.003 · Cited by 37 · see citing works
Kapoor, A., Kim, J., Zeng, X., Harris, S. T., & Anderson, A. (2020) · DIGITAL HEALTH · doi:10.1177/2055207620918715 · Cited by 33 · see citing works
The Use of Deep Learning Software in the Detection of Voice Disorders: A Systematic Review
Barlow, J., Sragi, Z., Rivera‐Rivera, G., Al‐Awady, A., Daşdöğen, Ü., Courey, M. S., & Kirke, D. N. (2024) · Otolaryngology–Head and Neck Surgery · doi:10.1002/ohn.636 · Cited by 25 · see citing works
Bricker, J., Miao, Z., Mull, K., Santiago-Torres, M., & Vock, D. M. (2023) · Journal of Medical Internet Research · doi:10.2196/43629 · Cited by 22 · see citing works
Rinderknecht, M. D., & Klopfenstein, Y. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00482-9 · Cited by 23 · see citing works
Arif, N. M. N. A., Roslan, N. S., Ismail, S. B., Nayak, R. D., Jamian, M. R., Mohamad Ali Roshidi, A. S., Edward, T. C., Kamal, M. A., Mohd Amin, M. M., Shaari, S., et al. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168446 · Cited by 24 · see citing works
Davies, C., Lee, J. Y., Walter, J., Kim, D., Yu, L., Park, J., Blake, S., Kalluri, L., Cziraky, M., Stanek, E., et al. (2022) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.10194 · Cited by 23 · see citing works
Campa, F., Bongiovanni, T., Rossi, A., Cerullo, G., Casolo, A., Martera, G., Trecroci, A., Moro, T., & Paoli, A. (2023) · Journal of Translational Medicine · doi:10.1186/s12967-023-04795-z · Cited by 16 · see citing works
“In general, an AUC of lower than 0.7 suggests no discrimination, from 0.7 to 0.8 is considered acceptable, from 0.8 to 0.9 is considered excellent, and more than 0.9 is considered outstanding [41].”
BackgroundInfluentialVillegas García, L., Patró, E., Barbero, J. D., Esteve-Valverde, E., Palao, D. J., Soria, V., Labad, J., & Cobo, J. (2025) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2024.107209 · Cited by 15 · see citing works
Large Language Model Uncertainty Measurement and Calibration for Medical Diagnosis and Treatment
Savage, T., Wang, J., Gallo, R., Boukil, A., Patel, V., Ahmad Safavi-Naini, S. A., Soroush, A., & Chen, J. H. (2024) · doi:10.1101/2024.06.06.24308399 · Cited by 11 · see citing works
Celepli, S., Bigat, I., Karakas, B., Tezcan, H. M., Yar, M. D., Celepli, P., Aksahin, M. F., Hancerliogullari, O., Yilmaz, Y. F., & Erogul, O. (2025) · Diagnostics · doi:10.3390/diagnostics15162065 · Cited by 6 · see citing works
Toward Safer Diagnoses: A SEIPS-Based Narrative Review of Diagnostic Errors
Yen, C., Epling, J. W., Rockwell, M., & Vaughn-Cooke, M. (2026) · Diagnostics · doi:10.3390/diagnostics16020347 · Cited by 6 · see citing works
- Authors
- Mandrekar, J. N.
- Affiliations
- Mayo Clinic (US)
- Corresponding author
- Mandrekar, Jayawant N.
- Venue
- Journal of Thoracic Oncology
- Publisher
- Elsevier BV
- Published
- 2010-09
- Volume
- 5
- Issue
- 9
- Pages
- 1315-1316
- ISSN
- 1556-0864 (ISSN-L)
- OpenAlex
- W1980276147
- Cited by
- 4,506
- References
- 2
- Citations by year
- 2026: 476 · 2025: 704 · 2024: 830 · 2023: 765 · 2022: 759 · 2021: 573 · 2020: 236 · 2019: 94 · 2018: 37 · 2017: 6 · 2016: 3 · 2015: 3 · 2014: 4 · 2013: 4 · 2012: 5
- FWCI
- 3.82
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Thoracic Oncology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- receiver operating characteristic, diagnostic performance, continuous predictors, decision threshold, area under the ROC curve, sensitivity and specificity, test comparison
- MeSH terms
- Diagnostic Tests, Routine, Humans, Neoplasms, Prognosis, ROC Curve, Sensitivity and Specificity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review