The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
OptimalCutpoints: An R Package for Selecting Optimal Cutpoints in Diagnostic Tests
Open accessAbstract
Continuous diagnostic tests are often used for discriminating between healthy and diseased populations. For the clinical application of such tests, it is useful to select a cutpoint or discrimination value c that defines positive and negative test results. In general, individuals with a diagnostic test value of c or higher are classified as diseased. Several search strategies have been proposed for choosing optimal cutpoints in diagnostic tests, depending on the underlying reason for this choice. This paper introduces an R package, known as OptimalCutpoints, for selecting optimal cutpoints in diagnostic tests. It incorporates criteria that take the costs of the different diagnostic decisions into account, as well as the prevalence of the target disease and several methods based on measures of diagnostic test accuracy. Moreover, it enables optimal levels to be calculated according to levels of given (categorical) covariates. While the numerical output includes the optimal cutpoint values and associated accuracy measures with their confidence intervals, the graphical output includes the receiver operating characteristic (ROC) and predictive ROC curves. An illustration of the use of OptimalCutpoints is provided, using a real biomedical dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · cited 1×
“…yielded by the summary method always includes: the matched call to the main function optimal.cutpoints ; the AUC value with its confidence interval (Delong, Delong, and Clarke-Pearson 1988); the method used for selecting the optimal value together with the number of optimal cutpoints (in some…”
MethodHosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
(1966) · Medical Entomology and Zoology · Cited by 10,293
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 8,990
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine
Zweig, M. H., & Campbell, G. (1993) · Clinical Chemistry · doi:10.1093/clinchem/39.4.561 · Cited by 6,569
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
THE USE OF CONFIDENCE OR FIDUCIAL LIMITS ILLUSTRATED IN THE CASE OF THE BINOMIAL
CLOPPER, C. J., & PEARSON, E. S. (1934) · Biometrika · doi:10.1093/biomet/26.4.404 · Cited by 5,003
Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions
Agresti, A., & Coull, B. A. (1998) · The American Statistician · doi:10.1080/00031305.1998.10480550 · Cited by 3,223
The Statistical Evaluation of Medical Tests for Classification and Prediction
Pepe, M. S. (2003) · doi:10.1093/oso/9780198509844.001.0001 · Cited by 2,198
Greiner, M., Pfeiffer, D., & Smith, R. D. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00115-x · Cited by 1,923
The Statistical Evaluation of Medical Tests for Classification and Prediction
(2005) · Technometrics · doi:10.1198/tech.2005.s278 · Cited by 1,899
Evaluating presence–absence models in ecology: the need to account for prevalence
Manel, S., Williams, H. C., & Ormerod, S. J. (2001) · Journal of Applied Ecology · doi:10.1046/j.1365-2664.2001.00647.x · Cited by 1,759
Perkins, N. J., & Schisterman, E. F. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj063 · Cited by 1,759
Bamber, D. (1975) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(75)90001-2 · Cited by 1,439
(1982) · Cited by 1,288
Schisterman, E. F., Perkins, N. J., Liu, A., & Bondell, H. (2005) · Epidemiology · doi:10.1097/01.ede.0000147512.81966.ba · Cited by 1,185
Dangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143
Lewis, J. D., Chuai, S., Nessel, L., Lichtenstein, G. R., Aberra, F. N., & Ellenberg, J. H. (2008) · Inflammatory Bowel Diseases · doi:10.1002/ibd.20520 · Cited by 964
Likelihood ratios with confidence: Sample size estimation for diagnostic test studies
Simel, D. L., Samsa, G. P., & Matchar, D. B. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90128-v · Cited by 932 · cited 4×
“…"Exact" (Clopper and Pearson 1934), "Quadratic" (Fleiss 1981), "Wald" (Wald and Wolfowitz 1939), "AgrestiCoull" (Agresti and Coull 1998), "RubinSchenker" (Rubin and Schenker 1987), "Transformed" (Simel, Samsa, and Matchar 1991), "NotTransformed" (Koop-man 1984) and "GartNam" (Gart and Nam 1998).”
MethodInfluentialAssessment of diagnostic tests when disease verification is subject to selection bias
Begg, C. B., & Greenes, R. A. (1983) · Biometrics · doi:10.2307/2530820 · Cited by 708
ROC Analysis Applied to the Evaluation of Medical Imaging Techniques
SWETS, J. A. (1979) · Investigative Radiology · doi:10.1097/00004424-197903000-00002 · Cited by 699
Kappa coefficients in medical research
Chmura Kraemer, H., Periyakoil, V. S., & Noda, A. (2002) · Statistics in Medicine · doi:10.1002/sim.1180 · Cited by 527
Maximally Selected Chi Square Statistics
Miller, R., & Siegmund, D. (1982) · Biometrics · doi:10.2307/2529881 · Cited by 490
Introduction to Medical Decision Making.
(1969) · Annals of Internal Medicine · doi:10.7326/0003-4819-70-5-1051_1 · Cited by 478
PresenceAbsence : An R Package for Presence Absence Analysis
Freeman, E. A., & Moisen, G. (2008) · Journal of Statistical Software · doi:10.18637/jss.v023.i11 · Cited by 461
The interpretation of diagnostic tests
Shapiro, D. E. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800203 · Cited by 383
Mazumdar, M., & Glassman, J. R. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000115)19:1<113::aid-sim245>3.0.co;2-o · Cited by 367
Logistic Regression Analysis and Reporting: A Primer
Peng, C.-Y. J., & So, T.-S. H. (2002) · Understanding Statistics · doi:10.1207/s15328031us0101_04 · Cited by 311
Evaluating cutpoints for the MHI-5 and MCS using the GHQ-12: a comparison of five different methods
Kelly, M. J., Dunstan, F. D., Lloyd, K., & Fone, D. L. (2008) · BMC Psychiatry · doi:10.1186/1471-244x-8-10 · Cited by 264
Interpreting Validity Indexes for Diagnostic Tests: An Illustration Using the Berg Balance Test
Riddle, D. L., & Stratford, P. W. (1999) · Physical Therapy · doi:10.1093/ptj/79.10.939 · Cited by 229
Gart, J. J., & Nam, J.-M. (1988) · Biometrics · doi:10.2307/2531848 · Cited by 189
Confidence Limits for Continuous Distribution Functions
Wald, A., & Wolfowitz, J. (1939) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177732209 · Cited by 146
Why do so many prognostic factors fail to pan out?
Hilsenbeck, S. G., Clark, G. M., & McGuire, W. L. (1992) · Breast Cancer Research and Treatment · doi:10.1007/bf01840833 · Cited by 138
Multivariate Interpretation of Clinical Laboratory Data.
Sampson, A. R., Albert, A., & Harris, E. K. (1988) · Journal of the American Statistical Association · doi:10.2307/2288902 · Cited by 136
Greiner, M. (1996) · Journal of Immunological Methods · doi:10.1016/0022-1759(96)00013-0 · Cited by 135
Correcting for verification bias in studies of a diagnostic test's accuracy
Zhou, X. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700403 · Cited by 134
Boyko, E. J. (1994) · Medical Decision Making · doi:10.1177/0272989x9401400210 · Cited by 127
Greiner, M. (1995) · Journal of Immunological Methods · doi:10.1016/0022-1759(95)00078-o · Cited by 122
Spatio-temporal interaction with disease mapping
Sun, D., Tsutakawa, R. K., Kim, H., & He, Z. (2000) · Statistics in Medicine · doi:10.1002/1097-0258(20000815)19:15<1997::aid-sim511>3.0.co;2-c · Cited by 115
Receiver Operating Characteristic Studies and Measurement Errors
Coffin, M., & Sukhatme, S. (1997) · Biometrics · doi:10.2307/2533545 · Cited by 113
Evaluating the effect of optimized cutoff values in the assessment of prognostic factors
Lausen, B., & Schumacher, M. C. (1996) · Computational Statistics & Data Analysis · doi:10.1016/0167-9473(95)00016-x · Cited by 107
Gallop, R., Crits-Christoph, P. F., Muenz, L. R., & Tu, X. M. (2003) · Understanding Statistics · doi:10.1207/s15328031us0204_01 · Cited by 88
Böhning, D., Holling, H., & Patilea, V. (2010) · Statistical Methods in Medical Research · doi:10.1177/0962280210374532 · Cited by 85
Comparison of receiver operating characteristic curves on the basis of optimal operating points
Halpern, E. J., Albert, M., Krieger, A. Μ., Metz, C. E., & Maidment, A. D. A. (1996) · Academic Radiology · doi:10.1016/s1076-6332(96)80451-x · Cited by 83
Maximum likelihood estimators of sensitivity and specificity corrected for verification bias
Zhou, X. (1993) · Communication in Statistics- Theory and Methods · doi:10.1080/03610929308831209 · Cited by 79
A package for statistical analysis in epidemiology
Carstensen, B., & Plummer, M. (2016) · Cited by 72
Constructing a cut‐off point for a quantitative diagnostic test
Schäfer, H. J. (1989) · Statistics in Medicine · doi:10.1002/sim.4780081110 · Cited by 69
Hoffman, R. M., Clanon, D. L., Littenberg, B., et al. (2000) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2000.90907.x · Cited by 67
Effect of verification bias on positive and negative predictive values
Zhou, X. (1994) · Statistics in Medicine · doi:10.1002/sim.4780131705 · Cited by 66
Strategies for graphical threshold determination
Vermont, J., Bosson, J., François, P., Robert, C., Rueff, A., & Jacques Demongeot, A. (1991) · Computer Methods and Programs in Biomedicine · doi:10.1016/0169-2607(91)90072-2 · Cited by 63
An Exponential Model Used for optimal Threshold selection on ROC Curues
England, W. L. (1988) · Medical Decision Making · doi:10.1177/0272989x8800800208 · Cited by 62
Outcome-oriented Cutpoints in Analysis of Quantitative Exposures
Schulgen, G., Lausen, B., Olsen, J. H., & Schumacher, M. C. (1994) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117227 · Cited by 62
Logit-Based Interval Estimation for Binomial Data Using the Jeffreys Prior
Rubin, D. B., & Nathaniel Schenker, A. (1987) · Sociological Methodology · doi:10.2307/271031 · Cited by 57
Bortheiry, A. L., Malerbi, D. A. C., & Franco, L. J. (1994) · Diabetes Care · doi:10.2337/diacare.17.11.1269 · Cited by 55
Aoki, K., Misumi, J., Kimura, T., Wenyuan, Z., & Xie, T. (1997) · Journal of Epidemiology · doi:10.2188/jea.7.143 · Cited by 49
PresenceAbsence: An R Package for Presence Absence Analysis
Freeman, E. A., Service, U. F., Moisen, G. G., et al. (2007) · Cited by 48
Abt, E. (2010) · Evidence-Based Dentistry · doi:10.1038/sj.ebd.6400746 · Cited by 32
Optimal Cutpoint Estimation With Censored Data
Sima, C. S., & Gönen, M. (2013) · Journal of Statistical Theory and Practice · doi:10.1080/15598608.2013.772022 · Cited by 15
Estimating the accuracy of psychological scales using longitudinal data
Rutter, C. M. (2003) · Biostatistics · doi:10.1093/biostatistics/4.1.97 · Cited by 14
(1950) · doi:10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3
Cited by
Genome-wide association studies
Uffelmann, E., Huang, Q. Q., Munung, N. S., de Vries, J., Okada, Y., Martin, A. R., Martin, H. C., Lappalainen, T., & Posthuma, D. (2021) · Nature Reviews Methods Primers · doi:10.1038/s43586-021-00056-9 · Cited by 1,849
Receiver operating characteristic curve: overview and practical use for clinicians
Nahm, F. S. (2022) · Korean Journal of Anesthesiology · doi:10.4097/kja.21209 · Cited by 1,484
Mossman, S. A., Luft, M. J., Schroeder, H. K., Varney, S. T., Fleck, D. E., Barzman, D. H., Gilman, R., DelBello, M. P., & Strawn, J. R. (2017) · Annals of Clinical Psychiatry · doi:10.1177/104012371702900404 · Cited by 471
“26 To maximize sensitivity and specificity for each cutpoint, the minimum sensitivity and specificity were determined, where m i represented the minimums across cutpoints.”
easyROC: An Interactive Web-tool for ROC Curve Analysis Using R Language Environment
Goksuluk, D., Korkmaz, S., Zararsiz, G., & Karaagaoglu, A. (2016) · The R Journal · doi:10.32614/rj-2016-042 · Cited by 377
Freeman, D., Loe, B. S., Kingdon, D., Startup, H., Molodynski, A., Rosebrock, L., Brown, P., Sheaves, B., Waite, F., & Bird, J. C. (2021) · Psychological Medicine · doi:10.1017/s0033291719003155 · Cited by 288
Metadata
- Authors
- López-Ratón, M., Rodríguez-Álvarez, M. X., Suárez, C. C., & Sampedro, F. G.
- Venue
- Journal of Statistical Software
- Publisher
- Foundation for Open Access Statistic
- Published
- 2014
- Volume
- 61
- Issue
- 8
- ISSN
- 1548-7660 (ISSN-L)
- OpenAlex
- W1829632160
- Cited by
- 548
- References
- 66
- FWCI
- 27.50
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.