The DOI matches its Crossref record.
ROCR: visualizing classifier performance in R
Open accessAbstract
UNLABELLED: ROCR is a package for evaluating and visualizing the performance of scoring classifiers in the statistical language R. It features over 25 performance measures that can be freely combined to create two-dimensional performance curves. Standard methods for investigating trade-offs between specific performance measures are available within a uniform framework, including receiver operating characteristic (ROC) graphs, precision/recall plots, lift charts and cost curves. ROCR integrates tightly with R's powerful graphics capabilities, thus allowing for highly adjustable plots. Being equipped with only three commands and reasonable default values for optional parameters, ROCR combines flexibility with ease of usage. AVAILABILITY: http://rocr.bioinf.mpi-sb.mpg.de. ROCR can be used under the terms of the GNU General Public License. Running within R, it is platform-independent. CONTACT: tobias.sing@mpi-sb.mpg.de.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
GSVA: gene set variation analysis for microarray and RNA-Seq data
Hänzelmann, S., Castelo, R., & Guinney, J. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-7 · Cited by 15,440
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
SoilGrids250m: Global gridded soil information based on machine learning
Hengl, T., Mendes de Jesus, J., Heuvelink, G. B. M., Ruiperez Gonzalez, M., Kilibarda, M., Blagotić, A., Shangguan, W., Wright, M. N., Geng, X., Bauer-Marschallinger, B., et al. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169748 · Cited by 4,733
McGinnis, C. S., Murrow, L. M., & Gartner, Z. J. (2019) · Cell Systems · doi:10.1016/j.cels.2019.03.003 · Cited by 4,521
REVEL: An Ensemble Method for Predicting the Pathogenicity of Rare Missense Variants
Ioannidis, N. M., Rothstein, J. H., Pejaver, V., Middha, S., McDonnell, S. K., Baheti, S., Musolf, A., Li, Q., Holzinger, E., Karyadi, D., et al. (2016) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2016.08.016 · Cited by 2,908
Multivariable association discovery in population-scale meta-omics studies
Mallick, H., Rahnavard, A., McIver, L. J., Ma, S., Zhang, Y., Nguyen, L. H., Tickle, T. L., Weingart, G., Ren, B., Schwager, E. H., et al. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009442 · Cited by 2,596
Extinction risk and conservation of the world’s sharks and rays
Dulvy, N. K., Fowler, S. L., Musick, J. A., Cavanagh, R. D., Kyne, P. M., Harrison, L. R., Carlson, J. K., Davidson, L. N., Fordham, S. V., Francis, M. P., et al. (2014) · eLife · doi:10.7554/elife.00590 · Cited by 2,047
Single-cell chromatin state analysis with Signac
Stuart, T., Srivastava, A., Madad, S., Lareau, C. A., & Satija, R. (2021) · Nature Methods · doi:10.1038/s41592-021-01282-5 · Cited by 1,769
A data-driven approach to preprocessing Illumina 450K methylation array data
Pidsley, R., Y Wong, C. C., Volta, M., Lunnon, K., Mill, J., & Schalkwyk, L. C. (2013) · BMC Genomics · doi:10.1186/1471-2164-14-293 · Cited by 1,143
Yu, K.-H., Zhang, C., Berry, G. J., Altman, R. B., Ré, C., Rubin, D. L., & Snyder, M. (2016) · Nature Communications · doi:10.1038/ncomms12474 · Cited by 972
Sela, I., Ashkenazy, H., Katoh, K., & Pupko, T. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv318 · Cited by 835
SWAN: Subset-quantile Within Array Normalization for Illumina Infinium HumanMethylation450 BeadChips
Maksimovic, J., Gordon, L., & Oshlack, A. (2012) · Genome Biology · doi:10.1186/gb-2012-13-6-r44 · Cited by 826
Biological interpretation of genome-wide association studies using predicted gene functions
Pers, T. H., Karjalainen, J. M., Chan, Y., Westra, H.-J., Wood, A. R., Yang, J., Lui, J. C., Vedantam, S., Gustafsson, S., Esko, T., et al. (2015) · Nature Communications · doi:10.1038/ncomms6890 · Cited by 787
Polymorphisms Associated with Cholesterol and Risk of Cardiovascular Events
Kathiresan, S., Melander, O., Anevski, D., Guiducci, C., Burtt, N. P., Roos, C., Hirschhorn, J. N., Berglund, G., Hedblad, B., Groop, L., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0706728 · Cited by 696
(2011) · Cited by 675
Uejio, C. K., Wilhelmi, O. V., Golden, J. S., Mills, D. M., Gulino, S. P., & Samenow, J. P. (2011) · Health & Place · doi:10.1016/j.healthplace.2010.12.005 · Cited by 504
Temporal Response of the Human Virome to Immunosuppression and Antiviral Therapy
De Vlaminck, I., Khush, K. K., Strehl, C., Kohli, B., Luikart, H., Neff, N. F., Okamoto, J., Snyder, T. M., Cornfield, D. N., Nicolls, M. R., et al. (2013) · Cell · doi:10.1016/j.cell.2013.10.034 · Cited by 471
Diagnosis-Independent Alzheimer Disease Biomarker Signature in Cognitively Normal Elderly People
De Meyer, G. (2010) · Archives of Neurology · doi:10.1001/archneurol.2010.179 · Cited by 462
PresenceAbsence : An R Package for Presence Absence Analysis
Freeman, E. A., & Moisen, G. (2008) · Journal of Statistical Software · doi:10.18637/jss.v023.i11 · Cited by 461
YAP1 Exerts Its Transcriptional Control via TEAD-Mediated Activation of Enhancers
Stein, C., Bardet, A. F., Roma, G., Bergling, S., Clay, I., Ruchti, A., Agarinis, C., Schmelzle, T., Bouwmeester, T., Schübeler, D., et al. (2015) · PLOS Genetics · doi:10.1371/journal.pgen.1005465 · Cited by 414
A Pilot Study of Circulating miRNAs as Potential Biomarkers of Early Stage Breast Cancer
Zhao, H., Shen, J., Medico, L., Wang, D., Ambrosone, C. B., & Liu, S. (2010) · PLoS ONE · doi:10.1371/journal.pone.0013735 · Cited by 398
Pirracchio, R., Petersen, M. L., Carone, M., Rigon, M. R., Chevret, S., & van der Laan, M. J. (2015) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(14)70239-5 · Cited by 374
Miller, R., Plessow, F., Kirschbaum, C., & Stalder, T. (2013) · Psychosomatic Medicine · doi:10.1097/psy.0000000000000002 · Cited by 376
In vivo high-throughput profiling of CRISPR-Cpf1 activity
Kim, H. K., Song, M., Lee, J., Menon, A. V., Jung, S., Kang, Y.-M., Choi, J. W., Woo, E., Koh, H. C., Nam, J.-W., et al. (2017) · Nature Methods · doi:10.1038/nmeth.4104 · Cited by 383
The genetic interpretation of area under the ROC curve in genomic profiling
Wray, N. R., Yang, J., Goddard, M. E., & Visscher, P. M. (2010) · PLoS Genetics · doi:10.1371/journal.pgen.1000864 · Cited by 334
Human pluripotent stem cell-derived neural constructs for predicting neural toxicity
Schwartz, M. P., Hou, Z., Propson, N. E., Zhang, J., Engstrom, C. J., Costa, V. S., Jiang, P., Nguyen, B. K., Bolin, J. M., Daly, W., et al. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1516645112 · Cited by 325
AF2Complex predicts direct physical interactions in multimeric proteins with deep learning
Gao, M., Nakajima An, D., Parks, J. M., & Skolnick, J. (2022) · Nature Communications · doi:10.1038/s41467-022-29394-2 · Cited by 252
Shung, D. L., Au, B., Taylor, R. A., Tay, J. K., Laursen, S. B., Stanley, A. J., Dalton, H. R., Ngu, J., Schultz, M., & Laine, L. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.09.009 · Cited by 227
Hicks, S., Wheeler, D. A., Plon, S. E., & Kimmel, M. (2011) · Human Mutation · doi:10.1002/humu.21490 · Cited by 223
Ultraconserved Enhancers Are Required for Normal Development
Dickel, D. E., Ypsilanti, A. R., Pla, R., Zhu, Y., Barozzi, I., Mannion, B. J., Khin, Y. S., Fukuda-Yuzawa, Y., Plajzer-Frick, I., Pickle, C. S., et al. (2018) · Cell · doi:10.1016/j.cell.2017.12.017 · Cited by 190
Hwang, S., Kwon, A.-Y., Jeong, J.-Y., Kim, S., Kang, H., Park, J., Kim, J.-H., Han, O. J., Lim, S. M., & An, H. J. (2020) · Scientific Reports · doi:10.1038/s41598-019-57218-9 · Cited by 215
Identification of a three-biomarker panel in urine for early detection of pancreatic adenocarcinoma
Radon, T. P., Massat, N. J., Jones, R., Alrawashdeh, W., Dumartin, L., Ennis, D., Duffy, S. W., Kocher, H. M., Pereira, S. P., Guarner (posthumous), L., et al. (2015) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-14-2467 · Cited by 203
Haas, J., Barbato, A., Behringer, D., Studer, G., Roth, S., Bertoni, M., Mostaguir, K., Gumienny, R., & Schwede, T. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25431 · Cited by 168
“2 was created with the ROCR package[35].”
Marchese Robinson, R. L., Palczewska, A., Palczewski, J., & Kidley, N. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00753 · Cited by 167
Kautzky, A., Dold, M., Bartova, L., Spies, M., Kranz, G. S., Souery, D., Montgomery, S., Mendlewicz, J., Zohar, J., Fabbri, C., et al. (2019) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12959 · Cited by 148
Verbakel, J. Y., Steyerberg, E. W., Uno, H., De Cock, B., Wynants, L., Collins, G. S., & Van Calster, B. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.028 · Cited by 138
Sex estimation using sexually dimorphic amelogenin protein fragments in human enamel
Parker, G. J., Yip, J. M., Eerkens, J. W., Salemi, M., Durbin-Johnson, B., Kiesow, C., Haas, R., Buikstra, J. E., Klaus, H., Regan, L. A., et al. (2019) · Journal of Archaeological Science · doi:10.1016/j.jas.2018.08.011 · Cited by 138
Serum neurofilament light chain levels are related to small vessel disease burden
Duering, M., Konieczny, M. J., Tiedt, S., Baykara, E., Tuladhar, A. M., Leijsen, E. V., Lyrer, P., Engelter, S. T., Gesierich, B., Achmüller, M., et al. (2018) · Journal of Stroke · doi:10.5853/jos.2017.02565 · Cited by 132
Caveats and pitfalls of ROC analysis in clinical microarray research (and how to avoid them)
Berrar, D., & Flach, P. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbr008 · Cited by 136
Walia, R. R., Xue, L. C., Wilkins, K., El-Manzalawy, Y., Dobbs, D., & Honavar, V. (2014) · PLoS ONE · doi:10.1371/journal.pone.0097725 · Cited by 132
Klimenko, N. S., Tyakht, A. V., Popenko, A. S., Vasiliev, A. S., Altukhov, I. A., Ischenko, D. S., Shashkova, T. I., Efimova, D. A., Nikogosov, D. A., Osipenko, D. A., et al. (2018) · Nutrients · doi:10.3390/nu10050576 · Cited by 123
Detection of selective sweeps in structured populations: a comparison of recent methods
Vatsiou, A. I., Bazin, E., & Gaggiotti, O. E. (2016) · Molecular Ecology · doi:10.1111/mec.13360 · Cited by 111
Pluta, J., Yushkevich, P., Das, S., & Wolk, D. (2012) · Journal of Alzheimer's Disease · doi:10.3233/jad-2012-111931 · Cited by 107
Generally applicable transcriptome-wide analysis of translation using anota2seq
Oertlin, C., Lorent, J., Murie, C., Furic, L., Topisirovic, I., & Larsson, O. (2019) · Nucleic Acids Research · doi:10.1093/nar/gkz223 · Cited by 107
LOGICOIL—multi-state prediction of coiled-coil oligomeric state
Vincent, T. L., Green, P. J., & Woolfson, D. N. (2013) · Bioinformatics · doi:10.1093/bioinformatics/bts648 · Cited by 103
Im, H. K., Gamazon, E. R., Nicolae, D. L., & Cox, N. J. (2012) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2012.02.008 · Cited by 103
Poulos, R. C., Olivier, J., & Wong, J. W. H. (2017) · Nucleic Acids Research · doi:10.1093/nar/gkx463 · Cited by 93
RNA‐seq Data: Challenges in and Recommendations for Experimental Design and Analysis
Williams, A. G., Thomas, S., Wyman, S. K., & Holloway, A. K. (2014) · Current Protocols in Human Genetics · doi:10.1002/0471142905.hg1113s83 · Cited by 93
HiNT: a computational method for detecting copy number variations and translocations from Hi-C data
Wang, S., Lee, S., Chu, C., Jain, D., Kerpedjiev, P., Nelson, G. M., Walsh, J. M., Alver, B. H., & Park, P. J. (2020) · Genome Biology · doi:10.1186/s13059-020-01986-5 · Cited by 99
Statistical Shape Modeling of the Left Ventricle: Myocardial Infarct Classification Challenge
Suinesiaputra, A., Dhooge, J., Duchateau, N., Ehrhardt, J., Frangi, A. F., Gooya, A., Grau, V., Lekadir, K., Lu, A., Mukhopadhyay, A., et al. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2652449 · Cited by 95
González-Torrecilla, E., Almendral, J., Arenal, A., Atienza, F., Atea, L. F., del Castillo, S., & Fernández-Avilés, F. (2009) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.02.059 · Cited by 89
Alexandre, P. M., Stewart, S. I., Mockrin, M. H., Keuler, N. S., Syphard, A. D., Bar-Massada, A., Clayton, M. K., & Radeloff, V. C. (2016) · Landscape Ecology · doi:10.1007/s10980-015-0257-6 · Cited by 83
Marko, N. F., Gonugunta, V. A., Hamrahian, A. H., Usmani, A., Mayberg, M. R., & Weil, R. J. (2009) · Journal of Neurosurgery · doi:10.3171/2008.12.jns081265 · Cited by 67
The effect of ligand-based tautomer and protomer prediction on structure-based virtual screening.
Kalliokoski, T., Salo, H. S., Lahtela-Kakkonen, M., & Poso, A. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900364w · Cited by 66
Laengsri, V., Shoombuatong, W., Adirojananon, W., Nantasenamat, C., Prachayasittikul, V., & Nuchnoi, P. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0929-2 · Cited by 66
Penman, S. H., Price, O. F., Penman, T. D., & Bradstock, R. A. (2019) · International Journal of Wildland Fire · doi:10.1071/wf18046 · Cited by 66
Turner, O. C., Aeffner, F., Bangari, D. S., High, W., Knight, B., Forest, T., Cossic, B., Himmel, L. E., Rudmann, D. G., Bawa, B., et al. (2020) · Toxicologic Pathology · doi:10.1177/0192623319881401 · Cited by 64
Self-citationC-Reactive Protein: A Differential Biomarker For Major Depressive Disorder and Bipolar II Disorder
Chang, H. H., Wang, T.-Y., Lee, I. H., Lee, S.-Y., Chen, K. C., Huang, S.-Y., Yang, Y. K., Lu, R.-B., & Chen, P. S. (2017) · The World Journal of Biological Psychiatry · doi:10.3109/15622975.2016.1155746 · Cited by 61
Stocco, A., Prat, C. S., Losey, D. M., Cronin, J. A., Wu, J., Abernethy, J. A., & Rao, R. P. N. (2015) · PLOS ONE · doi:10.1371/journal.pone.0137303 · Cited by 57
Kebede, M. M., Le Cornet, C., & Fortner, R. T. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1589 · Cited by 50
von Glischinski, M., Willutzki, U., Stangier, U., Hiller, W., Hoyer, J., Leibing, E., Leichsenring, F., & Hirschfeld, G. (2018) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2179 · Cited by 54
Ho, R., Workman, M. J., Mathkar, P., Wu, K., Kim, K. J., O’Rourke, J. G., Kellogg, M., Montel, V., Banuelos, M. G., Arogundade, O. A., et al. (2021) · Cell Systems · doi:10.1016/j.cels.2020.10.010 · Cited by 62
SCORER 2.0: an algorithm for distinguishing parallel dimeric and trimeric coiled-coil sequences
Armstrong, C. T., Vincent, T. L., Green, P. J., & Woolfson, D. N. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr299 · Cited by 49
Khoueiry, P., Girardot, C., Ciglar, L., Peng, P.-C., Gustafson, E. H., Sinha, S., & Furlong, E. E. (2017) · eLife · doi:10.7554/elife.28440 · Cited by 44
Norris, P. R., Canter, J. A., Jenkins, J. M., Moore, J. H., Williams, A. E., & Morris, J. A. (2009) · Annals of Surgery · doi:10.1097/sla.0b013e3181b8fb1f · Cited by 40
Conceptualizing vacation dedication
Ring, A., Yanamandram, V., Gretzel, U., & Dolnicar, S. (2020) · Current Issues in Tourism · doi:10.1080/13683500.2019.1678572 · Cited by 34
Gray, M. E., & Dickson, B. G. (2016) · Landscape Ecology · doi:10.1007/s10980-016-0353-2 · Cited by 31
How difficult is inference of mammalian causal gene regulatory networks?
Djordjevic, D., Yang, A., Zadoorian, A., Rungrugeecharoen, K., & Ho, J. W. K. (2014) · PLoS ONE · doi:10.1371/journal.pone.0111661 · Cited by 29
Multiscale consensus habitat modeling for landscape level conservation prioritization
Poor, E. E., Scheick, B. K., & Mullinax, J. M. (2020) · Scientific Reports · doi:10.1038/s41598-020-74716-3 · Cited by 27
Navigating a 2D Virtual World Using Direct Brain Stimulation
Losey, D. M., Stocco, A., Abernethy, J. A., & Rao, R. P. N. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00072 · Cited by 19
Yun, J. W., Lee, S., Kim, H. M., Chun, S., Engleman, E. G., Kim, H. C., & Kang, E.-S. (2019) · Cancers · doi:10.3390/cancers11081157 · Cited by 18
Huang, A. Y., Zhou, Z., Talukdar, M., Miller, M. B., Chhouk, B., Enyenihi, L., Rosen, I., Stronge, E., Zhao, B., Kim, D., et al. (2024) · doi:10.1101/2024.01.03.574078 · Cited by 16
Huang, A. Y., Zhou, Z., Talukdar, M., Enyenihi, L., Miller, M. B., Chhouk, B., Rosen, I., Zheng, M., Zhou, M., Yang, A., et al. (2026) · Cell · doi:10.1016/j.cell.2026.03.040 · Cited by 15
Dhami, B., Bhusal, A., Neupane, B., Kc, N., Lamichhane, S., Bhattarai, D., & Shrestha, B. (2023) · Global Ecology and Conservation · doi:10.1016/j.gecco.2023.e02648 · Cited by 11
Machine learning classification of new firearm injury encounters in the St Louis region: 2010-2020
Ancona, R. M., Cooper, B. P., Foraker, R., Kaser, T., Adeoye, O., & Mueller, K. L. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae173 · Cited by 7
GAO, H., ZHENG, Z., YUE, Z., LIU, F., ZHOU, L., & ZHAO, X. (2012) · International Journal of Molecular Medicine · doi:10.3892/ijmm.2012.1113 · Cited by 7
Metadata
- Authors
- Sing, T., Sander, O., Beerenwinkel, N., & Lengauer, T.
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2005-08-11
- Volume
- 21
- Issue
- 20
- Pages
- 3940-3941
- ISSN
- 0266-7061 (ISSN-L)
- 1367-4803
- 1460-2059
- OpenAlex
- W2145126338
- Cited by
- 3,200
- References
- 0
- FWCI
- 30.92
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.