The DOI matches its Crossref record; 245 of the 252 references it cites were found in a registry or an index.
Machine learning with a reject option: a survey
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In order to assess models' ability to refrain from answering a question when uncertain [14], we also include an additional multiple choice answer for each question:”
From LAB-Bench: Measuring Capabilities of Language Models for Biology Research (2024) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1996) · Cited by 20,179
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,759
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008) · 2008 Eighth IEEE International Conference on Data Mining · doi:10.1109/icdm.2008.17 · Cited by 6,601
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791
Methods of combining multiple classifiers and their applications to handwriting recognition
Xu, L., Krzyżak, A., & Suen, C. Y. (1992) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.155943 · Cited by 2,192
Kowalska, M. K., Woźniak, M., Kijek, M. M., Mitrosz, P., Szakiel, J., & Turek, P. (2022) · Scientific Reports · doi:10.1038/s41598-021-99269-x · Cited by 2,121
Support Vector Method for Novelty Detection
Schölkopf, B., Williamson, R. C., Smola, A. J., Shawe-Taylor, J. S., & Platt, J. C. (1999) · Cited by 2,038
Efficient algorithms for mining outliers from large data sets
Ramaswamy, S., Rastogi, R. R., & Shim, K. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335437 · Cited by 1,908
Pimentel, M. A. F., Clifton, D. A., Clifton, L., & Tarassenko, L. (2014) · Signal Processing · doi:10.1016/j.sigpro.2013.12.026 · Cited by 1,649
Research works printed but not linked: 13
- Alhoniemi, E., Himberg, J., & Vesanto, J. (1999). Probabilistic measures for responses of self-organizing map units. In H. Bothe, E. Oja, E. Massad and C. Haefke, (eds.), Proceedings of the international ICSC congress on computational intelligence methods and applications. ICSC Academic Press, Zurich, Switzerland, (pp. 286–290). ICSC Academic Press.
- Cao, Y., Cai, T., & An, B. (2022). Generalizing consistent multi-class classification with rejection to be compatible with arbitrary losses. (NeurIPS), (pp. 1–14).
- Ferri, C., & Hernández-Orallo, J. (2004). Cautious lassifiers. Proceedings of ROC analysis in artificial intelligence, 1st international workshop (ROCAI-2004), (pp. 27–36).
- Frélicot, C. (1997). Learning rejection thresholds for a class of fuzzy classifiers from possibilistic clustered noisy data. In 7th International fuzzy systems association world congress, IFSA, (vol. 3, pp. 111–116).
- Friedel, C. C., Rückert, U., & Kramer, S. (2006). Cost Curves for Abstaining Classifiers. In ICML 2006 - 3th workshop ROC analysis in ML, Pittsburgh, Pennsylvania.
- Hansen, L. K., Liisberg, C., & Salamon, P. (1997). The error-reject tradeoff. Open Systems & Information Dynamics, 4(2), 159–184. · DOI
- Landgrebe, T. C., Tax, D. M., Paclík, P., Duin, R. P., & Andrew, C. (2004). A combining strategy for ill-defined problems. In Proceedings of the 15th annual symposium of the pattern recognition association of South Africa, (pp. 57–62).
- Lee, J. K., Bu, Y., Rajan, D., Sattigeri, P., Panda, R., Das, S., & Wornell, G. W. (2021). Fair selective classification via sufficiency. In Proceedings of the 38th international conference on machine learning, (vol. 139, pp. 6076–6086).
- Nalisnick, E., Matsukawa, A., Whye, Y., Dilan, T., & Balaji, G. (2019). Hybrid models with deep and invertible features (ICML, 2019).
- Prasad, N. R., Almanza-Garcia, S., & Lu, T. T. (2009). Anomaly detection. Computers, Materials and Continua, 14(1), 1–22.
- Van der Plas, D., Meert, W., Verbraecken, J., & Davis, J. (2021). A reject option for automated sleep stage scoring. In ICML workshop on interpretable machine learning in healthcare, Virtual event.
- Vanschoren, J. (2018). Meta-learning: A survey. pp. 1–29.
- Zhou, L., Martínez-Plumed, F., Hernández-Orallo, J., Ferri, C., & Schellaert, W. (2022). Reject before you run: Small assessors anticipate big language models. In CEUR workshop proceedings, (vol. 3169).
Cited by
Sorted by citations
Policy advice and best practices on bias and fairness in AI
Álvarez, J. M., Colmenarejo, A. B., Elobaid, A., Fabbrizzi, S., Fahimi, M., Ferrara, A., Ghodsi, S., Mougan, C., Papageorgiou, I. T., Lobo, P. R., et al. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09746-w · Cited by 147
Methodology and evaluation in sports analytics: challenges, approaches, and lessons learned
Davis, J., Bransen, L., Devos, L., Jaspers, A., Meert, W., Robberechts, P., Haaren, J. V., & Roy, M. V. (2024) · Machine-mediated learning · doi:10.1007/s10994-024-06585-0 · Cited by 94
Self-citationTowards Human-AI Teaming to Mitigate Alert Fatigue in Security Operations Centres
Chhetri, M. B., Tariq, S., Singh, R., Jalalvand, F., Paris, C. L., & Nepal, . (2024) · ACM Transactions on Internet Technology · doi:10.1145/3670009 · Cited by 70
Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection
Yong, B. X., & Brintrup, A. M. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118196 · Cited by 55
A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities
Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., & Yang, J. (2024) · ACM Computing Surveys · doi:10.1145/3665926 · Cited by 77
Metadata
- Authors
- Hendrickx, K., Perini, L., Van der Plas, D., Meert, W., & Davis, J.
- Affiliations
- Siemens (Belgium) (BE)
- KU Leuven (BE)
- Corresponding author
- Hendrickx, Kilian
- Venue
- Machine Learning
- Published
- 2024-03-29
- Volume
- 113
- Issue
- 5
- Pages
- 3073-3110
- ISSN
- 0885-6125 (ISSN-L)
- 1573-0565
- OpenAlex
- W3183516990
- Cited by
- 134
- References
- 252
- Citations by year
- 2026: 59 · 2025: 31 · 2024: 23
- FWCI
- 16.68
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Machine Learning (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- agentschap innoveren en ondernemen (HBC.2017.0226, HBC.2019.2615)
- onderzoeksraad, ku leuven (C14/17/070)
- vlaamse overheid (Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- abstention, decision support
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
