Calibrated Learning to Defer with One-vs-All Classifiers
Open accessAbstract
The learning to defer (L2D) framework has the potential to make AI systems safer. For a given input, the system can defer the decision to a human if the human is more likely than the model to take the correct action. We study the calibration of L2D systems, investigating if the probabilities they output are sound. We find that Mozannar & Sontag's (2020) multiclass framework is not calibrated with respect to expert correctness. Moreover, it is not even guaranteed to produce valid probabilities due to its parameterization being degenerate for this purpose. We propose an L2D system based on one-vs-all classifiers that is able to produce calibrated probabilities of expert correctness. Furthermore, our loss function is also a consistent surrogate for multiclass L2D, like Mozannar & Sontag's (2020). Our experiments verify that not only is our system calibrated, but this benefit comes at no cost to accuracy. Our model's accuracy is always comparable (and often superior) to Mozannar & Sontag's (2020) model's in tasks ranging from hate speech detection to galaxy classification to diagnosis of skin lesions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The state-of-the-art approach to managing assignments in human-AI collaboration is learning to defer.
“The state-of-the-art approach to managing assignments in HAIC is learning to defer (L2D)4–13, in which algorithms choose whether to assign a task to a human expert or an ML model, leveraging their complementary strengths.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [11]
The Introduction (p. 1) describes L2D as a framework that assigns decisions to a human when the human is more likely than the model to be correct, enabling an efficient division of labor. But the paper does not establish L2D as the singular state-of-the-art approach; Experiments (p. 6) also refers to other state-of-the-art methods, including differentiable triage. (LLM, conf=0.82)
Experimental — accuracy not yet measuredLearning to Defer algorithms improve human-AI collaboration by deferring decisions to human experts when they are likely to be more accurate than the AI model.
“Learning to Defer (L2D) algorithms improve human-AI collaboration by deferring decisions to human experts when they are likely to be more accurate than the AI model.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [11]
The paper chose to create a total of 50 experts because this far surpasses the numbers previously considered in learning-to-defer research.
“We chose to create a total of 50 experts as this far surpasses the numbers previously considered in L2D research4–13.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [11]
Various authors use CIFAR-10, simulating experts with high accuracy on five classes but random accuracy on the remaining five.
“Various authors4,8,11 use CIFAR-1016, simulating experts with high accuracy on five classes but random accuracy on the remaining five.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [11]
For α=0, the probability of error is the same for all instances within a class, which is equivalent to class-dependent label noise.
“For α=0, the probability of error is the same for all instances within a class, which is equivalent to class-dependent label noise, as used in previous L2D testing4,8,11.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [11]
A common approach in learning-to-defer research involves using label noise to produce arbitrarily accurate predictions.
“A common approach involves using label noise to produce arbitrarily accurate predictions.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [11]
The Background (p. 2) describes learning to defer as training a classifier and a rejector, while the Introduction (p. 1) describes this paper's one-vs-all approach. Searches for “label noise” and “arbitrarily accurate predictions” found no passage saying that label noise is a common approach or that it produces arbitrarily accurate predictions. (LLM, conf=0.73)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 44
- Allwein, E. L., Schapire, R. E., and Singer, Y. Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research , 1: 113-141, 2001.
- Bamford, S. P., Nichol, R. C., Baldry, I. K., Land, K., Lintott, C. J., Schawinski, K., Slosar, A., Szalay, A. S., Thomas, D., Torki, M., Andreescu, D., Edmondson, E. M., Miller, C. J., Murray, P., Raddick, M. J., and Vandenberg, J. Galaxy Zoo: the dependence of morphology and colour on environment*. Monthly Notices of the Royal Astronomical Society , 393(4):1324-1352, 02 2009. ISSN 00358711. doi: 10.1111/j.1365-2966.2008.14252.x. · DOI
- Bansal, G., Nushi, B., Kamar, E., Horvitz, E., and Weld, D. S. Is the most accurate ai the best teammate? optimizing ai for teamwork. In AAAI Conference on Artificial Intelligence , 2021.
- Bartlett, P. L. and Wegkamp, M. H. Classification with a reject option using a hinge loss. Journal of Machine Learning Research , 9:1823-1840, jun 2008. ISSN 15324435.
- Bartlett, P. L., Jordan, M. I., and McAuliffe, J. D. Convexity, classification, and risk bounds. Journal of the American Statistical Association , 101(473):138-156, 2006.
- Chalkidis, I., Androutsopoulos, I., and Aletras, N. Neural legal judgment prediction in English. In Meeting of the Association for Computational Linguistics , 2019.
- Charoenphakdee, N., Cui, Z., Zhang, Y., and Sugiyama, M. Classification with rejection based on cost-sensitive classification. In International Conference on Machine Learning , 2021.
- Chow, C. K. An optimum character recognition system using decision functions. IRE Transactions on Electronic Computers , 6:247-254, 1957.
- Cortes, C., DeSalvo, G., and Mohri, M. Learning with rejection. In International Conference on Algorithmic Learning Theory , 2016a.
- Cortes, C., DeSalvo, G., and Mohri, M. Boosting with abstention. In Advances in Neural Information Processing Systems , 2016b.
- Davidson, T., Warmsley, D., Macy, M. W., and Weber, I. Automated hate speech detection and the problem of offensive language. In International AAAI Conference on Web and Social Media , 2017.
- Dietterich, T. G. and Bakiri, G. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research , 2(1):263-286, jan 1995. ISSN 1076-9757.
- Grandvalet, Y., Rakotomamonjy, A., Keshet, J., and Canu, S. Support vector machines with a reject option. In Advances in Neural Information Processing Systems , 2009.
- Grigorescu, S., Trasnea, B., Cocias, T., and Macesanu, G. A survey of deep learning techniques for autonomous driving. Journal of Field Robotics , 37(3):362-386, 2020. doi: https://doi.org/10.1002/rob.21918. · DOI · Paper page
- Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In International Conference on Machine Learning , 2017.
- Gupta, C. and Ramdas, A. Top-label calibration and multiclass-to-binary reductions. In International Conference on Learning Representations , 2022.
- He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition , 2016.
- Hendrycks, D. and Dietterich, T. G. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019.
- Jiang, H., Kim, B., Guan, M. Y., and Gupta, M. To trust or not to trust a classifier. In Advances in Neural Information Processing Systems , 2018.
- Joulin, A., Grave, E., Bojanowski, P., Douze, M., J´ egou, H., and Mikolov, T. Fasttext.zip: Compressing text classification models. arXiv preprint arXiv:1612.03651 , 2016. · arXiv
- Kim, Y. Convolutional neural networks for sentence classification. In Conference on Empirical Methods in Natural Language Processing , 2014.
- Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations , 2015.
- Kull, M., Perello-Nieto, M., K¨ angsepp, M., Filho, T. S., Song, H., and Flach, P. Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with dirichlet calibration. In Advances in Neural Information Processing Systems , 2019.
- Langford, J., Tti-Chicago, Net, J., and Beygelzimer, A. Sensitive error correcting output codes. In Conference on Learning Theory , 2005.
- Madras, D., Pitassi, T., and Zemel, R. Predict responsibly: Improving fairness and accuracy by learning to defer. In Advances in Neural Information Processing Systems , 2018.
- Mozannar, H. and Sontag, D. A. Consistent estimators for learning to defer to an expert. In International Conference on Machine Learning , 2020.
- Nguyen, A., Yosinski, J., and Clune, J. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Conference on Computer Vision and Pattern Recognition , 2015.
- Ni, C., Charoenphakdee, N., Honda, J., and Sugiyama, M. On the calibration of multiclass classification with rejection. In Advances in Neural Information Processing Systems , 2019.
- Okati, N., De, A., and Gomez-Rodriguez, M. Differentiable learning under triage. In Advances in Neural Information Processing Systems , 2021.
- Raghu, M., Blumer, K., Corrado, G., Kleinberg, J., Obermeyer, Z., and Mullainathan, S. The algorithmic automation problem: Prediction, triage, and human effort. arXiv preprint arXiv:1903.12220 , 2019. · arXiv
- Ramaswamy, H. G., Srinivasan Babu, B., Agarwal, S., and Williamson, R. C. On the consistency of output code based learning algorithms for multiclass learning problems. In Conference on Learning Theory , 2014.
- Ramaswamy, H. G., Tewari, A., and Agarwal, S. Consistent algorithms for multiclass classification with an abstain option. Electronic Journal of Statistics , 12:530-554, 2018.
- Reid, M. D. and Williamson, R. C. Composite binary losses. Journal of Machine Learning Research , 11(83): 2387-2422, 2010.
- Rifkin, R. and Klautau, A. In defense of one-vs-all classification. Journal of Machine Learning Research , 5: 101-141, 2004.
- Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Keysers, D., Uszkoreit, J., Lucic, M., and Dosovitskiy, A. Mlp-mixer: An all-mlp architecture for vision. In Advances in Neural Information Processing Systems , 2021.
- Tschandl, P., Rosendahl, C., and Kittler, H. The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data , 5(1):1-9, 2018.
- C., Soyer, H. P., Zalaudek, I., and Kittler, H. Human-computer collaboration for skin cancer recognition. Nature Medicine , pp. 1-6, 2020.
- Vaicenavicius, J., Widmann, D., Andersson, C., Lindsten, F., Roll, J., and Sch¨ on, T. Evaluating model calibration in classification. In Conference on Artificial Intelligence and Statistics , 2019.
- Wilder, B., Horvitz, E., and Kamar, E. Learning to complement humans. In International Joint Conference on Artificial Intelligence , 2020.
- Yuan, M. and Wegkamp, M. Classification methods with reject option based on convex risk minimization. Journal of Machine Learning Research , 11(5):111-130, 2010.
- Zagoruyko, S. and Komodakis, N. Wide residual networks. British Machine Vision Conference , 2016.
- Zhao, S., Kim, M., Sahoo, R., Ma, T., and Ermon, S. Calibrating predictions to decisions: A novel approach to multi-class calibration. Advances in Neural Information Processing Systems , 2021.
- Zhong, H., Guo, Z., Tu, C., Xiao, C., Liu, Z., and Sun, M. Legal judgment prediction via topological learning. In Conference on Empirical Methods in Natural Language Processing , 2018.
- Zoabi, Y., Deri-Rozov, S., and Shomron, N. Machine learning-based prediction of covid-19 diagnosis based on symptoms. npj Digital Medicine , 4(1):1-5, 2021.
Cited by
Sorted by citations
A benchmarking framework and dataset for learning to defer in human-AI decision-making
Alves, J. V., Leitão, D., Jesus, S., Sampaio, M. O. P., Liébana, J., Saleiro, P., Figueiredo, M. A. T., & Bizarro, P. (2025) · Scientific Data · doi:10.1038/s41597-025-04664-y · Cited by 10 · see citing works
“Various authors 4,8,11 use CIFAR-10 16 , simulating experts with high accuracy on five classes but random accuracy on the remaining five.”
- Authors
- Verma, R., & Nalisnick, E.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2202.03673
- OpenAlex
- W4221142770
- Cited by
- 5
- References
- 0
- Citations by year
- 2025: 2 · 2024: 2 · 2023: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- learning to defer, probability calibration, human-in-the-loop decision making, multi-class classification, surrogate loss functions, AI safety, hate speech detection, galaxy classification, skin lesion diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
