The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Predicting good probabilities with supervised learning
Abstract
We examine the relationship between the predictions made by different learning algorithms and true posterior probabilities. We show that maximum margin methods such as boosted trees and boosted stumps push probability mass away from 0 and 1 yielding a characteristic sigmoid shaped distortion in the predicted probabilities. Models such as Naive Bayes, which make unrealistic independence assumptions, push probabilities toward 0 and 1. Other models such as neural nets and bagged trees do not have these biases and predict well calibrated probabilities. We experiment with two ways of correcting the biased probabilities predicted by some learning methods: Platt Scaling and Isotonic Regression. We qualitatively examine what kinds of distortions these calibration methods are suitable for and quantitatively examine how much data they need to be effective. The empirical results show that after calibration boosted trees, random forests, and SVMs predict the best probabilities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There are certain machine learning models that do not meet this criterion such as SVM, though this issue can be potentially addressed through Platt Scaling or Isotonic Regression [24].”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [24]
Citations
Cites
UCI Repository of machine learning databases
Blake, C. L. (1998) · Medical Entomology and Zoology · Cited by 10,582
Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods
Platt, J. C. (1999) · Cited by 6,385
Making Large-Scale SVM Learning Practical
Joachims, T. (1998) · RePEc: Research Papers in Economics · doi:10.17877/de290r-14262 · Cited by 4,411
Order Restricted Statistical Inference.
Santner, T., Robertson, T., Wright, F. T., & Dykstra, R. L. (1990) · Journal of the American Statistical Association · doi:10.2307/2289813 · Cited by 1,586
Transforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280
An Empirical Distribution Function for Sampling with Incomplete Information
Ayer, M., Brunk, H. D., Ewing, G. M., Reid, W. T., & Silverman, E. (1955) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728423 · Cited by 904
Provost, F. J., & Fawcett, T. E. (1997) · Cited by 720
(2001) · Cited by 683
The Comparison and Evaluation of Forecasters
DeGroot, M. H., & Fienberg, S. E. (1983) · The Statistician · doi:10.2307/2987588 · Cited by 604 · cited 1×
“On real problems where the true conditional probabilities are not known, model calibration can be visualized with reliability diagrams (DeGroot & Fienberg, 1982).”
BackgroundTree Induction for Probability-Based Ranking
Provost, F., & Domingos, P. (2003) · Machine Learning · doi:10.1023/a:1024099825458 · Cited by 541
Cited by
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
An empirical comparison of supervised learning algorithms
Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143865 · Cited by 2,780
Self-citationDecision trees: a recent overview
Kotsiantis, S. B. (2013) · Artificial Intelligence Review · doi:10.1007/s10462-011-9272-4 · Cited by 1,568
Design challenges and misconceptions in named entity recognition
Ratinov, L., & Roth, D. (2009) · Proceedings of the Thirteenth Conference on Computational Natural Language Learning - CoNLL '09 · doi:10.3115/1596374.1596399 · Cited by 1,448
Artificial intelligence in cardiology
Johnson, K. W., Torres Soto, J., Glicksberg, B. S., Shameer, K., Miotto, R., Ali, M., Ashley, E., & Dudley, J. T. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.03.521 · Cited by 1,202
Metadata
- Authors
- Niculescu-Mizil, A., & Caruana, R.
- Affiliations
- Cornell University (US)
- ORCID
- Caruana, Rich
- Venue
- Proceedings of the 22nd international conference on Machine learning - ICML '05
- Publisher
- ACM Press
- Published
- 2005
- Pages
- 625-632
- OpenAlex
- W2098824882
- Cited by
- 1,884
- References
- 11
- Citations by year
- 2026: 410 · 2025: 188 · 2024: 160 · 2023: 173 · 2022: 152 · 2021: 188 · 2020: 149 · 2019: 104 · 2018: 80 · 2017: 49 · 2016: 46 · 2015: 40 · 2014: 22 · 2013: 14 · 2012: 21
- FWCI
- 10.70
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- probability calibration, predicted probabilities, posterior probability, boosted trees, naive Bayes, artificial neural networks, bagged trees, Platt scaling, isotonic regression, random forest, support vector machine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Predicting good probabilities with supervised learning. https://app.cochat.ai/papers/10.1145/1102351.1102430
- The paper
- Niculescu-Mizil, A., & Caruana, R. (2005). Predicting good probabilities with supervised learning. Proceedings of the 22nd international conference on Machine learning - ICML '05. https://doi.org/10.1145/1102351.1102430
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.