The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Calibrating predictive model estimates to support personalized medicine
Open accessAbstract
OBJECTIVE: Predictive models that generate individualized estimates for medically relevant outcomes are playing increasing roles in clinical care and translational research. However, current methods for calibrating these estimates lose valuable information. Our goal is to develop a new calibration method to conserve as much information as possible, and would compare favorably to existing methods in terms of important performance measures: discrimination and calibration. MATERIAL AND METHODS: We propose an adaptive technique that utilizes individualized confidence intervals (CIs) to calibrate predictions. We evaluate this new method, adaptive calibration of predictions (ACP), in artificial and real-world medical classification problems, in terms of areas under the ROC curves, the Hosmer-Lemeshow goodness-of-fit test, mean squared error, and computational complexity. RESULTS: ACP compared favorably to other calibration methods such as binning, Platt scaling, and isotonic regression. In several experiments, binning, isotonic regression, and Platt scaling failed to improve the calibration of a logistic regression model, whereas ACP consistently improved the calibration while maintaining the same discrimination or even improving it in some experiments. In addition, the ACP algorithm is not computationally expensive. LIMITATIONS: The calculation of CIs for individual predictions may be cumbersome for certain predictive models. ACP is not completely parameter-free: the length of the CI employed may affect its results. CONCLUSIONS: ACP can generate estimates that may be more suitable for individualized predictions than estimates that are calibrated using existing methods. Further studies are necessary to explore the limitations of ACP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302
Efron, B., & Tibshirani, R. (1986) · Statistical Science · doi:10.1214/ss/1177013815 · Cited by 6,033
Zimmerman, J. E., Kramer, A. A., McNair, D. S., & Malila, F. M. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000215112.84523.f0 · Cited by 1,764 · cited 5×
“Like subgroups in the HL test, we have two choices for constructing these bins: (1) equal number of elements in the bins, sorted by probability estimates, that is, Pn=10R elements per cell; or (2) fixed, equal length intervals of probability estimates, that is, 0<p<1⁄40.”
BackgroundMethodInfluentialTransforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280
Kramer, A. A., & Zimmerman, J. E. (2007) · Critical Care Medicine · doi:10.1097/01.ccm.0000275267.64078.b0 · Cited by 886
Research works printed but not linked: 24
- 10.1093/jnci/djr172 · DOI
- 10.1093/aje/kwh258 · DOI
- 10.1136/amiajnl-2011-000101 · DOI
- 10.1146/annurev.bioeng.8.061505.095842 · DOI
- Stone NJ Bilek S Rosenbaum S . Recent national cholesterol education program adult treatment panel III update: adjustments and options. Am J Cardiol 2005;96:53–9. · DOI · Paper page
- 10.1001/jama.285.19.2486 · DOI · Paper page
- 10.1097/SMJ.0b013e3181d39038 · DOI
- 10.1016/S0140-6736(86)91511-4 · DOI
- 10.1038/nrcardio.2010.53 · DOI
- Altman RB Miller KS . 2010 Translational bioinformatics year in review. J Am Med Inform Assoc 2010;18:358–66. · DOI
- 10.1136/amiajnl-2011-000245 · DOI
- 10.1148/radiology.143.1.7063747 · DOI · Paper page
- 10.1002/cncr.25081 · DOI
- 10.1023/A:1007413511361 · DOI · Paper page
- Zadrozny B Elkan C . Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. The Eighteenth International Conference on Machine Learning. 2001:609–16.
- Platt JC . Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers 1999:61–74. · DOI · Paper page
- Burdakov O Grimvall A Hussian M . A generalised pav algorithm for monotonic regression in several variables. International Conference on Computational Statistics. 2004:761–67.
- 10.1198/106186008X285627 · DOI
- 10.2307/2685209 · DOI
- 10.2307/2684406 · DOI
- 10.1002/(SICI)1097-0258(19970515)16:9<965::AID-SIM509>3.0.CO;2-O · doi:10.1002/(sici)1097-0258(19970515)16:9<965::aid-sim509>3.0.co;2-o
- 10.1016/j.jbi.2005.02.008 · DOI · Paper page
- 10.1002/jhm.895 · DOI
- 10.1093/oxfordjournals.eurheartj.a015035 · DOI
Cited by
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
“Alternatively, in automated health care, control should be passed on to human doctors when the confidence of a disease diagnosis network is low (Jiang et al., 2012).”
BackgroundA tutorial on calibration measurements and calibration models for clinical prediction models
Huang, Y., Li, W., Macheret, F., Gabriel, R. A., & Ohno-Machado, L. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz228 · Cited by 486
Self-citationDeep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
Papernot, N., & McDaniel, P. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04765 · Cited by 335
Calibration drift in regression and machine learning models for acute kidney injury
Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx030 · Cited by 312
Improving Calibration for Long-Tailed Recognition
Zhong, Z., Cui, J., Liu, S., & Jia, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01622 · Cited by 282
Metadata
- Authors
- Jiang, X., Osl, M., Kim, J., & Ohno-Machado, L.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2012-03
- Volume
- 19
- Issue
- 2
- Pages
- 263-274
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2137556846
- Cited by
- 204
- References
- 5
- FWCI
- 3.62
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Calibrating predictive model estimates to support personalized medicine. https://app.cochat.ai/papers/10.1136/amiajnl-2011-000291
- The paper
- Jiang, X., Osl, M., Kim, J., & Ohno-Machado, L. (2012). Calibrating predictive model estimates to support personalized medicine. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2011-000291
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.