A simple method to adjust clinical prediction models to local circumstances
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
INTRODUCTION: Clinical prediction models estimate the risk of having or developing a particular outcome or disease. Researchers often develop a new model when a previously developed model is validated and the performance is poor. However, the model can be adjusted (updated) using the new data. The updated model is then based on both the development and validation data. We show how a simple updating method may suffice to update a clinical prediction model. METHODS: A prediction model that preoperatively predicts the risk of severe postoperative pain was developed with multivariable logistic regression from the data of 1944 surgical patients in the Academic Medical Center Amsterdam, the Netherlands. We studied the predictive performance of the model in 1,035 new patients, scheduled for surgery at a later time in the University Medical Center Utrecht, the Netherlands. We assessed the calibration (agreement between predicted risks and the observed frequencies of an outcome) and discrimination (ability of the model to distinguish between patients with and without postoperative pain). When the incidence of the outcome is different, all predicted risks may be systematically over- or underestimated. Hence, the intercept of the model can be adjusted (updating). RESULTS: The predicted risks were systematically higher than the observed frequencies, corresponding to a difference in the incidence of postoperative pain between the development (62%) and validation set (36%). The updated model resulted in better calibration. DISCUSSION: When a clinical prediction model in new patients does not show adequate performance, an alternative to developing a new model is to update the prediction model with new data. The updated model will be based on more patient data, and may yield better risk estimates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773
Evaluating the Yield of Medical Tests
Harrell, F. E. (1982) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1982.03320430047030 · Cited by 3,490
Apfel, C. C., Läärä, E., Koivuranta, M., Greim, C.-A., & Roewer, N. (1999) · Anesthesiology · doi:10.1097/00000542-199909000-00022 · Cited by 2,121
A Proposal for a New Method of Evaluation of the Newborn Infant.
Apgar, V. (1953) · Anesthesia & Analgesia · doi:10.1213/00000539-195301000-00041 · Cited by 2,015
Clinical Prediction Rules: Applications and Methodological Standards
Wasson, J. H., Sox, H. C., Neff, R. K., & Goldman, L. (1985) · New England Journal of Medicine · doi:10.1056/nejm198509263131306 · Cited by 1,357
A general cardiovascular risk profile: The Framingham study
Kannel, W. B., McGee, D., & Gordon, T. (1976) · The American Journal of Cardiology · doi:10.1016/0002-9149(76)90061-8 · Cited by 1,189
Assessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Reilly, B. M., & Evans, A. T. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-3-200602070-00009 · Cited by 794
Laupacis, A. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03540300056034 · Cited by 793
External validation is necessary in prediction research:
Bleeker, S. E., Moll, H. A., Steyerberg, E. W., Donders, A. R. T., Derksen-Lubsen, G., Grobbee, D. E., & Moons, K. G. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00207-5 · Cited by 710
Self-citation
Cited by
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationRisk prediction models: II. External validation, model updating, and impact assessment
Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012) · Heart · doi:10.1136/heartjnl-2011-301247 · Cited by 1,195
Self-citationDebray, T. P. A., Vergouwe, Y., Koffijberg, H., Nieboer, D., Steyerberg, E. W., & Moons, K. G. M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.018 · Cited by 597
Self-citationAssessing calibration of prognostic risk scores
Crowson, C. S., Atkinson, E. J., & Therneau, T. M. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280213497434 · Cited by 289
Cook, N. R., Paynter, N. P., Eaton, C. B., Manson, J. E., Martin, L. W., Robinson, J. G., Rossouw, J. E., Wassertheil-Smoller, S., & Ridker, P. M. (2012) · Circulation · doi:10.1161/circulationaha.111.075929 · Cited by 263
Metadata
- Authors
- Janssen, K. J. M., Vergouwe, Y., Kalkman, C. J., Grobbee, D. E., & Moons, K. G. M.
- Venue
- Canadian Journal of Anesthesia/Journal canadien d'anesthésie
- Published
- 2009-02-07
- Volume
- 56
- Issue
- 3
- Pages
- 194-201
- ISSN
- 0832-610X (ISSN-L)
- 1496-8975
- OpenAlex
- W1551135786
- Cited by
- 126
- References
- 14
- FWCI
- 3.01
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A simple method to adjust clinical prediction models to local circumstances. https://app.cochat.ai/papers/10.1007/s12630-009-9041-x
- The paper
- Janssen, K. J. M., Vergouwe, Y., Kalkman, C. J., Grobbee, D. E., & Moons, K. G. M. (2009). A simple method to adjust clinical prediction models to local circumstances. Canadian Journal of Anesthesia/Journal canadien d'anesthésie. https://doi.org/10.1007/s12630-009-9041-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.