Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index.

External validation is necessary in prediction research:

Bleeker, S.E, Moll, H.A, Steyerberg, E.W, Donders, A.R.T, Derksen-Lubsen, G, Grobbee, D.E, Moons, K.G.M · 2003 · Journal of Clinical Epidemiology

Abstract

Background and objectives Prediction models tend to perform better on data on which the model was constructed than on new data. This difference in performance is an indication of the optimism in the apparent performance in the derivation set. For internal model validation, bootstrapping methods are recommended to provide bias-corrected estimates of model performance. Results are often accepted without sufficient regard to the importance of external validation. This report illustrates the limitations of internal validation to determine generalizability of a diagnostic prediction model to future settings. Methods A prediction model for the presence of serious bacterial infections in children with fever without source was derived and validated internally using bootstrap resampling techniques. Subsequently, the model was validated externally. Results In the derivation set (n=376), nine predictors were identified. The apparent area under the receiver operating characteristic curve (95% confidence interval) of the model was 0.83 (0.78-0.87) and 0.76 (0.67-0.85) after bootstrap correction. In the validation set (n=179) the performance was 0.57 (0.47-0.67). Conclusion For relatively small data sets, internal validation of prediction models by bootstrap techniques may not be sufficient and indicative for the model's performance in future patients. External validation is essential before implementing prediction models in clinical practice.

Overview

Integrity signals

Registry

Crossref · 10.1016/s0895-4356(03)00207-5

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 45 references

Research works printed but not linked: 1

  1. Steensel-Moll, 1996, A problem-oriented patient classification system for general pediatrics II, Tijdschr Kindergeneesk, 64, 99

Cited by

See all 729

Metadata

Authors
Bleeker, S. E., Moll, H. A., Steyerberg, E. W., Donders, A. R. T., Derksen-Lubsen, G., Grobbee, D. E., & Moons, K. G. M.
Affiliations
  • Erasmus MC (NL)
  • Utrecht University (NL)
  • University Medical Center Utrecht (NL)
Corresponding author
Bleeker, S.E
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2003-09
Volume
56
Issue
9
Pages
826-832
ISSN
0895-4356 (ISSN-L)
OpenAlex
W154377515
Cited by
710
References
45
Citations by year
2026: 32 · 2025: 40 · 2024: 50 · 2023: 46 · 2022: 51 · 2021: 45 · 2020: 49 · 2019: 33 · 2018: 32 · 2017: 31 · 2016: 31 · 2015: 31 · 2014: 31 · 2013: 36 · 2012: 25
FWCI
10.66
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • Journal of Clinical Epidemiology (publishedVersion)
  • PubMed (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
external validation, prediction model, internal validation, bootstrapping, optimism bias, generalizability, diagnostic prediction model, serious bacterial infection, fever without source, children, model performance
MeSH terms
Bacterial Infections, Child, Preschool, Fever of Unknown Origin, Humans, Infant, Predictive Value of Tests, Confidence Intervals, Likelihood Functions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: External validation is necessary in prediction research:. https://app.cochat.ai/papers/10.1016/s0895-4356(03)00207-5
The paper
Bleeker, S. E., Moll, H. A., Steyerberg, E. W., Donders, A. R. T., Derksen-Lubsen, G., Grobbee, D. E., & Moons, K. G. M. (2003). External validation is necessary in prediction research:. Journal of Clinical Epidemiology. https://doi.org/10.1016/s0895-4356(03)00207-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.