Evaluation (not validation) of quantitative models.
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
The present regulatory climate has led to increasing demands for scientists to attest to the predictive reliability of numerical simulation models used to help set public policy, a process frequently referred to as model validation. But while model validation may reveal useful information, this paper argues that it is not possible to demonstrate the predictive reliability of any model of a complex natural system in advance of its actual use. All models embed uncertainties, and these uncertainties can and frequently do undermine predictive reliability. In the case of lead in the environment, we may categorize model uncertainties as theoretical, empirical, parametrical, and temporal. Theoretical uncertainties are aspects of the system that are not fully understood, such as the biokinetic pathways of lead metabolism. Empirical uncertainties are aspects of the system that are difficult (or impossible) to measure, such as actual lead ingestion by an individual child. Parametrical uncertainties arise when complexities in the system are simplified to provide manageable model input, such as representing longitudinal lead exposure by cross-sectional measurements. Temporal uncertainties arise from the assumption that systems are stable in time. A model may also be conceptually flawed. The Ptolemaic system of astronomy is a historical example of a model that was empirically adequate but based on a wrong conceptualization. Yet had it been computerized--and had the word then existed--its users would have had every right to call it validated. Thus, rather than talking about strategies for validation, we should be talking about means of evaluation. That is not to say that language alone will solve our problems or that the problems of model evaluation are primarily linguistic. The uncertainties inherent in large, complex models will not go away simply because we change the way we talk about them. But this is precisely the point: calling a model validated does not make it valid. Modelers and policymakers must continue to work toward finding effective ways to evaluate and judge the quality of their models, and to develop appropriate terminology to communicate these judgments to the public whose health and safety may be at stake.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Limits to Growth: A report for the Club of Rome's Project on the Predicament of Mankind
Meadows, D. H., Meadows, D. L., Randers, J., & Behrens, W. W. (1972) · doi:10.1349/ddlp.1 · Cited by 4,268
Cited by
SoilGrids 2.0: producing soil information for the globe with quantified spatial uncertainty
Poggio, L., de Sousa, L. M., Batjes, N. H., Heuvelink, G. B. M., Kempen, B., Ribeiro, E., & Rossiter, D. (2021) · SOIL · doi:10.5194/soil-7-217-2021 · Cited by 2,333
Validation of species–climate impact models under climate change
Araújo, M. B., Pearson, R. G., Thuiller, W., & Erhard, M. (2005) · Global Change Biology · doi:10.1111/j.1365-2486.2005.01000.x · Cited by 1,669
The importance of biotic interactions for modelling species distributions under climate change
Araújo, M. B., & Luoto, M. (2007) · Global Ecology and Biogeography · doi:10.1111/j.1466-8238.2007.00359.x · Cited by 1,213
Using size-based indicators to evaluate the ecosystem effects of fishing
Shin, Y.-J., Rochet, M.-J., Jennings, S., Field, J. G., & Gislason, H. (2005) · ICES Journal of Marine Science · doi:10.1016/j.icesjms.2005.01.004 · Cited by 535
Simple spatially-distributed models for predicting flood inundation: A review
Hunter, N. M., Bates, P. D., Horritt, M. S., & Wilson, M. D. (2007) · Geomorphology · doi:10.1016/j.geomorph.2006.10.021 · Cited by 440
Metadata
- Authors
- Oreskes, N.
- Venue
- Environmental Health Perspectives
- Publisher
- American Chemical Society (ACS)
- Published
- 1998-12
- Volume
- 106
- Issue
- suppl 6
- Pages
- 1453-1460
- ISSN
- 0091-6765 (ISSN-L)
- 1552-9924
- OpenAlex
- W2082057801
- Cited by
- 200
- References
- 1
- FWCI
- 3.66
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.