ON THE VALIDATION OF MODELS
Abstract
Traditional methods of evaluating geographic models by statistical comparisons between observed and simulated variates are criticized. In particular, it is suggested that the correlation coefficient (r), its square and tests of their statistical significance are inadequate for such purposes. The root mean squared error (RMSE) and related measures as well as a new index of agreement (d) are alternatively presented as superior indices for making such comparisons. Arguments are made for increasing the number of digital algorithms and data plots being published.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MSS is a measure proposed by Willmott [39] that factors in both mean and variance differences and furnishes a comprehensive assessment of the agreement between the two datasets.
“MSS, a measure proposed by Willmott [39] that factors in both mean and variance differences, furnishes a comprehensive assessment of the agreement between the two datasets.”
From Baseline Climatology of the Canary Current Upwelling System and Evolution of Sea Surface Temperature (2024) · cites this paper as Willmott [39]
Cited by
Sorted by citations
Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations
Moriasi, D. N., Arnold, J. G., Liew, M. W. V., Bingner, R. L., Harmel, R. D., & Veith, T. L. (2007) · Transactions of the ASABE · doi:10.13031/2013.23153 · Cited by 13,446 · see citing works
Characterising performance of environmental models
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J., Marsili-Libelli, S., Newham, L. T. H., Norton, J. P., Perrin, C., et al. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.09.011 · Cited by 1,536 · see citing works
Rennie, J. J., Lawrimore, J. H., Gleason, B. E., Thorne, P. W., Morice, C. P., Menne, M. J., Williams, C. N., de Almeida, W. G., Christy, J. R., Flannery, M., et al. (2014) · Geoscience Data Journal · doi:10.1002/gdj3.8 · Cited by 123 · see citing works
Parameterization and evaluation of predictions of DSSAT/CANEGRO for Brazilian sugarcane
Marin, F. R., Jones, J. W., Royce, F., Suguitani, C., Donzeli, J. L., Filho, W. J. P., & Nassif, D. S. P. (2011) · Agronomy Journal · doi:10.2134/agronj2010.0302 · Cited by 118 · see citing works
Herbst, M., Hellebrand, H. J., Bauer, J., Huisman, J. A., Šimůnek, J., Weihermüller, L., Graf, A., Vanderborght, J., & Vereecken, H. (2008) · Ecological Modelling · doi:10.1016/j.ecolmodel.2008.02.007 · Cited by 80 · see citing works
A package of parameter estimation methods and implementation for the STICS crop-soil model
Wallach, D., Buis, S., Lecharpentier, P., Bourges, J., Clastre, P., Launay, M., Bergez, J.-E., Guerif, M., Soudais, J., & Justes, E. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.09.004 · Cited by 72 · see citing works
Hens, N., Habteab Ghebretinsae, A., Hardt, K., Van Damme, P., & Van Herck, K. (2014) · Vaccine · doi:10.1016/j.vaccine.2013.10.088 · Cited by 65 · see citing works
Khan, N., Shahid, S., Ismail, T. B., & Behlil, F. (2021) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-020-01963-1 · Cited by 59 · see citing works
(2023) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2023.08.016 · Cited by 60 · see citing works
Evaluation of CORDEX- South Asia regional climate models for heat wave simulations over India
Singh, S., Mall, R. K., Dadich, J., Verma, S., Singh, J. V., & Gupta, A. (2021) · Atmospheric Research · doi:10.1016/j.atmosres.2020.105228 · Cited by 35 · see citing works
Zhou, X. V., Clark, C. D., Nair, S. S., Hawkins, S. A., & Lambert, D. M. (2015) · Agricultural Water Management · doi:10.1016/j.agwat.2015.06.018 · Cited by 25 · see citing works
Baseline Climatology of the Canary Current Upwelling System and Evolution of Sea Surface Temperature
Mills, L., Janeiro, J., & Martins, F. (2024) · Remote Sensing · doi:10.3390/rs16030504 · Cited by 5 · see citing works
- Authors
- Willmott, C. J.
- Affiliations
- University of Delaware (US)
- Corresponding author
- Willmott, Cort J.
- Venue
- Physical Geography
- Publisher
- Informa UK Limited
- Published
- 1981-07
- Volume
- 2
- Issue
- 2
- Pages
- 184-194
- ISSN
- 0272-3646 (ISSN-L)
- 1930-0557
- OpenAlex
- W2111286455
- Cited by
- 4,916
- References
- 0
- Citations by year
- 2026: 352 · 2025: 305 · 2024: 340 · 2023: 323 · 2022: 385 · 2021: 344 · 2020: 312 · 2019: 316 · 2018: 272 · 2017: 215 · 2016: 237 · 2015: 180 · 2014: 191 · 2013: 159 · 2012: 151
- FWCI
- 13.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Physical Geography (publishedVersion)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Spatial and Panel Data Analysis
- Physical Sciences › Environmental Science › Environmental Engineering › Soil Geostatistics and Mapping
- Social Sciences › Geography, Planning and Development › Geography Education and Pedagogy
- Keywords
- model validation, geographic models, model evaluation, correlation coefficient, root mean square error, index of agreement, statistical significance tests, digital algorithms, data plots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
