The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
A Framework for Validation of Computer Models
Abstract
We present a framework that enables computer model evaluation oriented toward answering the question: Does the computer model adequately represent reality? The proposed validation framework is a six-step procedure based on Bayesian and likelihood methodology. The Bayesian methodology is particularly well suited to treating the major issues associated with the validation process: quantifying multiple sources of error and uncertainty in computer models, combining multiple sources of information, and updating validation assessments as new information is acquired. Moreover, it allows inferential statements to be made about predictive error associated with model predictions in untested situations. The framework is implemented in a test bed example of resistance spot welding, to provide context for each of the six steps in the proposed validation process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
McKay, M. D., Beckman, R. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271432 · Cited by 7,415
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Self-citationMonte Carlo Statistical Methods
Kim, H. (2000) · Technometrics · doi:10.1080/00401706.2000.10485722 · Cited by 5,508
Bayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222
Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences
Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994) · Science · doi:10.1126/science.263.5147.641 · Cited by 3,010
Verification and Validation in Computational Science and Engineering
Roache, P. J. (1998) · Cited by 2,157
Uncertainty in predictions of the climate response to rising levels of greenhouse gases
Stainforth, D. A., Aina, T., Christensen, C., Collins, M., Faull, N., Frame, D. J., Kettleborough, J. A., Knight, S., Martin, A., Murphy, J. M., et al. (2005) · Nature · doi:10.1038/nature03301 · Cited by 1,166
Probabilistic Sensitivity Analysis of Complex Models: A Bayesian Approach
Oakley, J. E., & O'Hagan, A. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.05304.x · Cited by 1,085
Marginal Likelihood From the Metropolis–Hastings Output
Chib, S., & Jeliazkov, I. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501750332848 · Cited by 1,062
Computer Model Calibration Using High-Dimensional Output
Higdon, D., Gattiker, J., Williams, B., & Rightley, M. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000000888 · Cited by 867
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 854
Self-citationMatrix Algebra Useful for Statistics
Schmee, J. (1985) · Technometrics · doi:10.1080/00401706.1985.10488022 · Cited by 775
Currin, C., Mitchell, T., Morris, M., & Ylvisaker, D. (1991) · Journal of the American Statistical Association · doi:10.1080/01621459.1991.10475138 · Cited by 753
Screening, Predicting, and Computer Experiments
Welch, W. J., Buck, R. J., Sacks, J., Wynn, H. P., Mitchell, T. J., & Morris, M. D. (1992) · Technometrics · doi:10.2307/1269548 · Cited by 630
Self-citationCombining Field Data and Computer Simulations for Calibration and Prediction
Higdon, D., Kennedy, M., Cavendish, J. C., Cafeo, J. A., & Ryne, R. D. (2004) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827503426693 · Cited by 623
Self-citationObjective Bayesian Analysis of Spatially Correlated Data
Berger, J. O., De Oliveira, V., & Sansó, B. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753382282 · Cited by 469
Self-citationBayesian inference for the uncertainty distribution of computer model outputs
Oakley, J. (2002) · Biometrika · doi:10.1093/biomet/89.4.769 · Cited by 323
Bayesian Inference for Non-Stationary Spatial Covariance Structure via Spatial Deformations
Schmidt, A. M., & O'Hagan, A. (2003) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00413 · Cited by 292
Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction
Morris, M., Mitchell, T., & Ylvisaker, D. (1991) · doi:10.2172/5836957 · Cited by 276
Bayesian Forecasting for Complex Systems Using Computer Simulators
Craig, P. S., Goldstein, M., Rougier, J. C., & Seheult, A. H. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753168370 · Cited by 240
Computer Model Validation with Functional Output
(2015) · Cited by 204
Experimental Design and Observation for Large Systems
Bates, R. A., Buck, R. J., Riccomagno, E., & Wynn, H. P. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02068.x · Cited by 200
Gustafson, P. (2005) · Statistical Science · doi:10.1214/088342305000000098 · Cited by 188
Craig, P. S., Goldstein, M., Seheult, A. H., & Smith, J. A. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2290-3_2 · Cited by 184
Caswell, H. (1976) · Elsevier eBooks · doi:10.1016/b978-0-12-547204-3.50050-6 · Cited by 116
Default priors for Gaussian processes
Paulo, R. (2005) · The Annals of Statistics · doi:10.1214/009053604000001264 · Cited by 111
Self-citationProbabilistic Formulations for Transferring Inferences from Mathematical Models to Physical Systems
Goldstein, M., & Rougier, J. (2004) · SIAM Journal on Scientific Computing · doi:10.1137/s106482750342670x · Cited by 108
Quantitative Methods for Current Environmental Issues
Anderson, C. W., Barnett, V., Chatwin, P. C., & El‐Shaarawi, A. H. (2002) · Springer eBooks · doi:10.1007/978-1-4471-0657-9 · Cited by 103
Estimating Percentiles of Uncertain Computer Code Outputs
Oakley, J. E. (2004) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1046/j.0035-9254.2003.05044.x · Cited by 80
Bayesian Design and Analysis of Computer Experiments: Use of Derivatives in Surface Prediction
Morris, M. D., Mitchell, T. J., & Ylvisaker, D. (1993) · Technometrics · doi:10.2307/1269517 · Cited by 55
Circuit Optimization via Sequential Computer Experiments: Design of an Output Buffer
Aslett, R., Buck, R. J., Duvall, S. G., Sacks, J., & Welch, W. J. (1998) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/1467-9876.00096 · Cited by 45
Self-citationEfficient models for correlated data via convolutions of intrinsic processes
Lee, H. K. H., Higdon, D., Calder, C. A., & Holloman, C. H. (2005) · Statistical Modelling · doi:10.1191/1471082x05st085oa · Cited by 38
Bayesian Analysis of Computer Model Outputs
Oakley, J. E., & O’Hagan, A. (2002) · Springer eBooks · doi:10.1007/978-1-4615-0813-7_10 · Cited by 36
Workshop on statistical approaches for the evaluation of complex computer models
Berk, R. A., Bickel, P. J., Campbell, K., Fovell, R. G., Keller‐McNulty, S., Kelly, E., Linn, R., Park, B., Perelson, A., Rouphail, N. M., et al. (2002) · Statistical Science · doi:10.1214/ss/1030550860 · Cited by 33
Self-citationSample Size Implications when Biases are Modelled Rather than Ignored
Gustafson, P. (2006) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2006.00436.x · Cited by 18
Editorial: (Post-normal) Statistical Science
Zidek, J. V. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2005.00393.x · Cited by 13
Cited by
Sensitivity analysis: A review of recent advances
Borgonovo, E., & Plischke, E. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.06.032 · Cited by 1,167
Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization
Peherstorfer, B., Willcox, K., & Gunzburger, M. (2018) · SIAM Review · doi:10.1137/16m1082469 · Cited by 1,061
Computer Model Calibration Using High-Dimensional Output
Higdon, D., Gattiker, J., Williams, B., & Rightley, M. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000000888 · Cited by 867
Comparing uncertainty analysis techniques for a SWAT application to the Chaohe Basin in China
Yang, J., Reichert, P., Abbaspour, K. C., Xia, J., & Yang, H. (2008) · Journal of Hydrology · doi:10.1016/j.jhydrol.2008.05.012 · Cited by 800
Bayesian analysis of computer code outputs: A tutorial
O’Hagan, A. (2006) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2005.11.025 · Cited by 755
Metadata
- Authors
- Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, J., Lin, C.-H., & Tu, J.
- Venue
- Technometrics
- Publisher
- Informa UK Limited
- Published
- 2007-05
- Volume
- 49
- Issue
- 2
- Pages
- 138-154
- ISSN
- 0040-1706 (ISSN-L)
- 1537-2723
- OpenAlex
- W2027319489
- Cited by
- 655
- References
- 36
- FWCI
- 26.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.