The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Quantification of Model Uncertainty: Calibration, Model Discrepancy, and Identifiability
Abstract
To use predictive models in engineering design of physical systems, one should first quantify the model uncertainty via model updating techniques employing both simulation and experimental data. While calibration is often used to tune unknown calibration parameters of a computer model, the addition of a discrepancy function has been used to capture model discrepancy due to underlying missing physics, numerical approximations, and other inaccuracies of the computer model that would exist even if all calibration parameters are known. One of the main challenges in model updating is the difficulty in distinguishing between the effects of calibration parameters versus model discrepancy. We illustrate this identifiability problem with several examples, explain the mechanisms behind it, and attempt to shed light on when a system may or may not be identifiable. In some instances, identifiability is achievable under mild assumptions, whereas in other instances, it is virtually impossible. In a companion paper, we demonstrate that using multiple responses, each of which depends on a common set of calibration parameters, can substantially enhance identifiability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Cressie, N. (1992) · Terra Nova · doi:10.1111/j.1365-3121.1992.tb00605.x · Cited by 8,823
McKay, M. D., Beckman, R. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271432 · Cited by 7,415
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Gaussian Processes in Machine Learning
Rasmussen, C. E. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-28650-9_4 · Cited by 5,397
Rees, M., & Cressie, N. (1993) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982871 · Cited by 5,101
Bayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222
Statistics for experimenters. An introduction to design, data analysis, and model building.
Box, G. E. P., Hunter, W. G., & Hunter, J. S. (1978) · Cited by 4,021
Comparative studies of metamodelling techniques under multiple modelling criteria
Jin, R., Chen, W., & Simpson, T. W. (2001) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-001-0160-4 · Cited by 1,709
Self-citationStatistical Methods for Spatial Data Analysis: Texts in Statistical Science
Schabenberger, O., & Gotway, C. A. (2017) · Chapman & Hall/CRC eBooks · doi:10.1201/9781315275086 · Cited by 964
Computer Model Calibration Using High-Dimensional Output
Higdon, D., Gattiker, J., Williams, B., & Rightley, M. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000000888 · Cited by 867
Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building
Parameswaran, R., Box, G. E. P., Hunter, W. G., & Hunter, J. S. (1979) · Journal of Marketing Research · doi:10.2307/3150696 · Cited by 684
A Framework for Validation of Computer Models
Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, J., Lin, C.-H., & Tu, J. (2007) · Technometrics · doi:10.1198/004017007000000092 · Cited by 655
Combining Field Data and Computer Simulations for Calibration and Prediction
Higdon, D., Kennedy, M., Cavendish, J. C., Cafeo, J. A., & Ryne, R. D. (2004) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827503426693 · Cited by 623
Evaluation of gaussian processes and other methods for non-linear regression
Hinton, G. E., & Rasmussen, C. E. (1997) · Library and Archives Canada (Government of Canada) · Cited by 429
Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments
Qian, P. Z. G., & Wu, C. F. J. (2008) · Technometrics · doi:10.1198/004017008000000082 · Cited by 336
Lawson, A., & Cressie, N. (1993) · The Statistician · doi:10.2307/2348117 · Cited by 306
Modularization in Bayesian analysis, with emphasis on analysis of computer models
Bayarri, M. J., Berger, J. O., & Liu, F. (2009) · Bayesian Analysis · doi:10.1214/09-ba404 · Cited by 200
Toward a Better Understanding of Model Validation Metrics
Liu, Y., Chen, W., Arendt, P., & Huang, H.-Z. (2011) · Journal of Mechanical Design · doi:10.1115/1.4004223 · Cited by 175
Self-citationUnderstanding the Effects of Model Uncertainty in Robust Design With Computer Experiments
Apley, D. W., Liu, J., & Chen, W. (2005) · Journal of Mechanical Design · doi:10.1115/1.2204974 · Cited by 160
Self-citationA better understanding of model updating strategies in validating engineering models
Xiong, Y., Chen, W., Tsui, K., & Apley, D. W. (2008) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2008.11.023 · Cited by 148
Self-citationImproving Identifiability in Model Calibration Using Multiple Responses
Arendt, P. D., Apley, D. W., Chen, W., Lamb, D., & Gorsich, D. (2012) · Journal of Mechanical Design · doi:10.1115/1.4007573 · Cited by 146
Self-citationCase studies in Gaussian process modelling of computer codes
Kennedy, M. C., Anderson, C. W., Conti, S., & O’Hagan, A. (2006) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2005.11.028 · Cited by 131
Bayesian Validation of Computer Models
Wang, S., Chen, W., & Tsui, K. (2009) · Technometrics · doi:10.1198/tech.2009.07011 · Cited by 128
Self-citationA Design-Driven Validation Approach Using Bayesian Prediction Models
Chen, W., Xiong, Y., Tsui, K., & Wang, S. (2007) · Journal of Mechanical Design · doi:10.1115/1.2809439 · Cited by 110
Self-citationCalibration and Uncertainty Analysis for Computer Simulations with Multivariate Output
McFarland, J., Mahadevan, S., Romero, V., & Swiler, L. P. (2008) · AIAA Journal · doi:10.2514/1.35288 · Cited by 102
Statistical Adjustments to Engineering Models
Joseph, V. R., & Melkote, S. N. (2009) · Journal of Quality Technology · doi:10.1080/00224065.2009.11917791 · Cited by 89
A Bayesian analysis of the thermal challenge problem
Liu, F., Bayarri, M. J., Berger, J. O., Paulo, R., & Sacks, J. (2008) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2007.05.032 · Cited by 65
Evaluation Of Gaussian Processes And Other Methods For Non-Linear Regression
Rasmussen, C. E. (1996) · Library and Archives Canada (Government of Canada) · Cited by 65
Calibration and Uncertainty Analysis for Computer Simulations with Multivariate Output
McFarland, J., & Swiler, L. (2010) · Wiley eBooks · doi:10.1002/9780470685853.ch9 · Cited by 52
A Unified Framework for Integrated Optimization Under Uncertainty
Wang, Z., Huang, H., & Liu, Y. (2010) · Journal of Mechanical Design · doi:10.1115/1.4001526 · Cited by 42
A kriging approach to the analysis of climate model experiments
Drignei, D. (2009) · Journal of Agricultural Biological and Environmental Statistics · doi:10.1198/jabes.2009.0006 · Cited by 26
Cited by
A paradigm for data-driven predictive modeling using field inversion and machine learning
Parish, E. J., & Duraisamy, K. (2016) · Journal of Computational Physics · doi:10.1016/j.jcp.2015.11.012 · Cited by 620
A review of uncertainty analysis in building energy assessment
Tian, W., Heo, Y., de Wilde, P., Li, Z., Yan, D., Park, C. S., Feng, X., & Augenbroe, G. (2018) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2018.05.029 · Cited by 453
A comprehensive review of digital twin — part 1: modeling and twinning enabling technologies
Thelen, A., Zhang, X., Fink, O., Lu, Y., Ghosh, S., Youn, B. D., Todd, M. D., Mahadevan, S., Hu, C., & Hu, Z. (2022) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-022-03425-4 · Cited by 441
SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics
Kaheman, K., Kutz, J. N., & Brunton, S. L. (2020) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2020.0279 · Cited by 276
Data-Driven Aerospace Engineering: Reframing the Industry with Machine Learning
Brunton, S. L., Nathan Kutz, J., Manohar, K., Aravkin, A. Y., Morgansen, K., Klemisch, J., Goebel, N., Buttrick, J., Poskin, J., Blom-Schieber, A. W., et al. (2021) · AIAA Journal · doi:10.2514/1.j060131 · Cited by 270
Metadata
- Authors
- Arendt, P. D., Apley, D. W., & Chen, W.
- Venue
- Journal of Mechanical Design
- Publisher
- ASME International
- Published
- 2012-09-28
- Volume
- 134
- Issue
- 10
- ISSN
- 0161-8458 (ISSN-L)
- 1050-0472
- 1528-9001
- OpenAlex
- W2072028796
- Cited by
- 332
- References
- 32
- FWCI
- 14.15
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.