The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come - Or Not
Abstract
The use of metamodeling techniques in the design and analysis of computer experiments has progressed remarkably in the past two decades, but how far have we really come? This is the question that we investigate in this paper, namely, the extent to which the use of metamodeling techniques in multidisciplinary design optimization have evolved in the two decades since the seminal paper on Design and Analysis of Computer Experiments by Sacks et al. As part of this review, we examine the motivation for advancements in metamodeling techniques from both a historical perspective and the research itself. Based on current thrusts in the field, we emphasize multi-level/multi-fidelity approximations and ensembles of metamodels, as well as the availability of metamodels within commercial software and for design space exploration and visualization in this review. Our closing remarks offer insight into future research directions – nearly the same ones that have motivated us in the past. I.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
Gunst, R. F., Myers, R. H., & Montgomery, D. C. (1996) · Technometrics · doi:10.2307/1270613 · Cited by 9,530
Cressie, N. (1992) · Terra Nova · doi:10.1111/j.1365-3121.1992.tb00605.x · Cited by 8,823
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
The method of moving asymptotes—a new method for structural optimization
Svanberg, K. (1987) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.1620240207 · Cited by 5,573
Journal of the American Statistical Association
Koehler, A. B. (1995) · International Journal of Forecasting · doi:10.1016/0169-2070(95)90067-5 · Cited by 5,337
Empirical Model Building and Response Surfaces
Freeny, A. (1988) · Technometrics · doi:10.1080/00401706.1988.10488371 · Cited by 5,235
Rees, M., & Cressie, N. (1993) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982871 · Cited by 5,101
Statistics for experimenters. An introduction to design, data analysis, and model building.
Box, G. E. P., Hunter, W. G., & Hunter, J. S. (1978) · Cited by 4,021
Journal of the Royal Statistical Society.
(1923) · The Economic Journal · doi:10.1093/ej/33.130.269 · Cited by 3,219
Support Vector Machines for Classification and Regression
Gunn, S. R. (1998) · Cited by 2,820
Multiquadric equations of topography and other irregular surfaces
Hardy, R. L. (1971) · Journal of Geophysical Research Atmospheres · doi:10.1029/jb076i008p01905 · Cited by 2,811
Engineering Design via Surrogate Modelling: A Practical Guide
Forrester, A. I. J., Sóbester, A., & Keane, A. J. (2008) · doi:10.1002/9780470770801 · Cited by 2,309
Metamodels for Computer-based Engineering Design: Survey and recommendations
Simpson, T. W., Poplinski, J. D., Koch, P. N., & Allen, J. K. (2001) · Engineering with Computers · doi:10.1007/pl00007198 · Cited by 1,951
Self-citationComparative studies of metamodelling techniques under multiple modelling criteria
Jin, R., Chen, W., & Simpson, T. W. (2001) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-001-0160-4 · Cited by 1,709
Self-citationReview of Metamodeling Techniques in Support of Engineering Design Optimization
Wang, G. G., & Shan, S. (2007) · Journal of Mechanical Design · doi:10.1115/1.2429697 · Cited by 1,706
Engineering Design via Surrogate Modelling: A Practical Guide
Keane, A., Forrester, A., & Sobester, A. (2008) · American Institute of Aeronautics and Astronautics eBooks · doi:10.2514/4.479557 · Cited by 1,686
Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization
Simpson, T. W., Mauery, T. M., Korte, J. J., & Mistree, F. (2001) · AIAA Journal · doi:10.2514/2.1234 · Cited by 1,097
Self-citationSpace Mapping: The State of the Art
Bandler, J. W., Cheng, Q. S., Dakroury, S. A., Mohamed, A. S., Bakr, M. H., Madsen, K., & Sondergaard, J. (2004) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2003.820904 · Cited by 1,064
Multidisciplinary aerospace design optimization: survey of recent developments
Sobieszczanski-Sobieski, J., & Haftka, R. T. (1997) · Structural Optimization · doi:10.1007/bf01197554 · Cited by 1,052
Multi-fidelity optimization via surrogate modelling
Forrester, A. I. J., Sóbester, A., & Keane, A. J. (2007) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2007.1900 · Cited by 1,018
A rigorous framework for optimization of expensive functions by surrogates
Booker, A. J., Dennis, J. E., Frank, P. D., Serafini, D. B., Torczon, V., & Trosset, M. W. (1999) · Structural Optimization · doi:10.1007/bf01197708 · Cited by 960
Space mapping technique for electromagnetic optimization
Bandler, J. W., Biernacki, R. M., Chen, S. H., Grobelny, P. A., & Hemmers, R. H. (1994) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/22.339794 · Cited by 697
A trust-region framework for managing the use of approximation models in optimization
Alexandrov, N. M., Dennis, J. E., Lewis, R. M., & Torczon, V. J. (1998) · Structural and Multidisciplinary Optimization · doi:10.1007/bf01197433 · Cited by 642
Structural optimization: A new dual method using mixed variables
Fleury, C., & Braibant, V. (1986) · International Journal for Numerical Methods in Engineering · doi:10.1002/nme.1620230307 · Cited by 612
Model Selection Via Multifold Cross Validation
Zhang, P. (1993) · The Annals of Statistics · doi:10.1214/aos/1176349027 · Cited by 539
Approximation methods in multidisciplinary analysis and optimization: a panel discussion
Simpson, T. W., Booker, A. J., Ghosh, D., Giunta, A. A., Koch, P. N., & Yang, R.-J. (2004) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-004-0389-9 · Cited by 412
Self-citationNumerical Procedures for Surface Fitting of Scattered Data by Radial Functions
Dyn, N., Levin, D., & Rippa, S. (1986) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0907043 · Cited by 402
Sampling Strategies for Computer Experiments
Simpson, T. W., Dennis, L. A., & Chen, W. (2001) · Cited by 396
Self-citationOverview of Modern Design of Experiments Methods for Computational Simulations (Invited)
Giunta, A., Wojtkiewicz, S., & Eldred, M. (2003) · 41st Aerospace Sciences Meeting and Exhibit · doi:10.2514/6.2003-649 · Cited by 388
Koehler, J. R., & Owen, A. B. (1996) · Handbook of Statistics · doi:10.1016/s0169-7161(96)13011-x · Cited by 322
A statistical method for global optimization
Cox, D. D., & John, S. (2003) · [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1992.271617 · Cited by 302
Combining Estimates in Regression and Classification
Leblanc, M., & Tibshirani, R. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476733 · Cited by 290
Computational Approaches for Aerospace Design: The Pursuit of Excellence
Keane, A. J., & Nair, P. B. (2005) · Wiley eBooks · doi:10.1002/0470855487 · Cited by 284
CONLIN: An efficient dual optimizer based on convex approximation concepts
Fleury, C. (1989) · Structural and Multidisciplinary Optimization · doi:10.1007/bf01637664 · Cited by 273
Computational Approaches for Aerospace Design: The Pursuit of Excellence
Keane, A. J., & Nair, P. B. (2005) · Cited by 256
Optimization with variable-fidelity models applied to wing design
Alexandrov, N. M., Lewis, R., Gumbert, C. R., et al. (2000) · AIAA Aerospace Sciences Meeting and Exhibit · doi:10.2514/6.2000-841 · Cited by 225
Giunta, A. (1997) · VTechWorks (Virginia Tech) · Cited by 220
Multiple Surrogate Modeling for Axial Compressor Blade Shape Optimization
Samad, A., Kim, K., Goel, T., Haftka, R. T., & Shyy, W. (2008) · Journal of Propulsion and Power · doi:10.2514/1.28999 · Cited by 210
Statistical Approximations for Multidisciplinary Design Optimization: The Problem of Size
Koch, P. N., Simpson, T. W., Allen, J. K., & Mistree, F. (1999) · Journal of Aircraft · doi:10.2514/2.2435 · Cited by 203
Self-citationSecond-Order Corrections for Surrogate-Based Optimization with Model Hierarchies
Eldred, M., Giunta, A., & Collis, S. S. (2004) · AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (MAO) · doi:10.2514/6.2004-4457 · Cited by 189
Combining global and local approximations
Haftka, R. T. (1991) · AIAA Journal · doi:10.2514/3.10768 · Cited by 185
Hybrid Variable Fidelity Optimization by Using a Kriging-Based Scaling Function
Gano, S., Renaud, J. E., & Sanders, B. (2005) · AIAA Journal · doi:10.2514/1.12466 · Cited by 169
Update strategies for kriging models used in variable fidelity optimization
Gano, S. E., Renaud, J. E., Martin, J. D., & Simpson, T. W. (2006) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-006-0025-y · Cited by 161
Self-citationPrediction Error and Its Estimation for Subset-Selected Models
Roecker, E. B. (1991) · Technometrics · doi:10.1080/00401706.1991.10484873 · Cited by 161
Metamodeling using extended radial basis functions: a comparative approach
Mullur, A. A., & Messac, A. (2005) · Engineering With Computers · doi:10.1007/s00366-005-0005-7 · Cited by 158
Wing Optimization Using Design of Experiment, Response Surface, and Data Fusion Methods
Keane, A. J. (2003) · Journal of Aircraft · doi:10.2514/2.3153 · Cited by 154
Robust Design Simulation: A Probabilistic Approach to Multidisciplinary Design
Mavris, D. N., Bandte, O., & DeLaurentis, D. A. (1999) · Journal of Aircraft · doi:10.2514/2.2437 · Cited by 152
Proceedings of the Royal Society
Paillot, R. (1892) · Journal de Physique Théorique et Appliquée · doi:10.1051/jphystap:018920010017701 · Cited by 123
REGRESSION WITH MULTIPLE CANDIDATE MODELS: SELECTING OR MIXING?Preprint
Yang, Y. (1999) · Iowa State University Digital Repository (Iowa State University) · Cited by 120
Formulations for Surrogate-Based Optimization with Data Fit, Multifidelity, and Reduced-Order Models
Eldred, M., & Dunlavy, D. (2006) · AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (MAO) · doi:10.2514/6.2006-7117 · Cited by 120
Toward an optimal ensemble of kernel-based approximations with engineering applications
Sánchez, E., Pintos, S. A., & Queipo, N. V. (2007) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-007-0159-6 · Cited by 112
Visual Steering Commands for Trade Space Exploration: User-Guided Sampling With Example
Stump, G. M., Lego, S. E., Yukish, M., et al. (2009) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.3243633 · Cited by 108
Utans, J., & Moody, J. E. (2002) · International Conference on Artificial Intelligence Applications on Wall Street · doi:10.1109/aiaws.1991.236576 · Cited by 102
Empirical Model-Building and Response Surfaces.
Shoesmith, E., Box, G. E. P., & Draper, N. R. (1988) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348383 · Cited by 102
A Knowledge-Based Approach To Response Surface Modelling in Multifidelity Optimization
Leary, S. J., Bhaskar, A., & Keane, A. J. (2003) · Journal of Global Optimization · doi:10.1023/a:1023283917997 · Cited by 101
Neural networks in structural analysis and design: An overview
Hajela, P., & Berke, L. (1992) · Computing Systems in Engineering · doi:10.1016/0956-0521(92)90138-9 · Cited by 84
First and second order convex approximation strategies in structural optimization
Fleury, C. (1989) · Structural and Multidisciplinary Optimization · doi:10.1007/bf01743804 · Cited by 82
A Constraint Mapping Approach to the Structural Optimization of an Expensive Model using Surrogates
Leary, S. J., Bhaskar, A., & Keane, A. J. (2001) · Optimization and Engineering · doi:10.1023/a:1016038305014 · Cited by 70
Multifidelity Optimization for Variable-Complexity Design
Robinson, T., Willcox, K. E., Eldred, M., & Haimes, R. (2006) · AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference (MAO) · doi:10.2514/6.2006-7114 · Cited by 62
Experimental Design and Structural Optimization
Schoofs, A. J. G. (1988) · Structural Optimization · doi:10.1007/978-94-009-1413-1_39 · Cited by 59
The effect of System Response Time on interactive computer aided problem solving
Goodman, T., & Spence, R. (1978) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/800248.807378 · Cited by 56
Multifidelity Response Surface Approximations for the Optimum Design of Diffuser Flows
Madsen, J. I., & Langthjem, M. A. (2001) · Optimization and Engineering · doi:10.1023/a:1016046606831 · Cited by 41
Multiple Surrogates for the Shape Optimization of Bluff Body-facilitated Mixing
Mack, Y., Goel, T., Shyy, W., Haftka, R. T., & Queipo, N. V. (2005) · AIAA Aerospace Sciences Meeting and Exhibit · doi:10.2514/6.2005-333 · Cited by 40
The effect of System Response Time on interactive computer aided problem solving
Goodman, T., & Spence, R. (1978) · ACM SIGGRAPH Computer Graphics · doi:10.1145/965139.807378 · Cited by 37
Simpson, T. W., Barron, K., Rothrock, L., Frecker, M., Barton, R. R., & Ligetti, C. (2007) · Research in Engineering Design · doi:10.1007/s00163-007-0033-y · Cited by 32
Self-citationResponse surface approximations for structural optimization
Roux, W. J., Stander, N., & Haftka, R. T. (1998) · International Journal for Numerical Methods in Engineering · doi:10.1002/(sici)1097-0207(19980615)42:3<517::aid-nme370>3.0.co;2-l · Cited by 19
Ligetti, C. B., & Simpson, T. W. (2005) · Journal of Computing and Information Science in Engineering · doi:10.1115/1.1794698 · Cited by 17
Self-citationSimpson, T. W., Iyer, P. S., Rothrock, L., Frecker, M., Barton, R. R., Barron, K., & Meckesheimer, M. (2005) · AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (SDM) · doi:10.2514/6.2005-2060 · Cited by 11
Self-citation
Cited by
Recent advances in surrogate-based optimization
Forrester, A. I. J., & Keane, A. J. (2009) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2008.11.001 · Cited by 2,357
Review of surrogate modeling in water resources
Razavi, S., Tolson, B. A., & Burn, D. H. (2012) · Water Resources Research · doi:10.1029/2011wr011527 · Cited by 983
Hierarchical Kriging Model for Variable-Fidelity Surrogate Modeling
Han, Z.-H., & Görtz, S. (2012) · AIAA Journal · doi:10.2514/1.j051354 · Cited by 490
Managing computational complexity using surrogate models: a critical review
Alizadeh, R., Allen, J. K., & Mistree, F. (2020) · Research in Engineering Design · doi:10.1007/s00163-020-00336-7 · Cited by 453
Surrogate modelling for sustainable building design – A review
Westermann, P., & Evins, R. (2019) · Energy and Buildings · doi:10.1016/j.enbuild.2019.05.057 · Cited by 340
Metadata
- Authors
- Simpson, T., Toropov, V., Balabanov, V., & Viana, F.
- Venue
- 12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
- Published
- 2008-06-14
- OpenAlex
- W2319488728
- Cited by
- 302
- References
- 72
- FWCI
- 16.71
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.