The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
An evaluation of adaptive surrogate modeling based optimization with two benchmark problems
Open accessAbstract
Surrogate modeling uses cheap “surrogates” to represent the response surface of simulation models. It involves several steps, including initial sampling, regression and adaptive sampling. This study evaluates an adaptive surrogate modeling based optimization (ASMO) method on two benchmark problems: the Hartman function and calibration of the SAC-SMA hydrologic model. Our results show that: 1) Gaussian Processes are the best surrogate model construction method. A minimum Interpolation Surface method is the best adaptive sampling method. Low discrepancy Quasi Monte Carlo methods are the most suitable initial sampling designs. Some 15–20 times the dimension of the problem may be the proper initial sample size; 2) The ASMO method is much more efficient than the widely used Shuffled Complex Evolution global optimization method. However, ASMO can provide only approximate optimal solutions, whose precision is limited by surrogate modeling methods and problem-specific features; and 3) The identifiability of model parameters is correlated with parameter sensitivity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Classification and Regression by randomForest
Liaw, A. I., & Wiener, M. C. (2007) · Cited by 18,594
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Efficient Global Optimization of Expensive Black-Box Functions
Jones, D. R., Schonlau, M., & Welch, W. J. (1998) · Journal of Global Optimization · doi:10.1023/a:1008306431147 · Cited by 8,070
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
McKay, M. D., Beckman, R. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271432 · Cited by 7,415
Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates
Sobol′, I. M. (2001) · Mathematics and Computers in Simulation · doi:10.1016/s0378-4754(00)00270-6 · Cited by 6,389
The Nature of Statistical Learning Theory
Sain, S. R. (1996) · Technometrics · doi:10.1080/00401706.1996.10484565 · Cited by 5,434
Some New Three Level Designs for the Study of Quantitative Variables
Box, G. E. P., & Behnken, D. W. (1960) · Technometrics · doi:10.1080/00401706.1960.10489912 · Cited by 4,177
Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter
Golub, G. H., Heath, M., & Wahba, G. (1979) · Technometrics · doi:10.1080/00401706.1979.10489751 · Cited by 3,934
Myers, R. H., & Montgomery, D. C. (1996) · IIE Transactions · doi:10.1080/15458830.1996.11770760 · Cited by 3,819
McKay, M. D., Beckman, R. J., & Conover, W. J. (1979) · Technometrics · doi:10.2307/1268522 · Cited by 3,757
Effective and efficient global optimization for conceptual rainfall‐runoff models
Duan, Q., Sorooshian, S., & Gupta, V. (1992) · Water Resources Research · doi:10.1029/91wr02985 · Cited by 3,310
Self-citationComputer Models of Watershed Hydrology
Singh, V. P. (1995) · Medical Entomology and Zoology · Cited by 3,273
On the distribution of points in a cube and the approximate evaluation of integrals
Sobol', I. M. (1967) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(67)90144-9 · Cited by 2,975
Random Number Generation and Quasi-Monte Carlo Methods.
Collings, B. J., & Niederreiter, H. (1993) · Journal of the American Statistical Association · doi:10.2307/2290359 · Cited by 2,884
Sensitivity Estimates for Nonlinear Mathematical Models
Sobol, I. M. (1990) · Medical Entomology and Zoology · Cited by 2,806
Random Number Generation and Quasi-Monte Carlo Methods
Niederreiter, H. (1992) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611970081 · Cited by 2,770
Surrogate-based analysis and optimization
Queipo, N. V., Haftka, R. T., Shyy, W., Goel, T., Vaidyanathan, R., & Kevin Tucker, P. (2005) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2005.02.001 · Cited by 2,446
Recent advances in surrogate-based optimization
Forrester, A. I. J., & Keane, A. J. (2009) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2008.11.001 · Cited by 2,357
The Nature of Statistical Learning
Vapnik, V. N. (1995) · Medical Entomology and Zoology · Cited by 2,295
A Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,159
Shuffled complex evolution approach for effective and efficient global minimization
Duan, Q. Y., Gupta, V. K., & Sorooshian, S. (1993) · Journal of Optimization Theory and Applications · doi:10.1007/bf00939380 · Cited by 1,722
Self-citationStatistics for experimenters: design, innovation, and discovery
(2006) · Choice Reviews Online · doi:10.5860/choice.43-3429 · Cited by 1,644
Characterising performance of environmental models
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J., Marsili-Libelli, S., Newham, L. T. H., Norton, J. P., Perrin, C., et al. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.09.011 · Cited by 1,536
Optimal use of the SCE-UA global optimization method for calibrating watershed models
Duan, Q., Sorooshian, S., & Gupta, V. K. (1994) · Journal of Hydrology · doi:10.1016/0022-1694(94)90057-4 · Cited by 1,508
Self-citationReview of surrogate modeling in water resources
Razavi, S., Tolson, B. A., & Burn, D. H. (2012) · Water Resources Research · doi:10.1029/2011wr011527 · Cited by 983
How to avoid a perfunctory sensitivity analysis
Saltelli, A., & Annoni, P. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.04.012 · Cited by 974
Multi-objective global optimization for hydrologic models
Yapo, P. O., Gupta, H. V., & Sorooshian, S. (1998) · Journal of Hydrology · doi:10.1016/s0022-1694(97)00107-8 · Cited by 908
Sensitivity analysis for non-linear mathematical models
Sobol, I. M. (1993) · Medical Entomology and Zoology · Cited by 878
(2024) · Multivariate Biomarker Discovery · doi:10.1017/9781009006767.012 · Cited by 809
Choosing the Sample Size of a Computer Experiment: A Practical Guide
Loeppky, J. L., Sacks, J., & Welch, W. J. (2009) · Technometrics · doi:10.1198/tech.2009.08040 · Cited by 758
Bayesian analysis of computer code outputs: A tutorial
O’Hagan, A. (2006) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2005.11.025 · Cited by 755
Duan, Q., Schaake, J., Andréassian, V., Franks, S., Goteti, G., Gupta, H. V., Gusev, Y. M., Habets, F., Hall, A., Hay, L., et al. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.031 · Cited by 701
Self-citationBoyle, D. P., Gupta, H. V., & Sorooshian, S. (2000) · Water Resources Research · doi:10.1029/2000wr900207 · Cited by 696
Algorithm 247: Radical-inverse quasi-random point sequence
Halton, J. H. (1964) · Communications of the ACM · doi:10.1145/355588.365104 · Cited by 601
Ajami, N. K., Duan, Q., & Sorooshian, S. (2007) · Water Resources Research · doi:10.1029/2005wr004745 · Cited by 597 · cited 1×
“It is also a benchmark problem for evaluation of many parameter estimation methods (Duan et al., 1992, 1994; Yapo et al., 1998; Ajami et al., 2007).”
BackgroundSelf-citationIman, R. L., Helton, J. C., & Campbell, J. E. (1981) · Journal of Quality Technology · doi:10.1080/00224065.1981.11978748 · Cited by 584
Sorooshian, S., Duan, Q., & Gupta, V. K. (1993) · Water Resources Research · doi:10.1029/92wr02617 · Cited by 510
Self-citationGlobal versus local search in constrained optimization of computer models
Schonlau, M., Welch, W. J., & Jones, D. R. (1998) · Institute of Mathematical Statistics Lecture Notes - Monograph Series · doi:10.1214/lnms/1215456182 · Cited by 435
Experiments: Planning, Analysis, and Optimization
Wu, C. F. J., & Hamada, M. (2021) · Wiley Series in Probability and Statistics · doi:10.1002/9781119470007 · Cited by 407
Proceedings of the Winter Simulation Conference
Rose, O., & Uhrmacher, A. M. (2012) · Winter Simulation Conference · Cited by 401
Jap, D., Stöttinger, M., & Bhasin, S. (2015) · Hardware and Architectural Support for Security and Privacy · doi:10.1145/2768566.2768568 · Cited by 347
A statistical method for global optimization
Cox, D. D., & John, S. (2003) · [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1992.271617 · Cited by 302
On the Design of Optimization Strategies Based on Global Response Surface Approximation Models
Sóbester, A., Leary, S. J., & Keane, A. J. (2005) · Journal of Global Optimization · doi:10.1007/s10898-004-6733-1 · Cited by 299
Gan, T. Y., & Biftu, G. F. (1996) · Water Resources Research · doi:10.1029/95wr02195 · Cited by 232
van Werkhoven, K., Wagener, T., Reed, P., & Tang, Y. (2009) · Advances in Water Resources · doi:10.1016/j.advwatres.2009.03.002 · Cited by 198
The construction and implementation of metamodels
Blanning, R. W. (1975) · SIMULATION · doi:10.1177/003754977502400606 · Cited by 196
Quantile-Based Optimization of Noisy Computer Experiments With Tunable Precision
Picheny, V., Ginsbourger, D., Richet, Y., & Caplin, G. (2013) · Technometrics · doi:10.1080/00401706.2012.707580 · Cited by 184
A Multistep Automatic Calibration Scheme for River Forecasting Models
Hogue, T. S., Sorooshian, S., Gupta, H. V., Holz, A., & Braatz, D. T. (2000) · Journal of Hydrometeorology · Cited by 163
A general framework for Dynamic Emulation Modelling in environmental problems
Castelletti, A. F., Galelli, S., Ratto, M., Soncini‐Sessa, R., & Young, P. C. (2012) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.01.002 · Cited by 150
Numerical assessment of metamodelling strategies in computationally intensive optimization
Razavi, S., Tolson, B. A., & Burn, D. H. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.09.010 · Cited by 145
Approximating SWAT Model Using Artificial Neural Network and Support Vector Machine1
Zhang, X., Srinivasan, R., & Liew, M. W. V. (2009) · JAWRA Journal of the American Water Resources Association · doi:10.1111/j.1752-1688.2009.00302.x · Cited by 137
Considerations for parameter optimization and sensitivity in climate models
Neelin, J. D., Bracco, A., Luo, H., McWilliams, J. C., & Meyerson, J. E. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1015473107 · Cited by 136
Bayesian Algorithms for One-Dimensional Global Optimization
Locatelli, M. (1997) · Journal of Global Optimization · doi:10.1023/a:1008294716304 · Cited by 122
Emulation techniques for the reduction and sensitivity analysis of complex environmental models
Ratto, M., Castelletti, A. F., & Pagano, A. J. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.11.003 · Cited by 113
A spatial regularization approach to parameter estimation for a distributed watershed model
Pokhrel, P., Gupta, H. V., & Wagener, T. (2008) · Water Resources Research · doi:10.1029/2007wr006615 · Cited by 106
A global search method for optimizing nonlinear systems
Stuckman, B. E. (1988) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.23094 · Cited by 77
Multilevel calibration strategy for complex hydrologic simulation models
Brazil, L. E. (1988) · Cited by 68
Castelletti, A. F., Pianosi, F., Soncini‐Sessa, R., & Antenucci, J. P. (2010) · Water Resources Research · doi:10.1029/2009wr008389 · Cited by 57
Factor screening for simulation with multiple responses: Sequential bifurcation
Shi, W., Kleijnen, J. P. C., & Liu, Z. (2014) · European Journal of Operational Research · doi:10.1016/j.ejor.2014.02.021 · Cited by 42
Moreno, H. A., Vivoni, E. R., & Gochis, D. (2012) · Journal of Hydrology · doi:10.1016/j.jhydrol.2012.03.019 · Cited by 35
Ajami, N., Duan, Q., & Sorooshian, S. (2006) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · Cited by 31
Self-citationEvaluating of path-dependent securities with low discrepancy methods
Krykova, I. (2004) · Digital WPI · Cited by 29
Evaluation of MARS modeling technique for sensitivity analysis of model output
Shahsavani, D., Tarantola, S., & Ratto, M. (2010) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2010.05.204 · Cited by 22
Cited by
AutoML: A survey of the state-of-the-art
He, X., Zhao, K., & Chu, X. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,691
Fatichi, S., Vivoni, E. R., Ogden, F. L., Ivanov, V. Y., Mirus, B., Gochis, D., Downer, C. W., Camporese, M., Davison, J. H., Ebel, B., et al. (2016) · Journal of Hydrology · doi:10.1016/j.jhydrol.2016.03.026 · Cited by 671
Managing computational complexity using surrogate models: a critical review
Alizadeh, R., Allen, J. K., & Mistree, F. (2020) · Research in Engineering Design · doi:10.1007/s00163-020-00336-7 · Cited by 453
Machine learning for hydrologic sciences: An introductory overview
Xu, T., & Liang, F. (2021) · WIREs Water · doi:10.1002/wat2.1533 · Cited by 267
Metadata
- Authors
- Wang, C., Duan, Q., Gong, W., Ye, A., Di, Z., & Miao, C.
- Venue
- Environmental Modelling & Software
- Publisher
- Elsevier BV
- Published
- 2014-10
- Volume
- 60
- Pages
- 167-179
- ISSN
- 1364-8152 (ISSN-L)
- OpenAlex
- W2038538989
- Cited by
- 271
- References
- 68
- FWCI
- 12.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.