Large sample behaviors of the generalized likelihood uncertainty estimation (GLUE) in assessing the uncertainty of rainfall‐runoff simulations
Open accessThe DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index.
Abstract
Several methods have been recently proposed for quantifying the uncertainty of hydrological models. These techniques are based upon different hypotheses, are diverse in nature, and produce outputs that can significantly differ in some cases. One of the favored methods for uncertainty assessment in rainfall‐runoff modeling is the generalized likelihood uncertainty estimation (GLUE). However, some fundamental questions related to its application remain unresolved. One such question is that GLUE relies on some explicit and implicit assumptions, and it is not fully clear how these may affect the uncertainty estimation when referring to large samples of data. The purpose of this study is to address this issue by assessing how GLUE performs in detecting uncertainty in the simulation of long series of synthetic river flows. The study aims to (1) discuss the hypotheses underlying GLUE and derive indications about their effects on the uncertainty estimation, and (2) compare the GLUE prediction limits with a large sample of data that is to be simulated in the presence of known sources of uncertainty. The analysis shows that the prediction limits provided by GLUE do not necessarily include a percentage close to their confidence level of the observed data. In fact, in all the experiments, GLUE underestimates the total uncertainty of the simulation provided by the hydrological model.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similarly studies such as Montanari (2005) have shown that GLUE can underestimate the posterior output uncertainty.”
From A theoretical and real world evaluation of two Bayesian techniques for the calibration of variety parameters in a sugarcane crop model (2016) · cites this paper as Montanari (2005)
Citations
Cites
River flow forecasting through conceptual models part I — A discussion of principles
Nash, J. E., & Sutcliffe, J. V. (1970) · Journal of Hydrology · doi:10.1016/0022-1694(70)90255-6 · Cited by 23,674 · cited 2×
“As examples, Cameron et al. [1999] used the Nash and Sutcliffe [1970] efficiency to evaluate the likelihood of the simulation of a continuous hydrograph, and Beven and Freer [2001] present a list of likelihood measures that can be used in rainfall-runoff modeling.”
MethodLocally Weighted Regression: An Approach to Regression Analysis by Local Fitting
Cleveland, W. S., & Devlin, S. J. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478639 · Cited by 5,742 · cited 1×
“Smoothing was conducted by using the loess smoother that operates through a local regression model [Cleveland and Devlin, 1988].”
MethodThe future of distributed models: model calibration and uncertainty prediction
Beven, K., & Binley, A. (1992) · Hydrological Processes · doi:10.1002/hyp.3360060305 · Cited by 4,527 · cited 1×
“[3] One of the most known and used uncertainty assessment methods in hydrological modeling is the generalized likelihood uncertainty estimation (GLUE), proposed by Beven and Binley [1992] , that is briefly described in section 2 of the present article.”
MethodGuidelines for predicting crop water requirements
Doorenbos, J. (1977) · Medical Entomology and Zoology · Cited by 3,331
Effective and efficient global optimization for conceptual rainfall‐runoff models
Duan, Q., Sorooshian, S., & Gupta, V. (1992) · Water Resources Research · doi:10.1029/91wr02985 · Cited by 3,310 · cited 2×
“[31] By using a 30-year-long record of uncorrupted synthetic data (years from 1 to 30 of the synthetic records), HYMOD was first calibrated in an optimality context by minimizing the squared difference between Qobs(t) and Qsim(t) through a generic algorithm [Duan et al., 1992].”
MethodStatistics for Long-Memory Processes.
Ray, W. D., & Beran, J. (1996) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983481 · Cited by 2,686 · cited 3×
“Thus model calibration can be carried out by minimizing [Beran, 1994] W qð Þ ¼ XN=2 j¼1 log fM lj; q þ fe lj;f þ J lj fM lj; q þ fe lj;f ( ) : ð10Þ In the above expression, the spectral density function of the model, fM(lj, q), has been estimated by computing the smoothed periodogram of a long…”
MethodA manifesto for the equifinality thesis
Beven, K. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.007 · Cited by 2,581 · cited 10×
“There is an implicit assumption in GLUE that the errors in the prediction will be similar to those in the evaluation period [ Beven, 2005 ].”
BackgroundResultInfluentialBeven, K., & Freer, J. (2001) · Journal of Hydrology · doi:10.1016/s0022-1694(01)00421-8 · Cited by 2,189 · cited 1×
“[7] It is important to remark that even in the ideal context this study refers to, one should not expect that a specified proportion of synthetic observations will lie within the GLUE prediction limits [Beven and Freer, 2001].”
BackgroundStatistics for Long-Memory Processes.
Percival, D. B., & Beran, J. (1996) · Journal of the American Statistical Association · doi:10.2307/2291761 · Cited by 1,480
Vrugt, J. A., Gupta, H. V., Bouten, W., & Sorooshian, S. (2003) · Water Resources Research · doi:10.1029/2002wr001642 · Cited by 1,196
Research works printed but not linked: 1
- Boyle D. P.(2000) Multicriteria calibration of hydrological models Ph.D. dissertation Dep. of Hydrol. and Water Resour. Univ. of Ariz. Tucson.
Cited by
A decade of Predictions in Ungauged Basins (PUB)—a review
Hrachowitz, M., Savenije, H. H. G., Blöschl, G., McDonnell, J. J., Sivapalan, M., Pomeroy, J. W., Arheimer, B., Blume, T., Clark, M. P., Ehret, U., et al. (2013) · Hydrological Sciences Journal · doi:10.1080/02626667.2013.803183 · Cited by 1,295
“Notwithstanding the significant advances in model uncertainty assessment discussed above, Wagener and Montanari (2011) pointed out that the focus needs to shift from reducing model uncertainty towards reducing the uncertainty in our understanding of how catchments function under given environmental boundary conditions…”
BackgroundMethodInfluentialSelf-citationEnsemble flood forecasting: A review
Cloke, H. L., & Pappenberger, F. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.06.005 · Cited by 1,139
Fatichi, S., Vivoni, E. R., Ogden, F. L., Ivanov, V. Y., Mirus, B., Gochis, D., Downer, C. W., Camporese, M., Davison, J. H., Ebel, B., et al. (2016) · Journal of Hydrology · doi:10.1016/j.jhydrol.2016.03.026 · Cited by 671
“However, applications of these approaches have been mostly carried out using coarse, conceptual models applied to watersheds (Beven and Freer, 2001; Montanari, 2005; Vrugt et al., 2005) or groundwater hydrology models (e.g., Hill and Tiedeman, 2007).”
Benchmarking observational uncertainties for hydrology: rainfall, river discharge and water quality
McMillan, H., Krueger, T., & Freer, J. (2012) · Hydrological Processes · doi:10.1002/hyp.9384 · Cited by 522
“…discussion and counter discussion about the merits for making strong and weak judgements about the nature of data, parameter and structural errors (Montanari, 2005; Mantovan and Todini, 2006; Beven et al., 2007; Mantovan et al., 2007; Beven et al., 2008; Stedinger et al., 2008; Beven, 2009; Vrugt…”
BackgroundGlobal‐scale regionalization of hydrologic model parameters
Beck, H. E., van Dijk, A. I. J. M., de Roo, A., Miralles, D. G., McVicar, T. R., Schellekens, J., & Bruijnzeel, L. A. (2016) · Water Resources Research · doi:10.1002/2015wr018247 · Cited by 511
“…approach [Beven and Binley, 1992], in that they lack a formal statistical foundation and involve several subjective choices (notably the calibration score threshold, the choice of 10 most similar donors, and the similarity criterion) [McIntyre et al., 2005; Montanari, 2005; Winsemius et al., 2009].”
BackgroundEquifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?
Vrugt, J. A., ter Braak, C. J. F., Gupta, H. V., & Robinson, B. A. (2009) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-008-0274-y · Cited by 455
“…the hydrologic literature have criticized GLUE for not being formally Bayesian, resulting in parameter and predictive distributions that are statistically incoherent, unreliable, and that should therefore not be used (Christensen 2004; Montanari 2005; Mantovan and Todini 2006; Vogel et al. 2008).”
MethodTime stability of catchment model parameters: Implications for climate impact analyses
Merz, R., Parajka, J., & Blöschl, G. (2011) · Water Resources Research · doi:10.1029/2010wr009505 · Cited by 432
“This large variability may be partly related to parameter identifiability issues [Beven and Binley ,1 992; Montanari, 2005 ].”
BackgroundAppraisal of the generalized likelihood uncertainty estimation (GLUE) method
Stedinger, J. R., Vogel, R. M., Lee, S. U., & Batchelder, R. (2008) · Water Resources Research · doi:10.1029/2008wr006822 · Cited by 386
“Christensen [2004], Montanari [2005] , Mantovan and Todini [2006] and this study provide such reviews.”
BackgroundMethodResultInfluentialVerification tools for probabilistic forecasts of continuous hydrological variables
Laio, F., & Tamea, S. (2007) · Hydrology and Earth System Sciences · doi:10.5194/hess-11-1267-2007 · Cited by 358
“…are nowadays commonly used to quantify the prediction uncertainty and to supplement the information provided by point-value predictions (Krzysztofowicz, 2001; Ferraris et al., 2002; To-dini, 2004; Montanari and Brath, 2004; Siccardi et al., 2005; Montanari, 2005; Tamea et al., 2005; Beven, 2006).”
BackgroundBlasone, R.-S., Vrugt, J. A., Madsen, H., Rosbjerg, D., Robinson, B. A., & Zyvoloski, G. A. (2008) · Advances in Water Resources · doi:10.1016/j.advwatres.2007.12.003 · Cited by 336
“Various likelihood functions have been proposed in the literature (e.g. Beven and Binley, 1992; Romanowicz et al., 1994; Christensen, 2004; Montanari, 2005) as measures that quantify the closeness between model simulations and observations.”
BackgroundMethodInfluentialJin, X., Xu, C.-Y., Zhang, Q., & Singh, V. P. (2010) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.12.028 · Cited by 324
“Previous studies (e.g., Montanari, 2005; Blasone et al., 2008b) have highlighted that resulting parameter uncertainty and model simulation results are sensitive to the choice of threshold values.”
ResultSo just why would a modeller choose to be incoherent?
Beven, K. J., Smith, P. J., & Freer, J. E. (2008) · Journal of Hydrology · doi:10.1016/j.jhydrol.2008.02.007 · Cited by 261
Bastola, S., Murphy, C., & Sweeney, J. (2011) · Advances in Water Resources · doi:10.1016/j.advwatres.2011.01.008 · Cited by 248
“To examine the capability of the prediction intervals to capture the observed values, an index defined as the ratio of the number of the observations falling within their respective prediction intervals to the total number of observations is normally used (e.g., Montanari, 2005).”
BackgroundMethodMontanari, A. (2007) · Hydrological Processes · doi:10.1002/hyp.6623 · Cited by 239
“Therefore, the obtained confidence limits do not necessarily encompass the true observation with a given frequency (Montanari, 2005).”
BackgroundMethodSelf-citationA blueprint for process‐based modeling of uncertain hydrological systems
Montanari, A., & Koutsoyiannis, D. (2012) · Water Resources Research · doi:10.1029/2011wr011412 · Cited by 235
Self-citationEstimating the uncertainty of hydrological forecasts: A statistical approach
Montanari, A., & Grossi, G. (2008) · Water Resources Research · doi:10.1029/2008wr006897 · Cited by 229
“runoff model that was introduced by Boyle [2000] and recently used by Wagener et al. [2001], Vrugt et al. [2003], and Montanari [2005] .”
BackgroundMethodSelf-citationRojas, R., Feyen, L., & Dassargues, A. (2008) · Water Resources Research · doi:10.1029/2008wr006908 · Cited by 219
“[16] Some controversy has recently been raised concerning the subjective nature of some decisions that have to be made in order to implement GLUE [see e.g., Montanari, 2005; Mantovan and Todini, 2006; Beven et al., 2007, 2008; Mantovan et al., 2007].”
BackgroundBlasone, R.-S., Madsen, H., & Rosbjerg, D. (2008) · Journal of Hydrology · doi:10.1016/j.jhydrol.2007.12.026 · Cited by 193
“Since its first introduction, it has been extensively applied in hydrology and environmental modelling to estimate the uncertainty associated with model outputs and parameter estimates (among others Beven and Binley, 1992; Freer et al., 1996; Lamb et al., 1998; Montanari, 2005).”
BackgroundWinsemius, H. C., Schaefli, B., Montanari, A., & Savenije, H. H. G. (2009) · Water Resources Research · doi:10.1029/2009wr007706 · Cited by 189
“We generally agree about the subjectivity of GLUE [Montanari, 2005] but nevertheless we are convinced that GLUE is potentially useful for reducing the uncertainty of hydrological model parameters in conditions of data scarcity, when a formal statistical assessment would not be reliable.”
BackgroundSelf-citationLi, L., Xia, J., Xu, C.-Y., & Singh, V. P. (2010) · Journal of Hydrology · doi:10.1016/j.jhydrol.2010.06.044 · Cited by 184
“On the other hand, other studies point out that prediction limits estimated by GLUE can be very different from prediction limits estimated by the Bayesian methods (Montanari, 2005; Mantovan and Todini, 2006; Stedinger et al., 2008).”
Convergence of approaches toward reducing uncertainty in predictions in ungauged basins
Wagener, T., & Montanari, A. (2011) · Water Resources Research · doi:10.1029/2010wr009469 · Cited by 180
“The differences between the two basic strategies to estimate predictive uncertainty (and their scientific validity) have been discussed elsewhere [see Montanari, 2005; Mantovan and Todini, 2006; Beven et al. ,2 008;Stedinger et al., 2008], and we do not want to repeat this discussion here.”
BackgroundSelf-citationScale effects in conceptual hydrological modeling
Merz, R., Parajka, J., & Blöschl, G. (2009) · Water Resources Research · doi:10.1029/2009wr007872 · Cited by 166
“[7] Both the estimation of model parameters and the issues of calibration and verification efficiencies are confounded by problems of parameter uncertainty [ Montanari, 2005, 2007; Refsgaard et al., 2006; Go¨tzinger and Ba´rdossy, 2008; Freer et al., 1996].”
BackgroundLiu, Y., Freer, J., Beven, K., & Matgen, P. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.01.016 · Cited by 162
Her, Y., & Chaubey, I. (2015) · Hydrological Processes · doi:10.1002/hyp.10487 · Cited by 158
“…2008; Liu et al., 2009; Winsemius et al., 2009) and statistical informality of a generalized likelihood function (Freer et al., 1996; Christensen, 2003; Montanari, 2005, 2007; Mantovan and Todini, 2006; Zheng and Keller, 2007; Jia and Culver, 2008; Stedinger et al., 2008; Sadegh and Vrugt, 2013).”
BackgroundXiong, L., Wan, M., WEI, X., & O’Connor, K. M. (2009) · Hydrological Sciences Journal · doi:10.1623/hysj.54.5.852 · Cited by 157
“These problems have led to some indepth appraisal of the GLUE methodology (Montanari, 2005; Mantovan & Todini, 2006; Xiong & O’Connor, 2008; Stedinger et al., 2008).”
BackgroundMethodInfluentialNepal, S., Krause, P., Flügel, W.-A., Fink, M., & Fischer, C. (2012) · Hydrological Processes · doi:10.1002/hyp.9627 · Cited by 150
“It has been widely applied in hydrology and environmental modelling to estimate the uncertainty associated with model outputs and parameters estimates (Beven and Binley, 1992; Beven and Freer, 2001; Montanari, 2005).”
MethodMontanari, A., Shoemaker, C. A., & van de Giesen, N. (2009) · Water Resources Research · doi:10.1029/2009wr008471 · Cited by 141
“…has raised the very relevant question about the capability of probabilistic and nonprobabilistic methods to correctly infer the frequency properties of hydrological simulations and predictions [see, e.g., Beven, 2006b; Montanari, 2005, 2007; Mantovan and Todini, 2006; Beven et al., 2007, 2008].”
BackgroundSelf-citationA drought index accounting for snow
Staudinger, M., Stahl, K., & Seibert, J. (2014) · Water Resources Research · doi:10.1002/2013wr015143 · Cited by 140
Kim, U., & Kaluarachchi, J. J. (2008) · Journal of Hydrology · doi:10.1016/j.jhydrol.2008.08.016 · Cited by 148
Spatiotemporal topological kriging of runoff time series
Skøien, J. O., & Blöschl, G. (2007) · Water Resources Research · doi:10.1029/2006wr005760 · Cited by 132
“However, as Blo¨schl [2005] noted, there is significant uncertainty in the estimates, mainly due to a lack of representativeness of catchment data, rainfall data as well as identifiability problems of the runoff model parameters [ Montanari, 2005b ].”
BackgroundMethodBaldassarre, G. D., Castellarin, A., Montanari, A., & Brath, A. (2009) · Natural Hazards · doi:10.1007/s11069-009-9355-6 · Cited by 127
“Montanari 2005 ). These other sources of uncertainty were neglected by assuming that these affect the scenarios considered in a similar fashion and therefore that the results of our comparison are still suitable in the more general case.”
ResultSelf-citationFreni, G., Mannina, G., & Viviani, G. (2007) · Water Research · doi:10.1016/j.watres.2007.12.014 · Cited by 133
Challenges in communicating and using ensembles in operational flood forecasting
Demeritt, D., Nobert, S., Cloke, H., & Pappenberger, F. (2010) · Meteorological Applications · doi:10.1002/met.194 · Cited by 132
“As in the peer review literature itself (e.g. Golding, 2000; Montanari, 2005; Jaun et al., 2008; Cloke and Pappenberger, 2009) there was a range of opinion among operational forecasters about how much confidence could be placed in the number and spread of ensemble members as a measure of the total…”
BackgroundOptimization of a similarity measure for estimating ungauged streamflow
Reichl, J. P. C., Western, A. W., McIntyre, N. R., & Chiew, F. H. S. (2009) · Water Resources Research · doi:10.1029/2008wr007248 · Cited by 124
“It is important to note, especially in light of recent debate on the subject [e.g., Beven, 2006; Montanari, 2005; Stedinger et al., 2008] that while this paper refers to ‘‘likelihoods’’ (described in this section and sections 3.3.1 and 3.3.2), we do not derive formal likelihood measures.”
BackgroundInfluentialSellami, H., Jeunesse, I. L., Benabdallah, S., & Vanclooster, M. (2013) · Hydrological Sciences Journal · doi:10.1080/02626667.2013.837222 · Cited by 123
“However, it is common for the 95% confidence interval to be used for uncertainty analysis, despite the fact that the interval may not be able to capture 95% of the observations and, hence, not represent all the uncertainty (Montanari 2005, Beven 2006).”
BackgroundAssessing the impact of different sources of topographic data on 1-D hydraulic modelling of floods
Ali, A. M., SOLOMATINE, D. P., & Baldassarre, G. D. (2015) · Hydrology and earth system sciences · doi:10.5194/hess-19-631-2015 · Cited by 137
Ding, W., Zhang, C., Peng, Y., Zeng, R., Zhou, H., & Cai, X. (2015) · Water Resources Research · doi:10.1002/2015wr017127 · Cited by 119
“ANALYTICAL FRAMEWORK FOR FLOOD WATER CONSERVATION 4705 I25 I 21e: (8) In equation (8), the forecast error (e) includes the input uncertainty, model structure, and parameter errors [Montanari, 2005; Nasseri et al., 2014].”
BackgroundA novel approach to parameter uncertainty analysis of hydrological models using neural networks
Shrestha, D. L., Kayastha, N., & SOLOMATINE, D. P. (2009) · Hydrology and earth system sciences · doi:10.5194/hess-13-1235-2009 · Cited by 122
“…of the popular methods for analysing parameter uncertainty in hydrological modelling and has been widely used over the past ten years to analyse and estimate predictive uncertainty, particularly in hydrological applications (see, e.g., Freer et al., 1996; Beven and Freer, 2001; Montanari, 2005).”
MethodResultDemirel, M. C., Booij, M. J., & Hoekstra, A. Y. (2013) · Water Resources Research · doi:10.1002/wrcr.20294 · Cited by 113
“[7] The drawbacks and advantages of the GLUE method have been enormously discussed in the hydrology literature [Beven and Young, 2003; Beven et al., 2007, 2008; Li et al., 2010; Mantovan and Todini, 2006; Montanari, 2005; Stedinger et al., 2008].”
MethodDotto, C. B. S., Kleidorfer, M., Deletić, A., Rauch, W. A., McCarthy, D., & Fletcher, T. D. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.03.013 · Cited by 118
Montanari, A., & Toth, E. (2007) · Water Resources Research · doi:10.1029/2006wr005184 · Cited by 106
Self-citationLiu, P., Li, L., Chen, G., & Rheinheimer, D. E. (2014) · Journal of Hydrology · doi:10.1016/j.jhydrol.2014.04.012 · Cited by 101
A hydraulic study on the applicability of flood rating curves
Baldassarre, G. D., & Claps, P. (2010) · Hydrology research · doi:10.2166/nh.2010.098 · Cited by 110
“Thirdly, the computed likelihoods are rescaled to produce a cumulative sum of 1, and then uncertainty bounds and the median simulation are derived by following the standard GLUE methodology (e.g. Montanari 2005).”
MethodEstimating uncertainty of streamflow simulation using Bayesian neural networks
Zhang, X., Liang, F., Srinivasan, R., & Liew, M. W. V. (2009) · Water Resources Research · doi:10.1029/2008wr007030 · Cited by 101
“Through incorporating variable model structure and prior knowledge into account, BNN-d obtained better estimation of 95% uncertainty interval, but it is worth noting that the probabilistic predictions at other coverage levels are also of interest (Krzysztofowicz, 2001; Montanari, 2005; Laio et al., 2007b).”
BackgroundMethodSrivastava, P. K., Han, D., Rico‐Ramirez, M. A., & Islam, T. (2013) · Hydrological Processes · doi:10.1002/hyp.9946 · Cited by 100
“The uncertainty in model calibration and validation has been estimated, and the plots are derived with NSE as a likelihood measure under the natural flow conditions.”
MethodInfluentialSellami, H., Jeunesse, I. L., Benabdallah, S., Baghdadi, N. N., & Vanclooster, M. (2014) · Hydrology and earth system sciences · doi:10.5194/hess-18-2393-2014 · Cited by 103
Perturbation of convection-permitting NWP forecasts for flash-flood ensemble forecasting
Vincendon, B., Ducrocq, V., Nuissier, O., & Vié, B. (2011) · Natural hazards and earth system sciences · doi:10.5194/nhess-11-1529-2011 · Cited by 97
Hassan, A. E., Bekhit, H. M., & Chapman, J. (2008) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2008.11.002 · Cited by 100
Begou, J. C., Jomaa, S., Benabdallah, S., Bazié, P., Afouda, A. A., & Rode, M. (2016) · Water · doi:10.3390/w8050178 · Cited by 100
Mirzaei, M., Huang, Y. F., El‐Shafie, A., & Shatirah, A. (2015) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-014-1000-6 · Cited by 96
Bridging the gap between GLUE and formal statistical approaches: approximate Bayesian computation
Sadegh, M., & Vrugt, J. A. (2013) · Hydrology and earth system sciences · doi:10.5194/hess-17-4831-2013 · Cited by 98
“…oppose incorrect usage of statistics in favor of coherent probabilistic approaches (Gupta et al., 1998; Beven and Young, 2003; Gupta et al., 2003; Christensen, 2004; Montanari, 2005; Mantovan and Todini, 2006; Stedinger et al., 2008; Beven et al., 2008; Beven, 2009; Vrugt et al., 2008b, c).”
BackgroundReal-time probabilistic forecasting of flood stages
Chen, S., & Yu, P. (2007) · Journal of Hydrology · doi:10.1016/j.jhydrol.2007.04.008 · Cited by 97
Xiong, L., & O’Connor, K. M. (2007) · Journal of Hydrology · doi:10.1016/j.jhydrol.2007.10.029 · Cited by 94
Efficient prediction uncertainty approximation in the calibration of environmental simulation models
Tolson, B. A., & Shoemaker, C. A. (2008) · Water Resources Research · doi:10.1029/2007wr005869 · Cited by 74
Sun, W. C., Ishidaira, H., & Bastola, S. (2010) · Hydrology and earth system sciences · doi:10.5194/hess-14-2011-2010 · Cited by 87
“This can partially be explained by the fact that judgments made by GLUE are only based on whether the likelihood value is good or not.”
BackgroundResultInfluentialMontanari, A., & Baldassarre, G. D. (2012) · Advances in Water Resources · doi:10.1016/j.advwatres.2012.09.007 · Cited by 82
Self-citationMcMillan, H., & Clark, M. (2009) · Water Resources Research · doi:10.1029/2008wr007288 · Cited by 84
Estimating the Uncertainty of Hydrological Predictions through Data-Driven Resampling Techniques
Sikorska‐Senoner, A. E., Montanari, A., & Koutsoyiannis, D. (2014) · Journal of Hydrologic Engineering · doi:10.1061/(asce)he.1943-5584.0000926 · Cited by 84
“Synthetic precipitation data of 50 years on five rain gauges located within the 372 catchment were generated using the generalized multivariate Neyman-Scott rectangular 373 pulses model (Cowpertwait 1995) which was previously calibrated on the available recorded 374 data from two years (1972 and 1973; (Montanari 2005)…”
BackgroundMethodInfluentialSelf-citationUncertainty assessment of a stochastic groundwater flow model using GLUE analysis
Hassan, A. E., Bekhit, H. M., & Chapman, J. (2008) · Journal of Hydrology · doi:10.1016/j.jhydrol.2008.08.017 · Cited by 82
“A variety of methods exist for achieving this purpose, among which are the generalized likelihood uncertainty estimation (GLUE) first developed by Beven and Binley (1992) and subsequently used in many studies (Franks and Beven, 1997; Freer et al., 1996; Lamb et al., 1998; Aronica et al., 1998; Cameron et al., 1999; Hu…”
MethodKoutsoyiannis, D., Montanari, A., Lins, H. F., & Cohn, T. A. (2009) · Hydrological Sciences Journal · doi:10.1623/hysj.54.2.394 · Cited by 78
“This is established practice within the hydrological community and has been recently re-confirmed in discussions related to the generalized likelihood uncertainty estimation (e.g. Montanari, 2005; Stedinger et al., 2008).”
BackgroundSelf-citationStochastic watershed models for hydrologic risk management
Vogel, R. M. (2017) · Water Security · doi:10.1016/j.wasec.2017.06.001 · Cited by 82
Multi‐method global sensitivity analysis (MMGSA) for modelling floodplain hydrological processes
Cloke, H., Pappenberger, F., & Renaud, J. ‐. (2007) · Hydrological Processes · doi:10.1002/hyp.6734 · Cited by 88
“The implications of using informal likelihood measures and retaining only behavioural simulations is discussed in length by Montanari (2005) and Beven et al. (2007).”
Moretti, G., & Montanari, A. (2008) · Hydrology and earth system sciences · doi:10.5194/hess-12-1141-2008 · Cited by 79
“This type of uncertainty assessment is subjective (for an extensive discussion see Montanari, 2005, 2007,and Beven, 2006).”
BackgroundSelf-citationZheng, Y., & Keller, A. A. (2007) · Water Resources Research · doi:10.1029/2006wr005345 · Cited by 75
“We have seen only one study [Muleta and Nicklow, 2005] using GLUE for a watershed water quality model (SWAT).”
MethodEvaluation of statistical models for forecast errors from the HBV model
Engeland, K., Renard, B., Steinsland, I., & Kolberg, S. A. (2010) · Journal of Hydrology · doi:10.1016/j.jhydrol.2010.01.018 · Cited by 68
A multimodel data assimilation framework via the ensemble Kalman filter
Xue, L., & Zhang, D. (2014) · Water Resources Research · doi:10.1002/2013wr014525 · Cited by 66
“…1999] and its Maximum Likelihood variant (MLBMA) [ Neuman , 2003] provide a coherent mechanism to account for model uncertainty [ Ye et al ., 2004, 2005, 2008, 2010; Poeter and Anderson , 2005; Duan et al ., 2007; Rojas et al ., 2008; Zhang et al ., 2009; Li and Tsai , 2009; Reeves et al ., 2010;…”
BackgroundExploring the potential of SRTM topographic data for flood inundation modelling under uncertainty
Yan, K., Baldassarre, G. D., & SOLOMATINE, D. P. (2013) · Journal of Hydroinformatics · doi:10.2166/hydro.2013.137 · Cited by 71
Bayesian calibration of a flood inundation model using spatial data
Hall, J. W., Manning, L. J., & Hankin, R. K. S. (2011) · Water Resources Research · doi:10.1029/2009wr008541 · Cited by 68
Sun, N., Hong, B., & Hall, M. (2013) · Hydrological Processes · doi:10.1002/hyp.9869 · Cited by 70
“…the Hydrologic Simulation Program-Fortran (HSPF) model (Jia and Culver, 2008), the Soil Moisture Accounting and Routing (SMAR) model (Xiong and O’Connor, 2008), the MOUSE Runoff Model (MOUSE) model (Thorndahl et al., 2008), and the HYdrologic MODel (HYMOD) rainfall–runoff model (Montanari, 2005).”
MethodUncertainty of Hydrological Predictions
Montanari, A. (2011) · Elsevier eBooks · doi:10.1016/b978-0-444-53199-5.00045-2 · Cited by 72
Self-citationParameter uncertainty analysis in watershed total phosphorus modeling using the GLUE methodology
Gong, Y., Shen, Z., Hong, Q., Liu, R., & Liao, Q. (2011) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2011.05.015 · Cited by 73
Uncertainty assessment of an integrated urban drainage model
Freni, G., Mannina, G., & Viviani, G. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.04.037 · Cited by 71
Uncertainty in design flood profiles derived by hydraulic modelling
Brandimarte, L., & Baldassarre, G. D. (2012) · Hydrology research · doi:10.2166/nh.2011.086 · Cited by 68
Zhang, X., Liang, F., Yu, B., & Zong, Z. (2011) · Journal of Hydrology · doi:10.1016/j.jhydrol.2011.09.002 · Cited by 63
Sexton, J., Everingham, Y., & Inman-Bamber, G. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.05.014 · Cited by 61
Uncertainty in high and low flows due to model structure and parameter errors
Tian, Y., Booij, M. J., & Xu, Y. (2013) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-013-0751-9 · Cited by 61
Lin, K., Zhang, Q., & Chen, X. (2010) · Journal of Hydrology · doi:10.1016/j.jhydrol.2010.09.012 · Cited by 58
Mustafa, S. M. T., Nossent, J., Ghysels, G., & Huysmans, M. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104654 · Cited by 61
“However, some researchers have criticized GLUE because it is not a 149 formal Bayesian method and may result in statistically incoherent and unreliable parameters 150 and predictive distributions (Mantovan & Todini, 2006; Montanari, 2005; Stedinger et al., 151 2008).”
MethodFloods in a Changing Climate: Inundation Modelling
Baldassarre, G. D. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9781139088411 · Cited by 75
Influence of errors in radar rainfall estimates on hydrological modeling prediction uncertainty
Borga, M., Esposti, S. D., & Norbiato, D. (2006) · Water Resources Research · doi:10.1029/2005wr004559 · Cited by 55
“: ERRORS IN RADAR RAINFALL ESTIMATES 5 of 14 W08409 fixed proportion of the simulations can be retained according to their ranked likelihoods so that the best n solutions are considered acceptable [Montanari, 2005].”
BackgroundBluecat: A Local Uncertainty Estimator for Deterministic Simulations and Predictions
Koutsoyiannis, D., & Montanari, A. (2022) · Water Resources Research · doi:10.1029/2021wr031215 · Cited by 55
“Water Resources Research KOUTSOYIANNIS AND MONTANARI 10.1029/2021WR031215 11 of 20 We apply to both case studies the rainfall-runoff model HyMod (Boyle, 2000; Montanari, 2005) with 5 parameters.”
BackgroundMethodSelf-citationA Bayesian-based integrated approach for identifying groundwater contamination sources
Yan, X., Dong, W., An, Y., & Lu, W. (2019) · Journal of Hydrology · doi:10.1016/j.jhydrol.2019.124160 · Cited by 54
Bayesian analysis for uncertainty estimation of a canopy transpiration model
Samanta, S., Mackay, D. S., Clayton, M. K., Kruger, E. L., & Ewers, B. E. (2007) · Water Resources Research · doi:10.1029/2006wr005028 · Cited by 52
Papacharalampous, G., Tyralis, H., Koutsoyiannis, D., & Montanari, A. (2019) · Advances in Water Resources · doi:10.1016/j.advwatres.2019.103470 · Cited by 55
“It has been discussed, for example, in Montanari (2005), Mantovan and Todini (2006), Stedinger et al. (2008), Vrugt et al. (2009b), and Sadegh and Vrugt (2013); see also the related comments in Todini (2007).”
BackgroundSelf-citationA Stochastic Method for Crop Models: Including Uncertainty in a Sugarcane Model
Marin, F. R., Jones, J. W., & Boote, K. J. (2017) · Agronomy Journal · doi:10.2134/agronj2016.02.0103 · Cited by 51
Shafii, M., Tolson, B. A., & Matott, L. S. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2015.01.051 · Cited by 46
Clark, M., Kavetski, D., & Fenicia, F. (2012) · Water Resources Research · doi:10.1029/2012wr012547 · Cited by 44
Dai, Q., Han, D., Rico‐Ramirez, M. A., & Srivastava, P. K. (2014) · Journal of Hydrology · doi:10.1016/j.jhydrol.2014.01.016 · Cited by 42
On the use of global sensitivity analysis for the numerical analysis of permeable pavements
Brunetti, G., Šimůnek, J., Turco, M., & Piro, P. (2018) · Urban Water Journal · doi:10.1080/1573062x.2018.1439975 · Cited by 43
“The GLUE analysis has been used extensively in the literature for the uncertainty assessment of various hydrological models (Beven and Freer 2001; Montanari 2005).”
Rojas, R., Feyen, L., & Dassargues, A. (2009) · Hydrological Processes · doi:10.1002/hyp.7231 · Cited by 43
Advances in the study of uncertainty quantification of large-scale hydrological modeling system
Song, X., Zhan, C., Kong, F., & Xia, J. (2011) · Journal of Geographical Sciences · doi:10.1007/s11442-011-0881-2 · Cited by 44
Jia, Y., & Culver, T. B. (2008) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)0733-9496(2008)134:2(97 · Cited by 40
Zhu, G., Li, X., Ma, J., Wang, Y., Liu, S., Huang, C., Zhang, K., & Hu, X. (2018) · Advances in Water Resources · doi:10.1016/j.advwatres.2018.02.007 · Cited by 39
Atencia, A., Mediero, L., Llasat, M. C., & Garrote, L. (2011) · Hydrology and earth system sciences · doi:10.5194/hess-15-3809-2011 · Cited by 38
Probabilistic calibration of a distributed hydrological model for flood forecasting
Mediero, L., Garrote, L., & Martin-Carrasco, F. J. (2011) · Hydrological Sciences Journal · doi:10.1080/02626667.2011.610322 · Cited by 41
“…the proportion of time intervals in which the observed flow falls within the confidence interval over the total number of time intervals of the event (Montanari 2005): CR (α) = N∑ k=1 I [yk] N (23) where I[yk] is 1 if the observed flow value falls within the confidence interval at time t, and is 0…”
BackgroundMethodProbabilistic development of shear strength model for reinforced concrete squat walls
Ning, C., & Li, B. (2016) · Earthquake Engineering & Structural Dynamics · doi:10.1002/eqe.2834 · Cited by 37
“According to the discussion by Montanari [26], the determination of the threshold value has considerable influence on the resulting parameter uncertainty and the model simulation results.”
BackgroundFreer, J. E., Beven, K. J., Neal, J. C., Schumann, G. J.-P., Hall, J. W., & Bates, P. (2013) · Cambridge University Press eBooks · doi:10.1017/cbo9781139047562.008 · Cited by 37
Uncertainty-based multi-criteria calibration of rainfall-runoff models: a comparative study
Shafii, M., Tolson, B. A., & Matott, L. S. (2014) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-014-0855-x · Cited by 41
“The use of GLUE without a formal likelihood function has been the subject of much debate (e.g., Beven et al. 2008; Mantovan and Todini 2006; Montanari 2005; Thiemann et al. 2001).”
MethodSellami, H., Benabdallah, S., Jeunesse, I. L., & Vanclooster, M. (2015) · Hydrological Sciences Journal · doi:10.1080/02626667.2015.1040801 · Cited by 39
“Indeed, the selection of NS as a threshold value to discriminate between ‘behavioural’ and ‘non-behavioural’ SWAT simulations in the GLUE method is very subjective as it influences the uncertainty prediction (Montanari 2005, Mantovan and Todini 2006).”
MethodJacquin, A. P., & Shamseldin, A. Y. (2007) · Water Resources Research · doi:10.1029/2006wr005072 · Cited by 35
“[15] As frequently noted in the literature, the prediction quantiles obtained with the GLUE methodology are dependent on a series of subjective choices [Beven et al., 2000; Beven and Freer, 2001; Beven, 2006; Montanari, 2005 ].”
BackgroundMethodInfluential
Metadata
- Authors
- Montanari, A.
- Affiliations
- University of Bologna (IT)
- Corresponding author
- Montanari, Alberto
- ORCID
- Montanari, Alberto
- Venue
- Water Resources Research
- Publisher
- American Geophysical Union (AGU)
- Published
- 2005-08
- Volume
- 41
- Issue
- 8
- ISSN
- 0043-1397 (ISSN-L)
- 1944-7973
- OpenAlex
- W2099329408
- Cited by
- 231
- References
- 45
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 5 · 2023: 5 · 2022: 10 · 2021: 19 · 2020: 4 · 2019: 11 · 2018: 9 · 2017: 9 · 2016: 11 · 2015: 10 · 2014: 22 · 2013: 16 · 2012: 18
- FWCI
- 14.08
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Water Resources Research (publishedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- Topics
- Physical Sciences › Environmental Science › Water Science and Technology › Hydrology and Watershed Management Studies
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Environmental Engineering › Groundwater flow and contamination studies
- Keywords
- generalized likelihood uncertainty estimation, adhesives, uncertainty analysis, rainfall-runoff modeling, hydrological model uncertainty, limits of prediction, confidence level, large sample behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.