Large sample behaviors of the generalized likelihood uncertainty estimation (GLUE) in assessing the uncertainty of rainfall‐runoff simulations
Open accessThe DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index.
Abstract
Several methods have been recently proposed for quantifying the uncertainty of hydrological models. These techniques are based upon different hypotheses, are diverse in nature, and produce outputs that can significantly differ in some cases. One of the favored methods for uncertainty assessment in rainfall‐runoff modeling is the generalized likelihood uncertainty estimation (GLUE). However, some fundamental questions related to its application remain unresolved. One such question is that GLUE relies on some explicit and implicit assumptions, and it is not fully clear how these may affect the uncertainty estimation when referring to large samples of data. The purpose of this study is to address this issue by assessing how GLUE performs in detecting uncertainty in the simulation of long series of synthetic river flows. The study aims to (1) discuss the hypotheses underlying GLUE and derive indications about their effects on the uncertainty estimation, and (2) compare the GLUE prediction limits with a large sample of data that is to be simulated in the presence of known sources of uncertainty. The analysis shows that the prediction limits provided by GLUE do not necessarily include a percentage close to their confidence level of the observed data. In fact, in all the experiments, GLUE underestimates the total uncertainty of the simulation provided by the hydrological model.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Similarly studies such as Montanari (2005) have shown that GLUE can underestimate the posterior output uncertainty.”
From A theoretical and real world evaluation of two Bayesian techniques for the calibration of variety parameters in a sugarcane crop model (2016) · cites this paper as Montanari (2005)
Citations
Cites
River flow forecasting through conceptual models part I — A discussion of principles
Nash, J. E., & Sutcliffe, J. V. (1970) · Journal of Hydrology · doi:10.1016/0022-1694(70)90255-6 · Cited by 23,674 · cited 2×
“As examples, Cameron et al. [1999] used the Nash and Sutcliffe [1970] efficiency to evaluate the likelihood of the simulation of a continuous hydrograph, and Beven and Freer [2001] present a list of likelihood measures that can be used in rainfall-runoff modeling.”
MethodLocally Weighted Regression: An Approach to Regression Analysis by Local Fitting
Cleveland, W. S., & Devlin, S. J. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478639 · Cited by 5,742 · cited 1×
“Smoothing was conducted by using the loess smoother that operates through a local regression model [Cleveland and Devlin, 1988].”
MethodThe future of distributed models: model calibration and uncertainty prediction
Beven, K., & Binley, A. (1992) · Hydrological Processes · doi:10.1002/hyp.3360060305 · Cited by 4,527 · cited 1×
“[3] One of the most known and used uncertainty assessment methods in hydrological modeling is the generalized likelihood uncertainty estimation (GLUE), proposed by Beven and Binley [1992] , that is briefly described in section 2 of the present article.”
MethodGuidelines for predicting crop water requirements
Doorenbos, J. (1977) · Medical Entomology and Zoology · Cited by 3,331
Effective and efficient global optimization for conceptual rainfall‐runoff models
Duan, Q., Sorooshian, S., & Gupta, V. (1992) · Water Resources Research · doi:10.1029/91wr02985 · Cited by 3,310 · cited 2×
“[31] By using a 30-year-long record of uncorrupted synthetic data (years from 1 to 30 of the synthetic records), HYMOD was first calibrated in an optimality context by minimizing the squared difference between Qobs(t) and Qsim(t) through a generic algorithm [Duan et al., 1992].”
MethodStatistics for Long-Memory Processes.
Ray, W. D., & Beran, J. (1996) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983481 · Cited by 2,686 · cited 3×
“Thus model calibration can be carried out by minimizing [Beran, 1994] W qð Þ ¼ XN=2 j¼1 log fM lj; q þ fe lj;f þ J lj fM lj; q þ fe lj;f ( ) : ð10Þ In the above expression, the spectral density function of the model, fM(lj, q), has been estimated by computing the smoothed periodogram of a long…”
MethodA manifesto for the equifinality thesis
Beven, K. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.007 · Cited by 2,581 · cited 10×
“There is an implicit assumption in GLUE that the errors in the prediction will be similar to those in the evaluation period [ Beven, 2005 ].”
BackgroundResultInfluentialBeven, K., & Freer, J. (2001) · Journal of Hydrology · doi:10.1016/s0022-1694(01)00421-8 · Cited by 2,189 · cited 1×
“[7] It is important to remark that even in the ideal context this study refers to, one should not expect that a specified proportion of synthetic observations will lie within the GLUE prediction limits [Beven and Freer, 2001].”
BackgroundStatistics for Long-Memory Processes.
Percival, D. B., & Beran, J. (1996) · Journal of the American Statistical Association · doi:10.2307/2291761 · Cited by 1,480
Vrugt, J. A., Gupta, H. V., Bouten, W., & Sorooshian, S. (2003) · Water Resources Research · doi:10.1029/2002wr001642 · Cited by 1,196
Prophecy, reality and uncertainty in distributed hydrological modelling
Beven, K. (1993) · Advances in Water Resources · doi:10.1016/0309-1708(93)90028-e · Cited by 1,169 · cited 1×
“The acceptance of the existence of multiple behavioral parameter sets has been called equifinality [Beven, 1993].”
BackgroundTheory and applications of long-range dependence
Doukhan, P., Oppenheim, G., & Taqqu, M. S. (2003) · Birkhäuser-Verlag eBooks · Cited by 1,002
Freer, J., Beven, K., & Ambroise, B. (1996) · Water Resources Research · doi:10.1029/95wr03723 · Cited by 763 · cited 2×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
MethodThe form of the instantaneous unit hydrograph
Nash, J. E. (1957) · NERC Open Research Archive (Natural Environment Research Council) · Cited by 691
How far can we go in distributed hydrological modelling?
Beven*, K. (2001) · Hydrology and Earth System Sciences · doi:10.5194/hess-5-1-2001 · Cited by 691
The probability-distributed principle and runoff production at point and basin scales
MOORE, R. J. (1985) · Hydrological Sciences Journal · doi:10.1080/02626668509490989 · Cited by 590 · cited 1×
“HYMOD consists of a relatively simple rainfall excess model, described in detail by Moore [1985] , that is connected with two series of linear reservoirs: three identical reservoirs for the quick response and a single reservoir for the slow response.”
MethodA framework for development and application of hydrological models
Wagener, T., Boyle, D. P., Lees, M. J., Wheater, H. S., Gupta, H. V., & Sorooshian, S. (2001) · Hydrology and Earth System Sciences · doi:10.5194/hess-5-13-2001 · Cited by 560 · cited 1×
“3.5. Brief Description of the HYMOD Model [29] HYMOD is a five-parameter conceptual rainfallrunoff model that was introduced by Boyle [2000] and recently used by Wagener et al. [2001] and Vrugt et al. [2003].”
MethodInfluentialEstimation and information in stationary time series
Whittle, P. (1953) · Arkiv för Matematik · doi:10.1007/bf02590998 · Cited by 558 · cited 3×
“Besides the Nash efficiency E, the mean absolute relative error MRe, the sum of squared errors SSq, and the likelihood function proposed by Whittle [1953] , W(q), were considered.”
BackgroundMethodInfluentialPappenberger, F., Beven, K., Horritt, M., & Blazkova, S. (2005) · Journal of Hydrology · doi:10.1016/j.jhydrol.2004.06.036 · Cited by 516 · cited 1×
“[48] The likelihood measure proposed by Whittle [1953] for a stationary time series is computed on the spectral density [see, e.”
MethodAronica, G., Bates, P. D., & Horritt, M. S. (2002) · Hydrological Processes · doi:10.1002/hyp.398 · Cited by 429 · cited 1×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
MethodTowards a coherent philosophy for modelling the environment
Beven, K. (2002) · Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2002.0986 · Cited by 383 · cited 2×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
BackgroundMethodBeven, K. (2002) · Hydrological Processes · doi:10.1002/hyp.343 · Cited by 358 · cited 2×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
BackgroundMethodZhao, R. J., Liu, X. R., & Singh, V. P. (1980) · Cited by 339
Giraitis, L., & Surgailis, D. (1990) · Probability Theory and Related Fields · doi:10.1007/bf01207515 · Cited by 339 · cited 2×
“[51] Under the assumption of zero mean and i.i.d. e(t), Whittle’s likelihood provides asymptotically consistent and normally distributed estimates in the case of non-Gaussian and linear models [Giraitis and Surgailis, 1990].”
BackgroundMontanari, A., Rosso, R., & Taqqu, M. S. (1997) · Water Resources Research · doi:10.1029/97wr00043 · Cited by 289 · cited 1×
“More details on FARIMA models and the simulation procedure herein applied are given by Montanari et al. [1997] .”
MethodSelf-citationA stochastic approach for assessing the uncertainty of rainfall‐runoff simulations
Montanari, A., & Brath, A. (2004) · Water Resources Research · doi:10.1029/2003wr002540 · Cited by 261
Self-citationConfronting input uncertainty in environmental modelling
Kavetski, D., Franks, S. W., & Kuczera, G. (2003) · Water Science and Application · doi:10.1029/ws006p0049 · Cited by 224
Flood frequency estimation by continuous simulation for a gauged upland catchment (with uncertainty)
Cameron, D. S., Beven, K. J., Tawn, J., Blazkova, S., & Naden, P. (1999) · Journal of Hydrology · doi:10.1016/s0022-1694(99)00057-8 · Cited by 195 · cited 3×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
MethodA seasonal fractional ARIMA Model applied to the Nile River monthly flows at Aswan
Montanari, A., Rosso, R., & Taqqu, M. S. (2000) · Water Resources Research · doi:10.1029/2000wr900012 · Cited by 184 · cited 2×
“[48] The likelihood measure proposed by Whittle [1953] for a stationary time series is computed on the spectral density [see, e.g., Beran, 1994; Montanari et al., 2000; 6 of 13 W08406 MONTANARI: ESTIMATING UNCERTAINTY BY USING GLUE W08406 Chouduri et al., 2004].”
MethodSelf-citationBayesian system for probabilistic river stage forecasting
Krzysztofowicz, R. (2002) · Journal of Hydrology · doi:10.1016/s0022-1694(02)00106-3 · Cited by 165
A generalized spatial-temporal model of rainfall based on a clustered point process
Cowpertwait, P. S. P. (1995) · Proceedings of the Royal Society of London Series A Mathematical and Physical Sciences · doi:10.1098/rspa.1995.0077 · Cited by 153 · cited 1×
“[19] The synthetic rainfall data were generated by using the generalized multivariate Neyman-Scott rectangular pulses model [Cowpertwait, 1995].”
MethodFranchini, M. (1996) · Hydrological Sciences Journal · doi:10.1080/02626669609491476 · Cited by 151 · cited 2×
“3.3. Generation of Synthetic River Flow Data [24] Synthetic river flow data were obtained by using the previously generated synthetic rainfall and temperature records as input to the lumped version of the rainfallrunoff model ADM (a distributed model [ Franchini, 1996 ]).”
MethodInfluentialBlazkova, S. D., & Beven, K. J. (2004) · Journal of Hydrology · doi:10.1016/j.jhydrol.2003.12.025 · Cited by 145 · cited 3×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
MethodResultFlood frequency estimation by continuous simulation under climate change (with uncertainty)
Cameron, D., Beven, K. J., & Naden, P. S. (2000) · Hydrology and earth system sciences · doi:10.5194/hess-4-393-2000 · Cited by 139 · cited 2×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
MethodBlazkova, S. D., & Beven, K. J. (2002) · Water Resources Research · doi:10.1029/2001wr000500 · Cited by 133 · cited 3×
“First, it has been already used in numerous real-world applications [see, e.g., Freer et al., 1996; Cameron et al., 1999, 2000; Blazkova and Beven, 2002, 2004; Aronica et al., 2002].”
MethodResultHydrologic uncertainty processor for probabilistic stage transition forecasting
Krzysztofowicz, R., & Maranzano, C. J. (2004) · Journal of Hydrology · doi:10.1016/j.jhydrol.2004.01.003 · Cited by 81
Freer, J. E., Beven, K. J., & Peters, N. E. (2003) · Water science and application · doi:10.1029/ws006p0069 · Cited by 79
Towards a coherent philosophy for environmental modelling.
Beven, K. J. (2002) · Lancaster EPrints (Lancaster University) · Cited by 51
Contiguity of the Whittle measure for a Gaussian time series
Choudhuri, N. (2004) · Biometrika · doi:10.1093/biomet/91.1.211 · Cited by 43 · cited 1×
“W08406 MONTANARI: ESTIMATING UNCERTAINTY BY USING GLUE W08406 Chouduri et al., 2004 ].”
BackgroundBayesian system for probabilistic stage transition forecasting
Krzysztofowicz, R., & Maranzano, C. J. (2004) · Journal of Hydrology · doi:10.1016/j.jhydrol.2004.02.013 · Cited by 33 · cited 1×
“For instance, Beven [2002a, 2005], Blazkova and Beven [2002, 2004], Krzysztofowicz [2002] , Krzysztofowicz and Maranzano [2004], Vrugt et al. [2003], Pappenberger et al. [2005], and Montanari and Brath [2004a, 2004b] are among the most recent studies.”
ResultThe Generation of Multiple Independent Sequences of Pseudorandom Numbers
Maclaren, N. M. (1989) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2348065 · Cited by 25 · cited 2×
“The parameters were generated by using a variant of the multiplicative congruential algorithm known as the Wichmann-Hill algorithm [Maclaren, 1989] and were allowed to vary all simultaneously from set to set.”
MethodStatistics for the environment
Thomas, R. (1995) · Health & Place · doi:10.1016/1353-8292(95)90051-9 · Cited by 12
Brath, A., Montanari, A., & Toth, E. (2004) · Cited by 3
Self-citationAssessing the uncertainty of rainfall-runoff simulations through a meta-Gaussian approach
Montanari, A., & Brath, A. (2004) · Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) · Cited by 2
Self-citation
Research works printed but not linked: 1
- Boyle D. P.(2000) Multicriteria calibration of hydrological models Ph.D. dissertation Dep. of Hydrol. and Water Resour. Univ. of Ariz. Tucson.
Cited by
A decade of Predictions in Ungauged Basins (PUB)—a review
Hrachowitz, M., Savenije, H. H. G., Blöschl, G., McDonnell, J. J., Sivapalan, M., Pomeroy, J. W., Arheimer, B., Blume, T., Clark, M. P., Ehret, U., et al. (2013) · Hydrological Sciences Journal · doi:10.1080/02626667.2013.803183 · Cited by 1,295
“Notwithstanding the significant advances in model uncertainty assessment discussed above, Wagener and Montanari (2011) pointed out that the focus needs to shift from reducing model uncertainty towards reducing the uncertainty in our understanding of how catchments function under given environmental boundary conditions…”
BackgroundMethodInfluentialSelf-citationEnsemble flood forecasting: A review
Cloke, H. L., & Pappenberger, F. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.06.005 · Cited by 1,139
Fatichi, S., Vivoni, E. R., Ogden, F. L., Ivanov, V. Y., Mirus, B., Gochis, D., Downer, C. W., Camporese, M., Davison, J. H., Ebel, B., et al. (2016) · Journal of Hydrology · doi:10.1016/j.jhydrol.2016.03.026 · Cited by 671
“However, applications of these approaches have been mostly carried out using coarse, conceptual models applied to watersheds (Beven and Freer, 2001; Montanari, 2005; Vrugt et al., 2005) or groundwater hydrology models (e.g., Hill and Tiedeman, 2007).”
Benchmarking observational uncertainties for hydrology: rainfall, river discharge and water quality
McMillan, H., Krueger, T., & Freer, J. (2012) · Hydrological Processes · doi:10.1002/hyp.9384 · Cited by 522
“…discussion and counter discussion about the merits for making strong and weak judgements about the nature of data, parameter and structural errors (Montanari, 2005; Mantovan and Todini, 2006; Beven et al., 2007; Mantovan et al., 2007; Beven et al., 2008; Stedinger et al., 2008; Beven, 2009; Vrugt…”
BackgroundGlobal‐scale regionalization of hydrologic model parameters
Beck, H. E., van Dijk, A. I. J. M., de Roo, A., Miralles, D. G., McVicar, T. R., Schellekens, J., & Bruijnzeel, L. A. (2016) · Water Resources Research · doi:10.1002/2015wr018247 · Cited by 511
“…approach [Beven and Binley, 1992], in that they lack a formal statistical foundation and involve several subjective choices (notably the calibration score threshold, the choice of 10 most similar donors, and the similarity criterion) [McIntyre et al., 2005; Montanari, 2005; Winsemius et al., 2009].”
Background
Metadata
- Authors
- Montanari, A.
- Affiliations
- University of Bologna (IT)
- Corresponding author
- Montanari, Alberto
- ORCID
- Montanari, Alberto
- Venue
- Water Resources Research
- Publisher
- American Geophysical Union (AGU)
- Published
- 2005-08
- Volume
- 41
- Issue
- 8
- ISSN
- 0043-1397 (ISSN-L)
- 1944-7973
- OpenAlex
- W2099329408
- Cited by
- 231
- References
- 45
- Citations by year
- 2026: 2 · 2025: 7 · 2024: 5 · 2023: 5 · 2022: 10 · 2021: 19 · 2020: 4 · 2019: 11 · 2018: 9 · 2017: 9 · 2016: 11 · 2015: 10 · 2014: 22 · 2013: 16 · 2012: 18
- FWCI
- 14.08
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Water Resources Research (publishedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- Topics
- Physical Sciences › Environmental Science › Water Science and Technology › Hydrology and Watershed Management Studies
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Environmental Engineering › Groundwater flow and contamination studies
- Keywords
- generalized likelihood uncertainty estimation, adhesives, uncertainty analysis, rainfall-runoff modeling, hydrological model uncertainty, limits of prediction, confidence level, large sample behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.