The DOI matches its Crossref record.
Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis
Abstract
Hydrological models (HMs) can be applied for different purposes, and a key step is model calibration using objective functions (OF) to quantify the agreement between observed and calculated discharges. Fully understanding the OF is important to properly take advantage of model calibration and interpret the results. This study evaluates 36 OF proposed in the literature, considering two watersheds of different hydrological regimes. Daily simulated streamflow time-series, using a distributed hydrological model (MGB-IPH), and ten daily streamflow synthetic time-series, generated from the observed and calculated streamflows, were used in the analysis of each watershed. These synthetic data were used to evaluate how does each metric evaluate hypothetical cases that present isolated very well known error behaviors. Despite of all NSE-derived (Nash-Sutcliffe efficiency) metrics that use the square of the residuals in their formulation have shown higher sensitivity to errors in high flows, the ones that use daily and monthly averages of flow rates in absolute terms were more stringent than the others to assess HMs performance. Low flow errors were better evaluated by metrics that use the flow logarithm. The constant presence of zero flow rates deteriorate them significantly, with the exception of the metrics TRMSE (Transformed root mean square error) did not demonstrate this problem. An observed limitation of the formulations of some metrics was that the errors of overestimation or underestimation are compensated. Our results reassert that each metric should be interpreted specifically thinking about the aspects it has been proposed for, and simultaneously taking into account a set of metrics would lead to a broader evaluation of HM ability (e.g. multiobjective model evaluation). We recommend that the use of synthetic time series as those proposed in this work could be useful as an auxiliary step towards better understanding the evaluation of a calibrated hydrological model for each study case, taking into account model capabilities and observed hydrologic regime characteristics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“MARE is a good index for low data values considering that it is less sensitive to high flow values (Ferreira et al., 2020).”
From Construction of a daily streamflow dataset for Peru using a similarity-based regionalization approach and a hybrid hydrological modeling framework (2023) · cites this paper as Ferreira et al., 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 50
- Akhtar M., 2009, Use of regional climate model simulations as input for hydrological models for the Hindukush–Karakorum–Himalaya region, Hydrology and Earth System Sciences, 13, 1075, 10.5194/hess-13-1075-2009 · DOI
- Bennett N. D., 2013, ANDREASSIAN, V. Characterising performance of environmental models, Environmental Modelling & Software, 40, 1, 10.1016/j.envsoft.2012.09.011 · DOI · Paper page
- Boyle D. P., 2000, Toward improved calibration of hydrologic models: combining the strengths of manual and automatic methods, Water Resources Research, 36, 3663, 10.1029/2000WR900207 · DOI · Paper page
- Bravo J. M., 2009, Incorporating forecasts of rainfall in two hydrologic models used for medium-range streamflow forecasting, Journal of Hydrologic Engineering, 14, 435, 10.1061/(ASCE)HE.1943-5584.0000014 · doi:10.1061/(asce)he.1943-5584.0000014
- Collischonn W., 2007, The MGB-IPH model for large-scale rainfall-runoff modelling, Hydrological Sciences Journal, 52, 878, 10.1623/hysj.52.5.878 · doi:10.1623/hysj.52.5.878
- Criss R. E., 2008, Do Nash values have value? Discussion and alternate proposals, Hydrological Processes, 22, 2723, 10.1002/hyp.7072 · DOI
- Dakhlaoui H., 2012, Toward a more efficient Calibration Schema for HBV rainfall-runoff model, Journal of Hydrology (Amsterdam), 444-445, 161, 10.1016/j.jhydrol.2012.04.015 · doi:10.1016/j.jhydrol.2012.04.015
- Efstratiadis A., 2010, One decade of multi-objective calibration approaches in hydrological modelling: a review Multi-objective calibration approaches in hydrological modelling, Hydrological Sciences Journal, 55, 58, 10.1080/02626660903526292 · DOI · Paper page
- Felix W. S., 2016, Representação dos processos hidrológicos em bacia hidrográfica do semiárido paraibano com modelagem hidrológica distribuída, Revista Brasileira de Recursos Hídricos- RBRH, 21, 556
- Fenícia F., 2007, Soft combination of local models in a multi-objective framework, Hydrology and Earth System Sciences, 11, 1797, 10.5194/hess-11-1797-2007 · DOI
- Fowler K., 2018, Improved rainfall-runoff calibration for drying climate: choice of objective function, Water Resources Research, 54, 3392, 10.1029/2017WR022466 · DOI
- Garcia F., 2017, Which objective function to calibrate rainfall–runoff models for low-flow index simulations?, Hydrological Sciences Journal, 62, 1149, 10.1080/02626667.2017.1308511 · DOI
- Gupta H. V., 1998, Toward improved calibration of hydrologic models: multiple and noncommensurable measures of information, Water Resources Research, 34, 751, 10.1029/97WR03495 · DOI · Paper page
- Gupta H. V., 1999, Status of automatic calibration for hydrologic models: comparison with multilevel expert calibration, Journal of Hydrologic Engineering, 4, 135, 10.1061/(ASCE)1084-0699(1999)4:2(135) · doi:10.1061/(asce)1084-0699(1999)4:2(135 · Paper page
- Gupta H. V., 2009, Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling, Journal of Hydrology (Amsterdam), 377, 80, 10.1016/j.jhydrol.2009.08.003 · DOI · Paper page
- Gutierrez J. C. T., 2019, A new stopping criterion for multi-objective evolutionary algorithms: application in the calibration of a hydrologic model, Computational Geosciences, 23, 1219, 10.1007/s10596-019-09870-3 · doi:10.1007/s10596-019-09870-3
- Herman M. R., 2018, Evaluating the role of evapotranspiration remote sensing data in improving hydrological modeling predictability, Journal of Hydrology (Amsterdam), 556, 39, 10.1016/j.jhydrol.2017.11.009 · DOI · Paper page
- Hogue T. S., 2000, A multistep automatic calibration scheme for river forecasting models, Journal of Hydrometeorology, 1, 524, 10.1175/1525-7541(2000)001<0524:AMACSF>2.0.CO;2 · DOI
- Janssen P. H. M., 1995, Calibration of process-oriented models, Ecological Modelling, 83, 55, 10.1016/0304-3800(95)00084-9 · DOI · Paper page
- Kollat J. B., 2012, When are multiobjective calibration trade-offs in hydrologic models meaningful?, Water Resources Research, 48, 1, 10.1029/2011WR011534 · doi:10.1029/2011wr011534
- Krause P., 2005, Comparison of different efficiency criteria for hydrological model assessment, Advances in Geosciences, 5, 89, 10.5194/adgeo-5-89-2005 · DOI · Paper page
- Legates D. R., 1999, Evaluating the use of “goodness-of-fit” measures in hydrologicv and hydroclimatic model validation, Water Resources Research, 35, 233, 10.1029/1998WR900018 · DOI · Paper page
- Li C., 2010, Effect of calibration data series length on performance and optimal parameters of hydrological model, Water Science and Engineering, 3, 378
- Lindström G., 1997, Development and test of the distributed HBV-96 hydrological model, Journal of Hydrology (Amsterdam), 201, 272, 10.1016/S0022-1694(97)00041-3 · DOI
- Madsen H., 2000, Automatic calibration of a conceptual rainfall–runoff model using multiple objectives, Journal of Hydrology (Amsterdam), 235, 276, 10.1016/S0022-1694(00)00279-1 · DOI · Paper page
- Molina-Navarro E., 2017, The impact of the objective function in multi-site and multi-variable calibration of the SWAT model, Environmental Modelling & Software, 93, 255, 10.1016/j.envsoft.2017.03.018 · DOI
- Moriasi D. N., 2007, Model evaluation guidelines for systematic quatification of accuracy in watershed simulations, American Society of Agricultural and Biological Engineers, 50, 885
- Muleta M. K., 2012, Model performance sensitivity to objective function during automated calibrations, Journal of Hydrologic Engineering, 17, 756, 10.1061/(ASCE)HE.1943-5584.0000497 · DOI
- Nelson A. M., 2017, Impact of length of dataset on streamflow calibration parameters and performance of apex model, Journal of the American Water Resources Association, 53, 1164, 10.1111/1752-1688.12564 · DOI
- Oliveira R. F., 2018, Hydrological calibration and validation of the MGBIPH model for water resource management in the upper Teles Pires River basin in the Amazon-Cerrado ecotone in Brazil, Acta Amazonica, 49, 54, 10.1590/1809-4392201800812 · doi:10.1590/1809-4392201800812
- Paiva R. C. D., 2013, Largescale hydrologic and hydrodynamic modeling of the Amazon River basin, Water Resources Research, 49, 1226, 10.1002/wrcr.20067 · DOI · Paper page
- Pappenberger F., 2004, Functional classification and evaluation of hydrographs based on Multicomponent Mapping (Mx), Intl. J. River Basin Management, 2, 89, 10.1080/15715124.2004.9635224 · DOI
- Parker S. R., 2019, Targeted hydrologic model calibration to improve prediction of ecologically-relevant flow metrics, Journal of Hydrology (Amsterdam), 573, 546, 10.1016/j.jhydrol.2019.03.081 · doi:10.1016/j.jhydrol.2019.03.081
- Pechlivanidis I. G., 2012, Using an informational entropy-based metric as a diagnostic of flow duration to drive model parameter identification, Journal Global NEST, 14, 325
- Pereira F. F., 2014, Implementation of a two-way coupled atmospherichydrological system for environmental modeling at regional scale, Hydrology Research, 45, 504, 10.2166/nh.2013.335 · doi:10.2166/nh.2013.335
- Pool S., 2018, Evaluating model performance:towards a non-parametric variant of the Kling-Gupta efficiency, Hydrological Sciences Journal, 63, 1941, 10.1080/02626667.2018.1552002 · DOI · Paper page
- Pushpalatha R., 2012, A review of efficiency criteria suitable for evaluating low-flow simulations, Journal of Hydrology (Amsterdam), 420-421, 171, 10.1016/j.jhydrol.2011.11.055 · DOI · Paper page
- Rajib M. A., 2016, YU; Z. Multi-objective calibration of a hydrologic model using spatially distributed emotely sensed/in-situ soil moisture, Journal of Hydrology (Amsterdam), 536, 192, 10.1016/j.jhydrol.2016.02.037 · DOI · Paper page
- Ribeiro Neto A., 2006, Hydrological modelling in Amazonia—use of the MGB-IPH model and alternative databases, 303, 246
- Rientjes T. H. M., 2013, Multi-variable calibration of a semi-distributed hydrological model using streamflow data and satellite-based evapotranspiration, Journal of Hydrology (Amsterdam), 505, 276, 10.1016/j.jhydrol.2013.10.006 · DOI
- Romanowicz R. J., 2013, On the choice of calibration periods and objective functions: a practical guide to model parameter identification, Acta Geophysica, 61, 1477, 10.2478/s11600-013-0157-6 · doi:10.2478/s11600-013-0157-6
- Rwetabula J., 2012, Simulation of hydrological processes in the Simiyu River, tributary of Lake Victoria, Tanzania, Water AS, 38, 623
- Schaefli B., 2007, Do Nash values have value?, Hydrological Processes, 21, 2075, 10.1002/hyp.6825 · DOI · Paper page
- Troin M., 2015, Performance and Uncertainty Evaluation of Snow Models on Snowmelt Flow Simulations over a Nordic Catchment (Mistassibi, Canada), Hydrology, 2, 289, 10.3390/hydrology2040289 · doi:10.3390/hydrology2040289
- Tucci C. E. M., 2008, Short- and long-term flow forecasting in the Rio Grande watershed (Brazil), Atmospheric Science Letters, 9, 53, 10.1002/asl.165 · doi:10.1002/asl.165
- Van Liew M. W., 2007, Suitability of SWAT for the Conservation Effects Assessment Project: Comparison on USDA Agricultural Research Service Watersheds, Journal of Hydrological Research, 12, 173, 10.1061/(ASCE)1084-0699(2007)12:2(173) · doi:10.1061/(asce)1084-0699(2007)12:2(173 · Paper page
- Westerberg I. K., 2011, Calibration of hydrological models using flow-duration curves, Hydrology and Earth System Sciences, 15, 2205, 10.5194/hess-15-2205-2011 · DOI · Paper page
- Wohling T., 2013, Evaluating multiple performance criteria to calibrate the distributed hydrological model of the upper Neckar catchment, Environmental Earth Sciences, 69, 453, 10.1007/s12665-013-2306-2 · DOI
- Zhang Y., 2016, Multi-metric calibration of hydrological model to capture overall flow regimes, Journal of Hydrology (Amsterdam), 539, 525, 10.1016/j.jhydrol.2016.05.053 · DOI
- Zink M., 2018, Conditioning a hydrologic model using patterns of remotely sensed land surface temperature, Water Resources Research, 54, 2976, 10.1002/2017WR021346 · DOI
Cited by
Llauca, H., Leon, K., & Lavado-Casimiro, W. (2023) · Journal of Hydrology: Regional Studies · doi:10.1016/j.ejrh.2023.101381 · Cited by 29
Metadata
- Authors
- Ferreira, P. M. D. L., Paz, A. R. D., & Bravo, J. M.
- Affiliations
- Universidade Federal da Paraíba, Brasil
- Universidade Federal do Rio Grande do Sul, Brasil
- Venue
- RBRH
- Publisher
- FapUNIFESP (SciELO)
- Published
- 2020
- Volume
- 25
- ISSN
- 2318-0331
- 1414-381X
- Cited by
- 21
- References
- 0
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- creativecommons.org/licenses/by/4.0/ (am)
- creativecommons.org/licenses/by/4.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis. https://app.cochat.ai/papers/10.1590/2318-0331.252020190155
- The paper
- Ferreira, P. M. D. L., Paz, A. R. D., & Bravo, J. M. (2020). Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis. RBRH. https://doi.org/10.1590/2318-0331.252020190155
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.