CoChat
Cited by 21

The DOI matches its Crossref record.

Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis

Ferreira, Paloma Mara de Lima, Paz, Adriano Rolim da, Bravo, Juan Martín · 2020 · RBRH

Abstract

Hydrological models (HMs) can be applied for different purposes, and a key step is model calibration using objective functions (OF) to quantify the agreement between observed and calculated discharges. Fully understanding the OF is important to properly take advantage of model calibration and interpret the results. This study evaluates 36 OF proposed in the literature, considering two watersheds of different hydrological regimes. Daily simulated streamflow time-series, using a distributed hydrological model (MGB-IPH), and ten daily streamflow synthetic time-series, generated from the observed and calculated streamflows, were used in the analysis of each watershed. These synthetic data were used to evaluate how does each metric evaluate hypothetical cases that present isolated very well known error behaviors. Despite of all NSE-derived (Nash-Sutcliffe efficiency) metrics that use the square of the residuals in their formulation have shown higher sensitivity to errors in high flows, the ones that use daily and monthly averages of flow rates in absolute terms were more stringent than the others to assess HMs performance. Low flow errors were better evaluated by metrics that use the flow logarithm. The constant presence of zero flow rates deteriorate them significantly, with the exception of the metrics TRMSE (Transformed root mean square error) did not demonstrate this problem. An observed limitation of the formulations of some metrics was that the errors of overestimation or underestimation are compensated. Our results reassert that each metric should be interpreted specifically thinking about the aspects it has been proposed for, and simultaneously taking into account a set of metrics would lead to a broader evaluation of HM ability (e.g. multiobjective model evaluation). We recommend that the use of synthetic time series as those proposed in this work could be useful as an auxiliary step towards better understanding the evaluation of a calibrated hydrological model for each study case, taking into account model capabilities and observed hydrologic regime characteristics.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1590/2318-0331.252020190155

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 50

  1. Akhtar M., 2009, Use of regional climate model simulations as input for hydrological models for the Hindukush–Karakorum–Himalaya region, Hydrology and Earth System Sciences, 13, 1075, 10.5194/hess-13-1075-2009 · DOI
  2. Bennett N. D., 2013, ANDREASSIAN, V. Characterising performance of environmental models, Environmental Modelling & Software, 40, 1, 10.1016/j.envsoft.2012.09.011 · DOI · Paper page
  3. Boyle D. P., 2000, Toward improved calibration of hydrologic models: combining the strengths of manual and automatic methods, Water Resources Research, 36, 3663, 10.1029/2000WR900207 · DOI · Paper page
  4. Bravo J. M., 2009, Incorporating forecasts of rainfall in two hydrologic models used for medium-range streamflow forecasting, Journal of Hydrologic Engineering, 14, 435, 10.1061/(ASCE)HE.1943-5584.0000014 · doi:10.1061/(asce)he.1943-5584.0000014
  5. Collischonn W., 2007, The MGB-IPH model for large-scale rainfall-runoff modelling, Hydrological Sciences Journal, 52, 878, 10.1623/hysj.52.5.878 · doi:10.1623/hysj.52.5.878
  6. Criss R. E., 2008, Do Nash values have value? Discussion and alternate proposals, Hydrological Processes, 22, 2723, 10.1002/hyp.7072 · DOI
  7. Dakhlaoui H., 2012, Toward a more efficient Calibration Schema for HBV rainfall-runoff model, Journal of Hydrology (Amsterdam), 444-445, 161, 10.1016/j.jhydrol.2012.04.015 · doi:10.1016/j.jhydrol.2012.04.015
  8. Efstratiadis A., 2010, One decade of multi-objective calibration approaches in hydrological modelling: a review Multi-objective calibration approaches in hydrological modelling, Hydrological Sciences Journal, 55, 58, 10.1080/02626660903526292 · DOI · Paper page
  9. Felix W. S., 2016, Representação dos processos hidrológicos em bacia hidrográfica do semiárido paraibano com modelagem hidrológica distribuída, Revista Brasileira de Recursos Hídricos- RBRH, 21, 556
  10. Fenícia F., 2007, Soft combination of local models in a multi-objective framework, Hydrology and Earth System Sciences, 11, 1797, 10.5194/hess-11-1797-2007 · DOI
  11. Fowler K., 2018, Improved rainfall-runoff calibration for drying climate: choice of objective function, Water Resources Research, 54, 3392, 10.1029/2017WR022466 · DOI
  12. Garcia F., 2017, Which objective function to calibrate rainfall–runoff models for low-flow index simulations?, Hydrological Sciences Journal, 62, 1149, 10.1080/02626667.2017.1308511 · DOI
  13. Gupta H. V., 1998, Toward improved calibration of hydrologic models: multiple and noncommensurable measures of information, Water Resources Research, 34, 751, 10.1029/97WR03495 · DOI · Paper page
  14. Gupta H. V., 1999, Status of automatic calibration for hydrologic models: comparison with multilevel expert calibration, Journal of Hydrologic Engineering, 4, 135, 10.1061/(ASCE)1084-0699(1999)4:2(135) · doi:10.1061/(asce)1084-0699(1999)4:2(135 · Paper page
  15. Gupta H. V., 2009, Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling, Journal of Hydrology (Amsterdam), 377, 80, 10.1016/j.jhydrol.2009.08.003 · DOI · Paper page
  16. Gutierrez J. C. T., 2019, A new stopping criterion for multi-objective evolutionary algorithms: application in the calibration of a hydrologic model, Computational Geosciences, 23, 1219, 10.1007/s10596-019-09870-3 · doi:10.1007/s10596-019-09870-3
  17. Herman M. R., 2018, Evaluating the role of evapotranspiration remote sensing data in improving hydrological modeling predictability, Journal of Hydrology (Amsterdam), 556, 39, 10.1016/j.jhydrol.2017.11.009 · DOI · Paper page
  18. Hogue T. S., 2000, A multistep automatic calibration scheme for river forecasting models, Journal of Hydrometeorology, 1, 524, 10.1175/1525-7541(2000)001<0524:AMACSF>2.0.CO;2 · DOI
  19. Janssen P. H. M., 1995, Calibration of process-oriented models, Ecological Modelling, 83, 55, 10.1016/0304-3800(95)00084-9 · DOI · Paper page
  20. Kollat J. B., 2012, When are multiobjective calibration trade-offs in hydrologic models meaningful?, Water Resources Research, 48, 1, 10.1029/2011WR011534 · doi:10.1029/2011wr011534
  21. Krause P., 2005, Comparison of different efficiency criteria for hydrological model assessment, Advances in Geosciences, 5, 89, 10.5194/adgeo-5-89-2005 · DOI · Paper page
  22. Legates D. R., 1999, Evaluating the use of “goodness-of-fit” measures in hydrologicv and hydroclimatic model validation, Water Resources Research, 35, 233, 10.1029/1998WR900018 · DOI · Paper page
  23. Li C., 2010, Effect of calibration data series length on performance and optimal parameters of hydrological model, Water Science and Engineering, 3, 378
  24. Lindström G., 1997, Development and test of the distributed HBV-96 hydrological model, Journal of Hydrology (Amsterdam), 201, 272, 10.1016/S0022-1694(97)00041-3 · DOI
  25. Madsen H., 2000, Automatic calibration of a conceptual rainfall–runoff model using multiple objectives, Journal of Hydrology (Amsterdam), 235, 276, 10.1016/S0022-1694(00)00279-1 · DOI · Paper page
  26. Molina-Navarro E., 2017, The impact of the objective function in multi-site and multi-variable calibration of the SWAT model, Environmental Modelling & Software, 93, 255, 10.1016/j.envsoft.2017.03.018 · DOI
  27. Moriasi D. N., 2007, Model evaluation guidelines for systematic quatification of accuracy in watershed simulations, American Society of Agricultural and Biological Engineers, 50, 885
  28. Muleta M. K., 2012, Model performance sensitivity to objective function during automated calibrations, Journal of Hydrologic Engineering, 17, 756, 10.1061/(ASCE)HE.1943-5584.0000497 · DOI
  29. Nelson A. M., 2017, Impact of length of dataset on streamflow calibration parameters and performance of apex model, Journal of the American Water Resources Association, 53, 1164, 10.1111/1752-1688.12564 · DOI
  30. Oliveira R. F., 2018, Hydrological calibration and validation of the MGBIPH model for water resource management in the upper Teles Pires River basin in the Amazon-Cerrado ecotone in Brazil, Acta Amazonica, 49, 54, 10.1590/1809-4392201800812 · doi:10.1590/1809-4392201800812
  31. Paiva R. C. D., 2013, Largescale hydrologic and hydrodynamic modeling of the Amazon River basin, Water Resources Research, 49, 1226, 10.1002/wrcr.20067 · DOI · Paper page
  32. Pappenberger F., 2004, Functional classification and evaluation of hydrographs based on Multicomponent Mapping (Mx), Intl. J. River Basin Management, 2, 89, 10.1080/15715124.2004.9635224 · DOI
  33. Parker S. R., 2019, Targeted hydrologic model calibration to improve prediction of ecologically-relevant flow metrics, Journal of Hydrology (Amsterdam), 573, 546, 10.1016/j.jhydrol.2019.03.081 · doi:10.1016/j.jhydrol.2019.03.081
  34. Pechlivanidis I. G., 2012, Using an informational entropy-based metric as a diagnostic of flow duration to drive model parameter identification, Journal Global NEST, 14, 325
  35. Pereira F. F., 2014, Implementation of a two-way coupled atmospherichydrological system for environmental modeling at regional scale, Hydrology Research, 45, 504, 10.2166/nh.2013.335 · doi:10.2166/nh.2013.335
  36. Pool S., 2018, Evaluating model performance:towards a non-parametric variant of the Kling-Gupta efficiency, Hydrological Sciences Journal, 63, 1941, 10.1080/02626667.2018.1552002 · DOI · Paper page
  37. Pushpalatha R., 2012, A review of efficiency criteria suitable for evaluating low-flow simulations, Journal of Hydrology (Amsterdam), 420-421, 171, 10.1016/j.jhydrol.2011.11.055 · DOI · Paper page
  38. Rajib M. A., 2016, YU; Z. Multi-objective calibration of a hydrologic model using spatially distributed emotely sensed/in-situ soil moisture, Journal of Hydrology (Amsterdam), 536, 192, 10.1016/j.jhydrol.2016.02.037 · DOI · Paper page
  39. Ribeiro Neto A., 2006, Hydrological modelling in Amazonia—use of the MGB-IPH model and alternative databases, 303, 246
  40. Rientjes T. H. M., 2013, Multi-variable calibration of a semi-distributed hydrological model using streamflow data and satellite-based evapotranspiration, Journal of Hydrology (Amsterdam), 505, 276, 10.1016/j.jhydrol.2013.10.006 · DOI
  41. Romanowicz R. J., 2013, On the choice of calibration periods and objective functions: a practical guide to model parameter identification, Acta Geophysica, 61, 1477, 10.2478/s11600-013-0157-6 · doi:10.2478/s11600-013-0157-6
  42. Rwetabula J., 2012, Simulation of hydrological processes in the Simiyu River, tributary of Lake Victoria, Tanzania, Water AS, 38, 623
  43. Schaefli B., 2007, Do Nash values have value?, Hydrological Processes, 21, 2075, 10.1002/hyp.6825 · DOI · Paper page
  44. Troin M., 2015, Performance and Uncertainty Evaluation of Snow Models on Snowmelt Flow Simulations over a Nordic Catchment (Mistassibi, Canada), Hydrology, 2, 289, 10.3390/hydrology2040289 · doi:10.3390/hydrology2040289
  45. Tucci C. E. M., 2008, Short- and long-term flow forecasting in the Rio Grande watershed (Brazil), Atmospheric Science Letters, 9, 53, 10.1002/asl.165 · doi:10.1002/asl.165
  46. Van Liew M. W., 2007, Suitability of SWAT for the Conservation Effects Assessment Project: Comparison on USDA Agricultural Research Service Watersheds, Journal of Hydrological Research, 12, 173, 10.1061/(ASCE)1084-0699(2007)12:2(173) · doi:10.1061/(asce)1084-0699(2007)12:2(173 · Paper page
  47. Westerberg I. K., 2011, Calibration of hydrological models using flow-duration curves, Hydrology and Earth System Sciences, 15, 2205, 10.5194/hess-15-2205-2011 · DOI · Paper page
  48. Wohling T., 2013, Evaluating multiple performance criteria to calibrate the distributed hydrological model of the upper Neckar catchment, Environmental Earth Sciences, 69, 453, 10.1007/s12665-013-2306-2 · DOI
  49. Zhang Y., 2016, Multi-metric calibration of hydrological model to capture overall flow regimes, Journal of Hydrology (Amsterdam), 539, 525, 10.1016/j.jhydrol.2016.05.053 · DOI
  50. Zink M., 2018, Conditioning a hydrologic model using patterns of remotely sensed land surface temperature, Water Resources Research, 54, 2976, 10.1002/2017WR021346 · DOI

Metadata

Authors
Ferreira, P. M. D. L., Paz, A. R. D., & Bravo, J. M.
Affiliations
  • Universidade Federal da Paraíba, Brasil
  • Universidade Federal do Rio Grande do Sul, Brasil
Venue
RBRH
Published
2020
Volume
25
ISSN
  • 2318-0331
  • 1414-381X
Cited by
21
References
0

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis. https://app.cochat.ai/papers/10.1590/2318-0331.252020190155
The paper
Ferreira, P. M. D. L., Paz, A. R. D., & Bravo, J. M. (2020). Objective functions used as performance metrics for hydrological models: state-of-the-art and critical analysis. RBRH. https://doi.org/10.1590/2318-0331.252020190155

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.