Do Nash values have value?

Abstract

How Do We Communicate Model Performance?The process of model performance evaluation is of primary importance, not only in the model development and calibration process, but also when communicating the results to other researchers and to stakeholders.The basic 'rule' is that every modelling result should be put into context, for example, by indicating the model performance using appropriate indicators, and by highlighting potential sources of uncertainty, and this practice has found its entry into the large majority of papers and conference presentations.While the question of how to communicate the performance of a model to potential end-users is currently receiving increasing interest (e.g.Pappenberger and Beven, 2006), we-as well as many other colleagues-observe regularly that researchers take much less care when communicating model performance amongst ourselves.We seem to assume that we are speaking about familiar performance concepts and that they have comparable significance for various types of model applications and case studies.In doing so, we do not pay sufficient attention to making clear what the values represented by our performance measures really mean.Even concepts as simple as the bias between an observed and a simulated time series need to be put into proper context: whereas a 10% bias in simulation of simulated discharge may be unacceptable in a climate change impact assessment, it may be of less concern in the context of real-time flood forecasting.While some performance measures can have an absolute meaning, such as the common measure of linear correlation, the vast majority of performance measures, and in particular quadratic-error-based measures, can only be properly interpreted when viewed in the context of a reference value.For hydrologists, the Nash-Sutcliffe efficiency measure (Nash and Sutcliffe, 1970) (Equation (1)) has become a common part of our everyday jargon when reporting the results of a catchment modelling study.Somehow, we seem to ignore the fact that modellers in other fields of environmental sciences are not often familiar with what a 'Nash value' is.More important, it is worth asking whether we ourselves know what it means when we report that a model has a 'Nash value of 0•87'.The reality is that the Nash efficiency value, while a convenient and normalized (-inf to 1•0) measure of model performance does not provide a reliable basis for comparing the results of different case studies.In stating this, we are not pointing out anything not already well known (e.g.Martinec and Rango, 1989; Legates and McCabe, 1999; Seibert, 2001), but we think that it is worth recalling, with the hope that the following discussion will encourage hydrologists to use performance measures in a more useful manner-which is to always provide appropriate reference values-so that reported Nash values can be properly interpreted. Nash Values Need a BaselineThe Nash-Sutcliffe performance measure (Nash and Sutcliffe, 1970), called, hereafter, the Nash-Sutcliffe efficiency (NSE), is computed as follows: