Comparing Simulated and Measured Values Using Mean Squared Deviation and its Components

Abstract

When output (x) of a mechanistic model is compared with measurement (y), it is common practice to calculate the correlation coefficient between x and y, and to regress y on x. There are, however, problems in this approach. The assumption of the regression, that y is linearly related to x, is not guaranteed and is unnecessary for the x–y comparison. The correlation and regression coefficients are not explicitly related to other commonly used statistics [e.g., root mean squared deviation (RMSD)]. We present an approach based on the mean squared deviation (MSD = RMSD 2) and show that it is better suited to the x–y comparison than regression. Mean squared deviation is the sum of three components: squared bias (SB), squared difference between standard deviations (SDSD), and lack of correlation weighted by the standard deviations (LCS). To show examples, the MSD‐based analysis was applied to simulation vs. measurement comparisons in literature, and the results were compared with those from regression analysis. The analysis of MSD clearly identified the simulation vs. measurement contrasts with larger deviation than others; the correlation–regression approach tended to focus on the contrasts with lower correlation and regression line far from the equality line. It was also shown that results of the MSD‐based analysis were easier to interpret than those of regression analysis. This is because the three MSD components are simply additive and all constituents of the MSD components are explicit. This approach will be useful to quantify the deviation of calculated values obtained with this model from measurements.