The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index; 4 of the 8 sources its claims cite were checked for support.
Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature
Open accessPublished version of a preprint
Abstract
Abstract. Both the root mean square error (RMSE) and the mean absolute error (MAE) are regularly employed in model evaluation studies. Willmott and Matsuura (2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and thus the MAE would be a better metric for that purpose. While some concerns over using RMSE raised by Willmott and Matsuura (2005) and Willmott et al. (2009) are valid, the proposed avoidance of RMSE in favor of MAE is not the solution. Citing the aforementioned papers, many researchers chose MAE over RMSE to present their model evaluation statistics when presenting or adding the RMSE measures could be more beneficial. In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009). The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian. In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric, whereas Willmott et al. (2009) indicated that the sums-of-squares-based statistics do not satisfy this rule. In the end, we discussed some circumstances where using the RMSE will be more beneficial. However, we do not contend that the RMSE is superior over the MAE. Instead, a combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This technical note argues against the recommendation by Willmott and Matsuura (2005) and Willmott et al. (2009) to abandon RMSE in favor of MAE, showing that RMSE is not ambiguous in meaning and is more appropriate when errors are expected to be Gaussian. The authors also demonstrate that RMSE satisfies the triangle inequality required of a distance metric, contrary to the earlier claims, while concluding that a combination of metrics is often needed.
Key findings
- RMSE is not ambiguous in its meaning, contrary to Willmott et al. (2009); it is more appropriate than MAE when the error distribution is expected to be Gaussian, whereas MAE suits uniformly distributed errors.
- Using randomly generated zero-mean, unit-variance Gaussian pseudo-errors, RMSEs converge to the true standard deviation (SE = 1) once n ≥ 100, with standard deviation within 5% of truth; MAEs converge to 0.8, approximating the expectation of √(2/π). Neither metric is robust at n = 4 or n = 10.
- RMSE satisfies the triangle inequality required for a distance metric: the example in Willmott et al. (2009) (d(a,c)=4, d(a,b)=2, d(b,c)=3) actually represents the MSE, not the RMSE, and the L2-norm proof in Eqs. (5)–(7) shows RMSE does satisfy it.
- RMSE has practical advantages: it avoids absolute values (useful for gradients/sensitivities and data assimilation cost functions) and is generally better at discriminating among model results by weighting unfavorable conditions more heavily.
Methods
- Technical note / commentary design: no new observational dataset; the argument is analytical, using norm theory (L1- and L2-norms linked to MAE and RMSE) and a proof of the triangle inequality for the L2-norm.
- A Monte Carlo-style demonstration: five sets of pseudo-errors of size n = 4, 10, 100, 1000, 10 000, and 1000 000 drawn from a zero-mean, unit-variance Gaussian distribution with different random seeds, comparing RMSEs and MAEs (Table 1).
- Reference to prior empirical example: Chai et al. (2009) reported RMSE-to-MAE ratios of 1.63 to 2.29 for model NO2 column predictions versus SCIAMACHY satellite observations.
Limitations
- Stated by the authors: the analysis assumes unbiased errors and does not consider uncertainties from observation errors or the method used to compare model and observations; all statistics are less useful with limited sample sizes (e.g., n = 4 or 10), where presenting the errors themselves may be more appropriate; for non-normal distributions, additional moments (mean, variance, skewness, flatness) are needed; outliers several orders larger than other samples may justifiably be removed when sample numbers are limited, and systematic errors may need removal before computing RMSEs.
- Evident from the text: the demonstration relies on synthetic Gaussian pseudo-errors rather than real model–observation comparisons, so the convergence results are conditional on the Gaussian assumption; the authors explicitly do not claim RMSE is superior to MAE, so the note offers guidance rather than a definitive metric selection rule.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
14 Own interpretation · 6 No basis given · 4 Supported · 3 Sources not found · 3 Own result · 1 No full text of the sources · 1 Not checked yet
Every source the paper cites for this claim was checked and supports it.
In the paper
“Using hypothetical sets of four errors, Willmott and Matsuura (2005) demonstrated that while keeping the MAE as a constant of 2.0, the RMSE varies from 2.0 to 4.0.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
What the sources say
- Supportedcites Willmott and Matsuura (2005) source
Table 1 identifies five hypothetical sets of four errors, and Section 3 says their RMSE increases as the error magnitudes become more variable. The paper gives the bounds as MAE ≤ RMSE ≤ √n × MAE; with four errors and an MAE of 2.0, those bounds are 2.0 and 4.0. Page numbers are unavailable in our copy. (LLM, conf=0.90)
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric, whereas Willmott et al. (2009) indicated that the sums-ofsquares-based statistics do not satisfy this rule.”
- Also stated in: § 3 Triangle inequality of a metric · boosted
“This proves that RMSE satisfies the triangle inequality required for a distance function metric.”
- Also stated in: § 4 Summary and discussion · boosted
“In addition, we demonstrate that the RMSE satisfies the triangle inequality required for a distance function metric.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > Abstract
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 3 Triangle inequality of a metric · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When the sample size reaches 100 or above, using the calculated RMSEs one can re-construct the error distribution close to its “truth” or “exact solution”, with its standard deviation within 5 % to its truth (i.e., SE = 1).”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 2 Interpretation of RMSE and MAE
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The errors are unbiased and follow a normal distribution when presenting the RMSE.
- The authors state this as an assumption: The uncertainties brought in by observation errors or the method used to compare model and observations are not considered.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For instance, Table 1 shows that neither the RMSEs nor the MAEs are robust when only 4 or 10 samples are used to calculate those values.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 2 Interpretation of RMSE and MAE
plainn = 10- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- The authors state this as an assumption: The errors are unbiased and follow a normal distribution when presenting the RMSE.
- The authors state this as an assumption: The uncertainties brought in by observation errors or the method used to compare model and observations are not considered.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian.”
- Also stated in: § 1 Introduction · plain
“When the error distribution is expected to be Gaussian and there are enough samples, the RMSE has an advantage over the MAE to illustrate the error distribution.”
- Also stated in: § 4 Summary and discussion · plain
“We present that the RMSE is not ambiguous in its meaning, and it is more appropriate to use than the MAE when model errors follow a normal distribution.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Model errors are likely to have a normal distribution rather than a uniform distribution.
- Also stated in: § 1 Introduction · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Giving higher weighting to the unfavorable conditions, the RMSE usually is better at revealing model performance differences.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 4 Summary and discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The L1-norm and L2-norm are closely related to the MAE and the RMSE, respectively, as shown in Eqs. ( 3) and ( 4): Horn and Johnson, 1990).”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 3 Triangle inequality of a metric
What the sources say
- Not matchedcites Horn and Johnson, 1990
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009).”
- Also stated in: § 4 Summary and discussion · plain
“We present that the RMSE is not ambiguous in its meaning, and it is more appropriate to use than the MAE when model errors follow a normal distribution.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > Abstract
What the sources say
- Not matchedcites Willmott et al. (2009)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 4 Summary and discussion · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One distinct advantage of RMSEs over MAEs is that RM-SEs avoid the use of absolute value, which is highly undesirable in many mathematical calculations.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 4 Summary and discussion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similarly, one can readily show that, for several sets of errors with the same RMSE, the MAE would vary from set to set.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 2
- The quote only partly states this claim. 1
Every source the paper cites for this claim was checked and supports it.
In the paper
“They summarized that the RMSE tends to become increasingly larger than the MAE (but not necessarily in a monotonic fashion) as the distribution of error magnitudes becomes more variable.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
What the sources say
- Supportedcites Willmott and Matsuura (2005) source
The Concluding Remarks state the attributed point directly, including the qualification that the increase need not be monotonic. (LLM, conf=0.95)
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“They concluded that the RMSE varies with the variability of the the error magnitudes and the total-error or averageerror magnitude (MAE), and the sample size n.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
What the sources say
- Supportedcites Willmott and Matsuura (2005) source
The Concluding Remarks state that RMSE depends on error-magnitude variability, MAE, and the square root of sample size. The claim says it varies with sample size n, which accurately conveys that dependence without asserting that the relationship is linear. (LLM, conf=0.93)
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Willmott and Matsuura (2005) have simply proved that the RMSE is not equivalent to the MAE, and one cannot easily derive the MAE value from the RMSE (and vice versa).”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
What the sources say
- Supportedcites Willmott and Matsuura (2005) source
Section 3 describes an inconsistent relationship between MAE and RMSE, including cases where RMSE changes with error-magnitude variance. Section 4 explains that RMSE depends on MAE, the distribution of error magnitudes, and the number of errors. This supports the claim that the measures are not equivalent and that one cannot generally derive either value from the other alone. (LLM, conf=0.90)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 1 shows that the MAEs converge to 0.8, an approximation to the expectation of 2 π .”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 2 Interpretation of RMSE and MAE
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
- The authors state this as an assumption: The errors are unbiased and follow a normal distribution when presenting the RMSE.
- The authors state this as an assumption: The uncertainties brought in by observation errors or the method used to compare model and observations are not considered.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When both metrics are calculated, the RMSE is by definition never smaller than the MAE.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The MAE is suitable to describe uniformly distributed errors.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 2 Interpretation of RMSE and MAE
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For other kinds of distributions, more statistical moments of model errors, such as mean, variance, skewness, and flatness, are needed to provide a complete picture of the model error variation.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 4 Summary and discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“The ratio of RMSE to MAE ranged from 1.63 to 2.29 (see Table 1 of Chai et al., 2009).”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites Chai et al., 2009 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In many of the model sensitivity studies that use only RMSE, a detailed interpretation is not critical because variations of the same model will have similar error distributions.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 4 Summary and discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As every statistical measure condenses a large number of data into a single value, it only provides one projection of the model errors emphasizing a certain aspect of the error characteristics of the model performance.”
- Also stated in: § 4 Summary and discussion · plain
“As stated earlier, any single metric provides only one projection of the model errors and, therefore, only emphasizes a certain aspect of the error characteristics.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Also stated in: § 4 Summary and discussion · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.”
- Also stated in: § 4 Summary and discussion · plain
“A combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Also stated in: § 4 Summary and discussion · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While the MAE gives the same weight to all errors, the RMSE penalizes variance as it gives errors with larger absolute values more weight than errors with smaller absolute values.”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The authors state this as an assumption.
The errors are unbiased and follow a normal distribution when presenting the RMSE.
“The underlying assumption when presenting the RMSE is that the errors are unbiased and follow a normal distribution.”
“We also assume the error sample set is unbiased.”
The paper relies on this without support.
Model errors are likely to have a normal distribution rather than a uniform distribution.
“Because model errors are likely to have a normal distribution rather than a uniform distribution, the RMSE is a better metric to present than the MAE for such a type of data.”
The authors state this as an assumption.
The uncertainties brought in by observation errors or the method used to compare model and observations are not considered.
“The uncertainties brought in by observation errors or the method used to compare model and observations are not considered here.”
The paper relies on this without support.
There are often hundreds of observations available to calculate model statistics.
“Fortunately, there are often hundreds of observations available to calculate model statistics, unlike the examples with n = 4 (Willmott and Matsuura, 2005) and n = 10 ( Willmott et al., 2009).”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 4 Summary and discussion
“| 4 | 0.92, 0.65, 1.48, 1.02, 0.79 0.70, 0.57, 1.33, 1.16, 0.76 | |”
§ Root mean square error (RMSE) or mean absolute error (MAE)? -Arguments against avoiding RMSE in the literature > 1 Introduction
“When both metrics are calculated, the RMSE is by definition never smaller than the MAE.”
Citations
Cites
Horn, R. A., & Johnson, C. R. (1985) · Cited by 22,203
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,707
Willmott, C., & Matsuura, K. (2005) · Climate Research · doi:10.3354/cr030079 · Cited by 6,512 · cited 4×
“Further, Willmott et al. (2009) concluded that the sums-of-squaresbased error statistics such as the RMSE and the standard error have inherent ambiguities and recommended the use of alternates such as the MAE.”
BackgroundMethodInfluentialHuber, P. J., & Ronchetti, E. M. (2009) · Wiley series in probability and statistics · doi:10.1002/9780470434697 · Cited by 1,823
Ambiguities inherent in sums-of-squares-based error statistics
Willmott, C. J., Matsuura, K., & Robeson, S. M. (2008) · Atmospheric Environment · doi:10.1016/j.atmosenv.2008.10.005 · Cited by 212 · cited 1×
“Fortunately, there are often hundreds of observations available to calculate model statistics, unlike the examples with n= 4 (Willmott and Matsuura, 2005) and n= 10 (Willmott et al., 2009).”
BackgroundMcKeen, S. A., Wilczak, J. M., Grell, G., et al. (2005) · Journal of Geophysical Research Atmospheres · doi:10.1029/2005jd005858 · Cited by 208
Savage, N., Agnew, P., Davis, L. S., Ordóñez, C., Thorpe, R., Johnson, C. E., O’Connor, F. M., & Dalvi, M. (2013) · Geoscientific model development · doi:10.5194/gmd-6-353-2013 · Cited by 128
Taylor, M. H., Lösch, M., Wenzel, M., et al. (2013) · Journal of Climate · doi:10.1175/jcli-d-13-00089.1 · Cited by 93
Chai, T., Kim, H. C., Lee, P., et al. (2013) · Geoscientific model development · doi:10.5194/gmd-6-1831-2013 · Cited by 87
Self-citationA multi-physics ensemble of present-day climate regional simulations over the Iberian Peninsula
Jerez, S., Montávez, J. P., Jiménez‐Guerrero, P., et al. (2012) · Climate Dynamics · doi:10.1007/s00382-012-1539-1 · Cited by 80 · cited 2×
“(2009) (e.g., Taylor et al., 2013; Chatterjee et al., 2013; Jerez et al., 2013).”
Method
Cited by
Sorted by citations
Chicco, D., Warrens, M. J., & Jurman, G. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.623 · Cited by 5,380
“…than MAE; in addition to this general note, several further considerations helping researchers in choosing the more suitable metric for evaluating a regression model given the available data and the target task can be drawn ( Chai & Draxler, 2014 ; Willmott & Matsuura, 2005 ; Wang & Lu, 2018 ).”
Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not
Hodson, T. (2022) · Geoscientific model development · doi:10.5194/gmd-15-5481-2022 · Cited by 2,786
“Chai and Draxler (2014) were correct that RMSE is optimal for normally distributed errors, though wrongly suggest that MAE only applies to uniformly distributed errors.”
BackgroundMethodInfluentialA survey on missing data in machine learning
Emmanuel, T., Maupong, T., Mpoeleng, D., Semong, T., Mphago, B., & Tabona, O. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00516-9 · Cited by 1,067
“However, [144] partially disputed the conclusions and introduced arguments against avoiding RMSE.”
BackgroundBotchkarev, A. (2019) · Interdisciplinary Journal of Information Knowledge and Management · doi:10.28945/4184 · Cited by 961
“Chai and Draxler (2014) clarified that “as every statistical measure condenses a large number of data into a single value, it only provides one projection of the model errors emphasizing a certain aspect of the error characteristics of the model performance.””
BackgroundInfluentialBeck, H. E., Vergopolan, N., Pan, M., Levizzani, V., van Dijk, A. I. J. M., Weedon, G. P., Brocca, L., Pappenberger, F., Huffman, G. J., & Wood, E. F. (2017) · Hydrology and earth system sciences · doi:10.5194/hess-21-6201-2017 · Cited by 932
Chong, E., Han, C., & Park, F. C. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.04.030 · Cited by 834
“For more interpretations, see Chai and Draxler (2014).”
SRDiff: Single image super-resolution with diffusion probabilistic models
Li, H., Yang, Y., Chang, M., Chen, S., Feng, H., Xu, Z., Li, Q., & Chen, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.029 · Cited by 777
Root mean square error or mean absolute error? Use their ratio as well
Karunasingha, D. S. K. (2021) · Information Sciences · doi:10.1016/j.ins.2021.11.036 · Cited by 818
Review on landslide susceptibility mapping using support vector machines
Huang, Y., & Zhao, L. (2018) · CATENA · doi:10.1016/j.catena.2018.03.003 · Cited by 749
Be aware of error measures. Further studies on validation of predictive QSAR models
Roy, K., Das, R. N., Ambure, P. S., & Aher, R. B. (2016) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2016.01.008 · Cited by 714
Yaseen, Z. M., Sulaiman, S. O., Deo, R. C., & Chau, K. (2018) · Journal of Hydrology · doi:10.1016/j.jhydrol.2018.11.069 · Cited by 704
Hoffmann, L., Günther, G., Li, D., Stein, O., Wu, X., Griessbach, S. G., Heng, Y., Konopka, P., Muller, R. H., Vogel, B., et al. (2019) · Atmospheric chemistry and physics · doi:10.5194/acp-19-3097-2019 · Cited by 670
“For a more detailed discussion on the advantages and disadvantages of using MADs versus standard deviations see Willmott and Matsuura (2005) and Chai and Draxler (2014).”
A novel hybrid artificial intelligence approach for flood susceptibility assessment
Chapi, K., Singh, V. P., Shirzadi, A., Shahabi, H., Bui, D. T., Pham, B. T., & Khosravi, K. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.06.012 · Cited by 659
Τσόκα, Σ., Tsikaloudaki, K., & Theodosiou, T. G. (2018) · Sustainable Cities and Society · doi:10.1016/j.scs.2018.08.009 · Cited by 615
A review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584
“L(Y, f (X)) = n∑ i=1 |yi − f (xi)| (9) The mean value of SAD is usually used as the regression loss, which is called the Mean Absolute Error (MAE) [22, 247].”
MethodYildiz, B., Bilbao, J. I., & Sproul, A. B. (2017) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.02.023 · Cited by 563
Ayvaz, S., & Alpay, K. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114598 · Cited by 616
Pham, B. T., Pradhan, B., Bui, D. T., Prakash, I., & Dholakia, M. B. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.07.005 · Cited by 563
A Review of Deep Learning Models for Time Series Prediction
Han, Z., Zhao, J., Leung, H., Ma, K. F., & Wang, W. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2923982 · Cited by 537
“Mean Absolute Percentage Error (MAPE) [211] and Root Mean Square Error (RMSE) [212] are deployed as indices for error statistics, which are defined as follows. where N denotes the length of predicted results, i.e., ˆ y i . y i refers to the real value.”
BackgroundSelecting appropriate machine learning methods for digital soil mapping
Khaledian, Y., & Miller, B. A. (2019) · Applied Mathematical Modelling · doi:10.1016/j.apm.2019.12.016 · Cited by 472
“Therefore, a combination of metrics can provide a more complete description of the model performance [84].”
BackgroundPredicting compressive strength of lightweight foamed concrete using extreme learning machine model
Yaseen, Z. M., Deo, R. C., Hilal, A. A., Abd, A. M., Cornejo-Bueno, L. M., Salcedo‐Sanz, S., & Nehdi, M. L. (2017) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2017.09.004 · Cited by 436
Gramatica, P., & Sangion, A. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00088 · Cited by 420
Performance metrics for the assessment of satellite data products: an ocean color case study
Seegers, B. N., Stumpf, R. P., Schaeffer, B. A., Loftin, K. A., & Werdell, P. J. (2018) · Optics Express · doi:10.1364/oe.26.007404 · Cited by 412
Ghimire, S., Deo, R. C., Raj, N., & Mi, J. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.113541 · Cited by 403
An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Şişman, B., Yamagishi, J., King, S., & Li, H. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 399
“The common objective evaluation metrics include Mel-cepstral distortion (MCD) [246] for spectrum, and PCC [247] and RMSE [248]–[250] for prosody.”
BackgroundInfluentialBui, D. T., Pradhan, B., Nampak, H., Bui, Q., Tran, Q.-A., & Nguyen, Q.-P. (2016) · Journal of Hydrology · doi:10.1016/j.jhydrol.2016.06.027 · Cited by 356
Tran, V. Q., Dang, V. Q., & Ho, L. S. (2022) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2022.126578 · Cited by 367
Artificial Intelligence Approaches for Prediction of Compressive Strength of Geopolymer Concrete
Dao, D. V., Ly, H., Trinh, S. H., Le, T.-T., & Pham, B. T. (2019) · Materials · doi:10.3390/ma12060983 · Cited by 350
“These measures can be calculated by the following equations [55]:”
BackgroundMethodInfluentialNiu, M., Kebreab, E., Hristov, A. N., Oh, J., Arndt, C., Bannink, A., Bayat, A. R., Brito, A. F., Boland, T. M., Casper, D. P., et al. (2018) · Global Change Biology · doi:10.1111/gcb.14094 · Cited by 343
“The MAE was calculated as shown in Equation (vi) to quantify the prediction error as suggested by Chai and Draxler (2014): In both cases, smaller RMSPE or MAE implies better model performance.”
Building Auto-Encoder Intrusion Detection System based on random forest feature selection
Li, X., Chen, W., Zhang, Q., & Wu, L. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101851 · Cited by 340
On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression
Qi, J., Du, J., Siniscalchi, S. M., Ma, X., & Lee, C.-H. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3016837 · Cited by 370
“An exception occurs when the expected error satisfies Gaussian-distributed and enough training samples are available [3].”
BackgroundForecasting air quality time series using deep learning
Freeman, B. S., Taylor, G. W., Gharabaghi, B., & Thé, J. V. G. (2018) · Journal of the Air & Waste Management Association · doi:10.1080/10962247.2018.1459956 · Cited by 338
“MAE and RMSE are widely used measures of continuous variables, with RMSE criticized for overbiasing toward large errors (Chai and Draxler 2014; Willmott and Matsuura 2005).”
MethodAL‐Musaylh, M. S., Deo, R. C., Adamowski, J., & Li, Y. (2017) · Advanced Engineering Informatics · doi:10.1016/j.aei.2017.11.002 · Cited by 302
Zhang, Z., Ye, L., Qin, H., Liu, Y., Wang, C., Yu, X. J., Yin, X., & Li, J. (2019) · Applied Energy · doi:10.1016/j.apenergy.2019.04.047 · Cited by 301
Khalil, M., McGough, A. S., Pourmirza, Z., Pazhoohesh, M., & Walker, S. L. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105287 · Cited by 331
Comparative Analysis of Machine Learning Techniques for Predicting Air Quality in Smart Cities
Ameer, S., Shah, M. A., Khan, A., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2925082 · Cited by 293
Vafaei, S., Soosani, J., Adeli, K., Fadaei, H., Naghavi, H., Pham, T. D., & Bui, D. T. (2018) · Remote Sensing · doi:10.3390/rs10020172 · Cited by 281
“However, the equation is significantly influenced by large values and outliers [84].”
BackgroundNguyen, K. T., Nguyen, Q. D., Lê, A. T., Shin, J., & Lee, K. (2020) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2020.118581 · Cited by 273
Meta-learning on Heterogeneous Information Networks for Cold-start Recommendation
Lu, Y., Fang, Y., & Shi, C. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403207 · Cited by 279
“performance [2], it is intuitive that the larger the loss value in a semantic space, the smaller the corresponding weight should be.”
BackgroundNam, K., Hwangbo, S., & Yoo, C. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109725 · Cited by 271
Novel approach for streamflow forecasting using a hybrid ANFIS-FFA model
Yaseen, Z. M., Ebtehaj, I., Bonakdari, H., Deo, R. C., Danandeh Mehr, A., Mohtar, W. H. M. W., Diop, L., El-shafie, A., & Singh, V. P. (2017) · Journal of Hydrology · doi:10.1016/j.jhydrol.2017.09.007 · Cited by 267
Review of medical image quality assessment
Chow, L. S., & Paramesran, R. (2016) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2016.02.006 · Cited by 266
Low-Cost Outdoor Air Quality Monitoring and Sensor Calibration
Concas, F., Mineraud, J., Lagerspetz, E., Varjonen, S., Liu, X., Puolamäki, K., Nurmi, P., & Tarkoma, S. (2021) · ACM Transactions on Sensor Networks · doi:10.1145/3446005 · Cited by 329
“ge distance of points from the regression model. RMSE and MSE weigh errors proportionally to their magnitude, whereas MAE weighs all errors equally. This makes RMSE and MSE more sensitive to outliers [125], suggesting that MAE is a better measure for measuring the average performance, whereas (R)MSE is useful for meas…”
BackgroundGramatica, P. (2020) · International Journal of Quantitative Structure-Property Relationships · doi:10.4018/ijqspr.20200701.oa1 · Cited by 266
“The different quality of these two parameters is commented in the literature (Chai & Draxler, 2014; Roy et al., 2016).”
BackgroundMohammadi, B., & Mehdizadeh, S. (2020) · Agricultural Water Management · doi:10.1016/j.agwat.2020.106145 · Cited by 255
Bhagat, S. K., Tung, T. M., & Yaseen, Z. M. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.119473 · Cited by 252
Deep learning as a tool for ecology and evolution
Borowiec, M. L., Dikow, R. B., Frandsen, P. B., McKeeken, A. J., Valentini, G., & White, A. E. (2022) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13901 · Cited by 295
Hannan, M. A., Lipu, M. S. H., Hussain, A., Ker, P. J., Mahlia, T. M. I., Mansor, M., Ayob, A., Saad, M. H. M., & Dong, Z. Y. (2020) · Scientific Reports · doi:10.1038/s41598-020-61464-7 · Cited by 244
“In this study, RMSE was chosen as the objective function because of the large number of sample variables and randomness behavior of SOC errors 27 .”
MethodVerification of deterministic solar forecasts
Yang, D., Alessandrini, S., Antonanzas, J., Antoñanzas-Torres, F., Bădescu, V., Beyer, H. G., Blaga, R., Boland, J. W., Bright, J. M., Coimbra, C. F. M., et al. (2020) · Solar Energy · doi:10.1016/j.solener.2020.04.019 · Cited by 244
“Furthermore, since there are countless publications that discuss and conclude that one metric is better than the other, it is not difficult to cite those articles that support any choice the author wishes to make (Chai and Draxler, 2014).”
BackgroundWang, X., Tolksdorf, V., Otto, M., et al. (2020) · International Journal of Climatology · doi:10.1002/joc.6686 · Cited by 238
“Since RMSE gives more weight to errors with larger absolute values and is more sensitive to outliers, it provides information about the variability of the error distribution (Chai and Draxler, 2014; Willmott and Matsuura, 2005).”
BackgroundDynamic Neural Network Modelling of Soil Moisture Content for Predictive Irrigation Scheduling
Adeyemi, O., Grove, I. G., Peets, S., Domun, Y., & Norton, T. (2018) · Sensors · doi:10.3390/s18103408 · Cited by 251
“The RMSE and MAE quantify the prediction errors in the same units of the variables.”
MethodInfluentialA Hybrid LSTM Neural Network for Energy Consumption Forecasting of Individual Households
Yan, K., Li, W., Ji, Z., Qi, M., & Du, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2949065 · Cited by 230
Kouadio, L. A., Deo, R. C., Byrareddy, V. M., Adamowski, J., Mushtaq, S., & Nguyen, V. P. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.10.014 · Cited by 232
Méndez-Morales, M., Maathuis, B. H. P., Hein‐Griggs, D., & Alvarado-Gamboa, L.-F. (2020) · Water · doi:10.3390/w12020482 · Cited by 234
Autopilot control unmanned aerial vehicle system for sewage defect detection using deep learning
Pandey, B. K., Pandey, D., & Sahani, K. (2024) · Engineering Reports · doi:10.1002/eng2.12852 · Cited by 220
Temporal Convolutional Networks with RNN approach for chaotic time series prediction
Düdükçü, H. V., Taşkıran, M., Taşkıran, Z. G. Ç., & Yıldırım, T. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109945 · Cited by 226
Arora, A., Arabameri, A., Pandey, M. K., Siddiqui, M. A., Shukla, U. K., Bui, D. T., Mishra, V. N., & Bhardwaj, A. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.141565 · Cited by 232
Ning, Y. D., Kazemi, H., & Tahmasebi, P. (2022) · Computers & Geosciences · doi:10.1016/j.cageo.2022.105126 · Cited by 218
Boukerche, A., Tao, Y., & Sun, P. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107484 · Cited by 215
Lee, S. M., & Lee, D.-H. (2018) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph15071322 · Cited by 215
“…the accuracy of prediction between the models was examined It is determined by the difference between the predicted and actual values measured by the model and is a good indicator of the average error [17,37,38]. by comparing the RMSE values commonly used as a measure of model performance [36].”
MethodPrasad, R., Deo, R. C., Li, Y., & Maraseni, T. (2018) · Geoderma · doi:10.1016/j.geoderma.2018.05.035 · Cited by 208
Sentiment analysis methods, applications, and challenges: A systematic literature review
Mao, Y., Liu, Q., & Zhang, Y. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102048 · Cited by 331
“Such as Mean Square Error(Verma et al., 2017), Ranking loss (Moghaddam and Ester, 2010), Receiver Operator Characteristic (ROC) (Krupinski, 2017), Area Under the Curve (AUC) (Bowers and Zhou, 2019), Root Mean Square Error (RMSE) (Chai and Draxler, 2014a) and mean absolute error (MAE) (Chai and Draxler, 2014b; Willmott…”
Nguyen, H., & Bui, X.-N. (2018) · Natural Resources Research · doi:10.1007/s11053-018-9424-1 · Cited by 206
DeRousseau, M. A., Laftchiev, E., Kasprzyk, J. R., Rajagopalan, B., & Srubar, W. V. (2019) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2019.08.042 · Cited by 213
Zhang, J., Li, D., & Wang, Y. (2020) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2020.120665 · Cited by 204
A Deep Learning Method for Short-Term Residential Load Forecasting in Smart Grid
Ye, H., Zhou, Y., Li, Q., Xu, W., & Zheng, X. (2020) · IEEE Access · doi:10.1109/access.2020.2981817 · Cited by 203
Bartlett, J. D., O’Connor, F. K., Pitchford, N. W., Torres-Ronda, L., & Robertson, S. J. (2016) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.2015-0791 · Cited by 201
PV power forecast using a nonparametric PV model
Almeida, M. P., Perpiñán, O., & Narvarte, L. (2015) · Solar Energy · doi:10.1016/j.solener.2015.03.006 · Cited by 208
“The Mean Absolute Error, MAE, (Equation 10), is less influenced by outliers, although the relative influence depends on the number of samples and the error distribution (Chai and Draxler, 2014).”
BackgroundIndividual mangrove tree measurement using UAV-based LiDAR data: Possibilities and challenges
Yin, D., & Wang, L. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.12.034 · Cited by 197
Generative Semi-supervised Learning for Multivariate Time Series Imputation
Miao, X., Wu, Y., Wang, J., Gao, Y., Mao, X., & Yin, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17086 · Cited by 210
“In the experiments, we utilize root mean square error (RMSE) (Jeffery, Garofalakis, and Franklin 2006) and mean absolute error (MAE) (Chai and Draxler 2014) to measure the imputation performance.”
MethodGervin, K., Salas, L. A., Bakulski, K. M., van Zelm, M. C., Koestler, D. C., Wiencke, J. K., Duijts, L., Moll, H. A., Kelsey, K. T., Kobor, M. S., et al. (2019) · Clinical Epigenetics · doi:10.1186/s13148-019-0717-y · Cited by 196
“For those investigators interested in predicting absolute cell 388 proportions from DNA methylation data, as RMSE measures how accurately the model predicts the 389 response, for this specific scenario with few outliers and a sample size >100, RMSE is the most 390 important criterion for fit[30] and IDOL is the prefer…”
MethodReview of Data Fusion Methods for Real-Time and Multi-Sensor Traffic Flow Analysis
Kashinath, S. A., Mostafa, S. A., Mustapha, A., Mahdin, H., Lim, D., Mahmoud, M. A., Mohammed, M. A., Al‐rimy, B. A. S., Fudzee, M. F. M., & Yang, T. J. (2021) · IEEE Access · doi:10.1109/access.2021.3069770 · Cited by 203
“This evaluation technique is scale-independent [140] and extremely useful for model evaluations and gives the same weight to the error [136].”
BackgroundMethodInfluentialGhorbani, M. A., Deo, R. C., Yaseen, Z. M., et al. (2017) · Theoretical and Applied Climatology · doi:10.1007/s00704-017-2244-0 · Cited by 192
“The results are evaluated using five performance criteria metrics: root mean square error (RMSE), mean absolute error (MAE), Nash-Sutcliffe efficiency (NS), and the Willmott’s Index (WI).”
InfluentialAssessing the accuracy of predictive models for numerical data: Not r nor r2, why not? Then what?
Li, J. (2017) · PLoS ONE · doi:10.1371/journal.pone.0183250 · Cited by 192
A study on water quality prediction by a hybrid CNN-LSTM model with attention mechanism
Yang, Y., Xiong, Q., Wu, C., Zou, Q., Yu, Y., Yi, H., & Gao, M. (2021) · Environmental Science and Pollution Research · doi:10.1007/s11356-021-14687-8 · Cited by 196
Goodness-of-fit criteria for hydrological models: Model calibration and performance assessment
Althoff, D., & Rodrigues, L. N. (2021) · Journal of Hydrology · doi:10.1016/j.jhydrol.2021.126674 · Cited by 193
Ghimire, S., Deo, R. C., Downs, N. J., et al. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2019.01.158 · Cited by 189
A New Criterion for Model Selection
Pham, H. (2019) · Mathematics · doi:10.3390/math7121215 · Cited by 190
H2O2 treatment enhanced the heavy metals removal by manure biochar in aqueous solutions
Wang, Y., & Liu, R. (2018) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2018.02.137 · Cited by 186
Aghighi, H., Azadbakht, M., Ashourloo, D., Shahrabi, H. S., & Radiom, S. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2823361 · Cited by 187
A systematic review of personal thermal comfort models
Martins, L. A., Soebarto, V. I., & Williamson, T. (2021) · Building and Environment · doi:10.1016/j.buildenv.2021.108502 · Cited by 205
Content-Noise Complementary Learning for Medical Image Denoising
Geng, M., Meng, X., Yu, J., Zhu, L., Jin, L., Jiang, Z., Qiu, B., Li, H., Kong, H., Yuan, J., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3113365 · Cited by 190
An overview of performance evaluation metrics for short-term statistical wind power forecasting
Sopeña, J. M. G., Pakrashi, V., & Ghosh, B. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.110515 · Cited by 193
Hossain Lipu, M. S., Hannan, M. A., Hussain, A., Saad, M. H., Ayob, A., & Uddin, M. N. (2019) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2019.2902532 · Cited by 188
“paper, the root mean square error (RMSE) is chosen as a fitness function due to high sample dataset and random distribution of error estimation [24].”
Hai, T., Diop, L., Bodian, A., et al. (2018) · Agricultural Water Management · doi:10.1016/j.agwat.2018.06.018 · Cited by 178
Alslaity, A., & Orji, R. (2022) · Behaviour and Information Technology · doi:10.1080/0144929x.2022.2156387 · Cited by 199
Prasad, R., Deo, R. C., Li, Y., & Maraseni, T. (2017) · Atmospheric Research · doi:10.1016/j.atmosres.2017.06.014 · Cited by 170
Lin, H., Long, E., Ding, X., Diao, H., Chen, Z., Liu, R., Huang, J., Cai, J., Xu, S., Zhang, X., et al. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002674 · Cited by 173
“Evaluation metrics Three evaluation metrics—the coefficient of determination (R(2)), the root mean square error (RMSE), and the mean absolute error (MAE)—were used to assess the performance of the regression algorithm in predicting a targeted SE [12].”
MethodPurity homophily in social networks.
Dehghani, M., JOHNSON, K., Hoover, J., Sagi, E., Garten, J., Parmar, N., Vaisey, S., Iliev, R. I., & Graham, J. (2016) · Journal of Experimental Psychology General · doi:10.1037/xge0000139 · Cited by 169
“To test the hypothesis that purity concerns are the best predictor of social distance, compared to other moral concerns, we compared the root-mean-squared errors (RMSE), a common metric of model fit (Chai & Draxler, 2014; Moriasi et al., 2007), generated from random permutation tests.”
Generating 3D city models without elevation data
Biljecki, F., Ledoux, H., & Stoter, J. (2017) · Computers, Environment and Urban Systems · doi:10.1016/j.compenvurbsys.2017.01.001 · Cited by 165
“The latter metric is sensitive to outliers, but we include it as well following the reasoning of Chai and Draxler (2014).”
BackgroundKim, J. R., Shim, W. H., Yoon, H. M., Hong, S. H., Lee, J. S., Cho, Y. A., & Kim, S. (2017) · American Journal of Roentgenology · doi:10.2214/ajr.17.18224 · Cited by 170
Himeur, Y., Alsalemi, A., Al‐Kababji, A., Bensaali, F., Amira, A., Sardianos, C., Dimitrakopoulos, G. J., & Varlamis, I. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.02.002 · Cited by 177
“The simplest and most popular error are Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) [105], which are defined as follows: where r ui is the actual rating of user u for item i and ˆ r ui is the predicted rating.”
BackgroundInfluentialComparison of unstructured kinetic bacterial growth models.
Muloiwa, M., Nyende‐Byakika, S., & Dinka, M. O. (2020) · South African Journal of Chemical Engineering · doi:10.1016/j.sajce.2020.07.006 · Cited by 166
Best Practices for Elevation-Based Assessments of Sea-Level Rise and Coastal Flooding Exposure
Gesch, D. B. (2018) · Frontiers in Earth Science · doi:10.3389/feart.2018.00230 · Cited by 164
Liu, Y., Jing, W., Wang, Q., & Xia, X. (2020) · Advances in Water Resources · doi:10.1016/j.advwatres.2020.103601 · Cited by 163
Near real-time wind speed forecast model with bidirectional LSTM networks
Joseph, L. P., Deo, R. C., Prasad, R., Salcedo‐Sanz, S., Raj, N., & Soar, J. (2023) · Renewable Energy · doi:10.1016/j.renene.2022.12.123 · Cited by 172
“The absolute computation in 𝑀𝐴𝐸 reduces the biases, making it reliable over 𝑟 and 𝑅𝑀𝑆𝐸 [87].”
Hosseini, M., McNairn, H. E., Merzouki, A., et al. (2015) · Remote Sensing of Environment · doi:10.1016/j.rse.2015.09.002 · Cited by 162
Liu, C., Sun, B. N., Zhang, C., et al. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115383 · Cited by 162
AL‐Musaylh, M. S., Deo, R. C., Li, Y., & Adamowski, J. (2018) · Applied Energy · doi:10.1016/j.apenergy.2018.02.140 · Cited by 162
Abd-Alhamid, F., Kent, M. G., Bennett, C., Calautit, J. K., & Wu, Y. (2019) · Building and Environment · doi:10.1016/j.buildenv.2019.106278 · Cited by 158
Metadata
- Authors
- Chai, T., & Draxler, R. R.
- Affiliations
- National Oceanic and Atmospheric Administration (US)
- Venue
- Geoscientific Model Development
- Publisher
- Copernicus GmbH
- Published
- 2014-06-30
- Volume
- 7
- Issue
- 3
- Pages
- 1247-1250
- ISSN
- 1991-959X (ISSN-L)
- 1991-9603
- OpenAlex
- W2102148524
- Semantic Scholar
- 8c42db1839098b46f18265907464c16e49bb7a1c
- Cited by
- 7,178
- References
- 12
- Citations by year
- 2026: 1015 · 2025: 1120 · 2024: 980 · 2023: 873 · 2022: 751 · 2021: 683 · 2020: 538 · 2019: 411 · 2018: 324 · 2017: 235 · 2016: 175 · 2015: 59 · 2014: 13
- FWCI
- 106.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Geoscientific model development (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Oceanic and Atmospheric Administration (NA09NES4400006)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Water Science and Technology › Hydrology and Watershed Management Studies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Keywords
- root mean square error, mean absolute error, model evaluation metrics, error distribution, Gaussian errors, triangle inequality, distance measures, model evaluation, hydrological model evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-17, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-17 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
