The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
The accuracy of extrapolation (time series) methods: Results of a forecasting competition
Abstract
In the last few decades many methods have become available for forecasting. As always, when alternatives exist, choices need to be made so that an appropriate forecasting method can be selected and used for the specific situation being considered. This paper reports the results of a forecasting competition that provides information to facilitate such choice. Seven experts in each of the 24 methods forecasted up to 1001 series for six up to eighteen time horizons. The results of the competition are presented in this paper whose purpose is to provide empirical evidence about differences found to exist among the various extrapolative (time series) methods used in the competition.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Time Series Analysis: Forecasting and Control
Geurts, M., Box, G. E. P., & Jenkins, G. M. (1977) · Journal of Marketing Research · doi:10.2307/3150485 · Cited by 19,186
Nonparametric Statistical Methods
(1999) · Technometrics · doi:10.1080/00401706.1999.10485954 · Cited by 6,090
Bates, J. M., & Granger, C. W. J. (1969) · Journal of the Operational Research Society · doi:10.1057/jors.1969.103 · Cited by 3,201
Forecasting methods and applications
(1993) · Computers & Operations Research · doi:10.1016/0305-0548(93)90040-p · Cited by 2,393
Time Series Analysis Forecasting and Control
Bartholomew, D. J. (1971) · Journal of the Operational Research Society · doi:10.1057/jors.1971.52 · Cited by 1,792
Forecasting Economic Time Series
(2000) · The Economic Journal · doi:10.1111/1468-0297.00513 · Cited by 1,768
Forecasting: Methods and Applications.
Wichern, D. W., Makridakis, S., & Wheelwright, S. C. (1979) · Journal of the American Statistical Association · doi:10.2307/2287014 · Cited by 1,237
Self-citationExperience with Forecasting Univariate Time Series and the Combination of Forecasts
Newbold, P., & Granger, C. W. J. (1974) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344546 · Cited by 848
Forecasting Economic Time Series
Clements, M. P., & Hendry, D. F. (1977) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 710
Harrison, P. J., & Stevens, C. F. (1976) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1976.tb01586.x · Cited by 649
Cited by
Another look at measures of forecast accuracy
Hyndman, R. J., & Koehler, A. B. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.03.001 · Cited by 5,757
“Nevertheless, the MSE was used by Makridakis et al. (1982), in the M-competition.”
BackgroundMethodTime series forecasting using a hybrid ARIMA and neural network model
Zhang, G. P. (2003) · Neurocomputing · doi:10.1016/s0925-2312(01)00702-0 · Cited by 4,428
Forecasting with artificial neural networks:
Zhang, G., Eddy Patuwo, B., & Y. Hu, M. (1998) · International Journal of Forecasting · doi:10.1016/s0169-2070(97)00044-7 · Cited by 4,229
Automatic Time Series Forecasting: The forecast Package for R
Hyndman, R. J., & Khandakar, Y. (2008) · Journal of Statistical Software · doi:10.18637/jss.v027.i03 · Cited by 3,511
Damage propagation modeling for aircraft engine run-to-failure simulation
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008) · 2008 International Conference on Prognostics and Health Management · doi:10.1109/phm.2008.4711414 · Cited by 1,954
The M3-Competition: results, conclusions and implications
Makridakis, S., & Hibon, M. (2000) · International Journal of Forecasting · doi:10.1016/s0169-2070(00)00057-1 · Cited by 1,695
Self-citationEXPERT SYSTEMS WITH APPLICATIONS
(2005) · Expert Systems with Applications · doi:10.1016/j.eswa.2004.10.010 · Cited by 1,656
25 years of time series forecasting
De Gooijer, J. G., & Hyndman, R. J. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.01.001 · Cited by 1,587
Combining forecasts: A review and annotated bibliography
(1990) · Long Range Planning · doi:10.1016/0024-6301(90)90316-v · Cited by 1,465
Exponential smoothing: The state of the art
Gardner, E. S. (1985) · Journal of Forecasting · doi:10.1002/for.3980040103 · Cited by 1,458
Error measures for generalizing about forecasting methods: Empirical comparisons
Armstrong, J. S., & Collopy, F. (1992) · International Journal of Forecasting · doi:10.1016/0169-2070(92)90008-w · Cited by 1,396
Chapter 4 Forecast Combinations
Timmermann, A. (2006) · Handbook of Economic Forecasting · doi:10.1016/s1574-0706(05)01004-9 · Cited by 1,314
Making and Evaluating Point Forecasts
Gneiting, T. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.r10138 · Cited by 1,154
A new metric of absolute percentage error for intermittent demand forecasts
Kim, S., & Kim, H. (2016) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2015.12.003 · Cited by 1,146
A state space framework for automatic forecasting using exponential smoothing methods
Hyndman, R. J., Koehler, A. B., Snyder, R. D., & Grose, S. (2002) · International Journal of Forecasting · doi:10.1016/s0169-2070(01)00110-8 · Cited by 1,066
Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing
De Livera, A. M., Hyndman, R. J., & Snyder, R. D. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm09771 · Cited by 1,049
Principles of Forecasting: A Handbook for Researchers and Practitioners
(2001) · International Series in Operations Research & Management Science · doi:10.1007/978-0-306-47630-3 · Cited by 1,041
Out-of-sample tests of forecasting accuracy: an analysis and review
Tashman, L. J. (2000) · International Journal of Forecasting · doi:10.1016/s0169-2070(00)00065-0 · Cited by 1,021
Neural network forecasting for seasonal and trend time series
Zhang, G. P., & Qi, M. (2005) · European Journal of Operational Research · doi:10.1016/j.ejor.2003.08.037 · Cited by 986
The M4 Competition: 100,000 time series and 61 forecasting methods
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.04.014 · Cited by 969
Self-citationExponential smoothing: The state of the art—Part II
Gardner, E. S. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.03.005 · Cited by 948
Forecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Ben Taieb, S., Bergmeir, C., Bessa, R. J., Bijak, J., Boylan, J. E., et al. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 921
Self-citationAccuracy measures: theoretical and practical concerns
Makridakis, S. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90079-3 · Cited by 869
Self-citationTime series prediction: Forecasting the future and understanding the past
Makridakis, S. (1994) · International Journal of Forecasting · doi:10.1016/0169-2070(94)90077-9 · Cited by 841
Self-citationInternational journal of public administration
Armstrong, J. S., & Brouthers, L. E. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90017-9 · Cited by 743
Jung, M., Reichstein, M., & Bondeau, A. (2009) · Biogeosciences · doi:10.5194/bg-6-2001-2009 · Cited by 712
The M4 Competition: Results, findings, conclusion and way forward
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2018) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2018.06.001 · Cited by 629
Self-citationAverages of Forecasts: Some Empirical Results
Makridakis, S., & Winkler, R. L. (1983) · Management Science · doi:10.1287/mnsc.29.9.987 · Cited by 621
Self-citationEvidence on Structural Instability in Macroeconomic Time Series Relations
Stock, J. H., & Watson, M. W. (1996) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1996.10524626 · Cited by 617
Evidence on Structural Instability in Macroeconomic Time Series Relations
Stock, J., & Watson, M. (1994) · National Bureau of Economic Research · doi:10.3386/t0164 · Cited by 600
Forecasting the novel coronavirus COVID-19
Petropoulos, F., & Makridakis, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0231236 · Cited by 567
Self-citationM5 accuracy competition: Results, findings, and conclusions
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.013 · Cited by 554
Self-citationJudgmental forecasting: A review of progress over the last 25years
Lawrence, M., Goodwin, P., O'Connor, M., & Önkal, D. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.03.007 · Cited by 534
Metrics for evaluating performance of prognostic techniques
Saxena, A., Celaya, J., Balaban, E., Goebel, K., Saha, B., Saha, S., & Schwabacher, M. (2008) · 2008 International Conference on Prognostics and Health Management · doi:10.1109/phm.2008.4711436 · Cited by 518
The accuracy of intermittent demand estimates
Syntetos, A. A., & Boylan, J. E. (2005) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2004.10.001 · Cited by 489
Evaluating time series forecasting models: an empirical study on performance estimation methods
Cerqueira, V., Torgo, L., & Mozetič, I. (2020) · Machine Learning · doi:10.1007/s10994-020-05910-7 · Cited by 485
Neural Network Models for Time Series Forecasts
Hill, T., O'Connor, M., & Remus, W. (1996) · Management Science · doi:10.1287/mnsc.42.7.1082 · Cited by 468
How effective are neural networks at forecasting and prediction? A review and evaluation
Adya, M., & Collopy, F. (1998) · Journal of Forecasting · doi:10.1002/(sici)1099-131x(1998090)17:5/6<481::aid-for709>3.0.co;2-q · Cited by 451
Artificial neural network models for forecasting and decision making
Hill, T., Marquez, L., O'Connor, M., & Remus, W. (1994) · International Journal of Forecasting · doi:10.1016/0169-2070(94)90045-0 · Cited by 450
Bandara, K., Bergmeir, C., & Smyl, S. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112896 · Cited by 435
A Comparison of Linear and Nonlinear Univariate Models for Forecasting Macroeconomic Time Series
Stock, J. H., & Watson, M. W. (1999) · Cointigration, Causality, and forecating · doi:10.1093/oso/9780198296836.003.0001 · Cited by 433
Journal of Business and Economic Statistics
Fildes, R. (1994) · International Journal of Forecasting · doi:10.1016/0169-2070(94)90039-6 · Cited by 413
Self-citationCombining neural network model with seasonal time series ARIMA model
Tseng, F.-M., Yu, H.-C., & Tzeng, G.-H. (2002) · Technological Forecasting and Social Change · doi:10.1016/s0040-1625(00)00113-x · Cited by 345
Prediction of hourly solar radiation using a novel hybrid model of ARMA and TDNN
Ji, W., & Chee, K. C. (2011) · Solar Energy · doi:10.1016/j.solener.2011.01.013 · Cited by 338
A simulation study of artificial neural networks for nonlinear time-series forecasting
Zhang, G. P., Patuwo, B. E., & Hu, M. Y. (2001) · Computers & Operations Research · doi:10.1016/s0305-0548(99)00123-9 · Cited by 336
Holt-Winters Forecasting: Some Practical Issues
Chatfield, C., & Yar, M. (1988) · The Statistician · doi:10.2307/2348687 · Cited by 330
Harvey, A. C., & Durbin, J. (1986) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981553 · Cited by 329
To combine or not to combine: selecting among forecasts and their combinations
Hibon, M., & Evgeniou, T. (2005) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2004.05.002 · Cited by 325
Self-citationJournal of business and economic statistics
Fildes, R. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90124-0 · Cited by 323
Self-citationCombining expert forecasts: Can anything beat the simple average?
Genre, V., Kenny, G., Meyler, A., & Timmermann, A. (2013) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2012.06.004 · Cited by 313
The application of neural networks to forecast fuzzy time series
Huarng, K., & Yu, T. H.-K. (2006) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/j.physa.2005.08.014 · Cited by 308
Probability Forecasting in Meteorology
Murphy, A. H., & Winkler, R. L. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478075 · Cited by 300
Self-citationMethods for combining the outputs of different rainfall–runoff models
Shamseldin, A. Y., O'Connor, K. M., & Liang, G. C. (1997) · Journal of Hydrology · doi:10.1016/s0022-1694(96)03259-3 · Cited by 296
Exponential smoothing with a damped multiplicative trend
Taylor, J. W. (2003) · International Journal of Forecasting · doi:10.1016/s0169-2070(03)00003-7 · Cited by 294
Neural network ensemble operators for time series forecasting
Kourentzes, N., Barrow, D. K., & Crone, S. F. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.12.011 · Cited by 292
Forecasting and operational research: a review
Fildes, R., Nikolopoulos, K., Crone, S. F., & Syntetos, A. A. (2008) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2602597 · Cited by 287
Self-citationA comparison of forecasting methods for hotel revenue management
Weatherford, L. R., & Kimes, S. E. (2003) · International Journal of Forecasting · doi:10.1016/s0169-2070(02)00011-0 · Cited by 287
Forecast combinations: An over 50-year review
Wang, X., Hyndman, R. J., Li, F., & Kang, Y. (2023) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2022.11.005 · Cited by 280
Exponential smoothing model selection for forecasting
Billah, B., King, M. L., Snyder, R. D., & Koehler, A. B. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2005.08.002 · Cited by 279
Dalrymple, D. J. (1987) · International Journal of Forecasting · doi:10.1016/0169-2070(87)90031-8 · Cited by 276
Forecasting Economic Time Series With Structural and Box-Jenkins Models: A Case Study
Harvey, A. C., & Todd, P. H. J. (1983) · Journal of Business & Economic Statistics · doi:10.1080/07350015.1983.10509355 · Cited by 271
Crone, S. F., Hibon, M., & Nikolopoulos, K. (2011) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2011.04.001 · Cited by 271
Self-citationThe evaluation of extrapolative forecasting methods
Fildes, R. (1992) · International Journal of Forecasting · doi:10.1016/0169-2070(92)90009-x · Cited by 265
Self-citationA new accuracy measure based on bounded relative error for time series forecasting
Chen, C., Twycross, J., & Garibaldi, J. M. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174202 · Cited by 264
The M2-competition: A real-time judgmentally based forecasting study
Makridakis, S., Chatfield, C., Hibon, M., Lawrence, M., Mills, T., Ord, K., & Simmons, L. F. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90044-n · Cited by 263
Self-citationKim, J., Kim, H., Kim, H., Lee, D., & Yoon, S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11223-9 · Cited by 263
A combined forecasting approach based on fuzzy soft sets
Xiao, Z., Gong, K., & Zou, Y. (2009) · Journal of Computational and Applied Mathematics · doi:10.1016/j.cam.2008.09.033 · Cited by 261
A novel nonlinear ensemble forecasting model incorporating GLAR and ANN for foreign exchange rates
Yu, L., Wang, S., & Lai, K. K. (2005) · Computers & Operations Research · doi:10.1016/j.cor.2004.06.024 · Cited by 259
A neural network ensemble method with jittered training data for time series forecasting
Zhang, G. P. (2007) · Information Sciences · doi:10.1016/j.ins.2007.06.015 · Cited by 196
The Impact of Empirical Accuracy Studies on Time Series Analysis and Forecasting
Fildes, R., & Makridakis, S. (1995) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1403481 · Cited by 172
Self-citationReproducibility in forecasting research
Boylan, J. E., Goodwin, P., Mohammadipour, M., & Syntetos, A. A. (2015) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2014.05.008 · Cited by 47
Theory and practice of multivariate arma forecasting
Riise, T., & Tjozstheim, D. (1984) · Journal of Forecasting · doi:10.1002/for.3980030308 · Cited by 41
Fitting Time Series Models by Minimizing Multistep-ahead Errors: A Frequency Domain Approach
Haywood, J., & Tunnicliffe Wilson, G. (1997) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00066 · Cited by 29
“We use time series number 596 from the collection of 1001 time series used by Makridakis et al. (1982) for a forecasting competition.”
MethodAn overview of revenue management and dynamic pricing models in hotel business
Bandalouski, A. M., Kovalyov, M. Y., Pesch, E., & Tarim, S. A. (2018) · RAIRO - Operations Research · doi:10.1051/ro/2018001 · Cited by 25
Mathematical Definition, Mapping, and Detection of (Anti)Fragility
Taleb, N. N., & Douady, R. (2012) · arXiv · doi:10.48550/arxiv.1208.1189
Metadata
- Authors
- Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Parzen, E., & Winkler, R.
- Venue
- Journal of Forecasting
- Publisher
- Wiley
- Published
- 1982-04
- Volume
- 1
- Issue
- 2
- Pages
- 111-153
- ISSN
- 0277-6693 (ISSN-L)
- 1099-131X
- OpenAlex
- W2154326182
- Cited by
- 1,462
- References
- 22
- FWCI
- 44.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.