An ensemble learning strategy for panel time series forecasting of excess mortality during the COVID-19 pandemic
Open accessAbstract
Quantifying and analyzing excess mortality in crises such as the ongoing COVID-19 pandemic is crucial for policymakers. Traditional measures fail to take into account differences in the level, long-term secular trends, and seasonal patterns in all-cause mortality across countries and regions. This paper develops and empirically investigates the forecasting performance of a novel, flexible and dynamic ensemble learning with a model selection strategy (DELMS) for the seasonal time series forecasting of monthly respiratory disease death data across a pool of 61 heterogeneous countries. The strategy is based on a Bayesian model averaging (BMA) of heterogeneous time series methods involving both the selection of the subset of best forecasters (model confidence set), the identification of the best holdout period for each contributed model, and the determination of optimal weights using out-of-sample predictive accuracy. A model selection strategy is also developed to remove the outlier models and to combine the models with reasonable accuracy in the ensemble. The empirical outcomes of this large set of experiments show that the accuracy of the BMA approach is significantly improved with DELMS when selecting a flexible and dynamic holdout period and removing the outlier models. Additionally, the forecasts of respiratory disease deaths for each country are highly accurate and exhibit a high correlation (94%) with COVID-19 deaths in 2020.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ensemble methods can provide superior predictive accuracy relative to single learners in several domain-specific areas.
“Empirical studies show that they can provide superior predictive accuracy relative to single learners in several domain-specific areas (Ashofteh et al. 2022; Ayuso et al. 2021; Bravo 2021; Kim and Baek 2022).”
From Modelling Motor Insurance Claim Frequency and Severity Using Gradient Boosting (2023) · cites this paper as Ashofteh et al. 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325 · see citing works
An interactive web-based dashboard to track COVID-19 in real time
Dong, E., Du, H., & Gardner, L. (2020) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30120-1 · Cited by 10,048 · see citing works
Prediction, learning, and games
Cesa-Bianchi, N., & Lugosi, G. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511546921 · Cited by 4,660 · see citing works
Extreme learning machine: a new learning scheme of feedforward neural networks
Huang, G.-B., Zhu, Q.-Y., & Siew, C. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1380068 · Cited by 4,248 · see citing works
Forecasting principles and practice
Hyndman, R. J., & Athanasopoulos, G. (n.d.) · Cited by 4,194 · see citing works
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735 · see citing works
Automatic Time Series Forecasting: The forecast Package for R
Hyndman, R. J., & Khandakar, Y. (2008) · Journal of Statistical Software · doi:10.18637/jss.v027.i03 · Cited by 3,548 · see citing works
Bates, J. M., & Granger, C. W. J. (1969) · Journal of the Operational Research Society · doi:10.1057/jors.1969.103 · Cited by 3,201 · see citing works
Using Bayesian model averaging to calibrate forecast ensembles.
Raftery, A. E., Gneiting, T., Balabdaoui, F., & Polakowski, M. (2005) · Monthly Weather Review · doi:10.1175/mwr2906.1 · Cited by 1,915 · see citing works
Introduction to time series analysis and forecasting
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0946 · Cited by 1,405 · see citing works
Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing
Livera, A. M. D., Hyndman, R. J., & Snyder, R. D. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.tm09771 · Cited by 1,056 · see citing works
Analysis of Financial Time Series
Tsay, R. S. (2010) · Wiley series in probability and statistics · doi:10.1002/9780470644560 · Cited by 1,026 · see citing works
The M4 Competition: 100,000 time series and 61 forecasting methods
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.04.014 · Cited by 1,000 · see citing works
Averages of Forecasts: Some Empirical Results
Makridakis, S., & Winkler, R. L. (1983) · Management Science · doi:10.1287/mnsc.29.9.987 · Cited by 622 · see citing works
Banerjee, A., Pasea, L., Harris, S., Gonzalez-Izquierdo, A., Torralbo, A., Shallcross, L., Noursadeghi, M., Pillay, D., Sebire, N., Holmes, C., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30854-0 · Cited by 518 · see citing works
Islam, N., Shkolnikov, V. M., Acosta, R. J., Klimkin, I., Kawachi, I., Irizarry, R. A., Alicandro, G., Khunti, K., Yates, T., Jdanov, D. A., et al. (2021) · BMJ · doi:10.1136/bmj.n1137 · Cited by 468 · see citing works
Kontis, V., Bennett, J. E., Rashid, T., Parks, R. M., Pearson-Stuttard, J., Guillot, M., Asaria, P., Zhou, B., Battaglini, M., Corsetti, G., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1112-0 · Cited by 410 · see citing works
Persistence in forecasting performance and conditional combination strategies
Aiolfi, M., & Timmermann, A. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2005.07.015 · Cited by 351 · see citing works
Coherent Mortality Forecasting: The Product-Ratio Method With Functional Time Series Models
Hyndman, R. J., Booth, H., & Yasmeen, F. (2012) · Demography · doi:10.1007/s13524-012-0145-5 · Cited by 287 · see citing works
Brazdil, P., Giraud-Carrier, C. G., Soares, C., & Vilalta, R. (2008) · Cognitive technologies · doi:10.1007/978-3-540-73263-1 · Cited by 265 · see citing works
Simple robust averages of forecasts: Some empirical results
Jose, V. R. R., & Winkler, R. L. (2008) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2007.06.001 · Cited by 242 · see citing works
Review of guidelines for the use of combined forecasts
de Menezes, L. M., Bunn, D. W., & Taylor, J. W. (2000) · European Journal of Operational Research · doi:10.1016/s0377-2217(98)00380-4 · Cited by 242 · see citing works
COVID-19: a need for real-time monitoring of weekly excess deaths
Leon, D. A., Школьников, В. М., Smeeth, L. L., Magnus, P. M., Pechholdová, M., & Jarvis, C. I. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30933-8 · Cited by 231 · see citing works
A stacking-based ensemble learning method for earthquake casualty prediction
Cui, S., Yin, Y., Wang, D., Li, Z., & Wang, Y. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.107038 · Cited by 228 · see citing works
Zhang, Z., & Hong, W. (2019) · Nonlinear Dynamics · doi:10.1007/s11071-019-05252-7 · Cited by 226 · see citing works
Aiolfi, M., Capistrán, C., & Timmermann, A. (2010) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780195398649.013.0013 · Cited by 215 · see citing works
Excess mortality during the COVID-19 outbreak in Italy: a two-stage interrupted time-series analysis
Scortichini, M., Schneider, R., de'Donato, F. K., Sario, M. D., Michelozzi, P., Davoli, M., Masselot, P., Sera, F., & Gasparrini, A. (2020) · International Journal of Epidemiology · doi:10.1093/ije/dyaa169 · Cited by 165 · see citing works
Park, H. J., Kim, Y., & Kim, H. Y. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108106 · Cited by 150 · see citing works
Andrawis, R. R., Atiya, A. F., & El‐Shishiny, H. (2011) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2010.09.005 · Cited by 145 · see citing works
Forecast CombinationsPreprint
Aiolfi, M., Capistrán, C., & Timmermann, A. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1609530 · Cited by 145 · see citing works
Short-Term Forecasting for Energy Consumption through Stacking Heterogeneous Ensemble Learning Model
Khairalla, M. A., Ning, X., AL-Jallad, N. T., & El-Faroug, M. O. (2018) · Energies · doi:10.3390/en11061605 · Cited by 79 · see citing works
Some theoretical results on forecast combinations
Felix Chan, A., & Pauwels, L. L. (2017) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2017.08.005 · Cited by 75 · see citing works
Weighted ensemble of statistical models
Pawlikowski, M., & Chorowska, A. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.03.019 · Cited by 71 · see citing works
Model Confidence Sets and forecast combination
Samuels, J. D., & Sekkel, R. (2016) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2016.07.004 · Cited by 65 · see citing works
A conservative approach for online credit scoring
Ashofteh, A., & Bravo, J. M. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114835 · Cited by 58 · see citing works
Self-citationHassani, H., & Mahmoudvand, R. (2018) · Palgrave Macmillan UK eBooks · doi:10.1057/978-1-137-40951-5 · Cited by 54 · see citing works
Mathonsi, T., & van Zyl, T. L. (2021) · Forecasting · doi:10.3390/forecast4010001 · Cited by 46 · see citing works
Automatic Indexation of the Pension Age to Life Expectancy: When Policy Design Matters
Ayuso, M., Bravo, J. M., Holzmann, R., & Palmer, E. (2021) · Risks · doi:10.3390/risks9050096 · Cited by 44 · see citing works
Self-citationA study on the quality of novel coronavirus (COVID-19) official datasets
Ashofteh, A., & Bravo, J. M. (2020) · Statistical Journal of the IAOS · doi:10.3233/sji-200674 · Cited by 40 · see citing works
Self-citationEnsemble approach for time series analysis in demand forecasting: Ensemble learning
Akyuz, A. O., Uysal, M., Bulbul, B. A., & Uysal, M. O. (2017) · 2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA) · doi:10.1109/inista.2017.8001123 · Cited by 38 · see citing works
A two-layer nested heterogeneous ensemble learning predictive method for COVID-19 mortality
Cui, S., Wang, Y., Wang, D., Sai, Q., Huang, Z., & Cheng, T. C. E. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107946 · Cited by 34 · see citing works
Wang, Z., Chen, H., Zhu, J., & Ding, Z. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108110 · Cited by 33 · see citing works
Dynamic selection of forecast combiners
Sergio, A. T., Lima, T. P., & Ludermir, T. B. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.08.072 · Cited by 30 · see citing works
Pricing longevity derivatives via Fourier transforms
Bravo, J. M., & Nunes, J. P. V. (2020) · Insurance Mathematics and Economics · doi:10.1016/j.insmatheco.2020.10.008 · Cited by 30 · see citing works
Self-citationExcess All-Cause Deaths during Coronavirus Disease Pandemic, Japan, January–May 20201
Kawashima, T., Nomura, S., Tanoue, Y., Yoneoka, D., Eguchi, A., Ng, C. F. S., Matsuura, K., Shi, S., Makiyama, K., Uryu, S., et al. (2021) · Emerging infectious diseases · doi:10.3201/eid2703.203925 · Cited by 30 · see citing works
Estimation of Excess All-cause Mortality during COVID-19 Pandemic in Korea
Shin, M. S., Sim, B., Jang, W. M., & Lee, J. Y. (2021) · Journal of Korean Medical Science · doi:10.3346/jkms.2021.36.e280 · Cited by 25 · see citing works
Comparative Study on Univariate Forecasting Methods for Meteorological Time Series
Phan, T.-T.-H., Caillault, É. P., & Bigand, A. (2018) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2018.8553576 · Cited by 21 · see citing works
Forecasting the Retirement Age: A Bayesian Model Ensemble Approach
Bravo, J. M., & Ayuso, M. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-72657-7_12 · Cited by 21 · see citing works
Self-citationFong, S. J., Li, G., Dey, N., Crespo, R. A. G., & Herrera‐Viedma, E. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106282 · Cited by 18 · see citing works
The manipulation of official statistics as corruption and ways of understanding it
Georgiou, A. V. (2021) · Statistical Journal of the IAOS · doi:10.3233/sji-200667 · Cited by 17 · see citing works
A comparative study for predictive monitoring of COVID-19 pandemic
Fatimah, B., Aggarwal, P., Singh, P., & Gupta, A. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.108806 · Cited by 15 · see citing works
Immunization Strategies for Funding Multiple Inflation-Linked Retirement Income Benefits
Simões, C. P. G., Oliveira, L., & Bravo, J. M. (2021) · Risks · doi:10.3390/risks9040060 · Cited by 14 · see citing works
Self-citationExcess All-Cause Mortality During the COVID-19 Outbreak in Japan
Yorifuji, T., Matsumoto, N., & Takao, S. (2020) · Journal of Epidemiology · doi:10.2188/jea.je20200492 · Cited by 13 · see citing works
Ashofteh, A., & Bravo, J. M. (2021) · Statistical Journal of the IAOS · doi:10.3233/sji-210841 · Cited by 13 · see citing works
Self-citationWijesinghe, G. W. R. I., & Rathnayaka, R. M. K. T. (2020) · International Conference on Information Technology Research (ICITR) · doi:10.1109/icitr51448.2020.9310826 · Cited by 12 · see citing works
Life Table Forecasting in COVID-19 Times: An Ensemble Learning Approach
Ashofteh, A., & Bravo, J. M. (2021) · Iberian Conference on Information Systems and Technologies (CISTI) · doi:10.23919/cisti52073.2021.9476583 · Cited by 11 · see citing works
Self-citationForecasting Subnational Demographic Data using Seasonal Time Series Methods
Bravo, J. M., & Coelho, E. (2019) · Journal of the Association for Information Systems · Cited by 9 · see citing works
Self-citationMethods, Challenges, and Practical Issues of COVID-19 Projection: A Data Science Perspective
Kim, M., Gu, Z., Yu, S., Wang, G., & Wang, L. (2021) · Journal of Data Science · doi:10.6339/21-jds1013 · Cited by 8 · see citing works
Flagged first, then by citations
4 of the 62 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [37]A. Ashofteh, Big Data for Credit Risk Analysis: Efficient Machine Learning Models Using PySpark, in: Proceedings of SIMSTAT 2019-10th International Workshop on Simulation and Statistics, 2019.
Cited by
Sorted by citations
Intergenerational actuarial fairness when longevity increases: Amending the retirement age
Bravo, J. M., Ayuso, M., Holzmann, R., & Palmer, E. (2023) · Insurance Mathematics and Economics · doi:10.1016/j.insmatheco.2023.08.007 · Cited by 35 · see citing works
Self-citationModelling Motor Insurance Claim Frequency and Severity Using Gradient Boosting
Clemente, C., Guerreiro, G. R., & Bravo, J. M. (2023) · Risks · doi:10.3390/risks11090163 · Cited by 28 · see citing works
Self-citationWang, X., Tian, Y., & Liu, C. (2023) · Frontiers in Environmental Science · doi:10.3389/fenvs.2023.1086300 · Cited by 23 · see citing works
Wang, L., Li, N., Xie, M., & Wu, L. (2023) · Nonlinear Dynamics · doi:10.1007/s11071-023-08296-y · Cited by 11 · see citing works
Soto‐Ferrari, M., Carrasco‐Pena, A., & Prieto, D. (2024) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-024-02889-8 · Cited by 11 · see citing works
Assessing the effectiveness of recent pension reforms: The French experiment
Bravo, J. M., Ayuso, M., & Mekkaoui, N. E. (2025) · Socio-Economic Planning Sciences · doi:10.1016/j.seps.2025.102335 · Cited by 5 · see citing works
Self-citationEnsemble Methods for Stock Market Prediction
Bravo, J. M. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-74643-7_31 · Cited by 3 · see citing works
Self-citationA novel online multi-task learning for COVID-19 multi-output spatio-temporal prediction
Wu, Z., Loo, C. K., Obaidellah, U. H. B., & Pasupa, K. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e18771 · Cited by 3 · see citing works
Credit Risk Scoring: A Stacking Generalization Approach
Raimundo, B., & Bravo, J. M. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-45642-8_38 · Cited by 3 · see citing works
Self-citationA hybrid recursive direct system for multi-step mortality rate forecasting
Duarte, F. C. D. L., Neto, P. S. G. D. M., & Firmino, P. R. A. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06182-x · Cited by 2 · see citing works
Prakash, S., Jalal, A. S., & Pathak, P. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02417-8 · Cited by 1 · see citing works
Guo, Z., Gong, H., Li, Y., Tao, C., & Zhou, L. (2022) · Research Square · doi:10.21203/rs.3.rs-2259096/v1 · Cited by 1 · see citing works
Karcıoğlu, A. A. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4594978
Ashofteh, A. (2023) · Springer proceedings in mathematics & statistics · doi:10.1007/978-3-031-39864-3_2
Self-citationLinear сombination of AR(1), MA(q) models with minimal forecast variance
Musatov, D., & Petrusevich, D. A. (2025) · AIP conference proceedings · doi:10.1063/5.0257103
dos Santos, J. L. B., Filho, E. A., Oliveira, M. G. F. D. M., Oliveira, A., & Oliveira, G. H. F. D. M. (2024) · doi:10.5753/eniac.2024.243677
Financing Long-Term Care in Portugal: Is There a Role for Home Equity Release Schemes?
Bravo, J. M., & Gonçalves, J. (2025) · Springer proceedings in business and economics · doi:10.1007/978-3-031-94487-1_9
Self-citationA Stacking Generalization Approach
Raimundo, B., & Bravo, J. M. (2024) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Self-citationPricing e-forwards: An Investigation Using Bayesian Model Ensembles and Stacking Regression
Bravo, J. M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-12882-9_36
Self-citationAccurate Temperature Prediction for Delhi Using Ensemble Deep Learning–LLM Architectures
Kaushik, V., Tyagi, M., Abhishek, Kumar, P. A., & Kumar, A. (2026) · International Conference on Innovations in Computational Intelligence (ICICI) · doi:10.1109/icici68867.2026.11565035
Convolution transformation for sufficient dimension reduction in time series
Alwis, T. P. D., Samadi, S. Y., & Ashofteh, A. (2026) · Japanese Journal of Statistics and Data Science · doi:10.1007/s42081-026-00364-y
Self-citation
- Authors
- Ashofteh, A., Bravo, J. M., & Ayuso, M.
- Affiliations
- Universidade Nova de Lisboa (PT)
- Iscte – Instituto Universitário de Lisboa (PT)
- Universitat de Barcelona (ES)
- Corresponding author
- Ashofteh, Afshin
- Venue
- Applied Soft Computing
- Publisher
- Elsevier BV
- Published
- 2022-10
- Volume
- 128
- Pages
- 109422
- ISSN
- 1568-4946 (ISSN-L)
- PMC
- PMC9341166
- OpenAlex
- W4289260730
- Cited by
- 21
- References
- 62
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 7 · 2023: 7 · 2022: 1
- FWCI
- 2.77
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Applied Soft Computing (publishedVersion, cc-by)
- SSRN Electronic Journal (acceptedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, other-oa)
- Funders
- Ministerio de Ciencia e Innovación (PID2019-105986GB-C21)
- National Institute of Science and Technology National Institute of Photonics Applied to Cell Biology
- Fundação para a Ciência e a Tecnologia
- Generalitat de Catalunya (2020-PANDE-00074)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Physical Sciences › Mathematics › Modeling and Simulation › COVID-19 epidemiological studies
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Keywords
- excess mortality, COVID-19 pandemic, ensemble learning, Bayesian model averaging, model confidence set, seasonal time series forecasting, all-cause mortality, model selection strategy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
