The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Comprehensive Review of Neural Network-Based Prediction Intervals and New Advances
Abstract
This paper evaluates the four leading techniques proposed in the literature for construction of prediction intervals (PIs) for neural network point forecasts. The delta, Bayesian, bootstrap, and mean-variance estimation (MVE) methods are reviewed and their performance for generating high-quality PIs is compared. PI-based measures are proposed and applied for the objective and quantitative assessment of each method's performance. A selection of 12 synthetic and real-world case studies is used to examine each method's performance for PI construction. The comparison is performed on the basis of the quality of generated PIs, the repeatability of the results, the computational requirements and the PIs variability with regard to the data uncertainty. The obtained results in this paper indicate that: 1) the delta and Bayesian methods are the best in terms of quality and repeatability, and 2) the MVE and bootstrap methods are the best in terms of low computational load and the width variability of PIs. This paper also introduces the concept of combinations of PIs, and proposes a new method for generating combined PIs using the traditional PIs. Genetic algorithm is applied for adjusting the combiner parameters through minimization of a PI-based cost function subject to two sets of restrictions. It is shown that the quality of PIs produced by the combiners is dramatically better than the quality of PIs obtained from each individual method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,010
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Neural networks for pattern recognition
(1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,538
Genetic Algorithms in Search, Optimization and Machine Learning
Goldberg, D. E., & Robson, D. (1988) · Cited by 18,391
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Goldberg, D. E., & Shakespeare, W. (2002) · Cited by 16,088
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
An Introduction to Genetic Algorithms
Mitchell, M. (1996) · doi:10.7551/mitpress/3927.001.0001 · Cited by 11,340
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Training feedforward networks with the Marquardt algorithm
Hagan, M. T., & Menhaj, M. B. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.329697 · Cited by 7,701
Bootstrap Methods and their Application
Davison, A. C., & Hinkley, D. V. (1997) · doi:10.1017/cbo9780511802843 · Cited by 7,350
Bootstrap Methods and Their Application
Loughin, T. M., Davison, A. C., & Hinkley, D. V. (1999) · Journal of the American Statistical Association · doi:10.2307/2669707 · Cited by 5,418
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260
Neural Information Processing Systems
(2018) · The Deep Learning Revolution · doi:10.7551/mitpress/11474.003.0014 · Cited by 3,730
25 years of time series forecasting
De Gooijer, J. G., & Hyndman, R. J. (2006) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2006.01.001 · Cited by 1,587
Neural networks and statistical techniques: A review of applications
Paliwal, M., & Kumar, U. A. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2007.10.005 · Cited by 792
Lower Upper Bound Estimation Method for Construction of Neural Network-Based Prediction Intervals
Khosravi, A., Nahavandi, S., Creighton, D., & Atiya, A. F. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2096824 · Cited by 758
Self-citationThe Evidence Framework Applied to Classification Networks
MacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.5.720 · Cited by 722
Estimating the mean and variance of the target probability distribution
Nix, D. A., & Weigend, A. S. (1994) · Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94) · doi:10.1109/icnn.1994.374138 · Cited by 700
Genetic Algorithms: Principles and Perspectives: A Guide to GA Theory
Reeves, C. R., & Rowe, J. E. (2002) · Thammasat University Digital Collections · Cited by 557
Neural Network Applications in Power Electronics and Motor Drives—An Introduction and Perspective
Bose, B. K. (2007) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2006.888683 · Cited by 520
Optimal Linear Combinations of Neural Networks
Hashem, S. (1997) · Neural Networks · doi:10.1016/s0893-6080(96)00098-6 · Cited by 499
Machine learning approaches for estimation of prediction interval for the model output
Shrestha, D. L., & Solomatine, D. P. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.01.012 · Cited by 428
Azlan Hussain, M. (1999) · Artificial Intelligence in Engineering · doi:10.1016/s0954-1810(98)00011-9 · Cited by 381
Prediction Intervals for Artificial Neural Networks
Hwang, J. T. G., & Ding, A. A. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474027 · Cited by 341
A constructive algorithm for training cooperative neural network ensembles
Islam, M. M., Yao, X., & Murase, K. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.813832 · Cited by 294
Practical Confidence and Prediction Intervals
Heskes, T. (1996) · doi:10.5555/2998981.2999006 · Cited by 288
A Comparison of Some Error Estimates for Neural Network Models
Tibshirani, R. (1996) · Neural Computation · doi:10.1162/neco.1996.8.1.152 · Cited by 285
Construction of Optimal Prediction Intervals for Load Forecasting Problems
Khosravi, A., Nahavandi, S., & Creighton, D. (2010) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2010.2042309 · Cited by 267
Self-citationDAISY : A database for identification of systems
Moor, B. D., Gersem, P. D., Schutter, B. D., & Favoreel, W. (1997) · Journal A · Cited by 264
Confidence estimation methods for neural networks: a practical comparison
Papadopoulos, G., Edwards, P. J., & Murray, A. F. (2001) · IEEE Transactions on Neural Networks · doi:10.1109/72.963764 · Cited by 206
A Statistical Approach for Interval Forecasting of the Electricity Price
Zhao, J. H., Dong, Z. Y., Xu, Z., & Wong, K. P. (2008) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2008.919309 · Cited by 184
Prediction Intervals for Neural Networks via Nonlinear Regression
Veaux, R. D. D., Schumi, J., Schweinsberg, J., & Ungar, L. (1998) · Technometrics · doi:10.1080/00401706.1998.10485556 · Cited by 183
Prediction Intervals to Account for Uncertainties in Travel Time Prediction
Khosravi, A., Mazloumi, E., Nahavandi, S., Creighton, D., & van Lint, J. W. C. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2106209 · Cited by 183
Self-citationClinical Applications of Artificial Neural Networks
(2001) · Cambridge University Press eBooks · doi:10.1017/cbo9780511543494 · Cited by 155
Bayesian committee of neural networks to predict travel times with confidence intervals
van Hinsbergen, C. P. I., van Lint, H., & van Zuylen, H. J. (2009) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2009.04.007 · Cited by 152
Evolving artificial neural network ensembles
Yao, X., & Islam, M. M. (2008) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2007.913386 · Cited by 144
Confidence intervals and prediction intervals for feedforward neural networks
Dybowski, R., & Roberts, S. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9780511543494.013 · Cited by 134
Benoit, D. F., & den Poel, D. V. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.01.031 · Cited by 132
A prediction interval-based approach to determine optimal structures of neural network metamodels
Khosravi, A., Nahavandi, S., & Creighton, D. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.07.059 · Cited by 123
Self-citationNew training strategies for constructive neural networks with application to regression problems
Ma, L., & Khorasani, K. (2004) · Neural Networks · doi:10.1016/j.neunet.2004.02.002 · Cited by 118
A genetic algorithm-based method for improving quality of travel time prediction intervals
Khosravi, A., Mazloumi, E., Nahavandi, S., Creighton, D., & van Lint, H. (2011) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2011.04.002 · Cited by 92
Self-citationConstruction of confidence intervals for neural networks based on least squares estimation
Rivals, I., & Personnaz, L. (2000) · Neural Networks · doi:10.1016/s0893-6080(99)00080-5 · Cited by 86
Forecasting nonlinear time series with neural network sieve bootstrap
Giordano, F., Rocca, M. L., & Perna, C. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.03.003 · Cited by 77
Using Combined Forecasts with Changing Weights for Electricity Demand Profiling
Taylor, J. W., & Majithia, S. (2000) · Journal of the Operational Research Society · doi:10.1057/palgrave.jors.2600856 · Cited by 62
Uncertainty analysis of a neural network used for fatigue lifetime prediction
Pierce, G., Worden, K., & Bezazi, A. (2008) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2007.12.004 · Cited by 53
Neural network modeling with confidence bounds: a case study on the solder paste deposition process
Ho, S. L., Xie, M., Tang, L. C., Xu, K. J., & Goh, T. N. (2001) · IEEE Transactions on Electronics Packaging Manufacturing · doi:10.1109/6104.980042 · Cited by 42
Backpropagation of pseudoerrors: neural networks that are adaptive to heterogeneous noise
Ding, A. A., & He, X. (2003) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2003.809428 · Cited by 38
Halberg, A., Teigen, K. H., & Fostervold, K. I. (2009) · Acta Psychologica · doi:10.1016/j.actpsy.2009.07.007 · Cited by 9
Cited by
Electricity price forecasting: A review of the state-of-the-art with a look into the future
Weron, R. (2014) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2014.08.008 · Cited by 1,646
Ahmed, R., Sreeram, V., Mishra, Y., & Arif, M. D. (2020) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2020.109792 · Cited by 1,259
“which have demonstrated utility under varied industrial applications [201,207,209].”
BackgroundMethodInfluentialA review of wind speed and wind power forecasting with deep neural networks
Wang, Y., Zou, R., Liu, F., Zhang, L., & Liu, Q. (2021) · Applied Energy · doi:10.1016/j.apenergy.2021.117766 · Cited by 959
Short-Term Load and Wind Power Forecasting Using Neural Network-Based Prediction Intervals
Quan, H., Srinivasan, D., & Khosravi, A. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2276053 · Cited by 614
Self-citationPractical options for selecting data-driven or physics-based prognostics algorithms with reviews
An, D., Kim, N. H., & Choi, J.-H. (2015) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2014.09.014 · Cited by 451
Metadata
- Authors
- Khosravi, A., Nahavandi, S., Creighton, D., & Atiya, A. F.
- Venue
- IEEE Transactions on Neural Networks
- Published
- 2011-09
- Volume
- 22
- Issue
- 9
- Pages
- 1341-1356
- ISSN
- 1045-9227 (ISSN-L)
- 1941-0093
- OpenAlex
- W2171666055
- Cited by
- 694
- References
- 54
- FWCI
- 22.54
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.