The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
Open accessAbstract
The Nonlinear autoregressive exogenous (NARX) model, which predicts the current value of a time series based upon its previous values as well as the current and past values of multiple driving (exogenous) series, has been studied for decades. Despite the fact that various NARX models have been developed, few of them can capture the long-term temporal dependencies appropriately and select the relevant driving series to make predictions. In this paper, we propose a dual-stage attention-based recurrent neural network (DA-RNN) to address these two issues. In the first stage, we introduce an input attention mechanism to adaptively extract relevant driving series (a.k.a., input features) at each time step by referring to the previous encoder hidden state. In the second stage, we use a temporal attention mechanism to select relevant encoder hidden states across all time steps. With this dual-stage attention scheme, our model can not only make predictions effectively, but can also be easily interpreted. Thorough empirical studies based upon the SML 2010 dataset and the NASDAQ 100 Stock dataset demonstrate that the DA-RNN can outperform state-of-the-art methods for time series prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · cited 1×
“Recurrent neural networks (RNNs) [Rumelhart et al., 1986; Werbos, 1990; Elman, 1991], a type of deep neural network specially designed for sequence modeling, have received a great amount of attention due to their flexibility in capturing nonlinear relationships.”
BackgroundHierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1174 · Cited by 4,712
Distributed representations, simple recurrent networks, and grammatical structure
Elman, J. L. (1991) · Machine Learning · doi:10.1007/bf00114844 · Cited by 1,354
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1176 · Cited by 1,232
Temporal Convolutional Networks: A Unified Approach to Action Segmentation
Lea, C., Vidal, R., Reiter, A., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_7 · Cited by 924
Learning long-term dependencies in NARX recurrent neural networks
Lin, T., Horne, B. G., Tino, P., & Giles, C. L. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.548162 · Cited by 784
Saliency detection via Cellular Automata
Qin, Y., Lu, H., Xu, Y., & Wang, H. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298606 · Cited by 460
Self-citationThe use of NARX neural networks to predict chaotic time series
Diaconescu, E. (2008) · Cited by 349
Hypothesis Testing in Time Series Analysis.
Moran, P. A., & Whittle, P. (1951) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2981095 · Cited by 322
A dual-stage two-phase model of selective attention.
Hübner, R., Steinhauser, M., & Lehle, C. (2010) · Psychological Review · doi:10.1037/a0019471 · Cited by 267
NARMAX time series model prediction: feedforward and recurrent fuzzy neural network approaches
Gao, Y., & Er, M. J. (2005) · Fuzzy Sets and Systems · doi:10.1016/j.fss.2004.09.015 · Cited by 239
Hypothesis-Testing in Time Series Analysis.
David, F. N., & Whittle, P. (1952) · Biometrika · doi:10.2307/2332487 · Cited by 182
Distributed Representations, Simple Recurrent Networks, and Grammatical Structure
Elman, J. L. (1991) · Connectionist Approaches to Language Learning · doi:10.1007/978-1-4615-4008-3_5 · Cited by 159
NARX-Based Nonlinear System Identification Using Orthogonal Least Squares Basis Hunting
Chen, S., Wang, X. X., & Harris, C. J. (2008) · IEEE Transactions on Control Systems Technology · doi:10.1109/tcst.2007.899728 · Cited by 80
Top Rank Supervised Binary Coding for Visual Search
Song, D., Liu, W., Ji, R., Meyer, D., & Smith, J. R. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.223 · Cited by 76
Self-citationIntegrated pre-processing for Bayesian nonlinear system identification with Gaussian processes
Frigola, R., & Rasmussen, C. E. (2013) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2013.6760734 · Cited by 51
Substructure Vibration NARX Neural Network Approach for Statistical Damage Inference
Yan, L., Elgamal, A., & Cottrell, G. W. (2011) · Journal of Engineering Mechanics · doi:10.1061/(asce)em.1943-7889.0000363 · Cited by 48
Self-citationFine-Grained Photovoltaic Output Prediction Using a Bayesian Ensemble
Chakraborty, P., Marwah, M., Arlitt, M. F., & Ramakrishnan, N. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8179 · Cited by 46
Song, D., Liu, W., & Meyer, D. (2016) · International Joint Conference on Artificial Intelligence · Cited by 23
Self-citation
Cited by
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17325 · Cited by 7,752
A State-of-the-Art Survey on Deep Learning Theory and Architectures
Alom, M. Z., Taha, T. M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M. S., Hasan, M., Van Essen, B. C., Awwal, A. A. S., & Asari, V. K. (2019) · Electronics · doi:10.3390/electronics8030292 · Cited by 1,620
Graph Neural Network-Based Anomaly Detection in Multivariate Time Series
Deng, A., & Hooi, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16523 · Cited by 1,413
“More recently, Generative Adversarial Networks (GANs) (Li et al. 2019) and LSTM-based approaches (Qin et al. 2017) have also re-ported promising performance for multivariate anomaly detection.”
BackgroundRecurrent Neural Networks for Time Series Forecasting: Current status and future directions
Hewamalage, H., Bergmeir, C., & Bandara, K. (2021) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2020.06.008 · Cited by 1,371
Temporal pattern attention for multivariate time series forecasting
Shih, S.-Y., Sun, F.-K., & Lee, H.-Y. (2019) · Machine Learning · doi:10.1007/s10994-019-05815-0 · Cited by 960
Metadata
- Authors
- Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., & Cottrell, G. W.
- Venue
- Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
- Published
- 2017-08
- Pages
- 2627-2633
- OpenAlex
- W2613328025
- Cited by
- 1,585
- References
- 31
- FWCI
- 111.73
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.