The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Temporal Fusion Transformers for interpretable multi-horizon time series forecasting
Open accessAbstract
Multi-horizon forecasting often contains a complex mix of inputs – including static (i.e. time-invariant) covariates, known future inputs, and other exogenous time series that are only observed in the past – without any prior information on how they interact with the target. Several deep learning methods have been proposed, but they are typically ‘black-box’ models that do not shed light on how they use the full range of inputs present in practical scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) – a novel attention-based architecture that combines high-performance multi-horizon forecasting with interpretable insights into temporal dynamics. To learn temporal relationships at different scales, TFT uses recurrent layers for local processing and interpretable self-attention layers for long-term dependencies. TFT utilizes specialized components to select relevant features and a series of gating layers to suppress unnecessary components, enabling high performance in a wide range of scenarios. On a variety of real-world datasets, we demonstrate significant performance improvements over existing benchmarks, and highlight three practical interpretability use cases of TFT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Of note, recent work in a time-series analysis by Google has demonstrated the promise of attention-based model architectures to combine both static and time-dependent inputs to achieve interpretable time-series forecasting.”
From Multimodal biomedical AI (2022) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Comaniciu, D., Ramesh, V., & Meer, P. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1195991 · Cited by 4,560
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Clevert, D.-A., Unterthiner, T., & Hochreiter, S. (2015) · arXiv · doi:10.48550/arxiv.1511.07289 · Cited by 4,513
Residual Attention Network for Image Classification
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.683 · Cited by 4,073
DeepAR: Probabilistic forecasting with autoregressive recurrent networks
Salinas, D., Flunkert, V., Gasthaus, J. A., & Januschowski, T. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.07.001 · Cited by 3,094
Convergence Results for Neural Networks via Electrodynamics
Panigrahy, R., Rahimi, A., Sachdeva, S., et al. (2018) · Leibniz international proceedings in informatics · doi:10.4230/lipics.itcs.2018.22 · Cited by 2,848
The Divergence and Bhattacharyya Distance Measures in Signal Selection
Kailath, T. (1967) · IRE Transactions on Communications Systems · doi:10.1109/tcom.1967.1089532 · Cited by 1,860
Language Modeling with Gated Convolutional NetworksPreprint
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08083 · Cited by 1,765
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series ForecastingPreprint
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.-X., & Yan, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.00235 · Cited by 1,414
A theoretically grounded application of dropout in recurrent neural networks
Gal, Y., & Ghahramani, Z. (2016) · Cambridge University Engineering Department Publications Database · Cited by 1,089
RETAIN: an interpretable predictive model for healthcare using reverse time attention mechanism
Choi, E., Bahadori, M. T., Kulas, J. A., Schuetz, A., Stewart, W. F., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1608.05745 · Cited by 1,029
The M4 Competition: 100,000 time series and 61 forecasting methods
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2019.04.014 · Cited by 1,000
(2017) · International Conference on Machine Learning · Cited by 758
A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series
Marcellino, M. G., Stock, J. H., & Watson, M. W. (2005) · Journal of Econometrics · doi:10.1016/j.jeconom.2005.07.020 · Cited by 736
Deep State Space Models for Time Series Forecasting
Rangapuram, S. S., Seeger, M., Gasthaus, J. A., Stella, L., Wang, Y., & Januschowski, T. (2018) · neural information processing systems · Cited by 503
Attend and Diagnose: Clinical Time Series Analysis Using Attention Models
Song, H., Rajan, D., Thiagarajan, J., & Spanias, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11635 · Cited by 468
Regime Changes and Financial Markets
Ang, A., & Timmermann, A. (2012) · Annual Review of Financial Economics · doi:10.1146/annurev-financial-110311-101808 · Cited by 452
TabNet: Attentive Interpretable Tabular Learning
Arık, S. Ö., & Pfister, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16826 · Cited by 434
Self-citationPhased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Neil, D., Pfeiffer, M., & Liu, S. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-149394 · Cited by 378
A Clockwork RNNPreprint
Koutník, J., Greff, K., Gomez, F. J., & Schmidhuber, J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.3511 · Cited by 345
Temporal regularized matrix factorization for high-dimensional time series prediction
Yu, H., Rao, N., & Dhillon, I. S. (2016) · neural information processing systems · Cited by 315
A Multi-Horizon Quantile Recurrent ForecasterPreprint
Wen, R., Torkkola, K., Narayanaswamy, B., & Madeka, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11053 · Cited by 272
Multiple-output modeling for multi-step-ahead time series forecasting
Taieb, S. B., Antti Sorjamaa, B., & Bontempi, G. (2010) · Neurocomputing · doi:10.1016/j.neucom.2009.11.030 · Cited by 254
Multi-Horizon Time Series Forecasting with Temporal Attention Learning
Fan, C., Zhang, Y., Pan, Y., Li, X., Zhang, C., Yuan, R., Wu, D., Wang, W., Pei, J., & Huang, H. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330662 · Cited by 228
A Genotype-First Approach to Defining the Subtypes of a Complex Disease
Stessman, H. A. F., Bernier, R., & Eichler, E. E. (2014) · Cell · doi:10.1016/j.cell.2014.02.002 · Cited by 227
Exploring Interpretable LSTM Neural Networks over Multi-Variable DataPreprint
Guo, T., Lin, T., & Antulov-Fantulin, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12034 · Cited by 154
Probabilistic demand forecasting at scale
Böse, J.-H., Flunkert, V., Gasthaus, J. A., Januschowski, T., Lange, D., Salinas, D., Schelter, S., Seeger, M., & Wang, Y. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137765.3137775 · Cited by 126
Deep Factors for ForecastingPreprint
Wang, Y., Smola, A. J., Maddix, D. C., Gasthaus, J. A., Foster, D. P., & Januschowski, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12417 · Cited by 120
Attentive State-Space Modeling of Disease Progression
Alaa, A. M., & van der Schaar, M. (2019) · neural information processing systems · Cited by 60
Multi-horizon inflation forecasts using disaggregated data
Capistrán, C., Constandse, C., & Ramos‐Francia, M. (2010) · Economic Modelling · doi:10.1016/j.econmod.2010.01.006 · Cited by 44
Multi-step prediction for influenza outbreak by an adjusted long short-term memory
Zhang, J., & Nawata, K. (2018) · Epidemiology and Infection · doi:10.1017/s0950268818000705 · Cited by 44
Courty, P., & Li, H. (1999) · The Journal of Business · doi:10.1086/209627 · Cited by 31
Temporal Causal Inference with Time Lag
Du, S., Song, G., Han, L., & Hong, H. (2017) · Neural Computation · doi:10.1162/neco_a_01028 · Cited by 14
RL-LIM: Reinforcement Learning-based Locally Interpretable ModelingPreprint
Yoon, J., Arık, S. Ö., & Pfister, T. (2019) · arXiv (Cornell University) · Cited by 8
Self-citationGıovanıs, E. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2479295 · Cited by 8
Distributed Lags and Dynamic Models
Baltagi, B. H. (2011) · Springer eBooks · doi:10.1007/978-3-642-20059-5_6 · Cited by 7
Flagged first, then by citations
Cited by
Sorted by citations
Time-series forecasting with deep learning: a survey
Lim, B. Y., & Zohren, S. (2021) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2020.0209 · Cited by 2,198
Self-citationAcosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365
A Transformer-based Framework for Multivariate Time Series Representation Learning
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467401 · Cited by 1,278
Transformers in Time Series: A Survey
Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., & Sun, L. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/759 · Cited by 1,260
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction
Yu, C., Ma, X., Ren, J., Zhao, H., & Yi, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58610-2_30 · Cited by 645
Metadata
- Authors
- Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T.
- Affiliations
- University of Oxford (GB)
- Google (United States) (US)
- Corresponding author
- Lim, Bryan
- Venue
- International Journal of Forecasting
- Publisher
- Elsevier BV
- Published
- 2021-10
- Volume
- 37
- Issue
- 4
- Pages
- 1748-1764
- ISSN
- 0169-2070 (ISSN-L)
- OpenAlex
- W2996331899
- Cited by
- 3,291
- References
- 41
- Citations by year
- 2026: 1359 · 2025: 827 · 2024: 508 · 2023: 362 · 2022: 153 · 2021: 66
- FWCI
- 126.76
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Journal of Forecasting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University)
- Funders
- Google Inc
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Transportation › Traffic Prediction and Management Techniques
- Keywords
- Temporal Fusion Transformer, interpretability, self-attention, gating mechanism, feature selection, exogenous variables, recurrent layers, long-range dependencies, temporal dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
