Time2Vec: Learning a Vector Representation of Time
Open accessAbstract
Time is an important feature in many applications involving events that occur synchronously and/or asynchronously. To effectively consume time information, recent studies have focused on designing new architectures. In this paper, we take an orthogonal but complementary approach by providing a model-agnostic vector representation for time, called Time2Vec, that can be easily imported into many existing and future architectures and improve their performances. We show on a range of models and problems that replacing the notion of time with its Time2Vec representation improves the performance of the final model.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These approaches, categorized as temporal representation, divide the entire framework into two stages: "temporal representation" and "temporal prediction."
“These approaches, categorized as temporal representation[105,106], divide the entire framework into two stages: “temporal representation” and “temporal prediction.””
From Advancements in the estimation of the state of charge of lithium-ion battery: a comprehensive review of traditional and deep learning approaches (2025) · cites this paper as [106]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
(2017) · Cited by 11,625 · see citing works
node2vec: Scalable Feature Learning for Networks
Grover, A., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939754 · Cited by 10,926 · see citing works
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
The Fourier Transform and Its Applications
Bracewell, R. N., & Kahn, P. B. (1966) · American Journal of Physics · doi:10.1119/1.1973431 · Cited by 5,775 · see citing works
Gaussian Processes in Machine Learning
Rasmussen, C. E. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28650-9_4 · Cited by 5,734 · see citing works
An introduction to hidden Markov models
Rabiner, L., & Juang, B. (1986) · IEEE ASSP Magazine · doi:10.1109/massp.1986.1165342 · Cited by 4,775 · see citing works
Support Vector Regression Machines
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A. J., & Vapnik, V. N. (1996) · Cited by 4,654 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 1× · see citing works
“Vector embedding has been previously successfully used for other domains such as text [40, 48], (knowledge) graphs [22, 46, 25], and positions [57, 17].”
MethodDynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,563 · see citing works
Fitting Autoregreesive Models for Prediction
Akaike, H. (1998) · Springer series in statistics · doi:10.1007/978-1-4612-1694-0_10 · Cited by 2,040 · see citing works
A Review of Relational Machine Learning for Knowledge Graphs
Nickel, M., Murphy, K. P., Tresp, V., & Gabrilovich, E. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2483592 · Cited by 1,684 · see citing works
Recurrent nets that time and count
Gers, F. A., & Schmidhuber, J. (2000) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2000.861302 · Cited by 656 · see citing works
Patient Subtyping via Time-Aware LSTM Networks
Baytaş, İ. M., Xiao, C., Zhang, X., Wang, F., Jain, A. K., & Zhou, J. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3097997 · Cited by 646 · see citing works
Recurrent Marked Temporal Point Processes
Du, N., Dai, H., Trivedi, R., Upadhyay, U., Gomez-Rodriguez, M., & Song, L. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939875 · Cited by 538 · see citing works
What to Do Next: Modeling User Behaviors by Time-LSTM
Zhu, Y., Li, H., Liao, Y., Wang, B., Guan, Z., Liu, H., & Cai, D. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/504 · Cited by 489 · see citing works
Nonlinear signal processing using neural networks: Prediction and system modelling
Lapedes, A. S., & Farber, R. M. (1987) · University of North Texas Digital Library (University of North Texas) · Cited by 373 · see citing works
Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences
Neil, D., Pfeiffer, M., & Liu, S. (2016) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-149394 · Cited by 361 · see citing works
Representation learning for dynamic graphs: A survey
Kazemi, S. M., Goel, R., Jain, K., Kobyzev, I., Sethi, A., Forsyth, P., & Poupart, P. (2019) · arXiv · doi:10.48550/arxiv.1905.11485 · Cited by 338 · see citing works
Self-citationKwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294 · see citing works
Sutton, C. A., McCallum, A., & Rohanimanesh, K. (2007) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.1145/1015330.1015422 · Cited by 288 · see citing works
Music recommendation and discovery in the long tail
Herrada, Ò. C. (2009) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 284 · see citing works
The neural hawkes process: a neurally self-modulating multivariate point process
Mei, H., & Eisner, J. M. (2017) · neural information processing systems · Cited by 201 · see citing works
SimplE embedding for link prediction in knowledge graphs
Kazemi, S. M., & Poole, D. (2018) · neural information processing systems · Cited by 147 · see citing works
Self-citationThe role of over-parametrization in generalization of neural networks
Neyshabur, B., Li, Z., Bhojanapalli, S., et al. (2019) · Cited by 143 · see citing works
Liu, P., Zeng, Z., & Wang, J. (2015) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2015.2461191 · Cited by 119 · see citing works
Wasserstein Learning of Deep Generative Point Process ModelsPreprint
Xiao, S., Farajtabar, M., Ye, X., Yan, J., Song, L., & Hongyuan Zha, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08051 · Cited by 115 · see citing works
Feature Representations for Neuromorphic Audio Spike Streams
Anumula, J., Neil, D., Delbrück, T., & Liu, S. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00023 · Cited by 107 · see citing works
Deep Reinforcement Learning of Marked Temporal Point ProcessesPreprint
Upadhyay, U., De, A., & Gomez-Rodriguez, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09360 · Cited by 69 · see citing works
Taming the waves: sine as activation function in deep neural networks
Parascandolo, G., Huttunen, H. J., & Virtanen, T. I. (2017) · Cited by 68 · see citing works
Sopena, J. M. G. (1999) · doi:10.1049/cp:19991129 · Cited by 65 · see citing works
Neural Decomposition of Time-Series Data for Effective Generalization
Godfrey, L. B., & Gashler, M. S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2709324 · Cited by 55 · see citing works
Learning Temporal Point Processes via Reinforcement LearningPreprint
Li, S., Xiao, S., Zhu, S., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.05016 · Cited by 50 · see citing works
Modeling time series data with deep Fourier neural networks
Gashler, M. S., & Ashmore, S. C. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.01.108 · Cited by 45 · see citing works
State-frequency memory recurrent neural networks
Hu, H., & Qi, G.-J. (2017) · Journal of International Crisis and Risk Communication Research · Cited by 43 · see citing works
Learning Conditional Generative Models for Temporal Point Processes
Xiao, S., Xu, H., Yan, J., et al. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12072 · Cited by 42 · see citing works
Hawkes ProcessesPreprint
Laub, P. J., Taimre, T., & Pollett, P. K. (2015) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1507.02822 · Cited by 35 · see citing works
Population Size Extrapolation in Relational Probabilistic Modelling
Poole, D., Buchman, D., Kazemi, S. M., Kristian Kersting, B., & Natarajan, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11508-5_25 · Cited by 28 · see citing works
Self-citationStreaming Graph Neural NetworksPreprint
Ma, Y., Guo, Z., Ren, Z., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10627 · Cited by 23 · see citing works
Wong, K., Leung, C.-S., & Chang, S. (2003) · doi:10.1109/icpr.2002.1047806 · Cited by 22 · see citing works
Relational Representation Learning for Dynamic (Knowledge) Graphs: A Survey.Preprint
Kazemi, S. M., Goel, R., Jain, K., Kobyzev, I., Sethi, A., Forsyth, P., & Poupart, P. (2019) · arXiv (Cornell University) · Cited by 21 · see citing works
Self-citationLearning Dynamic Embeddings from Temporal InteractionsPreprint
Kumar, S., Zhang, X., & Leskovec, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02289 · Cited by 19 · see citing works
FOURIER NEURAL NETWORKS: AN APPROACH WITH SINUSOIDAL ACTIVATION FUNCTIONS 1
Mintchev, M. P., López, L. F. D. M., Aslanyan, L. H., et al. (2003) · Cited by 15 · see citing works
evt_MNIST: A spike based version of traditional MNISTPreprint
Fatahi, M., Ahmadi, M., Shahsavari, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.06751 · Cited by 5 · see citing works
Time-Dependent Representation for Neural Event Sequence PredictionPreprint
Li, Y., Du, N., & Bengio, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.00065 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- [4][4] Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-learn in networks of spiking neurons. In NeurIPS , 2018. -
- [6][6] Víctor Campos, Brendan Jou, Xavier Giró-i Nieto, Jordi Torres, and Shih-Fu Chang. Skip rnn: Learning to skip state updates in recurrent neural networks. In ICLR , 2018. -
- [8][8] Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Neural Information Processing Systems (NeurIPS) , 2018. -
- [10][10] Edward Choi, Mohammad Taha Bahadori, Andy Schuetz, Walter F Stewart, and Jimeng Sun. Doctor AI: Predicting clinical events via recurrent neural networks. In Machine Learning for Healthcare Conference , pages 301-318, 2016. -
- [12][12] Daryl J Daley and David Vere-Jones. An introduction to the theory of point processes: volume II: general theory and structure . Springer Science & Business Media, 2007. -
- [36][36] Zachary C Lipton, David Kale, and Randall Wetzel. Directly modeling missing data in sequences with rnns: Improved classification of clinical time series. In Machine Learning for Healthcare Conference , pages 253-270, 2016. -
- [54][54] Corentin Tallec and Yann Ollivier. Can recurrent neural networks warp time? In International Conference on Learning Representation (ICLR) , 2018. -
- [55][55] Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs. In ICML , pages 3462-3471, 2017. -
Cited by
Sorted by citations
Wu, Y., Bai, D., Zhang, K., Li, Y., & Yang, F. (2025) · Journal of Materials Informatics · doi:10.20517/jmi.2024.84 · Cited by 20 · see citing works
- Authors
- Kazemi, S. M., Goel, R., Eghbali, S., Ramanan, J., Sahota, J., Thakur, S., Wu, S., Smyth, C., Poupart, P., & Brubaker, M.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1907.05321
- OpenAlex
- W2957988017
- Cited by
- 206
- References
- 51
- Citations by year
- 2026: 4 · 2025: 29 · 2024: 48 · 2023: 54 · 2022: 36 · 2021: 24 · 2020: 9 · 2019: 1 · 2018: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- Time2Vec, time embedding, event synchronization, asynchronous events, temporal modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
