CoChat

UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting

Open access
Liu, Xu, Hu, Junfeng, Li, Yuan, Diao, Shizhe, Liang, Yuxuan, Hooi, Bryan, Zimmermann, Roger · 2024 · Proceedings of the ACM Web Conference 2024

The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.

Abstract

Multivariate time series forecasting plays a pivotal role in contemporary web technologies. In contrast to conventional methods that involve creating dedicated models for specific time series application domains, this research advocates for a unified model paradigm that transcends domain boundaries. However, learning an effective cross-domain model presents the following challenges. First, various domains exhibit disparities in data characteristics, e.g., the number of variables, posing hurdles for existing models that impose inflexible constraints on these factors. Second, the model may encounter difficulties in distinguishing data from various domains, leading to suboptimal performance in our assessments. Third, the diverse convergence rates of time series domains can also result in compromised empirical performance. To address these issues, we propose UniTime for effective cross-domain time series learning. Concretely, UniTime can flexibly adapt to data with varying characteristics. It also uses domain instructions and a Language-TS Transformer to offer identification information and align two modalities. In addition, UniTime employs masking to alleviate domain convergence speed imbalance issues. Our extensive experiments demonstrate the effectiveness of UniTime in advancing state-of-the-art forecasting performance and zero-shot transferability.

Overview

Integrity signals

Registry

Crossref · 10.1145/3589334.3645434

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 22 references

Research works printed but not linked: 24

  1. Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. In Advances in neural information processing systems. 1877--1901.
  2. Shizhe Diao, Rui Pan, Hanze Dong, Ka Shun Shum, Jipeng Zhang, Wei Xiong, and Tong Zhang. 2023. Lmflow: An extensible toolkit for finetuning and inference of large foundation models. arXiv preprint arXiv:2306.12420 (2023). · arXiv
  3. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations.
  4. Nate Gruver Marc Finzi Shikai Qiu and Andrew Gordon Wilson. 2023. Large language models are zero-shot time series forecasters. In Advances in Neural Information Processing Systems.
  5. Xiaoxin He, Xavier Bresson, Thomas Laurent, and Bryan Hooi. 2023. Explanations as Features: LLM-Based Features for Text-Attributed Graphs. arXiv preprint arXiv:2305.19523 (2023). · arXiv
  6. Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2023. Large language models are zero-shot rankers for recommender systems. arXiv preprint arXiv:2305.08845 (2023). · arXiv
  7. Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue Xue Wang James Zhang Yi Wang Haifeng Chen Xiaoli Li et al. 2023. Large models for time series and spatio-temporal data: A survey and outlook. arXiv preprint arXiv:2310.10196 (2023). · arXiv
  8. Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan, and Qingsong Wen. 2024. Position Paper: What Can Large Language Models Tell Us about Time Series Analysis. arXiv preprint arXiv:2402.02713 (2024). · arXiv
  9. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT. 2.
  10. Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The efficient transformer. In International Conference on Learning Representations.
  11. Shiyang Li Xiaoyong Jin Yao Xuan Xiyou Zhou Wenhu Chen Yu-Xiang Wang and Xifeng Yan. 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Advances in neural information processing systems.
  12. Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. 2022b. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations.
  13. Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. 2022a. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting. In Advances in Neural Information Processing Systems. 9881--9893.
  14. Jun Ma and Bo Wang. 2023. Segment anything in medical images. arXiv preprint arXiv:2304.12306 (2023). · arXiv
  15. Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations.
  16. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research (2020), 5485--5551.
  17. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.
  18. Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. 2022. Etsformer: Exponential smoothing transformers for time-series forecasting. In arXiv preprint arXiv:2202.01381. · arXiv
  19. Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. 2023. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations.
  20. Junjie Zhang, Ruobing Xie, Yupeng Hou, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2023. Recommendation as instruction following: A large language model empowered recommendation approach. arXiv preprint arXiv:2305.07001 (2023). · arXiv
  21. Yunhao Zhang and Junchi Yan. 2022. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations.
  22. Haiteng Zhao, Shengchao Liu, Chang Ma, Hannan Xu, Jie Fu, Zhi-Hong Deng, Lingpeng Kong, and Qi Liu. 2023. GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning. In Advances in neural information processing systems.
  23. Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning. 27268--27286.
  24. Tian Zhou Peisong Niu Xue Wang Liang Sun and Rong Jin. 2023. One Fits All: Power General Time Series Analysis by Pretrained LM. In Advances in Neural Information Processing Systems.

Cited by

See all 126

Metadata

Authors
Liu, X., Hu, J., Li, Y., Diao, S., Liang, Y., Hooi, B., & Zimmermann, R.
Affiliations
  • National University of Singapore (SG)
  • Hong Kong University of Science and Technology (HK)
  • The Hong Kong University of Science and Technology (Guangzhou) (CN)
Venue
Proceedings of the ACM Web Conference 2024
Publisher
ACM
Published
2024-05-13
Pages
4095-4106
OpenAlex
W4396758709
Cited by
115
References
20
Citations by year
2026: 62 · 2025: 48 · 2024: 5
FWCI
43.13
Type
conference paper
Language
en
Open access
Locations
  • International World Wide Web Conference (WWW) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National University of Singapore (A-8000969-00-00)
  • Hong Kong University of Science and Technology
Topics
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
  • Social Sciences › Transportation › Traffic Prediction and Management Techniques
Keywords
multivariate time series, domain identification, masking, zero-shot transfer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting. https://app.cochat.ai/papers/10.1145/3589334.3645434
The paper
Liu, X., Hu, J., Li, Y., Diao, S., Liang, Y., Hooi, B., & Zimmermann, R. (2024). UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting. Proceedings of the ACM Web Conference 2024. https://doi.org/10.1145/3589334.3645434

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.