TSLA: A Multi-Task Time Series Language Model
Abstract
Real-world time series data often require analysis or interpretation from domain experts. Some tasks, like time series question answering, involve both time series and natural language questions, posing challenges for single-modality language models to understand their interaction. To this end, we present TSLA (Time Series Language Model), a framework designed to enhance the language model with the understanding of time series data for multi-modality tasks. TSLA comprises three key components. (1) Time Series Tokenizer learns how to represent time series data into discrete tokens, making it more manageable for language models. (2) Joint (Pre-)Training on task-agnostic time series and text data integrates time series tokens and text tokens to model the interplay between time series and language concepts. (3) Multi-task Instruction Tuning fine-tunes the pretrained TSLA for various downstream tasks relevant to user interests. For evaluation, we applied TSLA to time series data from human motions on four tasks: time series captioning, time series question answering, text-based time series synthesis, and text-based time series continuation. The results demonstrate TSLA’s effectiveness in handling multiple time series analysis tasks, pointing the way for future research endeavors.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Current large language models struggle to effectively incorporate time-series inputs due to challenges in temporal tokenization.
“However, current large language models struggle to effectively incorporate time-series inputs due to challenges in temporal tokenization [121, 6, 32, 121, 100].”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [32]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
(2004) · Cited by 9,718 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17325 · Cited by 7,752 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series ForecastingPreprint
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2201.12740 · Cited by 3,946 · see citing works
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729 · see citing works
BEiT: BERT Pre-Training of Image Transformers
Bao, H., Dong, L., Piao, S., & Wei, F. (2021) · arXiv · doi:10.48550/arxiv.2106.08254 · Cited by 1,392 · see citing works
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works
TimeLLM: Time Series Forecasting by Reprogramming Large Language Models
Jin, M., Pan, S., Li, Y.-F., Ma, L., Zhang, J. Y., Chu, Z., Wang, S., Shi, X., Chen, P., Wen, Q., et al. (2026) · CRC Press eBooks · doi:10.1201/9781003616719-7 · Cited by 614 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
Generating Diverse and Natural 3D Human Motions from Text
Guo, C., Zou, S., Zuo, X., Wang, S., Ji, W., Li, X., & Cheng, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00509 · Cited by 582 · see citing works
Can ChatGPT forecast stock price movements? Return predictability and large language models
Lopez-Lira, A., & Tang, Y. (2026) · Journal of Financial Economics · doi:10.1016/j.jfineco.2026.104335 · Cited by 425 · see citing works
DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection
Yang, Y., Zhang, C., Zhou, T., Wen, Q., & Sun, L. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599295 · Cited by 387 · cited 1× · see citing works
“Many groundbreaking studies have focused on time series perspective, like time series classification [38], forecasting [6, 13], and anomaly detection [37].”
One Fits All: Power General Time Series Analysis by Pretrained LM
Zhou, T., Niu, P., Wang, X., Sun, L., & Jin, R. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1877 · Cited by 367 · see citing works
Generating Human Motion from Textual Descriptions with Discrete Representations
Zhang, J., Zhang, Y., Cun, X., Zhang, Y., Zhao, H., Lu, H., Shen, X., & Ying, S. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01415 · Cited by 328 · see citing works
TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting
Ekambaram, V., Jati, A., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599533 · Cited by 326 · see citing works
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y., Liang, Y., Li, Y.-F., Pan, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01728 · Cited by 276
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255 · see citing works
Silliman, R. (2019) · University of California Press eBooks · doi:10.1525/9780520940420-020 · Cited by 221 · see citing works
MotionGPT: Human Motion as a Foreign Language
Jiang, B., Chen, X., Liu, W., Yu, J., Yu, G., & Chen, T. (2023) · arXiv · doi:10.48550/arxiv.2306.14795 · Cited by 149 · see citing works
MotionGPT: Human Motion as a Foreign Language
Jiang, B., Chen, X., Wen, L. I. U., Jingyi Yu, A., Yu, G., & Chen, T. (2023) · neural information processing systems · doi:10.52202/075280-0880 · Cited by 149 · see citing works
Sawhney, R., Agarwal, S., Wadhwa, A., & Shah, R. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.676 · Cited by 146 · see citing works
AutoTimes: Autoregressive Time Series Forecasters via Large Language Models
Liu, Y., Guo, Q., Huang, X., Wang, J., & Long, M. (2024) · neural information processing systems · doi:10.52202/079017-3882 · Cited by 92 · see citing works
Self-citationRobust and Accurate Anomaly Detection in ECG Artifacts Using Time Series Motif Discovery
Sivaraks, H., & Ratanamahatana, C. A. (2015) · Computational and Mathematical Methods in Medicine · doi:10.1155/2015/453214 · Cited by 79 · see citing works
Leveraging language foundation models for human mobility forecasting
Xue, H., Voutharoja, B. P., & Salim, F. D. (2022) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3557915.3561026 · Cited by 73 · see citing works
MOMENT: a family of open time-series foundation models
Goswami, M., Szafer, K., Choudhry, A., Cai, Y., Li, S., & Dubrawski, A. (2024) · arXiv · doi:10.48550/arxiv.2402.03885 · Cited by 41 · see citing works
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation
Xing, Y., Shi, Z., Zhao, M., Lakemeyer, G., Ma, Y., & Wattenhofer, R. P. (2021) · doi:10.18653/v1/2021.acl-long.44 · Cited by 39 · see citing works
Neural Data-Driven Captioning of Time-Series Line Charts
Spreafico, A., & Carenini, G. (2020) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3399715.3399829 · Cited by 24 · see citing works
Translating Human Mobility Forecasting through Natural Language Generation
Xue, H., Salim, F. D., Ren, Y., & Clarke, C. L. A. (2022) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3488560.3498387 · Cited by 20 · see citing works
FLAMES2Graph: An Interpretable Federated Multivariate Time Series Classification Framework
Younis, R., Ahmadi, Z., Hakmeh, A., & Fisichella, M. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599354 · Cited by 16 · see citing works
InstrucTime: Advancing Time Series Classification with Multimodal Language Modeling
Cheng, M., Chen, Y., Liu, Q., Liu, Z., Luo, Y., & Chen, E. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703499 · Cited by 11 · see citing works
Self-citationTruth-Conditional Captions for Time Series Data
Jhamtani, H., & Berg-Kirkpatrick, T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.55 · Cited by 10 · see citing works
Generating Fine-Grained Causality in Climate Time Series Data for Forecasting and Anomaly DetectionPreprint
Fu, D., Zhu, Y., Tong, H., Weldemariam, K., Bhardwaj, O., & He, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.04254 · Cited by 4 · see citing works
Flagged first, then by citations
2 of the 39 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Kong, Z., Li, Y., Zeng, F., Xin, L., Messica, S., Lin, X., Zhao, P., Kellis, M., Tang, H., & Zitnik, M. (2025) · arXiv · doi:10.48550/arxiv.2505.18227
“Future models should similarly prioritize informative frames and temporal segments, allowing them to reason over extended video sequences with greater efficiency and interpretability.”
- Authors
- Fang, L., Chen, Y., Yu, W., Liu, Y., Tang, L.-A., Torvik, V. I., & Chen, H.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Amazon (United States) (US)
- NEC (United States) (US)
- Venue
- ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2025-04-06
- Pages
- 1-5
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W4408354823
- Cited by
- 0
- References
- 39
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
- Funders
- Health
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- multi-task learning, instruction tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
