CoChat
Sign Up
Cited by 371Time Series Analysis and ForecastingEnergy Load and Power ForecastingStock Market Forecasting Methods

One Fits All: Power General Time Series Analysis by Pretrained LM

Open access
Zhou, Tian, Niu, Peisong, Wang, Xue, Sun, Liang, Jin, Rong·2023·Advances in Neural Information Processing Systems 36
DOI 10.52202/075280-1877
Open this paper in CoChat — free accountOpen PDF

Abstract

Although we have witnessed great success of pre-trained models in natural language processing (NLP) and computer vision (CV), limited progress has been made for general time series analysis. Unlike NLP and CV where a unified model can be used to perform different tasks, specially designed approach still dominates in each time series analysis task such as classification, anomaly detection, forecasting, and few-shot learning. The main challenge that blocks the development of pre-trained model for time series analysis is the lack of a large amount of data for training. In this work, we address this challenge by leveraging language or CV models, pre-trained from billions of tokens, for time series analysis. Specifically, we refrain from altering the self-attention and feedforward layers of the residual blocks in the pre-trained language or image model. This model, known as the Frozen Pretrained Transformer (FPT), is evaluated through fine-tuning on all major types of tasks involving time series. Our results demonstrate that pre-trained models on natural language or images can lead to a comparable or state-of-the-art performance in all main time series analysis tasks, as illustrated in Figure 1. We also found both theoretically and empirically that the self-attention module behaviors similarly to principle component analysis (PCA), an observation that helps explains how transformer bridges the domain gap and a crucial step towards understanding the universality of a pre-trained transformer.The code is publicly available at https://github.com/DAMO-DI-ML/One_Fits_All.

Integrity signals

Registry

Crossref · 10.52202/075280-1877

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MLP-HAR is another lightweight deep learning model that adopts purely fully connected layers for efficiency of HAR models on edge devices.

    “It refers to the another lightweight deep learning model that adopts purely fully connected layers for efficiency of HAR models on edge devices.”

    From Towards Customizable Foundation Models for Human Activity Recognition with Wearable Devices (2025) · cites this paper as [70]

Cites

Still being checked.

Cited by

Sorted by citations

Related

Authors
Zhou, T., Niu, P., Wang, X., Sun, L., & Jin, R.
ORCID
Niu, Peisong
Venue
Advances in Neural Information Processing Systems 36
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2023
Pages
43322-43355
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/075280-1877
OpenAlex
W7133215398
Cited by
367
References
0
Citations by year
2026: 186 · 2025: 121 · 2024: 59 · 2023: 1
FWCI
76.22
Type
conference paper
Language
en
Open access
Licence
cc-by-nc-sa
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Energy Load and Power Forecasting
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
Keywords
time series forecasting, pre-trained language models, power systems, time series classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review