The DOI matches its Crossref record; 223 of the 232 references it cites were found in a registry or an index; 86 of the 189 sources its claims cite were checked for support.
Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Open accessAbstract
A comprehensive survey of current state-of-the-art SSL methods for time series data, proposing a taxonomy categorizing methods into generative-based, contrastive-based, and adversarial-based across ten subcategories, along with benchmark datasets for forecasting, classification, anomaly detection, and clustering.
Overview
CoChat Summary
TL;DR
This survey reviews self-supervised learning (SSL) for time series, proposing a taxonomy of generative-based, contrastive-based, and adversarial-based methods divided into ten subcategories. It summarizes applications/datasets and future directions, and compiles quantitative comparisons showing no single SSL method dominates across tasks.
Key findings
- The paper proposes a taxonomy with three broad SSL paradigms and ten subcategories: autoregressive-based forecasting, autoencoder-based reconstruction, diffusion-based generation; sampling contrast, prediction contrast, augmentation contrast, prototype contrast, expert knowledge contrast; generation/imputation and auxiliary representation enhancement.
- For anomaly detection, contrastive methods such as Skip-CPC and Dcdetector are reported as achieving the best performance on four datasets; Dcdetector reaches F1 scores of 97.94 on PSM, 96.60 on MSL, 97.02 on SMAP, and 96.33 on SWaT, while Skip-CPC reaches F1 98.55 on PSM, 94.46 on SMD, and 92.75 on MSL.
- For forecasting, LaST and CoST show significant advantages; LaST reports ETTh H=24 MSE/MAE of 0.324/0.368, ETTm 0.218/0.289, Electricity 0.125/0.222, and Weather 0.105/0.134.
- For classification and clustering, results on 30 datasets from HAR, UCR, and UEA show that no single method achieves the best performance on all datasets, and most methods are based on self-supervised contrastive learning.
Methods
- Design: a comprehensive literature survey that reviews existing SSL and time series surveys, then organizes time series SSL methods by learning paradigm and by how positive/negative samples or pretext tasks are constructed.
- Data/resources: summarizes commonly used datasets for anomaly detection, forecasting, classification, and clustering, including PSM, SMD, MSL, SMAP, SWaT, WADI, ETTh, ETTm, Electricity, Weather, HAR, UCR 128, and UEA 30.
- Analysis: qualitative review of frameworks, mathematical formulations, advantages/disadvantages, plus compiled quantitative comparisons from prior work on anomaly detection (five datasets), forecasting (four datasets), and classification/clustering (30 datasets); no new experiments are reported.
Limitations
- The paper states that prior time series SSL surveys only discuss a small part of self-supervised contrastive learning, lack benchmark dataset summaries, and have scarce future directions; it positions itself as filling that gap.
- It also summarizes method-family limitations: generative-based methods are sensitive to rare instances and lack high-level semantic information; contrastive-based methods face sampling bias and augmentation uncertainty; adversarial-based methods suffer unstable training and are rarely used for feature extraction.
- Evident from the text: as a survey, it does not run a unified benchmark; the quantitative tables are collected from prior papers and contain missing entries (“-”), so comparisons are incomplete, and the paper itself calls for reliable benchmark evaluation as a future direction.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
66 Valid · 26 Not valid · 63 Not checked · 80 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Methods for constructing multiple contrastive views based on multi-granularity or multi-scale augmentations also include MRLF [131], CMLF [115], and SSLAPP [132].”
What the sources say
- Not checked — no full text of the sourcecites [131] source
- Not checked — no full text of the sourcecites [115] source
- Partly supportedcites [132] sourceExperimental — accuracy not yet measured
Sections 3.2 and 3.3 describe SSLAPP constructing two augmented views as a positive pair and learning from global and local features. But Section 3.2 explicitly says SSLAPP uses pairwise representation learning instead of contrastive learning; the global and local distinction applies to its loss terms, not to multi-scale augmentations. The citation identifies SSLAPP correctly but mischaracterizes its method. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“However, introducing reasonable inductive bias or prior is helpful for many deep neural networks-based modeling tasks [140], [196], [197].”
What the sources say
- Supportedcites [140] source
The Introduction (§1.1; page not provided) says integrating prior knowledge into a learning framework is effective for sparse data and describes its use across several applications. This supports the broad claim that suitable priors can help deep-learning modeling tasks. (LLM, conf=0.85)
- Supportedcites [196] source
The Network Architecture section says that adding domain-specific components to neural networks can boost performance and that task-targeted architectures can generalize better. It gives examples for different data and tasks, supporting the broad claim that appropriate inductive biases are helpful in neural-network modeling. (LLM, conf=0.91)
- Supportedcites [197] Chen, 2022, Integration of knowledge and data in machine learning source
The Introduction (page not available in our copy) says that embedding domain knowledge can improve model accuracy and robustness, and that it can improve machine-learning performance. Section 3.1 describes several ways to incorporate prior knowledge into neural networks across different applications, supporting the citing sentence’s broad, qualified claim. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Unlike previous frameworks, SSGAN's classification network makes full use of label information, which helps the model achieve more accurate imputations.”
What the sources say
- Supportedcites [162] source
The Classifier Network passage says SSGAN’s classifier uses self-training to make full use of label information in partially labeled time series and guides the generator during imputation. The Ablation Study reports that removing the classifier reduces imputation accuracy by an average of 1.36%, supporting its contribution to more accurate imputations. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“It is beneficial to introduce an adversarial-based SSL, which can further amplify the difference between normal and anomalous data, such as USAD [66] and DAEMON [184].”
What the sources say
- Partly supportedcites [66] sourceExperimental — accuracy not yet measured
Section 5.4 (Ablation Study; page unavailable) reports that adversarial training improves USAD’s F1 score and helps detect anomalies close to normal data. Section 3.2 describes USAD as an unsupervised autoencoder method with adversarial training, but the copy does not identify it as self-supervised learning (SSL). (LLM, conf=0.82)
- Partly supportedcites [184] sourceExperimental — accuracy not yet measured
The Abstract and §2.2 identify DAEMON as an unsupervised anomaly detector that uses adversarial training to learn normal time-series patterns; the model-variant results in §4.5.2 report a benefit from adversarial regularization. The paper does not describe DAEMON as self-supervised learning (SSL) or establish that adversarial training amplifies the difference between normal and anomalous data. The cited paper is DAEMON, so this verdict does not assess USAD. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In recent years, many scholars have also explored the potential of generative representation learning in time series generation and imputation, such as C-RNN-GAN [154], TimeGAN [155], TTS-GAN [156], and E 2 GAN[157].”
What the sources say
- Partly supportedcites [154] Mogren, 2016, C-RNN-GAN: Continuous recurrent neural networks with adversarial training sourceExperimental — accuracy not yet measured
The Introduction (p. 1) identifies C-RNN-GAN as an adversarial model that generates continuous sequences, demonstrated with classical music. This supports citing it as an example of sequence generation, but the paper does not describe time-series imputation or establish it as an imputation method. (LLM, conf=0.84)
- Not matchedcites [155] Yoon, Time-Series Generative Adversarial Networks
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [156] sourceExperimental — accuracy not yet measured
The TTS-GAN paper identifies itself as a transformer-based GAN for generating synthetic time-series data (title and abstract, p. 1). Citation [156] is attached to TTS-GAN as an example in the list; the paper supports its role as a time-series generation method. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
44 more claims · 15 with their cited sources checked
Create a free account to see all 235 claims and to verify the 10 references still open.
- Probably not a substantial claim. 158
- It may state two separate claims as one. 79
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ I.2 Correlation analysis between methods and tasks · p. 26
“| Method | 15 UCR Datasets | 15 UCR Datasets | 15 UCR Datasets | 15 UCR Datasets |”
§ I.2 Correlation analysis between methods and tasks · p. 26
“| | Beef | ChCo | Coffee | CricketX | DiPOA | DiPOC | ECG200 | ECGF | Car | Ham | Herring | Wine | Meat | MiPOA | Wafer | Yoga |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ I.1 Quantitative performance comparisons · p. 25
“| RANSynCoder [63] | 99 | 96 | 93 | 90 | 85 | 83 |”
§ H.1 Anomaly detection · p. 23
“F1-score is the harmonic mean of precision and recall, i.e.,”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ I.1 Quantitative performance comparisons · p. 25
“| ImDiffusion [101] | 98.11 | 97.53 | 97.81 | 95.2 | 95.09 | 94.88 |”
§ H.1 Anomaly detection · p. 23
“F1-score is the harmonic mean of precision and recall, i.e.,”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 2×
“With the help of well-designed pretext tasks, SSL has recently achieved great success in the domains of Computer Vision (CV) [7], [8], [9], [10] and Natural Language Processing (NLP) [11], [12].”
BackgroundMethodUnpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“The generator in the GAN can generate synthetic data close to the real data, so adversarial representation learning has a wide range of applications in the data generation field [141], especially in image generation [142]–[145].”
BackgroundAuto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · cited 1×
“The major categories include end-to-end [16], memory bank [34], momentum encoder [35], and clustering [36].”
BackgroundRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 1×
“Using the learning paradigm as a taxonomy is arguably the most popular among the existing SSL surveys, including [20], [22], [23], [24], [25], [26], [27].”
MethodKirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · cited 1×
“A similar phenomenon also appears in computational vision [203].”
BackgroundA Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 1×
“The generator in the GAN can generate synthetic data close to the real data, so adversarial representation learning has a wide range of applications in the data generation field [149], especially in image generation [150], [151], [152], [153].”
BackgroundA simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · cited 1×
“…various recent and typical adversarial attack and defense methods for fault classifiers in data-driven fault detection and classificationsystems,includingwhite-boxattack(FGSM[209], IGSM[210],C & W attack [211], DeepFool [212]) and gray-box and black-box attack (UAP [213], SPSA [214], Random noise).”
BackgroundTowards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859 · cited 1×
“…various recent and typical adversarial attack and defense methods for fault classifiers in data-driven fault detection and classificationsystems,includingwhite-boxattack(FGSM[209], IGSM[210],C & W attack [211], DeepFool [212]) and gray-box and black-box attack (UAP [213], SPSA [214], Random noise).”
BackgroundDenoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 1×
“Mask autoencoder (MAE) is a structure widely used in language models and vision models in recent years [11], [73].”
BackgroundIntriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 1×
“With the widespread use of deep neural networks in time series forecasting, classification, and anomaly detection, the vulnerability and robustness of deep models under adversarial have become a significant concern [204], [205], [206], [207].”
BackgroundInformer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17325 · Cited by 7,752
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · cited 1×
“Denoising autoencoder (DAE) is another widely used approach, which is based on the addition of noise to the input seriestocorruptthedata,andthenfollowedbythereconstruction task[68].”
MethodRepresentation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 3×
“LNT and TRL-CPC use the same structure as the original CPC [109] to build a representation learning model, and the purpose is to capture the local semantics across the time to detect the anomaly points.”
MethodDeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655 · cited 1×
“Therefore, an interesting direction is to construct a reasonable evaluation framework for time series data augmentation methods, then further select the optimal method or combination strategy.”
BackgroundLarge Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Supervised Contrastive Learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., & Krishnan, D. (2020) · arXiv · doi:10.48550/arxiv.2004.11362 · Cited by 3,569 · cited 3×
“Supervised contrastive learning (SCL) [107] effectively addresses sampling bias, so introducing the super-visedsignaltoidentifypositiveandnegativesamplesisafeasible solution.”
BackgroundInfluentialDiffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Deep learning for time series classification: a review
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-019-00619-1 · Cited by 3,342 · cited 1×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundSimCSE: Simple Contrastive Learning of Sentence Embeddings
Gao, T., Yao, X., & Chen, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.552 · Cited by 2,851 · cited 1×
“With the help of well-designed pretext tasks, SSL has recently achieved great success in the domains of Computer Vision (CV) [7], [8], [9], [10] and Natural Language Processing (NLP) [11], [12].”
MethodUniversal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000056 · Cited by 2,609 · cited 1×
“Variationalautoencoder(VAE)isamodelbasedonvariational inference [78], [79].”
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09882 · Cited by 2,499
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · cited 1×
“With the help of well-designed pretext tasks, SSL has recently achieved great success in the domains of Computer Vision (CV) [7], [8], [9], [10] and Natural Language Processing (NLP) [11], [12].”
MethodModeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
Lai, G., Chang, W.-C., Yang, Y., & Liu, H. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210006 · Cited by 2,152
Time-series Generative Adversarial Networks
Yoon, J., Jarrett, D., & van der Schaar, M. (2019) · neural information processing systems · Cited by 2,070
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036 · cited 2×
“JingandTian[18]summarize theself-supervisedfeaturelearningmethodsonimageand videodata,andfourcategoriesarediscussed:generation-based, context-based,freesemanticlabel-based,andcrossmodal-based,wherecross-modal-basedmethodsconstructlearning task using RGB frame sequence an optical flow sequence, which…”
BackgroundResultTian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997 · cited 1×
“With the help of well-designed pretext tasks, SSL has recently achieved great success in the domains of Computer Vision (CV) [7], [8], [9], [10] and Natural Language Processing (NLP) [11], [12].”
Method(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959
Score-Based Generative Modeling through Stochastic Differential Equations
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020) · arXiv · doi:10.48550/arxiv.2011.13456 · Cited by 1,953 · cited 2×
“For score SDEs, the diffusion operation is processed according to the stochastic differential equation (SDE) [97]: where f ( x , t ) and g ( t ) are diffusion function and drift function of the SDE, respectively, and w is a standard Wiener process.”
BackgroundMethodRobust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network
Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., & Pei, D. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330672 · Cited by 1,804 · cited 1×
“OmniAnomaly [81] combines VAE and Planar Normalizing Flow to propose an interpretable time series anomaly detection algorithm.”
MethodTime-series forecasting with deep learning: a survey
Lim, B., & Zohren, S. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0209 · Cited by 1,787 · cited 1×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundDetecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding
Hundman, K., Constantinou, V., Laporte, C., Colwell, I., & Soderstrom, T. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219845 · Cited by 1,771
Diffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704 · cited 1×
“As a new kind of deep generative model, diffusion models have achieved great success recently in many fields, including image synthesis, video generation, speech generation, bioinformatics, and natural language processing due to their powerful generating ability [88], [89], [90], [91], [92].”
BackgroundVillarraga Ossa, E. F. (2021) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · Cited by 1,694
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591 · cited 1×
“Using the learning paradigm as a taxonomy is arguably the most popular among the existing SSL surveys, including [20], [23]–[28].”
MethodGenerative Modeling by Estimating Gradients of the Data Distribution
Song, Y., & Ermon, S. (2019) · arXiv · doi:10.48550/arxiv.1907.05600 · Cited by 1,453
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435 · cited 1×
“The major categories include end-to-end [16], memory bank [34], momentum encoder [35], and clustering [36].”
BackgroundGraph Neural Network-Based Anomaly Detection in Multivariate Time Series
Deng, A., & Hooi, B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i5.16523 · Cited by 1,413 · cited 2×
“To be concrete, THOC [52] and GDN [55] employ autoregressive-based forecasting SSL framework, which assumes that anomalous sequences or time points are not predictable.”
MethodCSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series ImputationPreprint
Tashiro, Y., Song, J., Song, Y., & Ermon, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.03502 · Cited by 1,310
A Transformer-based Framework for Multivariate Time Series Representation Learning
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467401 · Cited by 1,278 · cited 1×
“Different from STEP, Zerveas et al. [75] performed a more complex masking operation on the time series, i.e., the multivariate time series was randomly divided into multiple non-overlapping segments of unequal length on each variable.”
MethodTransformers in Time Series: A Survey
Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., & Sun, L. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/759 · Cited by 1,260 · cited 2×
“The second category focuses on the key components of time series modeling based on deep neural networks, such as data augmentation [33], [45], [46], [47], model structure [33], [48], [49].”
BackgroundSelf-citation(2011) · Cited by 1,258
Discriminative Unsupervised Feature Learning with Convolutional Neural NetworksPreprint
Dosovitskiy, A., Springenberg, J. T., Riedmiller, M., & Brox, T. (2014) · arXiv (Cornell University) · Cited by 1,153
Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., & Keogh, E. (2019) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2019.1911747 · Cited by 1,128
A Review on Outlier/Anomaly Detection in Time Series Data
Blázquez-García, A., Conde, A., Mori, U., & Lozano, J. A. (2022) · ACM Computing Surveys · doi:10.1145/3444690 · Cited by 1,102 · cited 1×
“In the context of time series modeling, the commonly used pretext tasks include using the past series to forecast the future windows or specific time stamps, using the encoder and decoder to reconstruct the input, and forecasting the unseen part of the masked time series.”
MethodUSAD: UnSupervised Anomaly Detection on Multivariate Time Series
Audibert, J., Michiardi, P., Guyard, F., Marti, S., & Zuluaga, M. A. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403392 · Cited by 1,094
Self-supervised Learning: Generative or Contrastive
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3090866 · Cited by 1,087 · cited 1×
“The taxonomy is similar to the one proposed by Liu et al. [21] but specifically concentrated on time series data.”
MethodSelf-citationEsling, P., & Agon, C. (2012) · ACM Computing Surveys · doi:10.1145/2379776.2379788 · Cited by 997
(2015) · Cited by 989
Wang, T., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10242 · Cited by 921
Contrastive Representation Learning: A Framework and Review
Le-Khac, P. H., Healy, G., & Smeaton, A. F. (2020) · IEEE Access · doi:10.1109/access.2020.3031549 · Cited by 862 · cited 3×
“As summarized in [29], commonly used contrastive loss functions generally include scoring functions (cosine similarity), energy-based margin functions (pair loss and triplet loss), probabilistic NCE-based functions, and mutual information based functions.”
BackgroundMethodTS2Vec: Towards Universal Representation of Time Series
Yue, Z., Wang, Y., Duan, J., Yang, T., Huang, C., Tong, Y., & Xu, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20881 · Cited by 850 · cited 3×
“TS2Vec[118] generatesdifferentviewsthroughmaskingoperationsthatran-domlymaskoutsometimesteps.”
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800 · cited 2×
“However, introducing reasonable inductive bias or prior is helpful for many deep neural networks-based modeling tasks [140], [196], [197].”
BackgroundMethodSelf-citationFinancial time series forecasting with deep learning : A systematic literature review: 2005–2019
Sezer, Ö. B., Gudelek, M. U., & Özbayoğlu, A. M. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106181 · Cited by 793 · cited 1×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundAnomaly Detection for IoT Time-Series Data: A Survey
Cook, A. A., Misirli, G., & Fan, Z. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2958185 · Cited by 790 · cited 1×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundAutoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series ForecastingPreprint
Rasul, K., Seward, C., Schuster, I., & Vollgraf, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.12072 · Cited by 773
Maximum Likelihood Training of Score-Based Diffusion ModelsPreprint
Song, Y. S., Durkan, C., Murray, I., & Ermon, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.09258 · Cited by 759
A Dataset to Support Research in the Design of Secure Water Treatment Systems
Goh, J., Adepu, S., Junejo, K. N., & Mathur, A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-71368-7_8 · Cited by 729
Um, T. T., Pfister, F. M. J., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., & Kulić, D. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3136817 · Cited by 729
Time Series Data Augmentation for Deep Learning: A Survey
Wen, Q., Sun, L., Yang, F., Song, X., Gao, J., Wang, X., & Xu, H. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/631 · Cited by 727 · cited 5×
“The second category focuses on the key components of time series modeling based on deep neural networks, such as data augmentation [33], [45], [46], [47], model structure [33], [48], [49].”
BackgroundMethodInfluentialSelf-citationAn empirical survey of data augmentation for time series classification with neural networks
Iwana, B. K., & Uchida, S. (2021) · PLOS ONE · doi:10.1371/journal.pone.0254841 · Cited by 725
Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J. T., & Peng, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17037 · Cited by 714 · cited 1×
“Therefore,introducingclusteringconstraints intoexistingcontrastivelearningframeworksisanalternative, suchasCC[134],PCL[135],andSwAV[36].”
Time-Series Representation Learning via Temporal and Contextual Contrasting
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Li, X., & Guan, C. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/324 · Cited by 696 · cited 4×
“TS-TCC [116] and its extended version CA-TCC [117] designed a cross prediction task, which uses the context of x T 1 to predict the target in x T 2 , and vice versa uses the context of x T 2 to predict the target in x T 1 .”
MethodInfluentialUnderstanding Diffusion Models: A Unified PerspectivePreprint
Luo, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.11970 · Cited by 679 · cited 1×
“As a new kind of deep generative model, diffusion models have achieved great success recently in many fields, including image synthesis, video generation, speech generation, bioinformatics, and natural language processing due to their powerful generating ability [88], [89], [90], [91], [92].”
BackgroundMultivariate Time Series Imputation with Generative Adversarial Networks
Luo, Y., Cai, X., Zhang, Y., Xu, J., & Yuan, X. (2018) · neural information processing systems · Cited by 627
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
Zhang, R., Isola, P., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.76 · Cited by 616
Prototypical Contrastive Learning of Unsupervised Representations
Li, J., Zhou, P., Xiong, C., & Hoi, S. C. H. (2020) · arXiv · doi:10.48550/arxiv.2005.04966 · Cited by 611
Um, T. T. (n.d.) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 602
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 552
Self-citationWADI: a water distribution testbed for research in the design of secure cyber physical systems
Ahmed, C. M., Palleti, V. R., & Mathur, A. P. (2017) · Proceedings of the 3rd International Workshop on Cyber-Physical Systems for Smart Water Networks · doi:10.1145/3055366.3055375 · Cited by 550
FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.12740 · Cited by 533
Self-citationKirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.02643 · Cited by 527
Graph Self-Supervised Learning: A Survey
Liu, Y., Jin, M., Pan, S., Zhou, C., Zheng, Y., Xia, F., & Yu, P. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3172903 · Cited by 510 · cited 1×
“Using the learning paradigm as a taxonomy is arguably the most popular among the existing SSL surveys, including [20], [22], [23], [24], [25], [26], [27].”
MethodSelf-citationImproved Techniques for Training Score-Based Generative Models
Song, Y., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09011 · Cited by 482
Woo, G., Liu, C., Sahoo, D., Kumar, A., & Hoi, S. C. H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.01575 · Cited by 479
Multi-Time Attention Networks for Irregularly Sampled Time Series
Shukla, S. N., & Marlin, B. M. (2021) · PubMed · doi:10.48550/arxiv.2101.10318 · Cited by 470
Li, Z., Zhao, Y., Han, J., Su, Y., Jiao, R., Wen, X., & Pei, D. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467075 · Cited by 464 · cited 3×
“InterFusion[77], OmniAnomaly[78], GRELEN[79],”
BackgroundPyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting
Liu, S., Yu, H., Liao, C., Li, J., Lin, W., Liu, A. X., & Dustdar, S. (2022) · doi:10.34726/2945 · Cited by 464
Self-Supervised Representation Learning: Introduction, advances, and challenges
Ericsson, L., Gouk, H., Loy, C. C., & Hospedales, T. M. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3134634 · Cited by 433 · cited 1×
“For example, Ericsson et al. [28] provides a very comprehensive review of pretext tasksformultiplemodalities,includingimage,video,text,audio, timeseries,andgraph.”
BackgroundTimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., & Long, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.02186 · Cited by 431
Self-citationPractical Approach to Asynchronous Multivariate Time Series Anomaly Detection and Localization
Abdulaal, A., Liu, Z., & Lancewicki, T. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467174 · Cited by 423 · cited 3×
“Abdulaal et al. [63] focus on the complex asynchronous multivariate time series data and introduce the spectral analysis in the autoencoder model.”
BackgroundMethodSelf-Supervised Learning of Graph Neural Networks: A Unified Review
Xie, Y., Xu, Z., Zhang, J., Wang, Z., & Ji, S. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3170559 · Cited by 389 · cited 1×
“The SSL methods can be roughly divided into the following categories: generative-based, contrastive-based, and adversarial-based methods.”
BackgroundSelf-citationDCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection
Yang, Y., Zhang, C., Zhou, T., Wen, Q., & Sun, L. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599295 · Cited by 387 · cited 1×
“Recently, DCdetec-tor [126] proposes a dual attention contrastive representation framework for time series anomaly detection.”
BackgroundSelf-citationC-RNN-GAN: Continuous recurrent neural networks with adversarial training
Mogren, O. (2016) · arXiv · doi:10.48550/arxiv.1611.09904 · Cited by 386 · cited 2×
“C-RNN-GAN [154] is an early method of generating time series samples using GAN.”
BackgroundMethodDeep Learning for Time Series Forecasting: Tutorial and Literature Survey
Benidis, K., Rangapuram, S. S., Flunkert, V., Wang, Y., Maddix, D., Turkmen, C., Gasthaus, J., Bohlke-Schneider, M., Salinas, D., Stella, L., et al. (2023) · ACM Computing Surveys · doi:10.1145/3533382 · Cited by 382 · cited 2×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundMethodSelf-supervised speech representation learning: A review
Mohamed, A., Lee, H.-Y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., Kirchhoff, K., Li, S.-W., Livescu, K., Maaløe, L., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3207050 · Cited by 372 · cited 1×
“Using the learning paradigm as a taxonomy is arguably the most popular among the existing SSL surveys, including [20], [22], [23], [24], [25], [26], [27].”
MethodAdversarial Risk and the Dangers of Evaluating Against Weak Attacks
Uesato, J., O’Donoghue, B., van den Oord, A., & Kohli, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05666 · Cited by 353
TapNet: Multivariate Time Series Classification with Attentional Prototypical Network
Zhang, X., Gao, Y., Lin, J., & Lu, C.-T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6165 · Cited by 352
Sagheer, A., & Kotb, M. (2019) · Scientific Reports · doi:10.1038/s41598-019-55320-6 · Cited by 351 · cited 2×
“Then removing the decoder D ( · ) and leaving only the encoder E ( · ) that is used as a feature extractor, and the representation z is used for downstream tasks [60], [61], [62], [63].”
MethodData Augmentation techniques in time series domain: a survey and taxonomy
Iglesias, G., Talavera, E., González-Prieto, Á., Mozo, A., & Gómez-Canaval, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08459-3 · Cited by 347 · cited 2×
“Data augmentation is one of the effective methods to generate augmented views in SSCL [47], [193].”
BackgroundMethodDiffusion-based Time Series Imputation and Forecasting with Structured State Space ModelsPreprint
Alcaraz, J. M. L., & Strodthoff, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.09399 · Cited by 347 · cited 1×
“SSSD [102] combines diffusion models and structured state space models for time series imputation andforecastingtasks.”
Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting
Shao, Z., Zhang, Z., Wang, F., & Xu, Y. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539396 · Cited by 338 · cited 1×
“STEP[74]dividedthe seriesintomultiplenon-overlappingsegmentsofequallength andthenrandomlyselectedacertainproportionofthesegments formasking.”
Generative Adversarial Networks in Time Series: A Systematic Literature Review
Brophy, E., Wang, Z., She, Q., & Ward, T. (2023) · ACM Computing Surveys · doi:10.1145/3559540 · Cited by 337 · cited 3×
“The second category focuses on the key components of time series modeling based on deep neural networks, such as data augmentation [33], [45], [46], [47], model structure [33], [48], [49].”
BackgroundMethodBeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time Series
Zhou, B., Liu, S., Hooi, B., Cheng, X., & Ye, J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/616 · Cited by 333 · cited 1×
“Combining time series self-supervised learning and federated learning is a valuable research direction that can provide additional modeling tools for modern distributed systems.”
BackgroundTransfer learning for time series classification
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2018) · 2018 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2018.8621990 · Cited by 319
Unsupervised Scalable Representation Learning for Multivariate Time Series
Franceschi, J.-Y., Dieuleveut, A., & Jaggi, M. (2019) · arXiv · doi:10.48550/arxiv.1901.10738 · Cited by 305
A review on distance based time series classification
Abanda, A., Mori, U., & Lozano, J. A. (2019) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-018-0596-4 · Cited by 296 · cited 1×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundSelf-Supervised Learning on Graphs: Contrastive, Generative, or Predictive
Wu, L., Lin, H., Tan, C., Gao, Z., & Li, S. Z. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3131584 · Cited by 290 · cited 1×
“The SSL methods can be roughly divided into the following categories: generative-based, contrastive-based, and adversarial-based methods.”
BackgroundWind Turbine Fault Detection Using a Denoising Autoencoder With Temporal Information
Jiang, G., Xie, P., He, H., & Yan, J. (2018) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2017.2759301 · Cited by 288 · cited 1×
“In time series modeling, more than one method can add noise to the input, such as adding Gaussian noise [69], [70] and randomly setting some time steps to zero [71], [72].”
MethodFiLM: Frequency Improved Legendre Memory Model for Long-Term Time Series Forecasting
Zhou, T., Ma, Z., Wang, X., Wen, Q., Sun, L., Yao, T., Yin, W., & Jin, R. (2022) · neural information processing systems · doi:10.52202/068431-0921 · Cited by 282
Self-citationGenerative Adversarial Networks in Computer Vision: A Survey and Taxonomy
Wang, Z., She, Q., & Ward, T. E. (2022) · ACM Computing Surveys · doi:10.1145/3439723 · Cited by 267
E²GAN: End-to-End Generative Adversarial Network for Multivariate Time Series Imputation
Luo, Y., Zhang, Y., Cai, X., & Yuan, X. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/429 · Cited by 257 · cited 1×
“One interesting direction is to pay more attention to SSL, analyze its properties in time series modeling tasks, and give reliable benchmark evaluation.”
BackgroundCirstea, R.-G., Guo, C., Yang, B., Kieu, T., Dong, X., & Pan, S. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/277 · Cited by 255
Self-citationDiscrete Graph Structure Learning for Forecasting Multiple Time SeriesPreprint
Shang, C., Chen, J., & Bi, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.06861 · Cited by 254 · cited 1×
“Compared with RNNs and CNNs, Graph Neural Networks (GNNs) can better capture the correlation among variables and constituent in multivariate time series data, such as GDN [55] and GTS [56].”
MethodGenerative Time Series Forecasting with Diffusion, Denoise, and Disentanglement
Li, Y., Li, Y., Wang, Y., & Dou, D. (2022) · neural information processing systems · doi:10.52202/068431-1672 · Cited by 253
Forecast Methods for Time Series Data: A Survey
Liu, Z., Zhu, Z., Gao, J., & Xu, C. (2021) · IEEE Access · doi:10.1109/access.2021.3091162 · Cited by 247 · cited 2×
“The first category focuses on different tasks, such as classification [37], [38], forecasting [39], [40], [41], [42], and anomaly detection [43], [44].”
BackgroundMethodTimeNet: Pre-trained deep recurrent neural network for time series classificationPreprint
Malhotra, P., Tv, V., Vig, L., Agarwal, P., & Shroff, G. M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.08838 · Cited by 243
Integrating Machine Learning with Human Knowledge
Deng, C., Ji, X., Rainey, C., Zhang, J., & Lu, W. (2020) · iScience · doi:10.1016/j.isci.2020.101656 · Cited by 243 · cited 1×
“However, introducing reasonable inductive bias or prior is helpful for many deep neural networks-based modeling tasks [140], [196], [197].”
BackgroundSelf-citationRepresentation Learning: A Review and New PerspectivesPreprint
Bengio, Y., Courville, A. C., & Vincent, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5538 · Cited by 235
RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series
Wen, Q., Gao, J., Song, X., Sun, L., Xu, H., & Zhu, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015409 · Cited by 233 · cited 1×
“On the other hand, time series data usually has some available characteristics, such as seasonal, periodic, trend, and frequency domain biases [198], [199], [200].”
BackgroundSelf-citationAnomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
Xu, J., Wu, H., Wang, J., & Long, M. (2021) · arXiv (Cornell University) · doi:10.4230/lipics.time.2024.17 · Cited by 233
RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural NetworksPreprint
Gao, J., Song, X., Wen, Q., Wang, P., Sun, L., & Xu, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09545 · Cited by 227
Self-citationSelf-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C.-K., Li, X., & Guan, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3308189 · Cited by 224 · cited 2×
“TS-TCC [116] and its extended version CA-TCC [117] designed a cross prediction task, which uses the context of x T 1 to predict the target in x T 2 , and vice versa uses the context of x T 2 to predict the target in x T 1 .”
MethodTimeseries Anomaly Detection using Temporal Hierarchical One-Class Network
Shen, L., Li, Z., & Kwok, J. T.-Y. (2020) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 218
Generative Semi-supervised Learning for Multivariate Time Series Imputation
Miao, X., Wu, Y., Wang, J., Gao, Y., Mao, X., & Yin, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17086 · Cited by 210 · cited 2×
“SSGAN [162] is a semi-supervised framework for time series data imputation, which includes a generative network, a discriminative network, and a classification network.”
BackgroundMethodSelf-Supervised Contrastive Pre-Training for Time Series Via Time-Frequency Consistency
Zhang, X., Zhao, Z., Tsiligkaridis, T., & Žitnik, M. (2022) · neural information processing systems · doi:10.52202/068431-0288 · Cited by 203
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06377 · Cited by 198
Audio self-supervised learning: A survey
Liu, S., Mallol-Ragolta, A., Parada‐Cabaleiro, E., Qian, K., Jing, X., Kathan, A., Hu, B., & Schuller, B. W. (2022) · Patterns · doi:10.1016/j.patter.2022.100616 · Cited by 188 · cited 1×
“Using the learning paradigm as a taxonomy is arguably the most popular among the existing SSL surveys, including [20], [22], [23], [24], [25], [26], [27].”
MethodA Survey on Generative Diffusion ModelPreprint
Cao, H., Tan, C., Gao, Z.-Y., Xu, Y., Chen, G.-Y., Heng, P.-A., & Li, S. (2022) · arXiv.org · doi:10.48550/arxiv.2209.02646 · Cited by 187
TTS-GAN: A Transformer-Based Time-Series Generative Adversarial Network
Li, X., Metsis, V., Wang, H., & Ngu, A. H. H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-09342-5_13 · Cited by 176 · cited 6×
“Expert knowledge contrast Prototype contrast Augmentation contrast C-RNN-GAN[146], TimeGAN[147], TTS-GAN[148],”
BackgroundMethodInfluentialUnsupervised representation learning for time series with temporal neighborhood coding
Tonekaboni, S., Eytan, D., & Goldenberg, A. (2021) · arXiv · doi:10.48550/arxiv.2106.00750 · Cited by 170
ShapeNet: A Shapelet-Neural Network Approach for Multivariate Time Series Classification
Li, G., Choi, B., Xu, J., S Bhowmick, S., Chun, K.-P., & Wong, G. L.-H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.17018 · Cited by 169
DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series
Tan, Q., Ye, M., Yang, B., Liu, S., Ma, A. J., Yip, T. C., Wong, G. L., & Yuen, P. C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5440 · Cited by 168 · cited 1×
“…the mean square error (MSE), i.e., In the time series modeling with autoregressive-based forecasting task as a pretext task, Recurrent neural networks (RNNs) are widely used thanks to their strong capability in spatiotemporal dynamic behavior modeling or sequence prediction [41], [42], [50], [51].”
MethodFeature learning from incomplete EEG with denoising autoencoder
Li, J., Struzik, Z., Zhang, L., & Cichocki, A. (2015) · Neurocomputing · doi:10.1016/j.neucom.2014.08.092 · Cited by 163 · cited 1×
“In time series modeling, more than one method can add noise to the input, such as adding Gaussian noise [66], [67] and randomly setting some time steps to zero [68], [69].”
MethodImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly DetectionPreprint
Chen, Y.-H., Zhang, C., Ma, M.-H., Liu, Y., Ding, R., Li, B., He, S., Rajmohan, S., Lin, Q., & Zhang, D.-M. (2023) · Proceedings of the VLDB Endowment · doi:10.48550/arxiv.2307.00754 · Cited by 160
Self-citationAdversarial Sparse Transformer for Time Series Forecasting
Wu, S., Xiao, X., Ding, Q., Zhao, P., Wei, Y., & Huang, J. (2020) · neural information processing systems · Cited by 159
A Survey on Contrastive Self-Supervised Learning
Jaiswal, A., Babu, A. R., Zadeh, M. Z., Banerjee, D., & Makedon, F. (2020) · Technologies · doi:10.3390/technologies9010002 · Cited by 154 · cited 3×
“Jaiswal et al. [17], Le-Khac et al. [29] and Liu et al. [33] sort out the modules and components required in SSL from different perspectives.”
BackgroundResultDiffusion Models: A Comprehensive Survey of Methods and ApplicationsPreprint
Yang, L. Q., Zhang, Z., Song, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.00796 · Cited by 150
RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data
Xu, Y., Biswal, S., Deshpande, S. R., Maher, K. O., & Sun, J. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220051 · Cited by 144 · cited 1×
“T IME series data abound in many real-world scenarios [1], [2], including human activity recognition [3], industrial fault diagnosis [4], smart building management [5], and healthcare [6].”
BackgroundA survey on face data augmentation for the training of deep neural networks
Wang, X., Wang, K., & Lian, S. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-04748-3 · Cited by 141 · cited 1×
“Data augmentation is one of the effective methods to generate augmented views in SSCL [47], [193].”
MethodImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection
Chen, Y., Zhang, C., Ma, M., Liu, Y., Ding, R., Li, B., He, S., Rajmohan, S., Lin, Q., & Zhang, D. (2023) · Proceedings of the VLDB Endowment · doi:10.14778/3632093.3632101 · Cited by 139
Self-citationLearning Sparse Latent Graph Representations for Anomaly Detection in Multivariate Time Series
Han, S., & Woo, S. S. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539117 · Cited by 137 · cited 2×
“Furthermore,VGCRN[83]andFuSAGNet[67]combine twoframeworkstoachievemorerobustandaccuratere-sults.”
BackgroundTime Series Change Point Detection with Self-Supervised Contrastive Predictive Coding
Deldari, S., Smith, D. V., Xue, H., & Salim, F. D. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3449903 · Cited by 136 · cited 1×
“Following the architectureofCPC,LNT[110],TRL-CPC[111],TS-CP 2 [112], and Skip-Step CPC [113] were proposed.”
MethodContrastive learning based self-supervised time-series analysis
Pöppelbaum, J., Chadha, G. S., & Schwung, A. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2021.108397 · Cited by 131
DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Models
Wen, H., Lin, Y., Xia, Y., Wan, H., Wen, Q., Zimmermann, R., & Liang, Y. (2023) · Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems · doi:10.1145/3589132.3625614 · Cited by 130 · cited 1×
“DiffSTG[104] presentsthefirstshottopredicttheevolutionofspatio-temporal graphsusingDDPMs.”
Self-citationSelf-Supervised Transformer for Sparse and Irregularly Sampled Multivariate Clinical Time-Series
Tipirneni, S., & Reddy, C. K. (2022) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3516367 · Cited by 130 · cited 2×
“Moreover, STraTS [54] first encodes the time series data into triple representations to avoid the limitations of using basic RNN and CNN in modeling irregular and sparse time series data and then builds the transformer-based forecasting model for modeling multivariate medical clinical time series.”
BackgroundMethodCLOCS: Contrastive Learning of Cardiac Signals Across Space, Time, and PatientsPreprint
Kiyasseh, D., Zhu, T., & Clifton, D. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.13249 · Cited by 129
Tang, X., Yao, H., Sun, Y., Aggarwal, C., Mitra, P., & Wang, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6056 · Cited by 125 · cited 1×
“For example, LGnet [192] introduces adversarial training to enhance the modeling of global temporal distribution, which mitigates the impact of missing values on forecasting accuracy.”
BackgroundScaleformer: Iterative Multi-scale Refining Transformers for Time Series ForecastingPreprint
Shabani, A., Abdi, A., Meng, L., & Sylvain, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.04038 · Cited by 125
Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future TrendsPreprint
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00378 · Cited by 124 · cited 1×
“The SSL methods can be roughly divided into the following categories: generative-based, contrastive-based, and adversarial-based methods.”
BackgroundAdversarial Attacks on Time Series
Karim, F., Majumdar, S., & Darabi, H. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2986319 · Cited by 118 · cited 1×
“With the widespread use of deep neural networks in time series forecasting, classification, and anomaly detection, the vulnerability and robustness of deep models under adversarial have become a significant concern [204], [205], [206], [207].”
BackgroundWen, Q., Zhang, Z., Li, Y., & Sun, L. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403271 · Cited by 116 · cited 1×
“First, time series data exhibit unique properties such as seasonality, trend, and frequency domain information [13], [14], [15].”
BackgroundSelf-citationRobustPeriod: Robust Time-Frequency Mining for Multiple Periodicity Detection
Wen, Q., He, K., Sun, L., Zhang, Y., Ke, M., & Xu, H. (2021) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3448016.3452779 · Cited by 116 · cited 1×
“On the other hand, time series data usually has some available characteristics, such as seasonal, periodic, trend, and frequency domain biases [198], [199], [200].”
BackgroundSelf-citationTimeCLR: A self-supervised contrastive learning framework for univariate time series representation
Yang, X., Zhang, Z., & Cui, R. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.108606 · Cited by 114 · cited 1×
“TimeCLR [122] proposed DTW augmentation, which can not only simulate phase shifts and amplitude changes but also retain the structure and characteristics of the time series.”
MethodMixing up contrastive learning: Self-supervised representation learning for time series
Wickstrøm, K., Kampffmeyer, M., Mikalsen, K. Ø., & Jenssen, R. (2022) · Pattern Recognition Letters · doi:10.1016/j.patrec.2022.02.007 · Cited by 113 · cited 1×
“Allthe abovemethodsonlyneedasingletimeseriessampleinthe augmentationoperation,whileMixing-up[122]fusestwotime seriessamplestogenerateanewlyaugmentedview,whilethe pretexttaskistocorrectlypredicttheproportionoftwooriginal timeseriessamplesinaugmentedview.”
Modeling Irregular Time Series with Continuous Recurrent UnitsPreprint
Schirmer, M., Eltayeb, M., Lessmann, S., & Rudolph, M. (2021) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.2111.11344 · Cited by 107
Self-supervised Learning for Semi-supervised Time Series Classification
Jawed, S., Grabocka, J., & Schmidt-Thieme, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-47426-3_39 · Cited by 102 · cited 1×
“In addition to RNNs, the forecasting models based on Convolutional Neural Networks (CNNs) also have been developed [52].”
MethodLearning to Rotate: Quaternion Transformer for Complicated Periodical Time Series Forecasting
Chen, W., Wang, W., Peng, B., Wen, Q., Zhou, T., & Sun, L. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539234 · Cited by 101
Self-citationGRELEN: Multivariate Time Series Anomaly Detection from the Perspective of Graph Relational Learning
Zhang, W., Zhang, C., & Tsung, F. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/332 · Cited by 99 · cited 1×
“In order to better capture the dependencies between different variables in multivariate time series, GRELEN [82] and VGCRN [83] introduce the graph structure and in VAE.”
MethodSelf-citationUnsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral FusionPreprint
Yang, L. Q., & Hong, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.04770 · Cited by 99
Self-Supervised Contrastive Learning for Medical Time Series: A Systematic Review
Liu, Z., Alavi, A., Li, M., & Zhang, X. (2023) · Sensors · doi:10.3390/s23094221 · Cited by 90 · cited 6×
“The second category focuses on the key components of time series modeling based on deep neural networks, such as data augmentation [33], [45], [46], [47], model structure [33], [48], [49].”
BackgroundMethodInfluentialIncremental False Negative Detection for Contrastive LearningPreprint
Chen, T.-S., Hung, W.-C., Tseng, H.-Y., Chien, S., & Yang, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03719 · Cited by 89
The UEA multivariate time series classification archive, 2018
Bagnall, A., Dau, H. A., Lines, J., Flynn, M., Large, J., Bostrom, A., Southam, P., & Keogh, E. (2018) · arXiv · doi:10.48550/arxiv.1811.00075 · Cited by 87
Contrastive Learning for Unsupervised Domain Adaptation of Time SeriesPreprint
Özyurt, Y., Feuerriegel, S., & Zhang, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.06243 · Cited by 83
Self-citationUnsupervised Representation Learning from Pre-Trained Diffusion Probabilistic Models
Zhang, Z., Zhao, Z., & Lin, Z. (2022) · neural information processing systems · doi:10.52202/068431-1607 · Cited by 80
Peng, T., Shen, C., Sun, S., & Wang, D. (2022) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2021.3111567 · Cited by 76 · cited 2×
“Therefore,someworkscomprehensivelycompareand studytheaugmentationmethodsandfurtherevaluatetheper-formanceondifferenttasks[45],[119],[120],[121].”
MethodSmall perturbations are enough: Adversarial attacks on time series prediction
Wu, T., Wang, X., Qiao, S., Xian, X., Liu, Y., & Zhang, L. (2022) · Information Sciences · doi:10.1016/j.ins.2021.11.007 · Cited by 72 · cited 2×
“Wu et al. [206] propose an attack strategy for generating an adversarial time series by adding malicious perturbations to the original time series to deteriorate the performance of time series prediction models.”
BackgroundSelf-citationLabel-Efficient Time Series Representation Learning: A Review
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., & Li, X. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3430236 · Cited by 69
TARNet: Task-Aware Reconstruction for Time-Series Transformer
Chowdhury, R. R., Zhang, X., Shang, J., Gupta, R. K., & Hong, D. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539329 · Cited by 68 · cited 1×
“Unlike random masking, TARNet [77] considers the pre-trained model based on the masking strategy irrelevant to the downstream task, which leads to sub-optimal representations.”
MethodSemi-Supervised Contrastive Learning With Similarity Co-Calibration
Zhang, Y., Zhang, X., Li, J., Qiu, R. C., Xu, H., & Tian, Q. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3158069 · Cited by 67 · cited 1×
“An interesting trend is to fuse semi-supervised learning and contrastive-based SSL to guide the training of the encoder through a small amount of labeled data [146], [147].”
BackgroundGenerating Multivariate Time Series with COmmon Source CoordInated GAN (COSCI-GAN)
Seyfi, A., Rajotte, J.-F., & Ng, R. (2022) · neural information processing systems · doi:10.52202/068431-2375 · Cited by 65
Self-supervised on Graphs: Contrastive, Generative, or Predictive
Wu, L.-R., Lin, H., Gao, Z.-Y., Tan, C., & Li, S. (2021) · arXiv.org · doi:10.48550/arxiv.2105.07342 · Cited by 62
Multi-Granularity Residual Learning with Confidence Estimation for Time Series Prediction
Hou, M., Xu, C., Li, Z., Liu, Y., Liu, W., Chen, E., & Bian, J. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3512056 · Cited by 59 · cited 1×
“Thus one future direction is to consider more effective ways to induce inductive biases into time series SSL based on the understanding of time series data and characteristics of specific tasks.”
BackgroundSelf-citationNavigating the metric maze: a taxonomy of evaluation metrics for anomaly detection in time series
Sørbø, S., & Ruocco, M. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00988-8 · Cited by 58
Robust Time Series Analysis and Applications: An Industrial Perspective
Wen, Q., Yang, L., Zhou, T., & Sun, L. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3542612 · Cited by 58 · cited 1×
“T IME series data abound in many real-world scenarios [1], [2], including human activity recognition [3], industrial fault diagnosis [4], smart building management [5], and health-care [6].”
BackgroundSelf-citationWu, Y., Ni, J., Cheng, W., Zong, B., Song, D., Chen, Z., Liu, Y., Zhang, X., Chen, H., & Davidson, S. B. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i1.16145 · Cited by 57 · cited 1×
“The straightforward approach to deal with irregular and sparse time series data is to use interpolation algorithms to estimate missing values [161], [162], [202].”
MethodSelf-citationSemi-supervised Time Series Classification Model with Self-supervised Learning
Xi, L., Yun, Z., Liu, H., Wang, R., Huang, X., & Fan, H. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105331 · Cited by 56 · cited 1×
“Due to space limitations, relevant descriptions of evaluation metrics, examples, and task flow can be found in Appendix H, available online.”
BackgroundMHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time Series
Meng, Q., Qian, H., Liu, Y., Cui, L., Xu, Y., & Shen, Z. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26098 · Cited by 52
Self-citationMultivariate Time Series Missing Data Imputation Using Recurrent Denoising Autoencoder
Zhang, J., & Yin, P. (2019) · 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8982996 · Cited by 47 · cited 1×
“In time series modeling, more than one method can add noise to the input, such as adding Gaussian noise [69], [70] and randomly setting some time steps to zero [71], [72].”
MethodSelf-citationDenoising temporal convolutional recurrent autoencoders for time series classification
Zheng, Z., Zhang, Z., Wang, L., & Luo, X. (2022) · Information Sciences · doi:10.1016/j.ins.2021.12.061 · Cited by 45 · cited 1×
“In time series modeling, more than one method can add noise to the input, such as adding Gaussian noise [69], [70] and randomly setting some time steps to zero [71], [72].”
MethodAttack and Defense: Adversarial Security of Data-Driven FDC Systems
Zhuo, Y., Yin, Z., & Ge, Z. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3197190 · Cited by 45 · cited 1×
“Zhuo et al. [208] summarize and compare various recent and typical adversarial attack and defense methods for fault classifiers in data-driven fault detection and classificationsystems,includingwhite-boxattack(FGSM[209], IGSM[210],C & W attack [211], DeepFool [212]) and gray-box and black-box attack…”
BackgroundRAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring DataPreprint
Xu, Y., Biswal, S., Deshpande, S. R., Maher, K., & Sun, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08820 · Cited by 44
Neighborhood Contrastive Learning Applied to Online Patient MonitoringPreprint
Yèche, H., Dresdner, G., Locatello, F., Hüser, M., & Rätsch, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05142 · Cited by 42 · cited 2×
“In this section, we point out some critical problems in current studies and outline several research directions worthy of further investigation.”
BackgroundGT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks
Jeon, J., Kim, J., Song, H., Cho, S., & Park, N. (2022) · neural information processing systems · doi:10.52202/068431-2681 · Cited by 42
Integration of knowledge and data in machine learning
Chen, Y., & Zhang, D. (2022) · arXiv · doi:10.48550/arxiv.2202.10337 · Cited by 37 · cited 1×
“Generally speaking, this modeling framework incorporates expert prior knowledge or information into deep neural networks to guide model training [140], [141].”
MethodAdversarial Autoencoder for Unsupervised Time Series Anomaly Detection and Interpretation
Chen, X., Deng, L., Zhao, Y., & Zheng, K. (2023) · Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining · doi:10.1145/3539597.3570371 · Cited by 36
Hou, M., Xu, C., Liu, Y., Liu, W., Bian, J., Wu, L., Li, Z., Chen, E., & Liu, T.-Y. (2021) · Proceedings of the 30th ACM International Conference on Information & Knowledge Management · doi:10.1145/3459637.3482483 · Cited by 35 · cited 2×
“Methods for constructing multiple contrastive views based on multi-granularity or multi-scale augmentations also include MRLF [131], CMLF [115], and SSLAPP [132].”
BackgroundSelf-citationDiffLoad: Uncertainty Quantification in Electrical Load Forecasting With the Diffusion Model
Wang, Z., Wen, Q., Zhang, C., Sun, L., & Wang, Y. (2025) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2024.3449032 · Cited by 34
Self-citationLearning Representations for Incomplete Time Series Clustering
Ma, Q., Chen, C., Li, S., & Cottrell, G. W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17070 · Cited by 34 · cited 1×
“Combining time series self-supervised learning and federated learning is a valuable research direction that can provide additional modeling tools for modern distributed systems.”
BackgroundLearning from Irregularly-Sampled Time Series: A Missing Data PerspectivePreprint
Li, S. C.-X., & Marlin, B. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07599 · Cited by 33
Utilizing Expert Features for Contrastive Learning of Time-Series RepresentationsPreprint
Nonnenmacher, M., Oldenburg, L., Steinwart, I., & Reeb, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.11517 · Cited by 33
Zhang, K., Liu, Y., Gu, Y., Ruan, X., & Wang, J. (2021) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2021.3087503 · Cited by 28 · cited 1×
“T IME series data abound in many real-world scenarios [1], [2], including human activity recognition [3], industrial fault diagnosis [4], smart building management [5], and health-care [6].”
BackgroundSelf-citationDeldari, S., Xue, H., Saeed, A., He, J., Smith, D. V., & Salim, F. D. (2022) · arXiv · doi:10.48550/arxiv.2206.02353 · Cited by 26 · cited 3×
“The taxonomies proposed by Eldele et al. [19], Deldari et al. [20] and Liu et al. [33] are somewhat similar to our proposed taxonomy, i.e., three taxonomies involve time series self-supervised contrastive learning methods.”
MethodResultSelf-Supervised Pre-training for Time Series Classification
Shi, P., Ye, W., & Qin, Z. (2021) · 2021 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533426 · Cited by 25 · cited 1×
“Shi et al. [142] used the DTW distance of time series samples as prior information and believed that two samples with a small DTW distance have a higher similarity.”
BackgroundDebiased Contrastive Learning for Time-Series Representation Learning and Fault Detection
Zhang, K., Cai, R., Zhou, C., & Liu, Y. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3359409 · Cited by 24
Self-citationMulti-Variate Time Series Forecasting on Variable Subsets
Chauhan, J., Raghuveer, A., Saket, R., Nandy, J., & Ravindran, B. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539394 · Cited by 21 · cited 1×
“[73], who defined a new time series forecasting task called variable subset forecast (VSF).”
BackgroundDeep Unsupervised Binary Coding Networks for Multivariate Time Series Retrieval
Zhu, D., Song, D., Chen, Y., Lumezanu, C., Cheng, W., Zong, B., Ni, J., Mizoguchi, T., Yang, T., & Chen, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5497 · Cited by 20 · cited 1×
“DUBCNs [164] and CRLI [165] are used for series retrieval and clustering tasks, respectively.”
MethodSelf-citationAutowarp: Learning a Warping Distance from Unlabeled Time Series Using Sequence AutoencodersPreprint
Abid, A., & Zou, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10107 · Cited by 20
ACT: Adversarial Convolutional Transformer for Time Series Forecasting
Li, Y., Wang, H., Li, J., Liu, C., & Tan, J. (2022) · 2022 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892791 · Cited by 19 · cited 3×
“For example, introducing adversarial training in time series forecasting can improve the accuracy and capture long-term repeated patterns, such as AST [158] and ACT [159].”
BackgroundRelation Inference among Sensor Time Series in Smart Buildings with Metric Learning
Li, S., Hong, D., & Wang, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5900 · Cited by 19 · cited 1×
“T IME series data abound in many real-world scenarios [1], [2], including human activity recognition [3], industrial fault diagnosis [4], smart building management [5], and health-care [6].”
BackgroundLee, H., Seong, E., & Chae, D.-K. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/537 · Cited by 18 · cited 1×
“Methods for constructing multiple contrastive views based on multi-granularity or multi-scale augmentations also include MRLF [131], CMLF [115], and SSLAPP [132].”
BackgroundValve Stiction Detection Using Multitimescale Feature Consistent Constraint for Time-Series Data
Zhang, K., Liu, Y., Gu, Y., Wang, J., & Ruan, X. (2023) · IEEE/ASME Transactions on Mechatronics · doi:10.1109/tmech.2022.3227960 · Cited by 16 · cited 1×
“MTFCC [130] is another view generation method based on multi-scale characteristics, which samples time series samples at multiple scales and considers that the views from the same sample have similar representations, even if their scales are different.”
MethodSelf-citationPSA-GAN: Progressive Self Attention GANs for Synthetic Time SeriesPreprint
Jeha, P., Bohlke‐Schneider, M., Pedro, M., Kapoor, S., Rajbir, S. N., Flunkert, V., Gasthaus, J. A., & Januschowski, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.00981 · Cited by 14
Learning Discriminative Virtual Sequences for Time Series Classification
Dorle, A., Li, F., Song, W., & Li, S. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412099 · Cited by 11 · cited 1×
“TapNet [137] and DVSL [138] are explicit prototypes contrast because explicit prototypes are introduced.”
Background
Research works printed but not linked: 12
- Cleveland, 1990, STL: A seasonal-trend decomposition, J. Official Stat., 6, 3
- 10.5555/3524938.3525087 · doi:10.5555/3524938.3525087
- Dai, Graph-augmented normalizing flows for anomaly detection of multiple time series, 1
- Chen, Deep variational graph convolutional recurrent network for multivariate time series anomaly detection, 3621
- 10.1109/tkde.2024.3361474/mm1 · doi:10.1109/tkde.2024.3361474/mm1
- Zhang, Time series anomaly detection using skip-step contrastive predictive coding, 1
- 10.1371/journal.pone.0254841 · DOI · Paper page
- 10.5555/2969033.2969125 · doi:10.5555/2969033.2969125
- 10.1609/aaai.v35i10.17070 · DOI · Paper page
- Zhang, Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting, 1
- Wang, MICN: Multi-scale local and global context modeling for long-term series forecasting, 1
- Li, Moco SFL: Enabling cross-client collaborative self-supervised learning
Cited by
Jin, M., Koh, H. Y., Wen, Q., Zambon, D., Alippi, C., Webb, G. I., King, I., & Pan, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3443141 · Cited by 567
Self-citationKim, J., Kim, H., Kim, H., Lee, D., & Yoon, S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11223-9 · Cited by 263
Data-driven stock forecasting models based on neural networks: A review
Bao, W., Cao, Y., Yang, Y., Che, H., Huang, J., & Wen, S. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102616 · Cited by 211
Correlation-Aware Spatial–Temporal Graph Learning for Multivariate Time-Series Anomaly Detection
Zheng, Y., Koh, H. Y., Jin, M., Chi, L., Phan, K. T., Pan, S., Chen, Y. P., & Xiang, W. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3325667 · Cited by 158
“Due to the capacity of modeling interrelationship of different entities in various domains, GNNs have been widely used in domains and applications including traffic [35], [42], recommender systems [43], drug discovery [44], [45], and anomaly detection [46], [47].”
BackgroundMethodSelf-citationA Survey on Time-Series Pre-Trained Models
Ma, Q., Liu, Z., Zheng, Z., Huang, Z., Zhu, S., Yu, Z., & Kwok, J. T.-Y. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3475809 · Cited by 182
Metadata
- Authors
- Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., & Pan, S.
- Affiliations
- Zhejiang University (CN)
- Alibaba Group (China) (CN)
- Zhejiang Normal University (CN)
- Monash University (AU)
- Ant Group (China) (CN)
- Hong Kong University of Science and Technology (HK)
- Singapore Management University (SG)
- University of Connecticut (US)
- Griffith University (AU)
- ORCID
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2024-10
- Volume
- 46
- Issue
- 10
- Pages
- 6775-6794
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W4394699135
- Cited by
- 295
- References
- 232
- Citations by year
- 2026: 124 · 2025: 115 · 2024: 51
- FWCI
- 51.74
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, public-domain)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, cc-by-nc-nd)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Griffith Research Online (submittedVersion)
- arXiv (Cornell University)
- Funders
- Key R&D Project of Zhejiang Province (2024C01172)
- National Key R&D Program of China (2021YFB2012300)
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Keywords
- self-supervised learning, time series analysis, contrastive learning, adversarial learning, pre-training and fine-tuning, time series forecasting, time series classification, time series anomaly detection, time series clustering, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.