Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 38

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

TSGBench: Time Series Generation Benchmark

Ang, Yihao, Huang, Qiang, Bao, Yifan, Tung, Anthony K. H., Huang, Zhiyong · 2023 · Proceedings of the VLDB Endowment

Abstract

Synthetic Time Series Generation (TSG) is crucial in a range of applications, including data augmentation, anomaly detection, and privacy preservation. Although significant strides have been made in this field, existing methods exhibit three key limitations: (1) They often benchmark against similar model types, constraining a holistic view of performance capabilities. (2) The use of specialized synthetic and private datasets introduces biases and hampers generalizability. (3) Ambiguous evaluation measures, often tied to custom networks or downstream tasks, hinder consistent and fair comparison. To overcome these limitations, we introduce TSGBench, the inaugural Time Series Generation Benchmark, designed for a unified and comprehensive assessment of TSG methods. It comprises three modules: (1) a curated collection of publicly available, real-world datasets tailored for TSG, together with a standardized preprocessing pipeline; (2) a comprehensive evaluation measures suite including vanilla measures, new distance-based assessments, and visualization tools; (3) a pioneering generalization test rooted in Domain Adaptation (DA), compatible with all methods. We have conducted comprehensive experiments using TSGBench across a spectrum of ten real-world datasets from diverse domains, utilizing ten advanced TSG methods and twelve evaluation measures. The results highlight the reliability and efficacy of TSGBench in evaluating TSG methods. Crucially, TSGBench delivers a statistical analysis of the performance rankings of these methods, illuminating their varying performance across different datasets and measures and offering nuanced insights into the effectiveness of each method.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.14778/3632093.3632097

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 85

  1. Ahmed Alaa, Boris Van Breugel, Evgeny S Saveliev, and Mihaela van der Schaar. 2022. How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models. In ICML. 290--306.
  2. Ahmed M. Alaa, Alex James Chan, and Mihaela van der Schaar. 2021. Generative Time-series Modeling with Fourier Flows. In ICLR.
  3. Yihao Ang, Qiang Huang, Anthony KH Tung, and Zhiyong Huang. 2023. A Stitch in Time Saves Nine: Enabling Early Anomaly Detection with Correlation Analysis. In ICDE. 1832--1845.
  4. Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. 2018. The UEA multivariate time series classification archive, 2018. arXiv preprint arXiv:1811.00075 (2018). · arXiv
  5. André Bauer, Marwin Züfle, Simon Eismann, Johannes Grohmann, Nikolas Herbst, and Samuel Kounev. 2021. Libra: A Benchmark for Time Series Forecasting Methods. In ICPE. 189--200.
  6. Donald J Berndt and James Clifford. 1994. Using dynamic time warping to find patterns in time series. In KDD Workshop. 359--370.
  7. Ruichu Cai Jiawei Chen Zijian Li Wei Chen Keli Zhang Junjian Ye Zhuozhang Li Xiaoyan Yang and Zhenjie Zhang. 2021. Time Series Domain Adaptation via Sparse Associative Structure Alignment. In AAAI. 6859--6867. · DOI
  8. 10.14778/3494124.3494142 · DOI
  9. Luis Candanedo. 2017. Appliances energy prediction. UCI Machine Learning Repository. 10.24432/C5VC8G · doi:10.24432/c5vc8g
  10. Tian Qi Chen Yulia Rubanova Jesse Bettencourt and David Duvenaud. 2018. Neural Ordinary Differential Equations. In NeurIPS. 6572--6583.
  11. Razvan-Gabriel Cirstea Bin Yang Chenjuan Guo Tung Kieu and Shirui Pan. 2022. Towards Spatio- Temporal Aware Traffic Time Series Forecasting. In ICDE. 2900--2913. · DOI
  12. William Jay Conover and Ronald L Iman. 1979. On multiple-comparisons procedures. Los Alamos Sci. Lab. Tech. Rep. LA-7677-MS 1 (1979), 14.
  13. 10.1109/JAS.2019.1911747 · DOI · Paper page
  14. Ruizhi Deng, Bo Chang, Marcus A. Brubaker, Greg Mori, and Andreas M. Lehrmann. 2020. Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows. In NeurIPS. 7805--7815.
  15. Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. 2021. TimeVAE: A Variational Auto-Encoder for Multivariate Time Series Generation. arXiv preprint arXiv:2111.08095 (2021). · arXiv
  16. Daizong Ding Mi Zhang Yuanmin Huang Xudong Pan Fuli Feng Erling Jiang and Min Yang. 2022. Towards backdoor attack on deep learning based time series classification. In ICDE. 1274--1287. · DOI
  17. Laurent Dinh Jascha Sohl-Dickstein and Samy Bengio. 2016. Density estimation using Real NVP. In ICLR.
  18. Chris Donahue, Julian J. McAuley, and Miller S. Puckette. 2019. Adversarial Audio Synthesis. In ICLR.
  19. Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. 2017. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633 (2017). · arXiv
  20. Vincent Fortuin Dmitry Baranchuk Gunnar Rätsch and Stephan Mandt. 2020. GP-VAE: Deep Probabilistic Multivariate Time Series Imputation. In AISTATS. 1651--1661.
  21. Vincent Fortuin Matthias Hiiser Francesco Locatello Heiko Strathmann and Gunnar Rätsch.2019. SOM-VAE: Interpretable Discrete Representation Learning on Time Series. In ICLR.
  22. Jean-Yves Franceschi Aymeric Dieuleveut and Martin Jaggi. 2019. Unsupervised Scalable Representation Learning for Multivariate Time Series. In NeurIPS. 4652--4663.
  23. 10.1080/01621459.1937.10503522 · DOI · Paper page
  24. 10.1161/01.CIR.101.23.e215 · DOI · Paper page
  25. Ian J Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In NIPS. 2672--2680.
  26. Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander J. Smola. 2006. A Kernel Method for the Two-Sample-Problem. In NIPS. 513--520.
  27. Yuanzhe Hao Xiongpai Qin Yueguo Chen Yaru Li Xiaoguang Sun Yu Tao Xiao Zhang and Xiaoyong Du. 2021. TS-Benchmark: A Benchmark for Time Series Databases. In ICDE. 588--599. · DOI
  28. Martin Heusel Hubert Ramsauer Thomas Unterthiner Bernhard Nessler and Sepp Hochreiter. 2017. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In NIPS. 6629--6640.
  29. Hailin Hu, MingJian Tang, and Chengcheng Bai. 2020. Datsing: Data augmented time series forecasting with adversarial domain adaptation. In CIKM. 2061--2064.
  30. 10.24432/C51P50 · doi:10.24432/c51p50
  31. 10.14778/3476249.3476307 · DOI
  32. Daniel Jarrett Ioana Bica and Mihaela van der Schaar. 2021. Time-series Generation by Contrastive Imitation. In NeurIPS. 28968--28982.
  33. 10.1016/j.mlwa.2020.100001 · DOI
  34. Paul Jeha, Michael Bohlke-Schneider, Pedro Mercado, Shubham Kapoor, Rajbir Singh Nirwan, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. 2021. PSA-GAN: Progressive self attention GANs for synthetic time series. In ICLR.
  35. Jinsung Jeon Jeonghak Kim Haryong Song Seunghyeon Cho and Noseong Park. 2022. GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks. In NeurIPS. 36999--37010.
  36. 10.1038/sdata.2016.35 · DOI · Paper page
  37. James Jordon Jinsung Yoon and Mihaela Van Der Schaar. 2018. PATE-GAN: Generating synthetic data with differential privacy guarantees. In ICLR.
  38. Eamonn Keogh and Shruti Kasetty. 2002. On the need for time series data mining benchmarks: a survey and empirical demonstration. In KDD. 102--111. · DOI · Paper page
  39. Patrick Kidger James Foster Xuechen Li and Terry J Lyons. 2021. Neural SDEs as Infinite-Dimensional GANs. In ICML. 5453--5463.
  40. Diederik P. Kingma and Prafulla Dhariwal. 2018. Glow: Generative Flow with Invertible 1x1 Convolutions. In NeurIPS. 10236--10245.
  41. Diederik P Kingma Danilo J Rezende Shakir Mohamed and Max Welling. 2014. Semi-supervised learning with deep generative models. In NIPS. 3581--3589.
  42. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013). · arXiv
  43. Guokun Lai Wei-Cheng Chang Yiming Yang and Hanxiao Liu. 2018. Modeling Long-and Short-Term Temporal Patterns with Deep Neural Networks. In SIGIR. 95--104. · DOI · Paper page
  44. Kwei-Herng Lai Daochen Zha Junjie Xu Yue Zhao Guanchu Wang and Xia Hu. 2021. Revisiting Time Series Outlier Detection: Definitions and Benchmarks. In NeurIPS datasets and benchmarks track (round 1).
  45. Daesoo Lee Sara Malacarne and Erlend Aune. 2023. Vector Quantized Time Series Generation with a Bidirectional Prior Model. In AISTATS. 7665--7693.
  46. Guozhong Li, Byron Choi, Jianliang Xu, Sourav S Bhowmick, Daphne Ngar-yin Mah, and Grace LH Wong. 2022. IPS: Instance Profile for Shapelet Discovery for Time Series Classification. In ICDE. 1781--1793.
  47. Hongming Li Shujian Yu and Jose Principe. 2023. Causal Recurrent Variational Autoencoder for Medical Time Series Generation. In AAAI. 8562--8570. · DOI
  48. Xiaomin Li, Vangelis Metsis, Huangyingrui Wang, and Anne Hee Hiong Ngu. 2022. Tts-gan: A transformer-based time-series generative adversarial network. In AIME. 133--143.
  49. Haksoo Lim, Minjung Kim, Sewon Park, and Noseong Park. 2023. Regular Time-series Generation using SGM. arXiv preprint arXiv:2301.08518 (2023). · arXiv
  50. Zinan Lin Alankar Jain Chen Wang Giulia Fanti and Vyas Sekar. 2020. Using GANs for Sharing Networked Time Series Data: Challenges Initial Promise and Open Questions. In IMC. 464--483. · DOI · Paper page
  51. Spyros Makridakis, Evangelos Spiliotis, et al. 2021. The M5 competition and the future of human expertise in forecasting. Foresight: The International Journal of Applied Forecasting 60 (2021), 33--37.
  52. 10.1016/j.ijforecast.2018.06.001 · DOI · Paper page
  53. Wannes Meert, Kilian Hendrickx, Toon Van Craenendonck, Pieter Robberechts, Hendrik Blockeel, and Jesse Davis. 2020. DTAIDistance.
  54. Olof Mogren. 2016. C-RNN-GAN: A continuous recurrent neural network with adversarial training. In Constructive Machine Learning Workshop (CML) at NIPS 2016. 1.
  55. 10.1145/3490354.3494393 · doi:10.1145/3490354.3494393
  56. Hao Ni, Lukasz Szpruch, Magnus Wiese, Shujian Liao, and Baoren Xiao. 2020. Conditional Sig-Wasserstein GANs for Time Series Generation. arXiv preprint arXiv:2006.05421 (2020). · arXiv
  57. Alexander Nikitin, Letizia Iannucci, and Samuel Kaski. 2023. TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series. arXiv preprint arXiv:2305.11567 (2023). · arXiv
  58. Alan V Oppenheim. 1999. Discrete-time signal processing. Pearson Education India.
  59. 10.14778/3551793.3551830 · DOI
  60. 10.14778/3529337.3529354 · DOI
  61. Emanuel Parzen. 1963. On spectral analysis with missing observations and amplitude modulation. Sankhyā: The Indian Journal of Statistics, Series A (1963), 383--392.
  62. Hengzhi Pei Kan Ren Yuqing Yang Chang Liu Tao Qin and Dongsheng Li. 2021. Towards generating real-world time series data. In ICDM. 469--478. · DOI
  63. Mohamed Ragab, Emadeldeen Eldele, Wee Ling Tan, Chuan-Sheng Foo, Zhenghua Chen, Min Wu, Chee-Keong Kwoh, and Xiaoli Li. 2023. ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series Data. TKDD (2023), 1--18.
  64. Giorgia Ramponi, Pavlos Protopapas, Marco Brambilla, and Ryan Janssen. 2018. T-CGAN: Conditional Generative Adversarial Network for Data Augmentation in Noisy Time Series with Irregular Sampling. arXiv preprint arXiv:1811.08295 (2018). · arXiv
  65. Sangeeta Rani and Geeta Sikka. 2012. Recent Techniques of Clustering of Time Series Data: A Survey. International Journal of Computer Applications 52, 15 (2012).
  66. Carl Remlinger, Joseph Mikael, and Romuald Elie. 2022. Conditional Loss and Deep Euler Scheme for Time Series Generation. In AAAI, Vol. 36. 8098--8105. · DOI
  67. Jorge Reyes-Ortiz Davide Anguita Alessandro Ghio Luca Oneto and Xavier Parra. 2012. Human Activity Recognition Using Smartphones. UCI Machine Learning Repository. 10.24432/C54S4K · DOI
  68. Oliver Roesler. 2013. EEG Eye State. UCI Machine Learning Repository. 10.24432/C57G7J · doi:10.24432/c57g7j
  69. Yulia Rubanova Ricky T. Q. Chen and David K Duvenaud. 2019. Latent Ordinary Differential Equations for Irregularly-Sampled Time Series. In NeurIPS. 5320--5330.
  70. Ali Seyfi, Jean-Francois Rajotte, and Raymond T. Ng. 2022. Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN). In NeurIPS. 32777--32788.
  71. 10.1007/s10618-016-0455-0 · DOI · Paper page
  72. Kaleb E Smith and Anthony O Smith. 2020. Conditional GAN for timeseries generation. arXiv preprint arXiv:2006.16477 (2020). · arXiv
  73. Padmanaba Srinivasan and William J Knottenbelt. 2022. Time-series Transformer Generative Adversarial Networks. arXiv preprint arXiv:2205.11164 (2022). · arXiv
  74. 10.21105/joss.01169 · DOI · Paper page
  75. Aäron van den Oord Oriol Vinyals and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. In NIPS. 6306--6315.
  76. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing Data using t-SNE. JMLR 9, 11 (2008).
  77. Lei Wang Liang Zeng and Jian Li. 2023. AEC-GAN: Adversarial Error Correction GANs for Auto-Regressive Long Time-Series Generation. In AAAI. 10140--10148. · DOI
  78. 10.1080/14697688.2020.1730426 · DOI · Paper page
  79. Garrett Wilson, Janardhan Rao Doppa, and Diane J Cook. 2020. Multi-source deep domain adaptation with weak supervision for time-series sensor data. In KDD. 1768--1778.
  80. 10.14778/3503585.3503604 · DOI
  81. Tianlin Xu, Li Kevin Wenliang, Michael Munn, and Beatrice Acciaio. 2020. COTGAN: Generating Sequential Data via Causal Optimal Transport. In NeurIPS. 8798--8809.
  82. 10.1038/s41467-022-35295-1 · DOI
  83. Jinsung Yoon Daniel Jarrett and Mihaela van der Schaar. 2019. Time-series Generative Adversarial Networks. In NeurIPS. 5509--5519.
  84. Yu Zheng Xiuwen Yi Ming Li Ruiyuan Li Zhangqing Shan Eric Chang and Tianrui Li. 2015. Forecasting fine-grained air quality based on big data. In KDD. 2267--2276. · DOI · Paper page
  85. Linqi Zhou Michael Poli Winnie Xu Stefano Massaroli and Stefano Ermon. 2023. Deep Latent State Space Models for Time-Series Generation. In ICML. 42625--42643.

Metadata

Authors
Ang, Y., Huang, Q., Bao, Y., Tung, A. K. H., & Huang, Z.
Venue
Proceedings of the VLDB Endowment
Published
2023-11
Volume
17
Issue
3
Pages
305-318
ISSN
2150-8097
Cited by
38
References
2
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: TSGBench: Time Series Generation Benchmark. https://app.cochat.ai/papers/10.14778/3632093.3632097
The paper
Ang, Y., Huang, Q., Bao, Y., Tung, A. K. H., & Huang, Z. (2023). TSGBench: Time Series Generation Benchmark. Proceedings of the VLDB Endowment. https://doi.org/10.14778/3632093.3632097

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.