The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index; none of the 21 sources its claims cite is checked for support yet.
UniMTS: Unified Pre-training for Motion Time Series
Open accessAbstract
Motion time series collected from mobile and wearable devices such as smartphones and smartwatches offer significant insights into human behavioral patterns, with wide applications in healthcare, automation, IoT, and AR/XR due to their low-power, always-on nature. However, given security and privacy concerns, building large-scale motion time series datasets remains difficult, preventing the development of pre-trained models for human activity analysis. Typically, existing models are trained and tested on the same dataset, leading to poor generalizability across variations in device location, device mounting orientation and human activity type. In this paper, we introduce UniMTS, the first unified pre-training procedure for motion time series that generalizes across diverse device latent factors and activities. Specifically, we employ a contrastive learning framework that aligns motion time series with text descriptions enriched by large language models. This helps the model learn the semantics of time series to generalize across activities. Given the absence of large-scale motion time series data, we derive and synthesize time series from existing motion skeleton data with all-joint coverage. Spatio-temporal graph networks are utilized to capture the relationships across joints for generalization across different device locations. We further design rotation-invariant augmentation to make the model agnostic to changes in device mounting orientations. Our model shows exceptional generalizability across 18 motion time series classification benchmark datasets, outperforming the best baselines by 340% in the zero-shot setting, 16.3% in the few-shot setting, and 9.2% in the full-shot setting.
Overview
CoChat Summary
TL;DR
UniMTS is the first unified pre-training procedure for motion time series, using contrastive learning to align physics-simulated motion time series with LLM-augmented text descriptions so that a graph encoder generalizes across device location, orientation, and activity. Pre-trained only on simulated data, it achieves state-of-the-art results on 18 real-world motion time series classification benchmarks, improving over the best baselines by 340% zero-shot, 16.3% few-shot, and 9.2% full-shot.
Key findings
- Zero-shot: UniMTS averages 43.5 (17.9) accuracy, 34.3 (14.2) macro-F1, and 58.9 (19.3) R@2 across 18 datasets, versus the best baseline ImageBind at 19.3 (14.6) accuracy, 11.0 (7.8) F1, and 24.0 (20.0) R@2; the improvement is statistically significant (e.g., p = 8 × 10⁻⁶ for ImageBind).
- Full-shot: UniMTS averages 90.6 (10.6) accuracy and 87.5 (13.4) macro-F1, outperforming pre-trained, self-supervised, and conventional baselines (e.g., SHARE at 85.8 (13.2) accuracy; p = 0.018 for the best baseline).
- Ablations in the zero-shot setting reduce average accuracy from 43.5 to 27.1 without rotation-invariant augmentation, 39.8 without text augmentation, and 25.1 when the graph encoder is replaced by a CNN that concatenates joints.
- Efficiency: the graph encoder has 4.94M parameters versus 18.69M for ImageBind's IMU encoder; full-shot fine-tuning on UCI-HAR converges in ~1.3 minutes versus ~9.8 minutes for ImageBind; estimated power on an eNPU is ~22.64 mW versus ~702 mW.
Methods
- Design: contrastive learning (CLIP-style) aligning a spatio-temporal graph convolutional encoder over simulated motion time series with a frozen CLIP text encoder; rotation-invariant augmentation samples a random rotation matrix per joint per iteration; random joint masking (1–5 joints selected) emulates partial device coverage.
- Data: pre-training corpus is HumanML3D (14,616 3D motion skeletons, 28.59 hours, average sequence length 7.1 s, 44,970 text descriptions, SMPL 22-joint structure); accelerations and angular velocities are synthesized from skeleton positions/orientations via motion equations with added Gaussian noise; text is augmented with GPT-3.5 ("gpt-3.5-turbo") generating k = 3 paraphrases. Evaluation uses 18 real-world datasets spanning head, chest, back, arm, wrist, waist, hip, leg, knee, and ankle, grouped into easy (<10 activities), medium (10–20), and hard (>20) levels.
- Analysis: Adam optimizer, learning rate 0.0001, batch size 64, ~13 GB memory on a single NVIDIA A100; real data resampled to 20 Hz and normalized; metrics are accuracy, macro-F1, and R@2, with Wilcoxon signed-rank tests with Holm's α (5%).
Limitations
- Stated: simulated time series are only approximations of real signals, which are collected near rather than directly on body joints (e.g., smartwatch near the wrist); the framework is currently limited to classification and not extended to tasks such as inertial navigation; pre-training uses only existing motion datasets, with plans to add video-based pose estimation data; deployment optimization (quantization, pruning, distillation) is left to future work.
- Evident from the text: pre-training relies on a single skeleton corpus (HumanML3D) after KIT-ML and NTU RGB+D 120 yielded no improvement; the text encoder is frozen CLIP, so text-side adaptation is not learned; and the paper reports no evaluation on inertial navigation or other non-classification tasks.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 2 Not valid · 13 Not checked · 56 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Following the same Wilcoxon-signed rank test as in the zero-shot setting, we observe p-values far below 0.05 (e.g., p-value = 2 × 10 -25 for the best-performing baseline ImageBind), indicating the statistical significance of our improvement.”
§ UniMTS: Unified Pre-training for Motion Time Series > 4.3 Few-Shot Fine-tuning Results
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Following the same Wilcoxon-signed rank test, we observe p-values far below 0.05 (e.g., p-value = 0.018 for the best-performing baseline), indicating the statistical significance of our improvement.”
§ UniMTS: Unified Pre-training for Motion Time Series > 4.4 Full-Shot Results
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Simulated motion time series closely resemble patterns of the real PAMAP2 time series.”
§ UniMTS: Unified Pre-training for Motion Time Series > Broad Impact.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe consistently similar patterns between simulated motion time series and real PAMAP2 data across activities of various intensity levels, ranging from sitting to walking and rope jumping.”
§ UniMTS: Unified Pre-training for Motion Time Series > Additional information
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“UniMTS also demonstrates state-of-the-art performance in the few-shot fine-tuning setting, showing the effectiveness of pre-training.”
§ UniMTS: Unified Pre-training for Motion Time Series > 4.3 Few-Shot Fine-tuning Results
boostedn = 10 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
29 more claims
Create a free account to see all 71 claims and every source verdict.
- It may state two separate claims as one. 29
- The quote only partly states this claim. 3
- Probably not a substantial claim. 10
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01230 · Cited by 2,071
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding
Liu, J., Shahroudy, A., Perez, M., Wang, G., Duan, L.-Y., & Kot, A. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916873 · Cited by 1,818
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456
A Transformer-based Framework for Multivariate Time Series Representation Learning
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467401 · Cited by 1,278
Research works printed but not linked: 9
- Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pages 65-70, 1979.
- Rong Hu, Ling Chen, Shenghuan Miao, and Xing Tang. Swl-adapt: An unsupervised domain adaptation model with sample weight learning for cross-user wearable human activity recog- nition. In Proceedings of the AAAI Conference on artificial intelligence, volume 37, pages 6012-6020, 2023.
- Bing Li, Wei Cui, Wei Wang, Le Zhang, Zhenghua Chen, and Min Wu. Two-stream convolution augmented transformer for human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 286-293, 2021.
- Haojie Ma, Wenzhong Li, Xiao Zhang, Songcheng Gao, and Sanglu Lu. Attnsense: Multi-level attention mechanism for multimodal human activity recognition. In IJCAI, pages 3109-3115, 2019.
- Terry T Um, Franz MJ Pfister, Daniel Pichler, Satoshi Endo, Muriel Lang, Sandra Hirche, Urban Fietzek, and Dana Kulić. Data augmentation of wearable sensor data for parkinson's disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM international conference on multimodal interaction, pages 216-220, 2017.
- Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The eleventh international conference on learning representations, 2022.
- Huatao Xu, Pengfei Zhou, Rui Tan, and Mo Li. Practically adopting human activity recogni- tion. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, pages 1-15, 2023.
- Huatao Xu, Pengfei Zhou, Rui Tan, Mo Li, and Guobin Shen. Limu-bert: Unleashing the potential of unlabeled data for imu sensing applications. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, pages 220-233, 2021.
- Alexander D Young, Martin J Ling, and Damal K Arvind. Imusim: A simulation environment for inertial sensing algorithm design and evaluation. In Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks, pages 199-210. IEEE, 2011.
Cited by
Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals
Luo, Y., Chen, Y., Salekin, A., & Rahman, T. (2026) · ACM Transactions on Computing for Healthcare · doi:10.1145/3803808 · Cited by 52
Foundation Models for CPS-IoT: Opportunities and ChallengesPreprint
Baris, O., Chen, Y., Dong, G., Han, L., Kimura, T., Quan, P., Wang, R., Wang, T., Abdelzaher, T., Berges, M. E., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.16368 · Cited by 30
Large Language Model-guided Semantic Alignment for Human Activity Recognition
Yan, H., Tan, H., Ding, Y. C., Zhou, P., Namboodiri, V., & Yang, Y. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770652 · Cited by 31
Wei, Q., Huang, J., Gao, Y. C., & Dong, W. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749509 · Cited by 21
SHADE-AD: An LLM-Based Framework for Synthesizing Activity Data of Alzheimer’s Patients
Fu, H., Chen, H., Lin, S., & Xing, G. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722062 · Cited by 10
Metadata
- Authors
- Zhang, X., Teng, D., Chowdhury, R., Li, S., Hong, D., Gupta, R., & Shang, J.
- Venue
- Advances in Neural Information Processing Systems 37
- Published
- 2024
- Pages
- 107469-107493
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4415795924
- Cited by
- 8
- References
- 9
- Citations by year
- 2026: 7 · 2025: 1
- FWCI
- 3.05
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2333790, CAREER, 2239440)
- Semiconductor Research Corporation (JUMP 2.0)
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- pre-training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: UniMTS: Unified Pre-training for Motion Time Series. https://app.cochat.ai/papers/10.52202/079017-3414
- The paper
- Zhang, X., Teng, D., Chowdhury, R., Li, S., Hong, D., Gupta, R., & Shang, J. (2024). UniMTS: Unified Pre-training for Motion Time Series. Advances in Neural Information Processing Systems 37. https://doi.org/10.52202/079017-3414
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.