COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
Open accessPublished version of a preprint
The DOI matches its Crossref record; 62 of the 77 references it cites were found in a registry or an index, and 2 are not checked yet; 9 of the 26 sources its claims cite were checked for support.
Abstract
The goal of creating intelligent, human-centered wearable systems for continuous activity understanding faces a fundamental trade-off: Egocentric video-based models capture rich semantic information and have demonstrated strong performance in human activity recognition (HAR), but their high power consumption, privacy concerns, and dependence on lighting limit their feasibility for continuous on-device recognition. In contrast, inertial measurement unit (IMU) sensors offer an energy-efficient, privacy-preserving alternative, yet lack large-scale annotated datasets, leading to weaker generalization. To bridge this gap, we propose COMODO, a cross-modal self-supervised distillation framework that transfers semantic knowledge from video to IMU without requiring labels. COMODO leverages a pretrained and frozen video encoder to construct a dynamic instance queue to align the feature distributions of video and IMU embeddings. This enables the IMU encoder to inherit rich semantic structure from video while maintaining its efficiency for real-world applications. Experiments on multiple egocentric HAR datasets show that COMODO consistently improves downstream performance, matching or surpassing fully supervised models, and demonstrating strong cross-dataset generalization. Benefiting from its simplicity and flexibility, COMODO is compatible with diverse pretrained video and time-series models, offering the potential to leverage more powerful teacher and student foundation models in future ubiquitous computing research. The code is available at this repository: https://github.com/cruiseresearchgroup/COMODO.
Verified 2026-10-10 · 2 references not checked yet
Overview
CoChat Summary
TL;DR
COMODO is a cross-modal self-supervised distillation framework that transfers semantic knowledge from a frozen pretrained video encoder to a lightweight IMU encoder via a dynamic FIFO queue and similarity-distribution alignment, without IMU labels. Across three egocentric HAR datasets, it matches or surpasses fully supervised baselines and improves cross-dataset generalization while retaining IMU inference efficiency.
Key findings
- On IMU-based HAR, COMODO achieved A@1 of 59.13 on Ego4D, 84.92 on EgoExo4D, and 92.48 on MMEA, outperforming IMU2CLIP by 4.3%, 5.46%, and 1.07% A@1, respectively.
- In cross-dataset transfer, COMODO improved over IMU2CLIP, CrossHAR, and supervised fine-tuned Mantis in all cases; e.g., EgoExo4D→Ego4D A@1 was 56.62 vs. 51.75 for IMU2CLIP and 52.58 for Mantis†, and Ego4D→EgoExo4D A@1 was 82.54.
- Efficiency: COMODO (8.1M params) required 1.70G FLOPs and 75.7ms latency for a 5-second window, versus TimeSformer-Base-k400 (121.4M params) at 196.05G FLOPs and 3590.3ms, giving 47× speedup and 115× FLOPs reduction while retaining 78.8% accuracy vs. 89.8%.
- Ablations showed concat pooling outperformed mean pooling (e.g., MMEA A@1 +6.84%), FIFO sampling outperformed random sampling, and optimal queue size was dataset-specific (Ego4D 16384, EgoExo4D 1024, MMEA 2048); COMODO also outperformed L2 and InfoNCE baselines.
Methods
- Design: a frozen pretrained video teacher (default TimeSformer-base; VideoMAE-Base-k400 also tested) and a trainable IMU student (default Mantis; MOMENT-small also tested) are projected to L2-normalized embeddings; a FIFO queue stores video embeddings, and the IMU encoder is trained by minimizing cross-entropy between the softmax similarity distributions of video and IMU embeddings, equivalent to KL alignment.
- Data: three egocentric HAR datasets—Ego4D (31 activities; 253K train pairs), EgoExo4D (8 activities; 109K train pairs), and MMEA (32 activities; 4553 train pairs). Ego4D/EgoExo4D were preprocessed to 200 Hz IMU, 5-second non-overlapping windows, and 224×224 video at 10 FPS; MMEA used 25 Hz IMU and official splits.
- Analysis: trained for 20 epochs with batch size 128, learning rate 3e-4, τ_v=0.1, τ_x=0.05, on a single NVIDIA L40S GPU with seed 42; evaluated by training an RBF-kernel SVM on frozen IMU features and reporting Accuracy@1/@3/@5 against supervised and self-supervised baselines, including cross-dataset transfer.
Limitations
- Stated in the paper: COMODO relies on synchronized video-IMU data pairs during training; precise temporal alignment between heterogeneous modalities remains challenging, though similarity-distribution alignment mitigates it; and effectiveness is tied to the representational capacity of the pretrained video teacher.
- Stated in the paper: the current framework uses one-way knowledge transfer from a frozen teacher rather than mutual distillation, and does not explicitly consider personalization or contextual adaptation.
- Evident from the text: downstream evaluation still uses labels to train an SVM on extracted features, so the full pipeline is not label-free; cross-dataset generalization is tested only across the three egocentric datasets, and standalone IMU datasets cannot be used without paired video.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 2 Not valid · 8 Not checked · 141 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“while IMU2CLIP reports results on Ego4D, the IMU2CLIP paper does not provide official benchmark splits.”
What the sources say
- Partly supportedcites [40] [40] Seungwhan Moon, Andrea Madotto, Zhaojiang Lin, Aparajita Saraf, Amy Bearman, and Babak Damavandi. 2023. IMU2CLIP: Languagegrounded Motion Sensor Translation with Multimodal Contrastive Learning. In Findings of the Association for Computational Linguistics: EMNLP 2023 . 13246-13253. - sourceExperimental — accuracy not yet measured
The Results section reports Ego4D test-set results. The Methods section says the authors split the data into training, validation, and test sets by video ID, and Table 1 gives split sizes. Our copy does not say whether those splits were provided as official benchmark splits, so it supports the results portion but does not establish the claim about their absence. (LLM, conf=0.77)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Continuous wearable video is often untenable due to power consumption and privacy concerns [4, 36, 41, 43].”
What the sources say
- Not checked — no full text of the sourcecites [4] [4] Divyanshu Bhardwaj, Alexander Ponticello, Shreya Tomar, Adrian Dabrowski, and Katharina Krombholz. 2024. In Focus, Out of Privacy: The Wearer's Perspective on the Privacy Dilemma of Camera Glasses. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 577, 18 pages. doi:10.1145/3613904.3642242 - source
- Not checked — no full text of the sourcecites [36] [36] Si Liao, Hanwei He, Huangxun Chen, and Zhice Yang. 2025. Bystander Privacy in Video Sharing Era: Automated Consent Compliance through Platform Censorship. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25) . Association for Computing Machinery, New York, NY, USA, Article 902, 16 pages. doi:10.1145/3706598.3713391 - source
- Partly supportedcites [41] [41] Tamara Mujirishvili, Anton Fedosov, Kooshan Hashemifard, Pau Climent-Pérez, and Francisco Florez-Revuelta. 2024. 'I Don't Want to Become a Number': Examining Different Stakeholder Perspectives on a Video-Based Monitoring System for Senior Care with Inherent Privacy Protection (by Design).. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '24) . Association for Computing Machinery, New York, NY, USA, Article 774, 19 pages. doi:10.1145/3613904.3642164 - sourceExperimental — accuracy not yet measured
Section 2.3 (page unavailable) says cameras in private places raise privacy concerns that hinder their acceptability. It does not say that continuous wearable video is often untenable, or identify power consumption as a reason; its discussion of power efficiency in Section 3.0.1 concerns the hardware chosen for a senior-care monitoring system. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [43] [43] Octav Opaschi and Radu-Daniel Vatavu. 2020. Uncovering Practical Security and Privacy Threats for Connected Glasses with Embedded Video Cameras. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4, Article 167 (Dec. 2020), 26 pages. doi:10.1145/3432700 - source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As illustrated in Figure 6, COMODO achieves a superior balance between recognition accuracy and computational cost.”
plainstates an associationn = 100- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, despite sharing the same architecture and hardware footprint as IMU2CLIP, COMODO consistently achieves higher accuracy (+3.6% mean A@1), demonstrating that similarity distribution distillation enhances semantic representations without introducing additional inference overhead.”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is further evidenced by the fact that the combination of the stronger Mantis student with either teacher consistently yields the best and second-best results overall.”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
88 more claims
Create a free account to see all 157 claims and to verify the 6 references still open.
- Probably not a substantial claim. 31
- It may state two separate claims as one. 44
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.6 Ablation Study · p. 15
“The results in Table 5 reveal that COMODO consistently and substantially outperforms the IMU2CLIP across every teacher-student configuration on all datasets.”
§ 4.6 Ablation Study · p. 16
“| MOMENT 𝑆 | VideoMAE 𝐵 | IMU2CLIP | 41.39 | 65.82 | 76.45 | 69.77 | 94.61 | 98.81 | 78.42 | 91.19 | 94.60 | | | VideoMAE 𝐵 | COMODO | 57.74 | 76.66 | 83.79 | 81.60 | 97.68 | 99.48 | 79.48 | 91.19 | 94.45 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Baseline Implementation Detail · p. 28
“| CrossHAR | 512 | 8 4.97 | 96.55 | 98.11 |”
§ 4.2 Experimental Settings · p. 11
“| CrossHAR | ✗ | 50.96 | 72.66 | 81.53 | 77.14 | 96.95 | 99.31 | 84.97 | 96.55 | 97.74 |”
Citations
Cites
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · cited 2×
“[43] shows that multimodal students can out-perform unimodal teachers by learning richer cross-modal representations.”
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Zhou, H., Zhang, S.-H., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2020) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2012.07436 · Cited by 9,681
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17325 · Cited by 7,752
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 1×
“In this way, our representation learning distillation framework remains unsupervised, with class labels only used to train the SVM on the extracted features.”
Are Transformers Effective for Time Series Forecasting?
Zeng, A., Chen, M.-H., Zhang, L., et al. (2022) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2205.13504 · Cited by 4,870
Are Transformers Effective for Time Series Forecasting?
Zeng, A., Chen, M., Zhang, L., & Xu, Q. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i9.26317 · Cited by 3,320
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
FitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
Is Space-Time Attention All You Need for Video Understanding?
Bertasius, G., Wang, H., & Torresani, L. (2021) · arXiv · doi:10.48550/arxiv.2102.05095 · Cited by 1,663
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456
Contrastive Representation DistillationPreprint
Tian, Y., Krishnan, D., & Isola, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10699 · Cited by 1,328
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909
TS2Vec: Towards Universal Representation of Time Series
Yue, Z., Wang, Y., Duan, J., Yang, T., Huang, C., Tong, Y., & Xu, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20881 · Cited by 850
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 1×
“…and natural language processing, which have seen significant advancements driven by large-scale datasets, human activity recognition using wearable sensors still relies on relatively small datasets, many of which are collected in controlled environments with scripted activities [1, 6, 35, 47].”
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630 · cited 1×
“The goal of this task is to measure the capability of a model to generalize its learned representations to other datasets that differ significantly from its training set.”
Self-citationUSC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554
Unsupervised Scalable Representation Learning for Multivariate Time Series
Franceschi, J.-Y., Dieuleveut, A., & Jaggi, M. (2019) · arXiv · doi:10.48550/arxiv.1901.10738 · Cited by 305
SEED: Self-supervised Distillation For Visual RepresentationPreprint
Fang, Z., Wang, J., Wang, L., Zhang, L., Yang, Y., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04731 · Cited by 187
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Grauman, K., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, T., Ashutosh, K., Baiyya, V., Bansal, S., Boote, B., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01834 · Cited by 182
Kwon, H., Tong, C., Haresamudram, H., Gao, Y., Abowd, G. D., Lane, N. D., & Plötz, T. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411841 · Cited by 152
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Li, K., Wang, Y., Li, Y., Wang, Y., He, Y., Wang, L., & Qiao, Y. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01826 · Cited by 143 · cited 1×
“The availability of large-scale datasets [3, 5, 17–19] has fueled the success of deep learning models in this domain.”
SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Tang, C. I., Perez-Pozuelo, I., Spathis, D., Brage, S., Wareham, N., & Mascolo, C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448112 · Cited by 140
ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition
Jain, Y., Tang, C. I., Min, C., Kawsar, F., & Mathur, A. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3517246 · Cited by 113
Abedin, A., Ehsanpour, M., Shi, Q., Rezatofighi, H., & Ranasinghe, D. C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448083 · Cited by 107
COCOA: Cross Modality Contrastive Learning for Sensor Data
Deldari, S., Xue, H., Saeed, A., Smith, D. V., & Salim, F. D. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550316 · Cited by 92
Self-citationGeneralized Gradients: Priors on Minimization Flows
Charpiat, G., Maurel, P., Pons, J.-P., Keriven, R., & Faugeras, O. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-9966-2 · Cited by 91
Hong, Z., Li, Z., Zhong, S., Lyu, W., Wang, H., Ding, Y., He, T., & Zhang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659597 · Cited by 67 · cited 1×
“[24] introduced hierarchical self-supervised pretraining to improve cross-dataset generalization, and [15] handled missing modality conditions through a combination of modality dropout training and cross-modal contrastive learning.”
Self-citationXu, L., Wu, Q., Pan, L., Meng, F., Li, H., He, C., Wang, H., Cheng, S., & Dai, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3295899 · Cited by 66 · cited 1×
“The goal of this task is to measure the capability of a model to generalize its learned representations to other datasets that differ significantly from its training set.”
Multimodal Distillation for Egocentric Action Recognition
Radevski, G., Grujicic, D., Blaschko, M. B., Moens, M., & Tuytelaars, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00481 · Cited by 62 · cited 1×
“Following prior work on unsupervised representation learning [12, 16, 45], we evaluate the quality of the learned representations by training a Support Vector Machine (SVM) with an RBF kernel on the extracted features from the training subset, and then using it to predict the classes of the test…”
IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition
Leng, Z., Bhattacharjee, A., Rajasekhar, H., Zhang, L., Bruda, E., Kwon, H., & Plötz, T. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3678545 · Cited by 61 · cited 1×
“[25, 26] synthesized IMU signals from video and language models, demonstrating the potential for large-scale HAR by alleviating the need for extensive real-world IMU data collection.”
CroSSL: Cross-modal Self-Supervised Learning for Time-series through Latent Masking
Deldari, S., Spathis, D., Malekzadeh, M., Kawsar, F., Salim, F. D., & Mathur, A. (2024) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3616855.3635795 · Cited by 56 · cited 1×
“[8, 9] introduced contrastive cross-modal learning frameworks for multi-sensor time-series, with [9] improving upon [8] by replacing explicit negative pairs with latent masking.”
Self-citationDistillation Multiple Choice Learning for Multimodal Action Recognition
Garcia, N. C., Bargal, S. A., Ablavsky, V., Morerio, P., Murino, V., & Sclaroff, S. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00280 · Cited by 53
EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding
Tan, S., Nagarajan, T., & Grauman, K. L. (2023) · neural information processing systems · doi:10.52202/075280-1455 · Cited by 47
MMTSA: Multi-Modal Temporal Segment Attention Network for Efficient Human Activity Recognition
Gao, Z., Wang, Y., Chen, J., Xing, J., Patel, S., Liu, X., & Shi, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610872 · Cited by 44
Transfer Learning in Sensor-Based Human Activity Recognition: A Survey
Dhekane, S. G., & Ploetz, T. (2025) · ACM Computing Surveys · doi:10.1145/3717608 · Cited by 42
Multimodal Knowledge Expansion
Xue, Z., Ren, S., Gao, Z., & Zhao, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00089 · Cited by 41
MOMENT: a family of open time-series foundation models
Goswami, M., Szafer, K., Choudhry, A., Cai, Y., Li, S., & Dubrawski, A. (2024) · arXiv · doi:10.48550/arxiv.2402.03885 · Cited by 41
Zero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings
Tong, C., Ge, J., & Lane, N. D. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3494995 · Cited by 40 · cited 1×
“…all stored video embeddings: (cid:80) where P v ( i Q ) and P x ( i Q ) represent the similarity score distributions of the video and IMU embeddings, respectively. τ v and τ x are temperature scaling factors for the video and IMU, and ( ) represents the inner product be-tween the two embeddings.”
In Focus, Out of Privacy: The Wearer's Perspective on the Privacy Dilemma of Camera Glasses
Bhardwaj, D., Ponticello, A., Tomar, S., Dabrowski, A., & Krombholz, K. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642242 · Cited by 34
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Plötz, T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729467 · Cited by 33 · cited 1×
“A key challenge is that annotating IMU data is much more labor-intensive and costly than labeling images or text, as it often requires external reference signals and precise temporal alignment [21].”
Haresamudram, H., Tang, C. I., Suh, S., Lukowicz, P., & Ploetz, T. (2024) · arXiv · Cited by 31 · cited 1×
“A key challenge is that annotating IMU data is much more labor-intensive and costly than labeling images or text, as it often requires external reference signals and precise temporal alignment [21].”
Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognition
Hatano, M., Hachiuma, R., Fujii, R., & Saitô, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73414-4_11 · Cited by 31
Masked Video and Body-Worn IMU Autoencoder for Egocentric Action Recognition
Zhang, M., Huang, Y., Liu, R., & Sato, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_18 · Cited by 31
Resource-Efficient Continual Learning for Sensor-Based Human Activity Recognition
Leite, C. F. S., & Xiao, Y. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3530910 · Cited by 29
(2023) · doi:10.1109/cvpr52729.2023.00627 · Cited by 28 · cited 1×
“[24] introduced hierarchical self-supervised pretraining to improve cross-dataset generalization, and [15] handled missing modality conditions through a combination of modality dropout training and cross-modal contrastive learning.”
GOAT: A Generalized Cross-Dataset Activity Recognition Framework with Natural Language Supervision
Miao, S., & Chen, L. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699736 · Cited by 27
Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition
Miao, S., Chen, L., & Hu, R. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631415 · Cited by 23
Mujirishvili, T., Fedosov, A., Hashemifard, K., Climent-Pérez, P., & Florez-Revuelta, F. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642164 · Cited by 18
IMU2CLIP: Language-grounded Motion Sensor Translation with Multimodal Contrastive Learning
Moon, S., Madotto, A., Lin, Z., Saraf, A., Bearman, A., & Damavandi, B. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.883 · Cited by 18
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition
Fritsch, S. G., Oguz, C., Rey, V. F., Ray, L. S. S., Kiefer-Emmanouilidis, M., & Lukowicz, P. (2025) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom64205.2025.00020 · Cited by 16
Uncovering Practical Security and Privacy Threats for Connected Glasses with Embedded Video Cameras
Opaschi, O., & Vatavu, R.-D. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3432700 · Cited by 10
Bystander Privacy in Video Sharing Era: Automated Consent Compliance through Platform Censorship
Liao, S., He, H., Chen, H., & Yang, Z. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713391 · Cited by 4
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
Grauman, K., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, T., Ashutosh, K., Baiyya, V., Bansal, S., Boote, B., et al. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02557-6 · Cited by 3
ConGen: Unsupervised control and generalization distillation for sentence representation
Limkonchotiwat, P., Ponwitayarat, W., Lowphansirikul, L., Udomcharoenchaikit, C., Chuangsuwanich, E., & Nutanong, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.483 · Cited by 3
SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity RecognitionPreprint
Li, Z., Deldari, S., Chen, L., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.10624 · Cited by 1
Self-citationMantis: Lightweight Foundation Model for Time Series Classification
Feofanov, V., Wen, S., Xie, S., Roschmann, S., Alonso, M., Guo, H., Ilbert, R., Tiomoko, M., Bouniot, Q., Akata, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.15637 · Cited by 1
ZARA: Training-Free Motion Time-Series Reasoning via Evidence-Grounded LLM Agents
Li, Z., Chen, B., Xue, H., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.04038
Self-citationA public domain dataset for human activity recognition using smartphones
Davide, A. (2013) · Esann
UnresolvedVasilii Feofanov, Songkang Wen, Marius Alonso, Romain Ilbert, Hongbo Guo, Malik Tiomoko, Lujia Pan, Jianfeng Zhang, and levgen Redko. 2025. Mantis: Lightweight Calibrated Foundation Model for User-Friendly Time Series Classification. arXiv preprint arXiv:2502.15637 (2025).
Vasilii, F. (2025)
UnresolvedJean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. 2019. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems 32 (2019).
Jean-Yves, F. (2019)
UnresolvedMononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. 2024. MOMENT: a family of open time-series foundation models. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML'24). JMLR.org, Article 642, 38 pages.
Mononito, G. (2024)
UnresolvedYuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge Distillation of Large Language Models. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=5h0qf7IBZZ
Yuxian, G. (2024)
UnresolvedMasashi Hatano, Ryo Hachiuma, Ryo Fujii, and Hideo Saito. 2024. Multimodal cross-domain few-shot learning for egocentric action recognition. In European Conference on Computer Vision. Springer, 182–199.
Masashi, H. (2024)
UnresolvedZechen Li, Baiyu Chen, Hao Xue, and Flora D Salim. 2025. Zara: Training-free motion time-series reasoning via evidence-grounded llm agents. arXiv preprint arXiv:2508.04038 (2025).
Zechen, L. (2025)
UnresolvedVisualizing data using t-SNE
Laurens, V. D. M. (2008) · Journal of machine learning research 9
UnresolvedJianyuan Ni, Raunak Sarbajna, Yang Liu, Anne HH Ngu, and Yan Yan. 2022. Cross-modal knowledge distillation for vision-to-sensor action recognition. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4448–4452.
Jianyuan, N. (2022)
UnresolvedGerman I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural networks 113 (2019), 54–71.
Parisi German, I. (2019)
UnresolvedAlec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PmLR, 8748–8763.
Alec, R. (2021)
UnresolvedAdriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. FitNets: Hints for Thin Deep Nets. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and
Adriana, R. (2015)
UnresolvedHaixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. 2023. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In International Conference on Learning Representations.
Haixu, W. (2023)
UnresolvedZihui Xue Zhengqi Gao Sucheng Ren and Hang Zhao. 2023. The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation. In ICLR.
(n.d.)
UnresolvedMingfang Zhang, Yifei Huang, Ruicong Liu, and Yoichi Sato. 2024. Masked video and body-worn IMU autoencoder for egocentric action recognition. In European Conference on Computer Vision. Springer, 312–330.
Mingfang, Z. (2024)
Unresolved
Cited by
RightFeatKD: Selective Feature-Based Knowledge Distillation
Haque, S. T., Yan, Y., & Ngu, A. H. H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31927-2_43
Metadata
- Authors
- Chen, B., Wongso, W., Li, Z., Khaokaew, Y., Xue, H., & Salim, F. D.
- Affiliations
- UNSW Sydney (AU)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2026-06-15
- Volume
- 10
- Issue
- 2
- Pages
- 1-29
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W4414565371
- Semantic Scholar
- c1b81ba2d90f8ce02b66263c5af3fbe23ae65cdb
- Cited by
- 1
- References
- 63
- Citations by year
- 2026: 1
- FWCI
- 2.83
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- ARC Centre of Excellence for Automated Decision-Making and Society (CE200100005)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- inertial measurement unit, cross-modal knowledge distillation, self-supervised learning, wearable sensors, energy efficiency, privacy preservation, cross-dataset generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.