Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition

Miao, Shenghuan; Chen, Ling; Hu, Rong

doi:10.1145/3631415

Cited by 24Context-Aware Activity Recognition SystemsHuman Mobility and Location-Based AnalysisPhysical Activity and Health

Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition

Open access

Miao, Shenghuan, Chen, Ling, Hu, Rong·2023·Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

DOI 10.1145/3631415

Open this paper in CoChat — free account Open PDF

Abstract

The widespread adoption of wearable devices has led to a surge in the development of multi-device wearable human activity recognition (WHAR) systems. Nevertheless, the performance of traditional supervised learning-based methods to WHAR is limited by the challenge of collecting ample annotated wearable data. To overcome this limitation, self-supervised learning (SSL) has emerged as a promising solution by first training a competent feature extractor on a substantial quantity of unlabeled data, followed by refining a minimal classifier with a small amount of labeled data. Despite the promise of SSL in WHAR, the majority of studies have not considered missing device scenarios in multi-device WHAR. To bridge this gap, we propose a multi-device SSL WHAR method termed Spatial-Temporal Masked Autoencoder (STMAE). STMAE captures discriminative activity representations by utilizing the asymmetrical encoder-decoder structure and two-stage spatial-temporal masking strategy, which can exploit the spatial-temporal correlations in multi-device data to improve the performance of SSL WHAR, especially on missing device scenarios. Experiments on four real-world datasets demonstrate the efficacy of STMAE in various practical scenarios.

Integrity signals

Registry

Crossref · 10.1145/3631415

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

4 of 18 cited sources checked · 1 supported · 1 flagged · accuracy not yet measured

Premisecitation
Self-supervised learning provides a viable solution by pre-training a feature extractor on large unlabeled data and then optimizing a small classifier with limited labeled data.
“Self-supervised learning (SSL) provides a viable solution by first pre-training a feature extractor on a large amount of unlabeled data and then optimizing a small classifier with limited labeled data [13].”
Quote in the paperSubstantial claim1 of 1 sources checked
- cites [13] 13. Shohreh Deldari, Hao Xue, Aaqib Saeed, Jiayuan He, Daniel V Smith, and Flora D Salim. 2022. Beyond just vision: A review on self-supervised representation learning on multimodal and temporal data. arXiv preprint arXiv:2206.02353 (2022). source
  Section 3.1 (pp. 8–9) describes learning an encoder from unlabeled data and transferring it to downstream tasks with limited labeled data, including classification. It does not specify optimizing a small classifier as the second step; the encoder may instead be reused directly or fine-tuned. (LLM, conf=0.84)
  Experimental — accuracy not yet measured
Findingown data
Experiments on four real-world datasets demonstrate STMAE's efficacy in various practical scenarios.
“Experiments on four real-world datasets demonstrate the efficacy of STMAE in various practical scenarios.”
Quote in the paperSubstantial claim
Findingown data
STMAE outperforms state-of-the-art baselines on four public datasets.
“Our evaluation on four public datasets demonstrates that STMAE outperforms the state-of-the-art (SOTA) baselines.”
Quote in the paperSubstantial claim
Findingown data
The advantage of STMAE is particularly pronounced when only a small number of devices are available.
“The advantage of STMAE is particularly pronounced when only a small number of devices are available.”
Quote in the paperSubstantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.
Findingreasoning
Opportunity dataset does not reach stable performance even at the maximum labeling rate of 100%.
“However, Opportunity dataset does not seem to reach stable performance even at the maximum labeling rate of 100%.”
Quote in the paperSubstantial claim

Findingown data
For Opportunity and PAMAP datasets, the marco F1-score performance initially improves with an increasing number of CNN layers, but then gradually decreases.
“Specifically, the results indicate that for Opportunity and PAMAP datasets, the marco F1-score performance initially improves with an increasing number of CNN layers, but then gradually decreases.”
Quote in the paperSubstantial claim
Findingreasoning
The performance of w/o TM is completely poor.
“while the performance of w/o TM becomes completely poor.”
Quote in the paperSubstantial claim
Findingreasoning
The top-performing baselines are not those specifically designed for multi-device scenarios on all datasets.
“We note that the top-performing baselines are not those specifically designed for multi-device scenarios on all datasets.”
Quote in the paperNot a substantial claim
Findingreasoning
The advantage of STMAE over STMAE-Sup is more pronounced at lower labeling rates.
“In addition, the advantage of STMAE over STMAE-Sup is more pronounced in the case of lower labeling rates, as the supervised method is prone to underfitting due to insufficient training data, while STMAE has already learned some prior knowledge of wearable data during pre-training, reducing its reliance on labeled data during supervised learning.”
Quote in the paperSubstantial claim
Findingown data
As the labeling rate increases, the rate of increase in macro F1-score slows down.
“As the labeling rate increases, the rate of increase in macro F1-score slows down.”
Quote in the paperNot a substantial claim
Findingcitation
On Opportunity and Realdisp datasets, LIMU-BERT achieves the best performance.
“On Opportunity and Realdisp datasets, LIMU-BERT [71] achieves the best performance.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [71] 71. Huatao Xu, Pengfei Zhou, Rui Tan, Mo Li, and Guobin Shen. 2021. LIMU-BERT: Unleashing the potential of unlabeled data for IMU sensing applications. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems. 220-233. source
Findingown data
Compared to STMAE-Sup, STMAE achieves similar or better performance by optimizing only a single FC layer alone in supervised learning phase.
“Furthermore, compared to the supervised method STMAE-Sup, STMAE achieves similar or better performance by optimizing only a single FC layer alone in supervised learning phase.”
Quote in the paperSubstantial claim
Findingown data
STMAE consistently outperforms the other baselines across all datasets for various numbers of available devices.
“It can be seen that STMAE consistently outperforms the other baselines across all datasets for various numbers of available devices.”
Quote in the paperSubstantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.
Findingown data
The symmetrical autoencoder performs similarly to w/o SM on Opportunity and PAMAP datasets.
“For the symmetrical autoencoder, we find that it performs similarly to w/o SM on Opportunity and PAMAP datasets.”
Quote in the paperSubstantial claim
Findingreasoning
For Realdisp dataset, the performance decreases steadily with an increasing number of CNN layers.
“For Realdisp dataset, the performance decreases steadily with an increasing number of CNN layers.”
Quote in the paperNot a substantial claim
Findingreasoning
The performance of w/o SM is slightly inferior to that of STMAE on each dataset.
“The performance of w/o SM is slightly inferior to that of STMAE on each dataset, while the performance of w/o TM becomes completely poor.”
Quote in the paperSubstantial claim
Findingreasoning
For PAMAP dataset, the macro F1-score shows a relatively gradual change with a slight upward trend.
“For PAMAP dataset, the macro F1-score shows a relatively gradual change with a slight upward trend.”
Quote in the paperNot a substantial claim
Findingown data
Macro F1-score grows rapidly with increasing labeling rate in the low labeling rate interval.
“Our findings also reveal that macro F1-score grows rapidly with increasing labeling rate in the low labeling rate interval, where the model has a high data demand.”
Quote in the paperSubstantial claim
Findingown data
Masked reconstruction-based methods perform similarly to contrastive learning-based methods on average.
“In addition, we find that masked reconstruction-based methods perform similarly to contrastive learning-based methods on average, suggesting that there is still significant potential for research on masked reconstruction-based methods.”
Quote in the paperSubstantial claim
Findingown data
STMAE achieves the best performance on all datasets compared to other SSL baselines.
“STMAE achieves the best performance on all datasets compared to other SSL baselines, which demonstrates the effectiveness of spatial-temporal masking and encoding.”
Quote in the paperSubstantial claim
Findingreasoning
The sweet point is reached at a temporal masking probability of 90% for Realdisp.
“The sweet point is reached at a temporal masking probability of 90%, indicating that Realdisp dataset is not affected by the destruction of temporal properties due to high temporal masking probabilities.”
Quote in the paperNot a substantial claim
Findingown data
STMAE outperforms its variants on each dataset.
“The results, shown in Fig. 3, indicate that STMAE outperforms its variants on each dataset, demonstrating the effectiveness of the proposed spatial-temporal masking strategy.”
Quote in the paperSubstantial claim
Findingown data
On the remaining one dataset, the performance of STMAE is comparable to STMAE-Sup with a -0.79% gap.
“In the remaining one dataset, the performance of STMAE is comparable to STMAE-Sup (with -0.79% gap).”
Quote in the paperSubstantial claim
Findingown data
The temporal masking probability has a significant impact on the performance of STMAE.
“As shown in Fig. 4, in general, we observe that the temporal masking probability has a significant impact on the performance of STMAE.”
Quote in the paperNot a substantial claim
Findingown data
On three out of four datasets, STMAE achieves higher performance than the supervised method STMAE-Sup.
“On three out of four datasets, STMAE even achieves higher performance than the supervised method STMAE-Sup.”
Quote in the paperSubstantial claim
Findingreasoning
Realdisp dataset achieves stable performance at a labeling rate of 10%.
“Realdisp dataset achieves stable performance at a labeling rate of 10%.”
Quote in the paperNot a substantial claim
Findingown data
No single method emerges as the clear winner across all datasets.
“Based on the performance of SSL baselines, we observe that no single method emerges as the clear winner across all datasets.”
Quote in the paperNot a substantial claim
Findingreasoning
For Realdisp dataset, the macro F1-score consistently increases with increasing temporal masking probability over the entire range from 10% to 90%.
“For Realdisp dataset, the macro F1-score consistently increases with increasing temporal masking probability over the entire range (from 10% to 90%).”
Quote in the paperNot a substantial claim
Findingown data
STMAE outperforms all other SSL baselines at all labeling rates for each dataset.
“Overall, we find that STMAE outperforms all other SSL baselines at all labeling rates for each dataset, demonstrating its adaptability to various labeling rate situations.”
Quote in the paperSubstantial claim
Findingnone given
STMAE-Rand has the worst performance on all datasets.
“Expectedly, STMAE-Rand has the worst performance on all datasets.”
Quote in the paperNot a substantial claim
Findingnone given
The frozen setting alone also achieves acceptable results on all datasets.
“Furthermore, the frozen setting alone also achieves acceptable results on all datasets.”
Quote in the paperNot a substantial claim
Findingown data
Temporal masking is highly effective in various network structures, significantly improving the model's performance.
“We also discover that the temporal masking is highly effective in various network structures, significantly improving the model's performance.”
Quote in the paperSubstantial claim
Findingown data
On the other three datasets, there is a linear decline in marco F1-score performance with decreasing available devices.
“However, we observe a linear decline in marco F1-score performance with decreasing available devices on the other three datasets.”
Quote in the paperNot a substantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.
Findingown data
On Opportunity, Realdisp, and PAMAP datasets, the performance of STMAE significantly improves under the fine-tuned setting.
“On Opportunity, Realdisp, and PAMAP datasets, the performance of STMAE significantly improves under the fine-tuned setting.”
Quote in the paperNot a substantial claim
Findingown data
The Transformer structure achieves the best performance among the compared structures.
“It achieves the best performance among the compared structures”
Quote in the paperSubstantial claim
Findingreasoning
On RealWorld dataset, the performance of STMAE slightly decreases under the fine-tuned setting.
“However, on RealWorld dataset, the performance of STMAE slightly decreases under the fine-tuned setting.”
Quote in the paperNot a substantial claim
Findingown data
Evaluation on multiple public wearable datasets demonstrates the superiority of STMAE compared to the SOTA methods.
“Evaluation is conducted on multiple public wearable datasets, demonstrating the superiority of STMAE compared to the SOTA methods.”
Quote in the paperSubstantial claim
- Rests on an assumption: The four selected datasets enable a comprehensive and unbiased evaluation of the method.
Findingown data
STMAE achieves higher macro F1-score than the best SSL baselines on Opportunity, RealWorld, Realdisp, and PAMAP at the specified labeling rates.
“| STMAE | 56.72 | 65.68 | 87.38 | 72.54 |”
- Also stated in: § B METHODOLOGICAL TRANSPARENCY & REPRODUCIBILITY · plain
  “| Improvement | 5.41% | 2.81% | 3.95% | 5.00% |”
Quote in the paperSubstantial claim
Findingreasoning
LSTM demonstrates the worst performance among the compared structures.
“On the other hand, LSTM demonstrates the worst performance”
Quote in the paperNot a substantial claim
Findingown data
Even with partial device availability, STMAE can produce high macro F1-scores.
“The results indicate that even with partial device availability, STMAE can produce high macro F1-scores.”
Quote in the paperSubstantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.
Findingown data
On Realdisp dataset, the performance degradation of STMAE with decreasing available devices is relatively minimal.
“On Realdisp dataset, the performance degradation of STMAE with decreasing available devices is relatively minimal.”
Quote in the paperSubstantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.
Premisereasoning
Existing SSL WHAR methods do not consider missing device scenarios in multi-device WHAR.
“Most importantly, existing SSL WHAR methods do not consider missing device scenarios in multi-device WHAR.”
- Also stated in: § 2.2 SSL for WHAR · plain
  “none of them consider the missing device scenarios.”
Quote in the paperSubstantial claim
Premisereasoning
Contrastive learning is vulnerable to biased input instances because it strongly relies on data augmentation methods to generate positive and negative samples.
“Contrastive learning is vulnerable to biased input instances, as it strongly relies on data augmentation methods to generate positive and negative samples.”
Quote in the paperSubstantial claim
Premisecitation
The performance of supervised learning methods for WHAR is particularly dependent on large-scale labeled datasets, which require significant human efforts and financial costs.
“The performance of supervised learning methods is particularly dependent on large-scale labeled datasets, which require significant human efforts and financial costs [62].”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [62] 62. Timo Sztyler and Heiner Stuckenschmidt. 2016. On-body localization of wearable devices: An investigation of position-aware activity recognition. In Proceedings of the International Conference on Pervasive Computing and Communications. 1-9. source
Interpretationown data
High temporal masking probabilities can completely destroy the temporal information in wearable data, making it difficult for the TE to be effectively trained.
“On the other hand, high temporal masking probabilities can completely destroy the temporal information in wearable data, making it difficult for the TE to be effectively trained.”
Quote in the paperSubstantial claim
Interpretationreasoning
By taking advantage of spatial correlations, effective feature extraction can be achieved even when data from certain devices are unavailable.
“By taking advantage of these spatial correlations, we are able to achieve effective feature extraction even in cases where data from certain devices are unavailable.”
Quote in the paperNot a substantial claim
- Rests on an assumption: Spatial correlations between data from various wearable devices differ depending on the activity being performed.
- Rests on an assumption: Spatial correlations can be utilized to infer data from one device based on data from other devices, making them a powerful means of reducing data redundancy.
Interpretationown data
It is possible to achieve acceptable performance using only a simple autoencoder.
“It suggests that it is possible to achieve acceptable performance using only a simple autoencoder.”
Quote in the paperNot a substantial claim
Interpretationreasoning
Realdisp dataset is not affected by the destruction of temporal properties due to high temporal masking probabilities.
“indicating that Realdisp dataset is not affected by the destruction of temporal properties due to high temporal masking probabilities.”
Quote in the paperNot a substantial claim
Interpretationreasoning
The spatial and temporal encoders learned in pre-training phase are practical for downstream tasks.
“It shows that the spatial and temporal encoders learned in pre-training phase are practical for downstream tasks.”
Quote in the paperSubstantial claim
Interpretationreasoning
Capturing temporal correlations is crucial to obtain features that can fully characterize the current activity.
“Since different activities exhibit different periodic patterns, amplitude ranges, etc., in the time series, capturing temporal correlations is crucial to obtain features that can fully characterize the current activity.”
Quote in the paperSubstantial claim
- Rests on an assumption: Different activities exhibit different periodic patterns, amplitude ranges, etc., in time series.
Interpretationreasoning
The difference in stability is likely due to differences in types of activities in each dataset.
“This is likely due to the differences in types of activities in each dataset.”
Quote in the paperNot a substantial claim
Interpretationown data
The proposed spatial-temporal masking strategy is effective.
“demonstrating the effectiveness of the proposed spatial-temporal masking strategy.”
Quote in the paperSubstantial claim
Interpretationreasoning
The temporal information to be masked is not raw readings but high-level semantic features after spatial fusion.
“Fortunately, the temporal information we need to mask now is not the raw readings, but the high-level semantic features after spatial fusion.”
Quote in the paperSubstantial claim
Premisereasoning
The amount of available labeled data in the supervised learning phase is very limited.
“This guidance process is usually short for better user experience, so the amount of available labeled data are very limited.”
Quote in the paperSubstantial claim0 of 1 sources checked
- cites [51] 51. Xiaomin Ouyang, Xian Shuai, Jiayu Zhou, Ivy Wang Shi, Zhiyuan Xie, Guoliang Xing, and Jianwei Huang. 2022. Cosmo: Contrastive fusion learning with small data for multimodal human activity recognition. In Proceedings of the Annual International Conference on Mobile Computing And Networking. 324-337. source
Interpretationown data
The performance advantage of STMAE over Zero TM indicates the effectiveness of using mask tokens in temporal masking.
“The performance advantage of STMAE over Zero TM indicates the effectiveness of using mask tokens in temporal masking.”
Quote in the paperSubstantial claim
Interpretationown data
These experiments demonstrate the practicality of pre-trained encoders.
“These experiments demonstrate the practicality of pre-trained encoders.”
Quote in the paperSubstantial claim
Interpretationreasoning
In Opportunity dataset, mid-level gesture classes have more inter-class similarity and intra-class variability, resulting in confusion between similar activities.
“In Opportunity dataset, mid-level gesture classes have more inter-class similarity and intra-class variability in their activities, resulting in confusion between similar activities, e.g., “Open Drawer 1”, “Open Drawer 2”, and “Open Drawer 3” that differ only in the height of the drawer.”
Quote in the paperSubstantial claim
Interpretationreasoning
Low temporal masking probabilities provide insufficient information for the TE to learn temporal correlations, as the RD can easily recover the masked data.
“Low temporal masking probabilities provide insufficient information for the TE to learn the temporal correlations from wearable data, as the RD can easily recover the masked data.”
Quote in the paperSubstantial claim
Interpretationnone given
The mechanism of spatial masking and encoding allows raw wearable data to be fused spatially even in the absence of some device data, contributing to effective feature extraction at each time point.
“The mechanism of spatial masking and encoding allows the raw wearable data to be fused spatially even in the absence of some device data, contributing to the effective feature extraction at each time point.”
Quote in the paperSubstantial claim
- Rests on an assumption: Spatial correlations between data from various wearable devices differ depending on the activity being performed.
- Rests on an assumption: Spatial correlations can be utilized to infer data from one device based on data from other devices, making them a powerful means of reducing data redundancy.
Interpretationreasoning
PAMAP's higher sampling frequency results in smaller variations of reading within its time window, making it easier to reconstruct.
“Its higher sampling frequency results in smaller variations of reading within its time window, making it easier to reconstruct.”
Quote in the paperSubstantial claim
Interpretationreasoning
LSTM's worst performance is primarily due to difficulties in feature transmission within the LSTM under the masking strategy.
“primarily due to difficulties in feature transmission within the LSTM under the masking strategy.”
Quote in the paperNot a substantial claim
Interpretationown data
The combination of TE and TM plays a crucial role in the model's performance.
“This experiment demonstrates that the combination of TE and TM plays a crucial role in the model's performance”
Quote in the paperSubstantial claim
Interpretationnone given
STMAE leverages the spatial-temporal correlations in multi-device data to achieve superior results.
“STMAE leverages the spatial-temporal correlations in multi-device data to achieve superior results.”
Quote in the paperSubstantial claim
- Rests on an assumption: The four selected datasets enable a comprehensive and unbiased evaluation of the method.
Interpretationown data
Training both the encoders and classifier simultaneously during supervised learning phase is generally effective in improving model performance.
“It means that training both the encoders and classifier simultaneously during supervised learning phase is generally effective in improving model performance”
Quote in the paperNot a substantial claim
Interpretationown data
STMAE may still be applicable only in cases where the number of missing devices is relatively low.
“This suggests that while STMAE outperforms various baselines, it may still be applicable only in cases where the number of missing devices is relatively low.”
Quote in the paperSubstantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.
Interpretationown data
STMAE is robust in missing device scenarios, even with only a single FC layer for supervised learning.
“It demonstrates the robustness of STMAE in missing device scenarios, even with only a single FC layer for supervised learning.”
Quote in the paperSubstantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.

Classified as probably not a substantial claim. 19
May combine two claims. 30

Classified as probably not a substantial claim.
Study design and conductcitation
Leave-one-group-out cross-validation is used for performance evaluation.
“Leave-one-group-out cross-validation approach is used for performance evaluation follows the previous works [26,34] .”
Quote in the paperSubstantial claim1 of 2 sources checked
- cites [26] 26. Harish Haresamudram, Irfan Essa, and Thomas Plötz. 2021. Contrastive predictive coding for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1-26. source
  Section 4.1 (pp. 7–8) describes participant-based training, validation, and test splits, not leave-one-group-out cross-validation. It says 20% of participants are randomly selected for testing and specifies a fixed participant split for USC-HAD. (LLM, conf=0.88)
  Checked by a reasoning model — may be wrong.
  Experimental — accuracy not yet measured
- cites [34] 34. Yash Jain, Chi Ian Tang, Chulhong Min, Fahim Kawsar, and Akhil Mathur. 2022. ColloSSL: Collaborative self-supervised learning for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1-28. source
May combine two claims.
Premisecitation
Activity labeling in wearable data is difficult, reducing the number of available labels.
“This renders activity labeling difficult, reducing the number of available labels [8,23,30].”
Quote in the paperSubstantial claim1 of 3 sources checked
- cites [8] 8. Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. Computing Surveys 54, 4 (2021), 1-40. source
- cites [23] 23. Fuqiang Gu, Mu-Huan Chung, Mark Chignell, Shahrokh Valaee, Baoding Zhou, and Xue Liu. 2021. A survey on deep learning for human activity recognition. Computing Surveys 54, 8 (2021), 1-34.
  Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [30] 30. Rong Hu, Ling Chen, Shenghuan Miao, and Xing Tang. 2023. SWL-Adapt: An unsupervised domain adaptation model with sample weight learning for cross-user wearable human activity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 6012-6020. source
May combine two claims.
Findingown data
On RealWorld and PAMAP datasets, CPC and Masked Reconstruction achieve the best performance, respectively.
“On RealWorld and PAMAP datasets, CPC [26] and Masked Reconstruction [25] achieve the best performance, respectively.”
Quote in the paperSubstantial claim1 of 2 sources checked
- cites [26] 26. Harish Haresamudram, Irfan Essa, and Thomas Plötz. 2021. Contrastive predictive coding for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1-26. source
  The citation to this paper is attached to CPC’s performance on RealWorld, not to masked reconstruction on PAMAP. Section 5.1 (p. 10) reports CPC results for Mobiact, Motionsense, USC-HAD and UCI-HAR, but the passages found when searching for RealWorld do not establish its performance on that dataset. (LLM, conf=0.76)
  Experimental — accuracy not yet measured
- cites [25] 25. Harish Haresamudram, Apoorva Beedu, Varun Agrawal, Patrick L Grady, Irfan Essa, Judy Hoffman, and Thomas Plötz. 2020. Masked reconstruction based self-supervision for human activity recognition. In Proceedings of the International Symposium on Wearable Computers. 45-49. source
May combine two claims.
Findingown data
STMAE outperforms the best SSL baselines by 5.41%, 2.81%, 3.95%, and 5.00% on Opportunity, RealWorld, Realdisp, and PAMAP, respectively.
“STMAE outperforms the best SSL baselines by 5.41%, 2.81%, 3.95%, and 5.00% on Opportunity, RealWorld, Realdisp, and PAMAP, respectively.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
The macro F1-score fluctuates noticeably with changes in the temporal masking probability, and this behavior depends on the dataset.
“The macro F1-score fluctuates noticeably with changes in the temporal masking probability, and this behavior depends on the dataset.”
Quote in the paperNot a substantial claim
May combine two claims.
Findingown data
Increasing the number of CNN layers can enhance the representation capability of the network, but the impact on performance improvement is limited.
“Our results, as shown in Fig. 5, suggest that while increasing the number of CNN layers can enhance the representation capability of the network, the impact on performance improvement is limited.”
Quote in the paperSubstantial claim
May combine two claims.
Findingreasoning
For Opportunity and RealWorld datasets, the macro F1-score initially increases as the temporal masking probability increases, reaching a sweet point, and then decreases as the temporal masking probability continues to increase.
“For Opportunity and RealWorld datasets, the macro F1-score initially increases as the temporal masking probability increases, reaching a sweet point, and then decreases as the temporal masking probability continues to increase.”
Quote in the paperSubstantial claim
May combine two claims.
Findingreasoning
RealWorld dataset exhibits slight improvement in performance with an increasing number of CNN layers, but the improvement is not significant.
“On the other hand, RealWorld dataset exhibits slight improvement in performance with an increasing number of CNN layers, but the improvement is not significant.”
Quote in the paperNot a substantial claim
May combine two claims.
Findingown data
STMAE outperforms other variants and achieves the best performance.
“Specifically, STMAE outperforms other variants and achieves the best performance.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
The total training time of STMAE on Opportunity dataset using a Raspberry Pi 4 Model B was 16.82 min, with an associated energy consumption of 47.67 KJ.
“In supervised learning phase, the total training time was 16.82 min, with an associated energy consumption of 47.67 KJfoot_1 .”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
The Transformer structure is superior in both the complete masking setting and the without temporal masking setting.
“The results demonstrate the superiority of the Transformer structure in both masking settings.”
Quote in the paperSubstantial claim
May combine two claims.
Findingown data
In inference phase, each sample's inference time was 41.17 ms, with an associated energy consumption of 163.90 mJ.
“In inference phase, each sample's inference time was 41.17 ms, with an associated energy consumption of 163.90 mJ 1 .”
Quote in the paperSubstantial claim
May combine two claims.
Interpretationreasoning
The inter-activity similarity and intra-activity variability imply instances are biased easily in WHAR, exacerbating the difficulty of constructing positive and negative samples in contrastive learning.
“The inter-activity similarity and intra-activity variability imply instances are biased easily in WHAR, which exacerbate the difficulty of constructing positive and negative samples in contrastive learning.”
Quote in the paperSubstantial claim
May combine two claims.
Premisereasoning
The available devices in multi-device WHAR are an arbitrary subset of the total devices, which is unpredictable and would severely impact the recognition performance of existing methods.
“As a result, the available devices are an arbitrary subset of the total devices, which is unpredictable and would severely impact the recognition performance of existing methods.”
Quote in the paperSubstantial claim0 of 5 sources checked
- cites [19] 19. Berihun Fekade, Taras Maksymyuk, Maryan Kyryk, and Minho Jo. 2017. Probabilistic recovery of incomplete sensed data in IoT. IEEE Internet of Things Journal 5, 4 (2017), 2282-2292. source
- cites [33] 33. Farhad Imani, Changqing Cheng, Ruimin Chen, and Hui Yang. 2018. Nested gaussian process modeling for high-dimensional data imputation in healthcare systems. In Proceedings of the IIE Annual Conference. 1312-1317.
  Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [39] 39. Hua Kang, Qianyi Huang, and Qian Zhang. 2022. Augmented adversarial learning for human activity recognition with partial sensor sets. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 3 (2022), 1-30. source
- cites [42] 42. Shengzhong Liu, Shuochao Yao, Yifei Huang, Dongxin Liu, Huajie Shao, Yiran Zhao, Jinyang Li, Tianshi Wang, Ruijie Wang, Chaoqi Yang, et al. 2020. Handling missing sensors in topology-aware iot applications with gated graph neural network. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1-31. source
- cites [45] 45. Yang Liu, Zhenjiang Li, Zhidan Liu, and Kaishun Wu. 2019. Real-time arm skeleton tracking and gesture inference tolerant to missing wearable sensors. In Proceedings of the Annual International Conference on Mobile Systems, Applications, and Services. 287-299. source
May combine two claims.
Interpretationreasoning
STMAE captures discriminative activity representations by using an asymmetrical encoder-decoder structure and a two-stage spatial-temporal masking strategy, exploiting spatial-temporal correlations in multi-device data to improve SSL WHAR performance, especially in missing device scenarios.
“STMAE captures discriminative activity representations by utilizing the asymmetrical encoder-decoder structure and two-stage spatial-temporal masking strategy, which can exploit the spatial-temporal correlations in multi-device data to improve the performance of SSL WHAR, especially on missing device scenarios.”
Quote in the paperSubstantial claim
May combine two claims.
Premisenone given
There exists inter-activity similarity and intra-activity variability in WHAR.
“It is noteworthy that there exists inter-activity similarity and intra-activity variability in WHAR.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given
The asymmetrical encoder-decoder structure imposes a greater demand on the encoders for learning spatial-temporal correlations that benefits downstream tasks.
“Such an encoder-decoder structure design can impose a greater demand on the encoders for learning spatial-temporal correlations that benefits downstream tasks.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based).
Interpretationreasoning
The two-stage spatial-temporal masking strategy can handle missing device scenarios effectively.
“Such a masking strategy design can handle missing device scenarios effectively.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
In Realdisp dataset, fitness activities are highly cyclical and have more homogeneous sub-activities, making them easier for the model to learn.
“In Realdisp dataset, fitness activities are highly cyclical and have more homogeneous sub-activities, making them easier for the model to learn.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Premisecitation
Learning correlations across multiple devices benefits contrastive learning.
“The authors propose ColloSSL, which demonstrates that learning the correlations across multiple devices benefits contrastive learning.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [34] 34. Yash Jain, Chi Ian Tang, Chulhong Min, Fahim Kawsar, and Akhil Mathur. 2022. ColloSSL: Collaborative self-supervised learning for human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 1-28. source
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationreasoning
The learned spatial-temporal correlations can help extract versatile activity representations even when only part of devices are available, which is beneficial for downstream tasks.
“The learned spatial-temporal correlations can help to extract versatile activity representations even when only part of devices are available, which is beneficial for downstream tasks.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
The supervised method is prone to underfitting due to insufficient training data, while STMAE has already learned prior knowledge of wearable data during pre-training, reducing its reliance on labeled data during supervised learning.
“as the supervised method is prone to underfitting due to insufficient training data, while STMAE has already learned some prior knowledge of wearable data during pre-training, reducing its reliance on labeled data during supervised learning.”
Quote in the paperSubstantial claim
May combine two claims.
Interpretationown data
One layer of CNN can already meet the needs for spatial fusion, and significant improvement in performance is not necessarily achieved with an increasing number of layers.
“Overall, our findings suggest that one layer of CNN can already meet the needs for spatial fusion, and that the significant improvement in performance is not necessarily achieved with an increasing number of layers.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim. May combine two claims.
Interpretationnone given
Spatial masking primarily simulates missing device scenarios, while temporal masking prepares for learning temporal correlations by TE.
“Spatial masking primarily simulates missing device scenarios, while the role of temporal masking is to prepare for learning temporal correlations by TE.”
Quote in the paperSubstantial claim
- Rests on an assumption: It is impossible for every user to wear all devices at all time in daily life.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Premisecitation
In typical deployment of a WHAR model, pre-training is performed in the cloud and supervised learning is performed on local devices.
“In the typical deployment of a WHAR model [51], the pre-training phase is performed in the cloud to leverage the cloud computing power and large-scale wearable datasets.The supervised learning phase is performed in the local devices (e.g., smartphones) to personalize the WHAR model while ensuring user privacy.”
Quote in the paperNot a substantial claim0 of 1 sources checked
- cites [51] 51. Xiaomin Ouyang, Xian Shuai, Jiayu Zhou, Ivy Wang Shi, Zhiyuan Xie, Guoliang Xing, and Jianwei Huang. 2022. Cosmo: Contrastive fusion learning with small data for multimodal human activity recognition. In Proceedings of the Annual International Conference on Mobile Computing And Networking. 324-337. source
May combine two claims.
Interpretationreasoning
On PAMAP dataset, w/o TM performs particularly poorly, probably because its corresponding window size (300) is long, making the demand of learning temporal correlations even stronger.
“On PAMAP dataset, we find that w/o TM performs particularly poorly, probably because its corresponding window size (300) is long, making the demand of learning temporal correlations even stronger.”
Quote in the paperNot a substantial claim
Classified as probably not a substantial claim.
Study design and conductnone given
The final reported result is the average macro F1-score across five independent runs by varying seeds.
“The final reported result is the average macro F1-score across five independent runs by varying seeds.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone given
Multi-device WHAR offers various benefits but also introduces the problem of missing devices, which is an inevitable and widespread issue.
“Multi-device WHAR offers various benefits, but it also introduces the problem of missing devices.This is an inevitable and widespread issue.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning
The decrease in performance for Realdisp may be due to overfitting as the network becomes more complex with an increasing number of layers.
“This decrease in performance may be due to overfitting, as the network becomes more complex with an increasing number of layers.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S3 (load-bearing).
Interpretationreasoning
The slight improvement for RealWorld could be due to the larger size of RealWorld dataset, which can still be fit well with more layers.
“This could be due to the larger size of RealWorld dataset, which can still be fit well with more layers.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Premisenone given
Previous SSL WHAR research has largely focused on single-device scenarios and has not considered the challenges and opportunities presented by multi-device data.
“While previous research on SSL for WHAR has demonstrated promising results, these studies have largely focused on single-device scenarios and have not considered the challenges and opportunities presented by multi-device data.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
The Realdisp result can be attributed to the fine-grained nature of the fitness activities and the strong periodicity of the corresponding wearable data, allowing accurate reconstruction even at very high temporal masking probabilities.
“This can be attributed to the fine-grained nature of the fitness activities in Realdisp dataset and the strong periodicity of the corresponding wearable data, allowing for accurate reconstruction of the time series even at very high temporal masking probabilities.”
Quote in the paperSubstantial claim
May combine two claims.
Interpretationown data
The model is saturated with data and the current amount of data are sufficient for the model to fully exploit its potential.
“suggesting that the model is saturated with data and that the current amount of data are sufficient for the model to fully exploit its potential.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
Without temporal masking, temporal correlations are not competently learned, so the parameters of TE cannot be learned well in pre-training phase.
“This is because the structure of TE that corresponds to the temporal masking is Transformer encoders, which require sufficient learning to fully exploit the temporal correlations.Without the temporal masking, the temporal correlations are not competently learned, therefore the parameters of TE cannot be learned well in pre-training phase.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationreasoning
The underwhelming performance of STMAE-Rand indicates the challenging nature of the activity recognition tasks undertaken and underscores the necessity for a proficient feature extractor.
“The underwhelming performance of STMAE-Rand indicates the challenging nature of the activity recognition tasks we undertake, underscoring the necessity for a proficient feature extractor.”
Quote in the paperNot a substantial claim
May combine two claims.
Interpretationreasoning
The spatial masking corresponds to the SE (CNN structure), which has a small number of parameters and therefore has a smaller impact on the final performance.
“The spatial masking corresponds to the SE (CNN structure), which has a small number of parameters and therefore has a smaller impact on the final performance.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S4 (based).
Premisenone given
Lightweight downstream neural network structure is meaningful because supervised learning phase is usually done on edge but not on cloud.
“Lightweight downstream neural network structure is meaningful, as supervised learning phase is usually done on edge but not on cloud.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given
If RD can recover the masked wearable values well, it means that the spatial and temporal encoders capture the spatial-temporal correlations well.
“If RD can recover the masked wearable values well, it means that the spatial and temporal encoders capture the spatial-temporal correlations well.”
Quote in the paperNot a substantial claim
Probably not a substantial claim: may not pass S4 (based).
Interpretationnone given
Randomly selecting N_s out of N devices to mask all data from selected devices simulates the real-world scenario where some devices are not available during inference.
“We randomly select 𝑁 s out of 𝑁 devices in each input sample such that all data from the selected devices are masked.This simulates the real-world scenario where some devices are not available during inference phase.”
Quote in the paperNot a substantial claim
- Rests on an assumption: It is impossible for every user to wear all devices at all time in daily life.
May combine two claims.
Interpretationown data
These computational costs are relatively low and meet the requirements for practical deployment.
“These computational costs are relatively low and meet the requirements for practical deployment.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Interpretationreasoning
The labeled training data on RealWorld dataset are insufficient to fit the entire network structure of the encoders and classifier, which may explain the slight decrease in performance under the fine-tuned setting.
“This may because the labeled training data on RealWorld dataset are insufficient to fit the entire network structure of the encoders and classifier.”
Quote in the paperSubstantial claim
Classified as probably not a substantial claim.
Interpretationreasoning
The Transformer structure is capable of learning features more quickly and efficiently due to its parallel computation characteristic.
“It achieves the best performance among the compared structures and is capable of learning features more quickly and efficiently due to its parallel computation characteristic.”
Quote in the paperSubstantial claim
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationreasoning
The specific number of applicable devices varies depending on the dataset and usage scenario.
“The specific number of applicable devices varies depending on the dataset and usage scenario.”
Quote in the paperNot a substantial claim
- Rests on an assumption: Missing device scenarios are an inevitable problem for multi-device WHAR.

43 of 43

The paper relies on this without support.
Wearable data require annotation at the time of data collection, unlike image or text data that can be annotated after collection.
“Unlike image or text data that can be annotated through crowdsourcing or other means after data collection, wearable data require annotation at the time of data collection.”
The paper relies on this without support.
It is impossible for every user to wear all devices at all time in daily life.
“This situation is realistic, as it is impossible for every user to wear all devices at all time in daily life.”
The paper relies on this without support.
Spatial correlations between data from various wearable devices differ depending on the activity being performed.
“This is because the spatial correlations differ depending on the activity being performed.”
The paper relies on this without support.
Spatial correlations can be utilized to infer data from one device based on data from other devices, making them a powerful means of reducing data redundancy.
“These spatial correlations can also be utilized to infer data from one device based on the data from other devices, making them a powerful means of reducing data redundancy.”
The paper relies on this without support.
Each device has a relatively small number of raw features (e.g., an IMU device typically contains raw features from a 3-axis accelerometer, a 3-axis gyroscope, and a 3-axis magnetometer).
“Each device has a relatively small number of raw features.For example, an IMU device typically contains raw features from a 3-axis accelerometer, a 3-axis gyroscope, and a 3-axis magnetometer.”
The paper relies on this without support.
Wearable data contain a relatively large number of temporal features and rich temporal context information; the time dimension is an order of magnitude larger than the spatial dimension.
“However, wearable data contain a relatively large number of temporal features and rich temporal context information.For example, a 2-seconds input sample at a common sampling rate of 50 Hz contains 100 time points in the time dimension.It means that the features in the time dimension are an order of magnitude larger than those in the spatial dimension.”
The paper relies on this without support.
Due to the high sampling rate of wearable devices (typically 50 Hz–200 Hz), raw readings at two adjacent time points have a strong similarity.
“It is important to note that due to the high sampling rate of wearable devices (typically 50 Hz∼200 Hz), the raw readings at two adjacent time points have a strong similarity.”
The paper relies on this without support.
Different activities exhibit different periodic patterns, amplitude ranges, etc., in time series.
“Since different activities exhibit different periodic patterns, amplitude ranges, etc., in the time series, capturing temporal correlations is crucial to obtain features that can fully characterize the current activity.”
The paper relies on this without support.
Missing device scenarios are an inevitable problem for multi-device WHAR.
“We explore the performance of STMAE under missing device scenarios, which is an inevitable problem for multidevice WHAR.”
The paper relies on this without support.
The four selected datasets enable a comprehensive and unbiased evaluation of the method.
“These datasets have distinct characteristics in the number of subjects, the number of devices, the number and type of activities, and the sampling frequency, which opens the door to a comprehensive and unbiased evaluation of our method.”

10 of 10

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

The table reverses the stated labeling rates for RealWorld and Realdisp
Major
The experimental settings assign 1% labeled data to Realdisp and 10% to the other datasets. Table 2 instead assigns 1% to RealWorld and 10% to Realdisp, leaving the labeling rates behind the main comparison unclear.
The classifier input dimension does not match the TE output
Minor
The classifier is specified to take a D_rd-dimensional input, yet it directly receives the averaged TE output, which has dimension D_te. Table 3 sets these dimensions to 64 and 32, respectively, for PAMAP; they also differ for Opportunity and RealWorld.
RealWorld is not smaller than Realdisp
Minor
The text justifies Realdisp's lower labeling rate by saying the remaining datasets have much smaller sample sizes. Table 1 lists 39,281 RealWorld samples versus 6,280 Realdisp samples.

3 of 3

Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 3× · see citing works
“Recently, there has been notable advancement in learning general representations of images [3, 28], videos [18], and audio [31] through mask reconstruction-based SSL methods.”
BackgroundMethod
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · see citing works
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv · doi:10.48550/arxiv.1803.01271 · Cited by 4,655 · see citing works
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1× · see citing works
“DeepConvLSTM [50] first concatenates the multi-device data from each device together.”
Method
Transformers in Time Series: A SurveyPreprint
Wen, Q., Zhou, T., Zhang, C.-L., Chen, W.-Q., Ma, Z., Yan, J., & Sun, L. (2022) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.2202.07125 · Cited by 1,763 · see citing works
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 3× · see citing works
“PAMAP [57]: PAMAP dataset includes data from three IMUs and a heart rate monitor collected from 9 individuals during the performance of 12 realistic activities.”
BackgroundMethod
A Transformer-based Framework for Multivariate Time Series Representation Learning
Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A., & Eickhoff, C. (2021) · Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining · doi:10.1145/3447548.3467401 · Cited by 1,278 · see citing works
Large Scale Population Assessment of Physical Activity Using Wrist Worn Accelerometers: The UK Biobank Study
Doherty, A., Jackson, D., Hammerla, N., Plötz, T., Olivier, P., Granat, M. H., White, T., van Hees, V. T., Trenell, M. I., Owen, C. G., et al. (2017) · PLOS ONE · doi:10.1371/journal.pone.0169649 · Cited by 1,250 · see citing works
TS2Vec: Towards Universal Representation of Time Series
Yue, Z., Wang, Y., Duan, J., Yang, T., Huang, C., Tong, Y., & Xu, B. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20881 · Cited by 850 · see citing works
Collecting complex activity datasets in highly rich networked sensor environments
Roggen, D., Calatroni, A., Rossi, M., Holleczek, T., Forster, K., Troster, G., Lukowicz, P., Bannach, D., Pirkl, G., Ferscha, A., et al. (2010) · 2010 Seventh International Conference on Networked Sensing Systems (INSS) · doi:10.1109/inss.2010.5573462 · Cited by 788 · see citing works
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690 · see citing works
DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · see citing works
Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency ConsistencyPreprint
Zhang, X., Zhao, Z.-Y., Tsiligkaridis, T., et al. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2206.08496 · Cited by 561 · see citing works
On-body localization of wearable devices: An investigation of position-aware activity recognition
Sztyler, T., & Stuckenschmidt, H. (2016) · 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2016.7456521 · Cited by 369 · see citing works
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346 · see citing works
Masked Autoencoders as Spatiotemporal Learners
Feichtenhofer, C., Fan, H., Li, Y., & He, K. (2022) · neural information processing systems · doi:10.52202/068431-2605 · Cited by 343 · see citing works
Proceedings of the 2012 ACM Conference on Ubiquitous Computing
Dey, A. K., Chu, H.-H., & Hayes, G. R. (2012) · doi:10.1145/2370216 · Cited by 332 · see citing works
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314 · cited 1× · see citing works
“This renders activity labeling difficult, reducing the number of available labels [8, 23, 30].”
Background
A Survey on Deep Learning for Human Activity Recognition
Gu, F., Chung, M.-H., Chignell, M., Valaee, S., Zhou, B., & Liu, X. (2021) · ACM Computing Surveys · doi:10.1145/3472290 · Cited by 287 · see citing works
MultiMAE: Multi-modal Multi-task Masked Autoencoders
Bachmann, R., Mizrahi, D., Atanov, A., & Zamir, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_20 · Cited by 268 · see citing works
AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition
Ma, H., Li, W., Zhang, X., Gao, S., & Lu, S. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/431 · Cited by 266 · see citing works
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · arXiv · doi:10.48550/arxiv.2207.06405 · Cited by 206 · see citing works
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · neural information processing systems · doi:10.52202/068431-2081 · Cited by 206 · see citing works
Text Recognition in the Wild: A Survey
Chen, X., Jin, L., Zhu, Y., Luo, C., & Wang, T. (2022) · ACM Computing Surveys · doi:10.1145/3440756 · Cited by 205 · see citing works
A benchmark dataset to evaluate sensor displacement in activity recognition
Baños, O., Damas, M., Pomares, H., Rojas, I., Tóth, M. A., & Amft, O. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370437 · Cited by 163 · see citing works
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150 · cited 9× · see citing works
“Leave-one-group-out cross-validation approach is used for performance evaluation follows the previous works [26, 34] .”
BackgroundMethodInfluential
LIMU-BERT
Xu, H., Zhou, P., Tan, R., Li, M., & Shen, G. (2021) · Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/3485730.3485937 · Cited by 150 · cited 9× · see citing works
“Masked reconstruction tasks leverage encoders to extract valid features from partially masked input and decoders to recover the complete input by learning the correlation patterns within the data [25, 56, 71].”
BackgroundMethodResultInfluential
Masked reconstruction based self-supervision for human activity recognition
Haresamudram, H., Beedu, A., Agrawal, V., Grady, P. L., Essa, I., Hoffman, J., & Plötz, T. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414306 · Cited by 146 · see citing works
Multimodal autoencoder: A deep learning approach to filling in missing sensor data and enabling better mood prediction
Jaques, N., Taylor, S., Sano, A., & Picard, R. W. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273601 · Cited by 135 · see citing works
Untitled work
Peng (2018) · doi:10.1145/3214277 · Cited by 134 · see citing works
Self-citation
Position-aware activity recognition with wearable devices
Sztyler, T., Stuckenschmidt, H., & Petrich, W. H. (2017) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2017.01.008 · Cited by 131 · see citing works
Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness
Zappi, P., Stiefmeier, T., Farella, E., Roggen, D., Benini, L., & Troster, G. (2007) · 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information · doi:10.1109/issnip.2007.4496857 · Cited by 126 · see citing works
Untitled work
Zhang (2020) · doi:10.1145/3432192 · Cited by 126 · see citing works
Context Recognition In-the-Wild: Unified Model for Multi-Modal Sensors and Multi-Label Classification
Vaizman, Y., Weibel, N., & Lanckriet, G. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3161192 · Cited by 122 · see citing works
Probabilistic Recovery of Incomplete Sensed Data in IoT
Fekade, B., Maksymyuk, T., Kyryk, M., & Jo, M. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2730360 · Cited by 114 · see citing works
ColloSSL: Collaborative Self-Supervised Learning for Human Activity Recognition
Jain, Y., Tang, C. I., Min, C., Kawsar, F., & Mathur, A. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3517246 · Cited by 113 · see citing works
Attend and Discriminate: Beyond the State-of-the-Art for Human Activity Recognition Using Wearable Sensors
Abedin, A., Ehsanpour, M., Shi, Q., Rezatofighi, H., & Ranasinghe, D. C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448083 · Cited by 107 · see citing works
Assessing the State of Self-Supervised Human Activity Recognition using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · arXiv · Cited by 107 · see citing works
PMC: A Privacy-preserving Deep Learning Model Customization Framework for Edge Computing
Liu, B., Li, Y., Liu, Y., Guo, Y., & Chen, X. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3432208 · Cited by 95 · see citing works
Cosmo: contrastive fusion learning with small data for multimodal human activity recognition
Ouyang, X., Shuai, X., Zhou, J., Shi, I. W., Xie, Z., Xing, G., & Huang, J. (2022) · Proceedings of the 28th Annual International Conference on Mobile Computing And Networking · doi:10.1145/3495243.3560519 · Cited by 94 · see citing works
COCOA: Cross Modality Contrastive Learning for Sensor Data
Deldari, S., Xue, H., Saeed, A., Smith, D. V., & Salim, F. D. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550316 · Cited by 92 · see citing works
NeckSense
Zhang, S., Zhao, Y., Nguyen, D. T., Xu, R., Sen, S., Hester, J. D., & Alshurafa, N. I. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397313 · Cited by 92 · see citing works
On the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90 · see citing works
CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series ForecastingPreprint
Woo, G., Liu, C., Sahoo, D., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.01575 · Cited by 87 · see citing works
Contrastive Self-supervised Learning for Sensor-based Human Activity Recognition
Khaertdinov, B., Ghaleb, E., & Asteriadis, S. (2021) · 2021 IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/ijcb52358.2021.9484410 · Cited by 85 · cited 4× · see citing works
“Contrastive learning tasks aim to maximize agreement of an instance and its positive samples while minimizing agreement between the instance and its negative samples in feature space [14, 26, 34, 40, 51, 54, 58, 64].”
BackgroundMethodInfluential
Untitled work
Ling, C. (2020) · doi:10.1145/3381012 · Cited by 72 · see citing works
Self-citation
CoolMoves
Ahuja, K., Ofek, E., González-Franco, M., Holz, C., & Wilson, A. D. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463499 · Cited by 72 · see citing works
Real-time Arm Skeleton Tracking and Gesture Inference Tolerant to Missing Wearable Sensors
Liu, Y., Li, Z., Liu, Z., & Wu, K. (2019) · Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3307334.3326109 · Cited by 71 · see citing works
Quantifying the Causal Effect of Individual Mobility on Health Status in Urban Space
Zhang, Y., Xu, F., Xia, T., & Li, Y. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3494990 · Cited by 68 · see citing works
Exploring Contrastive Learning in Human Activity Recognition for Healthcare
Tang, C. I., Perez-Pozuelo, I., Spathis, D., & Mascolo, C. (2020) · arXiv · doi:10.48550/arxiv.2011.11542 · Cited by 67 · see citing works
n-Gage
Gao, N., Shao, W., Rahaman, M. S., & Salim, F. D. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3411813 · Cited by 61 · see citing works
BLEselect: Gestural IoT Device Selection via Bluetooth Angle of Arrival Estimation from Smart Glasses
Zhang, T., Lan, Z., Xu, C., Li, Y., & Chen, Y. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569482 · Cited by 58 · see citing works
Graph Contrastive Learning for Skeleton-based Action RecognitionPreprint
Huang, X.-H., Zhou, H., Wang, J., Feng, H., Han, J.-Y., Ding, E.-R., Wang, J.-D., Wang, X.-G., Liu, W.-Y., & Feng, B. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2301.10900 · Cited by 53 · see citing works
SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications
Yao, S., Zhao, Y., Shao, H., Liu, D., Liu, S., Hao, Y., Piao, A., Hu, S., Lu, S., & Abdelzaher, T. F. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737500 · Cited by 52 · see citing works
IF-ConvTransformer
Zhang, Y., Wang, L., Chen, H., Tian, A., Zhou, S., & Guo, Y. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534584 · Cited by 47 · see citing works
SWL-Adapt: An Unsupervised Domain Adaptation Model with Sample Weight Learning for Cross-User Wearable Human Activity Recognition
Hu, R., Chen, L., Miao, S., & Tang, X. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25743 · Cited by 44 · see citing works
Self-citation
Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous Data
Wang, C., Gao, Y., Mathur, A., Williams, A. C. D. C., Lane, N. D., & Bianchi‐Berthouze, N. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463508 · Cited by 41 · see citing works
ApneaDetector
Chen, X., Xiao, Y., Tang, Y., Fernández‐Mendoza, J., & Cao, G. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463514 · Cited by 39 · see citing works
Robust Inertial Motion Tracking through Deep Sensor Fusion across Smart Earbuds and Smartphone
Gong, J., Zhang, X., Huang, Y., Ren, J., & Zhang, Y. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463517 · Cited by 37 · see citing works
Towards a Dynamic Inter-Sensor Correlations Learning Framework for Multi-Sensor-Based Wearable Human Activity Recognition
Miao, S., Chen, L., Hu, R., & Luo, Y. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550331 · Cited by 36 · see citing works
Self-citation
Applying Machine Learning for Sensor Data Analysis in Interactive Systems: Common Pitfalls of Pragmatic Use and Ways to Avoid Them
PlÖtz, T. (2022) · ACM Computing Surveys · doi:10.1145/3459666 · Cited by 34 · see citing works
A Graph-Based Approach for Missing Sensor Data Imputation
Xiao, J., Tian, Z., & Li, K. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3106656 · Cited by 33 · see citing works
n-Gage: Predicting in-class Emotional, Behavioural and Cognitive Engagement in the WildPreprint
Gao, N., Shao, W., Rahaman, M. S., & Salim, F. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04831 · Cited by 31 · see citing works
Individual and Group-wise Classroom Seating Experience
Gao, N., Rahaman, M. S., Shao, W., Ji, K., & Salim, F. D. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3550335 · Cited by 27 · see citing works
Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data
Deldari, S., Xue, H., Saeed, A., He, J., Smith, D. V., & Salim, F. D. (2022) · arXiv · doi:10.48550/arxiv.2206.02353 · Cited by 26 · see citing works
A Multi-Sensor Approach to Automatically Recognize Breaks and Work Activities of Knowledge Workers in Academia
Lascio, E. D., Gashi, S., Hidalgo, J. S., Nale, B., Debus, M. E., & Silvia Santini, B. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3411821 · Cited by 25 · see citing works
Synthesizing and Reconstructing Missing Sensory Modalities in Behavioral Context Recognition
Saeed, A., Özçelebi, T., & Lukkien, J. J. (2018) · Sensors · doi:10.3390/s18092967 · Cited by 23 · see citing works
Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion
Rahimi Taghanaki, S., Rainbow, M. J., & Etemad, A. (2021) · 2021 International Symposium on Wearable Computers · doi:10.1145/3460421.3480417 · Cited by 22 · cited 7× · see citing works
“Masked reconstruction tasks leverage encoders to extract valid features from partially masked input and decoders to recover the complete input by learning the correlation patterns within the data [25, 56, 71].”
BackgroundMethodResultInfluential
SALIENCE: An Unsupervised User Adaptation Model for Multiple Wearable Sensors Based Human Activity Recognition
Chen, L., Zhang, Y., Miao, S., Zhu, S., Hu, R. J., Peng, L., & Lv, M. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3171312 · Cited by 22 · see citing works
Self-citation
Ubi-SleepNet
Zhai, B., Guan, Y., Catt, M., & Plötz, T. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3494961 · Cited by 21 · see citing works
SyncUp
Zhou, Z., Xu, A., & Yatani, K. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478120 · Cited by 20 · see citing works
Handling Missing Sensors in Topology-Aware IoT Applications with Gated Graph Neural Network
Liu, S., Yao, S., Huang, Y., Liu, D., Shao, H., Zhao, Y., Li, J., Wang, T., Wang, R., Yang, C., et al. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411818 · Cited by 20 · see citing works
GlucoMine
Bartolome, A., Shah, S., & Prioleau, T. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478109 · Cited by 16 · see citing works
Augmented Adversarial Learning for Human Activity Recognition with Partial Sensor Sets
Kang, H., Huang, Q., & Zhang, Q. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550285 · Cited by 16 · see citing works
Towards Early Detection and Burden Estimation of Atrial Fibrillation in an Ambulatory Free-living Environment
Zhang, H., Zhu, L., Nathan, V., Kuang, J., Kim, J., Gao, J. A., & Olgin, J. E. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463503 · Cited by 10 · see citing works
LAX-Score
Jung, W., Watson, A., Kuehn, S., et al. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478076 · Cited by 8 · see citing works
MotionBERT: A Unified Perspective on Learning Human Motion RepresentationsPreprint
Zhu, W., Ma, X., Liu, Z., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.06551 · Cited by 8 · see citing works
SwingNet
Jia, H. G., Hu, J., & Hu, W. S. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3478082 · Cited by 5 · see citing works
MHCCL: Masked Hierarchical Cluster-Wise Contrastive Learning for Multivariate Time SeriesPreprint
Meng, Q., Qian, H., Liu, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.01141 · Cited by 3 · see citing works
What Makes Good Contrastive Learning on Small-Scale Wearable-based Tasks?Preprint
Qian, H., Tian, T., & Miao, C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05998
Learning from the Best: Contrastive Representations Learning Across Sensor Locations for Wearable Activity RecognitionPreprint
Rey, V. F., Suh, S., & Lukowicz, P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.01459
Large scale population assessment of physical activity using wrist worn accelerometers: The UK biobank study
Doherty, A., Jackson, D., Hammerla, N., et al. (2017) · Newcastle University ePrints (Newcastle Univesity) · doi:10.25405/ncl.34308291
Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2022) · arXiv · doi:10.48550/arxiv.2206.02909
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Tang, C. I., Perez-Pozuelo, I., Spathis, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.06073
Nested Gaussian process modeling and imputation of high-dimensional incomplete data under uncertainty
Imani, F., Cheng, C., Chen, R., et al. (2019) · IISE Transactions on Healthcare Systems Engineering · doi:10.1080/24725579.2019.1583704

Flagged first, then by citations

Research works printed but not linked: 10

Pierre Baldi. 2012. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML Workshop on Unsupervised and Transfer Learning. 37--49.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning. 1597--1607.
Farhad Imani, Changqing Cheng, Ruimin Chen, and Hui Yang. 2018. Nested gaussian process modeling for high-dimensional data imputation in healthcare systems. In Proceedings of the IIE Annual Conference. 1312--1317.
10.1145/3432208 · DOI · Paper page
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. 2021. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In Proceedings of the International Conference on Learning Representations.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. 5998--6008.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Proceedings of the Advances in Neural Information Processing Systems, 22419--22430.
Ling Yang and Shenda Hong. 2022. Unsupervised time-series representation learning with iterative bilinear temporal-spectral fusion. In International Conference on Machine Learning. PMLR, 25038--25054.
Jinsung Yoon, James Jordon, and Mihaela Schaar. 2018. Gain: Missing data imputation using generative adversarial nets. In Proceedings of the International Conference on Machine Learning. 5689--5698.
Jingguang Zhou and Zili Huang. 2018. Recover missing sensor data with iterative imputing network. In Proceedings of the Workshops at the AAAI Conference on Artificial Intelligence.

Sorted by citations

Energy-aware human activity recognition for wearable devices: A comprehensive review
Contoli, C., Freschi, V., & Lattanzi, E. (2024) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2024.101976 · Cited by 43 · see citing works
Self-supervised Learning for Accelerometer-based Human Activity Recognition: A Survey
Logacjov, A. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699767 · Cited by 25 · see citing works
GOAT: A Generalized Cross-Dataset Activity Recognition Framework with Natural Language Supervision
Miao, S., & Chen, L. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699736 · Cited by 17 · see citing works
Self-citation
Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2024) · Sensors · doi:10.3390/s24041238 · Cited by 14 · see citing works
“Extending this setup, spatio-temporal masking was explored in [67].”
Background
PhyMask: An Adaptive Masking Paradigm for Efficient Self-Supervised Learning in IoT
Kara, D., Kimura, T., Chen, Y., Li, J., Wang, R., Chen, Y., Wang, T., Liu, S., & Abdelzaher, T. (2024) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3666025.3699325 · Cited by 11 · see citing works
MSMFT: Multi-Stream Multimodal Factorized Transformer for Human Activity Recognition
Zhou, X., Yuan, J., Fan, L., Niu, X., Zha, K., & Liu, X. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3529889 · Cited by 15 · see citing works
“Current HAR algorithms primarily focus on unimodal approaches using video, wearable devices, skeletons, Radar, and Wi-Fi [2], [3], [4], [5], [6], [7].”
MASTER: A Multi-modal Foundation Model for Human Activity Recognition
Zhu, G., Zhao, D. Y., Li, C., Zhao, M., Zhang, Z., Quan, H., & Ma, H. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749511 · Cited by 9 · see citing works
SemiCMT: Contrastive Cross-Modal Knowledge Transfer for IoT Sensing with Semi-Paired Multi-Modal Signals
Chen, Y., Hu, C., Kimura, T., Li, Q., Liu, S., Wu, F., & Chen, G. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699779 · Cited by 6 · see citing works
Long-term self-supervised learning for accelerometer-based sleep–wake recognition
Logacjov, A., Bach, K., & Mork, P. J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109758 · Cited by 4 · see citing works
OpenMAE: Efficient Masked Autoencoder for Vibration Sensing with Open-domain Data Enrichment
Hu, C., Chen, Y., Kara, D., Liu, S., Abdelzaher, T., Wu, F., & Chen, G. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729485 · Cited by 4 · see citing works
Foundation Models Defining A New Era In Sensor-based Human Activity Recognition: A Survey And Outlook
Bian, S., Liu, M., Ray, L. S. S., Zhou, B., Guo, B., Yu, Z., Plötz, T., Lukowicz, P., Yuan, S., & Rey, V. F. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3810230 · Cited by 7 · see citing works
A Conditional KAN Diffusion Network for Human Activity Recognition with Missing Sensor Signal Series
Xiong, H., Gong, J., Luo, H., Zhao, F., Gao, Y., Chen, R., & Xiao, M. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890337 · Cited by 1 · see citing works
“Second, existing methods often fail in scenarios with missing sensor data, which can lead to inaccurate predictions due to erroneous learning of spatiotemporal dependencies [16], [17].”
Multi-source sensing adaptation for human behavior modeling in fabric space
Yi, H., Wu, X., Hao, Y., & Chen, M. (2026) · Information Sciences · doi:10.1016/j.ins.2026.123381 · Cited by 1 · see citing works
Real-time Gait Event Tracker (RGET): An Innovative Approach to Digital Human Modeling through Integration of Inertial Measurement Unit and Dynamic Feature Extraction
Xia, T. (2024) · Journal of Electrical Systems · doi:10.52783/jes.1214 · Cited by 1 · see citing works
SCAGOT: Semi-Supervised Disentangling Context and Activity Features Without Target Data for Sensor-Based HAR
Duan, F., Zhu, T., Chen, L., Ning, H., Liu, C., & Wan, Y. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3543928 · Cited by 1 · see citing works
“With the rapid development of the Internet of Things, wireless, and mobile computing, sensor-based HAR has recently been widely used in various application scenarios, such as smart homes [8], [9], [10] and assisted living [11], [12], [13].”
Self-citation
Saga: Capturing Multi-granularity Semantics from Massive Unlabelled IMU Data
Li, Y., Hu, F., Zhu, H., Zhang, S., Zhang, L., Chang, S., & Guo, M. (2025) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs63083.2025.00090 · Cited by 1 · see citing works
COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
Chen, B., Wongso, W., Li, Z., Khaokaew, Y., Xue, H., & Salim, F. D. (2026) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3810218 · Cited by 1 · see citing works
A Washing Machine is All You Need? On the Feasibility of Machine Data for Self-Supervised Human Activity Recognition
Haresamudram, H., Essa, I. A., & Plötz, T. (2024) · International Conference on Activity and Behavior Computing (ABC) · doi:10.1109/abc61795.2024.10651688
“This setup was extended to probabilistically masking both temporally and across sensor channels [24].”
Method
Segment-Based Unsupervised Deep Learning for Human Activity Recognition using Accelerometer Data and SBOA based Channel Attention Networks
Janardhan, M., Neelima, A., Siri, D., Sathish Kumar, R., Balakrishna, N., Sreenivasa, N., Singasani, T. R., & Vatambeti, R. (2024) · International Research Journal of Multidisciplinary Technovation · doi:10.54392/irjmt2461
“The STMAE [19] classifier attained an accuracy of 92.50%, a recall of 96.40%, and a precision of 94.50%.”
BackgroundMethodInfluential
KCIN: A Kolmogorov-Arnold Convolutional Network-Based Model for Transportation Mode Recognition With Incomplete Sensor Series
Xiong, H., Luo, H., Gong, J., Zhao, F., Wang, J., & Ma, X. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00169
“Secondly, the majority of current methods do not account for scenarios where sensor data is missing [15].”
Elevator, Escalator, or Neither? Classifying Conveyor State Using Smartphone Under Arbitrary Pedestrian Behavior
He, T., Xia, Z., & Chan, S.-H. G. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3586618
Inertial Signal Forecasting With Foundation Model Techniques
Wieland, C., & Pankratius, V. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3603456
“Masking approaches, such as span masking [30] or spatial masking [35], learn to reconstruct signals with masked values.”
Can LLMs Learn the Language of Inertial Sensors in Human Activity Recognition?
Bigelli, L., Bogliolo, A., Calisti, L., Capellacci, P., Contoli, C., Kania, N., & Lattanzi, E. (2025) · International Conference on Machine Learning Technologies (ICMLT) · doi:10.1109/icmlt65785.2025.11193221
“Jungo et al. leveraged a masking technique for imputing missing wearable data in HAR for representation purposes [21], while Miao et al. proposed a spatiotemporal strategy to deal with missing device scenarios and construct an effective pretraining task in line with the masking approach [22].”
PFHAR: Practically Adopting Multi-Modal Foundation Model for Human Activity Recognition Through Edge-Cloud Collaborative Learning
Zhang, Z., Zhao, D. Y., Zhu, G., Li, X. Q., Li, C., Liu, B., Li, Y., & Ma, H. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3672513
“Different from existing mask reconstruction-based meth-ods [45], [56], which use an encoder-decoder architecture to restore masked embeddings, we employ supervised contrastive loss with the superclass information from the union dataset to train the foundation model, enabling it to extract…”
Influential

Comparing self-supervised learning techniques for wearable human activity recognition
Ek, S., Presotto, R., Civitarese, G., Portet, F., Lalanda, P., & Bettini, C. (2025) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-024-00182-9 · Cited by 15 · see citing works
Assessing the State of Self-Supervised Human Activity Recognition Using Wearables
Haresamudram, H., Essa, I., & Plötz, T. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550299 · Cited by 107 · see citing works
Lightweight and Generalizable Multi-Sensor Human Activity Recognition via Cascaded Fusion and Style-Augmented DecompositionPreprint
Chenglong, W., Yan, Z., Ding, W., & Xinlei, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.08910
Decomposing and Fusing Intra- and Inter-Sensor Spatio-Temporal Signal for Multi-Sensor Wearable Human Activity Recognition
Xie, H., Li, H., Zheng, C., Yuan, H., Liao, G., Liao, J., & Liu, L. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i13.33582 · Cited by 5 · see citing works
Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearables
Sheng, T., & Huber, M. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3397330 · Cited by 28 · see citing works

Authors: Miao, S., Chen, L., & Hu, R.
Affiliations: Zhejiang University (CN)
ORCID: Miao, Shenghuan
Chen, Ling
Hu, Rong
Venue: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Publisher: Association for Computing Machinery (ACM)
Published: 2023-12-19
Volume: 7
Issue: 4
Pages: 1-25
ISSN: 2474-9567 (ISSN-L)
DOI: 10.1145/3631415
OpenAlex: W4390821797
Cited by: 23
References: 63
Citations by year: 2026: 3 · 2025: 9 · 2024: 11
FWCI: 2.09
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
Social Sciences › Transportation › Human Mobility and Location-Based Analysis
Health Sciences › Medicine › Physiology › Physical Activity and Health
Keywords: self-supervised learning, masked autoencoder, unlabeled data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition

Abstract

Integrity signals

Claims the paper makes

Self-check

Cites

Cited by

Related

Record

About these checks