RobustHAR: Multi-scale Spatial-temporal Masked Self-supervised Pre-training for Robust Human Activity Recognition
Open accessAbstract
Human activity recognition (HAR) is prone to performance degradation in real-world applications due to data missing between intra-sensor and inter-sensor channels. Masked modeling, as one mainstream paradigm of self-supervised pre-training, can learn robust representations across sensors in the data missing scenario by reconstructing the masked content based on the unmasked part. However, the existing methods predominantly emphasize the temporal dynamics of human activities, which limits their ability to effectively capture the spatial interdependencies among multiple sensors. Besides, different human activities often span across various spatial-temporal scales, which results in activity recognizer failing to capture intricate spatial-temporal semantic information. To address these issues, we propose RobustHAR, a new HAR model with multi-scale spatial-temporal masked self-supervised pre-training designed to improve model performance on the data missing context. RobustHAR involves three main steps: (1) RobustHAR constructs location-inspired spatial-temporal 3D-variation modeling to capture spatial-temporal correlated information in human activity data. (2) RobustHAR then designs multi-scale spatial-temporal masked self-supervised pre-training with semantic-consistent multi-scale feature co-learning for learning robust features at different scales. (3) Finally, RobustHAR fine-tunes the pretraining model with adaptive multi-scale feature fusion for human activity recognition. Extensive experiments on three public multi-sensor datasets demonstrate that RobustHAR outperforms existing state-of-the-art methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 5 cited sources checked · 0 supported · 0 flagged
- Findingown data
RobustHAR attains the highest recognition accuracy on the Opportunity and RealWorld datasets when the balance coefficient is set to 0.5.
“As shown in Fig. 4(a), RobustHAR attains the highest recognition accuracy on the Opportunity and Real-World dataset when balance coefficient is set to 0.5.”
Quote in the paperNot a substantial claim - Findingown data
RobustHAR achieves the best performance on the CZU-MHAD dataset when the balance coefficient is set to 0.7.
“Meanwhile, RobustHAR achieves the best performance on the CZU-MHAD dataset when balance coefficient is set to 0.7.”
Quote in the paperNot a substantial claim - Findingnone given
The reconstruction performance applied at the channel-wise mask remains robust.
“Meanwhile, the reconstruction performance applied at the channel-wise mask remains robust.”
Quote in the paperNot a substantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
- Findingown data
When the masking rate is set to 0.5, RobustHAR performs best on the RealWorld dataset.
“As shown in Fig. 4(b), when masking rate is set to 0.3, RobustHAR achieves the highest recognition accuracy on the Opportunity and CZU-MHAD datasets; when the masking rate is set to 0.5, RobustHAR performs best on the RealWorld dataset.”
Quote in the paperNot a substantial claim - Findingown data
The robustness of self-supervised pre-trained models outperforms that of supervised models in data-missing scenarios.
“As can be seen from the table, the robustness of the self-supervised pre-trained model outperforms that of the supervised model.”
Quote in the paperSubstantial claim- Rests on an assumption: HAR models often face challenges when data is missing, either due to sensor failures or noise in the environment.
- Rests on an assumption: Randomly removing data points at different scales adequately simulates real-world data-missing scenarios.
- Findingown data
On the RealWorld dataset, removing multi-scale spatial-temporal masked pre-training results in a 10% reduction in recognition accuracy of RobustHAR.
“On the RealWorld dataset, this results in a 10% reduction in recognition accuracy of RobustHAR.”
Quote in the paperSubstantial claim - Findingnone given
At the segment-wise mask, RobustHAR can capture long-range dependencies between different activities.
“At the segment-wise mask, RobustHAR can capture long-range dependencies between different activities.”
Quote in the paperNot a substantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
- Findingreasoning
RobustHAR's computing complexity is slightly higher than other models (such as ST-MAE: O(LT N C)).
“It is slightly higher than other models (such as ST-MAE: O(LT N C)).”
Quote in the paperNot a substantial claim - Findingown data
Extensive experiments on three public multi-sensor datasets demonstrate that RobustHAR outperforms existing state-of-the-art methods.
“Extensive experiments on three public multi-sensor datasets demonstrate that RobustHAR outperforms existing state-of-the-art methods.”
- Also stated in: § 1 Introduction · boosted
“Besides, Extensive experiments conducted on three public multi-sensor datasets demonstrate that RobustHAR outperforms the current state-of-the-art algorithms.”
Quote in the paperSubstantial claim - Also stated in: § 1 Introduction · boosted
- Findingown data
RobustHAR exhibits varying degrees of reconstruction accuracy depending on the scale at which human activity data is masked.
“As illustrated in Fig. 5, RobustHAR exhibits varying degrees of reconstruction accuracy depending on the scale at which human activity data is masked.”
Quote in the paperNot a substantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
- Findingown data
The performance of RobustHAR is most significantly impacted when multi-scale spatial-temporal masked pre-training is not employed.
“Specifically, the experimental results indicate that the performance of RobustHAR is most significantly impacted when multi-scale spatial-temporal masked pre-training is not employed.”
Quote in the paperSubstantial claim - Findingown data
RobustHAR maintains robust performance for HAR in the data-missing scenario.
“Besides, experimental results demonstrate that RobustHAR surpasses state-of-the-art baselines, and maintains robust performance for HAR in the datamissing scenario.”
Quote in the paperSubstantial claim- Rests on an assumption: HAR models often face challenges when data is missing, either due to sensor failures or noise in the environment.
- Rests on an assumption: Randomly removing data points at different scales adequately simulates real-world data-missing scenarios.
- Findingown data
RobustHAR achieves superior performance compared to other baselines on the Opportunity, RealWorld, and CZU-MHAD datasets.
“The experimental results demonstrate that RobustHAR achieves superior performance compared to other baselines on the Opportunity, RealWorld, and CZU-MHAD datasets, as shown in the table 2.”
- Also stated in: § 5 Conclusion · boosted
“Besides, experimental results demonstrate that RobustHAR surpasses state-of-the-art baselines, and maintains robust performance for HAR in the datamissing scenario.”
Quote in the paperSubstantial claim - Also stated in: § 5 Conclusion · boosted
- Findingown data
When the masking rate is set to 0.3, RobustHAR achieves the highest recognition accuracy on the Opportunity and CZU-MHAD datasets.
“As shown in Fig. 4(b), when masking rate is set to 0.3, RobustHAR achieves the highest recognition accuracy on the Opportunity and CZU-MHAD datasets; when the masking rate is set to 0.5, RobustHAR performs best on the RealWorld dataset.”
Quote in the paperNot a substantial claim - Findingnone given
The overall computing complexity of RobustHAR is O(LT N C^2).
“The overall computing complexity of RobustHAR is O(LT N C 2 ), where L is the number of Transformer layers, T is the time step, N is the number of wearable sensors, and C is the dimensionality of the features generated by wearable sensors.”
Quote in the paperNot a substantial claim - Findingown data
RobustHAR achieves the best robust performance by capturing spatial-temporal dependencies at different scales.
“More importantly, RobustHAR achieves the best robust performance by capturing spatial-temporal dependencies at different scales.”
Quote in the paperSubstantial claim- Rests on an assumption: HAR models often face challenges when data is missing, either due to sensor failures or noise in the environment.
- Rests on an assumption: Randomly removing data points at different scales adequately simulates real-world data-missing scenarios.
- Premisecitation
Masked modeling can reconstruct masked wearable sensor data based on the unmasked part.
“In contrast, masked modeling can reconstruct the masked wearable sensor data based on the unmasked part.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Haresamudram et al., 2022
Not checked in the free run.
- Interpretationreasoning
The semantic consistency of sensor-wise features enables the model to better leverage a small amount of labeled data during the fine-tuning stage.
“This consistency enables the model to better leverage a small amount of labeled data during the fine-tuning stage.”
Quote in the paperSubstantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
- Interpretationown data
RobustHAR effectively mines multi-scale robust activity features with multi-scale spatial-temporal masked pre-training model.
“effectively mines multi-scale robust activity features with multi-scale spatial-temporal masked pretraining model.”
Quote in the paperSubstantial claim - Premisereasoning
Masked modeling facilitates learning robust activity representations without dependence on complete data.
“This approach facilitates the model's ability to learn robust activity representations without dependence on complete data.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Haresamudram et al., 2022
Not checked in the free run.
- Interpretationreasoning
RobustHAR captures temporal-spatial correlations information among intra-sensors and inter-sensors by location-inspired spatial-temporal 3D modeling.
“It not only captures temporal-spatial correlations information among intra-sensors and inter-sensors by locationinspired spatial-temporal 3D modeling”
Quote in the paperSubstantial claim - Interpretationnone given
RobustHAR fine-tunes the pre-training model with adaptive multi-scale feature fusion for human activity recognition.
“Finally, RobustHAR fine-tunes the pretraining model with adaptive multi-scale feature fusion for human activity recognition.”
- Also stated in: § 1 Introduction · plain
“RobustHAR fine-tunes the pre-training model with a small set of labeled data for human activity recognition by adaptive multiscale feature fusion.”
- Also stated in: § 1 Introduction · plain
“Finally, a small amount of labeled activity data is used to fine-tune the pre-training model for human activity recognition.”
Quote in the paperSubstantial claim - Also stated in: § 1 Introduction · plain
- Interpretationreasoning
RobustHAR can retain the channel-specific features for accurately recognizing human activities.
“This suggests that RobustHAR can retain the channel-specific features for accurately recognizing human activities.”
Quote in the paperSubstantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
- Interpretationreasoning
RobustHAR constructs location-inspired spatial-temporal 3D-variation modeling to capture spatial-temporal correlated information in human activity data.
“RobustHAR constructs location-inspired spatialtemporal 3D-variation modeling to capture spatialtemporal correlated information in human activity data.”
- Also stated in: § 1 Introduction · plain
“Location-inspired spatial-temporal 3D-variation modeling is constructed to mine spatial-temporal correlated information among human activity data.”
- Also stated in: § 3.1 Location-inspired spatial-temporal 3D-variation modeling · plain
“Therefore, RobustHAR leverages locationinspired spatial-temporal 3D-variation modeling to capture the spatial-temporal correlated information among wearable sensor-based human activity data.”
Quote in the paperSubstantial claim- Rests on an assumption: Human activity is temporally dynamic and spatially complex, and positional relationships between sensors significantly impact HAR performance.
- Also stated in: § 1 Introduction · plain
- Interpretationreasoning
RobustHAR ensures semantic consistency across activity features at different scales.
“Meanwhile, RobustHAR ensures semantic consistency across activity features at different scales.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Semantic-consistent multi-scale feature co-learning can enhance the robustness of the HAR model when wearable sensor data at one scale is incomplete, because the others can still provide sufficient semantic information.
“Besides, semantic-consistent multi-scale feature co-learning can enhance the robustness of HAR model when wearable sensor data at one scale is incomplete, the others can still provide sufficient semantic information.”
Quote in the paperNot a substantial claim - Premisereasoning
Contrastive learning encounters considerable limitations in addressing data missing issues prevalent in real-world applications, because it relies on complete sample pairs for training.
“This limitation arises from its reliance on complete sample pairs for training.”
Quote in the paperSubstantial claim- Rests on an assumption: Contrastive learning relies on complete sample pairs for training.
- Interpretationown data
An appropriately chosen masking rate strikes a balance between the complexity of the HAR task and the capacity of HAR to learn meaningful and generalized representations from human activity data.
“An appropriately chosen masking rate strikes a balance between the complexity of HAR task and the capacity of HAR to learn meaningful and generalized representations from human activity data.”
Quote in the paperSubstantial claim - Premisecitation
Existing masked modeling methods predominantly emphasize temporal dynamics, limiting their ability to capture spatial interdependencies among multiple sensors.
“However, the existing methods predominantly emphasize the temporal dynamics of human activities, which limits their ability to effectively capture the spatial interdependencies among multiple sensors.”
- Also stated in: § 1 Introduction · plain
“However, since semantic information of sensor-based action data is mainly contained in temporal variations [Dong et al., 2024], the existing masked modeling often ignores spatial correlated information among different wearable sensors.”
- Also stated in: § 2.2 Self-supervised pre-training learning for human activity recognition · plain
“Nevertheless, since semantic information in sensor-based activity data is often contained in temporal variations [Dong et al., 2024], most existing masked modeling often ignores spatially correlated information among different wearable sensors.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Dong et al., 2024
Not checked in the free run.
- Also stated in: § 1 Introduction · plain
- Interpretationreasoning
By considering the spatial distribution of sensors, we can better understand the relative relationships of each body part during movement, thereby extracting richer spatial-temporal activity features.
“By considering the spatial distribution of sensors, we can better understand the relative relationships of each body parts during movement, thereby extracting richer spatial-temporal activity features.”
Quote in the paperNot a substantial claim- Rests on an assumption: Human activity is temporally dynamic and spatially complex, and positional relationships between sensors significantly impact HAR performance.
- Interpretationreasoning
Multi-scale spatial-temporal masked is used to extract rich contextual semantics across diverse scales.
“To capture such fine-grained distinctions, multi-scale spatial-temporal masked is used to extract rich contextual semantics across diverse scales.”
Quote in the paperNot a substantial claim- Rests on an assumption: Different human activities span across various spatial-temporal scales, causing activity recognizers to fail to capture intricate spatial-temporal semantic information.
- Rests on an assumption: Human activities inherently exhibit multi-scale temporal and spatial characteristics.
- Interpretationreasoning
When performing masking operations on multi-scale data, the HAR model needs to ensure that the information representation at different scales remains consistent.
“When performing masking operations on multi-scale data (such as segment-wise mask, channel-wise mask, and sensorwise mask), HAR model needs to ensure that the information representation at different scales remains consistent.”
Quote in the paperSubstantial claim - Interpretationreasoning
RobustHAR designs multi-scale spatial-temporal masked self-supervised pre-training with semantic-consistent multi-scale feature co-learning for learning robust features at different scales.
“RobustHAR then designs multiscale spatial-temporal masked self-supervised pretraining with semantic-consistent multi-scale feature co-learning for learning robust features at different scales.”
- Also stated in: § 1 Introduction · plain
“Multi-scale spatial-temporal masked pre-training with semantic-consistent multi-scale feature co-learning is designed for capturing robust spatial-temporal contextual information at different scales.”
- Also stated in: § 1 Introduction · plain
“Secondly, multi-scale spatial-temporal masked pre-training enables activity recognition model to learn robust feature across various temporal and spatial scales, thereby improving its capacity to capture the contextual information inherent in human activity.”
Quote in the paperSubstantial claim- Rests on an assumption: Different human activities span across various spatial-temporal scales, causing activity recognizers to fail to capture intricate spatial-temporal semantic information.
- Rests on an assumption: Human activities inherently exhibit multi-scale temporal and spatial characteristics.
- Also stated in: § 1 Introduction · plain
- Interpretationreasoning
Sensor-wise features can ensure semantic coherence across different feature scales by semantic-consistent multi-scale feature co-learning.
“Although there is a discrepancy between the reconstructed human activity data (sensor-wise mask) and the original ground-truth data, sensor-wise features can ensure semantic coherence across different feature scales by semantic-consistent multi-scale feature co-learning.”
Quote in the paperNot a substantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
- Classified as probably not a substantial claim. 15
- May combine two claims. 13
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Findingnone givenRobustHAR's robustness and accuracy in data-missing scenarios far exceed those of the others.
“However, its robustness and accuracy in data-missing scenarios far exceed those of the others.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataBoth the location-inspired spatial-temporal 3D-variation modeling and fine-tuning the pre-training model with adaptive multi-scale feature fusion play crucial roles in enhancing the final recognition performance of RobustHAR.
“Furthermore, it is apparent that both the location-inspired spatialtemporal 3D-variation modeling and fine-tune pre-training model with adaptive multi-scale feature fusion play crucial roles in enhancing the final recognition performance.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenSelf-supervised learning-based human activity recognition models, particularly masked modeling methods such as SimMTM, MaskMAE, and ST-MAE, can enable the activity recognition model to learn global and contextual semantic features by masking portions of the input data.
“Compared to supervised learning models, self-supervised learning-based human activity recognition models, particularly in masked modeling (such as SimMTM, MaskMAE, and ST-MAE), these models can enable the activity recognition model to learn global and contextual semantic features by masking portions of the input data.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningData missing compromises the construction of positive and negative pairs in contrastive learning, leading to biased representations.
“Data missing compromises the construction of positive and negative pairs, leading to biased representations.”
Quote in the paperSubstantial claim- Rests on an assumption: Contrastive learning relies on complete sample pairs for training.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenSTD-MAE can model the heterogeneity of spatial-temporal activity data by spatial-temporal-decoupled masked pre-training.
“Meanwhile, STD-MAE can model the heterogeneity of spatial-temporal activity data by spatial-temporal-decoupled masked pre-training.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Premisenone givenTransformer-based action recognition models such as Reformer and Informer can leverage the modeling capacity of a Transformer with an architecture that can capture long-sequence semantic information of activity data.
“Transformer-based action recognition models (such as Reformer and Informer) can leverage the modeling capacity of a Transformer with an architecture that can capture long-sequence semantic information of activity data.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningMulti-scale spatial-temporal masked enhances the model's ability to learn comprehensive activity features and improves robustness in data-missing scenarios by leveraging spatial-temporal correlations at different scales.
“This approach not only enhances the model's ability to learn comprehensive activity features but also improves robustness in data-missing scenarios by leveraging spatial-temporal correlations at different scales.”
Quote in the paperSubstantial claim- Rests on an assumption: Different human activities span across various spatial-temporal scales, causing activity recognizers to fail to capture intricate spatial-temporal semantic information.
- Rests on an assumption: Human activities inherently exhibit multi-scale temporal and spatial characteristics.
Probably not a substantial claim: may not pass S4 (based).
Premisenone givenMasked modeling methods often fail to account for the influence of spatial-temporal features across different scales on HAR.
“While these masked modeling methods can effectively capture the spatial-temporal features of human activities, they often fail to account for the influence of spatial-temporal features across different scales on HAR.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAll experimental results are reported as mean values obtained from five independent trials.
“All experimental results are reported as mean values obtained from five independent trials.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningIf masking operations at each scale lead to inconsistent semantic information, the HAR model may lose its overall understanding of human activity, resulting in a decline in recognition performance.
“If the masking operations at each scale lead to inconsistent semantic information, HAR model may lose its overall understanding of human activity, resulting in a decline in recognition performance.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningWhen the masking rate is too low, the HAR model has access to an excessive amount of information, facilitating rapid convergence but potentially leading to overfitting.
“When the masking rate is too low, HAR model has access to an excessive amount of information, facilitating rapid convergence but potentially leading to overfitting.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningSuch a process enhances the model's robustness in the presence of data-missing interference, while facilitating the extraction of more meaningful features during training, thereby improving overall performance in activity recognition tasks.
“Such a process enhances the model's robustness in the presence of data-missing interference, while facilitating the extraction of more meaningful features during training, thereby improving the overall performance in activity recognition tasks.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenFeatures at different scales can capture distinct aspects of human activity; segment-wise features focus on subtle motion details, while sensor-wise features highlight the spatial correlations between different sensors.
“Features at different scales can capture distinct aspects of human activity.For example, segment-wise features focus on subtle motion details, while sensor-wise features highlight the spatial correlations between different sensors.”
Quote in the paperNot a substantial claim- Rests on an assumption: Features at different scales capture distinct aspects of human activity.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisereasoningTimeNet and MSGNet can improve the accuracy of activity recognition.
“Hence, TimeNet and MSGNet can improve the accuracy of activity recognition.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThe decoding process recovers masked spatial-temporal activity data and reinforces the model's comprehension of data integrity by progressively reconstructing human activity features.
“This decoding process not only recovers the masked spatial-temporal activity data, but also reinforces the model's comprehension of data integrity by progressively reconstructing human activity features.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningAt the sensor-wise level, RobustHAR can effectively exploit cross-sensor correlations to reconstruct the missing sensor values, demonstrating its ability to capture spatial dependencies between different sensor locations.
“Besides, at the sensor-wise level, where individual sensor data is masked, RobustHAR can effectively exploit cross-sensor correlations to reconstruct the missing sensor values, demonstrating its ability to capture spatial dependencies between different sensor locations.”
Quote in the paperSubstantial claim- Rests on an assumption: The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
May combine two claims.
Interpretationown dataRobustHAR can capture the spatial-temporal relationships among inter-sensor and intra-sensor at different scales, thereby ensuring the semantic consistency of features across various scales.
“Overall, RobustHAR can capture the spatial-temporal relationships among inter-sensor and intra-sensor at different scales, thereby ensuring the semantic consistency of features across various scales.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe multi-scale reconstruction loss ensures accurate reconstruction of the masked data across different scales, helping the model recover missing information and preserve the original structure of human activity data.
“The former is used to ensure the accurate reconstruction of the masked data across different scales, helping the model recover the missing information and preserve the original structure of human activity data.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Premisenone givenTransformer-based action recognition models overlook the varying inter-series correlations across different time scales.
“However, these methods overlook the varying inter-series correlations across different time scales.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenAll models, including baselines and RobustHAR, were carefully tuned to ensure fair comparisons.
“To ensure fair comparisons, we carefully tune all models, including both baselines and our proposed RobustHAR.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenThe spatial-temporal 3D variation X represents human activity data that are spatially adjacent and also adjacent in the semantic space, and this adjacency ensures the coherence and validity of human activity data.
“The spatial-temporal 3D variation X represents human activity data that are spatially adjacent and also adjacent in the semantic space.This adjacency ensures the coherence and validity of human activity data.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe semantic consistency loss enforces semantic consistency across different scales, ensuring that feature representations at each scale align in terms of their underlying semantic information.
“The latter enforces semantic consistency across the different scales, ensuring that the feature representations at each scale align in terms of their underlying semantic information.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationContrastive learning can effectively extract features by data augmentation.
“Although contrastive learning can effectively extract features by data augmentation”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Zhang et al., 2025a
Not checked in the free run.
May combine two claims.
PremisecitationExisting masked modeling often ignores spatial correlated information among different wearable sensors, because semantic information of sensor-based action data is mainly contained in temporal variations.
“However, since semantic information of sensor-based action data is mainly contained in temporal variations [Dong et al., 2024], the existing masked modeling often ignores spatial correlated information among different wearable sensors.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Dong et al., 2024
Not checked in the free run.
The paper relies on this without support.
Human activity recognition performance degrades in real-world applications due to data missing between intra-sensor and inter-sensor channels.
“Human activity recognition (HAR) is prone to performance degradation in real-world applications due to data missing between intra-sensor and intersensor channels.”
“In real-world applications, factors such as sensor malfunctions and signal loss can easily lead to the loss of collected human activity data.”
The paper relies on this without support.
Different human activities span across various spatial-temporal scales, causing activity recognizers to fail to capture intricate spatial-temporal semantic information.
“Besides, different human activities often span across various spatial-temporal scales, which results in activity recognizer failing to capture intricate spatial-temporal semantic information.”
“Furthermore, the varying spatial-temporal scales of human activities often hinder activity recognizers from capturing fine-grained spatiotemporal semantics.”
The paper relies on this without support.
Human activity is temporally dynamic and spatially complex, and positional relationships between sensors significantly impact HAR performance.
“Human activity is temporally dynamic and spatially complex, especially in multi-sensor environments, where the positional relationships and interactions between different sensors can significantly impact the performance of HAR.”
The paper relies on this without support.
Human activities inherently exhibit multi-scale temporal and spatial characteristics.
“Human activities inherently exhibit multi-scale temporal and spatial characteristics.”
The paper relies on this without support.
Contrastive learning relies on complete sample pairs for training.
“This limitation arises from its reliance on complete sample pairs for training.”
The paper relies on this without support.
Features at different scales capture distinct aspects of human activity.
“Features at different scales can capture distinct aspects of human activity.”
The paper relies on this without support.
HAR models often face challenges when data is missing, either due to sensor failures or noise in the environment.
“Human activity recognition (HAR) models often face challenges when data is missing, either due to sensor failures or noise in the environment.”
The paper relies on this without support.
Randomly removing data points at different scales adequately simulates real-world data-missing scenarios.
“we simulate data-missing scenarios by randomly removing data points at different scales.”
The paper relies on this without support.
The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.
“The reconstruction performance during the self-supervised pre-training phase plays a crucial role in understanding the intricate internal structure of human activity data.”
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 32
- [2]2. Wanlin Cai, Yuxuan Liang, Xianggen Liu, Jianshuai Feng, and Yuankai Wu. Msgnet: Learning multi- scale inter-series correlations for multivariate time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 11141-11149, 2024.
- [4]4. Xin Chao, Zhenjie Hou, and Yujian Mo. Czu-mhad: a multimodal dataset for human action recog- nition utilizing a depth camera and 10 wearable inertial sensors. IEEE Sensors Journal, 22(7):7034-7042, 2022.
- [6]6. Ling Chen, Yi Zhang, Shenghuan Miao, Sirou Zhu, Rong Hu, Liangying Peng, and Mingqi Lv. Salience: An unsupervised user adaptation model for mul- tiple wearable sensors based human activity recognition. IEEE Transactions on Mobile Computing, 22(9):5492- 5503, 2023.
- [8]8. Dongzhou Cheng, Lei Zhang, Lutong Qin, Shuoyuan Wang, Hao Wu, and Aiguo Song. Maskcae: Masked convolutional autoencoder via sen- sor data reconstruction for self-supervised human activity recognition. IEEE Journal of Biomedical and Health In- formatics, 28(5), 2024.
- [10]10. Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre-training framework for masked time-series modeling. In Proceedings of Advances in Neu- ral Information Processing Systems, volume 36, 2024.
- [11]11. Gao et al., 2024] Haotian Gao, Renhe Jiang, Zheng Dong, Jinliang Deng, and Xuan Song. Spatio-temporal- decoupled masked pre-training for traffic forecasting. In Proceedings of International Joint Conference on Artifi- cial Intelligence, 2024.
- [12]12. Guan et al., 2025] Renxiang Guan, Wenxuan Tu, Siwei Wang, Jiyuan Liu, Dayu Hu, Chang Tang, Yu Feng, Jun- hong Li, Baili Xiao, and Xinwang Liu. Structure-adaptive multi-view graph clustering for remote sensing data. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 16933-16941, 2025.
- [14]14. Harish Haresamudram, Irfan Essa, and Thomas Plötz. Contrastive predictive coding for human activity recognition. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technolo- gies, pages 1-26. ACM New York, NY, USA, 2021.
- [16]16. Harish Haresamudram, Irfan Essa, and Thomas Plötz. Assessing the state of self- supervised human activity recognition using wearables. In Proceedings of the ACM on Interactive, Mobile, Wear- able and Ubiquitous Technologies, pages 1-47. ACM New York, NY, USA, 2022.
- [18]18. Zhiqing Hong, Zelong Li, Shuxin Zhong, Wenjun Lyu, Haotian Wang, Yi Ding, Tian He, and Desheng Zhang. Crosshar: Generalizing cross-dataset hu- man activity recognition via hierarchical self-supervised pretraining. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, pages 1- 26. ACM New York, NY, USA, 2024.
- [19]19. Hou et al., 2019] Ruibing Hou, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Cross attention network for few-shot classification. In Proceedings of Advances in Neural Information Processing Systems, volume 32, 2019.
- [21]21. Yash Jain, Chi Ian Tang, Chulhong Min, Fahim Kawsar, and Akhil Mathur. Collossl: Collabora- tive self-supervised learning for human activity recogni- tion. In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, pages 1-28. ACM New York, NY, USA, 2022.
- [23]23. Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In Proceedings of International Conference on Learning Representations, pages 1-12, 2020.
- [24]24. Lalwani and Ramasamy, 2024] Pooja Lalwani and Gane- shan Ramasamy. Human activity recognition using a multi-branched cnn-bilstm-bigru model. Applied Soft Computing, 154(3):111344, 2024.
- [26]26. Shuangjian Li, Tao Zhu, Furong Duan, Lim- ing Chen, Huansheng Ning, Christopher Nugent, and Yap- ing Wan. Harmamba: efficient and lightweight wearable sensor human activity recognition based on bidirectional mamba. IEEE Internet of Things Journal, 12(3):2373- 2384, 2024.
- [27]27. Liu et al., 2024a] Jinfu Liu, Chen Chen, and Mengyuan Liu. Multi-modality co-learning for efficient skeleton-based ac- tion recognition. In Proceedings of the 32nd ACM Interna- tional Conference on Multimedia, pages 4909-4918, 2024.
- [28]28. Liu et al., 2024b] Xiao Liu, Guan Yuan, Rui Bing, Zhuo Cai, Shengshen Fu, and Yonghao Yu. When skeleton meets motion: adaptive multimodal graph representation fusion for action recognition. In Proceedings of 2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2024.
- [29]29. Liu et al., 2024c] Yang Liu, Fang Liu, Licheng Jiao, Qianyue Bao, Lingling Li, Yuwei Guo, and Puhua Chen. A knowledge-based hierarchical causal inference network for video action recognition. IEEE Transactions on Multi- media, 26(4):9135-9149, 2024.
- [31]31. Yang Liu, Lei Si, Zhongbin Wang, Dong Wei, Xin Li, and Jinheng Gu. Dual discriminator and adaptive multisource feature fusion wgan-gp for coal-rock properties recognition under limited infrared thermal im- ages. IEEE Transactions on Industrial Informatics, pages 1-12, 2025.
- [32]32. Miao et al., 2024] Shenghuan Miao, Ling Chen, and Rong Hu. Spatial-temporal masked autoencoder for multi- device wearable human activity recognition. In Proceed- ings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, pages 1-25. ACM New York, NY, USA, 2024.
- [33]33. Pandey and Kumar, 2024] Ajeet Pandey and Piyush Kumar. Residual deep gated recurrent unit-based attention frame- work for human activity recognition by exploiting dilated features. The Visual Computer, 40(2):1-20, 2024.
- [35]35. Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, et al. Collecting complex activity datasets in highly rich networked sensor environments. In Proceed- ings of International Conference on Networked Sensing Systems, pages 233-240. IEEE, 2010.
- [36]36. Sztyler and Stuckenschmidt, 2016] Timo Sztyler and Heiner Stuckenschmidt. On-body localization of wearable devices: An investigation of position-aware activity recog- nition. In Proceedings of IEEE International Conference on Pervasive Computing and Communications, pages 1-9. IEEE, 2016.
- [37]37. Wei et al., 2024] Yuecen Wei, Haonan Yuan, Xingcheng Fu, Qingyun Sun, Hao Peng, Xianxian Li, and Chunming Hu. Poincaré differential privacy for hierarchy-aware graph embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9160-9168, 2024.
- [38]38. Wei et al., 2025] Yuecen Wei, Xingcheng Fu, Lingyun Liu, Qingyun Sun, Hao Peng, and Chunming Hu. Prompt- based unifying inference attack on graph neural networks. In Proceedings of the AAAI Conference on Artificial Intel- ligence, pages 12836-12844, 2025.
- [40]40. Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In Proceedings of International Conference on Learning Representations, 2023.
- [42]42. Zongqian Wu, Yujing Liu, Mengmeng Zhan, Ping Hu, and Xiaofeng Zhu. Adaptive multi- modality prompt learning. In Proceedings of the ACM In- ternational Conference on Multimedia, pages 8672-8680, 2024.
- [43]43. Zhang et al., 2023] Guixian Zhang, Debo Cheng, and Shichao Zhang. Fpgnn: Fair path graph neural network for mitigating discrimination. World Wide Web, 26(5):3119- 3136, 2023.
- [44]44. Zhang et al., 2025a] Guixian Zhang, Guan Yuan, Debo Cheng, Lin Liu, Jiuyong Li, and Shichao Zhang. Disen- tangled contrastive learning for fair graph representations. Neural Networks, 181(1):106781, 2025.
- [45]45. Zhang et al., 2025b] Xuebin Zhang, Qicheng Xu, Fuyuan Feng, Xiaochen Lu, and Longting Xu. Fall-mamba: A multimodal fusion and masked mamba-based approach for fall detection. IEEE Internet of Things Journal, 12(8):10493-10505, 2025.
- [46]46. Zhou et al., 2021] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on Artificial Intelligence, pages 11106- 11115, 2021.
- [48]48. Yuxuan Zhou, Xudong Yan, Zhi-Qi Cheng, Yan Yan, Qi Dai, and Xian-Sheng Hua. Block- gcn: Redefine topology awareness for skeleton-based ac- tion recognition. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 2049-2058, 2024.
Cited by
Sorted by citations
Joint-level voting and action clustering for enhanced skeleton-based human action recognition
Bouguerra, A., Hemam, S. M., & Hioual, O. (2026) · The Visual Computer · doi:10.1007/s00371-025-04350-5
Zou, R., Yang, Z., Q, J., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-1465-5_35
- Authors
- Liu, X., Yuan, G., Zhang, Y., Liu, S., & Yan, Q.
- Affiliations
- China University of Mining and Technology (CN)
- Venue
- Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence
- Published
- 2025-09
- Pages
- 8563-8571
- OpenAlex
- W4414359565
- Semantic Scholar
- 14a183703b39f5ef23dd956a49cc150ff91e5307
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 2
- FWCI
- 1.51
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- Funders
- Government of Jiangsu Province
- China University of Mining and Technology
- Fundamental Research Funds for the Central Universities (2023XSCX048)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, multi-sensor data, missing data, multi-scale feature learning, spatio-temporal modeling, multi-scale feature fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
