SADeepSense: Self-Attention Deep Learning Framework for Heterogeneous On-Device Sensors in Internet of Things Applications
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Deep neural networks are becoming increasingly popular in Internet of Things (IoT) applications. Their capabilities of fusing multiple sensor inputs and extracting temporal relationships can enhance intelligence in a wide range of applications. However, one key problem is the missing of adaptation to heterogeneous on-device sensors. These low-end sensors on IoT devices possess different accuracies, granularities, and amounts of information, whose sensing qualities are heterogeneous and vary over time. The existing deep learning frameworks for IoT applications usually treat every sensor input equally over time or increase model capacity in an ad-hoc manner, lacking the ability to identify and exploit the sensor heterogeneities. In this work, we propose SADeepSense, a deep learning framework that can automatically balance the contributions of multiple sensor inputs over time by exploiting their sensing qualities. SADeepSense makes two key contributions. First, SADeepSense employs the self-attention mechanism to learn the correlations among different sensors over time with no additional supervision. The correlations are then applied to infer the sensing qualities and to reassign model concentrations in multiple sensors over time. Second, instead of directly learning the sensing qualities and contributions, SADeepSense generates the residual concentrations that are deviated from the equal contributions, which helps to stabilize the training process. We demonstrate the effectiveness of SADeepSense with two representative IoT sensing tasks: heterogeneous human activity recognition with motion sensors and gesture recognition with the wireless signal. SADeepSense consistently outperforms the state-of-the-art methods by a clear margin. In addition, we show that SADeepSense only imposes little additional resource-consumption burden on embedded devices compared to the corresponding state-of-the-art framework.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
A Survey of Context-Aware Mobile Computing Research
Chen, G., & Kotz, D. F. (2000) · Cited by 1,749
Stisen, A., Blunck, H., Bhattacharya, S., Prentow, T. S., Kjærgaard, M. B., Dey, A., Sonne, T., & Jensen, M. M. (2015) · Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems · doi:10.1145/2809695.2809718 · Cited by 714 · cited 5×
“The heterogeneity of on-device sensor qualities is one of these key issues, waiting to be resolved [15].”
BackgroundMethodInfluentialDeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · cited 6×
“Great efforts have been made on designing unified structures for fusing multiple sensing inputs and extracting temporal relationships [19], [11], compressing neural network structures for reducing resource consumptions on low-end devices [24], [20], [4], providing well-calibrated uncertainty estimations for neural net…”
BackgroundMethodSelf-citationPreprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · cited 1×
“It selects all popular time-domain and frequency domain features from [7] and ECDF features from [9].”
Background
Research works printed but not linked: 2
- 0, Weka for Android
- 0, Mobile TensorFlow
Cited by
Human activity recognition in artificial intelligence framework: a narrative review
Gupta, N., Gupta, S. K., Pathak, R. K., Jain, V., Rashidi, P., & Suri, J. S. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10116-x · Cited by 337
A survey on deploying mobile deep learning applications: A systemic and technical perspective
Wang, Y., Wang, J., Zhang, W., Zhan, Y., Guo, S., Zheng, Q., & Wang, X. (2022) · Digital Communications and Networks · doi:10.1016/j.dcan.2021.06.001 · Cited by 110
Radar-Based Human Activity Recognition Using Hybrid Neural Network Model With Multidomain Fusion
Ding, W., Guo, X., & Wang, G. (2021) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2021.3068436 · Cited by 99
Attention-Based Deep Learning Framework for Human Activity Recognition With User Adaptation
Buffelli, D., & Vandin, F. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3067690 · Cited by 68
Deep learning based multimodal complex human activity recognition using wearable devices
Chen, L., Liu, X., Peng, L., & Wu, M. (2020) · Applied Intelligence · doi:10.1007/s10489-020-02005-7 · Cited by 72
DeepMV: Multi-View Deep Learning for Device-Free Human Activity Recognition
Xue, H., Jiang, W., Miao, C., Ma, F., Wang, S., Yuan, Y., Yao, S., Zhang, A., & Su, L. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3380980 · Cited by 79
Self-citation(2020) · doi:10.1145/3380999 · Cited by 57
“We compare GlobalFusion to both the state-of-the-art non-attentional DeepSense [40] framework, and attentional frameworks, including SADeepSense [42], BANet [34], and attnLSTM [47].”
BackgroundMethodInfluentialSelf-citationIF-ConvTransformer: A Framework for Human Activity Recognition Using IMU Fusion and ConvTransformer
Zhang, Y., Wang, L., Chen, H., Tian, A., Zhou, S., & Guo, Y. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3534584 · Cited by 46
Agac, S., & İncel, Ö. D. (2023) · Diagnostics · doi:10.3390/diagnostics13111861 · Cited by 60
Robust Inertial Motion Tracking through Deep Sensor Fusion across Smart Earbuds and Smartphone
Gong, J., Zhang, X., Huang, Y., Ren, J., & Zhang, Y. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463517 · Cited by 39
Miao, S., Chen, L., Hu, R., & Luo, Y. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3550331 · Cited by 36
Kang, L., Gao, C., Li, B., & Liu, W. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.111146 · Cited by 42
Oleh, U., Obermaisser, R., & Ahammed, A. S. (2024) · Algorithms · doi:10.3390/a17100434 · Cited by 40
Centaur: Robust Multimodal Fusion for Human Activity Recognition
Xaviar, S., Yang, X., & Ardakanian, O. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3388893 · Cited by 44
Theodorou, P., Tsiligkos, K., & Μηλιώνης, Α. (2023) · Sensors · doi:10.3390/s23125411 · Cited by 32
ActSonic: Recognizing Everyday Activities from Inaudible Acoustic Wave Around the Body
Mahmud, S., Parikh, V., Liang, Q., Li, K., Zhang, R., Ajit, A., Gunda, V., Agarwal, D., Guimbretière, F. V., & Zhang, C. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3699752 · Cited by 43
Spatial-Temporal Masked Autoencoder for Multi-Device Wearable Human Activity Recognition
Miao, S., Chen, L., & Hu, R. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3631415 · Cited by 23
Self-Attention-Based Deep Convolution LSTM Framework for Sensor-Based Badminton Activity Recognition
Deng, J., Zhang, S., & Ma, J. (2023) · Sensors · doi:10.3390/s23208373 · Cited by 27
DeepMTD: Moving Target Defense for Deep Visual Sensing against Adversarial Examples
Song, Q., Yan, Z., & Tan, R. (2021) · ACM Transactions on Sensor Networks · doi:10.1145/3469032 · Cited by 21
Park, J., Lee, K., Lee, S., Zhang, M., & Ko, J. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610917 · Cited by 30
Handling Missing Sensors in Topology-Aware IoT Applications with Gated Graph Neural Network
Liu, S., Yao, S., Huang, Y., Liu, D., Shao, H., Zhao, Y., Li, J., Wang, T., Wang, R., Yang, C., et al. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411818 · Cited by 20
“Sensor fusion across all body positions is a mean pooling layer, instead of complex attention mechanisms [36, 61].”
BackgroundSelf-citationmilliEgo: Single-chip mmWave Radar Aided Egomotion Estimation via Deep Sensor FusionPreprint
Lu, C. X., Saputra, M. R. U., Zhao, P., Almalıoğlu, Y., de Gusmão, P. P. B., Chen, C., Sun, K., Trigoni, N., & Markham, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.02266 · Cited by 20
Guo, K., Zhou, H., Tian, Y., Zhou, W., Ji, Y., & Li, X. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796879 · Cited by 18
Warehouse LSTM-SVM-Based ECG Data Classification With Mitigated Device Heterogeneity
Khan, P., Ranjan, P., Singh, Y., & Kumar, S. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3116428 · Cited by 17
Agac, S., & İncel, Ö. D. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109274 · Cited by 19
Kaelin, V., Nilsson, I., & Lindgren, H. (2024) · Scandinavian Journal of Occupational Therapy · doi:10.1080/11038128.2024.2421355 · Cited by 26
EUMD: Efficient slot utilization based missing tag detection with unknown tags
Lin, K., Chen, H., Ai, X., Владимирович, Ш. В., Ni, L., Yu, J., & Li, Y. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102640 · Cited by 15
IoBT-OS: Optimizing the Sensing-to-Decision Loop for the Internet of Battlefield Things
Liu, D., Abdelzaher, T., Wang, T., Hu, Y. C., Li, J., Liu, S., Caesar, M., Kalasapura, D., Bhattacharyya, J., Srour, N., et al. (2022) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn54977.2022.9868920 · Cited by 17
Self-citationMasked Video and Body-Worn IMU Autoencoder for Egocentric Action Recognition
Zhang, M., Huang, Y., Liu, R., & Sato, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_18 · Cited by 31
Wang, X., Wang, H., Wu, F., Liu, T., Cao, Q., & Su, L. (2024) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3666025.3699360 · Cited by 24
Huang, Q., Xie, W., Li, C., Wang, Y., & Liu, Y. (2023) · Applied Sciences · doi:10.3390/app131910560 · Cited by 9
Ahmad, N., & Leung, H.-F. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643511 · Cited by 13
Measuring intrinsic human activity information using WiFi-based attention model
Zhou, Q., Xing, J., Yang, Q., Chen, Y., & Feng, B. (2022) · Measurement · doi:10.1016/j.measurement.2022.111084 · Cited by 8
A Diverse Noise-Resilient DNN Ensemble Model on Edge Devices for Time-Series Data
Shubha, S. S., Sen, T., Shen, H., & Normansell, M. (2021) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/secon52354.2021.9491607 · Cited by 9
IOTeeth: Intra-Oral Teeth Sensing System for Dental Occlusal Diseases Recognition
Hu, Z., Radmehr, A., Zhang, Y., Pan, S., & Nguyen, P. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3643516 · Cited by 12
Enhancing link prediction in dynamic social networks through hybrid GCN-LSTM models
Choudhary, S., & Kumar, G. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02430-5 · Cited by 9
Feng, Q., Duan, C., Xue, J., Li, C., Huang, F., Zhang, X., Weng, J., & Yu, P. S. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3592965 · Cited by 7
Self-Supervised Learning from Unlabeled IoT Data
Liu, D., & Abdelzaher, T. (2023) · Springer eBooks · doi:10.1007/978-3-031-40787-1_2 · Cited by 5
Self-citationHierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition
Tonmoy, M. T. H., Mahmud, S., Rahman, A. K. M. M., Amin, M. A., & Ali, A. A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-75768-7_28 · Cited by 22
Human activity recognition using CNN and LSTM for inertial sensors activity data
Gupta, N., Gupta, S. K., & Jain, V. (2024) · AIP conference proceedings · doi:10.1063/5.0198752 · Cited by 4
Ahmad, N., Leung, H.-F., & Farnia, F. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3555799 · Cited by 4
PianoWatch: An Intelligent Piano Understanding and Evaluation System Using Smartwatch
Guo, K., Zhou, H., Jing, S., Du, H., Luo, P., Hou, J., & Li, X. (2023) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/secon58729.2023.10287516 · Cited by 2
SenseMLP: a parallel MLP architecture for sensor-based human activity recognition
Li, W., Guo, J., & Wu, H. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01384-y · Cited by 2
Human Action Recognition in Multi-Level Convolutional Temporal Attention Network
Huang, Q., Chen, Z., Li, C., & Qian, W. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889140 · Cited by 1
NoiseSenseDNN: Modeling DNN for Sensor Data to Mitigate the Effect of Noise in Edge Devices
Sen, T., Shen, H., & Normansell, M. (2020) · IEEE International Conference on Mobile Ad-Hoc and Sensor Systems (MASS) · doi:10.1109/mass50613.2020.00053 · Cited by 2
Ensembled HAR Approach Using Stacking on Inertial Sensors Data
Gupta, N., Gupta, S. K., & Jain, V. (2023) · International Conference on Electrical, Electronics, Communication and Computers (ELEXCOM) · doi:10.1109/elexcom58812.2023.10370376 · Cited by 1
Deep ensembled voting framework for human activity recognition and validation on video sequences
Gupta, N., Gupta, S. K., Jain, V., Singh, N., & Suri, J. S. (2025) · Evolving Systems · doi:10.1007/s12530-025-09695-9 · Cited by 2
Modeling expert risk assessments in utility tunnels with deep learning
Xue, G., Gong, D., Ren, L., & Cui, Z. (2025) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2025.111523 · Cited by 6
Xie, W., Huang, Q., Li, C., Wang, Y., & Liu, Y. (2023) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3595916.3626425
RFA: Regularized Feature Alignment Method for Cross-Subject Human Activity Recognition
Yang, Z., Fu, H., Huan, R., Qu, M., & Pan, Y. (2025) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001425510085
Zhao, Z., He, C., Jiang, X., & Xu, X. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110583 · Cited by 3
Metadata
- Authors
- Yao, S., Zhao, Y., Shao, H., Liu, D., Liu, S., Hao, Y., Piao, A., Hu, S., Lu, S., & Abdelzaher, T. F.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- University of Washington (US)
- IBM (United States) (US)
- Venue
- IEEE INFOCOM 2019 - IEEE Conference on Computer Communications
- Publisher
- IEEE
- Published
- 2019-04
- Pages
- 1243-1251
- OpenAlex
- W2918488306
- Cited by
- 52
- References
- 23
- Citations by year
- 2025: 8 · 2024: 11 · 2023: 11 · 2022: 6 · 2021: 9 · 2020: 7
- FWCI
- 1.79
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Conference on Computer Communications (INFOCOM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
- Keywords
- heterogeneous sensors, self-attention, sensor fusion, residual concentration, human activity recognition, gesture recognition, wireless communication signals, embedded devices, Internet of Things
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.