Across-Sensor Feature Learning for Energy-Efficient Activity Recognition on Mobile Devices
Abstract
In this paper we propose across-sensor representation learning framework for improving power-accuracy trade-off in multi-sensor human activity recognition (HAR). The goal of the study is to achieve the level of performance comparable to one of multi-sensor HAR systems by using fewer or even single sensor. Such performance is achieved by learning relations between these sensors at training time and utilizing them at test time. These relations are learned by supervised deep models which use multi-sensor data during training only. The absence of need for having multiple sensors during test time allows turning these sensors off and replacing them with a single sensor coupled with learned across-sensor relations. These across-sensor relations make up for the information lost from the turned-off sensors. Using fewer sensors reduces energy consumption of HAR systems deployed on a smartphone. Moreover, it allows building HAR systems for situations when collection of multi-sensor data is possible only during training. This work presents preliminary results achieved with the proposed approach on the SHL dataset. Obtained results show an improvement of up to 14% in classification accuracy of single-sensor HAR.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Autoencoders are most commonly used for feature extraction and dimensionality reduction.
“As such, autoencoders are most commonly used for feature extraction and dimensionality reduction [120,122,123,124,125,126,133,134,135,136,137,138,139,140,141].”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [124]
Cites
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · see citing works
A review of wearable sensors and systems with application in rehabilitation
Patel, S., Park, H., Bonato, P., Chan, L., & Rodgers, M. (2012) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-9-21 · Cited by 2,396 · see citing works
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · cited 1× · see citing works
“In the final step, building a predictive model, a variety of methods were used, including CNN, RNN, DNN, RBM, Decision tree, SVM, Naive Bayes, KNN, etc. Review of employed methods was described in works of Wang et al. [28] and Lara et al. [29].”
MethodA tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668 · see citing works
Tutorial on Variational Autoencoders
Doersch, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05908 · Cited by 1,470 · see citing works
Wearable Sensors for Human Activity Monitoring: A Review
Mukhopadhyay, S. C. (2015) · IEEE Sensors Journal · doi:10.1109/jsen.2014.2370945 · Cited by 1,401 · see citing works
Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Rifai, S., Vincent, P., Muller, X., Glorot, X., & Bengio, Y. (2011) · Cited by 1,189 · see citing works
Anguita, D., Ghio, A., Oneto, L., Parra, X., & Reyes-Ortiz, J. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35395-6_30 · Cited by 998 · see citing works
Window Size Impact in Human Activity Recognition
Baños, O., Gálvez, J. M., Damas, M., Pomares, H., & Rojas, I. (2014) · Sensors · doi:10.3390/s140406474 · Cited by 656 · see citing works
Preprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578 · see citing works
Khan, A., Lee, Y.-K., Lee, S. Y., & Kim, T. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2010.2051955 · Cited by 518 · see citing works
Self-citationKeeping the Resident in the Loop: Adapting the Smart Home to the User
Rashidi, P., & Cook, D. J. (2009) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2009.2025137 · Cited by 443 · see citing works
Avci, A., Bosch, S., Marin-Perianu, M., Marin-Perianu, R., & Havinga, P. (2010) · University of Twente Research Information · Cited by 443 · see citing works
A framework of energy efficient mobile sensing for automatic user state recognition
Wang, Y., Lin, J., Annavaram, M., Jacobson, Q. A., Hong, J., Krishnamachari, B., & Sadeh, N. (2009) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/1555816.1555835 · Cited by 436 · see citing works
Activity recognition with smartphone sensors
Su, X. Y., Tong, H., & Ji, P. (2014) · Tsinghua Science & Technology · doi:10.1109/tst.2014.6838194 · Cited by 383 · see citing works
A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer
Chen, Y., & Xue, Y. (2015) · 2015 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.263 · Cited by 378 · see citing works
Wearable Activity Tracking in Car Manufacturing
Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., & Tr, G. (2008) · IEEE Pervasive Computing · doi:10.1109/mprv.2008.40 · Cited by 367 · see citing works
Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection
Zappi, P., Lombriser, C., Stiefmeier, T., Farella, E., Roggen, D., Benini, L., & Tröster, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-77690-1_2 · Cited by 329 · see citing works
A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone
Wang, A., Chen, G., Yang, J., Zhao, S., & Chang, C.-Y. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2545708 · Cited by 324 · see citing works
Increasing trend of wearables and multimodal interface for human activity monitoring: A review
Kumari, P., Mathew, L., & Syal, P. (2016) · Biosensors and Bioelectronics · doi:10.1016/j.bios.2016.12.001 · Cited by 298 · see citing works
Energy-accuracy trade-off for continuous mobile device location
Lin, K., Kansal, A., Lymberopoulos, D., & Zhao, F. (2010) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/1814433.1814462 · Cited by 277 · see citing works
Gjoreski, H., Ciliberto, M., Wang, L., Morales, F. J. O., Mekki, S., Valentin, S., & Roggen, D. (2018) · IEEE Access · doi:10.1109/access.2018.2858933 · Cited by 277 · see citing works
Energy-Efficient Continuous Activity Recognition on Mobile Phones: An Activity-Adaptive Approach
Yan, Z., Subbaraju, V., Chakraborty, D., Misra, A., & Aberer, K. (2012) · International Symposium on Wearable Computers · doi:10.1109/iswc.2012.23 · Cited by 254 · see citing works
Khan, A., Lee, Y.-K., Lee, S. Y., & Kim, T.-S. (2010) · International Conference on Future Information Technology · doi:10.1109/futuretech.2010.5482729 · Cited by 245 · see citing works
Self-citationThe mobile fitness coach: Towards individualized skill assessment using personalized mobile devices
Kranz, M., Möller, A., Hammerla, N., Diewald, S., Plötz, T., Olivier, P. L., & Roalter, L. (2012) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2012.06.002 · Cited by 197 · see citing works
Activity Recognition on Smartphones via Sensor-Fusion and KDA-Based SVMs
Khan, A., Tufail, A., Khattak, A. M., & Laine, T. H. (2014) · International Journal of Distributed Sensor Networks · doi:10.1155/2014/503291 · Cited by 149 · see citing works
Self-citationTrading off Prediction Accuracy and Power Consumption for Context-Aware Wearable Computing
Krause, A., Ihmig, M., Rankin, E., Leong, D., Gupta, S., Siewiorek, D. P., Smailagic, A., Deisher, M. E., & Sengupta, U. (2005) · Proceedings · doi:10.1109/iswc.2005.52 · Cited by 144 · see citing works
Human activity recognition in pervasive health-care: Supporting efficient remote collaboration
Osmani, V., Balasubramaniam, S., & Botvich, D. (2007) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2007.11.002 · Cited by 130 · see citing works
Khan, A., Siddiqi, M., & Lee, S. W. (2013) · Sensors · doi:10.3390/s131013099 · Cited by 114 · see citing works
Self-citationEnergy-Efficient Motion Related Activity Recognition on Mobile Devices for Pervasive Healthcare
Liang, Y., Zhou, X., Yu, Z., & Guo, B. (2013) · Mobile Networks and Applications · doi:10.1007/s11036-013-0448-9 · Cited by 99 · see citing works
Lee, M.-W., Khan, A., & Kim, T. (2011) · Personal and Ubiquitous Computing · doi:10.1007/s00779-011-0403-3 · Cited by 94 · see citing works
Self-citationAn effective deep autoencoder approach for online smartphone-based human activity recognition
Almaslukh, B., Artoli, A. M., & Al-Muhtadi, J. F. (2017) · Cited by 94 · see citing works
Khan, A., Lee, Y.-K., Lee, S., & Kim, T. (2010) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-010-0701-3 · Cited by 91 · see citing works
Self-citationKhan, A., Lee, Y. K., & Kim, T.-S. (2008) · doi:10.1109/iembs.2008.4650379 · Cited by 90 · see citing works
Self-citationUnsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors
Li, Y., Shi, D., Ding, B., & Liu, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13817-6_11 · Cited by 89 · see citing works
Mazilu, S., Blanke, U., Hardegger, M., Tröster, G., Gazit, E., & Hausdorff, J. M. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557278 · Cited by 84 · see citing works
A Deep Learning Framework Using Passive WiFi Sensing for Respiration Monitoring
Khan, U. M., Kabir, Z., Hassan, S. A., & Ahmed, S. H. (2017) · Globecom · doi:10.1109/glocom.2017.8255027 · Cited by 56 · see citing works
User-Independent Activity Recognition via Three-Stage GA-Based Feature Selection
Saputri, T. R. D., Khan, A., & Lee, S.-W. (2014) · International Journal of Distributed Sensor Networks · doi:10.1155/2014/706287 · Cited by 36 · see citing works
Self-citationA Novel Energy-Efficient Approach for Human Activity Recognition
Zheng, L., Wu, D., Ruan, X., Weng, S., Peng, A., Tang, B., Lü, H., Shi, H., & Zheng, H. (2017) · Sensors · doi:10.3390/s17092064 · Cited by 34 · see citing works
Lee, M.-W., Khan, A., Kim, J., Cho, Y.-S., & Kim, T. (2010) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2010.5626729 · Cited by 32 · see citing works
Self-citationGait fingerprinting-based user identification on smartphones
Ahmad, M. I., Khan, A., Brown, J. A., Protasov, S., & Khattak, A. M. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727588 · Cited by 29 · see citing works
Self-citationLearning to Infer the Status of Heavy-Duty Sensors for Energy-Efficient Context-Sensing
Li, X., Cao, H., Chen, E., & Tian, J. (2012) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2089094.2089111 · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“As such, autoencoders are most commonly used for feature extraction and dimensionality reduction [18, 27, 53, 73, 74, 80, 103, 137, 171, 172, 194, 234, 283, 287, 296, 298].”
BackgroundMethodOn the Fairness of Generative Adversarial Networks (GANs)
Kenfack, P. J., Arapovy, D. D., Hussain, R., Kazmi, S. M. A., & Khan, A. (2021) · International Conference Nonlinearity, Information and Robotics (NIR) · doi:10.1109/nir52917.2021.9666131 · Cited by 25 · see citing works
Self-citationAn Optimal Feature Selection Method for Human Activity Recognition Using Multimodal Sensory Data
Haider, T., Khan, M. H., & Farid, M. S. (2024) · Information · doi:10.3390/info15100593 · Cited by 9 · see citing works
Domain Generalization Using Ensemble Learning
Mesbah, Y., Ibrahim, Y. Y., & Khan, A. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-82193-7_15 · Cited by 8 · see citing works
Self-citationDeep Learning Models in Software Requirements EngineeringPreprint
Naumcheva, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.07771 · Cited by 6 · see citing works
Analyzing the Effectiveness of Image Augmentations for Face Recognition from Limited Data
Zhuchkov, A. (2021) · International Conference Nonlinearity, Information and Robotics (NIR) · doi:10.1109/nir52917.2021.9666135 · Cited by 5 · see citing works
Class-incremental Learning using a Sequence of Partial Implicitly Regularized Classifiers
Bobiev, S., Khusainova, A. M., Khan, A., & Kazmi, S. M. A. (2022) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.v35i.130549
Self-citation
- Authors
- Gavrilin, Y., & Khan, A.
- Affiliations
- Innopolis University (RU)
- Venue
- 2019 International Joint Conference on Neural Networks (IJCNN)
- Publisher
- IEEE
- Published
- 2019-07
- Volume
- 19
- Pages
- 1-7
- ISSN
- 2161-4407 (ISSN-L)
- OpenAlex
- W2979239438
- Cited by
- 8
- References
- 42
- Citations by year
- 2024: 1 · 2022: 3 · 2021: 4
- FWCI
- 0.57
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- International Joint Conference on Neural Networks (IJCNN) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, multi-sensor fusion, representation learning, energy efficiency, smartphone sensors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
