Real-Time Activity Recognition on Smartphones Using Deep Neural Networks
Abstract
Real-time activity recognition is an important research problem, which has drawn the attention of many researchers for many years. All previous works of real-time activity recognition needed to manually extract features and then fed these features into the classifiers as the inputs. However, due to the recognition time limit, they could only extract a small amount of features, which might not achieve good accuracy and could be improved. Besides, these features were manually extracted according to the experience of the researchers. Because researchers did not know which features worked better for classification, it was hard to choose suitable features, thus these manually extracted features might not be related to the specific classification task and have poor discrimination ability and could be improved. In this paper, we recommend deep neural networks for real-time activity recognition, which automatically learn suitable features and then perform classification. We collected accelerometer data of seven activities of interest on an android smartphone, including walking, running, standing, sitting, lying, walking upstairs and walking downstairs, and conducted experiments on our collected dataset to compare our method with traditional methods. Besides, we implemented a deep neural network on a smartphone and tested the recognition time of the model. The results showed that deep neural networks could achieve real-time performance and got higher accuracy than traditional methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In early exploratory work in 2011, a five-layer DBN trained on mobile phone acceleration data improved accuracy by 1% to 32% compared to traditional ML with manually extracted features.
“The accuracy improvement ranges from 1% to 32% when compared to traditional ML methods with manually extracted features.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [156]
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928 · see citing works
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · see citing works
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147 · see citing works
Activity recognition using cell phone accelerometers
Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1964897.1964918 · Cited by 2,912 · see citing works
A Practical Guide to Training Restricted Boltzmann Machines
Hinton, G. E. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_32 · Cited by 2,852 · see citing works
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · Cited by 1,839 · see citing works
Activity recognition from accelerometer data
Ravi, N., Dandekar, N., Mysore, P., & Littman, M. L. (2005) · Cited by 1,434 · see citing works
Anguita, D., Ghio, A., Oneto, L., Parra, X., & Reyes-Ortiz, J. L. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35395-6_30 · Cited by 998 · see citing works
Activity Classification Using Realistic Data From Wearable Sensors
Pärkkä, J., Ermes, M., Korpipää, P., Mäntyjärvi, J., Peltola, J., & Korhonen, I. (2006) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2005.856863 · Cited by 759 · see citing works
The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 1× · see citing works
“Previous datasets either contain a small amount of data for an individual user [9] [10] [11] [12], or consist of extremely imbalanced data of different activities for a single subject [13] [14] , thus could not represent the motion characteristics of a user.”
BackgroundAn Introduction to Restricted Boltzmann Machines
Fischer, A., & Igel, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33275-3_2 · Cited by 605 · see citing works
USC-HAD: a daily activity dataset for ubiquitous activity recognition using wearable sensors
Zhang, M., & Sawchuk, A. A. (2012) · Proceedings of the 2012 ACM Conference on Ubiquitous Computing · doi:10.1145/2370216.2370438 · Cited by 554 · see citing works
Fusion of Smartphone Motion Sensors for Physical Activity Recognition
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2014) · Sensors · doi:10.3390/s140610146 · Cited by 552 · see citing works
WEKA: The Waikato Environment for Knowledge Analysis
Garner, S. (1996) · Cited by 508 · see citing works
Feature Learning for Activity Recognition in Ubiquitous Computing
Plötz, T., Hammerla, N., & Olivier, P. (2011) · Newcastle University · doi:10.25405/ncl.34175319 · Cited by 337 · see citing works
Classification Accuracies of Physical Activities Using Smartphone Motion Sensors
Wu, W., Dasgupta, S., Ramirez, E. E., Peterson, C., & Norman, G. J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2208 · Cited by 316 · see citing works
A Review and Taxonomy of Activity Recognition on Mobile Phones
İncel, Ö. D., Köse, M., & Ersoy, C. (2013) · BioNanoScience · doi:10.1007/s12668-013-0088-3 · Cited by 234 · see citing works
A mobile platform for real-time human activity recognition
Lara, S. D., & Labrador, M. A. (2012) · 2012 IEEE Consumer Communications and Networking Conference (CCNC) · doi:10.1109/ccnc.2012.6181018 · Cited by 128 · see citing works
Online pose classification and walking speed estimation using handheld devices
Park, J., Patel, A., Curtis, D. W., Teller, S. J., & Ledlie, J. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370235 · Cited by 109 · see citing works
Advancing from offline to online activity recognition with wearable sensors
Ermes, M., Pärkkä, J., & Cluitmans, L. (2008) · doi:10.1109/iembs.2008.4650199 · Cited by 96 · see citing works
Ready-to-use activity recognition for smartphones
Siirtola, P., & Röning, J. (2013) · IEEE Symposium on Computational Intelligence and Data Mining (CIDM) · doi:10.1109/cidm.2013.6597218 · Cited by 69 · see citing works
MACRO-CLASS SELECTION FOR HIERARCHICAL K-NN CLASSIFICATION OF INERTIAL SENSOR DATA
McCall, C., Reddy, K., & Shah, M. A. (2012) · International Conference on Pervasive and Embedded Computing and Communication Systems · doi:10.5220/0003819101060114 · Cited by 22 · see citing works
Flagged first, then by citations
3 of the 28 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · see citing works
“For multi-modal signals, it is better to use CNN since (Bhattacharya and Lane, 2016) Body-worn, ambiemt RBM Gesture, ADL, transportation Self, D01 (Chen and Xue, 2015) Body-worn CNN ADL Self (Chen et al., 2016b) Body-worn CNN ADL D06 (Edel and K¨oppe, 2016) Body-worn RNN ADL D01, D04, Self (Fang and Hu, 2014) Object,…”
BackgroundMethodInfluentialNweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · see citing works
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“In early exploratory work back in 2011 [325], a 5-layer DBN is trained with the input acceleration data collected from mobile phones.”
MethodHuman Activity Recognition Using Inertial Sensors in a Smartphone: An Overview
Lima, W. S., Souto, E., El‐Khatib, K., Jalali, R., & da Gama, J. M. P. (2019) · Sensors · doi:10.3390/s19143213 · Cited by 260 · see citing works
Survey on Human Activity Recognition based on Acceleration Data
Slim, S. O., Atia, A., A., M. M., & M.Mostafa, M.-S. (2019) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2019.0100311 · Cited by 68 · see citing works
Human activity recognition with smartphone-integrated sensors: A survey
Dentamaro, V., Gattulli, V., Impedovo, D., & Manca, F. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123143 · Cited by 86 · see citing works
Few-shot learning-based human activity recognition
Feng, S., & Duarte, M. F. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.06.070 · Cited by 83 · see citing works
Iss2Image: A Novel Signal-Encoding Technique for CNN-Based Human Activity Recognition
Hur, T., Bang, J., Huynh‐The, T., Lee, J., Kim, J., & Lee, S. (2018) · Sensors · doi:10.3390/s18113910 · Cited by 74 · see citing works
Pires, I. M., Marques, G., García, N. M., Flórez‐Revuelta, F., Teixeira, M. C., Zdravevski, E., Spinsante, S., & Coimbra, M. T. (2020) · Electronics · doi:10.3390/electronics9030509 · Cited by 44 · see citing works
Activity, Plan, and Goal Recognition: A Review
Van-Horenbeke, F. A., & Peer, A. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.643010 · Cited by 42 · see citing works
Pires, I. M., García, N. M., Pombo, N. G. C. C., Flórez‐Revuelta, F., & Spinsante, S. (2018) · Sensors · doi:10.3390/s18020640 · Cited by 32 · see citing works
Leveraging Wearable Sensors for Human Daily Activity Recognition with Stacked Denoising Autoencoders
Ni, Q., Fan, Z., Zhang, L., Nugent, C., Cleland, I., Zhang, Y., & Zhou, N. Q. (2020) · Sensors · doi:10.3390/s20185114 · Cited by 32 · see citing works
Self-citationBurns, D., Razmjou, H., Shaw, J., Richards, R. R., McLachlin, S. D., Hardisty, M., Henry, P. D. G., & Whyne, C. M. (2020) · JMIR Research Protocols · doi:10.2196/17841 · Cited by 29 · see citing works
Da-Zhong, M., Wang, J., Sun, Q., & Hu, X. (2019) · IEEE Access · doi:10.1109/access.2019.2908015 · Cited by 27 · see citing works
Towards an integrated framework for air quality monitoring and exposure estimation—a review
Singla, S., Bansal, D., Misra, A., & Raheja, G. (2018) · Environmental Monitoring and Assessment · doi:10.1007/s10661-018-6940-8 · Cited by 25 · see citing works
Pires, I. M., Teixeira, M. C., Pombo, N. G. C. C., García, N. M., Flórez‐Revuelta, F., Spinsante, S., Goleva, R., & Zdravevski, E. (2018) · The Open Bioinformatics Journal · doi:10.2174/1875036201811010061 · Cited by 23 · see citing works
Human Activity Recognition Based on Acceleration Data From Smartphones Using HMMs
Iloga, S., Bordat, A., Kernec, J. L., & Romain, O. (2021) · IEEE Access · doi:10.1109/access.2021.3117336 · Cited by 24 · see citing works
Ni, Q., Fan, Z., Zhang, L., Zhang, B., Zheng, X., & Zhang, Y. (2022) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-021-00052-7 · Cited by 23 · see citing works
Self-citationAlMuhaideb, S., AlAbdulkarim, L., AlShahrani, D. M., AlDhubaib, H., & AlSadoun, D. E. (2024) · Sensors · doi:10.3390/s24165436 · Cited by 24 · see citing works
IoT System Based on parameter optimization of Deep Learning using Genetic Algorithm
Slim, S. O., Elfattah, M. M. A., Atia, A., & Mostafa, M.-S. (2021) · International journal of intelligent engineering and systems · doi:10.22266/ijies2021.0430.20 · Cited by 15 · see citing works
Lakoju, M., Ajienka, N., Khanesar, M. A., Burnap, P., & Branson, D. T. (2021) · Sensors · doi:10.3390/s21154991 · Cited by 14 · see citing works
Human Activity Recognition based on Acceleration sensor and Neural Network
Wenzheng, Z. (2020) · International Conference on Orange Technology (ICOT) · doi:10.1109/icot51877.2020.9468785 · Cited by 11 · see citing works
Efficient Frequency Domain Feature Extraction Model using EPS and LDA for Human Activity Recognition
Bhuiyan, R. A., Amiruzzaman, M., Ahmed, N., & Islam, M. R. (2020) · IEEE International Conference on Knowledge Innovation and Invention (ICKII) · doi:10.1109/ickii50300.2020.9318786 · Cited by 9 · see citing works
The Effect of Axis-Wise Triaxial Acceleration Data Fusion in CNN-Based Human Activity Recognition
Han, X., Ye, J., Luo, J., & Zhou, H. (2020) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2018edp7409 · Cited by 8 · see citing works
Deep Learning for Sensor-Based Activity Recognition: Recent Trends
Ahad, M. A. R., Antar, A. D., & Ahmed, M. (2020) · Intelligent systems reference library · doi:10.1007/978-3-030-51379-5_9 · Cited by 7 · see citing works
Effect of Reduced Dimensionality on Deep learning for Human Activity Recognition
Kumari, G., Chakraborty, J., & Nandy, A. (2020) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt49239.2020.9225419 · Cited by 7 · see citing works
A trustworthy model of recommender system using hyper-tuned restricted boltzmann machine
Jha, G. K., Gaur, M., Ranjan, P., & Thakur, H. K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11575-8 · Cited by 6 · see citing works
Uddin, M. H., Ara, J. M. K., Rahman, M. H., & Yang, S. (2021) · International Conference on Electrical Information and Communication Technology (EICT) · doi:10.1109/eict54103.2021.9733607 · Cited by 6 · see citing works
Sensor-Based Human Activity and Behavior Computing
Antar, A. D., Ahmed, M., & Ahad, M. A. R. (2021) · Intelligent systems reference library · doi:10.1007/978-3-030-75490-7_6 · Cited by 5 · see citing works
Behavior recognition and disaster detection by the abnormal analysis using SVM for ERESS
Nakajima, S., Yamasaki, T., Matsumoto, K., Uemura, K., Wada, T., & Ohtsuki, K. (2018) · International Conference on Information Networking · doi:10.1109/icoin.2018.8343198 · Cited by 3 · see citing works
(2019) · International Journal of Innovative Technology and Exploring Engineering · doi:10.35940/ijitee.f1111.0486s419 · Cited by 3 · see citing works
Implementation of Smartphone Based Activity Recognition
Dupare, S., Moundekar, Y., & Chandankhede, P. H. (2023) · International Conference for Advancement in Technology (ICONAT) · doi:10.1109/iconat57137.2023.10080702 · Cited by 2 · see citing works
A Neural Network Model to Identify Relative Movements from Wearable Devices
John, C. S., Mueller, Z., Prayaga, L., & Devulapalli, K. (2020) · IEEE SoutheastCon · doi:10.1109/southeastcon44009.2020.9368261 · Cited by 2 · see citing works
Ríos, M., Torroledo, C. A. G., Quintero, C. A., & Rodriguez, C. S. (2018) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2018.00099 · Cited by 1 · see citing works
Contributions to Human Motion Modeling and Recognition using Non-intrusive Wearable Sensors
Martín, M. (2022) · doi:10.20868/upm.thesis.70493 · Cited by 1 · see citing works
- Authors
- Zhang, L., Wu, X., & Luo, D.
- Affiliations
- Peking University (CN)
- Venue
- 2015 IEEE 12th Intl Conf on Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom)
- Publisher
- IEEE
- Published
- 2015-08
- Pages
- 1236-1242
- OpenAlex
- W2494054254
- Cited by
- 35
- References
- 28
- Citations by year
- 2024: 2 · 2023: 1 · 2022: 4 · 2021: 6 · 2020: 9 · 2019: 5 · 2018: 8
- FWCI
- 1.09
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Ubiquitous Intelligence and Computing, Autonomic and Trusted Computing, and Scalable Computing and Communications (UIC-ATC-ScalCom) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- real-time activity recognition, smartphone sensing, accelerometer data, deep neural network, automatic feature learning, wearable activity recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
