Multicolumn bidirectional long short-term memory for mobile devices-based human activity recognition
Abstract
The ever-growing popularity of mobile devices equipped with accelerometers has provided the opportunity to capture the semantic aspects of human activity and improve user experiences with behavior-based recommendations. These functions depend heavily on the accuracy of human activity recognition, and thus real applications that use mobile devices-based human activity recognition systems (MARSs) need to seamlessly incorporate the information carried by newly labeled training samples. Motivated by the success of the weightlessness feature, we propose a new two-directional feature for bidirectional long short-term memory (BLSTM) for incremental learning in human activity recognition. To further improve the performance, we also present a new ensemble classifier termed multicolumn BLSTM (MBLSTM), which effectively combines different acceleration signal features to further improve activity recognition accuracy. Experiments on the naturalistic mobile devices-based human activity dataset suggest that MBLSTM is superior to other state-of-the-art MARS methods.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bidirectional LSTMs, having both past and future recurrent connections, were used in [205,206] to classify activities.
“Bidirectional LSTMs, having both past and future recurrent connections, were used in [205,206] to classify activities.”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [206]
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Multi-column deep neural networks for image classification
Ciresan, D. C., Meier, U., & Schmidhuber, J. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248110 · Cited by 3,760 · see citing works
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147 · see citing works
A Novel Connectionist System for Unconstrained Handwriting Recognition
Graves, A., Liwicki, M., Fernandez, S., Bertolami, R., Horst Bunke, A., & Schmidhuber, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.137 · Cited by 1,984 · see citing works
Activity recognition from accelerometer data
Ravi, N., Dandekar, N., Mysore, P., & Littman, M. L. (2005) · Cited by 1,434 · see citing works
Classification with Noisy Labels by Importance Reweighting
Liu, T., & Tao, D. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2456899 · Cited by 796 · see citing works
Self-citationErmes, M., Pärkkä, J., Mäntyjärvi, J., & Korhonen, I. (2008) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2007.899496 · Cited by 707 · see citing works
Offline Arabic Handwriting Recognition with Multidimensional Recurrent Neural Networks
Graves, A. (2012) · Springer eBooks · doi:10.1007/978-1-4471-4072-6_12 · Cited by 685 · see citing works
Activity Recognition and Monitoring Using Multiple Sensors on Different Body Positions
Maurer, U., Smailagic, A., Siewiorek, D. P., & Deisher, M. E. (2006) · Wearable and Implantable Body Sensor Networks · doi:10.1109/bsn.2006.6 · Cited by 606 · see citing works
A Practical Approach to Recognizing Physical Activities
Lester, J., Choudhury, T., & Borriello, G. (2006) · Lecture notes in computer science · doi:10.1007/11748625_1 · Cited by 464 · see citing works
Activity Recognition Using a Single Accelerometer Placed at the Wrist or Ankle
MANNINI, A., INTILLE, S. S., ROSENBERGER, M., SABATINI, A. M., & HASKELL, W. (2013) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e31829736d6 · Cited by 383 · see citing works
Superpixel-Based Hand Gesture Recognition With Kinect Depth Camera
Wang, C., liu, Z. H., & Chan, S. C. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2374357 · Cited by 329 · see citing works
Simple and Complex Activity Recognition through Smart Phones
Dernbach, S., Das, B., Krishnan, N. C., Thomas, B. L., & Cook, D. J. (2012) · International Conference on Intelligent Environments (IE) · doi:10.1109/ie.2012.39 · Cited by 317 · see citing works
Large-Margin Multi-ViewInformation Bottleneck
Xu, C., Tao, D., & Xu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.2296528 · Cited by 255 · see citing works
Self-citationWearable sensor badge and sensor jacket for context awareness
Farringdon, J., Moore, A. J., Tilbury, N., Church, J., & Biemond, P. D. (2003) · International Symposium on Wearable Computers · doi:10.1109/iswc.1999.806681 · Cited by 239 · see citing works
An Activity Recognition System For Mobile Phones
Győrbíró, N., Fábián, Á., & Hományi, G. (2008) · Mobile Networks and Applications · doi:10.1007/s11036-008-0112-y · Cited by 237 · see citing works
Feature Selection and Activity Recognition from Wearable Sensors
Pirttikangas, S., Fujinami, K., & Nakajima, T. (2006) · Lecture Notes in Computer Science · doi:10.1007/11890348_39 · Cited by 229 · see citing works
Zeng, Z., Huang, T., & Zheng, W. X. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2054106 · Cited by 203 · see citing works
Wearable Computing: Accelerometers’ Data Classification of Body Postures and Movements
Ugulino, W. C., Cardador, D., Vega, K. C., Velloso, E., Milidiú, R. L., & Fukś, H. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34459-6_6 · Cited by 150 · see citing works
On the approximation capability of recurrent neural networks
Hammer, B. (2000) · Neurocomputing · doi:10.1016/s0925-2312(99)00174-5 · Cited by 132 · see citing works
Daily living activity recognition based on statistical feature quality group selection
Baños, O., Damas, M., Pomares, H., Prieto, A., & Rojas, I. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.01.164 · Cited by 127 · see citing works
Background-Modeling-Based Adaptive Prediction for Surveillance Video Coding
Zhang, X., Huang, T., Tian, Y., & Gao, W. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2294549 · Cited by 106 · see citing works
Physical Activity Recognition from Accelerometer Data Using a Multi-Scale Ensemble Method
Zheng, Y., Won, W.-K., Guan, X., & Trost, S. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i2.18997 · Cited by 84 · see citing works
Ensemble Manifold Rank Preserving for Acceleration-Based Human Activity Recognition
Tao, D., Jin, L., Yuan, Y., & Xue, Y. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2357794 · Cited by 77 · see citing works
Self-citationRecognizing context for annotating a live life recording
Kern, N., Schiele, B., & Schmidt, A. (2006) · Personal and Ubiquitous Computing · doi:10.1007/s00779-006-0086-3 · Cited by 74 · see citing works
Predicting Mode of Transport from iPhone Accelerometer Data
Nham, B., Siangliulue, K., & Yeung, S. Y. (2012) · Cited by 68 · see citing works
Zhang, X., Tian, Y., Huang, T., Dong, S., & Gao, W. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2352036 · Cited by 65 · see citing works
Weighted Subspace Filtering and Ranking Algorithms for Video Concept Retrieval
Lin, L., Chen, C., Shyu, M., et al. (2011) · IEEE Multimedia · doi:10.1109/mmul.2011.35 · Cited by 52 · see citing works
He, Z., Liu, Z., Jin, L., Zhen, L.-X., & Huang, J. (2008) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2008.4761688 · Cited by 52 · see citing works
Toward Blind Scheduling in Mobile Media Cloud: Fairness, Simplicity, and Asymptotic Optimality
Zhou, L., & Wang, H. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2241044 · Cited by 48 · see citing works
Space-time Facet Model for Human Activity Classification
Samanta, S., & Chanda, B. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2326734 · Cited by 42 · see citing works
Liang, D., Zhang, Z., & Peng, M. (2015) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2453419 · Cited by 35 · see citing works
Discriminative Temporal Smoothing for Activity Recognition from Wearable Sensors
Suutala, J., Pirttikangas, S., & Röning, J. (2007) · Springer eBooks · doi:10.1007/978-3-540-76772-5_15 · Cited by 33 · see citing works
Novel Sampling Algorithm for Human Mobility-Based Mobile Phone Sensing
Thejaswini, M., Rajalakshmi, P., & Desai, U. B. (2014) · IEEE Internet of Things Journal · doi:10.1109/jiot.2014.2388074 · Cited by 32 · see citing works
A Software-Based Sonar Ranging Sensor for Smart Phones
Graham, D., Simmons, G., Nguyen, D. T., et al. (2015) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2408451 · Cited by 31 · see citing works
A Unified Framework of Latent Feature Learning in Social Media
Yuan, Z., Sang, J., Xu, C., & Liu, Y. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2322338 · Cited by 26 · see citing works
Similar handwritten Chinese character recognition by kernel discriminative locality alignment
Tao, D., Liang, L., Jin, L., et al. (2012) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.06.014 · Cited by 26 · see citing works
Self-citationMobile Activity Recognition Using Ubiquitous Data Stream Mining
Gomes, J. B., Krishnaswamy, S. P., Gaber, M. M., et al. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-32584-7_11 · Cited by 23 · see citing works
MACRO-CLASS SELECTION FOR HIERARCHICAL K-NN CLASSIFICATION OF INERTIAL SENSOR DATA
McCall, C., Reddy, K., & Shah, M. A. (2012) · International Conference on Pervasive and Embedded Computing and Communication Systems · doi:10.5220/0003819101060114 · Cited by 22 · see citing works
Light-Weight Online Unsupervised Posture Detection by Smartphone Accelerometer
Yurur, O., Liu, C. H., & Moreno, W. A. (2015) · IEEE Internet of Things Journal · doi:10.1109/jiot.2015.2404929 · Cited by 21 · see citing works
Wearable assistant for load monitoring: recognition of on-body load placement from gait alterations
Benocci, M., Bächlin, M., Farella, E., et al. (2010) · doi:10.4108/icst.pervasivehealth2010.8894 · Cited by 18 · see citing works
New criteria for exponential stability of delayed recurrent neural networks
Xiao, J., Zeng, Z., & Wu, A. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.07.053 · Cited by 17 · see citing works
A Multimedia Semantic Retrieval Mobile System Based on HCFGs
Yang, Y., Ha, H.-Y., Fleites, F. C., et al. (2013) · IEEE Multimedia · doi:10.1109/mmul.2013.33 · Cited by 17 · see citing works
How Mobility Impacts Video Streaming over Multi-Hop Wireless Networks?
Zhou, L., Wang, H., & Guizani, M. (2012) · IEEE Transactions on Communications · doi:10.1109//tcomm.2012.051712.110165 · Cited by 16 · see citing works
Interruptions : using activity transitions to trigger proactive messages
Ho, J. (2004) · Cited by 14 · see citing works
How Mobility Impacts Video Streaming over Multi-Hop Wireless Networks?
Zhou, L., Wang, H., & Guizani, M. (2012) · IEEE Transactions on Communications · doi:10.1109/tcomm.2012.051712.110165 · Cited by 14 · see citing works
A Model-Assisted Cross-Layer Design of an Energy-Efficient Mobile Video Cloud
Lombardo, A., Panarello, C., & Schembra, G. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2350257 · Cited by 14 · see citing works
Guerreiro, T., Gamboa, R., & Jorge, J. A. (2008) · doi:10.1145/1473018.1473033 · Cited by 12 · see citing works
Jain, R. (2014) · Computer · doi:10.1109/mc.2014.336 · Cited by 4 · see citing works
Building Social Life Networks.
Jain, R., Jalali, L., Pongpaichet, S., et al. (2013) · IEEE Data(base) Engineering Bulletin · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“Bidirectional LSTMs, having both past and future recurrent connections, were used in [166, 271] to classify activities.”
MethodDeep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690 · see citing works
“al. 2017 [94]; Saeedi et al. 2017 [130]; Morales et al. 2016 [99]; Jiang et al. 2018 [66]; Khan et al. 2018 [72]; Zheng et al. 2019 [192]; Gjoreski et al. 2019 [43] Time Concept Drift Tao et al. 2016 [143]; Gudur et al.2019 [50] Concept Evolution Nair et al. 2019 [102] Open-Set Yang et al. 2019 [165] Sensor Instance M…”
BackgroundDemrozi, F., Pravadelli, G., Bihorac, A., & Rashidi, P. (2020) · IEEE Access · doi:10.1109/access.2020.3037715 · Cited by 368 · see citing works
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314 · see citing works
“al. 2017 [94]; Saeedi et al. 2017 [130]; Morales et al. 2016 [99]; Jiang et al. 2018 [66]; Khan et al. 2018 [72]; Zheng et al. 2019 [192]; Gjoreski et al. 2019 [43] Time Concept Drift Tao et al. 2016 [143]; Gudur et al.2019 [50] Concept Evolution Nair et al. 2019 [102] Open-Set Yang et al. 2019 [165] Sensor Instance M…”
BackgroundDistilling the Knowledge From Handcrafted Features for Human Activity Recognition
Chen, Z., Zhang, L., Cao, Z., & Guo, J. (2018) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2018.2789925 · Cited by 160 · see citing works
Deep Learning Techniques for Agronomy Applications
Chen, C.-H., Kung, H.-Y., & Hwang, F.-J. (2019) · Agronomy · doi:10.3390/agronomy9030142 · Cited by 29 · see citing works
Valarezo, A., Rivera, L., Park, H., Park, N., & Kim, T.-S. (2020) · Computer Science and Information Systems · doi:10.2298/csis190920005v · Cited by 12 · see citing works
- Authors
- Tao, D., Wen, Y., & Hong, R.
- Affiliations
- Yunnan University (CN)
- Nanyang Technological University (SG)
- Hefei University of Technology (CN)
- Venue
- IEEE Internet of Things Journal
- Published
- 2016-12
- Volume
- 3
- Issue
- 6
- Pages
- 1124-1134
- ISSN
- 2327-4662 (ISSN-L)
- OpenAlex
- W2526868004
- Cited by
- 83
- References
- 50
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 4 · 2023: 14 · 2022: 9 · 2021: 8 · 2020: 11 · 2019: 13 · 2018: 10 · 2017: 10
- FWCI
- 7.47
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- Funders
- National Natural Science Foundation of China (61572486, 61472116)
- Guangdong Natural Science Funds (2014A030310252)
- Shenzhen Technology Project (JCYJ20140901003939001)
- Singapore MOE Tier-1 (RG13/17)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Transportation › Human Mobility and Location-Based Analysis
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- human activity recognition, mobile devices, accelerometer, bidirectional LSTM, incremental learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
