Iss2Image: A Novel Signal-Encoding Technique for CNN-Based Human Activity Recognition
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
The most significant barrier to success in human activity recognition is extracting and selecting the right features. In traditional methods, the features are chosen by humans, which requires the user to have expert knowledge or to do a large amount of empirical study. Newly developed deep learning technology can automatically extract and select features. Among the various deep learning methods, convolutional neural networks (CNNs) have the advantages of local dependency and scale invariance and are suitable for temporal data such as accelerometer (ACC) signals. In this paper, we propose an efficient human activity recognition method, namely Iss2Image (Inertial sensor signal to Image), a novel encoding technique for transforming an inertial sensor signal into an image with minimum distortion and a CNN model for image-based activity classification. Iss2Image converts real number values from the X, Y, and Z axes into three color channels to precisely infer correlations among successive sensor signal values in three different dimensions. We experimentally evaluated our method using several well-known datasets and our own dataset collected from a smartphone and smartwatch. The proposed method shows higher accuracy than other state-of-the-art approaches on the tested datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Activity Recognition from User-Annotated Acceleration Data
Bao, L., & Intille, S. S. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24646-6_1 · Cited by 3,147
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761
Human activity recognition with smartphone sensors using deep learning neural networks
Ronao, C. A., & Cho, S.-B. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.04.032 · Cited by 1,215
Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors
Zeng, M., Nguyen, L. T., Yu, B., Mengshoel, O. J., Zhu, J., Wu, P., & Zhang, J. (2014) · Proceedings of the 6th International Conference on Mobile Computing, Applications and Services · doi:10.4108/icst.mobicase.2014.257786 · Cited by 876 · cited 1×
“CNNs are now widely used in inertial-sensor-based HAR and have two advantages over other techniques: local dependency and scale invariance [13].”
MethodHuman Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks
Jiang, W., & Yin, Z. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806333 · Cited by 676 · cited 1×
“[22] changed sensory data into image form by stacking the data row-by-row.”
BackgroundDeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052577 · Cited by 667 · cited 1×
“Many researchers have used deep learning techniques for HAR, such as a convolutional neural network (CNN) [7], an autoencoder [8], a restricted Boltzmann machine [9], a recurrent neural network (RNN) [10], and a hybrid method [11].”
MethodDeep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Preece, S. J., Goulermas, J. Y., Kenney, L. P. J., & Howard, D. (2009) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2008.2006190 · Cited by 551 · cited 1×
“The methods in [49,50] also extracted features from the acceleration signal in both the time and frequency domains; however, they each use a single classifier, decision tree and kNN, respectively.”
Method
Research works printed but not linked: 30
- Atasoy, 2016, Integrating features for accelerometer-based activity recognition, Procedia Comput. Sci., 98, 522, 10.1016/j.procs.2016.09.070 · DOI
- Zebin, T., Scully, P.J., and Ozanyan, K.B. (November, January 30). Human activity recognition with inertial sensors using a deep learning approach. Proceedings of the 2016 IEEE SENSORS, Orlando, FL, USA. · DOI · Paper page
- Xu, 2017, Learning multi-level features for sensor-based human action recognition, Pervasive Mob. Comput., 40, 324, 10.1016/j.pmcj.2017.07.001 · DOI
- Zhang, C., and Chen, T. (2003). From low level features to high level semantics. Handbook of Video Databases: Design and Applications, CRC Press.
- Ha, S., and Choi, S. (2016, January 24–29). Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. Proceedings of the 2016 International Joint Conference on Neural Networks, Vancouver, BC, Canada. · DOI · Paper page
- Almaslukh, 2017, An effective deep autoencoder approach for online smartphone-based human activity recognition, Int. J. Comput. Sci. Netw. Secur., 17, 160
- Inoue, 2018, Deep recurrent neural network for mobile human activity recognition with high throughput, Artif. Life Rob., 23, 173, 10.1007/s10015-017-0422-x · DOI · Paper page
- Bhattacharya, S., and Lane, N.D. (2016, January 14–18). From smart to deep: Robust activity recognition on smartwatches using deep learning. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communication Workshops, Sydney, Australia. · DOI · Paper page
- Uddin, 2018, Classification of Recurrence Plots’ Distance Matrices with a Convolutional Neural Network for Activity Recognition, Procedia Comput. Sci., 130, 157, 10.1016/j.procs.2018.04.025 · DOI
- Zhang, R., and Li, C. (2015). Motion sequence recognition with multi-sensors using deep convolutional neural network. Intelligent Data Analysis and Applications, Springer. · DOI
- Dehzangi, O., Taherisadr, M., and ChangalVala, R. (2017). IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor Fusion. Sensors, 17. · DOI · Paper page
- Alsheikh, M.A., Selim, A., Niyato, D., Doyle, L., Lin, S., and Tan, H.P. (2016, January 12). Deep Activity Recognition Models with Triaxial Accelerometers. Proceedings of the Artificial Intelligence Applied to Assistive Technologies and Smart Environments, Phoenix, AZ, USA.
- Lu, 2019, Deconvolutive short-time Fourier transform spectrogram, IEEE Signal Process Lett., 16, 576
- Ha, S., Yun, J.M., and Choi, S. (2015, January 9–12). Multi-modal convolutional neural networks for activity recognition. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China. · DOI · Paper page
- Yang, J., Nguyen, M.N., San, P.P., Li, X., and Krishnaswamy, S. (August, January 28). Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina.
- Baldominos, A., Saez, Y., and Isasi, P. (2018). Evolutionary Design of Convolutional Neural Networks for Human Activity Recognition in Sensor-Rich Environments. Sensors, 18. · DOI
- Cho, H., and Yoon, S.M. (2018). Divide and Conquer-Based 1D CNN Human Activity Recognition Using Test Data Sharpening. Sensors, 18. · DOI · Paper page
- Saez, Y., Baldominos, A., and Isasi, P. (2017). A comparison study of classifier algorithms for cross-person physical activity recognition. Sensors, 17. · DOI
- Vavoulas, G., Chatzaki, C., Malliotakis, T., Pediaditis, M., and Tsiknakis, M. (2016, January 21–22). The MobiAct Dataset: Recognition of Activities of Daily Living using Smartphones. Proceedings of the 2nd International Conference on Information and Communication Technologies for Ageing Well and e-Health, Rome, Italy. · DOI · Paper page
- Leutheuser, H., Schuldhaus, D., and Eskofier, B.M. (2013). Hierarchical, multi-sensor based classification of daily life activities: Comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE, 8. · DOI · Paper page
- Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24–26). A Public Domain Dataset for Human Activity Recognition Using Smartphones. Proceedings of the 21th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium.
- Penatti, O.A., and Santos, M.F. (arXiv, 2017). Human activity recognition from mobile inertial sensors using recurrence plots, arXiv.
- Zheng, X., Wang, M., and Ordieres-Meré, J. (2018). Comparison of Data Preprocessing Approaches for Applying Deep Learning to Human Activity Recognition in the Context of Industry 4.0. Sensors, 18. · DOI · Paper page
- Russakovsky, 2015, Imagenet large scale visual recognition challenge, Int. J. Comput. Vis., 115, 211, 10.1007/s11263-015-0816-y · DOI · Paper page
- Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
- Lin, M., Chen, Q., and Yan, S. (arXiv, 2013). Network in network, arXiv.
- Ahmer, M., Shah, M.Z.A., Shah, S.M.Z.S., Shah, S.M.S., Chowdhry, B.S., Shah, A., and Bhatti, K.H. (2017). Using Non-Linear Support Vector Machines for Detection of Activities of Daily Living. Indian J. Sci. Technol., 10. · DOI
- Zdravevski, 2017, Improving activity recognition accuracy in ambient-assisted living systems by automated feature engineering, IEEE Access, 5, 5262, 10.1109/ACCESS.2017.2684913 · DOI · Paper page
- Anguita, 2013, Energy Efficient Smartphone-Based Activity Recognition using Fixed-Point Arithmetic, J. Univ. Comput. Sci., 19, 1295
- Cao, 2018, GCHAR: An efficient Group-based Context—Aware human activity recognition on smartphone, J. Parallel Distrib. Comput., 118, 67, 10.1016/j.jpdc.2017.05.007 · DOI · Paper page
Cited by
Artificial intelligence for the metaverse: A survey
Huynh-The, T., Pham, Q.-V., Pham, X.-Q., Nguyen, T. T., Han, Z., & Kim, D.-S. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105581 · Cited by 710
Self-citationMetaverse for Healthcare: A Survey on Potential Applications, Challenges and Future Directions
Chengoden, R., Victor, N., Huynh-The, T., Yenduri, G., Jhaveri, R. H., Alazab, M., Bhattacharya, S., Hegde, P., Maddikunta, P. K. R., & Gadekallu, T. R. (2023) · IEEE Access · doi:10.1109/access.2023.3241628 · Cited by 508
Self-citationHuman Activity Recognition via Hybrid Deep Learning Based Model
Khan, I. U., Afzal, S., & Lee, J. W. (2022) · Sensors · doi:10.3390/s22010323 · Cited by 256
“Over the last two decades, CNNs have been actively used and achieved astonishing performance for various computer vision-related real-world problems that include activity recognition [29], object detection [30], speech recognition [31,32], and image enhancement [33].”
BackgroundSelf-citationHuynh-The, T., Hua, C.-H., Tu, N. A., & Kim, D.-S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3013272 · Cited by 72
“For example, Ordóñez and Roggen [43] simply plotted three acceleration signals separately in the time domain before learning deep visual features by a deep CNN, however, this sensory-data-to-image encoding scheme is fragile due to the loss of detail information [42].”
BackgroundMethodSelf-citationA review of AI Technologies for Wearable Devices
Jin, C. Y. (2019) · IOP Conference Series: Materials Science and Engineering · doi:10.1088/1757-899x/688/4/044072 · Cited by 32
Metadata
- Authors
- Hur, T., Bang, J., Huynh-The, T., Lee, J., Kim, J.-I., & Lee, S.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2018-11-13
- Volume
- 18
- Issue
- 11
- Pages
- 3910
- ISSN
- 1424-8220 (ISSN-L)
- OpenAlex
- W2901438495
- Cited by
- 74
- References
- 22
- FWCI
- 3.61
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- Institute for Information and communications Technology Promotion (2017-0-00655)
- Ministry of Science and ICT (IITP-2017-0-01629)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Iss2Image: A Novel Signal-Encoding Technique for CNN-Based Human Activity Recognition. https://app.cochat.ai/papers/10.3390/s18113910
- The paper
- Hur, T., Bang, J., Huynh-The, T., Lee, J., Kim, J.-I., & Lee, S. (2018). Iss2Image: A Novel Signal-Encoding Technique for CNN-Based Human Activity Recognition. Sensors. https://doi.org/10.3390/s18113910
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.