CoChat

A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning

Open access
Bouchabou, Damien, Nguyen, Sao Mai, Lohr, Christophe, LeDuc, Benoit, Kanellos, Ioannis · 2021 · Sensors
Cited by 210

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.

Abstract

Recent advances in Internet of Things (IoT) technologies and the reduction in the cost of sensors have encouraged the development of smart environments, such as smart homes. Smart homes can offer home assistance services to improve the quality of life, autonomy, and health of their residents, especially for the elderly and dependent. To provide such services, a smart home must be able to understand the daily activities of its residents. Techniques for recognizing human activity in smart homes are advancing daily. However, new challenges are emerging every day. In this paper, we present recent algorithms, works, challenges, and taxonomy of the field of human activity recognition in a smart home through ambient sensors. Moreover, since activity recognition in smart homes is a young field, we raise specific problems, as well as missing and needed contributions. However, we also propose directions, research opportunities, and solutions to accelerate advances in this field.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s21186037

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 23 references

Research works printed but not linked: 92

  1. Chan, 2008, A review of smart homes—Present state and future challenges, Comput. Methods Programs Biomed., 91, 55, 10.1016/j.cmpb.2008.02.001 · DOI · Paper page
  2. Hussain, Z., Sheng, M., and Zhang, W.E. (2019). Different Approaches for Human Activity Recognition: A Survey. arXiv.
  3. Beddiar, 2020, Vision-based human activity recognition: A survey, Multimed. Tools Appl., 79, 30509, 10.1007/s11042-020-09004-3 · DOI · Paper page
  4. Singh, D., Psychoula, I., Kropf, J., Hanke, S., and Holzinger, A. (2018). Users’ perceptions and attitudes towards smart home technologies. International Conference on Smart Homes and Health Telematics, Springer. · DOI
  5. Li, X., Zhang, Y., Marsic, I., Sarcevic, A., and Burd, R.S. (2016, January 14–16). Deep learning for rfid-based activity recognition. Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM, Stanford, CA, USA. · DOI · Paper page
  6. Gomes, L., Sousa, F., and Vale, Z. (2018). An intelligent smart plug with shared knowledge capabilities. Sensors, 18. · doi:10.3390/s18113961
  7. Chen, 2012, Sensor-based activity recognition, IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.), 42, 790, 10.1109/TSMCC.2012.2198883 · DOI · Paper page
  8. Aggarwal, 2014, Human activity recognition from 3d data: A review, Pattern Recognit. Lett., 48, 70, 10.1016/j.patrec.2014.04.011 · DOI · Paper page
  9. Vrigkas, M., Nikou, C., and Kakadiaris, I.A. (2015). A review of human activity recognition methods. Front. Robot. AI, 2. · DOI · Paper page
  10. Liciotti, 2020, A Sequential Deep Learning Application for Recognising Human Activities in Smart Homes, Neurocomputing, 396, 501, 10.1016/j.neucom.2018.10.104 · DOI · Paper page
  11. Gochoo, 2018, Unobtrusive activity recognition of elderly people living alone using anonymous binary sensors and DCNN, IEEE J. Biomed. Health Inform., 23, 693
  12. Yan, 2020, Using latent knowledge to improve real-time activity recognition for smart IoT, IEEE Trans. Knowl. Data Eng., 32, 574, 10.1109/TKDE.2019.2891659 · DOI
  13. Perkowitz, M., Philipose, M., Fishkin, K., and Patterson, D.J. (2004, January 17–20). Mining models of human activities from the web. Proceedings of the 13th International Conference on World Wide Web, New York, NY, USA. · DOI · Paper page
  14. Chen, 2008, A logical framework for behaviour reasoning and assistance in a smart home, Int. J. Assist. Robot. Mechatron., 9, 20
  15. Chen, L., and Nugent, C.D. (2019). Human Activity Recognition and Behaviour Analysis, Springer. · DOI
  16. Yamada, 2007, Applying ontology and probabilistic model to human activity recognition from surrounding things, IPSJ Digit. Cour., 3, 506, 10.2197/ipsjdc.3.506 · DOI
  17. Chen, L., Nugent, C., Mulvenna, M., Finlay, D., and Hong, X. (2009). Semantic smart homes: Towards knowledge rich assisted living environments. Intelligent Patient Management, Springer. · DOI
  18. Chen, 2009, Ontology-based activity recognition in intelligent pervasive environments, Int. J. Web Inf. Syst., 5, 410, 10.1108/17440080911006199 · DOI · Paper page
  19. Chen, 2011, A knowledge-driven approach to activity recognition in smart homes, IEEE Trans. Knowl. Data Eng., 24, 961, 10.1109/TKDE.2011.51 · DOI · Paper page
  20. Logan, B., Healey, J., Philipose, M., Tapia, E.M., and Intille, S. (2007, January 16–19). A long-term evaluation of sensing modalities for activity recognition. Proceedings of the International Conference on Ubiquitous Computing, Innsbruck, Austria.
  21. Vail, D.L., Veloso, M.M., and Lafferty, J.D. (2007, January 14–18). Conditional random fields for activity recognition. Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, Honolulu, HI, USA. · DOI · Paper page
  22. Fleury, 2009, SVM-based multimodal classification of activities of daily living in health smart homes: Sensors, algorithms, and first experimental results, IEEE Trans. Inf. Technol. Biomed., 14, 274, 10.1109/TITB.2009.2037317 · DOI · Paper page
  23. Brdiczka, 2008, Learning situation models in a smart home, IEEE Trans. Syst. Man Cybern. Part B (Cybern.), 39, 56, 10.1109/TSMCB.2008.923526 · DOI · Paper page
  24. van Kasteren, T., and Krose, B. (2007, January 24–25). Bayesian activity recognition in residence for elders. Proceedings of the 2007 3rd IET International Conference on Intelligent Environments, Ulm, Germany. · DOI · Paper page
  25. Cook, 2010, Learning setting-generalized activity models for smart spaces, IEEE Intell. Syst., 2010, 1
  26. Sedky, 2018, Evaluating machine learning techniques for activity classification in smart home environments, Int. J. Inf. Syst. Comput. Sci., 12, 48
  27. Chinellato, E., Hogg, D.C., and Cohn, A.G. (2016, January 14–16). Feature space analysis for human activity recognition in smart environments. Proceedings of the 2016 12th International Conference on Intelligent Environments (IE), London, UK. · DOI
  28. Cook, 2013, Activity discovery and activity recognition: A new partnership, IEEE Trans. Cybern., 43, 820, 10.1109/TSMCB.2012.2216873 · DOI · Paper page
  29. Yala, N., Fergani, B., and Fleury, A. (2015, January 2–4). Feature extraction for human activity recognition on streaming data. Proceedings of the 2015 International Symposium on Innovations in Intelligent SysTems and Applications (INISTA), Madrid, Spain. · DOI
  30. Aminikhanghahi, 2019, Enhancing activity recognition using CPD-based activity segmentation, Pervasive Mob. Comput., 53, 75, 10.1016/j.pmcj.2019.01.004 · DOI
  31. Pouyanfar, 2018, A survey on deep learning: Algorithms, techniques, and applications, ACM Comput. Surv. (CSUR), 51, 1
  32. Fang, 2014, Human activity recognition based on feature selection in smart home using back-propagation algorithm, ISA Trans., 53, 1629, 10.1016/j.isatra.2014.06.008 · DOI
  33. Irvine, N., Nugent, C., Zhang, S., Wang, H., and Ng, W.W. (2020). Neural network ensembles for sensor-based human activity recognition within smart environments. Sensors, 20. · DOI
  34. Tan, 2018, Multi-resident activity recognition in a smart home using RGB activity image and DCNN, IEEE Sens. J., 18, 9718, 10.1109/JSEN.2018.2866806 · DOI · Paper page
  35. Mohmed, G., Lotfi, A., and Pourabdollah, A. (July, January 30). Employing a deep convolutional neural network for human activity recognition based on binary ambient sensor data. Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments, Corfu, Greece.
  36. Singh, D., Merdivan, E., Hanke, S., Kropf, J., Geist, M., and Holzinger, A. (2017). Convolutional and recurrent neural networks for activity recognition in smart environment. Towards Integrative Machine Learning and Knowledge Extraction, Springer. · DOI
  37. Wang, A., Chen, G., Shang, C., Zhang, M., and Liu, L. (2016, January 3–5). Human activity recognition in a smart home environment with stacked denoising autoencoders. Proceedings of the International Conference on Web-Age Information Management, Nanchang, China. · DOI
  38. van Kasteren, T.L., Englebienne, G., and Kröse, B.J. (2011). Human activity recognition from wireless sensor network data: Benchmark and software. Activity Recognition in Pervasive Intelligent Environments, Springer. · DOI · Paper page
  39. Ghods, A., and Cook, D.J. (2019). Activity2vec: Learning adl embeddings from sensor data with a sequence-to-sequence model. arXiv.
  40. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, MIT Press.
  41. Bouchabou, D., Nguyen, S.M., Lohr, C., Kanellos, I., and Leduc, B. (2021, January 8). Fully Convolutional Network Bootstrapped by Word Encoding and Embedding for Activity Recognition in Smart Homes. Proceedings of the IJCAI 2020 Workshop on Deep Learning for Human Activity Recognition, Yokohama, Japan. · DOI
  42. Quigley, B., Donnelly, M., Moore, G., and Galway, L. (2018). A Comparative Analysis of Windowing Approaches in Dense Sensing Environments. Proceedings, 2. · DOI
  43. van Kasteren, T.L.M. (2011). Activity Recognition for Health Monitoring Elderly Using Temporal Probabilistic Models. [Ph.D. Thesis, Universiteit van Amsterdam].
  44. Zhang, 2018, Ensemble classifier of long short-term memory with fuzzy temporal windows on binary sensors for activity recognition, Expert Syst. Appl., 114, 441, 10.1016/j.eswa.2018.07.068 · DOI
  45. Hamad, 2019, Efficient activity recognition in smart homes using delayed fuzzy temporal windows on binary sensors, IEEE J. Biomed. Health Inform., 24, 387, 10.1109/JBHI.2019.2918412 · DOI
  46. Hamad, R.A., Yang, L., Woo, W.L., and Wei, B. (2020). Joint learning of temporal models to handle imbalanced data for human activity recognition. Appl. Sci., 10. · DOI
  47. Hamad, R.A., Kimura, M., Yang, L., Woo, W.L., and Wei, B. (2021). Dilated causal convolution with multi-head self attention for sensor human activity recognition. Neural Comput. Appl., 1–18. · DOI
  48. Al Machot, F., Mayr, H.C., and Ranasinghe, S. (2016, January 5–8). A windowing approach for activity recognition in sensor data streams. Proceedings of the 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN), Vienna, Austria. · DOI
  49. Cook, 2009, Assessing the quality of activities in a smart environment, Methods Inf. Med., 48, 480, 10.3414/ME0592 · DOI · Paper page
  50. Philipose, 2004, Inferring activities from interactions with objects, IEEE Pervasive Comput., 3, 50, 10.1109/MPRV.2004.7 · DOI · Paper page
  51. Singh, D., Merdivan, E., Psychoula, I., Kropf, J., Hanke, S., Geist, M., and Holzinger, A. (September, January 29). Human activity recognition using recurrent neural networks. Proceedings of the International Cross-Domain Conference for Machine Learning and Knowledge Extraction, Reggio, Italy.
  52. Park, J., Jang, K., and Yang, S.B. (2018, January 5–8). Deep neural networks for activity recognition with multi-sensor data in a smart home. Proceedings of the 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), Singapore. · DOI
  53. Hong, 2009, Evidential fusion of sensor data for activity recognition in smart homes, Pervasive Mob. Comput., 5, 236, 10.1016/j.pmcj.2008.05.002 · DOI · Paper page
  54. Asghari, P., Soelimani, E., and Nazerfard, E. (2019). Online Human Activity Recognition Employing Hierarchical Hidden Markov Models. arXiv. · DOI
  55. Devanne, M., Papadakis, P., and Nguyen, S.M. (2019, January 6–9). Recognition of Activities of Daily Living via Hierarchical Long-Short Term Memory Networks. Proceedings of the International Conference on Systems Man and Cybernetics, Bari, Italy. · DOI
  56. Wang, 2020, Human Activity Recognition Based on Wearable Sensor Using Hierarchical Deep LSTM Networks, Circuits Syst. Signal Process., 39, 837, 10.1007/s00034-019-01116-y · DOI
  57. Tayyub, J., Hawasly, M., Hogg, D.C., and Cohn, A.G. (2018, January 12–15). Learning Hierarchical Models of Complex Daily Activities from Annotated Videos. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, NV, USA. · DOI
  58. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv.
  59. Safyan, 2019, Ontology-driven semantic unified modelling for concurrent activity recognition (OSCAR), Multimed. Tools Appl., 78, 2073, 10.1007/s11042-018-6318-5 · DOI
  60. Li, X., Zhang, Y., Zhang, J., Chen, S., Marsic, I., Farneth, R.A., and Burd, R.S. (2017). Concurrent activity recognition with multimodal CNN-LSTM structure. arXiv.
  61. Alhamoud, A., Muradi, V., Böhnstedt, D., and Steinmetz, R. (2016, January 7–9). Activity recognition in multi-user environments using techniques of multi-label classification. Proceedings of the 6th International Conference on the Internet of Things, Stuttgart, Germany. · DOI
  62. Tran, S.N., Zhang, Q., Smallbon, V., and Karunanithi, M. (2018, January 19–23). Multi-resident activity monitoring in smart homes: A case study. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece. · DOI
  63. Natani, 2021, Sequential neural networks for multi-resident activity recognition in ambient sensing smart homes, Appl. Intell., 51, 6014, 10.1007/s10489-020-02134-z · DOI
  64. Cook, 2012, CASAS: A smart home in a box, Computer, 46, 62, 10.1109/MC.2012.328 · DOI · Paper page
  65. Sanchis, 2013, Activity recognition using hybrid generative/discriminative models on home environments using binary sensors, Sensors, 13, 5460, 10.3390/s130505460 · DOI · Paper page
  66. Wang, 2020, Activities of Daily Living Recognition With Binary Environment Sensors Using Deep Learning: A Comparative Study, IEEE Sens. J., 21, 5423, 10.1109/JSEN.2020.3035062 · DOI
  67. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
  68. Perslev, M., Jensen, M.H., Darkner, S., Jennum, P.J., and Igel, C. (2019). U-time: A fully convolutional network for time series segmentation applied to sleep staging. arXiv.
  69. Schlimmer, 1986, Incremental learning from noisy data, Mach. Learn., 1, 317, 10.1007/BF00116895 · DOI · Paper page
  70. Thrun, S., and Pratt, L. (1998). Learning to Learn, Springer US. · DOI · Paper page
  71. Duminy, N., Nguyen, S.M., Zhu, J., Duhaut, D., and Kerdreux, J. (2021). Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchy. Appl. Sci., 11. · DOI
  72. Weiss, 2016, A survey of transfer learning, J. Big Data, 3, 9, 10.1186/s40537-016-0043-6 · DOI · Paper page
  73. Gjoreski, 2015, Competitive live evaluations of activity-recognition systems, IEEE Pervasive Comput., 14, 70, 10.1109/MPRV.2015.3 · DOI
  74. Espinilla, M., Medina, J., and Nugent, C. (2018). UCAmI Cup. Analyzing the UJA Human Activity Recognition Dataset of Activities of Daily Living. Proceedings, 2. · DOI
  75. Alemdar, H., Ertan, H., Incel, O.D., and Ersoy, C. (2013, January 5–8). ARAS human activity datasets in multiple homes with multiple residents. Proceedings of the 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, Venice, Italy. · doi:10.4108/pervasivehealth.2013.252120
  76. Cumin, J., Lefebvre, G., Ramparany, F., and Crowley, J.L. (2017, January 7–10). A dataset of routine daily activities in an instrumented home. Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence, Philadelphia, PA, USA. · DOI
  77. Quero, 2018, Sensor-based datasets for human activity recognition—A systematic review of literature, IEEE Access, 6, 59192, 10.1109/ACCESS.2018.2873502 · DOI · Paper page
  78. Helal, S., Kim, E., and Hossain, S. (April, January 29). Scalable approaches to activity recognition research. Proceedings of the 8th International Conference Pervasive Workshop, Mannheim, Germany.
  79. Helal, S., Lee, J.W., Hossain, S., Kim, E., Hagras, H., and Cook, D. (2011, January 25–28). Persim-Simulator for human activities in pervasive spaces. Proceedings of the 2011 Seventh International Conference on Intelligent Environments, Nottingham, UK. · DOI
  80. Mendez-Vazquez, A., Helal, A., and Cook, D. (2021, July 02). Simulating events to generate synthetic data for pervasive spaces. Workshop on Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research, Available online: https://dl.acm.org/doi/abs/10.1145/1520340.1520735. · doi:10.1145/1520340.1520735
  81. Armac, I., and Retkowitz, D. (2007, January 15–20). Simulation of smart environments. Proceedings of the IEEE International Conference on Pervasive Services, Istanbul, Turkey. · DOI
  82. Fu, Q., Li, P., Chen, C., Qi, L., Lu, Y., and Yu, C. (2011, January 26–28). A configurable context-aware simulator for smart home systems. Proceedings of the 2011 6th International Conference on Pervasive Computing and Applications, Port Elizabeth, South Africa.
  83. Alshammari, N., Alshammari, T., Sedky, M., Champion, J., and Bauer, C. (2017). Openshs: Open smart home simulator. Sensors, 17. · DOI
  84. Lee, 2015, Persim 3d: Context-driven simulation and modeling of human activities in smart spaces, IEEE Trans. Autom. Sci. Eng., 12, 1243, 10.1109/TASE.2015.2467353 · DOI
  85. Synnott, J., Chen, L., Nugent, C.D., and Moore, G. (2014, January 26–30). The creation of simulated activity datasets using a graphical intelligent environment simulation tool. Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA. · DOI
  86. Synnott, 2015, Simulation of smart home activity datasets, Sensors, 15, 14162, 10.3390/s150614162 · DOI
  87. Richter, S.R., Hayder, Z., and Koltun, V. (2017, January 22–29). Playing for benchmarks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. · DOI · Paper page
  88. Roitberg, A., Schneider, D., Djamal, A., Seibold, C., Reiß, S., and Stiefelhagen, R. (2021). Let’s Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games. arXiv. · DOI
  89. Das, S., Dai, R., Koperski, M., Minciullo, L., Garattoni, L., Bremond, F., and Francesca, G. (November, January 27). Toyota smarthome: Real-world activities of daily living. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
  90. Katz, 1983, Assessing self-maintenance: Activities of daily living, mobility, and instrumental activities of daily living, J. Am. Geriatr. Soc., 31, 721, 10.1111/j.1532-5415.1983.tb03391.x · DOI · Paper page
  91. He, H., and Ma, Y. (2013). Imbalanced Learning: Foundations, Algorithms, and Applications, University of Rhode Island. · DOI · Paper page
  92. Bolleddula, N., Hung, G.Y.C., Ma, D., Noorian, H., and Woodbridge, D.M.k. (2020, January 20–24). Sensor Selection for Activity Classification at Smart Home Environments. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada. · doi:10.1109/embc44109.2020.9176631

Cited by

See all 210

Metadata

Authors
Bouchabou, D., Nguyen, S. M., Lohr, C., LeDuc, B., & Kanellos, I.
Venue
Sensors
Publisher
MDPI AG
Published
2021-09-09
Volume
21
Issue
18
Pages
6037
ISSN
1424-8220 (ISSN-L)
OpenAlex
W3196704696
Cited by
210
References
23
FWCI
12.08
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning. https://app.cochat.ai/papers/10.3390/s21186037
The paper
Bouchabou, D., Nguyen, S. M., Lohr, C., LeDuc, B., & Kanellos, I. (2021). A Survey of Human Activity Recognition in Smart Homes Based on IoT Sensors Algorithms: Taxonomies, Challenges, and Opportunities with Deep Learning. Sensors. https://doi.org/10.3390/s21186037

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.