The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
A Survey of Deep Learning-Based Human Activity Recognition in Radar
Open accessAbstract
Radar, as one of the sensors for human activity recognition (HAR), has unique characteristics such as privacy protection and contactless sensing. Radar-based HAR has been applied in many fields such as human–computer interaction, smart surveillance and health assessment. Conventional machine learning approaches rely on heuristic hand-crafted feature extraction, and their generalization capability is limited. Additionally, extracting features manually is time–consuming and inefficient. Deep learning acts as a hierarchical approach to learn high-level features automatically and has achieved superior performance for HAR. This paper surveys deep learning based HAR in radar from three aspects: deep learning techniques, radar systems, and deep learning for radar-based HAR. Especially, we elaborate deep learning approaches designed for activity recognition in radar according to the dimension of radar returns (i.e., 1D, 2D and 3D echoes). Due to the difference of echo forms, corresponding deep learning approaches are different to fully exploit motion information. Experimental results have demonstrated the feasibility of applying deep learning for radar-based HAR in 1D, 2D and 3D echoes. Finally, we address some current research considerations and future opportunities.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Deep Learning: Methods and Applications
Deng, L., & Yu, D. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000039 · Cited by 3,480
Why Does Unsupervised Pre-training Help Deep Learning?
Erhan, D., Bengio, Y., Courville, A. C., Manzagol, P.-A., Vincent, P., & Samy Bengio, A. (2010) · Journal of Machine Learning Research · Cited by 2,573
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014 · cited 3×
“Secondly, hand-crafted features often refer to some low-level statistical information including mean, variance, frequency and amplitude [32], which are task-specific.”
BackgroundMethodInfluentialDeep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
Micro-doppler effect in radar: phenomenon, model, and simulation study
Chen, V. C., Li, F., Ho, S.-S., & Wechsler, H. (2006) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2006.1603402 · Cited by 1,684
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
Human activity recognition with smartphone sensors using deep learning neural networks
Ronao, C. A., & Cho, S.-B. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.04.032 · Cited by 1,215
Soli: ubiquitous gesture sensing with millimeter wave radar
Lien, J., Gillian, N., Karagozler, M. E., Amihood, P., Schwesig, C., Olson, E., Raja, H., & Poupyrev, I. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925953 · Cited by 957
Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., & Penn, G. (2012) · 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2012.6288864 · Cited by 876
Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine
Kim, Y., & Ling, H. (2009) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2009.2012849 · Cited by 870
Transition-Aware Human Activity Recognition Using Smartphones
Reyes-Ortiz, J.-L., Oneto, L., Samà, A., Parra, X., & Anguita, D. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.07.085 · Cited by 740
Kim, Y., & Moon, T. (2016) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2491329 · Cited by 647
Hand gesture recognition with 3D convolutional neural networks
Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301342 · Cited by 543
From action to activity: Sensor-based activity recognition
Liu, Y., Nie, L., Liu, L., & Rosenblum, D. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.08.096 · Cited by 486
Wang, S., Song, J., Lien, J., Poupyrev, I., & Hilliges, O. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984565 · Cited by 433
Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors
Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2016) · Sensors · doi:10.3390/s16040426 · Cited by 423
Robust human activity recognition from depth video using spatiotemporal multi-fused features
Jalal, A., Kim, Y.-H., Kim, Y.-J., Kamal, S., & Kim, D. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.08.003 · Cited by 384
Hand Gesture Recognition Using Micro-Doppler Signatures With Convolutional Neural Network
Kim, Y., & Toomajian, B. (2016) · IEEE Access · doi:10.1109/access.2016.2617282 · Cited by 378
Squeeze and Excitation Rank Faster R-CNN for Ship Detection in SAR Images
Lin, Z., Ji, K., Leng, X., & Kuang, G. (2019) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2882551 · Cited by 378
A Review on Recent Progress of Portable Short-Range Noncontact Microwave Radar Systems
Li, C., Peng, Z., Huang, T.-Y., Fan, T., Wang, F.-K., Horng, T.-S., Munoz-Ferreras, J.-M., Gomez-Garcia, R., Ran, L., & Lin, J. (2017) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2017.2650911 · Cited by 372
Seyfioglu, M. S., Ozbayoglu, A. M., & Gurbuz, S. Z. (2018) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2018.2799758 · Cited by 352
Contextual Region-Based Convolutional Neural Network with Multilayer Fusion for SAR Ship Detection
Kang, M., Ji, K., Leng, X., & Lin, Z. (2017) · Remote Sensing · doi:10.3390/rs9080860 · Cited by 339
Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor
Zhang, Z., Tian, Z., & Zhou, M. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2808688 · Cited by 333
Fall Detection Using Deep Learning in Range-Doppler Radars
Jokanovic, B., & Amin, M. (2018) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2017.2740098 · Cited by 260
Multi-sensor system for driver's hand-gesture recognition
Molchanov, P., Gupta, S., Kim, K., & Pulli, K. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163132 · Cited by 238
Lin, Z., Ji, K., Kang, M., Leng, X., & Zou, H. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2698213 · Cited by 234
Short-range FMCW monopulse radar for hand-gesture sensing
Molchanov, P., Gupta, S., Kim, K., & Pulli, K. (2015) · 2015 IEEE Radar Conference (RadarCon) · doi:10.1109/radar.2015.7131232 · Cited by 220
Human behavior analysis in video surveillance: A Social Signal Processing perspective
Cristani, M., Raghavendra, R., Del Bue, A., & Murino, V. (2013) · Neurocomputing · doi:10.1016/j.neucom.2011.12.038 · Cited by 197
UWB Radar for Human Being Detection
Yarovoy, A. G., Ligthart, L. P., Matuzas, J., & Levitas, B. (2006) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/maes.2006.284354 · Cited by 185
Video analysis of human dynamics—a survey
Wang, J. J., & Singh, S. (2003) · Real-Time Imaging · doi:10.1016/j.rti.2003.08.001 · Cited by 174
Radar fall motion detection using deep learning
Jokanovic, B., Amin, M., & Ahmad, F. (2016) · 2016 IEEE Radar Conference (RadarConf) · doi:10.1109/radar.2016.7485147 · Cited by 171
Open-set human activity recognition based on micro-Doppler signatures
Yang, Y., Hou, C., Lang, Y., Guan, D., Huang, D., & Xu, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.030 · Cited by 164
Super Normal Vector for Human Activity Recognition with Depth Cameras
Yang, X., & Tian, Y. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2565479 · Cited by 162
Chen, Z., Li, G., Fioranelli, F., & Griffiths, H. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2806940 · Cited by 157
Gesture Recognition Using mm-Wave Sensor for Human-Car Interface
Smith, K., Csech, C., Murdoch, D., & Shaker, G. S. A. (2018) · IEEE Sensors Letters · doi:10.1109/lsens.2018.2810093 · Cited by 140
Human motion recognition exploiting radar with stacked recurrent neural network
Wang, M., Zhang, Y. D., & Cui, G. (2019) · Digital Signal Processing · doi:10.1016/j.dsp.2019.01.013 · Cited by 136
Gesture recognition for smart home applications using portable radar sensors
Wan, Q., Li, Y., Li, C., & Pal, R. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2014.6945096 · Cited by 127
Cao, P., Xia, W., Ye, M., et al. (2018) · IET Radar Sonar & Navigation · doi:10.1049/iet-rsn.2017.0511 · Cited by 126
Jalal, A., Kamal, S., & Kim, D. (2017) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2017.447 · Cited by 124
Park, J., Javier, R. J., Moon, T., & Kim, Y. (2016) · Sensors · doi:10.3390/s16121990 · Cited by 120
Seyfioğlu, M. S., & Gürbüz, S. Z. (2017) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2017.2771405 · Cited by 120
Synthetic Aperture Radar Target Recognition with Feature Fusion Based on a Stacked Autoencoder
Kang, M., Ji, K., Leng, X., Xing, X., & Zou, H. (2017) · Sensors · doi:10.3390/s17010192 · Cited by 116
Ren, Y., Zhu, C., & Xiao, S. (2018) · Remote Sensing · doi:10.3390/rs10091470 · Cited by 105
Portable Microwave Radar Systems for Short-Range Localization and Life Tracking: A Review
Peng, Z., & Li, C. (2019) · Sensors · doi:10.3390/s19051136 · Cited by 87
Micro Hand Gesture Recognition System Using Ultrasonic Active Sensing
Sang, Y., Shi, L., & Liu, Y. (2018) · IEEE Access · doi:10.1109/access.2018.2868268 · Cited by 86
Zhou, Z. W., Cao, Z., & Pi, Y. (2017) · Sensors · doi:10.3390/s18010010 · Cited by 81
Lin, Y., Le Kernec, J., Yang, S., Fioranelli, F., Romain, O., & Zhao, Z. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2872849 · Cited by 81
Trommel, R. P., Harmanny, R. I. A., Cifola, L., & Driessen, J. N. (2016) · European Radar Conference (EuRAD) · Cited by 78
Chen, L., Zhou, M., Wu, M., She, J., Liu, Z., Dong, F., & Hirota, K. (2018) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2018.2809691 · Cited by 78
Transfer Learning for Human Activities Classification Using Micro-Doppler Spectrograms
Du, H., He, Y., & Jin, T. (2018) · IEEE International Conference on Computational Electromagnetics · doi:10.1109/compem.2018.8496654 · Cited by 68
Self-citationPeak Detection Algorithm for Vital Sign Detection Using Doppler Radar Sensors
Kim, J., Park, J., Jang, S., & Yang, J. (2019) · Sensors · doi:10.3390/s19071575 · Cited by 62
Multi-target human gait classification using LSTM recurrent neural networks applied to micro-Doppler
Klarenbeek, G. J., Harmanny, R. I. A., & Cifola, L. (2017) · European Radar Conference (EuRAD) · doi:10.23919/eurad.2017.8249173 · Cited by 61
Millimeter-Wave Interferometric Angular Velocity Detection
Nanzer, J. A. (2010) · IEEE Transactions on Microwave Theory and Techniques · doi:10.1109/tmtt.2010.2086467 · Cited by 57
Kim, Y., & Toomajian, B. (2017) · European Conference on Antennas and Propagation · doi:10.23919/eucap.2017.7928465 · Cited by 53
An FMCW radar sensor for human gesture recognition in the presence of multiple targets
Peng, Z., Li, C., Muñoz‐Ferreras, J., & Gómez‐García, R. (2017) · IEEE MTT-S International Microwave Biomedical Conference (IMBioC) · doi:10.1109/imbioc.2017.7965798 · Cited by 53
Multiple joint-variable domains recognition of human motion
Jokanović, B., Amin, M. G., & Erol, B. (2017) · IEEE Radar Conference (RadarConf) · doi:10.1109/radar.2017.7944340 · Cited by 51
Indoor Device-Free Activity Recognition Based on Radio Signal
Huang, X., & Dai, M. (2016) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2016.2616883 · Cited by 47
Doppler-Radar Based Hand Gesture Recognition System Using Convolutional Neural Networks
Zhang, J., Tao, J., & Shi, Z. (2018) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-6571-2_132 · Cited by 40
Unsupervised Domain Adaptation for Micro-Doppler Human Motion Classification via Feature Fusion
Lang, Y., Wang, Q., Yang, Y., Hou, C., Huang, D., & Xiang, W. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2873776 · Cited by 38
Intelligent video surveillance for real-time detection of suicide attempts
Bouachir, W., Gouiaa, R., Li, B., & Noumeir, R. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.03.018 · Cited by 37
Enabling Edge Devices that Learn from Each Other: Cross Modal Training for Activity Recognition
Xing, T., Sandha, S. S., Balaji, B., Chakraborty, S., & Srivastava, M. (2018) · Proceedings of the 1st International Workshop on Edge Systems, Analytics and Networking · doi:10.1145/3213344.3213351 · Cited by 35
Human Motion Classification Based on Range Information with Deep Convolutional Neural Network
Shao, Y., Guo, S., Sun, L., et al. (2017) · International Conference on Information Science and Control Engineering · doi:10.1109/icisce.2017.317 · Cited by 35
Qi, F., Lv, H., Liang, F., Li, Z., Yu, X. L., & Wang, J. (2017) · Remote Sensing · doi:10.3390/rs9030260 · Cited by 33
Le, H. T., Phung, S. L., Bouzerdoum, A., & Tivive, F. H. C. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461847 · Cited by 31
Range information for reducing fall false alarms in assisted living
Erol, B., Amin, M. G., Zhou, Z., & Zhang, J. (2016) · IEEE Radar Conference (RadarConf) · doi:10.1109/radar.2016.7485176 · Cited by 23
Kim, Y., Park, J., & Moon, T. (2017) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2262719 · Cited by 17
Deep Learning Methods for Personnel Recognition based on Micro-Doppler Features
Shao, Y., Dai, Y., Yuan, L., et al. (2017) · International Conference on Signal Processing Systems (ICSPS) · doi:10.1145/3163080.3163095 · Cited by 13
Cited by
Human Action Recognition From Various Data Modalities: A Review
Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., & Liu, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183112 · Cited by 601
Integrated Sensing and Communications: Recent Advances and Ten Open Challenges
Lu, S., Liu, F., Li, Y., Zhang, K., Huang, H., Zou, J., Li, X., Dong, Y., Dong, F., Zhu, J., et al. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3361173 · Cited by 516
Self-citationDeep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“whose transmitters and receivers need to be placed on the opposite sides, radar transmitters and antennas are mounted on the same side of users. Doppler effect is the basis of the radar-based system [84, 134]. Current research mostly adopts spectrograms for representing the Doppler effect and utilizes deep learning me…”
BackgroundMethodInfluentialDeep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“whose transmitters and receivers need to be placed on the opposite sides, radar transmitters and antennas are mounted on the same side of users. Doppler effect is the basis of the radar-based system [84, 134]. Current research mostly adopts spectrograms for representing the Doppler effect and utilizes deep learning me…”
BackgroundMethodInfluentialKumar, P., & Chauhan, S. (2021) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-021-00063-5 · Cited by 13
Retracted
Metadata
- Authors
- Li, X., He, Y., & Jing, X.
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2019-05-06
- Volume
- 11
- Issue
- 9
- Pages
- 1068
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W2944605902
- Cited by
- 290
- References
- 77
- FWCI
- 13.97
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.