The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Bi-LSTM Network for Multimodal Continuous Human Activity Recognition and Fall Detection
Open accessAbstract
This paper presents a framework based on multilayer bi-LSTM network (bidirectional Long Short-Term Memory) for multimodal sensor fusion to sense and classify daily activities' patterns and high-risk events such as falls. The data collected in this work are continuous activity streams from FMCW radar and three wearable inertial sensors on the wrist, waist, and ankle. Each activity has a variable duration in the data stream so that the transitions between activities can happen at random times within the stream, without resorting to conventional fixed-duration snapshots. The proposed bi-LSTM implements soft feature fusion between wearable sensors and radar data, as well as two robust hard-fusion methods using the confusion matrices of both sensors. A novel hybrid fusion scheme is then proposed to combine soft and hard fusion to push the classification performances to approximately 96% accuracy in identifying continuous activities and fall events. These fusion schemes implemented with the proposed bi-LSTM network are compared with conventional sliding window approach, and all are validated with realistic “leaving one participant out” (L1PO) method (i.e. testing subjects unknown to the classifier). The developed hybrid-fusion approach is capable of stabilizing the classification performance among different participants in terms of reducing accuracy variance of up to 18.1% and increasing minimum, worst-case accuracy up to 16.2%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 1×
“Graves [32], [33] are chosen, as they can learn the backward and forward long-term dependencies between small”
BackgroundFramewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1×
“Rather than considering radar data [27] or wearable sensors data [28] in isolation, we investigate soft”
BackgroundWearable Sensors for Human Activity Monitoring: A Review
Mukhopadhyay, S. C. (2015) · IEEE Sensors Journal · doi:10.1109/jsen.2014.2370945 · Cited by 1,401
A Review of Data Fusion Techniques
Castanedo, F. (2013) · The Scientific World Journal · doi:10.1155/2013/704504 · Cited by 1,027
Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine
Kim, Y., & Ling, H. (2009) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2009.2012849 · Cited by 870
RT-Fall: A Real-Time and Contactless Fall Detection System with Commodity WiFi Devices
Wang, H., Zhang, D., Wang, Y., Ma, J., Wang, Y., & Li, S. (2017) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2016.2557795 · Cited by 640
Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition
Wu, D., Pigou, L., Kindermans, P.-J., Le, N. D.-H., Shao, L., Dambre, J., & Odobez, J.-M. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2537340 · Cited by 453
Gürbüz, S. Z., & Amin, M. G. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2018.2890128 · Cited by 421
Nweke, H. F., Teh, Y. W., Mujtaba, G., & Al-garadi, M. A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.06.002 · Cited by 394
Indoor Person Identification Using a Low-Power FMCW Radar
Vandersmissen, B., Knudde, N., Jalalvand, A., Couckuyt, I., Bourdoux, A., De Neve, W., & Dhaene, T. (2018) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2018.2816812 · Cited by 262
A weighted voting framework for classifiers ensembles
Kuncheva, L. I., & Rodríguez, J. J. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-012-0586-6 · Cited by 254
Wearable Sensor-Based Hand Gesture and Daily Activity Recognition for Robot-Assisted Living
Zhu, C., & Sheng, W. (2011) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2010.2093883 · Cited by 229
From Fall Detection to Fall Prevention: A Generic Classification of Fall-Related Systems
Chaccour, K., Darazi, R., El Hassani, A. H., & Andres, E. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2016.2628099 · Cited by 207
Radar and RGB-Depth Sensors for Fall Detection: A Review
Cippitelli, E., Fioranelli, F., Gambi, E., & Spinsante, S. (2017) · IEEE Sensors Journal · doi:10.1109/jsen.2017.2697077 · Cited by 198
Self-citationRadar Micro-Doppler Signatures: Processing and Applications
(2014) · Institution of Engineering and Technology eBooks · doi:10.1049/pbra034e · Cited by 193
A Real-Time Human Action Recognition System Using Depth and Inertial Sensor Fusion
Chen, C., Jafari, R., & Kehtarnavaz, N. (2016) · IEEE Sensors Journal · doi:10.1109/jsen.2015.2487358 · Cited by 187
Ding, C., Hong, H., Zou, Y., Chu, H., Zhu, X., Fioranelli, F., Le Kernec, J., & Li, C. (2019) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2019.2908758 · Cited by 183
Self-citationApplication of data fusion techniques and technologies for wearable health monitoring
King, R. C., Villeneuve, E., White, R. J., Sherratt, R. S., Holderbaum, W., & Harwin, W. S. (2017) · Medical Engineering & Physics · doi:10.1016/j.medengphy.2016.12.011 · Cited by 182
Multi-classification approaches for classifying mobile app traffic
Aceto, G., Ciuonzo, D., Montieri, A., & Pescapé, A. (2018) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2017.11.007 · Cited by 181
Kernec, J. L., Fioranelli, F., Ding, C., Zhao, H., Sun, L. G., Hong, H., Lorandel, J., & Romain, O. (2019) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2903715 · Cited by 163
Self-citationFioranelli, F., Ritchie, M., & Griffiths, H. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2439393 · Cited by 149
Self-citationHuman motion recognition exploiting radar with stacked recurrent neural network
Wang, M., Zhang, Y. D., & Cui, G. (2019) · Digital Signal Processing · doi:10.1016/j.dsp.2019.01.013 · Cited by 136
Multiple comparator classifier framework for accelerometer-based fall detection and diagnostic
Gibson, R. M., Amira, A., Ramzan, N., Casaseca‐de‐la‐Higuera, P., & Pervez, Z. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.10.062 · Cited by 118
Bennett, T. R., Wu, J., Kehtarnavaz, N., & Jafari, R. (2016) · IEEE Signal Processing Magazine · doi:10.1109/msp.2015.2499314 · Cited by 84
Knowledge Exploitation for Human Micro-Doppler Classification
Karabacak, C., Gürbüz, S. Z., Gürbüz, A. C., Guldogan, M. B., Hendeby, G., & Gustafsson, F. (2015) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2015.2452311 · Cited by 81
Operational assessment and adaptive selection of micro‐Doppler features
Gürbüz, S. Z., Erol, B., Çağlıyan, B., & Tekeli, B. (2015) · IET Radar Sonar & Navigation · doi:10.1049/iet-rsn.2015.0144 · Cited by 77
A Multisensory Approach for Remote Health Monitoring of Older People
Li, H., Shrestha, A., Heidari, H., Kernec, J. L., & Fioranelli, F. (2018) · IEEE Journal of Electromagnetics RF and Microwaves in Medicine and Biology · doi:10.1109/jerm.2018.2827099 · Cited by 66
Self-citationHuman Activity Detection and Coarse Localization Outdoors Using Micro-Doppler Signatures
Luo, F., Poslad, S., & Bodanese, E. (2019) · IEEE Sensors Journal · doi:10.1109/jsen.2019.2917375 · Cited by 64
CapsFall: Fall Detection Using Ultra-Wideband Radar and Capsule Network
Sadreazami, H., Bolić, M., & Rajan, S. (2019) · IEEE Access · doi:10.1109/access.2019.2907925 · Cited by 61
Magnetic and Radar Sensing for Multimodal Remote Health Monitoring
Li, H., Shrestha, A., Heidari, H., Kernec, J. L., & Fioranelli, F. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2872894 · Cited by 49
Self-citationFioranelli, F., Ritchie, M., & Griffiths, H. (2016) · IET Radar Sonar & Navigation · doi:10.1049/iet-rsn.2015.0493 · Cited by 48
Self-citation(2015) · doi:10.4230/oasics.slate.2021.8 · Cited by 1
Research works printed but not linked: 6
- 10.1109/TGRS.2012.2188411 · doi:10.1109/tgrs.2012.2188411
- wang, 2013, CW and pulse-Doppler radar processing based on FPGA for human sensing applications, IEEE Trans Geosci Remote Sens, 51, 3097
- erol, 0, Radar data cube processing for human activity recognition using multi subspace learning, IEEE Trans Aerosp Electron Syst
- 2018, Help At Home Use of Assistive Technology for Older People, 3
- 2008, WHO Global Report on Falls Prevention in Older Age
- 10.1049/iet-rsn.2016.0055 · doi:10.1049/iet-rsn.2016.0055
Cited by
Artificial intelligence for the metaverse: A survey
Huynh-The, T., Pham, Q.-V., Pham, X.-Q., Nguyen, T. T., Han, Z., & Kim, D.-S. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105581 · Cited by 710
ST-DeepHAR: Deep Learning Model for Human Activity Recognition in IoHT Applications
Abdel-Basset, M., Hawash, H., Chakrabortty, R. K., Ryan, M., Elhoseny, M., & Song, H. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3033430 · Cited by 136
A spatio-temporal graph neural network for fall prediction with inertial sensors
Wang, S., Li, X., Liao, G., Liu, J., Liao, C., Liu, M., Liao, J., & Liu, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111709 · Cited by 18
Metadata
- Authors
- Li, H., Shrestha, A., Heidari, H., Le Kernec, J., & Fioranelli, F.
- Venue
- IEEE Sensors Journal
- Published
- 2020-02-01
- Volume
- 20
- Issue
- 3
- Pages
- 1191-1201
- ISSN
- 1530-437X (ISSN-L)
- 1558-1748
- 2379-9153
- OpenAlex
- W2979637742
- Cited by
- 277
- References
- 33
- FWCI
- 10.95
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- Engineering and Physical Sciences Research Council (EP/R041679/1)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.