Deep Learning for RFID-Based Activity Recognition
Open accessThe DOI matches its Crossref record; 41 of the 42 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
We present a system for activity recognition from passive RFID data using a deep convolutional neural network. We directly feed the RFID data into a deep convolutional neural network for activity recognition instead of selecting features and using a cascade structure that first detects object use from RFID data followed by predicting the activity. Because our system treats activity recognition as a multi-class classification problem, it is scalable for applications with large number of activity classes. We tested our system using RFID data collected in a trauma room, including 14 hours of RFID data from 16 actual trauma resuscitations. Our system outperformed existing systems developed for activity recognition and achieved similar performance with process-phase detection as systems that require wearable sensors or manually-generated input. We also analyzed the strengths and limitations of our current deep learning architecture for activity recognition from RFID data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“We chose to have 3 convolutional layers in our network, with odd-number kernel sizes: 3×3×32, 3×3×64 and 3×3×128, and with stride 1, which have been shown as efficient in the VGG net [30].”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 7×
“We used the CNN with rectified linear units (ReLUs) because such units train several times faster than traditional tanh units [2,25].”
BackgroundMethodResultInfluentialAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“We initialized the learning rate at 0.01 and adjusted it based on the ADAM optimization (Adam Optimizer in TensorFlow) [32].”
MethodInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“Several parameters need to be determined for constructing the convolutional layers [29,2].”
Background(2014) · Cited by 33,419
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“The use of deep learning in recent years has led to great leaps in many fields, from image classification [2] to speech recognition [19].”
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853 · cited 1×
“Earlier research tried to accomplish the complex tasks such as object recognition or activity recognition by using manufactured features, or building a hierarchical model with several layers of classifiers to extract low-level features for final decision making [17,18].”
MethodTensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408 · cited 1×
“The network was developed with a Microsoft Azure cloud computing platform [8] and locally with Google TensorFlow [9].”
MethodTwo-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · cited 1×
“Some similarity exists with video data processing where image frames are temporally related and pixels in each frame are spatially related [28].”
BackgroundLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 2×
“The 3D convolution was recently used for video classification and object recognition [3,40].”
BackgroundMethodEvaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1×
“We used the common metrics of F -score, Informedness, Markedness [34] and Matthew Correlation Coefficient (MCC) [35].”
MethodVoxNet: A 3D Convolutional Neural Network for real-time object recognition
Maturana, D., & Scherer, S. (2015) · 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353481 · Cited by 3,458 · cited 1×
“The 3D convolution was recently used for video classification and object recognition [3,40].”
Method(2011) · Cited by 2,752
A Survey on Human Activity Recognition using Wearable Sensors
Lara, O. D., & Labrador, M. A. (2013) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2012.110112.00192 · Cited by 2,662 · cited 1×
“Unlike systems designed for recognizing simple physical activities of individuals, such as sitting, standing, or sleeping [37,20,21], recognizing complex teamwork activities is significantly more challenging.”
BackgroundUnderstanding Neural Networks Through Deep Visualization
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,358 · cited 3×
“We visualized and analyzed the neuron activations in each convolutional layer as has been done in image analysis [11].”
BackgroundMethodInfluential(2019) · 2019 International Conference "Quality Management, Transport and Information Security, Information Technologies" (IT&QM&IS) · doi:10.1109/itqmis.2019.8928339 · Cited by 954
Inferring Activities from Interactions with Objects
Philipose, M., Fishkin, K. P., Perkowitz, M., Patterson, D. J., Fox, D., Kautz, H., & Hahnel, D. (2004) · IEEE Pervasive Computing · doi:10.1109/mprv.2004.7 · Cited by 875 · cited 2×
“Various features have been proposed and classifiers tested for RFID systems in different application settings [6,7], which makes it unfeasible to compare their relative efficiency.”
BackgroundTagoram: real-time tracking of mobile RFID tags to high precision using COTS devices
Yang, L., Chen, Y., Li, X.-Y., Xiao, C., Li, M., & Liu, Y. (2014) · Proceedings of the 20th annual international conference on Mobile computing and networking · doi:10.1145/2639108.2639111 · Cited by 757 · cited 3×
“Examples include the Convolutional Neural Network (CNN), widely used for image classification [13] and recently for speech recognition [24], the Deep Neural Network (DNN), used for speech recognition and audio sensing [25], and a multimodal structure used for audio-visual speech recognition [26].”
BackgroundMethodSelf-citationExploring convolutional neural network structures and optimization techniques for speech recognition
Abdel-Hamid, O., Deng, L., & Yu, D. (2013) · Interspeech 2013 · doi:10.21437/interspeech.2013-744 · Cited by 384 · cited 2×
“As was the case with other applications [2,4], our deep learning-based activity recognition system also achieved better performance compared to systems that used traditional classifiers.”
BackgroundResultA Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer
Chen, Y., & Xue, Y. (2015) · 2015 IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2015.263 · Cited by 378 · cited 2×
“Earlier research showed that deep learning can be applied to data from mobile phone sensors or an accelerometer for recognition of person’s simple physical activities [20,21].”
BackgroundA Scalable Approach to Activity Recognition based on Object Use
Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., & Rehg, J. M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408865 · Cited by 324 · cited 3×
“We also compared the performance of our deep-learning system using RFID data for medical activity recognition with several state-of-the-art recognition systems for real-world application scenarios [14,38,39] (Table 6) .”
BackgroundMethodResultDeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning
Lane, N. D., Georgiev, P., & Qendro, L. (2015) · Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2750858.2804262 · Cited by 310 · cited 2×
“Examples include the Convolutional Neural Network (CNN), widely used for image classification [13] and recently for speech recognition [24], the Deep Neural Network (DNN), used for speech recognition and audio sensing [25], and a multimodal structure used for audio-visual speech recognition [26].”
MethodCan Deep Learning Revolutionize Mobile Sensing?
Lane, N. D., & Georgiev, P. (2015) · Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications · doi:10.1145/2699343.2699349 · Cited by 282 · cited 2×
“Consistent with the views of others [5], we believe that deep learning has the potential to be successful for mobile sensing.”
BackgroundMobile big data analytics using deep learning and apache spark
Alsheikh, M. A., Niyato, D., Lin, S., Tan, H.-P., & Han, Z. (2016) · IEEE Network · doi:10.1109/mnet.2016.7474340 · Cited by 272 · cited 2×
“Earlier research showed that deep learning can be applied to data from mobile phone sensors or an accelerometer for recognition of person’s simple physical activities [20,21].”
Deng, Z., Vahdat, A., Hu, H., & Mori, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.516 · Cited by 253 · cited 1×
“We also compared the performance of our deep-learning system using RFID data for medical activity recognition with several state-of-the-art recognition systems for real-world application scenarios [14,38,39] (Table 6) .”
ResultPerformance metrics for activity recognition
Ward, J. A., Lukowicz, P., & Gellersen, H. W. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1889681.1889687 · Cited by 182 · cited 1×
“To deal with inadequacies of traditional evaluation metrics, researches have designed a new set of metrics for activity recognition, such as frame metrics and 2SET metrics [41].”
MethodIDSense: A Human Object Interaction Detection System Based on Passive UHF RFID
Li, H., Ye, C., & Sample, A. P. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702178 · Cited by 160 · cited 2×
“We did not manually remove their corresponding data from our testing set as has been done in other research [12,16], but these “extraneous” data may cause false predictions because in some cases an object may be used in the labeled activities as well as in “other activities.””
BackgroundMethodDeep Structured Models For Group Activity Recognition
Deng, Z., Zhai, M., Chen, L., Liu, Y., Muralidharan, S., Javan, M., & Mori, G. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.179 · Cited by 110 · cited 1×
“We also compared the performance of our deep-learning system using RFID data for medical activity recognition with several state-of-the-art recognition systems for real-world application scenarios [14,38,39] (Table 6) .”
ResultMultiview activity recognition in smart homes with spatio-temporal features
Chen, W., Khalili, A. H., & Aghajan, H. K. (2010) · ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC) · doi:10.1145/1865987.1866010 · Cited by 88 · cited 1×
“Earlier research tried to accomplish the complex tasks such as object recognition or activity recognition by using manufactured features, or building a hierarchical model with several layers of classifiers to extract low-level features for final decision making [17,18].”
MethodPhase recognition during surgical procedures using embedded and body-worn sensors
Bardram, J. E., Doryab, A., Jensen, R. M., Lange, P. M., Nielsen, K. L. G., & Petersen, S. T. (2011) · 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom) · doi:10.1109/percom.2011.5767594 · Cited by 76 · cited 3×
“RFID was also used as the primary system for process-phase detection with wearable RFID antennas and other sensors [15].”
MethodResultModeling and Online Recognition of Surgical Phases Using Hidden Markov Models
Blum, T., Padoy, N., Feußner, H., & Navab, N. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85990-1_75 · Cited by 61 · cited 3×
“Our system is also advantageous compared to the system that takes as input a manually-generated activity log [10], or the system that takes input directly from medical equipment sets [36].”
BackgroundResultCBID: A Customer Behavior Identification System Using Passive Tags
Han, J., Ding, H., Qian, C., Ma, D., Xi, W., Wang, Z., Jiang, Z., & Shangguan, L. (2014) · IEEE International Conference on Network Protocols (ICNP) · doi:10.1109/icnp.2014.26 · Cited by 54 · cited 2×
“Attributes of RFID signal other than RSS, such as Doppler shift and phase angle, have been used for human-object interaction detection or people tracking [13,23].”
BackgroundMethodAutomatic phase prediction from low-level surgical activities
Forestier, G., Riffaud, L., & Jannin, P. (2015) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-015-1195-0 · Cited by 49 · cited 4×
“The experimental results showed that our system achieved performance on phase detection comparable to the system that used manually-generated log of executed tasks as input to phase recognition [10].”
BackgroundResultInfluentialMicrosoft Azure Machine Learning
Mund, S. (2015) · CERN Document Server (European Organization for Nuclear Research) · Cited by 34
Activity recognition for medical teamwork based on passive RFID
Li, X., Yao, D., Pan, X., Johannaman, J., Yang, J., Webman, R. B., Sarcevic, A., Marsic, I., & Burd, R. S. (2016) · IEEE International Conference on RFID · doi:10.1109/rfid.2016.7488002 · Cited by 33 · cited 10×
“As suggested [6], we selected a percentage of data from each class for training and used the remainder for testing.”
BackgroundMethodResultInfluentialSelf-citationDeep neural network for RFID-based activity recognition
Li, X., Zhang, Y., Li, M., Marsic, I., Yang, J., & Burd, R. S. (2016) · Proceedings of the Eighth Wireless of the Students, by the Students, and for the Students Workshop · doi:10.1145/2987354.2987355 · Cited by 27 · cited 1×
“We recently implemented a modified DNN structure using a similar dataset, but directly using high-dimensional RSS in the input layer remains a challenge [27].”
MethodSelf-citationPassive RFID for Object and Use Detection during Trauma Resuscitation
Parlak, S., Marsic, I., Sarcevic, A., Bajwa, W. U., Waterhouse, L. J., & Burd, R. S. (2015) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2015.2436393 · Cited by 18 · cited 2×
“We did not manually remove their corresponding data from our testing set as has been done in other research [12,16], but these “extraneous” data may cause false predictions because in some cases an object may be used in the labeled activities as well as in “other activities.””
BackgroundMethodSelf-citationDeep Structured Models For Group Activity RecognitionPreprint
Deng, Z., Zhai, M., Chen, L., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.04191 · Cited by 17 · cited 1×
“We also compared the performance of our deep-learning system using RFID data for medical activity recognition with several state-of-the-art recognition systems for real-world application scenarios [14,38,39] (Table 6) .”
ResultDeng, Z., Vahdat, A., Hu, H., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04196 · Cited by 13 · cited 1×
“We also compared the performance of our deep-learning system using RFID data for medical activity recognition with several state-of-the-art recognition systems for real-world application scenarios [14,38,39] (Table 6) .”
ResultUntitled work
Srivastava (2014) · doi:10.5555/2627435.2670313 · cited 1×
“First, we applied the “dropout” in fully-connected layers, which is widely used in CNNs to avoid overfitting during model training [31].”
Method
Research works printed but not linked: 4
- 10.5555/2999134.2999257 · doi:10.5555/2999134.2999257
- Abdel-Hamid, Ossama, Li Deng, and Dong Yu. "Exploring convolutional neural network structures and optimization techniques for speech recognition." In Interspeech, pp. 3366-3370. 2013..
- 10.5555/2968826.2968890 · doi:10.5555/2968826.2968890
- Abadi, Martın, et al. "TensorFlow: Large-scale machine learning on heterogeneous systems, 2015." Software available from tensorflow.org.
Cited by
Deep learning for sensor-based activity recognition: A survey
Wang, J., Chen, Y., Hao, S., Peng, X., & Hu, L. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.02.010 · Cited by 2,014
“(Li et al., 2016a) added pooling after the fully-connected layers to extract the important features.”
BackgroundInfluentialDeep Learning in Mobile and Wireless Networking: A Survey
Zhang, C., Patras, P., & Haddadi, H. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2904897 · Cited by 1,768
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“onal expenses [161]. In addition to wearable sensors, RFID-based activity recognition requires the fusion of multiple RFID signals as well, and CNNs are also commonly used for the early fusion manner [86]. Sensor-based Fusion (SF). In contrast to EF, SF fist considers each modality individually and then fuses differen…”
MethodHuman activity recognition in artificial intelligence framework: a narrative review
Gupta, N., Gupta, S. K., Pathak, R. K., Jain, V., Rashidi, P., & Suri, J. S. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10116-x · Cited by 337
Metadata
- Authors
- Li, X., Zhang, Y., Marsic, I., Sarcevic, A., & Burd, R. S.
- Affiliations
- Rutgers, The State University of New Jersey (US)
- Drexel University (US)
- Children's National (US)
- Venue
- Proceedings of the 14th ACM Conference on Embedded Networked Sensor Systems CD-ROM
- Publisher
- ACM
- Published
- 2016-11-14
- Volume
- 2016
- Pages
- 164-175
- PMC
- PMC6205502
- OpenAlex
- W2548765505
- Cited by
- 150
- References
- 39
- Citations by year
- 2026: 5 · 2025: 12 · 2024: 15 · 2023: 25 · 2022: 15 · 2021: 16 · 2020: 15 · 2019: 22 · 2018: 18 · 2017: 7
- FWCI
- 5.43
- Type
- conference paper
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- ACM Conference on Embedded Networked Sensor Systems (SenSys) (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health
- U.S. National Library of Medicine (R01LM011834)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Computer Networks and Communications › RFID technology advancements
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Indoor and Outdoor Localization Technologies
- Keywords
- activity recognition, passive RFID, deep convolutional neural networks, trauma resuscitation, multi-class classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.