DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Deep convolutional neural networks (DCNNs) are currently popular in human activity recognition (HAR) applications. However, in the face of modern artificial intelligence sensor-based games, many research achievements cannot be practically applied on portable devices (i.e., smart phone, VR/AR). DCNNs are typically resource-intensive and too large to be deployed on portable devices, and thus, this limits the practical application of complex activity detection. In addition, since portable devices do not possess high-performance graphic processing units, there is hardly any improvement in Action Game (ACT) experience. Besides, in order to deal with multi-sensor collaboration, all previous HAR models typically treated the representations from different sensor signal sources equally. However, distinct types of activities should adopt different fusion strategies. In this paper, a novel scheme is proposed. This scheme is used to train 2-bit CNNs with weights and activations constrained to {-0.5, 0, 0.5}. It takes into account the correlation between different sensor signal sources and the activity types. This model, which we refer to as DFTerNet, aims at producing a more reliable inference and better trade-offs for practical applications. It is known that quantization of weights and activations can substantially reduce memory size and use more efficient bitwise operations to replace floating or matrix operations to achieve much faster calculation and lower power consumption. Our basic idea is to exploit quantization of weights and activations directly in pre-trained filter banks and adopt dynamic fusion strategies for different activity types. Experiments demonstrate that by using a dynamic fusion strategy, it is possible to exceed the baseline model performance by up to ~5% on activity recognition data sets, such as the OPPORTUNITY and PAMAP2 data sets. Using the quantization method proposed, we were able to achieve performances closer to that of the full-precision counterpart. These results were also verified using the UniMiB-SHAR data set. In addition, the proposed method can achieve ~9x acceleration on CPUs and ~11x memory saving.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 1×
“(16/32)-bit floating point, 2-bit (even 8-bit) operations will not only reduce the energy costs for Hardware/IC Design (see Figure 1), but also halves the memory access costs and memory size requirements during runtime, which will greatly facilitate the deployment of binary/ternary convolutional neural networks on por…”
BackgroundDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 6×
“Recently, many researchers have adopted Convolutional Neural Networks (CNNs) to deploy human activity recognition system, such as [6], [22], and [26]–[28].”
BackgroundResultInfluentialEstimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 10×
“The accelerometer, 2Note that, following [26], the PAMAP2 dataset is downsampled to 100Hz3 = 33.33Hz, in order to have a temporal resolution comparable gyroscope, magnetometer, temperature and heart rate contain 40 sensors and are recorded from IMU over 10 hours in total.”
MethodInfluentialDeep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364 · cited 1×
“Some researchers (see [16], [48]) have proven that stochastic rounding has an unbiased expectation and has been success-”
BackgroundThe Opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. D. R., & Roggen, D. (2013) · Pattern Recognition Letters · doi:10.1016/j.patrec.2012.12.014 · Cited by 742 · cited 15×
“We set the segment parameters like [44], use a time-window of 2s on the OPPORTUNITY and PAMAP2 datasets, resulting in T=64, and σ = 3.”
MethodInfluential
Research works printed but not linked: 38
- zhou, 2017, Incremental network quantization Towards lossless cnns with low-precision weights
- hubara, 2016, Quantized neural networks Training neural networks with low precision weights and activations
- xu, 2015, Show, attend and tell: Neural image caption generation with visual attention, Proc Int Conf Mach Learn (ICML), 2048
- 10.1109/CVPR.2015.7298965 · DOI · Paper page
- 10.3115/v1/D14-1181 · DOI · Paper page
- 10.1109/ACCESS.2018.2833888 · DOI
- 10.1049/el:19901121 · DOI
- 10.1109/ICCV.2015.522 · DOI · Paper page
- 10.1109/ACCESS.2017.2777003 · DOI
- 10.1109/CVPR.2015.7298594 · DOI · Paper page
- cook, 2015, Activity Learning Discovering Recognizing and Predicting Human Behavior from Sensor Data
- 10.1145/2493988.2494349 · DOI · Paper page
- feldhorst, 2016, Motion classification for analyzing the order picking process using mobile sensors, Proc Int Conf Pattern Recognit Appl Methods, 706, 10.5220/0005828407060713 · DOI
- 10.1109/ICCI-CC.2015.7259385 · DOI · Paper page
- ioffe, 2015, Batch Normalization Accelerating Deep Network Training by Reducing Internal Covariate Shift
- zeiler, 2012, ADADELTA An Adaptive Learning Rate Method
- simonyan, 2014, Very Deep Convolutional Networks for Large-scale Image Recognition
- wen, 2017, TernGrad: Ternary gradients to reduce communication in distributed deep learning, Proc Adv Neural Inf Process Syst (NIPS), 1508
- esser, 2016, Convolutional Networks for Fast Energy-Efficient Neuromorphic Computing
- han, 2015, Deep compression Compressing deep neural networks with pruning trained quantization and huffman coding
- courbariaux, 2015, BinaryConnect: Training deep neural networks with binary weights during propagations, Proc Adv Neural Inf Process Syst (NIPS), 3123
- courbariaux, 2016, Binarized neural networks Training deep neural networks with weights and activations constrained to +1 or ?1
- rastegari, 2016, Xnor-net Imagenet classification using binary convolutional neural networks
- 10.1109/FPT.2014.7082765 · doi:10.1109/fpt.2014.7082765
- 10.1145/3123021.3123046 · DOI · Paper page
- 10.1109/ACCESS.2016.2608777 · DOI
- 10.1109/ACCESS.2017.2770178 · DOI
- yang, 2015, Deep convolutional neural networks on multichannel time series for human activity recognition, Proc Intern Joint Conf Artificial Intel (IJCAI), 3995
- 10.1109/ACCESS.2018.2810264 · DOI
- horowitz, 2014, Computing’s energy problem (and what we can do about it), IEEE Int Solid-State Circuits Conf Dig Tech Papers, 10
- 10.1109/ACCESS.2017.2753830 · DOI
- mu?a, 2016, Faster Population Counts using AVX2 Instructions
- szegedy, 2016, Inception-v4 inception-resnet and the impact of residual connections on learning
- 10.1007/978-3-319-46484-8_48 · DOI · Paper page
- rueda, 2018, Learning attribute representation for human activity recognition
- alistarh, 2017, QSGD: Communication-efficient SGD via gradient quantization and encoding, Proc Adv Neural Inf Process Syst (NIPS), 1707
- 10.1007/BF02678430 · DOI · Paper page
- bengio, 2013, Estimating or propagating gradients through stochastic neurons
Cited by
Deep Learning for Sensor-based Human Activity Recognition
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2022) · ACM Computing Surveys · doi:10.1145/3447744 · Cited by 690
“y-accumulate operations (require hundreds of logic gates at least) can be replaced by popcount-XNOR operations (only require a single logic gate), which are especially well suited for FPGAs and ASICs [166]. In [166], Yang et al. explored a 2-bit CNN with weights and activations constrained to {-0.5, 0, 0.5} for effici…”
MethodInfluentialMulti-input CNN-GRU based human activity recognition using wearable sensors
Dua, N., Singh, S. N., & Semwal, V. B. (2021) · Computing · doi:10.1007/s00607-021-00928-8 · Cited by 432
Deep Learning for Sensor-based Human Activity Recognition: Overview, Challenges and Opportunities
Chen, K., Zhang, D., Yao, L., Guo, B., Yu, Z., & Liu, Y. (2020) · arXiv · doi:10.48550/arxiv.2001.07416 · Cited by 314
“y-accumulate operations (require hundreds of logic gates at least) can be replaced by popcount-XNOR operations (only require a single logic gate), which are especially well suited for FPGAs and ASICs [166]. In [166], Yang et al. explored a 2-bit CNN with weights and activations constrained to {-0.5, 0, 0.5} for effici…”
MethodInfluentialKumar, P., & Chauhan, S. (2021) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-021-00063-5 · Cited by 13
Retracted
Metadata
- Authors
- Yang, Z., Raymond, O. I., Zhang, C., Wan, Y., & Long, J.
- Venue
- IEEE Access
- Published
- 2018
- Volume
- 6
- Pages
- 56750-56764
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2887355508
- Cited by
- 48
- References
- 19
- FWCI
- 2.74
- Type
- journal article
- Language
- en
- Open access
- gold
- Funders
- National Natural Science Foundation of China (61472450)
- Natural Science Foundation of Hunan Province (2017JJ3417)
- Science and Technology Plan of Hunan (2016TP1003)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition. https://app.cochat.ai/papers/10.1109/access.2018.2873315
- The paper
- Yang, Z., Raymond, O. I., Zhang, C., Wan, Y., & Long, J. (2018). DFTerNet: Towards 2-bit Dynamic Fusion Networks for Accurate Human Activity Recognition. IEEE Access. https://doi.org/10.1109/access.2018.2873315
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.