Sparsification and Separation of Deep Learning Layers for Constrained Resource Inference on Wearables
Open accessAbstract
Deep learning has revolutionized the way sensor data are analyzed and interpreted. The accuracy gains these approaches offer make them attractive for the next generation of mobile, wearable and embedded sensory applications. However, state-of-the-art deep learning algorithms typically require a significant amount of device and processor resources, even just for the inference stages that are used to discriminate high-level classes from low-level data. The limited availability of memory, computation, and energy on mobile and embedded platforms thus pose a significant challenge to the adoption of these powerful learning techniques. In this paper, we propose SparseSep, a new approach that leverages the sparsification of fully connected layers and separation of convolutional kernels to reduce the resource requirements of popular deep learning algorithms. As a result, SparseSep allows large-scale DNNs and CNNs to run efficiently on mobile and embedded hardware with only minimal impact on inference accuracy. We experiment using SparseSep across a variety of common processors such as the Qualcomm Snapdragon 400, ARM Cortex M0 and M3, and Nvidia Tegra K1, and show that it allows inference for various deep models to execute more efficiently; for example, on average requiring 11.3 times less memory and running 13.3 times faster on these representative platforms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
On the Opportunity dataset, InnoHAR improves F1-score by about 9% over CNN and by 3% over DeepConvLSTM.
“For Opportunit dataset, our proposed model has a signi cant increase of about 9% compared with the CNN model of Yang et al. [25], as well as a 3% improvement compared with DeepConvLSTM.”
From InnoHAR: A Deep Neural Network for Complex Human Activity Recognition (2019) · cites this paper as [25]
InnoHAR outperforms both CNN and DeepConvLSTM on the Opportunity, PAMAP2, and Smartphone datasets.
“In terms of overall performance, the experimental results show that our proposed InnoHAR model based on Inception-like module has better performance than both CNN [25] and DeepConvLSTM [19].”
From InnoHAR: A Deep Neural Network for Complex Human Activity Recognition (2019) · cites this paper as [25]
Most existing deep learning studies for HAR have used deep learning as a black box and scratched the data.
“Most of these existing research have only used deep learning as a black box, and the data has been scratched. For example, Ronao and Cho [22] and Yang et al. [25] perform feature extraction of the sliding window using only shallow convolutional neural networks.”
From InnoHAR: A Deep Neural Network for Complex Human Activity Recognition (2019) · cites this paper as [25]
CNN achieves an F1-score of 0.851 on the Opportunity dataset, 0.88 on the PAMAP2 dataset, and 0.915 on the Smartphone dataset.
“| CNN [25] | 0.851 | 0.88 | 0.915 |”
From InnoHAR: A Deep Neural Network for Complex Human Activity Recognition (2019) · cites this paper as [25]
On the PAMAP2 dataset, InnoHAR improves F1-score by nearly 5% over CNN and by 3% over DeepConvLSTM.
“For PAMAP2 dataset, we can also observe nearly a 5% gap between the best (proposed model) and worst (CNN [25]), and InnoHAR also has a 3% performance advantage over DeepConvLSTM.”
From InnoHAR: A Deep Neural Network for Complex Human Activity Recognition (2019) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Jolliffe, I. T. (2005) · Wiley eBooks · doi:10.1002/0470013192.bsa501 · Cited by 14,522 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · see citing works
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
Aharon, M., Elad, M., & Bruckstein, A. M. (2006) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.881199 · Cited by 9,464 · see citing works
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · see citing works
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642 · see citing works
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · Cited by 3,189 · see citing works
Efficient sparse coding algorithms
Lee, H., Battle, A. J., Raina, R., & Ng, A. Y. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,449 · see citing works
Introduction to Algorithms, third edition
Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2009) · Cited by 2,309 · see citing works
MAUI: making smartphones last longer with code offload
Cuervo, E., Balasubramanian, A., Cho, D.-K., Wolman, A., Saroiu, S., Chandra, R., & Bahl, P. (2010) · Proceedings of the 8th international conference on Mobile systems, applications, and services · doi:10.1145/1814433.1814441 · Cited by 2,281 · see citing works
Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158 · see citing works
Deep Neural Networks for Acoustic Modeling in Speech Recognition
Hinton, G. E., Deng, L., Yu, D., Dahl, G. E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T. N., et al. (2012) · Cited by 1,839 · see citing works
Deep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,785 · see citing works
Raina, R., Battle, A. J., Lee, H., Packer, B., & Ng, A. Y. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273592 · Cited by 1,514 · see citing works
One weird trick for parallelizing convolutional neural networksPreprint
Krizhevsky, A. (2014) · arXiv.org · doi:10.48550/arxiv.1404.5997 · Cited by 1,435 · see citing works
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/2541940.2541967 · Cited by 1,314 · see citing works
Speeding up Convolutional Neural Networks with Low Rank Expansions
Jaderberg, M., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.88 · Cited by 1,164 · see citing works
Deep neural networks for small footprint text-dependent speaker verification
Variani, E., Lei, X., McDermott, E., Moreno, I. L., & Gónzalez-Domínguez, J. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854363 · Cited by 1,073 · see citing works
Compressing Deep Convolutional Networks using Vector QuantizationPreprint
Gong, Y., Liu, L., Yang, M., & Bourdev, L. D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6115 · Cited by 1,047 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Small-footprint keyword spotting using deep neural networks
Chen, G., Parada, C., & Heigold, G. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854370 · Cited by 555 · see citing works
DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices
Lane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., Jiao, L., Qendro, L., & Kawsar, F. (2016) · 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2016.7460664 · Cited by 450 · see citing works
Self-citationRestructuring of deep neural network acoustic models with singular value decomposition
Xue, J., Li, J., & Gong, Y. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-552 · Cited by 433 · see citing works
Convolutional neural networks with low-rank regularizationPreprint
Tai, C., Xiao, T., Zhang, Y., Wang, X., & Weinan, E. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06067 · Cited by 373 · see citing works
Ra, M.-R., Sheth, A. N., Mummert, L., Pillai, P. S., Wetherall, D. J., & Govindan, R. (2011) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/1999995.2000000 · Cited by 360 · see citing works
Lane, N. D., Georgiev, P., & Qendro, L. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2750858.2804262 · Cited by 310 · see citing works
Self-citationCan Deep Learning Revolutionize Mobile Sensing?
Lane, N. D., & Georgiev, P. (2015) · Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications · doi:10.1145/2699343.2699349 · Cited by 282 · see citing works
Self-citationLane, N. D., Bhattacharya, S., Georgiev, P., Forlivesi, C., & Kawsar, F. (2015) · doi:10.1145/2820975.2820980 · Cited by 231 · see citing works
Self-citationRigamonti, R., Sironi, A., Lepetit, V., & Fua, P. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.355 · Cited by 213 · see citing works
Kang, S., Lee, J., Jang, H., Lee, H., Lee, Y., Park, S., Park, T., & Song, J. (2008) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/1378600.1378630 · Cited by 185 · see citing works
PD Disease State Assessment in Naturalistic Environments Using Deep Learning
Hammerla, N., Fisher, J., Andras, P., Rochester, L., Walker, R., & Ploetz, T. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9484 · Cited by 179 · see citing works
Nath, S. (2012) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2307636.2307640 · Cited by 177 · see citing works
Histogram of gradients of Time-Frequency Representations for Audio Scene Detection
Rakotomamonjy, A., & Gasso, G. (2014) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2014.2375575 · Cited by 174 · see citing works
Reshaping deep neural network for fast decoding by node-pruning
He, T., Fan, Y., Qian, Y., Tan, T., & Yu, K. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853595 · Cited by 119 · see citing works
Using unlabeled data in a sparse-coding framework for human activity recognition
Bhattacharya, S., Nurmi, P., Hammerla, N., & Plötz, T. (2014) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2014.05.006 · Cited by 116 · see citing works
Self-citationBalancing energy, latency and accuracy for mobile sensor data classification
Chu, D. C. Y., Lane, N. D., Lai, T. T.-T., Pang, C., Meng, X., Guo, Q., Li, F., & Zhao, F. (2011) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/2070942.2070949 · Cited by 111 · see citing works
Self-citationOn Vectorization of Deep Convolutional Neural Networks for Vision Tasks
Ren, J., & Li, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9488 · Cited by 82 · see citing works
Ju, Y., Lee, Y., Yu, J., et al. (2012) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/2426656.2426678 · Cited by 31 · see citing works
Moazzami, M.-M., Phillips, D. E., Tan, R., et al. (2015) · International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/2737095.2737098 · Cited by 13 · see citing works
Histogram of gradients of Time-Frequency Representations for Audio scene detectionPreprint
Rakotomamonjy, A., & Gasso, G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04909 · Cited by 7 · see citing works
Compressing LSTMs into CNNs.Preprint
Geras, K. J., Mohamed, A., Caruana, R., et al. (2015) · arXiv (Cornell University) · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Y. Bengio, I. J. Goodfellow, and A. Courville, "Deep learning," 2015, book in preparation for MIT Press. [Online]. Available: http://www.iro.umontreal.ca/˜bengioy/dlbook
- C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2007.
- T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to algorithms, 3rd ed. MIT press, 2009.
- H. Lee, A. Battle, R. Raina, and A. Y. Ng, "E cient sparse coding algorithms," in Neural Information Processing Systems (NIPS), 2007.
- M.-R. Ra, A. Sheth, L. B. Mummert, P. Pillai, D. Wetherall, and R. Govindan, "Odessa: enabling interactive perception applications on mobile devices." in MobiSys, A. K. Agrawala, M. D. Corner, and D. Wetherall, Eds. ACM, 2011, pp. 43-56. [Online]. Available: http://dblp.uni-trier.de/db/conf/mobisys/ mobisys2011.html#RaSMPWG11
- J. Xue, J. Li, and Y. Gong, "Restructuring of deep neural network acoustic models with singular value decomposition," in Interspeech, 2013. [Online]. Available: http://research.microsoft.com/apps/pubs/ default.aspx?
Cited by
Sorted by citations
Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing
Li, H., Ota, K., & Dong, M. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700202 · Cited by 1,566 · see citing works
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540 · see citing works
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“13, parameters pruning can be applied adaptively in model structure optimization as well [145]–[147].”
BackgroundDeep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances
Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., & Alshurafa, N. (2022) · Sensors · doi:10.3390/s22041476 · Cited by 625 · see citing works
“Lane et al. performed a small-scale exploration into the performance of DNNs for HAR applications on a mobile platforms in various configurations, including utilizing the phone’s CPU and DSP and offloading computation onto remote devices [158].”
BackgroundInfluentialEnsembles of Deep LSTM Learners for Activity Recognition using Wearables
Guan, Y., & Plötz, T. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3090076 · Cited by 511 · see citing works
DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters
Zhao, Z., Barijough, K. M., & Gerstlauer, A. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2858384 · Cited by 502 · see citing works
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436 · see citing works
DeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications
Huynh, L. N., Lee, Y., & Balan, R. K. (2017) · Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081360 · Cited by 329 · see citing works
InnoHAR: A Deep Neural Network for Complex Human Activity Recognition
Xu, C., Chai, D., He, J., Zhang, X., & Duan, S. (2019) · IEEE Access · doi:10.1109/access.2018.2890675 · Cited by 333 · see citing works
Multimodal Deep Learning for Activity and Context Recognition
Radu, V., Tong, C., Bhattacharya, S., Lane, N. D., Mascolo, C., Marina, M. K., & Kawsar, F. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3161174 · Cited by 281 · see citing works
Self-citationYao, S., Zhao, Y., Zhang, A., Su, L., & Abdelzaher, T. (2017) · Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems · doi:10.1145/3131672.3131675 · Cited by 188 · see citing works
On attention models for human activity recognition
Murahari, V. S., & Plötz, T. (2018) · Proceedings of the 2018 ACM International Symposium on Wearable Computers · doi:10.1145/3267242.3267287 · Cited by 157 · see citing works
DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning
Xu, M., Qian, F., Zhu, M., Huang, F., Pushp, S., & Liu, X. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2893250 · Cited by 138 · see citing works
Mathur, A., Lane, N. D., Bhattacharya, S., Boran, A., Forlivesi, C., & Kawsar, F. (2017) · Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3081333.3081359 · Cited by 128 · see citing works
Self-citationAbedin, A., Ehsanpour, M., Shi, Q., Rezatofighi, H., & Ranasinghe, D. C. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3448083 · Cited by 107 · see citing works
Adaptive deep learning model selection on embedded systems
Taylor, B., Marco, V. S., Wolff, W., Elkhatib, Y., & Wang, Z. (2018) · Proceedings of the 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems · doi:10.1145/3211332.3211336 · Cited by 90 · see citing works
On the role of features in human activity recognition
Haresamudram, H., Anderson, D. V., & Plötz, T. (2019) · Proceedings of the 23rd International Symposium on Wearable Computers · doi:10.1145/3341163.3347727 · Cited by 90 · see citing works
Mathur, A., Zhang, T., Bhattacharya, S., Velickovic, P., Joffe, L., Lane, N. D., Kawsar, F., & Lio, P. (2018) · 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1109/ipsn.2018.00048 · Cited by 70 · see citing works
Self-citationGeorgiev, P., Bhattacharya, S., Lane, N. D., & Mascolo, C. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3131895 · Cited by 67 · see citing works
Yao, S., Zhao, Y., Shao, H., Liu, D., Liu, S., Hao, Y., Piao, A., Hu, S., Lu, S., & Abdelzaher, T. F. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737500 · Cited by 52 · see citing works
Demystifying TensorRT: Characterizing Neural Network Inference Engine on Nvidia Edge Devices
Shafi, O., Rai, C., Sen, R., & Ananthanarayanan, G. (2021) · 2021 IEEE International Symposium on Workload Characterization (IISWC) · doi:10.1109/iiswc53511.2021.00030 · Cited by 51 · see citing works
Towards deep clustering of human activities from wearables
Abedin, A., Motlagh, F., Shi, Q., Rezatofighi, H., & Ranasinghe, D. (2020) · Proceedings of the 2020 International Symposium on Wearable Computers · doi:10.1145/3410531.3414312 · Cited by 27 · see citing works
The benefit of the doubt: Uncertainty aware sensing for edge computing platforms
Qendro, L., Chauhan, J., Ramos, A. G. C. P., & Mascolo, C. (2021) · arXiv · doi:10.48550/arxiv.2102.05956 · Cited by 6 · see citing works
- Authors
- Bhattacharya, S., & Lane, N. D.
- Affiliations
- Nokia Bell Labs
- Nokia (United Kingdom) (GB)
- Venue
- Proceedings of the 14th ACM Conference on Embedded Networked Sensor Systems CD-ROM
- Publisher
- ACM
- Published
- 2016-11-14
- Pages
- 176-189
- OpenAlex
- W2546536770
- Cited by
- 236
- References
- 41
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 13 · 2023: 20 · 2022: 19 · 2021: 39 · 2020: 41 · 2019: 30 · 2018: 35 · 2017: 18 · 2016: 1
- FWCI
- 9.72
- Type
- conference paper
- Language
- en
- Open access
- Locations
- ACM Conference on Embedded Networked Sensor Systems (SenSys) (publishedVersion)
- Apollo (University of Cambridge) (submittedVersion)
- Apollo
- Funders
- Nvidia
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- wearable devices, memory reduction, inference acceleration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
