PreprintCited by 974Advanced Neural Network ApplicationsStochastic Gradient Optimization TechniquesNeural Networks and Applications

The DOI matches its DataCite record; 51 of the 53 references it cites were found in a registry or an index.

Compressing Neural Networks with the Hashing Trick

Open access

Chen, Wenlin, Wilson, James T., Tyree, Stephen, Weinberger, Kilian Q., Chen, Yixin·2015·arXiv

DOI 10.48550/arxiv.1504.04788arXiv 1504.04788

Open this paper in CoChat — free account Open PDF

Abstract

As deep nets are increasingly used in applications suited for mobile devices, a fundamental dilemma becomes apparent: the trend in deep learning is to grow models to absorb ever-increasing data set sizes; however mobile devices are designed with very little memory and cannot store such large models. We present a novel network architecture, HashedNets, that exploits inherent redundancy in neural networks to achieve drastic reductions in model sizes. HashedNets uses a low-cost hash function to randomly group connection weights into hash buckets, and all connections within the same hash bucket share a single parameter value. These parameters are tuned to adjust to the HashedNets weight sharing architecture with standard backprop during training. Our hashing procedure introduces no additional memory overhead, and we demonstrate on several benchmark data sets that HashedNets shrink the storage requirements of neural networks substantially while mostly preserving generalization performance.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1504.04788

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Untitled work
(2014) · Cited by 33,419
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Untitled work
(2010) · International Conference on Machine Learning · Cited by 13,995
Term-Weighting Approaches in Automatic Text Retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 11,421
Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260

Show all 53 references

Sorted by citations

Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
“Furthermore, our results carry to other variants of dropout as well (such as drop-connect [29], multiplicative Gaussian noise [13], hashed neural networks [30], etc.”
Background
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
Self-citation
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
Luo, J.-H., Wu, J., & Lin, W. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.541 · Cited by 1,934
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and ActivationsPreprint
Hubara, I., Courbariaux, M., Soudry, D., El‐Yaniv, R., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.07061 · Cited by 1,663
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“In fact, model redundancies are common in DNNs [124], [125] and can be utilized to make model optimization possible.”
Background
Learning both Weights and Connections for Efficient Neural Networks
Han, S., Pool, J., Tran, J., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1506.02626 · Cited by 1,481
Deep Learning for IoT Big Data and Streaming Analytics: A Survey
Mohammadi, M., Al-Fuqaha, A., Sorour, S., & Guizani, M. (2018) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2018.2844341 · Cited by 1,471
Quantized Convolutional Neural Networks for Mobile Devices
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.521 · Cited by 1,273
A survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation
Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., & Hajishirzi, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_34 · Cited by 1,029
Pruning and quantization for deep neural network acceleration: A survey
Liang, T., Glossner, J., Wang, L., Shi, S., & Zhang, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.045 · Cited by 1,054
HRank: Filter Pruning Using High-Rank Feature Map
Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., & Shao, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00160 · Cited by 934
CondenseNet: An Efficient DenseNet Using Learned Group Convolutions
Huang, G., Liu, S., Maaten, L. V. D., & Weinberger, K. Q. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00291 · Cited by 904
Self-citation
A Survey of Model Compression and Acceleration for Deep Neural Networks
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2017) · arXiv · doi:10.48550/arxiv.1710.09282 · Cited by 857
One-Shot Learning for Semantic Segmentation
Shaban, A., Bansal, S., Liu, Z., Essa, I. A., & Boots, B. (2017) · British Machine Vision Conference (BMVC) · doi:10.5244/c.31.167 · Cited by 958
Sequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836
NISP: Pruning Networks Using Neuron Importance Score Propagation
Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V. I., Han, X., Gao, M., Lin, C., & Davis, L. S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00958 · Cited by 854
Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and Challenges
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765695 · Cited by 817
Dynamic Network Surgery for Efficient DNNsPreprint
Guo, Y., Yao, A., & Chen, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.04493 · Cited by 856
Self-citation
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications
Gündüz, D., Qin, Z., Aguerri, I. E., Dhillon, H. S., Yang, Z., Yener, A., Wong, K., & Chae, C. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3223408 · Cited by 893
Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile ApplicationsPreprint
Kim, Y., Park, E., Yoo, S., Choi, T., Lu, Y., & Shin, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06530 · Cited by 736
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
Zhang, D., Yang, J., Ye, D., & Hua, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_23 · Cited by 714
Fixed Point Quantization of Deep Convolutional Networks
Lin, D. D., Talathi, S. S., & Annapureddy, V. S. (2015) · arXiv · doi:10.48550/arxiv.1511.06393 · Cited by 684
Tensorizing Neural NetworksPreprint
Novikov, A., Podoprikhin, D., Osokin, A., & Vetrov, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06569 · Cited by 721
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gündüz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 577
Learning to Hash for Indexing Big Data—A Survey
Wang, J., Liu, W., Kumar, S., & Chang, S. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2487976 · Cited by 534
A comprehensive survey on model compression and acceleration
Choudhary, T., Mishra, V. K., Goswami, A., & Jagannathan, S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09816-7 · Cited by 555
Deep Learning with Low Precision by Half-Wave Gaussian Quantization
Cai, Z., He, X., Sun, J., & Vasconcelos, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.574 · Cited by 498
A Systematic DNN Weight Pruning Framework Using Alternating Direction Method of Multipliers
Zhang, T., Ye, S., Zhang, K., Tang, J., Wen, W., Fardad, M., & Wang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_12 · Cited by 492
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
BlockDrop: Dynamic Inference Paths in Residual Networks
Wu, Z., Nagarajan, T., Kumar, A., Rennie, S., Davis, L. S., Grauman, K., & Feris, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00919 · Cited by 465
Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?
Nurvitadhi, E., Venkatesh, G., Sim, J., Marr, D., Huang, R., Ong Gee Hock, J., Liew, Y. T., Srivatsan, K., Moss, D., Subhaschandra, S., et al. (2017) · Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/3020078.3021740 · Cited by 467
Group sparse regularization for deep neural networks
Scardapane, S., Comminiello, D., Hussain, A., & Uncini, A. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.02.029 · Cited by 467
Lednet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation
Wang, Y., Zhou, Q., Liu, J., Xiong, J., Gao, G., Wu, X., & Latecki, L. J. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803154 · Cited by 455
Quantization Networks
Yang, J., Shen, X., Xing, J., Tian, X., Li, H., Deng, B., Huang, J., & Hua, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00748 · Cited by 431
On Compressing Deep Models by Low Rank and Sparse Decomposition
Yu, X., Liu, T., Wang, X., & Tao, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.15 · Cited by 431
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436
Learning from Multiple Teacher Networks
You, S., Xu, C., Xu, C., & Tao, D. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098135 · Cited by 413
Fast ConvNets Using Group-Wise Brain Damage
Lebedev, V., & Lempitsky, V. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.280 · Cited by 403
Variational Convolutional Neural Network Pruning
Zhao, C., Ni, B., Zhang, J., Zhao, Q., Zhang, W., & Tian, Q. C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00289 · Cited by 379
Think Locally, Act Globally: Federated Learning with Local and Global RepresentationsPreprint
Liang, P. P., Liu, T., Ziyin, L., Allen, N. B., Auerbach, R. P., Brent, D., Salakhutdinov, R., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01523 · Cited by 712
Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video
Shaifee, M. J., Chywl, B., Li, F., & Wong, A. M. (2017) · Journal of Computational Vision and Imaging Systems · doi:10.15353/vsnl.v3i1.171 · Cited by 370
MCDNN
Han, S., Shen, H., Philipose, M., Agarwal, S., Wolman, A., & Krishnamurthy, A. (2016) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2906388.2906396 · Cited by 386
NoScope: optimizing neural network queries over video at scale
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137664 · Cited by 367
Data-Free Learning of Student Networks
Chen, H., Wang, Y., Xu, C., Yang, Z., Liu, C., Shi, B., Xu, C., Xu, C., & Tian, Q. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00361 · Cited by 387
MobileNetV2 Model for Image Classification
Dong, K., Zhou, C., Ruan, Y., & Li, Y. (2020) · International Conference on Information Technology and Computer Application (ITCA) · doi:10.1109/itca52113.2020.00106 · Cited by 379
Image Question Answering Using Convolutional Neural Network with Dynamic Parameter Prediction
Noh, H., Seo, P. H., & Han, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.11 · Cited by 359
Learning Affective Features With a Hybrid Deep Model for Audio–Visual Emotion Recognition
Zhang, S., Zhang, S., Huang, T., Gao, W., & Tian, Q. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2719043 · Cited by 346
MoDNN: Local distributed mobile computing system for Deep Neural Network
Mao, J., Chen, X., Nixon, K. W., Krieger, C., & Chen, Y. (2017) · Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017 · doi:10.23919/date.2017.7927211 · Cited by 331
Self-citation
Deep Learning Advances in Computer Vision with 3D Data
Ioannidou, A., Chatzilari, E., Nikolopoulos, S., & Kompatsiaris, I. Y. (2017) · ACM Computing Surveys · doi:10.1145/3042064 · Cited by 334
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
Ioannou, Y., Robertson, D., Cipolla, R., & Criminisi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.633 · Cited by 321
Local Binary Convolutional Neural Networks
Juefei-Xu, F., Boddeti, V. N., & Savvides, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.456 · Cited by 297
Web AR: A Promising Future for Mobile Augmented Reality—State of the Art, Challenges, and Insights
Qiao, X., Ren, P., Dustdar, S., Liu, L., Ma, H., & Chen, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2895105 · Cited by 302
Edge Intelligence: Empowering Intelligence to the Edge of Network
Xu, D., Li, T., Li, Y., Su, X. J., Tarkoma, S., Jiang, T., Crowcroft, J., & Hui, P. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3119950 · Cited by 296
ThiNet: Pruning CNN Filters for a Thinner Net
Luo, J.-H., Zhang, H., Zhou, H.-Y., Xie, C.-W., Wu, J., & Lin, W. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858232 · Cited by 269
Adaptive Neural Networks for Efficient Inference
Bolukbasi, T., Wang, J., Dekel, O., & Saligrama, V. (2017) · arXiv · doi:10.48550/arxiv.1702.07811 · Cited by 251
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey
Wang, Y., Yang, C., Lan, S., Zhu, L., & Zhang, Y. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3393230 · Cited by 269
A survey of application research based on blockchain smart contract
Lin, S., Zhang, L., Li, J., Ji, L.-L., & Sun, Y. (2022) · Wireless Networks · doi:10.1007/s11276-021-02874-x · Cited by 266
Methods for Pruning Deep Neural Networks
Vadera, S., & Ameen, S. (2022) · IEEE Access · doi:10.1109/access.2022.3182659 · Cited by 252
Cambricon-S: Addressing Irregularity in Sparse Neural Networks through A Cooperative Software/Hardware Approach
Zhou, X., Du, Z., Guo, Q., Liu, S., Liu, C., Wang, C., Zhou, X., Li, L., Chen, T., & Chen, Y. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00011 · Cited by 225
Self-citation
AutoPruner: An end-to-end trainable filter pruning method for efficient deep model inference
Luo, J.-H., & Wu, J. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107461 · Cited by 236
Convolutional Neural Network Pruning with Structural Redundancy Reduction
Wang, Z., Li, C., & Wang, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01467 · Cited by 228
Exploring Sparsity in Recurrent Neural NetworksPreprint
Narang, S., Elsen, E., Diamos, G., & Sengupta, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.05119 · Cited by 218
Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning
Yu, K., Dong, C., Lin, L., & Loy, C. C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00259 · Cited by 214
Deep Fried Convnets
Yang, Z., Moczulski, M., Denil, M., De Freitas, N., Song, L., & Wang, Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.173 · Cited by 215
Structured Transforms for Small-Footprint Deep LearningPreprint
Sindhwani, V., Sainath, T. N., & Kumar, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.01722 · Cited by 200
"Learning-Compression" Algorithms for Neural Net Pruning
Carreira-Perpiñán, M. Á., & Idelbayev, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00890 · Cited by 206
Boost Precision Agriculture with Unmanned Aerial Vehicle Remote Sensing and Edge Intelligence: A Survey
Liu, J. B., Xiang, J., Jin, Y., Liu, R., Yan, J., & Wang, L. (2021) · Remote Sensing · doi:10.3390/rs13214387 · Cited by 212
Filter-based deep-compression with global average pooling for convolutional networks
Hsiao, T.-Y., Chang, Y.-C., Chou, H.-H., & Chiu, C.-T. (2019) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2019.02.008 · Cited by 190
Data-Distortion Guided Self-Distillation for Deep Neural Networks
Xu, T.-B., & Liu, C. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015565 · Cited by 190
Compression of Neural Machine Translation Models via Pruning
See, A., Luong, M.-T., & Manning, C. D. (2016) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1029 · Cited by 220
Breast cancer histopathology image classification through assembling multiple compact CNNs
Zhu, C., Song, F., Wang, Y., Dong, H., Guo, Y., & Liu, J. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0913-x · Cited by 184
Pruning filters with L1-norm and capped L1-norm for CNN compression
Kumar, A., Shaikh, A. M., Li, Y., Bilal, H., & Yin, B. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01894-y · Cited by 171
Measuring the Intrinsic Dimension of Objective LandscapesPreprint
Li, C., Farkhoor, H., Liu, R., & Yosinski, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.08838 · Cited by 352
Curriculum Learning by Transfer Learning: Theory and Experiments with Deep NetworksPreprint
Weinshall, D., Cohen, G., & Amir, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03796 · Cited by 182
Deep Neural Network Approximation for Custom Hardware
Wang, E., Davis, J. J., Zhao, R., Ng, H.-C., Niu, X., Luk, W. W., Cheung, P. Y. K., & Constantinides, G. A. (2019) · ACM Computing Surveys · doi:10.1145/3309551 · Cited by 175
Building fast and compact convolutional neural networks for offline handwritten Chinese character recognition
Xiao, X., Jin, L., Yang, Y., Yang, W., Sun, J., & Chang, T. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.06.032 · Cited by 170
Holistic CNN Compression via Low-Rank Decomposition with Knowledge Transfer
Lin, S., Ji, R., Chen, C., Tao, D., & Luo, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2873305 · Cited by 164
CNNpack: packing convolutional neural networks in the frequency domain
Wang, Y., Xu, C., You, S., Tao, D., & Xu, C. (2016) · neural information processing systems · Cited by 148
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition
Meng, Y., Lin, C.-C., Panda, R., Sattigeri, P., Karlinsky, L., Oliva, A., Saenko, K., & Feris, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_6 · Cited by 167
An Entropy-based Pruning Method for CNN CompressionPreprint
Luo, J.-H., & Wu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05791 · Cited by 147
A Survey of FPGA-Based Neural Network AcceleratorPreprint
Guo, K., Zeng, S., Yu, J., Wang, Y., & Yang, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.08934 · Cited by 147
Ultimate tensorization: compressing convolutional and FC layers alikePreprint
Garipov, T., Podoprikhin, D., Novikov, A., & Vetrov, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03214 · Cited by 141
OpenEI: An Open Framework for Edge Intelligence
Zhang, X., Wang, Y., Lu, S., Liu, L., xu, L., & Shi, W. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00182 · Cited by 139
“Users are allowed to integrate the trained machine learning model into Apple products, such as Siri, Camera, and QuickType.”
Method
MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation
Qiu, Y., Liu, Y., Li, S., & Xu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i6.16617 · Cited by 138
Deep Learning for Mobile Multimedia: A Survey
Ota, K., Dao, M. S., Mezaris, V., & Natale, F. G. B. D. (2017) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3092831 · Cited by 135
SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks
Faraone, J., Fraser, N., Blott, M., & Leong, P. H. W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00452 · Cited by 134
Compressing Convolutional Neural Networks in the Frequency Domain
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939839 · Cited by 134
Self-citation
Regularization of deep neural networks with spectral dropout
Khan, S. H., Hayat, M., & Porikli, F. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.09.009 · Cited by 139
AGLNet: Towards real-time semantic segmentation of self-driving images via attention-guided lightweight network
Zhou, Q., Wang, Y., Fan, Y., Wu, X., Zhang, S., Kang, B., & Latecki, L. J. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106682 · Cited by 133
Network Pruning via Performance Maximization
Gao, S., Huang, F., Cai, W., & Huang, H. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00915 · Cited by 134
A survey on optimization techniques for edge artificial intelligence (ai)
Surianarayanan, C., Lawrence, J. J., Chelliah, P. R., Prakash, E., & Hewage, C. (2023) · Sensors · doi:10.3390/s23031279 · Cited by 131

Authors: Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y.
Affiliations: Washington University in St. Louis (US)
ORCID: Chen, Wenlin
Weinberger, Kilian Q.
Chen, Yixin
Venue: arXiv
Publisher: arXiv
Published: 2015
ISSN: 2331-8422 (ISSN-L)
DOI: 10.48550/arxiv.1504.04788
arXiv: 1504.04788
OpenAlex: W2172166488
Cited by: 974
References: 53
Citations by year: 2026: 2 · 2025: 9 · 2024: 29 · 2023: 59 · 2022: 69 · 2021: 118 · 2020: 167 · 2019: 159 · 2018: 184 · 2017: 114 · 2016: 44 · 2015: 19
Type: posted content
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords: neural network compression, parameter sharing, model size reduction, memory-efficient deep learning, mobile devices, parameter redundancy, hash functions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Compressing Neural Networks with the Hashing Trick

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks