The DOI matches its DataCite record; 51 of the 53 references it cites were found in a registry or an index.
Compressing Neural Networks with the Hashing Trick
Open accessAbstract
As deep nets are increasingly used in applications suited for mobile devices, a fundamental dilemma becomes apparent: the trend in deep learning is to grow models to absorb ever-increasing data set sizes; however mobile devices are designed with very little memory and cannot store such large models. We present a novel network architecture, HashedNets, that exploits inherent redundancy in neural networks to achieve drastic reductions in model sizes. HashedNets uses a low-cost hash function to randomly group connection weights into hash buckets, and all connections within the same hash bucket share a single parameter value. These parameters are tuned to adjust to the HashedNets weight sharing architecture with standard backprop during training. Our hashing procedure introduces no additional memory overhead, and we demonstrate on several benchmark data sets that HashedNets shrink the storage requirements of neural networks substantially while mostly preserving generalization performance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, structured update, sketched update can help enhance the communication efficiency when clients uploading updates to the server as well.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [202]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
(2014) · Cited by 33,419
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Neural networks for pattern recognition
Bishop, C. (1994) · Choice Reviews Online · doi:10.5860/choice.31-5500 · Cited by 18,537
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
(2010) · International Conference on Machine Learning · Cited by 13,995
Term-Weighting Approaches in Automatic Text Retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 11,421
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Cited by
Sorted by citations
Items 901–973 of 973
C3-FIow: Compute Compression Co-Design FIow for Deep Neural Networks
Sotoudeh, M., & Baghsorkhi, S. S. (2019)
Dynamic Unit Surgery for Deep Neural Network Compression and Acceleration
Kim, M., & Kwok, J. T.-Y. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852368
A Comparative Study of Neural Network CompressionPreprint
Baktash, H., Natale, E., & Viennot, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11144
Deep Convolutional Neural Networks Based on Image Data Augmentation for Visual Object Recognition
Jayech, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33607-3_51
Radius Adaptive Convolutional Neural NetworkPreprint
Rakhshanfar, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11079
Anchor & Transform: Learning Sparse Representations of Discrete ObjectsPreprint
Liang, P. P., Zaheer, M., Wang, Y., & Ahmed, A. A. (2019) · arXiv (Cornell University)
An Improving Framework of regularization for Network CompressionPreprint
Zhenqian, E., & Weiguo, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.05078
VLSI Implementation of Area-Efficient Parallelized Neural Network Accelerator Using Hashing Trick
Yoo, T. K., Park, J. K., & Kim, J. T. (2019) · International SoC Design Conference · doi:10.1109/isocc47750.2019.9027759
Design of multicycle path accelerator for neural network
Jo, C.-W., & Lee, K.-Y. (2019) · International SoC Design Conference · doi:10.1109/isocc47750.2019.9078500
Conditional Neural Architecture SearchPreprint
Kao, S.-C., Ramamurthy, A., Williams, R., & Krishna, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03969
Neural Network Pruning for ECG Arrhythmia Classification
Labarge, I. E. (2020) · doi:10.15368/theses.2020.76
ThriftyNets: Convolutional Neural Networks with Tiny Parameter Budget
Coiffier, G., Hacene, G. B., & Gripon, V. (2021) · IoT · doi:10.3390/iot2020012
Artificial Intelligence Inference in Edge
Wang, X., Han, Y., Leung, V. C. M., Niyato, D. T., Yan, X., & Chen, X. (2020) · Springer eBooks · doi:10.1007/978-981-15-6186-3_5
The Performance of Johnson-Lindenstrauss Transforms: Beyond the Classical Framework
Jagadeesan, M. (2020) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)
Ahmed, F., Mahmud, M. S., Moinuddin, K. A., Hyder, M. I., & Yeasin, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.12877
Pushkareva, M. M., & Karandashev, I. (2020) · Optical Memory and Neural Networks · doi:10.3103/s1060992x20030042
Be Your Own Best Competitor! Multi-Branched Adversarial Knowledge TransferPreprint
Ghorbani, M., Fooladgar, F., & Shohreh Kasaei, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04516
Continual learning using hash-routed convolutional neural networksPreprint
Berjaoui, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05880
Sensitiveness Based Strategy for Network Compression
Lu, Y., Feng, K., Li, H., Wu, Y., Gong, M., & Xu, Y. (2020) · International Conference on Industrial Artificial Intelligence (IAI) · doi:10.1109/iai50351.2020.9262195
Re-Training and Parameter Sharing with the Hash Trick for Compressing Convolutional Neural Networks
Gou, X., Qing, L., Wang, Y., & Xin, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62223-7_35
An Automatic Clustering Algorithm Leveraging Reinforcement Learning for Model Compression
Wang, W., Wu, F., & Zhao, J. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-70665-4_38
Identifying and Exploiting Structures for Reliable Deep LearningPreprint
Sanyal, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07083
A Practical Online Framework for Extracting Running Video Summaries under a Fixed Memory Budget
Lavania, C., Wei, K., Iyer, R., & Bilmes, J. (2021) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611976700.26
Deep Clustering and Deep Network Compression
Alqahtani, A. (2021) · Cronfa (Swansea University) · doi:10.23889/suthesis.57269
Empirical Study of Data-Free Iterative Knowledge Distillation
Shah, H., Vaswani, A., Dash, T., Hebbalaguppe, R., & Srinivasan, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86365-4_44
ES-ENAS: Efficient Evolutionary Optimization for Large Hybrid Search SpacesPreprint
Song, X., Choromański, K., Parker-Holder, J., Tang, Y., Zhang, Q., Peng, D., Jain, D., Gao, W., Pacchiano, A., Sarlós, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.07415 · Cited by 1
Student Helping Teacher: Teacher Evolution via Self-Knowledge DistillationPreprint
Li, Z., Li, X., Yang, L., Yang, J., & Pan, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00329 · Cited by 1
Robustness of Compressed Deep Neural Networks with Adversarial Training
Zhang, Y., Li, C., Wang, W., Yuting, Z., Zhang, X., & Yulin, Z. (2021) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa52424.2021.9555552
Effective Deep Learning Based Architecture for Pedestrian Detection from Digital Images
Gill, K., & Mangat, V. (2020) · International Journal of Engineering and Advanced Technology · doi:10.35940/ijeat.b4225.029320
Xu, J., Chen, X., Hu, S., Yu, J., Liu, X., & Meng, H. M. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.14836
Feature Hashing with Insertion and Deletion of Features
Pratap, R., Bhardwaj, S., Sarode, H. S., & Kulkarni, R. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9672011
Compact Deep Neural Networks via Soft and Smooth Filter Pruning
Fu, Z., Wang, X., Li, W., & Zeng, Z. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220017
Light Face: A Light Face Detector for Edge Devices
Khanehgir, S., Ghoreyshi, A. M., Akbari, A., Derakhshan, R., & Sabokrou, M. (2022) · Iranian Conference on Machine Vision and Image Processing · doi:10.1109/mvip53647.2022.9738740
Algorithm and hardware codesign of sparse binary network on-chip
Zhou, X., Liu, H., Shi, C., & Liu, J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-32-385783-3.00016-8
PREDICTIVE ANALYSIS OF NETWORK TRAFFIC RESOURCES UNDER CLOUD COMPUTING BASED ON NEURAL NETWORK
(2019) · International Journal of Mechatronics and Applied Mechanics · doi:10.17683/ijomam/issue6.19
A Filter Rank Based Pruning Method for Convolutional Neural Networks
Liu, H., Guan, Z., & Lei, P. (2021) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom53373.2021.00182
A systematic study on benchmarking AI inference accelerators
Jiang, Z., Li, J., Liu, F., Gao, W., Wang, L., Lan, C., Tang, F., Liu, L., & Li, T. (2022) · CCF Transactions on High Performance Computing · doi:10.1007/s42514-022-00105-z
Zhang, Y., Muscoloni, A., & Cannistraci, C. V. (2022) · Preprints.org · doi:10.20944/preprints202207.0139.v1
Low Bit Neural Networks with Channel Sparsity and Sharing
Long, X., Zeng, X., Liu, Y., & Qiao, M. (2022) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc55077.2022.9885798
Building Compact and Robust Deep Neural Networks with Toeplitz Matrices
Araujo, A. (2021) · theses.fr (ABES) · doi:10.70675/581a32b0zfa99z457ezab3fzd0f62f6fc8d8 · Cited by 1
Structured Compression of Convolutional Neural Networks for Specialized Tasks
Gabbay, F., Salomon, B. G., & Shomron, G. (2022) · Mathematics · doi:10.3390/math10193679
NeuroFabric: Hardware and ML Model Co-Design for A Priori Sparse Neural Network Training
Isakov, M., & Kinsy, M. A. (2022) · IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD) · doi:10.1109/iccd56317.2022.00088
A Channel-level Neural Network Compression Method Based on K-order Statistics
Cao, Y., Zhao, H., Xu, Z., Yu, H., Long, K., & Xu, W. (2022) · International Conference on Information Technology in Medicine and Education · doi:10.1109/itme56794.2022.00034
Efficient and Scalable Object Localization in 3D on Mobile Device
Gupta, N., & Khan, N. (2023) · doi:10.32920/22734395
Efficient and Scalable Object Localization in 3D on Mobile Device
Gupta, N., & Khan, N. (2023) · doi:10.32920/22734395.v1
Unbiased estimation of inner product via higher order count sketch
Verma, B. D., Pratap, R., & Thakur, M. (2023) · Information Processing Letters · doi:10.1016/j.ipl.2023.106407
Time-Reduced Model for Multilayer Spiking Neural Networks
Li, Y. (2023) · International Journal of Systems Engineering · doi:10.11648/j.ijse.20230701.11
Efficient Algorithms for Artificial Neural Networks and Explainable AI
Eshkiki, H. (2023) · Cronfa (Swansea University) · doi:10.23889/suthesis.63943
Kernel-wise difference minimization for convolutional neural network compression in metaverse
Chang, Y.-T. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2023.1200382
Deep Learning Model Optimization
Ren, J., & Xia, D. T. (2023) · Springer eBooks · doi:10.1007/978-981-99-2897-2_8
A Sequence Model Compression Method Based on Proper Orthogonal Decomposition
Xue, J., Wu, D., & Liu, Y. (2023) · International Conference on Computer Science and Blockchain (CCSB) · doi:10.1109/ccsb60789.2023.10398824
Information Fusion Channel Pruning for Accelerating Neural Networks
Zhu, Z., Zhai, Y., & Zhang, Y. (2023) · International Conference on Big Data Computing and Communications (BigCom) · doi:10.1109/bigcom61073.2023.00040
Filter pruning via schatten p-norm
Lin, Y., Bian, J., Lai, J., & Xie, X. (2024) · International Conference on Graphics and Image Processing (ICGIP) · doi:10.1117/12.3021622
Scalable Transmission of Convolutional Neural Network Weights in the Frequency Domain
Li, F., Xu, N., & Chen, X. (2023) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc59590.2023.10507320
A Feature Map Lossless Compression Framework for Convolutional Neural Network Accelerators
Zhang, Z., Jiao, X., & Xu, C. (2024) · IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas59952.2024.10595980
Kovalov, O. (2024) · Kharkiv Surgical School · doi:10.37699/2308-7005.4.2024.21
Pruning and Neural Architectures Redesigning for Deep Neural Networks Compression in Mobiles
Ferwana, I., Chaffar, S., & Belhouari, S. B. (2024) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-6684-3795-7.ch005
Human Intelligent-things Interaction Application Using 6G and Deep Edge LearningPreprint
Kedwan, F., & Rahman, M. A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5070397
Pruning CNN based on Combinational Filter Deletion
Huang, W., Wang, X., Zhao, Z., Su, L., Sui, S., & Wang, J. (2024) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon55916.2024.10905786
Development of Neuron Processing for DCNN with Weight and Activation for IoT
Vybhavi, G. Y., Sultana, A., Preethi, C., Vengal, D., Alabdeli, H. M., & S, S. C. (2025) · International Conference on Intelligent Control, Computing and Communications (IC3) · doi:10.1109/ic363308.2025.10956980
Xu, L., & Shi, W. (2025) · Wiley eBooks · doi:10.1002/9781394285860.ch4
Kozina, A., & Pikus, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-10202-7_28
IoU-based overlapping metric and aggregated loss function for morphological-linear neural networks
Hernández, G., Rodríguez-Molina, A., Gómez, E. Z., Rojas-López, A. G., & Villarreal-Cervantes, M. G. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3797
Deep Roots: Improving Cnn Efficiency With Hierarchical Filter Groups: Cvpr 2017 Poster
Ioannou, Y. (2017) · Figshare · doi:10.5281/zenodo.831417
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
Mishra, A., & Marr, D. (2017) · arXiv · doi:10.48550/arxiv.1711.05852 · Cited by 270
Scalable Methods for 8-bit Training of Neural Networks
Banner, R., Hubara, I., Hoffer, E., & Soudry, D. (2018) · arXiv · doi:10.48550/arxiv.1805.11046 · Cited by 193
On-device machine learning An algorithms and learning theory perspective
Dhar, S., Guo, J., Liu, J., Tripathi, S., Kurup, U., & Shah, M. (2019) · arXiv · doi:10.48550/arxiv.1911.00623 · Cited by 47
Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression
Wang, D., Zhou, L., Zhang, X., Bai, X., & Zhou, J. (2018) · arXiv · doi:10.48550/arxiv.1803.05729 · Cited by 42
Metadata
- Authors
- Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y.
- Affiliations
- Washington University in St. Louis (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1504.04788
- OpenAlex
- W2172166488
- Cited by
- 974
- References
- 53
- Citations by year
- 2026: 2 · 2025: 9 · 2024: 29 · 2023: 59 · 2022: 69 · 2021: 118 · 2020: 167 · 2019: 159 · 2018: 184 · 2017: 114 · 2016: 44 · 2015: 19
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Keywords
- neural network compression, parameter sharing, model size reduction, memory-efficient deep learning, mobile devices, parameter redundancy, hash functions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
