An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
The high computation and memory storage of large deep neural networks (DNNs) models pose intensive challenges to the conventional Von-Neumann architecture, incurring sub-stantial data movements in the memory hierarchy. The memristor crossbar array has emerged as a promising solution to mitigate the challenges and enable low-power acceleration of DNNs. Memristor-based weight pruning and weight quantization have been seperately investigated and proven effectiveness in reducing area and power consumption compared to the original DNN model. However, there has been no systematic investigation of memristor-based neuromorphic computing (NC) systems considering both weight pruning and weight quantization. In this paper, we propose an unified and systematic memristor-based framework considering both structured weight pruning and weight quantization by incorporating alternating direction method of multipliers (ADMM) into DNNs training. We consider hardware constraints such as crossbar blocks pruning, conductance range, and mismatch between weight value and real devices, to achieve high accuracy and low power and small area footprint. Our framework is mainly integrated by three steps, i.e., memristor-based ADMM regularized optimization, masked mapping and retraining. Experimental results show that our proposed framework achieves 29.81× (20.88×) weight compression ratio, with 98.38% (96.96%) and 98.29% (97.47%) power and area reduction on VGG-16 (ResNet-18) network where only have 0.5% (0.76%) accuracy loss, compared to the original DNN models. We share our models at anonymous link http://bit.ly/2Jp5LHJ.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Memristor—The Missing Circuit Element
Chua, L. (1971) · IEEE Transactions on Circuit Theory · doi:10.1109/tct.1971.1083337 · Cited by 10,268 · cited 1×
“, memristor crossbar) can be utilized to perform matrixvector multiplication in the analog domain and solve systems of linear equations in O(1) time complexity [20, 21].”
MethodMemristive switching mechanism for metal/oxide/metal nanodevices
Yang, J. J., Pickett, M. D., Li, X., Ohlberg, D. A. A., Stewart, D. R., & Williams, R. S. (2008) · Nature Nanotechnology · doi:10.1038/nnano.2008.160 · Cited by 2,987 · cited 1×
“Applying an electric field across the device over time would lead to the migration of oxygen vacancies and change the memristance state, which is called state drift [28].”
BackgroundThe chips are down for Moore’s law
Waldrop, M. M. (2016) · Nature · doi:10.1038/530144a · Cited by 2,397
Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine
Hu, M., Graves, C. E., Li, C., Li, Y., Ge, N., Montgomery, E., Davila, N., Jiang, H., Williams, R. S., Yang, J. J., et al. (2018) · Advanced Materials · doi:10.1002/adma.201705914 · Cited by 751 · cited 1×
“To maintain accuracy, the single memristor crossbar size in our design is no larger than 128 × 64 [36] and is identical for all DNN layers.”
Training deep neural networks with low precision multiplications
Courbariaux, M., Bengio, Y., & David, J.-P. (2014) · arXiv · doi:10.48550/arxiv.1412.7024 · Cited by 407
Research works printed but not linked: 30
- 10.1145/3007787.3001140 · DOI · Paper page
- ouyang, 2013, Stochastic alternating direction method of multipliers, ICML, 80
- boyd, 2011, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends� in Machine Learning
- pi, 2013, Cross point arrays of 8 nm x 8 nm memristive devices fabricated with nanoimprint lithography, Journal of Vacuum Science & Technology B Microelectronics and Nanometer Structures
- 10.23919/DATE.2018.8342044 · DOI
- 10.1109/ISQED.2016.7479230 · DOI · Paper page
- han, 2015, Learning both weights and connections for efficient neural network, NeurIPS
- dong, 2012, Nvsim: A circuit-level performance, energy, and area model for emerging nonvolatile memory, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 31
- wen, 2016, Learning structured sparsity in deep neural networks, NeurIPS, 2074
- zhang, 2018, Adam-admm: A unified, systematic framework of structured weight pruning for dnns, arXiv preprint arXiv 1807 11091
- niu, 2019, 26ms inference time for resnet-50: Towards real-time execution of all dnns on smartphone, arXiv preprint arXiv 1905 00571
- 10.1109/CVPR.2017.761 · DOI · Paper page
- 10.1109/CVPR.2016.521 · DOI · Paper page
- 10.1145/3194554.3194625 · DOI
- wang, 2018, Towards ultra-high performance and energy efficiency of deep learning systems: an algorithm-hardware co-optimization framework, AAAI2018
- 10.1145/2228360.2228448 · DOI · Paper page
- shrestha, 2019, Approximating back-propagation for a biologically plausible local learning rule in spiking neural networks, ICONS, 10.1145/3354265.3354275 · doi:10.1145/3354265.3354275
- 10.1007/978-3-7091-6244-6_18 · doi:10.1007/978-3-7091-6244-6_18
- 10.1109/ISQED.2018.8357306 · DOI
- 10.1109/ISQED.2019.8697495 · DOI
- 10.1109/IJCNN.2018.8489619 · DOI
- 10.1145/3123939.3124552 · DOI · Paper page
- goodfellow, 2016, Deep Learning
- 10.1145/3299874.3319492 · DOI
- 10.1109/ICCAD.2017.8203823 · DOI · Paper page
- 10.1109/MWSCAS.2017.8053125 · DOI
- 10.1109/ICSENS.2014.6985125 · DOI
- 10.1145/3061639.3062256 · DOI
- 10.1109/MWSCAS.2010.5548670 · DOI
- 10.1109/NVMSA.2017.8064465 · DOI
Cited by
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
“However, the negative side is that these are rather closed pieces of software, which has been partially solved by Ma et al. 230 and Yuan et al. 231 , by using PyTorch instead of TensorFlow, focusing in this case on the weight pruning and quantization effects.”
SPViT: Enabling Faster Vision Transformers via Latency-Aware Soft Token Pruning
Kong, Z., Dong, P., Ma, X., Meng, X., Niu, W., Sun, M., Shen, X., Yuan, G., Ren, B., Tang, H., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20083-0_37 · Cited by 169
“However, when huge amount of AI-powered applications are benefiting from the network pruning advantages [63,51,53,99,98,30,77,17,56,100,44,26,25,27], the applications of selfattention-based neural network pruning remain scarce [32,74,43,81,61].”
BackgroundSelf-citationNaveen, S., & Kounte, M. R. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20523-1 · Cited by 33
Metadata
- Authors
- Yuan, G., Ma, X., Ding, C., Lin, S., Zhang, T., Jalali, Z. S., Zhao, Y., Jiang, L., Soundarajan, S., & Wang, Y.
- Venue
- 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED)
- Publisher
- IEEE
- Published
- 2019-07
- Pages
- 1-6
- OpenAlex
- W2971533524
- Cited by
- 58
- References
- 6
- FWCI
- 15.01
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM. https://app.cochat.ai/papers/10.1109/islped.2019.8824944
- The paper
- Yuan, G., Ma, X., Ding, C., Lin, S., Zhang, T., Jalali, Z. S., Zhao, Y., Jiang, L., Soundarajan, S., & Wang, Y. (2019). An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM. 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED). https://doi.org/10.1109/islped.2019.8824944
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.