Strategies to Improve the Accuracy of Memristor-Based Convolutional Neural Networks
The DOI matches its Crossref record; 40 of the 41 references it cites were found in a registry or an index.
Abstract
In this article, we quantify several nonideal characteristics of memristor synaptic devices, such as the limited conductance states, write nonlinearities, and variations, and comprehensively investigate their effects on the convolutional neural network (CNN) performance. Our result shows that the available conductance states ( ${N}_{\text {state}}$ ), asymmetric write nonlinearities, and cycle-to-cycle (C2C) variation are critical factors to the learning accuracy, while symmetric write nonlinearities and device-to-device variation go trivial. We accordingly propose three strategies to mitigate their impacts on CNN performance: 1) limiting the weight range to improve the utilization of ${N}_{\text {state}}$ ; 2) adopting a new “with-read” update scheme to mitigate the effects of asymmetric write nonlinearities; and 3) employing multiple memristors for each kernel element to alleviate the impact of C2C variation. Our work would provide guidance for the hardware implementation and optimization of CNN in memristor crossbar.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, most of those studies in fact only measured the figures-of-merit of one/few devices and simulated the accuracy of an ANN via software62–67 in such type of studies the connection between the memristors fabricated and the ANN is relatively weak.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [66]
In the citing paper
“some articles focused on the fabrication and device-level characterization of one/few memristors62–70,186,187, also present some of the figures-of-merit generated by a software-based training ANN process (similar to the ones in Fig. 14) in order to claim that their devices exhibit potential for neuromorphic applications.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [66]
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 2×
“The rectified linear unit (ReLU) activation function and stochastic gradient descent algorithm are applied [23].”
BackgroundMethodRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Fukushima, K. (1980) · Biological Cybernetics · doi:10.1007/bf00344251 · Cited by 5,759 · cited 1×
“for neuromorphic computing have been demonstrated in recent years, such as multilayer perception (MLP) network [1], convolutional neural network (CNN) [5], spiking neural network (SNN) [6], and long short-term memory (LSTM) network [7].”
BackgroundDeep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503 · cited 1×
“image recognition, classification, and detection tasks [8].”
Training and operation of an integrated neuromorphic network based on metal-oxide memristors
Prezioso, M., Merrikh-Bayat, F., Hoskins, B. D., Adam, G. C., Likharev, K. K., & Strukov, D. B. (2015) · Nature · doi:10.1038/nature14441 · Cited by 3,001
Analogue signal and image processing with large memristor crossbars
Li, C., Hu, M., Li, Y., Jiang, H., Ge, N., Montgomery, E., Zhang, J., Song, W., Dávila, N., Graves, C. E., et al. (2017) · Nature Electronics · doi:10.1038/s41928-017-0002-z · Cited by 1,292 · cited 1×
“2(a)] [27], [28].”
Self-citationBurr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936
Face classification using electronic synapses
Yao, P., Wu, H., Gao, B., Eryilmaz, S. B., Huang, X., Zhang, W., Zhang, Q., Deng, N., Shi, L., Wong, H.-S. P., et al. (2017) · Nature Communications · doi:10.1038/ncomms15199 · Cited by 909 · cited 6×
“Here, the conductance range was normalized to [0, 1], therefore, the initial values of W+ and W− are randomly distributed from 0 to 1 (the range of W is [−1, 1]).”
BackgroundInfluentialMemristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine
Hu, M., Graves, C. E., Li, C., Li, Y., Ge, N., Montgomery, E., Davila, N., Jiang, H., Williams, R. S., Yang, J. J., et al. (2018) · Advanced Materials · doi:10.1002/adma.201705914 · Cited by 751 · cited 1×
“2(a)] [27], [28].”
Self-citationFerroelectric FET analog synapse for acceleration of deep neural network training
Jerry, M., Chen, P.-Y., Zhang, J., Sharma, P., Ni, K., Yu, S., & Datta, S. (2017) · 2017 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2017.8268338 · Cited by 599
Yu, S., Gao, B., Fang, Z., Yu, H., Kang, J., & Wong, H. ‐. P. (2013) · Advanced Materials · doi:10.1002/adma.201203680 · Cited by 527
Chen, P.-Y., Peng, X., & Yu, S. (2017) · 2017 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2017.8268337 · Cited by 435
Long short-term memory networks in memristor crossbar arrays
Li, C., Wang, Z., Rao, M., Belkin, D., Song, W., Jiang, H., Yan, P., Li, Y., Lin, P., Hu, M., et al. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0001-4 · Cited by 429 · cited 2×
“for neuromorphic computing have been demonstrated in recent years, such as multilayer perception (MLP) network [1], convolutional neural network (CNN) [5], spiking neural network (SNN) [6], and long short-term memory (LSTM) network [7].”
BackgroundSelf-citationImmunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices
Querlioz, D., Bichler, O., Dollfus, P., & Gamrat, C. (2013) · IEEE Transactions on Nanotechnology · doi:10.1109/tnano.2013.2250995 · Cited by 420
Reinforcement learning with analogue memristor arrays
Wang, Z., Li, C., Song, W., Rao, M., Belkin, D., Li, Y., Yan, P., Jiang, H., Lin, P., Hu, M., et al. (2019) · Nature Electronics · doi:10.1038/s41928-019-0221-6 · Cited by 370
Self-citationOptimization of Conductance Change in Pr1–xCaxMnO3-Based Synaptic Devices for Neuromorphic Systems
Jang, J.-W., Park, S., Burr, G. W., Hwang, H., & Jeong, Y.-H. (2015) · IEEE Electron Device Letters · doi:10.1109/led.2015.2418342 · Cited by 310
Improving Analog Switching in HfO x -Based Resistive Memory With a Thermal Enhanced Layer
Wu, W., Wu, H., Gao, B., Deng, N., Yu, S., & Qian, H. (2017) · IEEE Electron Device Letters · doi:10.1109/led.2017.2719161 · Cited by 282
Ambrogio, S., Balatti, S., Milo, V., Carboni, R., Wang, Z.-Q., Calderoni, A., Ramaswamy, N., & Ielmini, D. (2016) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2016.2526647 · Cited by 256
YodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration
Andri, R., Cavigelli, L., Rossi, D., & Benini, L. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2017.2682138 · Cited by 247
3-D Memristor Crossbars for Analog and Neuromorphic Computing Applications
Adam, G. C., Hoskins, B. D., Prezioso, M., Merrikh-Bayat, F., Chakrabarti, B., & Strukov, D. B. (2017) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2016.2630925 · Cited by 234 · cited 1×
“Moreover, the strategies to mitigate the effects of nonideal factors are proposed in terms of programming schemes and device optimization by previous works [18]–[21].”
MethodDesign implications of memristor-based RRAM cross-point structures
Xu, C., Dong, X., Jouppi, N. P., & Xie, Y. (2011) · 2011 Design, Automation & Test in Europe · doi:10.1109/date.2011.5763125 · Cited by 232
Mitigating effects of non-ideal synaptic device characteristics for on-chip learning
Chen, P.-Y., Lin, B., Wang, I.-T., Hou, T.-H., Ye, J., Vrudhula, S., Seo, J.-S., Cao, Y., & Yu, S. (2015) · 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) · doi:10.1109/iccad.2015.7372570 · Cited by 232
Recent progress in analog memory-based accelerators for deep learning
Tsai, H., Ambrogio, S., Narayanan, P., Shelby, R. M., & Burr, G. W. (2018) · Journal of Physics D: Applied Physics · doi:10.1088/1361-6463/aac8a5 · Cited by 225
Resistive memory device requirements for a neural algorithm accelerator
Agarwal, S., Plimpton, S. J., Hughart, D. R., Hsia, A. H., Richter, I., Cox, J. A., James, C. D., & Marinella, M. J. (2016) · 2016 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727298 · Cited by 213
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices
Gokmen, T., Onen, M., & Haensch, W. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00538 · Cited by 162
Demonstration of Convolution Kernel Operation on Resistive Cross-Point Array
Gao, L., Chen, P.-Y., & Yu, S. (2016) · IEEE Electron Device Letters · doi:10.1109/led.2016.2573140 · Cited by 146
Memristor crossbar deep network implementation based on a Convolutional neural network
Yakopcic, C., Alom, M. Z., & Taha, T. M. (2016) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2016.7727302 · Cited by 134
Signal and noise extraction from analog memory elements for neuromorphic computing
Gong, N., Idé, T., Kim, S., Boybat, I., Sebastian, A., Narayanan, V., & Ando, T. (2018) · Nature Communications · doi:10.1038/s41467-018-04485-1 · Cited by 109
Spike-timing dependent plasticity in a transistor-selected resistive switching memory
Ambrogio, S., Balatti, S., Nardi, F., Facchinetti, S., & Ielmini, D. (2013) · Nanotechnology · doi:10.1088/0957-4484/24/38/384012 · Cited by 107
Agarwal, S., Quach, T.-T., Parekh, O., Hsia, A. H., DeBenedictis, E. P., James, C. D., Marinella, M. J., & Aimone, J. B. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00484 · Cited by 96
Wang, Z., Ambrogio, S., Balatti, S., & Ielmini, D. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00438 · Cited by 93
Device and circuit optimization of RRAM for neuromorphic computing
Wu, H., Yao, P., Gao, B., Wu, W., Zhang, Q., Zhang, W., Deng, N., Wu, D., Wong, H.-S. P., Yu, S., et al. (2017) · 2017 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2017.8268372 · Cited by 86
Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing
Lin, Y.-H., Wang, C.-H., Lee, M.-H., Lee, D.-Y., Lin, Y.-Y., Lee, F.-M., Lung, H.-L., Wang, K.-C., Tseng, T.-Y., & Lu, C.-Y. (2019) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2019.2894273 · Cited by 75
LiSiOX-Based Analog Memristive Synapse for Neuromorphic Computing
Chen, J., Lin, C.-Y., Li, Y., Qin, C., Lu, K., Wang, J., Chen, C.-K., He, Y., Chang, T., Sze, S. M., et al. (2019) · IEEE Electron Device Letters · doi:10.1109/led.2019.2898443 · Cited by 71
Self-citationImproved Conductance Linearity and Conductance Ratio of 1T2R Synapse Device for Neuromorphic Systems
Moon, K., Kwak, M., Park, J., Lee, D., & Hwang, H. (2017) · IEEE Electron Device Letters · doi:10.1109/led.2017.2721638 · Cited by 70
Garbin, D., Bichler, O., Vianello, E., Rafhay, Q., Gamrat, C., Perniola, L., Ghibaudo, G., & DeSalvo, B. (2014) · 2014 IEEE International Electron Devices Meeting · doi:10.1109/iedm.2014.7047126 · Cited by 66
Wang, J., Hu, S., Zhan, X., Yu, Q., Liu, Z., Chen, T. P., Yin, Y., Hosaka, S., & Liu, Y. (2018) · Scientific Reports · doi:10.1038/s41598-018-30768-0 · Cited by 55
A Novel Read Scheme for Large Size One-Resistor Resistive Random Access Memory Array
Zackriya, V. M., Kittur, H. M., & Chin, A. (2017) · Scientific Reports · doi:10.1038/srep42375 · Cited by 50
Chang, C.-C., Liu, J., Shen, Y.-L., Chou, T., Chen, P.-C., Wang, I., Su, C.-C., Wu, M.-H., Hudec, B., Chang, C., et al. (2017) · Technical digest - International Electron Devices Meeting/Technical digest · doi:10.1109/iedm.2017.8268373 · Cited by 38
Self-citationVoltage-controlled reverse filament growth boosts resistive switching memory
Belmonte, A., Celano, U., Chen, Z., Radhaskrishnan, J., Redolfi, A., Clima, S., Richard, O., Bender, H., Kar, G. S., Vandervorst, W., et al. (2018) · Nano Research · doi:10.1007/s12274-018-1983-2 · Cited by 25
Cited by
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
A review of memristor: material and structure design, device performance, applications and prospects
Xiao, Y., Jiang, B., Zhang, Z., Ke, S., Jin, Y., Wen, X., & Ye, C. (2023) · Science and Technology of Advanced Materials · doi:10.1080/14686996.2022.2162323 · Cited by 295
“ing in the switching layer [177], design double layer structure [178,179] have been taken by some academics.”
BackgroundAmirsoleimani, A., Alibart, F., Yon, V., Xu, J., Pazhouhandeh, M. R., Ecoffey, S., Beilliard, Y., Genov, R., & Drouin, D. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000115 · Cited by 216
Multiply accumulate operations in memristor crossbar arrays for analog computing
Chen, J., Li, J., Li, Y., & Miao, X. (2021) · Journal of Semiconductors · doi:10.1088/1674-4926/42/1/013104 · Cited by 109
Self-citationTeixeira, H., Dias, C., Silva, A., & Ventura, J. O. (2024) · ACS Nano · doi:10.1021/acsnano.4c03264 · Cited by 75
Metadata
- Authors
- Pan, W.-Q., Chen, J., Kuang, R., Li, Y., He, Y.-H., Feng, G.-R., Duan, N., Chang, T.-C., & Miao, X.-S.
- Affiliations
- Wuhan National Laboratory for Optoelectronics (CN)
- National Sun Yat-sen University (TW)
- Venue
- IEEE Transactions on Electron Devices
- Published
- 2020-03
- Volume
- 67
- Issue
- 3
- Pages
- 895-901
- ISSN
- 0018-9383 (ISSN-L)
- 1557-9646
- OpenAlex
- W3001813906
- Cited by
- 76
- References
- 41
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 17 · 2023: 17 · 2022: 17 · 2021: 9 · 2020: 5
- FWCI
- 4.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Electron Devices (publishedVersion)
- Funders
- National Key Research and Development Plan of Ministry of Science and Technology (MOST) of China (2016YFA0203800)
- National Natural Science Foundation of China (61841404, 51732003, 61674061, 61774067)
- National Defense Advanced Research Program (31513040304)
- Major Special Projects for Technological Innovation of Hubei Province (2018AAA038)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Ferroelectric and Negative Capacitance Devices
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- memristors, convolutional neural networks, memristor crossbar, artificial synapses, conductance states, cycle-to-cycle variation, device-to-device variation, learning accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.