Analog architectures for neural network acceleration based on non-volatile memory
Open accessThe DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Abstract
Analog hardware accelerators, which perform computation within a dense memory array, have the potential to overcome the major bottlenecks faced by digital hardware for data-heavy workloads such as deep learning. Exploiting the intrinsic computational advantages of memory arrays, however, has proven to be challenging principally due to the overhead imposed by the peripheral circuitry and due to the non-ideal properties of memory devices that play the role of the synapse. We review the existing implementations of these accelerators for deep supervised learning, organizing our discussion around the different levels of the accelerator design hierarchy, with an emphasis on circuits and architecture. We explore and consolidate the various approaches that have been proposed to address the critical challenges faced by analog accelerators, for both neural network inference and training, and highlight the key design trade-offs underlying these techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215
Memristive devices for computing
Yang, J. J., Strukov, D. B., & Stewart, D. R. (2013) · Nature Nanotechnology · doi:10.1038/nnano.2012.240 · Cited by 3,885
Training and operation of an integrated neuromorphic network based on metal-oxide memristors
Prezioso, M., Merrikh-Bayat, F., Hoskins, B. D., Adam, G. C., Likharev, K. K., & Strukov, D. B. (2015) · Nature · doi:10.1038/nature14441 · Cited by 3,001
Fully hardware-implemented memristor convolutional neural network
Yao, P., Wu, H., Gao, B., Tang, J., Zhang, Q., Zhang, W., Yang, J. J., & Qian, H. (2020) · Nature · doi:10.1038/s41586-020-1942-4 · Cited by 2,179
Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364
Neuro-Inspired Computing With Emerging Nonvolatile Memorys
Yu, S. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2790840 · Cited by 1,170
Equivalent-accuracy accelerated neural-network training using analogue memory
Ambrogio, S., Narayanan, P., Tsai, H., Shelby, R. M., Boybat, I., di Nolfo, C., Sidler, S., Giordano, M., Bodini, M., Farinha, N. C. P., et al. (2018) · Nature · doi:10.1038/s41586-018-0180-5 · Cited by 1,151
EIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
Burr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936
Efficient and self-adaptive in-situ learning in multilayer memristor neural networks
Li, C., Belkin, D., Li, Y., Yan, P., Hu, M., Ge, N., Jiang, H., Montgomery, E., Lin, P., Wang, Z., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-04484-2 · Cited by 931
Face classification using electronic synapses
Yao, P., Wu, H., Gao, B., Eryilmaz, S. B., Huang, X., Zhang, W., Zhang, Q., Deng, N., Shi, L., Wong, H.-S. P., et al. (2017) · Nature Communications · doi:10.1038/ncomms15199 · Cited by 909
Neuromorphic computing with multi-memristive synapses
Boybat, I., Le Gallo, M., Nandakumar, S. R., Moraitis, T., Parnell, T., Tuma, T., Rajendran, B., Leblebici, Y., Sebastian, A., & Eleftheriou, E. (2018) · Nature Communications · doi:10.1038/s41467-018-04933-y · Cited by 899
ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.12 · Cited by 775
Parallel programming of an ionic floating-gate memory array for scalable neuromorphic computing
Fuller, E. J., Keene, S. T., Melianas, A., Wang, Z., Agarwal, S., Li, Y., Tuchman, Y., James, C. D., Marinella, M. J., Yang, J. J., et al. (2019) · Science · doi:10.1126/science.aaw5581 · Cited by 756
Self-citationMemristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine
Hu, M., Graves, C. E., Li, C., Li, Y., Ge, N., Montgomery, E., Davila, N., Jiang, H., Williams, R. S., Yang, J. J., et al. (2018) · Advanced Materials · doi:10.1002/adma.201705914 · Cited by 751
Large-scale deep unsupervised learning using graphics processors
Raina, R., Madhavan, A., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553486 · Cited by 742
A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations
Cai, F., Correll, J. M., Lee, S. H., Lim, Y., Bothra, V., Zhang, Z., Flynn, M. P., & Lu, W. D. (2019) · Nature Electronics · doi:10.1038/s41928-019-0270-x · Cited by 728
A review of emerging non-volatile memory (NVM) technologies and applications
Chen, A. (2016) · Solid-State Electronics · doi:10.1016/j.sse.2016.07.006 · Cited by 668
Pattern classification by memristive crossbar circuits using ex situ and in situ training
Alibart, F., Zamanidoost, E., & Strukov, D. B. (2013) · Nature Communications · doi:10.1038/ncomms3072 · Cited by 642
Ferroelectric FET analog synapse for acceleration of deep neural network training
Jerry, M., Chen, P.-Y., Zhang, J., Sharma, P., Ni, K., Yu, S., & Datta, S. (2017) · 2017 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2017.8268338 · Cited by 599
High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm
Alibart, F., Gao, L., Hoskins, B. D., & Strukov, D. B. (2012) · Nanotechnology · doi:10.1088/0957-4484/23/7/075201 · Cited by 557
Woo, J., Moon, K., Song, J., Lee, S., Kwak, M., Park, J., & Hwang, H. (2016) · IEEE Electron Device Letters · doi:10.1109/led.2016.2582859 · Cited by 512
Accurate deep neural network inference using computational phase-change memory
Joshi, V., Le Gallo, M., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. (2020) · Nature Communications · doi:10.1038/s41467-020-16108-9 · Cited by 490
Scaling for edge inference of deep neural networks
Xu, X., Ding, Y., Hu, S. X., Niemier, M., Cong, J., Hu, Y., & Shi, Y. (2018) · Nature Electronics · doi:10.1038/s41928-018-0059-3 · Cited by 462
Automated Systolic Array Architecture Synthesis for High Throughput CNN Inference on FPGAs
Wei, X., Yu, C. H., Zhang, P., Chen, Y., Wang, Y., Hu, H., Liang, Y., & Cong, J. (2017) · Proceedings of the 54th Annual Design Automation Conference 2017 · doi:10.1145/3061639.3062207 · Cited by 426
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltin, M., Williams, R. S., Faraboschi, P., Hwu, W.-M. W., Strachan, J. P., Roy, K., et al. (2019) · Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3297858.3304049 · Cited by 424
Recent Progress in Phase-ChangeMemory Technology
Burr, G. W., BrightSky, M. J., Sebastian, A., Cheng, H.-Y., Wu, J.-Y., Kim, S., Sosa, N. E., Papandreou, N., Lung, H.-L., Pozidis, H., et al. (2016) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2016.2547718 · Cited by 393
Bayat, F. M., Prezioso, M., Chakrabarti, B., Nili, H., Kataeva, I., & Strukov, D. (2018) · Nature Communications · doi:10.1038/s41467-018-04482-4 · Cited by 377
CNP: An FPGA-based processor for Convolutional Networks
Farabet, C., Poulet, C., Han, J. Y., & LeCun, Y. (2009) · 2009 International Conference on Field Programmable Logic and Applications · doi:10.1109/fpl.2009.5272559 · Cited by 367
A single-transistor silicon synapse
Diorio, C., Hasler, P., Minch, A., & Mead, C. A. (1996) · IEEE Transactions on Electron Devices · doi:10.1109/16.543035 · Cited by 360
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343
Suri, M., Bichler, O., Querlioz, D., Cueto, O., Perniola, L., Sousa, V., Vuillaume, D., Gamrat, C., & DeSalvo, B. (2011) · 2011 International Electron Devices Meeting · doi:10.1109/iedm.2011.6131488 · Cited by 314
XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks
Sun, X., Yin, S., Peng, X., Liu, R., Seo, J.-S., & Yu, S. (2018) · 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE) · doi:10.23919/date.2018.8342235 · Cited by 246
Accelerator-friendly neural-network training: Learning variations and defects in RRAM crossbar
Chen, L., Li, J., Chen, Y., Deng, Q., Shen, J., Liang, X., & Jiang, L. (2017) · Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017 · doi:10.23919/date.2017.7926952 · Cited by 245
How to Train a Compact Binary Neural Network with High Accuracy?
Tang, W., Hua, G., & Wang, L. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10862 · Cited by 241
Saberi, M., Lotfi, R., Mafinezhad, K., & Serdijn, W. A. (2011) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2011.2107214 · Cited by 236
3-D Memristor Crossbars for Analog and Neuromorphic Computing Applications
Adam, G. C., Hoskins, B. D., Prezioso, M., Merrikh-Bayat, F., Chakrabarti, B., & Strukov, D. B. (2017) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2016.2630925 · Cited by 234
Binary neural network with 16 Mb RRAM macro chip for classification and online training
Yu, S., Li, Z., Chen, P.-Y., Wu, H., Gao, B., Wang, D., Wu, W., & Qian, H. (2016) · 2016 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2016.7838429 · Cited by 219
High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays
Merrikh-Bayat, F., Guo, X., Klachko, M., Prezioso, M., Likharev, K. K., & Strukov, D. B. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2778940 · Cited by 167
An analog neural network processor with programmable topology
Boser, B. E., Sackinger, E., Bromley, J., Le Cun, Y., & Jackel, L. D. (1991) · IEEE Journal of Solid-State Circuits · doi:10.1109/4.104196 · Cited by 142
Three-Dimensional nand Flash for Vector–Matrix Multiplication
Wang, P., Xu, F., Wang, B., Gao, B., Wu, H., Qian, H., & Yu, S. (2019) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2882194 · Cited by 132
Ambrogio, S., Gallot, M., Spoon, K., Tsai, H., Mackin, C., Wesson, M., Kariyappa, S., Narayanan, P., Liu, C.-C., Kumar, A., et al. (2019) · 2019 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm19573.2019.8993482 · Cited by 76
An Energy-Efficient Digital ReRAM-Crossbar-Based CNN With Bitwise Parallelism
Ni, L., Liu, Z., Yu, H., & Joshi, R. V. (2017) · IEEE Journal on Exploratory Solid-State Computational Devices and Circuits · doi:10.1109/jxcdc.2017.2697910 · Cited by 74
Design of Ternary Neural Network With 3-D Vertical RRAM Array
Li, Z., Chen, P.-Y., Xu, H., & Yu, S. (2017) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2017.2697361 · Cited by 70
Design Considerations for Efficient Deep Neural Networks on Processing-in-Memory Accelerators
Yang, T.-J., & Sze, V. (2019) · 2019 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm19573.2019.8993662 · Cited by 55
Negrov, D., Karandashev, I., Shakirov, V., Matveyev, Y., Dunin-Barkowski, W., & Zenkevich, A. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.10.061 · Cited by 49
Tsai, H., Ambrogio, S., Mackin, C., Narayanan, P., Shelby, R. M., Rocki, K., Chen, A., & Burr, G. W. (2019) · 2019 Symposium on VLSI Technology · doi:10.23919/vlsit.2019.8776519 · Cited by 47
Accelerating machine learning with Non-Volatile Memory: Exploring device and circuit tradeoffs
Fumarola, A., Narayanan, P., Sanches, L. L., Sidler, S., Jang, J., Moon, K., Shelby, R. M., Hwang, H., & Burr, G. W. (2016) · 2016 IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc.2016.7738684 · Cited by 37
Cited by
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
Le Gallo, M., Khaddam-Aljameh, R., Stanisavljevic, M., Vasilopoulos, A., Kersting, B., Dazzi, M., Karunaratne, G., Brändli, M., Singh, A., Müller, S. M., et al. (2023) · Nature Electronics · doi:10.1038/s41928-023-01010-1 · Cited by 310
On the Accuracy of Analog Neural Network Inference Accelerators
Xiao, T. P., Feinberg, B., Bennett, C. H., Prabhakar, V., Saxena, P., Agrawal, V., Agarwal, S., & Marinella, M. J. (2022) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2022.3214409 · Cited by 76
Self-citation
Metadata
- Authors
- Xiao, T. P., Bennett, C. H., Feinberg, B., Agarwal, S., & Marinella, M. J.
- Venue
- Applied Physics Reviews
- Publisher
- AIP Publishing
- Published
- 2020-07-09
- Volume
- 7
- Issue
- 3
- ISSN
- 1931-9401 (ISSN-L)
- OpenAlex
- W3041897167
- Cited by
- 193
- References
- 55
- FWCI
- 8.95
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.