Deep learning acceleration based on in-memory computing
The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Performing computations on conventional von Neumann computing systems results in a significant amount of data being moved back and forth between the physically separated memory and processing units. This costs time and energy, and constitutes an inherent performance bottleneck. In-memory computing is a novel non-von Neumann approach, where certain computational tasks are performed in the memory itself. This is enabled by the physical attributes and state dynamics of memory devices, in particular, resistance-based nonvolatile memory technology. Several computational tasks such as logical operations, arithmetic operations, and even certain machine learning tasks can be implemented in such a computational memory unit. In this article, we first introduce the general notion of in-memory computing and then focus on mixed-precision deep learning training with in-memory computing. The efficacy of this new approach will be demonstrated by training the MNIST multilayer perceptron network achieving high accuracy. Moreover, we show how the precision of in-memory computing can be further improved through architectural and device-level innovations. Finally, we present system aspects, such as high-level system architecture, including core-to-core interconnect technologies, and high-level ideas and concepts of the software stack.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
In-memory computing with resistive switching devices
Ielmini, D., & Wong, H.-S. P. (2018) · Nature Electronics · doi:10.1038/s41928-018-0092-2 · Cited by 2,298
‘Memristive’ switches enable ‘stateful’ logic operations via material implication
Borghetti, J., Snider, G. S., Kuekes, P. J., Yang, J. J., Stewart, D. R., & Williams, R. S. (2010) · Nature · doi:10.1038/nature08940 · Cited by 1,929
Deep Learning with Limited Numerical Precision
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015) · arXiv · doi:10.48550/arxiv.1502.02551 · Cited by 1,364
Equivalent-accuracy accelerated neural-network training using analogue memory
Ambrogio, S., Narayanan, P., Tsai, H., Shelby, R. M., Boybat, I., di Nolfo, C., Sidler, S., Giordano, M., Bodini, M., Farinha, N. C. P., et al. (2018) · Nature · doi:10.1038/s41586-018-0180-5 · Cited by 1,151
Self-citationBurr, G. W., Shelby, R. M., Sidler, S., di Nolfo, C., Jang, J., Boybat, I., Shenoy, R. S., Narayanan, P., Virwani, K., Giacometti, E. U., et al. (2015) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2015.2439635 · Cited by 936
Self-citationNeuromorphic computing with multi-memristive synapses
Boybat, I., Le Gallo, M., Nandakumar, S. R., Moraitis, T., Parnell, T., Tuma, T., Rajendran, B., Leblebici, Y., Sebastian, A., & Eleftheriou, E. (2018) · Nature Communications · doi:10.1038/s41467-018-04933-y · Cited by 899
Self-citationMemristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine
Hu, M., Graves, C. E., Li, C., Li, Y., Ge, N., Montgomery, E., Davila, N., Jiang, H., Williams, R. S., Yang, J. J., et al. (2018) · Advanced Materials · doi:10.1002/adma.201705914 · Cited by 751
Gokmen, T., & Vlasov, Y. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00333 · Cited by 491
Mixed-precision in-memory computing
Le Gallo, M., Sebastian, A., Mathis, R., Manica, M., Giefers, H., Tuma, T., Bekas, C., Curioni, A., & Eleftheriou, E. (2018) · Nature Electronics · doi:10.1038/s41928-018-0054-8 · Cited by 485
Self-citationRecent Progress in Phase-ChangeMemory Technology
Burr, G. W., BrightSky, M. J., Sebastian, A., Cheng, H.-Y., Wu, J.-Y., Kim, S., Sosa, N. E., Papandreou, N., Lung, H.-L., Pozidis, H., et al. (2016) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2016.2547718 · Cited by 393
Self-citationBiswas, A., & Chandrakasan, A. P. (2018) · 2018 IEEE International Solid - State Circuits Conference - (ISSCC) · doi:10.1109/isscc.2018.8310397 · Cited by 337
Temporal correlation detection using computational phase-change memory
Sebastian, A., Tuma, T., Papandreou, N., Le Gallo, M., Kull, L., Parnell, T., & Eleftheriou, E. (2017) · Nature Communications · doi:10.1038/s41467-017-01481-9 · Cited by 241
Self-citationHigh-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays
Merrikh-Bayat, F., Guo, X., Klachko, M., Prezioso, M., Likharev, K. K., & Strukov, D. B. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2778940 · Cited by 167
Compressed Sensing With Approximate Message Passing Using In-Memory Computing
Le Gallo, M., Sebastian, A., Cherubini, G., Giefers, H., & Eleftheriou, E. (2018) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2018.2865352 · Cited by 99
Self-citationBuddy-RAM: Improving the Performance and Efficiency of Bulk Bitwise Operations Using DRAM
Seshadri, V., Lee, D., Mullins, T., Hassan, H., Boroumand, A., Kim, J., Kozuch, M. A., Mutlu, O., Gibbons, P. B., & Mowry, T. C. (2016) · arXiv · doi:10.48550/arxiv.1611.09988 · Cited by 54
Inherent stochasticity in phase-change memory devices
Le Gallo, M., Tuma, T., Zipoli, F., Sebastian, A., & Eleftheriou, E. (2016) · 2016 46th European Solid-State Device Research Conference (ESSDERC) · doi:10.1109/essderc.2016.7599664 · Cited by 35
Self-citationPhase-Change Memory Models for Deep Learning Training and Inference
Nandakumar, S. R., Boybat, I., Joshi, V., Piveteau, C., Le Gallo, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. (2019) · 2019 26th IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs46596.2019.8964852 · Cited by 34
Self-citationCompressed sensing recovery using computational memory
Le Gallo, M., Sebastian, A., Cherubini, G., Giefers, H., & Eleftheriou, E. (2017) · 2017 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2017.8268469 · Cited by 28
Self-citationBoybat, I., Le Gallo, M., Moraitis, T., Leblebici, Y., Sebastian, A., & Eleftheriou, E. (2017) · 2017 13th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME) · doi:10.1109/prime.2017.7974095 · Cited by 20
Self-citation
Cited by
Memory devices and applications for in-memory computing
Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., & Eleftheriou, E. (2020) · Nature Nanotechnology · doi:10.1038/s41565-020-0655-z · Cited by 2,289
Self-citationAn overview of phase-change memory device physics
Le Gallo, M., & Sebastian, A. (2020) · Journal of Physics D: Applied Physics · doi:10.1088/1361-6463/ab7794 · Cited by 460
Self-citationMehonic, A., Sebastian, A., Rajendran, B., Simeone, O., Vasilaki, E., & Kenyon, A. J. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000085 · Cited by 331
Self-citationAmirsoleimani, A., Alibart, F., Yon, V., Xu, J., Pazhouhandeh, M. R., Ecoffey, S., Beilliard, Y., Genov, R., & Drouin, D. (2020) · Advanced Intelligent Systems · doi:10.1002/aisy.202000115 · Cited by 216
MERRC: A Memristor-Enabled Reconfigurable Low-Power Reservoir Computing Architecture at the Edge
Nowshin, F., Huang, Y., Sarkar, M. R., Xia, Q., & Yi, Y. (2024) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2023.3329337 · Cited by 12
Metadata
- Authors
- Eleftheriou, E., Gallo, M. L., Nandakumar, S. R., Piveteau, C., Boybat, I., Joshi, V., Khaddam-Aljameh, R., Dazzi, M., Giannopoulos, I., Karunaratne, G., Kersting, B., Stanisavljevic, M., Jonnalagadda, V. P., Ioannou, N., Kourtis, K., Francese, P. A., & Sebastian, A.
- Venue
- IBM Journal of Research and Development
- Publisher
- IBM
- Published
- 2019-11-01
- Volume
- 63
- Issue
- 6
- Pages
- 7:1-7:16
- ISSN
- 0018-8646 (ISSN-L)
- OpenAlex
- W2979706619
- Cited by
- 40
- References
- 22
- FWCI
- 1.67
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.