The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
The marriage of training and inference for scaled deep learning analog hardware
Abstract
Resistive crossbar arrays are promising candidates for efficient execution of deep neural network (DNN) inference workloads. The weight matrices of a neural network are mapped to the conductance values on crossbar arrays and then used as vector-matrix multiply engines. Although this mapping seems straightforward, we show that for large scale DNNs the weights must come from a training procedure that accounts for hardware induced constraints, such as ADC, DAC, noise and device fails, for the inference task to run successfully on analog hardware composed of crossbar arrays.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Noise-aware training-simple injection of noise onto weights or activations during DNN training-can make DNNs more robust for AIMC deployment.
“A number of previous studies have shown that noise-aware training —simple injection of noise onto weights or activations during DNN training—can make DNNs more robust for AIMC deployment33,38–42.”
From Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators (2023) · cites this paper as [40]
Citations
Cites
Gokmen, T., & Vlasov, Y. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00333 · Cited by 491
Self-citationSteinbuch, K. (1961) · Kybernetik · doi:10.1007/bf00293853 · Cited by 346
Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices
Gokmen, T., Onen, M., & Haensch, W. (2017) · Frontiers in Neuroscience · doi:10.3389/fnins.2017.00538 · Cited by 162
Self-citation2017 Symposia on VLSI Technology and Circuits
(2017) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2017.2695360 · Cited by 14
Flagged first, then by citations
Research works printed but not linked: 4
- krizhevsky, 2012, NIPS, 1097
- mackin, 2019, Advances in Electric Mathematics
- lecun, 1988, 86, 2278
- chang, 2019, IBM Journal of R&D
Cited by
Sorted by citations
Deep physical neural networks trained with backpropagation
Wright, L. G., Onodera, T., Stein, M. M., Wang, T., Schachter, D. T., Hu, Z., & McMahon, P. L. (2022) · Nature · doi:10.1038/s41586-021-04223-6 · Cited by 799
An analog-AI chip for energy-efficient speech recognition and transcription
Ambrogio, S., Narayanan, P., Okazaki, A., Fasoli, A., Mackin, C., Hosokawa, K., Nomura, A., Yasuda, T., Chen, A., Friz, A., et al. (2023) · Nature · doi:10.1038/s41586-023-06337-5 · Cited by 269
“To make the network more resilient to analog noise 23–26 , we retrained it while including weight and activation noise, weight clipping, L2 regularization and bias removal (https://aihwkit.readthedocs.io/en/latest/).”
MethodDelocalized photonic deep learning on the internet’s edge
Sludds, A., Bandyopadhyay, S., Chen, Z., Zhong, Z., Cochrane, J., Bernstein, L., Bunandar, D., Dixon, P. B., Hamilton, S. A., Streshinsky, M., et al. (2022) · Science · doi:10.1126/science.abq8271 · Cited by 206
Rasch, M. J., Mackin, C., Le Gallo, M., Chen, A., Fasoli, A., Odermatt, F., Li, N., Nandakumar, S. R., Narayanan, P., Tsai, H., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-40770-4 · Cited by 213
“[36, 38, 24]), we set out to retrain these 11 DNNs in a hardware-aware (HWA) manner.”
BackgroundMethodInfluentialSelf-citationA Flexible and Fast PyTorch Toolkit for Simulating Training and Inference on Analog Crossbar Arrays
Rasch, M. J., Moreda, D., Gokmen, T., Le Gallo, M., Carta, F., Goldberg, C., El Maghraoui, K., Sebastian, A., & Narayanan, V. (2021) · 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas51828.2021.9458494 · Cited by 160
“…apparent that accuracy on chips that are designed for inference benefit greatly from “hardware-aware” training, where the DNN workload is trained (or fine-tuned) in software with some analog noise and non-idealities in the forward pass, but with otherwise perfect backward and update pass [4,8,10,14].”
Self-citationIn-Memory Computing with Resistive Memory Circuits: Status and Outlook
Pedretti, G., & Ielmini, D. (2021) · Electronics · doi:10.3390/electronics10091063 · Cited by 103
“1 V, an average RRAM resistance R = 100 kΩ, corresponding to a current I = 100 μA for each cell within a a 100× 100 crosspoint array with wire resistance r = 1 Ω, then the overall IR drop can be computed as rI + 2rI + 3rI + · · ·+ NrI = rIN2 2 = 5 mV corresponding to a 5% error in the summation current [66].”
Background120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training
Lin, Z., Shastri, B. J., Yu, S., Song, J., Zhu, Y., Safarnejadian, A., Cai, W., Lin, Y., Ke, W., Hammood, M., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-53261-x · Cited by 106
“3b. Compared with the expected dot product result, the error of the measured one has a standard deviation of 0.03 (6.04 bits)—more than the 4 bits of precision required for performing AI tasks (details can be found in Supplementary Note 4) [42].”
InfluentialIntegration and Co-design of Memristive Devices and Algorithms for Artificial Intelligence
Wang, W. J., Song, W., Yao, P., Li, Y., Nostrand, J. V., Qiu, Q., Ielmini, D., & Yang, J. J. (2020) · iScience · doi:10.1016/j.isci.2020.101809 · Cited by 84
“into consideration, namely, hardware aware training (Gokmen et al. 2019; Joshi et al., 2020).”
Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference
Gallo, M. L., Lammie, C., Büchel, J., Carta, F., Fagbohungbe, O. I., Mackin, C., Tsai, H., Narayanan, V., Sebastian, A., Maghraoui, K. E., et al. (2023) · APL Machine Learning · doi:10.1063/5.0168089 · Cited by 88
Self-citationOptimised weight programming for analogue memory-based deep neural networks
Mackin, C., Rasch, M. J., Chen, A., Timcheck, J., Bruce, R. L., Li, N., Narayanan, P., Ambrogio, S., Gallo, M. L., Nandakumar, S. R., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-31405-1 · Cited by 62
Self-citationPark, J., Kim, S., Song, M. S., Youn, S., Kim, K., Kim, T.-H., & Kim, H. (2024) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.3c13775 · Cited by 56
Saito, D., Doevenspeck, J., Cosemans, S., Oh, H., Perumkunnil, M., Papistas, I. A., Belmonte, A., Rassoul, N., Delhougne, R., Kar, G. S., et al. (2020) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2020.3025986 · Cited by 31
“This lower precision and noise resilience model were trained by using teacher aware training technique as reported in [11]–[13].”
MethodSpoon, K., Tsai, H., Chen, A., Rasch, M. J., Ambrogio, S., Mackin, C., Fasoli, A., Friz, A. M., Narayanan, P., Stanisavljevic, M., et al. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2021.675741 · Cited by 26
Self-citationAnalog Multiply-Accumulate Cell With Multi-Bit Resolution for All-Analog AI Inference Accelerators
Nägele, R., Finkbeiner, J., Stadtlander, V., Grözing, M., & Berroth, M. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3268728 · Cited by 24
“As described in [13], this effect must be taken into account during training.”
Background1.4 The Future of Computing: Bits + Neurons + Qubits
Gil, D., & Green, W. M. J. (2020) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc19947.2020.9062918 · Cited by 22
“Initial system-level simulations show that even for inference only, building a model for analog hardware needs to take into account the above-mentioned material properties in order to obtain the weights required to maintain floating-point classification accuracy, albeit only in the forward pass [24].”
BackgroundHan, L.-X., Huang, P., Wang, Y., Zhou, Z., Zhang, Y., Liu, X.-Y., & Kang, J.-F. (2022) · IEEE Transactions on Circuits and Systems Part 1: Regular Papers · doi:10.1109/tcsi.2022.3194918 · Cited by 12
“In order to decrease the influence of nonlinearity of improved-LIF circuit on neural network accuracy, hardware-aware retraining [44], [45] is adopted, in which the nonlinearity of improved-LIF circuit is considered during neural network training.”
MethodDoevenspeck, J., Degraeve, R., Fantini, A., Cosemans, S., Mallik, A., Debacker, P., Verkest, D., Lauwereins, R., & Dehaene, W. (2021) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2021.3068696 · Cited by 14
“Gokmen et al. [11] distributions which are not representative for specific memory technology, as shown in this work for OxRRAM.”
Cherupally, S. K., Meng, J., Rakin, A. S., Yin, S., Yeo, I., Yu, S., Fan, D., & Seo, J.-S. (2021) · Semiconductor Science and Technology · doi:10.1088/1361-6641/ac461f · Cited by 12
Yang, Z., Zhang, L., Aras, K., Efimov, I. R., & Adam, G. C. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200032 · Cited by 10
The Impact of Device Uniformity on Functionality of Analog Passively-Integrated Memristive Circuits
Fahimi, Z., Mahmoodi, M. R., Klachko, M., Nili, H., & Strukov, D. B. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3097282 · Cited by 7
Analog-memory-based 14nm Hardware Accelerator for Dense Deep Neural Networks including Transformers
Okazaki, A., Narayanan, P., Ambrogio, S., Hosokawa, K., Tsai, H.-Y. S., Nomura, A., Yasuda, T., Mackin, C., Friz, A. M., Ishii, M., et al. (2022) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas48785.2022.9937292 · Cited by 6
“To simulate the various components in a realistic analog accelerator system, we include various sources of non-ideality in the analog MAC operation, and conductance noise of PCM devices such as programming noise, conductance drift and 1 /f read noise, as described in [8].”
BackgroundMethodSelf-citationNoise tolerant ternary weight deep neural networks for analog in-memory inference
Doevenspeck, J., Vrancx, P., Laubeuf, N., Mallik, A., Debacker, P., Verkest, D., Lauwereins, R., & Dehaene, W. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533684 · Cited by 5
Computing with nonvolatile memories for artificial intelligence
Pedretti, G., & Ielmini, D. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-820758-1.00009-1 · Cited by 5
Impact of analog memory device failure on in-memory computing inference accuracy
Li, N., Tsai, H.-Y. S., Narayanan, V., & Rasch, M. J. (2023) · APL Machine Learning · doi:10.1063/5.0131797 · Cited by 5
Self-citationHosseinzadeh, S., Lancaster, S., Parvaresh, A., Silva, C., Mikolajick, T., Slesazeck, S., & Fey, D. (2024) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd64264.2024.00010 · Cited by 4
“As a result, full-precision models used for inference on analog hardware will lead to a considerable degradation in performance compared to a model obtained through HWA training [33], [34].”
MethodEmerging Neural Workloads and Their Impact on Hardware
Brooks, D., Frank, M. M., Gokmen, T., Gupta, U., Hu, X. S., Jain, S., Laguna, A. F., Niemier, M. T., O’Connor, I., Raghunathan, A., et al. (2020) · Design, Automation, and Test in Europe · doi:10.23919/date48585.2020.9116435 · Cited by 4
“The non-yielding, corrupt devices can be accommodated by hardware-aware training via randomly dropping connections [33].”
BackgroundSelf-citationThe Future of Computing: Bits + Neurons + QubitsPreprint
Gil, D., & Green, W. M. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08446 · Cited by 4
“…system-level simulations show that even for inference only, building a model for analog hardware needs to take into account the above-mentioned material properties in order to obtain the weights required to maintain floating-point classification accuracy, albeit only in the forward pass [24].”
BackgroundToward Efficient System-on-Module for Design-Space Exploration of Analog Spiking Neural Networks
El-Masry, M., Werner, T., Zjajo, A., & Weigel, R. (2024) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2024.3406522 · Cited by 3
“The detrimental effect of variability can be mitigated to some extent as shown in the papers [47], [48], [49], [50].”
BackgroundMitigating Imperfections in Mixed-Signal Neuromorphic CircuitsPreprint
Fahimi, Z., Mahmoodi, M. R., Klachko, M., Nili, H., Kim, H., & Strukov, D. B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04236 · Cited by 6
“[31], DropConnect regularization is introduced to enhance the accuracy drop originating from low yield without using nonideality models—at 98% yield, 17% accuracy drop on CIFAR-10 based on ResNet-18 model is reduced to 10%.”
MethodAll-optical temporal integration mediated by subwavelength heat antennas
Zhang, Y., Farmakidis, N., Roumpos, I., Moralis‐Pegios, M., Tsakyridis, A., Lee, J. S., Dong, B., He, Y., Aggarwal, S., Pleros, N., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-67726-0 · Cited by 2
“To evaluate the precision of this integration, we compute the standard deviation of the integration peaks across multiple trials.”
Realistic Noise-aware Training as a Component of the Holistic ACiM Development Platform
Kim, J., Park, S., Suh, H., Kwon, Y., Lee, S., Lee, Y., Kim, K., Han, E., Kim, J., Kim, K., et al. (2024) · IEEE International Memory Workshop (IMW) · doi:10.1109/imw59701.2024.10536981 · Cited by 2
“Noise-aware Training (NAT) is a tuning method which adds random noise to the weight during the training phase of NN [7-8].”
BackgroundMethods for Setting Device Specifications for Analog In‐Memory Computing Inference
Wu, Z., Su, X., Rasch, M. J., & Li, N. (2026) · Advanced Intelligent Systems · doi:10.1002/aisy.202501355 · Cited by 1
Self-citationHardware-Aware Model Design and Training of Silicon-Based Analog Neural Networks
Filippeschi, G., Brazzini, M., Mosquera, C., Lanuzza, M., Catania, A., Strangio, S., & Iannaccone, G. (2026) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2026.3705439 · Cited by 1
“In this way, HWA retraining compensates for device and circuit imperfections, and has proven to be e ff ective across several AIMC platforms [14], [15], [16], [17], [18], [19].”
Multiplexing methods for scaling up photonic logic
Hamerly, R., Bandyopadhyay, S., Sludds, A., Chen, Z., Bernstein, L., Vadlamani, S. K., Basani, J. R., Davis, R., & Englund, D. (2023) · AI and Optical Data Sciences · doi:10.1117/12.2650902 · Cited by 1
“005, corresponding to at least 7 bits of precision, higher than the ∼5 bits usually required for DNN inference.(24,25) As confirmation, we run a pre-trained DNN on a benchmark image classification task (MNIST) using the NetCast hardware, obtaining a classification accuracy of 98.”
MethodJo, H. J., Kang, M., Um, M., Kim, J.-H., Kwon, K., Kim, S., & Lee, H. (2024) · International Journal of Circuit Theory and Applications · doi:10.1002/cta.4126
A 1.2-fJ/MAC, 7-fJ/ReLU, 5.9-b ENOB, All-Analog Neural Network Computing Macro
Nägele, R., Finkbeiner, J., Wittlinger, M., Grözing, M., Berroth, M., & Rademacher, G. (2025) · Conference on Ph.D. Research in Microelectronics and Electronics · doi:10.1109/prime66228.2025.11203351
“In order to maintain a high accuracy in the inference process, hardware constraints should be introduced into the training process [13].”
Li, M. T., Yang, F., Liu, Y., Zhang, C., Li, E., Yang, J., Lin, C., Liu, H., Luo, C., Chien, H., et al. (2026) · Advanced Materials · doi:10.1002/adma.73440
Metadata
- Authors
- Gokmen, T., Rasch, M. J., & Haensch, W.
- Affiliations
- IBM (United States) (US)
- IBM Research AI,Yorktown Heights,NY,USA,10598
- Venue
- 2019 IEEE International Electron Devices Meeting (IEDM)
- Publisher
- IEEE
- Published
- 2019-12
- Pages
- 22.3.1-22.3.4
- ISSN
- 0163-1918 (ISSN-L)
- OpenAlex
- W3005589975
- Cited by
- 37
- References
- 4
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 6 · 2023: 6 · 2022: 7 · 2021: 8 · 2020: 4 · 2019: 1
- FWCI
- 7.96
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Technical digest - International Electron Devices Meeting/Technical digest (publishedVersion)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Analog and Mixed-Signal Circuit Design
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- deep neural network inference, weight mapping, hardware-aware training, device noise, device failure, large-scale neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
