Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index; 13 of the 29 sources its claims cite were checked for support.

Accurate deep neural network inference using computational phase-change memory

Open access
Joshi, Vinay, Le Gallo, Manuel, Haefeli, Simon, Boybat, Irem, Nandakumar, S. R., Piveteau, Christophe, Dazzi, Martino, Rajendran, Bipin, Sebastian, Abu, Eleftheriou, Evangelos·2020·Nature Communications

Abstract

In-memory computing using resistive memory devices is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to phase-change memory (PCM) devices. We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on CIFAR-10 and a top-1 accuracy of 71.6% on ImageNet benchmarks after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one-day period, where each of the 361,722 synaptic weights is programmed on just two PCM devices organized in a differential configuration.

Overview

CoChat Summary

TL;DR

The paper shows that ResNet-style CNNs can be trained in software to tolerate phase-change memory (PCM) nonidealities, enabling accurate analog in-memory inference after weight transfer. With their training and compensation methods, they report 93.7% accuracy on CIFAR-10 and 71.6% top-1 on ImageNet after mapping weights to PCM, and 93.75% experimental accuracy on CIFAR-10 hardware that stayed above 92.6% for one day.

Key findings

  • Training ResNet-32 with injected Gaussian synaptic-weight noise yielded 93.7% test accuracy on CIFAR-10 after transfer to PCM, which was less than 0.2% below the FP32 baseline.
  • For ResNet-34 on ImageNet, additive-noise training gave 71.6% top-1 accuracy after transfer to PCM, about 6% higher on PCM than FP32 and 4-bit training in their comparison.
  • In hardware/software CIFAR-10 experiments, all 361,722 synaptic weights of ResNet-32 were programmed on two PCM devices per weight; test accuracy was 93.75% at 25 s after programming and, with global drift compensation, stayed above 92.6% for 1 day.
  • Adaptive batch normalization statistics update (AdaBS) improved drift retention further: on CIFAR-10 it kept accuracy above 93.5% over one day, an improvement of 0.9% over GDC; on ImageNet it improved one-year accuracy by 7% vs. GDC and retained above 71% for one year when combined with digital first/last layers.

Methods

  • The study used ResNet-32 on CIFAR-10 and ResNet-34 on ImageNet, both trained with stochastic gradient descent and then mapped to PCM crossbar synapses.
  • For training, they injected Gaussian noise into synaptic weights during the forward pass to emulate hardware read/write noise, combined with weight initialization from a pretrained baseline, weight clipping, and learning-rate scheduling; the noise scale was estimated from a one-time hardware characterization.
  • Hardware validation used a prototype PCM chip with 1 million devices for the main experiments and 3 million devices in the platform; PCM weights were programmed with an iterative program-and-verify procedure, then read over time and evaluated in software inference, with GDC and AdaBS used for drift compensation.

Limitations

  • The paper states that AdaBS increases accuracy at the cost of additional digital computations during calibration and requires off-chip memory/data transfers for calibration images.
  • It also states that the experiments were not done on a fully integrated chip that supports all functions of deep learning inference; this is an explicit limitation in the Discussion.
  • Evident from the text: the hardware/software experiment covers the network weights on the PCM array, but some parts of the inference stack are still simulated in software, so the results do not reflect a complete end-to-end deployed system.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41467-020-16108-9

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Training and inference methodology. a ResNet-32 network architecture for CIFAR-10 image classification. The architecture of ResNet-32 used in this study is a slightly modified version of the original implementation24 with fewer input and output channels in ResNet blocks 2 and 3. This allows each layer to fit on a single crossbar array of size 512 × 512 in an eventual computational memory accelerator. b Training and inference of an example layer of ResNet-32 according to the methodology proposed in this paper. Software training is performed by injecting a random noise term to the weights used during the forward propagation, δWtr^l, which is representative of the combined read and write noise of the memristive devices used during inference (see section Training procedure). When transferring the weights of a convolution layer to memristive crossbars for inference, they are flattened into a 2-D matrix by collapsing each filter into a single vector programmed on a crossbar column, and stacking all filters on separate columns25. The weights are then programmed as the differential conductance of two memristive devices. Input activations al are applied as voltages on the crossbar rows. The output current from the column containing the G− devices is subtracted from the one from the column containing the G+ devices. The differential current output I from the crossbar then routes to a digital circuitry that performs batch normalization and the corresponding rectified linear unit (ReLU) activation function, in order to obtain the input activations for the next layer al+1. The final softmax activation function can be performed off-chip if required. An optional correction of the batch normalization parameters can be periodically performed to improve the accuracy retention over time (see sections Hardware/software inference experiment on CIFAR-10 and Adaptive batch normalization statistics update). The input image is padded with zero values at the border to ensure that the convolution operation preserves the height and width of the image. Therefore, considering an input image of size n × n, the convolution operation can be performed in n2 matrix-vector multiplication cycles.
Fig. 1 Training and inference methodology. a ResNet-32 network architecture for CIFAR-10 image classification. The architecture of ResNet-32 used in this study is a slightly modified version of the original implementation24 with fewer input and output channels in ResNet blocks 2 and 3. This allows each layer to fit on a single crossbar array of size 512 × 512 in an eventual computational memory accelerator. b Training and inference of an example layer of ResNet-32 according to the methodology proposed in this paper. Software training is performed by injecting a random noise term to the weights used during the forward propagation, δWtr^l, which is representative of the combined read and write noise of the memristive devices used during inference (see section Training procedure). When transferring the weights of a convolution layer to memristive crossbars for inference, they are flattened into a 2-D matrix by collapsing each filter into a single vector programmed on a crossbar column, and stacking all filters on separate columns25. The weights are then programmed as the differential conductance of two memristive devices. Input activations al are applied as voltages on the crossbar rows. The output current from the column containing the G− devices is subtracted from the one from the column containing the G+ devices. The differential current output I from the crossbar then routes to a digital circuitry that performs batch normalization and the corresponding rectified linear unit (ReLU) activation function, in order to obtain the input activations for the next layer al+1. The final softmax activation function can be performed off-chip if required. An optional correction of the batch normalization parameters can be periodically performed to improve the accuracy retention over time (see sections Hardware/software inference experiment on CIFAR-10 and Adaptive batch normalization statistics update). The input image is padded with zero values at the border to ensure that the convolution operation preserves the height and width of the image. Therefore, considering an input image of size n × n, the convolution operation can be performed in n2 matrix-vector multiplication cycles.CC BY · PubMed Central
Fig. 2 Impact of injecting weight noise during training and inference on network accuracy. a Test accuracy on CIFAR-10 obtained for different amounts of relative injected weight noise during training ηtr without inducing any perturbation during inference (ηinf=0). When ηtr>8%, the training convergence starts to become affected by the high noise and it is not possible anymore to reach the software baseline within the same number of epochs. The error bars represent the standard deviation over 10 training runs. b Test accuracy on CIFAR-10 obtained for the networks trained with different amounts of relative weight noise ηtr as a function of the weight noise injected during inference ηinf. In most cases, ηtr can be increased above ηinf up to a certain point and still lead to comparable or slightly higher (within ≈0.1%) test accuracy than for ηtr=ηinf. However, when ηtr becomes much higher than ηinf, the test accuracy decreases due to the inability of the network to achieve baseline accuracy when ηtr>8%. Each data point represents the average over 10 training runs and 100 inference runs. c Test accuracy on CIFAR-10 as a function of ηtr=ηinf. The error bars represent the standard deviation over 100 inference runs averaged over 10 training runs. d Top-1 accuracy on ImageNet as a function of ηtr=ηinf. The error bars represent the standard deviation over 10 inference runs on a single training run.
Fig. 2 Impact of injecting weight noise during training and inference on network accuracy. a Test accuracy on CIFAR-10 obtained for different amounts of relative injected weight noise during training ηtr without inducing any perturbation during inference (ηinf=0). When ηtr>8%, the training convergence starts to become affected by the high noise and it is not possible anymore to reach the software baseline within the same number of epochs. The error bars represent the standard deviation over 10 training runs. b Test accuracy on CIFAR-10 obtained for the networks trained with different amounts of relative weight noise ηtr as a function of the weight noise injected during inference ηinf. In most cases, ηtr can be increased above ηinf up to a certain point and still lead to comparable or slightly higher (within ≈0.1%) test accuracy than for ηtr=ηinf. However, when ηtr becomes much higher than ηinf, the test accuracy decreases due to the inability of the network to achieve baseline accuracy when ηtr>8%. Each data point represents the average over 10 training runs and 100 inference runs. c Test accuracy on CIFAR-10 as a function of ηtr=ηinf. The error bars represent the standard deviation over 100 inference runs averaged over 10 training runs. d Top-1 accuracy on ImageNet as a function of ηtr=ηinf. The error bars represent the standard deviation over 10 inference runs on a single training run.CC BY · PubMed Central
Fig. 3 Impact of weight transfer to PCM synapses on inference accuracy. a Cumulative distributions of 11 representative iteratively programmed conductance levels on 10,000 PCM devices per level. The vertical dashed lines denote the target conductance for each level. b Conductance standard deviation of the 11 levels as a function of target conductance. The inset shows a representative conductance distribution of one level. c Test accuracy on CIFAR-10 after software training and after weight transfer to PCM synapses for different training schemes. d Top-1 accuracy on ImageNet after software training and after weight transfer to PCM synapses for different training schemes. The weights of all layers of ResNet-34 are transferred to PCM synapses, including the first and the last. In c and d the error bars represent the standard deviation over 10 inference runs.
Fig. 3 Impact of weight transfer to PCM synapses on inference accuracy. a Cumulative distributions of 11 representative iteratively programmed conductance levels on 10,000 PCM devices per level. The vertical dashed lines denote the target conductance for each level. b Conductance standard deviation of the 11 levels as a function of target conductance. The inset shows a representative conductance distribution of one level. c Test accuracy on CIFAR-10 after software training and after weight transfer to PCM synapses for different training schemes. d Top-1 accuracy on ImageNet after software training and after weight transfer to PCM synapses for different training schemes. The weights of all layers of ResNet-34 are transferred to PCM synapses, including the first and the last. In c and d the error bars represent the standard deviation over 10 inference runs.CC BY · PubMed Central
Fig. 4 Inference experiment with ResNet-32 on CIFAR-10. a Scatter plot of weights programmed in the PCM chip versus target weights obtained after training. The inset shows the distribution of the relative error between programmed and target synaptic conductance, (Gij^l−GT,ij^l)/Gmax, and its standard deviation σ. b Distributions of programmed devices whose target conductance fall within 0.25 μS from five representative GT values. The distributions are shown at four different times spanning the experiment duration of one day. The filled bars are the measured hardware data, the black lines are the PCM model. c Individual device conductance evolution over time of four arbitrarily picked devices from the chip programmed to four distinct GT values, along with one PCM model realization for the same GT values. d Measured test accuracy over time from the weights of the PCM chip, along with the corresponding PCM model match. A global drift compensation (GDC) procedure is performed for every layer before performing inference on the test set. The filled areas from the PCM model correspond to one standard deviation over 25 inference runs. The baseline refers to the FP32 software accuracy of 93.87%.
Fig. 4 Inference experiment with ResNet-32 on CIFAR-10. a Scatter plot of weights programmed in the PCM chip versus target weights obtained after training. The inset shows the distribution of the relative error between programmed and target synaptic conductance, (Gij^l−GT,ij^l)/Gmax, and its standard deviation σ. b Distributions of programmed devices whose target conductance fall within 0.25 μS from five representative GT values. The distributions are shown at four different times spanning the experiment duration of one day. The filled bars are the measured hardware data, the black lines are the PCM model. c Individual device conductance evolution over time of four arbitrarily picked devices from the chip programmed to four distinct GT values, along with one PCM model realization for the same GT values. d Measured test accuracy over time from the weights of the PCM chip, along with the corresponding PCM model match. A global drift compensation (GDC) procedure is performed for every layer before performing inference on the test set. The filled areas from the PCM model correspond to one standard deviation over 25 inference runs. The baseline refers to the FP32 software accuracy of 93.87%.CC BY · PubMed Central
Fig. 5 Adaptive batch normalization statistics update with PCM synapses. a The AdaBS calibration procedure consists in updating the running mean μ and variance σ2 parameters of the batch normalization performed in the digital unit of the in-memory computing hardware. The calibration is performed periodically when the device is idle, and after calibration the values of μ and σ2 of every layer are updated for subsequent inference. Note that during the calibration phase, batch normalization is performed using the mini-batch mean μB and variance σB² instead of μ and σ2 (see Methods and Supplementary Note 3). b Test accuracy of ResNet-32 on CIFAR-10 with GDC and AdaBS using the PCM model, along with experimental test accuracy obtained by applying AdaBS on ResNet-32 with the measured weights from the PCM chip. The filled areas from the PCM model correspond to one standard deviation over 25 inference runs. c Top-1 accuracy of ResNet-34 on ImageNet with GDC and AdaBS computed using the PCM model. Implementations using PCM synapses for all layers as well as first and last layers in digital FP32, and PCM synapses for all other layers, are shown. In the latter, no noise is applied on the first and last layers during training. The filled areas correspond to one standard deviation over 10 inference runs. In b and c the baseline refers to the FP32 software accuracy.
Fig. 5 Adaptive batch normalization statistics update with PCM synapses. a The AdaBS calibration procedure consists in updating the running mean μ and variance σ2 parameters of the batch normalization performed in the digital unit of the in-memory computing hardware. The calibration is performed periodically when the device is idle, and after calibration the values of μ and σ2 of every layer are updated for subsequent inference. Note that during the calibration phase, batch normalization is performed using the mini-batch mean μB and variance σB² instead of μ and σ2 (see Methods and Supplementary Note 3). b Test accuracy of ResNet-32 on CIFAR-10 with GDC and AdaBS using the PCM model, along with experimental test accuracy obtained by applying AdaBS on ResNet-32 with the measured weights from the PCM chip. The filled areas from the PCM model correspond to one standard deviation over 25 inference runs. c Top-1 accuracy of ResNet-34 on ImageNet with GDC and AdaBS computed using the PCM model. Implementations using PCM synapses for all layers as well as first and last layers in digital FP32, and PCM synapses for all other layers, are shown. In the latter, no noise is applied on the first and last layers during training. The filled areas correspond to one standard deviation over 10 inference runs. In b and c the baseline refers to the FP32 software accuracy.CC BY · PubMed Central

Verification

Claims the paper makes

11 Valid · 1 Not valid · 9 Not checked · 88 Own results and interpretations

57 more claims · 1 with their cited sources checked

Create a free account to see all 109 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 40
  • Probably not a substantial claim. 10

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 51 references

Research works printed but not linked: 5

  1. Blundell, C., Cornebise, J., Kavukcuoglu, K. & Wierstra, D. Weight uncertainty in neural networks. In Proc. of the 32nd International Conference on Machine Learning, Vol. 37 of ICML’15 1613–1622 (JMLR.org, 2015).
  2. Gulcehre, C., Moczulski, M., Denil, M. & Bengio, Y. Noisy activation functions. Proc. 33rd Int. Conf. Mach. Learn. 48, 3059–3068 (2016).
  3. Jim, K., Horne, B. G. & Giles, C. L. Effects of noise on convergence and generalization in recurrent networks. In Proc. of the 7th International Conference on Neural Information Processing Systems 649–656 (MIT Press, 1994).
  4. Courbariaux, M., Bengio, Y. & David, J.-P. Binaryconnect: training deep neural networks with binary weights during propagations. In Advances in Neural Information Processing Systems (eds Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M. & Garnett, R.) 3123–3131 (Curran Associates, Inc., 2015).
  5. Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proc. of the 32nd International Conference on Machine Learning, Vol. 37 of ICML’15 (ed. Bach, F. & Blei, D.) 448–456 (PMLR, 2015).

Cited by

See all 490

Metadata

Authors
Joshi, V., Le Gallo, M., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E.
Affiliations
  • King's College London (GB)
  • ETH Zurich (CH)
  • IBM Research - Zurich (CH)
Corresponding author
Sebastian, Abu
Venue
Nature Communications
Published
2020-05-18
Volume
11
Issue
1
Pages
2473-2473
ISSN
2041-1723 (ISSN-L)
OpenAlex
W2948661249
Cited by
490
References
51
Citations by year
2026: 69 · 2025: 83 · 2024: 73 · 2023: 88 · 2022: 92 · 2021: 68 · 2020: 17
FWCI
22.73
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Research Portal (King's College London) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
European Commission (HORIZON2020, 682675)
Topics
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
Keywords
phase-change memory, in-memory computing, resistive memory devices, deep neural network inference, weight mapping, device variability, ResNet, energy-efficient hardware, non-von Neumann architecture

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Accurate deep neural network inference using computational phase-change memory. https://app.cochat.ai/papers/10.1038/s41467-020-16108-9
The paper
Joshi, V., Le Gallo, M., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. (2020). Accurate deep neural network inference using computational phase-change memory. Nature Communications. https://doi.org/10.1038/s41467-020-16108-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.