CoChat
Sign Up
PreprintCited by 78Advanced Neural Network ApplicationsLow-power high-performance VLSI designDomain Adaptation and Few-Shot Learning

Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference

Open access
McKinstry, Jeffrey L., Esser, Steven K., Appuswamy, Rathinakumar, Bablani, Deepika, Arthur, John V., Yildiz, Izzet B., Modha, Dharmendra S.·2018·arXiv
DOI 10.48550/arxiv.1809.04191arXiv 1809.04191
Open this paper in CoChat — free accountOpen PDF

Abstract

To realize the promise of ubiquitous embedded deep network inference, it is essential to seek limits of energy and area efficiency. To this end, low-precision networks offer tremendous promise because both energy and area scale down quadratically with the reduction in precision. Here we demonstrate ResNet-18, -34, -50, -152, Inception-v3, Densenet-161, and VGG-16bn networks on the ImageNet classification benchmark that, at 8-bit precision exceed the accuracy of the full-precision baseline networks after one epoch of finetuning, thereby leveraging the availability of pretrained models. We also demonstrate ResNet-18, -34, -50, -152, Densenet-161, and VGG-16bn 4-bit models that match the accuracy of the full-precision baseline networks -- the highest scores to date. Surprisingly, the weights of the low-precision networks are very close (in cosine similarity) to the weights of the corresponding baseline networks, making training from scratch unnecessary. We find that gradient noise due to quantization during training increases with reduced precision, and seek ways to overcome this noise. The number of iterations required by SGD to achieve a given training error is related to the square of (a) the distance of the initial solution from the final plus (b) the maximum variance of the gradient estimates. Therefore, we (a) reduce solution distance by starting with pretrained fp32 precision baseline networks and fine-tuning, and (b) combat gradient noise introduced by quantization by training longer and reducing learning rates. Sensitivity analysis indicates that these simple techniques, coupled with proper activation function range calibration to take full advantage of the limited precision, are sufficient to discover low-precision networks, if they exist, close to fp32 precision baseline networks. The results herein provide evidence that 4-bits suffice for classification.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1809.04191

Cites

27 of 27 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ResNet-34 on ImageNet recovers high accuracy extremely quickly when retraining with additive noise due to quick updates of the batch normalization parameters, obtaining satisfactory convergence after only 8 epochs.

    “Consistent with previous observations23,33, we found that the network recovers high accuracy extremely quickly when retraining with additive noise due to quick updates of the batch normalization parameters (see Supplementary Note 1), and obtained satisfactory convergence after only 8 epochs.”

    From Accurate deep neural network inference using computational phase-change memory (2020) · cites this paper as [33]

    The Abstract (p. 1) reports that 8-bit ResNet-34 on ImageNet exceeded its full-precision baseline after one epoch of fine-tuning, supporting the observation of rapid accuracy recovery. The paper describes quantization and fine-tuning, not recovery from additive-noise retraining caused by quick updates to batch-normalization parameters. The citing sentence presents convergence after eight epochs as the citing authors’ own result, not a result attributed to this source. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Retraining ResNet with additive noise results mainly in adapting the batch normalization parameters, whereas the weights stay close to the full-precision weights trained without noise.

    “we notice that retraining ResNet with additive noise results mainly in adapting the batch normalization parameters, whereas the weights stay close to the full-precision weights trained without noise.”

    From Accurate deep neural network inference using computational phase-change memory (2020) · cites this paper as [33]

    Figure 2 (section 4.7, p. 8) reports that fine-tuned 4-bit ResNet-18 weights remain close to the initial full-precision weights. The paper discusses noise from quantization, but the passages do not show retraining with additive noise or establish that adaptation occurs mainly in batch-normalization parameters. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • It is unlikely that reduced digital precision training procedures can generally be applied as-is to analog in-memory computing hardware due to the random nature of δG.

    “Although a large body of efficient techniques to train DNNs with reduced digital precision has been reported32,33, it is unlikely that such procedures can generally be applied as-is to analog in-memory computing hardware due to the random nature of .”

    From Accurate deep neural network inference using computational phase-change memory (2020) · cites this paper as [33]

  • Training with 4-bit digital weights improves performance after transfer to PCM with respect to FP32, but accuracy decreases by more than 1% after transferring the 4-bit weights to PCM.

    “Training with 4-bit digital weights (using the method described in the ref. 33), which is roughly the effective precision of our PCM devices40, improves the performance after transfer with respect to FP32, but nevertheless the accuracy decreases by more than 1% after transferring the 4-bit weights to PCM.”

    From Accurate deep neural network inference using computational phase-change memory (2020) · cites this paper as [33]

  • Training with additive noise increases ImageNet top-1 accuracy by approximately 6% on PCM compared with FP32 and 4-bit training.

    “Training with additive noise increases the accuracy by approximately 6% on PCM compared with FP32 and 4-bit33 training.”

    From Accurate deep neural network inference using computational phase-change memory (2020) · cites this paper as [33]

    The Discussion (p. 9) describes compensating for quantization-related noise while fine-tuning 4- and 8-bit networks, not training with additive noise. The ImageNet results in §1.1 (p. 2) compare quantized networks with FP32 baselines, but our copy gives no PCM result or approximately 6% accuracy increase. I searched for additive noise, PCM, and the stated comparison. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 28 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. Chaim Baskin, Eli Schwartz, Evgenii Zheltonozhskii, Natan Liss, Raja Giryes, Alexander M. Bronstein, and Avi Mendelson. UNIQ: uniform noise injection for the quantization of neural networks. CoRR , abs/1804.10969, 2018. URL http://arxiv.org/abs/1804.10969 . · arXiv
  2. Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems , pp. 3123-3131, 2015.
  3. Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning , volume 1. MIT press Cambridge, 2016.
  4. Szymon Migacz. Nvidia 8-bit inference with tensorrt. GPU Technology Conference, 2017.

Cited by

Sorted by citations

Related

Authors
McKinstry, J. L., Esser, S. K., Appuswamy, R., Bablani, D., Arthur, J. V., Yildiz, I. B., & Modha, D. S.
Affiliations
IBM Research - Almaden (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1809.04191
arXiv
1809.04191
OpenAlex
W2890616119
Cited by
74
References
28
Citations by year
2025: 4 · 2024: 6 · 2023: 7 · 2022: 11 · 2021: 15 · 2020: 18 · 2019: 11 · 2018: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
low-precision neural networks, 4-bit quantization, 8-bit quantization, ImageNet classification, pre-trained model fine-tuning, gradient noise, quantization noise, ResNet, InceptionV3, DenseNet-161

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review