CoChat

A fully hardware-based memristive multilayer neural network

Open access

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

Abstract

A fully hardware-based memristive neural network is capable of delivering high computing throughput and power efficiency.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1126/sciadv.abj4801

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. The two-layer memristive perceptron built with hardware neurons and synapses. (A) Schematic of the perceptron, in which the hidden layer calculates the weighted sum of the inputs (vi) and applies activation functions on the weighted sum. (B) Block diagram of the implemented perceptron. The weights [w1 and w2 as represented by arrows in (A)] are represented by the conductance of the memristive devices in the crossbar arrays. The activation functions are applied using the hidden neurons. (C) Circuit schematic of the perceptron. Crossbar arrays are composed of 1T1R cells, each connected in series between the selection line (SL) and bit line (BL) with the transistor gate connected to the word line (WL). Hidden neurons (ReLUs) and inverters provide both positive and negative inputs for the second layer. (D) An optical micrograph of the 1T1R array used as the left crossbar in (C) as the first layer. Scale bar, 1 mm. (E) A stack of four PCBs as the hardware neurons, each containing 16 channels of ReLU activation functions and 16 inverters to support 16 differential pairs for the second layer. (F) An optical micrograph of the 1T1R array used as the right crossbar in (C) as the second layer. Scale bar, 1 mm. Photo credit: Fatemeh Kiani, University of Massachusetts Amherst.
Fig. 1. The two-layer memristive perceptron built with hardware neurons and synapses. (A) Schematic of the perceptron, in which the hidden layer calculates the weighted sum of the inputs (vi) and applies activation functions on the weighted sum. (B) Block diagram of the implemented perceptron. The weights [w1 and w2 as represented by arrows in (A)] are represented by the conductance of the memristive devices in the crossbar arrays. The activation functions are applied using the hidden neurons. (C) Circuit schematic of the perceptron. Crossbar arrays are composed of 1T1R cells, each connected in series between the selection line (SL) and bit line (BL) with the transistor gate connected to the word line (WL). Hidden neurons (ReLUs) and inverters provide both positive and negative inputs for the second layer. (D) An optical micrograph of the 1T1R array used as the left crossbar in (C) as the first layer. Scale bar, 1 mm. (E) A stack of four PCBs as the hardware neurons, each containing 16 channels of ReLU activation functions and 16 inverters to support 16 differential pairs for the second layer. (F) An optical micrograph of the 1T1R array used as the right crossbar in (C) as the second layer. Scale bar, 1 mm. Photo credit: Fatemeh Kiani, University of Massachusetts Amherst.CC BY-NC · PubMed Central
Fig. 2. Design and performance of the hardware ReLU. (A) The designed ReLU circuit. Each unit/channel consists of three stages: a current rectifier, a voltage follower, and an inverting amplifier, respectively (labeled in the blue boxes). The first stage rectifies the input current. The resistor on the feedback (R1) is used to tune the gain of the ReLU, and the diodes (D1 and D2) are used to perform the rectification. The voltage follower, which is a unity gain voltage buffer, prevents the previous stage from loading by the next one. The third stage inverts its input to provide a positive voltage on the output as expected by a ReLU activation function. It also scales down the output by tuning the resistors R2 and R3. (B) Schematic of signal propagation from the input to the output. The input current (Iin) is rectified in the first stage and converted to the voltage using a gain equal to −R1. The voltage output of the first stage (V1) is duplicated to the output of the second stage (V2) using the voltage follower and then inverted and scaled in the last stage (Vout) by multiplying with −R3/R2 (−0.1 in this design). (C) DC characteristic of the hardware ReLU. The output voltage is positive and linearly proportional to the input when the input current is positive and is around zero on the negative side. The output was measured 25 times for each input, and the error bar was plotted, which shows that the error from the expected output values is very small (less than 0.072% for the positive inputs and less than 1.5 mV for the negative ones). Linear regression was used to fit the measured data, and the R2 was calculated to be 1, showing excellent linearity on the positive side.
Fig. 2. Design and performance of the hardware ReLU. (A) The designed ReLU circuit. Each unit/channel consists of three stages: a current rectifier, a voltage follower, and an inverting amplifier, respectively (labeled in the blue boxes). The first stage rectifies the input current. The resistor on the feedback (R1) is used to tune the gain of the ReLU, and the diodes (D1 and D2) are used to perform the rectification. The voltage follower, which is a unity gain voltage buffer, prevents the previous stage from loading by the next one. The third stage inverts its input to provide a positive voltage on the output as expected by a ReLU activation function. It also scales down the output by tuning the resistors R2 and R3. (B) Schematic of signal propagation from the input to the output. The input current (Iin) is rectified in the first stage and converted to the voltage using a gain equal to −R1. The voltage output of the first stage (V1) is duplicated to the output of the second stage (V2) using the voltage follower and then inverted and scaled in the last stage (Vout) by multiplying with −R3/R2 (−0.1 in this design). (C) DC characteristic of the hardware ReLU. The output voltage is positive and linearly proportional to the input when the input current is positive and is around zero on the negative side. The output was measured 25 times for each input, and the error bar was plotted, which shows that the error from the expected output values is very small (less than 0.072% for the positive inputs and less than 1.5 mV for the negative ones). Linear regression was used to fit the measured data, and the R2 was calculated to be 1, showing excellent linearity on the positive side.CC BY-NC · PubMed Central
Fig. 3. In situ training of the two-layer networks. (A) Training accuracy increases with the number of hidden neurons. The network is trained using the MNIST dataset with a batch size of 50 and 2400 weight updates in total. Three different cases, namely, pure software training, simulation using simulated arrays (the hardware parameters were extracted and used to model the arrays), and experimental implementation with partial software ReLUs, are compared. (B) Training accuracy increases with the number of weight updating. The difference in the training accuracy for the three different cases narrows as well. During training, the weights in each array were updated, and the network accuracy was measured after each batch (50 images).
Fig. 3. In situ training of the two-layer networks. (A) Training accuracy increases with the number of hidden neurons. The network is trained using the MNIST dataset with a batch size of 50 and 2400 weight updates in total. Three different cases, namely, pure software training, simulation using simulated arrays (the hardware parameters were extracted and used to model the arrays), and experimental implementation with partial software ReLUs, are compared. (B) Training accuracy increases with the number of weight updating. The difference in the training accuracy for the three different cases narrows as well. During training, the weights in each array were updated, and the network accuracy was measured after each batch (50 images).CC BY-NC · PubMed Central
Fig. 4. Inference results of the two-layer hardware perceptron. (A) Inference accuracy increases with the number of hidden neurons. The accuracy is measured in different cases of using pure software, simulated arrays on simulation, experimental implementation using partial software hidden neurons, and experimental implementation using all-hardware hidden neurons. The improvement in test accuracy is more noticeable in the case of experimental implementation using all-hardware hidden neurons. (B) Confusion matrix of the inference accuracy using all-hardware hidden neurons. The most frequently misclassified images are digit 2. (C) Two different images of digit 2 (left column) have been applied as input, and on the basis of the output current, the corresponding probability of each digit (right column) has been calculated using a SoftMax function. The digit number with the highest probability shows the classification result. Both images have been misclassified, the first one as digit 0 and the second one as digit 3. These images are hard for human eyes to recognize as well.
Fig. 4. Inference results of the two-layer hardware perceptron. (A) Inference accuracy increases with the number of hidden neurons. The accuracy is measured in different cases of using pure software, simulated arrays on simulation, experimental implementation using partial software hidden neurons, and experimental implementation using all-hardware hidden neurons. The improvement in test accuracy is more noticeable in the case of experimental implementation using all-hardware hidden neurons. (B) Confusion matrix of the inference accuracy using all-hardware hidden neurons. The most frequently misclassified images are digit 2. (C) Two different images of digit 2 (left column) have been applied as input, and on the basis of the output current, the corresponding probability of each digit (right column) has been calculated using a SoftMax function. The digit number with the highest probability shows the classification result. Both images have been misclassified, the first one as digit 0 and the second one as digit 3. These images are hard for human eyes to recognize as well.CC BY-NC · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

1 retracted

Show all 49 references

Research works printed but not linked: 8

  1. B. Razavi Fundamentals of Microelectronics (Wiley 2013).
  2. R. G. D. Steel J. H. Torrie Principles and Procedures of Statistics (McGraw-Hill Book Company 1960).
  3. I. Goodfellow Y. Bengio A. Courville Y. Bengio Deep Learning (The MIT Press 2016).
  4. Y. LeCun D. Touresky G. Hinton T. Sejnowski A theoretical framework for back-propagation in Proceedings of the 1988 Connectionist Models Summer School (1988) vol. 1 pp. 21–28.
  5. C. Igel M. Hüsken Improving the Rprop learning algorithm in Proceedings of the Second International ICSC Symposium on Neural Computation (ICSC Academic Press 2000) vol. 2000 pp. 115–121.
  6. R. V. K. OReddy B. S. Rao K. P. Raju Handwritten hindi digits recognition using convolutional neural network with rmsprop optimization in Second International Conference on Intelligent Computing and Control Systems (ICICCS) (IEEE 2018) pp. 45–51.
  7. T. Tieleman, G. Hinton, Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning 4, 26–31 (2012).
  8. H. Hung V. Adzic Monte carlo simulation of device variations and mismatch in analog intregrated circuits in Proceedings of the National Conference on Undergraduate Research (NCUR) (2006) pp. 1–8.

Cited by

Metadata

Authors
Kiani, F., Yin, J., Wang, Z., Yang, J. J., & Xia, Q.
Affiliations
University of Massachusetts Amherst (US)
Corresponding author
Xia, Qiangfei
Venue
Science Advances
Published
2021-11-26
Volume
7
Issue
48
Pages
eabj4801-eabj4801
ISSN
2375-2548 (ISSN-L)
OpenAlex
W3217269745
Cited by
97
References
45
Citations by year
2026: 20 · 2025: 21 · 2024: 24 · 2023: 23 · 2022: 9
FWCI
4.11
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Science Advances (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • ScholarWorks@UMassAmherst (University of Massachusetts Amherst) (submittedVersion, cc-by-nc)
Topics
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
Keywords
memristor crossbar, in-memory computing, analog neural networks, rectified linear units, multilayer perceptron, handwritten digit recognition, MNIST, energy efficiency, analog signal transmission

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.