Linear conductance update improvement of CMOS-compatible second-order memristors for fast and energy-efficient training of a neural network using a memristor crossbar array

Conventional oxide-based memristors exhibit severe non-linearity and asymmetry during LTP-LTD.

“However, severe non-linearity and asymmetry during the LTP-LTD are observed in Fig. 1b, similar to other conventional oxide-based memristors.23,32,33”

• cites [23] 23. C. Li, D. Belkin, Y. Li, P. Yan, M. Hu, N. Ge, H. Jiang, E. Montgomery, P. Lin, Z. Wang, W. Song, J. P. Strachan, M. Barnell, Q. Wu, R. S. Williams, J. J. Yang and Q. Xia, Efficient and self-adaptive in-situ learning in multilayer memristor neural networks, Nat. Commun., 2018, 9, DOI: 10.1038/s41467-018-04484-2. sourceSupported
• cites [32] 32. U. Celano, L. Goux, R. Degraeve, A. Fantini, O. Richard, H. Bender, M. Jurczak and W. Vandervorst, Imaging the three-dimensional conductive channel in filamentary-based oxide resistive switching memory, Nano Lett., 2015, 15, 7970-7975. sourceNot checked — no full text of the source
• cites [33] 33. S. Kim, S. Choi, J. Lee and W. D. Lu, Tuning resistive switching characteristics of tantalum oxide memristors through Si doping, ACS Nano, 2014, 8, 10262-10269. sourceNot checked — no full text of the source

Oxide-based memristors can be largely affected not only by the filament radius, but also by the device temperature.

“Second-order memristors have been actively demonstrated in previous studies, showing that oxide-based memristors can be largely affected not only by the filament radius, but also by the device temperature.34,37”

• cites [34] 34. W. Wu, H. Wu, B. Gao, P. Yao, X. Zhang, X. Peng, S. Yu and H. Qian, A Methodology to Improve Linearity of Analog RRAM for Neuromorphic Computing, 2018 IEEE Sympo- sium on VLSI Technology: 18-22, 2018. sourceNot checked — no full text of the source
• cites [37] 37. K. Yang, J. Joshua Yang, R. Huang and Y. Yang, Nonlinearity in memristors for neuromorphic dynamic systems, Small Science, 2022, 2, 2100049. sourceSupported

For non-identical pulse methods, an extra read operation is required to check the device's current conductance state before applying a SET or RESET pulse.

“For non-identical pulse methods, an extra read operation is first required to check the device's current conductance state.”

• cites [6] 6. S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. P. Farinha, B. Killeen, C. Cheng, Y. Jaoudi and G. W. Burr, Equivalent- accuracy accelerated neural-network training using analogue memory, Nature, 2018, 558, 60-67. sourceNot checked — no full text of the source
• cites [18] 18. S. O. Park, H. Jeong, J. Park, J. Bae and S. Choi, Experi- mental demonstration of highly reliable dynamic memris- tor for artificial neuron and neuromorphic computing, Nat. Commun., 2022, 13, DOI: 10.1038/s41467-022-30539-6. sourcePartly supported

  The Methods (Electrical measurement; page not provided) describe applying a read pulse after each bit to obtain the memristor’s conductance. They do not say that non-identical pulse methods require an extra read before a SET or RESET pulse to check the current state. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [46] 46. Y. Y. Chen, R. Roelofs, A. Redolfi, R. Degraeve, D. Crotti, A. Fantini, S. Clima, B. Govoreanu, M. Komura, L. Goux, L. Zhang, A. Belmonte, Q. Xie, J. Maes, G. Pourtois and M. Jurczak, Tailoring switching and endurance/retention reliability characteristics of HfO2/Hf RRAM with Ti, Al, Si dopants, 2014 Symposium on VLSI Technology Digest of Technical Papers, 2014. sourceNot checked — no full text of the source

The device is abruptly depressed from the low resistance state to the high resistance state.

“The device is abruptly potentiated from the high resistance state (HRS) to the low resistance state (LRS) and abruptly depressed from the LRS to the HRS due to abrupt filament growth and rupture, as shown in Fig. 1c.”

The fabricated device exhibits severe non-linearity and asymmetry during LTP-LTD.

“However, severe non-linearity and asymmetry during the LTP-LTD are observed in Fig. 1b, similar to other conventional oxide-based memristors.23,32,33”

A fully CMOS-compatible HfO2 memristor was fabricated on an 8-inch wafer.

“In this study, we fabricated a fully CMOS-compatible HfO 2 memristor on an 8-inch wafer and experimentally demonstrated a method to mitigate the severe non-linearity and asymmetry in the memristor.”

The heating pulse effect largely improves the potentiation linearity of the HfO2 memristor.

“By utilizing a previously reported heating pulse method, we confirmed that the CMOS-compatible HfO 2 memristor has a second-order memristor effect and demonstrated that the heating pulse effect largely improves the potentiation linearity of the HfO 2 memristor.24”

• cites [24] 24. J. H. Ryu and S. Kim, Artificial synaptic characteristics of TiO 2 /HfO 2 memristor with self-rectifying switching for brain-inspired computing, Chaos, Solitons Fractals, 2020, 140, DOI: 10.1016/j.chaos.2020.110236. sourceNot checked — no full text of the source

The CMOS-compatible HfO2 memristor has a second-order memristor effect.

“By utilizing a previously reported heating pulse method, we confirmed that the CMOS-compatible HfO 2 memristor has a second-order memristor effect and demonstrated that the heating pulse effect largely improves the potentiation linearity of the HfO 2 memristor.24”

• cites [24] 24. J. H. Ryu and S. Kim, Artificial synaptic characteristics of TiO 2 /HfO 2 memristor with self-rectifying switching for brain-inspired computing, Chaos, Solitons Fractals, 2020, 140, DOI: 10.1016/j.chaos.2020.110236. sourceNot checked — no full text of the source

The potentiation linearity and dynamic range are largely improved because the thermal effect increases as the device conductance increases.

“Therefore, the potentiation linearity and the dynamic range are largely improved because the thermal effect increases as the device conductance increases, which is similar to nonidentical pulse methods that apply a stronger potentiation pulse to the device near the LRS region while applying a weaker potentiation pulse to the device near the HRS region.”

The training accuracy of the memristor crossbar array-based neural network for handwritten digit classification is improved from 83.64% to 91.52%.

“The training accuracy of the memristor crossbar array-based neural network for hand written digit classification is improved from 83.64% to 91.52%, demonstrating that the linearity-improved two-terminal memristor crossbar array can be a good candidate for energyefficient, compact, and fast neural network systems.”

• AssumptionRests on an assumption: The model-based simulation realistically represents a memristor crossbar array-based neural network.

The device is abruptly potentiated from the high resistance state to the low resistance state.

“The device is abruptly potentiated from the high resistance state (HRS) to the low resistance state (LRS) and abruptly depressed from the LRS to the HRS due to abrupt filament growth and rupture, as shown in Fig. 1c.”

The fabricated device has a metal-insulator-metal structure (TiN/Ti/HfO2/TiN) on an eight-inch Si wafer.

“Fig. 1a and Fig. S1c, d (ESI †) show the metal-insulator-metal structure (TiN/Ti/HfO 2 /TiN) of the fabricated device on an eightinch Si wafer.”

The abrupt depression is remedied by voltage division with the voltage divider circuit.

“On the other hand, the abrupt depression is remedied by the voltage division with the voltage divider circuit, which is composed of a 680 O resistor (for this experiment), as shown in Fig. 1e.”

Severe non-linearity and asymmetry during the LTP-LTD are observed in the HfO2 memristor, similar to other conventional oxide-based memristors.

“However, severe non-linearity and asymmetry during the LTP-LTD are observed in Fig. 1b, similar to other conventional oxide-based”

Based on the second-order memristor effect and the voltage divider, the conductance update linearity and symmetry are largely improved without sacrificing the conductance on/off ratio.

“Based on the second-order memristor effect and the voltage divider, the conductance update linearity and symmetry are largely improved without sacrificing the conductance on/off ratio, as shown in Fig. 1g.”

The depression linearity of the HfO2 memristor is enhanced by connecting a simple passive resistor (voltage divider) serially with the device.

“In addition, the depression linearity is also enhanced by connecting a simple passive resistor (voltage divider) serially with the device.”

When the device is weakly potentiated, a larger Joule heat is observed because of the higher current flow.

“However, when the device is weakly potentiated, as shown in Fig. 2d, a larger Joule heat is observed because of the higher current flow.”

For a significantly large heating pulse amplitude, the device shows a non-linear potentiation curve.

“Therefore, for a significantly large heating pulse amplitude, the device shows a non-linear potentiation curve, and it is confirmed that the heating pulse amplitude must be optimized in a small range (0.1 to 0.4 V).”

During 500 epochs of training, the memristor-based neural network without the heating pulse and the voltage divider achieves 83.64% training accuracy.

“during 500 epochs of training, the memristor-based neural network without the heating pulse and the voltage divider achieves 83.64% training accuracy”

The potentiation linearity is improved by introducing a heating pulse to utilize the second-order memristor effect.

“The potentiation linearity is improved by introducing a heating pulse to utilize the second-order memristor effect”

The heating pulse effect becomes significant as the HfO2 thickness increases.

“The results show that the heating pulse barely affects the conductance update of a thin (5 nm) HfO 2 device, while the heating pulse effect becomes significant as the HfO 2 thickness increases.”

The memristor's conductance update becomes more linear and gradual when the heating pulse and the voltage divider are used.

“The memristor's conductance update becomes more linear and gradual when the heating pulse and the voltage divider are used”

The ideal case with zero non-linearity achieves 92.42% training accuracy.

“which is close to the training accuracy of the ideal case with zero non-linearity (92.42%)”

A larger heating pulse amplitude induces a larger on/off ratio.

“As shown in Fig. S6b (ESI †), the larger heating pulse amplitude induces a larger on/off ratio, but the device potentiation linearity becomes significantly degraded when the heating pulse amplitude exceeds 0.5 V.”

When the pulse interval decreased to 2 ms and 1 ms, the potentiation became much stronger as the pulse interval was reduced.

“However, when the pulse interval decreased to 2 ms and 1 ms, the potentiation became much stronger as the pulse interval was reduced.”

• AssumptionRests on an assumption: A second-order memristor responds differently to input frequency because the device temperature increases as the input frequency increases, while a first-order memristor is not affected by input frequency because it has less temperature dependency.

The improved characteristics with the second-order memristor effect and the voltage divider can be maintained during several cycles.

“Seven consecutive cycles of LTP-LTD curves of the memristor with and without the potentiation or depression linearity improvement are shown in Fig. S4 (ESI †), showing that the improved characteristics with the second-order memristor effect and the voltage divider can be maintained during several cycles.”

With the heating pulse and voltage divider present, the depression stage after potentiation shows gradual and linear conductance changes.

“The depression stage after potentiation also shows gradual and linear conductance changes.”

Hand-written digit classification of the memristor-based MLP using a simulator demonstrates that the linearity and symmetry improvements from the heating pulse and the voltage divider enable accurate training of a memristor-based neural network.

“Finally, hand-written digit classification of the memristor-based MLP using a simulator demonstrates that the linearity and symmetry improvements from the heating pulse and the voltage divider enable the accurate training of a memristor-based neural network.”

The heating pulse and voltage divider have little impact on the cycle-to-cycle variation.

“Through the 10 consecutive LTP-LTD cycles for the HfO 2 device with/without the linearity improvements (the heating pulse and voltage divider), the variation coefficients (s/m) of LRS and HRS are obtained and demonstrate that the heating pulse and voltage divider have little impact on the cycle-to-cycle variation.”

The neural network based on the memristor with the heating pulse and voltage dividers achieves 91.52% training accuracy.

“the neural network based on the memristor with the heating pulse and voltage dividers successfully trains the weights and achieves 91.52% accuracy”

The device-to-device variations with and without linearity improvements are similar.

“As shown in Fig. S10 (ESI †), the device-to-device variation is more significant than the cycle-to-cycle variation in both cases, but the device-to-device variations with/without linearity improvements are similar.”

The heating pulse barely affects the conductance update of a thin (5 nm) HfO2 device.

“The results show that the heating pulse barely affects the conductance update of a thin (5 nm) HfO 2 device, while the heating pulse effect becomes significant as the HfO 2 thickness increases.”

The fabricated memristor in this study shows abrupt depression when a single RESET pulse (À0.8 V, 1 ms) is applied to the device in the LRS.

“The fabricated memristor in this study also shows abrupt depression when a single RESET pulse (À0.8 V, 1 ms) is applied to the device in the LRS, as shown in Fig. 3a.”

The dynamic range increases as the heating pulse length increases.

“The results show that heating pulse lengths longer than 2 ms affect the potentiation, and the dynamic range increases as the heating pulse length increases, demonstrating the effectiveness of the heating pulse for the second-order memristor.”

The depression linearity is improved by connecting a voltage divider circuit composed of a resistor and two diodes to properly adjust the effective RESET voltage.

“while the depression linearity is improved by connecting a voltage divider circuit composed of a resistor and two diodes to properly adjust the effective RESET voltage.”

Adding the voltage divider does not significantly deteriorate the array integration density.

“It should be noted that adding the voltage divider does not significantly deteriorate the array integration density (Fig. S9, ESI †).”

The HfO2 memristor with a voltage divider is fully depressed to the HRS by 100 RESET pulses while maintaining good linearity and gradual depression properties.

“Compared with the depression curve without the voltage divider, Fig. 3d shows that the memristor is fully depressed to the HRS by 100 RESET pulses while maintaining good linearity and gradual depression properties.”

With the heating pulse and voltage divider present, arbitrary SET pulses gradually potentiate the conductance with a larger on/off ratio due to the second-order memristor effect.

“In this case, arbitrary SET pulses gradually potentiate the conductance with a larger on/off ratio due to the second-order memristor effect.”

The memristor was weakly potentiated during the 100 consecutive SET pulses when the pulse interval was 5 ms.

“As shown in Fig. S5 (ESI †), the memristor was weakly potentiated during the 100 consecutive SET pulses when the pulse interval was 5 ms.”

• AssumptionRests on an assumption: A second-order memristor responds differently to input frequency because the device temperature increases as the input frequency increases, while a first-order memristor is not affected by input frequency because it has less temperature dependency.

Heating pulse lengths longer than 2 ms affect the potentiation.

“The results show that heating pulse lengths longer than 2 ms affect the potentiation, and the dynamic range increases as the heating pulse length increases, demonstrating the effectiveness of the heating pulse for the second-order memristor.”

When the device is fully potentiated, about 800 K of Joule heat is detected in the simulation.

“Finally, when the device is fully potentiated as shown in Fig. 2e, about 800 K of Joule heat is detected in the simulation.”

• Also stated in: § This journal is © The Royal Society of Chemistry 2023 the device structure and material properties (Fig. 2c-e).39 To examine the effect of Joule heat for each conductance state of the memristor, simulations have been performed for three different conductance states (RESET, weak SET, and SET states), as shown in Fig. 2c-e.Because the filament is formed by removing oxygen anions from the HfO 2 layer, the filament is modeled as non-stoichiometric HfO x .31 In the RESET state, the oxygen anions move to the HfO 2 layer from the top electrode, and rupture part of the filament by removing the oxygen vacancy.Therefore, the HfO 2 gap is implemented in the filament for simulating the fully RESET state in Fig. 2c.The filament geometry is modeled by referring to previous studies about the filament shape in HfO 2 -based memristors, which demonstrate the relationship between the filament shape (diameter) and the device conductance.[27][28][29][30][31] When a 1 ms, 0.4 V pulse is applied to the HfO 2 memristor in the RESET state, a small amount of heat is observed in the filament due to the low current flow through the filament, and the thermal enhancing effect is not dominant.However, when the device is weakly potentiated, as shown in Fig. 2d, a larger Joule heat is observed because of the higher current flow.Finally, when the device is fully potentiated as shown in Fig. 2e, about 800 K of Joule heat is detected in the simulation.The simulation results support the improvement of the potentiation linearity through the heating pulse, as the thermal enhancing effect of the memristor becomes stronger as the device conductance and Joule heat increase, which agrees with the experimental results shown in Fig. 2a. The optimal range of the heating pulse amplitude and the effects of the HfO 2 thickness on the second-order memristor effect are revealed by applying various amplitudes of heating pulses to the fabricated second-order HfO 2 memristors with 5, 7.5, and 10 nm of HfO 2 thicknesses (Fig. S6, ESI †).The conductance of each device during 100 consecutive set pulses with various heating pulse amplitudes (0 to 0.5 V with 0.1 V interval), followed by 100 consecutive reset pulses.The results show that the heating pulse barely affects the conductance update of a thin (5 nm) HfO 2 device, while the heating pulse effect becomes significant as the HfO 2 thickness increases.These results demonstrate that the thermal effect is significant for a device with a thicker oxide layer because a thicker oxide layer prevents the generated Joule heat from escaping the device and makes the filament temperature higher than a thinner oxide case.In addition to the effect of the oxide thickness, the optimal range of the heating pulse amplitude is disclosed.As shown in Fig. S6b (ESI †), the larger heating pulse amplitude induces a larger on/off ratio, but the device potentiation linearity becomes significantly degraded when the heating pulse amplitude exceeds 0.5 V.This is because a significantly large heating pulse amplitude (0.5 V) directly makes oxygen anions move to the top electrode and changes the device conductance (Fig. S7, ESI †), as well as generates the Joule heating effect.Therefore, for a significantly large heating pulse amplitude, the device shows a non-linear potentiation curve, and it is confirmed that the heating pulse amplitude must be optimized in a small range (0.1 to 0.4 V). · plain

  “fully potentiated as shown in Fig. 2e, about 800 K of Joule heat is detected in the simulation.”

The device-to-device variation is more significant than the cycle-to-cycle variation in both cases.

“As shown in Fig. S10 (ESI †), the device-to-device variation is more significant than the cycle-to-cycle variation in both cases, but the device-to-device variations with/without linearity improvements are similar.”

Without the heating pulse and voltage divider, a single SET (RESET) pulse completely potentiates (depresses) the conductance state.

“As shown in Fig. 4b, a single SET (RESET) pulse completely potentiates (depresses) the conductance state, and it might deteriorate the accuracy of the neural network, as depicted in Fig. 4a.”

The device potentiation linearity becomes significantly degraded when the heating pulse amplitude exceeds 0.5 V.

“As shown in Fig. S6b (ESI †), the larger heating pulse amplitude induces a larger on/off ratio, but the device potentiation linearity becomes significantly degraded when the heating pulse amplitude exceeds 0.5 V.”

Memristor crossbar array-based neural networks usually suffer from low accuracy because of non-ideal factors of memristors such as non-linearity and asymmetry, which prevent weights from being programmed to their targeted values.

“However, memristor crossbar array-based neural networks usually suffer from low accuracy because of the non-ideal factors of memristors such as non-linearity and asymmetry, which prevent weights from being programmed to their targeted values.”

• Also stated in: § Introduction Artificial neural networks, powered by today's well-matured digital computers, enable several difficult tasks, such as autonomous driving, object detection, and natural language processing, and they have been widely utilized in various industries.[1][2][3][4] However, conventional von Neumann computing is not desirable for artificial neural networks because it requires data transfer between the memory and processing units.5 In a von Neumann computer, the data are stored in a memory unit, but processed in a processing unit that is physically separated from the memory.Usually, training artificial neural networks requires a large amount of training data, commonly called big data; thus, a huge amount of energy is consumed only for data movement.6 In addition, computations in an artificial neural network are composed of simple but massive vector-matrix multiplications.Conventional processing units, such as central processing units (CPUs) and graphics processing units (GPUs), are favored for fast and complex arithmetic calculations, but they can only process data in parallel with a limited number of cores.7 This means that they are not able to process big data in parallel due to the limited a The School of Electrical Engineering, Korea Advanced Institute of Science and number of cores and unnecessary big data movement.Therefore, new computing hardware that can process massive simple vector-matrix multiplications in parallel without data movement between memory and processing units has been extensively studied in the field of artificial neural network hardware. Memristors, which are two-terminal devices that change their electrical conductance and store analog values, have been widely employed as synapses in artificial neural networks due to their simple structure, non-volatile analog memory properties, and inherent ability to process vector-matrix multiplication when they are fabricated in a crossbar array structure.[8][9][10][11][12][13][14][15][16] Based on the desirable characteristics of the memristor crossbar array, various studies have demonstrated that the energy efficiency, hardware size, and inference speed can be effectively improved compared with the conventional CMOS-based processors 12,17,18 Despite the effectiveness of memristors as synapses in neural network systems, it is hard to train the weights of the neural network in a memristor crossbar array.Two-terminal memristors show long retention and fast speed, but they usually undergo severe non-linear and asymmetric update properties.6,8,9,19,20 These prevent a memristor from being programmed to a targeted conductance state because even a single potentiation or depression pulse can induce a catastrophic conductance change (Dw). To mitigate this non-linearity issue, non-identical pulse methods are usually utilized to program the memristor into the targeted value; for example, applying a short or small amplitude SET (RESET) pulse when the device is fully depressed (potentiated) to suppress abrupt potentiation (depression), and applying a long or high amplitude SET (RESET) pulse when the device is already potentiated (depressed) to enhance the potentiation (depression).[21][22][23] These non-identical pulse methods, called incremental pulse schemes, are effective at programming the device to the targeted value, but they require several peripheral circuits with additional steps and worsen the energy consumption, time consumption, and hardware size of memristor-based neural network systems.For these reasons, it is hard to commercialize memristor-based neural network systems if the energy, time, and area efficiencies are lower than those of conventional CMOS-based neural network systems. In this study, we fabricated a fully CMOS-compatible HfO 2 memristor on an 8-inch wafer and experimentally demonstrated a method to mitigate the severe non-linearity and asymmetry in the memristor.By utilizing a previously reported heating pulse method, we confirmed that the CMOS-compatible HfO 2 memristor has a second-order memristor effect and demonstrated that the heating pulse effect largely improves the potentiation linearity of the HfO 2 memristor.24 In addition, the depression linearity is also enhanced by connecting a simple passive resistor (voltage divider) serially with the device.The developed method effectively reduces the time and energy consumption for the write and erase processes while improving the update linearity and symmetry.The training accuracy of the memristor crossbar array-based neural network for hand written digit classification is improved from 83.64% to 91.52%, demonstrating that the linearity-improved two-terminal memristor crossbar array can be a good candidate for energyefficient, compact, and fast neural network systems. · plain

  “Two-terminal memristors show long retention and fast speed, but they usually undergo severe non-linear and asymmetric update properties.6,8,9,19,20 These prevent a memristor from being programmed to a targeted conductance state because even a single potentiation or depression pulse can induce a catastrophic conductance change (Dw).”

• cites [6] 6. S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. P. Farinha, B. Killeen, C. Cheng, Y. Jaoudi and G. W. Burr, Equivalent- accuracy accelerated neural-network training using analogue memory, Nature, 2018, 558, 60-67. sourceNot checked — no full text of the source
• cites [8] 8. S. Srivastava, J. P. Thomas and K. T. Leung, Programmable, electroforming-free TiO x /TaO x heterojunction-based non- volatile memory devices, Nanoscale, 2019, 11, 18159-18168. sourceNot checked — no full text of the source
• cites [9] 9. S. Choi, S. H. Tan, Z. Li, Y. Kim, C. Choi, P. Y. Chen, H. Yeon, S. Yu and J. Kim, SiGe epitaxial memory for neuromorphic computing with reproducible high perfor- mance based on engineered dislocations, Nat. Mater., 2018, 17, 335-340. sourceNot checked — no full text of the source
• cites [19] 19. F. Cai, J. M. Correll, S. H. Lee, Y. Lim, V. Bothra, Z. Zhang, M. P. Flynn and W. D. Lu, A fully integrated reprogram- mable memristor-CMOS system for efficient multiply-accu- mulate operations, Nat. Electron., 2019, 2, 290-299. sourceNot checked — no full text of the source
• cites [20] 20. J. Tang, C. He, J. Tang, K. Yue, Q. Zhang, Y. Liu, Q. Wang, S. Wang, N. Li, C. Shen, Y. Zhao, J. Liu, J. Yuan, Z. Wei, J. Li, K. Watanabe, T. Taniguchi, D. Shang, S. Wang, W. Yang, R. Yang, D. Shi and G. Zhang, A reliable all-2D materials artificial synapse for high energy-efficient neuromorphic computing, Adv. Funct. Mater., 2021, 31, DOI: 10.1002/adfm.202011083. sourceNot checked — no full text of the source

It is hard to train the weights of a neural network in a memristor crossbar array.

“Despite the effectiveness of memristors as synapses in neural network systems, it is hard to train the weights of the neural network in a memristor crossbar array.”

• cites [6] 6. S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. P. Farinha, B. Killeen, C. Cheng, Y. Jaoudi and G. W. Burr, Equivalent- accuracy accelerated neural-network training using analogue memory, Nature, 2018, 558, 60-67. sourceNot checked — no full text of the source
• cites [8] 8. S. Srivastava, J. P. Thomas and K. T. Leung, Programmable, electroforming-free TiO x /TaO x heterojunction-based non- volatile memory devices, Nanoscale, 2019, 11, 18159-18168. sourceNot checked — no full text of the source
• cites [9] 9. S. Choi, S. H. Tan, Z. Li, Y. Kim, C. Choi, P. Y. Chen, H. Yeon, S. Yu and J. Kim, SiGe epitaxial memory for neuromorphic computing with reproducible high perfor- mance based on engineered dislocations, Nat. Mater., 2018, 17, 335-340. sourceNot checked — no full text of the source
• cites [19] 19. F. Cai, J. M. Correll, S. H. Lee, Y. Lim, V. Bothra, Z. Zhang, M. P. Flynn and W. D. Lu, A fully integrated reprogram- mable memristor-CMOS system for efficient multiply-accu- mulate operations, Nat. Electron., 2019, 2, 290-299. sourceNot checked — no full text of the source
• cites [20] 20. J. Tang, C. He, J. Tang, K. Yue, Q. Zhang, Y. Liu, Q. Wang, S. Wang, N. Li, C. Shen, Y. Zhao, J. Liu, J. Yuan, Z. Wei, J. Li, K. Watanabe, T. Taniguchi, D. Shang, S. Wang, W. Yang, R. Yang, D. Shi and G. Zhang, A reliable all-2D materials artificial synapse for high energy-efficient neuromorphic computing, Adv. Funct. Mater., 2021, 31, DOI: 10.1002/adfm.202011083. sourceNot checked — no full text of the source

Conventional von Neumann computing is not desirable for artificial neural networks because it requires data transfer between the memory and processing units.

“However, conventional von Neumann computing is not desirable for artificial neural networks because it requires data transfer between the memory and processing units.5”

• cites [5] 5. Q. Xia and J. J. Yang, Memristive crossbar arrays for brain- inspired computing, Nat. Mater., 2019, 18, 309-323. sourceNot checked — no full text of the source

The thermal effect is enhanced as the device conductance increases due to Joule heating.

“This thermal effect is enhanced as the device conductance increases due to Joule heating.”

The voltage division makes the effective RESET voltage stronger as the device conductance decreases.

“This makes the effective RESET voltage stronger as the device conductance decreases, which is similar to the non-identical pulse methods applying a stronger RESET voltage as the device conductance is smaller.”

The abrupt potentiation and depression are due to abrupt filament growth and rupture.

“The device is abruptly potentiated from the high resistance state (HRS) to the low resistance state (LRS) and abruptly depressed from the LRS to the HRS due to abrupt filament growth and rupture, as shown in Fig. 1c.”

Improving the memristor device's linearity and symmetry opens up the possibility of a trainable memristor crossbar array-based neural network system that possesses great energy efficiency, high area efficiency, and high accuracy at the same time.

“By improving the memristor device's linearity and symmetry, our results open up the possibility of a trainable memristor crossbar array-based neural network system that possesses great energy efficiency, high area efficiency, and high accuracy at the same time.”

When a heating pulse is applied, the device's temperature increases and oxygen anions can be moved more easily with the following SET pulse relative to the case without a heating pulse.

“When the heating pulse is applied, the device's temperature increases and the oxygen anions can be moved more easily with the following SET pulse relative to the case without a heating pulse.”

The accuracy of the memristor-based neural network is largely degraded by the non-linearity and asymmetry.

“The non-linearity and asymmetry prevent the analog and gradual programming of the device to the targeted value; therefore, the accuracy of the memristor-based neural network is largely degraded.”

Severe non-linear and asymmetric update properties prevent a memristor from being programmed to a targeted conductance state because even a single potentiation or depression pulse can induce a catastrophic conductance change.

“These prevent a memristor from being programmed to a targeted conductance state because even a single potentiation or depression pulse can induce a catastrophic conductance change (Dw).”

• cites [6] 6. S. Ambrogio, P. Narayanan, H. Tsai, R. M. Shelby, I. Boybat, C. di Nolfo, S. Sidler, M. Giordano, M. Bodini, N. C. P. Farinha, B. Killeen, C. Cheng, Y. Jaoudi and G. W. Burr, Equivalent- accuracy accelerated neural-network training using analogue memory, Nature, 2018, 558, 60-67. sourceNot checked — no full text of the source
• cites [8] 8. S. Srivastava, J. P. Thomas and K. T. Leung, Programmable, electroforming-free TiO x /TaO x heterojunction-based non- volatile memory devices, Nanoscale, 2019, 11, 18159-18168. sourceNot checked — no full text of the source
• cites [9] 9. S. Choi, S. H. Tan, Z. Li, Y. Kim, C. Choi, P. Y. Chen, H. Yeon, S. Yu and J. Kim, SiGe epitaxial memory for neuromorphic computing with reproducible high perfor- mance based on engineered dislocations, Nat. Mater., 2018, 17, 335-340. sourceNot checked — no full text of the source
• cites [19] 19. F. Cai, J. M. Correll, S. H. Lee, Y. Lim, V. Bothra, Z. Zhang, M. P. Flynn and W. D. Lu, A fully integrated reprogram- mable memristor-CMOS system for efficient multiply-accu- mulate operations, Nat. Electron., 2019, 2, 290-299. sourceNot checked — no full text of the source
• cites [20] 20. J. Tang, C. He, J. Tang, K. Yue, Q. Zhang, Y. Liu, Q. Wang, S. Wang, N. Li, C. Shen, Y. Zhao, J. Liu, J. Yuan, Z. Wei, J. Li, K. Watanabe, T. Taniguchi, D. Shang, S. Wang, W. Yang, R. Yang, D. Shi and G. Zhang, A reliable all-2D materials artificial synapse for high energy-efficient neuromorphic computing, Adv. Funct. Mater., 2021, 31, DOI: 10.1002/adfm.202011083. sourceNot checked — no full text of the source

Non-linearity and asymmetry prevent analog and gradual programming of the device to the targeted value.

“The non-linearity and asymmetry prevent the analog and gradual programming of the device to the targeted value; therefore, the accuracy of the memristor-based neural network is largely degraded.”

The HfO2 memristor changes its conductance by growing or rupturing an internal conductive filament, which consists of oxygen vacancies.

“The HfO 2 memristor changes its conductance by growing or rupturing an internal conductive filament, which consists of oxygen vacancies (Fig. S2, ESI †).[25][26][27]”

• cites [25][26][27]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The linearity-improved two-terminal memristor crossbar array can be a good candidate for energy-efficient, compact, and fast neural network systems.

“The training accuracy of the memristor crossbar array-based neural network for hand written digit classification is improved from 83.64% to 91.52%, demonstrating that the linearity-improved two-terminal memristor crossbar array can be a good candidate for energyefficient, compact, and fast neural network systems.”

Potentiation linearity can be improved by introducing a heating pulse to utilize the device temperature as a second variable.

“A previous study has demonstrated that potentiation linearity can be improved by introducing a heating pulse to utilize the device temperature as a second variable.24”

• cites [24] 24. J. H. Ryu and S. Kim, Artificial synaptic characteristics of TiO 2 /HfO 2 memristor with self-rectifying switching for brain-inspired computing, Chaos, Solitons Fractals, 2020, 140, DOI: 10.1016/j.chaos.2020.110236. sourceNot checked — no full text of the source

It is hard to commercialize memristor-based neural network systems if their energy, time, and area efficiencies are lower than those of conventional CMOS-based neural network systems.

“For these reasons, it is hard to commercialize memristor-based neural network systems if the energy, time, and area efficiencies are lower than those of conventional CMOS-based neural network systems.”

In the RESET state, oxygen anions move to the HfO2 layer from the top electrode, and rupture part of the filament by removing the oxygen vacancy.

“In the RESET state, the oxygen anions move to the HfO 2 layer from the top electrode, and rupture part of the filament by removing the oxygen vacancy.”

The proposed method does not require any extra steps to SET or RESET the memristor, unlike non-identical pulse methods.

“The proposed method in this study, contrary to non-identical pulse methods, does not require any extra steps to SET (RESET) the memristor”

The simulation results support that the heating pulse improves potentiation linearity.

“The simulation results support the improvement of the potentiation linearity through the heating pulse, as the thermal enhancing effect of the memristor becomes stronger as the device conductance and Joule heat increase, which agrees with the experimental results shown in Fig. 2a.”

When the weight has catastrophic potentiation and depression characteristics, it is difficult for the weight to maintain the desired value even after saturation because a single potentiation or depression signal might completely change the weight value.

“However, when the weight has catastrophic potentiation and depression characteristics, it is difficult for the weight to maintain the desired value even after saturation because a single potentiation or depression signal might completely change the weight value, as shown in Fig. 4a.”

The heating pulse is effective for the second-order memristor.

“The results show that heating pulse lengths longer than 2 ms affect the potentiation, and the dynamic range increases as the heating pulse length increases, demonstrating the effectiveness of the heating pulse for the second-order memristor.”

The gradual and linear potentiation and depression processes enable a reliable artificial neural network based on the memristor crossbar array because catastrophic changes in the weights are eliminated.

“The gradual and linear potentiation and depression processes enable a reliable artificial neural network based on the memristor crossbar array because catastrophic changes in the weights are eliminated.”

The linearity and symmetry of the memristor are significant for the training of the neural network.

“emphasizing the significance of the linearity and symmetry of the memristor for the training of the neural network.”

When the RESET pulse is applied, the applied voltage is divided between the resistor, a PN diode (in series with the resistor), and the memristor.

“However, when the RESET pulse is applied, the applied voltage is divided between the resistor, a PN diode (in series with the resistor), and the memristor.”

The proposed method provides a way to realize energy-efficient, accurate, and fast training of memristor-based neural network systems.

“therefore, it provides a way to realize energy-efficient, accurate, and fast training of memristor-based neural network systems.”

• AssumptionRests on an assumption: Memristors' non-linearity and asymmetry during a conductance update make it difficult to accurately tune a memristor to a target value and therefore prevent the development of energy-efficient, compact, and time-efficient memristor-based neural network systems.

Proper selection of the series resistor is important for the depression linearity.

“The effect of the series resistor's resistance on the depression linearity is shown in Fig. S8 (ESI †), which demonstrates that the proper selection of the resistor is important for the depression linearity.”

The heating pulse amplitude must be optimized in a small range (0.1 to 0.4 V).

“Therefore, for a significantly large heating pulse amplitude, the device shows a non-linear potentiation curve, and it is confirmed that the heating pulse amplitude must be optimized in a small range (0.1 to 0.4 V).”

A single SET or RESET pulse might deteriorate the accuracy of the neural network.

“As shown in Fig. 4b, a single SET (RESET) pulse completely potentiates (depresses) the conductance state, and it might deteriorate the accuracy of the neural network, as depicted in Fig. 4a.”

Severe nonlinearity and asymmetry prevent the memristors from being programmed into proper states, causing the lower training accuracy.

“because severe nonlinearity and asymmetry prevent the memristors from being programmed into proper states.”

The effective voltage across the memristor increases as the device conductance decreases (resistance increases).

“This makes the effective voltage across the memristor increase as the device conductance (resistance) decreases (increases).”

The hardware burden to program the memristor is reduced.

“As a result, the hardware burden to program the memristor is reduced as the voltage divider can be connected to the row or column of the crossbar array and does not require a large area, and the heating pulse can be easily implemented by adding a small voltage pulse before the original SET pulse.”

The thermal effect is significant for a device with a thicker oxide layer because a thicker oxide layer prevents the generated Joule heat from escaping the device and makes the filament temperature higher than in a thinner oxide case.

“These results demonstrate that the thermal effect is significant for a device with a thicker oxide layer because a thicker oxide layer prevents the generated Joule heat from escaping the device and makes the filament temperature higher than a thinner oxide case.”

There is no reason to read the device conductance or select the proper programming pulse before applying it using energy-hungry read-out circuits and bulky peripheral circuitry.

“therefore, there is no reason to read the device conductance or select the proper programming pulse before applying it by using energy-hungry read-out circuits and a bulky peripheral circuitry.”

Several memristors suffer from the catastrophic depression phenomenon, in which the stored information in the memristor is completely erased by a single RESET pulse.

“Several memristors suffer from the catastrophic depression phenomenon, in which the stored information in the memristor is completely erased by a single RESET pulse.[40][41][42][43]”

• cites [40][41][42][43]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The proposed method to improve device linearity does not require a significant hardware burden.

“The proposed method to improve device linearity does not require a significant hardware burden”

The final array integration density will be smaller compared to the memristor crossbar array utilizing the non-identical pulse method.

“Therefore, considering that the developed method does not require peripheral circuits for the non-identical pulse method, the final array integration density will be smaller compared to the memristor crossbar array utilizing the non-identical pulse method.”

• AssumptionRests on an assumption: The non-identical pulse method requires peripheral circuits.

The thermal enhancing effect of the memristor becomes stronger as the device conductance and Joule heat increase.

“the thermal enhancing effect of the memristor becomes stronger as the device conductance and Joule heat increase”