The DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index.
A Flexible and Fast PyTorch Toolkit for Simulating Training and Inference on Analog Crossbar Arrays
Open accessAbstract
We introduce the IBM ANALOG HARDWARE ACCELERATION KIT, a new and first of a kind open source toolkit to simulate analog crossbar arrays in a convenient fashion from within PYTORCH (freely available at https://github.com/IBM/aihwkit). The toolkit is under active development and is centered around the concept of an “analog tile” which captures the computations performed on a crossbar array. Analog tiles are building blocks that can be used to extend existing network modules with analog components and compose arbitrary artificial neural networks (ANNs) using the flexibility of the PYTORCH framework. Analog tiles can be conveniently configured to emulate a plethora of different analog hardware characteristics and their non-idealities, such as device-to-device and cycle-to-cycle variations, resistive device response curves, and weight and output noise. Additionally, the toolkit makes it possible to design custom unit cell configurations and to use advanced analog optimization algorithms such as Tiki-Taka. Moreover, the backward and update behavior can be set to “ideal" to enable hardware-aware training features for chips that target inference acceleration only. To evaluate the inference accuracy of such chips over time, we provide statistical programming noise and drift models calibrated on phase-change memory hardware. Our new toolkit is fully GPU accelerated and can be used to conveniently estimate the impact of material properties and non-idealities of future analog technology on the accuracy for arbitrary ANNs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, the IBM toolkit [18] does not assess the hardware chip's runtime, latency, and power performance.”
From Framework for In-Memory Computing Based on Memristor and Memcapacitor for On-Chip Training (2023) · cites this paper as [18]
Citations
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
Memory devices and applications for in-memory computing
Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R., & Eleftheriou, E. (2020) · Nature Nanotechnology · doi:10.1038/s41565-020-0655-z · Cited by 2,289
Self-citationNeuromorphic computing using non-volatile memory
Burr, G. W., Shelby, R. M., Sebastian, A., Kim, S., Kim, S., Sidler, S., Virwani, K. R., Ishii, M., Narayanan, P., Fumarola, A., et al. (2016) · Advances in Physics X · doi:10.1080/23746149.2016.1259585 · Cited by 1,196
Self-citationGokmen, T., & Vlasov, Y. (2016) · Frontiers in Neuroscience · doi:10.3389/fnins.2016.00333 · Cited by 491
Self-citationAccurate deep neural network inference using computational phase-change memory
Joshi, V., Le Gallo, M., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. (2020) · Nature Communications · doi:10.1038/s41467-020-16108-9 · Cited by 490 · cited 3×
“It has become increasingly apparent that accuracy on chips that are designed for inference benefit greatly from “hardwareaware” training, where the DNN workload is trained (or finetuned) in software with some analog noise and non-idealities in the forward pass, but with otherwise perfect backward and update pass [3],…”
BackgroundMethodSelf-citationChen, P.-Y., Peng, X., & Yu, S. (2017) · Technical digest - International Electron Devices Meeting/Technical digest · doi:10.1109/iedm.2017.8268337 · Cited by 453
Accurate deep neural network inference using computational phase-change memory
Joshi, V., Gallo, M. L., Haefeli, S., Boybat, I., Nandakumar, S. R., Piveteau, C., Dazzi, M., Rajendran, B., Sebastian, A., & Eleftheriou, E. S. (2020) · RePEc: Research Papers in Economics · doi:10.3929/ethz-b-000416797 · Cited by 378
Self-citationPeng, X., Huang, S., Jiang, H., Lu, A., & Yu, S. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3043731 · Cited by 317
The Next Generation of Deep Learning Hardware: Analog Computing
Hänsch, W., Gokmen, T., & Puri, R. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2871057 · Cited by 244
Self-citationRxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars
Jain, S., Sengupta, A., Roy, K., & Raghunathan, A. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3000185 · Cited by 151
Signal and noise extraction from analog memory elements for neuromorphic computing
Gong, N., Idé, T., Kim, S., Boybat, I., Sebastian, A., Narayanan, V., & Ando, T. (2018) · Nature Communications · doi:10.1038/s41467-018-04485-1 · Cited by 112
Self-citationAlgorithm for Training Neural Networks on Resistive Device Arrays
Gokmen, T., & Hänsch, W. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.00103 · Cited by 105
Self-citationTsai, H., Ambrogio, S., Mackin, C., Narayanan, P., Shelby, R. M., Rocki, K., Chen, A., & Burr, G. W. (2019) · 2019 Symposium on VLSI Technology · doi:10.23919/vlsit.2019.8776519 · Cited by 47
The marriage of training and inference for scaled deep learning analog hardware
Gokmen, T., Rasch, M. J., & Haensch, W. (2019) · 2019 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm19573.2019.8993573 · Cited by 37 · cited 1×
“…apparent that accuracy on chips that are designed for inference benefit greatly from “hardware-aware” training, where the DNN workload is trained (or fine-tuned) in software with some analog noise and non-idealities in the forward pass, but with otherwise perfect backward and update pass [4,8,10,14].”
Self-citationNeural network accelerator design with resistive crossbars: Opportunities and challenges
Jain, S., Ankit, A., Chakraborty, I., Gokmen, T., Rasch, M. J., Hänsch, W., Roy, K., & Raghunathan, A. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2947011 · Cited by 27
Self-citation
Cited by
Sorted by citations
Deep physical neural networks trained with backpropagation
Wright, L. G., Onodera, T., Stein, M. M., Wang, T., Schachter, D. T., Hu, Z., & McMahon, P. L. (2022) · Nature · doi:10.1038/s41586-021-04223-6 · Cited by 799
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
Self-citationLe Gallo, M., Khaddam-Aljameh, R., Stanisavljevic, M., Vasilopoulos, A., Kersting, B., Dazzi, M., Karunaratne, G., Brändli, M., Singh, A., Müller, S. M., et al. (2023) · Nature Electronics · doi:10.1038/s41928-023-01010-1 · Cited by 310
Self-citationResearch Progress on Memristor: From Synapses to Computing Systems
Yang, X., Taylor, B., Wu, A., Chen, Y., & Chua, L. O. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3159153 · Cited by 207
Rasch, M. J., Mackin, C., Le Gallo, M., Chen, A., Fasoli, A., Odermatt, F., Li, N., Nandakumar, S. R., Narayanan, P., Tsai, H., et al. (2023) · Nature Communications · doi:10.1038/s41467-023-40770-4 · Cited by 213
“Here, clip b a ð x Þ constrains z between minimum a and maximum b , α is a scalar, per-crossbar value which determines the usable input range.”
BackgroundMethodSelf-citation
Metadata
- Authors
- Rasch, M. J., Moreda, D., Gokmen, T., Le Gallo, M., Carta, F., Goldberg, C., El Maghraoui, K., Sebastian, A., & Narayanan, V.
- Affiliations
- IBM (United States) (US)
- IBM Research
- Venue
- 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS)
- Publisher
- IEEE
- Published
- 2021-06-06
- Pages
- 1-4
- OpenAlex
- W3148526109
- Cited by
- 160
- References
- 16
- Citations by year
- 2026: 28 · 2025: 36 · 2024: 40 · 2023: 32 · 2022: 18 · 2021: 6
- FWCI
- 37.68
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Rensselaer Polytechnic Institute
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Ferroelectric and Negative Capacitance Devices
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Reservoir Computing
- Keywords
- analog neural networks, hardware-aware training, phase-change memory, device-to-device variation, cycle-to-cycle variation, hardware acceleration, GPU-accelerated simulation, open-source software toolkit
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
