Uncertainty Modeling of Emerging Device based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
emerging device based Computing-in-memory (CiM) has been proved to be a promising candidate for high energy efficiency deep neural network (DNN) computations. However, most emerging devices suffer uncertainty issues, resulting in a difference between actual data stored and the weight value it is design to be. This leads to an accuracy drop from trained models to actually deployed platforms. In this work, we offer a thorough analysis on the effect of such uncertainties induced changes in DNN models. To reduce the impact of device uncertainties, we propose UAE, a uncertainty-aware Neural Architecture Search scheme to identify a DNN model that is both accurate and robust against device uncertainties.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The design parameters search can be performed using reinforcement learning264,266, evolutionary algorithms259,260,263,265, or differential methods261.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [264]
The cited source says what the claim attributes to it.
In the citing paper
“References 263,264, take memristor device variations into consideration when searching for the optimum software-related neural network parameters.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [264]
The cited source says what the claim attributes to it.
In the citing paper
“Such optimization lies within the domain of hardware-aware neural architecture search, which optimizes the design parameters of the neural network considering hardware feedback261–264, or in some cases, searches for the optimum hardware parameters265,266.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [264]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“3.2, we train a LeNet model for the MNIST dataset [11] to state-of-the-art accuracy.”
MethodInfluentialLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Learning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00907 · Cited by 7,831
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017) · Proceedings of the IEEE · doi:10.1109/jproc.2017.2761740 · Cited by 4,215 · cited 1×
“CiM architectures offer reduced data movement by in-situ weight data access [17].”
BackgroundNeural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
In-memory computing with resistive switching devices
Ielmini, D., & Wong, H.-S. P. (2018) · Nature Electronics · doi:10.1038/s41928-018-0092-2 · Cited by 2,298 · cited 1×
“Emerging device based Compute-in-Memory (CiM) neural accelerators [5] offer a great opportunity to break the memory wall with special architectural advantages.”
BackgroundInternational Conference on Learning Representations (ICLR)
Abbas, M. Z. A., Kivinen, J., & Raiko, T. (2016) · Cited by 1,694
ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001139 · Cited by 1,536
FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Wu, B., Keutzer, K., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., & Jia, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01099 · Cited by 1,261
Chi, P., Li, S., Xu, C., Zhang, T., Zhao, J., Liu, Y., Wang, Y., & Xie, Y. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001140 · Cited by 1,231
ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.12 · Cited by 775
Chi, P., Li, S., Xu, C., Zhang, T., Zhao, J., Liu, Y., Wang, Y., & Xie, Y. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.13 · Cited by 573
High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm
Alibart, F., Gao, L., Hoskins, B. D., & Strukov, D. B. (2012) · Nanotechnology · doi:10.1088/0957-4484/23/7/075201 · Cited by 557 · cited 1×
“Write-and-verify [1, 15, 19] is a simple, accurate and widely used programming scheme for RRAMs.”
MethodHardware/Software Co-Exploration of Neural Architectures
Jiang, W., Yang, L., Sha, E. H.-M., Zhuge, Q., Gu, S., Dasgupta, S., Shi, Y., & Hu, J. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.2986127 · Cited by 150
Self-citationMaking Memristive Neural Network Accelerators Reliable
Feinberg, B., Wang, S., & Ipek, E. (2018) · 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00015 · Cited by 130
Understanding the trade-offs in multi-level cell ReRAM memory design
Xu, C., Niu, D., Muralimanohar, N., Jouppi, N. P., & Xie, Y. (2013) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/2463209.2488867 · Cited by 129
Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search
Jiang, W., Zhang, X., Sha, E. H.-M., Yang, L., Zhuge, Q., Shi, Y., & Hu, J. (2019) · Proceedings of the 56th Annual Design Automation Conference 2019 · doi:10.1145/3316781.3317757 · Cited by 121
Self-citationYang, L., Yan, Z., Li, M., Kwon, H., Lai, L., Krishna, T., Chandra, V., Jiang, W., & Shi, Y. (2020) · 2020 57th ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218676 · Cited by 118
Self-citationLi, Y., Hao, C., Zhang, X., Liu, X., Chen, Y., Xiong, J., Hwu, W., & Chen, D. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218749 · Cited by 106
Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators
Jiang, W., Lou, Q., Yan, Z., Yang, L., Hu, J., Hu, X. S., & Shi, Y. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2991575 · Cited by 96
Self-citationInvestigation of statistical retention of filamentary analog RRAM for neuromophic computing
Zhao, M., Wu, H., Gao, B., Zhang, Q., Wu, W., Wang, S., Xi, Y., Wu, D., Deng, N., Yu, S., et al. (2017) · 2017 IEEE International Electron Devices Meeting (IEDM) · doi:10.1109/iedm.2017.8268522 · Cited by 95 · cited 3×
“For device uncertainty specifications, we assume an ISAAClike [16] neural accelerator architecture and a four-bit RRAM device, whose behavioral model is extracted from [25].”
BackgroundMethodStanding on the Shoulders of Giants: Hardware and Neural Architecture Co-Search With Hot Start
Jiang, W., Yang, L., Dasgupta, S., Hu, J., & Shi, Y. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012863 · Cited by 88
Self-citationWhen Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies
Yan, Z., Shi, Y., Liao, W., Hashimoto, M., Zhou, X., & Zhuo, C. (2020) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045134 · Cited by 83
Self-citationOn Neural Architecture Search for Resource-Constrained Hardware PlatformsPreprint
Lu, Q. H., Jiang, W., Xu, X., Shi, Y., & Hu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00105 · Cited by 74
Self-citationLow power memristor-based ReRAM design with Error Correcting Code
Niu, D., Yang, X., & Xie, Y. (2012) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2012.6165062 · Cited by 72
MS-NAS: Multi-scale Neural Architecture Search for Medical Image Segmentation
Yan, X., Jiang, W., Shi, Y., & Zhuo, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_38 · Cited by 70
Self-citationCo-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence
Yang, L., Jiang, W., Liu, W., Sha, E. H. ‐., Shi, Y., & Hu, J. (2020) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac47756.2020.9045595 · Cited by 39
Self-citationAccuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture SearchPreprint
Jiang, W., Zhang, X., Sha, E. H. ‐., Yang, L., Zhuge, Q., Shi, Y., & Hu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11211 · Cited by 31
Self-citationNASS: Optimizing Secure Inference via Neural Architecture SearchPreprint
Bian, S., Jiang, W., Lu, Q. H., Shi, Y., & Sato, T. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11854 · Cited by 25
Self-citationYang, L., Yan, Z., Li, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04116 · Cited by 12
Self-citationStanding on the Shoulders of Giants: Hardware and Neural Architecture Co-Search with Hot StartPreprint
Jiang, W., Yang, L., Dasgupta, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.09087 · Cited by 9
Zeng, D., Jiang, W., Wang, T., Xu, X., Yuan, H., Huang, M., Zhuang, J., Hu, J., & Shi, Y. (2020) · doi:10.1145/3400302.3415789 · Cited by 9
Self-citationLi, Y., Hao, C., Zhang, X., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02563 · Cited by 8
Zeng, D., Jiang, W., Wang, T., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07071 · Cited by 1
Research works printed but not linked: 1
- [8] Weiwen Jiang, Lei Yang, Edwin H-M Sha, Qingfeng Zhuge, Shouzhen Gu, Sakyasingha Dasgupta, Yiyu Shi, and Jingtong Hu. 2020. Hardware/Software coexploration of neural architectures. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2020). -
Cited by
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
“References263,264,takememristor device variations into consideration when searching for the optimum software-related neural network parameters.”
Neural Architecture Search Survey: A Hardware Perspective
Chitty-Venkata, K. T., & Somani, A. K. (2023) · ACM Computing Surveys · doi:10.1145/3524500 · Cited by 106
“2: Network Architecture and Precision Co-search [39, 137] Section 9.”
BackgroundNeural architecture search for in-memory computing-based deep learning accelerators
Krestinskaya, O., Fouda, M. E., Benmeziane, H., Maghraoui, K. E., Sebastian, A., Lu, W. D., Lanza, M., Li, H. H., Kurdahi, F. J., Fahmy, S. A., et al. (2024) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-024-00052-7 · Cited by 47
Das, A., Palesi, M., Kim, J., & Pande, P. P. (2024) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2024.3445829 · Cited by 42
SWIM: selective write-verify for computing-in-memory neural accelerators
Yan, Z., Hu, X. S., & Shi, Y. (2022) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3489517.3530459 · Cited by 37
Self-citation
Metadata
- Authors
- Yan, Z., Juan, D.-C., Hu, X. S., & Shi, Y.
- Affiliations
- University of Notre Dame (US)
- National Tsing Hua University (TW)
- Venue
- Proceedings of the 26th Asia and South Pacific Design Automation Conference
- Publisher
- ACM
- Published
- 2021-01-18
- Pages
- 859-864
- OpenAlex
- W3127168173
- Cited by
- 18
- References
- 31
- Citations by year
- 2025: 2 · 2024: 6 · 2023: 3 · 2022: 7
- FWCI
- 6.27
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Asia and South Pacific Design Automation Conference (ASP-DAC) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- in-memory computing, neural network accelerators, emerging devices, deep neural network robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.