Efficient Softmax Hardware Architecture for Deep Neural Networks
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Deep neural network (DNN) has become a pivotal machine learning and object recognition technology in the big data era. The softmax layer is one of the key component layers for completing multi-classification tasks. However, the softmax layer contains complex exponential and division operations, resulting in low accuracy and long critical paths in hardware accelerator design. In order to solve the above issues, we present a softmax hardware architecture with proper accuracy, good trade-off and strong expansibility. We summarize the classification rules of neural network and balance the calculation accuracy between resource consumption. On this basis, we proposed an exponential calculation unit based on the group lookup table, and improve a natural logarithmic calculation unit based on the Maclaurin series and the data preprocessing scheme matching them. The experimental results show that the softmax hardware architecture proposed in this paper can achieve the calculation accuracy of 3 decimal fraction and the classification accuracy of $99.01%$. Theoretically, it can accomplish the classification task of infinite categories.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
In-Datacenter Performance Analysis of a Tensor Processing Unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., et al. (2017) · Proceedings of the 44th Annual International Symposium on Computer Architecture · doi:10.1145/3079856.3080246 · Cited by 4,537 · cited 1×
“In recent years, the emergence of neural network accelerators (e.g., DaDianNao [8], Eyeriss [9] and TPU [11]) have greatly promoted the hardware implementation of ANN.”
BackgroundEyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001177 · Cited by 1,182 · cited 2×
“In recent years, the emergence of neural network accelerators (e.g., DaDianNao [8], Eyeriss [9] and TPU [11]) have greatly promoted the hardware implementation of ANN.”
Ando, K., Ueyoshi, K., Orimo, K., Yonekawa, H., Sato, S., Nakahara, H., Takamaeda-Yamazaki, S., Ikebe, M., Asai, T., Kuroda, T., et al. (2018) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2017.2778702 · Cited by 160
A 1.06-to-5.09 TOPS/W reconfigurable hybrid-neural-network processor for deep learning applications
Yin, S., Ouyang, P., Tang, S., Tu, F., Li, X., Liu, L., & Wei, S. (2017) · 2017 Symposium on VLSI Circuits · doi:10.23919/vlsic.2017.8008534 · Cited by 93
Price, M., Glass, J., & Chandrakasan, A. P. (2017) · 2017 IEEE International Solid-State Circuits Conference (ISSCC) · doi:10.1109/isscc.2017.7870352 · Cited by 86 · cited 2×
“A speech recognition chip is proposed in [20].”
Background
Cited by
Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformers
Stevens, J. R., Venkatesan, R., Dai, S., Khailany, B., & Raghunathan, A. (2021) · 2021 58th ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586134 · Cited by 148
Efficient Precision-Adjustable Architecture for Softmax Function in Deep Learning
Zhu, D., Lu, S., Wang, M., Lin, J., & Wang, Z. (2020) · IEEE Transactions on Circuits and Systems II: Express Briefs · doi:10.1109/tcsii.2020.3002564 · Cited by 115
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
Geng, X., Wang, Z., Chen, C., Xu, Q., Xu, K., Jin, C., Gupta, M., Yang, X., Chen, Z., Sabry Aly, M. M., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3394494 · Cited by 26
Metadata
- Authors
- Du, G., Tian, C., Li, Z., Zhang, D., Yin, Y., & Ouyang, Y.
- Venue
- Proceedings of the 2019 Great Lakes Symposium on VLSI
- Publisher
- ACM
- Published
- 2019-05-13
- Pages
- 75-80
- OpenAlex
- W2946480111
- Cited by
- 53
- References
- 7
- FWCI
- 5.78
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Efficient Softmax Hardware Architecture for Deep Neural Networks. https://app.cochat.ai/papers/10.1145/3299874.3317988
- The paper
- Du, G., Tian, C., Li, Z., Zhang, D., Yin, Y., & Ouyang, Y. (2019). Efficient Softmax Hardware Architecture for Deep Neural Networks. Proceedings of the 2019 Great Lakes Symposium on VLSI. https://doi.org/10.1145/3299874.3317988
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.