SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference
Open accessAbstract
Transformers have shown remarkable performance in both natural language processing (NLP) and computer vision (CV) tasks. However, their real-time inference speed and efficiency are limited due to the inefficiency in Softmax and Layer Normalization (LayerNorm). Previous works based on function approximation suffer from inefficient implementation as they place emphasis on computation while disregarding memory overhead concerns. Moreover, such methods rely on retraining to compensate for approximation error which can be costly and inconvenient. In this paper, we present SOLE, a hardware-software co-design for Softmax and LayerNorm which is composed of E2Softmax and AILayerNorm. E2Softmax utilizes log2 quantization of exponent function and log-based division to approximate Softmax while AILayerNorm adopts low-precision statistic calculation. Compared with state-of-the-art designs, we achieve both low-precision calculation and low bit-width storage on Softmax and LayerNorm. Experiments show that SOLE maintains inference accuracy without retraining while offering orders of magnitude speedup and energy savings over GPU, achieving 3.04×, 3.86× energy-efficiency improvements and 2.82×, 3.32× area-efficiency improvements over prior state-of-the-art custom hardware for Softmax and LayerNorm, respectively.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
SOLE's E2Softmax Unit avoids large lookup tables and multiplications, uses log2 quantization on exponent function outputs, and stores intermediate results in 4-bit representations, reducing memory usage.
“As shown in Fig.11, its E2Softmax Unit avoids large lookup tables and multiplications, uses log2 quantization on exponent function outputs, and stores intermediate results in 4-bit representations, reducing memory usage.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [23]
SOLE designs custom hardware for efficient transformer inference, optimizing Softmax and LayerNorm operations.
“SOLE [23] designs custom hardware for efficient transformer inference, optimizes Softmax and LayerNorm operations.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [23]
SOLE designs a two-stage LayerNorm Unit that facilitates pipelined computations and flexible batch processing.
“This design facilitates pipelined computations and flexible batch processing, with robust error tolerance and reduced buffering and multiplication.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [23]
SOLE employs dynamic compression for Layer Normalization statistics via Power-of-Two Factor.
“Additionally, SOLE employs dynamic compression for Layer Normalization statistics via Power-of-Two Factor [43], and innovatively designs a two-stage LayerNorm Unit.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [23]
SOLE introduces E2Softmax, which processes 8-bit quantized inputs using fixed-point arithmetic, employs log2 quantization post-exponentiation and an approximate logarithmic division.
“SOLE [23] introduces E2Softmax, which processes 8-bit quantized inputs using fixed-point arithmetic. This method employs log2 quantization post-exponentiation and an approximate logarithmic division, eschewing traditional high-precision operations.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [23]
SOLE's AILayerNorm Unit uses low-precision statistic calculation, dynamic compression, Power-of-Two Factor quantization, employs a 16-entry lookup table for square functions and shift operations, and facilitates smooth data flow through ping-pong buffers.
“The AILayerNorm Unit, on the other hand, uses low-precision statistic calculation, dynamic compression, and Power-of-Two Factor quantization, employs a 16-entry lookup table for square functions and shift operations, and facilitates smooth data flow through pingpong buffers.”
From Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey (2024) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,745 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., Torr, P. H. S., et al. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00681 · Cited by 4,008 · cited 1× · see citing works
“In recent years, transformer-based networks [1] have achieved success in enhancing the performance of computer vision (CV) tasks [2]–[4] as well as natural language processing (NLP) tasks [5]–[7].”
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · see citing works
Deformable DETR: Deformable Transformers for End-to-End Object Detection
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., & Dai, J. (2020) · arXiv · doi:10.48550/arxiv.2010.04159 · Cited by 1,837 · see citing works
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690 · see citing works
Self-citationSukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015) · arXiv · doi:10.48550/arxiv.1503.08895 · Cited by 1,345 · see citing works
Computer Multiplication and Division Using Binary Logarithms
Mitchell, J. N. (1962) · IEEE Transactions on Electronic Computers · doi:10.1109/tec.1962.5219391 · Cited by 606 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Shen, S., Dong, Z., Ye, J., Ma, L., Yao, Z., Gholami, A., Mahoney, M. W., & Keutzer, K. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6409 · Cited by 437 · see citing works
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
Wang, H., Zhang, Z., & Han, S. (2021) · 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00018 · Cited by 427 · see citing works
Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation
Wu, H., Judd, P., Zhang, X., Isaev, M., & Micikevicius, P. (2020) · arXiv · doi:10.48550/arxiv.2004.09602 · Cited by 246 · see citing works
 3: Accelerating attention mechanisms in neural networks with approximation
Ham, T. J., Jung, S. J., Kim, S., Oh, Y. H., Park, Y., Song, Y., Park, J.-H., Lee, S., Park, K., Lee, J. W., et al. (2020) · 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00035 · Cited by 242 · see citing works
Sanger: A Co-Design Framework for Enabling Sparse Attention using Reconfigurable Architecture
Lu, L., Jin, Y., Bi, H., Luo, Z., Li, P., Wang, T., & Liang, Y. E. (2021) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3466752.3480125 · Cited by 232 · see citing works
A High-Speed and Low-Complexity Architecture for Softmax Function in Deep Learning
Wang, M., Lu, S., Zhu, D., Lin, J. T., & Wang, Z. (2018) · IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) · doi:10.1109/apccas.2018.8605654 · Cited by 217 · see citing works
FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
Lin, Y., Zhang, T., Sun, P., Li, Z., & Zhou, S. (2022) · Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/164 · Cited by 198 · cited 3× · see citing works
“Proposed by [22] for Layer-Norm quantization, PTF equips different channels with different factors to address the inter-channel variation in the inputs of LayerNorm layers.”
Self-citationInner Attention based Recurrent Neural Networks for Answer Selection
Wang, B., Liu, K., & Zhao, J. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1122 · Cited by 196 · see citing works
Post-Training Quantization for Vision Transformer
Liu, Z., Wang, Y., Han, K., Ma, S., & Gao, W. (2021) · arXiv · doi:10.48550/arxiv.2106.14156 · Cited by 192 · see citing works
I-BERT: Integer-only BERT Quantization
Kim, S., Gholami, A., Yao, Z., Mahoney, M. W., & Keutzer, K. (2021) · arXiv · doi:10.48550/arxiv.2101.01321 · Cited by 186 · see citing works
Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformer
Lu, S., Wang, M., Liang, S., Lin, J., & Wang, Z. (2020) · 2020 IEEE 33rd International System-on-Chip Conference (SOCC) · doi:10.1109/socc49529.2020.9524802 · Cited by 154 · see citing works
Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformers
Stevens, J. R., Venkatesan, R., Dai, S., Khailany, B., & Raghunathan, A. (2021) · 2021 58th ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586134 · Cited by 148 · see citing works
DOTA: detect and omit weak attentions for scalable transformer acceleration
Qu, Z., Liu, L., Tu, F., Chen, Z., Ding, Y., & Xie, Y. (2022) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3503222.3507738 · Cited by 134 · see citing works
Scaling vision transformers to 22 billion parameters
Dehghani, M., Djolonga, J., Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A., Caron, M., Geirhos, R., Alabdulmohsin, I., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.05442 · Cited by 112 · see citing works
Yu, J., Park, J., Park, S., Kim, M., Lee, S.-H., Lee, D. H., & Choi, J. (2022) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3489517.3530505 · Cited by 64 · see citing works
Full Stack Optimization of Transformer Inference: a SurveyPreprint
Kim, S., Hooper, C. R. C., Wattanawong, T., Kang, M., Yan, R., Genc, H., Dinh, G., Huang, Q., Keutzer, K., Mahoney, M. W., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.14017 · Cited by 38 · see citing works
A Deep Look into Logarithmic Quantization of Model Parameters in Neural Networks
Cai, J., Takemoto, M., & Nakajo, H. (2018) · Proceedings of the 10th International Conference on Advances in Information Technology · doi:10.1145/3291280.3291800 · Cited by 31 · see citing works
Online normalizer calculation for softmax
Milakov, M., & Gimelshein, N. (2018) · arXiv · doi:10.48550/arxiv.1805.02867 · Cited by 27 · cited 1× · see citing works
“An online normalization scheme [29] is added to further reduce latency.”
UNO: Virtualizing and Unifying Nonlinear Operations for Emerging Neural Networks
Wu, D., Li, J., Behroozi, S., Kim, Y., & Miguel, J. S. (2021) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped52811.2021.9502473 · Cited by 3 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
3 of the 39 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., "An image is worth 16x16 words: Transformers for image recognition at scale," arXiv preprint arXiv:2010.11929, 2020. · arXiv
- J. Yu, J. Park, S. Park, M. Kim, S. Lee, D. H. Lee, and J. Choi, "Nn-lut: neural approximation of non-linear operations for efficient transformer inference," in Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 577-582, 2022.
Cited by
Sorted by citations
Saha, S., & Xu, L. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130417 · Cited by 66 · see citing works
Dong, P., Zhuang, J., Yang, Z., Ji, S., Li, Y., Xu, D., Huang, H., Hu, J., Jones, A. K., Shi, Y., et al. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3443692 · Cited by 35 · see citing works
“We replace the natural constant e inside the Softmax with the power of 2 [63] with the integer inputs. i represents the i th token Log2Q With INT-Softmax 2 n : Similar to [64], we utilize Log2Q on the attention map.”
Hardware-oriented algorithms for softmax and layer normalization of large language models
Li, W., Lyu, D., Wang, G., Hu, A., Xu, N., & He, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4137-4 · Cited by 12 · see citing works
Hardware Accelerator for Approximation-Based Softmax and Layer Normalization in Transformers
Kim, R., Lee, D., Kim, J., Park, J., & Lee, S. E. (2025) · Electronics · doi:10.3390/electronics14122337 · Cited by 10 · see citing works
Zhang, C., You, Y., Wang, N., Park, J., & Zhang, L. (2025) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2025.3575272 · Cited by 11 · see citing works
Algorithm-Hardware Co-Design of a Unified Accelerator for Non-Linear Functions in Transformers
Du, H., Wen, C., Chen, Z., Zhang, L., Sun, Q., Yan, Z., & Zhuo, C. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992717 · Cited by 7 · see citing works
MBS: A High-Precision Approximation Method for Softmax and Efficient Hardware Implementation
Wu, Y., Xie, Z., Pan, H., & Wang, Y. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3559069 · Cited by 6 · see citing works
VEXP: A Low-Cost RISC-V ISA Extension for Accelerated Softmax Computation in Transformers
Wang, R., İslamoğlu, G., Belano, A., Potocnik, V., Conti, F., Garofalo, A., & Luca Benini, C. (2025) · IEEE Symposium on Computer Arithmetic (ARITH) · doi:10.1109/arith64983.2025.00016 · Cited by 5 · see citing works
Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey
Du, D., Gong, G., & Chu, X. (2024) · arXiv · doi:10.48550/arxiv.2405.00314 · Cited by 4 · see citing works
“On the hardware side, optimization of high-level quantization algorithms drives the design of more efficient processors, potentially incorporating more efficient data reuse modules for parallel processing of lower-bit data [22], [23].”
BackgroundMethodVQT-CiM: Accelerating Vector Quantization Enhanced Transformer with Ferroelectric Compute-in-Memory
Huang, X., Du, H., Zhou, M., Yan, Z., Zhuo, C., & Yin, X. (2025) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac63849.2025.11133264 · Cited by 4 · see citing works
HAAN: A Holistic Approach for Accelerating Normalization Operations in Large Language Models
Peng, T., Xia, T., Qin, J., & Zhang, S. Q. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10993005 · Cited by 4 · see citing works
Moon, S., Li, M., Chen, G. K., Knag, P. C., Krishnamurthy, R. K., & Seok, M. (2025) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2025.3603187 · Cited by 4 · see citing works
A Low-Complexity and Reconfigurable Design for Nonlinear Function Approximation in Transformers
Wu, Q.-X., Teng, S.-S., Shieh, M., Huang, C.-T., & Lu, J. (2025) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2025.3585109 · Cited by 3 · see citing works
Tian, C., Liao, C., Zhang, T., Ding, J., Xu, X., & Deng, L. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3606813 · Cited by 3 · see citing works
A Low-complexity and Reconfigurable Design for Nonlinear Function Approximation in Transformers
Wu, Q., Teng, S.-S., Shieh, M., Huang, C.-T., & Lu, J. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11044081 · Cited by 3 · see citing works
Shao, H., & Wang, Z. (2025) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2025.3561000 · Cited by 2 · see citing works
An Efficient Training Architecture for Nonlinear Softmax Function in Transformers
Shao, H., & Wang, Z. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11044130 · Cited by 2 · see citing works
Low-Cost FlashAttention with Fused Exponential and Multiplication Hardware Operators
Alexandridis, K., Titopoulos, V., & Dimitrakopoulos, G. (2025) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi65124.2025.11130263 · Cited by 6 · see citing works
Mixed Non-linear Quantization for Vision Transformers
Kim, G., Lee, J., Park, S., Kwon, Y., & Kim, H. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91979-4_10 · Cited by 1 · see citing works
ARNorm: Hardware-Efficient Normalization for Lightweight Edge Models
Kim, S. H., & Rhee, C. E. (2025) · International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) · doi:10.1109/itc-cscc66376.2025.11137703 · Cited by 1 · see citing works
Revisiting Reconfigurable Acceleration of Vision Transformer with Patch Pruning
Chen, H., Ni, Y., Huang, W., Oh, H., Das, T., Wen, F., & Imani, M. (2025) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped65674.2025.11261783 · Cited by 1 · see citing works
Sparse Approximation of Softmax: Hardware-Efficient Acceleration for Long Sequence Inference
Ma, L., Zhao, Z., Wu, X., Chen, G. P., & Yin, W. (2025) · IEEE International Conference on ASIC (ASICON) · doi:10.1109/asicon66040.2025.11326470 · Cited by 1 · see citing works
CoRN-LN: Compressed Reciprocal Newton Method for Efficient Layer Normalization
Choi, D., Kim, H., & Kim, J.-H. (2025) · IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) · doi:10.1109/apccas67402.2025.11376941 · Cited by 1 · see citing works
Vectorized FlashAttention with low-cost exponential computation in RISC-V vector processors
Titopoulos, V., Alexandridis, K., & Dimitrakopoulos, G. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08322-x · Cited by 1 · see citing works
Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
Choi, D., Kim, H., & Kim, J. (2026) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas66217.2026.11562304 · Cited by 1 · see citing works
Liu, H., Wang, Y., Huang, Z., Fan, S., Tang, C., Zhang, S., Lei, L., Yang, H., Liu, Y., & Jia, H. (2024) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc62300.2024.10737793 · Cited by 1 · see citing works
Self-citationKawamura, T., Masuda, Y., & Ishihara, T. (2025) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed65160.2025.11014475 · Cited by 1 · see citing works
Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference
Chen, C., Mun, H.-G., Meng, J., Abdelfattah, M. S., & Seo, J.-S. (2025) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped65674.2025.11261776 · Cited by 1 · see citing works
A Streaming Transformer Accelerator with Efficient On-Chip Normalization
Zhao, N. B., Chen, J., Lv, J., Xie, C., Du, Y., & Du, L. (2024) · IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) · doi:10.1109/apccas62602.2024.10808505
He, L., Wang, Y., Huang, Z., Fan, S., Tang, C., Zhang, S., Lei, L., Yang, H., Liu, Y., & Jia, H. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3561719
Self-citationIterL2Norm: Fast Iterative L2-Normalization
Ye, C., Sim, Y., Kim, Y., Jin, S., & Jeong, D. S. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992867
Koca, N. A., Do, A. T., & Chang, C.-H. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11044257
Jo, J., Jung, J., & Lee, K. J. (2025) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2025.3586657
High-Precision Softmax Division without Multipliers or Look-Up Tables
Park, J., & Rhee, C. E. (2025) · International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC) · doi:10.1109/itc-cscc66376.2025.11137704
H-FA: A Hybrid Floating-Point and Logarithmic Approach to Hardware Accelerated FlashAttention
Alexandridis, K., & Dimitrakopoulos, G. (2025) · IEEE transactions on circuits and systems for artificial intelligence. · doi:10.1109/tcasai.2025.3630164
Zhang, B., Wang, J., Zhu, X., Guo, Z., Gu, Z., Zhang, K., Zhang, Z., Zhang, K., Zhao, W. S., & Zhang, Y. (2025) · doi:10.1109/iccad66269.2025.11240737
DevTrace: Lightweight Plug-In Design for PCIe Transaction Tracing in Edge Intelligence Workloads
Huang, Z., Xu, K., Wei, Z., Deng, Y., Chen, C., Wang, Y., Liu, F., Xia, M., & Qi, Z. (2025) · doi:10.1109/iccad66269.2025.11240822
FLASH-D: FlashAttention with Hidden Softmax Division
Alexandridis, K., Titopoulos, V., & Dimitrakopoulos, G. (2025) · International Symposium on Low Power Electronics and Design · doi:10.1109/islped65674.2025.11261805 · Cited by 4 · see citing works
Integer Quantization of Nonlinear Operations towards Hardware-Friendly ViTs
Sun, T., Ma, T., Liu, J., Li, Z., Li, Q., Wang, Y., Liu, H., & Liu, S. (2025) · IEEE International Conference on Electronics, Circuits and Systems (ICECS) · doi:10.1109/icecs66544.2025.11270824
Du, H., Wen, C., Chen, Z., Zhang, L., Sun, Q. L., Yan, Z., & Zhuo, C. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3642737
He, X., Li, T., Wen, C., Liu, H., & Sun, Q. L. (2025) · International Conference on Intelligent Technology and Embedded Systems (ICITES) · doi:10.1109/icites66466.2025.11274294
Zhao, Z., Li, H., Liu, F., Lu, Y., Wang, Z., Yang, T., Jiang, L., & Guan, H. (2025) · doi:10.1109/iccad66269.2025.11240736
FD-MLLM: Fault Diagnosis Framework Based on Multimodal Data and Large Language Model
Li, D., Pang, Z., Yang, K., Luo, Y., & Zeng, Y. (2025) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin64977.2025.11278971
CLCO-S1: Efficient Softmax1 Architecture of Cross Level Collaborative Optimization for Transformer
Chen, Y., Geng, H., Liu, C., & Du, L. (2025) · IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) · doi:10.1109/apccas67402.2025.11376624
Softm e x : Lightweight Softmax Compute Engines Based on Exponentiation Units
Kim, T., Ha, J., Lee, W., & Ko, T. (2026) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2026.3671623
Mun, H., Meng, J., Hu, X., Liao, Y., Chen, C., & Seo, J.-S. (2026) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2026.3693653
APEX: Integer-only Non-linear Function Approximation for Efficient Cross-Modal Inference
Ni, P., Mo, Z., Liu, T., Wen, H., Zhu, Z. G., Pei, M., Si, J., Guan, W., Wang, P., Hu, Q. P., et al. (2026) · Design, Automation, and Test in Europe · doi:10.23919/date69613.2026.11539465
Neural Quantization-Aware Piecewise Linear Approximation for Nonlinear Functions on FPGA
Zeng, F., Li, C., Li, R., Guan, J., Lai, R., & Zhu, Z. (2026) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas66217.2026.11562458
harDyT: A Post-Training Hardware-Aware LayerNorm Replacement
Gauthier, L., Bony, F., & Perchoux, J. (2026) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas66217.2026.11562110
A Low-Resource Vision Transformer Accelerator for Edge Applications
Zhang, Y., Wei, X., Wang, B., & Pang, M. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-22500-9_11
Wei, K., Kusaka, T., & Tanaka, T. (2026) · Electronics · doi:10.3390/electronics15174014
A Decoder-Centric Co-Design for Edge Deployment of Transformer-Based Neural TSP Solvers on FPGAs
Ren, Y., Li, T., & Gu, S. (2026) · International Conference on Information Science and Technology · doi:10.1109/icist71072.2026.11727846
- Authors
- Wang, W., Zhou, S., Sun, W., Sun, P., & Liu, Y.
- Affiliations
- Tsinghua University (CN)
- Megvii (China) (CN)
- Venue
- 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD)
- Publisher
- IEEE
- Published
- 2023-10-28
- Pages
- 1-9
- OpenAlex
- W4389166736
- Cited by
- 51
- References
- 39
- Citations by year
- 2026: 10 · 2025: 37 · 2024: 4
- FWCI
- 5.45
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Numerical Methods and Algorithms
- Keywords
- hardware-software co-design, softmax approximation, transformer inference, low-precision arithmetic, energy-efficient inference, area efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
