Fused-layer CNN accelerators
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Deep convolutional neural networks (CNNs) are rapidly becoming the dominant approach to computer vision and a major component of many other pervasive machine learning tasks, such as speech recognition, natural language processing, and fraud detection. As a result, accelerators for efficiently evaluating CNNs are rapidly growing in popularity. The conventional approaches to designing such CNN accelerators is to focus on creating accelerators to iteratively process the CNN layers. However, by processing each layer to completion, the accelerator designs must use off-chip memory to store intermediate data between layers, because the intermediate data are too large to fit on chip. In this work, we observe that a previously unexplored dimension exists in the design space of CNN accelerators that focuses on the dataflow across convolutional layers. We find that we are able to fuse the processing of multiple CNN layers by modifying the order in which the input data are brought on chip, enabling caching of intermediate data between the evaluation of adjacent CNN layers. We demonstrate the effectiveness of our approach by constructing a fused-layer CNN accelerator for the first five convolutional layers of the VGGNet-E network and comparing it to the state-of-the-art accelerator implemented on a Xilinx Virtex-7 FPGA. We find that, by using 362KB of on-chip storage, our fused-layer accelerator minimizes off-chip feature map data transfer, reducing the total transfer by 95%, from 77MB down to 3.6MB per image.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the number of worker nodes increases from 2 to 4, MoDNN can speedup the DNN computation by 2.17-4.28×[130]”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [130]
In the citing paper
“With regard to specific DNN structures, e.g., CNN, a finer grid partitioning can be applied to minimize communication, synchronization, and memory overhead [130].”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [130]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“Table II shows the results of a similar comparison, but for the larger and newer VGGNet-E network [17].”
BackgroundMethodResultInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“(We observe a similar pattern in other common CNN structures, such as AlexNet [8].”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 3×
“Large Scale Visual Recognition Challenge (ILSVRC) [15], uses five convolutional layers, while VGG [17] (winner of a portion of ILSVRC 2014) uses up to 17 convolutional layers.”
MethodOptimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158 · cited 10×
“The computational dominance of the convolutional layers, coupled with the need for larger and deeper networks, has sparked significant interest in the design and optimization of accelerator structures for these layers [19], [10].”
BackgroundMethodResultInfluentialDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492 · cited 3×
“As a result, there has been significant interest in developing and adapting hardware accelerators for CNNs [7] such as GPUs [3], FPGAs [13], [19], [10], [1], and ASICs [2].”
BackgroundMethodTorch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,186
cuDNN: Efficient Primitives for Deep Learning
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 943 · cited 2×
“As a result, there has been significant interest in developing and adapting hardware accelerators for CNNs [7] such as GPUs [3], FPGAs [13], [19], [10], [1], and ASICs [2].”
MethodA reconfigurable fabric for accelerating large-scale datacenter services
Putnam, A., Caulfield, A. M., Chung, E. S., Chiou, D., Constantinides, K., Demme, J., Esmaeilzadeh, H., Fowers, J., Gopal, G. P., Gray, J., et al. (2014) · 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2014.6853195 · Cited by 576
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369 · cited 1×
“In [6], [5], [16], systolic implementations were proposed.”
Cited by
Items 201–300 of 779
Mo, H., Zhu, W., Hu, W., Li, Q., Li, A., Yin, S., Wei, S., & Liu, L. (2021) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2021.3113569 · Cited by 32
FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive ReviewPreprint
Jiang, J., Zhou, Y., Gong, Y., Yuan, H., & Liu, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.13461 · Cited by 32
“Alwani et al. [56] introduced a pyramid-shaped multi-layer sliding window that processes input feature maps, allowing multiple layer results to be computed in advance.”
Logical/Physical Topology-Aware Collective Communication in Deep Learning Training
Cho, S.-H., Son, H., & Kim, J. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071117 · Cited by 31
“Computation time of each layer also decreases since the feature map of each layer, which is a major bottleneck of CNN [8], becomes smaller to extract the feature of input.”
BackgroundSoftware-Defined Design Space Exploration for an Efficient DNN Accelerator Architecture
Yu, Y., Li, Y., Che, S., Jha, N. K., & Zhang, W. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2983694 · Cited by 31
“Similar to the designs in [6], [7], [26], [48], we assume the architectural template uses dedicated separate weight and activation on-chip buffers.”
BackgroundReconfigurable Hardware Accelerators: Opportunities, Trends, and ChallengesPreprint
Wang, C., Lou, W., Gong, L., Jin, L., Tan, L., Hu, Y., Li, X., & Zhou, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.04771 · Cited by 32
“CNN accelerator, which is equipped with a flexible data buffer to ensure the balance of input and weight conversion bandwidth. It could effectively reduce the overall bandwidth requirement. Reference [56] presents a Fused-Layer, which focuses on the problem of data flow in the convolutional layer and reduces the trans…”
BackgroundA Survey of Artificial Neural Network Computing Systems
Foukalas, F. T. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10383-0 · Cited by 30
“Specifically, in [93], authors used an FPGA to develop a platform that modifies the input sequence of a neural network to exploit dataflow parallelism across the convolutional layers.”
Enabling Efficient Fast Convolution Algorithms on GPUs via MegaKernels
Jia, L., Liang, Y. E., Li, X., Lu, L., & Yan, S. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2973144 · Cited by 30
“In [32], the authors also use kernel fusion to eliminate data transfer with off-chip memory, but their fusion is cross-layers, which is different from our technique that apply kernel fusion within a single layer.”
MethodAdaPrune: An Accelerator-Aware Pruning Technique for Sustainable CNN Accelerators
Li, J., & Louri, A. (2021) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2021.3060690 · Cited by 29
SECDA-TFLite: A toolkit for efficient development of FPGA-based DNN accelerators for edge inference
Haris, J., Gibson, P., Cano, J., Agostini, N. B., & Kaeli, D. R. (2022) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2022.11.005 · Cited by 29
“There are a variety of tools available for developing FPGAbased DNN accelerators for edge devices [3,25], some of which feature model-specific tuning [28,45,50].”
Duan, X., Chen, Y., Li, M., Rong, Y., Xie, R., & Han, J. S. (2023) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2023.3261060 · Cited by 29
“Because the feature map of phase1 with a large spatial size requires a large amount of memory access, the fused-layer dataflow [37] should be used to access the feature map only through on-chip memory.”
BackgroundExploiting Parallelism for CNN Applications on 3D Stacked Processing-In-Memory Architecture
Wang, Y., Chen, W., Yang, J., & Li, T. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2868062 · Cited by 29
“FPGA implementations [20], [21], [22], [23] to provide computationally and power efficient systems [24].”
MethodA Primer on Deep Learning Architectures and Applications in Speech Processing
Ogunfunmi, T., Ramachandran, R. P., Togneri, R. B., Zhao, Y., & Xia, X. (2019) · Circuits Systems and Signal Processing · doi:10.1007/s00034-019-01157-3 · Cited by 29
Batch Normalization Processor Design for Convolution Neural Network Training and Inference
Ting, Y.-S., Teng, Y.-F., & Chiueh, T. (2021) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas51556.2021.9401434 · Cited by 28
Ristretto: An Atomized Processing Architecture for Sparsity-Condensed Stream Flow in CNN
Li, G., Xu, W., Song, Z., Jing, N., Cheng, J., & Liang, X. (2022) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00097 · Cited by 28
“…network has diverse numerical precisions and irregular sparse patterns across the layers, which is difficult to witness performance gains *Equal Contribution from conventional general-purpose processors and dedicated accelerators designed for single-precision dense models [5, 8, 9, 26, 43].”
BackgroundBlock convolution: Towards memory-efficient inference of large-scale CNNs on FPGA
Li, G., Li, F., Zhao, T., & Cheng, J. (2018) · Design, Automation, and Test in Europe · doi:10.23919/date.2018.8342188 · Cited by 28
“If multiple successive layers can be fused by skipping several bottleneck layers, the problem then can be alleviated [10], [11].”
BackgroundFraud analytics: A decade of research
Bockel-Rickermann, C., Verdonck, T., & Verbeke, W. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120605 · Cited by 27
SAC: Sharing-Aware Caching in Multi-Chip GPUs
Zhang, S., Naderan-Tahan, M., Jahre, M., & Eeckhout, L. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589078 · Cited by 27
“Continuing to scale GPU performance hence requires moving beyond a single chip [1, 7, 39].”
BackgroundTowards Memory-Efficient Allocation of CNNs on Processing-in-Memory Architecture
Wang, Y., Chen, W., Yang, J., & Li, T. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2791440 · Cited by 27
“There have been a number of FPGA implementations [1], [12], [13], [14] and ASIC accelerators [15], [16].”
BackgroundHigh-Throughput and Energy-Efficient FPGA-Based Accelerator for All Adder Neural Networks
Zhang, N., Ni, S., Chen, L., Wang, T., & Chen, H. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3543213 · Cited by 26
“To achieve high throughput, Alwani et al. [33] introduced a convolution layer fusion method to reduce off-chip data access by merging the adjacent layers of a CNN. Additionally, Sun et al. [34] optimized the data flow of the convolution layer through loop unrolling and tiling, improving the…”
Self-citationDeng, C., Liao, S., & Yuan, B. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2981068 · Cited by 26
“In [27], [28], [29], effi-cient dataflows are studied and developed to improve the hardware performance of DNN accelerators.”
BackgroundMini-batch Serialization: CNN Training with Inter-layer Data ReusePreprint
Lym, S., Behroozi, A., Wen, W., Li, G., Kwon, Y., & Erez, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00307 · Cited by 26
“While effective for the networks evaluated in (Alwani et al., 2016), Fused-Layer CNN can not be applied to training modern deep CNNs, because: (1) convolution layers with small feature maps and large channel counts and weight data (deeper layers in modern CNNs) do not exhibit sufficient inter-tile…”
MethodFuseKNA: Fused Kernel Convolution based Accelerator for Deep Neural Networks
Yang, J., Zhang, Z., Liu, Z., Zhou, J. L., Liu, L., Wei, S., & Yin, S. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00079 · Cited by 25
“To efficiently deploy DNNs, more and more customized accelerators are designed for higher performance and energy efficiency [11-21].”
BackgroundLee, S. (2022) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2022.104554 · Cited by 25
Kim, S., Jang, B., Lee, J., Bae, H., Jang, H., & Park, I. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3234640 · Cited by 25
“Some previous works [39], [40], [41], [42], [43] have used the depth-first processing technique that processes the CNNs in the depth-first order instead of the layer-first order [47], [48], [49], [50], so as to reduce the off-chip memory access and increase the throughput.”
MethodNicePIM: Design Space Exploration for Processing-In-Memory DNN Accelerators With 3-D Stacked-DRAM
Wang, J., Ge, M., Ding, B., Xu, Q., Chen, S., & Kang, Y. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3342605 · Cited by 25
Fast Convolutional Neural Networks in Low Density FPGAs Using Zero-Skipping and Weight Pruning
Véstias, M. P., Duarte, R. P., de Sousa, J. T., & Neto, H. C. (2019) · Electronics · doi:10.3390/electronics8111321 · Cited by 25
“The approach permits the optimization of the architecture for each layer but requires other techniques such as fused layers [17] to account for the extra memory required to store intermediate maps and weights.”
MethodKim, Y., Kim, H., Yadav, N., Li, S., & Choi, K. K. (2020) · Electronics · doi:10.3390/electronics9030478 · Cited by 25
Graph processing and machine learning architectures with emerging memory technologies: a survey
Qian, X. (2021) · Science China Information Sciences · doi:10.1007/s11432-020-3219-6 · Cited by 25
“The second trend is the design of hardware accelerators [34–42] 1) .”
BackgroundDadu-Eye: A 5.3 TOPS/W, 30 fps/1080p High Accuracy Stereo Vision Accelerator
Feng, M., Xu, H., Wang, Y., Wang, Y., Li, J., Zou, X., Li, B., & Han, Y. (2021) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3101296 · Cited by 25
F-DNA: Fast Convolution Architecture for Deconvolutional Network Acceleration
Mao, W., Lin, J., & Wang, Z. (2020) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2020.3000519 · Cited by 25
“It is noted that our reconfigurable accelerator also has a satisfactory throughput for convolutions when compared with some traditional CNN accelerators, such as the architectures in [36]–[38].”
MethodA Survey on Real-Time Object Detection on FPGAs
Hozhabr, S. H., & Giorgi, R. (2025) · IEEE Access · doi:10.1109/access.2025.3544515 · Cited by 24
“The operator fusion technique, also known as kernel or layer fusion technique, aims to reduce data exchanges between computation units and external memory by finding data dependencies between different parts of an object detection model [262], [263].”
Hardware accelerator systems for artificial intelligence and machine learning
(2020) · Advances in computers · doi:10.1016/bs.adcom.2020.11.005 · Cited by 24
Stream: A Modeling Framework for Fine-grained Layer Fusion on Multi-core DNN Accelerators
Symons, A., Mei, L., Colleman, S., Houshmand, P., Karl, S., & Verhelst, M. (2023) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass57527.2023.00051 · Cited by 24
“To overcome this, several works have studied fine-grained scheduling strategies of deeply fused DNN layers [1], [7], [14], [21].”
BackgroundFlashGNN: An In-SSD Accelerator for GNN Training
Niu, F., Yue, J., Shen, J., Liao, X., & Jin, H. (2024) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca57654.2024.00035 · Cited by 24
“We draw inspiration from [3] in developing this approach.”
MethodParana: A Parallel Neural Architecture Considering Thermal Problem of 3D Stacked Memory
Yin, S., Tang, S., Lin, X., Ouyang, P., Tu, F., Liu, L., Zhao, J., Xu, C., Li, S., Xie, Y., et al. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2858230 · Cited by 24
“Others [8], [26], [36], [58] exploit”
Instruction Driven Cross-layer CNN Accelerator for Fast Detection on FPGA
Yu, J., Ge, G., Hu, Y., Ning, X., Qiu, J., Guo, K., Wang, Y., & Yang, H. (2018) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3283452 · Cited by 24
“Alwani [1] introduces a fuse-layer policy and reduces the transfer of intermediate data, but Alwani implements the fuse-layer policy with pipeline structure.”
BackgroundMethodInfluentialA Review of FPGA‐Based Custom Computing Architecture for Convolutional Neural Network Inference
Xiyuan, P., Yu, J., Yao, B., Liansheng, L., & Peng, Y. (2021) · Chinese Journal of Electronics · doi:10.1049/cje.2020.11.002 · Cited by 24
Zhang, X., Wei, X. S., Sang, Q., Chen, H., & Xie, Y. (2020) · Electronics · doi:10.3390/electronics9091344 · Cited by 24
“[21] effectively reduced off-chip access by optimizing the order of input feature maps and fusing multiple adjacent convolutional layers.”
Self-citationRole of Reconfigurable Computing in speeding up Machine Learning Algorithms
Monir, N. I., Akter, F. Y., & Sayed, S. R. K. (2025) · SCCTS Transactions on Reconfigurable Computing · doi:10.31838/rcc/02.02.02 · Cited by 23
High performance accelerators for deep neural networks: A review
Akhoon, M. S., Suandi, S. A., Alshahrani, A. S., Saad, A. H. Y., Albogamy, F. R., Abdullah, M. Z., & Loan, S. A. (2021) · Expert Systems · doi:10.1111/exsy.12831 · Cited by 23
“Alwani et al. (2016) have designed and implemented fused layer CNN accelerators for the first time.”
BackgroundMethodInfluentialEDCompress: Energy-Aware Model Compression for Dataflows
Wang, Z., Luo, T., Goh, R. S. M., & Zhou, J. T. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3172941 · Cited by 23
“Dataflow Applied by Dataflow Applied by X : Y [4] [31] FX : FY [5] X : FX [6] [32] [33] CI : CO [7] [34] [35] [36] [37] [38]”
MethodLi, Z., Kim, S., Im, D., Han, D., & Yoo, H. (2022) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2022.3224964 · Cited by 23
“However, they cannot implement much output feature reuse, such as layer fusion [15], [26], [27], since the footprint required for output feature reuse increases as the output reuse times.”
BackgroundMemory optimization at Edge for Distributed Convolution Neural Network
Naveen, S., & Kounte, M. R. (2022) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.4648 · Cited by 23
“FTP includes fusing multiple layers 48 and partitioning fused layers into multiple simultaneous execution tasks to save off-chip memory transfer.”
MethodHardware Compilation of Deep Neural Networks: An Overview
Zhao, R., Liu, S., Ng, H.-C., Wang, E., Davis, J. J., Niu, X., Wang, X., Shi, H.-F., Constantinides, G. A., Cheung, P. Y. K., et al. (2018) · IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP) · doi:10.1109/asap.2018.8445088 · Cited by 23
Delmás, A., Sharify, S., Judd, P., & Moshovos, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09068 · Cited by 23
“Layer fusion can further reduce off-chip communication and create additional parallelism [26].”
Dong, P., Tan, Y., Liu, X., Luo, P., Liu, Y., Liang, L., Zhou, Y., Pang, D., Yung, M., Zhang, D., et al. (2025) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc49661.2025.10904499 · Cited by 22
ANNETTE: Accurate Neural Network Execution Time Estimation With Stacked Models
Wess, M., Ivanov, M., Unger, C., Nookala, A., Wendt, A., & Jantsch, A. (2020) · IEEE Access · doi:10.1109/access.2020.3047259 · Cited by 22
“It is crucial for the execution time of these layers, whether they are executed in isolation or connection with a convolution layer [31].”
BackgroundPiyasena, D., Wickramasinghe, R., Paul, D., Lam, S.-K., & Wu, M. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00063 · Cited by 22
“In existing work, stream based CNN accelerators have been realized by various techniques: using bitstream reconfiguration [4], fusing multiple layers [12], [13], changing tile granularity [11], quantized CNNs (QNN) [7] and more recently using FPGA clusters [6], [8].”
MethodDNN pruning with principal component analysis and connection importance estimation
Riera, M., Arnau, J.-M., & González, A. M. (2021) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2021.102336 · Cited by 22
“Some of the most popular applications of MLPs are speech recognition and machine translation [42,43], whereas CNNs are commonly used for image classification [44].”
BackgroundKim, S., Lee, J., Kang, S., Han, D., Jo, W., & Yoo, H. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3138092 · Cited by 22
Nekooei, A., & Safari, S. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.02.024 · Cited by 22
A high-performance convolution block oriented accelerator for MBConv-Based CNNs
Shang, J., Zhang, K., Zhang, Z., Li, C., & Liu, H. (2022) · Integration · doi:10.1016/j.vlsi.2022.10.012 · Cited by 21
DepFiN: A 12-nm Depth-First, High-Resolution CNN Processor for IO-Efficient Inference
Goetschalckx, K., Wu, F., & Verhelst, M. (2022) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2022.3210591 · Cited by 21
“…2) Cache: Second, instead of recomputing them, one could cache the pixels of the overlap regions off-chip (for large patches with limited overlap), on-chip (for small patches [8], [15], [17]), or both [8], [18] (different memories for both window sweeping directions and/or extra levels of tiling).”
BackgroundInfluentialA Low-Power Hardware Architecture for Real-Time CNN Computing
Liu, X., Cao, C., & Duan, S. (2023) · Sensors · doi:10.3390/s23042045 · Cited by 21
“[14] utilize layer fusion techniques to connect and fuse multiple consecutive convolutional layers of VGG, through multiple buffers, minimizing the number of feature data transferred from the off-chip memory.”
BackgroundMethodUnleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search
Lou, W., Gong, L., Wang, C., Qian, J., Wang, X., Li, C., & Zhou, X. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3391688 · Cited by 21
“Specifically, layer fusion can effectively reduce off-chip memory access and BW pressure, making it widely employed in FPGA-based NN accelerators [42], [43], [44].”
BackgroundZhang, J. J., Raj, P., Zarar, S., Ambardekar, A., & Garg, S. (2019) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3358178 · Cited by 21
“We note that for CNNs, convolution layers account more than 90% of the computational load and therefore dominate energy consumption [2, 26, 28].”
BackgroundGeng, T., Wang, T., Wu, C., Yang, C., Wu, W., Li, A., & Herbordt, M. (2019) · International Conference on Supercomputing · doi:10.1145/3330345.3330386 · Cited by 21
“For large storage demand, we adopt a layer-fusion technique, as referred to in [1].”
MethodTensorFlow to Cloud FPGAs: Tradeoffs for Accelerating Deep Neural Networks
Hadjis, S., & Olukotun, K. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00064 · Cited by 21
“Fusion is a state-of-the-art technique for DNN accelerators which minimizes DRAM bandwidth [13], [14].”
BackgroundAWARE-CNN: Automated Workflow for Application-Aware Real-Time Edge Acceleration of CNNs
Sanchez, J., Sawant, A., Neff, C., & Tabkhi, H. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2990215 · Cited by 21
“These approaches utilize the idea of fused layers [32].”
MethodFSA: A Fine-Grained Systolic Accelerator for Sparse CNNs
Li, F., Li, G., Mo, Z., He, X., & Cheng, J. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012212 · Cited by 21
Yang, S., Bhattacharjee, D., Kumar, V., Chatterjee, S., De, S., Debacker, P., Verkest, D., Mallik, A., & Catthoor, F. (2022) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2022.3171826 · Cited by 21
“3(c)), aiming to reduce intermediate data transfer at memory hierarchies [20].”
BackgroundPractical Mechanisms for Reducing Processor–Memory Data Movement in Modern Workloads
Boroumand, A. (2021) · Figshare · doi:10.1184/r1/14642340 · Cited by 21
Review on FPGA-Based Accelerators in Deep learning
Wu, Y. (2023) · IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) · doi:10.1109/itnec56291.2023.10082175 · Cited by 20
“Alwani [19] proposed a fused-layer CNN accelerator, fusing the processing of multiple CNN layers.”
MethodETTE: Efficient Tensor-Train-based Computing Engine for Deep Neural Networks
Gong, Y. D., Yin, M., Huang, L., Xiao, J., Sui, Y., Deng, C., & Yuan, B. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589103 · Cited by 20
Configurable 2D–3D CNNs Accelerator for FPGA-Based Hyperspectral Imagery Classification
He, W., Yang, Y., Mei, S., Hu, J., Xu, W., & Hao, S. (2023) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2023.3321965 · Cited by 20
“To support the efficient execution of CNNs on a hardware platform with limited resources, the loop computation needs to be optimized with optimization techniques like loop unrolling, loop tiling, and loop interchange proposed in [46], [47], and [48].”
MethodPflow: An end-to-end heterogeneous acceleration framework for CNN inference on FPGAs
Wan, Y., Xie, X., Yi, L., Jiang, B., Chen, J., & Jiang, Y. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103113 · Cited by 20
Ahmad, H., Arif, T., Hanif, M. A., Hafiz, R., & Shafique, M. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012865 · Cited by 20
“Thus, while the techniques mentioned above explore multilayer fusion, they do not perform a generic DSE exploring multiple reuse patterns for minimizing off-chip memory access volume [26], [27], and neither consider the opportunity for model compression.”
BackgroundFCNNLib: A Flexible Convolution Algorithm Library for Deep Learning on FPGAs
Liang, Y. E., Xiao, Q., Lu, L., & Xie, J. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3108065 · Cited by 20
Optimizing Off-Chip Memory Access for Deep Neural Network Accelerator
Zheng, Y., Yang, H., Shu, Y., Jia, Y., & Huang, Z. (2022) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2022.3150030 · Cited by 20
“Setting the size of ofmap buffer between 16KB to 512KB, we measured the DRAM access volume when using ofmap reuse oriented (ORO) scheduling scheme [8], weight reuse oriented (WRO) scheduling scheme [9] and our proposed adaptive scheduling scheme.”
BackgroundMethodCUF: Continuous Upsampling Filters
Vasconcelos, C. N., Öztireli, C., Matthews, M. A., Hashemi, M., Swersky, K., & Tagliasacchi, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00964 · Cited by 20
Power-Driven DNN Dataflow Optimization on FPGA
Sun, Q. L., Chen, T., Miao, J., & Yu, B. (2019) · doi:10.1109/iccad45719.2019.8942085 · Cited by 19
“A more effective strategy is to fuse several neighboring convolution layers together with pooling and ReLU layers, which further reduces the size of data transfer [23].”
BackgroundData Motion Acceleration: Chaining Cross-Domain Multi Accelerators
Wang, S., Xu, H., Mamandipoor, A., Mahapatra, R., Ahn, B. H., Ghodrati, S., Kailas, K. K., Alian, M., & Esmaeilzadeh, H. (2024) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca57654.2024.00083 · Cited by 19
Optimus: towards optimal layer-fusion on deep learning processors
Cai, X., Wang, Y., & Zhang, L. (2021) · ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES) · doi:10.1145/3461648.3463848 · Cited by 19
“Comparing with w/o MinCost (use the cost function of FusedLayer [1]), HaFS benefits from the memory-cost model𝑀𝑖𝑛𝐶𝑜𝑠𝑡 (·) in Section 3.2, because it takes into consideration that the parameters of fused layergroups may not fit into the on-chip buffer and captures the minimum overall off-chip memory access accurately w…”
BackgroundMethodInfluentialDNN Feature Map Compression Using Learned Representation over GF(2)
Gudovskiy, D., Hodgkinson, A., & Rigazio, L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_41 · Cited by 19
Accurate and Bandwidth Efficient Architecture for CNN-based Full-HD Super-Resolution
Chih, C.-Y., Wu, S.-S., Klopp, J. P., & Chen, L. (2018) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2018.8351288 · Cited by 19
“The layer fusion accelerator [1] is proposed to merge layers for a general CNN processor with the aim of reducing input and output bandwidth.”
BackgroundLin, X., Yin, S., Tu, F., Liu, L., Li, X., & Wei, S. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3195970.3196067 · Cited by 18
“Work [1] (FL) fuses adjacent layers to reduce the data movement, and work [6] (MCLP) partitions the target FPGA into multiple processors, each of which runs a different subset of the layers.”
BackgroundMethodInfluentialTinyTS: Memory-Efficient TinyML Model Compiler Framework on Microcontrollers
Liu, Y.-Y., Zheng, H.-S., Hu, Y. F., Hsu, C.-N., & Yeh, T. T. (2024) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca57654.2024.00070 · Cited by 18
“Furthermore, the Fused-layer CNN accelerator [2] combines and reorders multiple CNN layers to increase the caching of intermediate tensors.”
MethodA Communication-Centric Approach for Designing Flexible DNN Accelerators
Kwon, H., Samajdar, A., & Krishna, T. (2018) · IEEE Micro · doi:10.1109/mm.2018.2877289 · Cited by 18
“4 – 6 This makes the hardening of DNN accelerators into an IP or a discrete chip that is future-proof for the rapidly evolving DNN landscape impractical.”
BackgroundA Multi-Task Hardwired Accelerator for Face Detection and Alignment
Mo, H., Liu, L., Zhu, W., Li, Q., Liu, H., Yin, S., & Wei, S. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2955463 · Cited by 18
“The total number of required multipliers is at least 216 while ours is 126, which is 41.7% less than [25] as shown in Fig.”
MethodInfluentialTang, W., Yang, Q., & Yan, W. (2019) · Asia-Pacific Power and Energy Engineering Conference (APPEEC) · doi:10.1109/appeec45492.2019.8994713 · Cited by 18
CNN Acceleration With Hardware-Efficient Dataflow for Super-Resolution
Lee, S., Joo, S., Ahn, H. K., & Jung, S. (2020) · IEEE Access · doi:10.1109/access.2020.3031055 · Cited by 18
“Alwani et al . observed a fusing relationship between adjacent layers in the CNN inference [21].”
BackgroundMulti-Bank On-Chip Memory Management Techniques for CNN Accelerators
Kang, D., Kang, D., & Ha, S. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3076987 · Cited by 18
How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures
Venieris, S. I., Panopoulos, I., Leontiadis, I., & Venieris, I. S. (2021) · IEEE International Conference on Application-specific Systems, Architectures and Processors (ASAP) · doi:10.1109/asap52443.2021.00022 · Cited by 18
Huang, C.-T. (2021) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca52012.2021.00089 · Cited by 18
“One is feature reusing, such as fused-layer [4] and Shortcut Mining [5], and the other one is recomputing, like eCNN [21].”
MethodChang, L., Zhang, S., Du, H., Chen, Y., & Wang, S. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2021.3109580 · Cited by 18
Hardware-Efficient Residual Neural Network Execution in Line-Buffer Depth-First Processing
Shi, M., Houshmand, P., Mei, L., & Verhelst, M. (2021) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2021.3120103 · Cited by 18
Deep Learning for Computer Architects
Reagen, B., Adolf, R., Whatmough, P. N., Wei, G.-Y., & Brooks, D. M. (2017) · Synthesis lectures on computer architecture · doi:10.2200/s00783ed1v01y201706cac041 · Cited by 18
Edge–Artificial Intelligence-Powered Parking Surveillance With Quantized Neural Networks
Zhuang, Y., Pu, Z., Yang, H., & Wang, Y. (2022) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2022.3182358 · Cited by 18
“This layer fusion can improve the inference speed by reducing the computing complexity by simplifying two-step work to one-step work [47].”
MethodHuffDuff: Stealing Pruned DNNs from Sparse Accelerators
Yang, D., Nair, P. J., & Lis, M. (2023) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3575693.3575738 · Cited by 18
“We also exclude accelerators that execute multiple layers on-chip [3, 18, 38]; indeed, we are unaware of any sparse DNN accelerators that do that.”
BackgroundMethodInfluentialSystolic Array Based Accelerator and Algorithm Mapping for Deep Learning Algorithms
Yang, Z., Wang, L., Ding, D., Zhang, X., Deng, Y. L., Li, S., & Dou, Q. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-05677-3_16 · Cited by 18
“M Alwani et al constructed a fused-layer CNN accelerator for the (cid:12)rst (cid:12)ve convolutional layers of the VGGNet-E network [8].”
BackgroundRaj, R. D. A., Yanamala, R. M. R., Pallakonda, A., Aghaei, J., & Pouresmaeil, E. (2025) · Sustainable Energy Technologies and Assessments · doi:10.1016/j.seta.2025.104725 · Cited by 17
Accelerating Convolutional Neural Network by Exploiting Sparsity on GPUs
Xu, W., Sun, Y., Fan, S., Yu, H., & Fu, X. (2023) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3600092 · Cited by 17
“To address this problem, some works [4, 21, 36, 62, 79] consider integrating different CNN layers into one GPU kernel to decrease communication overhead.”
BackgroundToward High-Accuracy and Real-Time Two-Stage Small Object Detection on FPGA
Li, S., Zhu, Z., Sun, H., Ning, X., Dai, G., Hu, Y., Yang, H., & Wang, Y. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3385121 · Cited by 17
“So single-task FPGA-based accelerators are widely used to accelerate these algorithms [37], [38], [39].”
MethodSmall Memory Footprint Neural Network Accelerators
Seto, K., Nejatollahi, H., An, J., Kang, S., & Dutt, N. D. (2019) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed.2019.8697641 · Cited by 17
“Differently from the previous work [3] [4] which apply tasklevel pipelining to the loops that iterates over the primary output pixels as in Fig.”
BackgroundMethodInfluentialFast-ABC: A Fast Architecture for Bottleneck-Like Based Convolutional Neural Networks
Xie, X., Sun, F., Lin, J. T., & Wang, Z. (2019) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi.2019.00010 · Cited by 17
FPGA Architecture Exploration for DNN Acceleration
Roorda, E., Rasoulinezhad, S. A., Leong, P. H. W., & Wilton, S. J. E. (2022) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3503465 · Cited by 17
“The bank of memories containing weights can optionally be double buffered, a technique used in many existing accelerators [1, 14, 16, 28, 36] to reduce bottlenecks associated with off-chip memory accesses.”
MethodChang, L., Zhao, X., & Zhou, J. (2022) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3570927 · Cited by 17
“Layer fusion can decrease data access between on and of-chip memory to relieve the pressure of bandwidth [1].”
BackgroundAccelerating Deep Learning Inference with Cross-Layer Data Reuse on GPUs
Wang, X., Li, G., Dong, X. M., Li, J., Liu, L., & Feng, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57675-2_14 · Cited by 17
Lin, X., Yin, S., Tu, F., Liu, L., Li, X., & Wei, S. (2018) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac.2018.8465777 · Cited by 17
Why is FPGA-GPU Heterogeneity the Best Option for Embedded Deep Neural Networks?Preprint
Carballo-Hernández, W., Pelcat, M., & Berry, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01343 · Cited by 17
“• Fused-Layer: It was first introduced in [16] as a method to store intermediate weights and neuron activity in cache from adjacent layers in depth.”
Method
Metadata
- Authors
- Alwani, M., Chen, H., Ferdman, M., & Milder, P.
- Affiliations
- Stony Brook University (US)
- Venue
- 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
- Publisher
- IEEE
- Published
- 2016-10
- Pages
- 1-12
- ISSN
- 1072-4451 (ISSN-L)
- OpenAlex
- W4249932213
- Cited by
- 662
- References
- 20
- Citations by year
- 2026: 20 · 2025: 51 · 2024: 72 · 2023: 67 · 2022: 87 · 2021: 103 · 2020: 72 · 2019: 90 · 2018: 76 · 2017: 23 · 2016: 1
- FWCI
- 21.98
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/ACM International Symposium on Microarchitecture (MICRO) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- CNN accelerator, layer fusion, data flow design, FPGA implementation, convolutional neural network inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.