Fused-layer CNN accelerators
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Deep convolutional neural networks (CNNs) are rapidly becoming the dominant approach to computer vision and a major component of many other pervasive machine learning tasks, such as speech recognition, natural language processing, and fraud detection. As a result, accelerators for efficiently evaluating CNNs are rapidly growing in popularity. The conventional approaches to designing such CNN accelerators is to focus on creating accelerators to iteratively process the CNN layers. However, by processing each layer to completion, the accelerator designs must use off-chip memory to store intermediate data between layers, because the intermediate data are too large to fit on chip. In this work, we observe that a previously unexplored dimension exists in the design space of CNN accelerators that focuses on the dataflow across convolutional layers. We find that we are able to fuse the processing of multiple CNN layers by modifying the order in which the input data are brought on chip, enabling caching of intermediate data between the evaluation of adjacent CNN layers. We demonstrate the effectiveness of our approach by constructing a fused-layer CNN accelerator for the first five convolutional layers of the VGGNet-E network and comparing it to the state-of-the-art accelerator implemented on a Xilinx Virtex-7 FPGA. We find that, by using 362KB of on-chip storage, our fused-layer accelerator minimizes off-chip feature map data transfer, reducing the total transfer by 95%, from 77MB down to 3.6MB per image.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the number of worker nodes increases from 2 to 4, MoDNN can speedup the DNN computation by 2.17-4.28×[130]”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [130]
In the citing paper
“With regard to specific DNN structures, e.g., CNN, a finer grid partitioning can be applied to minimize communication, synchronization, and memory overhead [130].”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [130]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“Table II shows the results of a similar comparison, but for the larger and newer VGGNet-E network [17].”
BackgroundMethodResultInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“(We observe a similar pattern in other common CNN structures, such as AlexNet [8].”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 3×
“Large Scale Visual Recognition Challenge (ILSVRC) [15], uses five convolutional layers, while VGG [17] (winner of a portion of ILSVRC 2014) uses up to 17 convolutional layers.”
MethodOptimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158 · cited 10×
“The computational dominance of the convolutional layers, coupled with the need for larger and deeper networks, has sparked significant interest in the design and optimization of accelerator structures for these layers [19], [10].”
BackgroundMethodResultInfluentialDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492 · cited 3×
“As a result, there has been significant interest in developing and adapting hardware accelerators for CNNs [7] such as GPUs [3], FPGAs [13], [19], [10], [1], and ASICs [2].”
BackgroundMethodTorch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,186
cuDNN: Efficient Primitives for Deep Learning
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 943 · cited 2×
“As a result, there has been significant interest in developing and adapting hardware accelerators for CNNs [7] such as GPUs [3], FPGAs [13], [19], [10], [1], and ASICs [2].”
MethodA reconfigurable fabric for accelerating large-scale datacenter services
Putnam, A., Caulfield, A. M., Chung, E. S., Chiou, D., Constantinides, K., Demme, J., Esmaeilzadeh, H., Fowers, J., Gopal, G. P., Gray, J., et al. (2014) · 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2014.6853195 · Cited by 576
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369 · cited 1×
“In [6], [5], [16], systolic implementations were proposed.”
Cited by
Items 301–400 of 779
Kao, S.-C., Huang, X., & Krishna, T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.11218 · Cited by 17
“Some prior researchers [1, 24] have shown that a rarely explored mapping dimension could potentially ameliorate this off-chip access challenge —inter-layer dataflow (or so-called layer fusion).”
BackgroundInfluentialLoopTree: Enabling Exploration of Fused-layer Dataflow Accelerators
Gilbert, M., Wu, Y. N., Parashar, A., Sze, V., & Emer, J. (2023) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass57527.2023.00038 · Cited by 16
“We chose [7], [5], and [8] which represent designs with various tiling, scheduling, and reuse/recompute choices.”
BackgroundMethodInfluentialCocco: Hardware-Mapping Co-Exploration towards Memory Capacity-Communication Optimization
Tan, Z., Zhu, Z., & Ma, K. (2024) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3617232.3624865 · Cited by 16
“Fused-CNN [4] and SR-CNN [38] fuse large contiguous layers for plain networks using manually-designed strategies.”
BackgroundMethodInfluentialA Survey of AI Inference Technologies for On-Device Systems
Wang, W., Li, K., Ji, B., Liu, X., Yu, J., & Wu, Q. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3613691 · Cited by 16
“Within these groups, downstream operators are scheduled to immediately consume intermediate feature maps, thereby avoiding the generation of unnecessary intermediate results and reducing redundant data scans [64].”
Kim, S., Kim, S., Lee, J., & Yoo, H. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2021.3137259 · Cited by 16
Peng, J., Liu, H., Zhao, Z., Li, Z., Liu, S., & Li, Q. (2022) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2022.3197495 · Cited by 16
“port of high-performance hardware devices [1], [2], [3].”
BackgroundAn Experimental Study on Deep Learning Based on Different Hardware Configurations
Li, J., Zhang, C., Cao, Q., Qi, C., Huang, J., & Xie, C. (2017) · IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas.2017.8026843 · Cited by 16
“To accelerate deep learning applications, a series of work [13] [20] [17] [14] [6] developed FPGAs, GPUs, and some”
MethodKim, Y., Han, D., Kim, C., & Yoo, H. (2020) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2020.2980022 · Cited by 16
“To reduce the EMA, layer fusion is adopted [18].”
MethodunzipFPGA: Enhancing FPGA-based CNN Engines with On-the-Fly Weights Generation
Venieris, S. I., Fernández-Marqués, J., & Lane, N. D. (2021) · IEEE International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm51124.2021.00027 · Cited by 16
“Focusing on activations, [69] fuses adjacent layers to cache intermediate activations, while Eyeriss [70], [71] and [52] employ encoding schemes to minimise their bandwidth footprint.”
BackgroundMethodGeneral Reuse-Centric CNN Accelerator
Cicek, N. M., Ning, L., Öztürk, Ö., & Shen, X. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3064608 · Cited by 16
“Alwani and other researchers [37] propose a new CNN evaluation strategy on FPGA, which modifying the order in which the original input data are brought on chip.”
BackgroundTao, Y. (2022) · Deep Blue (University of Michigan) · doi:10.7302/4622 · Cited by 16
Ni, S., Wei, X., Zhang, N., & Chen, H. (2023) · Remote Sensing · doi:10.3390/rs15245784 · Cited by 15
Self-citationCai, J., Wang, X., Gao, M., Peng, S., Zhu, Z., Wei, Y., Wu, Z., & Ma, K. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00048 · Cited by 15
“Several works leverage buffer resources to reduce DRAM accesses under the paradigm of “layer fusion” [3], [7], [8], [29], [33], [37], [49], [60].”
Dazzi, M., Sebastian, A., Parnell, T. P., Francese, P. A., Benini, L., & Eleftheriou, E. S. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3073255 · Cited by 15
“Similarly, dataflows that execute CNNs partially in a layer-by-layer fashion and partially in an inter-layer pipelined fashion [30] have been shown to reduce off-chip memory accesses in FPGA-based accelerators [31].”
BackgroundHERALD: Optimizing Heterogeneous DNN Accelerators for Edge Devices.Preprint
Kwon, H., Lai, L., Krishna, T., & Chandra, V. (2019) · arXiv (Cornell University) · Cited by 15
Fusing Depthwise and Pointwise Convolutions for Efficient Inference on GPUs
Qararyah, F. M., Azhar, M. W., Maleki, M. A., & Trancoso, P. (2024) · doi:10.1145/3677333.3678153 · Cited by 14
“However, others [2], [6] have shown that redundant computations can be avoided at the cost of affordable extra buffering.”
BackgroundInfluentialMiao, T., Sha, L., Huang, K., Li, Y., & Liu, B. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108692 · Cited by 14
Efficient Design of Pruned Convolutional Neural Networks on FPGA
Véstias, M. P. (2020) · Journal of Signal Processing Systems · doi:10.1007/s11265-020-01606-2 · Cited by 14
“The approach removes efficiencies caused by different window sizes but requires other techniques such as fused layers [24] to account for the extra memory required to store intermediate maps and weights.”
MethodAn Energy-Efficient GAN Accelerator With On-Chip Training for Domain-Specific Optimization
Kim, S., Kang, S., Han, D., Kim, S., Kim, S., & Yoo, H. (2021) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2021.3094469 · Cited by 14
DDAM: D ata D istribution- A ware M apping of CNNs on Processing-In-Memory Systems
Wang, J., Du, H., Ding, B., Xu, Q., Chen, S., & Kang, Y. (2022) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3576196 · Cited by 14
“Manoj Alwani et al.[2] developed an efective caching strategy by changing the execution process of CNNs to a pyramid-shaped multi-layer sliding window.”
MethodTensor slicing and optimization for multicore NPUs
de Sousa, R. T., Pereira, M. M., Kwon, Y., Kim, T., Jung, N., Kim, C.-S., Frank, M., & Araujo, G. C. S. D. (2022) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2022.12.008 · Cited by 14
“The work proposed by Alwani et al. [51] and Xiao et al. [52] focuses on data-flow across multiple Conv-layers.”
MethodRe-configurable parallel Feed-Forward Neural Network implementation using FPGA
El‐Sharkawy, M., Wael, M., Mashaly, M. E., & Azab, E. (2024) · Integration · doi:10.1016/j.vlsi.2024.102176 · Cited by 14
A Novel 1D-Convolution Accelerator for Low-Power Real-time CNN processing on the Edge
Sanchez, J., Soltani, N., Chamarthi, R. V., Sawant, A., & Tabkhi, H. (2018) · IEEE High Performance Extreme Computing Conference · doi:10.1109/hpec.2018.8547530 · Cited by 14
“While there are exceptions [16], these design principles are prevalent in many architectures and ultimately ill-suited for real-time stream processing at the edge, where latency is critical and memory scarce.”
BackgroundMethodPolymorphic Accelerators for Deep Neural Networks
Mazreah, A. A., & Chen, L. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2020.3048624 · Cited by 14
Memory-Throughput Trade-off for CNN-Based Applications at the Edge
Minakova, S., & Stefanov, T. P. (2022) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3527457 · Cited by 14
“Intervals [2,3] and [2,4] overlap, which means that edges e1 23 and e 1 24 use the platform memory simultaneously and cannot be assigned to one buffer.”
BackgroundMethodBus Width Aware Off-Chip Memory Access Minimization for CNN Accelerators
Tewari, S., Kumar, A., & Paul, K. (2020) · IEEE Computer Society Annual Symposium on VLSI · doi:10.1109/isvlsi49217.2020.00051 · Cited by 14
“Number of FPGA [1]–[6], GPU [7] and ASIC [8]–[11] accelerators are proposed to meet the performance and energy requirements of CNNs.”
BackgroundREQ-YOLO: A Resource-Aware, Efficient Quantization Framework for Object Detection on FPGAsPreprint
Ding, C., Wang, S., Liu, N., Xu, K., Wang, Y., & Liang, Y. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.13396 · Cited by 14
“The second one is efficient hardware implementations, including FPGAs and ASICs [1, 7, 31, 32, 36, 49, 50, 53, 54].”
MethodFused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network InferencePreprint
Stahl, R., Mueller-Gritschneder, D., & Schlichtmann, U. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.17878 · Cited by 14
“Another application of tiling is the partitioning of DNNs so that they can be run distributed over several devices [32] or can be offloaded to one or more accelerators [2].”
BackgroundCoActo: CoActive Neural Network Inference Offloading with Fine-grained and Concurrent Execution
Bin, K., Park, J.-K., Park, C., Kim, S., & Lee, K. (2024) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3643832.3661885 · Cited by 13
LoopTree: Exploring the Fused-Layer Dataflow Accelerator Design Space
Gilbert, M., Wu, Y. N., Emer, J., & Sze, V. (2024) · IEEE transactions on circuits and systems for artificial intelligence. · doi:10.1109/tcasai.2024.3461716 · Cited by 13
“Prior work referred to this as a compounding effect in recomputation [16].”
InfluentialFPGA-based CNN accelerator using Convolutional Processing Element to reduce idle states
Dehnavi, M., Ghasemi, A., & Alizadeh, B. (2025) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2025.103468 · Cited by 13
VSA: Reconfigurable Vectorwise Spiking Neural Network Accelerator
Lien, H.-H., Hsu, C.-W., & Chang, T. (2021) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas51556.2021.9401181 · Cited by 13
“To reduce this overhead, we adopt the layer fusion [13] that executes two layers successively inside the chip and then stores output to external.”
MethodA Formalism of DNN Accelerator Flexibility
Kao, S.-C., Kwon, H., Pellauer, M., Parashar, A., & Krishna, T. (2022) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3530907 · Cited by 13
Fast Algorithms for Exact IR Drop De-Embedding in Analog Multiply–Accumulate Computing
Gao, S., Yang, F., Lu, C., & Zhao, Y. (2022) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2022.3197105 · Cited by 13
“We adapted the basic idea of fused layer to solve the issue of on-chip memory capacity limitation [16].”
MethodSE-CNN: Convolution Neural Network Acceleration via Symbolic Value Prediction
Yao, Y. (2023) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2023.3244767 · Cited by 13
“In [20], to reduce memory reload of intermediate data, fused-layer CNN is proposed in which intermediate data is cached using additional registers in FPGAs.”
MethodPin or Fuse? Exploiting Scratchpad Memory to Reduce Off-Chip Data Transfer in DNN Accelerators
Jeong, H.-J., Yeo, J., Bahk, C., & Park, J. (2023) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1145/3579990.3580017 · Cited by 13
“Alwani et al. [4] applied the notion of operator fusion to CNN processing.”
BackgroundReconfigurable Processor for Deep Learning in Autonomous Vehicles
Han, S., Wang, Y., Liang, S., Yao, S., Luo, H., Shan, Y., & Peng, J. (2017) · Cited by 13
A 1.17 TOPS/W, 150fps Accelerator for Multi-Face Detection and Alignment
Mo, H., Liu, L., Zhu, W., Li, Q., Liu, H., Hu, W., Wang, Y., & Wei, S. (2019) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3316781.3317736 · Cited by 13
“Because of Max-pooling(MP)’s 2 × 2 down-sampling, 2 × 2 × 10 new output activations, rather than 10× 10× 10 activations in [1], of CONV1 layer should be ready to produce 1× 1× 16 activations for CONV2 layer.”
BackgroundMethodInfluentialCSCNN: Algorithm-hardware Co-design for CNN Accelerators using Centrosymmetric Filters
Li, J., Louri, A., Karanth, A., & Bunescu, R. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00058 · Cited by 13
“Although many neural network accelerators have been proposed to optimize computation [14]- [19], [82], memory [9], [11], [21]- [26], [83]- [88] and data reuse [12], [27], [28], [68], [89], [90], they cannot benefit from sparsity due to lack of dedicated support.”
BackgroundMaleki, M. A., Nabipour-Meybodi, A., Kamal, M., Afzali‐Kusha, A., & Pedram, M. (2021) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3460972 · Cited by 13
Gong, Y. D., Yin, M., Huang, L., Deng, C., & Yuan, B. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3212642 · Cited by 13
“To accelerate the execution of convolutional neural networks (CNNs) in a real-time and energy-efficient way, a series of CNN hardware architecture have been proposed in [2], [4], [24].”
MethodEnabling Large Neural Networks on Tiny Microcontrollers with Swapping
Miao, H., & Lin, F. X. (2021) · arXiv · doi:10.48550/arxiv.2101.08744 · Cited by 12
“A proven wisdom is to swap tiles of NN layers between memory tiers as NN is being executed [23].”
BackgroundWang, F., Shen, M., Ding, Y., & Xiao, N. (2024) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca59077.2024.00076 · Cited by 12
“We fuse two C2Ds for ResNet18 and two GEMMs for BERT similar to previous works [62], [63].”
MethodAccelerating DNNs inference with predictive layer fusion
Olyaiy, M., Ng, C., & Lis, M. (2021) · International Conference on Supercomputing · doi:10.1145/3447818.3460378 · Cited by 12
“Likewise, the FusedCNN [4] accelerator also does not fuse the computation as we do in this paper; instead, layer computations are performed as normal while keeping the intermediate activations in on-chip storage.”
BackgroundAccelerating Deep Neural Networks on Mobile Multicore NPUs
Jung, H., Ji, H., Pushchin, A., Ostapenko, M., Niu, W., Palachev, I., Qu, Y., Fedin, P., Gribov, Y., Nam, H., et al. (2023) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1145/3579990.3580015 · Cited by 12
“Also, [3, 44, 46, 50] propose a layer fusing (stratum) technique for a distributed computing environment, and [45] introduces a retraining technique, fully decomposable spatial partition (FDSP), to overcome a drawback of spatial partitioning: the borderline data should be shared across devices.”
Memory and Interconnect Optimizations for Peta-Scale Deep Learning Systems
Venkataramani, S., Srinivasan, V., Choi, J., Heidelberger, P., Chang, L., & Gopalakrishnan, K. (2019) · IEEE International Conference on High Performance Computing, Data, and Analytics · doi:10.1109/hipc.2019.00036 · Cited by 12
“Inter-layer optimizations for additional data reuse opportunities such as feature computation partitioning across layers to retain intermediate output of part of featuremaps for the use as input for the next layer [28], [29] have also been studied.”
BackgroundPhysical Synthesis for Advanced Neural Network Processors
He, Z., Liao, P., Liu, S., Ma, Y., Lin, Y., & Yu, B. (2021) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/3394885.3431625 · Cited by 12
“[2] fuse the processing of adjacent convolutional layers with the data on-chip and use a pyramid-shaped multi-layer sliding window to minimize off-chip transfer; Ma et al.”
MethodNASA: Accelerating Neural Network Design with a NAS Processor
Ma, X., Si, C., Wang, Y., Liu, C., & Zhang, L. (2021) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca52012.2021.00067 · Cited by 12
“To exploit the data reuse and computation parallelism within a single neural network, various optimized dataflows are proposed [1], [14], [15], [18], [26], [41], [44].”
MethodAccelerating the SM3 hash algorithm with CPU‐FPGA Co‐Designed architecture
Huang, X., Guo, Z., Song, M., & Zeng, X. (2021) · IET Computers & Digital Techniques · doi:10.1049/cdt2.12034 · Cited by 12
A Survey of Open-source Tools for FPGA-based Inference of Artificial Neural Networks
Lebedev, M., & Belecky, P. (2021) · Ivannikov Memorial Workshop (IVMEM) · doi:10.1109/ivmem53963.2021.00015 · Cited by 12
MERCURY: Accelerating DNN Training By Exploiting Input Similarity
Janfaza, V., Weston, K., Razavi, M., Mandal, S., Mahmud, F., Hilty, A., & Muzahid, A. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071051 · Cited by 12
“Due to the versatility of DNN models, special hardware accelerators have been proposed and built [3], [9], [10], [12], [13], [21], [33], [40], [55].”
BackgroundBrainSlug: Transparent Acceleration of Deep Learning Through Depth-First ParallelismPreprint
Weber, N., Schmidt, F., Niepert, M., & Huici, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.08378 · Cited by 12
“he way variables are used – either by all or only single SIMD units – and replacing the outer loops with the ISPC specific foreach(...) instruction. ISPC can target different instruction sets, e.g. SSE[2,4], AVX[1,2] and AVX512 for Intel’s Knight’s Landing. We use the default values from the compiler. ISPC further supp…”
MethodChen, H., Song, M., Zhao, J., Dai, Y., & Li, T. (2019) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3307650.3322260 · Cited by 11
“It is used not only in DianNao [12] and DaDianNao, but also in many state-of-the-art FPGA accelerators [56][57][58].”
BackgroundMethodSelf-citationRetractedBinarized Encoder-Decoder Network and Binarized Deconvolution Engine for Semantic Segmentation
Kim, H., Kim, J., Choi, J., Lee, J., & Song, Y. H. (2020) · IEEE Access · doi:10.1109/access.2020.3048375 · Cited by 11
“6a [45].”
A Construction Kit for Efficient Low Power Neural Network Accelerator Designs
Jokic, P., Azarkhish, E., Bonetti, A., Pons, M., Emery, S., & Benini, L. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3520127 · Cited by 11
“94x) - improved - [123] 2016) VGGNet-E on FPGA (AlexNet: only 28% ext.”
BackgroundA Multi-Cache System for On-Chip Memory Optimization in FPGA-Based CNN Accelerators
Pacini, T., Rapuano, E., Dinelli, G., & Fanucci, L. (2021) · Electronics · doi:10.3390/electronics10202514 · Cited by 11
“Many works in the literature focus on this hybrid storage system, either by optimizing the data transfers with the external memory [40,41] or by presenting solutions to improve the resource management [42,43].”
BackgroundA Real Time Super Resolution Accelerator with Tilted Layer Fusion
Huang, A.-J., Hsu, K., & Chang, T. (2022) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas48785.2022.9937448 · Cited by 11
“The proposed approach will not keep all these boundary data as in [14] or discard them as in [15] to avoid large buffer size or information loss.”
BackgroundMethodInfluentialAivoTTA: an energy efficient programmable accelerator for CNN-based object recognition
IJzerman, J., Viitanen, T. T., Jääskeläinen, P., Kultala, H. O., Lehtonen, L. K., Peemen, M., Corporaal, H., & Takala, J. (2018) · International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation · doi:10.1145/3229631.3229637 · Cited by 11
“However, this approach cannot utilize flexible ways to optimize memory access patterns [27], layer fusion [1] or other techniques that change the order of operations in favor of energy efficiency and low memory transfer costs.”
MethodFeatherNet: An Accelerated Convolutional Neural Network Design for Resource-constrained FPGAs
Morcel, R., Hajj, H. M., Saghir, M. A. R., Akkary, H. H., Artail, H. A., Khanna, R., & Keshavamurthy, A. S. (2019) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3306202 · Cited by 11
“Consequently, ConvNets have two types of computational stages: convolutional and fully connected, and, therefore, in this architecture, there are only two types of Data-flow computational engines: (1) convolutional and (2) fully connected engines.”
BackgroundResultInfluentialData Locality Optimization of Depthwise Separable Convolutions for CNN Inference Accelerators
Wu, H., & Huang, C.-T. (2019) · Design, Automation, and Test in Europe · doi:10.23919/date.2019.8715097 · Cited by 11
“Recently, a specialized accelerator has been presented in [15] to fuse multiple CNN layers.”
BackgroundMethodArgus: An End-to-End Framework for Accelerating CNNs on FPGAs
Shen, Y., Ji, T., Ferdman, M., & Milder, P. A. (2019) · IEEE Micro · doi:10.1109/mm.2019.2930607 · Cited by 11
Self-citationFCNNLib: An Efficient and Flexible Convolution Algorithm Library on FPGAs
Xiao, Q., Lu, L., Xie, J., & Liang, Y. E. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218748 · Cited by 11
“As for conventional algorithm, most of the previous FPGA CNN accelerators are based on it [18].”
BackgroundEfficient Tensor Cores support in TVM for Low-Latency Deep learning
Sun, W. M., Sioutas, S., Stuijk, S., Nelson, A., & Corporaal, H. (2021) · Design, Automation, and Test in Europe · doi:10.23919/date51398.2021.9473984 · Cited by 11
EGCN: An Efficient GCN Accelerator for Minimizing Off-Chip Memory Access
Han, Y., Park, K., Jung, Y., & Kim, L. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3211413 · Cited by 11
“The solu-tion for this problem has already been proposed by [2] for CNNs, but it is not easy to implement on GCNs due to a large number of nodes and irregular connections.”
MethodTowards Enabling Dynamic Convolution Neural Network Inference for Edge Intelligence
Adeyemo, A. A., Sandefur, T., Odetola, T. A., & Hasan, S. R. (2022) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas48785.2022.9937326 · Cited by 11
“In [12], dataflow across convolutional layers was investigated.”
BOOST: Block Minifloat-Based On-Device CNN Training Accelerator with Transfer Learning
Guo, C., Lou, B., Liu, X., Boland, D., Leong, P. H. W., & Zhuo, C. (2023) · doi:10.1109/iccad57390.2023.10323638 · Cited by 11
“Layer fusion [33] is commonly used to combine Batch Nor-malization (BN) with the previous Convolutional (Conv) layer to reduce latency during inference.”
MethodA None-Sparse Inference Accelerator that Distills and Reuses the Computation Redundancy in CNNs
Wang, Y., Liang, S., Li, H., & Li, X. (2019) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3316781.3317749 · Cited by 11
“To further reduce the size of cache banks, fused-layer technique is adopted [15].”
MethodDesign space exploration for layer-parallel execution of convolutional neural networks on CGRAs
Heidorn, C. J. A., Hannig, F., & Teich, J. (2020) · International Workshop on Software and Compilers for Embedded Systems (SCOPES) · doi:10.1145/3378678.3391878 · Cited by 11
Deep Learning Inference Frameworks BenchmarkPreprint
Pochelu, P. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.04323 · Cited by 11
“is possible to merge a sequence of convolutions [13] and a”
BackgroundSimultaneous and Heterogenous Multithreading
Hsu, K.-C., & Tseng, H. (2023) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3613424.3614285 · Cited by 10
A 28-nm 3.32-nJ/Frame Compute-in-Memory CNN Processor With Layer Fusion for Always-on Applications
Zhao, Y., He, P., Zhu, Y., Martins, R. P., Chan, C., & Zhang, M. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3542590 · Cited by 10
“Conventional AI processors [1], [2], [3], [4], [5], [6] separate the memory and multiply-accumulate (MAC) units, leading to substantial power consumption and latency related to memory access.”
An FPGA based Tiled Systolic Array Generator to Accelerate CNNs
Devaraddi, V. S., & Rao, N. (2022) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd57027.2022.00050 · Cited by 10
Occam: Optimal Data Reuse for Convolutional Neural Networks
Gondimalla, A., Liu, J., Thottethodi, M. S., & Vijaykumar, T. N. (2022) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3566052 · Cited by 10
“The output maps’ dependence ancestors transitively extend to include the corresponding input maps of the earlier layers, as observed by Layer Fusion [4, 25] (the smaller rectangular black-and-gray boxes within each layer in Figure 3).”
BackgroundInfluentialHiMA: A Fast and Scalable History-based Memory Access Engine for Differentiable Neural Computer
Tao, Y., & Zhang, Z. (2021) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3466752.3480052 · Cited by 10
“NN accelerators [1, 5, 20, 21, 26, 31] cannot run DNC due to the lack of capability to handle elaborate memory operations.”
BackgroundExploration of Balanced Design in Resource-Constrained Edge Device for Efficient CNNs
Wang, X., Tian, T., Zhao, L., Wu, W., & Jin, X. (2022) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2022.3183258 · Cited by 10
“Some previous works also did some research on fusing layers, for example, [6] fused layers of AlexNet and VGG, and [8] fused convolutional blocks of MobileNetV2.”
BackgroundResultFastTrack: Leveraging Heterogeneous FPGA Wires to Design Low-Cost High-Performance Soft NoCs
Kapre, N., & Krishna, T. (2018) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2018.00067 · Cited by 10
“elements [1–3], system-level endpoints, and others.”
MethodDLFusion: An Auto-Tuning Compiler for Layer Fusion on Deep Neural Network Accelerator
Liu, Z., Leng, J., Chen, Q., Li, C., Zheng, W., Li, L., & Guo, M. (2020) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom51426.2020.00041 · Cited by 10
“Prior work explores the layer fusion as an architecture optimization [1] while we use it for compiler optimization.”
MethodPex: Memory-efficient Microcontroller Deep Learning through Partial ExecutionPreprint
Liberis, E., & Lane, N. D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.17246 · Cited by 10
Hecaton: Training Large Language Models with Scalable Chiplet SystemsPreprint
Huang, Z., Fan, S., Tang, C., Lin, X., Deng, S., & Liu, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.05784 · Cited by 10
“To compensate its lower bandwidth, we exploit layer fusion [1] and on-and-off-package overlap to orchestrate computation and DRAM access.”
Advancing Neuromorphic Architecture Toward Emerging Spiking Neural Network on FPGA
Gao, Y., Wang, T., Yang, Y., Gong, L., Chen, X., Wang, C., Li, X., & Zhou, X. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3547275 · Cited by 9
EasyViT: An Adaptive Collaborative Edge Computing Framework for Vision Transformer
Wen, D., Liang, G., Li, T., Chen, L., Li, J., & Li, T. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3578605 · Cited by 9
“As demonstrated in [41], [42],and [43], raw image data undergoes spatial partitioning across collaborating devices for concurrent processing.”
An FPGA-Based YOLOv6 Accelerator for High-Throughput and Energy-Efficient Object Detection
Xingan, S. H. A., Yanagisawa, M., & Shi, Y. (2024) · IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences · doi:10.1587/transfun.2024vlp0009 · Cited by 9
“In [14], Yu et al. deployed YOLOv1 [4] on FPGA, and the layer fusion technique [15] is proposed to reduce the DRAM access.”
MethodMoteNN: Memory Optimization via Fine-grained Scheduling for Deep Neural Networks on Tiny Devices
Chen, R., Ding, Z., Zheng, S., Li, M., & Liang, Y. E. (2024) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3649329.3655922 · Cited by 9
“Top-𝛽 new states are kept for future iterations (line 13-16).”
A Tightly Coupled AI-ISP Vision Processor
Zhang, H., Li, S., Gui, Y., Li, Z., Xu, S., Lu, Y., Niu, D., Zheng, H., Chen, Y.-K., Xie, Y., et al. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3510939 · Cited by 9
“To tackle the challenge of excessive EMA, Alwani et al. [39] proposed to fuse multiple convolutional layers processing to facilitate intermediate feature reuse.”
Lo, C.-Y., Huang, P.-T., & Hwang, W. (2020) · IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS) · doi:10.1109/aicas48895.2020.9073944 · Cited by 9
“With the fused-layer technique [6], the memory size of weights can be reduced by 49%.”
MethodValpreda, E., Mori, P., Fasfous, N., Vemparala, M. R., Frickenstein, A., Frickenstein, L., Stechele, W., Passerone, C., Masera, G., & Martina, M. (2022) · Electronics · doi:10.3390/electronics11182933 · Cited by 9
Ding, Y.-C., Lin, K.-P., Weng, C.-W., Wang, L., Wang, H.-C., Lin, C.-Y., Chen, Y., & Huang, C.-T. (2022) · European Solid-State Circuits Conference (ESSCIRC) · doi:10.1109/esscirc55480.2022.9911515 · Cited by 9
“In [1], [5] convolution layers are calculated at once until the final output patch is generated.”
BackgroundResultFused DSConv: Optimizing Sparse CNN Inference for Execution on Edge Devices
Guo, J., Teodorescu, R., & Agrawal, G. (2021) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid51090.2021.00064 · Cited by 9
NASA+: Neural Architecture Search and Acceleration for Multiplication-Reduced Hybrid Networks
Shi, H., You, H., Wang, Z., & Lin, Y. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3256700 · Cited by 9
“In parallel, various innovative accelerators [14], [15], [16], [17], [18], [19] have been proposed to boost the hardware efficiency of DNNs, promoting their real-world deployment from the hardware perspective.”
BackgroundEDA: Energy-Efficient Inter-Layer Model Compilation for Edge DNN Inference Acceleration
Pao, B. R., Chen, I.-C., Chang, E.-H., & Yeh, T. T. (2025) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca61900.2025.00050 · Cited by 8
“Unlike TFLM, EDA changes the on-chip [9], [42], [62], EDA reduces the operator traversal depth in each subgraph of a DNN model.”
InfluentialDISSEC: A distributed deep neural network inference scheduling strategy for edge clusters
Li, Q., Huang, L., Tong, Z., Du, T., Zhang, J., & Wang, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.084 · Cited by 8
Kang, H., & Yang, B. (2023) · Sensors · doi:10.3390/s23198104 · Cited by 8
“Some structures process a few layers at the same time [13,33], but they store the intermediate data between the layer blocks, too.”
BackgroundDesigning Deep Learning Models on FPGA with Multiple Heterogeneous Engines
Reis, M., Véstias, M. P., & Neto, H. C. (2023) · ACM Transactions on Reconfigurable Technology and Systems · doi:10.1145/3615870 · Cited by 8
“By fusing layers[4] several can be processed in the same iteration of the engine thus decreasing the number of intermediate memory accesses.”
BackgroundGlint, T., Awasthi, M., & Mekie, J. (2024) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac58780.2024.10473935 · Cited by 8
CIMR-V: An End-to-End SRAM-based CIM Accelerator with RISC-V for AI Edge Device
Guo, Y.-C., Chang, T., Lin, C.-S., Chiou, B.-C., Lai, C.-M., Sheu, S.-S., Lo, W., & Chang, S.-C. (2024) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas58744.2024.10558177 · Cited by 8
Murthy, N. S., Catthoor, F., & Verhelst, M. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103236 · Cited by 8
Lee, K., Cha, M., Kim, S., Nguyen, X. T., & Lee, H. (2025) · International Conference on Consumer Electronics · doi:10.1109/icce63647.2025.10930191 · Cited by 8
“To address the model’s computational demands, various techniques such as layer fusion, quantization, and Winograd convolution have been adopted [14]–[16].”
Towards memory-efficient processing-in-memory architecture for convolutional neural networks
Wang, Y., Zhang, M., & Yang, J. (2017) · ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES) · doi:10.1145/3078633.3081032 · Cited by 8
“The intermediate data will be streamed back to the same processing engine upon the completion of each layer [3].”
BackgroundRodrigues, C. F., Riley, G. D., & Luján, M. (2019) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3310273.3323162 · Cited by 8
“However, unlike these previous approaches there is a lack of studies or tools to explore cross-layer dependencies (a notable exception is [1]).”
BackgroundInSight: An FPGA-Based Neuromorphic Computing System for Deep Neural Networks
Hong, T., Kang, Y.-S., & Chung, J. (2020) · Journal of Low Power Electronics and Applications · doi:10.3390/jlpea10040036 · Cited by 8
“To cope with these issues, there has been an enormous amount of research efforts put into CNN acceleration hardware such as GPUs [3], field-programmable gate arrays (FPGAs) [4,5] and ASICs [6,7] very recently.”
Background
Metadata
- Authors
- Alwani, M., Chen, H., Ferdman, M., & Milder, P.
- Affiliations
- Stony Brook University (US)
- Venue
- 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
- Publisher
- IEEE
- Published
- 2016-10
- Pages
- 1-12
- ISSN
- 1072-4451 (ISSN-L)
- OpenAlex
- W4249932213
- Cited by
- 662
- References
- 20
- Citations by year
- 2026: 20 · 2025: 51 · 2024: 72 · 2023: 67 · 2022: 87 · 2021: 103 · 2020: 72 · 2019: 90 · 2018: 76 · 2017: 23 · 2016: 1
- FWCI
- 21.98
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/ACM International Symposium on Microarchitecture (MICRO) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- CNN accelerator, layer fusion, data flow design, FPGA implementation, convolutional neural network inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.