Fused-layer CNN accelerators
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Deep convolutional neural networks (CNNs) are rapidly becoming the dominant approach to computer vision and a major component of many other pervasive machine learning tasks, such as speech recognition, natural language processing, and fraud detection. As a result, accelerators for efficiently evaluating CNNs are rapidly growing in popularity. The conventional approaches to designing such CNN accelerators is to focus on creating accelerators to iteratively process the CNN layers. However, by processing each layer to completion, the accelerator designs must use off-chip memory to store intermediate data between layers, because the intermediate data are too large to fit on chip. In this work, we observe that a previously unexplored dimension exists in the design space of CNN accelerators that focuses on the dataflow across convolutional layers. We find that we are able to fuse the processing of multiple CNN layers by modifying the order in which the input data are brought on chip, enabling caching of intermediate data between the evaluation of adjacent CNN layers. We demonstrate the effectiveness of our approach by constructing a fused-layer CNN accelerator for the first five convolutional layers of the VGGNet-E network and comparing it to the state-of-the-art accelerator implemented on a Xilinx Virtex-7 FPGA. We find that, by using 362KB of on-chip storage, our fused-layer accelerator minimizes off-chip feature map data transfer, reducing the total transfer by 95%, from 77MB down to 3.6MB per image.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the number of worker nodes increases from 2 to 4, MoDNN can speedup the DNN computation by 2.17-4.28×[130]”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [130]
In the citing paper
“With regard to specific DNN structures, e.g., CNN, a finer grid partitioning can be applied to minimize communication, synchronization, and memory overhead [130].”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [130]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 6×
“Table II shows the results of a similar comparison, but for the larger and newer VGGNet-E network [17].”
BackgroundMethodResultInfluentialImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 4×
“(We observe a similar pattern in other common CNN structures, such as AlexNet [8].”
BackgroundMethodInfluentialGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 3×
“Large Scale Visual Recognition Challenge (ILSVRC) [15], uses five convolutional layers, while VGG [17] (winner of a portion of ILSVRC 2014) uses up to 17 convolutional layers.”
MethodOptimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks
Zhang, C., Li, P., Sun, G., Guan, Y., Xiao, B., & Cong, J. (2015) · Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays · doi:10.1145/2684746.2689060 · Cited by 2,158 · cited 10×
“The computational dominance of the convolutional layers, coupled with the need for larger and deeper networks, has sparked significant interest in the design and optimization of accelerator structures for these layers [19], [10].”
BackgroundMethodResultInfluentialDaDianNao: A Machine-Learning Supercomputer
Chen, Y., Luo, T., Liu, S., Zhang, S., He, L., Wang, J., Li, L., Chen, T., Xu, Z., Sun, N., et al. (2014) · 2014 47th Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1109/micro.2014.58 · Cited by 1,492 · cited 3×
“As a result, there has been significant interest in developing and adapting hardware accelerators for CNNs [7] such as GPUs [3], FPGAs [13], [19], [10], [1], and ASICs [2].”
BackgroundMethodTorch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,186
cuDNN: Efficient Primitives for Deep Learning
Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J. J., Catanzaro, B., & Shelhamer, E. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0759 · Cited by 943 · cited 2×
“As a result, there has been significant interest in developing and adapting hardware accelerators for CNNs [7] such as GPUs [3], FPGAs [13], [19], [10], [1], and ASICs [2].”
MethodA reconfigurable fabric for accelerating large-scale datacenter services
Putnam, A., Caulfield, A. M., Chung, E. S., Chiou, D., Constantinides, K., Demme, J., Esmaeilzadeh, H., Fowers, J., Gopal, G. P., Gray, J., et al. (2014) · 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2014.6853195 · Cited by 576
NeuFlow: A runtime reconfigurable dataflow processor for vision
Farabet, C., Martini, B., Corda, B., Akselrod, P., Culurciello, E., & LeCun, Y. (2011) · CVPR 2011 WORKSHOPS · doi:10.1109/cvprw.2011.5981829 · Cited by 369 · cited 1×
“In [6], [5], [16], systolic implementations were proposed.”
Cited by
Items 101–200 of 779
Chimera: An Analytical Optimizing Framework for Effective Compute-intensive Operators Fusion
Zheng, S., Chen, S., Song, P., Chen, R., Li, X., Yan, S., Lin, D., Leng, J., & Liang, Y. E. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071018 · Cited by 74
“FusedLayer [10], FixyNN [51], FixyFPGA [33], and Tangram [18], Ascend [30] implement efficient accelerators that can pipeline different DNN layers to gain inter-layer and intra-layer parallelism.”
BackgroundNEBULA: A Neuromorphic Spin-Based Ultra-Low Power Architecture for SNNs and ANNs
Singh, S., Sarma, A., Jao, N., Pattnaik, A., Lu, S. K., Yang, K., Sengupta, A., Narayanan, V., & Das, C. R. (2020) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca45697.2020.00039 · Cited by 73
“Despite several hardware advancements with accelerators like GPUs [49], [60], [84], TPUs [39] and other custom ASICs [3], [53], [65], [67], [91], present day computing system is far from achieving the unparalleled power efficiency of a ∼ 20W human brain [19], [86].”
BackgroundPUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning InferencePreprint
Ankit, A., Hajj, I. E., Chalamalasetti, S. R., Ndu, G., Foltín, M., Williams, R. S., Faraboschi, P., Hwu, W., Strachan, J. P., Roy, K., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10351 · Cited by 73
Lee, J., Lee, J., & Yoo, H. (2020) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2020.3014454 · Cited by 72
“Even though the SC 2 LF requires slightly higher the feature map bandwidth ( 0.15 GB/s) than previous methods [18], it can reduce 93.2 % of the required on-chip memory footprint than [18].”
BackgroundMethodInfluentialRestructuring Batch Normalization to Accelerate CNN TrainingPreprint
Jung, W., Jung, D.-J., Kim, A. B., Lee, S., Rhee, W., & Ahn, J. H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01702 · Cited by 72
“Fused-layer [3] is similar to our work as it fuses multiple CNN layers to exploit inter-layer data reusability; experiments on the VGGNet-E with FPGAs have shown a 95% memory transfer reduction per image.”
BackgroundMethodInfluentialTileFlow: A Framework for Modeling Fusion Dataflow via Tree-based Analysis
Zheng, S., Chen, S., Gao, S., Jia, L., Sun, G., Wang, R., & Liang, Y. E. (2023) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3613424.3623792 · Cited by 70
“Fused-Fuse two convolutions with Layer [2] height and width dimensions tiled.”
BackgroundMethodInfluentialWei, X., Liang, Y. E., & Cong, J. (2019) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3316781.3317875 · Cited by 70
“Previous designs [1, 2, 19, 22] attempt to keep part or all the parameters or activations on chip to avoid offchip memory data transfer by layer fusion.”
BackgroundBreaking High-Resolution CNN Bandwidth Barriers With Enhanced Depth-First Execution
Goetschalckx, K., & Verhelst, M. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2905361 · Cited by 69
“However, [11] relies on two-dimensional region-of-influence pyramids.”
BackgroundMethodResultInfluentialMemory Trojan Attack on Neural Network Accelerators
Zhao, Y. K., Hu, X., Li, S., Ye, J., Deng, L., Ji, Y., Xu, J., Wu, D., & Xie, Y. (2019) · Design, Automation, and Test in Europe · doi:10.23919/date.2019.8715027 · Cited by 69
“[26] propose a dataflow across multiple convolutional layers.”
BackgroundMethodInfluentialAdaptable butterfly accelerator for attention-based NNs via hardware and algorithm co-design
Fan, H., Chau, T., Venieris, S. I., Lee, R., Kouris, A., Luk, W., Lane, N. D., & Abdelfattah, M. S. (2022) · 2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00050 · Cited by 66
“The fine-grained intra- and inter-layer pipeline techniques [43], [44] are used to optimize the hardware performance.”
MethodFPGA-based accelerator for object detection: a comprehensive survey
Zeng, K., Ma, Q., Wu, J. W., Chen, Z., Shen, T., & Yan, C. C. (2022) · The Journal of Supercomputing · doi:10.1007/s11227-022-04415-5 · Cited by 66
“Alwani et al. proposed a fusion layer computing architecture that can perform multilayer convolutions at the same time, and the intermediate results are directly used in the next layer [136].”
BackgroundNeural networks on microcontrollers: saving memory at inference via operator reorderingPreprint
Liberis, E., & Lane, N. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05110 · Cited by 66
“Also, Alwani et al. [23] propose not to materialise an output tensor of a large convolution operation in memory at all, and compute its individual output elements as needed by succeeding operators.”
BackgroundGradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent
Kim, H., Park, H., Kim, T., Cho, K., Lee, E., Ryu, S., Lee, H., Choi, K., & Lee, J. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00030 · Cited by 65
“, 8bit gradient/32bit master weights) and a few state-of-the-art data reuse techniques [17], [50], [71], the relative portion of memory accesses from parameter updates dramatically increases.”
BackgroundQiu, Y. Y., Niu, J., Zhu, X., Zhu, K., Yao, Y., Ren, B., & Ren, T. (2022) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan11040057 · Cited by 65
“Some researchers have optimized AI models for parameter pruning and sharing, caching intermediate data between adjacent layers to reduce data movement [120].”
BackgroundA Low-Latency FPGA Accelerator for YOLOv3-Tiny With Flexible Layerwise Mapping and Dataflow
Kim, M., Oh, K., Cho, Y., Seo, H. S., Nguyen, X. T., & Lee, H. (2023) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2023.3335949 · Cited by 65
“Meanwhile, the fused layer design [31] pipelines a few early layers to reduce external memory accesses, especially for input/output feature maps (IFM/OFM) that can be forwarded from one layer to the next without accessing external memory.”
BackgroundMethodResultInfluentialLayerweaver: Maximizing Resource Utilization of Neural Processing Units via Layer-Wise Scheduling
Oh, Y. H., Kim, S., Jin, Y., Son, S., Bae, J., Lee, J.-S., Park, Y., Kim, D. U., Ham, T. J., & Lee, J. W. (2021) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca51647.2021.00056 · Cited by 65
“On the other hand, small-sized NPUs traditionally count on double buffering [32], [33] and dataflow optimization to minimize off-chip DRAM accesses [34]–[36].”
BackgroundMei, L., Goetschalckx, K., Symons, A., & Verhelst, M. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071098 · Cited by 63
“On the tile size, [23], [24] always treated one row of an image as a tile, whereas [1], [12] adopted a square tile size and [7] has a preferred tile size of 128 pixels along an image row.”
BackgroundInfluentialNeuRex: A Case for Neural Rendering Acceleration
Lee, J., Choi, K., Lee, J., Lee, S., Whangbo, J., & Sim, J. (2023) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3579371.3589056 · Cited by 62
“Small MLP weights and a large input dimension lead to a huge opportunity for layer fusion , as observed in other work [3].”
BackgroundResultTowards Efficient Microarchitectural Design for Accelerating Unsupervised GAN-Based Deep Learning
Song, M., Zhang, J., Chen, H., & Li, T. (2018) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00016 · Cited by 62
Self-citationScale-out acceleration for machine learning
Park, J., Sharma, H., Mahajan, D., Kim, J. K., Olds, P., & Esmaeilzadeh, H. (2017) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3123939.3123979 · Cited by 61
Nguyen, D. T., Je, H., Nguyen, T. N., Ryu, S., Lee, K., & Lee, H. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3153288 · Cited by 60
“Hardware accelerators use various dataflows or optimizations to increase resource utilization, for example, weight stationary [7], [8], [16], [28], [48], [53], output stationary [49], and row stationary [13], [50], [51].”
BackgroundThe Heterogeneous Deep Neural Network Processor With a Non-von Neumann Architecture
Shin, D., & Yoo, H. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2897076 · Cited by 60
“Therefore, it is very important to have enough internal memory and use it efficiently with the optimized processing and dataflow [58], [86].”
BackgroundFully Distributed Deep Learning Inference on Resource-Constrained Edge Devices
Stahl, R., Zhao, Z., Mueller-Gritschneder, D., Gerstlauer, A., & Schlichtmann, U. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-27562-4_6 · Cited by 60
“The distribution of the model was also tackled in a different context for hardware acceleration of DNNs in [1].”
MethodeCNN: A Block-Based and Highly-Parallel CNN Accelerator for Edge Inference
Huang, C.-T., Ding, Y.-C., Wang, H.-C., Weng, C.-W., Lin, K.-P., Wang, L.-W., & Chen, L.-D. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358263 · Cited by 60
“In contrast to layer fusion [4], we recompute blockoverlapped features to avoid huge on-chip SRAM.”
MethodDepFiN: A 12nm, 3.8TOPs depth-first CNN processor for high res. image processing
Goetschalckx, K., & Verhelst, M. (2021) · Symposium on VLSI Circuits · doi:10.23919/vlsicircuits52068.2021.9492338 · Cited by 59
Atomic Dataflow based Graph-Level Workload Orchestration for Scalable DNN Accelerators
Zheng, S., Zhang, X., Liu, L., Wei, S., & Yin, S. (2022) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca53966.2022.00042 · Cited by 58
Efficient Object Detection Framework and Hardware Architecture for Remote Sensing Images
Li, L., Zhang, S., & Wu, J. (2019) · Remote Sensing · doi:10.3390/rs11202376 · Cited by 58
“reduced the transfer of off-chip feature map data by modifying the order of input data and fusing multiple continuous convolutional layer processing [38].”
MethodToward Full-Stack Acceleration of Deep Convolutional Neural Networks on FPGAs
Liu, S., Fan, H., Ferianc, M., Niu, X., Shi, H.-F., & Luk, W. W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3055240 · Cited by 57
“From the hardware level perspective, researchers [25]–[27] have proposed: 1) a quantization method, which captures the specialty of FPGAs with capability of custom precision support to save computation resources and 2) loop unrolling strategies for multiple parallel processing.”
MethodEfficient depthwise separable convolution accelerator for classification and UAV object detection
Li, G., Zhang, J., Zhang, M., Wu, R., Cao, X., & Liu, W. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.071 · Cited by 57
Optimus: An Operator Fusion Framework for Deep Neural Networks
Cai, X., Wang, Y., & Zhang, L. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3520142 · Cited by 56
“Since the energy cost of accessing the main memory is orders of magnitude higher than arithmetic operations, of-chip memory accesses will account for most of the energy consumption in the DNNs accelerators [2, 7, 9, 29, 40] and become the performance bottleneck.”
BackgroundMethodInfluentialFPGA-based implementation of classification techniques: A survey
Saidi, A., Othman, S. B., Dhouibi, M., & Saoud, S. B. (2021) · Integration · doi:10.1016/j.vlsi.2021.08.004 · Cited by 55
Accelerating applications using edge tensor processing units
Hsu, K.-C., & Tseng, H. (2021) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3458817.3476177 · Cited by 55
Laius: An 8-Bit Fixed-Point CNN Hardware Inference Engine
Li, Z., Wang, L., Guo, S., Deng, Y. L., Dou, Q., Zhou, H., & Lu, W. (2017) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa/iucc.2017.00030 · Cited by 54
GNA: Reconfigurable and Efficient Architecture for Generative Network Acceleration
Yan, J., Yin, S., Tu, F., Liu, L., & Wei, S. (2018) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2018.2857258 · Cited by 54
“Fused-Layer CNN accelerator [13] explores fused layer technique toward cascade neural networks without branch structures such as Alexnet, VGGNET, etc.”
MethodHardware Acceleration of a Generalized Fast 2-D Convolution Method for Deep Neural Networks
Ansari, A., & Ogunfunmi, T. (2022) · IEEE Access · doi:10.1109/access.2022.3149505 · Cited by 53
Review of neural network model acceleration techniques based on FPGA platforms
Liu, F., Li, H., Hu, W., & He, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128511 · Cited by 54
Kim, S., Li, Z., Um, S., Jo, W., Ha, S., Lee, J., Kim, S., Han, D., & Yoo, H. (2023) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2023.3319962 · Cited by 52
“In addition, layer fusion [20] is also considered for minimizing external memory access by an intermediate feature map.”
BackgroundHIDA: A Hierarchical Dataflow Compiler for High-Level Synthesis
Ye, H., Jun, H., & Chen, D. (2024) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3617232.3624850 · Cited by 52
“To understand the scalability of HIDA, we perform an ablation study on ResNet-18 by sweeping the maximum parallel factor from 1 to 256 and tile size from 2 to 32.”
MethodASV: Accelerated Stereo Vision System
Feng, Y., Whatmough, P. N., & Zhu, Y. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358253 · Cited by 52
“port. We propose novel software optimizations to eliminate the compute inefficiencies without hardware changes. Data-ReuseOptimizationExploitingdata-reuse(throughtiling) is critical to DNN efficiency [12, 18, 19, 24, 25, 32, 39, 44, 46, 52, 70, 71, 74, 75]. Orthogonal to generic data-reuse, we identify a new reusedime…”
Najmaei, S., Glasmann, A., Schroeder, M. A., Sarney, W. L., Chin, M. L., & Potrepka, D. M. (2022) · Materials Today · doi:10.1016/j.mattod.2022.08.017 · Cited by 51
Trapezoid: A Versatile Accelerator for Dense and Sparse Matrix Multiplications
Yang, Y., Emer, J., & Sánchez, D. (2024) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca59077.2024.00072 · Cited by 51
“In dense/sparse neural networks, accelerators for CNNs [1, 2, 8, 16, 17, 22, 59, 74, 80], transformers [25, 26, 35, 47, 68, 76, 77, 78], point clouds [13, 14, 45] and beyond have been proposed.”
BackgroundTian, J. W., & Hong, S. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304038 · Cited by 51
“Moreover, [2] used convolution tiling on FPGA to reduce off-chip memory access of DNN, but their work also claims difficulties in applying such technique to general purpose processors.”
MethodShortcut Mining: Exploiting Cross-Layer Shortcut Reuse in DCNN Accelerators
Mazreah, A. A., & Chen, L. (2019) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2019.00030 · Cited by 51
“Another interesting approach, fused-CNN [2], cascades multiple layers together to avoid off-chip accesses.”
BackgroundMethodInfluentialSqueezeFlow: A Sparse CNN Accelerator Exploiting Concise Convolution Rules
Li, J., Jiang, S., Gong, S., Wu, J., Yan, J., Yan, G., & Li, X. (2019) · IEEE Transactions on Computers · doi:10.1109/tc.2019.2924215 · Cited by 51
“Hence, a number of customized accelerators [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34] have been proposed to deliver high computational throughput.”
Towards an Efficient Accelerator for DNN-Based Remote Sensing Image Segmentation on FPGAs
Liu, S., & Luk, W. W. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00037 · Cited by 51
“Layer fusion methods are also used to accelerate CNNs in previous work [16], [17] by caching all intermediate feature maps in on-chip memories to reduce the external bandwidth requirements.”
MethodIm, D., Han, D., Choi, S., Kang, S., & Yoo, H. (2020) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2020.2991189 · Cited by 51
“The ROI based image segmentation with the layer fusion [13] reduces the overall external memory access by 94.2%.”
BackgroundMethodInfluentialZhong, Z., Bao, W., Wang, J., Zhu, X., & Zhang, X. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3514501 · Cited by 50
“Usually, model optimization methods can be divided into four types: Compact/ transferred convolution filters [11, 26], low-rank factorization, parameter pruning sharing [2, 32], and knowledge distillation [34, 42].”
MethodQiu, K., Jao, N., Zhao, M., Mishra, C. S., Akbulut, G. G., Jose, S., Sampson, J., Kandemir, M., & Narayanan, V. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00034 · Cited by 50
“Challenge 2: Software controlled dynamic RCA activation and scheduling The idea of loop tiling has been widely leveraged in RCA design to either increase system throughput by smoothing the pipelining or reduce memory accesses by improving data locality [ 34], [3].”
MethodO3BNN-R: An Out-of-Order Architecture for High-Performance and Regularized BNN Inference
Geng, T., Li, A., Wang, T., Wu, C., Li, Y., Shi, R., Wu, W., & Herbordt, M. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3013637 · Cited by 50
“For large storage demand, we adopt a layer-fusion technique, as referred to in [1].”
MethodSECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inference
Haris, J., Gibson, P., Cano, J., Agostini, N. B., & Kaeli, D. R. (2021) · International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) · doi:10.1109/sbac-pad53543.2021.00015 · Cited by 49
ShapeShifter: Enabling Fine-Grain Data Width Adaptation in Deep Learning
Lascorz, A. D., Sharify, S., Edo, I., Stuart, D. M., Awad, O. M., Judd, P., Mahmoud, M., Nikolić, M., Siu, K., Poulos, Z., et al. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358295 · Cited by 49
“We evaluated the benefits of ShapeShifter compression in combination with layer fusion [67].”
MethodCommunication Lower Bound in Convolution Accelerators
Chen, X., Han, Y., & Wang, Y. (2020) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca47549.2020.00050 · Cited by 49
“, the fused-layer approach [35] that optimizes data movement between convolutional layers).”
BackgroundSGCN: Exploiting Compressed-Sparse Features in Deep Graph Convolutional Network Accelerators
Yoo, M., Song, J., Lee, J., Kim, N., Kim, Y., & Lee, J. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071102 · Cited by 48
“These methods have been successfully applied to inference [3], [28], training [52], and batch normalization [35].”
BackgroundMethodDeep reuse: streamline CNN inference on the fly via coarse-grained computation reuse
Ning, L., & Shen, X. (2019) · International Conference on Supercomputing · doi:10.1145/3330345.3330384 · Cited by 48
“FPGA based accelerators [1, 19, 22, 24, 25, 36] also show great potential on accelerating DNN inference.”
BackgroundBenchIP: Benchmarking Intelligence Processors
Tao, J.-H., Du, Z., Guo, Q., Lan, H., Zhang, L., Zhou, S., Xu, L., Liu, C., Liu, H.-F., Tang, S., et al. (2018) · Journal of Computer Science and Technology · doi:10.1007/s11390-018-1805-8 · Cited by 48
“However, with the chaos of benchmarks, it is difficult to compare and quantize the various IPs, especially the rapidly increasing works emerging at top architecture conferences [12, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30].”
BackgroundLee, J., Shin, D., Lee, J., Lee, J., Kang, S., & Yoo, H. (2019) · Symposium on VLSI Circuits · doi:10.23919/vlsic.2019.8778104 · Cited by 48
“5% than [4], 2) memory compaction scheme to reduce the on-chip memory footprint further by 31.”
BackgroundMethodInfluentialHardware-Software Co-Design for an Analog-Digital Accelerator for Machine Learning
Ambrosi, J., Ankit, A., Antunes, R., Chalamalasetti, S. R., Chatterjee, S., Hajj, I. E., Fachini, G. J. D. A., Faraboschi, P., Foltín, M., Huang, S., et al. (2018) · IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc.2018.8638612 · Cited by 47
“Many works propose additional architecture optimizations for digital-only accelerators such as reducing weight/input precision [78], [79], pruning weights to exploit sparsity [16], [80], [81], [82], [70], [83], [84], [85], and others [86], [87], [88], [89], [90], [91].”
BackgroundShen, J., Huang, Y. R., Wen, M., & Zhang, C. (2019) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2019.2912894 · Cited by 47
“In addition, to reduce the off-chip memory access, the previous study [20] has proposed the idea of layer fusion which successfully retains all the interlayer data of the fused layers on-chip and minimizes the data movement.”
BackgroundMethodInfluentialTowards Efficient Convolutional Neural Network for Domain-Specific Applications on FPGA
Zhao, R., Ng, H.-C., Luk, W. W., & Niu, X. (2018) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2018.00033 · Cited by 46
Estimating crowd density with edge intelligence based on lightweight convolutional neural networks
Wang, S., Pu, Z., Li, Q., & Wang, Y. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117823 · Cited by 46
Noise-Resilient DNN: Tolerating Noise in PCM-Based AI Accelerators via Noise-Aware Training
Kariyappa, S., Tsai, H., Spoon, K., Ambrogio, S., Narayanan, P., Mackin, C., Chen, A., Qureshi, M., & Burr, G. W. (2021) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2021.3089987 · Cited by 45
SOFA: A Compute-Memory Optimized Sparsity Accelerator via Cross-Stage Coordinated Tiling
Wang, H., Fang, J., Tang, X., Yue, Z., Li, J., Qin, Y., Guan, S., Yang, Q., Wang, Y., Li, C., et al. (2024) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro61859.2024.00093 · Cited by 45
Efficient Scheduling of Irregular Network Structures on CNN Accelerators
Zheng, S., Zhang, X., Ou, D., Tang, S., Liu, L., Wei, S., & Yin, S. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012215 · Cited by 45
“Evidently, enumeration [12] becomes impractical.”
BackgroundInfluentialA Framework for Behavioral Biometric Authentication Using Deep Metric Learning on Mobile Devices
Wang, C., Xiao, Y., Gao, X. H., Li, L., & Wang, J. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3072608 · Cited by 45
“Optimizations at the architecture level are more effective, such as improving the intra-layer [86], [87], cross-layer data locality [88], [89] and reducing data transfer via zero-value compression [90].”
BackgroundFull-stack optimization for accelerating CNNs using powers-of-two weights with FPGA validation
McDanel, B., Zhang, S. Q., Kung, H. T., & Dong, X. (2019) · International Conference on Supercomputing · doi:10.1145/3330345.3330385 · Cited by 44
ISOSceles: Accelerating Sparse CNNs through Inter-Layer Pipelining
Yang, Y., Emer, J., & Sánchez, D. (2023) · IEEE International Symposium on High-Performance Computer Architecture (HPCA) · doi:10.1109/hpca56546.2023.10071080 · Cited by 44
“Over the dense Fused-Layer [3] accelerator, ISOSceles is gmean 7.5 × (up to 18.0 × ) faster, and reduces off-chip traffic by 3.6 × by exploiting sparsity.”
BackgroundMethodInfluentialPArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency
Ghosh, S. K., Raha, A., Raghunathan, V., & Raghunathan, A. (2023) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3630266 · Cited by 44
Optimizing the Deployment of Tiny Transformers on Low-Power MCUs
Jung, V. J. B., Burrello, A., Scherer, M., Conti, F., & Benini, L. (2024) · IEEE Transactions on Computers · doi:10.1109/tc.2024.3500360 · Cited by 43
“The idea behind this is to tile several layers together, effectively using Layer-Wise Tiling but on a group of layers [35], [36].”
Practical Attacks on Deep Neural Networks by Memory Trojaning
Hu, X., Zhao, Y. K., Deng, L., Liang, L., Zuo, P., Ye, J., Lin, Y., & Xie, Y. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.2995347 · Cited by 43
“Alwani et al. [73] proposed the optimization to fuse multiple Conv layers at a time to reduce feature map data movement.”
BackgroundConvFusion: A Model for Layer Fusion in Convolutional Neural Networks
Waeijen, L., Sioutas, S., Peemen, M., Lindwer, M., & Corporaal, H. (2021) · IEEE Access · doi:10.1109/access.2021.3134930 · Cited by 43
“The layer fusion proposed by Alwani et al. [7] suffers from another shortcoming, aptly dubbed the computation pyramid by the authors.”
BackgroundInfluentialInstruction driven cross-layer CNN accelerator with winograd transformation on FPGA
Yu, J., Hu, Y., Ning, X., Qiu, J., Guo, K., Wang, Y., & Yang, H. (2017) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/fpt.2017.8280147 · Cited by 42
“The FPGA accelerator in [5] can only run a specific network and needs to be reprogrammed to run another network.”
BackgroundMethodInfluentialLerGAN: A Zero-Free, Low Data Movement and PIM-Based GAN Architecture
Mao, H., Song, M., Li, T., Dai, Y., & Shu, J. (2018) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00060 · Cited by 42
“NN accelerators Many recent works accelerate NN based on FPGAs [67] [45] [69] [57] [3] and ASICs [18] [23] [68] [40] [12] [43] [55] [44] [2].”
BackgroundAutomatic Generation of Multi-Precision Multi-Arithmetic CNN Accelerators for FPGAs
Zhao, Y., Gao, X., Guo, X., Liu, J., Wang, E., Mullins, R., Cheung, P. Y. K., Constantinides, G. A., & Xu, C. (2019) · International Conference on Field-Programmable Technology (FPT) · doi:10.1109/icfpt47387.2019.00014 · Cited by 42
“Unfortunately, they are limited to either small models [27], bi-narized networks [20], or only a few layers of a large CNN [1], which are unsuitable for the speed and task performance on large datasets.”
BackgroundJoint multi-user DNN partitioning and task offloading in mobile edge computing
Liao, Z., Hu, W., Huang, J., & Wang, J. (2023) · Ad Hoc Networks · doi:10.1016/j.adhoc.2023.103156 · Cited by 42
Boroumand, A., Ghose, S., Akin, B., Narayanaswami, R., Junior, G. F. D. O., Ma, X., Shiu, E., & Mutlu, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.00768 · Cited by 41
“Many prior works [1,2,5,6,7,13,15,22,23,33,39,42,45,46, 48,50,51,57] look at a specific model type (predominantly CNNs).”
BackgroundAccelerating Mobile Applications at the Network Edge with Software-Programmable FPGAs
Jiang, S., He, D., Yang, C., Xu, C., Luo, G., Chen, Y., Liu, Y., & Jiang, J. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8485850 · Cited by 40
“This is reasonable because there is no specific acceleration for the binarized operation on general CPUs.”
BackgroundInfluentialReconfigurability, Why It Matters in AI Tasks Processing: A Survey of Reconfigurable AI Chips
Wei, S., Lin, X., Tu, F., Wang, Y., Liu, L., & Yin, S. (2022) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2022.3228860 · Cited by 40
“Many early reconfigurable DNN processors are designed to have a single engine and fixed dataflow (SEFD) [24], [52], [74], [75], [76], [77].”
BackgroundStream: Design Space Exploration of Layer-Fused DNNs on Heterogeneous Dataflow Accelerators
Symons, A., Mei, L., Colleman, S., Houshmand, P., Karl, S., & Verhelst, M. (2024) · IEEE Transactions on Computers · doi:10.1109/tc.2024.3477938 · Cited by 40
“An alternative approach is layer-fused scheduling [12], also known as depth-first [13] or cascaded [14] scheduling (Fig.”
Hong, H., Choi, D., Kim, N.-J., Lee, H., Kang, B., Kang, H., & Kim, H. (2024) · Journal of Real-Time Image Processing · doi:10.1007/s11554-024-01442-8 · Cited by 39
“First, we introduce the fused-layer architecture [63–66], which fuses and processes operations of adjacent layers, and the multi-PU architecture [14, 67–69], which uses multiple processors to efficiently handle operations of various shapes.”
Yüzügüler, A. C., Sönmez, C., Drumond, M. P., Oh, Y., Falsafi, B., & Frossard, P. (2022) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3572917 · Cited by 39
“With the increase in the popularity of DNNs in the last decade, many hardware and software techniques have been proposed to accelerate DNN workloads.”
MethodInfluentialKoppula, S., Orosa, L., Yağlıkçı, A. G., Azizi, R., Shahroodi, T., Kanellopoulos, K., & Mutlu, O. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05340 · Cited by 39
Pentecost, L., Donato, M., Reagen, B., Gupta, U., Ma, S., Wei, G.-Y., & Brooks, D. M. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358258 · Cited by 38
High-Utilization, High-Flexibility Depth-First CNN Coprocessor for Image Pixel Processing on FPGA
Colleman, S., & Verhelst, M. (2021) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2020.3046125 · Cited by 38
“The DF principle can naively be deployed in a tile-based manner without reuse of intermediate features [19], as shown in Fig.”
BackgroundMethodEvolutionary Multi-Objective Model Compression for Deep Neural Networks
Wang, Z., Luo, T., Li, M., Zhou, J. T., Goh, R. S. M., & Zhen, L. (2021) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2021.3084393 · Cited by 39
“: X Y [16] [17] : C C I O [18] [19]”
DeepCuts: a deep learning optimization framework for versatile GPU workloads
Jung, W., Dao, T. T., & Lee, J. (2021) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/3453483.3454038 · Cited by 37
“Alwani et al. [10] propose a fusion technique that keeps the intermediate result of the previous kernel in on-chip memory and uses it for the next kernel.”
BackgroundScaling the Cascades: Interconnect-Aware FPGA Implementation of Machine Learning Problems
Samajdar, A., Garg, T., Krishna, T., & Kapre, N. (2019) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl.2019.00061 · Cited by 37
Shi, H., Xin, C., Mao, W., & Wang, Z. (2024) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2024.3422684 · Cited by 36
“These oversights hinder the acceleration potential through techniques like layer fusion and pipeline processing [17], thus limiting ViTs’ achievable hardware efficiency and real-world applications.”
BackgroundSamajdar, A., Qin, E., Pellauer, M., & Krishna, T. (2022) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3489517.3530506 · Cited by 36
DeepBurning-SEG: Generating DNN Accelerators of Segment-Grained Pipeline Architecture
Cai, X., Wang, Y., Ma, X., Han, Y., & Zhang, L. (2022) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro56248.2022.00094 · Cited by 36
“To alleviate the problem of resource utilization, an effective way is to map and pipeline the entire DNN over the full pipeline accelerator [1], [46], [69], [74], [83], [84] as shown in Fig.”
BackgroundMethodInfluentialA dedicated hardware accelerator for real-time acceleration of YOLOv2
Xu, K., Wang, X., Liu, X., Cao, C., Li, H., Peng, H., & Wang, D. (2020) · Journal of Real-Time Image Processing · doi:10.1007/s11554-020-00977-w · Cited by 35
Mind the Gap: Attainable Data Movement and Operational Intensity Bounds for Tensor Algorithms
Huang, Q., Tsai, P.-A., Emer, J., & Parashar, A. (2024) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca59077.2024.00021 · Cited by 34
“For example, the tensor A[5][4] has 2 ranks with shapes 5 and 4 respectively.”
Yu, Y., & Jha, N. K. (2020) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2020.3003328 · Cited by 34
Demand Layering for Real-Time DNN Inference with Minimized Memory Usage
Ji, M., Yi, S., Koo, C., Ahn, S., Seo, D., Dutt, N. D., & Kim, J.-C. (2022) · IEEE Real-Time Systems Symposium (RTSS) · doi:10.1109/rtss55097.2022.00033 · Cited by 34
“[43] reduces off-chip memory usage by fusing multiple CNN layers to utilize on-chip memory efficiently.”
Real-time object detection towards high power efficiency
Yu, J., Guo, K., Hu, Y., Ning, X., Qiu, J., Mao, H., Yao, S., Tang, T., Li, B., Wang, Y., et al. (2018) · Design, Automation, and Test in Europe · doi:10.23919/date.2018.8342100 · Cited by 34
“Alwani [20] introduces a fuse-layer policy, enabling caching of intermediate data between the adjacent CNN layers, and reduces the transfer of intermediate data.”
BackgroundIntelligence Beyond the Edge: Inference on Intermittent Embedded SystemsPreprint
Gobieski, G., Beckmann, N., & Lucia, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.07751 · Cited by 34
“of the neural network. SONIC optimizes DNN inference by ensuring that these loops execute correctly on intermittent power while adding much less overhead than prior task-based systems. 7 SONIC TAILS a[7] += b[7] !c a[4] += b[4] !c Tile -5 Tile -12 a[0] += b[0] !c a[9] += b[9] !c a[0] += b[0] !c ! ! a[5] += b[5] !c ! E…”
BackgroundFlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model TrainingPreprint
Lym, S., & Erez, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.13027 · Cited by 34
“The execution time of these layers changes depending on accelerator design choices (off-chip memory BW and on-chip buffer design) and workload scheduling methods (multi-layer fusion [48], and layer fusion [21], [49]).”
BackgroundA Fast and Scalable Architecture to Run Convolutional Neural Networks in Low Density FPGAs
Véstias, M. P., Duarte, R. P., de Sousa, J. T., & Neto, H. C. (2020) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2020.103136 · Cited by 33
Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips
Dagli, I., & Belviranli, M. E. (2024) · ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3627535.3638502 · Cited by 32
“This grouping considers several factors: 1) Preserving layer optimizations: Layer/operator fusion [2, 7, 43] merges multiple layers into a single layer, optimizing the computational load and minimizing memory accesses.”
BackgroundZCOMP: Reducing DNN Cross-Layer Memory Footprint Using Vector Extensions
Akin, B., Chishti, Z. A., & Alameldeen, A. R. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358305 · Cited by 32
“Fused-layer CNN accelerator merges multiple layers to minimize intermediate data buffering [13].”
BackgroundAccelerating Inference of Convolutional Neural Networks Using In-memory Computing
Dazzi, M., Sebastian, A., Benini, L., & Eleftheriou, E. S. (2021) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2021.674154 · Cited by 32
“To this end, custom hardware platforms such as application-specific integrated circuits (ASICs) are being designed (Chen et al., 2016) for low power and efficient execution of DNNs. ASICs are quite energy efficient in terms of TOPS/W and can reach state-of-the art accuracy and throughput in a variety of…”
Method
Metadata
- Authors
- Alwani, M., Chen, H., Ferdman, M., & Milder, P.
- Affiliations
- Stony Brook University (US)
- Venue
- 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
- Publisher
- IEEE
- Published
- 2016-10
- Pages
- 1-12
- ISSN
- 1072-4451 (ISSN-L)
- OpenAlex
- W4249932213
- Cited by
- 662
- References
- 20
- Citations by year
- 2026: 20 · 2025: 51 · 2024: 72 · 2023: 67 · 2022: 87 · 2021: 103 · 2020: 72 · 2019: 90 · 2018: 76 · 2017: 23 · 2016: 1
- FWCI
- 21.98
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/ACM International Symposium on Microarchitecture (MICRO) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- CNN accelerator, layer fusion, data flow design, FPGA implementation, convolutional neural network inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.