BranchyNet: Fast inference via early exiting from deep neural networks
The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Deep neural networks are state of the art methods for many learning tasks due to their ability to extract increasingly better features at each network layer. However, the improved performance of additional layers in a deep network comes at the cost of added latency and energy usage in feedforward inference. As networks continue to get deeper and larger, these costs become more prohibitive for real-time and energy-sensitive applications. To address this issue, we present BranchyNet, a novel deep network architecture that is augmented with additional side branch classifiers. The architecture allows prediction results for a large portion of test samples to exit the network early via these branches when samples can already be inferred with high confidence. BranchyNet exploits the observation that features learned at an early layer of a network may often be sufficient for the classification of many data points. For more difficult samples, which are expected less frequently, BranchyNet will use further or all network layers to provide the best likelihood of correct prediction. We study the BranchyNet architecture using several well-known networks (LeNet, AlexNet, ResNet) and datasets (MNIST, CIFAR10) and show that it can both improve accuracy and significantly reduce the inference time of the network.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In contrast, EE-ensemble method employs only half as many exits (only EEs) compared to QUTE, and the EE's regularization effect helps network accuracy (Teerapittayanon et al., 2016).”
From QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoring (2024) · cites this paper as Teerapittayanon et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
(2010) · Cited by 14,134
Cited by
Items 101–200 of 1,000
Towards Real-time Cooperative Deep Inference over the Cloud and Edge End Devices
Zhang, S., Li, Y., Liu, X., Guo, S., Wang, W., Wang, J., Ding, B., & Wu, D. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3397315 · Cited by 99
CLIO: enabling automatic compilation of deep learning pipelines across IoT and cloud
Huang, J., Samplawski, C., Ganesan, D., Marlin, B. M., & Kwon, H. (2020) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3372224.3419215 · Cited by 99
Wang, T., Zhou, W., Zeng, Y., & Zhang, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.873 · Cited by 99
Zygarde: Time-Sensitive On-Device Deep Inference and Adaptation on Intermittently-Powered Systems
Islam, B., & Nirjon, S. (2020) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3411808 · Cited by 97
Channel Selection Using Gumbel Softmax
Herrmann, C., Bowen, R. S., & Zabih, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58583-9_15 · Cited by 97
Mobile Foundation Model as Firmware
Yuan, J., Yang, C., Cai, D., Wang, S., Yuan, X., Zhang, Z., Li, X., Zhang, D., Mei, H., Jia, X., et al. (2024) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3636534.3649361 · Cited by 96
Width & Depth Pruning for Vision Transformers
Yu, F., Huang, K., Wang, M., Cheng, Y., Chu, W., & Cui, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20222 · Cited by 95
MobiSR: Efficient On-Device Super-Resolution through Heterogeneous Mobile Processors
Lee, R., Venieris, S. I., Dudziak, Ł., Bhattacharya, S., & Lane, N. D. (2019) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3300061.3345455 · Cited by 94
A survey: Distributed Machine Learning for 5G and beyond
Nassef, O., Sun, W., Purmehdi, H., Tatipamula, M., & Mahmoodi, T. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108820 · Cited by 94
Confident adaptive language modeling
Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Bahri, D., Tran, V., Tay, Y., & Metzler, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1269 · Cited by 91
Head2Toe: Utilizing Intermediate Representations for Better Transfer LearningPreprint
Evci, U., Dumoulin, V., Larochelle, H., & Mozer, M. C. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.03529 · Cited by 91
FlexDNN: Input-Adaptive On-Device Deep Learning for Efficient Mobile Vision
Fang, B., Zeng, X., Zhang, F., Xu, H. F., & Zhang, M. (2020) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec50012.2020.00014 · Cited by 90
Pyramid: Enabling Hierarchical Neural Networks with Edge Computing
He, Q., Dong, Z., Chen, F., Deng, S., Liang, W., & Yang, Y. J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511990 · Cited by 90
Guo, Y., Peng, W., Chen, Y., Zhang, L., Liu, X., Huang, X., & Ma, Z. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109639 · Cited by 90
DynamicDet: A Unified Dynamic Architecture for Object Detection
Lin, Z., Wang, Y., Zhang, J., & Chu, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00608 · Cited by 90
SEENN: Towards Temporal Spiking Early Exit Neural Networks
Li, Y., Geller, T., Li, Y., & Panda, P. (2023) · neural information processing systems · doi:10.52202/075280-2764 · Cited by 90
Integrating Edge Intelligence and Blockchain: What, Why, and How
Wang, X., Ren, X., Qiu, C., Xiong, Z., Yao, H., & Leung, V. C. M. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2022.3189962 · Cited by 89
When Does Confidence-Based Cascade Deferral Suffice?
Jitkrittum, W., Gupta, N., Menon, A. K., Narasimhan, H., Rawat, A. S., & Kumar, S. (2023) · neural information processing systems · doi:10.52202/075280-0431 · Cited by 89
Convolutional Networks with Adaptive Inference Graphs
Veit, A., & Belongie, S. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01190-4 · Cited by 89
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
Wang, C., Chen, X., Zhang, N., Tian, B., Xu, H., Deng, S., & Chen, H. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.11779 · Cited by 91
Deep learning for compressive sensing: a ubiquitous systems perspective
Machidon, A. L., & Pejović, V. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10259-5 · Cited by 87
LeeBERT: Learned Early Exit for BERT with cross-level optimization
Zhu, W. (2021) · doi:10.18653/v1/2021.acl-long.231 · Cited by 85
Neural Network Distiller: A Python Package For DNN Compression Research
Zmora, N., Jacob, G., Zlotnik, L., Elharar, B., & Novik, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12232 · Cited by 86
Hu, T.-K., Chen, T., Wang, H., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10025 · Cited by 50
DeeCap: Dynamic Early Exiting for Efficient Image Captioning
Fei, Z., Yan, X., Wang, S., & Tian, Q. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01190 · Cited by 83
Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for FreePreprint
Wang, H., Chen, T., Gui, S., Hu, T.-K., Liu, J., & Wang, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11828 · Cited by 83
Wu, Y., Wang, Z., Jia, Z., Shi, Y., & Hu, J. (2020) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18072.2020.9218526 · Cited by 81
ApproxDet: content and contention-aware approximate object detection for mobiles
Xu, R., Zhang, C.-L., Wang, P., Lee, J., Mitra, S., Chaterji, S., Li, Y., & Bagchi, S. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3431159 · Cited by 81
Efficient federated learning for fault diagnosis in industrial cloud-edge computing
Wang, Q., Li, Q., Wang, K., Wang, H., & Zeng, P. (2021) · Computing · doi:10.1007/s00607-021-00970-6 · Cited by 81
Batch-Shaping for Learning Conditional Channel Gated NetworksPreprint
Bejnordi, B. E., Blankevoort, T., & Welling, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.06627 · Cited by 81
Consistent Accelerated Inference via Confident Adaptive Transformers
Schuster, T., Fisch, A., Jaakkola, T., & Barzilay, R. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.406 · Cited by 80
Efficient AI System Design With Cross-Layer Approximate Computing
Venkataramani, S., Sun, X. N., Wang, N., Chen, C., Choi, J., Kang, M., Agarwal, A., Oh, J., Jain, S., Babinsky, T., et al. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3029453 · Cited by 81
Learning to Weight Samples for Dynamic Early-Exiting Networks
Han, Y., Pu, Y., Lai, Z., Wang, C., Song, S., Cao, J., Huang, W., Deng, C., & Huang, G. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20083-0_22 · Cited by 79
Scalable Quantitative Verification for Deep Neural Networks
Baluta, T., Chua, Z. L., Meel, K. S., & Saxena, P. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00039 · Cited by 78
Edge intelligence-driven digital twin of CNC system: Architecture and deployment
Yu, H., Yu, D. I., Wang, C., Hu, Y., & Li, Y. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2022.102418 · Cited by 79
A Panda? No, It's a Sloth: Slowdown Attacks on Adaptive Multi-Exit Neural Network InferencePreprint
Hong, S., Kaya, Y., Modoranu, I.-V., & Dumitraş, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02432 · Cited by 77
Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection
Yang, M., Chen, Y., & Kim, H.-S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19839-7_14 · Cited by 77
Enabling All In-Edge Deep Learning: A Literature Review
Joshi, P., Hasanuzzaman, M., Thapa, C., Afli, H., & Scully, T. (2023) · IEEE Access · doi:10.1109/access.2023.3234761 · Cited by 77
Incremental On-Device Tiny Machine Learning
Disabato, S., & Roveri, M. (2020) · International Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things (AIChallengeIoT) · doi:10.1145/3417313.3429378 · Cited by 76
EdgeML: An AutoML Framework for Real-Time Deep Learning on the Edge
Zhao, Z., Wang, K., Ling, N., & Xing, G. (2021) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1145/3450268.3453520 · Cited by 75
Computation-efficient deep learning for computer vision: A survey
Wang, Y., Han, Y., Wang, C., Song, S., Tian, Q., & Huang, G. M. (2026) · Cybernetics and Intelligence · doi:10.26599/cai.2024.9390002 · Cited by 74
Lee, R., Venieris, S. I., & Lane, N. D. (2021) · ACM Computing Surveys · doi:10.1145/3469094 · Cited by 73
CNNPC: End-Edge-Cloud Collaborative CNN Inference With Joint Model Partition and Compression
Yang, S., Zhang, Z., Zhao, C., Song, X., Guo, S., & Li, H. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3177782 · Cited by 73
Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU HeterogeneityPreprint
Griggs, T., Liu, X., Yu, J., Kim, D., Chiang, W.-L., Cheung, A., & Stoica, I. G. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.14527 · Cited by 73
Dynamic Recursive Neural Network
Guo, Q., Yu, Z., Wu, Y., Liang, D., Qin, H., & Yan, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00529 · Cited by 72
Dynamic Dual Gating Neural Networks
Li, F., Li, G., He, X., & Cheng, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00528 · Cited by 72
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D ParallelismPreprint
Chen, Y., Pan, X., Li, Y., Ding, B., & Zhou, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.04916 · Cited by 72
LegoDNN: block-grained scaling of deep neural networks for mobile vision
Han, R., Zhang, Q., Liu, C. H., Wang, G., Tang, J., & Chen, L. Y. (2021) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3447993.3483249 · Cited by 71
Any-Precision Deep Neural Networks
Yu, H., Li, H., Shi, H., Huang, T. S., & Hua, G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17286 · Cited by 71
LGViT: Dynamic Early Exiting for Accelerating Vision Transformer
Xu, G., Hao, J., Shen, L., Hu, H., Luo, Y., Lin, H., & Shen, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611762 · Cited by 70
Yao, H., Yu, W., & Wang, X. (2022) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2022.3204368 · Cited by 69
SplitGP: Achieving Both Generalization and Personalization in Federated Learning
Han, D.-J., Kim, D.-Y., Choi, M., Brinton, C. G., & Moon, J. J. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10229027 · Cited by 69
DynExit: A Dynamic Early-Exit Strategy for Deep Residual Networks
Wang, M., Mo, J., Lin, J. T., Wang, Z., & Du, L. (2019) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips47522.2019.9020551 · Cited by 68
Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge SynergyPreprint
Li, E., Zhou, Z., & Chen, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07840 · Cited by 68
Zhao, W., Han, Y., Tang, J., Wang, K., Song, Y., Huang, G. S., Wang, F., & You, Y. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.03456 · Cited by 68
AppealNet: An Efficient and Highly-Accurate Edge/Cloud Collaborative Architecture for DNN Inference
Li, M., Li, Y., Tian, Y., Jiang, L., & Xu, Q. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586176 · Cited by 67
PCEE-BERT: Accelerating BERT Inference via Patient and Confident Early Exiting
Zhang, Z., Zhu, W., Zhang, J., Wang, P., Jin, R., & Chung, T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.25 · Cited by 67
Human Activity Recognition on Microcontrollers with Quantized and Adaptive Deep Neural Networks
Daghero, F., Burrello, A., Xie, C., Castellano, M., Gandolfi, L., Calimera, A., Macii, E., Poncino, M., & Pagliari, D. J. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3542819 · Cited by 67
Lee, J., Mukhanov, L., Molahosseini, A. S., Minhas, U. I., Yang, H., del Rincón, J. M., Dichev, K., Hong, C.-H., & Vandierendonck, H. (2023) · ACM Computing Surveys · doi:10.1145/3587095 · Cited by 69
Zero time waste in pre-trained early exit neural networks
Wójcik, B., Przewięźlikowski, M., Szatkowski, F., Wołczyk, M., Bałazy, K., Krzepkowski, B., Podolak, I. T., Tabor, J., Śmieja, M., & Trzcinski, T. P. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.10.003 · Cited by 67
A survey on deep learning for challenged networks: Applications and trends
Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S. M., Medeiros, D. S. V., & Campista, M. E. M. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103213 · Cited by 66
Hastening Stream Offloading of Inference via Multi-Exit DNNs in Mobile Edge Computing
Liu, Z., Song, J., Qiu, C., Wang, X., Chen, X., He, Q., & Sheng, H. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3218724 · Cited by 66
HAPI: hardware-aware progressive inference
Laskaridis, S., Venieris, S. I., Kim, H., & Lane, N. D. (2020) · doi:10.1145/3400302.3415698 · Cited by 66
ALERT: Accurate Learning for Energy and TimelinessPreprint
Wan, C., Santriaji, M. H., Rogers, E. A., Hoffmann, H., Maire, M., & Lu, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00119 · Cited by 66
Zhang, X., Zhu, X., Wang, J., Yan, H., Chen, H., & Bao, W. (2020) · Information Sciences · doi:10.1016/j.ins.2020.05.137 · Cited by 65
Early Exiting BERT for Efficient Document Ranking
Xin, J., Nogueira, R. F., Yu, Y., & Lin, J. (2020) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2020.sustainlp-1.11 · Cited by 65
Scalable Video Object Segmentation With Identification Mechanism
Yang, Z., Miao, J., Wei, Y., Wang, W., Wang, X., & Yang, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3383592 · Cited by 65
A Dynamic Frame Selection Framework for Fast Video Recognition
Wu, Z., Li, H., Xiong, C., Jiang, Y., & Davis, L. S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3029425 · Cited by 64
A survey of secure semantic communications
Meng, R., Gao, S., Fan, D., Gao, H., Wang, Y., Xu, X., Wang, B., Lv, S., Zhang, Z., Sun, M., et al. (2025) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2025.104181 · Cited by 64
A Truthful and Efficient Incentive Mechanism for Demand Response in Green Datacenters
Zhou, Z., Liu, F., Chen, S., & Li, Z. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2882174 · Cited by 64
Advancements in Accelerating Deep Neural Network Inference on AIoT Devices: A Survey
Cheng, L., Gu, Y., Liu, Q., Yang, L., Liu, C., & Wang, Y. (2024) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2024.3353176 · Cited by 64
SubFlow: A Dynamic Induced-Subgraph Strategy Toward Real-Time DNN Inference and Training
Lee, S., & Nirjon, S. (2020) · IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS) · doi:10.1109/rtas48715.2020.00-20 · Cited by 63
HiTDL: High-Throughput Deep Learning Inference at the Hybrid Mobile Edge
Wu, J. L., Wang, L., Pei, Q., Cui, X., Liu, F., & Yang, T. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3195664 · Cited by 62
Toward Carbon-Neutral Edge Computing: Greening Edge AI by Harnessing Spot and Future Carbon Markets
Ma, H., Zhou, Z., Zhang, X., & Chen, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3268339 · Cited by 62
Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference
Shen, J., Wang, Y., Xu, P., Fu, Y., Wang, Z., & Lin, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6025 · Cited by 62
Fully Dynamic Inference with Deep Neural Networks
Xia, W., Yin, H., Dai, X., & Jha, N. K. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2021.3056031 · Cited by 61
A Global Past-Future Early Exit Method for Accelerating Inference of Pre-trained Language Models
Liao, K., Zhang, Y., Ren, X., Su, Q., Sun, X., & He, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.162 · Cited by 61
Path Independent Equilibrium Models Can Better Exploit Test-Time Computation
Anil, C., Pokle, A., Liang, K., Treutlein, J., Wu, Y., Bai, S., Kolter, J. Z., & Grosse, R. (2022) · neural information processing systems · doi:10.52202/068431-0566 · Cited by 60
Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search
Kim, G., & Cho, K. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.508 · Cited by 59
“Escaping a network early (Teerapittayanon et al., 2016; Huang et al., 2017) based on the confidence of the prediction (Xin et al.”
BackgroundMulti-Dimensional QoS Evaluation and Optimization of Mobile Edge Computing for IoT: A Survey
Huang, J., Liu, F., & Zhang, J. (2024) · Chinese Journal of Electronics · doi:10.23919/cje.2023.00.264 · Cited by 59
Dynamic Neural Network Structure: A Review for its Theories and Applications
Guo, J., Chen, C. L. P., Liu, Z., & Yang, X. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3377194 · Cited by 58
Characterizing LLM Inference Energy-Performance Tradeoffs Across Workloads and GPU Scaling
Maliakel, P. J., Ilager, S., & Brandić, I. (2026) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid68966.2026.00013 · Cited by 59
AdaDet: An Adaptive Object Detection System Based on Early-Exit Neural Networks
Yang, L., Zheng, Z., Wang, J., Song, S., Huang, G. M., & Li, F. (2023) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2023.3274214 · Cited by 57
Ding, Y., Zuo, L., Jing, M., He, P. Y., & Xiao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.29066 · Cited by 57
Single-layer vision transformers for more accurate early exits with less overhead
Bakhtiarnia, A., Zhang, Q., & Iosifidis, A. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.06.038 · Cited by 58
Liu, J., Wang, Q., Wang, J., & Cai, X. (2024) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2024.findings-acl.179 · Cited by 56
ePerceptive: energy reactive embedded intelligence for batteryless sensors
Montanari, A., Sharma, M., Jenkus, D., Alloulah, M., Qendro, L., & Kawsar, F. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3430782 · Cited by 56
Self-Regulation for Semantic Segmentation
Zhang, D., Zhang, H., Tang, J., Hua, X., & Sun, Q. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00687 · Cited by 56
Energy-aware human activity recognition for wearable devices: A comprehensive review
Contoli, C., Freschi, V., & Lattanzi, E. (2024) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2024.101976 · Cited by 55
Deep model compression for mobile platforms: A survey
Nan, K., Liu, S., Du, J., & Liu, H. (2019) · Tsinghua Science & Technology · doi:10.26599/tst.2018.9010103 · Cited by 55
Ngo, M. V., Luo, T., & Quek, T. Q. S. (2021) · ACM Transactions on Internet of Things · doi:10.1145/3480172 · Cited by 54
Dai, P., Han, B., Li, K., Xu, X., Xing, H., & Liu, K. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3457793 · Cited by 54
UltraTrail: A Configurable Ultralow-Power TC-ResNet AI Accelerator for Efficient Keyword Spotting
Bernardo, P. P., Gerum, C., Frischknecht, A., Lübeck, K., & Bringmann, O. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3012320 · Cited by 54
CDFKD-MFS: Collaborative Data-Free Knowledge Distillation via Multi-Level Feature Sharing
Hao, Z.-W., Luo, Y., Wang, Z., Hu, H., & An, J. (2022) · IEEE transactions on multimedia · doi:10.1109/tmm.2022.3192663 · Cited by 42
Li, X., Lou, C., Chen, Y., Zhu, Z., Shen, Y., Ma, Y., & Zou, A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.26042 · Cited by 53
Ren, P., Qiao, X., Huang, Y., Liu, L., Dustdar, S., & Chen, J. (2020) · IEEE Network · doi:10.1109/mnet.011.1900305 · Cited by 53
A Lightweight Collaborative Deep Neural Network for the Mobile Web in Edge Cloud
Huang, Y., Qiao, X., Ren, P., Liu, L., Pu, C., Dustdar, S., & Chen, J. (2020) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3043051 · Cited by 53
Transkimmer: Transformer Learns to Layer-wise Skim
Guan, Y., Li, Z., Leng, J., Lin, Z., & Guo, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.502 · Cited by 53
Zhang, W., Zeadally, S., Li, W., Zhang, H., Hou, J., & Leung, V. C. M. (2022) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2022.3175725 · Cited by 52
Han, D.-J., Kim, D.-Y., Choi, M., Nickel, D., Moon, J. J., Chiang, M., & Brinton, C. G. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2023.3331690 · Cited by 52
Metadata
- Authors
- Teerapittayanon, S., McDanel, B., & Kung, H. T.
- Affiliations
- Harvard University (US)
- Venue
- 2016 23rd International Conference on Pattern Recognition (ICPR)
- Publisher
- IEEE
- Published
- 2016-12
- Pages
- 2464-2469
- ISSN
- 1041-3278 (ISSN-L)
- OpenAlex
- W2962677625
- Cited by
- 1,326
- References
- 27
- Citations by year
- 2026: 182 · 2025: 195 · 2024: 184 · 2023: 219 · 2022: 160 · 2021: 161 · 2020: 119 · 2019: 69 · 2018: 23 · 2017: 12 · 2016: 2
- FWCI
- 12.26
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition (publishedVersion)
- Funders
- Intel Corporation
- Nvidia
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- early exit, inference acceleration, deep learning architectures, inference latency reduction, energy-efficient inference, image classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.