FALCON: Feature driven selective classification for energy-efficient image recognition
Open accessAbstract
Machine-learning algorithms have shown outstanding image recognition/classification performance for computer vision applications. However, the compute and energy requirement for implementing such classifier models for large-scale problems is quite high. In this paper, we propose feature driven selective classification (FALCON) inspired by the biological visual attention mechanism in the brain to optimize the energy-efficiency of machine-learning classifiers. We use the consensus in the characteristic features (color/texture) across images in a dataset to decompose the original classification problem and construct a tree of classifiers (nodes) with a generic-to-specific transition in the classification hierarchy. The initial nodes of the tree separate the instances based on feature information and selectively enable the latter nodes to perform object specific classification. The proposed methodology allows selective activation of only those branches and nodes of the classification tree that are relevant to the input while keeping the remaining nodes idle. Additionally, we propose a programmable and scalable neuromorphic engine (NeuE) that utilizes arrays of specialized neural computational elements to execute the FALCON-based classifier models for diverse datasets. The structure of FALCON facilitates the reuse of nodes while scaling up from small classification problems to larger ones thus allowing us to construct classifier implementations that are significantly more efficient. We evaluate our approach for a 12-object classification task on the Caltech101 dataset and ten-object task on CIFAR-10 dataset by constructing FALCON models on the NeuE platform in 45-nm technology. Our results demonstrate up to 3.66× improvement in energy-efficiency for no loss in output quality, and even higher improvements of up to 5.91× with 3.9% accuracy loss compared to an optimized baseline network. In addition, FALCON shows an improvement in training time of up to 1.96× as compared to the traditional classification approach.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In iCaRL, a ResNet-32 is retrained with new data and only 2000 samples of old data (exemplars) at every stage; it recovers good performance compared to LwF but still falls short of state-of-the-art by 18%.
“In 'iCarl' [16], a ResNet-32 is retrained with new data and only 2000 samples of old data (called 'exemplars') at every stage. It is able to recover a good amount performance, compared to 'LwF' but still falls short of state-of-the-art by 18%.”
From Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning (2020) · cites this paper as [16]
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244 · see citing works
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,050 · see citing works
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261 · see citing works
Nanoscale Memristor Device as Synapse in Neuromorphic Systems
Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., & Lu, W. (2010) · Nano Letters · doi:10.1021/nl904092h · Cited by 4,174 · cited 1× · see citing works
“The second is the use of emerging post-CMOS device such as resistive RAM [26], memristive crossbars [27], and spin-tronics [28], to realize the individual computational elements: neurons and synapses more efficiently.”
Fei-Fei, L., Fergus, R., & Perona, P. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2005.09.012 · Cited by 2,359 · see citing works
Mechanisms of visual attention in the human cortex
Ungerleider, S. K. A. L. G. (2000) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.23.1.315 · Cited by 2,159 · see citing works
Wide Residual NetworksPreprint
Zagoruyko, S., & Komodakis, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07146 · Cited by 1,850 · see citing works
A survey of content-based image retrieval with high-level semantics
Liu, Y., Zhang, D., Lu, G., & Ma, W.-Y. (2007) · Pattern Recognition · doi:10.1016/j.patcog.2006.04.045 · Cited by 1,696 · cited 1× · see citing works
“Using semantics or feature information for improving the accuracy of content-based image retrieval systems has been an active area of research [10].”
Deep Convolutional Network Cascade for Facial Point Detection
Sun, Y. J., Wang, X., & Tang, X. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.446 · Cited by 1,405 · see citing works
Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers
Allwein, E. L., Schapire, R. E., & Singer, Y. (2000) · Cited by 914 · see citing works
On optimization methods for deep learning
Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., Le, Q. V., & Ng, A. Y. (2011) · International Conference on Machine Learning · Cited by 868 · see citing works
14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks
Chen, Y., Krishna, T., Emer, J., & Sze, V. (2016) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc.2016.7418007 · Cited by 713 · see citing works
Sharing features: efficient boosting procedures for multiclass object detection
Torralba, A. B., Murphy, K. P., & Freeman, W. T. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315241 · Cited by 581 · see citing works
Torch: a modular machine learning software library
Collobert, R., Bengio, S., & Mariéthoz, J. (2002) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 443 · see citing works
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · Proceedings of the 37th annual international symposium on Computer architecture · doi:10.1145/1815961.1815993 · Cited by 343 · see citing works
Object detection using gabor filters
Jain, A. K., Ratha, N., & Lakshmanan, S. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00068-4 · Cited by 302 · see citing works
Label Embedding Trees for Large Multi-Class Tasks
Bengio, S., Weston, J., & Grangier, D. (2011) · Cited by 290 · see citing works
Venkataramani, S., Ranjan, A., Roy, K., & Raghunathan, A. (2014) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/2627369.2627613 · Cited by 259 · see citing works
Ensemble deep learning for speech recognition
Deng, L., & Platt, J. C. (2014) · Interspeech 2014 · doi:10.21437/interspeech.2014-433 · Cited by 248 · see citing works
Content Based Image Retrieval using Color and Texture
Singha, M. (2012) · Signal & Image Processing An International Journal · doi:10.5121/sipij.2012.3104 · Cited by 215 · see citing works
Fast image segmentation based on K-Means clustering with histograms in HSV color space
Chen, T.-W., Chen, Y., & Chien, S. (2008) · IEEE Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2008.4665097 · Cited by 196 · see citing works
Attribute Discovery via Predictable Discriminative Binary Codes
Rastegari, M., Farhadi, A., & Forsyth, D. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33783-3_63 · Cited by 179 · see citing works
Specifications of Nanoscale Devices and Circuits for Neuromorphic Computational Systems
Rajendran, B., Liu, Y., Seo, J.-S., Gopalakrishnan, K., Chang, L., Friedman, D. J., & Ritter, M. B. (2012) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2012.2227969 · Cited by 158 · see citing works
Computer science: The learning machines
Jones, N. (2014) · Nature · doi:10.1038/505146a · Cited by 155 · see citing works
GLHS: A Generalized Lightness, Hue, and Saturation Color Model
Levkowitz, H., & Herman, G. T. (1993) · CVGIP Graphical Models and Image Processing · doi:10.1006/cgip.1993.1019 · Cited by 152 · see citing works
A dynamically configurable coprocessor for convolutional neural networks
Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010) · ACM SIGARCH Computer Architecture News · doi:10.1145/1816038.1815993 · Cited by 151 · see citing works
Discriminative learning of relaxed hierarchy for large-scale visual recognition
Gao, T., & Koller, D. (2011) · International Conference on Computer Vision · doi:10.1109/iccv.2011.6126481 · Cited by 144 · see citing works
Fast and Balanced: Efficient Label Tree Learning for Large Scale Object Recognition
Deng, J., Satheesh, S., Berg, A. C., & Fei-Fei, L. (2011) · Cited by 144 · see citing works
Identification Using Encrypted Biometrics
Haghighat, M., Zonouz, S., & Abdel-Mottaleb, M. S. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40246-3_55 · Cited by 120 · see citing works
Exploiting Object Hierarchy: Combining Models from Different Category Levels
Zweig, A., & Weinshall, D. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4409064 · Cited by 111 · see citing works
Automatic Video Genre Categorization using Hierarchical SVM
Yuan, X., Lai, W., Mei, T., Hua, X., Wu, X., & Li, S. (2006) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2006.313037 · Cited by 110 · see citing works
Ramasubramanian, S. G., Venkatesan, R., Sharad, M., Roy, K., & Raghunathan, A. (2014) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/2627369.2627625 · Cited by 84 · see citing works
Martín, K. R. (1981) · The Arithmetic Teacher · doi:10.5951/at.29.3.0041 · Cited by 70 · see citing works
Gabor Filtering of Complex Hue/Saturation Images for Color Texture Classification
Palm, C., Keysers, D., Lehmann, T., & Spitzer, K. (2000) · Cited by 61 · see citing works
Conditional Probability Tree Estimation Analysis and AlgorithmsPreprint
Beygelzimer, A., Langford, J. C., Lifshits, Y., Sorkin, G. B., & Strehl, A. L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.2031 · Cited by 54 · see citing works
Energy-Efficient and Improved Image Recognition with Conditional Deep Learning
Panda, P., Sengupta, A., & Roy, K. (2017) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3007192 · Cited by 47 · see citing works
A Real-Time Gabor Primal Sketch for Visual Attention
Bernardino, A. J. M., & Santos-Victor, J. (2005) · Lecture notes in computer science · doi:10.1007/11492429_41 · Cited by 42 · see citing works
Panda, P., Sengupta, A., Sarwar, S. S., Srinivasan, G., Venkataramani, S., Raghunathan, A., & Roy, K. (2016) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/2897937.2905009 · Cited by 42 · see citing works
Neuroscience: Toward Unbinding the Binding Problem
Whitney, D. C. (2009) · Current Biology · doi:10.1016/j.cub.2009.01.047 · Cited by 31 · see citing works
Beyond charge-based computation: Boolean and non-Boolean computing with spin torque devices
Roy, K., Sharad, M., Fan, D., & Yogendra, K. (2013) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped.2013.6629282 · Cited by 8 · see citing works
Object Detection using Semantic Decomposition for Energy-Efficient Neural Computing.Preprint
Panda, P., Sengupta, A., Venkataramani, S., Raghunathan, A., & Roy, K. (2015) · arXiv (Cornell University) · Cited by 2 · see citing works
Flagged first, then by citations
3 of the 46 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning
Roy, D., Panda, P., & Roy, K. (2020) · Neural Networks · doi:10.1016/j.neunet.2019.09.010 · Cited by 257 · see citing works
Incremental Learning in Deep Convolutional Neural Networks Using Partial Network Sharing
Sarwar, S. S., Ankit, A., & Roy, K. (2019) · IEEE Access · doi:10.1109/access.2019.2963056 · Cited by 127 · see citing works
Ankit, A., Sengupta, A., Panda, P., & Roy, K. (2017) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3061639.3062311 · Cited by 93 · see citing works
AI on the edge: a comprehensive review
Su, W., Li, L., Liu, F., He, M., & Liang, X. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10141-4 · Cited by 86 · see citing works
Designing adaptive neural networks for energy-constrained image classification
Stamoulis, D., Chin, T.-W. (., Prakash, A. K., Fang, H., Sajja, S., Bognar, M., & Marculescu, D. (2018) · Proceedings of the International Conference on Computer-Aided Design · doi:10.1145/3240765.3240796 · Cited by 63 · see citing works
Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices
Li, G., Ma, X., Wang, X., Liu, L., Xue, J., & Feng, X. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3013050 · Cited by 42 · see citing works
Modular Neural Networks for Low-Power Image Classification on Embedded Devices
Goel, A., Aghajanzadeh, S., Tung, C., Chen, S.-H., Thiruvathukal, G. K., & Lu, Y.-H. (2020) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3408062 · Cited by 25 · see citing works
Efficient Computer Vision on Edge Devices with Pipeline-Parallel Hierarchical Neural Networks
Goel, A., Tung, C., Hu, X., Thiruvathukal, G. K., Davis, J. C., & Lu, Y.-H. (2022) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/asp-dac52403.2022.9712574 · Cited by 21 · see citing works
Adapting Neural Networks at Runtime: Current Trends in At-Runtime Optimizations for Deep Learning
Sponner, M., Waschneck, B., & Kumar, A. S. (2024) · ACM Computing Surveys · doi:10.1145/3657283 · Cited by 15 · see citing works
Ankit, A., Sengupta, A., Panda, P., & Roy, K. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.06064 · Cited by 14 · see citing works
An Agile Precision-Tunable CNN Accelerator based on ReRAM
He, Y., Wang, Y., Wang, Y., Li, H., & Li, X. (2019) · doi:10.1109/iccad45719.2019.8942163 · Cited by 13 · see citing works
Energy-Efficient Object Detection Using Semantic Decomposition
Panda, P., Venkataramani, S., Sengupta, A., Raghunathan, A., & Roy, K. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2707077 · Cited by 7 · see citing works
Low-power object counting with hierarchical neural networks
Goel, A., Tung, C., Aghajanzadeh, S., Ghodgaonkar, I., Ghosh, S., Thiruvathukal, G. K., & Lu, Y.-H. (2020) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3370748.3406569 · Cited by 7 · see citing works
A Fast Precision Tuning Solution for Always-On DNN Accelerators
Wang, Y., He, Y., Cheng, L., Li, H., & Li, X. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3089667 · Cited by 5 · see citing works
Tang, Y., Wang, Y., Li, H., & Li, X. (2019) · ACM Journal on Emerging Technologies in Computing Systems · doi:10.1145/3358696 · Cited by 5 · see citing works
Directed Acyclic Graph-based Neural Networks for Tunable Low-Power Computer Vision
Goel, A., Tung, C., Eliopoulos, N., Hu, X., Thiruvathukal, G. K., Davis, J. C., & Lu, Y.-H. (2022) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1145/3531437.3539723 · Cited by 3 · see citing works
Energy-efficient and Robust Cumulative Training with Net2Net Transformation
Feng, A., & Panda, P. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207451 · Cited by 3 · see citing works
Image offset density distribution model and recognition of hand knuckle
Yang, S., Gong, L., & Qiao, D. (2019) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-019-0422-y · Cited by 2 · see citing works
BlocTrain: Block-Wise Conditional Training and Inference for Efficient Spike-Based Deep Learning
Srinivasan, G., & Roy, K. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.603433 · Cited by 2 · see citing works
Сергеевич, О. Д., & Усатиков, С. В. (2017) · Scientific and technical journal of information technologies mechanics and optics · doi:10.17586/2226-1494-2017-17-4-685-693
Deep Neural Networks with an Adaptive Runtime for Eye Tracking on Smart Glasses
Fuhl, W. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228980
- Authors
- Panda, P., Ankit, A., Wijesinghe, P., & Roy, K.
- Affiliations
- Purdue University West Lafayette (US)
- Venue
- IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Published
- 2017-12
- Volume
- 36
- Issue
- 12
- Pages
- 2017-2029
- ISSN
- 0278-0070 (ISSN-L)
- 1937-4151
- OpenAlex
- W2529958663
- Cited by
- 21
- References
- 46
- Citations by year
- 2025: 1 · 2024: 1 · 2022: 3 · 2021: 2 · 2020: 4 · 2019: 5 · 2018: 1 · 2017: 4
- FWCI
- 1.59
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- C-SPIN, one of the six centers of StarNet, a Semiconductor Research Corporation Program through MARCO and DARPA
- Semiconductor Research Corporation
- National Science Foundation
- Intel Corporation
- Vannevar Bush Faculty Fellowship
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- selective prediction, color and texture features, neuromorphic hardware, 45 nm CMOS, training time reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
