The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
FLASH: F ast Neura l A rchitecture S earch with H ardware Optimization
Abstract
Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs). As the performance requirements of ML applications grow continuously, the hardware accelerators start playing a central role in DNN design. This trend makes NAS even more complicated and time-consuming for most real applications. This paper proposes FLASH, a very fast NAS methodology that co-optimizes the DNN accuracy and performance on a real hardware platform. As the main theoretical contribution, we first propose the NN-Degree, an analytical metric to quantify the topological characteristics of DNNs with skip connections (e.g., DenseNets, ResNets, Wide-ResNets, and MobileNets). The newly proposed NN-Degree allows us to do training-free NAS within one second and build an accuracy predictor by training as few as 25 samples out of a vast search space with more than 63 billion configurations. Second, by performing inference on the target hardware, we fine-tune and validate our analytical models to estimate the latency, area, and energy consumption of various DNN architectures while executing standard ML datasets. Third, we construct a hierarchical algorithm based on simplicial homology global optimization (SHGO) to optimize the model-architecture co-design process, while considering the area, latency, and energy consumption of the target hardware. We demonstrate that, compared to the state-of-the-art NAS approaches, our proposed hierarchical SHGO-based algorithm enables more than four orders of magnitude speedup (specifically, the execution time of the proposed algorithm is about 0.1 seconds). Finally, our experimental evaluations show that FLASH is easily transferable to different hardware architectures, thus enabling us to do NAS on a Raspberry Pi-3B processor in less than 3 seconds.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Such optimization lies within the domain of hardware-aware neural architecture search, which optimizes the design parameters of the neural network considering hardware feedback261–264, or in some cases, searches for the optimum hardware parameters265,266.”
From Hardware implementation of memristor-based artificial neural networks (2024) · cites this paper as [262]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · Proceedings of IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1993.341010 · Cited by 14,265
Barabási, A.-L., & Bonabeau, E. (2003) · Scientific American · doi:10.1038/scientificamerican0503-60 · Cited by 2,494
1.1 Computing's energy problem (and what we can do about it)
Horowitz, M. (2014) · 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) · doi:10.1109/isscc.2014.6757323 · Cited by 2,069
Neural Architecture Search: A Survey
Elsken, T., Metzen, J. H., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1808.05377 · Cited by 1,763
ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars
Shafiee, A., Nag, A., Muralimanohar, N., Balasubramonian, R., Strachan, J. P., Hu, M., Williams, R. S., & Srikumar, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001139 · Cited by 1,536
Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks
Chiang, W.-L., Liu, X., Si, S., Li, Y., Bengio, S., & Hsieh, C.-J. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330925 · Cited by 1,331
Research works printed but not linked: 32
- Mohamed S. Abdelfattah , Abhinav Mehrotra , Łukasz Dudziak , and Nicholas Donald Lane . 2021 . Zero-cost proxies for lightweight NAS . In International Conference on Learning Representations. Mohamed S. Abdelfattah, Abhinav Mehrotra, Łukasz Dudziak, and Nicholas Donald Lane. 2021. Zero-cost proxies for lightweight NAS. In International Conference on Learning Representations.
- Bowen Baker , Otkrist Gupta , Nikhil Naik , and Ramesh Raskar . 2016. Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 ( 2016 ). Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar. 2016. Designing neural network architectures using reinforcement learning. arXiv preprint arXiv:1611.02167 (2016). · arXiv
- Hadjer Benmeziane et al . 2021. A comprehensive survey on hardware-aware neural architecture search. arXiv preprint arXiv:2101.09336 ( 2021 ). Hadjer Benmeziane et al. 2021. A comprehensive survey on hardware-aware neural architecture search. arXiv preprint arXiv:2101.09336 (2021). · arXiv
- Han Cai , Ligeng Zhu , and Song Han . 2019 . ProxylessNAS: Direct neural architecture search on target task and hardware . In International Conference on Learning Representations. Han Cai, Ligeng Zhu, and Song Han. 2019. ProxylessNAS: Direct neural architecture search on target task and hardware. In International Conference on Learning Representations.
- Thomas Chau , Łukasz Dudziak , Mohamed S. Abdelfattah , Royson Lee , Hyeji Kim , and Nicholas D . Lane . 2020 . BRP-NAS: Prediction-based NAS using GCNs . arXiv preprint arXiv:2007.08668 (2020). Thomas Chau, Łukasz Dudziak, Mohamed S. Abdelfattah, Royson Lee, Hyeji Kim, and Nicholas D. Lane. 2020. BRP-NAS: Prediction-based NAS using GCNs. arXiv preprint arXiv:2007.08668 (2020). · arXiv
- Wuyang Chen , Xinyu Gong , and Zhangyang Wang . 2021 . Neural architecture search on ImageNet in four GPU hours: A theoretically inspired perspective . In International Conference on Learning Representations. Wuyang Chen, Xinyu Gong, and Zhangyang Wang. 2021. Neural architecture search on ImageNet in four GPU hours: A theoretically inspired perspective. In International Conference on Learning Representations.
- Matthieu Courbariaux , Itay Hubara , Daniel Soudry , Ran El-Yaniv , and Yoshua Bengio . 2016. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 ( 2016 ). Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.02830 (2016). · arXiv
- Xuanyi Dong and Yi Yang . 2020. NAS-Bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326 ( 2020 ). Xuanyi Dong and Yi Yang. 2020. NAS-Bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326 (2020). · arXiv
- Boris Grot and Stephen W. Keckler . 2008. Scalable on-chip interconnect topologies . In 2nd Workshop on Chip Multiprocessor Memory Systems and Interconnects. Boris Grot and Stephen W. Keckler. 2008. Scalable on-chip interconnect topologies. In 2nd Workshop on Chip Multiprocessor Memory Systems and Interconnects.
- Song Han , Huizi Mao , and William J . Dally . 2015 . Deep compression: Compressing de ep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015). Song Han, Huizi Mao, and William J. Dally. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015). · arXiv
- Geoffrey Hinton , Oriol Vinyals , and Jeff Dean . 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 ( 2015 ). Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015). · arXiv
- Chi-Hung Hsu et al . 2018 . Monas: Multi-objective neural architecture search using reinforcement learning. arXiv preprint arXiv:1806.10332 (2018). Chi-Hung Hsu et al. 2018. Monas: Multi-objective neural architecture search using reinforcement learning. arXiv preprint arXiv:1806.10332 (2018). · arXiv
- Gokul Krishnan et al . 2021 . Interconnect-centric benchmarking of in-memory acceleration for DNNS . In 2021 China Semiconductor Technology International Conference (CSTIC). IEEE, 1–4. Gokul Krishnan et al. 2021. Interconnect-centric benchmarking of in-memory acceleration for DNNS. In 2021 China Semiconductor Technology International Conference (CSTIC). IEEE, 1–4.
- 10.5555/3437539.3437669 · doi:10.5555/3437539.3437669
- Hanxiao Liu , Karen Simonyan , and Yiming Yang . 2018 . Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018). Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018). · arXiv
- Jovita Lukasik , David Friede , Heiner Stuckenschmidt , and Margret Keuper . 2020. Neural architecture performance prediction using graph neural networks. arXiv preprint arXiv:2010.10024 ( 2020 ). Jovita Lukasik, David Friede, Heiner Stuckenschmidt, and Margret Keuper. 2020. Neural architecture performance prediction using graph neural networks. arXiv preprint arXiv:2010.10024 (2020). · arXiv
- 10.5555/311445 · doi:10.5555/311445
- Volodymyr Mnih et al . 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 ( 2013 ). Volodymyr Mnih et al. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013). · arXiv
- 10.5555/1137830 · doi:10.5555/1137830
- Hieu Pham , Melody Guan , Barret Zoph , Quoc Le , and Jeff Dean . 2018 . Efficient neural architecture search via parameters sharing . In International Conference on Machine Learning. PMLR, 4095–4104 . Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018. Efficient neural architecture search via parameters sharing. In International Conference on Machine Learning. PMLR, 4095–4104.
- 10.5555/3305890.3305981 · doi:10.5555/3305890.3305981
- Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014). · arXiv
- Dimitrios Stamoulis et al . 2019. Single-path NAS: Designing hardware-efficient convnets in less than 4 hours. arXiv preprint arXiv:1904.02877 ( 2019 ). Dimitrios Stamoulis et al. 2019. Single-path NAS: Designing hardware-efficient convnets in less than 4 hours. arXiv preprint arXiv:1904.02877 (2019). · arXiv
- Mingxing Tan et al . 2019 . Mnasnet: Platform-aware neural architecture search for mobile . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2820–2828 . Mingxing Tan et al. 2019. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2820–2828.
- 10.5555/3157096.3157158 · doi:10.5555/3157096.3157158
- Duncan J. Watts and Steven H . Strogatz . 1998 . Collective dynamics of ‘small-world’ networks. Nature 393, 6684 (1998), 440–442. Duncan J. Watts and Steven H. Strogatz. 1998. Collective dynamics of ‘small-world’ networks. Nature 393, 6684 (1998), 440–442.
- Bichen Wu et al . 2019 . Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10734–10742 . Bichen Wu et al. 2019. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10734–10742.
- Sirui Xie , Hehui Zheng , Chunxiao Liu , and Liang Lin . 2019 . SNAS: Stochastic neural architecture search . In International Conference on Learning Representations. Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. 2019. SNAS: Stochastic neural architecture search. In International Conference on Learning Representations.
- Jiahui Yu et al . 2020. BigNAS: Scaling up neural architecture search with big single-stage models . In Computer Vision – ECCV 2020 . 702–717. Jiahui Yu et al. 2020. BigNAS: Scaling up neural architecture search with big single-stage models. In Computer Vision – ECCV 2020. 702–717.
- Sergey Zagoruyko and Nikos Komodakis . 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 ( 2016 ). Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016). · arXiv
- Barret Zoph and Quoc V . Le . 2016 . Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016). Barret Zoph and Quoc V. Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016). · arXiv
- Barret Zoph , Vijay Vasudevan , Jonathon Shlens , and Quoc V. Le . 2018. Learning transferable architectures for scalable image recognition . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8697–8710 . Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le. 2018. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8697–8710.
Cited by
Hardware implementation of memristor-based artificial neural networks
Aguirre, F., Sebastian, A., Le Gallo, M., Song, W., Wang, T., Yang, J. J., Lu, W., Chang, M.-F., Ielmini, D., Yang, Y., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-45670-9 · Cited by 564
Neural Architecture Search Survey: A Hardware Perspective
Chitty-Venkata, K. T., & Somani, A. K. (2023) · ACM Computing Surveys · doi:10.1145/3524500 · Cited by 106
Systematic review on neural architecture search
Avval, S. S. P., Eskue, N., Groves, R. M., & Yaghoubi, V. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11058-w · Cited by 141
Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities
Li, G.-H., Hoang, D.-T., Bhardwaj, K., Lin, M., Wang, Z.-Y., & Mărculescu, R. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3395423 · Cited by 69
Self-citationGeneric Neural Architecture Search via RegressionPreprint
Li, Y., Hao, C., Li, P., Xiong, J., & Chen, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01899 · Cited by 24
AnalogNAS: A Neural Network Design Framework for Accurate Inference with Analog In-Memory Computing
Benmeziane, H., Lammie, C., Boybat, I., Rasch, M., Gallo, M. L., Tsai, H., Muralidhar, R., Niar, S., Hamza, O., Narayanan, V., et al. (2023) · International Conference on Edge Computing [Services Society] · doi:10.1109/edge60047.2023.00045 · Cited by 20
High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System
Hu, X., Huang, H., Li, X., Zheng, X. W., Ren, Q., He, J., & Xiong, X. (2022) · ACM Transactions on Embedded Computing Systems · doi:10.1145/3530818 · Cited by 9
Mazumder, A. N., & Mohsenin, T. (2023) · IEEE Micro · doi:10.1109/mm.2023.3316433 · Cited by 5
CIMNet: Joint Search for Neural Network and Computing-in-Memory Architectures
Chen, X.-J., & Yang, C.-L. (2024) · IEEE Micro · doi:10.1109/mm.2024.3409068 · Cited by 3
Efficient Resource-Aware Neural Architecture Search with a Neuro-Symbolic Approach
Bellodi, E., Bertozzi, D., Bizzarri, A., Favalli, M., Fraccaroli, M., & Zese, R. (2023) · IEEE International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC) · doi:10.1109/mcsoc60832.2023.00034 · Cited by 2
Neural architecture search applying optimal stopping theory
Sheehan, M., & Yakimenko, O. A. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1643088 · Cited by 1
Chen, X.-J., Kuan, C., & Yang, C.-L. (2023) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac56929.2023.10247782 · Cited by 1
Efficient Hardware-aware Neural Architecture Search for Edge Computing
Benmeziane, H. (2023) · theses.fr (ABES) · doi:10.70675/12a3f1afza163z4f13zb3cfz9a3425c3d45d · Cited by 2
Cache and Reuse: Rethinking the Efficiency of On-device Transfer Learning
Yang, Y., Chiang, H.-Y., Li, G., Marculescu, D., & Marculescu, R. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00803
Self-citationCortes, A. I. L., Rodríguez, J. R. G., Luna, C. E. B. Y. G., Villalvazo, A. J. L., Sandoval-Aréchiga, R., & Rodríguez-Abdalá, V. I. (2026) · Applied Sciences · doi:10.3390/app16136436
Metadata
- Authors
- Li, G., Mandal, S. K., Ogras, U. Y., & Marculescu, R.
- Affiliations
- The University of Texas at Austin (US)
- University of Wisconsin–Madison (US)
- The University of Texas at Austin, Austin, Texas, USA
- Venue
- ACM Transactions on Embedded Computing Systems
- Published
- 2021-09-17
- Volume
- 20
- Issue
- 5s
- Pages
- 1-26
- ISSN
- 1539-9087 (ISSN-L)
- 1558-3465
- OpenAlex
- W3200067639
- Cited by
- 15
- References
- 24
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 4 · 2023: 5 · 2022: 2 · 2021: 1
- FWCI
- 1.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Embedded Computing Systems (publishedVersion)
- Funders
- US National Science Foundation (CNS-2007284)
- Semiconductor Research Corporation (GRC 2939.001 and 3012.001)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- neural architecture search, hardware-aware neural network design, load forecasting, hierarchical optimization, skip connections, Raspberry Pi
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.