The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index.
How to Evaluate Deep Neural Network Processors: TOPS/W (Alone) Considered Harmful
Open accessAbstract
A significant amount of specialized hardware has been developed for processing deep neural networks (DNNs) in both academia and industry. This article aims to highlight the key concepts required to evaluate and compare these DNN processors. We discuss existing challenges, such as the flexibility and scalability needed to support a wide range of neural networks, as well as design considerations for both the DNN processors and the DNN models themselves. We also describe specific metrics that can be used to evaluate and compare existing solutions beyond the commonly used tera-operations per second per watt (TOPS/W). This article is based on the tutorial "How to Understand and Evaluate Deep Learning Processors" that was given at the 2020 International Solid-State Circuits Conference, as well as excerpts from the book, Efficient Processing of Deep Neural Networks [36].
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The energy consumption of executing a DNN model is highly dictated by the computation architecture and methods (e.g., ultra-low power compute-in-memory AI accelerator) at the edge computation nodes [249], the architecture of DNN models [250], and the wireless transmission for data exchange during the model inference procedure.”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [249]
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
Self-citationWilliams, S., Waterman, A., & Patterson, D. A. (2009) · Communications of the ACM · doi:10.1145/1498765.1498785 · Cited by 2,798
A Proof for the Queuing Formula: L = λ W
Little, J. D. C. (1961) · Operations Research · doi:10.1287/opre.9.3.383 · Cited by 2,777
The Case for Energy-Proportional Computing
Barroso, L. A., & Hölzle, U. (2007) · Computer · doi:10.1109/mc.2007.443 · Cited by 2,491
1.1 Computing's energy problem (and what we can do about it)
Horowitz, M. (2014) · 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC) · doi:10.1109/isscc.2014.6757323 · Cited by 2,069
EIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937
Eyeriss: a spatial architecture for energy-efficient dataflow for convolutional neural networks
Chen, Y.-H., Emer, J., & Sze, V. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001177 · Cited by 1,182
Self-citation
Research works printed but not linked: 5
- 0, NVIDIA NVLink High-speed GPU interconnect
- 2013, High-bandwidth Memory (HBM) DRAM
- lecun, 0, The MNIST Database of Handwritten Digits
- 0, Deepbench
- chen, 20, Introducing the CVPR 2018 On-device Visual Intelligence Challenge Google AI blog
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“…a DNN model is highly dictated by the computation architecture and methods (e.g., ultra-low power compute-in-memory AI accelerator) at the edge computation nodes [249], the architecture of DNN models [250], and the wireless transmission for data exchange during the model inference procedure.”
MethodAn optical neural network using less than 1 photon per multiplication
Wang, T., Ma, S.-Y., Wright, L. G., Onodera, T., Richard, B. C., & McMahon, P. L. (2022) · Nature Communications · doi:10.1038/s41467-021-27774-8 · Cited by 268
Single chip photonic deep neural network with accelerated training
Bandyopadhyay, S., Sludds, A., Krastanov, S., Hamerly, R., Harris, N., Bunandar, D., Streshinsky, M., Hochberg, M., & Englund, D. (2022) · arXiv · doi:10.48550/arxiv.2208.01623 · Cited by 265
Lightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y.-H., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 259
Single-chip photonic deep neural network with forward-only training
Bandyopadhyay, S., Sludds, A., Krastanov, S., Hamerly, R., Harris, N., Bunandar, D., Streshinsky, M., Hochberg, M., & Englund, D. (2024) · Nature Photonics · doi:10.1038/s41566-024-01567-z · Cited by 201
Metadata
- Authors
- Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S.
- Affiliations
- Massachusetts Institute of Technology (US)
- National Taiwan University (TW)
- Venue
- IEEE Solid-State Circuits Magazine
- Published
- 2020
- Volume
- 12
- Issue
- 3
- Pages
- 28-41
- ISSN
- 1943-0582 (ISSN-L)
- 1943-0590
- OpenAlex
- W3080976800
- Cited by
- 80
- References
- 34
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 18 · 2023: 16 · 2022: 11 · 2021: 17 · 2020: 1
- FWCI
- 4.38
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- IEEE Solid-State Circuits Magazine (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- Funders
- Defense Advanced Research Projects Agency (HR0011-18-3-0007, NSF E2CDA 1639921)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Keywords
- energy efficiency metrics, processor evaluation, scalability, hardware flexibility, special-purpose hardware
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to Evaluate Deep Neural Network Processors: TOPS/W (Alone) Considered Harmful. https://app.cochat.ai/papers/10.1109/mssc.2020.3002140
- The paper
- Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2020). How to Evaluate Deep Neural Network Processors: TOPS/W (Alone) Considered Harmful. IEEE Solid-State Circuits Magazine. https://doi.org/10.1109/mssc.2020.3002140
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.