Algorithmic progress in computer vision
Open accessAbstract
We investigate algorithmic progress in image classification on ImageNet, perhaps the most well-known test bed for computer vision. We estimate a model, informed by work on neural scaling laws, and infer a decomposition of progress into the scaling of compute, data, and algorithms. Using Shapley values to attribute performance improvements, we find that algorithmic improvements have been roughly as important as the scaling of compute for progress computer vision. Our estimates indicate that algorithmic innovations mostly take the form of compute-augmenting algorithmic advances (which enable researchers to get better performance from less compute), not data-augmenting algorithmic advances. We find that compute-augmenting algorithmic advances are made at a pace more than twice as fast as the rate usually associated with Moore's law. In particular, we estimate that compute-augmenting innovations halve compute requirements every nine months (95\% confidence interval: 4 to 25 months).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Algorithmic improvements have historically decreased the compute requirements for a fixed performance level and can substitute for compute limitations.
“However, algorithmic improvements, which have historically decreased the compute requirements for a fixed performance level [48] [49], can substitute for compute limitations.”
From Measuring AI Ability to Complete Long Software Tasks (2025) · cites this paper as [48]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 14
- Deng, Jia et al. (2009). 'Imagenet: A large-scale hierarchical image database'. In: 2009 IEEE conference on computer vision and pattern recognition . Ieee, pp. 248-255.
- Fichte, Johannes K, Markus Hecher, and Stefan Szeider (2020). 'A time leap challenge for SAT-solving'. In: International Conference on Principles and Practice of Constraint Programming . Springer, pp. 267-285.
- Hart, Sergiu (1989). 'Shapley value'. In: Game theory . Springer, pp. 210-216.
- Hernandez, Danny and Tom B Brown (2020). 'Measuring the algorithmic efficiency of neural networks'. In: arXiv preprint arXiv:2005.04305 . · arXiv
- Hoffmann, Jordan et al. (2022). 'Training Compute-Optimal Large Language Models'. In: arXiv preprint arXiv:2203.15556 . · arXiv
- Jiang, Zi-Hang et al. (2021). 'All tokens matter: Token labeling for training better vision transformers'. In: Advances in Neural Information Processing Systems 34, pp. 18590-18602.
- Koch, Thorsten et al. (2022). 'Progress in mathematical programming solvers from 2001 to 2020'. In: EURO Journal on Computational Optimization , p. 100031.
- Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E Hinton (2012). 'Imagenet classification with deep convolutional neural networks'. In: Advances in neural information processing systems 25.
- LeCun, Yann et al. (1989). 'Backpropagation applied to handwritten zip code recognition'. In: Neural computation 1.4, pp. 541-551.
- Neusel, Mara D and Larry Smith (2002). Invariant theory of finite groups . Vol. 94. American Mathematical Society Providence, RI.
- Rasley, Jeff et al. (2020). 'Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters'. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pp. 3505-3506.
- Sherry, Yash and Neil C Thompson (2021). 'How Fast Do Algorithms Improve?[Point of View]'. In: Proceedings of the IEEE 109.11, pp. 1768-1777.
- Thompson, Neil C et al. (2020). 'The computational limits of deep learning'. In: arXiv preprint arXiv:2007.05558 . · arXiv
- Tillet, Philippe, Hsiang-Tsung Kung, and David Cox (2019). 'Triton: an intermediate language and compiler for tiled neural network computations'. In: Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages , pp. 10-19.
Cited by
Sorted by citations
Seger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., Ó hÉigeartaigh, S., Korinek, A., et al. (2023) · doi:10.2139/ssrn.4596436 · Cited by 31 · see citing works
Measuring AI Ability to Complete Long Software Tasks
Kwa, T., West, B., Becker, J., Deng, A., Garcia, K., Hasin, M., Jawhar, S., Kinniment, M., Rush, N., Von Arx, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.14499 · Cited by 14 · see citing works
Jones, C. R., & Bergen, B. K. (2024) · arXiv · doi:10.48550/arxiv.2412.17128 · Cited by 7 · see citing works
- Authors
- Erdil, E., & Besiroglu, T.
- Affiliations
- Massachusetts Institute of Technology (US)
- ORCID
- Besiroglu, Tamay
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2212.05153
- OpenAlex
- W4311407484
- Cited by
- 7
- References
- 0
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 3 · 2023: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- image classification, ImageNet, neural scaling laws, Shapley value, Moore's law, deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
