The DOI matches its OpenAlex record; 37 of the 37 references it cites were found in a registry or an index.
Revisiting ResNets: Improved Training and Scaling Strategies
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Self-citationFeature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Self-citationMobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Sheer scale was exhaustively demonstrated to improve performance of neural language models (Kaplan et al., 2020) which motivated the design of ever larger models including GPT-3 (Brown et al., 2020) and Switch Transformer (Fedus et al., 2021).”
MethodThe pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · cited 20×
“Automated search strategies for designing architectures have further pushed the state-of-the-art, notably with NasNet-A (Zoph et al., 2018), AmoebaNet-A (Real et al., 2019) and EfficientNet (Tan & Le, 2019).”
BackgroundMethodInfluentialSqueeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Cited by
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00721 · Cited by 13,198
YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2022) · arXiv · doi:10.48550/arxiv.2207.02696 · Cited by 13,205
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
MLP-Mixer: An all-MLP Architecture for Vision
Tolstikhin, I., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.01601 · Cited by 2,087
“[4], who show that canonical ResNets are still near state-of-the-art, if one uses recent training and augmentation strategies.”
BackgroundBottleneck Transformers for Visual Recognition
Srinivas, A., Lin, T.-Y., Parmar, N., Shlens, J., Abbeel, P., & Vaswani, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01625 · Cited by 1,385
Self-citationThe Medical Segmentation Decathlon
Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., Ronneberger, O., Summers, R. M., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-30695-9 · Cited by 1,384
“Note that similar findings, albeit in a different context and applied to ResNet, have been recently observed [36].”
ResultEfficientNetV2: Smaller Models and Faster Training
Tan, M., & Le, Q. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00298 · Cited by 1,098
(2022) · doi:10.1007/978-3-031-20053-3_27 · Cited by 1,060
Zhai, X., Kolesnikov, A., Houlsby, N., & Beyer, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01179 · Cited by 891
ResMLP: Feedforward networks for image classification with data-efficient training
Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.03404 · Cited by 807
ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training
Touvron, H., Bojanowski, P., Caron, M., Cord, M., El-Nouby, A., Grave, E., Izacard, G., Joulin, A., Synnaeve, G., Verbeek, J., et al. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3206148 · Cited by 807
“This is in line with recent work pointing out the importance of the training strategy over the model choice [3, 48].”
ResultContextual Transformer Networks for Visual Recognition
Li, Y., Yao, T., Pan, Y., & Mei, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3164083 · Cited by 749
MobileNetV4: Universal Models for the Mobile Ecosystem
Qin, D., Leichner, C., Delakis, M., Fornoni, M., Luo, S., Yang, F., Wang, W., Banbury, C., Ye, C., Akin, B., et al. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73661-2_5 · Cited by 663
How Do Vision Transformers Work?
Park, N., & Kim, S. (2022) · arXiv · doi:10.48550/arxiv.2202.06709 · Cited by 356
Simple Open-Vocabulary Object Detection
Minderer, M., Gritsenko, A., Stone, A., Neumann, M., Weissenborn, D., Dosovitskiy, A., Mahendran, A., Arnab, A., Dehghani, M., Shen, Z., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20080-9_42 · Cited by 348
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers
Steiner, A., Kolesnikov, A., Zhai, X., Wightman, R., Uszkoreit, J., & Beyer, L. (2021) · arXiv · doi:10.48550/arxiv.2106.10270 · Cited by 292
Residual Local Feature Network for Efficient Super-Resolution
Kong, F., Li, M., Liu, S., Liu, D., He, J., Bai, Y., Chen, F., & Fu, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00092 · Cited by 285
Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging
Azizi, S., Culp, L., Freyberg, J., Mustafa, B., Baur, S., Kornblith, S., Chen, T., Tomasev, N., Mitrović, J., Strachan, P., et al. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01049-7 · Cited by 250
TransCNN: Hybrid CNN and transformer mechanism for surveillance anomaly detection
Ullah, W., Hussain, T., Ullah, F. U. M., Lee, M. Y., & Baik, S. W. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106173 · Cited by 128
When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations
Chen, X., Hsieh, C.-J., & Gong, B. (2021) · arXiv · doi:10.48550/arxiv.2106.01548 · Cited by 128
“The performance of an architecture is often conflated with the training strategies (Bello et al., 2021), where data augmentations play a key role (Cubuk et al., 2019; 2020; Zhang et al., 2018; Xie et al., 2020; Chen et al., 2021b).”
MethodBenchmarking and scaling of deep learning models for land cover image classification
Papoutsis, I., Bountos, N. I., Zavras, A., Michail, D., & Tryfonopoulos, C. (2023) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2022.11.012 · Cited by 86
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Tay, Y., Dehghani, M., Rao, J., Fedus, W., Abnar, S., Chung, H. W., Narang, S., Yogatama, D., Vaswani, A., & Metzler, D. (2021) · arXiv · doi:10.48550/arxiv.2109.10686 · Cited by 63
“While there have been many proposed scaling protocols for ConvNets (Tan and Le, 2019; Bello et al., 2021), there is still limited advice on scaling of transformer architectures, apart from (Kaplan et al.”
BackgroundResultInfluentialSelf-citationDehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv · doi:10.48550/arxiv.2107.07002 · Cited by 48
Engelmann, J., McTrusty, A. D., MacCormick, I. J. C., Pead, E., Storkey, A., & Bernabeu, M. O. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00566-5 · Cited by 37
Metadata
- Authors
- Bello, I., Fedus, W., Du, X., Cubuk, E. D., Srinivas, A., Lin, T.-Y., Shlens, J., & Zoph, B.
- Venue
- arXiv (Cornell University)
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2103.07579
- OpenAlex
- W3138994021
- Cited by
- 199
- References
- 37
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.