What Do Compressed Deep Neural Networks Forget?
Open accessThe DOI matches its OpenAlex record; 33 of the 33 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 1×
“…[12, 35, 22], efforts to start with a network that is more compact with fewer parameters, layers or computations (architecture design) [34, 37, 44] and student networks with fewer parameters that learn from a larger teacher model (model distillation) [32] and finally pruning by setting a…”
BackgroundTHE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED
WELCH, B. L. (1947) · Biometrika · doi:10.1093/biomet/34.1-2.28 · Cited by 4,135
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00286 · Cited by 4,114
Energy and Policy Considerations for Deep Learning in NLP
Strubell, E., Ganesh, A., & McCallum, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1355 · Cited by 3,684 · cited 2×
“Networks A “bigger is better” race in the number of model parameters has gripped the field of machine learning (Canziani et al., 2016; Strubell et al., 2019).”
BackgroundSimple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · cited 2×
“However, recent work has highlighted that deep neural networks produce output probabilities that are uncalibrated [20, 40, 45] and thus cannot be interpreted as a measure of certainty.”
Background
Cited by
Hoefler, T., Alistarh, D., Ben-Nun, T., Dryden, N., & Peste, A. (2021) · ETH Zurich · doi:10.3929/ethz-b-000519202 · Cited by 423
Pervasive label errors in test sets destabilize machine learning benchmarks
Northcutt, C. G., Athalye, A., & Mueller, J. (2021) · arXiv · doi:10.48550/arxiv.2103.14749 · Cited by 293
“in the ImageNet dataset, 29 are well-known to contain errors [16, 31, 38], labeled data in test sets is often considered “correct” as 30 long as it is drawn from the same distribution as the train set — this is a fallacy — machine learning 31 test sets can, and do, contain pervasive errors and these errors can destabi…”
BackgroundMethodDoes learning require memorization? a short tale about a long tail
Feldman, V. (2020) · Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing · doi:10.1145/3357713.3384290 · Cited by 287
Hooker, S. (2021) · Communications of the ACM · doi:10.1145/3467017 · Cited by 211
Self-citationHooker, S. (2020) · arXiv · doi:10.48550/arxiv.2009.06489 · Cited by 185
Self-citation
Metadata
- Authors
- Hooker, S., Courville, A. C., Clark, G., Dauphin, Y., & Frome, A.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2019-11-13
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1911.05248
- OpenAlex
- W3042879175
- Cited by
- 84
- References
- 33
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What Do Compressed Deep Neural Networks Forget? https://app.cochat.ai/papers/10.48550/arxiv.1911.05248
- The paper
- Hooker, S., Courville, A. C., Clark, G., Dauphin, Y., & Frome, A. (2019). What Do Compressed Deep Neural Networks Forget? arXiv (Cornell University). https://doi.org/10.48550/arxiv.1911.05248
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.