The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index; 8 of the 22 sources its claims cite were checked for support.
MEMO: Test Time Robustness Via Adaptation and Augmentation
Open accessAbstract
While deep neural networks can attain good accuracy on in-distribution test points, many applications require robustness even in the face of unexpected perturbations in the input, changes in the domain, or other sources of distribution shift. We study the problem of test time robustification, i.e., using the test input to improve model robustness. Recent prior works have proposed methods for test time adaptation, however, they each introduce additional assumptions, such as access to multiple test points, that prevent widespread adoption. In this work, we aim to study and devise methods that make no assumptions about the model training process and are broadly applicable at test time. We propose a simple approach that can be used in any test setting where the model is probabilistic and adaptable: when presented with a test example, perform different data augmentations on the data point, and then adapt (all of) the model parameters by minimizing the entropy of the model's average, or marginal, output distribution across the augmentations. Intuitively, this objective encourages the model to make the same prediction across different augmentations, thus enforcing the invariances encoded in these augmentations, while also maintaining confidence in its predictions. In our experiments, we evaluate two baseline ResNet models, two robust ResNet-50 models, and a robust vision transformer model, and we demonstrate that this approach achieves accuracy gains of 1-8\% over standard model eva…
Overview
CoChat Summary
TL;DR
MEMO is a test-time robustness method that adapts all parameters of a pretrained probabilistic model on a single test point by minimizing the entropy of its marginal output distribution across augmentations. It requires no changes to training and improves accuracy across CIFAR-10 and ImageNet distribution-shift benchmarks, including state-of-the-art single-test-point results on ImageNet-C, ImageNet-R, and among ResNet-50 models on ImageNet-A.
Key findings
- MEMO improves over standard evaluation by 1–8% across evaluated models and generally outperforms prior augmentation and adaptation strategies. On CIFAR-10-C, ResNet-26 average error drops from 22.5% to 19.6% (−2.9), versus 19.9% for TTA and 21.5% for TTT.
- On ImageNet with a baseline ResNet-50, MEMO reduces ImageNet-C mCE from 76.7 to 69.9 (−6.8), ImageNet-R error from 63.9% to 58.8% (−5.1), and ImageNet-A error from 100.0% to 99.1% (−0.9). With DeepAugment+AugMix, MEMO reaches 49.8 mCE on ImageNet-C, 49.2% error on ImageNet-R, and 94.8% error on ImageNet-A.
- For RVT∗-small, MEMO improves ImageNet-C mCE from 49.4 to 40.6 (−8.8), ImageNet-R error from 52.3% to 43.8% (−8.5), and ImageNet-A error from 73.9% to 69.8% (−4.1). On ResNext-101 with WSL pretraining, MEMO reduces ImageNet-A error from 54.9% to 43.2% (−11.7).
- Ablations indicate both invariance across augmentations and confidence are important: MEMO generally outperforms pairwise cross-entropy and conditional-entropy adaptation objectives. Small numbers of augmentations such as B=4 or 8 already provide significant gains.
Methods
- Test-time procedure: given a trained differentiable probabilistic model and one test point, sample B augmentations (AugMix by default), compute the marginal predictive distribution across augmented copies, take one gradient step to minimize its entropy, then predict on the original test point.
- Evaluated on CIFAR-10, CIFAR-10.1, CIFAR-10-C, ImageNet-C, ImageNet-R, and ImageNet-A using ResNet-26, ResNet-50, RVT∗-small, and ResNext-101 models, including robust pretrained models.
- Compared against TTA, single-point BN adaptation, BN adaptation with multiple test points, Tent online/episodic, and TTT where applicable; hyperparameters selected on corruption validation sets.
Limitations
- The paper states that MEMO inference is more computationally expensive than standard inference due to augmentation and adaptation, though smaller B values offer more favorable efficiency–accuracy tradeoffs.
- The paper states that in preliminary continual-adaptation experiments, MEMO tended to lead to degenerate solutions, such as predicting a constant label with maximal confidence; possible remedies include adapting only selected parameters or regularizing toward the pretrained model.
- Evident from the text: MEMO assumes access to a differentiable, adaptable probabilistic model and a set of augmentation functions, and it does not assume ground-truth labels or multiple test points.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
8 Valid · 2 Not valid · 7 Not checked · 68 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“MEMO consistently outperforms non adaptive marginal distribution predictions (between 1-10% improvement) on the ImageNet-C [12] and ImageNet-R [14] test sets”
What the sources say
- Not matchedcites [12] [12] D. Hendrycks and T. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations (ICLR) , 2019. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [14] [14] D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, D. Song, J. Steinhardt, and J. Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. In IEEE International Conference on Computer Vision (ICCV) , 2021. - source
The citation is attached to “ImageNet-R,” identifying the test set rather than sourcing the MEMO comparison. Section 3.1 says the authors created the ImageNet-R dataset; this citation does not establish the stated 1–10% improvement. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“Also, MEMO is, to the best of our knowledge, the first adaptation method to improve performance (by 1-4% over standard model evaluation) on the ImageNet-A test set [15].”
What the sources say
- Supportedcites [15] [15] D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song. Natural adversarial examples. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2021. - source
The citation follows “ImageNet-A test set,” and the paper identifies ImageNet-A as a dataset it curated (Introduction; §3.1, Design). It serves as a citation for the test set, not for the citing authors’ claim about MEMO’s improvement. (LLM, conf=0.92)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As Table 2 and prior work both show [40, 46], accessing multiple test points can be powerful for benchmarks such as ImageNet-C and ImageNet-R, in which inferred statistics from the test input distribution may aid in prediction.”
What the sources say
- Supportedcites [40] [40] S. Schneider, E. Rusak, L. Eck, O. Bringmann, W. Brendel, and M. Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems (NeurIPS) , 2020. - source
The Abstract says multiple unlabeled examples can be used to estimate statistics from corrupted images and improve robustness on ImageNet-C. The Results report improved ImageNet-C performance with batches of test samples, and the section discussing ImageNet-R reports lower error after adaptation, including when statistics are calculated from more than 32 samples. (LLM, conf=0.93)
- Not matchedcites [46] [46] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations (ICLR) , 2021. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Thus, the ResNet-26 model we use for our method closely follows the modifications that Sun et al. [44] propose, in order to provide a fair point of comparison.”
What the sources say
- Supportedcites [44] [44] Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt. Test-time training with selfsupervision for generalization under distribution shifts. In International Conference on Machine Learning (ICML) , 2020. - source
Appendix A4.5 (p. 16) of our copy explicitly lists the architectural modifications Sun et al. propose relative to Hendrycks et al.: Group Normalization instead of Batch Normalization, splitting after the second residual group, and using a ResNet-26 instead of a 40-2 Wide ResNet. Section 3 'Network details' (p. 2-3) confirms the ResNet-26 for CIFAR-10 with GN and the split point at the end of the second group. The citation is therefore an identification of the source that proposed those ResNet-26 modifications, which our copy supports. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Crucially, we find that, in practice, the set of test augmentations A does not have to match the augmentations that were used to train the model.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
34 more claims
Create a free account to see all 85 claims and to verify the 6 references still open.
- It may state two separate claims as one. 37
- Probably not a substantial claim. 13
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Experimental Protocol · p. 15
“- Weight decay: 0 . 0 , 0 . 1 .”
§ A Experimental Protocol · p. 15
“Based on the ImageNet-C validation sets, we use AdamW [27] as the update rule G , with learning rate η = 0 . 00001 and weight decay 0 . 01 , and B = 64 .”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 1 Introduction · p. 2
“MEMO consistently outperforms non adaptive marginal distribution predictions (between 1-10% improvement) on the ImageNet-C [12] and ImageNet-R [14] test sets”
§ 4.1 Main Results · p. 7
“| + TTA | 53 . 0 (+3 . 6) | 49 . 0 ( - 3 . 3) | 68 . 9 ( - 5 . 0 ) | | + Single point BN | 48 . 0 ( - 1 . 4) | 51 . 1 ( - 1 . 2) | 74 . 4 (+0 . 5) | | + MEMO(ours) | 40 . 6 ( - 8 . 8 ) | 43 . 8 ( - 8 . 5 ) | 69 . 8 ( - 4 . 1) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3.2 Composing MEMO with Prior Methods · p. 5
“We find in our experiments that this technique never degrades, and generally improves, the performance of test time adaptation, thus we combine MEMO with this technique by default whenever applicable.”
§ 4.1 Main Results · p. 8
“WSL model, we did not use single point BN adaptation for MEMO as we found this technique to be actually harmful to performance, and this corroborates previous findings [40].”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.1 Main Results · p. 6
“Across all corruption types in CIFAR-10-C, MEMO consistently improves test error compared to the baselines, non adaptive TTA, and TTT.”
§ C Full CIFAR-10-C and ImageNet-C Results · p. 17
“| + TTA | 43 . 4 | 39 . 6 | 42 . 9 | 28 . 3 | 44 . 7 | 26 . 3 | 26 . 3 | 21 . 4 | 28 . 5 | 23 . 3 | 12 . 1 | 32 . 9 | 21 . 7 | 43 . 2 | 21 . 7 | | + MEMO(ours) | 43 . 5 | 39 . 8 | 43 . 3 | 26 . 4 | 44 . 4 | 25 . 1 | 25 . 0 | 20 . 9 | 28 . 3 | 22 . 8 | 11 . 9 | 28 . 3 | 21 . 1 | 42 . 8 | 21 . 7 |”
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 9
- [5][5] G. Blanchard, G. Lee, and C. Scott. Generalizing from several related classification tasks to a new unlabeled sample. In Advances in Neural Information Processing Systems (NIPS) , 2011. -
- [8][8] Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems (NIPS) , 2005. -
- [16][16] W. Hu, G. Niu, I. Sato, and M. Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning (ICML) , 2018. -
- [18][18] Y. Iwasawa and Y. Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems (NeurIPS) , 2021. -
- [26][26] Y. Liu, P. Kothari, B. van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi. TTT++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems (NeurIPS) , 2021. -
- [27][27] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR) , 2019. -
- [36][36] J. Quiñonero Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence. Dataset Shift in Machine Learning . The MIT Press, 2009. -
- [41][41] H. Shimodaira. Improving predictive inference under covariate shift by weighting the loglikelihood function. Journal of Statistical Planning and Inference (JSPI) , 2000. -
- [46][46] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations (ICLR) , 2021. -
Cited by
Domain Generalization: A Survey
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., & Loy, C. C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3195549 · Cited by 1,315
Continual Test-Time Domain Adaptation
Wang, Q., Fink, O., Van Gool, L., & Dai, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00706 · Cited by 502
A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
Liang, J., He, R., & Tan, T. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02181-w · Cited by 271
Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-Time Adaptation
Tang, Y., Chen, S., Kan, Z., Zhang, Y., Guo, Q., & He, Z. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3443633 · Cited by 7
“C S is the short-term instance-specific token that captures the local variations of individual image instances.”
Zhang, J., Luo, X., Gao, L., Zou, D., Shen, H. T., & Song, J. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3674435 · Cited by 23
“Finally, data augmentation strategies focus on modifying and rectifying the distribution of input data; representative methods include enforcing consistency over augmentations [59], [60], [61], or optimizing input prompts for vision-language models [62].”
Metadata
- Authors
- Zhang, M., Levine, S., & Finn, C.
- Venue
- Advances in Neural Information Processing Systems 35
- Published
- 2022
- Pages
- 38629-38642
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W7133206185
- Cited by
- 189
- References
- 0
- Citations by year
- 2026: 58 · 2025: 40 · 2024: 52 · 2023: 33 · 2022: 6
- FWCI
- 24.69
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Canadian Institute for Advanced Research
- National Defense Science and Engineering Graduate
- Army Research Office (W911NF-21-1-, W911NF-17, W911NF-21-1-0097, W911NF, W911NF-17-2-0181)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Hardware and Architecture › VLSI and Analog Circuit Testing
- Keywords
- domain adaptation, data augmentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MEMO: Test Time Robustness Via Adaptation and Augmentation. https://app.cochat.ai/papers/10.52202/068431-2799
- The paper
- Zhang, M., Levine, S., & Finn, C. (2022). MEMO: Test Time Robustness Via Adaptation and Augmentation. Advances in Neural Information Processing Systems 35. https://doi.org/10.52202/068431-2799
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.