Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index; 8 of the 22 sources its claims cite were checked for support.

MEMO: Test Time Robustness Via Adaptation and Augmentation

Open access
Zhang, Marvin, Levine, Sergey, Finn, Chelsea · 2022 · Advances in Neural Information Processing Systems 35

Abstract

While deep neural networks can attain good accuracy on in-distribution test points, many applications require robustness even in the face of unexpected perturbations in the input, changes in the domain, or other sources of distribution shift. We study the problem of test time robustification, i.e., using the test input to improve model robustness. Recent prior works have proposed methods for test time adaptation, however, they each introduce additional assumptions, such as access to multiple test points, that prevent widespread adoption. In this work, we aim to study and devise methods that make no assumptions about the model training process and are broadly applicable at test time. We propose a simple approach that can be used in any test setting where the model is probabilistic and adaptable: when presented with a test example, perform different data augmentations on the data point, and then adapt (all of) the model parameters by minimizing the entropy of the model's average, or marginal, output distribution across the augmentations. Intuitively, this objective encourages the model to make the same prediction across different augmentations, thus enforcing the invariances encoded in these augmentations, while also maintaining confidence in its predictions. In our experiments, we evaluate two baseline ResNet models, two robust ResNet-50 models, and a robust vision transformer model, and we demonstrate that this approach achieves accuracy gains of 1-8\% over standard model eva…

Overview

CoChat Summary

TL;DR

MEMO is a test-time robustness method that adapts all parameters of a pretrained probabilistic model on a single test point by minimizing the entropy of its marginal output distribution across augmentations. It requires no changes to training and improves accuracy across CIFAR-10 and ImageNet distribution-shift benchmarks, including state-of-the-art single-test-point results on ImageNet-C, ImageNet-R, and among ResNet-50 models on ImageNet-A.

Key findings

  • MEMO improves over standard evaluation by 1–8% across evaluated models and generally outperforms prior augmentation and adaptation strategies. On CIFAR-10-C, ResNet-26 average error drops from 22.5% to 19.6% (−2.9), versus 19.9% for TTA and 21.5% for TTT.
  • On ImageNet with a baseline ResNet-50, MEMO reduces ImageNet-C mCE from 76.7 to 69.9 (−6.8), ImageNet-R error from 63.9% to 58.8% (−5.1), and ImageNet-A error from 100.0% to 99.1% (−0.9). With DeepAugment+AugMix, MEMO reaches 49.8 mCE on ImageNet-C, 49.2% error on ImageNet-R, and 94.8% error on ImageNet-A.
  • For RVT∗-small, MEMO improves ImageNet-C mCE from 49.4 to 40.6 (−8.8), ImageNet-R error from 52.3% to 43.8% (−8.5), and ImageNet-A error from 73.9% to 69.8% (−4.1). On ResNext-101 with WSL pretraining, MEMO reduces ImageNet-A error from 54.9% to 43.2% (−11.7).
  • Ablations indicate both invariance across augmentations and confidence are important: MEMO generally outperforms pairwise cross-entropy and conditional-entropy adaptation objectives. Small numbers of augmentations such as B=4 or 8 already provide significant gains.

Methods

  • Test-time procedure: given a trained differentiable probabilistic model and one test point, sample B augmentations (AugMix by default), compute the marginal predictive distribution across augmented copies, take one gradient step to minimize its entropy, then predict on the original test point.
  • Evaluated on CIFAR-10, CIFAR-10.1, CIFAR-10-C, ImageNet-C, ImageNet-R, and ImageNet-A using ResNet-26, ResNet-50, RVT∗-small, and ResNext-101 models, including robust pretrained models.
  • Compared against TTA, single-point BN adaptation, BN adaptation with multiple test points, Tent online/episodic, and TTT where applicable; hyperparameters selected on corruption validation sets.

Limitations

  • The paper states that MEMO inference is more computationally expensive than standard inference due to augmentation and adaptation, though smaller B values offer more favorable efficiency–accuracy tradeoffs.
  • The paper states that in preliminary continual-adaptation experiments, MEMO tended to lead to degenerate solutions, such as predicting a constant label with maximal confidence; possible remedies include adapting only selected parameters or regularizing toward the pretrained model.
  • Evident from the text: MEMO assumes access to a differentiable, adaptable probabilistic model and a set of augmentation functions, and it does not assume ground-truth labels or multiple test points.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.52202/068431-2799

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 2 Not valid · 7 Not checked · 68 Own results and interpretations

34 more claims

Create a free account to see all 85 claims and to verify the 6 references still open.

See all claims — free account
  • It may state two separate claims as one. 37
  • Probably not a substantial claim. 13
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 9

  1. [5][5] G. Blanchard, G. Lee, and C. Scott. Generalizing from several related classification tasks to a new unlabeled sample. In Advances in Neural Information Processing Systems (NIPS) , 2011. -
  2. [8][8] Y. Grandvalet and Y. Bengio. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems (NIPS) , 2005. -
  3. [16][16] W. Hu, G. Niu, I. Sato, and M. Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning (ICML) , 2018. -
  4. [18][18] Y. Iwasawa and Y. Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems (NeurIPS) , 2021. -
  5. [26][26] Y. Liu, P. Kothari, B. van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi. TTT++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems (NeurIPS) , 2021. -
  6. [27][27] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR) , 2019. -
  7. [36][36] J. Quiñonero Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence. Dataset Shift in Machine Learning . The MIT Press, 2009. -
  8. [41][41] H. Shimodaira. Improving predictive inference under covariate shift by weighting the loglikelihood function. Journal of Statistical Planning and Inference (JSPI) , 2000. -
  9. [46][46] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations (ICLR) , 2021. -

Cited by

See all 189

Metadata

Authors
Zhang, M., Levine, S., & Finn, C.
Venue
Advances in Neural Information Processing Systems 35
Published
2022
Pages
38629-38642
ISSN
1049-5258 (ISSN-L)
OpenAlex
W7133206185
Cited by
189
References
0
Citations by year
2026: 58 · 2025: 40 · 2024: 52 · 2023: 33 · 2022: 6
FWCI
24.69
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Canadian Institute for Advanced Research
  • National Defense Science and Engineering Graduate
  • Army Research Office (W911NF-21-1-, W911NF-17, W911NF-21-1-0097, W911NF, W911NF-17-2-0181)
Topics
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Hardware and Architecture › VLSI and Analog Circuit Testing
Keywords
domain adaptation, data augmentation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: MEMO: Test Time Robustness Via Adaptation and Augmentation. https://app.cochat.ai/papers/10.52202/068431-2799
The paper
Zhang, M., Levine, S., & Finn, C. (2022). MEMO: Test Time Robustness Via Adaptation and Augmentation. Advances in Neural Information Processing Systems 35. https://doi.org/10.52202/068431-2799

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.