The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Continual Test-Time Domain Adaptation
Open accessAbstract
Test-time domain adaptation aims to adapt a source pre-trained model to a target domain without using any source data. Existing works mainly consider the case where the target domain is static. However, real-world machine perception systems are running in non-stationary and continually changing environments where the target domain distribution can change over time. Existing methods, which are mostly based on self-training and entropy regularization, can suffer from these non-stationary environments. Due to the distribution shift over time in the target domain, pseudo-labels become unreliable. The noisy pseudo-labels can further lead to error accumulation and catastrophic forgetting. To tackle these issues, we propose a continual test-time adaptation approach (CoTTA) which comprises two parts. Firstly, we propose to reduce the error accumulation by using weight-averaged and augmentation-averaged predictions which are often more accurate. On the other hand, to avoid catastrophic forgetting, we propose to stochastically restore a small part of the neurons to the source pre-trained weights during each iteration to help preserve source knowledge in the longterm. The proposed method enables the longterm adaptation for all parameters in the network. CoTTA is easy to implement and can be readily incorporated in off-the-shelf pre-trained models. We demonstrate the effectiveness of our approach on four classification tasks and a segmentation task for continual test-time adaptation, on which we outperform existing methods. Our code is available at https://gin.ee/cotta.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
A survey on Image Data Augmentation for Deep Learning
Shorten, C., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0197-0 · Cited by 13,203
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Zagoruyko, S., & Komodakis, N. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.87 · Cited by 7,244
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., & Luo, P. (2021) · arXiv · doi:10.48550/arxiv.2105.15203 · Cited by 5,090
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753
Learning Transferable Features with Deep Adaptation Networks
Long, M., Cao, Y., Wang, J., & Jordan, M. I. (2015) · arXiv · doi:10.48550/arxiv.1502.02791 · Cited by 4,505
Tarvainen, A., & Valpola, H. (2017) · arXiv · doi:10.48550/arxiv.1703.01780 · Cited by 4,275
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., & Abbeel, P. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8202133 · Cited by 3,124 · cited 1×
“Domain randomization is one of the most popular methods which improves the model generalizability by learning from different synthesis parameters of simulation environments [56, 57].”
BackgroundContinual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117
AutoAugment: Learning Augmentation Strategies From Data
Cubuk, E. D., Zoph, B., Mané, D., Vasudevan, V., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00020 · Cited by 2,733
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., Efros, A. A., & Darrell, T. (2017) · arXiv · doi:10.48550/arxiv.1711.03213 · Cited by 2,656
Unsupervised Domain Adaptation by Backpropagation
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 2,521
Acceleration of Stochastic Approximation by Averaging
Polyak, B. T., & Juditsky, A. B. (1992) · SIAM Journal on Control and Optimization · doi:10.1137/0330046 · Cited by 1,827
Continual Learning Through Synaptic Intelligence
Zenke, F., Poole, B., & Ganguli, S. (2017) · arXiv · doi:10.48550/arxiv.1703.04200 · Cited by 1,784
A continual learning survey: Defying forgetting in classification tasks
De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1909.08383 · Cited by 1,647
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-training
Zou, Y., Yu, Z., Vijaya Kumar, B. V. K., & Wang, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_18 · Cited by 1,464
Learning to Adapt Structured Output Space for Semantic Segmentation
Tsai, Y.-H., Hung, W.-C., Schulter, S., Sohn, K., Yang, M.-H., & Chandraker, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00780 · Cited by 1,430
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Hendrycks, D., & Dietterich, T. (2019) · arXiv · doi:10.48550/arxiv.1903.12261 · Cited by 1,335
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00823 · Cited by 1,250
FDA: Fourier Domain Adaptation for Semantic Segmentation
Yang, Y., & Soatto, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00414 · Cited by 1,119
Visual Domain Adaptation: A survey of recent advances
Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2347059 · Cited by 899
Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization
Tremblay, J., Prakash, A., Acuna, D., Brophy, M., Jampani, V., Anil, C., To, T., Cameracci, E., Boochoon, S., & Birchfield, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00143 · Cited by 884
Domain Generalization via Invariant Feature Representation
Muandet, K., Balduzzi, D., & Schölkopf, B. (2013) · arXiv · doi:10.48550/arxiv.1301.2115 · Cited by 771
Hoyer, L., Dai, D., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00969 · Cited by 622
Self-citationAdaptive batch normalization for practical domain adaptation
Li, Y., Wang, N., Shi, J., Hou, X., & Liu, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.03.005 · Cited by 601
ACDC: The Adverse Conditions Dataset with Correspondences for Semantic Driving Scene Understanding
Sakaridis, C., Dai, D., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01059 · Cited by 572
Self-citationModel Adaptation: Unsupervised Domain Adaptation Without Source Data
Li, R., Jiao, Q., Cao, W., Wong, H.-S., & Wu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00966 · Cited by 487
Progressive Feature Alignment for Unsupervised Domain Adaptation
Chen, C., Xie, W., Huang, W., Rong, Y., Ding, X., Huang, Y., Xu, T., & Huang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00072 · Cited by 481
Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A. A., & Hardt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.13231 · Cited by 393
Revisiting Batch Normalization For Practical Domain Adaptation
Li, Y., Wang, N., Shi, J., Liu, J., & Hou, X. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.04779 · Cited by 340
RobustBench: a standardized adversarial robustness benchmark
Croce, F., Andriushchenko, M., Sehwag, V., Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., & Hein, M. (2020) · arXiv · doi:10.48550/arxiv.2010.09670 · Cited by 284
Tent: Fully Test-time Adaptation by Entropy Minimization
Wang, D., Shelhamer, E., Liu, S., Olshausen, B. A., & Darrell, T. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10726 · Cited by 278
Source-Free Domain Adaptation for Semantic Segmentation
Liu, Y., Zhang, W., & Wang, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00127 · Cited by 273
Generalized Source-free Domain Adaptation
Yang, S., Wang, Y., van de Weijer, J., Herranz, L., & Jui, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00885 · Cited by 267
Universal Source-Free Domain Adaptation
Nath Kundu, J., Venkat, N., Rahul, M. V., & Venkatesh Babu, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00460 · Cited by 231
Domain Impression: A Source Data Free Domain Adaptation Method
Kurmi, V. K., Subramanian, V. K., & Namboodiri, V. P. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00066 · Cited by 218
Lian, Q., Duan, L., Lv, F., & Gong, B. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00686 · Cited by 213
MEMO: Test Time Robustness Via Adaptation and Augmentation
Zhang, M., Levine, S., & Finn, C. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2799 · Cited by 189
Test-time adaptable neural networks for robust medical image segmentation
Karani, N., Erdil, E., Chaitanya, K., & Konukoglu, E. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101907 · Cited by 185
Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation
Wang, Q., Dai, D., Hoyer, L., Gool, L. V., & Fink, O. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00840 · Cited by 181
Self-citationAdversarial Continual Learning
Ebrahimi, S., Meier, F., Calandra, R., Darrell, T., & Rohrbach, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_23 · Cited by 162
On Feature Normalization and Data Augmentation
Li, B., Wu, F., Lim, S.-N., Belongie, S., & Weinberger, K. Q. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01220 · Cited by 160
Generalize then Adapt: Source-Free Domain Adaptive Semantic Segmentation
Kundu, J. N., Kulkarni, A., Singh, A. K., Jampani, V., & Babu, R. V. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00696 · Cited by 154
Incremental Adversarial Domain Adaptation for Continually Changing Environments
Wulfmeier, M., Bewley, A., & Posner, I. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460982 · Cited by 148
Continuous Manifold Based Adaptation for Evolving Visual Domains
Hoffman, J., Darrell, T., & Saenko, K. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.116 · Cited by 134
The Task Rehearsal Method of Life-Long Learning: Overcoming Impoverished Data
Silver, D. L., & Mercer, R. E. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47922-8_8 · Cited by 127
SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation
Prabhu, V., Khare, S., Kartik, D., & Hoffman, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00844 · Cited by 126
Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Iwasawa, Y., & Matsuo, Y. (2021) · neural information processing systems · Cited by 116
SoFA: Source-data-free Feature Alignment for Unsupervised Domain Adaptation
Yeh, H.-W., Yang, B., Yuen, P. C., & Harada, T. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00052 · Cited by 108
Test-Time Adaptation to Distribution Shift by Confidence Maximization and Input TransformationPreprint
Mummadi, C. K., Hutmacher, R., Rambach, K., Levinkov, E., Brox, T., & Metzen, J. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.14999 · Cited by 97
Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning
Volpi, R., Larlus, D., & Rogez, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00442 · Cited by 82
Test-time Batch Statistics Calibration for Covariate ShiftPreprint
You, F., Li, J., & Zhao, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04065 · Cited by 71
Adapting to Continuously Shifting Domains
Bobu, A., Tzeng, E., Hoffman, J., & Darrell, T. (2018) · International Conference on Learning Representations · Cited by 63
Greedy Policy Search: A Simple Baseline for Learnable Test-Time AugmentationPreprint
Molchanov, D., Lyzhov, A., Molchanova, Y., Ashukha, A., & Vetrov, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09103 · Cited by 59
MixNorm: Test-Time Adaptation Through Online Normalization EstimationPreprint
Hu, X., Uzunbas, G., Chen, S., Wang, R., Shah, A., Nevatia, R., & Lim, S.-N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11478 · Cited by 58
Fully Test-Time Adaptation for Image Segmentation
Hu, M., Song, T., Gu, Y., Luo, X., Chen, J., Chen, Y., Zhang, Y., & Zhang, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87199-4_24 · Cited by 44
Continuous Domain Adaptation with Variational Domain-Agnostic Feature ReplayPreprint
Lao, Q., Jiang, X., Havaei, M., & Bengio, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04382 · Cited by 27
Training on Test Data with Bayesian Adaptation for Covariate ShiftPreprint
Zhou, A., & Levine, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.12746 · Cited by 12
KATANA: Simple Post-Training Robustness Using Test Time Augmentations.Preprint
Cohen, G., & Giryes, R. (2021) · arXiv (Cornell University) · Cited by 1
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
Hendrycks, D., Mu, N., Cubuk, E. D., Zoph, B., Gilmer, J., & Lakshminarayanan, B. (2019) · arXiv · doi:10.48550/arxiv.1912.02781
Improving robustness against common corruptions by covariate shift adaptation
Schneider, S., Rusak, E., Eck, L., Bringmann, O., Brendel, W., & Bethge, M. (2020) · arXiv · doi:10.48550/arxiv.2006.16971
Cited by
Hoyer, L., Dai, D., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00969 · Cited by 622
Self-citationHRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation
Hoyer, L., Dai, D., & Van Gool, L. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20056-4_22 · Cited by 285
Self-citationA Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
Liang, J., He, R., & Tan, T. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-024-02181-w · Cited by 271
A biologically inspired architecture with switching units can learn to generalize across backgrounds
Voina, D., Shea-Brown, E., & Mihalas, S. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.014 · Cited by 2
Metadata
- Authors
- Wang, Q., Fink, O., Van Gool, L., & Dai, D.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 7191-7201
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312897837
- Cited by
- 502
- References
- 70
- FWCI
- 53.77
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.