Understanding the Failure Modes of Out-of-Distribution Generalization
Open accessAbstract
Empirical studies suggest that machine learning models often rely on features, such as the background, that may be spuriously correlated with the label only during training time, resulting in poor accuracy during test-time. In this work, we identify the fundamental factors that give rise to this behavior, by explaining why models fail this way {\em even} in easy-to-learn tasks where one would expect these models to succeed. In particular, through a theoretical study of gradient-descent-trained linear classifiers on some easy-to-learn tasks, we uncover two complementary failure modes. These modes arise from how spurious correlations induce two kinds of skews in the data: one geometric in nature, and another, statistical in nature. Finally, we construct natural modifications of image classification datasets to understand when these failure modes can arise in practice. We also design experiments to isolate the two failure modes when training modern neural networks on these datasets.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior work establishes that in many tasks, in-distribution loss is a strong predictor of out-of-distribution generalization.
“Prior work on out-of-distribution generalization establishes that in many tasks, in-distribution loss is a strong predictor of out-of-distribution generalization [14].”
From Exploring Length Generalization in Large Language Models (2022) · cites this paper as [14]
The Introduction (p. 1) describes minimizing in-distribution training loss, while Section 3 (p. 4) shows an example in which an ERM classifier’s out-of-distribution accuracy deteriorates. Neither passage assesses whether in-distribution loss is a strong predictor of out-of-distribution generalization across many tasks. Searches for that relationship and for loss–accuracy correlations found no passage establishing the attributed claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Wu, Y., & Vapnik, V. N. (1999) · Technometrics · doi:10.2307/1271368 · Cited by 27,236 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318 · see citing works
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831 · see citing works
Shortcut learning in deep neural networks
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00257-z · Cited by 2,224 · see citing works
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
Hendrycks, D., & Dietterich, T. (2019) · arXiv · doi:10.48550/arxiv.1903.12261 · Cited by 1,335 · see citing works
Learning to Generalize: Meta-Learning for Domain Generalization
Li, D., Yang, Y., Song, Y.-Z., & Hospedales, T. M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11596 · Cited by 1,313 · see citing works
A Closer Look at Memorization in Deep Networks
Arpit, D., Jastrzębski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al. (2017) · arXiv · doi:10.48550/arxiv.1706.05394 · Cited by 1,307 · see citing works
Robustness May Be at Odds with AccuracyPreprint
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 1,240 · see citing works
Adversarial Examples Are Not Bugs, They Are Features
Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., & Madry, A. (2019) · arXiv · doi:10.48550/arxiv.1905.02175 · Cited by 971 · see citing works
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944 · see citing works
Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019) · arXiv · doi:10.48550/arxiv.1907.02893 · Cited by 927 · see citing works
Mixup: Beyond empirical risk minimizationPreprint
Zhang, H. (2024) · arXiv (Cornell University) · doi:10.57702/dcy1c3gw · Cited by 923 · see citing works
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1× · see citing works
“It can also lead to unfair biases and poor performance on minority groups (Dixon et al., 2018; Zhao et al., 2017; Sagawa et al., 2020b).”
BackgroundDomain Generalization via Invariant Feature Representation
Muandet, K., Balduzzi, D., & Schölkopf, B. (2013) · arXiv · doi:10.48550/arxiv.1301.2115 · Cited by 771 · see citing works
In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep LearningPreprint
Neyshabur, B., Tomioka, R., & Srebro, N. (2014) · International Conference on Learning Representations · doi:10.48550/arxiv.1412.6614 · Cited by 742 · see citing works
Self-citationRecognition in Terra Incognita
Beery, S., Van Horn, G., & Perona, P. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01270-0_28 · Cited by 679 · see citing works
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · see citing works
Exploring Generalization in Deep Learning
Neyshabur, B., Bhojanapalli, S., McAllester, D., & Srebro, N. (2017) · neural information processing systems · Cited by 670 · see citing works
Self-citationDeep Domain Generalization via Conditional Invariant Adversarial Networks
Li, Y., Tian, X., Gong, M., Liu, Y., Liu, T., Zhang, K., & Tao, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_38 · Cited by 622 · see citing works
Sagawa, S., Koh, P. W., Hashimoto, T., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.08731 · Cited by 563 · see citing works
Asirra: A captcha that exploits interest-aligned manual image categorization
Elson, J., Douceur, J. R., Howell, J., & Saul, J. (2007) · Cited by 479 · see citing works
In Search of Lost Domain Generalization
Gulrajani, I., & Lopez-Paz, D. (2020) · arXiv · doi:10.48550/arxiv.2007.01434 · Cited by 440 · cited 4× · see citing works
“As discussed in Gulrajani & Lopez-Paz (2020), there are also many other existing non-ERM based methods, including that of meta-learning (Li et al., 2018a), parameter-sharing (Sagawa et al., 2020a) and data augmentation (Zhang et al., 2018).”
BackgroundInfluentialOut-of-Distribution Generalization via Risk Extrapolation (REx)
Krueger, D., Caballero, E., Jacobsen, J.-H., Zhang, A., Binas, J., Zhang, D., Priol, R. L., & Courville, A. (2020) · arXiv · doi:10.48550/arxiv.2003.00688 · Cited by 407 · see citing works
The Implicit Bias of Gradient Descent on Separable DataPreprint
Soudry, D., Hoffer, E., Nacson, M. S., Gunasekar, S., & Srebro, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.10345 · Cited by 387 · see citing works
Unbiased Metric Learning: On the Utilization of Multiple Datasets and Web Images for Softening Bias
Chen, F., Xu, Y., & Rockmore, D. N. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.208 · Cited by 337 · see citing works
Exploring Generalization in Deep LearningPreprint
Neyshabur, B., Bhojanapalli, S., McAllester, D., & Srebro, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.08947 · Cited by 282 · see citing works
Self-citation(2011) · Cited by 277 · see citing works
Training Behavior of Deep Neural Network in Frequency Domain
Xu, Z. J., Zhang, Y., & Xiao, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-36708-4_22 · Cited by 241 · see citing works
The Elephant in the RoomPreprint
Rosenfeld, A., Zemel, R. S., & Tsotsos, J. K. (2018) · arXiv (Cornell University) · doi:10.13140/rg.2.2.15295.20642 · Cited by 140 · see citing works
Palechor, F. M., & Manotas, A. D. L. H. (2019) · Data in Brief · doi:10.1016/j.dib.2019.104344 · Cited by 135 · see citing works
An Investigation of Why Overparameterization Exacerbates Spurious CorrelationsPreprint
Sagawa, S., Raghunathan, A., Koh, P. W., & Liang, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.04345 · Cited by 132 · see citing works
The Pitfalls of Simplicity Bias in Neural NetworksPreprint
Shah, H., Tamuly, K., Raghunathan, A., Jain, P., & Netrapalli, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07710 · Cited by 121 · see citing works
Uniform convergence may be unable to explain generalization in deep learning
Nagarajan, V., & Kolter, J. Z. (2019) · arXiv · doi:10.48550/arxiv.1902.04742 · Cited by 118 · see citing works
Self-citationSGD on Neural Networks Learns Functions of Increasing ComplexityPreprint
Nakkiran, P., Kaplun, G., Kalimeris, D., Yang, T., Edelman, B., Zhang, F., & Barak, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11604 · Cited by 94 · see citing works
Causal Inference by using Invariant Prediction: Identification and Confidence Intervals
Peters, J., Bühlmann, P., & Meinshausen, N. (2016) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12167 · Cited by 88 · see citing works
Generalizing to unseen domains via distribution matchingPreprint
Albuquerque, I., Monteiro, J., Darvishi-Bayazi, M.-J., Falk, T. H., & Mitliagkas, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00804 · Cited by 87 · see citing works
What shapes feature representations? Exploring datasets, architectures, and trainingPreprint
Hermann, K. L., & Lampinen, A. K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12433 · Cited by 66 · see citing works
Risk and parameter convergence of logistic regressionPreprint
Ji, Z., & Telgarsky, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.07300 · Cited by 61 · see citing works
Cross-Entropy Loss and Low-Rank Features Have Responsibility for Adversarial ExamplesPreprint
Nar, K., Öçal, O., Sastry, S., & Ramchandran, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08360 · Cited by 51 · see citing works
Generalization in Deep Networks: The Role of Distance from InitializationPreprint
Nagarajan, V., & Kolter, J. Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01672 · Cited by 48 · see citing works
Self-citationOn the Spectral Bias of Deep Neural NetworksPreprint
Rahaman, N., Arpit, D., Baratin, A., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y., & Courville, A. C. (2018) · arXiv (Cornell University) · Cited by 40 · see citing works
Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately
Khani, F., & Liang, P. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445883 · Cited by 38 · see citing works
On the Learning Dynamics of Deep Neural NetworksPreprint
Combes, R. T. D., Pezeshki, M., Shabanian, S., Courville, A. C., & Bengio, Y. (2018) · arXiv (Cornell University) · Cited by 19 · see citing works
Towards Shape Biased Unsupervised Representation Learning for Domain GeneralizationPreprint
Asadi, N., Sarfi, A. M., Hosseinzadeh, M., Karimpour, Z., & Eftekhari, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08245 · Cited by 18 · see citing works
On the Learning Dynamics of Deep Neural NetworksPreprint
Combes, R. T. D., Pezeshki, M., Shabanian, S., Courville, A. C., & Bengio, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.06848 · Cited by 8 · see citing works
Feature Noise Induces Loss Discrepancy Across GroupsPreprint
Khani, F., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09876 · Cited by 4 · see citing works
Flagged first, then by citations
3 of the 51 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Invariance Principle Meets Information Bottleneck for Out-of-Distribution GeneralizationPreprint
Ahuja, K., Caballero, E., Zhang, D., Gagnon-Audet, J.-C., Bengio, Y., Mitliagkas, I., & Rish, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.06607 · Cited by 118 · see citing works
Invariant Information Bottleneck for Domain Generalization
Li, B., Shen, Y., Wang, Y., Zhu, W., Reed, C., Li, D., Keutzer, K., & Zhao, H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i7.20703 · Cited by 107 · see citing works
Toward Understanding and Boosting Adversarial Transferability From a Distribution Perspective
Zhu, Y. T., Chen, Y., Li, X., Chen, K., He, Y., Tian, X., Zheng, B., Chen, Y., & Huang, Q. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3211736 · Cited by 78 · see citing works
Model-Based Domain GeneralizationPreprint
Robey, A., Pappas, G. J., & Hassani, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.11436 · Cited by 57 · see citing works
Towards a Theoretical Framework of Out-of-Distribution GeneralizationPreprint
Ye, H., Xie, C., Cai, T., Li, R., Li, Z., & Wang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.04496 · Cited by 46 · see citing works
Exploring Length Generalization in Large Language Models
Anil, C., Wu, Y., Andreassen, A., Lewkowycz, A., Misra, V., Ramasesh, V., Slone, A., Gur-Ari, G., Dyer, E., & Neyshabur, B. (2022) · arXiv · doi:10.48550/arxiv.2207.04901 · Cited by 44 · see citing works
“Prior work on out-of-distribution generalization establishes that in many tasks, in-distribution loss is a strong predictor of out-of-distribution generalization [14].”
BackgroundSelf-citationSliceTeller : A Data Slice-Driven Approach for Machine Learning Model Validation
Zhang, X., Ono, J. P., Song, H., Gou, L., Ma, K., & Ren, L. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209465 · Cited by 43 · see citing works
Does Invariant Risk Minimization Capture Invariance?Preprint
Kamath, P., Tangella, A., Sutherland, D. J., & Srebro, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01134 · Cited by 37 · see citing works
Cross Contrasting Feature Perturbation for Domain Generalization
Li, C., Zhang, D., Huang, W., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00128 · Cited by 32 · see citing works
Enhanced Long-Tailed Recognition With Contrastive CutMix Augmentation
Pan, H., Guo, Y. F., Yu, M., & Chen, J. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3425148 · Cited by 31 · see citing works
Modeling the Distributional Uncertainty for Salient Object Detection Models
Tian, X., Zhang, J., Xiang, M., & Dai, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01883 · Cited by 31 · see citing works
Towards out of distribution generalization for problems in mechanics
Yuan, L., Park, H. S., & Lejeune, E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.115569 · Cited by 31 · see citing works
Out-of-distribution Detection via Frequency-regularized Generative Models
Cai, M., & Li, Y. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00548 · Cited by 29 · see citing works
Watson, M., Hasan, B. A. S., & Moubayed, N. A. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00159 · Cited by 27 · see citing works
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?Preprint
Zhang, D., Ahuja, K., Xu, Y., Wang, Y., & Courville, A. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02890 · Cited by 26 · see citing works
Nonlinear Invariant Risk Minimization: A Causal ApproachPreprint
Lu, C., Wu, Y., Hernández-Lobato, J. M., & Schölkopf, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12353 · Cited by 29 · see citing works
On the role of artificial intelligence in psychiatry
Rocheteau, E. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2022.132 · Cited by 21 · see citing works
Probe microscopy is all you need *
Kalinin, S. V., Vasudevan, R. K., Liu, Y., Ghosh, A., Roccapriore, K. M., & Ziatdinov, M. A. (2023) · Machine Learning Science and Technology · doi:10.1088/2632-2153/acccd5 · Cited by 20 · see citing works
FedGen: Generalizable Federated Learning for Sequential Data
Venkateswaran, P., Isahagian, V., Muthusamy, V., & Venkatasubramanian, N. (2023) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud60044.2023.00044 · Cited by 17 · see citing works
Dangers of Bayesian Model Averaging under Covariate ShiftPreprint
Izmailov, P., Nicholson, P., Lotfi, S., & Wilson, A. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.11905 · Cited by 16 · see citing works
Training Debiased Subnetworks with Contrastive Weight Pruning
Park, G. Y., Lee, S., Lee, S. W., & Ye, J. C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00766 · Cited by 15 · see citing works
I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at PitchPreprint
Turian, J. P., & Henry, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.04572 · Cited by 15 · see citing works
A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation.Preprint
Ulmer, D. (2021) · arXiv (Cornell University) · Cited by 15 · see citing works
Simple Convolutional-Based Models: Are They Learning the Task or the Data?
Sa-Couto, L., & Wichert, A. (2021) · Neural Computation · doi:10.1162/neco_a_01446 · Cited by 13 · see citing works
Open-Scenario-Oriented Human Gait Recognition Using Radar Micro-Doppler Signatures
Yang, Y., Zhao, D., Yang, X., Li, B., Wang, X., & Lang, Y. (2024) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2024.3403077 · Cited by 12 · see citing works
Duboudin, T., Dellandréa, E., Abgrall, C., Hénaff, G., & Chen, L. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00012 · Cited by 11 · see citing works
Causally motivated Shortcut Removal Using Auxiliary LabelsPreprint
Makar, M., Packer, B., Moldovan, D., Blalock, D. W., Halpern, Y., & D’Amour, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.06422 · Cited by 11 · see citing works
Conditional entropy minimization principle for learning domain invariant representation features
Nguyen, T. D., Lyu, B., Ishwar, P., Scheutz, M. J., & Aeron, S. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956548 · Cited by 9 · see citing works
Failure Modes of Domain Generalization Algorithms
Galstyan, T., Harutyunyan, H., Khachatrian, H., Steeg, G. V., & Galstyan, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01849 · Cited by 8 · see citing works
Sayadi, H., He, Z., Miari, T., & Aliasgari, M. (2024) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas58744.2024.10558202 · Cited by 7 · see citing works
Transformer spectral optimization: From gradient frequency analysis to adaptive spectral integration
Huang, Z., Chen, M., & Zheng, S. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113637 · Cited by 6 · see citing works
Experimental Standards for Deep Learning in Natural Language Processing Research
Ulmer, D., Bassignana, E., Müller-Eberstein, M., Varab, D., Zhang, M., van der Goot, R., Hardmeier, C., & Plank, B. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.196 · Cited by 6 · see citing works
How Does Distribution Matching Help Domain Generalization: An Information-Theoretic Analysis
Dong, Y., Gong, T., Chen, H., Song, S., Zhang, W., & Li, C. (2025) · IEEE Transactions on Information Theory · doi:10.1109/tit.2025.3531136 · Cited by 7 · see citing works
Unlock the Potential of Counterfactually-Augmented Data in Out-Of-Distribution Generalization
Fan, C., Chen, W., Tian, J., Li, Y., He, H., & Jin, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122066 · Cited by 5 · see citing works
Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach
Chen, Z., Zheng, Y., Lai, Z., Guan, Q., & Lin, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.29028 · Cited by 5 · see citing works
Loaliyan, S. S., & Steeg, G. V. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00496 · Cited by 5 · see citing works
Sun, Y. Q., Pahlavan, H. A., Chattopadhyay, A., Hassanzadeh, P., Lubis, S. W., Alexander, M. J., Gerber, E. P., Sheshadri, A., & Guan, Y. (2023) · doi:10.22541/essoar.170365371.16222393/v1 · Cited by 5 · see citing works
Self-Supervised Debiasing Using Low Rank Regularization
Park, G. Y., Jung, C., Lee, S., Ye, J. C., & Lee, S. W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01178 · Cited by 4 · see citing works
Understanding and Testing Generalization of Deep Networks on Out-of-Distribution DataPreprint
Hu, R., Sang, J., Wang, J., Hu, R., & Jiang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09190 · Cited by 4 · see citing works
Diffusing Gaussian Mixtures for Generating Categorical Data
Regol, F., & Coates, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26145 · Cited by 3 · see citing works
He, J., Chen, K., Meng, G., Zhang, J., & Li, C. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616605 · Cited by 3 · see citing works
Experimental Standards for Deep Learning in Natural Language Processing ResearchPreprint
Ulmer, D., Bassignana, E., Müller-Eberstein, M., Varab, D., Zhang, M., van der Goot, R., Hardmeier, C., & Plank, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.06251 · Cited by 5 · see citing works
Domain Generalization Study of Empirical Risk Minimization From Causal Perspectives
Mo, Z., Zhang, Z., & Tsui, K. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535276 · Cited by 2 · see citing works
Trade-off between reconstruction loss and feature alignment for domain generalization
Nguyen, T. D., Lyu, B., Ishwar, P., Scheutz, M. J., & Aeron, S. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00132 · Cited by 2 · see citing works
Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data
Zou, X. G., & Liu, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i15.29673 · Cited by 2 · see citing works
Gradual sensory maturation promotes abstract representation learningPreprint
Cheon, J., & Paik, S. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.24.661295 · Cited by 2 · see citing works
Modeling the Q-Diversity in a Min-max Play Game for Robust Optimization
Wu, T., Zheng, R., Gui, T., Zhang, Q., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.8 · Cited by 1 · see citing works
Kaur, G. (2023) · The British Journal of Psychiatry · doi:10.1192/bjp.2022.130
Evaluation of Categorical Generative Models - Bridging the Gap Between Real and Synthetic Data
Regol, F., Kroon, A., & Coates, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10097150
Denoising in Representation Space via Data-Dependent Regularization for Better Representation
Chen, M., Wang, D., Feng, S., & Zhang, Y. (2023) · Mathematics · doi:10.3390/math11102327
Exploring and Utilizing Pattern Imbalance
Mei, S., Zhao, C., Yuan, S., & Ni, B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00731
On The Failure of Invariant Risk Minimization and an Effective fix via Classification Error Control
Nguyen, T. D., Scheutz, M. J., & Aeron, S. (2023) · Machine learning for signal processing ... · doi:10.1109/mlsp55844.2023.10286000
The Timing of Suboptimality: How Temporal Uncertainty Drives Procrastination BehaviorPreprint
Sokratous, K., Hintze, A., Fitch, A. K., & Kvam, P. D. (2024) · doi:10.31219/osf.io/trkv9
Towards a better evaluation of out-of-domain generalization
Hwang, D., Kang, S., Eo, M., Kim, J., & Rhee, W. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107434
- Authors
- Nagarajan, V., Andreassen, A., & Neyshabur, B.
- Affiliations
- Carnegie Mellon University (US)
- Google (United States) (US)
- ORCID
- Andreassen, Anders
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2010.15775
- OpenAlex
- W3094749184
- Cited by
- 55
- References
- 51
- Citations by year
- 2025: 5 · 2024: 8 · 2023: 17 · 2022: 11 · 2021: 13 · 2020: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- out-of-distribution generalization, spurious correlations, failure modes, linear classifiers, gradient descent, image classification dataset, artificial neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
