CoChat

Conditional variance penalties and domain shift robustness

Open access
Heinze-Deml, Christina, Meinshausen, Nicolai · 2021 · Machine Learning
Cited by 97

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Abstract

When training a deep neural network for image classification, one can broadly distinguish between two types of latent features of images that will drive the classification. We can divide latent features into (i) ‘core’ or ‘conditionally invariant’ features$$C$$Cwhose distribution$$C\vert Y$$C|Y, conditional on the class Y, does not change substantially across domains and (ii) ‘style’ features$$S$$Swhose distribution$$S\vert Y$$S|Ycan change substantially across domains. Examples for style features include position, rotation, image quality or brightness but also more complex ones like hair color, image quality or posture for images of persons. Our goal is to minimize a loss that is robust under changes in the distribution of these style features. In contrast to previous work, we assume that the domain itself is not observed and hence a latent variable. We do assume that we can sometimes observe a typically discrete identifier or “$$\mathrm {ID}$$IDvariable”. In some applications we know, for example, that two images show the same person, and$$\mathrm {ID}$$IDthen refers to the identity of the person. The proposed method requires only a small fraction of images to have$$\mathrm {ID}$$IDinformation. We group observations if they share the same class and identifier$$(Y,\mathrm {ID})=(y,\mathrm {id})$$(Y,ID)=(y,id)and penalize the conditional variance of the prediction or the loss if we condition on$$(Y,\mathrm {ID})$$(Y,ID). Using a causal framework, this conditional variance regularization (C o R e) is shown to protect asymptotically against shifts in the distribution of the style variables in a partially linear structural equation model. Empirically, we show that the C o R e penalty improves predictive accuracy substantially in settings where domain changes occur in terms of image quality, brightness and color while we also look at more complex changes such as changes in movement and posture.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10994-020-05924-1

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 18 references

Cited by

See all 97

Metadata

Authors
Heinze-Deml, C., & Meinshausen, N.
Venue
Machine Learning
Published
2020-11-23
Volume
110
Issue
2
Pages
303-348
ISSN
  • 0885-6125 (ISSN-L)
  • 1573-0565
OpenAlex
W2795857097
Cited by
97
References
18
FWCI
7.58
Type
journal article
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Conditional variance penalties and domain shift robustness. https://app.cochat.ai/papers/10.1007/s10994-020-05924-1
The paper
Heinze-Deml, C., & Meinshausen, N. (2021). Conditional variance penalties and domain shift robustness. Machine Learning. https://doi.org/10.1007/s10994-020-05924-1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.