Cited by 51Domain Adaptation and Few-Shot LearningAdvanced Neural Network ApplicationsReinforcement Learning in Robotics

Pseudo Labels for Unsupervised Domain Adaptation: A Review

Open access
Li, Yundong, Guo, Longxia, Ge, Yizheng·2023·Electronics

Abstract

Conventional machine learning relies on two presumptions: (1) the training and testing datasets follow the same independent distribution, and (2) an adequate quantity of samples is essential for achieving optimal model performance during training. Nevertheless, meeting these two assumptions can be challenging in real-world scenarios. Domain adaptation (DA) is a subfield of transfer learning that focuses on reducing the distribution difference between the source domain (Ds) and target domain (Dt) and subsequently applying the knowledge gained from the Ds task to the Dt task. The majority of current DA methods aim to achieve domain invariance by aligning the marginal probability distributions of the Ds. and Dt. Recent studies have pointed out that aligning marginal probability distributions alone is not sufficient and that alignment of conditional probability distributions is equally important for knowledge migration. Nonetheless, unsupervised DA presents a more significant difficulty in aligning the conditional probability distributions because of the unavailability of labels for the Dt. In response to this issue, there have been several proposed methods by researchers, including pseudo-labeling, which offer novel solutions to tackle the problem. In this paper, we systematically analyze various pseudo-labeling algorithms and their applications in unsupervised DA. First , we summarize the pseudo-label generation methods based on the single and multiple classifiers and actions taken to deal with the problem of imbalanced samples. Second, we investigate the application of pseudo-labeling in category feature alignment and improving feature discrimination. Finally, we point out the challenges and trends of pseudo-labeling algorithms. As far as we know, this article is the initial review of pseudo-labeling techniques for unsupervised DA.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/electronics12153325

Cites

68 of 81 references verified0 citation errors · 13 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The absence of strict constraints in direct regression as pseudo-label filtering criteria hinders the application of self-training techniques during the cross-domain adaptation process.

    “Moreover, the absence of strict constraints in direct regression as pseudo-label filtering criteria hinders the application of self-training techniques during the cross-domain adaptation process [20].”

    From Pose Estimation for Cross-Domain Non-Cooperative Spacecraft Based on Spatial-Aware Keypoints Regression (2024) · cites this paper as [20]

    Section 2.2 describes self-training with confident pseudo-label predictions, and sections 3.2 and 4.1 discuss thresholds and pseudo-label selection. None of the passages found addresses direct regression as a filtering criterion or says that a lack of strict constraints hinders self-training in cross-domain adaptation (searched: direct regression, strict constraints, filtering criteria). (LLM, conf=0.73)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. Lee, D.-H. Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks. In Proceedings of the ICML 2013 Workshop: Challenges in Representation Learning (WREPL), Atlanta, GA, USA, 16-21 June 2013.
  2. Wang, F.; Han, Z.; Yin, Y. Source Free Robust Domain Adaptation Based on Pseudo Label Uncertainty Estimation. J. Softw. 2022, 33, 1183-1199. [CrossRef] · DOI
  3. Venkat, N.; Kundu, J.; Singh, D.K.; Revanur, A.; VenkateshBabu, R. Your Classifier can Secretly Suffice Multi-Source Domain Adaptation. arXiv 2021. [CrossRef] · DOI
  4. Liu, H.; Wang, J.; Long, M. Cycle Self-Training for Domain Adaptation. arXiv 2021. [CrossRef] · DOI
  5. Tanwani, A.K. DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer. arXiv 2021. [CrossRef] · DOI
  6. Jiang, X.; Lao, Q.; Matwin, S.; Havaei, M. Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation. arXiv 2020. [CrossRef] · DOI
  7. Yu, Z.; Li, J.; Du, Z.; Zhu, L.; Shen, H.T. A Comprehensive Survey on Source-free Domain Adaptation. arXiv 2023. [CrossRef] · DOI

Cited by

Sorted by citations

Authors
Li, Y., Guo, L., & Ge, Y.
Affiliations
North China University of Technology (CN)
Corresponding author
Li, Yundong
Venue
Electronics
Publisher
MDPI AG
Published
2023-08-03
Volume
12
Issue
15
Pages
3325
ISSN
2079-9292 (ISSN-L)
OpenAlex
W4385543003
Cited by
51
References
68
Citations by year
2026: 11 · 2025: 25 · 2024: 14 · 2023: 1
FWCI
4.51
Type
journal article
Language
en
Open access
Locations
Electronics (publishedVersion, cc-by)
Funders
National Natural Science Foundation of China (62071006)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
unsupervised domain adaptation, pseudo-labeling, domain invariance, transfer learning, limited labeled data, feature discrimination, marginal distribution alignment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review