Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.

Semi-Supervised Multimodal Representation Learning Through a Global Workspace

Open access
Devillers, Benjamin, Maytié, Léopold, VanRullen, Rufin · 2025 · IEEE Transactions on Neural Networks and Learning Systems

Abstract

Recent deep learning models can efficiently combine inputs from different modalities (e.g., images and text) and learn to align their latent representations or to translate signals from one domain to another (as in image captioning or text-to-image generation). However, current approaches mainly rely on brute-force supervised training over large multimodal datasets. In contrast, humans (and other animals) can learn useful multimodal representations from only sparse experience with matched cross-modal data. Here, we evaluate the capabilities of a neural network architecture inspired by the cognitive notion of a "global workspace" (GW): a shared representation for two (or more) input modalities. Each modality is processed by a specialized system (pretrained on unimodal data and subsequently frozen). The corresponding latent representations are then encoded to and decoded from a single shared workspace. Importantly, this architecture is amenable to self-supervised training via cycle-consistency: encoding-decoding sequences should approximate the identity function. For various pairings of vision-language modalities and across two datasets of varying complexity, we show that such an architecture can be trained to align and translate between two modalities with very little need for matched data (from four to seven times less than a fully supervised approach). The GW representation can be used advantageously for downstream classification and cross-modal retrieval tasks and for robust transfer learning. Ablation studies reveal that both the shared workspace and the self-supervised cycle-consistency training are critical to the system's performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tnnls.2024.3416701

Cites

65 of 65 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 56 references

Research works printed but not linked: 2

  1. Jaegle, 2021, Perceiver: General Perception With Iterative Attention
  2. 2023, Open-Source Mobile Robot Simulation Software

Cited by

See all 6

Metadata

Authors
Devillers, B., Maytié, L., & VanRullen, R.
Affiliations
  • Centre National de la Recherche Scientifique (FR)
  • CerCo, CNRS UMR 5549, Université de Toulouse, Toulouse, France
Venue
IEEE Transactions on Neural Networks and Learning Systems
Published
2025-05
Volume
36
Issue
5
Pages
7843-7857
ISSN
  • 2162-237X (ISSN-L)
  • 2162-2388
OpenAlex
W4400229207
Cited by
5
References
56
Citations by year
2026: 1 · 2025: 3
FWCI
0.70
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Neural Networks and Learning Systems (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • HAL (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
  • High Performance Computing (HPC) Resources from CALMIP (2020-p20032)
  • Agence Nationale de la Recherche (ANR) grant Natural Language Programming for Conversational Cobots (ANR-21-FAI2-0005)
  • Region Occitanie grant Cobots Conversationnels pour Processus Industriels et Logistiques
  • “Défi-clé Robotique centrée sur l’humain” Ph.D.-thesis grant
  • Artificial and Natural Intelligence Toulouse Institute (ANITI) Chair (ANR-19-PI3A-004)
  • European Research Council (ERC) Advanced Grant Global Latent Workspace (101096017)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
Keywords
global workspace, multimodal representation learning, cross-modal alignment, cross-modal translation, self-supervised learning, cycle consistency, vision-language models, transfer learning, cross-modal retrieval, shared latent representation, semi-supervised learning
MeSH terms
Supervised Machine Learning, Deep Learning, Algorithms, Animals, Humans, Databases, Factual, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Semi-Supervised Multimodal Representation Learning Through a Global Workspace. https://app.cochat.ai/papers/10.1109/tnnls.2024.3416701
The paper
Devillers, B., Maytié, L., & VanRullen, R. (2025). Semi-Supervised Multimodal Representation Learning Through a Global Workspace. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/tnnls.2024.3416701

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.