Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.

Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes

Open access
Madsen, Jens, Parra, Lucas C·2022·PNAS Nexus

Published version of a preprint

Abstract

Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain–body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/pnasnexus/pgac020

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. ISC of neural, physiological, and behavioral signals during passive video watching. (A) Signals for each of the modalities simultaneously recorded during Experiment 1. The 2 subjects shown (green and orange) have the highest ISC values measured for each modality. The EEG signal is the first component extracted from the 64-channel EEG using correlated component analysis. The gaze position signal is the horizontal gaze position. (B) Pearson correlation matrix between all pairs of the 92 subjects for each of the modalities. Subjects are sorted by increasing average correlation values. The correlation matrix for gaze position is the average correlation of gaze position in the horizontal and vertical direction. The correlation matrix for EEG is the sum of the correlation values obtained for 9 components extracted with correlated component analysis. (C) ISC values are the average of pairwise correlations for each subject, i.e. the mean over column of the correlation matrix in (B), excluding the diagonal, and averaged over the 3 videos presented (10 minutes total duration). Subjects are ordered by their ISC values (same as in B). Filled points indicate statistically significant ISC values and nonfilled points indicate they are not statistically significant. Statistical significance is determined using circular shuffle statistics (10,000 shuffles and corrected for multiple comparisons with FDR of 0.01). Circular shuffle means that the signal of each subject is randomly shifted in time, thus removing any intersubject relation.
Fig. 1. ISC of neural, physiological, and behavioral signals during passive video watching. (A) Signals for each of the modalities simultaneously recorded during Experiment 1. The 2 subjects shown (green and orange) have the highest ISC values measured for each modality. The EEG signal is the first component extracted from the 64-channel EEG using correlated component analysis. The gaze position signal is the horizontal gaze position. (B) Pearson correlation matrix between all pairs of the 92 subjects for each of the modalities. Subjects are sorted by increasing average correlation values. The correlation matrix for gaze position is the average correlation of gaze position in the horizontal and vertical direction. The correlation matrix for EEG is the sum of the correlation values obtained for 9 components extracted with correlated component analysis. (C) ISC values are the average of pairwise correlations for each subject, i.e. the mean over column of the correlation matrix in (B), excluding the diagonal, and averaged over the 3 videos presented (10 minutes total duration). Subjects are ordered by their ISC values (same as in B). Filled points indicate statistically significant ISC values and nonfilled points indicate they are not statistically significant. Statistical significance is determined using circular shuffle statistics (10,000 shuffles and corrected for multiple comparisons with FDR of 0.01). Circular shuffle means that the signal of each subject is randomly shifted in time, thus removing any intersubject relation.CC BY · PubMed Central
Fig. 2. ISC is co-modulated in different signal modalities. ISC computed here for multiple signals recorded on 92 subjects while they watched instructional videos (same as in Fig. 1). Each point represents ISC between 1 individual and the group. The diagonal of the plot matrix is the histogram of each modality. Significant Pearson's correlation of ISC between each modality is indicated by a * (P < 0.01, bonferroni corrected). Lines indicate the linear least-squares prediction of vertical from horizontal axis (while points are the same in upper and lower triangles after flipping axes, prediction lines are not).
Fig. 2. ISC is co-modulated in different signal modalities. ISC computed here for multiple signals recorded on 92 subjects while they watched instructional videos (same as in Fig. 1). Each point represents ISC between 1 individual and the group. The diagonal of the plot matrix is the histogram of each modality. Significant Pearson's correlation of ISC between each modality is indicated by a * (P < 0.01, bonferroni corrected). Lines indicate the linear least-squares prediction of vertical from horizontal axis (while points are the same in upper and lower triangles after flipping axes, prediction lines are not).CC BY · PubMed Central
Fig. 3. Time scales of signal fluctuations. (A) Frequency-resolved ISC: intersubject coherence spectrum computed by first band-pass filtering signals at different frequencies and then computing ISC averaged over videos and subjects. Band-pass filtering used center frequency on a logarithmic scale and a bandwidth of 0.2 octaves. Blue shading indicates SEM across N = 92 subjects. Significance of ISC values above 0 are established in each band using t test, corrected for multiple comparisons using 1D cluster statistics (light gray area indicates P < 0.01, cluster corrected). (B) Magnitude of fluctuations captured by the power spectrum, i.e. the power of the band-pass filtered signals. (C) Correlation between time-resolved ISC of EEG and time-resolved ISC of different modalities computed separately for each subject. Time-resolved ISC is computed in 10-second time windows with 50% overlap. Each dot is a subject. ** indicate P < 0.01 for a t test for nonzero mean ISC, uncorrected). The density is computed using kernel density estimation using a Gaussian kernel. The box plot shows the median and 25th and 75th percentile. Data aggregated over ∼15 minutes of video of Experiment 1.
Fig. 3. Time scales of signal fluctuations. (A) Frequency-resolved ISC: intersubject coherence spectrum computed by first band-pass filtering signals at different frequencies and then computing ISC averaged over videos and subjects. Band-pass filtering used center frequency on a logarithmic scale and a bandwidth of 0.2 octaves. Blue shading indicates SEM across N = 92 subjects. Significance of ISC values above 0 are established in each band using t test, corrected for multiple comparisons using 1D cluster statistics (light gray area indicates P < 0.01, cluster corrected). (B) Magnitude of fluctuations captured by the power spectrum, i.e. the power of the band-pass filtered signals. (C) Correlation between time-resolved ISC of EEG and time-resolved ISC of different modalities computed separately for each subject. Time-resolved ISC is computed in 10-second time windows with 50% overlap. Each dot is a subject. ** indicate P < 0.01 for a t test for nonzero mean ISC, uncorrected). The density is computed using kernel density estimation using a Gaussian kernel. The box plot shows the median and 25th and 75th percentile. Data aggregated over ∼15 minutes of video of Experiment 1.CC BY · PubMed Central
Fig. 4. Common factor modulating ISC predicts memory recall: (A) variance explained by the principal components computed across all ISC modalities. (B) Loadings on the first 2 principal components, i.e. how much of each ISC modality is captured by each of the principal components. (C) The ISC projected onto the 2 first principal coefficients (PC Score) and the test taking performance of students (%-correct). (D) ISC of each modality and the test taking performance of students. Panels (A) and (B) include all data from Fig. 3 (Experiment 1) Memory performance for panels (C) and (D) was available only for a subset of 43 subjects. P-values are uncorrected.
Fig. 4. Common factor modulating ISC predicts memory recall: (A) variance explained by the principal components computed across all ISC modalities. (B) Loadings on the first 2 principal components, i.e. how much of each ISC modality is captured by each of the principal components. (C) The ISC projected onto the 2 first principal coefficients (PC Score) and the test taking performance of students (%-correct). (D) ISC of each modality and the test taking performance of students. Panels (A) and (B) include all data from Fig. 3 (Experiment 1) Memory performance for panels (C) and (D) was available only for a subset of 43 subjects. P-values are uncorrected.CC BY · PubMed Central
Fig. 5. Attentional modulation of ISC and Power resolved by frequency. Here, subjects watch instructional videos (31 minutes total duration, Experiment 2) attending normally (blue) or distracted (purple) by performing a mental arithmetic task. (A) Coherence spectra measure ISC resolved by frequency. The color shaded areas indicate SEM across subjects. Significant differences between attending and distracted conditions are established for each band using a t test, corrected for multiple comparisons with 1D cluster statistics (gray-shaded frequency range indicates P < 0.01, N = 29, and cluster threshold corrected). (B) Power spectra measure power of signal fluctuations resolved by frequency. Shaded areas as in B. (C) Effect size is measured as Cohen's d’ with variance estimated across subjects. ISC and power are computed on band-pass filtered signals (as in Fig. 3.) and averaged over 6 videos.
Fig. 5. Attentional modulation of ISC and Power resolved by frequency. Here, subjects watch instructional videos (31 minutes total duration, Experiment 2) attending normally (blue) or distracted (purple) by performing a mental arithmetic task. (A) Coherence spectra measure ISC resolved by frequency. The color shaded areas indicate SEM across subjects. Significant differences between attending and distracted conditions are established for each band using a t test, corrected for multiple comparisons with 1D cluster statistics (gray-shaded frequency range indicates P < 0.01, N = 29, and cluster threshold corrected). (B) Power spectra measure power of signal fluctuations resolved by frequency. Shaded areas as in B. (C) Effect size is measured as Cohen's d’ with variance estimated across subjects. ISC and power are computed on band-pass filtered signals (as in Fig. 3.) and averaged over 6 videos.CC BY · PubMed Central
Fig. 6. Within-subject coupling between EEG and other signal modalities during viewing of videos. WSC between EEG and gaze position, pupil size, HR, and respiration. The WSC is computed as the correlation between each signal modality and spatially filtered EEG. The spatial filters are found as the least square fit of scalp electrode potentials with the respective signal modality (computed separately for each frequency band; see supplement for variation across bands). (A) WSC computed by combining the 3 stimuli used in Experiment 1. Blue-shaded area indicates SEM over subjects. Gray-shaded frequency-band indicates significant difference from 0 (P < 0.01, N = 92 subjects, and cluster corrected t test on test data). (B) WSC values are computed separately for the attentive and distracted viewing conditions. WSC values are the average across the 5 stimuli used in Experiment 2 with N = 29 subjects, and the shaded area around the average WSC values is the SEM over subjects. Gray-shaded frequency range indicates a significant difference between attending and distracted conditions (P < 0.01, paired t test N = 29, and cluster corrected).
Fig. 6. Within-subject coupling between EEG and other signal modalities during viewing of videos. WSC between EEG and gaze position, pupil size, HR, and respiration. The WSC is computed as the correlation between each signal modality and spatially filtered EEG. The spatial filters are found as the least square fit of scalp electrode potentials with the respective signal modality (computed separately for each frequency band; see supplement for variation across bands). (A) WSC computed by combining the 3 stimuli used in Experiment 1. Blue-shaded area indicates SEM over subjects. Gray-shaded frequency-band indicates significant difference from 0 (P < 0.01, N = 92 subjects, and cluster corrected t test on test data). (B) WSC values are computed separately for the attentive and distracted viewing conditions. WSC values are the average across the 5 stimuli used in Experiment 2 with N = 29 subjects, and the shaded area around the average WSC values is the SEM over subjects. Gray-shaded frequency range indicates a significant difference between attending and distracted conditions (P < 0.01, paired t test N = 29, and cluster corrected).CC BY · PubMed Central
Fig. 7. Predictions and test for saccade rate and head velocity. Here, using data from Experiment 2 from 29 subjects we test predictions on saccade rate and head velocity. (A) EEG has significant WSC with saccade rate but not head velocity, and neither difference with attentional state (attend: blue, distract: purple). (B) Significant ISC is observed for all subjects for saccade rate (29/29) but only a subset of subjects for head velocity (8/29). (C) Saccade rate, but not head velocity are predictive of memory performance across subjects. (D) Attention modulates ISC of saccade rate but not head velocity. Saccade rate is computed similarly to HR. Head velocity is calculated as the magnitude of the Hilbert transform of head position, combining horizontal, vertical, and depth directions with Euclidean norm. Significance established using the same methods as above.
Fig. 7. Predictions and test for saccade rate and head velocity. Here, using data from Experiment 2 from 29 subjects we test predictions on saccade rate and head velocity. (A) EEG has significant WSC with saccade rate but not head velocity, and neither difference with attentional state (attend: blue, distract: purple). (B) Significant ISC is observed for all subjects for saccade rate (29/29) but only a subset of subjects for head velocity (8/29). (C) Saccade rate, but not head velocity are predictive of memory performance across subjects. (D) Attention modulates ISC of saccade rate but not head velocity. Saccade rate is computed similarly to HR. Head velocity is calculated as the magnitude of the Hilbert transform of head position, combining horizontal, vertical, and depth directions with Euclidean norm. Significance established using the same methods as above.CC BY · PubMed Central
Fig. 8. Schematic summary of results and proposed common mechanism. A video stimulus is processed cognitively by the brain, similarly in different subjects (dashed arrows—indicate cognitive processing encompassing perception and cognition; obviously signals enter the brain through eyes and ears). This cognitive processing causes similar fluctuations in signals of the body that exhibit robust brain–body coupling (solid arrows). Therefore, brain, physiological, and behavioral signals correlate between subjects (bold arrows). Subjects' attention variably engages with the stimulus, such that cognitive processing as a mediator of common fluctuations varies, across subjects and time. This variation results in a co-modulation of ISC in different signals. Links indicated in blue have been measured in this study as correlation. Narrow arrows are hypothesized causal effects (note that brain–body links may be bidirectional, which is obvious for the eyes and is discussed below for the heart as well).
Fig. 8. Schematic summary of results and proposed common mechanism. A video stimulus is processed cognitively by the brain, similarly in different subjects (dashed arrows—indicate cognitive processing encompassing perception and cognition; obviously signals enter the brain through eyes and ears). This cognitive processing causes similar fluctuations in signals of the body that exhibit robust brain–body coupling (solid arrows). Therefore, brain, physiological, and behavioral signals correlate between subjects (bold arrows). Subjects' attention variably engages with the stimulus, such that cognitive processing as a mediator of common fluctuations varies, across subjects and time. This variation results in a co-modulation of ISC in different signals. Links indicated in blue have been measured in this study as correlation. Narrow arrows are hypothesized causal effects (note that brain–body links may be bidirectional, which is obvious for the eyes and is discussed below for the heart as well).CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 88 references

Research works printed but not linked: 1

  1. Parra, 2019, Correlated components analysis – extracting reliable dimensions in multivariate data, Neurons Behav Data Anal Theory, 2, 7125

Cited by

See all 91

Metadata

Authors
Madsen, J., & Parra, L. C.
Affiliations
City College of New York (US)
Corresponding author
Madsen, Jens
Venue
PNAS Nexus
Published
2022-03
Volume
1
Issue
1
Pages
pgac020-pgac020
ISSN
2752-6542 (ISSN-L)
OpenAlex
W4220920836
Cited by
91
References
88
Citations by year
2026: 20 · 2025: 27 · 2024: 19 · 2023: 21 · 2022: 4
FWCI
16.75
Type
journal article
Language
en
Open access
Locations
  • PNAS Nexus (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
National Science Foundation (DRL-1660548)
Topics
  • Social Sciences › Psychology › Social Psychology › Action Observation and Synchronization
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
Keywords
interpersonal synchrony, inter-subject correlation, attention, information recall, pupil size, heart rate, respiration, head movement, EEG

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes. https://app.cochat.ai/papers/10.1093/pnasnexus/pgac020
The paper
Madsen, J., & Parra, L. C. (2022). Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes. PNAS Nexus. https://doi.org/10.1093/pnasnexus/pgac020

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.