Modeling Team-level Multimodal Dynamics during Multiparty Collaboration
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
We adopt a multimodal approach to investigating team interactions in the context of remote collaborative problem solving (CPS). Our goal is to understand multimodal patterns that emerge and their relation with collaborative outcomes. We measured speech rate, body movement, and galvanic skin response from 101 triads (303 participants) who used video conferencing software to collaboratively solve challenging levels in an educational physics game. We use multi-dimensional recurrence quantification analysis (MdRQA) to quantify patterns of team-level regularity, or repeated patterns of activity in these three modalities. We found that teams exhibit significant regularity above chance baselines. Regularity was unaffected by task factors. but had a quadratic relationship with session time in that it initially increased but then decreased as the session progressed. Importantly, teams that produce more varied behavioral patterns (irregularity) reported higher emotional valence and performed better on a subset of the problem solving tasks. Regularity did not predict arousal or subjective perceptions of the collaboration. We discuss implications of our findings for the design of systems that aim to improve collaborative outcomes by monitoring the ongoing collaboration and intervening accordingly.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Autonomic nervous system activity distinguishes among emotions
Ekman, P., Levenson, R. W., & Friesen, W. V. (1983) · Science · doi:10.1126/science.6612338 · Cited by 2,692
The Construction of Shared Knowledge in Collaborative Problem Solving.
Roschelle, J., & Teasley, S. D. (1995) · Computer Supported Collaborative Learning · doi:10.1007/978-3-642-85098-1_5 · Cited by 2,274 · cited 2×
“Effective CPS is dependent on a team’s ability to establish common ground [74], jointly develop a solution that accommodates multiple perspectives [74], and monitor progress toward the goal [29, 75].”
BackgroundThe Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818 · cited 1×
“Social cues and emotional aspects of the collaboration are conveyed through acoustic-prosodic information [28] (e.g., pitch) and facial expressions [26].”
BackgroundDefining Twenty-First Century Skills
Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012) · Assessment and Teaching of 21st Century Skills · doi:10.1007/978-94-007-2324-5_2 · Cited by 1,809
Critchley, H. D., Elliott, R., Mathias, C. J., & Dolan, R. J. (2000) · The Journal of Neuroscience · doi:10.1523/jneurosci.20-08-03033.2000 · Cited by 813
Mutual interpersonal postural constraints are involved in cooperative conversation.
Shockley, K., Santana, M.-V., & Fowler, C. A. (2003) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.29.2.326 · Cited by 681 · cited 2×
“RQA has traditionally been used to quantify recurrent patterns within one signal (e.g., [89, 93]) or to study alignment between two signals (cross-RQA (e.g., [18, 40, 46, 50, 71, 73, 83]).”
BackgroundMethodRichardson, D. C., & Dale, R. (2005) · Cognitive Science · doi:10.1207/s15516709cog0000_29 · Cited by 641
Synchronized arousal between performers and related spectators in a fire-walking ritual
Konvalinka, I., Xygalatas, D., Bulbulia, J., Schjødt, U., Jegindø, E.-M., Wallot, S., Van Orden, G., & Roepstorff, A. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1016955108 · Cited by 632 · cited 2×
“RQA has traditionally been used to quantify recurrent patterns within one signal (e.g., [89, 93]) or to study alignment between two signals (cross-RQA (e.g., [18, 40, 46, 50, 71, 73, 83]).”
BackgroundMethodInterpersonal autonomic physiology: a systematic review of the literature
Palumbo, R. V., Marraccini, M. E., Weyandt, L. L., Wilder-Smith, O., McGee, H. A., Liu, S., & Goodwin, M. S. (2017) · Personality and Social Psychology Review · doi:10.1177/1088868316628405 · Cited by 566
Group Cohesion and Performance: A Meta-Analysis
Evans, C. R., & Dion, K. L. (1991) · Small Group Research · doi:10.1177/1046496491222002 · Cited by 535 · cited 2×
“Successful communication relies on socio-cognitive-affective processes such as conversational grounding, turn-taking, emotion co-regulation, and joint attention, to name a few [8, 27, 39, 48, 51, 59].”
BackgroundEmotional coregulation in close relationships
Butler, E. A., & Randall, A. K. (2013) · Emotion Review · doi:10.1177/1754073912451630 · Cited by 476
Laughlin, P. R., Hatch, E. C., Silver, J. S., & Boh, L. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.90.4.644 · Cited by 390 · cited 1×
“With respect to the first, prior work has investigated how team composition (e.g., group size [55, 56], ability/motivation cohesiveness [27, 37, 52]) and features of the task (e.g., problem structure [54, 81]) predict team performance.”
BackgroundModeling dominance in group conversations using nonverbal activity cues
Jayagopi, D. B., Hung, H., Yeo, C., & Gatica-Perez, D. (2009) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/tasl.2008.2008238 · Cited by 217
Assessing cross-partner associations in physiological responses via coupled oscillator models.
Helm, J. L., Sbarra, D., & Ferrer, E. (2012) · Emotion · doi:10.1037/a0025036 · Cited by 179 · cited 2×
“RQA has traditionally been used to quantify recurrent patterns within one signal (e.g., [89, 93]) or to study alignment between two signals (cross-RQA (e.g., [18, 40, 46, 50, 71, 73, 83]).”
BackgroundMethodAutomatic Detection of Learning-Centered Affective States in the Wild
Bosch, N., D'Mello, S., Baker, R., Ocumpaugh, J., Shute, V., Ventura, M., Wang, L., & Zhao, W. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701397 · Cited by 178 · cited 1×
“, Newton’s laws, energy transfer, and torque) and has been found to be highly engaging [13, 60].”
BackgroundSelf-citationWallot, S., Roepstorff, A., & Mønster, D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.01835 · Cited by 140 · cited 3×
“We then use multidimensional recurrence quantification analysis [92] (MdRQA) to jointly model three team-level signals: speech rate to index verbal contribution, body movement features to index nonverbal communication, and galvanic skin response as a measure of autonomic arousal.”
MethodInfluentialMovement Coordination during Conversation
Latif, N., Barbosa, A. V., Vatiokiotis-Bateson, E., Castelhano, M. S., & Munhall, K. G. (2014) · PLoS ONE · doi:10.1371/journal.pone.0105036 · Cited by 72
Social psychophysiological compliance in a four-person research team
Henning, R. A., Armstead, A. G., & Ferris, J. K. (2009) · Applied Ergonomics · doi:10.1016/j.apergo.2009.04.009 · Cited by 69
Peñarroja, V., Orengo, V., & Zornoza, A. (2017) · Journal of Applied Social Psychology · doi:10.1111/jasp.12449 · Cited by 47
Cited by
Ouyang, F., Xu, W., & Cukurova, M. (2023) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-023-09387-z · Cited by 124
Bobko, P., Hirshfield, L., Eloy, L., Spencer, C., Doherty, E., Driscoll, J., & Obolsky, H. (2023) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2022.2086644 · Cited by 33
Self-citationD’Mello, S. K., Duran, N., Michaels, A., & Stewart, A. E. B. (2024) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09387-6 · Cited by 14
Self-citation
Metadata
- Authors
- Eloy, L., E.B. Stewart, A., Jean Amon, M., Reinhardt, C., Michaels, A., Sun, C., Shute, V., Duran, N. D., & D'Mello, S.
- Venue
- 2019 International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2019-10-14
- Pages
- 244-258
- OpenAlex
- W2981040244
- Cited by
- 35
- References
- 19
- FWCI
- 8.04
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.