Multimodal Language Processing in Human Communication

Cited by (433)

Items 201–300 of 433

1. The place of language in multimodal communication in humans and other primates
   Michael Sharwood Smith · 2023 · Cognitive Systems Research
2. “Talking heads” in Portuguese sign and spoken languages
   Marisa Cruz, Sónia Frota · 2025 · Language and Cognition
3. Perceiving emotions in visual stimuli: social verbal context facilitates emotion detection of words but not of faces
   Stephanie S. A. H. Blom, Henk Aarts, Gün R. Semin · 2020 · Experimental Brain Research
4. Specific facial signals associate with categories of social actions conveyed through questions
   Naomi Nota, James P. Trujillo, Judith Holler · 2022
5. Introduction to the Special Issue on Iconicity and Sound Symbolism
   Aleksandra Ćwiek, Jody E. Kreiman, Susanne I. Fuchs · 2025 · The Journal of the Acoustical Society of America
6. Gesture Reduces Mapping Difficulties in the Development of Spatial Language Depending on the Complexity of Spatial Relations
   Ercenur Ünal, Kevser Kırbaşoğlu, Dilay Z. Karadöller, Beyza Sümer, Aslı Özyürek · 2025 · Cognitive Science
7. Data Collection in Multimodal Language and Communication Research: A Flexible Decision Framework
   Anastasia Bauer, Patrick C. Trettenbrein, Federica Amici, Aleksandra Ćwiek, Lisa-Marie Krause, Anna Kuder, Silva H. Ladewig, Marc Schulder, Petra B. Schumacher, Door Spruijt et al. · 2026 · Advances in Methods and Practices in Psychological Science
8. Comparative EEG reveals general and conspecific vocalization sensitivities in evolutionarily distant mammal species
   Boglárka Morvai, Marianna Boros, Elodie Ferrando, Lilla Magyari, Attila Andics · 2025 · NeuroImage
9. Psycholinguistic perspectives on face-to-face conversation
   Judith Holler, Anna K. Kuhlen · 2026 · Nature Reviews Psychology
10. Pragmatic and cognitive abilities in children and adolescents after pediatric brain tumor
    Agata Marchetti Guerrini, Ilaria Gabbatore, Geraldina Poggi, Maura Massimino, Alessandra Bardoni, Sandra Strazzer, Francesca M. Bosco · 2026 · Journal of Neurolinguistics
11. Assessment Battery for Communication (ABaCo): An Extension of Normative Data to Elderly Individuals
    Romina Angeleri, Ilaria Gabbatore, Ariele Merlini, Francesca M. Bosco · 2026 · Journal of Psycholinguistic Research
12. Multimodal pragmatic assessment in healthy aging: The interplay with Executive Functioning, Theory of Mind and Cognitive Reserve
    Alberto Parola, Ilaria Gabbatore, Dize Hilviu, Andrea Marini, Francesca M. Bosco · 2026 · Acta Psychologica
13. Presupposition processing in language comprehension
    Qi YANG, Xiaoming Jiang, Xiaolin ZHOU · 2022 · Advances in Psychological Science
14. Speakers’ Gestures and Semantic Analysis
    Alan J. Cienki · 2023 · Cognitive Semantics
15. How to test gesture-speech integration in ten minutes
    Matteo Maran, Hans Rutger Bosker · 2024 · Speech prosody
16. Gestural Iconicity and Alignment as Steps in the Evolution of Language
    Erica A. Cartmill · 2025 · Topics in Cognitive Science
17. Multimodal analysis of vocal emotion and prosody in subclinical depressed individuals
    Liusheng Wang, Haiyan Zhang, Dongfang Lin · 2025 · Current Psychology
18. Effective Design for Experiments on Small-Group Conversation: Insights From an Example Study
    Raluca Nicoras, Bryony Buck, Rosa-Linde Fischer, Matthew Godfrey, Lauren V. Hadley, Karolina Smeds, Graham Naylor · 2025 · American Journal of Audiology
19. Beating stress: Evidence for recalibration of word stress perception
    Ronny Bujok, David Peeters, Antje S. Meyer, Hans Rutger Bosker · 2025 · Attention Perception & Psychophysics
20. Trends in Usage-Based and Pragmatic Language Processing and Learning: A Bibliometric Analysis on Psycholinguistics and Second-Language Acquisition Studies
    Xiaoming Jiang · 2020 · IntechOpen eBooks
21. Investigating EFL Students’ Tense Errors in Writing Narrative Text
    Veneza Firda Resita Fanny, Juhansar Juhansar, Ulfa Amalia, Ari Prasetyoaji · 2022 · International Journal of Business Humanities Education and Social Sciences (IJBHES)
22. Getting others to share goods in Polish and Norwegian: Material and moral anchors for request conventions
    Paweł Urbanik · 2020 · Intercultural Pragmatics
23. Modality-specific dysfunctional neural processing of social-abstract and non-social-concrete information in schizophrenia
    Yifei He, Miriam Steines, Gebhard Sammer, Arne Nagels, Tilo T. J. Kircher, Benjamín Straube · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
24. Gesture’s body orientation modulates the N400 during semantic integration of gesture and visual sentence
    Yifei He, Svenja Luell, R. Muralikrishnan, Benjamin Straube, Arne Nagels · 2020 · bioRxiv (Cold Spring Harbor Laboratory)
25. Communicative efficiency in multimodal language
    Beata J. Grzyb, Stefan L. Frank, Gabriella Vigliocco · 2022
26. Development of Multimodal Turn Coordination in Conversations: Evidence for Adult-like behavior in Middle Childhood
    Abhishek Agrawal, Jing Liu, Kübra Bodur, Benoît Favre, Abdellah Fourtassi · 2023
27. Used to Be a Dime, Now It's a Dollar: Revised Speech Perception in Noise Key Word Predictability Revisited 40 Years On.
    Alexina Whitley, G. Naylor, L. Hadley · 2024 · Journal of Speech, Language and Hearing Research
28. Four-year-olds’ visuospatial cognitive abilities and their relation to observer‑viewpoint gestures across three communicative tasks
    Ulrich J. Boden, Friederike Kern, Sofia Koutalidis, Olga Abramov, Anne Németh, Stefan Kopp, Katharina J. Rohlfing · 2024 · Pragmatics & Cognition
29. Imagined versus practiced professional identity development: a multimodal (inter)action analysis of a Chinese novice english teacher
    Jing Kun Zhou, Lewis Tennant, Tui Matelau · 2024 · Multimodal Communication
30. Interpreters’ use of embodied resources to manage rapport challenge in video remote interpreting
    Dries Cavents, Jelena Vranjes, July De Wilde · 2025 · Perspectives
31. Adaptions in eye-movement behavior during face-to-face communication in noise
    Valeska Slomianka, Tobias May, Torsten Dau · 2025 · Frontiers in Psychology
32. Large Language Models as Coders of Pragmatic Competence in Healthy Aging: Preliminary Results on Reliability, Limits, and Implications for Human-Centered AI
    Arianna Boldi, Ilaria Gabbatore, Francesca M. Bosco · 2025 · Electronics
33. Will I speak louder if I see you struggling to understand? Speech modifications in response to non-verbal visual cues of listening effort
    Elena Gessa, Chiara Valzolgher, Elena Giovanelli, Tommaso Rosi, Clément Desoche, Alessandro Farnè, Francesco Pavani · 2026 · Psychonomic Bulletin & Review
34. Enhancing Pragmatic Communication in Healthy Aging: A Preliminary Investigation of the Long-Term Effects of the Advanced Cognitive Pragmatic Treatment
    Ilaria Gabbatore, Giorgia Vegna, Dize Hilviu, Ariele Merlini, Francesca M. Bosco · 2026 · Journal of Psycholinguistic Research
35. The Conversational Discourse Unit: Identification and Its Role in Conversational Turn-taking Management
    Junfei Hu, Liesbeth Degand · 2023 · Dialogue & Discourse
36. Identification of commonalities across different languages
    Kieran Green · 2023 · Frontiers in Language Sciences
37. Pictorial Irony and Sarcasm
    Albert N. Katz · 2023 · Cambridge University Press eBooks
38. Discontinuist and continuist approaches of language evolution… and beyond
    Lise Habib-Dassetto, Alban Lemasson, Cristel Portès, Marie Montant · 2023 · Revue de primatologie
39. Safe Multimodal Communication in Human-Robot Collaboration
    Davide Ferrari, Andrea Pupa, Alberto Signoretti, Cristian Secchi · 2024 · Springer proceedings in advanced robotics
40. Motion-Tracking Technology for the Study of Gesture
    James P. Trujillo · 2024 · Cambridge University Press eBooks
41. The human in the loop Perspectives and challenges for RoboCup 2050
    Alessandra Rossi, Maike Paetzel, Merel Keijsers, Michael Anderson, Susan Anderson, Daniel Barry, Jan Gutsche, Justin Hart, Luca Iocchi, A. A. Kokkelmans et al. · 2024 · Autonomous Robots
42. How to identify patient perception of AI voice robots in the follow-up scenario? A multimodal identity perception method based on deep learning
    Mingjie Liu, Kuiyou Chen, Qing Ye, Hong Wu · 2024 · Journal of Biomedical Informatics
43. Virtual reality as a tool to study language evolution
    Jonas Nölle, David Peeters · 2025 · Oxford University Press eBooks
44. The Impact of Word–Touch Synchrony on Early Language Development: Preliminary Findings From Korean Mother–Child Interactions
    Jongmin Jung, Eon‐Suk Ko · 2025 · Journal of Speech Language and Hearing Research
45. Speech planning depends on next-speaker selection: evidence from pupillometry in question–answer sequences in naturalistic triadic conversation
    Christoph Rühlemann, Mathias Barthel · 2025 · Discourse Processes
46. Which word gets the nuclear stress in a turn-at-talk?
    Christoph Rühlemann, Martin Schweinberger · 2021 · Journal of Pragmatics
47. On General Language Understanding
    David Schlangen · 2023 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
48. Visual Linguistics With R: A Practical Introduction to Quantitative Interactional Linguistics
    Christoph Rühlemann · 2020
49. Sociosemiotic approach to teaching foreign language listening: linguodidactic implications of adopting views on interrelation of verbal and non-verbal semiotic resources in communication.
    Mikhail Vyacheslavovich Danilin · 2021 · Samara Journal of Science
50. Towards the Description of the Cultural Images Translation of Li Qingzhao's: A Study of the Medio – Translatology
    Cheng Liang, Sang Seong Goh · 2022 · World Journal of English Language
51. Real-time inference in communication across cultures: evidence from a non-industrialized society
    Rachel Anna Ryskin, Miguel Angel Salinas, Steven T. Piantadosi, Edward A. F. Gibson · 2021
52. Language is less arbitrary than one thinks: Iconicity and Indexicality in real-world language learning and processing
    Margherita Murgiano, Yasamin Motamedi, Gabriella Vigliocco · 2020
53. Oxytocin facilitates communicative adjustments by upregulating broadband aperiodic activity in the prefrontal cortex
    Rui Liu, Arjen Stolk, Miriam de Boer, Robert Oostenveld, Ivan Toni · 2021
54. Introduction
    Anja Stukenbrock, Elisabeth Zima · 2025 · Pragmatics & beyond. New series
55. Contribution of working memory to gesture production in toddlers
    Naziye Güneş‐Acar, İ. Ercan, Ayli̇n C. Küntay, Ayhan Aksu-Koç · 2021 · Cognitive Development
56. Multimodal Language Data and Platform Construction for LCTLs Teaching
    Yuan Wang, Kai Jin, Wuying Liu · 2023 · International Conference on Machine Learning and Computing (ICMLC)
57. Visual channel facilitates the comprehension of the intonation of Brazilian Portuguese wh-questions and wh-exclamations: evidence from congruent and incongruent stimuli
    Luma da Silva Miranda, João Antônio de Moraes, Albert Olivier Blaise Rilliard · 2024 · Language and Cognition
58. The role of pointing gestures and eye gaze in second language vocabulary learning
    Paula Janjić, Gozdem Arikan, Harmen B. Gudde, Joseph J.C. Murphy, Laksha Sivaram, Kenny R. Coventry · 2024 · Discourse Processes
59. Facial expressions in different communication settings: A case of whispering and speaking with a face mask in Farsi
    Nasim Mahdinazhad Sardhaei, Marzena Żygis, Hamid Sharifzadeh · 2024 · Language and Cognition
60. Construction Grammar, multimodal communication, and design features of language
    Arie Verhagen · 2025 · Constructional approaches to language
61. Retellings of emotional videos: the dynamics of gesture characteristics in diverse emotion categories
    Süleyman Can Ceylan, Demet Özer, Tilbe Göksun · 2025 · Cognition & Emotion
62. Effects of subliminal emotional facial expressions on language comprehension as revealed by event-related brain potentials
    Miguel Rubianes, Laura Jiménez‐Ortega, Francisco Muñoz, Linda Drijvers, Tatiana Almeida-Rivera, José Sánchez-García, Sabela Fondevila, Pilar Casado, Manuel Martín‐Loeches · 2025 · Scientific Reports
63. Prior expectations guide multisensory integration during face-to-face communication
    Giulia Mazzi, Ambra Ferrari, Maria Laura Mencaroni, Chiara Valzolgher, Mirko Tommasini, Francesco Pavani, Stefania Benetti · 2025 · PLoS Computational Biology
64. The lexicalization of emojis: the influence of frequency and functions of emojis in sentences on this process—a study based on eye movement tracking
    Wanhong Lu, Haoge Du, Feng Gu, Jianghua Han · 2025 · Frontiers in Psychology
65. Head Gestures Do Not Serve as Precursors of Prosodic Focus Marking in the Second Language as They Do in the First Language
    Lieke van Maastricht, Núria Esteve‐Gibert · 2025 · Language Learning
66. Alignment and Repair Work in Extended Reality
    Noak Petersson, Donald McMillan, Harko Verhagen, Jordi Solsona Belenguer · 2025 · International Academic Mindtrek Conference (Mindtrek)
67. The neurobiology of human sequential signal prediction: Insights from language, music, and mathematics
    Tomoya Nakai, Tatsuya Daikoku, Yohei Oseki · 2025 · Neuroscience Research
68. The timing of an avatar’s beat gestures biases lexical stress perception in vocoded speech
    Matteo Maran, Renske M. J. Uilenreef, Roos Rossen, Hans Rutger Bosker · 2025 · Applied Psycholinguistics
69. Structural alignment leads to lower cognitive load in a collaborative task
    Marek Placiński, Theresa Matzinger · 2025 · Interaction Studies Social Behaviour and Communication in Biological and Artificial Systems
70. Children Sustain Their Attention on Spatial Scenes When Planning to Describe Spatial Relations Multimodally in Speech and Gesture
    Ercenur Ünal, Dilay Z. Karadöller, Aslı Özyürek · 2026 · Developmental Science
71. Hallmarks of social action in the vocal turn-taking of wild common marmosets (Callithrix jacchus)
    Filipa Abreu, Nicola Schiel, Antonio Souto, Simone Pika · 2026 · Scientific Reports
72. Cognitive Vulnerability and Multimodal Communicative-Pragmatic Difficulty in Subclinical Aging: A MoCA-Guided Exploration
    Giorgia Vegna, Ilaria Gabbatore, Andrea Marini, Francesca M. Bosco · 2026 · Experimental Aging Research
73. Communicative Function of Eye Blinks of Virtual Avatars May Not Translate onto Physical Platforms
    Djamari Oetringer, Pieter Wolfert, Jorre Deschuyteneer, Serge Thill, Tony Belpaeme · 2021 · ACM/IEEE International Conference on Human-Robot Interaction (HRI)
74. The role of rhythm in auditory speech understanding
    Liangjie CHEN, Lei LIU, Zhongshu Ge, Xiaodong YANG, Liang Li · 2022 · Advances in Psychological Science
75. Laughter Meaning Construction and Use in Development: Children and Spoken Dialogue Systems
    Chiara Mazzocconi · 2022 · Lecture notes in computer science
76. The effect of visual speech information on linguistic release from masking
    Brittany T. Williams, Navin Viswanathan, Susanne M. Brouwer · 2023 · The Journal of the Acoustical Society of America
77. From sentential syntax to syntax of interaction
    Pavel Ozerov · 2023 · Journal of Speech Sciences
78. The Effect of Familiarity on Conversation Turns in Dyadic Conversation Among Chinese Older Adults
    Wen-Shin Chang, Meng-Ju Tsai · 2023 · SAGE Open
79. Information presentation and gaze-following for grounding in simulated anaesthesia emergencies: A multimodal analysis
    Keiko Tsuchiya, Hitoshi Sato, Kyota Nakamura, Takeru Abe, Arisa Fujii, Atsushi Miyazaki, Yuka Okuyama, Daisuke Kuwabara · 2023 · Language and Health
80. Cognition-Emotion Interaction during L2 Sentence Comprehension: The Correlation of ERP and GSR Responses to Sense Combinations
    Reza Pishghadam, Shaghayegh Shayesteh, Farveh Daneshvarfard, Nasim Boustani, Zahra Seyednozadi, Mohammad Zabetipour, Morteza Pishghadam · 2024 · Journal of Psycholinguistic Research
81. The effect of native language and bilingualism on multimodal perception in speech: A study of audio-aerotactile integration
    Haruka Saito, Mark Tiede, D. H. Whalen, Lucie Ménard · 2024 · The Journal of the Acoustical Society of America
82. Selective View of the Last 50 Years of the Experimental Psycholinguistics of Idiom, Metaphor, and Irony: A Commentary
    Albert N. Katz · 2024 · Discourse Processes
83. Histograms and density plots
    2020 · John Benjamins Publishing Company eBooks
84. Situating multimodal language acquisition in the real-world: A commentary on Karadöller, Sümer and Özyürek
    Gabriella Vigliocco · 2025 · First Language
85. Cross-situational learning of sign-like gestures in children and adults: a behavioural and event-related potential study
    Arianna Colombani, Varghese Peter, Qian Yin Mai, Amanda Saksida, Natalie Boll‐Avetisyan, Outi Tuomainen, Mridula Sharma · 2025 · Language Cognition and Neuroscience
86. Visible lexical stress cues on the face do not influence audiovisual speech perception
    Ronny Bujok, Antje S. Meyer, Hans Rutger Bosker · 2022 · Speech prosody
87. Improving the communication ethics of dental students through learning about local culture
    Silvia Rosa, Ivonne Olivia, Yasmadi Yasmadi, Endang Fatmawati · 2022 · Cypriot Journal of Educational Sciences
88. Pointing in gesture and sign
    Kensy Cooperrider, Kate Mesh · 2020
89. The role of co-speech gestures in retrieval and prediction during naturalistic multimodal narrative processing
    Sergio Osorio, Benjamin Straube, Lars Meyer, Yifei He · 2022
90. Understanding Children's Multimodal Conversational Development: Challenges and Opportunities
    Abdellah Fourtassi · 2023
91. Vocal-visual combinations in wild chimpanzees
    Joseph G. Mine, Claudia Wilke, Chiara Zulberti, Melika Bejhati, Sabine Stoll, Zarin P Machanda, Andri Manser, Katie E. Slocombe, Simon William Townsend · 2023 · bioRxiv (Cold Spring Harbor Laboratory)
92. Editorial: Multimodal communication and multimodal computing
    Alexander Mehler, Andy Lücking, Tiansi Dong · 2023 · Frontiers in Artificial Intelligence
93. Individual variations in McGurk illusion susceptibility reflect different integration-segregation strategies of audiovisual speech perception
    Chenjie Dong, Zhengye Wang, Ruqin Li, Suiping Wang · 2023 · bioRxiv (Cold Spring Harbor Laboratory)
94. Models of human learning should capture the complexity of natural communication
    Jessica Elizabeth Kosie, Mira L. Nencheva, Justin A. Junge, Casey Lew‐Williams · 2024
95. Predicting meaning in the dyad.
    Emilia Kerr, Benjamin Morillon, Kristof Strijkers · 2024
96. Synergistic Perception: Adaptive Multimodal Sentiment Analysis via Multi-Task Learning
    Tao Cao, Baojian Ren, Hongfei Cao, Ang Chen, Shuchen Bai, Na Liu · 2025
97. The Point of Pointing: Deictic Gestures Modulate Attentional Shifts and Cognitive Load in Simultaneous Interpreting
    PAN Dongpeng, Kilian Seeber · 2026 · Research Square
98. Moving Emotions: An examination of the role of facial mimicry in emotion understanding.
    Stephanie S. A. H. Blom · 2020
99. Speech and Gaze during Parent-Child Interactions
    Gijs A. Holleman, Ignace T. C. Hooge, Jorg Huijding, Maja Deković, Chantal Kemner, Roy S. Hessels · 2020 · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion)
100. Testing the prerequisites for hand gestures playing a potential predictive role in communication: A corpus study of the speech-gesture timing in English conversation.
     N.C.N.P. Nguyen Phuc · 2020