Hand and mind: what gestures reveal about thought
The DOI matches its Crossref record.
Abstract
What is the relation between gestures and speech? In terms of symbolic forms, of course, the spontaneous and unwitting gestures we make while talking differ sharply from spoken language itself. Whereas spoken language is linear, segmented, standardized, and arbitrary, gestures are global, synthetic, idiosyncratic, and imagistic. In Hand and Mind, David McNeill presents a bold theory of the essential unity of speech and the gestures that accompany it. This long-awaited, provocative study argues that the unity of gestures and language far exceeds the surface level of speech noted by previous researchers and in fact also includes the semantic and pragmatic levels of language. In effect, the whole concept of language must be altered to take into account the nonsegmented, instantaneous, and holistic images conveyed by gestures. McNeill and his colleagues carefully devised a standard methodology for examining the speech and gesture behavior of individuals engaged in narrative discourse. A research subject is shown a cartoon like the 1950 Canary Row--a classic Sylvester and Tweedy Bird caper that features Sylvester climbing up a downspout, swallowing a bowling ball and slamming into a brick wall. After watching the cartoon, the subject is videotaped recounting the story from memory to a listener who has not seen the cartoon. Painstaking analysis of the videotapes revealed that although the research subjects--children as well as adults, some neurologically impaired--represented a wide variety of linguistic groupings, the gestures of people speaking English and a half dozen other languages manifest the same principles. Relying on data from more than ten years of research, McNeill shows thatgestures do not simply form a part of what is said and meant but have an impact on thought itself. He persuasively argues that because gestures directly transfer mental images to visible forms, conveying ideas that language cannot always express, we must examine language and gesture
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
A survey of socially interactive robots
Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(02)00372-x · Cited by 3,162
Gallagher, S. (2005) · doi:10.1093/0199271941.001.0001 · Cited by 3,311
Action and embodiment within situated human interaction
Goodwin, C. (2000) · Journal of Pragmatics · doi:10.1016/s0378-2166(99)00096-x · Cited by 3,073
Cognitive Grammar: A Basic Introduction
Langacker, R. (2008) · doi:10.1093/acprof:oso/9780195331967.001.0001 · Cited by 2,987
Multimodality: A Social Semiotic Approach to Contemporary Communication
Forceville, C. J. (2011) · Journal of Pragmatics · doi:10.1016/j.pragma.2011.06.013 · Cited by 2,771
McNeill, D. (2005) · doi:10.7208/chicago/9780226514642.001.0001 · Cited by 2,157
The Visual Analysis of Human Movement: A Survey
Gavrila, D. M. (1999) · Computer Vision and Image Understanding · doi:10.1006/cviu.1998.0716 · Cited by 1,782
Vision based hand gesture recognition for human computer interaction: a survey
Rautaray, S. S., & Agrawal, A. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9356-9 · Cited by 1,526
Grammar, Gesture, and Meaning in American Sign Language
Liddell, S. K. (2003) · Cambridge University Press eBooks · doi:10.1017/cbo9780511615054 · Cited by 1,387
Establishing and maintaining long-term human-computer relationships
Bickmore, T. W., & Picard, R. W. (2005) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1067860.1067867 · Cited by 1,333
User-defined gestures for surface computing
Wobbrock, J. O., Morris, M. R., & Wilson, A. D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518866 · Cited by 1,317
Negative Feedback as Regulation and Second Language Learning in the Zone of Proximal Development
ALJAAFREH, A., & LANTOLF, J. P. (1994) · The Modern Language Journal · doi:10.1111/j.1540-4781.1994.tb02064.x · Cited by 1,199
Arbib, M. A. (2005) · Behavioral and Brain Sciences · doi:10.1017/s0140525x05000038 · Cited by 1,164
“Vygotsky’s Neglected Legacy”: Cultural-Historical Activity Theory
Roth, W.-M., & Lee, Y.-J. (2007) · Review of Educational Research · doi:10.3102/0034654306298273 · Cited by 1,113
Language as shaped by the brain
Christiansen, M. H., & Chater, N. (2008) · Behavioral and Brain Sciences · doi:10.1017/s0140525x08004998 · Cited by 1,063
Mondada, L. (2018) · Research on Language and Social Interaction · doi:10.1080/08351813.2018.1413878 · Cited by 1,056
Tomasello, M., Carpenter, M., & Liszkowski, U. (2007) · Child Development · doi:10.1111/j.1467-8624.2007.01025.x · Cited by 1,039
Multimodal human–computer interaction: A survey
Jaimes, A., & Sebe, N. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.019 · Cited by 965
Visible embodiment: Gestures as simulated action
HOSTETTER, A. B., & ALIBALI, M. W. (2008) · Psychonomic Bulletin & Review · doi:10.3758/pbr.15.3.495 · Cited by 861
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Children Creating Core Properties of Language: Evidence from an Emerging Sign Language in Nicaragua
Senghas, A., Kita, S., & Özyürek, A. (2004) · Science · doi:10.1126/science.1100199 · Cited by 779
Challenges of multimodality: Language and the body in social interaction
Mondada, L. (2016) · Journal of Sociolinguistics · doi:10.1111/josl.1_12177 · Cited by 768
(2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202 · Cited by 763
BEAT: the Behavior Expression Animation Toolkit
Cassell, J., Vilhjálmsson, H. H., & Bickmore, T. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383315 · Cited by 739
Embodiment in Mathematics Teaching and Learning: Evidence From Learners' and Teachers' Gestures
Alibali, M. W., & Nathan, M. J. (2012) · Journal of the Learning Sciences · doi:10.1080/10508406.2011.611446 · Cited by 691
Ten myths of multimodal interaction
Oviatt, S. (1999) · Communications of the ACM · doi:10.1145/319382.319398 · Cited by 689
Action-based language: A theory of language acquisition, comprehension, and production
Glenberg, A. M., & Gallese, V. (2012) · Cortex · doi:10.1016/j.cortex.2011.04.010 · Cited by 680
Two Sides of the Same Coin: Speech and Gesture Mutually Interact to Enhance Comprehension
Kelly, S. D., Özyürek, A., & Maris, E. (2010) · Psychological Science · doi:10.1177/0956797609357327 · Cited by 669
Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., & Stone, M. (1994) · Proceedings of the 21st annual conference on Computer graphics and interactive techniques - SIGGRAPH '94 · doi:10.1145/192161.192272 · Cited by 665
Gesture and the process of speech production: We think, therefore we gesture
Alibali, M. W., Kita, S., & Young, A. J. (2000) · Language and Cognitive Processes · doi:10.1080/016909600750040571 · Cited by 633
Intercultural competence: Interpersonal communication across cultures
Cranmer, D. J. (1995) · International Journal of Intercultural Relations · doi:10.1016/0147-1767(95)90057-8 · Cited by 629
Affective Body Expression Perception and Recognition: A Survey
Kleinsmith, A., & Bianchi-Berthouze, N. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.16 · Cited by 607
Fitch, W. T. (2010) · doi:10.1017/cbo9780511817779 · Cited by 594
Social Eye Gaze in Human-Robot Interaction: A Review
Admoni, H., & Scassellati, B. (2017) · Journal of Human-Robot Interaction · doi:10.5898/jhri.6.1.admoni · Cited by 588
Parametric hidden Markov models for gesture recognition
Wilson, A. D., & Bobick, A. F. (1999) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.790429 · Cited by 574
Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning
Ong, S. C. W., & Ranganath, S. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.112 · Cited by 562
When do gestures communicate? A meta-analysis.
Hostetter, A. B. (2011) · Psychological Bulletin · doi:10.1037/a0022128 · Cited by 558
From Perception to Meaning: Image Schemas in Cognitive Linguistics
(2005) · doi:10.1515/9783110197532 · Cited by 533
Billinghurst, M., Clark, A., & Lee, G. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781601989215 · Cited by 529
Lexical gestures and lexical access: a process model
Krauss, R. M., Chen, Y., & Gottesman, R. F. (2000) · Language and Gesture · doi:10.1017/cbo9780511620850.017 · Cited by 517
Dead Geographies—And How to Make Them Live
Thrift, N., & Dewsbury, J.-D. (2000) · Environment and Planning D: Society and Space · doi:10.1068/d1804ed · Cited by 511
Gestures: Their Role in Teaching and Learning
Roth, W.-M. (2001) · Review of Educational Research · doi:10.3102/00346543071003365 · Cited by 496
Oviatt, S. (2007) · Human Factors and Ergonomics · doi:10.1201/9781410615862.ch21 · Cited by 485
Gesture, Speech, and Lexical Access: The Role of Lexical Movements in Speech Production
Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996) · Psychological Science · doi:10.1111/j.1467-9280.1996.tb00364.x · Cited by 478
Embodied cognition and the magical future of interaction design
Kirsh, D. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2442106.2442109 · Cited by 465
A review of verbal and non-verbal human–robot interactive communication
Mavridis, N. (2015) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2014.09.031 · Cited by 453
Gesture and speech in interaction: An overview
Wagner, P., Malisz, Z., & Kopp, S. (2014) · Speech Communication · doi:10.1016/j.specom.2013.09.008 · Cited by 456
Ape gestures and language evolution
Pollick, A. S., & de Waal, F. B. M. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0702624104 · Cited by 452
Pragmatics and Non-Verbal Communication
Wharton, T. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511635649 · Cited by 450
On the Meaning of Words and Dinosaur Bones: Lexical Knowledge Without a Lexicon
Elman, J. L. (2009) · Cognitive Science · doi:10.1111/j.1551-6709.2009.01023.x · Cited by 446
Iconicity as a general property of language: evidence from spoken and signed languages
Perniss, P., Thompson, R. L., & Vigliocco, G. (2010) · Frontiers in Psychology · doi:10.3389/fpsyg.2010.00227 · Cited by 442
Metaphor in Pictures and Multimodal Representations
Forceville, C. (2008) · The Cambridge Handbook of Metaphor and Thought · doi:10.1017/cbo9780511816802.028 · Cited by 438
Gesture's Role in Speaking, Learning, and Creating Language
Goldin-Meadow, S., & Alibali, M. W. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143802 · Cited by 437
Wilson, D., & Wharton, T. (2006) · Journal of Pragmatics · doi:10.1016/j.pragma.2005.04.012 · Cited by 436
Multimodal Language Processing in Human Communication
Holler, J., & Levinson, S. C. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2019.05.006 · Cited by 441
The tangle of space and time in human cognition
Núñez, R., & Cooperrider, K. (2013) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2013.03.008 · Cited by 427
Research-Based Design of Pedagogical Agent Roles: a Review, Progress, and Recommendations
Kim, Y., & Baylor, A. L. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0055-y · Cited by 422
SOCIOCULTURAL THEORY AND L2: State of the Art
Lantolf, J. P. (2006) · Studies in Second Language Acquisition · doi:10.1017/s0272263106060037 · Cited by 422
Immersive VR and Education: Embodied Design Principles That Include Gesture and Hand Controls
Johnson-Glenberg, M. C. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00081 · Cited by 416
To err is human(-like): effects of robot gesture on perceived anthropomorphism and likability
Salem, M., Eyssel, F., Rohlfing, K., Kopp, S., & Joublin, F. (2013) · International Journal of Social Robotics · doi:10.1007/s12369-013-0196-9 · Cited by 407
AREAL LINGUISTICS AND MAINLAND SOUTHEAST ASIA
Enfield, N. J. (2005) · Annual Review of Anthropology · doi:10.1146/annurev.anthro.34.081804.120406 · Cited by 410
Looking at people: sensing for ubiquitous and wearable computing
Pentland, A. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824823 · Cited by 403
Bucholtz, M., & Hall, K. (2016) · Sociolinguistics · doi:10.1017/cbo9781107449787.009 · Cited by 401
Making children gesture brings out implicit knowledge and leads to learning
Broaders, S. C., Cook, S. W., Mitchell, Z., & Goldin-Meadow, S. (2007) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.136.4.539 · Cited by 397
Alibali, M. W., Heath, D. C., & Myers, H. J. (2001) · Journal of Memory and Language · doi:10.1006/jmla.2000.2752 · Cited by 397
Neural evidence for the interplay between language, gesture, and action: A review
Willems, R. M., & Hagoort, P. (2007) · Brain and Language · doi:10.1016/j.bandl.2007.03.004 · Cited by 393
The EMOTE model for effort and shape
Chi, D., Costa, M., Zhao, L., & Badler, N. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.352172 · Cited by 393
Krahmer, E., & Swerts, M. (2007) · Journal of Memory and Language · doi:10.1016/j.jml.2007.06.005 · Cited by 386
The origin of human multi-modal communication
Levinson, S. C., & Holler, J. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0302 · Cited by 385
Beyond languages, beyond modalities: transforming the study of semiotic repertoires
Kusters, A., Spotti, M., Swanwick, R., & Tapio, E. (2017) · International Journal of Multilingualism · doi:10.1080/14790718.2017.1321651 · Cited by 379
Oviatt, S., Cohen, P., Wu, L., Duncan, L., Suhm, B., Bers, J., Holzman, T., Winograd, T., Landay, J., Larson, J., et al. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1504_1 · Cited by 375
From communication to language in two modalities
Iverson, J. M., Capirci, O., & Caselli, M. C. (1994) · Cognitive Development · doi:10.1016/0885-2014(94)90018-3 · Cited by 373
(2008) · Routledge eBooks · doi:10.4324/9780203892299 · Cited by 371
Anchoring, Iconicity, and Orientation in Guugu Yimithirr Pointing Gestures
Haviland, J. B. (1993) · Journal of Linguistic Anthropology · doi:10.1525/jlin.1993.3.1.3 · Cited by 371
Instructing people for training gestural interactive systems
Fothergill, S., Mentis, H., Kohli, P., & Nowozin, S. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208303 · Cited by 368
Depicting as a method of communication.
Clark, H. H. (2016) · Psychological Review · doi:10.1037/rev0000026 · Cited by 365
Gesture, sign, and language: The coming of age of sign language and gesture studies
Goldin-Meadow, S., & Brentari, D. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15001247 · Cited by 363
The capacity for music: What is it, and what’s special about it?
Jackendoff, R., & Lerdahl, F. (2006) · Cognition · doi:10.1016/j.cognition.2005.11.005 · Cited by 362
Integration and synchronization of input modes during multimodal human-computer interaction
Oviatt, S., DeAngeli, A., & Kuhn, K. (1997) · Proceedings of the ACM SIGCHI Conference on Human factors in computing systems · doi:10.1145/258549.258821 · Cited by 360
How Words Mean: Lexical Concepts, Cognitive Models, and Meaning Construction
Evans, V. (2009) · doi:10.1093/acprof:oso/9780199234660.001.0001 · Cited by 357
(2008) · Gesture Studies · doi:10.1075/gs.3 · Cited by 348
The bridge of iconicity: from a world of experience to the experience of language
Perniss, P., & Vigliocco, G. (2014) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2013.0300 · Cited by 345
A Competent Speaker Who Can't Speak: The Social Life of Aphasia
GOODWIN, C. (2004) · Journal of Linguistic Anthropology · doi:10.1525/jlin.2004.14.2.151 · Cited by 342
How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis.
Kita, S., Alibali, M. W., & Chu, M. (2017) · Psychological Review · doi:10.1037/rev0000059 · Cited by 342
When Language Meets Action: The Neural Integration of Gesture and Speech
Willems, R. M., Özyürek, A., & Hagoort, P. (2007) · Cerebral Cortex · doi:10.1093/cercor/bhl141 · Cited by 342
Human Conversation as a System Framework: Designing Embodied Conversational Agents
Cassell, J., Bickmore, T., Campbell, L., Vilhjálmsson, H., & Yan, H. (2000) · Embodied Conversational Agents · doi:10.7551/mitpress/2697.003.0004 · Cited by 342
Hagoort, P., & van Berkum, J. (2007) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2007.2089 · Cited by 338
The identification of target and source in pictorial metaphors
Forceville, C. (2002) · Journal of Pragmatics · doi:10.1016/s0378-2166(01)00007-8 · Cited by 337
Metaphor, Gesture, and Thought
Cienki, A., & Müller, C. (2008) · The Cambridge Handbook of Metaphor and Thought · doi:10.1017/cbo9780511816802.029 · Cited by 336
Learning Individual Styles of Conversational Gesture
Ginosar, S., Bar, A., Kohavi, G., Chan, C., Owens, A., & Malik, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00361 · Cited by 335
Nuckolls, J. B. (1999) · Annual Review of Anthropology · doi:10.1146/annurev.anthro.28.1.225 · Cited by 333
From manual gesture to speech: A gradual transition.
Gentilucci, M., & Corballis, M. C. (2006) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2006.02.004 · Cited by 332
Action’s Influence on Thought: The Case of Gesture
Goldin-Meadow, S., & Beilock, S. L. (2010) · Perspectives on Psychological Science · doi:10.1177/1745691610388764 · Cited by 329
Speech gesture generation from the trimodal context of text, audio, and speaker identity
Yoon, Y., Cha, B., Lee, J.-H., Jang, M., Lee, J., Kim, J., & Lee, G. (2020) · ACM Transactions on Graphics · doi:10.1145/3414685.3417838 · Cited by 328
Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks
Fussell, S. R., Setlock, L. D., Yang, J., Ou, J., Mauer, E., & Kramer, A. D. I. (2004) · Human–Computer Interaction · doi:10.1207/s15327051hci1903_3 · Cited by 328
GOLDINMEADOW, S., & WAGNER, S. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.03.006 · Cited by 327
How Robots Influence Humans: A Survey of Nonverbal Communication in Social Human–Robot Interaction
Saunderson, S., & Nejat, G. (2019) · International Journal of Social Robotics · doi:10.1007/s12369-019-00523-0 · Cited by 325
Social Dialongue with Embodied Conversational Agents
Bickmore, T., & Cassell, J. (2005) · Text, Speech and Language Technology · doi:10.1007/1-4020-3933-6_2 · Cited by 324
Mapping the information flow from one brain to another during gestural communication
Schippers, M. B., Roebroeck, A., Renken, R., Nanetti, L., & Keysers, C. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1001791107 · Cited by 323
Behavior Matching in Multimodal Communication Is Synchronized
Louwerse, M. M., Dale, R., Bard, E. G., & Jeuniaux, P. (2012) · Cognitive Science · doi:10.1111/j.1551-6709.2012.01269.x · Cited by 323
Metadata
- Venue
- Choice Reviews Online
- Publisher
- American Library Association
- Published
- 1993-01-01
- Volume
- 30
- Issue
- 05
- Pages
- 30-2492-30-2492
- ISSN
- 0009-4978 (ISSN-L)
- 1523-8253
- OpenAlex
- W2118163921
- Cited by
- 5,094
- References
- 0
- Type
- book review
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.