Multimodality and Expressivity in Videogames
Open accessAbstract
This article addresses multimodality and expressivity in digital games, from the perspective of audiovisual communication, discussing essential critical dimensions regarding game literacy and experience. An overview on how we make meaning from audiovisual media is presented, to frame the focus on the multimodal cognitive nature of digital games. After comparing film and digital games, we discuss how and why the latter are rapidly becoming more relevant than television, discussing media learning differences between vicarious and enactive representations, and the creative need for children to access networks so as to interact in multimodal communication environments.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies on the multimodal aspects of video games and on how these aspects impact the expressive functions of digital games are generally scarce and mostly resort to theories of multimodality borrowed from distantly related disciplines such as film studies.
“However, as reported by Dunne (2014), Zagalo (2019) and, with a different terminology, Backe (2020), studies on the multimodal aspects of video games and on how these aspects impact the expressive functions of digital games are generally scarce and mostly resorting to theories of multimodality borrowed from distantly related disciplines, such as film studies (cf.e.g.Burn, 2016).”
From Interactive Digital Narratives as Complex Expressive Means (2022) · cites this paper as Zagalo (2019)
Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredVideo games are complex audio-visual objects that are variable and infinite, because interacting with them can always yield a slightly different "product."
“In a similar fashion, Zagalo, (2019) talks of video games as complex audio-visual objects that are variable and infinite, for it is always possible to obtain a slightly different “product” (Koenitz, 2015) by interacting with them.”
From Interactive Digital Narratives as Complex Expressive Means (2022) · cites this paper as Zagalo, (2019)
The Conclusions describe digital games as interactive audiovisual experiences, and Games and Social Learning says players can choose and change stories. The paper does not say that games are infinite or that interacting with them always yields a different product; its example of Life is Strange says players cannot change most of its outcomes. Page numbers are unavailable in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measuredVideo games are one of the most all-encompassing multimodal means of communication.
“Not least for this reason, video games are one of the most allencompassing multimodal means of communication (as noted also by Toh, 2018;and Zagalo, 2019) to the point of being possibly considered examples of Gesamtkunstwerk, due to the synergy of different arts they often exhibit (cf.Backe, 2020; and in some regards also by; Smith, 2007).”
From Interactive Digital Narratives as Complex Expressive Means (2022) · cites this paper as Zagalo, 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Intrinsic Motivation and Self-Determination in Human Behavior
Deci, E. L., & Ryan, R. M. (1985) · doi:10.1007/978-1-4899-2271-7 · Cited by 26,772 · see citing works
(1985) · Medical Entomology and Zoology · Cited by 19,180 · see citing works
Intrinsic Motivation and Self-Determination in Human Behavior
Miller, K. A. J., Deci, E. L., & Ryan, R. M. (1988) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2070638 · Cited by 15,361 · see citing works
(1994) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,542 · see citing works
THE ROLE OF TUTORING IN PROBLEM SOLVING <sup>*</sup>
Wood, D., Bruner, J. S., & Ross, G. (1976) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1976.tb00381.x · Cited by 8,851 · see citing works
Visual Pleasure and Narrative Cinema
Mulvey, L. (1975) · Screen · doi:10.1093/screen/16.3.6 · Cited by 8,680 · see citing works
What Video Games Have to Teach Us about Learning and Literacy
(2004) · Education + Training · doi:10.1108/et.2004.00446dae.002 · Cited by 6,674 · see citing works
What video games have to teach us about learning and literacy
Gee, J. P. (2003) · Computers in Entertainment · doi:10.1145/950566.950595 · Cited by 5,735 · see citing works
Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification
Hamari, J., Koivisto, J., & Sarsa, H. (2014) · 2014 47th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2014.377 · Cited by 5,201 · see citing works
(1966) · Medical Entomology and Zoology · Cited by 4,505 · see citing works
Positive emotions broaden the scope of attention and thought‐action repertoires
Fredrickson, B. L., & Branigan, C. (2005) · Cognition & Emotion · doi:10.1080/02699930441000238 · Cited by 3,521 · see citing works
Mirror neurons and the simulation theory of mind-reading
Gallese, V (1998) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(98)01262-5 · Cited by 3,201 · see citing works
Toward a Theory of Instruction
McGuire, M. T. (1966) · Psychosomatic Medicine · doi:10.1097/00006842-196611000-00010 · Cited by 2,950 · see citing works
The benefits of playing video games.
Granic, I., Lobel, A., & Engels, R. C. M. E. (2014) · American Psychologist · doi:10.1037/a0034857 · Cited by 2,456 · see citing works
Bruner, J. S. (1987) · Social research · Cited by 1,811 · see citing works
Cope, B., & Kalantzis, M. (2005) · Routledge eBooks · doi:10.4324/9780203979402 · Cited by 1,752 · see citing works
O'Brien, C., Bordwell, D., & Branigan, E. (1986) · SubStance · doi:10.2307/3684717 · Cited by 1,487 · see citing works
Bolt, R. A. (1980) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/800250.807503 · Cited by 1,445 · see citing works
Time in the mind: Using space to think about time
Casasanto, D. J., & Boroditsky, L. (2007) · Cognition · doi:10.1016/j.cognition.2007.03.004 · Cited by 1,188 · see citing works
Visual Pleasure and Narrative Cinema
Mulvey, L. (1975) · doi:10.1007/978-1-349-14428-0_27 · Cited by 1,140 · see citing works
Bordwell, D. (2013) · Routledge eBooks · doi:10.4324/9781315002163 · Cited by 1,104 · see citing works
The storytelling animal: how stories make us human
(2012) · Choice Reviews Online · doi:10.5860/choice.50-0062 · Cited by 948 · see citing works
(2009) · Cited by 849 · see citing works
Reclaiming conversation: the power of talk in a digital age
(2016) · Choice Reviews Online · doi:10.5860/choice.195873 · Cited by 842 · see citing works
Kress, G. (2009) · Routledge eBooks · doi:10.4324/9780203970034 · Cited by 765 · see citing works
An update on the effects of playing violent video games
Anderson, C. A. (2003) · Journal of Adolescence · doi:10.1016/j.adolescence.2003.10.009 · Cited by 740 · see citing works
Deterding, S. (2012) · interactions · doi:10.1145/2212877.2212883 · Cited by 725 · see citing works
Louder than words: the new science of how the mind makes meaning
(2013) · Choice Reviews Online · doi:10.5860/choice.50-5297 · Cited by 718 · see citing works
Metaphor Interpretation as Embodied Simulation
Gibbs, R. W. (2006) · Mind & Language · doi:10.1111/j.1468-0017.2006.00285.x · Cited by 637 · see citing works
Zhong, C., & Leonardelli, G. J. (2008) · Psychological Science · doi:10.1111/j.1467-9280.2008.02165.x · Cited by 632 · see citing works
Window shopping: cinema and the postmodern
(1993) · Choice Reviews Online · doi:10.5860/choice.31-0840 · Cited by 592 · see citing works
Post-Theory. Reconstructing Film Studies
Curtis, S., Bordwell, D., & Carroll, N. (1997) · The Journal of the Midwest Modern Language Association · doi:10.2307/1315434 · Cited by 522 · see citing works
Multimodal integration for the representation of space in the posterior parietal cortex
Andersen, R. A. (1997) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1997.0128 · Cited by 515 · see citing works
After Virtue: A Study in Moral Theory.
Wax, M. L., & MacIntyre, A. (1982) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2067167 · Cited by 482 · see citing works
Drive the surprising truth about what motivates us
Pink, D. H. 1. (2009) · Cited by 456 · see citing works
Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked
Cramer, B. W. (2018) · Journal of Information Policy · doi:10.5325/jinfopoli.8.1.0314 · Cited by 415 · see citing works
Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents
Bateman, J. A. (2008) · Cited by 410 · see citing works
Bateman, J. A. (2008) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230582323 · Cited by 362 · see citing works
Passionate views: film, cognition, and emotion
Plantinga, C. (1999) · Choice Reviews Online · doi:10.5860/choice.37-2077 · Cited by 362 · see citing works
A weak scientific basis for gaming disorder: Let us err on the side of caution
van Rooij, A. J., Ferguson, C. J., Colder Carras, M., Kardefelt-Winther, D., Shi, J., Aarseth, E., Bean, A. M., Bergmark, K. H., Brus, A., Coulson, M., et al. (2018) · Journal of Behavioral Addictions · doi:10.1556/2006.7.2018.19 · Cited by 346 · see citing works
Violent Video Games and Aggression
Ferguson, C. J., Rueda, S. M., Cruz, A. M., Ferguson, D. E., Fritz, S., & Smith, S. M. (2008) · Criminal Justice and Behavior · doi:10.1177/0093854807311719 · Cited by 344 · see citing works
Eduardo Salas, A., Bowers, C. A., & Rhodenizer, L. G. (1998) · International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0803_2 · Cited by 299 · see citing works
Reclaiming conversation the power of talk in a digital age
Turkle, S. 1. (2015) · Cited by 260 · see citing works
The Long History of Gaming in Military Training
Smith, R. D. (2009) · Simulation & Gaming · doi:10.1177/1046878109334330 · Cited by 238 · see citing works
Gee, J. P. (2008) · Games and Culture · doi:10.1177/1555412008317309 · Cited by 234 · see citing works
(2017) · Cited by 209 · see citing works
In the blink of an eye : a perspective on film editing
Murch, W. (2001) · Cited by 205 · see citing works
Stetina, B. U., Kothgassner, O. D., Lehenbauer, M., & Kryspin‐Exner, I. (2010) · Computers in Human Behavior · doi:10.1016/j.chb.2010.09.015 · Cited by 191 · see citing works
Major Film Theories: An Introduction
Purcell, J. M., & Andrew, J. D. (1977) · Journal of Aesthetics and Art Criticism · doi:10.2307/430773 · Cited by 149 · see citing works
Human Communication as Narration
Fisher, W. R. (2021) · University of South Carolina Press eBooks · doi:10.2307/j.ctv1nwbqtk · Cited by 114 · see citing works
Kuss, D. J., Griffiths, M. D., & Pontes, H. M. (2017) · Journal of Behavioral Addictions · doi:10.1556/2006.6.2017.032 · Cited by 112 · see citing works
Post-Theory: Reconstructing Film Studies
Knight, D., Bordwell, D., Carroll, N., & Carroll, N. (1998) · Journal of Aesthetic Education · doi:10.2307/3333563 · Cited by 62 · see citing works
What are we downloading for our children? Best-selling children’s apps in four European countries
Uğurlu, B. S., Takács, Z. K., & Bus, A. G. (2017) · Journal of Early Childhood Literacy · doi:10.1177/1468798417744057 · Cited by 56 · see citing works
Film Editing: The Art of the Expressive
Orpen, V. (2003) · Cited by 50 · see citing works
Post-theory reconstructing film studies
Bordwell, D., & Carroll, N. (1996) · Cited by 41 · see citing works
Purposes for literacy in children's use of the online virtual world Club Penguin
Marsh, J. (2012) · Journal of Research in Reading · doi:10.1111/j.1467-9817.2012.01530.x · Cited by 39 · see citing works
Making machinima: animation, games, and multimodal participation in the media arts
Burn, A. R. (2015) · Learning Media and Technology · doi:10.1080/17439884.2015.1107096 · Cited by 37 · see citing works
Mavoa, J., Carter, M., & Gibbs, M. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3116595.3116638 · Cited by 24 · see citing works
The major film theories : an introduction
Andrew, J. D. (1976) · Oxford University Press eBooks · Cited by 16 · see citing works
Towards automatic evaluation of multimodal user interfaces
Coutaz, J., Salber, D., & Balbo, S. (1993) · Knowledge-Based Systems · doi:10.1016/0950-7051(93)90018-o · Cited by 14 · see citing works
Daniels, K. (2017) · Springer eBooks · doi:10.1007/978-981-10-4364-2_12 · Cited by 9 · see citing works
Narrative Design of Sadness in Heavy Rain
Zagalo, N. (2017) · Journal of Science and Technology of the Arts · doi:10.7559/citarj.v9i2.246 · Cited by 8 · see citing works
Self-citationElementos do design de videojogos que fomentam o interesse dos jogadores
Zagalo, N., Carvalho, A. A. A., & Araújo, I. (2021) · Educação Sociedade & Culturas · doi:10.34626/esc.vi48.180 · Cited by 8 · see citing works
Self-citationMetz, C. (1966) · Communications · doi:10.3406/comm.1966.1102 · Cited by 6 · see citing works
Film Editing: The Art of the Expressive
LoBrutto, V. (2005) · Film Quarterly · doi:10.1525/fq.2005.59.1.56 · Cited by 2 · see citing works
Building digital construction zones for young learners
Kopelke, K., & Gower, S. (2017) · Educating Young Children: Learning and Teaching in the Early Childhood Years · Cited by 1 · see citing works
A weak scientific basis for gaming disorder: Let us err on the side of caution
van Rooij, A. J., Ferguson, C., Carras, M. C., Kardefelt‐Winther, D., Shi, J., & Przybylski, A. (2018) · Open Science Framework · doi:10.17605/osf.io/kc7r9 · Cited by 1 · see citing works
Flagged first, then by citations
9 of the 76 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- Boyd, B. (2010). On the Origin of Stories. Cambridge, Massachusetts: Harvard University Press.
- Egenfeldt-Nielsen, S. (2007). Educational potential of computer games. Journal of Educational Multimedia and Hypermedia, 16(3), 263-281.
- Gould, S. J. (1994). So Near and Yet So Far. The New York Review of Books. Retrieved from https://www.nybooks.com/articles/1994/10/20/so-near-and-yet-so-far/
- Kaakinen, J., & Arwood, E. (2009). Systematic Review of Nursing Simulation Literature for Use of Learning Theory. International Journal of Nursing Education Scholarship, 6(1), https://doi.org/10.2202/1548923X.1688 · DOI
- Kress, G. (2000). Multimodality. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures (pp. 179-200). London: Routledge.
Cited by
Sorted by citations
Interactive Digital Narratives as Complex Expressive Means
Bellini, M. (2022) · Frontiers in Virtual Reality · doi:10.3389/frvir.2022.854960 · Cited by 13 · see citing works
“Not least for this reason, video games are one of the most all-encompassing multimodal means of communication (as noted also by Toh, 2018; and Zagalo, 2019) to the point of being possibly considered examples of Gesamtkunstwerk , due to the synergy of different arts they often exhibit (cf. Backe,…”
BackgroundGame Design as an Autonomous Research Subject
Neves, P. P., & Zagalo, N. (2021) · Information · doi:10.3390/info12090367 · Cited by 12 · see citing works
Self-citationUnderstanding Multimodal Assessment Practices in Higher Education to Improve Equity
Midgette, E., Stewart, O. G., & August, I. (2025) · Education Sciences · doi:10.3390/educsci15111523 · Cited by 4 · see citing works
Zagalo, N. (2020) · Human-computer interaction series · doi:10.1007/978-3-030-37085-5_2 · Cited by 2 · see citing works
Self-citationThe Hidden Danger in Digital Children Games: Gambling
Erdal, K., & ERDAL, G. (2020) · Journal of History Culture and Art Research · doi:10.7596/taksad.v9i3.2709 · Cited by 2 · see citing works
Villanueva, E., & Ensslin, A. (2021) · Pragmatics & beyond. New series · doi:10.1075/pbns.327.09vil · Cited by 2 · see citing works
The pragmatics of critical thinking
Bardzokas, V., & Desilla, L. (2025) · Internet Pragmatics · doi:10.1075/ip.00118.bar · Cited by 1 · see citing works
Rahardi, R. K., & Firdaus, W. (2023) · Aksara · doi:10.29255/aksara.v35i2.4160.211--227 · Cited by 1 · see citing works
- Authors
- Zagalo, N.
- Affiliations
- University of Aveiro (PT)
- Corresponding author
- Zagalo, Nelson
- ORCID
- Zagalo, Nelson
- Venue
- Observatorio (OBS*)
- Published
- 2019-03-04
- Volume
- 13
- Issue
- 1
- ISSN
- 1646-5954 (ISSN-L)
- OpenAlex
- W2916949239
- Cited by
- 8
- References
- 76
- Citations by year
- 2025: 2 · 2023: 1 · 2022: 1 · 2021: 2 · 2020: 2
- FWCI
- 14.02
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc
- Locations
- Observatorio (OBS*) (publishedVersion, cc-by-nc)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, cc-by-nc)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, cc-by-sa)
- Funders
- European Commission
- European Regional Development Fund
- Topics
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Literacy, Media, and Education
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Keywords
- multimodality, expressivity, video games, audiovisual communication, game literacy, game experience, multimodal cognition, media learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review