Investigating the Impact of Annotation Interfaces on Player Performance in Distributed Multiplayer Games
Open accessAbstract
In distributed multiplayer games, it can be difficult to communicate strategic information for planning game moves and player interactions. Often, players spend extra time communicating, reducing their engagement in the game. Visual annotations in game maps and in the gameworld can address this problem and result in more efficient player communication. We studied the impact of real-time feedback on planning annotations, specifically two different annotation types, in a custom-built, third-person, multiplayer game and analyzed their effects on player performance, experience, workload, and annotation use. We found that annotations helped engage players in collaborative planning, which reduced frustration, and shortened goal completion times. Based on these findings, we discuss how annotating in virtual game spaces enables collaborative planning and improves team performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Annotation interfaces such as markers and waypoints help ground information in physical space and support the development of reconnaissance and sensemaking skills.
“Providing annotation interfaces (e.g., markers and waypoints (Figure 4)), help ground information in physical space [2], and support development of reconnaissance and sensemaking skills.”
From Mapping in the Wild: Toward Designing to Train Search & Rescue Planning (2018) · cites this paper as [2]
The Waypoint Support and Value of Annotating the Gameworld sections describe annotations that mark locations and turn plans into situated actions in a virtual gameworld, supporting the spatial-grounding point. They do not establish grounding in physical space. The Future Work section proposes studying whether annotations improve sensemaking skills; it does not report demonstrated development of those skills or address reconnaissance skills. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Gioia, D. A., Mehra, A., & Weick, K. E. (1996) · Academy of Management Review · doi:10.2307/259169 · Cited by 15,604 · see citing works
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · see citing works
Toward a Theory of Situation Awareness in Dynamic Systems
Endsley, M. R. (1995) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049543 · Cited by 8,875 · see citing works
Rules of Play: Game Design Fundamentals
Salen, K., & Zimmerman, E. (2003) · Medical Entomology and Zoology · Cited by 5,321 · see citing works
Plans and Situated Actions: The Problem of Human-Machine Communication
Kantowitz, B. H., & Suchman, L. A. (1990) · The American Journal of Psychology · doi:10.2307/1423221 · Cited by 5,306 · see citing works
A Pattern Language: Towns, Buildings, Construction
Alexander, C., Ishikawa, S., Silverstein, M., Jacobson, M., Fiksdahl-King, I., & Angel, S. (1977) · Cited by 4,555 · see citing works
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 1× · see citing works
“To assess workload, NASA-TLX [26] was chosen as it is the most commonly used and the most widely validated of the various tools available for measuring workload [25].”
MethodLabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210 · see citing works
The influence of shared mental models on team process and performance.
Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Eduardo Salas, A., & Cannon‐Bowers, J. A. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.2.273 · Cited by 2,634 · see citing works
A Descriptive Framework of Workspace Awareness for Real-Time Groupware
Gutwin, C. A., & Greenberg, S. (2002) · Computer Supported Cooperative Work (CSCW) · doi:10.1023/a:1021271517844 · Cited by 1,189 · see citing works
Toward an understanding of team performance and training.
Eduardo Salas, A., Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I. (1992) · Cited by 909 · see citing works
Theoretical Underpinnings of Situation Awareness, A Critical Review
Endsley, M. R. (2000) · doi:10.1201/b12461-7 · Cited by 902 · see citing works
Nasa-task load index (NASA-TLX); 20 years later: (577632012-009)
Hart, S. G. (2006) · PsycEXTRA Dataset · doi:10.1037/e577632012-009 · Cited by 838 · see citing works
Hamilton (2014) · doi:10.1145/2556288.2557048 · Cited by 674 · see citing works
Situation Awareness in Team Performance: Implications for Measurement and Training
Eduardo Salas, A., Prince, C., Baker, D. P., & Shrestha, L. (1995) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872095779049525 · Cited by 601 · see citing works
Understanding Information Age Warfare
Alberts, D. S., Garstka, J. J., Hayes, R. E., & Signori, D. A. (2001) · doi:10.21236/ada386374 · Cited by 352 · see citing works
Workspace Awareness in Real-Time Distributed Groupware: Framework, Widgets, and Evaluation
Gutwin, C. A., Greenberg, S., & Roseman, M. (1996) · Springer eBooks · doi:10.1007/978-1-4471-3588-3_18 · Cited by 335 · see citing works
Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks
Fussell, S. R., Setlock, L. D., Yang, J., Ou, J., Mauer, E., & Kramer, A. D. I. (2004) · Human-Computer Interaction · doi:10.1207/s15327051hci1903_3 · Cited by 328 · see citing works
(2013) · TU/e Research Portal · Cited by 265 · see citing works
Game Mechanics: Advanced Game Design
Adams, E., & Dormans, J. (2012) · Data Archiving and Networked Services (DANS) · Cited by 254 · see citing works
Communication overhead: The hidden cost of team cognition.
MacMillan, J., Entin, E. E., & Serfaty, D. (2004) · American Psychological Association eBooks · doi:10.1037/10690-004 · Cited by 234 · see citing works
The use of visual information in shared visual spaces
Kraut, R. E., Gergle, D., & Fussell, S. R. (2002) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/587078.587084 · Cited by 196 · see citing works
Brown, B. A. T., & Bell, M. (2004) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1031607.1031666 · Cited by 154 · see citing works
A comparison of two scoring procedures with the NASA task load index in a simulated flight task
Moroney, W. F., Biers, D. W., Eggemeier, F. T., & Mitchell, J. A. (2003) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon.1992.220513 · Cited by 128 · see citing works
Action as language in a shared visual space
Gergle, D., Kraut, R. E., & Fussell, S. R. (2004) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1031607.1031687 · Cited by 125 · see citing works
Leavitt, A., Keegan, B., & Clark, J. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858132 · Cited by 111 · see citing works
Björk, S. L., Lundgren, S., & Holopainen, J. P. (2003) · doi:10.26503/dl.v2003i1.60 · Cited by 110 · see citing works
Mental Models: Knowledge in the Head and Knowledge in the World.
Jonassen, D. H., & Henning, P. (1996) · doi:10.22318/icls1996.433 · Cited by 108 · see citing works
Jørgensen, K. (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9780262026864.001.0001 · Cited by 107 · see citing works
Minneman, S., & Bly, S. A. (1991) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/108844.108893 · Cited by 96 · see citing works
Target assistance for subtly balancing competitive play
Bateman, S., Mandryk, R. L., Stach, T., & Gutwin, C. A. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979287 · Cited by 89 · see citing works
Stabilized Annotations for Mobile Remote Assistance
Fakourfar, O., Ta, K., Tang, R., Bateman, S., & Tang, A. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858171 · Cited by 89 · see citing works
Improving player balancing in racing games
Cechanowicz, J. E., Gutwin, C. A., Bateman, S., Mandryk, R. L., & Stavness, I. (2014) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2658537.2658701 · Cited by 77 · see citing works
Implicit coordination in firefighting practice
Dugas, P. O. T., & Kerne, A. (2007) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1240624.1240734 · Cited by 75 · see citing works
Now You Can Compete With Anyone
Vicencio-Moreira, R., Mandryk, R. L., & Gutwin, C. A. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702242 · Cited by 71 · see citing works
The Social Behaviors of Experts in Massive Multiplayer Online Role-Playing Games
Huffaker, D. A., Wang, J., Treem, J. W., Ahmad, M. A., Fullerton, L., Williams, D. C., Poole, M. S., & Contractor, N. (2009) · doi:10.1109/cse.2009.13 · Cited by 64 · see citing works
Voice in Virtual Worlds: The Design, Use, and Influence of Voice Chat in Online Play
Wadley, G., Carter, M., & Gibbs, M. (2014) · Human-Computer Interaction · doi:10.1080/07370024.2014.987346 · Cited by 63 · see citing works
A framework for cooperative communication game mechanics from grounded theory
Dugas, P. O. T., Hammer, J., Hamilton, W. A., Jarrah, A., Graves, W. J., & Garretson, O. (2014) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2658537.2658681 · Cited by 61 · see citing works
Gergle, D., Millen, D. R., Kraut, R. E., & Fussell, S. R. (2004) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985747 · Cited by 59 · see citing works
Co-located Collaborative Sensemaking on a Large High-Resolution Display with Multiple Input Devices
Vogt, K., Bradel, L., Andrews, C., North, C. L., Endert, A., & Hutchings, D. R. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23771-3_44 · Cited by 54 · see citing works
Fussell, S. R., Setlock, L. D., & Parker, E. M. (2003) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/765891.765980 · Cited by 52 · see citing works
Bonsignore, E., Hansen, D. L., Dugas, P. O. T., Nacke, L. E., Salter, A., & Lutters, W. G. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2141512.2141517 · Cited by 46 · see citing works
Self-citationVerbal coordination in first person shooter games
Tang, A., Massey, J., Wong, N. L., Reilly, D. F., & Edwards, W. K. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145292 · Cited by 34 · see citing works
The Effects of Navigation Assistance on Spatial Learning and Performance in a 3D Game
Johanson, C., Gutwin, C. A., & Mandryk, R. L. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3116595.3116602 · Cited by 31 · see citing works
Why Players use Pings and Annotations in Dota 2
Wuertz, J., Bateman, S., & Tang, A. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025967 · Cited by 31 · see citing works
A Framework Supporting Selecting Space to Make Place in Spatial Mixed Reality Play
Sharma, H. N., Alharthi, S. A., Dolgov, I., & Dugas, P. O. T. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3116595.3116612 · Cited by 28 · see citing works
Self-citationCommunication channels and awareness cues in collocated collaborative time-critical gaming
Cheung, V., Chang, Y.-L. B., & Scott, S. D. (2012) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2145204.2145291 · Cited by 28 · see citing works
Playing at Planning: Game Design Patterns from Disaster Response Practice
Toups Dugas, P. O., Hamilton, W. A., & Alharthi, S. A. (2016) · Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/2967934.2968089 · Cited by 26 · see citing works
Self-citationSupporting Implicit Coordination Between Distributed Teams in Disaster Management
Baber, C., Cross, J., Smith, P. A., & Robinson, D. (2007) · Springer eBooks · doi:10.1007/978-3-540-75668-2_5 · Cited by 17 · see citing works
Riegler, M., Lux, M., Charvillat, V., Carlier, A., Vliegendhart, R., & Larson, M. (2014) · International Conference on Multimedia Retrieval · doi:10.1145/2578726.2582621 · Cited by 14 · see citing works
Team work: collaboration and control in London Underground line control rooms
Heath, C., & Luff, P. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511489839.005 · Cited by 9 · see citing works
Investigating the Impact of Cooperative Communication Mechanics on Player Performance in Portal 2
Vaddi, D., Toups, Z. O., Dolgov, I., Wehbe, R. R., & Nacke, L. E. (2016) · Graphics Interface · doi:10.5555/3076132.3076143 · Cited by 8 · see citing works
Self-citationAlharthi, S. A., Torres, R., Khalaf, A. S., & Dugas, P. O. T. (2017) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3130859.3130864 · Cited by 7 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Herbert H. Clark. 1996. Using Language. Cambridge University Press.
Cited by
Sorted by citations
Harris, J., & Hancock, M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300239 · Cited by 89 · see citing works
Implicit Communication of Actionable Information in Human-AI teams
Liang, C., Proft, J., Andersen, E., & Knepper, R. A. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300325 · Cited by 81 · see citing works
Making Maps Available for Play
Dugas, P. O. T., LaLone, N. J., Alharthi, S. A., Sharma, H. N., & Webb, A. M. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3336144 · Cited by 50 · see citing works
Self-citationModeling Individual and Team Behavior through Spatio-temporal Analysis
Ahmad, S. U., Bryant, A., Kleinman, E., Teng, Z., Nguyen, T.-H. D., & El‐Nasr, M. S. (2019) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3311350.3347188 · Cited by 49 · see citing works
Leveling Up Teamwork in Esports
Musick, G., Zhang, R., McNeese, N. J., Freeman, G., & Hridi, A. P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449123 · Cited by 48 · see citing works
Synthesizing Game Levels for Collaborative Gameplay in a Shared Virtual Environment
Liu, H., Choi, M., Kao, D., & Mousas, C. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3558773 · Cited by 30 · see citing works
Dislocated Boardgames: Design Potentials for Remote Tangible Play
Maurer, B., & Fuchsberger, V. (2019) · Multimodal Technologies and Interaction · doi:10.3390/mti3040072 · Cited by 24 · see citing works
Park, E., Lee, S., Ham, A., Choi, M., Kim, S., & Lee, B. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445217 · Cited by 23 · see citing works
I See You: Examining the Role of Spatial Information in Human-Agent Teams
Schelble, B. G., Flathmann, C., Musick, G., McNeese, N. J., & Freeman, G. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555099 · Cited by 22 · see citing works
Dechant, M., Poeller, S., Johanson, C., Wiley, K., & Mandryk, R. L. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376734 · Cited by 22 · see citing works
An Activity Theory Analysis of Search & Rescue Collective Sensemaking and Planning Practices
Alharthi, S. A., LaLone, N. J., Sharma, H. N., Dolgov, I., & Dugas, P. O. T. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445272 · Cited by 20 · see citing works
Self-citationA Vision of Augmented Reality for Urban Search and Rescue
LaLone, N. J., Alharthi, S. A., & Dugas, P. O. T. (2019) · Halfway to the Future Symposium · doi:10.1145/3363384.3363466 · Cited by 19 · see citing works
Self-citationToward Understanding the Effects of Cognitive Styles on Collaboration in Multiplayer Games
Alharthi, S. A., Raptis, G. E., Katsini, C., Dolgov, I., Nacke, L. E., & Dugas, P. O. T. (2018) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3272973.3274047 · Cited by 18 · see citing works
Self-citationMapping in the Wild: Toward Designing to Train Search & Rescue Planning
Alharthi, S. A., Hamilton, W. A., Dolgov, I., & Toups Dugas, P. O. (2018) · Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3272973.3274039 · Cited by 14 · see citing works
“, markers and waypoints (Figure 4)), help ground information in physical space [2], and support development of reconnaissance and sensemaking skills.”
BackgroundSelf-citationSynthesizing Shared Space Virtual Reality Fire Evacuation Training Drills
Liu, H., Choi, M., Yu, L., Koilias, A., Yu, L.-F., & Mousas, C. (2022) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct57072.2022.00097 · Cited by 9 · see citing works
Lachowicz, M., Serweta-Pawlik, A., Żurek, A., Jamro, D., Cieśliński, W., Kociuba, M., & Żurek, G. (2025) · Virtual Reality · doi:10.1007/s10055-025-01209-x · Cited by 7 · see citing works
Collaborative Mixed Reality Games
Alharthi, S. A., Spiel, K., Hamilton, W. A., Bonsignore, E., & Dugas, P. O. T. (2018) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3272973.3273013 · Cited by 7 · see citing works
Self-citationRyskeldiev, B., Igarashi, T., Zhang, J., Ochiai, Y., Cohen, M., & Herder, J. (2018) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3272973.3274100 · Cited by 5 · see citing works
CoMap: A Collaborative 3D Sketch Mapping Game to Engage Spatial Communication in Search and Rescue
Xiao, T., Zhong, S., Kiefer, P., Mizuki, M., Dugas, P. O. T., & Raubal, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791175 · Cited by 1 · see citing works
Modeling Individual and Team Behavior through Spatio-temporal AnalysisPreprint
Ahmad, S. U., Bryant, A., Kleinman, E., Teng, Z., Nguyen, T.-H. D., & El‐Nasr, M. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.11199
Neto, F. A. L., da Silva, E. J. L. A., de Araújo, C. S., & Júnior, J. M. D. S. (2026) · Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames) · doi:10.5753/sbgames.2026.25615
- Authors
- Alharthi, S. A., Torres, R. C., Khalaf, A. S., Toups Dugas, P. O., Dolgov, I., & Nacke, L. E.
- Affiliations
- New Mexico State University (US)
- New Mexico State University, Las Cruces, NM, USA
- University of Waterloo (CA)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-21
- Pages
- 1-13
- OpenAlex
- W2795493509
- Cited by
- 21
- References
- 54
- Citations by year
- 2026: 2 · 2025: 1 · 2022: 3 · 2021: 3 · 2020: 2 · 2019: 6 · 2018: 4
- FWCI
- 11.43
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation (IIS-1651532, IIS-1619273)
- Natural Sciences and Engineering Research Council of Canada (RGPIN-418622-2012)
- Canada Foundation for Innovation (35819)
- Social Sciences and Humanities Research Council of Canada (895-2011-1014)
- Mitacs (IT07255)
- Topics
- Social Sciences › Sociology and Political Science › Digital Games and Media
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Keywords
- collaborative planning, player communication, real-time feedback, player performance, player engagement, team performance, frustration, workload, game maps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
