PANDALens: Towards AI-Assisted In-Context Writing on OHMD During Travels
Open accessAbstract
While effective for recording and sharing experiences, traditional in-context writing tools are relatively passive and unintelligent, serving more like instruments rather than companions. This reduces primary task (e.g., travel) enjoyment and hinders high-quality writing. Through formative study and iterative development, we introduce PANDALens, a Proactive AI Narrative Documentation Assistant built on an Optical See-Through Head Mounted Display that supports personalized documentation in everyday activities. PANDALens observes multimodal contextual information from user behaviors and environment to confirm interests and elicit contemplation, and employs Large Language Models to transform such multimodal information into coherent narratives with significantly reduced user effort. A real-world travel scenario comparing PANDALens with a smartphone alternative confirmed its effectiveness in improving writing quality and travel enjoyment while minimizing user effort. Accordingly, we propose design guidelines for AI-assisted in-context writing, highlighting the potential of transforming them from tools to intelligent companions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
An Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · see citing works
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 1× · see citing works
“Interaction Distraction and Naturalness to primary task (i.e., travel) [17] (7-point Likert scale), usability ( SUS [13]), and perceived task load ( RTLX [30]), were collected to evaluate interaction quality of moment capture in the travel.”
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · see citing works
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · see citing works
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · cited 2× · see citing works
“Few-shot prompts (i.e., Auto Mode Selection in Figure 6) enabled LLM to discern whether to generate questions, compile a moment selection list, or create a full blog based on the input format, and additional prompt engineering techniques [70, 87] were utilized to enhance its output.”
The Power of Interest for Motivation and Engagement
Renninger, K. A., & Hidi, S. E. (2015) · Routledge eBooks · doi:10.4324/9781315771045 · Cited by 919 · see citing works
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 893 · see citing works
Wu, T. S., Terry, M., & Cai, C. (2021) · International Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 757 · see citing works
Exploring the Psychology of Interest
Silvia, P. J. (2006) · doi:10.1093/acprof:oso/9780195158557.001.0001 · Cited by 734 · see citing works
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · cited 1× · see citing works
“Creating a good user experience when interacting with advanced AI (e.g., LLMs), especially when uncertain about its capabilities, is also challenging [91].”
BackgroundUnsupervised Video Summarization with Adversarial LSTM Networks
Mahasseni, B., Lam, M., & Todorović, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.318 · Cited by 620 · see citing works
PaLM-E: An Embodied Multimodal Language Model
Driess, D., Xia, F., Sajjadi, M. S. M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03378 · Cited by 591 · see citing works
Allen, J. E., Guinn, C. I., & Horvtz, E. (1999) · IEEE Intelligent Systems · doi:10.1109/5254.796083 · Cited by 442 · see citing works
Beyond total capture: a constructive critique of lifelogging
Sellen, A. J., & Whittaker, S. (2010) · Communications of the ACM · doi:10.1145/1735223.1735243 · Cited by 377 · see citing works
Using context-aware computing to reduce the perceived burden of interruptions from mobile devices
Ho, J., & Intille, S. S. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055100 · Cited by 367 · see citing works
A review of speaker diarization: Recent advances with deep learning
Park, T., Kanda, N., Dimitriadis, D., Han, K. J., Watanabe, S., & Narayanan, S. S. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101317 · Cited by 363 · see citing works
Autonomous travel decision-making: An early glimpse into ChatGPT and generative AI
Wong, I. A., Lian, Q. L., & Sun, D. (2023) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2023.06.022 · Cited by 361 · see citing works
Fine-tuning language models to find agreement among humans with diverse preferencesPreprint
Bakker, M. A., Chadwick, M., Sheahan, H., et al. (2022) · Neural Information Processing Systems · doi:10.48550/arxiv.2211.15006 · Cited by 353 · see citing works
Lee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345 · cited 2× · see citing works
“For subjective measures, self-rated Post-Editing Effort , quality of Language , Creativity & Appeal , and meeting preferred Writing Style for final writing (7-point Likert scale) [49] were collected.”
Rezwana, J., & Maher, M. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3519026 · Cited by 342 · see citing works
Multimodal Deep Learning for Activity and Context Recognition
Radu, V., Tong, C., Bhattacharya, S., Lane, N. D., Mascolo, C., Marina, M. K., & Kawsar, F. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3161174 · Cited by 281 · see citing works
Co-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265 · cited 1× · see citing works
“Similarly, although AI helps in post-processing, including editing [75, 88], a lack of proper user guidance can lead to misleading information [7, 38, 85] and templated output [49, 65, 84] that deviates from user expectations.”
BackgroundCreative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · see citing works
Privacy Concerns for Use of Voice Activated Personal Assistant in the Public Space
Easwara Moorthy, A., & Vu, K.-P. L. (2015) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2014.986642 · Cited by 250 · see citing works
Su, L., Cheng, J., & Swanson, S. R. (2020) · Tourism Management · doi:10.1016/j.tourman.2020.104138 · Cited by 213 · see citing works
An Application of Travel Blog Narratives to Explore Memorable Tourism Experiences
Chandralal, L., Rindfleish, J., & Valenzuela, F. (2015) · Asia Pacific Journal of Tourism Research · doi:10.1080/10941665.2014.925944 · Cited by 190 · see citing works
Ranking Domain-Specific Highlights by Analyzing Edited Videos
Sun, M., Farhadi, A., & Seitz, S. M. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10590-1_51 · Cited by 187 · see citing works
Exploring the Tourist Experience
Park, S., & Santos, C. A. (2016) · Journal of Travel Research · doi:10.1177/0047287515624017 · Cited by 172 · see citing works
Towards Indistinguishable Augmented Reality
Itoh, Y., Langlotz, T., Sutton, J., & Plopski, A. (2021) · ACM Computing Surveys · doi:10.1145/3453157 · Cited by 152 · see citing works
Toward Storytelling From Visual Lifelogging: An Overview
Bolaños, M., Dimiccoli, M., & Radeva, P. I. (2016) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2616296 · Cited by 136 · see citing works
Fine-Tuning Language Models to Find Agreement Among Humans with Diverse Preferences
Bakker, M. A., Chadwick, M. J., Sheahan, H., Tessler, M. H., Campbell-Gillingham, L., Balaguer, J., McAleese, N., Glaese, A., Aslanides, J., Botvinick, M., et al. (2022) · neural information processing systems · doi:10.52202/068431-2766 · Cited by 133 · see citing works
Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511599 · Cited by 119 · see citing works
The influence of the components of storytelling blogs on readers’ travel intentions
Hsiao, K., Lu, H., & Lan, W. (2013) · Internet Research · doi:10.1108/10662241311313303 · Cited by 112 · see citing works
Summarization of Egocentric Videos: A Comprehensive Survey
del Molino, A. G., Tan, C., Lim, J., & Tan, A. (2016) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2623480 · Cited by 111 · see citing works
InSense: Interest-Based Life Logging
Blum, M., Pentland, A. ‘., & Tröster, G. (2006) · IEEE Multimedia · doi:10.1109/mmul.2006.87 · Cited by 104 · see citing works
AngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580907 · Cited by 98 · see citing works
Valencia, S., Cave, R., Kallarackal, K., Seaver, K., Terry, M., & Kane, S. K. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581560 · Cited by 97 · cited 1× · see citing works
“Similarly, although AI helps in post-processing, including editing [75, 88], a lack of proper user guidance can lead to misleading information [7, 38, 85] and templated output [49, 65, 84] that deviates from user expectations.”
120-01-02 Department of Political Science
Bin, Y. (2013) · Exhibit - A Showcase of Scholarship, Creativity and Preservation Provided by Xavier University Library (Xavier University) · Cited by 96 · see citing works
A framework of memory management and tourism experiences
Tung, V. W. S., Lin, P. M. C., Zhang, H. Q., & Zhao, A. (2016) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2016.1260521 · Cited by 94 · see citing works
XAIR: A Framework of Explainable AI in Augmented Reality
Xu, X., Yu, A., Jonker, T. R., Todi, K., Lu, F., Qian, X., Belo, J. M. E., Wang, T., Li, M., Mun, A., et al. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581500 · Cited by 84 · see citing works
Proactive Conversational Agents in the Post-ChatGPT World
Liao, L., Yang, G. H., & Shah, C. S. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3594250 · Cited by 83 · see citing works
A Survey of Attention Management Systems in Ubiquitous Computing Environments
Anderson, C., Hübener, I., Seipp, A.-K., Ohly, S., David, K., & Pejović, V. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3214261 · Cited by 80 · see citing works
Li, R., Wu, J., & Starner, T. E. (2019) · Augmented Human International Conference · doi:10.1145/3311823.3311831 · Cited by 73 · see citing works
Effects of Proactive Dialogue Strategies on Human-Computer Trust
Kraus, M., Wagner, N., & Minker, W. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3340631.3394840 · Cited by 72 · see citing works
Alallah, F. S., Neshati, A., Sakamoto, Y., Hasan, K., Lank, E., Bunt, A., & Irani, P. (2018) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3281505.3281541 · Cited by 70 · see citing works
Matching attentional draw with utility in interruption
Gluck, J., Bunt, A., & McGrenere, J. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240631 · Cited by 70 · see citing works
Managing mobile text in head mounted displays
Orlosky, J., Kiyokawa, K., & Takemura, H. (2014) · ACM SIGMOBILE Mobile Computing and Communications Review · doi:10.1145/2636242.2636246 · Cited by 64 · see citing works
Yang, R., & Tung, V. W. S. (2017) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2017.1363685 · Cited by 63 · see citing works
Tse, T. S. M., & Zhang, E. Y. (2012) · Asia Pacific Journal of Tourism Research · doi:10.1080/10941665.2012.658413 · Cited by 58 · see citing works
Exploring multisensory place experiences through cruise blog analysis
Buzova, D., Cervera, A., & Blas, S. S. (2019) · Psychology and Marketing · doi:10.1002/mar.21286 · Cited by 50 · see citing works
Overview of Lifelogging: Current Challenges and Advances
Ksibi, A., Alluhaidan, A. S., Salhi, A., & Rahman, S. A. E. (2021) · IEEE Access · doi:10.1109/access.2021.3073469 · Cited by 50 · see citing works
Factors influencing learning effectiveness of educational travel: A case study in China
Ping, L., & Huimin, L. (2020) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2019.10.001 · Cited by 46 · see citing works
Heads-Up Computing Moving Beyond the Device-Centered Paradigm
Zhao, S., Tan, F. F.-Y., & Fennedy, K. (2023) · Communications of the ACM · doi:10.1145/3571722 · Cited by 34 · see citing works
Self-citationJanaka, N. N. P. K., Haigh, C. D. S. Y., Kim, H., Zhang, S., & Zhao, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502127 · Cited by 33 · see citing works
Self-citationMultitasking with Play Write, a Mobile Microproductivity Writing Tool
Iqbal, S. T., Teevan, J. B., Liebling, D. J., & Thompson, A. L. (2018) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3242587.3242611 · Cited by 31 · see citing works
Kang, B., Min, C., Kim, W., Hwang, I., Park, C., Lee, S., Lee, S.-J., & Song, J. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998186 · Cited by 31 · see citing works
Walber, T., Ansgar Scherp, A., & Staab, S. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557025 · Cited by 29 · see citing works
ParaGlassMenu: Towards Social-Friendly Subtle Interactions in Conversations
Cai, R., Janaka, N. N. P. K., Zhao, S., & Sun, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581065 · Cited by 28 · see citing works
Self-citationPrice, B. A., Stuart, A., Çalıklı, G., McCormick, C., Mehta, V., Hutton, L., Bandara, A. K., Levine, M., & Nuseibeh, B. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3090087 · Cited by 27 · see citing works
Can Icons Outperform Text? Understanding the Role of Pictograms in OHMD Notifications
Janaka, N. N. P. K., Zhao, S., & Sapkota, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580891 · Cited by 23 · see citing works
Self-citationTan, A. L., Gillies, R. M., & Jamaludin, A. B. (2021) · Education Sciences · doi:10.3390/educsci11080379 · Cited by 22 · see citing works
van Berkel, N., Opie, J., Ahmad, O. F., Lovat, L. B., Stoyanov, D. V., & BLANDFORD, A. E. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3480247 · Cited by 22 · see citing works
Sapkota, S., Ram, A., & Zhao, S. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447526.3472035 · Cited by 22 · see citing works
Self-citationMapping the Design Space of Human-AI Interaction in Text Summarization
Cheng, R., Smith, A., Zhang, K., Tetreault, J., & Jaimes-Larrarte, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.33 · Cited by 22 · see citing works
Janaka, N. N. P. K., Gao, J., Zhu, L. Z., Zhao, S., Lyu, L., Xu, P., Nabokow, M., Wang, S., & Ong, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610931 · Cited by 21 · see citing works
Self-citationImproving the Usability of Voice User Interfaces: A New Set of Ergonomic Criteria
Nowacki, C., Gordeeva, A., & Lizé, A.-H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49760-6_8 · Cited by 20 · see citing works
Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture
Krosnick, R., Anderson, F., Matejka, J., Oney, S., Lasecki, W. S., Grossman, T., & Fitzmaurice, G. W. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445066 · Cited by 17 · see citing works
Chang, Y., Zhao, Y., Dong, M., Wang, Y., Lu, Y., Lv, Q., Dick, R. P., Lu, T., Gu, N., & Shang, L. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3463509 · Cited by 15 · see citing works
Soleymani, M., & Mortillaro, M. (2018) · Frontiers in ICT · doi:10.3389/fict.2018.00017 · Cited by 14 · see citing works
Zhou, C., Fennedy, K., Tan, F. F.-Y., Zhao, S., & Shao, Y. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581430 · Cited by 14 · see citing works
Self-citationEnabling Self-Reflection with LifelogExplorer: Generating Simple Views from Complex Data
Kocielnik, R., Maggi, F., & Sidorova, N. (2013) · Proceedings of the ICTs for improving Patients Rehabilitation Research Techniques · doi:10.4108/icst.pervasivehealth.2013.251934 · Cited by 13 · see citing works
Coste, D., & Pier, J. H. (2009) · De Gruyter eBooks · doi:10.1515/9783110217445.295 · Cited by 12 · see citing works
HighlightMe: Detecting Highlights from Human-Centric Videos
Bhattacharya, U., Wu, G., Petrangeli, S., Swaminathan, V., & Manocha, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00805 · Cited by 10 · see citing works
Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks
Ding, Z., & Chan, J. (2023) · arXiv · doi:10.48550/arxiv.2303.06430 · Cited by 9 · see citing works
User Attention with Head-Worn Displays
Farve, N., Achituv, T., & Maes, P. (2016) · doi:10.1145/2851581.2892530 · Cited by 7 · see citing works
LiveSnippets: Voice-based Live Authoring of Multimedia Articles about Experiences
Kim, H., Zhao, S., Liu, C., & Hara, K. (2020) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3379503.3403556 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 13
- Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023). · arXiv
- John Brooke. 1996. SUS -A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 7.
- Vannevar Bush et al. 1945. As we may think. The atlantic monthly 176, 1 (1945), 101-108.
- Yuhu Chang, Yingying Zhao, Mingzhi Dong, Yujiang Wang, Yutian Lu, Qin Lv, Robert P Dick, Tun Lu, Ning Gu, and Li Shang. 2021. MemX: An attention-aware smart eyewear system for personalized moment auto-capture. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2 (2021), 1-23.
- Jakob Engel, Kiran Somasundaram, Michael Goesele, Albert Sun, Alexander Gamino, Andrew Turner, Arjang Talattof, Arnie Yuan, Bilal Souti, Brighid Mered- ith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel DeTone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Duncan Frost, Edward Miller, Elias Mueggler, Evgeniy Oleinik, Fan Zhang, Guruprasad Somasundaram, Gustavo So- laira, Harry Lanaras, Henry Howard-Jenkins, Huixuan Tang, Hyo Jin Kim, Jaime Rivera, Ji Luo, Jing Dong, Julian Straub, Kevin Bailey, Kevin Eckenhof, Lingni Ma, Luis Pesqueira, Mark Schwesinger, Maurizio Monge, Nan Yang, Nick Charron, Nikhil Raina, Omkar Parkhi, Peter Borschowa, Pierre Moulon, Prince Gupta, Raul Mur-Artal, Robbie Pennington, Sachin Kulkarni, Sagar Miglani, Santosh Gondi, Saransh Solanki, Sean Diener, Shangyi Cheng, Simon Green, Steve Saarinen, Suvam Patra, Tassos Mourikis, Thomas Whelan, Tripti Singh, Vasileios Balntas, Vijay Baiyya, Wilson Dreewes, Xiaqing Pan, Yang Lou, Yipu Zhao, Yusuf Man- sour, Yuyang Zou, Zhaoyang Lv, Zijian Wang, Mingfei Yan, Carl Ren, Renzo De Nardi, and Richard Newcombe. 2023. Project Aria: A New Tool for Egocentric Multi-Modal AI Research. arXiv:2308.13561 [cs.HC] · arXiv
- Sanchit Gandhi, Patrick von Platen, and Alexander M. Rush. 2023. Distil- Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling. arXiv:2311.00430 [cs.CL] · arXiv
- Seppo E Iso-Ahola. 1983. Towards a social psychology of recreational travel. Leisure studies 2, 1 (1983), 45-56.
- Richard E Mayer and Roxana Moreno. 2003. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist 38, 1 (2003), 43-52.
- Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. 2023. Capabilities of GPT-4 on Medical Challenge Problems. arXiv:2303.13375 [cs.CL] · arXiv
- Donald A. Norman. 2013. The design of everyday things (revised and expanded edition ed.). Basic Books.
- Claire Petitmengin, Martijn Van Beek, Michel Bitbol, Jean-Michel Nissou, and Andreas Roepstorf. 2019. Studying the experience of meditation through micro- phenomenology. Current opinion in psychology 28 (2019), 54-59.
- John Sweller. 1988. Cognitive load during problem solving: Efects on learning. Cognitive science 12, 2 (1988), 257-285.
- Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision). arXiv:2309.17421 [cs.CV] · arXiv
Cited by
Sorted by citations
Tang, Y., Situ, J., Cui, A. Y., Wu, M., & Huang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714224 · Cited by 44 · see citing works
Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
Reza, M., Thomas-Mitchell, J., Dushniku, P., Laundry, N., Williams, J. J., & Kuzminykh, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757566 · Cited by 37 · see citing works
“Finally, reviewing support was limited, and focused on evaluation using quantitative feedback like readability metrics [87], and visual feedback such as progress bars [89].”
Cai, R., Janaka, N. N. P. K., Kim, H., Chen, Y., Zhao, S., Huang, Y., & Hsu, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713953 · Cited by 29 · see citing works
Self-citationInie, N., Falk, J., & Selvan, R. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714227 · Cited by 26 · see citing works
Vision-Based Multimodal Interfaces: A Survey and Taxonomy for Enhanced Context-Aware System Design
Hu, Y. '., Tang, J., Gong, X., Zhou, Z., Zhang, S., Elvitigala, D. S., Mueller, F., Hu, W. S., & Quigley, A. J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714161 · Cited by 28 · see citing works
Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations
Kim, Y., Lee, S.-J., & Donahue, C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713818 · Cited by 23 · see citing works
agentAR: Creating Augmented Reality Applications with Tool-Augmented LLM-based Autonomous Agents
Zhu, C., Hsia, S.-K., Hu, X., Liu, Z., Shi, J., & Karthik Ramani, A. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747676 · Cited by 13 · see citing works
PilotAR: Streamlining Pilot Studies with OHMDs from Concept to Insight
Janaka, N. N. P. K., Cai, R., Ram, A., Zhu, L. Z., Zhao, S., & Yong, K. Q. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3678576 · Cited by 11 · see citing works
Self-citationExperiencing the World through Imperfect Lenses: An Autoethnography of Living in Mixed Reality
Sun, Y. C., & Schöning, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735841 · Cited by 10 · see citing works
Empowering Smart Glasses with Large Language Models: Towards Ubiquitous AGI
Zhang, D., Li, Y., He, Z., & Li, X. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678992 · Cited by 9 · see citing works
SketchGPT: A Sketch-based Multimodal Interface for Application-Agnostic LLM Interaction
Huang, Z., Gao, C., Shan, Y., Hu, H., Li, Q., Deng, X., Ma, C., Lai, Y., Liu, Y., Tian, F., et al. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747598 · Cited by 8 · see citing works
InteRecon: Towards Reconstructing Interactivity of Personal Memorable Items in Mixed Reality
Li, Z., Li, J., Xiong, Z., Zhang, S., Faruqi, F., Mueller, S., Liang, C., Ma, X., & Fan, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713882 · Cited by 10 · see citing works
Rajaram, S., Surale, H. B., McConkey, C., Rognon, C., Mehta, H., Glueck, M., & Collins, C. M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714310 · Cited by 6 · see citing works
Exploring the Innovation Opportunities for Pre-trained Models
Park, M., Forlizzi, J., & Zimmerman, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735753 · Cited by 5 · see citing works
Janaka, N. N. P. K., Cai, R., Zhao, S., & Hsu, D. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3648644 · Cited by 5 · see citing works
Self-citationCross-Format Retrieval-Augmented Generation in XR with LLMs for Context-Aware Maintenance Assistance
Nagy, A., Spyridis, Y., & Argyriou, V. (2025) · IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR) · doi:10.1109/aixvr63409.2025.00067 · Cited by 5 · see citing works
Kundu, R. K., Ahmed, I., & Hoque, K. A. (2025) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct68609.2025.00060 · Cited by 4 · see citing works
Investigating Context-Aware Collaborative Text Entry on Smartphones using Large Language Models
Chen, W., Shi, Y., Wang, Y., Shi, W., Chen, M., Cheng, G., Mei, Y., Zhu, Y., Zhang, J., & Yu, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713944 · Cited by 4 · see citing works
Towards Intelligent Wearable Assistants
Janaka, N. N. P. K. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678989 · Cited by 3 · see citing works
Self-citationDesignerlyLoop: Forming Design Intent through Curated Reasoning for Human-LLM Alignment
Wang, A., Li, Z., Tong, X., & Hui, P. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812885 · Cited by 4 · see citing works
Gu, Y., Cai, R., Ram, A., Li, Y., Zhang, H., & Zhao, S. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3590876 · Cited by 1 · see citing works
Self-citationTravelTales: Reflecting on Meaningful Travel through Digital Scrapbooking and Journaling
Yin, M., & Xiao, R. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749512 · Cited by 1 · see citing works
An Exploratory Study on AI-driven Visualisation Techniques on Decision Making in Extended Reality
Dong, Z., Han, B., Zhang, J., Wen, R., Ens, B., Clark, A. J., & Piumsomboon, T. (2024) · Australasian Computer-Human Interaction Conference · doi:10.1145/3726986.3727036 · Cited by 4 · see citing works
Viago: Exploring Visual-Audio Modality Transitions for Social Media Consumption on the Go
Chang, R.-C., Grossman, T., Rognon, C., Glueck, M., Collins, C. M., Karlson, A. K., & Surale, H. B. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747761 · Cited by 1 · see citing works
Ouyang, Y., Gao, S., Wang, R., Zhu, H., Shao, Y., Gu, X., & Li, Q. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790530 · Cited by 1 · see citing works
Mind-Archive: A Hybrid Framework for Real-Time Affective Lifelogging in Extended Reality
Han, B., Zawar‐Reza, P., & Piumsomboon, T. (2025) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct68609.2025.00171
WACU: Multi-Modal Wristband Assistant for Contextual Understanding
Patsch, C., Goter, J., Greer, J., Ma, L., & Sodhi, R. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00749
LAPPI: Interactive Optimization With LLM-Assisted Preference-Based Problem Instantiation
Kuroki, S., Nakagawa, M., Yoshida, S., Koyama, Y., & Kozuno, T. (2026) · IEEE Access · doi:10.1109/access.2026.3679052 · Cited by 1 · see citing works
Fan, H., Yuan, Y., Hsia, C., Liang, C., & Peng, Z. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798509
Lost in Translation: The Value of Verbalizations in Interpreting 3D Computer-Aided Design Workflows
Cheng, K., Vermeulen, J., Fitzmaurice, G. W., & Matejka, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791084
Wang, J. P., Cai, R., Xu, S., Hu, T., Qu, H., Zhao, S., & Yuan, L.-P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791348
Self-citationEmpowered XR through Generative AI: Balancing Superpowers and Risks
Tang, Y., Wu, M., Situ, J., Liu, J., Cui, A. Y., & Huang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791307
Dong, Z., Han, B., Zhang, J., Ens, B., Clark, A. J., & Piumsomboon, T. (2025) · International Conference on Advancements in Computational Sciences · doi:10.1145/3789418.3789450
CityNarrator: A Multimodal Narrative Agent for Citywalk
Zhang, S. T., & Bu, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30549-7_15
Hautasaari, A., Kibayashi, Y., Chujo, R., Hatada, Y., & Naemura, T. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816234
Wang, D., Qiu, L., Cai, L., Jiang, Y., Fan, M., & Wang, Z. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2713822
- Authors
- Cai, R., Janaka, N., Chen, Y., Wang, L., Zhao, S., & Liu, C.
- Affiliations
- National University of Singapore (SG)
- Massachusetts Institute of Technology (US)
- City University of Hong Kong (HK)
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-24
- OpenAlex
- W4396832991
- Cited by
- 36
- References
- 78
- Citations by year
- 2026: 10 · 2025: 21 · 2024: 5
- FWCI
- 11.57
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- CityU Scholars (submittedVersion, cc-by)
- Funders
- National Research Foundation Singapore (AISG2-RP-2020-016)
- City University of Hong Kong
- Ministry of Education - Singapore (MOE-T2EP20221-0010)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Keywords
- AI-assisted writing, optical see-through head-mounted displays, travel writing, narrative documentation, large language models, user effort reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
