The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making
Abstract
Our study of 20 knowledge workers revealed a common challenge: the difficulty of synthesizing unstructured information scattered across multiple platforms to make informed decisions. Drawing on their vision of an ideal knowledge synthesis tool, we developed Yodeai, an AI-enabled system, to explore both the opportunities and limitations of AI in knowledge work. Through a user study with 16 product managers, we identified three key requirements for Generative AI in knowledge work: adaptable user control, transparent collaboration mechanisms, and the ability to integrate background knowledge with external information. However, we also found significant limitations, including overreliance on AI, user isolation, and contextual factors outside the AI's reach. As AI tools become increasingly prevalent in professional settings, we propose design principles that emphasize adaptability to diverse workflows, accountability in personal and collaborative contexts, and context-aware interoperability to guide the development of human-centered AI systems for product managers and knowledge workers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Braun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
Prospect Theory: An Analysis of Decision Under Risk
Kahneman, D., & Tversky, A. (2013) · World Scientific Handbook in Financial Economics Series · doi:10.1142/9789814417358_0006 · Cited by 3,091
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 3×
“This literature emphasizes the nuances of joint decision-making, as user background knowledge can affect the problem-solving process and results [28], with trust and explainability playing key roles in the adoption of generative AI applications [1].”
Experimental evidence on the productivity effects of generative artificial intelligence
Noy, S., & Zhang, W. (2023) · Science · doi:10.1126/science.adh2586 · Cited by 2,150
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.12712 · Cited by 1,526
Exploratory search: from finding to understanding
Marchionini, G. (2006) · Communications of the ACM · doi:10.1145/1121949.1121979 · Cited by 1,506
Unifying Large Language Models and Knowledge Graphs: A Roadmap
Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3352100 · Cited by 1,376
Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4573321 · Cited by 794
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764 · cited 1×
“Furthermore, the potential for hallucinations in the model [42] and biases [82] raise concerns about the validity of purely AI-generated outputs.”
Simon, H. A. (1990) · Utility and Probability · doi:10.1007/978-1-349-20568-4_5 · Cited by 705
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · cited 2×
“The design of human-AI interactions presents challenges due to uncertainty in the capabilities and the complexity of the output [95].”
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29720 · Cited by 556
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Podstawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.09687 · Cited by 556
Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
(2025) · doi:10.1145/3706598 · Cited by 534
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493 · cited 1×
“AI tools must support an interactive feedback loop for prioritization [81], enabling PMs to delve deeper into feature or bug details to understand how they align with broader business goals or evaluate trade-offs, such as when an engineer notes the high implementation cost of a large feature.”
Security and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232 · cited 1×
“The flexibility to create multiple instances of each widget, with varying levels of user control from fully automated to user-directed, encourages experimentation and iteration [37, 83, 84].”
Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models
Suh, S., Min, B., Palani, S., & Xia, H. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606756 · Cited by 151
Sensecape: Enabling Multilevel Exploration and Sensemaking with Large Language Models
Suh, S., Min, B., Palani, S., & Xia, H. (2023) · arXiv · doi:10.48550/arxiv.2305.11483 · Cited by 151
Deductive and Inductive Approaches to Qualitative Data Analysis: The Five-Cycle Process
Bingham, A. (2021) · Proceedings of the 2021 AERA Annual Meeting · doi:10.3102/1682697 · Cited by 148
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137
Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information Seeking
Sharma, N., Liao, Q. V., & Xiao, Z. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642459 · Cited by 132
Inkpen, K., Chappidi, S., Mallari, K., Nushi, B., Ramesh, D., Michelucci, P., Mandava, V., Vepřek, L. H., & Quinn, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3534561 · Cited by 132 · cited 1×
“This literature emphasizes the nuances of joint decision-making, as user background knowledge can affect the problem-solving process and results [28], with trust and explainability playing key roles in the adoption of generative AI applications [1].”
Social Dynamics of AI Support in Creative Writing
Gero, K. I., Long, T., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580782 · Cited by 129
Rundo, L., Pirrone, R., Vitabile, S., Sala, E., & Gambino, O. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103479 · Cited by 113
Citation: A Key to Building Responsible and Accountable Large Language Models
Huang, J., & Chang, K. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.31 · Cited by 72
Kobiella, C., Flores López, Y. S., Waltenberger, F., Draxler, F., & Schmidt, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641964 · Cited by 57
Kim, T. S., Lee, Y., Chang, M., & Kim, J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606833 · Cited by 56
How Knowledge Workers Use and Want to Use LLMs in an Enterprise Context
Brachman, M., El-Ashry, A., Dugan, C., & Geyer, W. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650841 · Cited by 37
Supporting Metacognition during Exploratory Search with the OrgBox
Crescenzi, A., Ward, A. R., Li, Y., & Capra, R. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3462955 · Cited by 25
Research works printed but not linked: 69
- Marc Annacchino. 2003. New product development: from initial idea to product management. Butterworth-Heinemann.
- Ofer Bergman Ruth Beyth-Marom and Rafi Nachmias. 2003. The user-subjective approach to personal information management systems. Journal of the American Society for Information Science and Technology 54 9 (2003) 872–878. · DOI
- Michael Bernstein Max Van Kleek David Karger and MC Schraefel. 2008. Information scraps: How and why information eludes our personal information management tools. ACM Transactions on Information Systems (TOIS) 26 4 (2008) 1–46. · DOI
- 10.1145/985692.985766 · DOI · Paper page
- 10.1145/3379337.3415865 · DOI
- 10.1145/3180308.3180362 · DOI · Paper page
- 10.1145/3331184.3331309 · DOI
- 10.1145/3526113.3545672 · DOI
- Peter J. Denning. 2006. Infoglut. Commun. ACM 49 7 (jul 2006) 15–19. 10.1145/1139922.1139936 · doi:10.1145/1139922.1139936
- Eva Eigner and Thorsten Händler. 2024. Determinants of LLM-assisted Decision-Making. arXiv preprint arXiv:https://arXiv.org/abs/2402.17385 (2024). · arXiv
- Tyna Eloundou Sam Manning Pamela Mishkin and Daniel Rock. 2023. Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:https://arXiv.org/abs/2303.10130 (2023). · arXiv
- Daniel A Epstein Clara Caldeira Mayara Costa Figueiredo Xi Lu Lucas M Silva Lucretia Williams Jong Ho Lee Qingyang Li Simran Ahuja Qiuer Chen et al. 2020. Mapping and taking stock of the personal informatics literature. Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies 4 4 (2020) 1–38. · DOI · Paper page
- 10.1145/2702123.2702561 · DOI · Paper page
- Md Mahadi Hassan Alex Knipper and Shubhra Kanti Karmaker Santu. 2023. Chatgpt as your personal data scientist. arXiv preprint arXiv:https://arXiv.org/abs/2305.13657 (2023). · arXiv
- 10.1017/CBO9781139644082 · DOI · Paper page
- 10.1145/3607720.3607785 · DOI
- Joel Jang, Seonghyeon Ye, and Minjoon Seo. 2023. Can large language models truly understand prompts? a case study with negated prompts. In Transfer learning for natural language processing workshop. PMLR, 52–62.
- 10.1145/3586183.3606737 · DOI
- William Jones. 2010. Keeping found things found: The study and practice of personal information management. Morgan Kaufmann.
- William P Jones and Jaime Teevan. 2007. Personal information management. University of Washington Press.
- 10.1145/3526113.3545660 · DOI
- 10.1145/3586183.3606759 · DOI
- Shivani Kapania Ruiyi Wang Toby Jia-Jun Li Tianshi Li and Hong Shen. 2024. " I’m categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices. arXiv preprint arXiv:https://arXiv.org/abs/2403.19876 (2024). · arXiv
- Vivian Lai Chacha Chen Q Vera Liao Alison Smith-Renner and Chenhao Tan. 2021. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:https://arXiv.org/abs/2112.11471 (2021). · arXiv
- 10.1145/3603555.3603565 · DOI · Paper page
- Binbin Li Tianxin Meng Xiaoming Shi Jie Zhai and Tong Ruan. 2023. MedDM: LLM-executable clinical guidance tree for clinical decision-making. arXiv preprint arXiv:https://arXiv.org/abs/2312.02441 (2023). · arXiv
- 10.1145/1753326.1753409 · DOI · Paper page
- Jie Li Hancheng Cao Laura Lin Youyang Hou Ruihao Zhu and Abdallah El Ali. 2023. User Experience Design Professionals’ Perceptions of Generative Artificial Intelligence. arXiv preprint arXiv:https://arXiv.org/abs/2309.15237 (2023). · arXiv
- 10.1145/3332165.3347908 · DOI
- Michael Xieyang Liu Tongshuang Wu Tianying Chen Franklin Mingzhe Li Aniket Kittur and Brad A Myers. 2023. Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models. arXiv preprint arXiv:https://arXiv.org/abs/2310.02161 (2023). · arXiv
- Tao Long Katy Ilonka Gero and Lydia B Chilton. 2024. Not Just Novelty: A Longitudinal Study on Utility and Customization of AI Workflows. arXiv preprint arXiv:https://arXiv.org/abs/2402.09894 (2024). · DOI
- Paul Benjamin Lowry, Tom Roberts, and Bryce Eryn Caine. 2005. The effect of familiar logos on trust of websites designed for small user interfaces. In 11th International Conference on Human-Computer Interaction (HCII 2005), Las Vegas, Nevada, July, Citeseer. 22–27.
- Xiao Ma Swaroop Mishra Ariel Liu Sophie Su Jilin Chen Chinmay Kulkarni Heng-Tze Cheng Quoc Le and Ed Chi. 2023. Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses. arXiv preprint arXiv:https://arXiv.org/abs/2312.00763 (2023). · arXiv
- 10.1145/3586182.3625115 · doi:10.1145/3586182.3625115
- Catherine C Marshall. 2008. Rethinking personal digital archiving Part 1: Four challenges from the field. D-Lib Magazine 14 3 (2008) 2. · doi:10.1045/march2008-marshall-pt1
- Catherine C Marshall. 2008. Rethinking personal digital archiving part 2: implications for services applications and institutions. D-Lib Magazine 14 3 (2008) 3. · doi:10.1045/march2008-marshall-pt2
- Imani Munyaka Zahra Ashktorab Casey Dugan James Johnson and Qian Pan. 2023. Decision Making Strategies and Team Efficacy in Human-AI Teams. Proceedings of the ACM on Human-Computer Interaction 7 CSCW1 (2023) 1–24. · DOI
- 10.1145/3526113.3545696 · DOI
- 10.1145/3491102.3517672 · DOI
- 10.5555/2785659 · doi:10.5555/2785659
- Vijay Pereira Elias Hadjielias Michael Christofi and Demetris Vrontis. 2023. A systematic literature review on the impact of artificial intelligence on workplace outcomes: A multi-process perspective. Human Resource Management Review 33 1 (2023) 100857. · DOI · Paper page
- Jos Pierson An Jacobs Katrien Dreessen B Lievens I Van den Broeck and Wendy Van den Broeck. 2006. Walking the interface: uncovering practices through proxy technology assessment. Proc. of EPIC 2006 (2006) 40–54. · doi:10.1111/j.1559-8918.2006.tb00033.x
- Crystal Qian and James Wexler. 2024. Take It Leave It or Fix It: Measuring Productivity and Trust in Human-AI Collaboration. arXiv preprint arXiv:https://arXiv.org/abs/2402.18498 (2024). · arXiv
- Sukumar Rathnam and Michael V Mannino. 1995. Tools for building the human-computer interface of a decision support system. Decision Support Systems 13 1 (1995) 35–59. · DOI
- Gary R Schirr. 2013. User research for product innovation: Qualitative methods. The PDMA handbook of new product development (2013) 231. · doi:10.1002/9781118466421.ch14
- 10.1145/3351095.3372827 · DOI · Paper page
- Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. 2023. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning. PMLR, 31210–31227.
- Yang Shi Tian Gao Xiaohan Jiao and Nan Cao. 2023. Understanding design collaboration between designers and artificial intelligence: A systematic literature review. Proceedings of the ACM on Human-Computer Interaction 7 CSCW2 (2023) 1–35. · DOI · Paper page
- Rolf Sint, Sebastian Schaffert, Stephanie Stroka, and Roland Ferstl. 2009. Combining unstructured, fully structured and semi-structured information in semantic wikis. In CEUR workshop proceedings , Vol. 464. Citeseer, 73–87.
- 10.1007/978-3-031-49425-3_5 · DOI
- 10.5555/38407 · doi:10.5555/38407
- Sangho Suh Meng Chen Bryan Min Toby Jia-Jun Li and Haijun Xia. 2023. Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation. arXiv preprint arXiv:https://arXiv.org/abs/2310.12953 (2023). · arXiv
- Justin D Weisz Michael Muller Jessica He and Stephanie Houde. 2023. Toward general design principles for generative AI applications. arXiv preprint arXiv:https://arXiv.org/abs/2301.05578 (2023). · arXiv
- Matt Welsh. 2022. The end of programming. Commun. ACM 66 1 (2022) 34–35. · DOI
- Steve Whittaker. 2011. Personal information management: from information consumption to curation. Annual review of information science and technology 45 1 (2011) 1. · DOI · Paper page
- Steve Whittaker Victoria Bellotti and Jacek Gwizdka. 2006. Email in personal information management. Commun. ACM 49 1 (2006) 68–73. · DOI · Paper page
- Steve Whittaker and Julia Hirschberg. 2001. The character value and management of personal paper archives. ACM Transactions on Computer-Human Interaction (TOCHI) 8 2 (2001) 150–170. · DOI · Paper page
- Steve Whittaker and Candace Sidner. 1996. Email overload: exploring personal information management of email. In Proceedings of the SIGCHI conference on Human factors in computing systems. 276–283.
- Tom D Wilson. 1999. Models in information behaviour research. Journal of documentation 55 3 (1999) 249–270. · DOI · Paper page
- Allison Woodruff Renee Shelby Patrick Gage Kelley Steven Rousso-Schindler Jamila Smith-Loud and Lauren Wilcox. 2023. How Knowledge Workers Think Generative AI Will (Not) Transform Their Industries. arXiv preprint arXiv:https://arXiv.org/abs/2310.06778 (2023). · DOI
- Siye Wu Jian Xie Jiangjie Chen Tinghui Zhu Kai Zhang and Yanghua Xiao. 2024. How Easily do Irrelevant Inputs Skew the Responses of Large Language Models? arXiv preprint arXiv:https://arXiv.org/abs/2404.03302 (2024). · arXiv
- Tianbao Xie Fan Zhou Zhoujun Cheng Peng Shi Luoxuan Weng Yitao Liu Toh Jing Hua Junning Zhao Qian Liu Che Liu et al. 2023. Openagents: An open platform for language agents in the wild. arXiv preprint arXiv:https://arXiv.org/abs/2310.10634 (2023). · arXiv
- 10.1145/3526113.3545647 · DOI
- Ryan Yen Nicole Sultanum and Jian Zhao. 2024. To Search or To Gen? Exploring the Synergy between Generative AI and Web Search in Programming. arXiv preprint arXiv:https://arXiv.org/abs/2402.00764 (2024). · DOI
- Ryan Yen Jiawen Zhu Sangho Suh Haijun Xia and Jian Zhao. 2023. CoLadder: Supporting Programmers with Hierarchical Code Generation in Multi-Level Abstraction. arXiv preprint arXiv:https://arXiv.org/abs/2310.08699 (2023). · arXiv
- Yuliang Yun Dexin Ma and Meihong Yang. 2021. Human–computer interaction-based decision support system with applications in data mining. Future Generation Computer Systems 114 (2021) 285–289. · DOI
- 10.1145/3544548.3581388 · DOI · Paper page
- Shao Zhang Jianing Yu Xuhai Xu Changchang Yin Yuxuan Lu Bingsheng Yao Melanie Tory Lace M Padilla Jeffrey Caterino Ping Zhang et al. 2023. Rethinking human-ai collaboration in complex medical decision making: A case study in sepsis diagnosis. arXiv preprint arXiv:https://arXiv.org/abs/2309.12368 (2023). · arXiv
- Yizhen Zheng Huan Yee Koh Jiaxin Ju Anh TN Nguyen Lauren T May Geoffrey I Webb and Shirui Pan. 2023. Large language models for scientific synthesis inference and explanation. arXiv preprint arXiv:https://arXiv.org/abs/2310.07984 (2023). · arXiv
Metadata
- Authors
- Yun, B., Feng, D., Chen, A. S., Nikzad, A., & Salehi, N.
- Venue
- Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2025-04-25
- Pages
- 1-19
- Cited by
- 38
- References
- 31
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.