AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Open accessAbstract
The rise of powerful large language models (LLMs) brings about tremendous opportunities for innovation but also looming risks for individuals and society at large. We have reached a pivotal moment for ensuring that LLMs and LLM-infused applications are developed and deployed responsibly. However, a central pillar of responsible AI -- transparency -- is largely missing from the current discourse around LLMs. It is paramount to pursue new approaches to provide transparency for LLMs, and years of research at the intersection of AI and human-computer interaction (HCI) highlight that we must do so with a human-centered perspective: Transparency is fundamentally about supporting appropriate human understanding, and this understanding is sought by different stakeholders with different goals in different contexts. In this new era of LLMs, we must develop and design approaches to transparency by considering the needs of stakeholders in the emerging LLM ecosystem, the novel types of LLM-infused applications being built, and the new usage patterns and challenges around LLMs, all while building on lessons learned about how people process, interact with, and make use of information. We reflect on the unique challenges that arise in providing transparency for LLMs, along with lessons learned from HCI and responsible AI research that has taken a human-centered perspective on AI transparency. We then lay out four common approaches that the community has taken to achieve transparency -- model reporting, publishing evaluation results, providing explanations, and communicating uncertainty -- and call out open questions around how these approaches may or may not be applied to LLMs. We hope this provides a starting point for discussion and a useful roadmap for future research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
End-user studies suggest that prompting is challenging, with non-expert users making various errors and adopting ineffective strategies-a reflection of the demand on users' metacognitive monitoring and control.
“End-user studies suggest that prompting is challenging, with nonexpert users making various errors and adopting ineffective strategies-a reflection of the demand on users' metacognitive monitoring and control [33,41,43,85,103,174,205].”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [103]
The available copy’s abstract (p. 2) discusses human-centered LLM transparency but does not address end-user prompting errors, ineffective strategies, or metacognitive monitoring and control. Searches for those terms returned only the abstract and publication metadata; the linked article text is not present in the indexed copy, so the claim cannot be checked here. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical UsagePreprint
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v1 · Cited by 429 · see citing works
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 280 · see citing works
“Well-adjusted confidence in prompting ability Metacognitive flexibility to adapt prompting strategy”
BackgroundInfluentialFriha, O., Ferrag, M. A., Kantarcı, B., Çakmak, B., Ozgun, A., & Ghoualmi‐Zine, N. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3456549 · Cited by 209 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v4 · Cited by 173 · see citing works
Explainable Generative AI (GenXAI): a survey, conceptualization, and research agenda
Schneider, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10916-x · Cited by 184 · see citing works
“Many of our novel aspects, in particular, cannot be found in prior works.”
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shankar, S., Zamfirescu-Pereira, J. D., Hartmann, B., Parameswaran, A., & Arawjo, I. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676450 · Cited by 151 · see citing works
A Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136 · see citing works
Suchikova, Y., Tsybuliak, N., Teixeira da Silva, J. A., & Nazarovets, S. (2026) · Accountability in Research · doi:10.1080/08989621.2025.2544331 · Cited by 70 · see citing works
“A well-structured taxonomy delineates the tasks of both humans and GAIs, ensuring clarity and transparency for researchers, institutions, and journal editors adopting AI-driven technology in their processes (Khalifa and Albadawy 2024; Liao and Wortman Vaughan 2024).”
Ironies of Generative AI: Understanding and Mitigating Productivity Loss in Human-AI Interaction
Simkute, A., Tankelevitch, L., Kewenig, V., Scott, A. E., Sellen, A., & Rintel, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2405782 · Cited by 89 · see citing works
“This is exacerbated by GenAI tools’ ability to produce outputs at a capacity too demanding for adequate evaluation, with questionable reliability, and with poor explainability (Chen et al., 2023; Liao & Vaughan, 2023; Schellaert et al., 2023).”
InfluentialHadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., et al. (2023) · doi:10.36227/techrxiv.23589741 · Cited by 88 · see citing works
Navigating LLM ethics: advancements, challenges, and future directions
Jiao, J., Afroogh, S., Xu, Y., & Phillips, C. (2025) · AI and Ethics · doi:10.1007/s43681-025-00814-5 · Cited by 125 · see citing works
Knowledge Editing for Large Language Models: A Survey
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., & Li, J. (2025) · ACM Computing Surveys · doi:10.1145/3698590 · Cited by 79 · see citing works
“In KME, the update of a specific piece of knowledge in LLMs is typically formulated as an edit , such as rectifying the answer to “ Who is the president of the USA?”
Cultural Bias in Large Language Models: A Comprehensive Analysis and Mitigation Strategies
Liu, Z. (2025) · Journal of Transcultural Communication · doi:10.1515/jtc-2023-0019 · Cited by 77 · see citing works
“Liao & Vaughan (2023) argues that transparency is crucial for the responsible development and deployment of LLMs, despite the challenges posed by their complex capabilities, massive architectures, proprietary nature, diverse stakeholders, and evolving public perception.”
How Beginning Programmers and Code LLMs (Mis)read Each Other
Nguyen, S., Babe, H. M., Zi, Y., Guha, A., Anderson, C. J., & Feldman, M. Q. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642706 · Cited by 81 · see citing works
Beyond Traditional Assessment: Exploring the Impact of Large Language Models on Grading Practices
Fagbohun, O., Iduwe, N. P., Abdullahi, M., Ifaturoti, A., & Nwanna, O. (2024) · Journal of Artificial Intelligence, Machine Learning and Data Science · doi:10.51219/jaimld/oluwole-fagbohun/19 · Cited by 68 · see citing works
“The development of automatic scoring algorithms began over a decade ago, primarily using token-based models in student responses, often referred to as the ‘bag of words’ approach 15,27 .”
Barman, K. G., Wood, N. G., & Pawłowski, P. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09778-2 · Cited by 69 · see citing works
The Art of Creative Inquiry—From Question Asking to Prompt Engineering
Sasson-Lazovsky, G., Raz, T., & Kenett, Y. N. (2024) · The Journal of Creative Behavior · doi:10.1002/jocb.671 · Cited by 69 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v3 · Cited by 62 · see citing works
Hoque, M. N., Mashiat, T., Ghai, B., Shelton, C. D., Chevalier, F., Kraus, K., & Elmqvist, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641895 · Cited by 48 · see citing works
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46 · see citing works
Hadi, M. U., Al-Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Hassan, S. Z., et al. (2023) · doi:10.36227/techrxiv.23589741.v2 · Cited by 47 · see citing works
Rapp, A., Lodovico, C. D., & Di, L. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103471 · Cited by 49 · see citing works
Leveraging generative AI for clinical evidence synthesis needs to ensure trustworthiness
Zhang, G., Jin, Q., Jered McInerney, D., Chen, Y., Wang, F., Cole, C. L., Yang, Q., Wang, Y., Malin, B. A., Peleg, M., et al. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104640 · Cited by 40 · see citing works
Designing Home Automation Routines Using an LLM-Based Chatbot
Giudici, M., Padalino, L., Paolino, G., Paratici, I., Pascu, A. I., & Garzotto, F. (2024) · Designs · doi:10.3390/designs8030043 · Cited by 40 · see citing works
“Finally, following the principles presented by Liao and Vaughan [59], the application design aimed to keep users well informed about ongoing processes through essential and straightforward visuals.”
Method“As an AI language model, I cannot”: Investigating LLM Denials of User Requests
Wester, J., Schrills, T., Pohl, H., & van Berkel, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642135 · Cited by 40 · see citing works
“Therefore, it is important that users are provided with clear and relatable explanations [27], an aspect that was absent in baseline denials.”
BackgroundSafeguarding human values: rethinking US law for generative AI’s societal impacts
Cheong, I., Caliskan, A., & Kohno, T. (2024) · AI and Ethics · doi:10.1007/s43681-024-00451-4 · Cited by 39 · see citing works
“This blind trust poses a fundamental threat to individual autonomy, especially considering the difficulty of identifying information sources and potential bias in LLMs compared to other machine learning models, where explainability techniques have been developed in recent years [191].”
RELIC: Investigating Large Language Model Responses using Self-Consistency
Cheng, F., Zouhar, V., Arora, S., Sachan, M., Strobelt, H., & El‐Assady, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641904 · Cited by 40 · see citing works
Ridley, M. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24889 · Cited by 51 · see citing works
“As Liao and Vaughan point out, which version of the LLM is being explained (e.g., the pre-trained model, the adapted model, or the application based on either of the models) determines the explanatory requirements (Liao & Vaughan, 2023).”
InfluentialLarge language models for whole-learner support: opportunities and challenges
Mannekote, A., Davies, A., Pinto, J. D., Zhang, S., Olds, D., Schroeder, N. L., Lehman, B., Zapata-Rivera, D., & Zhai, C.-X. (2024) · Frontiers Artif. Intell. · doi:10.3389/frai.2024.1460364 · Cited by 42 · see citing works
'I'm Categorizing LLM as a Productivity Tool': Examining Ethics of LLM Use in HCI Research Practices
Kapania, S., Wang, R., Li, T. J.-J., Li, T., & Shen, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711000 · Cited by 36 · see citing works
Applications of Generative AI in Healthcare: algorithmic, ethical, legal and societal considerationsPreprint
Okonji, O. R., Yunusov, K., & Gordon, B. (2024) · doi:10.36227/techrxiv.171527587.75649430/v1 · Cited by 47 · see citing works
“Vera Liao et al. research [90] on Large Language Models (LLMs) identified challenges in transparency due to complex architectures and proprietary technologies.”
BackgroundMcLean, A. L., Wu, Y., McLean, A. C. L., & Hristidis, V. (2024) · Journal of Cancer Research and Clinical Oncology · doi:10.1007/s00432-024-05673-x · Cited by 33 · see citing works
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33 · see citing works
The impact of AI perceived transparency on trust in AI recommendations in healthcare applications
Shabankareh, M., Sahne, S. S. K., Nazarian, A., & Foroudi, P. (2025) · Asia-Pacific Journal of Business Administration · doi:10.1108/apjba-12-2024-0690 · Cited by 35 · see citing works
Jin, Y., & Ma, J. (2024) · Sustainable Cities and Society · doi:10.1016/j.scs.2024.105940 · Cited by 28 · see citing works
Fostering effective hybrid human-LLM reasoning and decision making
Passerini, A., Gema, A. P., Minervini, P., Sayin, B., & Tentori, K. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1464690 · Cited by 35 · see citing works
“(Dellermann et al., 2019; Akata et al., 2020; Lawrence, 2024; Wang et al., 2024; Ma et al., 2024; Liao and Wortman Vaughan, 2024).”
Guesmi, M., Chatti, M. A., Joarder, S., Ain, Q. U., Alatrash, R., Siepmann, C., & Vahidi, T. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2262797 · Cited by 29 · see citing works
“Therefore, the design choices in XAI should be driven by users’ explanation needs and goals (Chatti et al., 2022; Ehsan & Riedl, 2020; Liao & Varshney, 2021; Liao & Vaughan, 2023; D. Shin, 2021).”
Bridging large language model disparities: skill tagging of multilingual educational content
Kwak, Y., & Pardos, Z. A. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13465 · Cited by 25 · see citing works
Haurogné, J., Basheer, N., & Islam, S. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100598 · Cited by 33 · see citing works
Generative AI and the Metaverse: A Scoping Review of Ethical and Legal Challenges
Tabassum, A., Elmahjub, E., Padela, A. I., Zwitter, A., & Qadir, J. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3536082 · Cited by 26 · see citing works
“Scholars stress the need for AI systems to provide understandable and interpretable rationales for their decisions, integrating transparency with explainability [26], [17].”
Practices, opportunities and challenges in the fusion of knowledge graphs and large language models
Cai, L., Yu, C., Kang, Y., Fu, Y., Zhang, H., & Zhao, Y. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1590632 · Cited by 30 · see citing works
Gong, Y., Shang, L., & Wang, D. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3404236 · Cited by 26 · see citing works
“This research gap is significant given the current limitations of GPT, such as tendencies toward hallucinations and producing unreliable outputs [135].”
BackgroundSecret Use of Large Language Model (LLM)
Zhang, Z., Shen, C., Yao, B., Wang, D., & Li, T. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711061 · Cited by 24 · see citing works
“Transparency, as one of the responsible AI principles, has been significantly impacted by LLMs [57].”
He, G., Demartini, G., & Gadiraju, U. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713218 · Cited by 23 · see citing works
“Due to the capability of generating coherent, knowledgeable, and high-quality responses to diverse human input [106], a wide community of human-computer interaction researchers has paid attention to large language models [59].”
New sources of inaccuracy? A conceptual framework for studying AI hallucinations
Shao, A. (2025) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-182 · Cited by 22 · see citing works
“Communication researchers can explore not just what is true but why systems generate certain responses and how people interpret them (Liao & Vaughan, 2024; Ozmen Garibay et al., 2023; Schäfer, 2023).”
AI agent in healthcare: applications, evaluations, and future directions
Zhao, L., Liu, S., Xin, T., Tan, J., Wang, X., Li, Y., Bian, Z., Chen, Y., Kong, F., Bian, J., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00076-4 · Cited by 26 · see citing works
Examining Humanness as a Metaphor to Design Voice User Interfaces
Desai, S., Dubiel, M., & Leiva, L. A. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665535 · Cited by 21 · see citing works
“We report on a study with 240 participants examining the effects of human versus non-human metaphors on user perceptions within health and finance domains.”
BackgroundExploring Large Language Models for Personalized Recipe Generation and Weight-Loss Management
Ataguba, G., & Orji, R. (2025) · ACM Transactions on Computing for Healthcare · doi:10.1145/3712709 · Cited by 20 · see citing works
Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability
de Souza, R. P., Skluzacek, T. J., Wilkinson, S. R., et al. (2023) · IEEE International Conference on eScience · doi:10.1109/e-science58273.2023.10254822 · Cited by 20 · see citing works
“Provenance (a.k.a. lineage) data management provides valuable support for reproducing, tracing, understanding, and explaining data, models, and their transformation processes [7], [8].”
BackgroundCanvil: Designerly Adaptation for LLM-Powered User Experiences
Feng, K. J. K., Liao, Q. V., Xiao, Z., Vaughan, J. W., Zhang, A. X., & McDonald, D. W. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713139 · Cited by 28 · see citing works
“Our work opens new avenues for processes and tools that foreground designers’ human-centered expertise when developing LLM-powered applications.”
Self-citationBuilding Appropriate Mental Models: What Users Know and Want to Know about an Agentic AI Chatbot
Brachman, M., Kunde, S., Miller, S., Fucs, A., Dempsey, S., Jabbour, J., & Geyer, W. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712071 · Cited by 18 · see citing works
Private Yet Social: How LLM Chatbots Support and Challenge Eating Disorder Recovery
Choi, R., Kim, T., Park, S., Kim, J. G., & Lee, S.-J. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713485 · Cited by 19 · see citing works
The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment
Johnson, N., Moharana, S., Harrington, C., Andalibi, N., Heidari, H., & Eslami, M. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658910 · Cited by 20 · see citing works
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · doi:10.31219/osf.io/ac4sp · Cited by 51 · see citing works
Dehkordi, M. K. H., Perl, Y., Deek, F. P., He, Z., Keloth, V. K., Liu, H., Elhanan, G., & Einstein, A. J. (2025) · JMIR Medical Informatics · doi:10.2196/66476 · Cited by 16 · see citing works
Ding, Z., Smith, A., Zhang, W., Tetreault, J., & Jaimes, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.217 · Cited by 20 · see citing works
Explainability for Transparent Conversational Information-Seeking
Łajewska, W., Spina, D., Trippas, J. R., & Balog, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657768 · Cited by 18 · see citing works
“…limitations or potential pitfalls in the response generation needed to enable appropriate understanding and assessment, in contrast to functional understanding of what the system can do, by exposing its capabilities and limitations or mechanistic understanding focused on how the system works [33].”
BackgroundMapping the individual, social and biospheric impacts of Foundation Models
Hernández, A. D., Krishna, S., Perini, A., Katell, M. A., Bennett, S., Borda, A., Hashem, Y., Hadjiloizou, S., Mahomed, S., Jayadeva, S., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658939 · Cited by 20 · see citing works
Kellogg, K. C., Lifshitz‐Assaf, H., Randazzo, S., Mollick, E., Dell’Acqua, F., McFowland, E., Candelon, F., & Lakhani, K. R. (2025) · Information and Organization · doi:10.1016/j.infoandorg.2025.100559 · Cited by 18 · see citing works
“…be taken by system deployers to mitigate accuracy risks: a) communicate to users the intended conditions under which GenAI could be used reliably (Liao & Vaughan, 2023); b) provide co-audit tools (Mündler et al., 2023); c) create a prompt library (Svendsen & Garvey, 2023); d) continually assess…”
InfluentialImproving Governance Outcomes Through AI Documentation: Bridging Theory and Practice
Winecoff, A., & Bogen, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713814 · Cited by 16 · see citing works
“Transparency in AI is widely regarded as essential for calibrating trust and supporting accountability [42].”
Self-Directing AI: The Road to Fully Autonomous AI Agents
Jovanovic, M., & Campbell, M. (2025) · Computer · doi:10.1109/mc.2024.3507513 · Cited by 15 · see citing works
APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users
Desolda, G., Greco, F., & Vigano, L. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3733049 · Cited by 16 · see citing works
“…LLM using prompt engineering are different from those generated by other more traditional methods (like eXplainable AI) and surely have several drawbacks [7, 12, 43, 61]; e.g., LLMs may suffer from hallucinations, the generated explanations may not reflect the internal processes of the model, etc.”
Transparency and Accountability
Pappachan, P., Moslehpour, M., Bansal, R., & Rahaman, M. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-3860-5.ch006 · Cited by 15 · see citing works
Harnessing the power of language models in cybersecurity: A comprehensive review
Kaur, R., Klobučar, T., & Gabrijelčič, D. (2024) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2024.100315 · Cited by 16 · see citing works
Tang, B., Sun, K., Curran, N. T., Schaub, F., & Shin, K. G. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770640 · Cited by 12 · see citing works
“The LLM must be able to adapt on-the-fly to user’s ad requests and maintain a consistent record of the set of products served as advertisements to the user.”
Challenges and Opportunities for Responsible Prompting
de Santana, V. F. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636268 · Cited by 12 · see citing works
Perera, M., Ananthanarayan, S., Goncu, C., & Marriott, K. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713634 · Cited by 14 · see citing works
Future Shock: Grappling With the Generative AI Revolution
Leslie, D., & Meng, X. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.fad6d25c · Cited by 11 · see citing works
A Longitudinal Study of Child Wellbeing Assessment via Online Interactions with a Social Robot
Abbasi, N. I., Laban, G., Ford, T., Jones, P. B., & Güneş, H. (2025) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3722123 · Cited by 16 · see citing works
Turri, V., Morrison, K., Robinson, K.-M., Abidi, C., Perer, A., Forlizzi, J., & Dzombak, R. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658985 · Cited by 10 · see citing works
CUI@CHI 2024: Building Trust in CUIs—From Design to Deployment
Desai, S., Wei, C. Z., Sin, J., Dubiel, M., Zargham, N., Ahire, S., Porcheron, M., Kuzminykh, A., Lee, M., Candello, H. C. D. S. P., et al. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636287 · Cited by 9 · see citing works
“Given the ‘black-box’ nature of conversational interfaces, users frequently rely on anthropic tendencies [28].”
BackgroundHow Can I Signal You To Trust Me: Investigating AI Trust Signalling in Clinical Self-Assessments
Kollerup, N. K., Wester, J., Skov, M. B., & van Berkel, N. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661612 · Cited by 9 · see citing works
“While LLMs show promise in a range of domains and tasks, challenges and risks are increasingly highlighted in HCI [30, 42, 75].”
BackgroundAnalyzing Multimodal Interaction Strategies for LLM-Assisted Manipulation of 3D Scenes
Chen, J., Grubert, J., & Kristensson, P. O. (2025) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr59515.2025.00045 · Cited by 9 · see citing works
“This corroborates findings from Liao et al. [23] who state that interaction with LLM systems with only the natural language modality can be easily affected by subtle language and communication cues.”
MetricMate: An Interactive Tool for Generating Evaluation Criteria for LLM-as-a-Judge Workflow
Gebreegziabher, S. A., Chiang, C., Wang, Z., Ashktorab, Z., Brachman, M., Geyer, W., Li, T. J.-J., & Gómez-Zará, D. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729199 · Cited by 10 · see citing works
“LLM evaluation and auditing involve domain experts or crowd workers rating the model’s outputs [32, 51].”
Gururaja, S., Bertsch, A., Na, C., Widder, D. G., & Strubell, E. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.822 · Cited by 12 · see citing works
“More recently, there has been discussion of the increasing prevalence of closed-source models in NLP and how that will shape the field and its innovations (Rogers, 2023; Solaiman, 2023; Liao and Vaughan, 2023).”
BackgroundExploring Trust and Transparency in Retrieval-Augmented Generation for Domain Experts
Ravi, D., & Sindhgatta, R. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719985 · Cited by 9 · see citing works
The problem of atypicality in LLM-powered psychiatry
Garcia, B., Chua, E. Y. S., & Brah, H. (2025) · Journal of Medical Ethics · doi:10.1136/jme-2025-110972 · Cited by 9 · see citing works
“26 This has partly to do with the opacity of LLMs 42 43 — because they have to operate with so many variables, we have to settle for experimenting and later figure out what techniques show better performance.”
Rokhshad, R., Tichý, A., Ducret, M., Chaurasia, A., Uribe, S. E., Gambelin, O., & Schwendicke, F. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.106187 · Cited by 8 · see citing works
Using large language models for survey research in communication: opportunities and challenges
Valenzuela, S., Winter, S., & Rivera, S. (2025) · Communication and Change · doi:10.1007/s44382-025-00014-z · Cited by 8 · see citing works
“A significant distinction exists between closed proprietary models like ChatGPT and more open models like Llama, each presenting different advantages and limitations regarding transparency and accessibility (Liao & Vaughan, 2023).”
Ethical AI in the workplace: ensuring fairness and transparency
Rauhala, M., Drake, M., & Saaranen, P. (2025) · AI and Ethics · doi:10.1007/s43681-025-00903-5 · Cited by 8 · see citing works
Human-Centered Evaluation and Auditing of Language Models
Liu, Y. L., Deng, W. H., Lam, M. S., Eslami, M., Kim, J., Liao, Q. V., Xu, W., Novikova, J., & Xiao, Z. (2025) · Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706599.3706729 · Cited by 7 · see citing works
“Collectively, members of the organizing committee contributed to HCI and AI on various topics, including Explainable AI [29, 31, 58 Wei Xu is an Associate Professor in the College of Computing at Georgia Tech.”
Self-citationEmerging Data Practices: Data Work in the Era of Large Language Models
Garcia, A. A., Candello, H. C. D. S. P., Badillo-Urquiola, K. A., & Wong-Villacrés, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714069 · Cited by 8 · see citing works
“Technical reports and machine learning research suggest that LLM practitioners are also facing decision tensions on other fronts [15, 67, 76, 120, 155].”
Lyo, S. K., & Cook, T. S. (2025) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2025.07.003 · Cited by 7 · see citing works
Designing an LLM-based copilot for manufacturing equipment selection
Werheid, J., Melnychuk, O., Zhou, H. A., et al. (2025) · Manufacturing Letters · doi:10.1016/j.mfglet.2025.10.017 · Cited by 6 · see citing works
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 6 · see citing works
Lei, D., He, Y., & Zeng, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60606-9_13 · Cited by 5 · see citing works
Less or More: Towards Glanceable Explanations for LLM Recommendations Using Ultra-Small Devices
Wang, X., Yu, M., Nguyen, H., Iuzzolino, M. L., Wang, T., Tang, P., Lynova, N., Tran, C., Zhang, T., Sendhilnathan, N., et al. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712074 · Cited by 5 · see citing works
“The emergence of LLMs raised unique challenges when applying traditional XAI methods to explain them due to the size of their training datasets and non-deterministic outputs [34].”
Leveraging LLMs for Collaborative Human-AI Decision Making
Favier, A., Verma, P., La, N., & Shah, J. (2025) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v5i1.35556 · Cited by 5 · see citing works
Koçyiğit, E., Rossi, A., Sergeeva, A., Negri-Ribalta, C., Farjami, A., & Lenzini, G. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732129 · Cited by 5 · see citing works
Towards Expert Models Deployment Cost Optimization in Edge Computing Networks
Ren, J., Wang, C., Zhong, Y., et al. (2025) · International Conference on Communications · doi:10.1109/icc52391.2025.11161045 · Cited by 4 · see citing works
“From the observations in [2], (i) unlike human clients accessing LLMs sporadically and randomly, machine clients invoke LLMs with higher frequencies, greater volumes, and longer lifecycles; and (ii) the quantity of machine-type LLM communication requests will exceed that of human clients by 10 to…”
Fede, G. D., Alrabie, L., & Andolina, S. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_49-2 · Cited by 4 · see citing works
Transformation of University Students’ Educational Agency in the Era of AI Technologies
Konkol, M. M., & Эдуардовна, Ш. Е. (2026) · Высшее образование в России · doi:10.31992/0869-3617-2026-35-2-36-52 · Cited by 7 · see citing works
Responsible Generative AI: A Comprehensive Study to Explain LLMs
Shruti, I., Kumar, A., Seth, A., et al. (2024) · doi:10.1109/icecet61485.2024.10698306 · Cited by 4 · see citing works
Wencker, T., Borst, J., & Niekler, A. (2025) · Evaluation · doi:10.1177/13563890251330911 · Cited by 5 · see citing works
Investigating Context-Aware Collaborative Text Entry on Smartphones using Large Language Models
Chen, W., Shi, Y., Wang, Y., Shi, W., Chen, M., Cheng, G., Mei, Y., Zhu, Y., Zhang, J., & Yu, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713944 · Cited by 4 · see citing works
“Users may choose to ignore this information after becoming familiar with the system, but its presence is crucial for providing transparency, establishing trust in the AI assistant, and reducing metacognitive demands, particularly in complex or dynamic contexts [2, 13, 34, 35, 53].”
Fede, G. D., Alrabie, L., & Andolina, S. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_49-1 · Cited by 4 · see citing works
Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance
Srinivasan, T., & Thomason, J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789136 · Cited by 7 · see citing works
Braga, V. (2023) · Journal of Engineering Science · doi:10.52326/jes.utm.2023.30(3).07 · Cited by 3 · see citing works
“Data and Model Transparency : transparency is key to the responsible use of LLMs [58].”
BackgroundFanfiction in the Age of AI: Community Perspectives on Creativity, Authenticity and Adoption
Alfassi, R., Cooper, A., Mitchell, Z., Calabro, M., Shaer, O., & Mokryn, O. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2531272 · Cited by 4 · see citing works
“However, when user-generated content is repur-posed as training data for commercial AI models, these participatory dynamics are subverted.”
- Authors
- Liao, Q. V., & Vaughan, J. W.
- Affiliations
- Microsoft Research New York City (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Issue
- Special Issue 5
- ISSN
- 2644-2353 (ISSN-L)
- arXiv
- 2306.01941
- OpenAlex
- W4392297945
- Semantic Scholar
- 3dcf2db20082b480c6c091eea025465cc4fe57a6
- Cited by
- 209
- References
- 0
- Citations by year
- 2026: 85 · 2025: 70 · 2024: 42 · 2023: 12
- FWCI
- 29.13
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Harvard Data Science Review (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- AI transparency, large language models, responsible AI, human-centered design, explanation, uncertainty communication, stakeholder needs, human-computer interaction, research roadmap
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
