Don’t Just Tell Me, Ask Me: AI Systems that Intelligently Frame Explanations as Questions Improve Human Logical Discernment Accuracy over Causal AI explanations
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Critical thinking is an essential human skill. Despite the importance of critical thinking, research reveals that our reasoning ability suffers from personal biases and cognitive resource limitations, leading to potentially dangerous outcomes. This paper presents the novel idea of AI-framed Questioning that turns information relevant to the AI classification into questions to actively engage users’ thinking and scaffold their reasoning process. We conducted a study with 204 participants comparing the effects of AI-framed Questioning on a critical thinking task; discernment of logical validity of socially divisive statements. Our results show that compared to no feedback and even causal AI explanations of an always correct system, AI-framed Questioning significantly increase human discernment of logically flawed statements. Our experiment exemplifies a future style of Human-AI co-reasoning system, where the AI becomes a critical thinking stimulator rather than an information teller.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Examples include asking individuals to make independent predictions before receiving AI input [16, 101, 105] or using 'slow algorithms' to reduce reliance on AI. Additionally, providing AI explanations without concrete recommendations [42] and using AI-framed questioning [25] have been shown to enhance critical thinking.”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [25]
In the citing paper
“It enhances AI assistants by fostering deeper analytical thinking, encouraging users to form independent opinions before seeing AI suggestions (as in the Cognitive Forcing Function approach [16]) and stimulating critical thinking through AI-generated questions (similar to [25]).”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [25]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“To generate the causal AI explanations in our study, we used the large language model "GPT-3"[6].”
MethodAn integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 3×
“This paper deploys the defnition of trust used in [40] as "the willingness of a [user] to be vulnerable to the actions of [an AI system] based on the expectation that the [AI system] will perform a particular action important to the [user], irrespective of the ability to monitor or control [the AI system]".”
BackgroundMethodCognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Perreault, K. (2011) · Manual Therapy · doi:10.1016/j.math.2010.09.003 · Cited by 4,759
Pennycook, G., & Rand, D. G. (2019) · Cognition · doi:10.1016/j.cognition.2018.06.011 · Cited by 2,022 · cited 2×
“First, it has been shown to be more efective in changing people’s beliefs than if they are merely presented with corrective information[47].”
BackgroundBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · cited 1×
“Since the reasoning of a system is often abstract and can be hard to understand, additional research into what makes a proper explanation is required so as to design and engineer AI generated explanations that are more natural and user friendly[9, 14, 33, 43].”
BackgroundPennycook, G., & Rand, D. G. (2020) · Journal of Personality · doi:10.1111/jopy.12476 · Cited by 879
Assessing miserly information processing: An expansion of the Cognitive Reflection Test
Toplak, M. E., West, R. F., & Stanovich, K. E. (2014) · Thinking & Reasoning · doi:10.1080/13546783.2013.844729 · Cited by 864 · cited 3×
“To measure the level of critical thinking of subjects, we used cognitive refection test (CRT), a task designed to measure a person’s ability to refect on a question and resist reporting the frst response that comes to mind [13].”
MethodInfluentialDesigning Experiments and Analyzing Data: A Model Comparison Perspective.
Serlin, R. C., Maxwell, S. E., & Delaney, H. D. (1991) · Journal of the American Statistical Association · doi:10.2307/2289744 · Cited by 812 · cited 1×
“Further post hoc tests with Benjamini-Hochberg correction were conducted to identify how intervention conditions difer from each other[39].”
Method
Research works printed but not linked: 21
- Roberto Confalonieri, Tarek R Besold, Tillman Weyde, Kathleen Creel, Tania Lombrozo, Shane Mueller, and Patrick Shafto. 2019. What makes a good expla- nation? Cognitive dimensions of explaining intelligent machines. CogSci 2019: Creativity+ Cognition+ Computation (2019).
- Edward M Glaser. 1985. Educating for responsible citizenship in a democracy. In National Forum, Vol. 65. Honor Society of Phi Kappa Phi, 24.
- Ivan Habernal, Rafael Hannemann, Christian Pollak, Christopher Klamm, Patrick Pauli, and Iryna Gurevych. 2017. Argotario: Computational argumentation meets serious games. arXiv preprint arXiv:1707.06002 (2017). · arXiv
- Jonathan Haidt. 2001. The emotional dog and its rational tail: a social intuitionist approach to moral judgment. Psychological review 108, 4 (2001), 814.
- Jonathan Howard. 2019. Hasty Generalization, Survival Bias, Special Pleading, and Burden of Proof. In Cognitive Errors and Diagnostic Mistakes. Springer, 211-246.
- Daniel Kahneman. 2011. Thinking, fast and slow. Macmillan.
- Emily R Lai. 2011. Critical thinking: A literature review. Pearson's Research Reports 6, 1 (2011), 40-41.
- David MJ Lazer, Matthew A Baum, Yochai Benkler, Adam J Berinsky, Kelly M Greenhill, Filippo Menczer, Miriam J Metzger, Brendan Nyhan, Gordon Penny- cook, David Rothschild, Michael Schudson, Steven A. Sloman, Cass R. Sunstein, Emily A. Thorson, Duncan J. Watts, and Jonathan L. Zittrain. 2018. The science of fake news. Science 359, 6380 (2018), 1094-1096.
- Jennifer S Lerner and Philip E Tetlock. 1999. Accounting for the efects of accountability. Psychological bulletin 125, 2 (1999), 255.
- Matthew D Lieberman. 2000. Intuition: a social cognitive neuroscience approach. Psychological bulletin 126, 1 (2000), 109.
- Mark McCafrey, Paige Hayes, Jason Wagner, and Matt Hobbs. 2018. Consumer Intelligence Series: Prepare for the voice revolution. pwc. com (2018).
- Divyam Mehta, Aniket Dwivedi, Arunabha Patra, and M Anand Kumar. 2021. A transformer-based architecture for fake news classifcation. Social network analysis and mining 11, 1 (2021), 1-12. CHI '23, April 23-28, 2023, Hamburg, Germany
- Richard Paul and Linda Elder. 2007. Critical thinking: The art of Socratic ques- tioning. Journal of developmental education 31, 1 (2007), 36.
- Gordon Pennycook, Ziv Epstein, Mohsen Mosleh, Antonio A Arechar, Dean Eckles, and David G Rand. 2021. Shifting attention to accuracy can reduce misinformation online. Nature 592, 7855 (2021), 590-595.
- Herbert A Simon. 1956. Rational choice and the structure of the environment. Psychological review 63, 2 (1956), 129.
- Herbert Alexander Simon. 1997. Models of bounded rationality: Empirically grounded economic reason. Vol. 3. MIT press.
- Jay J Van Bavel and Andrea Pereira. 2018. The partisan brain: An Identity-based model of political belief. Trends in cognitive sciences 22, 3 (2018), 213-224.
- Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. science 359, 6380 (2018), 1146-1151.
- Darrell M West. 2018. The future of work: Robots, AI, and automation. Brookings Institution Press.
- Cassandra Xia. 2014. A game-based intervention for the reduction of statistical cognitive biases. Ph. D. Dissertation. Massachusetts Institute of Technology.
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. Advances in neural information processing systems 32 (2019).
Cited by
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490
A Roadmap to Pluralistic AlignmentPreprint
Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.05070 · Cited by 485
“…to represent multiple different perspectives can be valuable in creative production (Shanahan & Clarke, 2023), psychological inquiry (Shanahan et al., 2023), simulating social systems (Park et al., 2022), and deliberative discourse (Danry et al., 2023; Landemore & Page, 2015; Page, 2019; 2008).”
BackgroundAI makes you smarter but none the wiser: The disconnect between performance and metacognition
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108779 · Cited by 188
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645199 · Cited by 142
“In this study, we explore whether LLMs can contribute to decision making processes by playing the role of devil’s advocate to encourage human decision makers to engage in constructive deliberation, through generating critique questions and comments to provoke human reasoning [21].”
BackgroundThe Role of Causality in Explainable Artificial Intelligence
Carloni, G., Berti, A., & Colantonio, S. (2025) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70015 · Cited by 129
Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2024) · arXiv · doi:10.48550/arxiv.2409.16708 · Cited by 71
Buçinca, Z., Swaroop, S., Paluch, A. E., Doshi‐Velez, F., & Gajos, K. Z. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713229 · Cited by 76
“In terms of design, approaches where the AI critiques each decision after it is made ( e.g. , contrastive after) may undermine users’ feelings of competence and autonomy, as they could perceive the AI more as a micromanager than a supportive tool, constantly pointing out flaws and dictating its…”
Park, S., Subramonyam, H., & Kulkarni, C. E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.06024 · Cited by 62
DATATALES: Investigating the use of Large Language Models for Authoring Data-Driven Articles
Sultanum, N., & Srinivasan, A. (2023) · 2023 IEEE Visualization and Visual Analytics (VIS) · doi:10.1109/vis54172.2023.00055 · Cited by 52
“…we should consider ways to better equip authors with critical thinking skills: from facilitating fact checking and encouraging all statements to be fact-checked, to nudging techniques that encourage refection, e.g., phrasings that “ask a question” instead of stating “what and why” [6].”
BackgroundAI as Extraherics: Fostering Higher-order Thinking Skills in Human-AI InteractionPreprint
Yatani, K., Sramek, Z., & Yang, C.-L. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.09218 · Cited by 64
“Danry et al. demonstrate the positive effects of this strategy on the critical thinking task of evaluating the logical soundness of statements that can create social division [29].”
BackgroundMei, P., Brewis, D. N., Nwaiwu, F., Sumanathilaka, D., Alva-Manchego, F., & Demaree-Cotton, J. (2025) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2025.100140 · Cited by 50
“…has, however, looked at the problem through interrogating adjacent processes, such as investigating whether framing GenAI outputs as questions (Danry et al., 2023) or forcing a ‘pause ’ (Buçinca et al., 2021) can improve elements of critical thinking, whether LLMs can be used to promote…”
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
Curiosity in Consumer Behavior: A Systematic Literature Review and Research Agenda
Strzelecki, A., Jaciow, M., & Wolny, R. (2024) · International Journal of Consumer Studies · doi:10.1111/ijcs.70001 · Cited by 56
Offline Reinforcement Learning for Adaptive Support in AI-Assisted Decision-Making
Buçinca, Z., Swaroop, S., Paluch, A. E., Murphy, S. A., & Gajos, K. Z. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3815418 · Cited by 54
“Each of these interaction techniques aims to address the pitfalls of SXAI, with the goal of eliciting cognitive engagement to reduce overreliance on AI support [13, 28, 81], or to develop decision-makers’ skills in the long-term [15, 40].”
InfluentialGöndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 52
Sarkar, A. (2024) · Communications of the ACM · doi:10.1145/3649404 · Cited by 40
“Rather than optimise speed and efficiency, a provocateur engages in discussions, offers counterarguments, and asks questions [8] to stimulate our thinking.”
"It makes you think": Provocations Help Restore Critical Thinking to AI-Assisted Knowledge WorkPreprint
Drosos, I., Sarkar, A., Xiaotong, Xu, & Toronto, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.17247 · Cited by 43
Danry, V., Pataranutaporn, P., Groh, M., & Epstein, Z. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713408 · Cited by 35
“This experiment exemplifies a future style of Human-AI co-reasoning system [12, 13], where the AI becomes a critical thinking stimulator rather than a mere decision-maker [13].”
InfluentialSelf-citationde Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Guo, A., Pataranutaporn, P., & Maes, P. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650892 · Cited by 32
“…during a session [27], what a more customized UI or assistant could do to improve the process, or when the AI system proactively asks questions [7] and makes suggestions to guide the ideation process, and how user perception of ownership and agency affects the life of the brainstormed idea…”
Self-citationA Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and Creativity
Li, J., Li, J., & Su, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60615-1_5 · Cited by 44
Amplifying Minority Voices: AI-Mediated Devil's Advocate System for Inclusive Group Decision-Making
Lee, S., Kim, M., Hwang, S., Kim, D.-J., & Lee, K. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708557.3716334 · Cited by 42
Lee, S., Hwang, S., Kim, D.-J., & Lee, K. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719792 · Cited by 39
Dango: A Mixed-Initiative Data Wrangling System using Large Language Model
Chen, W., Tong, W., Case, A., & Zhang, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714135 · Cited by 41
“They often struggle with finding the right narratives to effectively communicate with LLMs.”
Prabhudesai, S., Kasi, A. P., Mansingh, A., Antar, A. D., Shen, H., & Banović, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713714 · Cited by 35
Exploring the Potential of Metacognitive Support Agents for Human-AI Co-Creation
Gmeiner, F., Luo, K., Wang, Y., Holstein, K., & Martelaro, N. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735785 · Cited by 30
Tankelevitch, L., Glassman, E. L., He, J., Kittur, A., Lee, M., Palani, S., Sarkar, A., Ramos, G., Rogers, Y., & Subramonyam, H. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.21036 · Cited by 22
“Sitting at one extreme are designs intended to directly challenge our thinking through provocation [14, 33] or questioning [18, 23], discussed below in §3.1.”
InfluentialPareek, S., Schömbs, S., Velloso, E., & Gonçalves, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713468 · Cited by 27
“…model to generate scenario texts allowed us to maintain consistency across scenarios, minimising unintended variability and ensuring variations in participant responses can be attributed to our manipulations rather than random differences in stimulus wording, following past research [8, 15].”
Kaur, H., Conrad, M. R., Rule, D., Lampe, C. A. C., & Gilbert, É. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637354 · Cited by 33
“…[16, 95, 128, 137]; how to present explanations to lay users, novices, and experts [100, 133]; which types of uncertainty information are helpful [48, 103, 139]; what causes information overload [13, 119]; whether explanations should presented as questions, answers, or a dialogue [23, 69, 70]; etc.”
BackgroundVibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaboration
Li, J., Hou, Y., Lin, L., Zhu, R., Cao, H., & Ali, A. E. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808062 · Cited by 23
“Teams adopt new governance strategies such as rotating “AI editor” roles, shared prompting conventions, and lightweight mechanisms to track model use [19, 57, 76], especially in light of disclosure requirements [24].”
The New Calculator? Practices, Norms, and Implications of Generative AI in Higher EducationPreprint
Simkute, A., Kewenig, V. N., Sellen, A. J., Rintel, S., & Tankelevitch, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.08864 · Cited by 22
Ferguson, S. A., Aoyagui, P. A., Rizvi, R., Kim, Y., & Kuzminykh, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687056 · Cited by 21
Countering Misinformation in Private Messaging Groups: Insights From a Fact-checking Chatbot
Lee, T.-H., & Fussell, S. R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701189 · Cited by 21
“For example, the AI system developed by Danry et al. [21] urges people to assess and make decisions for themselves rather than accepting AI systems’ suggestions at face value.”
Language Models as Critical Thinking Tools: A Case Study of PhilosophersPreprint
Ye, A., Moore, J., Novick, R., & Zhang, A. X. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.04516 · Cited by 21
“Park & Kulkarni (2024) show how LMs as “thinking assistants” can facilitate human self-reflection by asking, instead of answering, questions.”
BackgroundImpact of Voice Fidelity on Decision Making: A Potential Dark Pattern?
Dubiel, M., Sergeeva, A., & Leiva, L. A. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645202 · Cited by 16
“Alternatively, the system could offer more incentives for users to reassess their decisions, promoting deliberate and user-centred choices [18, 23].”
BackgroundTang, H., Lenzini, G., Greiff, S., Rohles, B., & Sergeeva, A. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661510 · Cited by 23
“For instance, a system that prompts users to question the characteristics of information can enhance their defense against incorrect details [35].”
BackgroundFramework for human–XAI symbiosis: extended self from the dual-process theory perspective
Litvinova, Y., Mikalef, P., & Luo, X. R. (2024) · Journal of Business Analytics · doi:10.1080/2573234x.2024.2396366 · Cited by 20
“For example, Danry et al. (2020, 2023) conducted two studies on an explainable wearable reasoner designed to help users evaluate arguments on different topics.”
Exploring the Impacts of ChatGPT on Future Scientific WorkPreprint
Irons, J. L., Mason, C., Cooper, P. S., Sidra, S., Reeson, A. F., & Paris, C. L. (2023) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/j2u9x · Cited by 18
Yamamoto, Y. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1382234 · Cited by 15
Behavioral Indicators of Overreliance During Interaction with Conversational Language Models
Liu, C., Zhou, Q., Shen, X., Liu, X. B., Wu, T., & Chen, X. '. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790332 · Cited by 25
Tang, H., Sun, S., Nie, K., Li, A., Sergeeva, A., & Lc, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3748600 · Cited by 14
“Although other attempts have used LLMs to help users learn about misinformation [29, 58] (including gamified attempts [137]), in these approaches, AI was mostly the source of correcting information.”
Lee, S.-H., Hwang, S., & Lee, K. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.11125 · Cited by 16
“AI systems can enhance decision-making processes by encouraging reflective thinking; instead of providing direct answers, they can ask questions that stimulate deeper analysis [11].”
BackgroundHow Can I Signal You To Trust Me: Investigating AI Trust Signalling in Clinical Self-Assessments
Kollerup, N. K., Wester, J., Skov, M. B., & van Berkel, N. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661612 · Cited by 16
“A signifcant amount of research within human-AI trust-building relies on Mayer’s proposed defnition of trust [47, 66]—with further examples such as behavioural perceptions of trust [28, 31] and ABI as trust measurement questionnaire [22].”
BackgroundChen, B., Wen, R., Tan, S., & Li, Y. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720067 · Cited by 19
A Taxonomy of Questions for Critical Reflection in Machine-Assisted Decision-Making
Fischer, S. W. S., Schraffenberger, H., Thill, S., & Haselager, P. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i1.36602 · Cited by 15
“As we will show, we utilise the mapping from questions to XAI techniques and reverse the process to re-purpose explanations to formulate possible questions, e.g., “Is it relevant to focus on factor x ?”.”
InfluentialLi, H. X., Appleby, G., & Suh, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2404.01425 · Cited by 16
“Along both of these lines of work, future research should address the ‘best ways’ for an LLM to explain itself [20] when returning results from its interactions with the KG.”
BackgroundScott, A. E., Tankelevitch, L., Panda, P., Vanukuru, R., Chen, X., & Rintel, S. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729204 · Cited by 17
PaperTrail: A Claim-Evidence Interface for Grounding Provenance in LLM-based Scholarly Q&A
Martin, A., Leckey, C. A. C., Brown, M. C., & Kaur, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791101 · Cited by 18
Aoyagui, P. A., Stemmler, K., Ferguson, S. A., Kim, Y., & Kuzminykh, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713248 · Cited by 18
“…and calls for trust calibration mechanisms [115, 121], one popular approach suggests it might be appropriate, in some use cases, for AI to offer the pros and cons [81] or pose questions [22] that will stimulate critical thinking [95] and potentially lead to appropriate reliance levels [19].”
Mining Evidence about Your Symptoms: Mitigating Availability Bias in Online Self-Diagnosis
Zhang, J., Zhu, Z., Li, J., & Lee, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713805 · Cited by 14
Lin, H.-C. K., Feng, C.-H., & Liu, Y.-L. E. (2025) · International Journal of Technology and Design Education · doi:10.1007/s10798-025-09993-y · Cited by 13
“These results align with Danry et al. (2023), who observed that applying learned information does not inherently require the critical reflection GAI tools can stimulate.”
Ma, Z., Mei, Y., Bruderlein, C., Gajos, K. Z., & Pan, W. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729194 · Cited by 28
Impact of Agent-Generated Rationales on Online Social Conformity
de Jong, S., Jacobsen, R. M., Wester, J., Wijenayake, S., Gonçalves, J., & van Berkel, N. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732217 · Cited by 13
“An effective way to enhance the diversity of ideas is to have LLMs actively challenge users’ assumptions, fostering reflection and critical thinking [21, 45, 68, 104].”
Faas, C., Kerstan, S., Uth, R., Langer, M., & Feit, A. M. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789100 · Cited by 15
Genç, H. U., Chandrasegaran, S. K., Dingler, T., & Verma, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713579 · Cited by 16
“This approach is supported by research into belief bias, which shows that users tend to over-rely on AI explanations when they align with their prior beliefs, making it crucial for adaptive systems to counterbalance such biases by promoting critical engagement [35, 51].”
Prasongpongchai, T., Pataranutaporn, P., Lertsutthiwong, M., & Maes, P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3715579 · Cited by 14
Self-citationPassi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 15
Understanding the Effects of AI-Assisted Critical Thinking on Human-AI Decision Making
Tian, H. Y., Amin, H., & Yin, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790785 · Cited by 14
Saleh, M., & Tabatabaei, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.13199 · Cited by 13
“This section will study this issue in selected Vision Language tasks.”
Lovett, N. (2026) · Human Resource Development Review · doi:10.1177/15344843261470602 · Cited by 9
Impacts of AI on Human Designers: A Systematic Literature Review
Sinlapanuntakul, P., & Zachry, M. R. (2025) · IEEE Transactions on Professional Communication · doi:10.1109/tpc.2025.3588655 · Cited by 11
“Future research should identify strategies and design systems that promote divergent thinking, challenge designers to explore alternative perspectives, and mitigate fixation risk while leveraging AI’s benefits [81], [82], [83].”
Reinhard, P., Li, M. M., Peters, C., & Leimeister, J. M. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10716-4 · Cited by 13
“Additionally, Naiseh et al. (2023) measured if users switched from the AI recommendation, and Danry et al. (2023) observed weighted discernment.”
Bertrand, A., Eagan, J., Maxwell, W., & Brand, J. L. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642326 · Cited by 10
Koçyiğit, E., Rossi, A., Sergeeva, A., Negri-Ribalta, C., Farjami, A., & Lenzini, G. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732129 · Cited by 9
Provoking critical thinking: Using counter-arguments in online discussion summarisation
Li, S., Han, L., & Demartini, G. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104258 · Cited by 9
Lyu, W., Wang, Y., Yue, M., Sun, Y., Suh, J., Kier, M. W., Yao, Z., & Zhang, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791138 · Cited by 11
“Table 3 summarizes the observed tensions in AI-assisted CPS (RQ1) to student-proposed responses from PD (RQ2).”
Dialogues with AI Reduce Beliefs in Misinformation but Build No Lasting Discernment Skills
Rani, A., Danry, V., Liang, P. P., Lippman, A., & Maes, P. (2025) · International Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790656 · Cited by 11
“Participants using AI systems that confidently “tell” rather than “ask”, report going along with the system because it sounds knowledgeable, thereby reducing their own engagement in critical evaluation [18].”
InfluentialSelf-citationUnderstanding and Supporting Formal Email Exchange by Answering AI-Generated Questions
Miura, Y., Yang, C.-L., Kuribayashi, M., Matsumoto, K., Kuzuoka, H., & Morishima, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714016 · Cited by 17
“AI-driven questioning, inspired by Socratic methods, has been shown to promote critical thinking and improve users’ ability to identify logical fallacies [19].”
Joshi, N., & Vogel, D. (2026) · Graphics Interface · doi:10.48550/arxiv.2507.03670 · Cited by 8
Adaptive Human-Agent Teaming: A Review of Empirical Studies from the Process Dynamics PerspectivePreprint
Wang, M., Wu, J., Ma, S., Li, N., Zhang, P., Gu, N., & Lu, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.10918 · Cited by 8
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6
“Users in clinical and non-clinical settings perceived systems with these cognitive forcing strategies as less useful than other AI systems [8, 25].”
Danry, V., Pataranutaporn, P., Cui, C., Hung, J.-T., Blanchard, L., Buçinca, Z., Tan, C., Starner, T. E., & Maes, P. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706719 · Cited by 9
Self-citationSarkar, A., Xiaotong, Xu, Toronto, N., Drosos, I., & Poelitz, C. (2024) · arXiv · doi:10.48550/arxiv.2412.15030 · Cited by 5
The interaction between emotion dynamics and opinion changes in the era of generative AI
Li, S., Fan, S., & Demartini, G. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100722 · Cited by 7
Sun, Y., Yue, W., Jin, X., Ma, S., Ma, X., & Wang, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757705 · Cited by 6
“…syndrome differentiation skills through iterative, contextualized reasoning, we suggest adopting existing educational methods (e.g., “Socratic” question-based interaction method [19, 122], problem-based learning [46]) to enhance students’ active thinking in diagnostic and therapeutic plans.”
Leveraging imperfection with MEDLEY: a multi-model approach harnessing bias in medical AI
Abtahi, F., Astaraki, M., & Seoane, F. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1701665 · Cited by 5
Lee, S., Hwang, S., Kim, D. S., & Lee, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.14263 · Cited by 6
Socratic Methodologies and Artificial Intelligence against Disinformation: A Philosophical Approach
Stamatellos, G. (2025) · The International Journal of the Platonic Tradition · doi:10.1163/18725473-12341567 · Cited by 4
Shape n’ Swarm: Hands-on, Shape-aware Generative Authoring with Swarm UI and LLMs
Jeung, M., Sathya, A., Qian, W., Arellano, S., Jimenez, L., & Nakagaki, K. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747781 · Cited by 8
“” These observations provide unique insight into how tangible interaction can assist user ideation when working with LLMs. Recent research in Human-AI Interaction has explored LLM-based systems that nudge users to think critically [8, 66].”
Gong, Y., Liu, Y., Shang, L., Wei, N., & Wang, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757577 · Cited by 6
“Following the method suggested by Danry et al. [16], we calculated the WMDA score by considering for both the binary accuracy rating and a confidence level on a scale of 1-7 (1 = “Not confident at all” and 7 = “Very confident”).”
Son, K., Choi, D., Kim, T. S., Kim, Y.-H., Yun, S., & Kim, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791680 · Cited by 7
“To overcome these limitations and effectively capture professionals’ cognitive decision-making processes in their worklfow, we derive the following three design goals: DG1: Structure the professional’s workflow based on their cognitive decision-making process through their self- explanations,…”
Xu, Z., Zhang, J., Tang, A., & Lee, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790298 · Cited by 4
“This finding resonates with prior work [20, 65] and Piaget’s “cognitive conflict” theory [53].”
Korosec-Serfaty, M., Léger, P., Parent‐Rocheleau, X., & Sénécal, S. (2026) · Information Systems Frontiers · doi:10.1007/s10796-025-10691-2 · Cited by 4
Danry, V., Hernandez, J., Wilson, A. D., Maes, P., & Amores, J. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830698 · Cited by 4
Self-citationPanigrahi, I., Kim, S. S. Y., Liaqat, A., Jinturkar, R., Russakovsky, O., Fong, R., & Abtahi, P. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719730 · Cited by 5
Frater, M., & Mushininga, R. (2025) · International Journal of Research in Business and Social Science (2147-4478) · doi:10.20525/ijrbs.v14i2.3678 · Cited by 6
“…technologies like Robotic Process Automation (RPA) and AI-based decision support systems (Roa et al, 59 2021:296; Ruiz-Real et al, 2020:108), while individual users benefit from tools like OpenAI's ChatGPT for knowledge access and problem-solving (Danry et al, 2023:2-5; Grimes et al, 2023:1617).”
WordCraft: Scaffolding the Keyword Method for L2 Vocabulary Learning with Multimodal LLMs
Shao, Y., Xiong, J., Wu, C., Wang, X., Zhou, Z., Ouyang, Y., Tao, Q., & Li, Q. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790668 · Cited by 3
Questioning AI: Promoting Decision-Making Autonomy Through ReflectionPreprint
Fischer, S. W. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.10250 · Cited by 4
“Although this system has not yet been fullyimplemented and evaluated, another study found that framing causal explanations as questions can improve human judgement [5].”
BackgroundReflexa: Uncovering How LLM-Supported Reflection Scaffolding Reshapes Creativity in Creative CodingPreprint
Wang, A., Li, Z., Luo, L., Tong, X., & Hui, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.17769 · Cited by 3
Yong, S., Yang, C.-L., Hashimoto, A., Kuzuoka, H., & Yoshida, S. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757646 · Cited by 3
The Influence of Chat GPT Use on University Students’ Critical Thinking Skills
Seo, J.-S., & Lim, S.-Y. (2025) · 한국가정관리학회지 · doi:10.7466/jfbl.2025.43.3.109 · Cited by 2
Generative AI in Foreign Language Inquiry-Based Learning: A Qualitative Study of Student Experiences
Chen, M. (2025) · International Conference on Digital Society and Artificial Intelligence (DSAI) · doi:10.1145/3748825.3748851 · Cited by 2
Guided Reflection in AI-Assisted Decision-Making: Effects on AI Overreliance and Decision Accuracy
Li, S., Li, J., Li, H., Zhu, H., & Li, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790632 · Cited by 8
Understanding Compliance and Conversion Dynamics in Multi-Agent Collectives
Lee, S., & Lee, K. (2026) · International Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790385 · Cited by 1
“On the other hand, when agents ask questions instead of giving answers, they foster metacognition and critical thinking, which helps reduce blind conformity [17, 72].”
Stankowski, A., Varab, D., Clarke, M., Gschwendtner, P., & Avramidis, E. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3813061 · Cited by 2
“Socratic questioning approaches identify dialogic affordances for scaffolding thought, where systems support users in articulating reasoning rather than providing answers [20].”
Confidence Without Competence in AI-Assisted Knowledge Work
Eleftheriou, E., Pallis, G., & Constantinides, M. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808055 · Cited by 2
Liu, J., Wu, D., Sun, G., & Deng, Y. (2024) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24969 · Cited by 2
“The reasoning question is considered an effective method to evoke users' thinking (Danry et al., 2023).”
Innovative Question Types and Formats
Trajkovski, G., & Hayes, H. (2025) · Digital education and learning · doi:10.1007/978-3-031-88252-4_3 · Cited by 2
Jiménez, P. P., Andrés Mejía, D., & Pascual, J. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016607 · Cited by 3
“AI-framed questioning actively engages reasoning, improves logical evaluation, and outperforms traditional feedback methods [24].”
Lee, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.15328 · Cited by 4
Metadata
- Authors
- Danry, V., Pataranutaporn, P., Mao, Y., & Maes, P.
- Affiliations
- MIT Media Lab (US)
- Columbia University (US)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-13
- OpenAlex
- W4366590585
- Cited by
- 110
- References
- 38
- Citations by year
- 2026: 39 · 2025: 50 · 2024: 19 · 2023: 2
- FWCI
- 19.53
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Keywords
- critical thinking, cognitive biases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.