Improving Human Verification of LLM Reasoning through Interactive Explanation Interfaces
Open accessThe DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
The reasoning capabilities of Large Language Models (LLMs) have led to their increasing employment in several critical applications, particularly education, where they support problem-solving, tutoring, and personalized study. While there are a plethora of works showing the effectiveness of LLMs in generating step-by-step solutions through chain-of-thought (CoT) reasoning on reasoning benchmarks, little is understood about whether the generated CoT is helpful for end-users in improving their ability to comprehend mathematical reasoning problems and detect errors/hallucinations in LLM-generated solutions. To address this gap and contribute to understanding how reasoning can improve human-AI interaction, we present three new interactive reasoning interfaces: interactive CoT (iCoT), interactive Program-of-Thought (iPoT), and interactive Graph (iGraph), and a novel framework that generates the LLM’s reasoning from traditional CoT to alternative, interactive formats. Across 125 participants, we found that interactive interfaces significantly improved performance. Specifically, the iGraph interface yielded the highest clarity and error detection rate (\(85.6\%\)), followed by iPoT (\(82.5\%\)), iCoT (\(80.6\%\)), all outperforming standard CoT (\(73.5\%\)). Interactive interfaces also led to faster response times, where participants using iGraph were fastest (57.9 secs), compared to iCoT and iPoT (60 secs), and the standard CoT baseline (64.7 secs). Furthermore, participants prefer…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048 · cited 1×
“Our design aligns with the results from Sweller’s study, which claims that working memory has a limited capacity and instructional designs should aim to minimize unnecessary mental effort [48].”
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Response strategies for coping with the cognitive demands of attitude measures in surveys
Krosnick, J. A. (1991) · Applied Cognitive Psychology · doi:10.1002/acp.2350050305 · Cited by 2,460 · cited 1×
“Prior research indicates that prolonged verification tasks lead to declining attention and accuracy [23, 24, 62].”
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · cited 2×
“Large language models (LLMs) are now woven into high-stakes, real-world workflows, including education, where they assist with problem solving, tutoring, and personalized study [12, 22, 53].”
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863 · cited 1×
“In contrast, we prompt LLMs to generate an interactive HTML/JavaScript application wrapped around the reasoning.”
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · cited 1×
“However, most existing research on CoT focuses on improving the reasoning process within the model rather than optimizing the presentation of the reasoning [51, 54].”
Explainability for Large Language Models: A Survey
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681 · cited 1×
“This gap motivates rethinking how reasoning is surfaced to end-users in contexts like education, where clarity, verification, and engagement are essential.”
The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study
Zhou, L., Bao, J., Setiawan, I. M. A., Saptono, A., & Parmanto, B. (2019) · JMIR mHealth and uHealth · doi:10.2196/11500 · Cited by 675 · cited 1×
“The question sets are inspired by established measures of engagement, usability, and interpretability in human-AI interaction research and are adapted for the design features of our interfaces [21, 45, 60].”
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · cited 1×
“…(XAI) literature shows that explanation techniques are frequently designed around what researchers can produce rather than what end users need; users instead report wanting practically useful information that improves collaboration, trust calibration, and feedback loops with AI systems[1, 17, 32].”
Research works printed but not linked: 16
- Karl Cobbe Vineet Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun Lukasz Kaiser Matthias Plappert Jerry Tworek Jacob Hilton Reiichiro Nakano et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:https://arXiv.org/abs/2110.14168 (2021). · arXiv
- Ali Forootani. 2025. A survey on mathematical reasoning and optimization with large language models. arXiv preprint arXiv:https://arXiv.org/abs/2503.17726 (2025). · arXiv
- Jenny Kunz and Marco Kuhlmann. 2024. Properties and challenges of llm-generated explanations. arXiv preprint arXiv:https://arXiv.org/abs/2402.10532 (2024). · arXiv
- Khai Le-Duc Duy MH Nguyen Phuong TH Trinh Tien-Phat Nguyen Nghiem T Diep An Ngo Tung Vu Trinh Vuong Anh-Tien Nguyen Mau Nguyen et al. 2025. S-chain: Structured visual chain-of-thought for medicine. arXiv preprint arXiv:https://arXiv.org/abs/2510.22728 (2025). · arXiv
- Xiaoyuan Li Wenjie Wang Moxin Li Junrong Guo Yang Zhang and Fuli Feng. 2024. Evaluating mathematical reasoning of large language models: A focus on error identification and correction. arXiv preprint arXiv:https://arXiv.org/abs/2406.00755 (2024). · arXiv
- Giang Nguyen Ivan Brugere Shubham Sharma Sanjay Kariyappa Anh Totti Nguyen and Freddy Lecue. 2024. Interpretable llm-based table question answering. arXiv preprint arXiv:https://arXiv.org/abs/2412.12386 (2024). · arXiv
- Giang Nguyen, Mohammad Reza Taesiri, Sunnie SY Kim, and Anh Nguyen. 2024. Allowing humans to interactively guide machines where to look does not always improve human-AI team’s classification accuracy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8169–8175.
- Tin Nguyen Logan Bolton Mohammad Reza Taesiri and Anh Totti Nguyen. 2025. HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputs. arXiv preprint arXiv:https://arXiv.org/abs/2503.02003 (2025). · arXiv
- Mahsan Nourani Chiradeep Roy Tahrima Rahman Eric D Ragan Nicholas Ruozzi and Vibhav Gogate. 2020. Don’t explain without verifying veracity: an evaluation of explainable ai with video activity recognition. arXiv preprint arXiv:https://arXiv.org/abs/2005.02335 (2020). · arXiv
- Suleyman Ozdel Can Sarpkaya Efe Bozkir Hong Gao and Enkelejda Kasneci. 2025. Examining the Role of LLM-Driven Interactions on Attention and Cognitive Engagement in Virtual Classrooms. arXiv preprint arXiv:https://arXiv.org/abs/2505.07377 (2025). · arXiv
- Aaron Springer and Steve Whittaker. 2018. Progressive disclosure: designing for effective transparency. arXiv preprint arXiv:https://arXiv.org/abs/1811.02164 (2018). · arXiv
- Boshi Wang Sewon Min Xiang Deng Jiaming Shen You Wu Luke Zettlemoyer and Huan Sun. 2022. Towards understanding chain-of-thought prompting: An empirical study of what matters. arXiv preprint arXiv:https://arXiv.org/abs/2212.10001 (2022). · arXiv
- Shen Wang Tianlong Xu Hang Li Chaoli Zhang Joleen Liang Jiliang Tang Philip S Yu and Qingsong Wen. 2024. Large language models for education: A survey and outlook. arXiv preprint arXiv:https://arXiv.org/abs/2403.18105 (2024). · arXiv
- Yuhao Zhang Jiaxin An Ben Wang Yan Zhang and Jiqun Liu. 2025. Human-Centered Explainability in Interactive Information Systems: A Survey. arXiv preprint arXiv:https://arXiv.org/abs/2507.02300 (2025). · arXiv
- Zhuosheng Zhang Aston Zhang Mu Li and Alex Smola. 2022. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:https://arXiv.org/abs/2210.03493 (2022). · arXiv
- Joerg Zumbach. 2006. Cognitive overhead in hypertext learning reexamined: Overcoming the myths. Journal of Educational Multimedia and Hypermedia 15 4 (2006) 411–432.
Cited by
Architecting Trust in Artificial Epistemic AgentsPreprint
Marchal, N., Chan, S., Franklin, M., Revel, M., Keeling, G., Fischli, R., Chandra, B., & Gabriel, I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.02960 · Cited by 19
Atakishiyev, S., Babiker, H. K. B., Dai, J., Farruque, N., Hayashi, T., Hriti, N. S., Rahman, M. A., Smith, I., Kim, M.-Y., Zai͏̈ane, O. R., et al. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.135281 · Cited by 8
Understanding the mechanism of human–AI interaction: a distributed cognition perspective
Zhao, H., & Han, Z. (2026) · Journal of Documentation · doi:10.1108/jd-01-2026-0003 · Cited by 8
InteractiveKG: Transparent and user-controllable knowledge graph reasoning
Kim, M., Zhao, Y., Ju, J., Seo, J., & Park, H. (2026) · Information Visualization · doi:10.1177/14738716261435574 · Cited by 4
“73 More broadly, knowledge graphs have been argued to provide a richer explainability substrate than purely numerical attribution methods by preserving both entities and their relationships.”
SketchVLM: Vision language models can annotate images to explain thoughts and guide users
Collins, B., Bolton, L., Nguyen, H. H., Taesiri, M. R., Bui, T., & Nguyen, A. T. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.2449 · Cited by 3
Self-citationWhen the Chain Breaks: Interactive Diagnosis of LLM Chain‐of‐Thought Reasoning Errors
Chen, S., Sritharan, N., Wen, X., Zhang, C., Wang, X., & Wang, Y. (2026) · Computer Graphics Forum · doi:10.1111/cgf.70439 · Cited by 1
“To fill the research gap, we propose ReasonDiag , an interactive visual analytics approach to help LLM users effectively interpret and diagnose CoT reasoning traces.”
ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Models
Zhang, J., Zheng, J., Cao, B., Lu, Y., Lin, H., Zheng, J., Han, X., & Sun, L. G. (2026) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/2026.emnlp-demos.4 · Cited by 1
HANSEL: Interactive Evidence for Post-Hoc Verification of Web AgentsPreprint
Zhang, Y., Hwang, Y., & Nam, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.18671 · Cited by 1
“However, effective verification is fundamentally interactive; users need to re-sort, probe filters, and cross-check directly [38, 54].”
Justified or Just Convincing? Error Verifiability as a Dimension of LLM QualityPreprint
Zhu, X., Truong, K. L., Fogliato, R., Swamy, G., Zhang, W., Yang, M., Ye, L., Liu, B., Liu, M., Ilyas, A., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.04418
“Prior work shows that reducing the difficulty of LLM explanations reduces overreliance on incorrect predictions (Vasconcelos et al., 2023) and that restructuring explanations into more readable formats improves human error detection (Zhou et al., 2026).”
PageGuide: Browser extension to assist users in navigating a webpage and locating informationPreprint
Nguyen, T., Truong, T. T., Zhou, R., Bui, T., Agarwal, C., & Nguyen, A. T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.23772
“💡 You will be prompted to confirm your password Step 3: User pause to input their password”
Self-citationKaksivaiheinen LLM-järjestelmä automaattiseen vaatimustenmukaisuuden arviointiin
Ahola, T. (2026) · Aaltodoc (Aalto University)
Do Reasoning Representations Help Humans Evaluate LLM Outputs?
Lim, J., Shin, S., & Hong, S. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.521
Why and How People Check Generative AI Output for Mistakes
Kelley, P. G., Feldmann, D., Jana, R., Thompson-Kuhn, C., & Woodruff, A. (2026) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v9i2.43242
“Several interfaces have been proposed, including leveraging warnings (Nahar et al. 2024), chain-of-thought (CoT) reasoning (Zhou et al. 2026), sincere apologies (Mah-mood et al. 2022), or a warn-verify-audit interface (Laban et al. 2024), often showing improvements on measures such as error…”
MemCanvas: A Structured Memory Canvas for Evolving Intent Alignment in LLM Dialogues
Ye, J., Fei, Y., Lu, Y., Zhou, Z., & You, W. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830596
“Similarly, recent works like iGraph [68] and Interactive Reasoning [38] further restructure the model’s step-by-step explanations or deductive paths into node-link diagrams to facilitate logic inspection and error detection, while IdeaSynth [40] and Guardrails [10] employ diagrams to structure…”
Project Progress Desk: Know Where All Your Claude Code and Codex Projects Stand at a GlancePreprint
Ye, W., Wang, Y., He, S., Sun, G., Wang, Z., He, Y., Shen, Z., Liu, Z., Tian, B., 정다은, et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.13287
Metadata
- Authors
- Zhou, R., Nguyen, G., Kharya, N., Nguyen, A., & Agarwal, C.
- Affiliations
- University of Virginia (US)
- GuideLabs, GuideLabs, San Francisco, California, USA
- Independent Researcher, Charlottesville, Virginia, USA
- Auburn University (US)
- Venue
- Proceedings of the 31st International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2026-03-22
- Pages
- 456-473
- OpenAlex
- W7133356125
- Cited by
- 2
- References
- 39
- Citations by year
- 2026: 2
- FWCI
- 17.87
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- large language models, chain-of-thought reasoning, hallucination detection, mathematical reasoning, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.