Question-Driven Design Process for Explainable AI User Experiences
Open accessAbstract
A pervasive design issue of AI systems is their explainability--how to provide appropriate information to help users understand the AI. The technical field of explainable AI (XAI) has produced a rich toolbox of techniques. Designers are now tasked with the challenges of how to select the most suitable XAI techniques and translate them into UX solutions. Informed by our previous work studying design challenges around XAI UX, this work proposes a design process to tackle these challenges. We review our and related prior work to identify requirements that the process should fulfill, and accordingly, propose a Question-Driven Design Process that grounds the user needs, choices of XAI techniques, design, and evaluation of XAI UX all in the user questions. We provide a mapping guide between prototypical user questions and exemplars of XAI techniques to reframe the technical space of XAI, also serving as boundary objects to support collaboration between designers and AI engineers. We demonstrate it with a use case of designing XAI for healthcare adverse events prediction, and discuss lessons learned for tackling design challenges of AI systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants' questions reflected their desire for an actionable or utility-oriented understanding to support their end goal of optimizing code generation and programming productivity.
“Participants' questions reflected their desire for an actionable or utility-oriented understanding to support their end goal of optimizing code generation and programming productivity [18,50,52], such as asking the Input, Output, How and How-to XAI questions (described in Section 4) to facilitate strategies to get better outputs from the AI.”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [50]
In “Question Analysis” (pp. 13–14), participants’ questions are linked to practical goals, including deciding actionable next steps for patient management. The paper’s participants were healthcare providers considering patient-risk predictions; it does not support the claim that their goal was optimizing code generation or programming productivity. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · see citing works
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1× · see citing works
“[22, 42, 46, 47]) increasingly accessible to practitioners.”
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581 · see citing works
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · see citing works
Self-citationDesigning Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · see citing works
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · see citing works
<i>Why and why not</i> explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · see citing works
Conversational processes and causal explanation.
Hilton, D. J. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.1.65 · Cited by 456 · see citing works
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · see citing works
XPLAIN: a system for creating and explaining expert consulting programs
Swartout, W. (1983) · Artificial Intelligence · doi:10.1016/s0004-3702(83)80014-9 · Cited by 394 · see citing works
UX Design Innovation: Challenges for Working with Machine Learning as a Design Material
Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391 · see citing works
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 1× · see citing works
“Others studied explainability features to help decision-makers such as physicians [54, 56] and recruiters [21] to better understand AI recommendations and make informed decisions.”
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Arya, V., Bellamy, R. K. E., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2019) · arXiv · doi:10.48550/arxiv.1909.03012 · Cited by 268 · cited 2× · see citing works
“…there is no agreed-upon definition or criteria of AI explainability, and there are different related terms used in the literature such as interpretability and transparency, a common goal shared by XAI work is to make AI’s decisions or functions understandable by people [8, 11, 12, 30, 41, 44, 45].”
Self-citationBringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250 · see citing works
Investigating How Experienced UX Designers Effectively Work with Machine Learning
Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., & Steinfeld, A. M. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196730 · Cited by 230 · see citing works
Explainability in human–agent systems
Rosenfeld, A., & Richardson, A. (2019) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09408-y · Cited by 228 · see citing works
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 225 · cited 1× · see citing works
“For model uncertainty, both the properties of the underlying model and user characteristics should guide the selection of both the algorithm to quantify uncertainty and the method to communicate uncertainty [16].”
Self-citationHohman, F., Head, A., Caruana, R., DeLine, R. A., & Drucker, S. M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225 · see citing works
The effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220 · see citing works
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · see citing works
Evaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163 · see citing works
Explainability scenarios: towards scenario-based XAI design
Wolf, C. T. (2019) · International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302317 · Cited by 153 · see citing works
Xie, Y., Chen, M., Kao, D., Gao, G., & Chen, X. '. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376807 · Cited by 141 · see citing works
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems
Tomsett, R., Braines, D., Harborne, D., Preece, A. D., & Chakraborty, S. (2018) · ORCA Online Research @Cardiff (Cardiff University) · doi:10.48550/arxiv.1806.07552 · Cited by 127 · see citing works
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123 · see citing works
Explainable Active Learning (XAL): Toward AI Explanations as Interfaces for Machine Teachers
Ghai, B., Liao, Q. V., Zhang, Y., Bellamy, R., & Mueller, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432934 · Cited by 106 · see citing works
Self-citationBogers, S., Frens, J. W., van Kollenburg, J., Deckers, E., & Hummels, C. (2016) · Designing Interactive Systems Conference · doi:10.1145/2901790.2901855 · Cited by 88 · see citing works
Hind, M. (2019) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/3313096 · Cited by 61 · see citing works
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415227 · Cited by 57 · see citing works
Prototyping ways of prototyping AI
van Allen, P. (2018) · interactions · doi:10.1145/3274566 · Cited by 51 · see citing works
Yu, B., Yuan, Y., Terveen, L., Wu, Z. S., Forlizzi, J., & Zhu, H. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395528 · Cited by 49 · see citing works
Lessons Learned in Designing AI for Autistic Adults
Begel, A., Tang, J., Andrist, S., Barnett, M., Carbary, T., Choudhury, P., Cutrell, E., Fung, A., Junuzovic, S., McDuff, D., et al. (2020) · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3373625.3418305 · Cited by 34 · see citing works
When User Experience Designers Partner with Data Scientists.
Girardin, F., & Lathia, N. (2017) · National Conference on Artificial Intelligence · Cited by 29 · see citing works
The Emerging Landscape of Explainable AI Planning and Decision MakingPreprint
Chakraborti, T., Sreedharan, S., & Subbarao Kambhampati, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11697 · Cited by 29 · see citing works
Creative Data Work in the Design Process
Kun, P., Mulder, I. J., GÖTZEN, A. D., & Kortuem, G. (2019) · Creativity and Cognition · doi:10.1145/3325480.3325500 · Cited by 27 · see citing works
Profiling Artificial Intelligence as a Material for User Experience Design
Yang, Q. (2021) · Figshare · doi:10.1184/r1/14376731 · Cited by 12 · see citing works
Flagged first, then by citations
1 of the 52 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- [22][22] Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. 2018. Explanations based on the missing: towards contrastive explanations with pertinent negatives. In Proceedings of the 32nd International Conference on Neural Information Processing Systems . 590-601. -
- [23][23] Jonathan Dodge, Q Vera Liao, Yunfeng Zhang, Rachel KE Bellamy, and Casey Dugan. 2019. Explaining models: an empirical study of how explanations impact fairness judgment. In Proceedings of the 24th International Conference on Intelligent User Interfaces . 275-285. -
- [31][31] David Gunning. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web 2, 2 (2017). -
- [34][34] Fred Hohman, Andrew Head, Rich Caruana, Robert DeLine, and Steven M Drucker. 2019. Gamut: A design probe to understand how data scientists understand machine learning models. In Proceedings of the 2019 CHI conference on human factors in computing systems . 1-13. -
- [50][50] Lingyun Sun, Zhibin Zhou, Wenqi Wu, Yuyang Zhang, Rui Zhang, and Wei Xiang. 2020. Developing a Toolkit for Prototyping Machine LearningEmpowered Products: The Design and Evaluation of ML-Rapid. International Journal of Design 14, 2 (2020), 35. -
Cited by
Sorted by citations
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863 · see citing works
Investigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219 · see citing works
Self-citationExplainable AI for Bioinformatics: Methods, Tools and Applications
Karim, M. R., Islam, T., Shajalal, M., Beyan, O., Lange, C., Cochez, M., Rebholz‐Schuhmann, D., & Decker, S. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad236 · Cited by 209 · see citing works
Responsible AI Pattern Catalogue: A Collection of Best Practices for AI Governance and Engineering
Lu, Q., Zhu, L., Xu, X., Whittle, J., Zowghi, D., & Jacquet, A. (2023) · ACM Computing Surveys · doi:10.1145/3626234 · Cited by 168 · see citing works
Charting the Sociotechnical Gap in Explainable AI: A Framework to Address the Gap in XAI
Ehsan, U., Saha, K., Choudhury, M. D., & Riedl, M. O. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579467 · Cited by 92 · see citing works
Beyond self-diagnosis: how a chatbot-based symptom checker should respond
You, Y., Tsai, C.-H., Li, Y., Ma, F., Heron, C., & Gui, X. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589959 · Cited by 60 · see citing works
Vianello, A., Laine, S., & Tuomi, E. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2095478 · Cited by 33 · see citing works
Pradeep, P., Caro-Martínez, M., & Wijekoon, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124733 · Cited by 28 · see citing works
Gambo, I., Massenon, R., Lin, C., Ogundokun, R. O., Agarwal, S., & Pak, W. (2024) · IEEE Access · doi:10.1109/access.2024.3443527 · Cited by 19 · see citing works
Benjamin, J. J., Kinkeldey, C., Müller-Birn, C., Korjakow, T., & Herbst, E.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3492858 · Cited by 19 · see citing works
Explainable Artificial Intelligence Technology for Predictive Maintenance
Walker, C. M., USDOE Office of Nuclear Energy (NE), Agarwal, V., Lin, L., Hill, R., Hill, R., Boring, R. L., Mortenson, T., & Lybeck, N. (2023) · doi:10.2172/1998555 · Cited by 17 · see citing works
Information that matters: Exploring information needs of people affected by algorithmic decisions
Schmude, T., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103380 · Cited by 13 · see citing works
Turri, V., Morrison, K., Robinson, K.-M., Abidi, C., Perer, A., Forlizzi, J., & Dzombak, R. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658985 · Cited by 10 · see citing works
A Comparison of Explainable AI Models on Numeric and Graph-Structured Data
Avinash, A., Harikumar, A., Nair, A., Pai, S. K., Surendran, S., & George, L. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.04.088 · Cited by 11 · see citing works
Leveraging Context-Sensitivity for User-Centered Explainability
Khaled, Y., & Ehab, N. (2025) · The European Journal on Artificial Intelligence · doi:10.1177/30504554251331568 · Cited by 4 · see citing works
QB4AIRA: A Question Bank for Responsible AI Risk Assessment
Lee, S. U., Perera, H., Xia, B., Liu, Y., Lu, Q., Zhu, L., Salvado, O., & Whittle, J. (2024) · IEEE Software · doi:10.1109/ms.2024.3512577 · Cited by 3 · see citing works
Schmude, T., Koesten, L. M., Möller, T. B., & Tschiatschek, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2650562 · Cited by 4 · see citing works
Polishchuk, K., & Brezhniev, E. (2023) · International Conference on Dependable Systems, Services and Technologies (DESSERT) · doi:10.1109/dessert61349.2023.10416528 · Cited by 2 · see citing works
Mathur, N., Rahman, H., & Desai, S. (2026) · Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772363.3798685 · Cited by 1 · see citing works
Sahithi, K. K., Riener, A., Wintersberger, P., Wang, M., Asteriou, P. M. M. P., & Lee, S. C. (2026) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3828158.3834796
Human-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120 · see citing works
Self-citation
- Authors
- Liao, Q. V., Pribić, M., Han, J., Miller, S., & Sow, D.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2104.03483
- OpenAlex
- W3146248196
- Cited by
- 18
- References
- 52
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 6 · 2023: 5 · 2022: 3 · 2021: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- explainable AI, user experience design, boundary objects, XAI evaluation, user questions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
