AI Should Challenge, Not Obey
Open accessAbstract
Let’s transform our robot secretaries into Socratic gadflies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Critical thinking has been operationalised in detail in certain specific disciplines, such as academic history, clinical psychology, and nursing.
“As previously noted [116,119], critical thinking has been operationalised in detail in certain specific disciplines, such as academic history, clinical psychology, and nursing.”
From The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers (2025) · cites this paper as [116]
In “Critical Thinking for Knowledge Work” (page not given), the paper says critical thinking is well-defined in history, nursing, and psychology, where it is formally taught. That supports the general point, but the paper does not specify clinical psychology. (LLM, conf=0.86)
Experimental — accuracy not yet measuredRecent work has motivated the need for critical thinking support in AI-assisted knowledge work.
“Recent work has motivated the need for critical thinking support in AI-assisted knowledge work [116,119].”
From The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers (2025) · cites this paper as [116]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956) · Cited by 9,342 · see citing works
Machines as teammates: A research agenda on AI in team collaboration
Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G.-J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., et al. (2020) · Information & Management · doi:10.1016/j.im.2019.103174 · Cited by 835 · see citing works
Concept mapping, mind mapping and argument mapping: what are the differences and do they matter?
Davies, M. (2011) · Higher Education · doi:10.1007/s10734-010-9387-6 · Cited by 804 · see citing works
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388 · see citing works
Critical thinking in nursing education: Literature review
Simpson, E., & Mary Courtney, R. N. (2002) · International Journal of Nursing Practice · doi:10.1046/j.1440-172x.2002.00340.x · Cited by 351 · see citing works
The brain in your pocket: Evidence that Smartphones are used to supplant thinking
Barr, N., Pennycook, G., Stolz, J. A., & Fugelsang, J. A. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.029 · Cited by 327 · see citing works
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 280 · cited 1× · see citing works
“Researchers have called for metacognitive support in AI tools [ 13], and to “educate people to be critical information seekers and users” [ 14].”
Self-citationLearner-Centered Design of Computing Education: Research on Computing for Everyone
Guzdial, M. (2015) · Synthesis lectures on human-centered informatics · doi:10.2200/s00684ed1v01y201511hci033 · Cited by 252 · see citing works
Critical thinking in E-learning environments
Saadé, R. G., Morin, D., & Thomas, J. D. E. (2012) · Computers in Human Behavior · doi:10.1016/j.chb.2012.03.025 · Cited by 192 · see citing works
How to design programs: an introduction to programming and computing
Felleisen, M., Findler, R. B., Flatt, M., & Krishnamurthi, S. (2001) · doi:10.14507/er.v0.173 · Cited by 189 · see citing works
Challenges and Prospects for Canadian Social Studies
Tupper, J. (2010) · Journal of international social studies · Cited by 142 · see citing works
Elaborating Team Roles for Artificial Intelligence-based Teammates in Human-AI Collaboration
Siemon, D. (2022) · Group Decision and Negotiation · doi:10.1007/s10726-022-09792-z · Cited by 136 · see citing works
Friend, Collaborator, Student, Manager
Guzdial, M., Liao, N., Chen, J., Chen, S.-Y., Shah, S., Shah, V., Reno, J. O., Smith, G., & Riedl, M. O. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300854 · Cited by 134 · see citing works
AI in Reverse: Computer Tools That Turn Cognitive
Salomon, G. (1988) · Journal of Educational Computing Research · doi:10.2190/4lu7-vw23-egb1-aw5g · Cited by 130 · see citing works
Danry, V., Pataranutaporn, P., Mao, Y., & Maes, P. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580672 · Cited by 110 · see citing works
Who/What Is My Teammate? Team Composition Considerations in Human–AI Teaming
McNeese, N. J., Schelble, B. G., Canonico, L. B., & Demir, M. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3086018 · Cited by 106 · see citing works
Kivunja, C. (2015) · Creative Education · doi:10.4236/ce.2015.63037 · Cited by 104 · see citing works
Pan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86 · see citing works
Critical Thinking in Psychology
(2019) · Cambridge University Press eBooks · doi:10.1017/9781108684354 · Cited by 82 · see citing works
Teaching Argument: An Introduction to the Toulmin Model
Kneupper, C. W. (1978) · College Composition and Communication · doi:10.2307/356935 · Cited by 77 · see citing works
Dehallucinating Large Language Models Using Formal Methods Guided Iterative Prompting
Jha, S., Jha, S. K., Lincoln, P. D., Bastian, N. D., Velasquez, A., & Neema, S. K. (2023) · IEEE International Conference on Assured Autonomy (ICAA) · doi:10.1109/icaa58325.2023.00029 · Cited by 73 · see citing works
Sarkar, A. (2023) · Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work · doi:10.1145/3596671.3597650 · Cited by 50 · see citing works
Self-citationEnough With “Human-AI Collaboration”
Sarkar, A. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3582735 · Cited by 47 · see citing works
Self-citationFostering Youth’s Critical Thinking Competency About AI through Exhibition
Lee, S., Choi, D., Lee, M., Choi, J., & Lee, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581159 · Cited by 42 · see citing works
Extending a Human-AI Collaboration Framework with Dynamism and Sociality
Muller, M., & Weisz, J. (2022) · 2022 Symposium on Human-Computer Interaction for Work · doi:10.1145/3533406.3533407 · Cited by 37 · see citing works
Iverson, K. E. (2007) · ACM eBooks · doi:10.1145/1283920.1283935 · Cited by 30 · see citing works
Mochizuki, T., Nishimori, T., Tsubakimoto, M., Oura, H., Sato, T., Johansson, H., Nakahara, J., & Yamauchi, Y. (2019) · Educational Technology Research and Development · doi:10.1007/s11423-019-09676-1 · Cited by 22 · see citing works
Should Computers Be Easy To Use? Questioning the Doctrine of Simplicity in User Interface Design
Sarkar, A. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3582741 · Cited by 19 · see citing works
Self-citationTeaching Argument: An Introduction to the Toulmin Model
Kneupper, C. W. (1978) · College Composition and Communication · doi:10.58680/ccc197816301 · Cited by 11 · see citing works
Critical Thinking in Collaboration
Sun, N., Yuan, C. W., Rosson, M. B., Wu, Y., & Carroll, J. M. (2017) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3027063.3053250 · Cited by 6 · see citing works
Towards Mobile Blended Interaction Fostering Critical Thinking
Holzer, A., Govaerts, S., Bendahan, S., & Gillet, D. (2015) · International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI Adjunct) · doi:10.1145/2786567.2793695 · Cited by 4 · see citing works
Conversational Agent as a Black Hat: Can Criticising Improve Idea Generation?
Cvetković, I., Rosenberg, V., & Bittner, E. A. C. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.023 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- Ian Drosos, Advait Sarkar, Xiaotong Xu, Carina Negreanu, Sean Rintel, and Lev Tankelevitch. "It's like a rubber duck that talks back": Understanding Generative AI-Assisted Data Analysis Workflows through a Participatory Prompting Study. In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work, CHIWORK '24, New York, NY, USA, 2024. Association for Computing Machinery.
- Chris Argyris. Double loop learning in organizations. Harvard business review, 55(5):115-125, 1977.
- Allen Cypher, Daniel C. Halbert, David Kurlander, Henry Lieberman, David Maulsby, Brad A. Myers, and Alan Turransky, editors. Watch what I do: programming by demonstration. MIT Press, Cambridge, MA, USA, 1993.
- Henry Lieberman, editor. Your Wish is My Command. Interactive Technologies. Morgan Kaufmann, San Francisco, 2001.
- Jiahao Yu, Xingwei Lin, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto- generated jailbreak prompts. arXiv preprint arXiv:2309.10253, 2023. · arXiv
- Andrew D. Gordon, Carina Negreanu, José Cambronero, Rasika Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson, Advait Sarkar, Stephanie Simmons, Jack Williams, and Ben Zorn. Co-audit: tools to help humans double-check AI-generated content. Proceedings of the 14th annual workshop on the intersection of HCI and PL (PLATEAU 2024), 5 2024.
- Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022. · arXiv
- Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. Explainability for large language models: A survey. arXiv preprint arXiv:2309.01029, 2023. · arXiv
- Advait Sarkar, Xiaotong (Tone) Xu, Neil Toronto, Ian Drosos, and Christian Poelitz. When Copilot Becomes Autopilot: Generative AI's Critical Risk to Knowledge Work and a Critical Solution. In Proceedings of the Annual Conference of the European Spreadsheet Risks Interest Group (EuSpRIG 2024), 2024.
Cited by
Sorted by citations
Lee, H.-P. (., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713778 · Cited by 490 · see citing works
“Recent work has motivated the need for critical thinking support in AI-assisted knowledge work [116, 119].”
Self-citationNatali, C., Marconi, L., Dias Duran, L. D., & Cabitza, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11352-1 · Cited by 143 · see citing works
Kazemitabaar, M., Huang, O., Suh, S., Henley, A. Z., & Grossman, T. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712104 · Cited by 48 · see citing works
Tankelevitch, L., Glassman, E. L., He, J., Kazemitabaar, M., Kittur, A., Lee, M., Palani, S., Sarkar, A., Ramos, G., Rogers, Y., et al. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706745 · Cited by 24 · see citing works
Self-citationXu, X. (., Konnova, A., Gao, B., Peng, C., Vo, D., & Dow, S. P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713649 · Cited by 20 · see citing works
Lee, S., Hwang, S., Kim, D.-J., & Lee, K. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719792 · Cited by 19 · see citing works
Park, S. O., Kim, H., & Lim, Y.-K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713722 · Cited by 18 · see citing works
AI Could Have Written This: Birth of a Classist Slur in Knowledge Work
Sarkar, A. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3716239 · Cited by 12 · see citing works
Self-citationWolf, S., Grundgeiger, T., Zähringer, R., Shishkova, L., Maas, F., Dilling, C., & Happel, O. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713372 · Cited by 10 · see citing works
Dynamic Prompt Middleware: Contextual Prompt Refinement Controls for Comprehension Tasks
Drosos, I., Williams, J., Sarkar, A., Wilson, N., Rintel, S., & Panda, P. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729203 · Cited by 10 · see citing works
Self-citationWhy AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild
Kumar, A., Bajpai, Y., Gulwani, S., Soares, G., & Murphy-Hill, E. R. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00043 · Cited by 8 · see citing works
Prasongpongchai, T., Pataranutaporn, P., Lertsutthiwong, M., & Maes, P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3715579 · Cited by 7 · see citing works
Scott, A. E., Tankelevitch, L., Panda, P., Vanukuru, R., Chen, X., & Rintel, S. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729204 · Cited by 7 · see citing works
Mastrianni, A., Kim, M. S., Sullivan, T. M., Sippel, G. J., Burd, R. S., Gajos, K. Z., & Sarcevic, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757512 · Cited by 6 · see citing works
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 6 · see citing works
Huang, E. J., Easterday, M. W., & Gerber, E. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757549 · Cited by 6 · see citing works
Yun, S., & Lim, Y.-K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713265 · Cited by 5 · see citing works
Aoyagui, P. A., Stemmler, K., Ferguson, S. A., Kim, Y., & Kuzminykh, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713248 · Cited by 4 · see citing works
Ghosh, P., & Rintel, S. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720142 · Cited by 4 · see citing works
Nishal, S., Lee, M., Diakopoulos, N. A., & Vaughan, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790372 · Cited by 3 · see citing works
Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding
Chou, Y.-H., Jiang, B., Chen, Y. W., Weng, M., Jackson, V., Zimmermann, T., & Jones, J. A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797105 · Cited by 3 · see citing works
A Reflection Machine to Support Critical Reflection During Decision-Making
Fischer, S. W. S. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708557.3716151 · Cited by 1 · see citing works
The End of Programming as We Know It
Jönsson, M., Tholander, J., Rost, M., Sarkar, A., Fernaeus, Y., Kharrufa, A., & Ludwig, T. G. (2025) · Decennial Aarhus Conference · doi:10.1145/3737609.3747093 · Cited by 1 · see citing works
Self-citationEyes on the Code: Mapping Critical Thinking Through Eye-Tracking for Student-LLM Coding Interactions
Sqalli, M. T. (2025) · Biannual Conference of the Italian SIGCHI Chapter (CHItaly) · doi:10.1145/3750069.3750397 · Cited by 1 · see citing works
Augmenting Human Cognition Through Generative AI
Rogers, Y., Reicherts, L., Zhang, Z. T., & Hassib, M. (2026) · Springer eBooks · doi:10.1007/978-3-031-61050-9_10-1 · Cited by 1 · see citing works
Generative Design and Vibe Coding: Rethinking The Design-Development Divide for UI Prototyping
Zhang, X., Subramonyam, H., Sarkar, A., Drosos, I., Wang, Z., Lee, K., Pimenova, V., Chen, X. '., & Lukoff, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3778802 · Cited by 1 · see citing works
Self-citationWho Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Chappidi, S., Singh, J. P., & Krauze, A. V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791428 · Cited by 2 · see citing works
Jenkins, A. D. G., Bagchi, P., Curtis, H., Seymour, W., Coté, M., & Such, J. M. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812975 · Cited by 1 · see citing works
Human-AI Collaborative Intelligence and Pedagogies of Hybrid Cognition in Learning Ecosystems
Alam, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7036-1.ch010 · Cited by 1 · see citing works
Beyond Compliance: How AI Could Help Creative Writers by Refusing Them
Qin, H. X., Zhu, G., Fan, M., & Hui, P. (2026) · Creativity and Cognition · doi:10.1145/3803784.3807571 · Cited by 1 · see citing works
Przybyszewski, J., Li, J., & Cuffe, P. (2025) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2025.3617543 · Cited by 1 · see citing works
Adamu, M. S., Lazem, S., Nkwo, M. S., & Peters, A. N. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5347484
Rubinstein, B., & Matos, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05176-9_24
Zhang, Z. T., von Felten, N., Reicherts, L., Tankelevitch, L., Guan, Z., Rintel, S., Fu, Y., He, J., Holstein, K., Sarkar, A., et al. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3778733
Self-citationAdavi, K. A. K., Ghosh, P., Banks, R., Sarkar, A., & Lindley, S. E. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808058 · Cited by 1 · see citing works
Self-citationThe First Reflection in Creative Experience (RiCE) Workshop
Ford, C., Sutskova, O., Cox, S., Sterman, S., Kreminski, M., van Koningsbruggen, R., Wang, A.-Q., Otenen, E., Ross, K., Fede, G. D., et al. (2026) · Creativity & Cognition · doi:10.1145/3803784.3804465
Gardella, N., Prather, J. E., Leinonen, J., Denny, P. C., Pettit, R., & Riggs, S. L. (2026) · ACM Conference on International Computing Education Research (ICER) · doi:10.1145/3765964.3811664
From Explanations to Reflective Questions in Human-AI Decision-Making
Fischer, S. W. S., Holmberg, L., Thill, S., & Schraffenberger, H. (2026) · Mensch und Computer (MuC) · doi:10.1145/3820253.3831391
The Reflection Penalty: when managers accept leadership coaching from a human but not an AI agent
Forshaw, T., Hale, J., & Longmire, M. (2026) · Human-Computer Interaction · doi:10.1080/07370024.2026.2744236
When disclosure rules are not enough: authorial warrant and judgment in ai-supported scholarship
Mirończuk, M. (2026) · Ethics and Information Technology · doi:10.1007/s10676-026-09928-8
- Authors
- Sarkar, A.
- Affiliations
- University of Cambridge (GB)
- Corresponding author
- Sarkar, Advait
- ORCID
- Sarkar, Advait
- Venue
- Communications of the ACM
- Published
- 2024-09-26
- Volume
- 67
- Issue
- 10
- Pages
- 18-21
- ISSN
- 0001-0782 (ISSN-L)
- 1557-7317
- OpenAlex
- W4402596704
- Cited by
- 40
- References
- 31
- Citations by year
- 2026: 15 · 2025: 25
- FWCI
- 16.65
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
- Keywords
- artificial intelligence, Socratic method, critical thinking, AI ethics, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
