The Metacognitive Demands and Opportunities of Generative AI

In the paper

“Generative AI (GenAI) systems offer unprecedented opportunities for transforming professional and personal work, yet present challenges around prompting, evaluating and relying on outputs, and optimizing workflows.”

What the sources say

• ?Not checked — no full text of the sourcecites [208] 208. J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1-21. https://doi.org/10.1145/ 3544548.3581388 source
• ?Could not verify

  The Discussion addresses dogmatism, uncertainty, and information search, not the use of generative AI. Searches for GenAI prompting, output evaluation, and workflow optimization found no passage addressing those challenges in our copy. (LLM, conf=0.78)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured
• Partly supported

  Sections 4.2.3 and 4.2.5 describe difficulties prompting Copilot and validating its suggestions; section 6.2 discusses suggestions that disrupt programmers’ workflow. These findings support the claim for AI-assisted programming, but the paper does not establish them for GenAI systems across professional and personal work generally. The indexed passages provide no page numbers. (LLM, conf=0.86)

  cites [14] 14. Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code-Generating Models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (April 2023), 78:85-78:111. https://doi.org/10.1145/3586030 sourceExperimental — accuracy not yet measured
• Partly supported

  The paper identifies difficulty writing effective prompts (Section 7.1, p. 10), evaluating generated code (Section 5, p. 7), and avoiding over-reliance on it (Section 5, p. 7). These findings concern AI-assisted programming; the passages do not establish the same challenges for GenAI systems generally, and the discussion of workflows (Section 7.3, p. 13) describes their use rather than a challenge of optimizing them. (LLM, conf=0.84)

  cites [154] 154. Advait Sarkar. 2023. Exploring Perspectives on the Impact of Artificial In- telligence on the Creativity of Knowledge Work: Beyond Mechanised Pla- giarism and Stochastic Parrots. In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work (CHIWORK '23). Association for Computing Machinery, New York, NY, USA, 1-17. https: //doi.org/10.1145/3596671.3597650 sourceExperimental — accuracy not yet measured

In the paper

“However, these same properties also pose a challenge for designing GenAI systems to be human-centered [34].”

What the sources say

• Supported

  Section 2.2, “Assimilating Human Intents” (page not provided), says GenAI’s open-ended inputs make it more challenging to capture human intentions and describes difficulties with prompts and controlling outputs. This supports the claim that properties of GenAI pose challenges for human-centered system design. (LLM, conf=0.90)

  cites [34] 34. Xiang 'Anthony' Chen, Jeff Burke, Ruofei Du, Matthew K. Hong, Jennifer Jacobs, Philippe Laban, Dingzeyu Li, Nanyun Peng, Karl D. D. Willis, Chien-Sheng Wu, and Bolei Zhou. 2023. Next Steps for Human-Centered Generative AI: A Tech- nical Perspective. https://doi.org/10.48550/arXiv.2306.15774 arXiv:2306.15774 [cs]. source

In the paper

“User studies reveal a range of usability challenges around prompting [208], evaluating and relying on outputs [156], and deciding on an automation strategy: whether and how to integrate GenAI into workflows [14,154].”

What the sources say

• ?Not checked — no full text of the sourcecites [208] 208. J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1-21. https://doi.org/10.1145/ 3544548.3581388 source
• ?Could not verify

  Citation [156] is attached to the claim about evaluating and relying on outputs. The Materials and Methods describe judgments about flickering-dot displays, while the Results and Discussion address confidence and information-seeking decisions. Searches for output evaluation, GenAI, and automation found no passage about usability challenges in evaluating or relying on generated outputs. (LLM, conf=0.76)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured
• Supported

  Section 4 (Theory; page unavailable) identifies two ways programmers use Copilot—acceleration and exploration—and reports that they switch between them while working. Section 6.2 (Better Output; page unavailable) also reports a participant turning Copilot off because it disrupted their workflow. Together, these support the workflow-integration point for which this source is cited; the sentence assigns prompting and output evaluation to other citations. (LLM, conf=0.87)

  cites [14] 14. Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code-Generating Models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (April 2023), 78:85-78:111. https://doi.org/10.1145/3586030 source
• Supported

  For the part cited to this paper—deciding how to integrate AI into programming workflows—Section 6 (p. 9) reports a user study identifying two modes of Copilot use, acceleration and exploration, and a difficulty with long suggestions disrupting programmers’ flow. Section 7.3 (p. 13) also describes contexts in which programmers develop workflows heavily dependent on AI assistance. (LLM, conf=0.85)

  cites [154] 154. Advait Sarkar. 2023. Exploring Perspectives on the Impact of Artificial In- telligence on the Creativity of Knowledge Work: Beyond Mechanised Pla- giarism and Stochastic Parrots. In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work (CHIWORK '23). Association for Computing Machinery, New York, NY, USA, 1-17. https: //doi.org/10.1145/3596671.3597650 source

In the paper

“Alongside the local interactions with GenAI systems, the generality of GenAI poses another, higherlevel metacognitive demand: the challenge of knowing whether and how to incorporate GenAI into workflows-i.e., one's 'automation strategy' (see also [154]).”

What the sources say

• Partly supported

  Section 7.3 (p. 13) reports that some programmers developed AI-dependent workflows for particular tasks, and Section 8.2 (p. 15) describes the wide range of tasks a prompt can specify. These passages support the idea that programmers make choices about how to use AI assistance, but they do not establish the broader claim that GenAI’s generality creates a higher-level metacognitive demand to decide whether and how to incorporate it into workflows. (LLM, conf=0.78)

  cites [154] 154. Advait Sarkar. 2023. Exploring Perspectives on the Impact of Artificial In- telligence on the Creativity of Knowledge Work: Beyond Mechanised Pla- giarism and Stochastic Parrots. In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work (CHIWORK '23). Association for Computing Machinery, New York, NY, USA, 1-17. https: //doi.org/10.1145/3596671.3597650 sourceExperimental — accuracy not yet measured

In the paper

“Firstly, given that metacognitive abilities can be taught [45,52,126], we can improve users' metacognition via metacognitive support strategies that can be integrated into GenAI systems.”

What the sources say

• Supported

  The Introduction (section 1, page unavailable) describes teaching students to plan, monitor, and evaluate their learning as instruction in metacognitive skills. The Scientific and practical contributions section (4.4, page unavailable) reports that students maintained newly acquired learning skills after metacognitive strategy instruction. This supports the cited premise that metacognitive abilities can be taught; the proposed use in GenAI systems is the citing authors’ own conclusion. (LLM, conf=0.93)

  cites [45] 45. Hester de Boer, Anouk S. Donker, Danny D. N. M. Kostons, and Greetje P. C. van der Werf. 2018. Long-term effects of metacognitive strategy instruction on student academic performance: A meta-analysis. Educational Research Review 24 (June 2018), 98-115. https://doi.org/10.1016/j.edurev.2018.03.002 source
• Supported

  The Descriptives section (3.1; page unavailable) reports that interventions frequently taught metacognitive approaches such as planning and monitoring, and that about half explicitly addressed metacognitive knowledge. This supports the cited premise that metacognitive abilities can be taught; the proposed use of GenAI support is the citing authors’ own application. (LLM, conf=0.90)

  cites [52] 52. A. S. Donker, H. de Boer, D. Kostons, C. C. Dignath van Ewijk, and M. P. C. van der Werf. 2014. Effectiveness of learning strategy instruction on academic performance: A meta-analysis. Educational Research Review 11 (Jan. 2014), 1-26. https://doi.org/10.1016/j.edurev.2013.11.002 source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [126] 126. Daniel Muijs and Christian Bokhove. 2020. Metacognition and Self-Regulation: Evidence Review. Technical Report. Education Endowment Foundation. https:// eric.ed.gov/?id=ED612286 Publication Title: Education Endowment Foundation ERIC Number: ED612286.

In the paper

“Recent HCI work has begun to pursue this direction [173,204], albeit without explicitly grounding it in metacognition; we suggest that a metacognitive lens offers new research and design directions for augmenting GenAI system usability.”

What the sources say

• Partly supported

  Section 3.2 describes helping users understand a model’s limitations and decide how to incorporate its output into further actions, which is related to the claimed direction. But the paper addresses interpretable machine learning, not integrating metacognitive support into GenAI systems; it does not substantiate that more specific attribution. (LLM, conf=0.76)

  cites [173] 173. Sangho Suh, Bryan Min, Srishti Palani, and Haijun Xia. 2023. Sensecape: En- abling Multilevel Exploration and Sensemaking with Large Language Models. https://doi.org/10.48550/arXiv.2305.11483 arXiv:2305.11483 [cs]. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Abstract and Conclusion discuss human confidence judgments and error monitoring, not HCI or GenAI systems. Searches for HCI, generative AI, system design, and usability found no passage addressing whether recent HCI work integrates metacognitive support. (LLM, conf=0.78)

  cites [204] 204. Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. AI Chains: Trans- parent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, 1-22. https://doi.org/10.1145/3491102.3517582 sourceExperimental — accuracy not yet measured

In the paper

“People often implicitly and unintentionally rely on heuristics to guide their metacognitive monitoring and control [3].”

What the sources say

• Partly supported

  The Introduction says metacognitive judgments rely on heuristic cues, are assumed to occur spontaneously, and guide control over effort allocation. It does not establish that reliance on the cues is implicit or unintentional; spontaneity of the judgments alone does not settle that part of the claim. No page numbers are available in our copy. (LLM, conf=0.84)

  cites [3] 3. Rakefet Ackerman. 2019. Heuristic Cues for Meta-Reasoning Judgments: Review and Methodology. Psihologijske teme 28, 1 (May 2019), 1-20. https://doi.org/10. 31820/pt.28.1.1 Publisher: Filozofski fakultet u Rijeci. sourceExperimental — accuracy not yet measured

In the paper

“Confidence and its adjustment are central to decision-making and reasoning, especially in many aspects of human-AI interaction [4, 171] ( §3.1, §3.2, and §3.3).”

What the sources say

• ?Not checked — no full text of the sourcecites [4] 4. Rakefet Ackerman and Valerie Thompson. 2017. Meta-Reasoning: Shedding meta-cognitive light on reasoning research. 1-15. source
• Supported

  In Challenge 2 (page unavailable), the paper describes reliance on AI as depending on the decision-maker’s confidence and the AI’s expressed confidence, and discusses how people calibrate that reliance. Challenge 3 (page unavailable) likewise explains that confidence measures help people assess AI predictions and decide whether to follow AI advice. These passages support the claim’s emphasis on confidence in human–AI decision-making. (LLM, conf=0.88)

  cites [171] 171. Mark Steyvers and Aakriti Kumar. 2023. Three Challenges for AI-Assisted Decision-Making. Perspectives on Psychological Science: A Journal of the As- sociation for Psychological Science (July 2023), 17456916231181102. https: //doi.org/10.1177/17456916231181102 source

In the paper

“Nevertheless, the field of humancomputer interaction (HCI) has so far considered metacognition mainly in the context of computer science education [107,142].”

What the sources say

• ?Not checked — no full text of the sourcecites [107] 107. Dastyni Loksa, Lauren Margulieux, Brett A. Becker, Michelle Craig, Paul Denny, Raymond Pettit, and James Prather. 2022. Metacognition and Self-Regulation in Programming Education: Theories and Exemplars of Use. ACM Transactions on Computing Education 22, 4 (Sept. 2022), 39:1-39:31. https://doi.org/10.1145/ 3487050 source
• ?Could not verify

  Section 2 discusses metacognition in relation to mind wandering and creativity, while Section 4 discusses its development in school. Searches for HCI, human-computer interaction, and computer science education found no passage assessing where HCI has mainly considered metacognition. (LLM, conf=0.75)

  cites [142] 142. James Prather, Brett A. Becker, Michelle Craig, Paul Denny, Dastyni Loksa, and Lauren Margulieux. 2020. What Do We Think We Think We Are Doing?: Metacognition and Self-Regulation in Programming. In Proceedings of the 2020 ACM Conference on International Computing Education Research. ACM, Virtual Event New Zealand, 2-13. https://doi.org/10.1145/3372782.3406263 sourceExperimental — accuracy not yet measured

In the paper

“Metacognitive interventions, such as training through feedback on metacognitive performance [30], can, for example, increase judgment accuracy-the ability to distinguish between one's own correct and incorrect metacognitive processing.”

What the sources say

• Supported

  The Results (Metacognitive Efficiency and Bias; page unavailable) define metacognitive efficiency as the ability to distinguish correct from incorrect trials and report a significant increase after training in the group receiving feedback on metacognitive judgments, but not in the control group. This supports the claim that this form of feedback training can increase judgment accuracy. (LLM, conf=0.93)

  cites [30] 30. Jason Carpenter, Maxine T. Sherman, Rogier A. Kievit, Anil K. Seth, Hakwan Lau, and Stephen M. Fleming. 2019. Domain-General Enhancements of Metacognitive Ability Through Adaptive Training. Journal of Experimental Psychology. General 148, 1 (Jan. 2019), 51-64. https://doi.org/10.1037/xge0000505 source

In the paper

“By contrast, programmers who were highly confident in their own ability actively questioned the AI code assistant when it produced confusing output [198].”

What the sources say

• Supported

  Section 5.1 (p. 8) reports that participants were more confident in their own code-translation ability than in the model and questioned its confusing confidence indications. P0 explicitly said they would write a line of code with “100% confidence” and questioned why the translator was uncertain about it. The confusing output here was the assistant’s confidence feedback, rather than necessarily the translated code itself. (LLM, conf=0.88)

  cites [198] 198. Justin D. Weisz, Michael Muller, Stephanie Houde, John Richards, Steven I. Ross, Fernando Martinez, Mayank Agarwal, and Kartik Talamadupula. 2021. Perfection Not Required? Human-AI Partnerships in Code Translation. In 26th International Conference on Intelligent User Interfaces (IUI '21). Association for Computing Machinery, New York, NY, USA, 402-412. https://doi.org/10.1145/ 3397481.3450656 source

In the paper

“which presents what Sarkar et al. [156] term a 'fuzzy abstraction matching' problem-it becomes difficult for users to discern a system's capabilities and to match one's intent and prompting accordingly”

What the sources say

• ?Could not verify

  The Abstract and Discussion address uncertainty-guided information search and dogmatism in a perceptual decision task; they do not address GenAI capabilities or prompting. Searches for “fuzzy abstraction matching” and for GenAI model flexibility found no passage supporting the attributed claim. The cited paper’s authors also do not match the Sarkar et al. named in the citing sentence. (LLM, conf=0.78)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured

In the paper

“This is exacerbated by the non-determinism of GenAI, particularly when tweaking one aspect of the prompt might unintentionally change a different aspect of the output [33].”

What the sources say

• ?Could not verify

  The Exit survey passage says the framework can treat model decision boundaries and errors probabilistically, but it does not discuss GenAI or prompt changes. The passages on prediction shown and trust address human understanding of model error, not whether changing one part of a prompt unintentionally changes another part of the output. Searches for prompt non-determinism and confidence adjustment found no passage addressing that claim. (LLM, conf=0.75)

  cites [33] 33. Chacha Chen, Shi Feng, Amit Sharma, and Chenhao Tan. 2023. Machine Expla- nations and Human Understanding. https://doi.org/10.48550/arXiv.2202.04092 arXiv:2202.04092 [cs]. sourceExperimental — accuracy not yet measured

In the paper

“Evaluating these outputs for quality therefore becomes far more important and effortful (in terms of cognitive load) compared to 'auto-complete' phrase suggestions or intelligent code completion [156,157].”

What the sources say

• ?Could not verify

  The Results (Measuring Information Search) examine decisions about whether to seek additional evidence in a dot-judgment task. The Discussion considers uncertainty and effort in metacognitive control, but does not compare the cognitive load of evaluating generated outputs with autocomplete or code completion. Searches for those terms and that comparison found no supporting passage in our copy; page numbers were unavailable. (LLM, conf=0.74)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured
• Partly supported

  The paper treats validating whether an output is fit for purpose as a cost (Table 2, System) and says users’ cognitive effort matters when evaluating multimodal systems (§2, “A New Kind of Cognitive Tool”). It does not establish that assessing their outputs is more important or more cognitively effortful than assessing auto-complete suggestions or intelligent code completion. (LLM, conf=0.84)

  cites [157] 157. Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Sean Ó hÉigeartaigh, Anna Korho- nen, and José Hernández-Orallo. 2023. Your Prompt is My Command: On Assess- ing the Human-Centred Generality of Multimodal Models. Journal of Artificial Intelligence Research 77 (June 2023), 377-394. https://doi.org/10.1613/jair.1.14157 sourceExperimental — accuracy not yet measured

In the paper

“Programmers were reluctant to deeply review and repair AI-generated code, preferring instead to re-write the entire code themselves [188].”

What the sources say

• Partly supported

  In “Granularity of uncertainty highlighting” (page unavailable), one participant said they would probably discard and rewrite a completion if its score were low. That conditional remark does not establish that programmers generally preferred rewriting over reviewing and repairing generated code; Section 5.2 instead reports participants making targeted edits. (LLM, conf=0.86)

  cites [188] 188. Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, 1-7. https://doi.org/10.1145/3491101.3519665 sourceExperimental — accuracy not yet measured

In the paper

“GenAI can produce (often extensive) content, such as entire emails, presentations, or software.”

What the sources say

• ?Could not verify

  The Materials and Methods passages describe an online dot-discrimination task, while the References concern information seeking and related research. Searches for GenAI and for generating emails, presentations, or software found no passage in our copy addressing those capabilities. (LLM, conf=0.78)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured
• Partly supported

  The Introduction describes generative models producing flexible outputs, and §2, “A New Kind of Cognitive Tool,” says they can generate novel content. The modalities table includes code generation. Our copy does not establish the more specific claim that they produce entire emails, presentations, or software; §4 instead gives an auto-completed piece of code as an example. (LLM, conf=0.84)

  cites [157] 157. Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Sean Ó hÉigeartaigh, Anna Korho- nen, and José Hernández-Orallo. 2023. Your Prompt is My Command: On Assess- ing the Human-Centred Generality of Multimodal Models. Journal of Artificial Intelligence Research 77 (June 2023), 377-394. https://doi.org/10.1613/jair.1.14157 sourceExperimental — accuracy not yet measured

In the paper

“Beyond coding, manufacturing designers who successfully learned to co-create with GenAI abstracted and explained the problem to themselves [70]”

What the sources say

• Supported

  Section 8.3.2 states that participants more successful at co-creating with the AI tools abstracted and explained their problems, including to themselves. Section 5.3.2 describes designers sketching and explaining a design problem to themselves to overcome it. (LLM, conf=0.94)

  cites [70] 70. Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, and Nikolas Martelaro. 2023. Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. In Pro- ceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, 1-20. https://doi.org/10.1145/3544548.3580999 source

In the paper

“These difficulties were exacerbated by the many parameters available to users who grappled with understanding and using them effectively [70,83,204].”

What the sources say

• Supported

  Section 5.2.3 describes designers struggling to understand the meaning and effects of parameters, sometimes relying on defaults or making incorrect assumptions. Section 5.1 reports that only a few set the relevant parameters correctly on their first attempt. Together, these passages support the attributed difficulty in understanding and using the tools' parameters. (LLM, conf=0.87)

  cites [70] 70. Frederic Gmeiner, Humphrey Yang, Lining Yao, Kenneth Holstein, and Nikolas Martelaro. 2023. Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. In Pro- ceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, 1-20. https://doi.org/10.1145/3544548.3580999 source
• ?Could not verify

  Section 3.2 discusses parameters as knowledge stored within a model, while section 5.2.2 discusses controls for generation. Neither addresses whether users faced many available parameters or struggled to understand and use them effectively. Searches for user-configurable settings and difficulty using them found no passage supporting the claim. (LLM, conf=0.76)

  cites [83] 83. Dhanya Jayagopal, Justin Lubin, and Sarah E. Chasins. 2022. Exploring the Learnability of Program Synthesizers by Novice Programmers. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST '22). Association for Computing Machinery, New York, NY, USA, 1-15. https://doi.org/10.1145/3526113.3545659 sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Shared limitations section discusses whether models of confidence and error monitoring apply to real-world decisions, not whether users struggle with numerous available parameters. Searches for users’ difficulty understanding and using parameters found no passage addressing that claim; the paper’s discussion of model parameters does not establish it. (LLM, conf=0.76)

  cites [204] 204. Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. AI Chains: Trans- parent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, 1-22. https://doi.org/10.1145/3491102.3517582 sourceExperimental — accuracy not yet measured

In the paper

“One relevant type of cue-'processing fluency',”

What the sources say

• Supported

  The Level 3: Momentary Experience section defines processing fluency as the subjective ease of performing a cognitive task. It also describes people using that ease when judging how confident they are that their solutions are correct. (LLM, conf=0.90)

  cites [3] 3. Rakefet Ackerman. 2019. Heuristic Cues for Meta-Reasoning Judgments: Review and Methodology. Psihologijske teme 28, 1 (May 2019), 1-20. https://doi.org/10. 31820/pt.28.1.1 Publisher: Filozofski fakultet u Rijeci. source
• ?Could not verify

  The Introduction (p. 2) and Study Results (p. 6) discuss LLM prompt chaining, not processing fluency or judgments of information accuracy. Searches for “processing fluency” and for ease of processing affecting confidence found no passage addressing the claim in our copy. (LLM, conf=0.76)

  cites [201] 201. Jennifer Wiley, Thomas D. Griffin, Allison J. Jaeger, Andrew F. Jarosz, Patrick J. Cushen, and Keith W. Thiede. 2016. Improving metacomprehension accuracy in an undergraduate course context. Journal of Experimental Psychology: Applied 22, 4 (Dec. 2016), 393-405. https://doi.org/10.1037/xap0000096 sourceExperimental — accuracy not yet measured

In the paper

“The generative nature of GenAI-its 'originality' [157]-means that many user workflows will or have shifted from users generating content to evaluating it [149,154], as already documented in programming [156] and manufacturing design [70].”

What the sources say

• Partly supported

  Section 4, “Elements of the Prompting Process” (page unavailable), describes generated images and code as potential outputs that save users from producing them themselves. Its “System” table also identifies user validation of an answer as a step. These passages support a generate-and-evaluate workflow, but do not establish that many user workflows have shifted, or will shift, from content creation to evaluation. (LLM, conf=0.82)

  cites [157] 157. Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Sean Ó hÉigeartaigh, Anna Korho- nen, and José Hernández-Orallo. 2023. Your Prompt is My Command: On Assess- ing the Human-Centred Generality of Multimodal Models. Journal of Artificial Intelligence Research 77 (June 2023), 377-394. https://doi.org/10.1613/jair.1.14157 sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Abstract and 'How does confidence guide behavior?' section discuss evaluating one’s own cognitive performance, not evaluating GenAI-generated content. Searches for GenAI, content creation, and workflow changes found no passage addressing a shift from generating content to evaluating it. Page numbers were unavailable in our copy. (LLM, conf=0.76)

  cites [149] 149. Evan F. Risko and Megan O. Kelly. 2023. Thinking in the digital age: Everyday cognition and the dawn of a new age of metacognition research. Applied Cognitive Psychology 37, 4 (2023), 785-788. https://doi.org/10.1002/acp.4102 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/acp.4102. sourceExperimental — accuracy not yet measured
• Supported

  Section 7.2 (p. 11) explicitly describes a shift in programming from faster code authoring toward spending more time checking generated code. This supports the claim’s programming example, for which this paper is cited. (LLM, conf=0.92)

  cites [154] 154. Advait Sarkar. 2023. Exploring Perspectives on the Impact of Artificial In- telligence on the Creativity of Knowledge Work: Beyond Mechanised Pla- giarism and Stochastic Parrots. In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work (CHIWORK '23). Association for Computing Machinery, New York, NY, USA, 1-17. https: //doi.org/10.1145/3596671.3597650 source
• ?Could not verify

  The Discussion describes how dogmatism relates to information search under uncertainty, not how GenAI changes user workflows. Searches for GenAI, content generation, and a shift toward evaluating generated content found no passage addressing the attributed claim. (LLM, conf=0.76)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured

In the paper

“For example, simple textual prompts promoting self-awareness significantly decreased participants' susceptibility to incorrect (albeit realistic) information [153].”

What the sources say

• ?Could not verify

  The section “Towards a Doctrine of Negotiated Complexity” discusses simple in-app prompts for teaching software features, but not prompts promoting self-awareness or a measured reduction in susceptibility to incorrect information. Searches for those concepts and for misinformation interventions found no passage in our copy reporting the claimed result. (LLM, conf=0.76)

  cites [153] 153. Nikita A. Salovich and David N. Rapp. 2021. Misinformed and unaware? Metacognition and the influence of inaccurate information. Journal of Ex- perimental Psychology: Learning, Memory, and Cognition 47, 4 (2021), 608-624. https://doi.org/10.1037/xlm0000977 Place: US Publisher: American Psychologi- cal Association. sourceExperimental — accuracy not yet measured

In the paper

“For example, people can engage in multi-level planning, where hours, days, and weeks have to be considered simultaneously [6].”

What the sources say

• Supported

  Section 5.3.4 describes participants keeping daily lists alongside weekly or monthly project lists to coordinate deadlines, while estimating how long individual tasks take. This supports the citing sentence’s example of considering shorter and longer time horizons together in multi-level planning. (LLM, conf=0.87)

  cites [6] 6. Yoana Ahmetoglu, Duncan P. Brumby, and Anna L. Cox. 2021. To Plan or Not to Plan? A Mixed-Methods Diary Study Examining When, How and Why Knowledge Work Planning is Inaccurate. Proceedings of the ACM on Human- Computer Interaction 4, CSCW3 (Jan. 2021), 222:1-222:20. https://doi.org/10. 1145/3432921 source

In the paper

“Moreover, as noted, GenAI models are non-deterministic [136].”

What the sources say

• ?Could not verify

  The Introduction discusses imperfect machine-learning algorithms, and the Procedure describes a machine-vision algorithm used to estimate jelly-bean counts. Neither passage addresses generative AI or whether models produce different outputs for the same input; searches for non-determinism, randomness, and repeated-input behavior found no supporting passage in our copy. (LLM, conf=0.74)

  cites [136] 136. Shuyin Ouyang, Jie M. Zhang, Mark Harman, and Meng Wang. 2023. LLM is Like a Box of Chocolates: the Non-determinism of ChatGPT in Code Generation. http://arxiv.org/abs/2308.02828 arXiv:2308.02828 [cs]. sourceExperimental — accuracy not yet measured

In the paper

“As it stands, this requires developing an expertise and a well-adjusted confidence that is distinct from existing domain expertise, with, for example,”

What the sources say

• Partly supported

  The Discussion describes confidence-guided information search as metacognitive control and suggests that training could target it. It does not establish that people must develop a separate expertise or well-adjusted confidence distinct from their existing domain expertise; the training proposal is tentative. (LLM, conf=0.76)

  cites [156] 156. Advait Sarkar, Andrew D. Gordon, Carina Negreanu, Christian Poelitz, Sruti Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with ar- tificial intelligence? https://doi.org/10.48550/arXiv.2208.06213 arXiv:2208.06213 [cs]. sourceExperimental — accuracy not yet measured

In the paper

“Research on real-world GenAI workflows, primarily in AI-assisted coding, suggests that tools like GitHub Copilot alter users' workflows in diverse ways [166].”

What the sources say

• Supported

  Section 2.2.2 describes several ways programmers change their workflows around Copilot, including editing suggestions, breaking down code, and combining suggestions. Section 2.1 also describes a shift from writing code to evaluating generated output. Together, these passages support the claim that tools like Copilot alter workflows in diverse ways. (LLM, conf=0.91)

  cites [166] 166. Auste Simkute, Lev Tankelevitch, Viktor Kewenig, Ava Elizabeth Scott, Abigail Sellen, and Sean Rintel. 2024. Ironies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions. https://doi.org/10. 48550/arXiv.2402.11364 arXiv:2402.11364 [cs]. source

In the paper

“Self-evaluation is particularly promising for augmenting GenAI explainability, where it can help increase users' receptiveness to explanations that seek to update their mental models of GenAI systems [4,171] (see §4.2.1 for more on explainability).”

What the sources say

• ?Not checked — no full text of the sourcecites [4] 4. Rakefet Ackerman and Valerie Thompson. 2017. Meta-Reasoning: Shedding meta-cognitive light on reasoning research. 1-15. source
• Partly supported

  In 'Improving the Assessment of Mental Models' (page unavailable), the paper suggests using metacognition to understand how people assess their own confidence. In 'What information should the AI present to the human DM?' (page unavailable), it suggests that interventions increasing cognitive effort might help people use explanations effectively. It does not say that self-evaluation increases receptiveness to explanations or helps explanations update users’ mental models of GenAI. (LLM, conf=0.78)

  cites [171] 171. Mark Steyvers and Aakriti Kumar. 2023. Three Challenges for AI-Assisted Decision-Making. Perspectives on Psychological Science: A Journal of the As- sociation for Psychological Science (July 2023), 17456916231181102. https: //doi.org/10.1177/17456916231181102 sourceExperimental — accuracy not yet measured

In the paper

“We argue that this imposes a distinct metacognitive demand on users that must make these decisions, akin to [149], who make a similar argument for digital storage and memory.”

What the sources say

• ?Could not verify

  The Abstract defines metacognition as monitoring and controlling cognitive processes, including memory and decisions, but does not discuss digital storage. Searches for digital storage and decisions about what to store found no passage making the similar argument attributed to this source. (LLM, conf=0.76)

  cites [149] 149. Evan F. Risko and Megan O. Kelly. 2023. Thinking in the digital age: Everyday cognition and the dawn of a new age of metacognition research. Applied Cognitive Psychology 37, 4 (2023), 785-788. https://doi.org/10.1002/acp.4102 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/acp.4102. sourceExperimental — accuracy not yet measured

In the paper

“Similar systems have been shown to improve users' planning and other metacognitive processes [38].”

What the sources say

• Partly supported

  Section 5.4 reports that OrgBox users perceived significantly greater support for monitoring and evaluation than users of the comparison tool. Planning support was rated higher, but the difference was not statistically significant (p=.059). The study measured perceived support, not demonstrated improvements in users’ metacognitive processes. (LLM, conf=0.88)

  cites [38] 38. Anita Crescenzi, Austin R. Ward, Yuan Li, and Rob Capra. 2021. Supporting Metacognition during Exploratory Search with the OrgBox. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '21). Association for Computing Machinery, New York, NY, USA, 1197-1207. https://doi.org/10.1145/3404835.3462955 sourceExperimental — accuracy not yet measured

In the paper

“Users' confidence and their ability to adjust it may also be challenged by the fact that GenAI tools can have multiple and often non-intuitive failure modes [33,157].”

What the sources say

• Partly supported

  Section 5.2 discusses model errors arising from invalid input–output relationships, including complex nonlinear ones. The section on explanations without task-specific intuitions says users may be unable to identify model errors. These passages support difficulty recognizing AI errors, but they do not establish that GenAI tools have multiple, often non-intuitive failure modes. (LLM, conf=0.78)

  cites [33] 33. Chacha Chen, Shi Feng, Amit Sharma, and Chenhao Tan. 2023. Machine Expla- nations and Human Understanding. https://doi.org/10.48550/arXiv.2202.04092 arXiv:2202.04092 [cs]. sourceExperimental — accuracy not yet measured
• Supported

  Section 2, “A New Kind of Cognitive Tool,” says failures can be unexpected and may arise from prompt phrasing, system capabilities, or the presence or absence of reference material in training data. This supports the claim that these tools have multiple, non-intuitive failure modes; the passage does not quantify how often they occur. (LLM, conf=0.90)

  cites [157] 157. Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Sean Ó hÉigeartaigh, Anna Korho- nen, and José Hernández-Orallo. 2023. Your Prompt is My Command: On Assess- ing the Human-Centred Generality of Multimodal Models. Journal of Artificial Intelligence Research 77 (June 2023), 377-394. https://doi.org/10.1613/jair.1.14157 source

In the paper

“However, feedforward information can also increase cognitive load [42], so the right balance, particularly for complex systems, remains to be explored.”

What the sources say

• Supported

  Section 6.2.2 (p. 11) says users likely needed extra time to attend to feedforward previews and that some felt distracted by them. This supports the hedged claim that feedforward information can impose cognitive costs, although the study did not find a significant difference in perceived workload. (LLM, conf=0.85)

  cites [42] 42. Hai Dang, Lukas Mecke, and Daniel Buschek. 2022. GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, 1-15. https://doi.org/10.1145/3491102.3502141 source

In the paper

“but the quality of generated content and its uses may be more difficult to evaluate objectively (e.g., consider how one would objectively evaluate the quality of an LLM-generated email) [87,99].”

What the sources say

• Partly supported

  The Affective metacognition section (page unavailable) says that some subjective evaluations lack an objective ground truth, supporting the general difficulty of objective assessment. It does not address the quality or uses of generated content, including LLM-generated emails. (LLM, conf=0.82)

  cites [87] 87. Sucharit Katyal and Stephen Fleming. 2023. Construct validity in metacognition research: balancing the tightrope between rigor of measurement and breadth of construct. https://doi.org/10.31234/osf.io/etjqh sourceExperimental — accuracy not yet measured
• Partly supported

  Section 4.2.2 (p. 11) says there are few metrics for determining the quality of generated explanations, making it hard to decide when to disregard them. This supports the difficulty of evaluating explanations and their use, but the paper does not extend that point to generated content generally or to LLM-generated emails. (LLM, conf=0.84)

  cites [99] 99. Vivian Lai, Chacha Chen, Alison Smith-Renner, Q. Vera Liao, and Chenhao Tan. 2023. Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 1369-1385. https: //doi.org/10.1145/3593013.3594087 sourceExperimental — accuracy not yet measured

In the paper

“Likewise, self-explanation prompts improved students' accuracy in evaluating their own understanding of information [201].”

What the sources say

• ?Could not verify

  The Study Design (p. 5) describes four designers and developers building LLM prompt chains, not students responding to self-explanation prompts. The Study Results (pp. 5–6) discuss chain construction and debugging, but do not assess students’ accuracy in judging their own understanding. Searches for self-explanation and metacognitive accuracy found no relevant passage. (LLM, conf=0.76)

  cites [201] 201. Jennifer Wiley, Thomas D. Griffin, Allison J. Jaeger, Andrew F. Jarosz, Patrick J. Cushen, and Keith W. Thiede. 2016. Improving metacomprehension accuracy in an undergraduate course context. Journal of Experimental Psychology: Applied 22, 4 (Dec. 2016), 393-405. https://doi.org/10.1037/xap0000096 sourceExperimental — accuracy not yet measured

In the paper

“Beyond the metacognitive demands implied in the local interaction with GenAI, the generality of GenAI-its applicability to a wide range of tasks [157]-poses a higher-level question to end-users about their workflow automation strategy”

What the sources say

• Supported

  The citation is attached to the description of GenAI’s generality as applicability to a wide range of tasks, not to the citing authors’ subsequent question about workflow automation. Section 3, “Human-Centred Generality,” defines generality in terms of a user’s ability to use a system for a wide range of relevant cognitive tasks. (LLM, conf=0.93)

  cites [157] 157. Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Sean Ó hÉigeartaigh, Anna Korho- nen, and José Hernández-Orallo. 2023. Your Prompt is My Command: On Assess- ing the Human-Centred Generality of Multimodal Models. Journal of Artificial Intelligence Research 77 (June 2023), 377-394. https://doi.org/10.1613/jair.1.14157 source

In the paper

“Interventions that cue users to reflect on their goals and strategies via question prompts or conversational interfaces have been shown to improve outcomes in the workplace [92,122,209], education [51], and other domains [17,147].”

What the sources say

• ?Not checked — no full text of the sourcecites [92] 92. Rafal Kocielnik, Lillian Xiao, Daniel Avrahami, and Gary Hsieh. 2018. Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 2 (July 2018), 1-26. https://doi.org/10.1145/3214273 source
• Partly supported

  Section 5.3.2 (Dialogue; page unavailable) describes explanatory dialogue systems that use probing questions, including in intelligent tutoring. It does not show that prompts cue users to reflect on their goals and strategies or that these systems improve outcomes; it says the evaluation of EDGE was anecdotal, with no formal comparison. (LLM, conf=0.82)

  cites [122] 122. André N. Meyer, Gail C. Murphy, Thomas Zimmermann, and Thomas Fritz. 2021. Enabling Good Work Habits in Software Developers through Reflective Goal- Setting. IEEE Transactions on Software Engineering 47, 9 (Sept. 2021), 1872-1885. https://doi.org/10.1109/TSE.2019.2938525 Conference Name: IEEE Transactions on Software Engineering. sourceExperimental — accuracy not yet measured
• Partly supported

  The Conclusion (p. not available) likens code completion to a chatbot conversation and links Copilot usage to developers’ perceived productivity. It does not study prompts or interfaces that cue reflection on goals and strategies, or show that such interventions improve workplace outcomes. (LLM, conf=0.80)

  cites [209] 209. Nima Zargham, Leon Reicherts, Michael Bonfert, Sarah Theres Voelkel, Johannes Schoening, Rainer Malaka, and Yvonne Rogers. 2022. Understanding Circum- stances for Desirable Proactive Behaviour of Voice Assistants: The Proactivity Dilemma. In Proceedings of the 4th Conference on Conversational User Interfaces (CUI '22). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3543829.3543834 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [51] 51. Anneline Devolder, Johan van Braak, and Jo Tondeur. 2012. Supporting self- regulated learning in computer-based learning environments: Systematic review of effects of scaffolding in the domain of science education. Journal of Computer Assisted Learning 28 (Dec. 2012). https://doi.org/10.1111/j.1365-2729.2011.00476. x source
• Partly supported

  Section 5.2.2 describes conversational systems that ask questions to stimulate reflection, including a workplace chatbot. The Introduction also describes reflection-supporting systems across diverse settings. But the paper does not establish that prompts about goals and strategies improve outcomes in the workplace, education, and other domains; the Introduction says which techniques are likely to be effective remains unclear. (LLM, conf=0.85)

  cites [17] 17. Marit Bentvelzen, Paweł W. Woźniak, Pia S.F. Herbes, Evropi Stefanidi, and Jasmin Niess. 2022. Revisiting Reflection in HCI: Four Design Resources for Technologies that Support Reflection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (March 2022), 2:1-2:27. https: //doi.org/10.1145/3517233 sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction describes questions about people’s goals for their photographs, but not an intervention designed to improve outcomes. Section 3 discusses monitoring and control when using digital tools; neither it nor the passages found by searching for question prompts, conversational interfaces, and interventions reports improved outcomes from reflective prompts in any domain. (LLM, conf=0.74)

  cites [147] 147. Leon Reicherts and Yvonne Rogers. 2020. Do Make me Think! How CUIs Can Support Cognitive Processes. In Proceedings of the 2nd Conference on Conver- sational User Interfaces (CUI '20). Association for Computing Machinery, New York, NY, USA, 1-4. https://doi.org/10.1145/3405755.3406157 sourceExperimental — accuracy not yet measured

In the paper

“The question of how much control to give users-system customizability, or how many 'knobs' a user can adjust-is another design choice that can moderate the metacognitive demands of GenAI [42,119].”

What the sources say

• Partly supported

  Section 5.10 (pp. 8–9) reports that more sliders increased perceived workload and made decisions about which slider to adjust next harder, supporting the point that the number of controls affects user demands. The study tested an image-generation task, however, and did not directly measure metacognitive demands across GenAI systems. (LLM, conf=0.83)

  cites [42] 42. Hai Dang, Lukas Mecke, and Daniel Buschek. 2022. GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22). Association for Computing Machinery, New York, NY, USA, 1-15. https://doi.org/10.1145/3491102.3502141 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [119] 119. Andrew M Mcnutt, Chenglong Wang, Robert A Deline, and Steven M. Drucker. 2023. On the Design of AI-powered Code Assistants for Notebooks. In Pro- ceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, 1-16. https://doi.org/10.1145/3544548.3580940 source

In the paper

“One could be wellcalibrated on average (e.g., one's overall level of confidence matches one's overall level of accuracy) but still have poor resolution (i.e., one's confidence level does not vary much between correct and incorrect answers)[66].”

What the sources say

• Supported

  In “Quantifying the accuracy of probability judgments,” the paper defines calibration as the fit between confidence ratings and the proportion correct, and resolution as the extent to which correct and incorrect answers receive different ratings. It treats them as distinct contributors to accuracy and illustrates that one can be poor while the other is good, supporting the claimed converse. (LLM, conf=0.91)

  cites [66] 66. Stephen M. Fleming and Hakwan C. Lau. 2014. How to measure metacognition. Frontiers in Human Neuroscience 8 (2014). https://www.frontiersin.org/articles/ 10.3389/fnhum.2014.00443 source

In the paper

“Analogously, as we offload more of our cognition to today's GenAI systems, the demand for our metacognition increases [149].”

What the sources say

• ?Could not verify

  The Abstract defines metacognition as evaluating and controlling one’s own cognitive processes, and “How does confidence guide behavior?” discusses its role in learning and cognitive control. Neither passage addresses whether offloading cognition to GenAI increases the demand for metacognition. Searches for cognitive offloading, external tools, and increased metacognitive demand found no passage making that connection. (LLM, conf=0.76)

  cites [149] 149. Evan F. Risko and Megan O. Kelly. 2023. Thinking in the digital age: Everyday cognition and the dawn of a new age of metacognition research. Applied Cognitive Psychology 37, 4 (2023), 785-788. https://doi.org/10.1002/acp.4102 _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/acp.4102. sourceExperimental — accuracy not yet measured

In the paper

“Although the perspective of metacognition is equally relevant for understanding the usability of search engines and similar technologies, we focus our scope to GenAI systems as they are relevantly distinct from that of search engines[157].”

What the sources say

• Supported

  The paper says that interacting with prompt-commanded AI differs from other ways of interacting with machines (Modalities Models). Section 2, “A New Kind of Cognitive Tool,” contrasts its flexibility with tools restricted to searching the internet for existing content. Together, these passages support the cited distinction from search engines; page numbers are unavailable in our copy. (LLM, conf=0.88)

  cites [157] 157. Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Sean Ó hÉigeartaigh, Anna Korho- nen, and José Hernández-Orallo. 2023. Your Prompt is My Command: On Assess- ing the Human-Centred Generality of Multimodal Models. Journal of Artificial Intelligence Research 77 (June 2023), 377-394. https://doi.org/10.1613/jair.1.14157 source

In the paper

“The two aspects of confidence can be independent.”

What the sources say

• Supported

  The Introduction distinguishes metacognitive sensitivity from metacognitive bias and says a person can have high overall confidence but poor sensitivity. It also says the two can be quantified separately, supporting the claim that these aspects can be independent. (LLM, conf=0.92)

  cites [66] 66. Stephen M. Fleming and Hakwan C. Lau. 2014. How to measure metacognition. Frontiers in Human Neuroscience 8 (2014). https://www.frontiersin.org/articles/ 10.3389/fnhum.2014.00443 source

In the paper

“The design of intuitive interfaces for GenAI systems might therefore inadvertently give users a misleading sense of competence, increasing the risk of errors (likewise discussed in cognitive psychology [153]).”

What the sources say

• Partly supported

  Section 2 (‘How Did We Get Here?’, page unavailable) warns that software can be made to appear more usable while becoming less useful, supporting the general concern about intuitive design. The paper does not establish that GenAI interfaces give users a misleading sense of their own competence or increase errors. (LLM, conf=0.78)

  cites [153] 153. Nikita A. Salovich and David N. Rapp. 2021. Misinformed and unaware? Metacognition and the influence of inaccurate information. Journal of Ex- perimental Psychology: Learning, Memory, and Cognition 47, 4 (2021), 608-624. https://doi.org/10.1037/xlm0000977 Place: US Publisher: American Psychologi- cal Association. sourceExperimental — accuracy not yet measured

In the paper

“There is a risk that strategies to address metacognitive demands may also increase cognitive load, due to the additional information that users have to process, such as self-reflective prompts, a set of sub-goals resulting from task decomposition, or model explanations [46].”

What the sources say

• Supported

  In Research Question 3 (page unavailable), the paper says that making metacognitive prompts more explicit could increase cognitive load and risk overloading learners because they may spend effort analyzing additional cues. This supports the quoted risk and its proposed mechanism. (LLM, conf=0.90)

  cites [46] 46. Anique B. H. de Bruin, Julian Roelle, Shana K. Carpenter, Martine Baars, and EFG-MRE. 2020. Synthesizing Cognitive Load and Self-regulation Theory: a Theoretical Framework and Research Agenda. Educational Psychology Review 32, 4 (Dec. 2020), 903-915. https://doi.org/10.1007/s10648-020-09576-4 source

In the paper

“Having a well-calibrated confidence does not necessarily imply high confidence resolution.”

What the sources say

• Supported

  In 'Judgments of probability > Quantifying the accuracy of probability judgments' (page not provided), the paper defines calibration as how well confidence ratings match the corresponding frequency of correct responses, and resolution as the ability to assign correct and incorrect answers to different probability categories. These are distinct properties: confidence can match overall accuracy without distinguishing correct from incorrect answers. (LLM, conf=0.88)

  cites [66] 66. Stephen M. Fleming and Hakwan C. Lau. 2014. How to measure metacognition. Frontiers in Human Neuroscience 8 (2014). https://www.frontiersin.org/articles/ 10.3389/fnhum.2014.00443 source

In the paper

“Finally, indicating model uncertainty [19,174,198], for example, by means of color-coding [167], and an explanation for that uncertainty [5,190], can also help users disentangle the role of their prompting and output evaluation ability from that of models' capabilities, further supporting confidence adjustment [197].”

What the sources say

• Supported

  The source supports the portion attributed to it: indicating model uncertainty can help users calibrate their trust in a model. Section 3.3 says that a well-communicated uncertainty estimate should help users form appropriate trust and distinguish among model capabilities. The sentence attaches separate citations to explanations of uncertainty and to the prompting-related claim; those are not attributed to this source. (LLM, conf=0.88)

  cites [19] 19. Umang Bhatt, Javier Antorán, Yunfeng Zhang, Q. Vera Liao, Prasanna Sattigeri, Riccardo Fogliato, Gabrielle Melançon, Ranganath Krishnan, Jason Stanley, Omesh Tickoo, Lama Nachman, Rumi Chunara, Madhulika Srikumar, Adrian Weller, and Alice Xiang. 2021. Uncertainty as a Form of Transparency: Mea- suring, Communicating, and Using Uncertainty. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21). Association for Computing Machinery, New York, NY, USA, 401-413. https://doi.org/10.1145/ 3461702.3462571 source
• ?Not checked — no full text of the sourcecites [174] 174. Jiao Sun, Q. Vera Liao, Michael Muller, Mayank Agarwal, Stephanie Houde, Kartik Talamadupula, and Justin D. Weisz. 2022. Investigating Explainability of Generative AI for Code through Scenario-based Design. In 27th International Conference on Intelligent User Interfaces (IUI '22). Association for Computing Machinery, New York, NY, USA, 212-228. https://doi.org/10.1145/3490099. 3511119 source
• Supported

  The citation to this paper is attached to providing an explanation for model uncertainty, not to the later claim about disentangling users’ abilities from model capabilities. Section 5.2.2 (p. 9) reports that alternate translations explained why the model highlighted tokens as low-confidence; Section 5.1 (p. 8) says such explanations could reduce confusion about confidence discrepancies. (LLM, conf=0.91)

  cites [198] 198. Justin D. Weisz, Michael Muller, Stephanie Houde, John Richards, Steven I. Ross, Fernando Martinez, Mayank Agarwal, and Kartik Talamadupula. 2021. Perfection Not Required? Human-AI Partnerships in Code Translation. In 26th International Conference on Intelligent User Interfaces (IUI '21). Association for Computing Machinery, New York, NY, USA, 402-412. https://doi.org/10.1145/ 3397481.3450656 source
• ?Not checked — no full text of the sourcecites [167] 167. Sofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, and Jake M. Hofman. 2023. Comparing Traditional and LLM-based Search for Con- sumer Choice: A Randomized Experiment. http://arxiv.org/abs/2307.03744 arXiv:2307.03744 [cs]. source
• Partly supported

  Section 2.1.2 (p. 2) reports that users struggled to understand when and why the code-translation model lacked confidence, and Section 4.2 (p. 4) identifies a need for interpretable confidence values. This supports the cited point about explaining uncertainty, but the paper does not show that such explanations help users separate their prompting or output-evaluation ability from model capability. (LLM, conf=0.82)

  cites [5] 5. Mayank Agarwal, Kartik Talamadupula, Stephanie Houde, Fernando Martinez, Michael Muller, John Richards, Steven Ross, and Justin D. Weisz. 2021. Quality Estimation & Interpretability for Code Translation. https://doi.org/10.48550/ arXiv.2012.07581 arXiv:2012.07581 [cs]. sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [190] 190. Helena Vasconcelos, Gagan Bansal, Adam Fourney, Q. Vera Liao, and Jen- nifer Wortman Vaughan. 2023. Generation Probabilities Are Not Enough: Exploring the Effectiveness of Uncertainty Highlighting in AI-Powered Code Completions. https://doi.org/10.48550/arXiv.2302.07248 arXiv:2302.07248 [cs]. source

In the paper

“Drawing on research in psychology and cognitive science, and recent GenAI user studies, we illustrate how GenAI systems impose metacognitive demands on users, requiring a high degree of metacognitive monitoring and control.”

In the paper

“We posit that these metacognitive demands are induced by GenAI's model flexibility, generality, and originality.1”

In the paper

“Moreover, it often requires that a task be broken up into sub-tasks (”

In the paper

“This demand for self-awareness and task decomposition is exacerbated by a particular type of model flexibility in GenAI: today's systems afford many parameters for end-users to adjust; these can be formal parameters like the model temperature, or a range of unspecified parameters that can be adjusted through text prompting (e.g., the tone, level of detail, or structure of a piece of text).”

In the paper

“For example, as we illustrate below, prompting in current GenAI systems imposes a high metacognitive demand due to the need for self-awareness of goals, increasing cognitive load, while the interaction method of typing (rather than speaking) further increases cognitive load, albeit without much associated metacognitive demand.”

In the paper

“Here, we argue that metacognition-the psychological ability to monitor and control one's own thought processes [4,62,131,181]-offers a valuable and unexplored perspective to understand and design for the usability challenges of GenAI.”

In the paper

“Many GenAI systems require specification that the email is to a senior colleague and needs an appropriate tone.”

In the paper

“Next, iterative output evaluation and adjustment (prompt iteration) depends on users' confidence in their prompting ability, and metacognitive flexibility to adapt their prompting strategy (Figure 2b).”

In the paper

“This model flexibility and control afforded to users requires knowing what one wants to achieve and convey that explicitly and effectively to the system.”

In the paper

“Monitoring and control are interrelated [93,95].”

What the sources say

• ?Not checked — no full text of the sourcecites [93] 93. Asher Koriat. 2007. Metacognition and consciousness. In The Cambridge handbook of consciousness. Cambridge University Press, New York, NY, US, 289-325. https://doi.org/10.1017/CBO9780511816789.012 source
• ?Not checked — no full text of the sourcecites [95] 95. Asher Koriat, Hilit Ma'ayan, and Ravit Nussinson. 2006. The intricate rela- tionships between monitoring and control in metacognition: Lessons for the cause-and-effect relation between subjective experience and behavior. Jour- nal of Experimental Psychology: General 135, 1 (Feb. 2006), 36-69. https: //doi.org/10.1037/0096-3445.135.1.36 source

In the paper

“Analogously, the ability of GenAI systems to quickly and easily generate extensive content may serve as a cue that misleadingly increases users' confidence, not only in the output itself, but also in their own ability to evaluate it.”

In the paper

“The challenge of output evaluation is present even in interaction modes without user prompting, such as in GitHub Copilot, which includes automated suggestions.”

In the paper

“Systems that provide automated suggestions such as GitHub Copilot alleviate some of the demands associated with prompting.”

In the paper

“In [208], non-expert participants used an LLM-based tool to improve a chatbot through prompting.One of the challenges they experienced was a struggle getting started.7”

What the sources say

• ?Not checked — no full text of the sourcecites [208] 208. J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1-21. https://doi.org/10.1145/ 3544548.3581388 source

In the paper

“To be clear, these challenges partly stem from a lack of feedback in the system about prompting effectiveness, leaving users to grapple with the fuzzy abstraction matching problem without support”

In the paper

“How this will affect users' selfconfidence and AI reliance remains unclear, but metacognition research suggests that increased effort requirements may discourage users from appropriate evaluation [4].”

What the sources say

• ?Not checked — no full text of the sourcecites [4] 4. Rakefet Ackerman and Valerie Thompson. 2017. Meta-Reasoning: Shedding meta-cognitive light on reasoning research. 1-15. source

In the paper

“We note that, although self-awareness and task decomposition are often required to some extent when interacting with GenAI systems, their pertinence increases as users concretize their usage intentions (e.g., achieving work or personal goals).”

In the paper

“Lu and Yin [108] show that, in the absence of AI accuracy information, humans rely on their agreement with AI as a heuristic for their reliance on it, albeit only when they have a high self-confidence.”

What the sources say

• ?Not checked — no full text of the sourcecites [108] 108. Zhuoran Lu and Ming Yin. 2021. Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21). Association for Computing Machinery, New York, NY, USA, 1-16. https://doi. org/10.1145/3411764.3445562

In the paper

“Evaluating and relying on AI output requires users to maintain a well-adjusted confidence in their own domain expertise and ability to evaluate output (i.e., self-confidence; Figure 2c).”

In the paper

“We posit that GenAI exacerbates the demand for a well-adjusted confidence in output evaluation.”

In the paper

“The importance of this metacognitive demand is evidenced in recent research on AIassisted decision-making, which finds that users' self-confidence is a key determinant of their reliance on AI responses”

In the paper

“In fact, such interaction modes may arguably make output evaluation more challenging due to the need to infer the intent behind systems' suggestions [74].”

What the sources say

• ?Not checked — no full text of the sourcecites [74] 74. Ken Gu, Madeleine Grunde-McLaughlin, Andrew M. McNutt, Jeffrey Heer, and Tim Althoff. 2023. How Do Data Analysts Respond to AI Assistance? A Wizard- of-Oz Study. https://doi.org/10.48550/arXiv.2309.10108 arXiv:2309.10108 [cs]. source

In the paper

“Expert programmers have advanced metacognitive monitoring and control, in that they are able to identify their specific goals and decompose them into concrete tasks [60].”

What the sources say

• ?Not checked — no full text of the sourcecites [60] 60. Anneli Eteläpelto. 1993. Metacognition and the Expertise of Computer Pro- gram Comprehension. Scandinavian Journal of Educational Research 37, 3 (Jan. 1993), 243-254. https://doi.org/10.1080/0031383930370305 Publisher: Routledge _eprint: https://doi.org/10.1080/0031383930370305. source

In the paper

“The relative ease with which GenAI can produce extensive output may also affect output evaluation and reliance via potentially misleading cues that people implicitly rely on to update their confidence and guide their subsequent metacognitive control [4].”

What the sources say

• ?Not checked — no full text of the sourcecites [4] 4. Rakefet Ackerman and Valerie Thompson. 2017. Meta-Reasoning: Shedding meta-cognitive light on reasoning research. 1-15. source

In the paper

“Novices in a domain or in GenAI may be particularly vulnerable, as one participant commented on long code suggestions:”

In the paper

“self-evaluation interventions can help users reflect on their task goals and approaches to task decomposition [74].”

What the sources say

• ?Not checked — no full text of the sourcecites [74] 74. Ken Gu, Madeleine Grunde-McLaughlin, Andrew M. McNutt, Jeffrey Heer, and Tim Althoff. 2023. How Do Data Analysts Respond to AI Assistance? A Wizard- of-Oz Study. https://doi.org/10.48550/arXiv.2309.10108 arXiv:2309.10108 [cs]. source

In the paper

“Such an understanding can in turn mitigate the risks of inappropriate evaluation of, and confidence in, AI-generated output.”

In the paper

“Self-evaluation can be used to support effective prompting.”

In the paper

“More importantly, changes in confidence can affect people's approach to evaluating GenAI output.”

In the paper

“The benefits of generated content may also be diffuse and indirect.”

In the paper

“More flexible interfaces can support the crafting of a complex prompting strategy by enabling open-ended exploration of task goals and the relationships between them during the planning process.”

In the paper

“However, tasks may nevertheless be complex or ambiguous, often requiring gathering, organizing, and synthesizing information in a nonlinear manner distinct from the linear conversational interfaces in most GenAI systems today.”

In the paper

“For example, participants co-writing with an LLM found that seeing the LLM's suggestions was helpful even when they did not implement them [207].”

What the sources say

• ?Not checked — no full text of the sourcecites [207] 207. Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces (IUI '22). Association for Computing Machinery, New York, NY, USA, 841-852. https://doi.org/10.1145/3490099.3511105 source

In the paper

“Adjusting confidence in output evaluation typically requires objective measures of performance for comparison (e.g., the number of errors a user correctly detected in the output)”

In the paper

“By increasing people's confidence, such cues can affect their metacognitive control, leading people to decrease the effort they invest into further deliberate processing, as measured by thinking time and changes-of-mind [4,183,184].”

What the sources say

• ?Not checked — no full text of the sourcecites [4] 4. Rakefet Ackerman and Valerie Thompson. 2017. Meta-Reasoning: Shedding meta-cognitive light on reasoning research. 1-15. source
• ?Not checked — no full text of the sourcecites [183] 183. Valerie Thompson and Kinga Morsanyi. 2012. Analytic thinking: do you feel like it? Mind & Society 11, 1 (June 2012), 93-105. https://doi.org/10.1007/s11299- 012-0100-6 source
• ?Not checked — no full text of the sourcecites [184] 184. Valerie A. Thompson, Jamie A. Prowse Turner, Gordon Pennycook, Linden J. Ball, Hannah Brack, Yael Ophir, and Rakefet Ackerman. 2013. The role of answer fluency and perceptual fluency as metacognitive cues for initiating analytic thinking. Cognition 128, 2 (Aug. 2013), 237-251. https://doi.org/10.1016/j. cognition.2012.09.012 source

In the paper

“Planning-related interventions can also support users directly in task decomposition, improving prompt effectiveness via more explicit and discrete instructions.”

In the paper

“When applied to the prompting and output evaluation process itself, these approaches can make users aware of where they are in a task.”

In the paper

“The relationship between metacognition and cognitive load is an active research area [46,163,193,194], but it is plausible that, although cognitive load may increase due to the processing requirements of many metacognitive support interventions, the improvement in metacognition may be accompanied by a simultaneous and larger reduction in cognitive load, resulting in a net decrease in cognitive”

In the paper

“From the perspective of metacognitive demand, by providing contextual and performance information alongside the system inputs and outputs, explainability should help partly offload metacognitive processing from users' minds and onto the system interface.”

In the paper

“On one hand, increasing customizability can increase the demand for self-awareness (e.g., “are any of the settings relevant to my task?”), well-adjusted confidence (e.g., “did any of the settings affect the output quality, or is it related to my prompting?”),task decomposition (e.g., “what is the right order to adjust settings for each of my subtasks?”),and metacognitive flexibility (e.g., “which”

In the paper

“Support getting started is a key user request for GenAI explainability; see §4.2.1.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites §4.2.1

In the paper

“As we aim to demonstrate here, all of these features place unique demands on users' metacognition and inform the design space of solutions to address these demands, a design space which necessarily extends beyond that of current search engines.”

In the paper

“This may increase cognitive load, particularly for novice users.”

In the paper

“On the other hand, increased customizability can support metacognition, particularly for more advanced users.”

In the paper

“As we aim to demonstrate here, all of these features place unique demands on users' metacognition and inform the design space of solutions to address these demands, a design space which necessarily extends beyond that of current search engines.”

In the paper

“As per §3.2, processing fluency can lead to inflated confidence without necessarily improving objective accuracy.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites §3.2

In the paper

“The usability challenges of prompting make it a key target for explainability, as per §4.2.1.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites §4.2.1