CoChat
Sign Up
Cited by 6Decision-Making and Behavioral EconomicsInnovative Teaching and Learning MethodsMemory Processes and Influences

Systematic metacognitive reflection helps people discover far-sighted decision strategies: A process-tracing experiment

Open access
Becker, Frederic, Wirzberger, Maria, Pammer-Schindler, Viktoria, Srinivas, Srinidhi, Lieder, Falk·2023·Judgment and Decision Making
DOI 10.1017/jdm.2023.16
Open this paper in CoChat — free accountOpen PDF

Abstract

Short-sighted decisions can have devastating consequences, and teaching people to make their decisions in a more far-sighted way is challenging. Previous research found that reflecting on one’s behavior can boost learning from success and failure. Here, we explore the potential benefits of guiding people to reflect on whether and how they thought about what to do (i.e.,systematic metacognitive reflection). We devised a series of Socratic questions that prompt people to reflect on their decision-making and tested their effectiveness in a process-tracing experiment with a 5-step planning task ($N=265$). Each participant went through several cycles of making a series of decisions and then either reflecting on how they made those decisions, answering unrelated questions, or moving on to the next decision right away. We found that systematic metacognitive reflection helps people discover adaptive, far-sighted decision strategies faster. Our results suggest that systematic metacognitive reflection is a promising approach to boosting people’s decision-making competence.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1017/jdm.2023.16

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Enabling users to critically evaluate AI outputs can build self-awareness by forcing users to rationalize their decisions to the GenAI system, enhance metacognitive flexibility by providing users with different perspectives on their task, and adjust confidence by adapting explanations to the users' needs and the models' confidence.

    “Enabling users to critically evaluate AI outputs can build self-awareness by forcing users to rationalize their decisions to the GenAI system [156], enhance metacognitive flexibility by providing users with different perspectives on their task [15], and adjust confidence by adapting explanations to the users' needs and the models' confidence [110].”

    From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [15]

    Split into 3 atoms — 0 confirmed, 3 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 51

  1. Ackerman, R., & Thompson, V. A. (2017). Meta-reasoning: Monitoring and control of thinking and reasoning. Trends in Cognitive Sciences, 21(8), 607-617. https://doi.org/10.1016/j.tics.2017.05.004 · DOI · Paper page
  2. Anseel, F., Lievens, F., & Schollaert, E. (2009). Reflection as a strategy to enhance task performance after feedback. Organizational Behavior and Human Decision Processes, 110(1), 23-35. https://doi.org/10.1016/j.obhdp.2009.05.003 · DOI
  3. Azevedo, R. (2005). Computer environments as metacognitive tools for enhancing learning. Educational Psychologist, 40(4), 193-197. https://doi.org/10.1207/s15326985ep4004_1 · DOI
  4. Bannert, M., Hildebrand, M., & Mengelkamp, C. (2009). Effects of a metacognitive support device in learning environments. Computers in Human Behavior, 25(4), 829-835. https://doi.org/10.1016/j.chb.2008.07.002 · DOI
  5. Bannert, M., & Reimann, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193-211. https://doi.org/10.1007/s11251-011-9167-4 · DOI
  6. Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289-300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x · DOI · Paper page
  7. Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48(3), 306-307. https://doi.org/10.1207/s15327752jpa4803_13 · DOI · Paper page
  8. Callaway, F., Lieder, F., Krueger, P. M., & Griffiths, T. L. (2017). Mouselab-MDP: A new paradigm for tracing how people plan. In The 3rd multidisciplinary conference on reinforcement learning and decision making. https://doi.org/10.31219/osf.io/ 7wcya · DOI
  9. Ellis, S., Carette, B., Anseel, F., & Lievens, F. (2014). Systematic reflection: Implications for learning from failures and successes. Current Directions in Psychological Science, 23(1), 67-72. https://doi.org/10.1177/0963721413504106 · DOI
  10. Ellis, S., & Davidi, I. (2005). After-event reviews: Drawing lessons from successful and failed experience. Journal of Applied Psychology, 90(5), 857-871. https://doi.org/10.1037/0021-9010.90.5.857 · DOI
  11. Ellis, S., Mendel, R., & Nir, M. (2006). Learning from successful and failed experience: The moderating role of kind of after- event review. Journal of Applied Psychology, 91(3), 669-680. https://doi.org/10.1037/0021-9010.91.3.669 · DOI
  12. Erev, I., & Barron, G. (2005). On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychologi- cal Review, 112(4), 912-931. https://doi.org/10.1037/0033-295X.112.4.912 · DOI
  13. Ford, J., Schmitt, N., Schechtman, S. L., Hults, B. M., & Doherty, M. L. (1989). Process tracing methods: Contributions, problems, and neglected research questions. Organizational Behavior and Human Decision Processes, 43(1), 75-117. https://doi.org/10.1016/0749-5978(89)90059-9 · DOI
  14. Goda, G. S., Levy, M., Manchester, C. F., Sojourner, A., & Tasoff, J. (2019). Predicting retirement savings using survey measures of exponential-growth bias and present bias. Economic Inquiry, 57(3), 1636-1658. https://doi.org/10.1111/ecin.12792 · DOI
  15. Griffiths, T. L., Callaway, F., Chang, M. B., Grant, E., Krueger, P. M., & Lieder, F. (2019). Doing more with less: Meta-reasoning and meta-learning in humans and machines. Current Opinion in Behavioral Sciences, 29, 24-30. https://doi.org/10.1016/j.cobeha.2019.01.005 · DOI
  16. He, R., Jain, Y. R., & Lieder, F. (2021). Measuring and modelling how people learn how to plan and how people adapt their planning strategies to the structure of the environment. In International conference on cognitive modeling. https://is.mpg.de/uploadsfile/attachment/attachment/671/20210720ICCMsubmissionfinal.pdf
  17. He, R., & Lieder, F. (2022a). Learning-induced changes in people's planning strategies. Preprint. https://doi.org/10.13140/RG.2.2.12721.66407 · DOI
  18. He, R., & Lieder, F. (2022b). Where do adaptive planning strategies come from? Preprint. https://doi.org/10.13140/RG.2.2.28966.60487 · DOI
  19. Hertwig, R., & Grüne-Yanoff, T. (2017). Nudging and boosting: Steering or empowering good decisions. Perspectives on Psychological Science, 12(6), 973-986. https://doi.org/10.1177/1745691617702496 · DOI
  20. Hertwig, R., & Wulff, D. U. (2022). A description-experience framework of the psychology of risk. Perspectives on Psychological Science, 17(3), 631-651.
  21. Hilliger, I., De Laet, T., Henríquez, V., Guerra, J., Ortiz-Rojas, M., Zuñiga, M. Á., Baier, J., & Pérez-Sanagustín, M. (2020). For learners, with learners: Identifying indicators for an academic advising dashboard for students. In C. Alario-Hoyos, M. J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S. M. Dennerlein (Eds.), Addressing global challenges and quality education (pp. 117-130). Cham: Springer International Publishing.
  22. Irwin, J. R., & McClelland, G. H. (2001). Misleading heuristics and moderated multiple regression models. Journal of Marketing Research, 38(1), 100-109. https://doi.org/10.1509/jmkr.38.1.100.18835 · DOI
  23. Jain, Y. R., Callaway, F., Griffiths, T. L., Dayan, P., He, R., Krueger, P. M., & Lieder, F. (2022). A computational process- tracing method for measuring people's planning strategies and how they change over time. Behavior Research Methods. https://doi.org/10.3758/s13428-022-01789-5 · DOI
  24. Jain, Y. R., Gupta, S., Rakesh, V., Dayan, P., Callaway, F., & Lieder, F. (2019). How do people learn how to plan? In 2019 conference on cognitive computational neuroscience (pp. 826-829). https://doi.org/10.32470/CCN.2019.1313-0 · DOI
  25. Johnson, E. J., Camerer, C., Sen, S., & Rymon, T. (2002). Detecting failures of backward induction: Monitoring information search in sequential bargaining. Journal of Economic Theory, 104(1), 16-47. https://doi.org/10.1006/jeth.2001.2850 · DOI
  26. Johnson, E. J., Schulte-Mecklenbeck, M., & Willemsen, M. C. (2008). Process models deserve process data: Comment on Brand- stätter, Gigerenzer, and Hertwig (2006). Psychological Review, 115, 263-272. https://doi.org/10.1037/0033-295X.115.1.263 · DOI
  27. Kinnier, R. T., & Metha, A. T. (1989). Regrets and priorities at three stages of life. Counseling and Values, 33(3), 182-193. https://doi.org/10.1002/j.2161-007X.1989.tb00761.x · DOI
  28. Kirschner, P., Sweller, J., & Clark, R. E. (2006). Why unguided learning does not work: An analysis of the failure of discovery learning, problem-based learning, experiential learning and inquiry-based learning. Educational Psychologist, 41(2), 75-86. https://www.asec.purdue.edu/LCT/HBCU/documents/AnalysisoftheFailureofDiscoveryPBLExperientialInquiryLearning. pdf
  29. Kocielnik, R., Xiao, L., Avrahami, D., & Hsieh, G. (2018). Reflection companion: A conversational system for engaging users in reflection on physical activity. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(2), 1-26. https://doi.org/10.1145/3214273 · DOI · Paper page
  30. Krueger, P. M., Lieder, F., & Griffiths, T. (2017). Enhancing metacognitive reinforcement learning using reward structures and feedback. In Proceedings of the 39th annual conference of the cognitive science society. https://doi.org/10.13140/RG.2.2.33793.58729 · DOI
  31. Larrick, R. P. (2004). Debiasing. In D. J. Koehler & N. Harvey (Eds.), Blackwell handbook of judgment and decision making (pp. 316-338). Malden, MA , Blackwell Publishing Ltd. https://doi.org/10.1002/9780470752937. · DOI · Paper page
  32. ch16 · DOI · Paper page
  33. Lejarraga, T., Ranganathan, K., & Wulff, D. U. (2022). Can simulated experience be harnessed to help people make investment decisions? PsyArXiv Preprint. https://doi.org/10.31234/osf.io/cw8f4 · DOI
  34. Lieder, F., & Griffiths, T. L. (2017). Strategy selection as rational metareasoning. Psychological Review, 124(6), 762-794. https://doi.org/10.1037/rev0000075 · DOI
  35. Loftus, G. R. (1978). On interpretation of interactions. Memory & Cognition, 6(3), 312-319. https://doi.org/10.3758/BF03197461 · DOI
  36. Pammer-Schindler, V., & Prilla, M. (2021). The reflection object: An activity-theory informed concept for designing for reflection. Interacting with Computers, 33(3), 295-310. https://doi.org/10.1093/iwc/iwab027 · DOI
  37. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1988). Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14(3), 534-552. https://doi.org/10.1037/0278-7393.14.3.534 · DOI · Paper page
  38. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge: Cambridge University Press.
  39. Renner, B., Wesiak, G., Pammer-Schindler, V., Prilla, M., Müller, L., Morosini, D., Mora, S., Faltin, N., & Cress, U. (2020). Computer-supported reflective learning: How apps can foster reflection at work. Behaviour & Information Technology, 39(2), 167-187. https://doi.org/10.1080/0144929X.2019.1595726 · DOI
  40. Rieskamp, J., & Otto, P. E. (2006). SSL: A theory of how people learn to select strategies. Journal of Experimental Psychology: General, 135(2), 207-236. https://doi.org/10.1037/0096-3445.135.2.207 · DOI
  41. Ryan, M., & Ryan, M. (2013). Theorising a model for teaching and assessing reflective learning in higher education. Higher Education Research & Development, 32(2), 244-257. https://doi.org/10.1080/07294360.2012.661704 · DOI
  42. Shrager, J., & Siegler, R. S. (1998). Scads: A model of children's strategy choices and strategy discoveries. Psychological Science, 9(5), 405-410. https://doi.org/10.1111/1467-9280.00076 · DOI
  43. Siegler, R., & Jenkins, E. A. (2014). How children discover new strategies (1st ed.). New York: Psychology Press. https://doi.org/10.4324/9781315807744 · DOI
  44. Siegler, R. S. (1999). Strategic development. Trends in Cognitive Sciences, 3(11), 430-435. https://doi.org/10.1016/S1364-6613(99)01372-8 · DOI
  45. Spiller, S. A., Fitzsimons, G. J., John G., Lynch, J., & Mcclelland, G. H. (2013). Spotlights, floodlights, and the magic number zero: Simple effects tests in moderated regression. Journal of Marketing Research, 50(2), 277-288. https://doi.org/10.1509/jmr.12.0420 · DOI
  46. Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1(1), 3-14. https://doi.org/10.1007/s11409-006-6893-0 · DOI · Paper page
  47. Wang, Y., & Sloan, F. A. (2018). Present bias and health. Journal of Risk and Uncertainty, 57(2), 177-198. https://doi.org/10.1007/s11166-018-9289-z · DOI
  48. Willemsen, M. C., & Johnson, E. J. (2011). Visiting the decision factory: Observing cognition with mouselabweb and other information acquisition methods. In M. Schulte-Mecklenbeck, A. Kuehberger, & J. G. Johnson (Eds.), A handbook of process tracing methods for decision research (pp. 21-42). New York: Psychology Press.
  49. Wolfbauer, I., Pammer-Schindler, V., & Rosé, C. (2020). Rebo junior: Analysis of dialogue structure quality for a reflection guidance chatbot. In EC-TEL impact paper proceedings 2020: 15th European conference on technology enhanced learning. https://par.nsf.gov/biblio/10174707
  50. Wood Daudelin, M. (1996). Learning from experience through reflection. Organizational Dynamics, 24(3), 36-48. https://doi.org/10.1016/S0090-2616(96)90004-2 · DOI
  51. Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3-17. https://doi.org/10.1207/ · DOI · Paper page

Cited by

Sorted by citations

Authors
Becker, F., Wirzberger, M., Pammer-Schindler, V., Srinivas, S., & Lieder, F.
Affiliations
  • Max Planck Institute for Intelligent Systems (DE)
  • University of Stuttgart (DE)
Corresponding author
Becker, Frederic
ORCID
Venue
Judgment and Decision Making
Publisher
Cambridge University Press (CUP)
Published
2023
Volume
18
ISSN
1930-2975 (ISSN-L)
DOI
10.1017/jdm.2023.16
OpenAlex
W4377080201
Cited by
6
References
7
Citations by year
2026: 2 · 2025: 2 · 2024: 1 · 2023: 1
FWCI
1.26
Type
journal article
Language
en
Open access
Licence
Locations
  • Judgment and Decision Making (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
Keywords
metacognitive reflection, Socratic questioning, decision strategies, adaptive decision-making, decision-making capacity, planning tasks, process tracing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review