Eight-minute self-regulation intervention raises educational attainment at scale in individualist but not collectivist cultures
Open accessAbstract
Significance: High attrition from educational programs is a major obstacle to social mobility and a persistent source of economic inefficiency. Over two-thirds of students entering a 2-y institution fail to earn a credential in the United States. In online courses, attrition rates are even higher. In two large field experiments, we tested the conditions under which a writing activity that facilitates goal commitment and goal-directed behavior reduces attrition in online courses. The activity raised completion rates by up to 78% for members of individualist cultures and primarily for those who contended with predictable and surmountable obstacles in the form of everyday obligations, but it was ineffective in collectivist cultures and for people contending with other types of obstacles.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Kizilcec and Cohen (2017) found MCII could increase individual goal attainment (MOOCs completion rates) in individualist cultures but not collectivist cultures.
“For example, Kizilcec and Cohen (2017) found that MCII could increase individual goal attainment (massive open online courses, abbreviated by MOOCs, completion rates) in individualist cultures but not collectivist cultures.”
From A Meta-Analysis of the Effects of Mental Contrasting With Implementation Intentions on Goal Attainment (2021) · cites this paper as Kizilcec and Cohen (2017)
Kizilcec and Cohen's (2017) studies on completing MOOCs found MCII to be effective for learners in individualist cultures but not learners in collectivist cultures.
“Kizilcec and Cohen's (2017) studies on completing MOOCsincluded Western and Eastern subjects and found MCII to be effective for learners in individualist cultures but not learners in collectivist cultures.”
From A Meta-Analysis of the Effects of Mental Contrasting With Implementation Intentions on Goal Attainment (2021) · cites this paper as Kizilcec and Cohen's (2017)
In the moderator analysis, Kizilcec and Cohen (2017) was excluded because of the large sample size in their study, which might render the results of moderator effect unreliable.
“Moreover, in the moderator analysis, we excluded Kizilcec and Cohen (2017) because of the large sample size in their study, which might render the results of moderator effect unreliable.”
From A Meta-Analysis of the Effects of Mental Contrasting With Implementation Intentions on Goal Attainment (2021) · cites this paper as Kizilcec and Cohen (2017)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Culture and the self: Implications for cognition, emotion, and motivation.
Markus, H. R., & Kitayama, S. (1991) · Psychological Review · doi:10.1037/0033-295x.98.2.224 · Cited by 21,995 · see citing works
Motivational and self-regulated learning components of classroom academic performance.
Pintrich, P. R., & De Groot, E. V. (1990) · Journal of Educational Psychology · doi:10.1037/0022-0663.82.1.33 · Cited by 8,332 · see citing works
Social cognitive theory of self-regulation
Bandura, A. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90022-l · Cited by 6,875 · see citing works
High self-control predicts good adjustment, less pathology, better grades, and interpersonal success
Tangney, J. P., Baumeister, R. F., & Boone, A. L. (2004) · Journal of Personality · doi:10.1111/j.0022-3506.2004.00263.x · Cited by 6,557 · see citing works
Implementation intentions: Strong effects of simple plans.
Gollwitzer, P. M. (1999) · American Psychologist · doi:10.1037/0003-066x.54.7.493 · Cited by 5,478 · see citing works
Culture and systems of thought: Holistic versus analytic cognition.
Nisbett, R. E., Peng, K., Choi, I., & Norenzayan, A. (2001) · Psychological Review · doi:10.1037/0033-295x.108.2.291 · Cited by 4,263 · see citing works
Differences Between Tight and Loose Cultures: A 33-Nation Study
Gelfand, M. J., Raver, J. L., Nishii, L., Leslie, L. M., Lun, J., Lim, B. C., Duan, L., Almaliach, A., Ang, S., Arnadottir, J., et al. (2011) · Science · doi:10.1126/science.1197754 · Cited by 3,252 · see citing works
Implementation intentions and goal achievement: a meta-analysis of effects and processes
Gollwitzer, P. M., & Sheeran, P. (2006) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(06)38002-1 · Cited by 3,236 · see citing works
Implementation intentions and effective goal pursuit.
Gollwitzer, P. M., & Brandstätter, V. (1997) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.73.1.186 · Cited by 1,343 · see citing works
<b>topicmodels</b> : An <i>R</i> Package for Fitting Topic Models
Grün, B., & Hornik, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v040.i13 · Cited by 1,072 · see citing works
Kizilcec, R. F., Pérez-Sanagustín, M., & Maldonado, J. J. (2017) · Computers & Education · doi:10.1016/j.compedu.2016.10.001 · Cited by 1,003 · see citing works
Self-citationDeconstructing disengagement: analyzing learner subpopulations in massive open online courses
Kizilcec, R. F., Piech, C., & Schneider, E. (2013) · Proceedings of the Third International Conference on Learning Analytics and Knowledge · doi:10.1145/2460296.2460330 · Cited by 984 · see citing works
Self-citationChanging Mass Priorities: The Link between Modernization and Democracy
Inglehart, R., & Welzel, C. (2010) · Perspectives on Politics · doi:10.1017/s1537592710001258 · Cited by 750 · see citing works
Self-regulation of goal-setting: turning free fantasies about the future into binding goals
Oettingen, G., Pak, H.-J., & Schnetter, K. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.80.5.736 · Cited by 619 · see citing works
Future thought and behaviour change
Oettingen, G. (2012) · European Review of Social Psychology · doi:10.1080/10463283.2011.643698 · Cited by 504 · see citing works
Modernization and Postmodernization: Cultural, Economic and Political Change in 43 Societies
de Graaf, N. D., & Inglehart, R. (1998) · British Journal of Sociology · doi:10.2307/591300 · Cited by 410 · see citing works
Duckworth, A. L., Grant, H., Loew, B. A., Oettingen, G., & Gollwitzer, P. M. (2010) · Educational Psychology · doi:10.1080/01443410.2010.506003 · Cited by 353 · see citing works
Attrition and Achievement Gaps in Online Learning
Kizilcec, R. F., & Halawa, S. (2015) · Proceedings of the Second (2015) ACM Conference on Learning @ Scale · doi:10.1145/2724660.2724680 · Cited by 307 · see citing works
Self-citationMotivation as a Lens to Understand Online Learners
Kizilcec, R. F., & Schneider, E. (2015) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2699735 · Cited by 253 · see citing works
Self-citationAdriaanse, M. A., Oettingen, G., Gollwitzer, P. M., Hennes, E. P., de Ridder, D. T. D., & de Wit, J. B. F. (2010) · European Journal of Social Psychology · doi:10.1002/ejsp.730 · Cited by 247 · see citing works
Stadler, G., Oettingen, G., & Gollwitzer, P. M. (2010) · Health Psychology · doi:10.1037/a0018644 · Cited by 233 · see citing works
Splitting a Predictor at the Upper Quarter or Third and the Lower Quarter or Third
Gelman, A., & Park, D. K. (2009) · The American Statistician · doi:10.1198/tast.2009.0001 · Cited by 198 · see citing works
Stadler, G., Oettingen, G., & Gollwitzer, P. M. (2009) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.09.021 · Cited by 188 · see citing works
Goal Setting and Goal Striving
Oettingen, G., & Gollwitzer, P. M. (2001) · Wiley eBooks · doi:10.1002/9780470998519.ch15 · Cited by 156 · see citing works
Duckworth, A. L., Kirby, T. A., Gollwitzer, A., & Oettingen, G. (2013) · Social Psychological and Personality Science · doi:10.1177/1948550613476307 · Cited by 152 · see citing works
Waldrop, M. M. (2013) · Nature · doi:10.1038/495160a · Cited by 147 · see citing works
Christiansen, S., Oettingen, G., Dahme, B., & Klinger, R. (2010) · Pain · doi:10.1016/j.pain.2009.12.015 · Cited by 141 · see citing works
Self‐regulation of time management: Mental contrasting with implementation intentions
Oettingen, G., Kappes, H. B., Guttenberg, K. B., & Gollwitzer, P. M. (2015) · European Journal of Social Psychology · doi:10.1002/ejsp.2090 · Cited by 132 · see citing works
Committing to implementation intentions: Attention and memory effects for selected situational cues
Achtziger, A., Bayer, U. C., & Gollwitzer, P. M. (2011) · Motivation and Emotion · doi:10.1007/s11031-011-9261-6 · Cited by 100 · see citing works
The emergence of goal pursuit: Mental contrasting connects future and reality
Kappes, A., & Oettingen, G. (2014) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2014.03.014 · Cited by 100 · see citing works
Oettingen, G., Mayer, D., & Thorpe, J. S. (2009) · Psychology and Health · doi:10.1080/08870440903079448 · Cited by 87 · see citing works
Promoting integrative bargaining: mental contrasting with implementation intentions
Kirk, D., Oettingen, G., & Gollwitzer, P. M. (2013) · International Journal of Conflict Management · doi:10.1108/10444061311316771 · Cited by 73 · see citing works
Mental contrasting facilitates academic performance in school children
Gollwitzer, A., Oettingen, G., Kirby, T. A., Duckworth, A. L., & Mayer, D. (2011) · Motivation and Emotion · doi:10.1007/s11031-011-9222-0 · Cited by 71 · see citing works
Gawrilow, C., Morgenroth, K., Schultz, R., Oettingen, G., & Gollwitzer, P. M. (2012) · Motivation and Emotion · doi:10.1007/s11031-012-9288-3 · Cited by 60 · see citing works
Mental Contrasting and Transfer of Energization
Sevincer, A. T., Busatta, P. D., & Oettingen, G. (2013) · Personality and Social Psychology Bulletin · doi:10.1177/0146167213507088 · Cited by 39 · see citing works
Oettingen, G. (1997) · Culture & Psychology · doi:10.1177/1354067x9733008 · Cited by 35 · see citing works
Mental contrasting promotes integrative bargaining
Kirk, D., Oettingen, G., & Gollwitzer, P. M. (2011) · International Journal of Conflict Management · doi:10.1108/10444061111171341 · Cited by 35 · see citing works
Self-Regulation: Principles and Tools
Oettingen, G., & Gollwitzer, P. M. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781139565790.002 · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Reich J (2014) MOOC completion and retention in the context of student intent. Educause Review Online. Available at er.educause.edu/articles/2014/12/mooc- completion-and-retention-in-the-context-of-student-intent. Accessed June 1, 2016.
- Reich J (2015) Education research. Rebooting MOOC research. Science 347(6217): 34-35.
- Orbeil S, Hodgldns S, Sheeran P (1997) Implementation intentions and the theory of planned behavior. Pers Soc Psychol Bull 23(9):945-954.
- Hofstede G, Hofstede GJ, Minkov M (2010) Cultures and Organizations: Software of the Mind (McGraw-Hill, New York), 3rd Ed.
Cited by
Sorted by citations
Wise interventions: Psychological remedies for social and personal problems.
Walton, G. M., & Wilson, T. D. (2018) · Psychological Review · doi:10.1037/rev0000115 · Cited by 617 · see citing works
Guo, Y.-M., & Xiao, Y.-D. (2019) · International Conference on Artificial Intelligence in Education · doi:10.1007/978-3-030-23207-8 · Cited by 492 · see citing works
“Studies showed that interaction with peers [9] or intervention from the course [13] need to be designed differently by students’ cultural background or the learning style that they are accustomed to [12].”
Maldonado‐Mahauad, J., Pérez‐Sanagustín, M., Kizilcec, R. F., Morales, N., & Muñoz-Gama, J. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.11.011 · Cited by 301 · see citing works
Self-citationScaling up behavioral science interventions in online education
Kizilcec, R. F., Reich, J., Yeomans, M., Dann, C., Brunskill, E., Lopez, G., Turkay, S., Williams, J. J., & Tingley, D. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1921417117 · Cited by 208 · see citing works
“%) for students in individualist countries (such as the United States and Germany) (14).”
BackgroundSelf-citationDavis, D., Jivet, I., Kizilcec, R. F., Chen, G., Hauff, C., & Houben, G. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027411 · Cited by 136 · see citing works
Self-citationStudent success prediction in MOOCs
Gardner, J., & Brooks, C. (2018) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-018-9203-z · Cited by 210 · see citing works
“2017), interventions to improve student completion (Kizilcec and Cohen 2017), demographics and participation rates and the relationship to course activity (Guo andReinecke 2014), and student plagiarism and academic honesty issues (Alexandron et al.”
BackgroundActive learning with online video: The impact of learning context on engagement
Seo, K., Dodson, S., Harandi, N. M., Roberson, N. D., Fels, S., & Roll, I. (2021) · Computers & Education · doi:10.1016/j.compedu.2021.104132 · Cited by 159 · see citing works
Wong, J., Khalil, M., Baars, M., de Koning, B. B., & PAAS, F. G. W. C. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.103595 · Cited by 136 · see citing works
Facilitating goal setting and planning to enhance online self-regulation of learning
Wong, J., Baars, M., He, M., de Koning, B. B., & PAAS, F. G. W. C. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106913 · Cited by 119 · see citing works
“While MCII, to our knowledge, has not been examined in video-based learning environments, studies on MCII in other learning contexts suggest that MC together with II can support the goal setting process and enhance learning in environments that demand SRL (Kizilcec & Cohen, 2017).”
BackgroundAnalysis of the Factors Influencing Learners’ Performance Prediction With Learning Analytics
Moreno-Marcos, P. M., Pong, T.-C., Muñoz‐Merino, P. J., & Kloos, C. D. (2020) · IEEE Access · doi:10.1109/access.2019.2963503 · Cited by 91 · see citing works
“The reason is that academic failure in MOOCs is very high and completion rates hardly ever The associate editor coordinating the review of this manuscript and approving it for publication was Bilal Alatas . exceed 25% for highly committed learners [2].”
BackgroundWang, G., Wang, Y., & Gai, X. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.565202 · Cited by 62 · see citing works
“For example, Kizilcec and Cohen (2017) found that MCII could increase individual goal attainment (massive open online courses, abbreviated by MOOCs, completion rates) in individualist Frontiers in Psychology | www.frontiersin.org 2 May 2021 | Volume 12 | Article 565202 cultures but not collectivist…”
BackgroundMethodInfluentialPromoting the Self-Regulation of Stress in Health Care Providers: An Internet-Based Intervention
Gollwitzer, P. M., Mayer, D., Frick, C. D., & Oettingen, G. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00838 · Cited by 74 · see citing works
“Regarding MCII interventions taught online, Kizilcec and Cohen (2017) found in two studies that MCII delivered as an 8-min online intervention in the context of massive open online courses (MOOC) substantially increased completion rates of the online courses.”
BackgroundAdaptasi dan properti psikometrik skala kontrol diri ringkas versi Indonesia
Arifin, H. H., & Milla, M. N. (2020) · doi:10.7454/jps.2020.18 · Cited by 57
Li, X.-Y., Han, M., Cohen, G. L., & Markus, H. (2021) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2016964118 · Cited by 54 · see citing works
“In these cultural contexts, trying to leverage the power of passion—the interests, enjoyment, and efficacy—of the independent self without addressing its social reality in the relevant cultural context is not sufficient for improving achievement (76).”
BackgroundSelf-citationCross, A., & Sheffield, D. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1594332 · Cited by 97 · see citing works
“In a sample of 17, 983 online learners, Kizilceca and Cohen (2017) found that MCII significantly increased online course completion rates for learners from individualist cultures where the obstacle concerned an everyday obligation.”
BackgroundRoll, I., Russell, D. M., & Gašević, D. (2018) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-018-0170-7 · Cited by 49 · see citing works
“The large diversity of learners means that we know more about how learners with different attributes engage and learn, and thus are able to offer diversified educational experiences that match learners’ goals and background (Kizilcec and Cohen 2017; Roll and Winne 2015).”
BackgroundMindfulness meditation and psychedelics: potential synergies and commonalities
Holas, P., & Kamińska, J. (2023) · Pharmacological Reports · doi:10.1007/s43440-023-00551-8 · Cited by 43 · see citing works
The motivating effect of monetary over psychological incentives is stronger in WEIRD cultures
Medvedev, D., Davenport, D., Talhelm, T., & Li, Y. (2024) · Nature Human Behaviour · doi:10.1038/s41562-023-01769-5 · Cited by 44 · see citing works
Ali, A. D., & Hanna, W. K. (2021) · Journal of Educational Computing Research · doi:10.1177/07356331211056178 · Cited by 35 · see citing works
“LA can produce reports of data analyses that depict elements of SRL attributes in online environments (Kizilcec & Cohen, 2017; Roll & Winne, 2015).”
BackgroundCulture in Computer-Based Learning Systems: Challenges and Opportunities
Baker, R., Ogan, A. E., Madaio, M. M. M., & Walker, E. W. E. (2026) · Computer-Based Learning in Context · doi:10.35510/cblc.2019.1.1.1 · Cited by 29
“The relationship between course completion and the individualism of learners’ culture replicated across two studies, even after evaluating potential confounding factors (Kizilcec and Cohen, 2017).”
BackgroundViberg, O., Kizilcec, R. F., Jivet, I., Martínez-Monés, A., Oh, A., Mutimukwe, C., Hrastinski, S., & Scheffel, M. (2023) · Computers in Human Behavior Reports · doi:10.48550/arxiv.2312.02093 · Cited by 29
“Researchers have studied cultural differences in the use and impact of learning analytics applications (e.g., Cho et al., 2021; Davis et al., 2017; Kizilcec & Cohen, 2017; Mittelmeier et al., 2016), but not the influence of culture on students’ privacy concerns.”
BackgroundSelf-citationEmbedding Experiments: Staking Causal Inference in Authentic Educational Contexts
Motz, B. A., Carvalho, P. F., de Leeuw, J. R., & Goldstone, R. L. (2018) · Journal of Learning Analytics · doi:10.18608/jla.2018.52.4 · Cited by 31 · see citing works
“The “mileage” of any intervention may vary between different situations, and controlled experiments can help prevent overgeneralization by providing clear estimates of a causal effect within a specific context (e.g., Kizilcec & Cohen, 2017).”
BackgroundEnculturating the Science of International Development: Beyond the WEIRD Independent Paradigm
Thomas, C., & Markus, H. R. (2023) · Journal of Cross-Cultural Psychology · doi:10.1177/00220221221128211 · Cited by 28 · see citing works
“Related research in the United States showed that framing donations to charity as a communal, collaborative action (“Let’s save a life together,” “Join your community”) motivated more interdependent, lower income people to donate more compared with when it was framed as an independent, individual action (“You=life-sav…”
BackgroundResultInfluentialYilmaz-Na, E., & Sönmez, E. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104683 · Cited by 28 · see citing works
Designing Culturally Aware Learning Analytics: A Value Sensitive Perspective
Viberg, O., Jivet, I., & Scheffel, M. (2022) · arXiv.org · doi:10.48550/arxiv.2212.09645 · Cited by 24
“When analysing the effectiveness of interventions in the context of massive open online courses (MOOCs), Kizilcec & Cohen (2017) showed that a writing activity that facilitates goal-commitment and goal-directed behaviour raised educational attainment at scale in individualist (US and Germany) but…”
BackgroundPreregistration and registered reports
Reich, J. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1900851 · Cited by 30 · see citing works
“…planning prompt (Yeomans & Reich, 2017), a value relevance affirmation (Kizilcec et al., 2017), a mental contrasting with implementation intentions (Kizilcec & Cohen, 2017), and a social accountability intervention—had shown moderate to large positive effects on persistence and completion in a…”
BackgroundHow Learner Support Services Affect Student Engagement in Online Learning Environments
He, H., Zheng, Q., Di, D., & Dong, B. (2019) · IEEE Access · doi:10.1109/access.2019.2910589 · Cited by 23 · see citing works
Bootstrap Thompson Sampling and Sequential Decision Problems in the Behavioral Sciences
Eckles, D., & Kaptein, M. C. (2019) · SAGE Open · doi:10.1177/2158244019851675 · Cited by 24 · see citing works
“For example, motivational interventions might work differently for students from collectivist versus individualist cultures (Kizilcec & Cohen, 2017).”
BackgroundKim, J. Y., Brockner, J., & Block, C. J. (2022) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2022.104118 · Cited by 23 · see citing works
Are MOOC learning designs culturally inclusive (enough)?
Rizvi, S., Rienties, B., Rogaten, J., & Kizilcec, R. F. (2023) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12883 · Cited by 16 · see citing works
“These frameworks are often used to understand cross-cultural differences in learners' behaviours (see, e.g., Baker et al., 2020; Joy & Kolb, 2009; Kizilcec & Cohen, 2017; Mensah & Chen, 2013).”
Self-citationRiddell, H., Sedikides, C., Gucciardi, D. F., Jackson, B., Thøgersen‐Ntoumani, C., & Ntoumanis, N. (2022) · Journal of Applied Social Psychology · doi:10.1111/jasp.12915 · Cited by 23 · see citing works
“For example, its utility for individuals living in collectivist societies may be reduced, as individuals in these cultures are more likely to perceive implementation intentions as an over-simplification of potentially uncontrollable obligations (Kizilcec & Cohen, 2017).”
The Promises and Pitfalls of Self-regulated Learning Interventions in MOOCs
Вилкова, К. (2021) · Technology Knowledge and Learning · doi:10.1007/s10758-021-09580-9 · Cited by 23 · see citing works
“…SRL interventions showed gains in learners’ achievement (Jansen et al., 2020; Wong et al., 2021; Yeomans & Reich, 2017), some raised educational attainment for a particular group of learners (Kizilcec & Cohen, 2017), and others demonstrated no effect (Davis et al., 2016; Kizilcec et al., 2016).”
BackgroundMethodInfluentialUsing machine learning to predict UK and Japanese secondary students' life satisfaction in PISA 2018
Pan, Z., & Cutumisu, M. (2023) · British Journal of Educational Psychology · doi:10.1111/bjep.12657 · Cited by 22 · see citing works
Effects of automated interventions in programming assignments
Teusner, R., Hille, T., & Staubitz, T. (2018) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3231644.3231650 · Cited by 23 · see citing works
“Pre-emptive interventions, such as plan making interventions [31], affirma-tion interventions [13] and self-regulation interventions [12] are suited to prevent undesired behavior before negative consequences come to effect.”
BackgroundPandemic response policies’ democratizing effects on online learning
Kizilcec, R. F., Makridis, C. A., & Sadowski, K. C. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2026725118 · Cited by 22 · see citing works
“Behavioral science interventions that facilitate strategic plan making and time management could therefore help workers achieve their professional learning goals (19, 20).”
BackgroundSelf-citationGeneralizing Predictive Models of Admission Test Success Based on Online Interactions
Moreno-Marcos, P. M., Laet, T. D., Muñoz‐Merino, P. J., Soom, C. V., Broos, T., Verbert, K., & Kloos, C. D. (2019) · Sustainability · doi:10.3390/su11184940 · Cited by 20 · see citing works
Davis, D., Kizilcec, R. F., Hauff, C., & Houben, G. (2018) · International Conference on Learning Analytics and Knowledge · doi:10.1145/3170358.3170383 · Cited by 18 · see citing works
“Given the wealth of scientific evidence on the benefits of retrieval practice (cf. Section 2) and the adaptability of digital learning platforms, in this paper we explore to what extent the testing effect holds in one of today’s most popular digital learning settings: Massive Open Online Courses (MOOCs).”
MethodInfluentialSelf-citationCultural Defaults in the Time of COVID: Lessons for the Future
Markus, H. R., Tsai, J. L., Uchida, Y., Yang, A. M., & Maitreyi, A. (2024) · Psychological Science in the Public Interest · doi:10.1177/15291006241277810 · Cited by 16 · see citing works
Designing Inclusive Learning Environments
Brooks, C. A., Kizilcec, R. F., & Dowell, N. (2020) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3386527.3405935 · Cited by 19 · see citing works
“The effectiveness of interventions may vary with cultural differences in the learner population [11].”
BackgroundSelf-citationTaking Responsibility for Others and Use of Mental Contrasting
Sevincer, A. T., Musik, T., Degener, A., Greinert, A., & Oettingen, G. (2020) · Personality and Social Psychology Bulletin · doi:10.1177/0146167219898569 · Cited by 16 · see citing works
“…may be particularly effective for pursuing individualist wishes in individualist cultures and collectivist wishes in collectivist cultures (Kizilcec & Cohen, 2017; Oettingen et al., 2008), future research may look at the degree to which people from individualist versus collectivist…”
BackgroundRe-Defining, Analyzing and Predicting Persistence Using Student Events in Online Learning
Moreno-Marcos, P. M., Muñoz‐Merino, P. J., Alario‐Hoyos, C., & Kloos, C. D. (2020) · Applied Sciences · doi:10.3390/app10051722 · Cited by 16 · see citing works
“Dropout is an important issue because its rates can sometimes be very high, both at course and academic (whole degree) level [20].”
BackgroundViberg, O., & Grönlund, Å. (2021) · Online Learning Analytics · doi:10.1201/9781003194620-2 · Cited by 14 · see citing works
“…(e.g., in terms of power distance) that may influence the adoption and the effectiveness of LA interventions (see e.g., Davis et al., 2017; Kizilcec & Cohen, 2017). perspective, in today’s society such goals should focus not only on the improvement of students’ subject knowledge, but also…”
BackgroundA Learning Engineering Model for Learner-Centered Adaptive Systems
Goodell, J., & Thai, K.-P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60128-7_41 · Cited by 11 · see citing works
Martenstyn, J. A., & Grant, A. M. (2021) · Coaching An International Journal of Theory Research and Practice · doi:10.1080/17521882.2021.1890166 · Cited by 10 · see citing works
“Several studies also included three ‘if–then plans’ in their MCII experimental procedure (Duckworth et al., 2011; Kizilcec & Cohen, 2017; Marquardt et al., 2017; Oettingen et al., 2015; Stadler et al., 2009, 2010).”
BackgroundMethodAhmed, A. E., Awadallah, A. A., Tagelsir, M., Suliman, M. A., Eltigani, A., Elsafi, H., Hamdelnile, B. D., Mukhtar, M. A., & Fadlelmola, F. M. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz004 · Cited by 14 · see citing works
“Also, for learners from less-developed countries, success stories from previous course alumni have been shown to improve performance [29] • Career progress and capacity building: The IBT aims to equip African researchers with the skills and knowledge to launch their careers and establish their science.”
BackgroundMethodWu, Y., Zhao, B., Wei, B., & Li, Y. (2022) · Personality and Individual Differences · doi:10.1016/j.paid.2021.111497 · Cited by 9 · see citing works
Not Only the Intention to Complete: The Role of Action-Oriented Intentions in MOOC Completion
Semenova, T. (2021) · Technology Knowledge and Learning · doi:10.1007/s10758-021-09534-1 · Cited by 13 · see citing works
Mcreynolds, A. A., Naderzad, S. P., Goswami, M., & Mostow, J. (2020) · ACM Conference on Learning @ Scale · doi:10.1145/3386527.3405920 · Cited by 10
“Some experiments have studied MOOCs in non-WEIRD settings [4, 6].”
BackgroundEl Proceso de Implementación de Analíticas de Aprendizaje
Ruipérez‐Valiente, J. A. (2020) · RIED Revista Iberoamericana de Educación a Distancia · doi:10.5944/ried.23.2.26283 · Cited by 15 · see citing works
“Como último ejemplo, el análisis de causas, donde frecuentemente se realiza experimentación y comparan los grupos de control y tratamiento (Kizilcec y Cohen, 2017), y en algunas ocasiones se aplican algoritmos específicos para realizar un análisis causal (Koedinger, Kim, Jia, McLaughlin y Bier,…”
Integrating Students’ Behavioral Signals and Academic Profiles in Early Warning System
Nam, S., & Samson, P. J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-23204-7_29 · Cited by 15 · see citing works
The effectiveness of the flipped classroom: A second-order meta-analysis
Ma, Y., Fan, H., & Ning, B. (2025) · International Journal of Educational Research · doi:10.1016/j.ijer.2025.102855 · Cited by 8 · see citing works
Leveraging cultural forms in human‐centred learning analytics design
Campos, F., Nguyen, H., Ahn, J., & Jackson, K. J. (2023) · British Journal of Educational Technology · doi:10.1111/bjet.13384 · Cited by 13 · see citing works
Strategies for Creating Inclusive Learning Environments through a Social Justice Lens
Dawson, B. A., Kilgore, W., & Rawcliffe, R. M. (2022) · Journal of Educational Research and Practice · doi:10.5590/jerap.2022.12.0.02 · Cited by 23 · see citing works
“Kizilcec and Cohen (2017) described a writing exercise designed to increase motivation and reduce attrition; it was 78% successful in learners who come from individualist cultures but ineffective for those from collectivist cultures.”
Pérez‐Sanagustín, M., Álvarez, R. A. P., Maldonado‐Mahauad, J., Villalobos, E., Hilliger, I., Hernández, J., Sapunar‐Opazo, D., Moreno-Marcos, P. M., Muñoz‐Merino, P. J., Kloos, C. D., et al. (2021) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3430895.3460991 · Cited by 10 · see citing works
Cho, J. Y., & Kizilcec, R. F. (2021) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3430895.3460138 · Cited by 8 · see citing works
“In the first field experiment testing MCII at large scale [24], learners in the control condition, which typically provides participants with an innocuous task to serve as the baseline in psychology experiments, wrote down their expectations for the course and their plans for taking the course.”
BackgroundSelf-citationThe Effect of Self-Regulation in a MOOC Environment on the Completion and Performance of Learners
Γιασιράνης, Σ., & Sofos, L. (2020) · Open Journal of Social Sciences · doi:10.4236/jss.2020.810010 · Cited by 5 · see citing works
Scaling Effective Learning Strategies: Retrieval Practice and Long-Term Knowledge Retention in MOOCs
Davis, D., Kizilcec, R. F., Hauff, C., & Houben, G. (2018) · Journal of Learning Analytics · doi:10.18608/jla.2018.53.3 · Cited by 8 · see citing works
“…case with most MOOC studies in the past (Rosen et al., 2017; van der Zee et al., 2018; Davis, Chen, van der Zee et al., 2016; Davis et al., 2017; Kizilcec & Cohen, 2017; Kizilcec, Saltarelli et al., 2017; Yeomans & Reich, 2017; Gamage et al., 2017), we found noncompliance to be a major…”
BackgroundSelf-citationCho, J., Li, Y., Kudryavtsev, A., Krasny, M. E., & Kizilcec, R. F. (2022) · Computer-Based Learning in Context · doi:10.31234/osf.io/v83cu · Cited by 6 · see citing works
“The cultural diversity in the online learning environment offers a rare opportunity for advancing our understanding of how people in different contexts respond to norm messages and similar behavioral science interventions (Kizilcec & Cohen, 2017; Kizilcec et al., 2020).”
BackgroundResultSelf-citationOn the limits of algorithmic prediction across the globe
Li, X.-Y., Song, D.-F., Han, M., Zhang, Y., & Kizilcec, R. F. (2021) · arXiv.org · doi:10.48550/arxiv.2103.15212 · Cited by 7
Self-citationWho Will Continue Using MOOCs in the Future? Personality Traits Perspective
Abdullatif, H. A. A., & Velázquez‐Iturbide, J. Á. (2020) · IEEE Access · doi:10.1109/access.2020.2979180 · Cited by 7 · see citing works
“Kizilcec and Cohen [19] and Kizlicec and Halawa [18] underline the role of culture in course completion, educational attainment, and satisfaction.”
BackgroundAbdulla, A., & Woods, R. (2021) · School Psychology International · doi:10.1177/01430343211000399 · Cited by 5 · see citing works
“MCII can be performed by students in just eight minutes (Kizilcec & Cohen, 2017).”
BackgroundResultInfluentialWhen to grit and when to quit: (How) should grit be taught in K-12 classrooms?
Audley, S., & Donaldson, M. (2022) · Theory Into Practice · doi:10.1080/00405841.2022.2096377 · Cited by 8 · see citing works
“Mental contrasting with implementation intentions (MCII) MCII is a self-regulation strategy that can enhance student academic commitment and goal attainment in under 10 minutes (Kizilcec & Cohen, 2017) and has been shown to modestly improve academic achievement (e.g., Duckworth et al., 2013).”
Davis, D. J. (2019) · doi:10.4233/uuid:b8be8302-84a0-4b29-a6fe-761a3f872420 · Cited by 5
“In the series of research efforts presented here, we tackle this problem by first referencing the research literature to discover what types of interventions or improvements can be made to large-scale online learning environments to address the identified issues; conducting exploratory studies on the natural behavior…”
BackgroundMethodInfluentialKemp, A. M., & O’Brien, K. H. (2025) · American Journal of Speech-Language Pathology · doi:10.1044/2024_ajslp-24-00100 · Cited by 5 · see citing works
A Multi-Method Approach to the Future of the Scholarship of Teaching and Learning
Glazier, R. A. (2024) · Journal of Political Science Education · doi:10.1080/15512169.2024.2343715 · Cited by 4 · see citing works
“For instance, teaching interventions might work differently among students in more collectivist vs. more individualistic societies (Joy and Kolb 2009; Kizilcec and Cohen 2017).”
Which Planning Tactics Predict Online Course Completion?
Cho, J. Y., Tao, Y., Yeomans, M., Tingley, D. H., & Kizilcec, R. F. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636891 · Cited by 2 · see citing works
“The same year another type of planning intervention tested in two large MOOCs was also found to have significant benefits for course completion, but only in countries with an individualistic culture such as the United States [32].”
BackgroundMethodSelf-citationExploring Cross-Country Prediction Model Generalizability in MOOCs
Andres-Bray, J., Hutt, S., & Baker, R. (2023) · ACM Conference on Learning @ Scale · doi:10.1145/3573051.3593380 · Cited by 4 · see citing works
“It may be possible to select factors for consideration based on studies that investigate the effectiveness of educational findings across different groups of students, such as socio-economic status [9], national wealth [24], or whether the student comes from a collectivist or individualist cultural background [23].”
BackgroundThe Impact of Culture on Learner Behavior in Visual Debuggers
Thayer, K., Guo, P. J., & Reinecke, K. (2018) · IEEE Symposium on Visual Languages / Human-Centric Computing Languages and Environments · doi:10.1109/vlhcc.2018.8506556 · Cited by 4 · see citing works
“One of the reasons for these concerns is that a country’s national culture has been found to influence how people learn [7]–[9].”
BackgroundParticipation of the Arab World in MOOCs
Ruipérez-Valiente, J. A., & Reich, J. (2018) · Learning With MOOCS · doi:10.1109/lwmoocs.2018.8534650 · Cited by 4
“For instance, educators have experimented with using MOOCs to equip Tanzanian students with marketrelevant IT skills [7], and researchers have investigated how different instructional messages are received differently in countries with different cultural traditions [8].”
BackgroundA statistical test for the benefits of personalizing interventions
Li, Z., & Brunskill, E. (2026) · Science · doi:10.1126/science.aeb9506
Tao, Y., Maidi, N., Yu, R., & Kizilcec, R. F. (2025) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3698205.3729552 · Cited by 2 · see citing works
“Instructors of these courses could incorporate instructional design strategies proven to foster self-regulation, such as prompting students to reflect on their learning approaches, adding activities on planning and time management [14], and providing exemplars of effective study strategies [20].”
Self-citationJiang, Y., & Yang, F. (2024) · Work · doi:10.3233/wor-240041 · Cited by 5 · see citing works
Rüschenpöhler, L., Gong, X.-Y., Wang, C., & Markic, S. (2024) · International Journal of Science Education · doi:10.1080/09500693.2024.2365461 · Cited by 3
“For instance, it has been shown that some educational interventions are effective in individualist cultures but ineffective in collectivist cultures (Kizilcec & Cohen, 2017).”
Walck-Shannon, E. M., Rowell, S. F., Bednarski, A. E., Barber, A. M., Yuan, G., & Frey, R. F. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.23-05-0092 · Cited by 5 · see citing works
“…on health behaviors, but recent work has applied MC-II to various educational settings, including K–12 (Duckworth et al. , 2011, 2013), vocational schools (Oettingen et al. , 2015), massive open-enrollment courses (Kizilcec and Cohen, 2017), and medical schools (Saddawi-Konefka et al. , 2017).”
Analyzing Students’ Behavior in a MOOC Course: A Process-Oriented Approach
Bernal, F., Maldonado‐Mahauad, J., Villalba‐Condori, K. O., Zúñiga‐Prieto, M., Veintimilla-Reyes, J., & Mejía-Pesántez, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60128-7_24 · Cited by 6 · see citing works
Watted, A., & Ahmad, M. Y. A. (2026) · Multicultural Education Review · doi:10.1080/2005615x.2026.2618918 · Cited by 1 · see citing works
“However, limited learner-instructor and peer interactions in some MOOCs can hinder knowledge construction (Kizilcec & Cohen, 2017).”
How culturally wise psychological interventions can help reduce poverty
Thomas, C., Premand, P., Bossuroy, T., Sambo, S. A., Markus, H., & Walton, G. M. (2025) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2505694122 · Cited by 2 · see citing works
“One large - scale cross - cultural experiment found this type of intervention boosted academic achievement in the United States, consistent with past studies, yet not in India or China ( 32 ).”
Mental Contrasting Strategies Promote the Pursuit of Difficult Goals: Japanese Cultural Context
Toyama, M., Nagamine, M., & Tang, L. (2024) · Personality and Social Psychology Bulletin · doi:10.1177/01461672241247481 · Cited by 2 · see citing works
“Although Kizilcec and Cohen (2017) explored MCII’s effectiveness in collectivist cultures, their study lacked a comprehensive analysis of goal attainability.”
Measuring Cultural Dimensions of Learning in Online Courses
Ji, Y.-C., Li, Y., Krasny, M., & Kizilcec, R. F. (2022) · ACM Conference on Learning @ Scale · doi:10.1145/3491140.3528333 · Cited by 2 · see citing works
“Several studies found behavioral heterogeneity in MOOCs between learners from different nations and cultures, including differences in in-course navigation strategies [4], in the distribution of learning activities that are used [13], and in selfregulated learning strategies [7, 26].”
BackgroundSelf-citationUsing Social Norms to Promote Actions Beyond the Course
Cho, J. Y., Li, Y., Armstrong, A. K., Russ, A., Krasny, M. E., & Kizilcec, R. F. (2021) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3430895.3460144 · Cited by 2 · see citing works
“Finally, completion rates were not significantly different between conditions, which is not too surprising given that the intervention targeted actions outside of the course rather than course completion, which has been the target of prior successful interventions [27, 19, 49].”
BackgroundSelf-citationAhmed, A. E., Awadallah, A. A., Elmahdi, M. T., Suliman, M. A., Khalil, A., Elsafi, H., Hamdelnile, B. D., Abdullateif, M., & Fadlelmola, F. M. (2018) · bioRxiv · doi:10.1101/431361 · Cited by 2
“Fromoneperspective, testimonialsfrom successful alumni have been shown to improve the sense of belonging in MOOCs, and there by learners’ performance [Kizilcec et al. , 2017].”
BackgroundInfluentialDas, S., Delavallade, C., Fashogbon, A., & Papineni, S. (2026) · Journal of Development Economics · doi:10.1596/1813-9450-11325 · Cited by 1
Planning promotes studying – The higher the plan quality the better: A micro-randomized trial
Schaaf, M., Brod, G., & Breitwieser, J. (2025) · Contemporary Educational Psychology · doi:10.17605/osf.io/bhmjx · Cited by 1
“…should boost motivation, for instance by highlighting the personal relevance and long-term benefits of studying, including mental contrasting to raise commitment (Gawrilow et al., 2013; Kizilcec & Cohen, 2017), and making practice sessions more enjoyable and effective for quicker rewards.”
Supporting student agency through student-centred learning across educational contexts
Lee, E., Baird, T. D., & Lee, D. (2025) · Innovations in Education and Teaching International · doi:10.1080/14703297.2025.2555961 · Cited by 3 · see citing works
“This aligns with previous findings that self-regulation interventions are more effective in individualistic cultures than in collectivist ones (Kizilcec & Cohen, 2017).”
Kapuza, A., & Andriasian, L. (2025) · Open Learning The Journal of Open Distance and e-Learning · doi:10.1080/02680513.2025.2545925 · Cited by 1 · see citing works
“Over the past few years, researchers have been discussing the theory of SRL and its implications for those studying in MOOCs, and conducting qualitative and quantitative research (Araka et al., 2020; Kizilcec & Cohen, 2017; Kizilcec et al., 2020; Vilkova, 2022).”
Khamidulina, M. S., & Maloshonok, N. (2025) · Высшее образование в России · doi:10.31992/0869-3617-2025-34-1-106-127 · Cited by 1 · see citing works
Straand, I. J., Følstad, A., & Wuensche, B. C. (2024) · JMIR Formative Research · doi:10.2196/59158 · Cited by 3 · see citing works
Wang, M., Kizilcec, R. F., & Baiocchi, M. (2025) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986251342292
“Recent work has highlighted that peer context should be taken into consideration as a driver of treatment effect heterogeneity for behavioral and psychological interventions (Yeager et al., 2019; Wal-ton and Yeager, 2020; Kizilcec et al., 2020; Kizilcec and Cohen, 2017).”
Self-citationWiita, F., Ho, A. K., & Weinstein, N. (2024) · JMIR Formative Research · doi:10.2196/52883 · Cited by 1 · see citing works
“Exclusions were applied where participants spent <8 minutes or had not completed a writing condition task; a previous online writing intervention lasting approximately 8 minutes was found to elicit improved outcomes [34].”
Introducing Practicable Learning Analytics
Viberg, O., & Grönlund, Å. (2023) · arXiv.org · doi:10.48550/arxiv.2301.13043 · Cited by 1
Effects of Framing Professional Development as a Career Growth Opportunity on Course Completion
Kizilcec, R. F., Mimno, J. A., & Karhan, A. J. (2022) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3491140.3528304 · Cited by 1 · see citing works
“This kind of self-regulatory intervention has shown promise in promoting online course completion [8, 10, 14].”
BackgroundSelf-citationAssessing Arabic Teachers’ Assessment Methods in Evaluating Students’ Literacy
Riwanda, A., Nor, S., Ridha, M., & Islamy, M. I. (2022) · Advances in Social Science, Education and Humanities Research/Advances in social science, education and humanities research · doi:10.2991/assehr.k.220104.030 · Cited by 2 · see citing works
Does mental contrasting with implementation intentions (MCII) help to increase physical activity?
Kędra, M., & Łaguna, M. (2026) · Current Issues in Personality Psychology · doi:10.5114/cipp/225999
“On the other hand, participants of the online intervention may generate inadequate outcomes, formidable obstacles, or ineffective plans for overcoming these obstacles (Kizilcec & Cohen, 2017), which might decrease the MCII effects.”
The role of AI for self-regulated learning: a systematic literature review
Jadhav, V. S., & Singh, S. (2026) · Journal of Computers in Education · doi:10.1007/s40692-026-00406-0
Qiu, C.-L., Cui, K., Wang, Z., & Hou, L. (2026) · Big Data · doi:10.1177/2167647x261463927
McNamara, D. S., Banawan, M., Balyan, R., Roscoe, R. D., & Arner, T. (2026) · Chinese/English Journal of Educational Measurement and Evaluation · doi:10.59863/zvlt7884
Lascoña, E., & Salazar, D. A. (2026) · Indonesian Journal of Education Research (IJoER) · doi:10.37251/ijoer.v7i2.2402
Messaging teachers to boost student EdTech use
Araya, R., Cristia, J., Escalante, L., Fabregas, R., Méndez, C., & Ríos, G. (2025) · Economics of Education Review · doi:10.1016/j.econedurev.2025.102706
BOOSTING LANGUAGE LEARNING WITH MCII STRATEGY: A QUASI-EXPERIMENTAL STUDY ON SECONDARY STUDENTS
Caner, S. (2024) · Journal of Nusantara Studies (JONUS) · doi:10.24200/jonus.vol9iss2pp218-240
Empowering Teacher in the Era of Information Technology through Self-Regulation Strategy
Irawati, M. D., & Heng, P. H. (2024) · Journal of Communication in Scientific Inquiry (JCSI) · doi:10.58915/jcsi.v5i1.1086
“The MCII intervention promotes goal attainment by combining two complementary self-regulation techniques (Kizilcec et al., 2017), namely: (a) mental contrast (MC), this technique directs goal setting and how our commitment is formed within it.”
Method
- Authors
- Kizilcec, R. F., & Cohen, G. L.
- Affiliations
- Stanford University (US)
- Corresponding author
- Kizilcec, René F.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2017-04-10
- Volume
- 114
- Issue
- 17
- Pages
- 4348-4353
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC5410783
- OpenAlex
- W2605718259
- Cited by
- 113
- References
- 0
- Citations by year
- 2026: 10 · 2025: 12 · 2024: 13 · 2023: 11 · 2022: 12 · 2021: 20 · 2020: 14 · 2019: 7 · 2018: 12 · 2017: 2
- FWCI
- 19.38
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Education, Achievement, and Giftedness
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- self-regulation, educational attainment, individualism-collectivism, cross-cultural comparison, course completion
- MeSH terms
- Academic Success, Adolescent, Adult, Attention, Culture, Humans, Individuality, Learning, Middle Aged, Students, United States, Universities, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
