Enhancing argumentative writing with automated feedback and social comparison nudging
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
The advantages offered by natural language processing (NLP) and machine learning enable students to receive automated feedback on their argumentation skills, independent of educator, time, and location. Although there is a growing amount of literature on formative argumentation feedback, empirical evidence on the effects of adaptive feedback mechanisms and novel NLP approaches to enhance argumentative writing remains scarce. To help fill this gap, the aim of the present study is to investigate whether automated feedback and social comparison nudging enable students to internalize and improve logical argumentation writing abilities in an undergraduate business course. We conducted a mixed-methods study to investigate the impact of argumentative writing on 71 students in a field experiment. Students in treatment group 1 completed their assignment while receiving automated feedback, whereas students in treatment group 2 completed the same assignment while receiving automated feedback with a social comparison nudge that indicated how other students performed on the same assignment. Students in the control group received generalized feedback based on rules of syntax. We found that participants who received automated argumentation feedback with a social comparison nudge wrote more convincing texts with higher-quality argumentation compared to the two benchmark groups (p < 0.05). The measured self-efficacy, perceived ease of use, and qualitative data provide valuable insights that help explain this effect. The results suggest that embedding automated feedback in combination with social comparison nudges enables students to increase their argumentative writing skills by triggering psychological processes. Receiving only automated feedback in the form of in-text argumentative highlighting without any further guidance appears not to significantly influence students’ writing abilities when compared to syntactic feedback.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Technology Acceptance Model 3 and a Research Agenda on Interventions
Venkatesh, V., & Bala, H. (2008) · Decision Sciences · doi:10.1111/j.1540-5915.2008.00192.x · Cited by 8,159
Social cognitive theory of self-regulation
Bandura, A. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90022-l · Cited by 6,875 · cited 7×
“We believe that self-regulated learning theory supports our hypothesis that automated argumentation feedback increases students ’ self-efficacy and thus their learning of argumentation skills (Bandura, 1991; Zimmerman & Schunk, 2001).”
BackgroundMethodResultInfluentialSocial Influence: Compliance and Conformity
Cialdini, R. B., & Goldstein, N. J. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142015 · Cited by 6,077 · cited 4×
“The concept of social comparison (Cialdini & Goldstein, 2004), also called social nudging, is especially promising when combining learning feedback and goal setting (e.g., by providing a feedback message such as “ other users have used more arguments in the same task ” ).”
BackgroundInfluentialAgarwal, R., & Karahanna, E. (2000) · MIS Quarterly · doi:10.2307/3250951 · Cited by 4,726
Formative assessment and the design of instructional systems
Sadler, D. R. (1989) · Instructional Science · doi:10.1007/bf00117714 · Cited by 4,285
Self-Regulated Learning: Beliefs, Techniques, and Illusions
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013) · Annual Review of Psychology · doi:10.1146/annurev-psych-113011-143823 · Cited by 1,468 · cited 2×
“…user studies (Afrin et al., 2021; Wambsganss et al., 2020), a review of the extant literature related to the provision of feedback related to failure (e.g., Bjork et al. (2013); Metcalfe (2017); Lawrence and Reed (2019)), and self-regulated learning theory (Bandura, 1991; Zimmerman & Schunk, 2001).”
BackgroundProceedings of the 2021 CHI Conference on Human Factors in Computing Systems
(2021) · doi:10.1145/3411764 · Cited by 1,240
Toward a script theory of guidance in computer-supported collaborative learning
Fischer, F., Kollar, I., Stegmann, K., & Wecker, C. (2013) · Educational Psychologist · doi:10.1080/00461520.2012.748005 · Cited by 525
Damgaard, M. T., & Nielsen, H. S. (2018) · Economics of Education Review · doi:10.1016/j.econedurev.2018.03.008 · Cited by 321
Zhu, M., Liu, O. L., & Lee, H.-S. (2020) · Computers & Education · doi:10.1016/j.compedu.2019.103668 · Cited by 229 · cited 7×
“Hence, this study expands prior research around formative feedback and digital nudging that used mainly exploratory or non-automated approaches (e.g., Zhu et al. (2020); Afrin et al. (2021); Wambsganss et al. (2020); Lin et al. (2020)).”
BackgroundResultInfluentialStegmann, K., Wecker, C., Weinberger, A., & Fischer, F. (2012) · Instructional Science · doi:10.1007/s11251-011-9174-5 · Cited by 149
Noroozi, O., Banihashem, S. K., Taghizadeh Kerman, N., Parvaneh Akhteh Khaneh, M., Babaee, M., Ashrafi, H., & Biemans, H. J. A. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2022.2034887 · Cited by 105
AL: An Adaptive Learning Support System for Argumentation Skills
Wambsganss, T., Niklaus, C., Cetto, M., Söllner, M., Handschuh, S., & Leimeister, J. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376732 · Cited by 101 · cited 10×
“In designing the tool, we followed the findings of recent user studies (Afrin et al., 2021; Wambsganss et al., 2020), a review of the extant literature related to the provision of feedback related to failure (e.g., Bjork et al. (2013); Metcalfe (2017); Lawrence and Reed (2019)), and self-regulated…”
BackgroundMethodInfluentialSelf-citationAutomated Educative Nudges to Reduce Missed Assignments in College
Motz, B. A., Mallon, M. G., & Quick, J. D. (2021) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2021.3064613 · Cited by 40
Adaptive Empathy Learning Support in Peer Review Scenarios
Wambsganss, T., Soellner, M., Koedinger, K. R., & Leimeister, J. M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517740 · Cited by 24 · cited 2×
“We applied the annotation scheme for argumentative knowledge construction described by Weinberger and Fischer (2006). This annotation scheme has been applied in several studies and has demonstrated high objectivity, reliability, and validity (e.”
MethodSelf-citation
Cited by
Impact of AI assistance on student agency
Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D., & Siemens, G. (2024) · Computers & Education · doi:10.1016/j.compedu.2023.104967 · Cited by 548
“For instance, nudging systems can remind students of upcoming deadlines (Damgaard & Nielsen, 2018) and AI-powered tools can provide automated personalised feedback on student work or suggest improvements in their writing (Wambsganss, Janson, & Leimeister, 2022).”
BackgroundGuo, K., & Wang, D. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12146-0 · Cited by 449
“Thanks to the advancement of artificial intelligence (AI) and natural language processing (NLP), machines’ capabilities of understanding, analysing, and evaluating human written texts have greatly improved (Wambsganss et al., 2022; Zhu et al., 2020).”
A systematic review of AI-based automated written feedback research
Shi, H., & Aryadoust, V. (2024) · ReCALL · doi:10.1017/s0958344023000265 · Cited by 222
Harnessing Generative AI (GenAI) for Automated Feedback in Higher Education: A Systematic Review
Lee, S. S., & Moore, R. L. (2024) · Online Learning · doi:10.24059/olj.v28i3.4593 · Cited by 181
Yang, Y., Luo, J., Yang, M., Yang, R., & Chen, J. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2327003 · Cited by 129
“The thoughtful and efficient application of GenAI by resourceful learners is consistent with research that highlights the significance of adjusting to new technologies in order to achieve improved performance (Wambsganss, Janson, and Leimeister 2022).”
BackgroundResult
Metadata
- Authors
- Wambsganss, T., Janson, A., & Leimeister, J. M.
- Venue
- Computers & Education
- Publisher
- Elsevier BV
- Published
- 2022-12
- Volume
- 191
- Pages
- 104644
- ISSN
- 0360-1315 (ISSN-L)
- OpenAlex
- W4300716842
- Cited by
- 71
- References
- 19
- FWCI
- 27.52
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.