AL: An Adaptive Learning Support System for Argumentation Skills

“These action steps provide the user with orientation and context to improve their writing quality [54, 27].”

“To develop skills such as argumentation, it is of great importance for the individual student to receive continuous feedback throughout their learning journey, also called formative feedback [5, 27].”

“For example, to assist critical writing, designers can add features such as the persuasiveness of the arguments (Wambsganss et al., 2020) and recommended similar arguments from published papers (Peng et al., 2020; Hui et al., 2018).”

“This has been investigated across a variety of felds, including law [49], science [45, 64], conversational argumentation [14, 78] and business reviews [73].”

“The relevant aspects depend on the task at hand: argumentation structures (eg, claims, premises and their connections; Rapanta & Walton, 2016; Wambsganss et al., 2020) are a key requirement for argumentative essay writing; pre-service teachers' initial solutions about a case vignette can be…”

“Examples might include pedagogical scenarios (Janson et al. 2020; Wambsganss et al. 2020), e.”

“The expert evaluators were given a set of assessment criteria adapted from the official GRE website16, Re3 [83] and Adaptive Learning system [73], where they rated each submission on the following six dimensions on a scale of 1 to 10.”

“[128] for students to develop argumentation quality in writing by providing feedback.”

“In designing the tool, we followed the findings of recent user studies (Afrin et al., 2021; Wambsganss et al., 2020), a review of the extant literature related to the provision of feedback related to failure (e.g., Bjork et al. (2013); Metcalfe (2017); Lawrence and Reed (2019)), and self-regulated…”

“Wambsganss et al. (2020) presented a feature-based argument mining system using linguistic features and traditional classifiers such as SVM.”

“Other tools support auxiliary processes during writing such as ideation [31, 32, 75, 116], revision [25, 56], and reflection [20, 91].”

“The authors in Wambsganss et al. (2020) used a questionnaire with 38 items to evaluate the usability, usefulness, adaptivity and effectiveness of an adaptive dialog-based tutoring system for augmenta-tion skills.”

“However, experts who created examples still faced the challenges of effectively adapting and contextualizing into the current learning step.”

“One recent work experiments with a text editor to support the student argumentation skills [24].”

“Finally, following the bottom-up approach in the prior study, [6], [46], [47], [48], we summarized the major causes of the unexpected system results from the aspects of attributes, tasks, and visual encodings.”

“This requirement echos one previous research [68].”

“Finally, reviewing support was limited, and focused on evaluation using quantitative feedback like readability metrics [87], and visual feedback such as progress bars [89].”

“, argumentation annotated corpora for argumentation skill learning [65, 66]), design knowledge for a valuesensitive perspective on CAs might encourage designers and research towards a more ethical design of these novel ISs.”

“Past research has presented general artifact design studies mainly in the fields of NLP and HCI (Scheuer et al. 2012, Scheuer 2015, Lippi and Torroni 2016, Chernodub et al. 2019, Lauscher et al. 2019, Afrin et al. 2021); however, it has lacked rigorous studies about the embedding and impact of these IS on students’ ar…”

“IS recently joined the discourse on pedagogical use of DAs by identifying potential for DA in IS education, studying the organizational consequences of using DA in higher education and vocational training, and empowering teachers to integrate DA in their teaching [36, 88, 106, 108, 158, 185, 190, 191, 193].”

“For example, Wambsganss et al. (2020) developed an adaptive Argumentative Learning system to provide students with instant feedback on the structure of an argument with text.”

“Individual empathy training is only available for a limited number of students since individual feedback through a student’s learning journey is often hindered due to large-scale lectures or the growing field of distance learning scenarios such as Massive Open Online Classes (MOOCs) (Seaman et al., 2018; Hattie and Ti…”

“While a number of well-defined models exist in the field of AM (Freeman, 2001; Perelman, 1971; Pollock, 1995; Walton and Macagno, 2015; Walton, 1996; Wambsganss et al., 2020a), there is no general argumentation annotation scheme across all domains and genres of texts.”

“For each tool, we adopt the technology acceptance model from [65, 81, 84] to measure its usefulness (four items; Cronbach’s 𝛼 = 0.919); easy to use (two items; Cronbach’s 𝛼 = 0.701); and intention to use (two items; Cronbach’s 𝛼 = 0.901).”

“gumentative components & relations and providing the evaluation scores [125].”

“AL (Wambsganss et al., 2020) used natural language processing to provide feedback on the quality of essays in terms of their argument structure, readableness, and coherence.”

“Researchers, particularly in educational technology, have developed systems to support students with persuasive and structured writing [50, 76, 78].”

“Inspired by Stab and Gurevych (2017a); Wambsganss et al. (2020c), our guideline consisted of 26 pages, including definitions and rules for what is an ar-gument, which annotation scheme is to be used, and how argument components, argumentative relations, and the qualitative attributes are to be…”

“Since metacognition skills are complex pedagogical constructs, we relied on a proven method to measure metacognition skills in textual data (e.g., done for argumentation skills in [2, 96, 101] or for problem-solving skills in [103]).”

“Numerous tools have recently been developed to enhance writing work-flows with LLMs, spanning various areas such as story writing [16, 17, 100], screenplay writing [67], poetry [31], dictation [57], and argumentative [96, 102] and scientific [28] writing.”

“Metacognition is helpful, or even essential, in a wide variety of HCI tasks, such as using computer programming tools [8, 39, 46], writing argumentative text on the web [60], creating and assessing visualizations [33], and learning via technology [4].”

“Recent advances in writing instruction tools have started to support writing structure and organization by demonstrating the use of machine learning in guiding how to write an argumentative essay [92, 94] and evaluating whether a learner has succeeded or failed at particular rhetorical moves [93, 95, 96].”

“Thus, research in fields such as HCI or educational technology has shown a growing interest in developing argumentation systems to support individuals [2, 26, 110, 113].”

“Advanced technologies can also be used as argumentative writing tutors that aid students in crafting arguments (Hoffmann, 2015) and provide adaptive feedback on the quality of arguments generated by students (Scheuer et al., 2010; Wambsganss et al., 2020).”

“, to enhance individual learning of students with adaptive writing support systems [8] or adaptive tutoring through a smart personal assistant [9].”

“The problems and constraints encountered in employing assorted and layered digital critical thinking teaching technologies are often thought to result from the people using the technologies (students, teachers and institutions) (for example, Arend ; Wambsganss et al. 2020).”

“…have already introduced many different assistants, which, for example, support students to learn to write program code, strengthen their argumentation skills, answer FAQs, or support study course selections (Bouaiachi et al., 2014; Hobert, 2019; Ranoliya et al., 2017; Wambsganss et al., 2020).”

“One promising solution to better support students in their writing process and to overcome the limitations in law courses would be the use of writing support systems that could provide individualized feedback to students (Wambsganss et al., 2020a).”

“These contributions complement existing research that has worked on longer argumentative essays (Wambsganss et al., 2020), which has differently conceptualized argumentation quality that is however less suitable for direct feedback within a conversational agent, and broader work on argumentation mining on identifying…”

“For instance, AL [70] visualizes argumentative sentence relationships and scores persuasiveness, while ArgRewrite [84] tracks and assesses sentence-level changes.”

“The dashboard was inspired by similar writing assistant works [30, 39, 48, 57, 60, 62] and was developed using the third-party code scripts available in the Spacy and NLTK code libraries 5 .”

“Barrot (2023) and Wambsganss et al. (2020) both signi fi cantly improved students ’ writing abilities and promoted the development of their attention and reasoning skills by providing adaptive feedback during the writing process.”

“For each system interface, we adapt the technology acceptance model [67, 70] to measure the perceived usefulness (four items, e.g., “I fnd the vocabulary learning support system useful in my vocabulary learning process by story retelling”; Cronbach’s = 0.921); easiness to use (four items, e.g., “My…”

“In another study, an adaptive learning support system, ‘AL’, designed and implemented by Wambsganss et al. (2020) is capable of using NLP and machine learning to give feedback to students to improve their ability in argumentative writing skills in German language; AL points out which claims are not…”

“Although Wambsganß et al. (2020) and Wamb-sganss et al. (2022a) propose different granularity levels of explanations, their study is restrained to students from their university.”

“Unlike AI-generated feedback (CodeTailor[60],AL [140],Surgment [141], and many other systems [31, 71, 84, 106, 123, 129]), our design leverages peer gaze focus aligned with course materials, inspired by peer effect theory showing students’ behaviors are influenced by peers [18, 38, 47, 59, 111,…”

“Our proposed enthymeme-related tasks complement these attempts, having the same ultimate goal, namely to learn support systems for argumentative writing (Wambsganss et al., 2020).”

“, general essay writing [7], argumentation [9], or writing styles), our understanding is yet limited regarding the granular effects of different writing support modalities (e.”

“, for argumentation skill training (Wambsganss et al., 2020a) or empathy skills (Wambsganss et al.”

“The two pre-survey constructs we chose have been employed in prior studies focusing on user-centric systems [101, 102] to check for valid randomization.”

“Pedagogical conversational agents’ functions and purposes have been manifold and include developing argumentation skills (Wambsganss et al., 2020, 2021), idea generation (Vladova et al., 2019), math education (Cai et al., 2021), programming skills (Hobert, 2019b), information acquisition (Meyer von…”

“…based on the learning from errors paradigm, as well as investigating how more recent NLP models (e.g., GPT) are able to be effective in teaching argumentation skills to learners, compared to the classifier-based approaches with task-specific models done in previous works [135, 139, 140].”

“…trending (“hot") posts ( Range from 2024.8.18 to 2024.8.25 ) as our source data for data annotation for two primary reasons: (1) the platform’s established reputation in facilitating structured argumentation, as evidenced by its extensive use in prior research on online deliberation [59, 79, 82].”

“While prior works on text revision [8, 15, 20, 28] have proposed machine collaborative writing interfaces, they focus on collecting human-machine interaction data to better train neural models, rather than understanding the underlying processes of text revision.”

“Wambsganss et al. (2020) proposed a support system that analyzes an argumentation made by learners, displays the argumentation structure, and highlights unsupported claims.”

“5 learning systems and the way students want to learn and improve their argumentation skills (more information in [40]).”

“The potential of student peer feedbacks and metacognition skill training has been successfully shown for other skill training based on NLP such as argumentations skills [43].”

“PCAs are effective in a variety of educational contexts, e.g., to support the development of better argumentation and reasoning skills (Wambsganss et al., 2020, 2021), to enhance students’ problem-solving skills (Winkler et al., 2019), or to teach programming skills with a scaffolding-based PCA…”

“Moreover, with the importance of feedback in learning processes [8, 47], recent advances in this field utilize explainable AI (XAI) to provide additional explanations on top of pure training samples to achieve better learning performances [14].”

“Each mode of the ICAP framework involves different types of behaviors and learning processes that imply learning outcomes augmenting the engagement level (Chi & Wylie, 2014; Wambsganss et al., 2020).”

“This builds upon prior work that has already demonstrated the benefits of traditional writing assistants, which feature a text area and feedback module as their primary interface elements [39, 77].”

“This makes TAP an adaptive tool in evaluating argumentative skills according to students' cognitive levels (Wambsganss et al., 2020).”

“By interweaving pronunciation, grammatical, and pragmatic analysis, it charts a comprehensive route to balanced communicative competence [13].”

“We assess each strategy’s strengths and limitations based on our axial coding process, linking reported user experiences to their intended rhetorical appeals.”

“…and has been shown to provide support across various domains, such as decision-making (Vassiliades et al., 2021; Walker et al., 2018), writing support (Wamb-sganss and Rietsche, 2019; Wambsganss et al., 2020), and argument generation (Khatib et al., 2021; Hua et al., 2019; Slonim et al., 2021).”

“The tool Artist is an adapted version based on the tool Argumentation Learning developed at the University of St.Gallen by Wambsganss et al. (2020).”

“However, providing individualized, high-quality essay feedback is time-consuming [29, 99, 108], thus limiting teachers’ capacity to provide detailed input to all students and thereby reducing opportunities for effective practice [26].”

“For each tool, we adopt the technology acceptance model from [85, 104, 110] to measure its usefulness (four items; Cronbach’s 𝛼 = 0.950 ), ease of use (four items; Cronbach’s 𝛼 = 0.948 ), and intent to use (two items; Cronbach’s 𝛼 = 0.954 ).”

“More recently, HCI researchers have examined the use of machine learning and GenAI as tools for supporting students’ writing, such as to guide the composition of argumentative essays [108, 110] and assess whether a learner has effectively executed specific rhetorical strategies [109, 111, 112].”

“…has focused on automatically identifying and analyzing argu-mentative structures in diverse texts, including essays, [Sazid and Mercer 2022, Wambsganss et al. 2020a, Abkenar et al. 2021, Guo et al. 2024], scientific papers [Michael, Fromm et al. 2020, Al Khatib et al. 2021, Stylianou and…”

“In order to effectively address RQ 3, it is crucial to emphasize the tool's positive impact on the development of students' writing skills within the scholarly environment, according to feedback (technology acceptance metric) collected from students (Wambsganss et al. 2020).”

“These strategies can provide learners with feedback and guidance on their language use, help track their progress, and identify areas for improvement (Wambsganss et al., 2020).”

“The findings of this study aligned with the previous research [27].”

“For terms and conditions of usage please see: http://creativecommons.org 225 un valido supporto all’apprendimento del procedere argomentativo (Wambsganss et al., 2020).”

“The application of adaptive learning provides feedback based on the learners' learning path [5].”