Student engagement with teacher and automated feedback on L2 writing

In the paper

“AWE feedback has discernible advantages over teacher feedback in terms of timeliness, convenience, multiple drafting, and even potential learner autonomy, which have also been shown in the previous studies (Chen & Cheng, 2008;Dikli, 2006).”

What the sources say

• Partly supported

  The Discussion (p. 14) says AWE feedback is delivered instantly, and the Pedagogical Foundation (pp. 3–4) describes repeated drafting and the theoretical potential for learner autonomy. The paper does not establish convenience as an advantage over teacher feedback; it also says whether AWE develops learner autonomy is uncertain and emphasizes the need for human feedback (Discussion, p. 14). (LLM, conf=0.84)

  cites Chen & Cheng, 2008 sourceExperimental — accuracy not yet measured
• Partly supported

  The Summary and Discussion section says AES systems provide immediate feedback, easing the time burden of teacher responses, and let students save and revise successive drafts. The MY Access! section describes a web-based tool intended to make writing more feasible. Our copy does not establish increased learner autonomy or show that these advantages were demonstrated in prior studies. (LLM, conf=0.84)

  cites Dikli, 2006Experimental — accuracy not yet measured

In the paper

“I looked at online dictionaries and corrected some spelling errors and nouns from the feedback but I was not sure how to correct the others… I didn't ask my classmates for help because I felt embarrassed with the score.”

In the paper

“Although she turned to the same online dictionary as David to improve her writing, she went beyond this by seeking to diagnose what was incorrect.”

In the paper

“The questionnaire data from 206 students showed that many students did not value teacher feedback, although the more highly motivated students claimed that they tried to engage with it and incorporate it into their drafts.”

In the paper

“We can see in the screenshot in Figure 6 that David only resubmitted his essay once after receiving AWE feedback and only spent less than 14 minutes on the task.”

In the paper

“This detailed description of how the two learners engaged with both teacher and AWE feedback on their writing shows that the highly engaged learner (Flora) outperformed the moderately engaged learner (David) in her response to both sources of feedback as well as in making more improvements to her text.”

In the paper

“We characterise Flora as a student who demonstrated deep and full engagement with both teacher and AWE feedback on her writing.”

In the paper

“In Flora's case, she also mentioned that the improvement of the score served as a positive confirmation, telling her she was on the right track.”

In the paper

“In contrast, teacher feedback did not allow the same kind of flexibility and the absence of immediate feedback created greater uncertainty then the learners came to correct errors and rewrite their texts.”

In the paper

“David was less adept at revising to effectively respond to both sources of feedback, failing to correct the grammatical errors noted by the AWE feedback, for example, and so was demotivated to work on the second draft.”

In the paper

“She also substituted simplistic terms and phrases with more sophisticated ones.”

In the paper

“Some errors noted by the AWE feedback Flora deliberately ignored, making no revisions at all.She did this intentionally to avoid being overwhelmed by the thinking and revising process.”

In the paper

“Firstly, Flora effectively corrected all the linguistic errors explicitly identified by the teacher, but missed the one error which the teacher only implicitly indicated.”

In the paper

“Flora evinced a strong emotional reaction to the feedback from both sources, as the teacher feedback elicited admiration for the teacher's assistance as well as some same about the errors she had made, encouraging revision.She also responded affectively to the more neutral delivered commentary of the AWE programme and the upgraded scores on her drafts, each gain encouraging her to improve further.”

In the paper

“Figure 4, from the AWE record, shows that Flora resubmitted her essay 13 times, and the score rose from 79 to 90 (100 as the full score).”

In the paper

“We found a very limited range of revision operations in the second draft and out of 35 errors seven were corrected properly.Five errors were deleted, one was substituted, one correction was added and 21 errors were not addressed.”

In the paper

“He was overwhelmed by the errors identified in his work and discouraged by his inability to respond to all but the most explicitly corrected of them.”

In the paper

“we found that Flora enjoyed more autonomy when she responded to AWE feedback.This allowed her to regulate her revising time and the pace of her rewriting according to her own learning plan.”

In the paper

“David said that he was overwhelmed by the amount of corrective feedback and felt helpless about how to address the problems.”

In the paper

“His response to both sources of feedback was more passive, and demonstrated lower levels of self-starting and participation.”

In the paper

“Most conspicuously, Flora rewrote her draft numerous times in the multiple-draft assignment, not only correcting errors but improving the style and explicitness of the text.”

In the paper

“He also mentioned in the interview that the teacher did not advise the students how many times he expected them to revise their essays or the overall score he expected them to get.”

In the paper

“Neither student was used to receiving detailed teacher feedback so they were often unsure about how to engage with it.”

In the paper

“In the interview, David expressed his confusion over this feedback:”

In the paper

“When she got the AWE feedback on her first draft, Flora did not revise her draft immediately but started the following day, with a five-hour gap between her second and third submission.”

In the paper

“In the interview, David did say that he appreciated the feedback he received and there was obviously an affective connection with the teacher”

In the paper

“Compared to David, Flora was more active in that she invested more time-on-task and more positive attitudes in the revision process.”

In the paper

“She also addressed confusing parts of her essay by deleting unnecessary words and sentences, and adding more explanations to increase clarity.”

In the paper

“When I saw some circles, underlines, and question marks, I was actually bewildered.I didn't know what the teacher meant… I prefer to be told.I think the teacher should offer us more help on how to revise our essays.”

In the paper

“I felt very disappointed about the score.I never got a score below 70.It was so frustrating that I didn't feel able to go back to it again to change the essay.”

In the paper

“While more facilitative and less directive than the teacher feedback, he felt embarrassed and demoralised by the low score he received from the AWE and this discouraged his full engagement with it.”

In the paper

“Neither reorganization nor rewriting was evident.”

In the paper

“Flora, in contrast, is a higher proficiency student and had relatively little difficulty interpreting both teacher and AWE feedback on her work.”

In the paper

“In all, the changes were minimal and he neither reorganized nor rewrote any part of his essay.”

In the paper

“a close look at the second draft reveals that instead of directly addressing the problem areas circled and underlined, he simply deleted some of the offending text.”

In the paper

“She took considerable time over her revisions, made numerous changes and submissions to the AWE and employed a repertoire of different stratagems: planning, goal-setting, monitoring, evaluating, and so on.”

In the paper

“These students varied in the techniques they were able to deploy, a factor related to their behavioural and cognitive engagement with feedback.”

In the paper

“So, behaviorally, David attempted to attend to the teacher feedback and made revisions”

In the paper

“Redundant words and phrases that were either circled or underlined were substituted with more concise ones.”

In the paper

“Teacher feedback addressed more error types (16) than the AWE feedback (8), although differences in the two assignments may have influenced this, with factors such as difficulty and length causing students to commit different types of errors.”

In the paper

“Moreover, this study shows that experienced ESL teachers can offer more comprehensive corrective feedback on student writing, giving feedback on such issues as number usage and abbreviation which tend to be outside the scope of AWE feedback routines.”

In the paper

“The different styles of engagement adopted by these students led us to identify one as a highly engaged learner and the other as a moderately engaged learner, and this helped us to see how each responded to teacher and AWE feedback differently.”

In the paper

“As a highly engaged learner, Flora gravitated more towards”

In the paper

“In contrast, David, responded mechanically to both teacher and AWE feedback, with a preference for spoonfeeding and a sceptical attitude to automated feedback.”

In the paper

“The highly engaged learner tended to participate actively and reflect more deeply with both teacher and AWE feedback on her writing, while the moderately engaged learner was less motivated and showed less willingness to use the feedback he received from both sources.”

In the paper

“Engagement also seems to involve students in devoting cognitive resources to understanding or memorizing the feedback they are given (Sachs & Polio, 2007).”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Sachs & Polio, 2007

In the paper

“A great deal of this research, however, has focused on error correction and limited itself to particular modes of delivery, examining the effectiveness of teacher written feedback (Ferris, 1997), oral conferencing (Weissberg, 2006), peer feedback (Mendonca & Johnson, 1994) or machine marking (Warschauer & Ware, 2006).”

What the sources say

• ?Not checked — no full text of the sourcecites Ferris, 1997 source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Weissberg, 2006
• ?Not checked — no full text of the sourcecites Mendonca & Johnson, 1994 source

In the paper

“As Stevenson and Phakiti (2014) have noted, students' behavioral engagement with AWE feedback (using AWE feedback to revise their drafts) tells us little about whether students have adopted cognitive and metacognitive strategies to notice, evaluate and finally improve their writing.”

What the sources say

• ?Not checked — no full text of the sourcecites Stevenson and Phakiti (2014) source

In the paper

“Grimes (2005) also found that students engaged only superficially with AWE feedback and identified a typical revision pattern whereby students submitted their first drafts, made a few corrections after receiving corrective feedback, and resubmitted their drafts as quickly as possible.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Grimes (2005)

In the paper

“In their study of two L2 writers' engagement with teacher feedback given as reformulations, for example, Qi and Lapkin (2001) argue that the quality of noticing, related to the depth of processing feedback, is a key factor in improvement in students' revisions.”

What the sources say

• ?Not checked — no full text of the sourcecites Qi and Lapkin (2001) source

In the paper

“They suggest that not only do affective factors affect the actions adopted by learners in responding to the feedback, they also have an impact on their willingness to accept and retain the feedback.”

What the sources say

• ?Not checked — no full text of the sourcecites Storch and Wigglesworth (2010) source

In the paper

“The bulk of AWE feedback research, however, has focused on students' written products, with little attention paid to students' writing and revising process.”

In the paper

“In summary, while open to multiple interpretations and beset by difficulties of measurement, student engagement is generally believed to correlate positively with academic achievement and has great potential to improve students' school outcomes.”

In the paper

“Encouraging is the fact that student engagement is malleable and can be affected by teachers and parents, and shaped by wider teaching practices and school policies (Fredricks et al., 2004;Willms, 2003).”

What the sources say

• ?Not checked — no full text of the sourcecites Fredricks et al., 2004 source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Willms, 2003

In the paper

“It is important to keep in mind that engagement is a multidimensional construct and student engagement with written feedback goes beyond error corrections.”

In the paper

“Feedback on second language writing is widely acknowledged to offer considerable learning benefits, providing writers with a sense of audience and an understanding what readers' value in a text (Goldstein, 2005;K. Hyland, 2016).”

What the sources say

• ?Not checked — no full text of the sourcecites Goldstein, 2005 source
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites K. Hyland, 2016

In the paper

“These studies are provocative and inspirational, but almost all the previous research on student engagement looks at human feedback on writing, focuses on just one or two dimensions of engagement or is narrowly confined to error corrections.”

In the paper

“This allowed us to ensure the two participants had the same AWE assignments and would receive the same teaching and feedback from the same teacher.”

In the paper

“Having collected the written data we first identified all the teacher feedback points (F.Hyland, 1998) and coded them according to a modified version of Ferris's (2006) taxonomy.We then followed the same categorization scheme to code the AWE feedback.”

What the sources say

• ?Not checked — no full text of the sourcecites F.Hyland, 1998 source
• ?Not checked — no full text of the sourcecites Ferris's (2006) source

In the paper

“F. Hyland (2003), for example, found that the extent her two case study students engaged with form-focused feedback varied greatly, so that one lower-intermediate student's response to teacher feedback was strongly influenced by her emotional reactions to the teachers' comments on her work.”

What the sources say

• ?Not checked — no full text of the sourcecites F. Hyland (2003) source

In the paper

“We suggest that student engagement is a key factor in the success of writing development and how these different sources of feedback have great potential in facilitating L2 students' involvement in their writing.”

In the paper

“A number of studies have suggested that student engagement with written corrective feedback facilitates language acquisition and writing development.”

In the paper

“We suggest that student engagement is a key factor in the success of writing development and how these different sources of feedback have great potential in facilitating L2 students' involvement in their writing.”

In the paper

“It is important to keep in mind that engagement is a multidimensional construct and student engagement with written feedback goes beyond error corrections.”

In the paper

“His lack of linguistic knowledge therefore impacted on his ability to correct errors and discouraged him from doing so, thus his cognitive, behavioural and affective engagement was reduced.”

In the paper

“Her proficiency thus enabled her to cognitively engage and encouraged her motivation (affect) and successful revisions (behavioural).”

In the paper

“He seemed less willing to reflect on the data provided to him on his writing or take the initiative to improve his work without the guidance or intervention of a teacher.”

In the paper

“This also had an impact on how he engaged cognitively and behaviourally with the feedback as he used more deletion that did not involve much evaluating, planning or monitoring.”

In the paper

“It is clear that the two students engaged with teacher and AWE feedback on their writing differently and we believe there were two main factors in this: the type of feedback the students received and individual learner traits.”

In the paper

“This kind of mechanical revising according to the answers provided by the teacher is simply copying and can be done automatically without deep engagement.”

In the paper

“this deletion strategy suggests that David's lack of English proficiency restricted his understanding of the feedback and contributed to a lack of deep engagement with teacher feedback.”

In the paper

“Perhaps the most significant of these was language proficiency, which is related to cognitive engagement.”

In the paper

“Affectively, the feedback created considerable anxiety for David and as a result he rejected its validity”

In the paper

“After recursive passes through the feedback and students' writing we classified revisions into seven categories: (1) correction, (2) no correction, (3) deletion, (4) substitution, (5) addition, (6) reorganization and (7) rewriting.”

In the paper

“This lack of teacher support, in addition to the negative emotions created by his experience of the AWE score combined to discourage David from engaging either behaviorally or cognitively with the AWE feedback.”

In the paper

“To address his problem areas during revision would require more cognitive processing, evaluating, and monitoring, so despite producing a second draft with fewer errors, the revision was primarily at a surface level.”

In the paper

“Together with her improved score, this indicated that Flora spent some time revising her essay and made efforts to improve the quality of her writing.”

In the paper

“which explains why he found the feedback overwhelming.”

In the paper

“Overall, we can see here a multidirectional interaction among the three types of engagement and a bi-directional interaction between the use of strategies and deep engagement.”

In the paper

“A number of learner factors are also likely to have influenced their responses to feedback and indicate the degree of engagement they were prepared, or able, to invest.”

In the paper

“Flora's greater repertoire of learning strategies enabled her to interpret and use both teacher and AWE feedback, making better use of the online dictionary, for instance, to identify and correct errors.”

In the paper

“although his inability to decode the symbols limited his engagement and ability to respond”

In the paper

“But we believe it is not the mere possession of these qualities which leads to good responses to feedback, it is the active student engagement which produces an effective response.”

In the paper

“The ability to engage with feedback on writing, therefore, is not only determined by language ability as it also requires the motivation and interest to invest time and energy in the process as well as a repertoire of effective revision and writing operations to carry it out.”

In the paper

“The timestamp, moreover, shows that she wrote her revisions over two days with one-week between them, indicating some reflection about the process between submissions.”

In the paper

“Her greater proficiency allowed her to thoroughly process both teacher and AWE comments, and use these to make effective revisions.”

In the paper

“There is no consensus in the literature regarding the effectiveness of such indirect feedback on learning, this process can help raise students' awareness of their mistakes (Ferris, 2002).”

What the sources say

• ?Not checked — no full text of the sourcecites Ferris, 2002

In the paper

“Learners seem eager to receive immediate feedback on their writing and engage in the revision process when the task is still fresh in their memory (K.Hyland, 2013).”

What the sources say

• ?Not checked — no full text of the sourcecites K.Hyland, 2013 source

In the paper

“and a better understanding of how students engage with teacher and AWE feedback differently may help us make better-informed decisions regarding the sources of feedback we offer our students.”

In the paper

“With the help of AWE feedback, teachers can adjust their feedback focus and allocate more time to instruction or to formative assessment aimed at the rhetorical development of student drafts.”

In the paper

“Thus instead of capitalizing on the advantages of computer-assisted language learning he was more concerned about its drawbacks, thus influencing the way he engaged with the AWE feedback.”

In the paper

“The examination of two types of feedback on L2 student writing indicates that it might be more effective to integrate the two types of feedback in classroom contexts.”

In the paper

“This belief prompted her to actively use both cognitive strategies (e.g.monitoring and synthesizing) and affective strategies (e.g.adjusting emotions and using motivation) to improve both her draft and her writing skills.”

In the paper

“Both the source of the feedback and individual learner factors can shape the process so that language proficiency and knowledge of learning strategies can play a pivotal role, not only in helping students to understand and interpret feedback, but also in evaluating and monitoring the revision process.”

In the paper

“Learners can be empowered with the responsibility to revise their essays according to their own schedule, enjoying the autonomy offered by AWE feedback and, by capitalizing on the multiple drafting and revision opportunities to improve their drafts, can internalize language points in the revision process.”

In the paper

“However, teacher feedback also has advantages.Most importantly, teachers' comments on content and organization are highly valued by student writers, partly because they are a human response to their writing and not a pattern-matching algorithm designed to highlight errors.”

In the paper

“More highly engaged learners thus tend to spend more time working with feedback, show more positive attitudes toward it, and employ more revising strategies, demonstrating that behavioral, affective, and cognitive engagement dynamically interact with each other.”

In the paper

“Such engagement is a dynamic process whereby behavioral, affective, and cognitive responses are simultaneously at work.”