The development of secondary students' feedback literacy: peer assessment as an intervention

Classified as probably not a substantial claim.

Carless and Boud introduced a feedback literacy framework describing the competences needed to participate in feedback processes.

“Carless and Boud (2018) introduced a framework of feedback literacy describing the competences necessary to participate in feedback processes; it has generated considerable interest and encouraged further research.”

• cites Carless & Boud, 2018 sourceSupported

Classified as probably not a substantial claim.

Carless and Boud called for research on the development of students' feedback literacy combined with intervention.

“Carless and Boud (2018) expressed the need for research on the development of students ' feedback literacy combined with intervention.”

• cites Carless & Boud, 2018 sourceSupported

May combine two claims.

Understanding corrective feedback as a mediator of improvement is pivotal in peer assessment, and the results of this study support that idea.

“Understanding corrective feedback as a mediator of improvement is pivotal in peer assessment (Ketonen et al., 2020; McConlogue, 2015; Tasker & Herrenkohl, 2016), and the results of this study support that idea.”

• cites Ketonen et al., 2020 sourceSupported
• cites McConlogue, 2015 sourceNot checked — no full text of the source
• cites Tasker & Herrenkohl, 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable).

The development of feedback literacy will be of value in university studies and workplaces.

“The development of their feedback literacy will be of value in their university studies (O ' Donovan, 2017) and workplaces (Carless & Boud, 2018).”

• cites O ' Donovan, 2017Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Carless & Boud, 2018 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

The first level of engaging with and interpreting received feedback is a lack of interest in feedback.

“The first level is not being interested in feedback at all”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Students at the first level of understanding why feedback is given do not appreciate any feedback.

“The students on the first level do not appreciate any kind of feedback.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The third level is active interpretation and evaluation of received feedback.

“The third level comprises active interpretation and evaluation of the received feedback.”

May combine two claims.

Improvement was observed in all three categories: understanding why feedback is given in five cases, engaging with and interpreting received feedback in four cases, and engaging in revisions in seven cases.

“Improvement was noticed in all categories: understanding of the purpose of feedback (five cases), engagement and interpretation of received feedback (four cases), and engagement in making revisions (seven cases).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In PA1, Student 10 enjoyed positive feedback but could not say it was useful.

“The student stated that she had enjoyed the positive feedback. She could not actually say that it was useful, but nevertheless, she was contented with receiving positive feedback.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Students at the third level welcome corrective feedback and view it as helping learning and improvement.

“Conversely, students on the third level welcome corrective feedback, as they see it as a mediator of learning and improvement.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The second level is reading feedback while remaining a passive receiver.

“The second level describes the student reading the feedback but staying in the passive role of a receiver.”

The quote only partly states this claim.

The study identified three levels of revision engagement: no improvement, superficial improvement, and investment in improvement.

“| Level 1 | No improvement | Student does not improve his/her work.”

• Also stated in: § Categories and levels for students ' feedback literacy · p. 10 · plain

  “| Level 2 | Superficial improvement | Student makes improvements that do not require much activity nor learning new knowledge or skills.”

• Also stated in: § Categories and levels for students ' feedback literacy · p. 10 · plain

  “| Level 3 | Investment in improvement | Student improves work as a result of his/her own activity, for example, seeking new feedback, guidance or new knowledge, or practicing new skills.”

Classified as probably not a substantial claim. May combine two claims.

Students had varying feedback literacy skills, which were defined and described in an iterative process.

“During the first part of the analysis, we noticed students had varying feedback literacy skills, and we defined and described them in an iterative process.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Students at the second level appreciate positive feedback and see feedback as a judgment.

“The second level, appreciation of positive feedback, reflects the understanding of feedback as a judgment.”

May combine two claims.

Student 10 advanced from Level 2 to Level 3 in understanding why feedback is given and in engaging with and interpreting received feedback from PA1 to PA2, while already being at the highest level in revision engagement in PA1.

“She was already on the highest of three levels on ' engagement in making revisions ' in PA1, but in ' understanding of the purpose of feedback ' and ' engagement and interpretation of received feedback, ' she advanced from Level 2 to Level 3.”

May combine two claims.

A year later, Student 10 was no longer happy with positive feedback and hoped for guiding feedback that would help her improve her work.

“A year later, the student was no longer happy with positive feedback. She stated that she had hoped for guiding feedback that would have helped her to improve her work.”

May combine two claims.

The levels of the first category show that the students were not necessarily inclined to read feedback or use it for improvement.

“The levels of the first category show that the students were not necessarily inclined to read feedback or use it for improvement.”

May combine two claims.

Students' development was shown through increased interest in feedback and appreciation of criticism.

“The students ' development was shown through their increased interest in feedback and appreciation of criticism.”

May combine two claims.

In PA1, Student 10 made only the one change suggested in the feedback; in PA2, she actively sought feedback and used feedback received by her work partner and feedback from assessing another student's work.

“In PA1, the student only made the one change that was suggested in the feedback. In PA2, she actively looked for feedback. Even though her peer assessor did not provide guidance, she utilized the feedback that her work partner had received and the feedback that assessing the other student ' s work provided her.”

May combine two claims.

Student 12 did not improve his work after PA1, but after PA2 he made two meaningful improvements that he judged important to make.

“The student did not improve his work after PA1, but after PA2, he made two meaningful improvements that he had judged important to make.”

May combine two claims.

Even when interested in feedback, most students were not initially prepared to invest in interpreting it but expected feedback to tell them unambiguously how to proceed.

“Even when interested in it, most of the students were not initially prepared to invest in interpreting it but expected the feedback to tell them unambiguously how to proceed.”

May combine two claims.

Student 12 superficially performed his role as a peer assessor and did not revise his work after PA1.

“This meant that he superficially performed his role as a peer assessor and did not revise his work.”

May combine two claims.

After PA2, Student 12 evaluated which feedback comments were most important and ignored less important ones, and he considered part of the feedback wrong and ignored it (Level 3).

“The student explained that he had evaluated which were the most important feedback comments and ignored less important ones. In addition, he stated that he had considered part of the feedback comments wrong and therefore ignored them (Level 3).”

May combine two claims.

Student 12's narration of feedback was contradictory; he had looked at the feedback but had not worked to understand its meaning and took it as general guidance for the future (Level 2).

“The student ' s narration of the feedback is contradictory. He had received a feedback comment that the work was hard to interpret. Student 12 had looked at the feedback but not worked to understand its meaning. Instead of comparing the feedback to his work and deciding to reject the feedback or to improve the work, he explained that he had taken the feedback as general guidance for the future (Le”

May combine two claims.

In PA1, Student 12 was at Level 1 in revision engagement and at Level 2 in understanding why feedback is given and in engaging with and interpreting received feedback.

“In PA1, Student 12 was on Level 1 on ' engagement in making revision ' and Level 2 on ' understanding of the purpose of feedback ' and ' engagement and interpretation of received feedback. '”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims. The quote only partly states this claim.

The authors engaged in peer negotiation to test the levels and categories, considering it appropriate to examine the reliability of coding.

“This study ' s authors engaged in peer negotiation to test the levels and categories. Coding students ' skills was a complex task that required precise interpretation and the management of versatile data because students expressed their feedback literacy skills in diverse parts of the interview, and as explained previously, the comments were interpreted with other data. Therefore, negotiation”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In two cases, a skill level was created from data of only one student, and this was enriched with interviews with additional students to test the rubric and find more examples.

“In two cases, a skill level was created from the data of only one student, and we enriched that data with interviews with additional students in the spring of Grade 7 (Table 1). This enabled us to test our rubric and find more examples to describe the levels.”

May combine two claims.

Peer assessment is generally considered useful in encouraging students to act on feedback, especially when integrated in a deeper conversation about assessment.

“As explained previously, peer assessment can help address these issues and is generally considered useful in encouraging students to act on feedback (Jonsson, 2013), especially when integrated in a deeper conversation about assessment (Cartney, 2010).”

• cites Jonsson, 2013 sourceNot checked — no full text of the source
• cites Cartney, 2010 sourceNot checked — no full text of the source

May combine two claims.

The number of participants was kept small to enable knowing the students and gathering rich data, because the study aimed to understand and explain phenomena rather than make broad generalizations.

“The study aimed at understanding and explaining the phenomena of peer assessment and feedback literacy instead of making broad generalizations, and the number of participants was kept small in order to enable knowing the students and gathering of rich data.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The training focused on issues that previous research identified as important for successful peer assessment: understanding formative assessment, understanding useful feedback qualities, internalizing criteria, comparing criteria with work, social rules, and practising peer assessment.

“The training focused on issues that previous research noted as important for successful peer assessment, specifically understanding the idea of formative assessment, understanding of the qualities of useful feedback, internalizing assessment criteria, comparing the criteria with the work, social rules, and practicing actual peer assessment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The decision to leave one skill out of the results in two cases (Students 4 and 8) was made in the meeting.

“The decision to leave one skill out of the results in two cases (Students 4 and 8) was made in the meeting.”

Classified as probably not a substantial claim. May combine two claims.

A case study design was adopted because the research was exploratory, and it allowed the researchers to know all students and gather versatile data.

“A case study design was adopted for the exploratory nature of the research. The case study approach allowed us to know all the students and gather versatile data.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Feedback must be accepted, processed, and acted on by the receiver.

“The feedback needs to be accepted, processed, and acted on by the receiver.”

Classified as probably not a substantial claim. May combine two claims.

Altogether, 13 students provided a satisfying dataset and were included in the analysis.

“Altogether, 13 students provided a satisfying dataset, and they were included in the analysis of the study.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The authors mutually agreed that the challenging cases were considerably more difficult than the regular cases and that definitively determining the level of students' skills was not possible.

“We had a mutual agreement that, due to scant interviewee input from students, these cases were considerably more challenging than the regular cases, and definitively determining the level of students ' skills was not possible.”

Classified as probably not a substantial claim. May combine two claims.

Within each category, case groups with similar skill levels were formed, examined, described, and revised until new rounds produced no changes, creating a criteria-based rubric from basic to advanced.

“Within each category, we formed case groups with similar skill levels. Then, we examined and described the groups and returned to the data with revised descriptions. We revised the groups until new rounds did not produce changes to the groups or their descriptions. While remaining sensitive to the theory, we organized the groups from the most basic to the most advanced, thereby creating a criteria”

Classified as probably not a substantial claim.

To ensure reliability, students' narratives were compared to their own work, their received and provided feedback, and the work they had assessed.

“and ' Engagement and interpretation of received feedback ' skills using interviews as a primary data source, and to ensure reliability, we compared students ' narratives to their own work, their received and provided feedback, and the work that they had assessed.”

Classified as probably not a substantial claim. May combine two claims.

The two peer assessments (PA1 and PA2) were similar in task difficulty, lab report form, and assessment criteria, although their topics differed.

“The topics (determining the speed of an object and determining the resistance of a wire) were different due to the curricula, but the difficulty of the tasks, the form of the lab reports, and the assessment criteria were similar.”

Classified as probably not a substantial claim.

Feedback literacy highlights the need for students' own activity in feedback processes.

“Feedback literacy highlights the need for students ' own activity in feedback processes.”

Classified as probably not a substantial claim.

Peer assessment can serve as an intervention that gives students opportunities to reflect on and practise feedback processes and thereby develop feedback literacy.

“Peer assessment is used not only as a tool to uncover students ' perceptions about assessment and feedback, but also as an intervention that provides students with the opportunities to reflect and practice feedback processes and hence to develop their feedback literacy.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The researcher and teacher planned pairings so that students were not partnered with someone from their inquiry group and with sensitivity to social issues.

“The researcher and the teacher planned the pairings so that students were not partnered with someone they conducted the inquiry with and with sensitivity to social issues, such as pairing vulnerable students with considerate classmates.”

Classified as probably not a substantial claim. May combine two claims.

Thematic analysis was chosen for its flexibility: it is not bound to a specific theoretical framework, works well with various data forms, can find similarities and differences, and can produce unexpected insights.

“A thematic analysis was chosen for its flexibility: it is not bound to any specific theoretical framework, and it works well in analyzing data consisting of various forms. It can be used to find similarities and differences within the data, and it can produce unexpected insights.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

After coding the whole dataset, codes were gathered into preliminary categories, the distinctiveness of categories was examined, categories were adjusted and named, and data were recoded until no further changes were made.

“After coding the whole dataset, the codes were gathered into preliminary categories. The distinctiveness of the categories was then examined, and the categories were adjusted and named. The data were recoded, and categories were re-examined and readjusted until no further changes were undertaken.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In two cases (Students 4 and 8), the data did not provide enough information to determine the category level, so these skills were left out of those students' results.

“In two cases (Students 4 and 8), even these data did not provide enough information to determine the category level, and these skills were left out of these students ' results (Figure 4).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The teacher and researcher had prolonged experience in co-teaching at the school, which helped co-planning the interventions and made sharing classroom responsibility natural.

“The teacher and the researcher had prolonged experience in co-teaching at the school, which helped co-planning the interventions and made sharing the responsibility in the classroom natural.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In the third phase, category criteria were used to evaluate seventh- and eighth-grade students' feedback literacy, and multiple data sources were used.

“In the third phase of analysis, the category criteria were used in evaluating seventh- and eighth-grade students ' feedback literacy, and multiple data sources were also used.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Tasker and Herrenkohl (2016) found that students learn to value critical feedback.

“Tasker and Herrenkohl (2016) found that students learn to value critical feedback”

• cites Tasker and Herrenkohl (2016) sourceNot checked — no full text of the source

May combine two claims.

In formative peer assessment, students are persuaded to focus on improvement and hence gain experience with corrective feedback and find it desirable.

“In formative peer assessment, students are persuaded to focus on improvement and hence gain experience with corrective feedback and find it desirable.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study's specific context focused on two student groups at the same school.

“The second limitation is this study ' s specific context, as it focused on two student groups at the same school.”

May combine two claims.

Features common with prior studies were explicit discussion of feedback processes and use of peer feedback, implying these are promising practices to promote development of students' feedback literacy.

“Features in common with the present study were the explicit discussion of the feedback processes and the use of peer feedback, implying that these are promising practices to promote the development of students ' feedback literacy.”

• cites Han & Xu, 2019bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Hey-Cunningham et al., 2020 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Han and Xu (2019b) reported a cultivated willingness in students to ask for peers' opinions.

“Han and Xu (2019b) reported a cultivated willingness in students ' to ask for peers ' opinions.”

• cites Han and Xu (2019b)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Several studies on secondary students mentioned their struggles to act on peer feedback, and the authors suggest the reason might be students' undeveloped feedback literacy skills.

“Several studies on secondary students mentioned their struggles to act on peer feedback (Anker-Hansen & Andr ee, 2019; Hovardas et al., 2014; Tseng & Tsai, 2007), and we suggest that the reason might be students ' undeveloped feedback literacy skills.”

• cites Anker-Hansen & Andr ee, 2019 sourceNot checked — no full text of the source
• cites Hovardas et al., 2014 sourceNot checked — no full text of the source
• cites Tseng & Tsai, 2007 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Other feedback literacy categories could probably be found with more data sources.

“However, other feedback literacy categories could probably be found with more data sources.”

May combine two claims.

Assessment practices in Finland are prevalently summative, and students are accustomed to receiving evaluative feedback at the end of courses when criticism cannot be used to improve performance.

“Assessment practices in Finland are prevalently summative (Atjonen et al., 2019), and students are accustomed to receiving evaluative feedback at the end of courses when criticism cannot be used to improve their performance.”

• cites Atjonen et al., 2019Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The criteria for the levels of students' feedback literacy were found functional in coding their skills, and these criteria can be used and elaborated in other studies.

“The criteria for the levels of the students ' feedback literacy were found functional in the coding of their skills, and these criteria can be used and elaborated on in other studies.”

May combine two claims.

The research is not experimental, so it is unclear to what extent the use of peer assessment affected the development of students' feedback literacy.

“The first is that the research is not experimental; therefore, it is unclear to what extent the use of peer assessment affected the development of students ' feedback literacy.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

During the school year, the students naturally faced experiences other than this intervention.

“Physics and chemistry contexts comprise only 5% of studies in lower secondary schools, and during the school year, the students naturally faced experiences other than this intervention.”