The nature of feedback: How peer feedback features affect students’ implementation rate and quality of revisions.
The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Abstract
Although feedback is often seen as a critical component of the learning process, many open questions about how specific feedback features contribute to the effectiveness of feedback remain—especially in regards to peer feedback of writing. Nelson and Schunn (2009) identified several important features of peer feedback in their nature of feedback model. In the current study, we test an updated theoretical model that includes a broader set of features and considers not only students’ likelihood of implementing a comment but also the quality of their revisions. To empirically test the updated theoretical model, we analyze over 7,500 comments from 351 reviewers to 189 authors. Each comment was coded for the presence of praise, a problem description, a suggested solution, localization, focus (i.e., low prose, high prose, substance), implementation, and revision quality. To account for the cross-classified nesting of data, we used a 2-level, cross-classified, hierarchical logistic regression model. Only 2 feedback features increased students’ likelihood of implementation (i.e., overall praise and localization), while several feedback features reduced students’ likelihood of implementation (i.e., mitigating praise, solutions, and high-prose comments). Overlapping feedback features affected students’ ability to revise and in opposing directions from their effect on likelihood of implementation: Revisions were less likely to improve the quality of their paper when implementing comments that included a specific location in the text, but they were more likely to improve the quality of their paper when implementing comments that focused on high-prose and substance issues. Implications of these findings are discussed. (PsycINFO Database Record (c) 2016 APA, all rights reserved)
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“a study by Patchan etal. (2016) found that the quality of feedback, including its specificity, clarity, and detail was a significant predictor of students' writing improvement in online peer feedback activities.”
From Online peer feedback in higher education: A synthesis of the literature (2024) · cites this paper as Patchan et al., 2016
Citations
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
The ICAP framework: Linking cognitive engagement to active learning outcomes
Chi, M. T. H., & Wylie, R. (2014) · Educational Psychologist · doi:10.1080/00461520.2014.965823 · Cited by 2,911
Cognitive Load Theory and Complex Learning: Recent Developments and Future Directions
van Merriënboer, J. J. G., & Sweller, J. (2005) · Educational Psychology Review · doi:10.1007/s10648-005-3951-0 · Cited by 2,206
Contextual prerequisites for understanding: Some investigations of comprehension and recall
Bransford, J. D., & Johnson, M. K. (1972) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(72)80006-9 · Cited by 2,131
Consequences of individual feedback on behavior in organizations.
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.4.349 · Cited by 1,822
Self‐Explanations: How Students Study and Use Examples in Learning to Solve Problems
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989) · Cognitive Science · doi:10.1207/s15516709cog1302_1 · Cited by 1,714
The instructional effect of feedback in test-like events.
Bangert-Drowns, R. L., Kulik, C.-L. C., Kulik, J. A., & Morgan, M. (1991) · Review of Educational Research · doi:10.3102/00346543061002213 · Cited by 1,313
Paas, F., Renkl, A., & Sweller, J. (2004) · Instructional Science · doi:10.1023/b:truc.0000021806.17516.d0 · Cited by 1,054
The effectiveness of corrective feedback in SLA: A meta-analysis
Li, S. (2010) · Language Learning · doi:10.1111/j.1467-9922.2010.00561.x · Cited by 941
The effect of different types of corrective feedback on ESL student writing
Bitchener, J., Young, S., & Cameron, D. (2005) · Journal of Second Language Writing · doi:10.1016/j.jslw.2005.08.001 · Cited by 900
A typology of written corrective feedback types
Ellis, R. (2008) · ELT Journal · doi:10.1093/elt/ccn023 · Cited by 872
Do students value feedback? Student perceptions of tutors’ written responses
Weaver, M. R. (2006) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930500353061 · Cited by 807
Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012) · Cognitive Science · doi:10.1111/j.1551-6709.2012.01245.x · Cited by 772
Controlling the Information Flow: Effects on Consumers' Decision Making and Preferences
Ariely, D. (2000) · Journal of Consumer Research · doi:10.1086/314322 · Cited by 722
The influence of teacher commentary on student revision
Ferris, D. R. (1997) · TESOL Quarterly · doi:10.2307/3588049 · Cited by 618 · cited 1×
“Turning to the quality of revisions, studies conducted in second language writing (Ferris, 1997) as well as in clinical note-writing (Parkes et al., 2013) found that praise almost never impacted performance quality and in general, the effect size of praise is typically quite small (Cohen's d ϭ…”
BackgroundImproving the effectiveness of peer feedback for learning
Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010) · Learning and Instruction · doi:10.1016/j.learninstruc.2009.08.007 · Cited by 615 · cited 1×
“…many different ages of learners about how to provide helpful feedback often suggests including praise (Grimm, 1986; Nilson, 2003; Saddler & Andrade, 2004), and one common criterion of peer feedback quality is whether praise was offered (Gielen et al., 2010; Prins, Sluijsmans, & Kirschner, 2006).”
BackgroundThe nature of feedback: How different types of peer feedback affect writing performance
Nelson, M. M., & Schunn, C. D. (2009) · Instructional Science · doi:10.1007/s11251-008-9053-x · Cited by 602
Self-citationFrey, M. C., & Detterman, D. K. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00687.x · Cited by 585
Student Reactions to Teacher Response in Multiple-Draft Composition Classrooms
Ferris, D. R. (1995) · TESOL Quarterly · doi:10.2307/3587804 · Cited by 512
Mory, E. H. (2013) · doi:10.4324/9781410609519-40 · Cited by 494
Evidence on the Effectiveness of Comprehensive Error Correction in Second Language Writing
Van Beuningen, C. G., De Jong, N. H., & Kuiken, F. (2012) · Language Learning · doi:10.1111/j.1467-9922.2011.00674.x · Cited by 449
Mathematical problem solving by analogy.
Novick, L. R., & Holyoak, K. J. (1991) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037//0278-7393.17.3.398 · Cited by 413
Kellogg, R. T. (1999) · doi:10.1093/oso/9780195129083.001.0001 · Cited by 344
Formative Assessment and Writing: A Meta-Analysis
Graham, S., Hebert, M., & Harris, K. R. (2015) · The Elementary School Journal · doi:10.1086/681947 · Cited by 340
Written corrective feedback for individual L2 writers
Ferris, D. R., Liu, H., Sinha, A., & Senna, M. (2013) · Journal of Second Language Writing · doi:10.1016/j.jslw.2012.09.009 · Cited by 328
A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction
Azevedo, R., & Bernard, R. M. (1995) · Journal of Educational Computing Research · doi:10.2190/9lmd-3u28-3a0g-ftqt · Cited by 308
On-line peer assessment and the role of the peer feedback: A study of high school computer course
Tseng, S.-C., & Tsai, C.-C. (2007) · Computers & Education · doi:10.1016/j.compedu.2006.01.007 · Cited by 293 · cited 1×
“…receiving solutions helped these learners improve their writing performance (AbuSeileek & Abualsha'r, 2014; Bitchener, 2012; Bitchener , Young, & Cameron, 2005; Nelson & Schunn, 2009; Shintani, Ellis, & Suzuki, 2014; Sugita, 2006; Tseng & Tsai, 2007; Van Beuningen, De Jong, & Kuiken, 2012).”
Corbett, A. T., & Anderson, J. R. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365111 · Cited by 280
‘Feed‐forward’: improving students' use of tutors' comments
Duncan, N. (2007) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930600896498 · Cited by 272
DIFFERENTIAL EFFECTS OF ORAL AND WRITTEN CORRECTIVE FEEDBACK IN THE ESL CLASSROOM
Sheen, Y. (2010) · Studies in Second Language Acquisition · doi:10.1017/s0272263109990507 · Cited by 265
Shintani, N., Ellis, R., & Suzuki, W. (2014) · Language Learning · doi:10.1111/lang.12029 · Cited by 260
Orsmond, P., & Merry, S. (2011) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930903201651 · Cited by 246
Cho, K., Schunn, C. D., & Charney, D. (2006) · Written Communication · doi:10.1177/0741088306289261 · Cited by 235
Self-citationMaximizing Student Performance in Summary Writing: Managing Cognitive Load
Kirkland, M. R., & Saunders, M. A. P. (1991) · TESOL Quarterly · doi:10.2307/3587030 · Cited by 212
Feedback Specificity, Exploration, and Learning.
Goodman, J. S., Wood, R. E., & Hendrickx, M. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.2.248 · Cited by 201
Iselin, E. R. (1988) · Accounting, Organizations and Society · doi:10.1016/0361-3682(88)90041-4 · Cited by 187
Patchan, M. M., & Schunn, C. D. (2015) · Instructional Science · doi:10.1007/s11251-015-9353-x · Cited by 181 · cited 1×
“…learn from peer assessment, including the benefits for the author of receiving feedback (Patchan & Schunn, in press) and the benefits for the reviewer of providing feedback (Patchan & Schunn, 2015), in contrast to the current focus on the relative effectiveness of different forms of peer feedback.”
BackgroundSelf-citationFeedback on learning achievement: rhetoric and reality
Orrell, J. (2006) · Teaching in Higher Education · doi:10.1080/13562510600874235 · Cited by 176
Improving student peer feedback
Nilson, L. B. (2003) · College Teaching · doi:10.1080/87567550309596408 · Cited by 172
Beaufort, A. (2004) · Research in the Teaching of English · doi:10.58680/rte20044467 · Cited by 168
Responding to L2 Students in College Writing Classes: Teacher Perspectives
Ferris, D. R., BROWN, J., Liu, H., & STINE, M. E. A. (2011) · TESOL Quarterly · doi:10.5054/tq.2011.247706 · Cited by 153
Feedback for General Practitioners in Training: Quality, Styles, and Preferences
Prins, F. J., Sluijsmans, D. M. A., & Kirschner, P. A. (2006) · Advances in Health Sciences Education · doi:10.1007/s10459-005-3250-z · Cited by 147
Learner uptake of teacher electronic feedback in ESL composition
Ene, E., & Upton, T. A. (2014) · System · doi:10.1016/j.system.2014.07.011 · Cited by 147
Matsumura, L. C., Patthey-Chavez, G. G., Valdés, R., & Garnier, H. (2002) · The Elementary School Journal · doi:10.1086/499713 · Cited by 145
Developing the teaching instinct, 1: Feedback
Hesketh, E. A., & Laidlaw, J. M. (2002) · Medical Teacher · doi:10.1080/014215902201409911 · Cited by 122
Feedback Specificity, Learning Opportunities, and Learning.
Goodman, J. S., & Wood, R. E. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.5.809 · Cited by 122
Feedback Models for Learning, Teaching and Performance
Molloy, E., & Boud, D. J. (2013) · Springer eBooks · doi:10.1007/978-1-4614-3185-5_33 · Cited by 115
A machine for automatic teaching of drill material.
Pressey, S. L. (1927) · School and Society · Cited by 112
Feedback sandwiches affect perceptions but not performance
Parkes, J., Abercrombie, S., & McCarty, T. A. (2012) · Advances in Health Sciences Education · doi:10.1007/s10459-012-9377-9 · Cited by 92
Written Corrective Feedback for L2 Development: Current Knowledge and Future Research
Bitchener, J. (2012) · TESOL Quarterly · doi:10.1002/tesq.62 · Cited by 90
Redefining Revision for Freshmen
Wallace, D. L., & Hayes, J. R. (2020) · Reading Empirical Research Studies: The Rhetoric of Research · doi:10.4324/9781315044620-37 · Cited by 83
Patchan, M. M., Charney, D., & Schunn, C. D. (2009) · Journal of Writing Research · doi:10.17239/jowr-2009.01.02.2 · Cited by 81
Self-citationServing up the feedback sandwich.
Dohrenwend, A. (2003) · PubMed · Cited by 79
Students' and teachers' views of written feedback at undergraduate level: A literature review
Agius, N. M., & Wilkinson, A. (2014) · Nurse Education Today · doi:10.1016/j.nedt.2013.07.005 · Cited by 72
Using peer computer-mediated corrective feedback to support EFL learners' writing
AbuSeileek, A., & Abualsha'r, A. (2014) · Language Learning & Technology · doi:10.64152/10125/44355 · Cited by 72
The effects of skill diversity on commenting and revisions
Patchan, M. M., Hawk, B. N., Stevens, C. A., & Schunn, C. D. (2012) · Instructional Science · doi:10.1007/s11251-012-9236-3 · Cited by 69
Self-citationThe impact of teachers' comment types on students' revision
Sugita, Y. (2006) · ELT Journal · doi:10.1093/elt/cci079 · Cited by 61
Voerman, L., Korthagen, F. A. J., Meijer, P. C., & Simons, R. J. (2014) · Teaching and Teacher Education · doi:10.1016/j.tate.2014.06.005 · Cited by 60
Lexical issues in writing center tutorials with international and US-educated multilingual writers
Nakamaru, S. (2010) · Journal of Second Language Writing · doi:10.1016/j.jslw.2010.01.001 · Cited by 58
Feedback specificity, information processing, and transfer of training
Goodman, J. S., Wood, R. E., & Chen, Z. (2011) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2011.01.001 · Cited by 54
Saddler, B., & Andrade, H. (2004) · Educational leadership · Cited by 50
Symonds, P. M., & Chase, D. H. (1992) · Journal of Educational Psychology · doi:10.1037/0022-0663.84.3.282 · Cited by 43
Understanding the benefits of receiving peer feedback: A case of matching ability in peer-review
Patchan, M. M., & Schunn, C. D. (2016) · Journal of Writing Research · doi:10.17239/jowr-2016.08.02.03 · Cited by 39
Self-citationNatural language processing techniques for researching and improving peer feedback
Xiong, W., Litman, D., & Schunn, C. (2012) · Journal of Writing Research · doi:10.17239/jowr-2012.04.02.3 · Cited by 36
Self-citationKnowledge of results as an incentive in school room practice.
Brown, F. J. (1932) · Journal of Educational Psychology · doi:10.1037/h0074392 · Cited by 29
Evaluation as a dynamic process.
LeBaron, S., & Jernick, J. (2000) · PubMed · Cited by 24
Improving Students' Responses to their Peers' Essays
Grimm, N. M. (1986) · College Composition and Communication · doi:10.2307/357386 · Cited by 22
A Study of Corrective Feedback and Learner's Uptake in Classroom Interactions
Esmaeili, F., & Behnam, B. (2014) · International Journal of Applied Linguistics & English Literature · doi:10.7575/aiac.ijalel.v.3n.4p.204 · Cited by 22
Patchan, M. M., Schunn, C. D., & Clark, R. J. (2011) · Journal of Writing Research · doi:10.17239/jowr-2011.02.03.4 · Cited by 20
Self-citationThe Role of Form-focused Feedback on Developing Students’ Writing Skill
Zohrabi, M., & Rezaie, P. (2012) · Theory and Practice in Language Studies · doi:10.4304/tpls.2.7.1514-1519 · Cited by 11
Classroom Evaluation of a Scaffolding Intervention for Improving Peer Review Localization
Nguyen, H. V., Xiong, W., & Litman, D. J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07221-0_34 · Cited by 9
Error-Flagging Support for Testing and Its Effect on Adaptation
Kumar, A. N. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13388-6_40 · Cited by 9
Improving Peer Feedback Prediction: The Sentence Level is Right
Nguyen, H. V., & Litman, D. J. (2014) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.3115/v1/w14-1812 · Cited by 8
An Evaluation of the Effectiveness of Just-In-Time Hints
Hausmann, R. G. M., Vuong, A., Towle, B., Fraundorf, S. H., Murray, R. C., & Connelly, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_114 · Cited by 6
The Interface of Error Types, Teacher’s Feedback, and Students’ Uptake
Zoghi, A., & Nikoopour, J. (2013) · International Journal of Applied Linguistics & English Literature · doi:10.7575/aiac.ijalel.v.3n.1p.54 · Cited by 5
Cited by
Does peer assessment promote student learning? A meta-analysis
Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2019.1620679 · Cited by 348
“This occurs in instances where students reflect on their own learning by providing feedback to others and improve their own work by incorporating feedback from others (Patchan, Schunn, and Correnti 2016).”
BackgroundFeedback sources in essay writing: peer-generated or AI-generated feedback?
Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J., & Drachsler, H. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00455-4 · Cited by 282
“…look at the findings of the first research question discloses that the feedback generated by ChatGPT comprehensively encompassed all essential components characterizing high-quality feedback, including affective, cognitive, and constructive dimensions (Kerman et al., 2022; Patchan et al., 2016).”
BackgroundMethodBanihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. A. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100489 · Cited by 246
Tian, L., & Zhou, Y. (2020) · System · doi:10.1016/j.system.2020.102247 · Cited by 210
Patchan, M. M., & Schunn, C. D. (2015) · Instructional Science · doi:10.1007/s11251-015-9353-x · Cited by 181
Self-citation
Metadata
- Authors
- Patchan, M. M., Schunn, C. D., & Correnti, R. J.
- Venue
- Journal of Educational Psychology
- Published
- 2016-11
- Volume
- 108
- Issue
- 8
- Pages
- 1098-1120
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W2328994266
- Semantic Scholar
- d9cc62a10825881c75c2c7a4b5491b9a2f81bdf1
- Cited by
- 165
- References
- 77
- Citations by year
- 2026: 14 · 2025: 35 · 2024: 23 · 2023: 32 · 2022: 17 · 2021: 7 · 2020: 10 · 2019: 10 · 2018: 12 · 2017: 4
- FWCI
- 38.47
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Psychology (publishedVersion)
- Topics
- Social Sciences › Education › Student Assessment and Feedback
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Education › Evaluation of Teaching Practices
- Keywords
- peer feedback, writing revision, praise, localization, problem description, student writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.