Assessing the Impact of Student Peer Review in Writing Instruction by Using the Normalized Compression Distance

Abstract

Research Problem: Studies identify peer review as an effective instructional method to improve student writing. Some teachers, however, avoid peer review, perhaps due to the workload required for assessing and correcting peer reviews. Previous studies have not proposed any method to reduce teacher workload by using an objective means to analyze the effects of peer review. Research Questions: This study assesses the degree of similarity between student essay drafts using normalized compression distance (NCD), a compression-based classification algorithm. How does peer review affect student essays, as measured by the NCD? What were the changes in essay length and holistic scores? How did students respond to peer essays? How did peer review affect students during revision? What were the NCD results? How did holistic scoring correspond to NCD results? Literature Review: Studies of pharmacists and engineers indicate that English language technical communication skills are important. Studies of peer review in language education indicate that peer comments are valuable but cultural differences and lack of confidence may impede making or using comments. Studies of NCD applied to web data, figures, and images indicate useful results. Methodology: This quantitative study used anonymous peer review and compared the results of traditional holistic scoring against a novel NCD measure. The researchers conducted the study with 35 student volunteers at a pharmaceutical university in Tokyo, Japan. The students had at least nine years of previous English instruction and previous peer-review experience. In class, students wrote an essay, anonymously reviewed a peer's essay according to instructions, then revised their own essays based on peer comments. An assessor graded the two drafts using a holistic scoring rubric. The researchers used NCD to quantify the change between drafts. Results and Discussion: Sixty percent of revisions contained more words than the originals. 51% percent of revisions received higher scores, 40% had no change, and 8.5% percent had reduced scores. Eleven percent of reviewers with low English proficiency did not identify obvious errors. Three revised essays had lower grades because the writers did not know how to incorporate peer comments. Anonymous peer review could lead to poor results where students had poor reviewing skills or did not know how to use peer comments. NCD helps teachers identify which revised essays to re-evaluate after peer review by indicating those with large quantities of changes. The study was limited by its small group of participants. Future research will examine longer essays, more participants, varied backgrounds, web delivery of NCD, and finding more factors to indicate the quality of written work to reduce teacher workload.