Peer-review in practice: eight years of Aropä
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Articles discussing and analysing student peer-review activities proliferate the educational literature, typically describing one or more class exercises where students provide feedback on each other’s work. These papers usually focus on a peer-review activity designed as a scholarly study, and make conclusions about its success or otherwise. There is not one standard model for ‘peer-review’, and information on the many different assessment designs used is distributed over an increasing number of publications and websites. This paper provides a meta-review of peer-review activities as they are implemented in practice, using configuration data from over a thousand assignments conducted using an online peer-review system during an eight-year period. We present data on the wide variety of assignment designs and the parameters that comprise them, their rubrics, and comparisons between subject areas. Information on the norms and range of all decisions to be made will encourage instructors (both new to and experienced in conducting peer-review activities) to reflect on and justify the choices they make.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Peer Assessment between Students in Colleges and Universities
Topping, K. (1998) · Review of Educational Research · doi:10.2307/1170598 · Cited by 2,173 · cited 4×
“Topping (2009) comments that students react differently to feedback from ‘adults’ and peers, and that it is a good idea for instructors to check some of the peer reviews against their own assessments. Pearce, Mulder, and Baik (2009) describe a case study where one of the reviews received was always Table 7.”
InfluentialStudent Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks
Falchikov, N., & Goldfinch, J. (2000) · Review of Educational Research · doi:10.3102/00346543070003287 · Cited by 1,170 · cited 1×
“While there have been analyses of peer review activities as published in the literature (Falchikov and Goldfinch (2000) compared numerical marks awarded by peers and instructors in 48 published studies, Topping (1998) analysed 109 articles), only Pearce, Mulder, and Baik (2009) report on…”
Rethinking feedback practices in higher education: a peer review perspective
Nicol, D., Thomson, A., & Breslin, C. (2014) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2013.795518 · Cited by 966
Topping, K. J. (2009) · Theory Into Practice · doi:10.1080/00405840802577569 · Cited by 785 · cited 5×
“Topping (1998) suggests peers might be matched according to complementary ability (that is, ‘more expert assess those less expert’), yet also describes a case study (Topping 2009) where review teams are groups of three students with ‘roughly similar ability in writing’.”
InfluentialPeer reviewers learn from giving comments
Cho, Y. H., & Cho, K. (2011) · Instructional Science · doi:10.1007/s11251-010-9146-1 · Cited by 294
Cheng, K.-H., Liang, J.-C., & Tsai, C.-C. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.02.001 · Cited by 174
Rotsaert, T., Panadero, E., & Schellens, T. (2018) · European Journal of Psychology of Education · doi:10.1007/s10212-017-0339-8 · Cited by 126
Demiraslan Çevik, Y. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.05.056 · Cited by 42
Brill, J. M. (2016) · Educational Technology Research and Development · doi:10.1007/s11423-015-9421-6 · Cited by 40 · cited 3×
“While the many case study articles on peer review (see also Li, Lui, and Steckelberg 2010; Brill 2016; Weaver et al. 2016) provide interesting information and analysis of individual cases that have led to scholarly publication, there is no comprehensive analysis of the design of peer review activities in practice. Whi…”
Peters, O., Körndle, H., & Narciss, S. (2018) · European Journal of Psychology of Education · doi:10.1007/s10212-017-0344-y · Cited by 37
Cited by
Assessment in the age of artificial intelligence
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100075 · Cited by 409
Incorporating AI and learning analytics to build trustworthy peer assessment systems
Darvishi, A., Khosravi, H., Sadiq, S., & Gašević, D. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13233 · Cited by 120
Evaluating the Quality of Learning Resources: A Learnersourcing Approach
Abdi, S., Khosravi, H., Sadiq, S., & Demartini, G. (2021) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2021.3058644 · Cited by 40
Incorporating Training, Self-monitoring and AI-Assistance to Improve Peer Feedback Quality
Darvishi, A., Khosravi, H., Abdi, S., Sadiq, S., & Gašević, D. (2022) · Proceedings of the Ninth ACM Conference on Learning @ Scale · doi:10.1145/3491140.3528265 · Cited by 31
Metadata
- Authors
- Purchase, H., & Hamer, J.
- Venue
- Assessment & Evaluation in Higher Education
- Publisher
- Informa UK Limited
- Published
- 2018-02-12
- Volume
- 43
- Issue
- 7
- Pages
- 1146-1165
- ISSN
- 0260-2938 (ISSN-L)
- 1469-297X
- OpenAlex
- W2792809572
- Cited by
- 22
- References
- 10
- FWCI
- 7.30
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Peer-review in practice: eight years of Aropä. https://app.cochat.ai/papers/10.1080/02602938.2018.1435776
- The paper
- Purchase, H., & Hamer, J. (2018). Peer-review in practice: eight years of Aropä. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2018.1435776
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.