Peering into large lectures: examining peer and expert mark agreement using peerScholar, an online peer assessment tool
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
As class sizes increase, methods of assessments shift from costly traditional approaches (e.g. expert‐graded writing assignments) to more economic and logistically feasible methods (e.g. multiple‐choice testing, computer‐automated scoring, or peer assessment). While each method of assessment has its merits, it is peer assessment in particular, especially when made available online through a Web‐based interface (e.g. our peerScholar system), that has the potential to allow a reintegration of open‐ended writing assignments in any size class – and in a manner that is pedagogically superior to traditional approaches. Many benefits are associated with peer assessment, but it was the concerns that prompted two experimental studies (n = 120 in each) using peerScholar to examine mark agreement between and within groups of expert (graduate teaching assistants) and peer (undergraduate students) markers. Overall, using peerScholar accomplished the goal of returning writing into a large class, while producing grades similar in level and rank order as those provided by expert graders, especially when a grade accountability feature was used.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlled and automatic human information processing: I. Detection, search, and attention.
Schneider, W., & Shiffrin, R. M. (1977) · Psychological Review · doi:10.1037/0033-295x.84.1.1 · Cited by 5,902
Formative assessment and the design of instructional systems
Sadler, D. R. (1989) · Instructional Science · doi:10.1007/bf00117714 · Cited by 4,285 · cited 4×
“Specifically, seeing and critiquing peers’ work is said to encourage deeper analyses of the student’s own work (Sims 1989), a process that is required for the improvement of work quality (Sadler 1989).”
BackgroundMethodInfluentialPeer Assessment between Students in Colleges and Universities
Topping, K. (1998) · Review of Educational Research · doi:10.2307/1170598 · Cited by 2,173 · cited 3×
“These findings suggest that peerScholar can be used both effectively and fairly in any class context, as has been found previously to be true of pencil and paper versions of peer assessment (e.g. Topping 1998).”
BackgroundStudent Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks
Falchikov, N., & Goldfinch, J. (2000) · Review of Educational Research · doi:10.3102/00346543070003287 · Cited by 1,170 · cited 1×
“In his review of past studies of peer assessment, Topping (1998) (see also Falchikov and Goldfinch 2000) concluded that peer assessment is a © 2008 The Authors.”
BackgroundEnnis, R. H. (1993) · Theory Into Practice · doi:10.1080/00405849309543594 · Cited by 1,101
Distributed and massed practice from laboratory to classroom
Seabrook, R., Brown, G. D. A., & Solity, J. E. (2005) · Applied Cognitive Psychology · doi:10.1002/acp.1066 · Cited by 211
Cited by
Incorporating AI and learning analytics to build trustworthy peer assessment systems
Darvishi, A., Khosravi, H., Sadiq, S., & Gašević, D. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13233 · Cited by 120
Freeman, S., & Parks, J. W. (2010) · CBE—Life Sciences Education · doi:10.1187/cbe.10-03-0017 · Cited by 44
“CBE—Life Sciences Education Vol. 9, 482–488, Winter 2010 482 (Robinson, 2001) and the peerScholar software created at the University of Toronto (Paré and Joordens, 2008).”
BackgroundMethodInfluentialEvaluating the Quality of Learning Resources: A Learnersourcing Approach
Abdi, S., Khosravi, H., Sadiq, S., & Demartini, G. (2021) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2021.3058644 · Cited by 40
Probabilistic Graphical Models for Boosting Cardinal and Ordinal Peer Grading in MOOCs
Mi, F., & Yeung, D.-Y. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9210 · Cited by 34
Incorporating Training, Self-monitoring and AI-Assistance to Improve Peer Feedback Quality
Darvishi, A., Khosravi, H., Abdi, S., Sadiq, S., & Gašević, D. (2022) · Proceedings of the Ninth ACM Conference on Learning @ Scale · doi:10.1145/3491140.3528265 · Cited by 31
Metadata
- Authors
- Paré, D. E., & Joordens, S.
- Venue
- Journal of Computer Assisted Learning
- Publisher
- Wiley
- Published
- 2008-10-27
- Volume
- 24
- Issue
- 6
- Pages
- 526-540
- ISSN
- 0266-4909 (ISSN-L)
- 1365-2729
- OpenAlex
- W2148189517
- Cited by
- 113
- References
- 6
- FWCI
- 17.17
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Peering into large lectures: examining peer and expert mark agreement using peerScholar, an online peer assessment tool. https://app.cochat.ai/papers/10.1111/j.1365-2729.2008.00290.x
- The paper
- Paré, D. E., & Joordens, S. (2008). Peering into large lectures: examining peer and expert mark agreement using peerScholar, an online peer assessment tool. Journal of Computer Assisted Learning. https://doi.org/10.1111/j.1365-2729.2008.00290.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.