The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Reliability and validity of rubrics for assessment through writing
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In that same vein, Rezaei and Lovorn found that without training regarding effective rubric use, reliability or validity will likely not improve.36”
From Beyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education (2020) · cites this paper as Rezaei and Lovorn, 2010
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Church, J. A. (2001) · Journal of Government Information · doi:10.1016/s1352-0237(01)00307-0 · Cited by 9,153
A Brief History of Neoliberalism
(2020) · The Anti-Capitalist Chronicles · doi:10.2307/j.ctv17ppcd0.8 · Cited by 9,074
The use of scoring rubrics: Reliability, validity and educational consequences
Jonsson, A., & Svingby, G. (2007) · Educational Research Review · doi:10.1016/j.edurev.2007.05.002 · Cited by 1,509 · cited 5×
“The intra-rater reliability (consistency of grading a given writing by the same rater twice) is reported to be higher (Cronbach’s alpha about .70) than the inter-rater reliability when a rubric is used (Jonsson & Svingby, 2007).”
InfluentialSachs, J. D. (2005) · Cited by 614
Capra, F. (2002) · Cited by 378
Assessing Second Language Writing in Academic Contexts
Kunnan, A. J., & Hamp-Lyons, L. (1993) · The Modern Language Journal · doi:10.2307/328948 · Cited by 363
Rater characteristics and rater bias: implications for training
Lumley, T., & McNamara, T. F. (1995) · Language Testing · doi:10.1177/026553229501200104 · Cited by 343
Assessment criteria in a large-scale writing test: what do they really mean to the raters?
Lumley, T. (2002) · Language Testing · doi:10.1191/0265532202lt230oa · Cited by 321
Globalization and belonging: the politics of identity in a changing world
(2004) · Choice Reviews Online · doi:10.5860/choice.41-6801 · Cited by 290
Andrade, H. L., Du, Y., & Wang, X. (2008) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2008.00118.x · Cited by 256
Hafner, J., & Hafner, P. (2003) · International Journal of Science Education · doi:10.1080/0950069022000038268 · Cited by 255
Diagnostic assessment of writing: A comparison of two rating scales
Knoch, U. (2009) · Language Testing · doi:10.1177/0265532208101008 · Cited by 215
The Validity of Using Holistic Scoring to Evaluate Writing: A Critical Overview
Charney, D. (1984) · Research in the Teaching of English · doi:10.58680/rte198415687 · Cited by 199
Validating holistic scoring for writing assessment : theoretical and empirical foundations
Williamson, M. M., & Huot, B. (1993) · Hampton Press eBooks · Cited by 168
Rethinking Rubrics in Writing Assessment
Wilson, M., & Kohn, A. (2006) · Cited by 157
Assessing Second Language Writing in Academic Contexts
Leki, I., & Hamp‐Lyons, L. (1993) · College Composition and Communication · doi:10.2307/358997 · Cited by 147
On a Scale: A Social History of Writing Assessment in America
Elliot, N. (2005) · Cited by 135
Re-training writing raters online: How does it compare with face-to-face training?
Knoch, U., Read, J., & von Randow, J. (2007) · Assessing Writing · doi:10.1016/j.asw.2007.04.001 · Cited by 119
Wolfe, E. W. (1997) · Assessing Writing · doi:10.1016/s1075-2935(97)80006-2 · Cited by 108
Dempsey, M. S., PytlikZillig, L. M., & Bruning, R. H. (2009) · Assessing Writing · doi:10.1016/j.asw.2008.12.003 · Cited by 107
(1977) · Elsevier eBooks · doi:10.1016/b978-0-12-697750-9.50001-x · Cited by 106
The scope of writing assessment
Hamp‐Lyons, L. (2002) · Assessing Writing · doi:10.1016/s1075-2935(02)00029-6 · Cited by 105
Speaking My Mind: The Trouble with Rubrics
Kohn, A. (2006) · The English Journal · doi:10.2307/30047080 · Cited by 96
Verbal ability and delinquency: testing the moderating role of psychopathic traits
Centifanti, L. C. M., Frick, P. J., Kimonis, E. R., & Aucoin, K. J. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01847.x · Cited by 92
Johnson, R. L., Penny, J., & Gordon, B. (2000) · Applied Measurement in Education · doi:10.1207/s15324818ame1302_1 · Cited by 91
WRITING NEATNESS, COMPOSITION ERRORS, AND ESSAY GRADES
Marshall, J. C., & Powers, J. M. (1969) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1969.tb00665.x · Cited by 87
Mabry, L. (1999) · Phi Delta Kappan · Cited by 84
Skaalvik, E. M., & Skaalvik, S. (2004) · Psychological Reports · doi:10.2466/pr0.95.3f.1187-1202 · Cited by 82
Read, B., Francis, B., & Robson, J. (2005) · Assessment & Evaluation in Higher Education · doi:10.1080/02602930500063827 · Cited by 80
Influences of Handwriting Quality on Teacher Evaluation of Written Work
Markham, L. R. (1976) · American Educational Research Journal · doi:10.3102/00028312013004277 · Cited by 75
Why I Won't Be Using Rubrics to Respond to Students' Writing
Wilson, M. (2007) · The English Journal · doi:10.2307/30047167 · Cited by 70
Huang, J. (2008) · Assessing Writing · doi:10.1016/j.asw.2008.10.002 · Cited by 67
Kohn, A. (2006) · The English Journal · Cited by 62
On the Uses of Rubrics: Reframing the Great Rubric Debate
Turley, E. D., & Gallagher, C. W. (2008) · The English Journal · doi:10.58680/ej20086301 · Cited by 56
THE INFLUENCE OF HANDWRITING ON ASSESSMENT
Briggs, D. (1970) · Educational Research · doi:10.1080/0013188700130107 · Cited by 52
What Does Educational Research Say about the Judging of Writing Ability?
McColly, W. (1970) · The Journal of Educational Research · doi:10.1080/00220671.1970.10884123 · Cited by 50
Teachers' use of rubrics to score non‐traditional tasks: factors related to discrepancies in scoring
Meier, S. L., Rich, B. S., & Cady, J. (2006) · Assessment in Education Principles Policy and Practice · doi:10.1080/09695940600563512 · Cited by 49
The Effect of Selected Composition Errors on Grades Assigned to Essay Examinations
Scannell, D. P., & Marshall, J. C. (1966) · American Educational Research Journal · doi:10.2307/1161913 · Cited by 44
Malouff, J. M. (2008) · College Teaching · doi:10.3200/ctch.56.3.191-192 · Cited by 43
Using Rating Augmentation to Expand the Scale of an Analytic Rubric
Penny, J., Johnson, R. L., & Gordon, B. (2000) · The Journal of Experimental Education · doi:10.1080/00220970009600096 · Cited by 38
East, M. (2006) · Assessing Writing · doi:10.1016/j.asw.2006.11.001 · Cited by 35
Rubric Development And Inter Rater Reliability Issues In Assessing Learning Outcomes
Newell, J. A., Newell, H. L., & Dahm, K. D. (2020) · doi:10.18260/1-2--10943 · Cited by 35
The effects of attractiveness of writer and penmanship on essay grades
BULL, R., & STEVENS, J. (1979) · Journal of Occupational Psychology · doi:10.1111/j.2044-8325.1979.tb00440.x · Cited by 28
The Effect of Handwriting upon Grading
James, H. W. (1927) · The English Journal · doi:10.2307/803599 · Cited by 19
Brookhart, S. M. (2005) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2005.00007.x · Cited by 12
Research works printed but not linked: 11
- Chase, 1968, The Impact of some obvious variables on essay test scores, Journal of Educational Measurement, 5, 315, 10.1111/j.1745-3984.1968.tb00644.x · doi:10.1111/j.1745-3984.1968.tb00644.x
- Coffman, 1971, On the reliability of ratings of essay examinations in English, Research in the Teaching of English, 5, 24, 10.58680/rte197120156 · doi:10.58680/rte197120156
- Feldt, 1999, Variability in reliability coefficients and the standard error of measurement from school district to district, Applied Measurement in Education, 12, 367, 10.1207/S15324818AME1204_3 · doi:10.1207/s15324818ame1204_3
- Finson, 1998, Rubrics and their use in inclusive science, Intervention in School and Clinic, 34, 79, 10.1177/105345129803400203 · doi:10.1177/105345129803400203
- Follman, 1993, Critical thinking and verbal ability, ACEHI Journal, 19, 71
- Ross-Fisher, 2005, Developing effective success rubrics, Kappa Delta Pi, 41, 131, 10.1080/00228958.2005.10518823 · doi:10.1080/00228958.2005.10518823
- Sheppard, 1929, The effects of quality of penmanship on grades, Journal of Educational Research, 19, 102, 10.1080/00220671.1929.10879922 · doi:10.1080/00220671.1929.10879922
- Silvestri, 2006, Using rubrics to increase the reliability of assessment in health classes, International Electronic Journal of Health Education, 9, 25
- Spandel, 2006, In defense of rubrics, English Journal, 96, 19, 10.2307/30046656 · doi:10.2307/30046656
- Tomkins, 2003, Trouble comes in threes, Times Educational Supplement, 4547, 23
- Wiggins, 1994, The constant danger of sacrificing validity to reliability: Making writing assessment serve writers, Assessing Writing, 1, 129, 10.1016/1075-2935(94)90008-6 · doi:10.1016/1075-2935(94)90008-6
Cited by
The quality and effectiveness of descriptive rubrics
Brookhart, S. M., & Chen, F. (2015) · Educational Review · doi:10.1080/00131911.2014.929565 · Cited by 266
Appropriate criteria: key to effective rubrics
Brookhart, S. M. (2018) · Frontiers in Education · doi:10.3389/feduc.2018.00022 · Cited by 243
A critical review of the arguments against the use of rubrics
Panadero, E., & Jonsson, A. (2020) · Educational Research Review · doi:10.1016/j.edurev.2020.100329 · Cited by 191
Wang, Y.-J., Shang, H.-F., & Briody, P. (2013) · Computer Assisted Language Learning · doi:10.1080/09588221.2012.655300 · Cited by 185
“Writing is an important tool in evaluation where English proficiency needs to be assessed, but it also allows measurements of many other skills and subjects (Rezaei & Lovorn, 2010).”
BackgroundMethodBeyond Fairness and Consistency in Grading: The Role of Rubrics in Higher Education
Ragupathi, K., & Lee, A. (2020) · Diversity and Inclusion in Global Higher Education · doi:10.1007/978-981-15-1628-3_3 · Cited by 107
Metadata
- Authors
- Rezaei, A. R., & Lovorn, M.
- Venue
- Assessing Writing
- Publisher
- Elsevier BV
- Published
- 2010-01
- Volume
- 15
- Issue
- 1
- Pages
- 18-39
- ISSN
- 1075-2935 (ISSN-L)
- OpenAlex
- W1976511561
- Cited by
- 299
- References
- 45
- FWCI
- 31.27
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.