The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Calibrated Self-Assessment
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“a recently proposed game-theoretic mechanism for incentivizing students to accurately self-assess their work (Labutov & Studer, 2016) can inherently lead to artificial deflation of lower-performing students' grades.”
From Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education? (2021) · cites this paper as Labutov & Studer, 2016
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Peer and self assessment in massive online classes
Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., Koller, D., & Klemmer, S. R. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2505057 · Cited by 390
Research works printed but not linked: 16
- J. E. Bickel. Scoring rules and decision analysis education. Decision Analysis, 7(4):346-357, 2010.
- G. W. Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
- T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-378, 2007.
- G. W. Harrison, J. Martínez-Correa, and J. T. Swarthout. Inducing risk neutral preferences with binary lotteries: A reconsideration. Journal of Economic Behavior & Organization, 94:145-159, 2013.
- A. S. Lan, A. E. Waters, C. Studer, and R. G. Baraniuk. Sparse factor analysis for learning and content analytics. The Journal of Machine Learning Research, 15(1):1959-2008, 2014.
- J. Park and K. Williams. The effects of peer-and self-assessment on the assessors. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education, pages 249-254. ACM, 2016.
- A. Peysakhovich and M. Plagborg-Møller. Proper scoring rules and risk aversion. Available at SSRN 2019078, 2012.
- C. Piech, J. Huang, Z. Chen, C. Do, A. Ng, and D. Koller. Tuned models of peer assessment in moocs. arXiv preprint arXiv:1307.2579, 2013. · arXiv
- K. Raman and T. Joachims. Methods for ordinal peer grading. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1037-1046. ACM, 2014.
- P. M. Sadler and E. Good. The impact of self-and peer-grading on student learning. Educational assessment, 11(1):1-31, 2006.
- L. J. Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66(336):783-801, 1971.
- R. Selten. Axiomatic characterization of the quadratic scoring rule. Experimental Economics, 1(1):43-62, 1998.
- R. Selten, A. Sadrieh, and K. Abbink. Money does not induce risk neutral behavior, but binary lotteries do even worse. Theory and Decision, 46(3):213-252, 1999.
- B. Strong, M. Davis, and V. Hawks. Self-grading in large general education classes: A case study. College Teaching, 52(2):52-57, 2004.
- K. Topping. Self and peer assessment in school and university: Reliability, validity and utility. In Optimising new modes of assessment: In search of qualities and standards, pages 55-87. Springer, 2003.
- C. Zhu, R. H. Byrd, P. Lu, and J. Nocedal. Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software (TOMS), 23(4):550-560, 1997.
Cited by
Sorted by citations
Holstein, K., & Doroudi, S. (2021) · arXiv · doi:10.48550/arxiv.2104.12920 · Cited by 29
Metadata
- Authors
- Labutov, I., & Studer, C.
- Venue
- ETH Zurich
- Publisher
- ETH Zurich
- Published
- 2016
- References
- 1
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
