The Measurement of Student Ability in Modern Assessment Systems
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Economists often use test scores to measure a student’s performance or an adult’s human capital. These scores reflect nontrivial decisions about how to measure and scale student achievement, with important implications for secondary analyses. For example, the scores computed in several major testing regimes, including the National Assessment of Educational Progress (NAEP), depend not only on the examinees’ responses to test items, but also on their background characteristics, including race and gender. As a consequence, if a black and white student respond identically to questions on the NAEP assessment, the reported ability for the black student will be lower than for the white student—reflecting the lower average performance of black students. This can bias many secondary analyses. Other assessments use different measurement models. This paper aims to familiarize applied economists with the construction and properties of common cognitive score measures and the implications for research using these measures.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072
Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014) · American Economic Review · doi:10.1257/aer.104.9.2633 · Cited by 1,717
Formulating, Identifying and Estimating the Technology of Cognitive and Noncognitive Skill Formation
Cunha, F., & Heckman, J. J. (2008) · Journal of Human Resources · doi:10.3368/jhr.43.4.738 · Cited by 1,468
Parametric empirical Bayes inference: theory and applications
Morris, C. N. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477920 · Cited by 1,448 · cited 1×
“In this concluding section, we discuss their implications for several of the running examples discussed throughout this article.”
The Role of Premarket Factors in Black-White Wage Differences
Neal, D. A., & Johnson, W. R. (1996) · Journal of Political Economy · doi:10.1086/262045 · Cited by 1,340
Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014) · American Economic Review · doi:10.1257/aer.104.9.2593 · Cited by 1,241
Understanding the Black-White Test Score Gap in the First Two Years of School
Fryer, R. G., & Levitt, S. D. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323031049 · Cited by 936
Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement*
Rothstein, J. (2010) · Quarterly Journal of Economics · doi:10.1162/qjec.2010.125.1.175 · Cited by 735
Self-citationThe Impact of No Child Left Behind on Student Achievement
Dee, T. S., & Jacob, B. (2011) · Journal of Policy Analysis and Management · doi:10.1002/pam.20586 · Cited by 487
Self-citationThe Black-White Test Score Gap Through Third Grade
Fryer, R. G. (2006) · American Law and Economics Review · doi:10.1093/aler/ahl003 · Cited by 422
(2010) · Quantitative Economics · doi:10.3982/qe8 · Cited by 304 · cited 1×
“These considerations motivate use of more complex performance measures, typically based on what is known as “item response theory.””
MethodTesting for Racial Differences in the Mental Ability of Young Children
Fryer, R. G., & Levitt, S. D. (2013) · American Economic Review · doi:10.1257/aer.103.2.981 · Cited by 106
Permanent Income and the Black-White Test Score Gap
Rothstein, J., & Wozny, N. (2013) · Journal of Human Resources · doi:10.3368/jhr.48.3.510 · Cited by 41
Self-citation
Cited by
PISA 2018 Results (Volume I): What Students Know and Can Do
OECD (2019) · PISA · doi:10.1787/5f07c754-en · Cited by 2,350
Blanden, J., Doepke, M., & Stuhler, J. (2022) · National Bureau of Economic Research · doi:10.3386/w29979 · Cited by 72
“Test scores are a noisy measure of achievement (Jacob and Rothstein, 2016), such that the positive coefficient on family background conditional on test scores may still reflect differences in abilities (i.”
BackgroundThe learning gain over one school year among 15-year-olds: An international comparison based on PISA
Avvisati, F., & Givord, P. (2023) · Labour Economics · doi:10.1016/j.labeco.2023.102365 · Cited by 32
Metadata
- Authors
- Jacob, B., & Rothstein, J.
- Venue
- Journal of Economic Perspectives
- Publisher
- American Economic Association
- Published
- 2016-08-01
- Volume
- 30
- Issue
- 3
- Pages
- 85-108
- ISSN
- 0895-3309 (ISSN-L)
- OpenAlex
- W2498388617
- Cited by
- 104
- References
- 13
- FWCI
- 13.14
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.