Methodological Concerns About the Education Value-Added Assessment System
The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Abstract
Value-added models help to evaluate the knowledge that school districts, schools, and teachers add to student learning as students progress through school. In this article, the well-known Education Value-Added Assessment System (EVAAS) is examined. The author presents a practical investigation of the methodological issues associated with the model. Specifically, she argues that, although EVAAS is probably the most sophisticated value-added model, it has flaws that must be addressed before widespread adoption. She explores in depth the shortage of external reviews and validity studies of the model, its insufficient user-friendliness, and methodological issues about missing data, regression to the mean, and student background variables. She also examines a paradigm case in which the model was used to advance unfounded assertions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition to assuming that teacher effectiveness is wholly and fully captured by students' test scores-a clear threat to substantive validity [2]”
From Measurement and Fairness (2021) · cites this paper as [2]
In the citing paper
“Many value-added models, including the EVAAS MRM, lack convergent validity [2].”
From Measurement and Fairness (2021) · cites this paper as [2]
In the citing paper
“Many school districts have used such scores to make decisions about resource distribution and even teachers' continued employment, often without any way to contest these decisions [2,3].”
From Measurement and Fairness (2021) · cites this paper as [2]
Citations
Cites
EQUALITY OF EDUCATIONAL OPPORTUNITY
Coleman, J. S. (1968) · Equity & Excellence in Education · doi:10.1080/0020486680060504 · Cited by 5,308
Equality of Educational Opportunity.
Sewell, W. H., Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., & York, R. L. (1967) · American Sociological Review · doi:10.2307/2091094 · Cited by 2,753
Standards for Educational and Psychological Testing.
García, E. J. (2017) · Revista Española de Pedagogía · doi:10.22550/2174-0909.3051 · Cited by 2,217
Handbook of Complementary Methods in Education Research
(2012) · Routledge eBooks · doi:10.4324/9780203874769 · Cited by 2,029
The relation between socioeconomic status and academic achievement.
White, K. R. (1982) · Psychological Bulletin · doi:10.1037//0033-2909.91.3.461 · Cited by 1,650
Equality of Educational Opportunity.
Pfautz, H. W., Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., & York, R. L. (1967) · American Sociological Review · doi:10.2307/2091096 · Cited by 1,278
Teacher and Classroom Context Effects on Student Achievement: Implications for Teacher Evaluation
Sanders, W. L., Wright, S. P., & Horn, S. P. (1997) · Journal of Personnel Evaluation in Education · doi:10.1023/a:1007999204543 · Cited by 1,044
Sanders, W. L., & Horn, S. P. (1998) · Journal of Personnel Evaluation in Education · doi:10.1023/a:1008067210518 · Cited by 590
Evaluating Value-Added Models for Teacher Accountability
McCaffrey, D. F., Lockwood, J. R., Koretz, D. M., & Hamilton, L. S. (2003) · PsycEXTRA Dataset · doi:10.1037/e658712010-001 · Cited by 541
Models for Value-Added Modeling of Teacher Effects
McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T. A., & Hamilton, L. (2004) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986029001067 · Cited by 513
Research works printed but not linked: 13
- Battelle for Kids, 2006, Achievement gains study of SOAR pilot and matched districts: Does SOAR make a difference? A summary of the final report to Battelle for Kids
- Braun HI, 2004, Value-added modeling: What does due diligence require?
- Lockwood JR, 2003, R News, 3, 17
- Morgan JP, 2002, Multiple choices: Testing students in Tennessee
- Northwest Evaluation Association, 2006, Achievement gaps: An examination of differences in student achievement and growth
- Ross SM, 2001, Paper presented at the annual meeting of the American Educational Research Association
- Sanders WL, 1998, The School Administrator, 55, 24
- SandersWLHow can value-added assessment lead to greater accountability?2004[Biographical sketch with article]. Report presented at First Annual Policy Conference of the New York State Educational Conference Board (Investment and Accountability for Student Success), Albany, NY. Retrieved February 10, 2007, fromhttp://www.nysecb.org/2004conference/04sanders.html
- Sanders WL, 1996, Cumulative and residual effects of teachers on future student academic achievement
- Sanders WL, 2002, Effects of building change on indicators of student achievement growth: Tennessee Value-Added Assessment System
- Stroup W, 1995, Assessment of the statistical methodology used in the Tennessee Value-Added Assessment System
- Walberg HJ, 1997, Grading teachers, grading schools: Is student achievement a valid evaluation measure?, 169
- 10.1037/0033-2909.91.3.461 · DOI · Paper page
Cited by
Neuman, S. B., & Cunningham, L. (2009) · American Educational Research Journal · doi:10.3102/0002831208328088 · Cited by 522
“Students of National Board–certified teachers, for example, make about 1 month’s greater growth than students of regularly certified teachers (Amrein-Beardsley, 2008).”
BackgroundApproaches to Evaluating Teacher Effectiveness: A Research Synthesis.
Goe, L., Bell, C. A., & Little, O. M. (2008) · Cited by 661
What Makes Great Teaching? Review of the Underpinning Research.
Coe, R., Aloisi, C., Higgins, S., & Major, L. E. (2014) · Durham Research Online (Durham University) · Cited by 645
Jacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362
“r teaching history and status in the community. If these attributes are relevant to our understanding of teacher quality, then this model lacks content validity as well. Furthermore, Amrein-Beardsley [3] demonstrates the score’s lack of construct validity|particularly the lack of convergent validity and content validi…”
BackgroundMethodA Validity Argument Approach to Evaluating Teacher Value-Added Scores
Hill, H. C., Kapitula, L., & Umland, K. (2011) · American Educational Research Journal · doi:10.3102/0002831210387916 · Cited by 269
Metadata
- Authors
- Amrein-Beardsley, A.
- Affiliations
- AUDREY AMREIN-BEARDSLEY is an assistant professor at Arizona State University, College of Teacher Education and Leadership, P.O. Box 37100, Phoenix, AZ 85069; . Her areas of interest are educational policy, research methods, and tests and assessments
- Venue
- Educational Researcher
- Published
- 2008-03
- Volume
- 37
- Issue
- 2
- Pages
- 65-75
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W2113233414
- Cited by
- 199
- References
- 48
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 7 · 2023: 5 · 2022: 3 · 2021: 8 · 2020: 19 · 2019: 8 · 2018: 14 · 2017: 14 · 2016: 17 · 2015: 9 · 2014: 18 · 2013: 19 · 2012: 18
- FWCI
- 26.20
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Researcher (publishedVersion)
- Topics
- Social Sciences › Education › Educational Assessment and Improvement
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Public Administration › Evaluation and Performance Assessment
- Keywords
- value-added models, student achievement improvement, school accountability, teacher evaluation, external validation, missing data, regression to the mean, student background, methodological critique
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methodological Concerns About the Education Value-Added Assessment System. https://app.cochat.ai/papers/10.3102/0013189x08316420
- The paper
- Amrein-Beardsley, A. (2008). Methodological Concerns About the Education Value-Added Assessment System. Educational Researcher. https://doi.org/10.3102/0013189x08316420
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.