CoChat
Sign Up
Cited by 57School Choice and PerformanceEducation, Achievement, and GiftednessEducational Assessment and Improvement

Using Value-Added Models to Measure Teacher Effects on Students’ Motivation and Achievement

Open access
Ruzek, Erik A., Domina, Thurston, Conley, AnneMarie M., Duncan, Greg J., Karabenick, Stuart A.·2015·The Journal of Early Adolescence
DOI 10.1177/0272431614525260
Open this paper in CoChat — free accountOpen PDF

Abstract

Value-added (VA) models measure teacher contributions to student learning and are increasingly employed in educational reform efforts. Using data from 35 seventh-grade teachers and 2,026 students across seven schools, we employ VA methods to measure teacher contributions to students’ motivational orientations (mastery and performance achievement goals) and their mathematics performance. The analysis suggests less variation in teachers’ contributions to students’ achievement goals than mathematics achievement. However, during a time when most students’ mastery motivation is declining sharply, a one standard deviation increase in teacher contributions to student mastery orientation is associated with a 40% smaller decline in student mastery goals. Teacher mastery contributions are also associated with gains in a student’s seventh-grade mathematics achievement ( d = .11). In addition to using VA measures to focus on improving student achievement, these measures can be used to orient teachers, schools, and districts on the enhancement of students’ motivation to learn.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/0272431614525260

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Ruzek et al. (2015) estimated a correlation of 0.50 between teacher effects on achievement versus effects on students' motivation in math class.

    “Ruzek et al. (2015) estimated a correlation of 0.50 between teacher effects on achievement versus effects on students’ motivation in math class.”

    From Teacher and Teaching Effects on Students’ Attitudes and Behaviors (2017) · cites this paper as Ruzek et al. (2015)

    In Changes in Student Motivation (Research Question 2), the paper reports a correlation of r = .50 between teacher contributions to seventh-grade mathematics achievement and to students’ mastery goals. The claim calls this an effect on students’ motivation generally, but the .50 estimate applies specifically to mastery goals, not the other motivation measures reported in Table 4. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Our findings align with previous studies that identify teacher effects on students' attitudes and behaviors and weak relationships between different measures of teacher effectiveness.

    “In many ways, our findings align with conclusions drawn from previous studies that also identify teacher effects on students’ attitudes and behaviors (Jennings & DiPrete, 2010; Kraft & Grace, 2016; Ruzek et al., 2015), as well as weak relationships between different measures of teacher effectiveness (Gershenson, 2016; Jackson, 2012; Kane & Staiger, 2012).”

    From Teacher and Teaching Effects on Students’ Attitudes and Behaviors (2017) · cites this paper as Ruzek et al., 2015

    The Present Study and Changes in Student Motivation sections report teacher-associated changes in seventh-graders’ achievement goals, supporting the point about students’ attitudes or motivation. The paper does not establish teacher effects on students’ behaviors; it describes achievement goals as associated with learning behaviors. The sentence cites other papers, not Ruzek et al., for weak relationships between measures of teacher effectiveness. Page numbers are unavailable in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Ruzek et al. (2015) found small but meaningful teacher effects on students' motivation between 0.03 sd and 0.08 sd among seventh graders.

    “In a study of 35 middle school math teachers, Ruzek et al. (2015) found small but meaningful teacher effects on students’ motivation between 0.03 sd and 0.08 sd among seventh graders.”

    From Teacher and Teaching Effects on Students’ Attitudes and Behaviors (2017) · cites this paper as Ruzek et al. (2015)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 63

  1. Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the Chicago public high schools. Journal of Labor Economics, 25, 95-135. doi:10.1086/508733 · DOI
  2. Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, 261-271. doi:10.1037//0022-0663.84.3.261 · DOI · Paper page
  3. Ames, C., & Ames, R. (1984). Systems of student and teacher motivation: Toward a qualitative definition. Journal of Educational Psychology, 76, 535-556. doi:10.1037//0022-0663.76.4.535 · DOI
  4. Ames, C., & Archer, J. (1988). Achievement goals in the classroom: Students' learn- ing strategies and motivation processes. Journal of Educational Psychology, 80, 260-267. doi:10.1037//0022-0663.80.3.260 · DOI
  5. Anderman, E. M., & Maehr, M. L. (1994). Motivation and schooling in the middle grades. Review of Educational Research, 64, 287-309. doi:10.3102/00346543064002287 · DOI
  6. Anderman, E. M., Maehr, M. L., & Midgley, C. (1999). Declining motivation after the transition to middle school: Schools can make a difference. Journal of Research and Development in Education, 32, 131-147.
  7. Anderman, E. M., & Midgley, C. (1997). Changes in achievement goal orientations, perceived academic competence, and grades across the transition to middle-level schools. Contemporary Educational Psychology, 22, 269-298. doi:10.1006/ ceps.1996.0926 · DOI
  8. Anderman, E. M., & Young, A. J. (1994). Motivation and strategy use in science: Individual differences and classroom effects. Journal of Research in Science Teaching, 31, 811-831. doi:10.1002/tea.3660310805 · DOI
  9. Anderman, L. H., & Anderman, E. M. (1999). Social predictors of changes in stu- dents' achievement goal orientations. Contemporary Educational Psychology 25, 21-37. doi:10.1006/ceps.1998.0978 · DOI
  10. Blyth, D. A., Simmons, R. G., & Carlton-Ford, S. (1983). The adjustment of early adolescents to school transitions. Journal of Early Adolescence, 3, 105-120. doi:10.1177/027243168331008 · DOI
  11. Bong, M. (2005). Within-grade changes in Korean girls' motivation and perceptions of the learning environment across domains and achievement levels. Journal of Educational Psychology, 97, 656-672. doi:10.1037/0022-0663.97.4.656 · DOI
  12. Bowen, W. G., Chingos, M. M., & McPherson, M. S. (2009). Crossing the finish line: Completing college at America's public universities. Princeton, NJ: Princeton University Press.
  13. Boyd, D. J., Grossman, P. L., Lankford, H., Loeb, S., & Wyckoff, J. (2009). Teacher preparation and student achievement. Education Evaluation and Policy Analysis, 31, 416-440. doi:10.3102/0162373709353129 · DOI · Paper page
  14. Braun, H., Chudowsky, N., & Koenig, J. (2010). Getting value out of value-added: Report of a workshop. Washington, DC: National Academies Press.
  15. Brophy, J. (2005). Goal theorists should move on from performance goals. Educational Psychologist, 40, 167-176. doi:10.1207/s15326985ep4003_3 · DOI
  16. Buddin, R., & Zamarro, G. (2009). Teacher qualifications and student achieve- ment in urban elementary schools. Journal of Urban Economics, 66, 103-115. doi:10.1016/j.jue.2009.05.001 · DOI
  17. Byrnes, V., & Ruby, A. (2007). Comparing achievement between K-8 and middle schools: A large-scale empirical study. American Journal of Education, 114, 101-135. doi:10.1086/520693 · DOI
  18. Carolan, B. V., Weiss, C. C., & Matthews, J. S. (2013). Which middle school model works best? Evidence from the Early Childhood Longitudinal Study. Youth & Society. doi:10.1177/0044118X13478625 · DOI
  19. Conley, A. M. (2012). Patterns of motivation beliefs: Combining achievement goal and expectancy-value perspectives. Journal of Educational Psychology, 104, 32- 47. doi:10.1037/a0026042 · DOI
  20. Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., . . .Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43, 1428-1446. doi:10.1037/0012-1649.43.6.1428 · DOI · Paper page
  21. Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motiva- tion and personality. Psychological Review, 95, 256-273. doi:10.1037//0033- 295X.95.2.256 · DOI · Paper page
  22. Eccles, J. S., & Midgley, C. (1989). Stage-environment fit: Developmentally appro- priate classrooms for young adolescents. In C. Ames & R. Ames (Eds.), Research on motivation in education: Goals and cognitions (Vol. 3, pp. 139-186). New York, NY: Academic Press.
  23. Eccles, J. S., Midgley, C., Wigfield, A., Buchanan, C. M., Reuman, D., Flanagan, C., & Mac Iver, D. (1993). Development during adolescence: The impact of stage-environment fit on young adolescents' experiences in schools and families. American Psychologist, 48, 90-101. doi:10.1037//0003-066X.48.2.90 · DOI
  24. Elliot, A. J., & Harackiewicz, J. M. (1996). Approach and avoidance achievement goals and intrinsic motivation: A mediational analysis. Journal of Personality and Social Psychology, 70, 461-475. doi:10.1037/0022-3514.70.3.461 · DOI
  25. Farrington, C. A., Roderick, M., Allensworth, E., Nagaoka, J., Keyes, T. S., Johnson, D. W., & Beechum, N. O. (2012). Teaching adolescents to become learners. The role of noncognitive factors in shaping school performance: A critical litera- ture review. Chicago, IL: University of Chicago Consortium on Chicago School Research.
  26. Gottfried, A. E., Fleming, J. S., & Gottfried, A. W. (2001). Continuity of academic intrinsic motivation from childhood through late adolescence: A longitudinal study. Journal of Educational Psychology, 93, 3-13.
  27. Gottfried, A. E., Marcoulides, G. A., Gottfried, A. W., Oliver, P. H., & Guerin, D. W. (2007). Multivariate latent change modeling of developmental decline in academic intrinsic math motivation and achievement: Childhood through adolescence. International Journal of Behavioral Development, 31, 317-327. doi:10.1177/0165025407077752 · DOI
  28. Grossman, P., Loeb, S., Cohen, J., Hammerness, K., Wyckoff, J., Boyd, D., & Lankford, H. (2010). Measure for measure: The relationship between measures of instructional practice in middle school English language arts and teach- ers' value-added scores (NBER Working Paper No. 16015). Cambridge, MA: National Bureau of Economic Research.
  29. Harackiewicz, J. M., Barron, K. E., Pintrich, P. R., Elliot, A. J., & Thrash, T. M. (2002). Revision of achievement goal theory: Necessary and illuminating. Journal of Educational Psychology, 94, 638-645. doi:10.1037//0022-0663.94.3.638 · DOI
  30. Harris, D. N., & Sass, T. R. (2011). Teacher training, teacher quality and student achievement. Journal of Public Economics, 95(7-8), 798-812. doi:10.1016/j. jpubeco.2010.11.009 at UNIV CALIFORNIA IRVINE on August 4, 2015 jea.sagepub.com Downloaded from · DOI
  31. Hill, H. C., Kapitula, L., & Umland, K. (2010). A validity argument approach to evaluating teacher value-added scores. American Educational Research Journal, 48, 1-38. doi:10.3102/0002831210387916 · DOI
  32. Jacob, B. A., & Lefgren, L. (2008). Can principals identify effective teachers? Evidence on subjective performance evaluation in education. Journal of Labor Economics, 26, 101-136. doi:10.1086/522974 · DOI · Paper page
  33. Jennings, J. L., & DiPrete, T. A. (2010). Teacher effects on social and behav- ioral skills in early elementary school. Sociology of Education, 83, 135-159. doi:10.1177/0038040710368011 · DOI · Paper page
  34. Kaplan, A., & Maehr, M. L. (2007). The contributions and prospects of goal orienta- tion theory. Educational Psychology Review, 19, 141-184. doi:10.1007/s10648- 006-9012-5 · DOI
  35. Karabenick, S. A. (2003). Seeking help in large college classes: A person-centered approach. Contemporary Educational Psychology, 28, 37-58. doi:10.1016/ S0361-476X(02)00012-7 · DOI · Paper page
  36. Keys, T. D., Conley, A. M., Duncan, G. J., & Domina, T. D. (2012). The role of goal o orientations for adolescent mathematics achievement. Contemporary Educational Psychology, 37, 47-54. doi:10.1016/j.cedpsych.2011.09.002. · DOI
  37. Larson, R. (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170-183. doi:10.1037//0003-066X.55.1.170 · DOI
  38. Lau, S., & Nie, Y. (2008). Interplay between personal goals and classroom goal struc- tures in predicting student outcomes: A multilevel analysis of person-context interactions. Journal of Educational Psychology, 100, 15-29. doi:10.1037/0022- 0663.100.1.15 · DOI
  39. Linnenbrink, E. A. (2005). The dilemma of performance-approach goals: The use of multiple goal contexts to promote students' motivation and learning. Journal of Educational Psychology, 97, 197-213. doi:10.1037/0022-0663.97.2.197 · DOI
  40. Linnenbrink, E. A., & Pintrich, P. R. (2002). Motivation as an enabler for academic success. School Psychology Review, 31, 313-327.
  41. Meece, J. L., Anderman, E. M., & Anderman, L. H. (2006). Classroom goal structure, student motivation, and academic achievement. Annual Review of Psychology, 57, 487-503. doi:10.1146/annurev.psych.56.091103.070258 · DOI · Paper page
  42. Midgley, C., Kaplan, A., & Middleton, M. (2001). Performance-approach goals: Good for what, for whom, under what circumstances, and at what cost? Journal of Educational Psychology, 93, 77-86. doi:10.1037//0022-0663.93.1.77 · DOI
  43. Murayama, K., & Elliot, A. J. (2009). The joint influence of personal achievement goals and classroom goal structures on achievement-related outcomes. Journal of Instructional Psychology, 101, 432-447. doi:10.1037/a0014221 · DOI
  44. Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, 328-346. doi:10.1037//0033-295X.91.3.328 · DOI · Paper page
  45. Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237-257. doi:10.3102/01623737026003237 · DOI
  46. Pastor, D. A., Barron, K. E., Miller, B. J., & Davis, S. L. (2007). A latent profile analy- sis of college students' achievement goal orientation. Contemporary Educational Psychology, 32, 8-47. doi:10.1016/j.cedpsych.2006.10.003 · DOI
  47. Patrick, H. (2004). Re-examining classroom mastery goal structure. In P. R. Pintrich & M. L. Maehr (Eds.), Advances in motivation: Vol. 13. Motivating students, improving schools: The legacy of Carol Midgley (pp. 233-263). Amsterdam, The Netherlands: Elsevier.
  48. Patrick, H., Ryan, A. M., & Kaplan, A. (2007). Early adolescents' perceptions of the classroom social environment, motivational beliefs, and engagement. Journal of Educational Psychology, 99, 83-98. doi:10.1037/0022-0663.99.1.83 · DOI
  49. In M. Shinn (Ed.), Toward positive youth development: Transforming schools and community programs (pp. 21-39). New York, NY: Oxford University Press.
  50. Pintrich, P. R. (2003). A motivational science perspective on the role of student moti- vation in learning and teaching contexts. Journal of Educational Psychology, 95, 667-686. doi:10.1037/0022-0663.95.4.667 · DOI
  51. Rabe-Hesketh, S., & Skrondal, A. (2008). Multilevel and longitudinal modeling using Stata. Austin, TX: Stata Press.
  52. Raphael, L. M., Pressley, M., & Mohan, L. (2008). Engaging instruction in middle school classrooms: An observational study of nine teachers. The Elementary School Journal, 109, 61-81. doi:10.1086/592367 · DOI
  53. Rockoff, J. E. (2004). The impact of individual teachers on student achieve- ment: Evidence from panel data. American Economic Review, 94, 247-252. doi:10.1257/0002828041302244 · DOI · Paper page
  54. Rockoff, J. E., & Lockwood, B. B. (2010). Stuck in the middle: Impacts of grade configuration in public schools. Journal of Public Economics, 94, 1051-1061. doi:10.1016/j.jpubeco.2010.06.017 · DOI
  55. Roeser, R. W., Eccles, J. S., & Sameroff, A. J. (1998). Academic and emotional func- tioning in early adolescence: Longitudinal relations, patterns, and prediction by experience in middle school. Development and Psychopathology, 10, 321-352. doi:10.1017/S0954579498001631 · DOI
  56. Roeser, R. W., Midgley, C., & Urdan, T. C. (1996). Perceptions of the school psy- chological environment and early adolescents' psychological and behavioral functioning in school: The mediating role of goals and belonging. Journal of Educational Psychology, 88, 408-422. doi:10.1037//0022-0663.88.3.408 · DOI
  57. Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. The Quarterly Journal of Economics, 125, 175-214. doi:10.1162/qjec.2010.125.1.175 · DOI · Paper page
  58. Ryan, A. M., & Patrick, H. (2001). The classroom social environment and changes in adolescent motivation and engagement during middle school. American Educational Research Journal, 38, 437-460. doi:10.3102/00028312038002437 · DOI · Paper page
  59. Schwartz, A. E., Stiefel, L., Rubenstein, R., & Zabel, J. (2011). The path not taken: How does school organization affect eighth-grade achievement? Educational Evaluation and Policy Analysis, 33, 293-317. doi:10.3102/0162373711407062 · DOI
  60. Senko, C., Hulleman, C. S., & Harackiewicz, J. M. (2011). Achievement goal the- ory at the crossroads: Old controversies, current challenges, and new directions. Educational Psychologist, 46, 26-47. doi:10.1080/00461520.2012.538646 · DOI
  61. Shim, S. S., Ryan, A. M., & Anderson, C. J. (2008). Achievement goals and achieve- ment during early adolescence: Examining time-varying predictor and outcome variables in growth-curve analysis. Journal of Educational Psychology, 100(3), 655-671. doi: 10.1037/0022-0663.100.3.65 · DOI
  62. Weiss, C. C., & Kipnes, L. (2006). Reexamining middle school effects: A comparison of middle grades students in middle schools and K-8 schools. American Journal of Education, 112, 239-272. doi:10.1086/498996 · DOI
  63. Wolters, C. A. (2004). Advancing achievement goal theory: Using goal structures and goal orientations to predict students' motivation, cognition, and achieve- ment. Journal of Educational Psychology, 96, 236-250. doi:10.1037/0022- 0663.96.2.236 · DOI

Cited by

Sorted by citations

Authors
Ruzek, E. A., Domina, T., Conley, A. M., Duncan, G. J., & Karabenick, S. A.
Affiliations
  • University of Virginia (US)
  • University of California, Irvine (US)
  • University of Michigan (US)
ORCID
Venue
The Journal of Early Adolescence
Publisher
SAGE Publications
Published
2014-03-12
Volume
35
Issue
5-6
Pages
852-882
ISSN
  • 0272-4316 (ISSN-L)
  • 1552-5449
DOI
10.1177/0272431614525260
OpenAlex
W2164537090
Cited by
57
References
2
Citations by year
2026: 1 · 2025: 3 · 2024: 3 · 2023: 3 · 2022: 6 · 2021: 7 · 2020: 7 · 2019: 12 · 2018: 1 · 2017: 5 · 2016: 7 · 2015: 2
FWCI
24.00
Type
journal article
Language
en
Open access
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
  • The Journal of Early Adolescence (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion)
Funders
National Science Foundation (0335369, 0928103)
Topics
  • Social Sciences › Education › School Choice and Performance
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Education, Achievement, and Giftedness
  • Social Sciences › Decision Sciences › Information Systems and Management › Educational Assessment and Improvement
Keywords
value-added models, teacher effects, student motivation, mastery goals, performance goals, mathematics achievement, seventh grade students, early adolescence, educational reform

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review