CoChat
Sign Up
Cited by 148School Choice and PerformanceSchool Leadership and Teacher PerformanceTeacher Education and Leadership Studies

Learning Job Skills from Colleagues at Work: Evidence from a Field Experiment Using Teacher Performance Data

Open access
Papay, John P., Taylor, Eric S., Tyler, John H., Laski, Mary E.·2020·American Economic Journal: Economic Policy
DOI 10.1257/pol.20170709
Open this paper in CoChat — free account

Abstract

We study a program designed to encourage learning from coworkers among school teachers. In an experiment, we document gains in job performance when high- and low-skilled teachers are paired and asked to work together on improving their skills. Pairs are matched on specific skills measured in prior evaluations. Each pair includes a target teacher who scores low in one or more of 19 skills and a partner who scores high in (many of) the target’s deficient skills. Student achievement improved 0.12 standard deviations in low-skilled teachers’ classrooms. Improvements are likely the result of target teachers learning skills from their partner. (JEL I21, J24, J45, M53)

AI snapshot

Written from the paper’s full text.

TL;DR

A randomized field experiment in Tennessee paired low-performing teachers with high-performing colleagues and encouraged them to work together on specific instructional skills. The intervention raised student achievement, especially in target teachers’ classrooms, and the pattern of results points toward peer skill learning rather than effort or resource-sharing.

Key findings

  • Students in treatment schools scored about 0.06σ higher overall in math and reading/language arts than control-school students (p = 0.08 with covariates); math gains were 0.07σ (p = 0.040) and reading/language arts gains were 0.05σ (p = 0.072).
  • Effects were concentrated among low-performing “target” teachers: their students gained 0.12σ, while partner and no-role teacher estimates were smaller and not statistically significant. With 61% target-role participation, the implied treatment-on-treated effect was about 0.20σ.
  • One year later, the target-teacher effect was 0.25σ and marginally significant (p = 0.068); Lee bounds for target teachers were 0.21σ to 0.34σ.
  • Mechanism tests favored skill learning: effects were larger when partners’ strengths matched targets’ weaknesses, and target teachers improved 0.27 teacher SD in matched observation skills. Tests for motivation/effort and shared resources/tasks did not support those mechanisms.

Methods

  • Design: 14 elementary and middle schools in a medium-sized Tennessee district were blocked into seven pairs and randomly assigned to treatment or control (7 treatment, 7 control). In treatment schools, low-performing target teachers were matched to high-performing partners using prior evaluation micro-data across 19 instructional skill areas; principals encouraged pairs to observe, discuss, and follow up on improvement.
  • Data/sample: about 3,000 students in grades 4–8 and 136 math/reading teachers during 2013–14; outcomes came from state standardized tests and teacher evaluation observation scores, with administrative records on student demographics, teacher experience, and prior value-added.
  • Analysis: school-level intent-to-treat regressions with randomization-block fixed effects and covariates; role-specific treatment interactions; mechanism tests using pair characteristics; wild cluster bootstrap-t inference because there were only 14 school clusters; Lee bounds for year-two attrition.

Limitations

  • The paper states that its mechanism tests are partial, that the three mechanisms are not mutually exclusive, and that it cannot rule out any mechanism; it especially cannot test all predictions for the effort/motivation hypothesis.
  • The paper notes that classroom observations were often conducted by principals who knew about the program, so observation-score results could be affected by differential scoring; it also notes lower survey response rates in treatment schools (about 45% vs. 66%), making attitude results cautious.
  • The paper states that year-two repeat-target estimates are endogenous, that the year-two sample fell to 96 of 136 teachers, and that control schools may have adopted similar pairing practices, which would bias effects downward. Evident from the text: the study is a single-district encouragement design with only 14 school clusters and principal discretion over participation and matching.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1257/pol.20170709

Cites

52 of 52 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

13 of 38 cited sources checked · 7 supported · 1 flagged · accuracy not yet measured

  • Interpretationcitation

    The 0.12 standard deviation gain is at least as large as the gap in performance between a novice and a teacher with five to ten years of experience.

    “A gain of 0.12σ is also at least as large as the difference in performance between a novice teacher and a 5 to 10 year veteran (Rockoff 2004, Papay and Kraft 2015).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Rockoff 2004 source

      Rockoff reports that beginning teachers and teachers with ten years of experience differ by about 0.20 standard deviations in reading scores (Introduction, p. 3; Empirical Results, p. 15). At the ten-year end of the claimed range, that gap is larger than 0.12, not no larger than it. (LLM, conf=0.86)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites Papay and Kraft 2015 source
  • Interpretationcitation

    Average treatment effects are about one-quarter as large as the estimated benefit of a 30 percent reduction in elementary class size, or of doubling middle and high school math class time.

    “Additionally, these average treatment effects are also roughly one-quarter the estimated gain from reducing class size by 30 percent in elementary grades (Kruger 1999), or the estimated gain from doubling the amount of class time middle and high school students spend in math (Taylor 2014, Cortes, Goodman, and Nomi 2015).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Kruger 1999

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Taylor 2014 source
    • cites Cortes, Goodman, and Nomi 2015 source
  • Premisecitation

    Experience on the job is a consistent source of differences in teacher performance.

    “One seemingly consistent source of differences in teacher performance is experience on the job (Rockoff 2004, Harris and Sass 2011, Papay and Kraft 2015).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Rockoff 2004 source

      The Introduction (p. 3) and Empirical Results (p. 15) report that teaching experience significantly improves reading test scores. But the Empirical Results (p. 16) find little evidence of gains in math, so this paper does not establish experience as a consistent source of performance differences across subjects. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Harris and Sass 2011 source
    • cites Papay and Kraft 2015 source
  • Premisecitation

    Estimated differences due to experience are much larger than differences in formal pre-service or in-service training.

    “Estimated differences due to experience are much larger than differences in formal pre-service or in-service training (Jackson, Rockoff, and Staiger 2014).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Jackson, Rockoff, and Staiger 2014 source

      In “Can Teacher Characteristics Explain Teacher Effects?” (page not available), the review says teacher effectiveness improves with experience while education and certification explain little variation. But “Mentoring and Feedback” (page not available) reports mixed findings for in-service training, including substantial gains from an intensive program. The paper does not establish that estimated experience differences are much larger than those from both pre-service and in-service training. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Premisecitation

    There is substantial variability in teacher job performance, measured in the short run by students' test scores and in the long run by students' economic and social success as adults.

    “And there is substantial variability in teacher job performance: measured both in the short-run with students' test scores (see Jackson, Rockoff, and Staiger 2014 for a review) and the long-run with students' economic and social success years later as adults (Chetty, Friedman, and Rockoff 2014).”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Jackson, Rockoff, and Staiger 2014 source
    • cites Chetty, Friedman, and Rockoff 2014 source
  • May combine two claims. 46
  • Classified as probably not a substantial claim. 29
  • The quote only partly states this claim. 4

  • May combine two claims.

    Interpretationcitation

    The 0.06σ average treatment effect is roughly one-third of a standard deviation in teacher performance, which is typically estimated at 0.15-0.20σ in math and somewhat smaller in reading.

    “Gains of 0.06σ represent roughly one-third of a standard deviation in teacher performance, which is typically estimated at 0.15-0.20σ in math and somewhat smaller in reading (Hanushek and Rivkin 2010, Jackson, Staiger, and Rockoff 2014).”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Hanushek and Rivkin 2010 source
    • cites Jackson, Staiger, and Rockoff 2014 source

      In “What Are Teacher Effects and How Are They Measured?” (page unavailable), the review gives a typical teacher value-added standard deviation of 0.1–0.2 student-achievement standard deviations and says teacher effects vary more in math than in reading. That makes the comparison of 0.06σ to roughly one-third of a teacher-performance standard deviation plausible, but the source does not give the more specific 0.15–0.20σ range for math. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    This context may partly explain why positive effects were found here while other research does not consistently find effects of formal mentoring or formal on-the-job training for teachers.

    “This context may partly explain why we find positive effects in this case while other research dose not consistently find effects of formal mentoring or formal on-the-job training for teachers (see reviews by Jackson, Rockoff, and Staiger 2014, and Yoon et al. 2007).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Jackson, Rockoff, and Staiger 2014 source
    • cites Yoon et al. 2007

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    Treatment gains were concentrated among target teachers; in target teachers' classrooms students scored 0.12σ higher.

    “The gains are concentrated among”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The improvements in teacher performance persisted, and perhaps grew, in the school year following treatment; in year two the effect for target teachers was a marginally significant 0.25σ.

    “These improvements in teacher performance persist, and perhaps grow, in the school year following treatment; in year two the effect for target teachers is a marginally significant 0.25σ.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    At the end of the school year, the average student in a treatment school scored 0.06σ higher on standardized math and reading/language arts tests than in a control school, regardless of whether the student's teacher participated in a partnership; the difference was marginally significant (p=0.08) with pre-treatment covariates.

    “At the end of the school year, the average student in a treatment school scores 0.06σ (student standard deviations) higher on standardized math and reading/language arts tests than she would have in a control school, regardless of whether her teacher participated in a partnership.”
    • Also stated in: § 2.1 Average Treatment Effects on Student Achievement · plain
      “The difference is marginally statistically significant (𝑝 = 0.08) when we control for pre-treatment covariates.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For teachers in non-matched grades and subjects, estimated treatment effects remain large, statistically significant, and about the same size as in Column 1.

    “Furthermore, the estimated treatment effects for teachers in non-matched grades and subjects remains large, statistically significant, and of approximately the same magnitude as in Column 1.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The estimated average treatment effect one year after treatment is 0.11 standard deviations, with a p-value of 0.22.

    “The estimated average treatment effect is also larger at 0.11σ, though the p-value is 0.22 (Column 1 Panel A).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The count of a target teacher's weak skills alone does not predict a larger treatment effect; the effect is larger only when those weak skills are matched.

    “The target teacher's number of weak skills does not itself predict a larger treatment effect; the effect is larger only when weak skills are matched.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The coefficient on the proportion of skills matched is positive and statistically significant at p = 0.052.

    “Consistent with peer learning, the coefficient is positive and statistically significant (𝑝 = 0.052).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    One year after treatment, the estimated effect for low-performing target teachers is 0.25 standard deviations, marginally significant at p = 0.068.

    “The estimated effect for low-performing target teachers is 0.25σ, and marginally significant (𝑝 = 0.068, Column 1 Panel B).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The reading estimates are more sensitive to inclusion of pre-treatment covariates, though equivalence cannot be rejected.

    “Additionally, the reading estimates are more sensitive to the inclusion of pre-treatment covariates, though we cannot reject that the reading estimates in Column 1 and 2 are equivalent statistically.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The 0.25 standard deviation improvement applies to teachers who were targets for only one year, though it is not statistically significant.

    “The 0.25σ improvement holds for teachers who were targets just one year, though it is not statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Teacher performance improvements continue, and may even increase, one year after treatment.

    “The improvements in teacher performance persist, and perhaps grow, in the year after treatment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In this setting, inference using conventional cluster-robust standard errors is quite similar.

    “However, in this setting inference using conventional cluster-robust standard errors (as implemented in Stata) turns out to be quite similar.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The less parametric specification likewise shows larger performance gains when target and partner teachers are better matched on skills, and its estimates are more precise.

    “This less-parametric approach also shows larger performance improvements when target and partner teachers are better matched on skills, and the estimates are more precise.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Partner-teacher coefficients are similar in size, but estimates for both partner and no-role teachers have very wide confidence intervals.

    “The coefficients for partner teachers are similar in magnitude, but estimates for both partner and norole teachers have very wide (implicit) confidence intervals.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The point estimate for the difference in prior-year classroom observation scores is small, negative, and not statistically significant.

    “The point estimate is small, negative, and not statistically significant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Target teachers' performance rose by 0.12 student standard deviations during the treatment year, and perhaps twice that in the following year.

    “Target teachers' performance improved 0.12 student standard deviations in the year of treatment, and perhaps double that much in the following year.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Among high-performing partner teachers, estimated treatment effects are positive, smaller than those for targets, and not statistically significant.

    “The estimates for high-performing partner teachers are positive, but much smaller and not statistically significant.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The coefficient on proportion skills matched is more positive than in Column 2, while the coefficient on observation scores is more negative than in Column 6.

    “If anything, the coefficient on proportion skills matched is more positive than in Column 2, and the coefficient on observation scores more negative than in Column 6.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    These performance gains were larger when partnerships were better matched on strengths and weaknesses, but otherwise there is little evidence of heterogeneity.

    “These performance improvements were larger when teacher partnerships were better matched on strong and weak skills, but otherwise we find little evidence of heterogeneity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Two teachers were recommended for the target role in both years.

    “Two teachers were recommended as target teachers in both years.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For target teachers with over ten years of experience, the treatment effect was 0.03 standard deviations larger, though far from statistically significant.

    “If anything, the treatment effect was 0.03σ larger for target teachers with more than ten years of experience, though the result is far from statistically significant (Appendix Table A2).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    When the proportion of skills matched is below the median, the estimated treatment effect is positive at 0.036 standard deviations (p = 0.340), but not statistically significant and similar in magnitude to estimates for partner and no-role teachers.

    “The estimated treatment effect when the proportion skills matched is below median (Column 5 Row 1) is positive (0.036σ; 𝑝 = 0.340), but not statistically significant and similar in size to the point estimates for partner and no assigned role teachers.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For low-performing target teachers, the treatment raised student test scores by about 0.12 standard deviations, a statistically significant effect.

    “Treatment * low-performing target | 0.123 | 0.133 | 0.121 | 0.128 | 0.036 | 0.124 | 0.138 | 0.124 | 0.139 | 0.112 | 0.113”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A subset of the 136 sampled teachers changed schools between the 2013-14 and 2014-15 school years.

    “Some of the 136 teachers in the sample switched schools between 2013-14 and 2014-15.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The treatment had no statistically significant effect on leaving the same school after one or two years, or on leaving Tennessee after two years.

    “All teachers 0.084 0.067 0.089 0.036 [0.184] [0.384] [0.012] [0.556]”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

    Findingnone given

    For teachers with no assigned role, the treatment effect on student test scores is not statistically significant.

    “Treatment * no assignment | 0.029 | 0.027 | 0.028 | 0.026 | 0.028 | 0.029 | 0.027 | 0.029 | 0.028 | 0.029 | 0.030”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The treatment school that did not participate received the match lists and materials like the other six, but decided not to proceed.

    “The non-participating treatment school received the partnership match lists and program materials just as the other six schools did, but chose not to move forward.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Six of the seven treatment schools participated in the Evaluation Partnership Program; one did not participate at all.

    “Only one of the seven treatment schools did not participate at all in the Evaluation Partnership Program.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Student achievement data were not used in matching teachers or in communication with teachers about partnership goals.

    “It is important to note that, while certainly used in the state's formal evaluation scores, student achievement data were not used in the matching of teachers or communication with teachers about the goals of the teacher partnerships.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    The pattern of larger effects for target teachers and persistence over time is highly suggestive of skill growth.

    “Larger effects for target teachers, effects which persist over time, are highly suggestive of skill growth, but could also result if partnering increased target teachers' motivation or effort, or provided new opportunities to share resources or tasks (Jackson and Bruegmann 2009).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Jackson and Bruegmann 2009 source

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Target and partner roles were assigned by algorithm for both treatment and control schools prior to randomization.

    “While the”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The performance improvements suggest teachers can learn job skills from colleagues, providing empirical evidence of the benefit of skilled coworkers in human capital development.

    “The performance improvements documented in this paper suggest teachers can learn job skills from their colleagues-empirical evidence of the intuitive benefit of skilled coworkers in human capital development.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The reported estimates are intent-to-treat estimates based on algorithm-assigned roles.

    “The estimates in the previous paragraph are intent-to-treat estimates based on algorithm-assigned roles.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Results are robust to assigning students to the one teacher with the highest weight when students have multiple teachers.

    “But our results are robust to assigning students to the one teacher with the highest weight.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Jackson and Bruegmann (2009) found that a teacher's productivity improves when a higher-performing colleague arrives at her school, and that the improvements persist after she no longer works with the same colleague.

    “Jackson and Bruegmann (2009) find a teacher's productivity, as measured by contribution to her students' test score growth, improves when a new higher-performing colleague arrives at her school; then, consistent with peer learning, the improvements persist after she is no longer working with the same colleague (i.e., teaching the same grade in the same school).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Jackson and Bruegmann (2009) source

  • Probably not a substantial claim: may not pass S2 (checkable).

    Study design and conductreasoning

    The rubric language describes relatively specific skills and behaviors, not vague general assessments of teaching effectiveness.

    “As the Figure 1 example suggests, the rubric language describes relatively specific skills and behaviors, not vague general assessments of teaching effectiveness.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    ITT estimates use only original assignments, not school-specific adjustments.

    “While these adjustments aided in the practical implementation of the program, our ITT estimates use only the original assignments of teachers, not the assignments after these school-specific adjustments.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Mas and Moretti (2009) report that low-performing grocery checkout clerks raise effort when a higher-performing peer works simultaneously and can observe them, while high performers are unaffected by other high performers.

    “Mas and Moretti (2009) find that low-performing grocery check-out clerks increase their work effort when a higher-performing peer works at the same time and can observe the low-performer's effort; but high-performers are not affected by other high-performers.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mas and Moretti (2009) source

  • May combine two claims.

    Premisecitation

    Both Jackson and Bruegmann (2009) and Taylor and Tyler (2012) report that performance gains persist; in Taylor and Tyler the effects grow from 0.05 standard deviations in the peer-evaluation year to 0.10 in later years.

    “Both Jackson and Bruegmann (2009) and Taylor and Tyler (2012) report that the gains in performance are sustained into the future; indeed in the latter case the effects grow from 0.05σ during the peer evaluation year to 0.10σ in the years after the peer evaluation year.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Jackson and Bruegmann (2009) source
    • cites Taylor and Tyler (2012) source

  • May combine two claims.

    Study design and conductown data

    The reported effects are school-level intent-to-treat effects and do not use variation in assigned teacher roles or treatment take-up.

    “These are school-level intent-to-treat effects, and do not use any variation in assigned teacher roles or treatment take-up.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Target teachers' gains do not necessarily replace gains from experience.

    “However, the gains for target teachers are not necessarily substituting for experience gains.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The study documents meaningful job-performance improvements among treatment teachers, and the pattern of gains suggests they learned job skills from colleagues.

    “We document meaningful improvements in job performance among treatment teachers, and the patterns of performance gains suggest teachers learned job skills from their colleagues.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    The assumption of no baseline differences rests largely on successful random assignment, with evidence in Table 2.

    “This assumption rests largely on the success of random assignment; evidence in support of successful random assignment is presented in Table 2.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All estimates include randomization block fixed effects.

    “All estimates in the paper include randomization block fixed effects, 𝜋 𝑏(𝑠) .”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The evaluation context likely influenced principals' and teachers' willingness to participate and the nature of their participation.

    “These connections to formal evaluation, and its stakes, likely influenced principals' and teachers' willingness to participate and the nature of their participation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The year-two sample contains 96 of the original 136 treatment and control teachers.

    “The sample for Table 5 Column 1 includes 96 of the original 136 treatment and control teachers.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The empirical tests are partial and the three mechanisms are not mutually exclusive, so none can be ruled out.

    “Our empirical tests are partial and the three mechanisms are not mutually exclusive, so we cannot rule out any of these mechanisms.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The relatively low-performing teachers targeted by the intervention, and ultimately their students, benefited substantially from partnering with a higher-performing colleague at their school.

    “The relatively low-performing teachers targeted by our intervention-and ultimately their students-benefited substantially from partnering with a higher-performing colleague at their school.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    The 14 schools were blocked in seven pairs based on level and enrollment, and randomly assigned one to treatment within each pair.

    “We blocked the 14 schools in seven pairs based on level (elementary or middle) and student enrollment, and randomly assigned one to treatment within each pair.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The positive average treatment effects are educationally and economically meaningful.

    “These positive average treatment effects are educationally and economically meaningful.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    Treatment probably raised interaction with one colleague, but the composition of coworkers and usual coworker interactions did not differ between treatment and control schools.

    “the treatment likely increased the degree of interaction with one coworker, but the mix of coworkers and typical coworker interactions were the same in treatment and control schools.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Column 4 offers evidence consistent with target teachers' job-performance gains continuing, and possibly growing, after treatment ends.

    “Column 4 does provide evidence consistent with improvements in job performance for target teachers persist, perhaps grow, after treatment ends.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The pattern of larger math effects and smaller reading effects is consistent with typical schooling research.

    “This is consistent with the typical pattern in empirical research on schooling: most general interventions affect reading achievement less than math achievement.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    Performance growth among experienced teachers is rare in the literature, but this result aligns with Taylor and Tyler (2012).

    “Performance growth for experienced teachers is rare in the literature, but this result is consistent with results in Taylor and Tyler (2012).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Taylor and Tyler (2012) source

  • May combine two claims.

    Interpretationreasoning

    The persistence of target teachers' performance gains is robust to attrition, partly perhaps because attrition was fairly balanced.

    “In short, the continued performance improvement among target teachers is robust to attrition, in part, perhaps, because attrition was relatively well balanced.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Cameron and coauthors show conventional clustering can have rejection rates up to 10 percent for a nominal 0.05 test, while wild cluster bootstrap-t approaches 5 percent.

    “Cameron and coauthors show that rejection rates can be as high as 10 percent for a nominally 𝛼 = 0.05 test using conventional clustering methods; using the wild cluster bootstrap-t method rejection rates approach 5 percent.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Cameron, Gelbach, and Miller (2008) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Taylor and Tyler (2012) found that veteran teachers evaluated by and receiving feedback from experienced high-performing teachers had productivity improvements that persisted for years after peer evaluation ended.

    “Taylor and Tyler (2012) study veteran teachers who were evaluated by and received feedback from experienced, high-performing teachers; the resulting improvements in teacher productivity persisted for years after the peer evaluation ended.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Taylor and Tyler (2012) source

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The links to formal evaluation probably shaped both principals' and teachers' decisions to participate and how they participated.

    “These connections to formal evaluation likely influenced principals' and teachers' willingness to participate, and the nature of their participation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    The teacher value-added scores are expressed in teacher standard-deviation units rather than student standard-deviation units.

    “Note that the teacher value-added scores are in teacher standard deviation units, not student standard deviations as is often the case.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The one-on-one personalized on-the-job training studied here appears much more successful and much less costly.

    “By contrast, the one-on-one personalized approach to on-the-job training we study in this paper is apparently much more successful and much less costly.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Study design and conductown data

    Survey response rates were lower in treatment schools, about 45 percent.

    “survey response rates were lower in treatment schools”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Because principals perform observations, they possessed all information needed to form such pairs.

    “Because principals conduct the observations, they had on hand all of the information necessary to conduct such pairings.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The cost is low partly because the high-performing colleague supplies the teaching expertise.

    “The cost is low in part because the high-performing coworker provides the teaching expertise.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The opportunity cost of that time matters, though it could plausibly replace time teachers would otherwise spend in formal professional development.

    “The opportunity cost of that time is important, although it could plausibly substitute for the time teachers would have spent in formal professional development courses.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisenone given

    Every teacher in the study, regardless of treatment or role, was covered by Tennessee's new formal evaluation system.

    “All teachers in our study-treatment and control, target and partner and no role-are subject to Tennessee's new formal performance evaluation system.”
    Quote in the paperSubstantial claim
67 of 67

  • The authors state this as an assumption.

    At the beginning of the experiment, there was no difference in potential outcomes between treatment and control samples.

    “At the beginning of the experiment, there was no difference in potential outcomes-student achievement growth, teacher or school performance, etc.-between treatment and control samples.”

  • The authors state this as an assumption.

    Comparison of performance between paired teachers induces greater effort.

    “on the assumption that the comparison of performance induces greater effort (as in Mas and Moretti 2009).”

  • The authors state this as an assumption.

    Even without treatment, shared production activities are easier or yield higher returns when teachers teach the same grade level or subject.

    “The assumption motivating this test is that, even absent the treatment, shared production activities are easier or higher-return when teachers teach the same grade-level or subject.”
3 of 3

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Lee bounds method is assigned two different years

    Minor

    The discussion attributes the bounds approach to Lee (2011), while the note to the bounds table attributes it to Lee (2009).

  • Lucas citation has conflicting publication years

    Minor

    The paper cites Lucas as a 1988 work, but the corresponding reference lists its year as 2988.

2 of 2

Cites

Flagged first, then by citations

2 of the 54 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Papay, J. P., Taylor, E. S., Tyler, J. H., & Laski, M. E.
Affiliations
  • Brown University (US)
  • Harvard University (US)
  • National Bureau of Economic Research (US)
ORCID
Venue
American Economic Journal: Economic Policy
Publisher
American Economic Association
Published
2020-02-01
Volume
12
Issue
1
Pages
359-388
ISSN
  • 1945-7731 (ISSN-L)
  • 1945-774X
DOI
10.1257/pol.20170709
OpenAlex
W2276452652
Cited by
148
References
54
Citations by year
2026: 5 · 2025: 13 · 2024: 11 · 2023: 23 · 2022: 18 · 2021: 19 · 2020: 17 · 2019: 16 · 2018: 11 · 2017: 6 · 2016: 8 · 2015: 1
FWCI
60.67
Type
journal article
Language
en
Open access
Locations
  • American Economic Journal Economic Policy (publishedVersion)
  • National Bureau of Economic Research (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • Bill and Melinda Gates Foundation
  • Tennessee Department of Education
  • Harvard Graduate School of Education
Topics
  • Social Sciences › Education › School Choice and Performance
  • Social Sciences › Education › School Leadership and Teacher Performance
  • Social Sciences › Education › Teacher Education and Leadership Studies
Keywords
teacher performance, peer learning, academic performance, teacher professional development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review