The Impacts of Virtual Charter High Schools on Secondary and Postsecondary Outcomes: Opening Up the Black Box
Open accessAbstract
As demand for virtual schools continues to grow in the wake of the COVID-19 pandemic, there is a pressing need to understand how these schools influence long-term student outcomes. Utilizing administrative data from Indiana, this article estimates the impacts of virtual charter high schools on high school graduation, dropout, and college enrollment and contrasts these results to the brick-and-mortar charter high schools in the state. Using a matching cell fixed-effect design, we find that students who switched from a traditional public school in eighth grade into a virtual charter high school experienced large, negative impacts across all these outcomes. A mediation analysis found that variation in class size and school-level course-taking patterns explain a substantial portion of these impacts.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
3 of 31 cited sources checked · 2 supported · 1 flagged · accuracy not yet measured
- Study design and conductown data
The study used an ordinary least squares model controlling for baseline achievement to net out residual baseline differences not addressed by matching.
“In our analysis described in the following, we net out any small remaining differences on test scores and a host of other baseline covariates not part of the matching process by employing an ordinary least squares model controlling for baseline achievement.”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Angrist et al., 2016 source
The Empirical Framework describes a lottery-based two-stage least squares (2SLS) analysis, with risk-set, year, and baseline demographic controls—not an ordinary least squares model used to correct differences left by matching. It also reports that baseline differences between offered and nonoffered students were small and statistically insignificant. The quoted citing sentence describes the citing authors’ own analysis rather than a method used in this source. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured - cites Cordes, 2024 source
- cites Dobbie & Fryer, 2013, 2020
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- The authors state this as an assumption: Matching cell fixed effects account for unobserved, time-invariant differences between cells.
- Premisecitation
Charter school authorizers have been hesitant to shut down popular virtual charter schools.
“Thus far, it appears that charter school authorizers are hesitant to shut down popular virtual charter schools (Ferrare et al., 2023)”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Ferrare et al., 2023 source
- Findingown data
Virtual charter high school attendance was associated with 4-year college attendance of 30.3%, compared with 39.5% for matched traditional public school peers.
“lower 4-year college attendance rates (30.3% to 39.5%)”
Quote in the paperSubstantial claim - Findingown data
For students in public schools at baseline, virtual charter high school attendance was associated with a 4-year graduation rate of 76.9%, versus 89.0% for matched traditional public school peers.
“Students who attended a virtual charter high school have much lower 4-year graduation rates than their matched TPS peers (76.9% to 89.0%)”
Quote in the paperSubstantial claim - Findingown data
For students in Indiana, attending a virtual charter high school rather than a traditional public high school had a large adverse effect on college enrollment.
“Using a matching cell fixed-effect design, we find that students who switched from a traditional public school in eighth grade into a virtual charter high school experienced large, negative impacts across all these outcomes.”
- Also stated in: § Abstract · plain
“Our results show that Indiana's virtual charter high schools have large, negative impacts on high school graduation, dropout, and college enrollment, whereas the brick-and-mortar charter high schools generally produce null outcomes.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The matching characteristics used (sex, race/ethnicity, baseline achievement, and chronic absenteeism or disciplinary incidents) sufficiently explain selection into virtual or brick-and-mortar charter schools.
- Rests on an assumption: Indiana virtual charter schools do not restrict enrollment capacity and do not use enrollment lotteries.
- The authors state this as an assumption: Matching cell fixed effects account for unobserved, time-invariant differences between cells.
- Also stated in: § Abstract · plain
- Findingown data
In intent-to-treat estimates, virtual charter high school students were 6.4 percentage points less likely to attend any college than matched traditional public school peers.
“These students were also FIGURE 1. Depiction of mediation analysis of charter effects on attainment outcomes.6.4 percentage points less likely to attend any college in general and 8.0 percentage points less likely to attend a 4-year college, in particular.”
Quote in the paperSubstantial claim - Findingown data
Class size accounts for roughly three-quarters of the adverse effects of virtual charter schools on high school dropout.
“Meanwhile, we find that class size mediates about three-quarters of the negative virtual impacts on high school dropout.”
Quote in the paperSubstantial claim- Rests on an assumption: If virtual charter attendance affects high school graduation, it does so largely through the unique contexts and practices of those schools.
- Findingown data
All attainment outcomes for brick-and-mortar charter high school students were null in the intent-to-treat estimates.
“By contrast, all attainment outcomes among the brickand-mortar charter high school students were null.”
Quote in the paperSubstantial claim - Findingown data
For students in Indiana, attending a virtual charter high school rather than a traditional public high school had a large adverse effect on graduating high school within four years.
“Using a matching cell fixed-effect design, we find that students who switched from a traditional public school in eighth grade into a virtual charter high school experienced large, negative impacts across all these outcomes.”
- Also stated in: § Abstract · plain
“Our results show that Indiana's virtual charter high schools have large, negative impacts on high school graduation, dropout, and college enrollment, whereas the brick-and-mortar charter high schools generally produce null outcomes.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The matching characteristics used (sex, race/ethnicity, baseline achievement, and chronic absenteeism or disciplinary incidents) sufficiently explain selection into virtual or brick-and-mortar charter schools.
- Rests on an assumption: Indiana virtual charter schools do not restrict enrollment capacity and do not use enrollment lotteries.
- The authors state this as an assumption: Matching cell fixed effects account for unobserved, time-invariant differences between cells.
- Also stated in: § Abstract · plain
- Findingnone given
The median class size in brick-and-mortar charter high schools was 20.
“The median class size in brick-and-mortar charters is 20.”
Quote in the paperNot a substantial claim - Findingown data
Mean curricular intensity was 0.45 standard deviations lower in virtual charter high schools than in comparison traditional public schools.
“The mean curricular intensity is nearly a half (-0.45) a standard deviation lower in virtual charters.”
Quote in the paperSubstantial claim - Findingown data
Virtual charter high school attendance was associated with any-college attendance of 51.3%, compared with 63.3% for matched traditional public school peers.
“lower college attendance rates at any college (51.3% to 63.3%)”
Quote in the paperSubstantial claim - Findingown data
Descriptive comparisons showed no statistically significant differences between brick-and-mortar charter high school students and their matched traditional public school peers.
“Meanwhile, there are no statistically significant descriptive differences between brick-and-mortar charter high school students and their matched TPS peers.”
Quote in the paperSubstantial claim - Findingown data
Brick-and-mortar charter high schools were slightly less likely than comparison traditional public schools to have students enrolled in calculus (77% vs. 96% of schools).
“but were slightly less likely to have students enrolled in a calculus course (77% v. 96% of schools, respectively).”
Quote in the paperNot a substantial claim - Findingown data
Mediation of the any-college enrollment effect was smaller, ranging from 20.3% to 39.0%.
“The mediation of the effects on any college enrollment was smaller in magnitude, ranging from 20.3% to 39.0%.”
Quote in the paperSubstantial claim - Findingown data
For students in Indiana, attending a virtual charter high school rather than a traditional public high school had a large adverse effect on the high school dropout outcome.
“Using a matching cell fixed-effect design, we find that students who switched from a traditional public school in eighth grade into a virtual charter high school experienced large, negative impacts across all these outcomes.”
- Also stated in: § Abstract · plain
“Our results show that Indiana's virtual charter high schools have large, negative impacts on high school graduation, dropout, and college enrollment, whereas the brick-and-mortar charter high schools generally produce null outcomes.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The matching characteristics used (sex, race/ethnicity, baseline achievement, and chronic absenteeism or disciplinary incidents) sufficiently explain selection into virtual or brick-and-mortar charter schools.
- Rests on an assumption: Indiana virtual charter schools do not restrict enrollment capacity and do not use enrollment lotteries.
- The authors state this as an assumption: Matching cell fixed effects account for unobserved, time-invariant differences between cells.
- Also stated in: § Abstract · plain
- Findingown data
For the same baseline public school students, virtual charter high school attendance was associated with a high school dropout rate of 18.8%, versus 9.7% for matched traditional public school peers.
“higher high school dropout rates (18.8% to 9.7%)”
Quote in the paperSubstantial claim - Findingown data
Adding average curricular intensity increased the reduction in the 4-year college enrollment estimate to 51.3%, yielding -3.9 percentage points.
“The change in our estimate increases to 51.3% when the average curricular intensity factor is included (-3.9 percentage points).”
Quote in the paperSubstantial claim - Findingown data
School-level curricular intensity accounts for roughly half of the adverse college-going effects of virtual charter schools, particularly for enrollment in a four-year college.
“School-level curricular intensity—a measure of the rigor of course-taking at a school (Austin, 2020)—mediates about half of the negative college-going effects among virtual charters, especially as it relates to 4-year college enrollment.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Austin, 2020 source
- Rests on an assumption: If virtual charter attendance affects high school graduation, it does so largely through the unique contexts and practices of those schools.
- Findingown data
Variation in class size and school-level course-taking patterns accounts for a substantial share of the effects of virtual charter high schools.
“A mediation analysis found that variation in class size and school-level course-taking patterns explain a substantial portion of these impacts.”
Quote in the paperSubstantial claim- Rests on an assumption: If virtual charter attendance affects high school graduation, it does so largely through the unique contexts and practices of those schools.
- Findingown data
In intent-to-treat estimates, virtual charter high school students had a dropout rate 9.7 percentage points higher than matched traditional public school peers.
“Virtual charter high school students had 4-year high school graduation and dropout rates that were, respectively, 12.6 percentage points lower and 9.7 percentage points higher than their matched TPS peers.”
Quote in the paperSubstantial claim - Findingown data
Brick-and-mortar charter high schools had statistically higher curricular intensity than comparison traditional public schools.
“In contrast, the brick-and-mortar charter high schools had statistically higher curricular intensity as the comparison TPSs”
Quote in the paperNot a substantial claim - Findingown data
The average number of AP courses taken per student was 0.17 in virtual charter high schools, compared with 0.38 in comparison traditional public schools.
“the average number of AP courses taken are far lower (0.17 vs 0.38 mean courses taken per student) than the TPSs (see Table 1).”
Quote in the paperSubstantial claim - Findingown data
Lower average AP course-taking in virtual charter high schools reduced the main effect estimate on 4-year college enrollment by 33.8%, from -8.0 to -5.3 percentage points.
“For instance, the context of lower average number of AP courses taken at virtual charter high schools results in a 33.8% reduction in the main effect estimate on enrollment in a 4-year college (from -8.0 to -5.3 percentage points).”
Quote in the paperSubstantial claim - Findingown data
In intent-to-treat estimates, virtual charter high school students were 8.0 percentage points less likely to attend a 4-year college than matched traditional public school peers.
“These students were also FIGURE 1. Depiction of mediation analysis of charter effects on attainment outcomes.6.4 percentage points less likely to attend any college in general and 8.0 percentage points less likely to attend a 4-year college, in particular.”
Quote in the paperSubstantial claim - Findingown data
In intent-to-treat estimates, virtual charter high school students had a 4-year graduation rate 12.6 percentage points lower than matched traditional public school peers.
“Virtual charter high school students had 4-year high school graduation and dropout rates that were, respectively, 12.6 percentage points lower and 9.7 percentage points higher than their matched TPS peers.”
Quote in the paperSubstantial claim - Premisecitation
Most prior research on virtual charter schools has examined student test scores.
“Prior research on virtual charter schools has largely focused on student test scores (Ahn & McEachin, 2017;Fitzpatrick et al., 2020;Ford, 2022;Zimmer et al., 2009).”
Quote in the paperSubstantial claim0 of 3 sources checked - Study design and conductown data
An Oster (2019) assessment indicated that unobservables would need to be at least three times as related to selection as observables to invalidate the main virtual charter findings.
“In our analysis, we found that unobservables are unlikely to invalidate our main findings for students attending virtual charter high schools because the ratio of unobservables to observables needed to invalidate our findings is by at least 3 times (see Appendix Table C1, available on the journal website).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Oster (2019) source
- Interpretationcitation
The high school-level negative outcomes for virtual charter students-4-year graduation (-12.6 pp) and dropout (9.7 pp)-were similar in magnitude to findings from Pennsylvania.
“The negative outcomes at the high school level-graduation in 4 years (-12.6 percentage points) and dropout (9.7 percentage points)-were similar in magnitude as the findings from Pennsylvania (Cordes, 2024).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Cordes, 2024 source
- Study design and conductown data
A bounding exercise that set the ratio of unobservables to observables at 1 produced lower-bound estimates similar to the preferred model estimates.
“The second assessment involves setting the ratio of unobservables to observables at 1 and reestimating the treatment effect.This serves as a bounding exercise, and from this assessment, we find the revised lower bound estimates are similar to our preferred model estimates.”
Quote in the paperSubstantial claim - Premisecitation
Secondary course-taking patterns have a substantial impact on students' high school and longer-term outcomes.
“evidence suggests that secondary course-taking patterns have a substantial impact on students' high school and longer term outcomes (Austin, 2020;Goodman, 2019;Long et al., 2012).”
Quote in the paperSubstantial claim0 of 3 sources checked - Premisecitation
Cordes (2024) found that students who enrolled in a Pennsylvania virtual charter school in ninth grade were 9.5 percentage points less likely to graduate high school.
“In fact, the only published study to date comes from Pennsylvania, where Cordes (2024) found that students who enrolled in one of the state's virtual charter schools in ninth grade were 9.5 percentage points less likely to graduate high school.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Cordes (2024) source
- Interpretationown data
Course-taking mediation operated primarily through the 4-year college-going pathway.
“Thus, it appears that the mediation of course-taking primarily works through the 4-year college-going trajectory.”
Quote in the paperSubstantial claim - Interpretationcitation
The any-college (-6.4 pp) and 4-year college (-8.0 pp) effects for virtual charter students were smaller in magnitude than the corresponding Pennsylvania effects (-16.8 and -15.8 pp).
“The likelihood of attending any college (-6.4 percentage points ) or 4-year college (-8.0 percentage points), however, was smaller in magnitude than the same outcomes in Pennsylvania (-16.8 and -15.8 percentage points, respectively).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Cordes, 2024 source
- Interpretationown data
The lower average number of AP credits taken played an important role in mediating the negative effect on 4-year college enrollment.
“The lower average number of AP credits taken, in particular, seems to play an important role in this mediation.”
Quote in the paperSubstantial claim - Interpretationown data
The higher dropout rates among virtual charter high school students appeared to operate through processes linked to the substantially larger class sizes in those schools.
“In this regard, the higher rates of dropout among virtual charter high school students appear to operate through processes associated with the substantially larger class sizes of these schools.”
Quote in the paperSubstantial claim - Premisecitation
Courses in elementary and middle virtual charter schools tend to have exceptionally large teacher-to-student ratios.
“Prior research has also shown that courses in elementary and middle virtual charter schools tend to have exceptionally large teacherto-student ratios (Fitzpatrick et al., 2020;Molnar, 2023)”
Quote in the paperSubstantial claim0 of 2 sources checked- cites Fitzpatrick et al., 2020 source
- cites Molnar, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
The Indiana findings remained large and negative compared with the generally null findings for brick-and-mortar charter high schools in the state.
“Nevertheless, these findings in Indiana remain large and negative relative to the generally null findings of the brick-and-mortar charter high schools in the state.”
Quote in the paperSubstantial claim - Interpretationown data
About half of the negative effect of virtual charter high school attendance on 4-year college enrollment was mediated by the lower average course-taking intensity in those schools.
“This suggests that half of the negative impact of attending a virtual charter high school on 4-year college enrollment is mediated by the lower average intensity of the course-taking environment in these schools.”
Quote in the paperSubstantial claim - Study design and conductown data
A falsification test using eighth-grade math and ELA scores found no statistically significant differences between either charter group and matched peers.
“However, this test resulted in no statistically significant differences between either group of charter students and their matched peers (see Appendix Table C2, available on the journal website), thus providing greater confidence in our research design.”
Quote in the paperSubstantial claim - Premisecitation
Ahn and McEachin (2017) found that virtual charter school students were less likely than traditional public students and other charter students to pass Ohio's high school graduation exams.
“The latter study also provided a first glimpse of the effects of virtual charters on high school outcomes, in which the authors found that virtual charter school students were less likely than traditional public students (TPSs) and those in other charter schools to pass Ohio's high school graduation exams.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Ahn and McEachin (2017) source
- Study design and conductown data
The main effect estimates for virtual charter high school outcomes remained robust across multiple checks.
“Our main effects estimation of the virtual charter high school outcomes holds up to multiple robustness checks.”
Quote in the paperSubstantial claim - Premisecitation
Knowledge is limited about the curricular and instructional factors that could explain virtual charter schools' effects on student outcomes.
“In addition, little is known about the curricular and instructional factors that might explain virtual charter schools' impacts on student outcomes-a limitation that extends to a wider scope of the charter school literature (Gleason, 2017).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Gleason, 2017 source
- Interpretationown data
Virtual charter high schools had substantial negative impacts on student attainment outcomes overall.
“Overall, we found that virtual charter high schools had substantial negative impacts on student attainment outcomes.”
Quote in the paperSubstantial claim - Interpretationown data
In Indiana's virtual charter high schools, just over half of the negative effect on 4-year college enrollment was mediated by the lower-intensity course-taking context.
“It appears that in the case of Indiana's virtual charter high schools, a substantial (just over half ) amount of the negative effect on 4-year college enrollment is mediated by the context of lower intensity course-taking in these schools.”
Quote in the paperSubstantial claim - Premisecitation
Average class size in virtual charter schools is 3 to 4 times as large as in all other schools.
“Informed by previous research where average class size was found to be 3 to 4 times as large in virtual charter schools compared to all other schools (Fitzpatrick et al., 2020;Molnar, 2023), we include median high school class size as another mediation measure.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Fitzpatrick et al., 2020 source
- cites Molnar, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationown data
Students who switched into virtual charter high schools entered a social environment where peers took less rigorous courses than in the traditional public schools they would otherwise have attended.
“Students who switch into these schools enter a social environment where their peers take less rigorous courses relative to the TPSs they would have otherwise attended.”
Quote in the paperSubstantial claim - Interpretationown data
Most of the negative impact on 4-year high school graduation and dropout in virtual charter high schools was mediated through the large class sizes of those schools.
“Our mediation analysis also revealed that most of the negative impact on high school graduation within 4 years and dropout in virtual charter high schools is mediated through the large class sizes of these schools.”
Quote in the paperSubstantial claim - Study design and conductown data
The assessment of selection on unobservables indicated that even if lower course-taking intensity partly reflected selection bias, that bias was unlikely to invalidate the findings.
“Although this is possible, our assessment of selection on unobservables indicates that even if this is true to an extent, this source of bias is unlikely to invalidate the findings.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 9
- May combine two claims. 15
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationZimmer and colleagues (2009) found that the estimated impact of virtual charter schools in Ohio was -0.44 standard deviations in mathematics and -0.25 standard deviations in reading.
“In Ohio, for example, Zimmer and colleagues (2009) used a student fixed-effects model and found that the impact estimate for virtual charter schools was -0.44 SD in mathematics and -0.25 SD in reading.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Zimmer et al. (2009) source
May combine two claims.
Findingown dataAccounting for median high school class size reduced the virtual charter dropout estimate by 72.2%, from 9.7 percentage points to 2.7 percentage points, and the new estimate was not statistically significant.
“When accounting for a student's median high school class size, we find that three quarters of the dropout rate is explained away: Our estimate decreases by 72.2%.This marks a change in the estimated impact of virtual charter schools on high school dropout from 9.7 percentage points more likely to only 2.7 percentage points, and this new estimate is no longer statistically significant.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe median class size in virtual charter high schools was 180, more than seven times the 24.5 median in comparison traditional public schools.
“Also, the median class size of the virtual charter high schools is 180, more than 7 times that of the comparison TPSs (24.5).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataBrick-and-mortar charter high schools in Indiana generally had null effects on high school graduation, dropout, and college enrollment.
“Our results show that Indiana's virtual charter high schools have large, negative impacts on high school graduation, dropout, and college enrollment, whereas the brick-and-mortar charter high schools generally produce null outcomes.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The matching characteristics used (sex, race/ethnicity, baseline achievement, and chronic absenteeism or disciplinary incidents) sufficiently explain selection into virtual or brick-and-mortar charter schools.
- Rests on an assumption: Indiana virtual charter schools do not restrict enrollment capacity and do not use enrollment lotteries.
- The authors state this as an assumption: Matching cell fixed effects account for unobserved, time-invariant differences between cells.
May combine two claims.
Findingown dataClass size mediated the higher dropout rate in virtual charter high schools but was not a significant predictor of college-going outcomes.
“In contrast to the course-taking measures, class size mediates the higher rate of high school dropout in virtual charter schools but is not a significant predictor of college-going outcomes.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataStudents in virtual charter high schools experienced a less rigorous course-taking environment and larger class sizes than the traditional public schools from which the analytical sample switched.
“Overall, students attending virtual charter high schools encounter a less rigorous course-taking environment and larger class sizes than those in the TPSs from which our analytical sample of students switched.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataSchool-level course-taking patterns explained a substantial portion of the 4-year college enrollment finding, but did not appear to mediate high school dropout or any-college-going rates.
“We find that schoollevel patterns of course-taking explain a substantial amount of our findings related to 4-year college enrollment, although it does not appear to mediate high school dropout or any college-going rates.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe authors found no evidence that highest math course, English credits, or science credits mediated the key outcomes.
“We did not find any evidence of mediation for these factors, as we did for the mean number of AP courses and the latent measure of curricular intensity.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataVirtual schools had lower average curricular intensity and less variance in curricular intensity than traditional public schools (SD = 0.18 vs. 0.32).
“Virtual schools not only have a lower average curricular intensity but less variance as well (virtual: SD = 0.18; TPS: SD = 0.32).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
PremisecitationLonger-term outcomes such as high school graduation and college enrollment are considered at least as important as test scores and may be a stronger indicator of student success.
“there is broad recognition that longer term outcomes, such as high school graduation and college enrollment, are at least as important and may ultimately be a stronger measure of student success than test scores (Sass et al., 2016)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Sass et al., 2016 source
Classified as probably not a substantial claim. May combine two claims.
Study design and conductcitationThe study conducted an assessment of selection on unobservables and a falsification test for the main effects estimates.
“We conducted an assessment of selection on unobservables (Oster, 2019) and a falsification test for our main effects estimates, which we describe further in the results and in Appendix C, available on the journal website.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Oster, 2019 source
- The authors state this as an assumption: Matching cell fixed effects account for unobserved, time-invariant differences between cells.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe Center for Research on Education Outcomes (2023) found that virtual students in 214 virtual charter schools had annual achievement losses of 0.21 standard deviations in math and 0.10 standard deviations in reading.
“They found that virtual students in these schools had annual achievement losses of -0.21 SD in math and -0.10 SD in reading.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Center for Research on Education Outcomes (2023)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationFitzpatrick et al. (2020) concluded that lower qualifications and higher student-teacher ratios among virtual charter teachers explained some of the negative impacts, but most of the effect remained unexplained.
“The authors concluded that the overall lower qualifications and higher student-teacher ratio of virtual charter teachers compared to traditional public teachers explained some of the negative impacts, yet the majority of this negative effect remained unexplained.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Fitzpatrick et al. (2020) source
Classified as probably not a substantial claim. May combine two claims.
PremisecitationCordes (2024) found that virtual charter students were 16.8 percentage points less likely to enroll in college and 15.2 percentage points less likely to persist in college beyond one semester.
“In addition, virtual charter students were 16.8 percentage points less likely to enroll in college and 15.2 percentage points less likely to persist in college beyond one semester.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Cordes (2024) source
May combine two claims.
Study design and conductreasoningThe study matched charter and public high school students within the same eighth-grade school cohort on sex, race/ethnicity, and baseline achievement, and additionally on chronic absenteeism for virtual charter students or disciplinary incidents for brick-and-mortar charter students.
“we matched together students who attended charter and public high schools in ninth grade who were part of the same eighth-grade school cohort (same school and year, whether in a public or charter school at baseline).”
- Also stated in: § Research Design · plain
“Therefore, we included these factors as additional matching criteria for finding public comparison students for virtual charter students (chronic absenteeism) and brick-and-mortar charter students (disciplinary background).”
Quote in the paperSubstantial claim- The authors state this as an assumption: The matching characteristics used (sex, race/ethnicity, baseline achievement, and chronic absenteeism or disciplinary incidents) sufficiently explain selection into virtual or brick-and-mortar charter schools.
- Also stated in: § Research Design · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationCurrent enrollment trends suggest an ongoing expansion of the virtual school sector across the country.
“Current enrollment trends are highly suggestive of an ongoing expansion of the virtual school sector across the country (Molnar, 2023;Veney & Jacobs, 2021).”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites Molnar, 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Veney & Jacobs, 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPrior research shows that student performance in virtual course-taking varies widely across contexts.
“Prior research demonstrates that student performance in virtual course-taking varies widely across contexts (Johnson et al., 2022)”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Johnson et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationcitationThere may still be successful models of virtual schooling even though existing virtual charter schools largely fail to have a positive impact on student outcomes.
“In this sense, there may still be successful models of virtual schooling even though existing virtual charter schools largely fail to have a positive impact on student outcomes.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPrior research shows that virtual course-taking can in some cases have positive impacts on student outcomes.
“and in some cases, can have positive impacts on student outcomes (Hart et al., 2019).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Hart et al., 2019 source
The authors state this as an assumption.
Students in the cohorts progress through grades normally.
“Assuming normal grade progression, this is a sufficient number of years of data to conduct our analyses on attainment outcomes along with necessary falsification and other robustness tests.”
The authors state this as an assumption.
The matching characteristics used (sex, race/ethnicity, baseline achievement, and chronic absenteeism or disciplinary incidents) sufficiently explain selection into virtual or brick-and-mortar charter schools.
“We believe that these limited characteristics sufficiently help to explain why a given student may select into either a virtual or brick-and-mortar charter school.”
The paper relies on this without support.
Indiana virtual charter schools do not restrict enrollment capacity and do not use enrollment lotteries.
“Virtual charter schools in Indiana enroll students without any semblance of capacity restrictions and therefore do not implement enrollment lotteries.”
The authors state this as an assumption.
Matching cell fixed effects account for unobserved, time-invariant differences between cells.
“The cell fixed effects account for unobserved, time-invariant between-cell differences.”
The paper relies on this without support.
If virtual charter attendance affects high school graduation, it does so largely through the unique contexts and practices of those schools.
“if switching to a virtual charter school influences a student's likelihood of graduating high school, it is likely because the latter schools have unique contexts and practices relative to a traditional public school.”
The paper relies on this without support.
The mediation analysis assumes that the mediators—school-level course-taking and class size—are not confounded with the outcomes by unmeasured variables.
“We find that schoollevel patterns of course-taking explain a substantial amount of our findings related to 4-year college enrollment”
“class size mediates the higher rate of high school dropout in virtual charter schools”
The paper relies on this without support.
Indiana's school choice landscape is rapidly expanding, with near-universal private school vouchers that include two private virtual schools.
“Indiana's virtual charter schools operate in a state that is rapidly expanding its school choice options, including near-universal access to private school vouchers that now include two participating private virtual schools.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Main estimates are attributed to a narrower student group
MinorThe results describe Table 2's estimates as applying to virtual-charter students who attended public schools at baseline. The research design says the main findings combine students from both public and charter baseline schools, with separate estimates reported elsewhere.
“Our intent-to-treat model estimates of attainment outcomes show similar patterns among virtual charter high school students who were enrolled in public schools at baseline as the descriptives (see Table 2).”
“Our main findings combine all charter students regardless of baseline school sector, and we provide separate descriptive estimates for these two groups in Appendix B, available on the journal website, and model estimates in Appendix C, available on the journal website.”
Cites
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.51.6.1173 · Cited by 91,542 · see citing works
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532 · see citing works
Mediation in experimental and nonexperimental studies: New procedures and recommendations.
Shrout, P. E., & Bolger, N. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.4.422 · Cited by 11,335 · see citing works
Unobservable Selection and Coefficient Stability: Theory and Evidence
Oster, E. (2019) · Journal of Business & Economic Statistics · doi:10.1080/07350015.2016.1227711 · Cited by 5,367 · see citing works
Explaining Charter School Effectiveness
Angrist, J. D., Pathak, P. A., & Walters, C. (2011) · American Economic Journal Applied Economics · doi:10.1257/app.5.4.1 · Cited by 454 · see citing works
Getting Beneath the Veil of Effective Schools: Evidence From New York City
Dobbie, W., & Fryer, R. G. (2013) · American Economic Journal: Applied Economics · doi:10.1257/app.5.4.28 · Cited by 352 · see citing works
Making College Worth It: A Review of the Returns to Higher Education
Oreopoulos, P., & Petronijevic, U. (2013) · The Future of Children · doi:10.1353/foc.2013.0001 · Cited by 341 · see citing works
Effects of High School Course-Taking on Secondary and Postsecondary Success
Long, M. C., Conger, D., & Iatarola, P. (2012) · American Educational Research Journal · doi:10.3102/0002831211431952 · Cited by 305 · see citing works
Angrist, J. D., Cohodes, S. R., Dynarski, S. M., Pathak, P. A., & Walters, C. R. (2016) · Journal of Labor Economics · doi:10.1086/683665 · Cited by 294 · see citing works
Charter Schools in Eight States: Effects on Achievement, Attainment, Integration, and Competition
Zimmer, R., Gill, B., Booker, K., Lavertu, S., Sass, T., & Witte, J. (2009) · doi:10.7249/mg869 · Cited by 264 · see citing works
Self-citationMaking College Worth It: A Review of Research on the Returns to Higher Education
Oreopoulos, P., & Petronijevic, U. (2013) · National Bureau of Economic Research · doi:10.3386/w19053 · Cited by 176 · see citing works
(2023) · doi:10.3102/00346543221105550 · Cited by 159 · see citing works
Is Free and Reduced-Price Lunch a Valid Measure of Educational Disadvantage?
Domina, T., Pharris-Ciurej, N., Penner, A. M., Penner, E. K., Brummet, Q., Porter, S. R., & Sanabria, T. (2018) · Educational Researcher · doi:10.3102/0013189x18797609 · Cited by 148 · see citing works
Student Enrollment Patterns and Achievement in Ohio’s Online Charter Schools
Ahn, J., & McEachin, A. (2017) · Educational Researcher · doi:10.3102/0013189x17692999 · Cited by 146 · see citing works
Online Learning, Offline Outcomes: Online Course Taking and High School Student Performance
Hart, C. M. D., Berger, D., Jacob, B., Loeb, S., & Hill, M. (2019) · AERA Open · doi:10.1177/2332858419832852 · Cited by 140 · see citing works
Bifulco, R. (2012) · Journal of Policy Analysis and Management · doi:10.1002/pam.20637 · Cited by 110 · see citing works
Michelmore, K. M., & Dynarski, S. (2017) · AERA Open · doi:10.1177/2332858417692958 · Cited by 108 · see citing works
Fitzpatrick, B. R., Berends, M., Ferrare, J. J., & Waddington, R. J. (2020) · Educational Researcher · doi:10.3102/0013189x20909814 · Cited by 77 · see citing works
Self-citationThe Labor of Division: Returns to Compulsory High School Math Coursework
Goodman, J. (2019) · Journal of Labor Economics · doi:10.1086/703135 · Cited by 70 · see citing works
Explaining Charter School EffectivenessPreprint
Angrist, J. D., Pathak, P. A., & Walters, C. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2045678 · Cited by 58 · see citing works
Charter Schools and Labor Market Outcomes
Dobbie, W., & Fryer, R. G. (2019) · Journal of Labor Economics · doi:10.1086/706534 · Cited by 55 · see citing works
Charter High Schools’ Effects on Long‐Term Attainment and Earnings
Sass, T. R., Zimmer, R. W., Gill, B. P., & Booker, T. K. (2016) · Journal of Policy Analysis and Management · doi:10.1002/pam.21913 · Cited by 53 · see citing works
Self-citationBricks and Mortar vs. Computers and Modems: The Impacts of Enrollment in K-12 Virtual SchoolsPreprint
Bueno, C. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3642969 · Cited by 36 · see citing works
Peeking Into the Black Box of School Turnaround: A Formal Test of Mediators and Suppressors
Henry, G. T., Pham, L. D., Kho, A., & Zimmer, R. (2020) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373720908600 · Cited by 35 · see citing works
Self-citationFortson, K., Gleason, P., Kopa, E., & Verbitsky‐Savitz, N. (2014) · Economics of Education Review · doi:10.1016/j.econedurev.2014.11.001 · Cited by 28 · see citing works
Scott-Clayton, J., & Wen, Q. (2019) · Evaluation Review · doi:10.1177/0193841x18803247 · Cited by 27 · see citing works
Michelmore, K. M., & Dynarski, S. (2016) · National Bureau of Economic Research · doi:10.3386/w22474 · Cited by 23 · see citing works
Measuring High School Curricular Intensity over Three Decades
Austin, M. (2020) · Sociology of Education · doi:10.1177/0038040719885123 · Cited by 16 · see citing works
Gleason, P. M. (2017) · Journal of School Choice · doi:10.1080/15582159.2017.1395620 · Cited by 14 · see citing works
Cordes, S. A. (2024) · Education Finance and Policy · doi:10.1162/edfp_a_00399 · Cited by 10 · see citing works
Using “Broken” Lotteries to Check the Validity of Charter School Evaluations Using Matching Designs
Foreman, L. M., Anderson, K. P., Ritter, G. W., et al. (2017) · Educational Policy · doi:10.1177/0895904817741543 · Cited by 6 · see citing works
Insufficient Accountability? Heterogeneous Effects of Charter Schools Across Authorizing Agencies
Ferrare, J. J., Waddington, R. J., Fitzpatrick, B. R., & Berends, M. (2023) · American Educational Research Journal · doi:10.3102/00028312231167802 · Cited by 4 · see citing works
Self-citationVirtual Charter Students Have Worse Labor Market Outcomes as Young Adults
Yoo, P. Y. (2024) · doi:10.3102/ip.24.2141854 · Cited by 4 · see citing works
Going Virtual: The Impacts of a Virtual Charter School Governance Reform
Ford, M. R. (2022) · Public Performance & Management Review · doi:10.1080/15309576.2022.2036203 · Cited by 2 · see citing works
Bifulco, R. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1805890 · Cited by 1 · see citing works
Charter Schools and Labor Market Outcomes
Dobbie, W., & Fryer, R. G. (2019) · RePEc: Research Papers in Economics · doi:10.26153/tsw/59835
https://www.edworkingpapers.com/ai20-250
Bueno, C. (2020) · EdWorkingPapers.com · doi:10.26300/kahb-5v62
Virtual Charter Students Have Worse Labor Market Outcomes as Young Adults
(2023) · edworkingpapers.com · doi:10.26300/7n80-pv38
Online Teaching in K–12 Education: A Systematic Review
Johnson, C. (2022) · Proceedings of the 2022 AERA Annual Meeting · doi:10.3102/1890925
Flagged first, then by citations
Research works printed but not linked: 5
- Adelman C. (2006). The toolbox revisited: Paths to degree completion from high school to college. U.S. Department of Education.
- Center for Research on Education Outcomes. (2015). Online charter school study. Center for Research on Education Outcomes, Stanford University.
- Molnar A. (Ed.) (2023). Virtual schools in the U.S. 2023. National Education Policy Center, University of Colorado Boulder.
- U.S. Government Accountability Office. (2022). K-12 education: Department of Education should help states address student testing issues and financial risks associated with virtual schools, particularly virtual charter schools (GAO-22-104444).
- Veney D., Jacobs D. (2021). Voting with their feet: A state-level analysis of public charter school and district public school trends. National Alliance for Public Charter Schools.
Cited by
Sorted by citations
Negative Selection into Virtual Charter Schools
Scafidi, B. P. (2025) · Journal of School Choice · doi:10.1080/15582159.2025.2609047
Teacher Effectiveness in Remote Instruction
Lawson, M. C., & Sass, T. R. (2026) · Education Finance and Policy · doi:10.1162/edfp.a.437
- Authors
- Ferrare, J. J., Waddington, R. J., Zimmer, R., & Berends, M.
- Affiliations
- University of Washington Bothell (US)
- University of Notre Dame (US)
- University of Kentucky (US)
- Corresponding author
- Ferrare, Joseph J.
- Venue
- Educational Researcher
- Published
- 2025-05-14
- Volume
- 54
- Issue
- 8
- Pages
- 449-460
- ISSN
- 0013-189X (ISSN-L)
- 1935-102X
- OpenAlex
- W4410374254
- Cited by
- 2
- References
- 35
- Citations by year
- 2026: 1 · 2025: 1
- FWCI
- 6.22
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Educational Researcher (publishedVersion)
- Funders
- Institute of Education Sciences (R305A190340)
- Topics
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Education › Online and Blended Learning
- Social Sciences › Education › Education during COVID-19 pandemic
- Keywords
- virtual schools, charter schools, high school graduation, high school dropout, college enrollment, postsecondary outcomes, class size, course-taking patterns, student outcomes, COVID-19 pandemic, school choice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-30.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-30.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
