ORAL FEEDBACK IN CLASSROOM SLA: A Meta-Analysis

Classified as probably not a substantial claim.

Any single group's standard deviation was viewed as especially prone to sampling error because most studies in the domain had small samples.

“The standard deviation of any single group was considered particularly susceptible to sampling error, due to the small sample sizes of most studies within research domain”

• cites Norris and Ortega's (2000) sourceCould not verify

  The supplied passages and the passages retrieved under this DOI are labeled as a different work, “Self-Monitoring and Self-Reflection in an English Blended Learning Environment.” Its Limitations section mentions a small sample in that study, but no passage addresses single-group standard deviations or their susceptibility to sampling error. Searches for the claim’s wording and for sampling error, standard deviation, and small sample sizes did not verify the attribution to Norris and Ortega. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Length of instruction alone is not a reliable index for SLA processes, as suggested by Norris and Ortega's acknowledgement that small differences between shorter and longer treatments were likely due to other study variables.

“However, it might be the case that length of instruction alone is not a reliable index for SLA processes, as suggested by Norris and Ortega's ( 2000 ) acknowledgement that the small differences they observed between shorter and longer term treatments were likely due to a number of study variables”

• cites Norris and Ortega's ( 2000 ) sourceCould not verify

  The supplied passages and further indexed results are labeled “Self-Monitoring and Self-Reflection in an English Blended Learning Environment,” not the cited Norris and Ortega paper. Searches for shorter versus longer treatments and instructional duration returned no passage addressing the attributed acknowledgement. The claim cannot be verified from the available copy. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Norris and Ortega (2000) noted that many studies used outcome measures favoring explicit treatments because these measures required explicit declarative knowledge under controlled conditions without much need for fluent spontaneous contextualized language.

“Norris and Ortega ( 2000 ) noted that many studies employed outcome measures that favor the effectiveness of explicit treatments because these measures required”

• cites Norris and Ortega ( 2000 ) sourceCould not verify

  The supplied passages and additional searches return text headed “Self-Monitoring and Self-Reflection in an English Blended Learning Environment,” rather than passages from Norris and Ortega’s synthesis. Searches for explicit treatments, declarative knowledge, and the claimed outcome-measure bias found no passage addressing the claim, so the indexed copy cannot verify it. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Meta-analysis has been widely recognized as a trustworthy tool for research synthesis in instructed SLA since Norris and Ortega's seminal study.

“The usefulness of meta-analysis as a trustworthy tool for research synthesis has been widely recognized in the area of instructed SLA studies since Norris and Ortega's ( 2000 ) seminal study”

• cites Norris and Ortega's (2000) sourceCould not verify

  The passages supplied and retrieved under this DOI are headed “Self-Monitoring and Self-Reflection in an English Blended Learning Environment,” rather than the cited Norris and Ortega paper. Searches for “meta-analysis” and its use in instructed SLA returned no relevant passage, so this copy cannot verify the claimed recognition of the method since their study. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

The instructional treatments in the 15 included studies differ substantially.

“It should be noted, however, that the instructional treatments employed in the 15 studies differ in substantial ways”

• AssumptionRests on an assumption: The 15 included classroom studies are sufficiently comparable in their instructional treatments to be combined in one meta-analysis despite substantial differences.

May combine two claims.

In the included studies, recasts and prompts were generally delivered in communicatively driven teacher-student interaction, whereas explicit correction was operationalized with less consistency and across a broader spectrum of instructional activities.

“Whereas recasts and prompts in the studies included in this metaanalysis were generally delivered in the context of communicatively driven teacher-student interaction, explicit correction was operationalized with slightly less consistency and across a broader spectrum of instructional activities.”

May combine two claims. The quote only partly states this claim.

In a meta-analysis of 15 classroom-based studies with 827 learners, oral corrective feedback produced statistically significant and lasting gains in the target language.

“Results revealed that CF had signifi cant and durable effects on target language development.”

May combine two claims.

Explicit correction did not differ significantly from recasts or prompts in either between-group or within-group comparisons.

“Whereas explicit correction did not prove signifi cantly different from either recasts or promptsneither in between-nor within-group contrasts”

May combine two claims.

Free constructed-response effects were significantly larger than metalinguistic judgment effects in between-group comparisons, and the difference almost reached significance in within-group comparisons.

“The analysis of confidence intervals shows that the effects of free constructed-response measures are signifi cantly larger than those of metalinguistic judgments in the between-group contrast, and the difference between these two measures almost reached signifi cance, with slight overlapping confidence intervals in the within-group contrasts.”

May combine two claims.

Corrective feedback is effective when used in response to learners' nontargetlike production because it promotes target language development over time.

“it is effective to employ CF in response to students' nontargetlike production because it contributes to target language development over time.”

May combine two claims.

Long treatments were significantly more effective than short-to-medium treatments in between-group comparisons, while the value of brief treatments remained unclear.

“Whereas the value of brief treatments remains unclear, the analysis of confi dence intervals shows that long treatments (0.82-1.44) are signifi cantly more effective than short-to-medium treatments (0.44-0.70) in the between-group contrasts”

• Also stated in: § CONCLUSION · hedged

  “The effects of long treatments proved to be signifi cantly larger than those of short-to-medium treatments but not distinguishable from brief treatments, thus preventing any conclusions about length of instruction”

May combine two claims. The quote only partly states this claim.

The current meta-analysis found smaller recast effects than Mackey and Goo (2007), who reported a much larger recast effect (d = 0.96).

“Our results are different from those of Mackey and Goo ( 2007 ), who found much larger effects for recasts ( d = 0.96)”

• cites Mackey and Goo ( 2007 )Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Recasts, prompts, and explicit correction were all significantly effective in L2 classrooms in both between-group and within-group comparisons.

“Taken together, with confi dence intervals above zero, recasts, prompts, and explicit correction are all signifi cantly effective in L2 classrooms both in the between-and within-group contrasts.”

May combine two claims.

Immediate and delayed CF effects were medium-to-large with overlapping confidence intervals, indicating no significant decline in gains between immediate and delayed posttests.

“Both immediate and delayed effects of CF thus exhibit medium-to-large effect sizes, with overlapping confi dence intervals as illustrated in Figure 4 , which indicates that there is no signifi cant decline in gains between immediate and delayed posttests.”

• Also stated in: § Durability · plain

  “which shows that CF has an immediate impact on L2 performance and that the magnitude of the effectiveness does not decline signifi cantly during the interval between the CF treatment and delayed posttests.”

May combine two claims.

Free constructed-response measures showed large effects, while constrained constructed-response measures and metalinguistic judgments showed medium-to-large effects.

“The means indicate large effect sizes for free constructed-response measures and medium-to-large effect sizes for constrained constructedresponse measures and metalinguistic judgments.”

Classified as probably not a substantial claim.

Recasts, explicit correction, and prompts are compared because their definitions are sufficiently categorical to permit relatively clear comparisons.

“Recasts, explicit correction, and prompts are compared here because their defi nitions are suffi ciently categorical to allow for relatively clear comparisons.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Prompts are distinguishable from recasts and explicit correction by their demand, the degree of conversational urgency exerted on interlocutors to react to negative feedback.

“Prompts range from implicit to explicit but are distinguishable from recasts and explicit correction in terms of what Ortega ( 2009 ) called demand -that is, “the degree of conversational urgency exerted upon interlocutors to react to the negative feedback” (p.75).”

• cites Ortega (2009)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Russell and Spada's meta-analysis found that corrective feedback facilitates second language development, with a very large effect size of 1.16.

“Russell and Spada found that CF is facilitative of L2 development; they identifi ed a very large effect size of 1.16.”

• cites Russell and Spada (2006)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Comparing prompts that withhold correct forms with reformulation types that provide correct forms has ecological validity because teachers can decide online whether to withhold or provide correct forms in response to errors, but they may not be able to decide online whether to deliver corrective feedback explicitly or implicitly.

“the comparison of prompts that withhold correct forms with other CF types that provide correct forms has considerable ecological validity because teachers are able to make online decisions about whether to withhold or to provide correct forms in response to students' errors”

May combine two claims.

All 15 CF studies in the current meta-analysis included control groups, so no baseline groups had to be chosen, enabling independent analysis of between- and within-group contrasts.

“all 15 CF studies in the current meta-analysis included control groups; therefore, no baseline groups (i.e., using least attentionfocused groups when there is no control group) for between-group differences had to be chosen, which enabled an independent analysis of between-and within-group contrasts.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The classification of prompts at discrete points along the implicit-explicit continuum is crude and open to further refinement.

“The classifi cation of prompts at discrete points along this continuum is rather crude and open to further refi nement”

• AssumptionRests on an assumption: The classification of corrective feedback types into recasts, explicit correction, and prompts is reliable enough for meta-analytic comparison.

May combine two claims.

Mackey and Goo reported medium effect sizes for classroom studies (d = 0.57 immediate, d = 0.76 long delayed) and larger effect sizes for laboratory studies (d = 0.96 immediate, d = 1.20 long delayed).

“They reported medium effect size values for classroom studies ( d = 0.57 for immediate posttests and d = 0.76 for long delayed posttests) and much larger effect sizes for laboratory studies ( d = 0.96 for immediate posttests and d = 1.20 for long delayed posttests).”

• cites Mackey and Goo (2007)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Mackey and Goo found that corrective feedback in second language interaction had a medium effect size of 0.71 on immediate posttests and a large effect size of 1.09 on delayed posttests.

“Similarly, Mackey and Goo discovered that providing CF in L2 interaction has a medium effect size of 0.71 in immediate posttests and a large effect size of 1.09 in delayed posttests.”

• cites Mackey and Goo (2007)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Explicit correction supplies both negative and positive evidence, recasts supply positive evidence and possibly negative evidence, and prompts supply only negative evidence.

“In terms of linguistic evidence, therefore, explicit correction provides both negative and positive evidence, recasts provide positive evidence and possibly also negative evidence, and prompts provide only negative evidence.”

May combine two claims.

Learners in laboratory settings may be more responsive to corrective feedback because intensity and consistency are well controlled, whereas classroom teachers may struggle to deliver corrective feedback consistently to specific linguistic targets.

“Learners in laboratory settings may be more sensitive to CF because variables such as intensity and consistency are well controlled for; in the case of classroom settings, teachers may have diffi culty delivering CF following specifi c linguistic targets in a consistent manner (Ellis & Sheen; Nicholas et al.).”

• cites Ellis & SheenNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Nicholas et al.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The distinguishing feature of prompts in this classification is withholding correct reformulations rather than varying degrees of implicitness or explicitness.

“the distinguishing feature of prompts in this classifi cation pertains less to their varying degrees of implicitness or explicitness and more to their common trait of withholding correct reformulations.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The overall impact of corrective feedback in classroom settings may be greater for younger learners than for older learners.

“the overall impact of CF in classroom settings might be greater for younger learners than for older learners”

Probably not a substantial claim: may not pass S2 (checkable).

The meta-analysis approach permits meaningful comparisons of multifaceted results from previous studies.

“Unlike a narrative review of CF effectiveness, the meta-analysis approach adopted here to synthesize previous studies entails a statistical procedure that permits meaningful comparisons of their multifaceted results.”

May combine two claims.

When CF is provided with relative intensity during form-focused activities designed to elicit target forms in oral interaction, its effects are sufficiently robust to appear immediately after treatment and to be sustained over 2-6 weeks.

“It may be the case that when CF is provided in this way, with relative intensity during form-focused activities designed to elicit target forms in the context of oral interaction, its effects are suffi ciently robust not only to appear immediately after the instructional treatment but also to be sustained over time, which in the present analysis ranged from 2-6 weeks.”

Classified as probably not a substantial claim. May combine two claims.

The results should be interpreted with caution and considered tentative because of the small N, the broad confidence intervals and standard deviations, and the inclusion of several k from one sample study.

“However, it is important to emphasize the need to interpret the results with caution and to consider them tentative, given the obvious limitations: the small N , the relatively broad range of confi dence intervals and standard deviations in all contexts, and the inclusive contribution of several k from one sample study.”

Classified as probably not a substantial claim.

The age-related data need to be interpreted with caution because of the small number of child and adult CF studies.

“Although the data need to be interpreted with caution due to the small number of child and adult CF studies”

May combine two claims.

Classroom learners appear to benefit from the positive evidence in recasts and the opportunities recasts provide to infer negative evidence, but they appear to benefit even more from the negative evidence in prompts and the greater demand prompts impose for modified output.

“classroom learners appear to benefi t from the positive evidence available in recasts as well as from the opportunities they provide to infer negative evidence, but these learners seem to benefi t even more from the negative evidence available in prompts and from the greater demand they impose for producing modifi ed output.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Control-group gains in within-group comparisons may be due to test-retest effects or to control participants receiving classroom instruction without intentionally designed CF.

“This result could be attributed to test-retest effects or to the fact that even participants in control groups, by virtue of being in classroom settings, also received instruction but without intentionally designed CF treatments.”

May combine two claims.

The difference in recast effects between the current study and Mackey and Goo (2007) is attributed to inclusion criteria: the current study included only classroom studies (n = 15), whereas Mackey and Goo included laboratory (n = 15) and classroom (n = 5) studies.

“we attribute this to differences in the inclusion criteria between the two studies: Our study included only classroom studies ( n = 15) in the meta-analysis, whereas Mackey and Goo included CF studies conducted in both laboratory ( n = 15) and classroom settings ( n = 5).”

• cites Mackey and Goo ( 2007 )Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Studies accepted for publication tend to have statistically significant results, so studies without significance remain unpublished, producing the file-drawer problem.

“The fact that studies accepted for publication tend to have statistically signifi cant results gives rise to the so-called fi le-drawer problem whereby studies without statistical significance remain unpublished (see Lipsey & Wilson, 2001 , for details).”

• cites Lipsey & Wilson, 2001Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Mitchell and Myles (2004) argued that the underlying learning processes are essentially the same for more local and more remote target languages, despite differing learning purposes and circumstances.

“As Mitchell and Myles ( 2004 ) argued,”

• cites Mitchell and Myles ( 2004 )Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The explanations for outcome-measure effects are speculative because of the small N and relatively large confidence intervals and standard deviations.

“However, such explanations are speculative, due to the small N and relatively large confidence intervals and standard deviations.”

May combine two claims.

Prompts may be more effective than recasts for younger learners, while prompts and recasts may be similarly effective for older learners.

“which indicates that prompts might be more effective than recasts for younger learners, whereas prompts and recasts might be similarly effective for older learners”

May combine two claims.

CF is facilitative of L2 development irrespective of instructional settings, and its impact is sustained at least until delayed posttests.

“Results showed that, irrespective of instructional settings, CF is facilitative of L2 development and that its impact is sustained at least until delayed posttests.”

Classified as probably not a substantial claim. May combine two claims.

The finding of no SL/FL difference suggests either that the distinction between SL and FL settings is too fluid to yield distinguishable results or that cognitive processes triggered by CF are similar across instructional settings.

“This fi nding suggests either that distinctions between SL and FL settings are too fl uid to yield distinguishable results (see Berns, 1990 ;Block, 2003 ) or that cognitive processes triggered by CF are similar across instructional settings.”

• cites Berns, 1990Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Block, 2003Not checked — no full text of the source