The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field
Open accessAbstract
Despite its critical role for the development of the field, little is known about replication in second language (L2) research. To better understand replication practice, we first provide a narrative review of challenges related to replication, drawing on recent developments in psychology. This discussion frames and motivates a systematic review, building on syntheses of replication in psychology, education, and L2 research. We coded 67 self‐labeled L2 replication studies found across 26 journals for 136 characteristics. We estimated a mean rate of 1 published replication study for every 400 articles, with a mean of 6.64 years between initial and replication studies and a mean of 117 citations of the initial study before a replication was published. Replication studies had an annual mean of 7.3 citations, much higher than averages in linguistics and education. Overlap in authorship between initial and replication studies and the availability of the initial materials both increased the likelihood of a replication supporting the initial findings. Our sample contained no direct (exact) replication attempts, and changes made to initial studies were numerous and wide ranging, which likely obscured, if not undermined, the interpretability of replication studies. To improve the amount and quality of L2 replication research, we propose 16 recommendations relating to rationale, nomenclature, design, infrastructure, and incentivization for collaboration and publication. Open Practices: This article has been awarded Open Materials and Open Data badges. All materials and data are publicly accessible via the IRIS Repository at https://www.iris-database.org/iris/app/home/detail?id=york:934328. Learn more about the Open Practices badges from the Center for Open Science:https://osf.io/tvyxz/wiki.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1×
“…to submit statistically significant findings) is a widely acknowledged problem, and null findings are confined to the “file drawer” (a term coined by Rosenthal, 1979; the phenomenon documented by, for example, Bakker, van Dijk, & Wicherts, 2012; Schmidt & Oh, 2016; Sterling, Rosenbaum, & Weinkam,…”
BackgroundEstimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 8×
“…or significant alterations of the initial study – considered “the means of establishing reproducibility of a finding with new data” (Open Science Collaboration, 2015, p. 1) and (b) ‘conceptual’, where there is intentional adaptation of the initial study to investigate generalizability to…”
BackgroundMethodResultInfluentialSimmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“in the initial study) is considered the more publishable and citable outcome (Ioannidis, 2005; Luijendijk & Koolman, 2012).”
BackgroundA manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis
Norris, J. M., & Ortega, L. (2000) · Language Learning · doi:10.1111/0023-8333.00136 · Cited by 2,569 · cited 16×
“The term ‘reproducible/reproducibility’ is used in a more marked way to refer only to (b), the extent to which the results of a replication study are similar to those of an initial study (i.e., reproduction of findings), in line with the recent developments in the field of psychology (e.g., Open Science Collaboration,…”
BackgroundMethodResultInfluentialHARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
Cited by
ELICITED IMITATION TASKS AS A MEASURE OF L2 PROFICIENCY: A META-ANALYSIS
Kostromitina, M., & Plonsky, L. (2022) · Studies in Second Language Acquisition · doi:10.1017/s0272263121000395 · Cited by 61
KASPROWICZ, R. E., MARSDEN, E., & SEPHTON, N. (2019) · The Modern Language Journal · doi:10.1111/modl.12586 · Cited by 47
Self-citationGao, J., & Ma, S. (2021) · Studies in Second Language Acquisition · doi:10.1017/s0272263120000686 · Cited by 12
Open Science and Transparency in Applied Linguistics Research
Marsden, E. (2019) · The Encyclopedia of Applied Linguistics · doi:10.1002/9781405198431.wbeal1493 · Cited by 6
Self-citation
Metadata
- Authors
- Marsden, E., Morgan‐Short, K., Thompson, S., & Abugaber, D.
- Venue
- Language Learning
- Publisher
- Wiley
- Published
- 2018-03-05
- Volume
- 68
- Issue
- 2
- Pages
- 321-391
- ISSN
- 0023-8333 (ISSN-L)
- 1467-9922
- OpenAlex
- W2807962483
- Cited by
- 268
- References
- 26
- FWCI
- 5.36
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
