Generalizing to a Language Population
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Many studies of verbal behavior have little scientific point if their conclusions have to be restricted to the specific language materials that were used in the experiment. It has not been customary, however, to perform significance tests that permit generalization beyond these specific materials, and thus there is little statistical evidence that such studies could be successfully replicated if a different sample of language materials were used. Three tests are described that will allow generalization to a population of language materials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Approximate Distribution of Estimates of Variance Components
Satterthwaite, F. E. (1946) · Biometrics Bulletin · doi:10.2307/3002019 · Cited by 4,373
Complex Analyses of Variance: General Problems
Green, B. F., & Tukey, J. W. (1960) · Psychometrika · doi:10.1007/bf02288577 · Cited by 163
Cited by
Mixed-effects modeling with crossed random effects for subjects and items
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008) · Journal of Memory and Language · doi:10.1016/j.jml.2007.12.005 · Cited by 8,540
The language-as-fixed-effect fallacy: A critique of language statistics in psychological research
Clark, H. H. (1973) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(73)80014-3 · Cited by 2,222
Null hypothesis significance testing: a review of an old and continuing controversy
Nickerson, R. S. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.241 · Cited by 1,090
The case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931
“Statistical significance is generally interpreted as having some relationship to replication (see, for example, Coleman, 1964; Melton, 1962), and replication is the cornerstone of science (Bauernfeind, 1968; Smith, 1970).”
BackgroundWestfall, J., Kenny, D. A., & Judd, C. M. (2014) · Journal of Experimental Psychology: General · doi:10.1037/xge0000014 · Cited by 928
“…I error rates if in fact one seeks generalization not only to other samples of participants but also to other samples of stimuli (e.g., particular White and African American target photos) that might be used (Baayen, Davidson, & Bates, 2008; Clark, 1973; Coleman, 1964; Santa, Miller, & Shaw, 1979).”
BackgroundYarkoni, T. (2022) · Behavioral and Brain Sciences · doi:10.1017/s0140525x20001685 · Cited by 701
Best practice guidance for linear mixed-effects models in psychological science
Meteyard, L., & Davies, R. A. I. (2020) · Journal of Memory and Language · doi:10.1016/j.jml.2020.104092 · Cited by 698
Raaijmakers, J. G. W., Schrijnemakers, J. M. C., & Gremmen, F. (1999) · Journal of Memory and Language · doi:10.1006/jmla.1999.2650 · Cited by 463
Keep it real: rethinking the primacy of experimental control in cognitive neuroscience
Nastase, S. A., Goldstein, A., & Hasson, U. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.117254 · Cited by 344
De Boeck, P. (2008) · Psychometrika · doi:10.1007/s11336-008-9092-x · Cited by 272
GENERALIZING ABOUT MESSAGES: SUGGESTIONS FOR DESIGN AND ANALYSIS OF EXPERIMENTS
JACKSON, S., & JACOBS, S. (1983) · Human Communication Research · doi:10.1111/j.1468-2958.1983.tb00691.x · Cited by 254
Understanding Mixed-Effects Models Through Data Simulation
DeBruine, L. M., & Barr, D. J. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920965119 · Cited by 148
“The problem, and the solutions to the problem, has been known in psycholinguistics for more than 50 years (Clark, 1973; Coleman, 1964), and most psycholinguistic journals require authors to demonstrate generalizability of findings over stimuli as well as over subjects.”
BackgroundForster, K. I., & Dickinson, R. G. (1976) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/0022-5371(76)90014-1 · Cited by 105
Metadata
- Authors
- Coleman, E. B.
- Venue
- Psychological Reports
- Publisher
- SAGE Publications
- Published
- 1964-02
- Volume
- 14
- Issue
- 1
- Pages
- 219-226
- ISSN
- 0033-2941 (ISSN-L)
- 1558-691X
- OpenAlex
- W2116581177
- Cited by
- 111
- References
- 2
- FWCI
- 3.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.