Hawthorne Control Procedures in Educational Experiments: A Reconsideration of Their Use and Effectiveness
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
This article reports on a descriptive analysis of research practices and a meta-analysis of effect sizes associated with control groups employed to address Hawthorne effects in educational experiments. The descriptive analysis of 86 studies revealed a diversity of practices designed to control one of three artifact variables: special attention, activity related to the experimental task, or awareness of participation in an experiment. The meta-analysis provided no evidence for a Hawthorne versus no-treatment control difference. Moreover, a detailed analysis of these studies by their control procedure, and subsequently by other moderator variables, also revealed no systematic trends to suggest a specific artifact source. A within-study analysis of the pattern of treatment/Hawthorne/control group effect sizes suggested that the artifact controls were of limited utility. Increased research is urged into other artifacts and alternative control procedures that the educational researcher should consider.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 2×
“for the no-treatment control group (NTC) from the appropriate mean in each case, dividing by the pooled standard deviation, and multiplying by Hedges' correction to remove the bias associated with small samples (Hedges, 1981).”
MethodThe Hawthorne effect: A reconsideration of the methodological artifact.
Adair, J. G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.2.334 · Cited by 1,095 · cited 4×
“Such an artifact has been proposed as an alternative explanation for the Hawthorne effect (Adair, 1984; Schneiderman, 1977).”
BackgroundMethodInfluentialSelf-citation
Cited by
McCambridge, J., Witton, J., & Elbourne, D. R. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.015 · Cited by 2,748
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033
One Hundred Years of Social Psychology Quantitatively Described
Richard, F. D., Bond, C. F., & Stokes-Zoota, J. J. (2003) · Review of General Psychology · doi:10.1037/1089-2680.7.4.331 · Cited by 1,558
Lintott, C. J., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., Raddick, M. J., Nichol, R. C., Szalay, A., Andreescu, D., et al. (2008) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2008.13689.x · Cited by 1,488
Albanese, M. (2000) · Medical Education · doi:10.1046/j.1365-2923.2000.00753.x · Cited by 418
Effects of augmented reality on learning and cognitive load in university physics laboratory courses
Thees, M., Kapp, S., Strzys, M. P., Beil, F., Lukowicz, P., & Kuhn, J. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106316 · Cited by 351
Levitt, S. D., & List, J. A. (2011) · American Economic Journal: Applied Economics · doi:10.1257/app.3.1.224 · Cited by 297
Using vignettes to collect data for nursing research studies: how valid are the findings?
GOULD, D. (1996) · Journal of Clinical Nursing · doi:10.1111/j.1365-2702.1996.tb00253.x · Cited by 290
Ziegelmann, J. P., Lippke, S., & Schwarzer, R. (2006) · Psychology & Health · doi:10.1080/1476832050018891 · Cited by 269
Hawthorne effects and research into professional practice
Holden, J. D. (2001) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2001.00280.x · Cited by 263
Szumski, G., Smogorzewska, J., & Karwowski, M. (2017) · Educational Research Review · doi:10.1016/j.edurev.2017.02.004 · Cited by 214
Colom, R., Román, F. J., Abad, F. J., Shih, P. C., Privado, J., Froufe, M., Escorial, S., Martínez, K., Burgaleta, M., Quiroga, M. A., et al. (2013) · Intelligence · doi:10.1016/j.intell.2013.09.002 · Cited by 112
Barrera, M., Atenafu, E., Nathan, P. C., Schulte, F., & Hancock, K. (2018) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsy040 · Cited by 22
Metadata
- Authors
- Adair, J. G., Sharpe, D., & Huynh, C.-L.
- Venue
- Review of Educational Research
- Publisher
- SAGE Publications
- Published
- 1989
- Volume
- 59
- Issue
- 2
- Pages
- 215
- ISSN
- 0034-6543 (ISSN-L)
- OpenAlex
- W1984857358
- Cited by
- 91
- References
- 2
- FWCI
- 7.55
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.