Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments: A Meta-Analysis of Testing Mode Effects
The DOI matches its Crossref record, with a small difference in title or year; 53 of the 55 references it cites were found in a registry or an index.
Abstract
In recent years, computer-based testing (CBT) has grown in popularity, is increasingly being implemented across the United States, and will likely become the primary mode for delivering tests in the future. Although CBT offers many advantages over traditional paper-and-pencil testing, assessment experts, researchers, practitioners, and users have expressed concern about the comparability of scores between the two test administration modes. To help provide an answer to this issue, a meta-analysis was conducted to synthesize the administration mode effects of CBTs and paper-and-pencil tests on K—12 student reading assessments. Findings indicate that the administration mode had no statistically significant effect on K—12 student reading achievement scores. Four moderator variables—study design, sample size, computer delivery algorithm, and computer practice—made statistically significant contributions to predicting effect size. Three moderator variables—grade level, type of test, and computer delivery method—did not affect the differences in reading scores between test modes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“this difference between reading media was larger in studies that used fixed linear computerized tests ( n =37) than in those that used adaptive computerized tests ( n =5).”
From Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension (2018) · cites this paper as Wang et al. (2007)
In the citing paper
“They found better reading outcomes in paper-based testing than in digital-based testing. The mean effect size (0.08) was significant, but small”
From Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension (2018) · cites this paper as Wang et al. (2007)
In the citing paper
“The results of the two meta-analyses in the present study yield a clear picture of screen inferiority, with lower reading comprehension outcomes for digital texts compared to printed texts, which corroborates and extends previous research (Kong et al., 2018; Singer & Alexander, 2017b; Wang et al., 2007).”
From Don't throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension (2018) · cites this paper as Wang et al., 2007
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Methods for Meta-Analysis
Wasserman, S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,326
(2000) · Cited by 9,198
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · cited 1×
“The standardized mean difference or ES g for any individual study is defined (Hedges, 1981; Hedges & Olkin, 1985) as follows: where M CBT and M PPT are the means of CBT and PPT scores and SD p is the pooled estimate of the standard deviation, where n CBT and n PPT are the sample sizes of CBT and PPT…”
Method(1904) · The American Journal of Psychology · doi:10.2307/1412635 · Cited by 4,187
A nation at risk: the imperative for educational reform
Denning, P. J. (1983) · Communications of the ACM · doi:10.1145/358150.358154 · Cited by 3,925
Food, M.-D. (2011) · In Search of Research Excellence · doi:10.4337/9781849807630.00020 · Cited by 3,062
Statistical Methods for Meta-Analysis
(1985) · Elsevier eBooks · doi:10.1016/c2009-0-03396-0 · Cited by 2,521
Morris, S. B., & DeShon, R. P. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.105 · Cited by 2,502
Research works printed but not linked: 34
- American Council on Education., 1995, Guidelines for computerized adaptive test development and use in education
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education., 1999, Standards for educational and psychological testing
- American Psychological Association Committee on Professional Standards and Committee on Psychological Tests and Assessment., 1986, Guidelines for computer-based tests and interpretations
- Arce-Ferrer, A., 2004, Comparison of paper-and-pencil and online versions of Grade 5 mathematics and Grade 8 reading and writing tests
- Association of Test Publishers., 2000, Guidelines for computer-based testing
- Bennett, R.E., 2001, Education Policy Analysis Archive,, 9
- Bennett, R.E., 2002, Journal of Technology, Learning, and Assessment,, 1
- Bergstrom, B., Paper presented at the annual meeting of the American Educational Research Association
- Boo, J., Paper presented at the annual meeting of the National Council on Measurement in Education
- Choi, S.W., Paper presented at the annual meeting of the National Council on Measurement in Education
- Folk, V.G., Paper presented at the ETS-sponsored colloquium on Computer-Based Tests: Building the Foundation for Future Assessments
- Godwin, J., Paper presented at the annual meeting of the National Council on Measurement in Education
- Goldberg, A., 2003, Journal of Technology, Learning, and Assessment,, 2
- 10.1037/0003-066X.42.5.443 · doi:10.1037/0003-066x.42.5.443 · Paper page
- Ito, K., Paper presented at the annual meeting of the American Educational Research Association
- Johnson, M., Paper presented at the British Educational Research Association Annual Conference
- Kim, J., Paper presented at the annual meeting of the Mid-Western Educational Research Association
- Kingsbury, G.G., Paper presented at the annual meeting of the American Educational Research Association
- Kingsbury, G.G., Paper presented at the annual meeting of the American Educational Research Association
- Mead, A.D., 1993, Psychological Bulletin,, 9, 287
- National Association of State Boards of Education., 2001, Any time, any place, any path, any pace: Taking the lead on e-learning policy
- National Center for Education Statistics., 2000, Teacher use of computers and the Internet in public schools (NCES 2000-090)
- Park, J., 2003, Education Week, 22, 15
- Poggio, J., Paper presented at the annual meeting of the National Council on Measurement in Education
- Pommerich, M., 2004, Journal of Technology, Learning, and Assessment,, 2
- Pommerich, M., Paper presented at the annual meeting of the National Council on Measurement in Education
- Pomplun, M., Paper presented at the annual meeting of the American Educational Research Association
- Russell, M., 1999, Educational Policy Analysis Archives, 7
- Russell, M., 1997, Educational Policy Analysis Archives, 5
- Russell, M., 2001, Teachers College Record
- Schwarz, R.D., Paper presented at the annual meeting of the National Council on Measurement in Education
- Stenner, A.J., Paper presented at the California Comparability Symposium
- Vispoel, W.P., Paper presented at the annual meeting of the AERA
- Wang, S., 2000, AT&T aptitude test equivalence study: A comparison of computer and paper-and-pencil employment examinations
Cited by
Delgado, P., Vargas, C., Ackerman, R., & Salmerón, L. (2018) · Educational Research Review · doi:10.1016/j.edurev.2018.09.003 · Cited by 664
Reading from paper compared to screens: A systematic review and meta‐analysis
Clinton, V. (2019) · Journal of Research in Reading · doi:10.1111/1467-9817.12269 · Cited by 436
Coiro, J. (2021) · Reading Research Quarterly · doi:10.1002/rrq.302 · Cited by 216
Technological Issues for Computer-Based Assessment
Csapó, B., Ainley, J., Bennett, R. E., Latour, T., & Law, N. W. Y. (2011) · Springer eBooks · doi:10.1007/978-94-007-2324-5_4 · Cited by 171
Reiss, K., Weis, M., Klieme, E., Köller, O., & OECD (2019) · Waxmann Verlag GmbH eBooks · doi:10.31244/9783830991007 · Cited by 209
Metadata
- Authors
- Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J.
- Affiliations
- Harcourt Assessment, Inc., shudong_wang@ harcourt.com
- Harcourt Assessment, Inc.
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2007-09-12
- Volume
- 68
- Issue
- 1
- Pages
- 5-24
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- OpenAlex
- W2079754662
- Cited by
- 186
- References
- 55
- Citations by year
- 2026: 9 · 2025: 9 · 2024: 10 · 2023: 11 · 2022: 15 · 2021: 15 · 2020: 14 · 2019: 13 · 2018: 13 · 2017: 7 · 2016: 5 · 2015: 20 · 2014: 6 · 2013: 9 · 2012: 11
- FWCI
- 5.74
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational and Psychological Measurement (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- computer-based testing, paper-and-pencil tests, test comparability, reading assessment, K-12 students, reading achievement, computer practice, test format effects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments: A Meta-Analysis of Testing Mode Effects. https://app.cochat.ai/papers/10.1177/0013164407305592
- The paper
- Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2008). Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments: A Meta-Analysis of Testing Mode Effects. Educational and Psychological Measurement. https://doi.org/10.1177/0013164407305592
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.