CoChat

Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments: A Meta-Analysis of Testing Mode Effects

Shudong Wang, Hong Jiao, Young, Michael J., Brooks, Thomas, Olson, John · 2008 · Educational and Psychological Measurement

The DOI matches its Crossref record, with a small difference in title or year; 53 of the 55 references it cites were found in a registry or an index.

Abstract

In recent years, computer-based testing (CBT) has grown in popularity, is increasingly being implemented across the United States, and will likely become the primary mode for delivering tests in the future. Although CBT offers many advantages over traditional paper-and-pencil testing, assessment experts, researchers, practitioners, and users have expressed concern about the comparability of scores between the two test administration modes. To help provide an answer to this issue, a meta-analysis was conducted to synthesize the administration mode effects of CBTs and paper-and-pencil tests on K—12 student reading assessments. Findings indicate that the administration mode had no statistically significant effect on K—12 student reading achievement scores. Four moderator variables—study design, sample size, computer delivery algorithm, and computer practice—made statistically significant contributions to predicting effect size. Three moderator variables—grade level, type of test, and computer delivery method—did not affect the differences in reading scores between test modes.

Overview

Integrity signals

Registry

Crossref · 10.1177/0013164407305592

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 53 references

Research works printed but not linked: 34

  1. American Council on Education., 1995, Guidelines for computerized adaptive test development and use in education
  2. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education., 1999, Standards for educational and psychological testing
  3. American Psychological Association Committee on Professional Standards and Committee on Psychological Tests and Assessment., 1986, Guidelines for computer-based tests and interpretations
  4. Arce-Ferrer, A., 2004, Comparison of paper-and-pencil and online versions of Grade 5 mathematics and Grade 8 reading and writing tests
  5. Association of Test Publishers., 2000, Guidelines for computer-based testing
  6. Bennett, R.E., 2001, Education Policy Analysis Archive,, 9
  7. Bennett, R.E., 2002, Journal of Technology, Learning, and Assessment,, 1
  8. Bergstrom, B., Paper presented at the annual meeting of the American Educational Research Association
  9. Boo, J., Paper presented at the annual meeting of the National Council on Measurement in Education
  10. Choi, S.W., Paper presented at the annual meeting of the National Council on Measurement in Education
  11. Folk, V.G., Paper presented at the ETS-sponsored colloquium on Computer-Based Tests: Building the Foundation for Future Assessments
  12. Godwin, J., Paper presented at the annual meeting of the National Council on Measurement in Education
  13. Goldberg, A., 2003, Journal of Technology, Learning, and Assessment,, 2
  14. 10.1037/0003-066X.42.5.443 · doi:10.1037/0003-066x.42.5.443 · Paper page
  15. Ito, K., Paper presented at the annual meeting of the American Educational Research Association
  16. Johnson, M., Paper presented at the British Educational Research Association Annual Conference
  17. Kim, J., Paper presented at the annual meeting of the Mid-Western Educational Research Association
  18. Kingsbury, G.G., Paper presented at the annual meeting of the American Educational Research Association
  19. Kingsbury, G.G., Paper presented at the annual meeting of the American Educational Research Association
  20. Mead, A.D., 1993, Psychological Bulletin,, 9, 287
  21. National Association of State Boards of Education., 2001, Any time, any place, any path, any pace: Taking the lead on e-learning policy
  22. National Center for Education Statistics., 2000, Teacher use of computers and the Internet in public schools (NCES 2000-090)
  23. Park, J., 2003, Education Week, 22, 15
  24. Poggio, J., Paper presented at the annual meeting of the National Council on Measurement in Education
  25. Pommerich, M., 2004, Journal of Technology, Learning, and Assessment,, 2
  26. Pommerich, M., Paper presented at the annual meeting of the National Council on Measurement in Education
  27. Pomplun, M., Paper presented at the annual meeting of the American Educational Research Association
  28. Russell, M., 1999, Educational Policy Analysis Archives, 7
  29. Russell, M., 1997, Educational Policy Analysis Archives, 5
  30. Russell, M., 2001, Teachers College Record
  31. Schwarz, R.D., Paper presented at the annual meeting of the National Council on Measurement in Education
  32. Stenner, A.J., Paper presented at the California Comparability Symposium
  33. Vispoel, W.P., Paper presented at the annual meeting of the AERA
  34. Wang, S., 2000, AT&T aptitude test equivalence study: A comparison of computer and paper-and-pencil employment examinations

Cited by

See all 186

Metadata

Authors
Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J.
Affiliations
  • Harcourt Assessment, Inc., shudong_wang@ harcourt.com
  • Harcourt Assessment, Inc.
Venue
Educational and Psychological Measurement
Published
2007-09-12
Volume
68
Issue
1
Pages
5-24
ISSN
  • 0013-1644 (ISSN-L)
  • 1552-3888
OpenAlex
W2079754662
Cited by
186
References
55
Citations by year
2026: 9 · 2025: 9 · 2024: 10 · 2023: 11 · 2022: 15 · 2021: 15 · 2020: 14 · 2019: 13 · 2018: 13 · 2017: 7 · 2016: 5 · 2015: 20 · 2014: 6 · 2013: 9 · 2012: 11
FWCI
5.74
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Educational and Psychological Measurement (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
  • Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
Keywords
computer-based testing, paper-and-pencil tests, test comparability, reading assessment, K-12 students, reading achievement, computer practice, test format effects

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments: A Meta-Analysis of Testing Mode Effects. https://app.cochat.ai/papers/10.1177/0013164407305592
The paper
Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2008). Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments: A Meta-Analysis of Testing Mode Effects. Educational and Psychological Measurement. https://doi.org/10.1177/0013164407305592

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.