CoChat

Technological Innovations in Large-Scale Assessment

Zenisky, April L., Sireci, Stephen G. · 2002 · Applied Measurement in Education
Cited by 138

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Computers have had a tremendous impact on assessment practices over the past half century. Advances in computer technology have substantially influenced the ways in which tests are made, administered, scored, and reported to examinees. These changes are particularly evident in computer-based testing, where the use of computers has allowed test developers to re-envision what test items look like and how they are scored. By integrating technology into assessments, it is increasingly possible to create test items that can sample as broad or as narrow a range of behaviors as needed while preserving a great deal of fidelity to the construct of interest. In this article we review and illustrate some of the current technological developments in computer-based testing, focusing on novel item formats and automated scoring methodologies. Our review indicates that a number of technological innovations in performance assessment are increasingly being researched and implemented by testing programs. In some cases, complex psychometric and operational issues have successfully been dealt with, but a variety of substantial measurement concerns associated with novel item types and other technological aspects impede more widespread use. Given emerging research, however, there appears to be vast potential for expanding the use of more computerized constructed-response type items in a variety of testing contexts.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1207/s15324818ame1504_02

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 3 references

Research works printed but not linked: 52

  1. Ackerman T. A., 1994, Applied Measurement in Education, 7, 255, 10.1207/s15324818ame0704_1 · DOI · Paper page
  2. Ackerman, T. A., Evans, J., Park, K., Tamassia, C. & Turner, R. (1999). Computer assessment using visual stimuli: A test of dermatological skin disorders. In F. Drasgow and J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 137-150). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
  3. Ashworth, S. D. & Joyce, T. M. (1994, April). Developing scoring protocols for a computerized multimedia in-basket exercise. Paper presented at the Ninth Annual Conference of the Society for Industrial and Organizational Psychology, Nashville, TN.
  4. 10.1037//0021-9010.76.4.522 · DOI
  5. 10.1177/01466210022031769 · DOI
  6. 10.1207/S15324818AME1303_5 · doi:10.1207/s15324818ame1303_5
  7. Bennett R. E., 1999, Journal of Educational Measurement, 36, 233, 10.1111/j.1745-3984.1999.tb00556.x · doi:10.1111/j.1745-3984.1999.tb00556.x
  8. Bennett R. E., 1995, Journal of Educational Measurement, 32, 19, 10.1111/j.1745-3984.1995.tb00454.x · doi:10.1111/j.1745-3984.1995.tb00454.x
  9. Bennett R. E., 1996, Applied Measurement in Education, 9, 133, 10.1207/s15324818ame0902_3 · doi:10.1207/s15324818ame0902_3
  10. Bennett R. E., 1997, Journal of Educational Measurement, 34, 64, 10.1111/j.1745-3984.1997.tb00507.x · DOI
  11. Bennett R. E., 1991, Applied Psychological Measurement, 15, 227, 10.1177/014662169101500302 · DOI
  12. Bennett R. E., 1997, Journal of Educational Measurement, 34, 162, 10.1111/j.1745-3984.1997.tb00512.x · DOI
  13. Braun H. I., 1990, Journal of Educational Measurement, 27, 93, 10.1111/j.1745-3984.1990.tb00736.x · doi:10.1111/j.1745-3984.1990.tb00736.x
  14. Brennan R. L., 1995, Educational Measurement: Issues and Practice, 14, 25, 10.1111/j.1745-3992.1995.tb00882.x · DOI
  15. Burstein, J. & Chodorow, M. (2002). Directions in automated essay scoring analysis. In R. Kaplan (Ed.), Oxford handbook of applied linguistics (pp. 487-497). Oxford, England: Oxford University Press.
  16. Burstein, J., Kukich, K., Wolff, S., Lu, C. & Chodorow, M. (1998, April). Computer analysis of essays. Paper presented at the NCME Symposium on Automated Scoring, San Diego, CA. · DOI
  17. 10.1016/S0747-5632(99)00032-1 · DOI
  18. 10.1177/01466210022031778 · DOI
  19. Clauser B. E., 2000, Journal of Educational Measurement, 37, 245, 10.1111/j.1745-3984.2000.tb01085.x · DOI
  20. Clauser B. E., 1997, Journal of Educational Measurement, 34, 141, 10.1111/j.1745-3984.1997.tb00511.x · DOI
  21. Crocker L., 1997, Applied Measurement in Education, 10, 83, 10.1207/s15324818ame1001_5 · DOI
  22. Davey T., 1997, Journal of Educational Measurement, 34, 21, 10.1111/j.1745-3984.1997.tb00505.x · doi:10.1111/j.1745-3984.1997.tb00505.x
  23. Desmarais, L. B., Dyer, P. J., Midkiff, K. R., Barbera, K. M., Curtis, J. R., Esrig, F. H. & Masi, D. L. (1992, May). Scientific uncertainties in the development of a multimedia test: Trade-offs and decisions. Paper presented at the Seventh Annual Conference of the Society for Industrial and Organizational Psychology, Montreal, Quebec, Canada.
  24. Drasgow, F., Olson-Buchanan, J. B. & Moberg, P. J. (1999). Development of an interactive video assessment: Trials and tribulations. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 197-219). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. · DOI
  25. Enright M. K., 1998, Journal of Educational Measurement, 35, 250, 10.1111/j.1745-3984.1998.tb00538.x · doi:10.1111/j.1745-3984.1998.tb00538.x
  26. Foltz P. W., 1998, Discourse Processes, 25, 285, 10.1080/01638539809545029 · DOI · Paper page
  27. Glaser, R. (1991). Expertise and assessment. In M. C. Wittrock & E. L. Baker (Eds.), Testing and cognition (pp. 17-30). Englewood Cliffs, NJ: Prentice Hall.
  28. Guion R. M., 1995, Educational Measurement: Issues and Practice, 14, 25, 10.1111/j.1745-3992.1995.tb00886.x · DOI
  29. Hambleton, R. K. (1997, October). Promising GMAT item formats for the 21st century. Invited presentation at the international workshop on the GMAT, Paris, France.
  30. 10.1016/S0747-5632(99)00026-6 · DOI
  31. Huff K. L., 2001, Educational Measurement: Issues and Practice, 20, 16, 10.1111/j.1745-3992.2001.tb00066.x · DOI
  32. 10.1016/S0346-251X(98)00034-7 · DOI
  33. Kane M. T., 1992, Evaluation and the Health Professions, 15, 163, 10.1177/016327879201500203 · DOI · Paper page
  34. Kane M., 1999, Educational Measurement: Issues and Practice, 18, 5, 10.1111/j.1745-3992.1999.tb00010.x · DOI · Paper page
  35. Landauer, T. K., Laham, D., Rehder, B. & Schreiner, M. E. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans. In G. Shafto & P. Langley (Eds.), Proceedings of the 19th annual meeting of the Cognitive Science Society (pp. 412-417). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
  36. Larkey, L. S. (1998). Automatic essay grading using text categorization techniques. In the Proceedings of the 21st Annual International Conference of the Association for Computing Machinery-Special Interest Group on Information Retrieval, Melbourne, Australia, 90-95. · DOI · Paper page
  37. Martinez M. E., 1991, Journal of Educational Measurement, 28, 131, 10.1111/j.1745-3984.1991.tb00349.x · DOI
  38. Martinez M. E., 1992, Applied Measurement in Education, 5, 151, 10.1207/s15324818ame0502_5 · doi:10.1207/s15324818ame0502_5
  39. Mills, C. (2000, February). Unlocking the promise of CBT. Keynote address presented at a conference of the Association of Test Publishers, Carmel Valley, CA.
  40. Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Almond, R. G. & Johnson, L. (1999, September 16-17). Making sense of data from complex assessments. Paper presented at the 1999 CRESST Conference, Los Angeles, CA. · DOI
  41. Page E. B., 1995, Phi Delta Kappan, 76, 561
  42. Page E. B., 1994, Journal of Experimental Education, 62, 127, 10.1080/00220973.1994.9943835 · DOI · Paper page
  43. Parshall C. G., 2001, Educational Measurement: Issues and Practice, 20, 5, 10.1111/j.1745-3992.2001.tb00058.x · DOI
  44. Parshall, C. G., Davey, T. & Pashley, P. (2000). Innovative item types for computerized testing. In W. J. van der Linden & C. Glas (Eds.), Computer-adaptive testing: Theory and practice (pp. 129-148). Boston: Kluwer Academic. · DOI
  45. Rehder B., 1998, Discourse Processes, 25, 337, 10.1080/01638539809545031 · DOI · Paper page
  46. 10.1037//0021-9010.76.6.856 · doi:10.1037//0021-9010.76.6.856
  47. Shavelson R. J., 1991, Applied Measurement in Education, 4, 347, 10.1207/s15324818ame0404_7 · DOI · Paper page
  48. van der Linden, W. J. & Hambleton, R. K. (Eds.). (1997). Handbook of modern item response theory. New York: Springer. · DOI · Paper page
  49. Vispoel, W. P. (1999). Creating computerized adaptive tests of music aptitude: Problems, solutions, and future directions. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 151-176). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
  50. Williamson D. M., 1999, Journal of Educational Measurement, 36, 158, 10.1111/j.1745-3984.1999.tb00552.x · DOI · Paper page
  51. 10.1207/S15324818AME1504_04 · DOI · Paper page
  52. Zenisky, A. L. & Sireci, S. (2001). Feasibility review of selected performance assessment item types for the computerized Uniform CPA Exam (Laboratory of Psychometric and Evaluative Research Rep. No. 405). Amherst: School of Education, University of Massachusetts.

Cited by

See all 138

Metadata

Authors
Zenisky, A. L., & Sireci, S. G.
Venue
Applied Measurement in Education
Published
2002-10
Volume
15
Issue
4
Pages
337-362
ISSN
  • 0895-7347 (ISSN-L)
  • 1532-4818
OpenAlex
W2041638702
Cited by
138
References
3
FWCI
5.96
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Technological Innovations in Large-Scale Assessment. https://app.cochat.ai/papers/10.1207/s15324818ame1504_02
The paper
Zenisky, A. L., & Sireci, S. G. (2002). Technological Innovations in Large-Scale Assessment. Applied Measurement in Education. https://doi.org/10.1207/s15324818ame1504_02

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.