The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Addressing the assessment challenge with an online system that tutors as it assesses
Open accessAbstract
Secondary teachers across the United States are being asked to use formative assessment data (Black and Wiliam 1998a,b; Roediger and Karpicke 2006) to inform their classroom instruction. At the same time, critics of US government’s No Child Left Behind legislation are calling the bill “No Child Left Untested”. Among other things, critics point out that every hour spent assessing students is an hour lost from instruction. But, does it have to be? What if we better integrated assessment into classroom instruction and allowed students to learn during the test? We developed an approach that provides immediate tutoring on practice assessment items that students cannot solve on their own. Our hypothesis is that we can achieve more accurate assessment by not only using data on whether students get test items right or wrong, but by also using data on the effort required for students to solve a test item with instructional assistance. We have integrated assistance and assessment in the ASSISTment system. The system helps teachers make better use of their time by offering instruction to students while providing a more detailed evaluation of student abilities to the teachers, which is impossible under current approaches. Our approach for assessing student math proficiency is to use data that our system collects through its interactions with students to estimate their performance on an end-of-year high stakes state test. Our results show that we can do a reliably better job predicting student end-of-year exam scores by leveraging the interaction data, and the model based on only the interaction information makes better predictions than the traditional assessment model that uses only information about correctness on the test items.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152 · cited 1×
“Assistance and assessment are integrated in the system ( " ASSISTment " 4 ) that offers instruction to students while providing a more detailed evaluation of their abilities to the teacher than is possible under current approaches.”
BackgroundBayesian model selection in social research
Raftery, A. E. (1995) · Sociological Methodology · doi:10.2307/271063 · Cited by 6,823
Inside the Black Box: Raising Standards through Classroom Assessment
Black, P., & Wiliam, D. (2010) · Phi Delta Kappan · doi:10.1177/003172171009200119 · Cited by 4,231
Handbook of Modern Item Response Theory
(1997) · Springer eBooks · doi:10.1007/978-1-4757-2691-6 · Cited by 2,305
The power of testing memory: Basic research and implications for educational practice
Roediger, H. L., & Karpicke, J. D. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00012.x · Cited by 2,037
RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY
TATSUOKA, K. K. (1983) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1983.tb00212.x · Cited by 920
Human Abilities: An Information Processing Approach
Sternberg, R. J. (1985) · Medical Entomology and Zoology · Cited by 608
Dynamic testing: the nature and measurement of learning potential
(2003) · Choice Reviews Online · doi:10.5860/choice.40-4136 · Cited by 529
Cognitively Diagnostic Assessment
(2012) · Routledge eBooks · doi:10.4324/9780203052969 · Cited by 490
Detecting Student Misuse of Intelligent Tutoring Systems
Baker, R. S., Corbett, A. T., & Koedinger, K. R. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30139-4_50 · Cited by 353
Self-citationGrigorenko, E. L., & Sternberg, R. J. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.1.75 · Cited by 310
"Kappan Classic": Inside the Black Box--Raising Standards through Classroom Assessment.
Black, P., & Wiliam, D. (2010) · Phi Delta Kappan · Cited by 232
Recovery of Two- and Three-Parameter Logistic Item Characteristic Curves: A Monte Carlo Study
Hulin, C. L., Lissak, R. I., & Drasgow, F. (1982) · Applied Psychological Measurement · doi:10.1177/014662168200600301 · Cited by 204
Opening the Door to Non-programmers: Authoring Intelligent Tutor Behavior by Demonstration
Koedinger, K. R., Aleven, V. A. W. M. M., Heffernan, N. T., McLaren, B. M., & Hockenberry, M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30139-4_16 · Cited by 161
Self-citationLin, T. Y. (2002) · Educational Technology & Society · Cited by 149
Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems
Walonoski, J. A., & Heffernan, N. T. (2006) · Lecture notes in computer science · doi:10.1007/11774303_38 · Cited by 139
Self-citationGrigorenko, E. L., & Sternberg, R. J. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.3.443 · Cited by 115
Corbett, A. T., & Bhatnagar, A. (1997) · Springer eBooks · doi:10.1007/978-3-7091-2670-7_25 · Cited by 96
Massachusetts Department of Education
(1923) · Journal of Education · doi:10.1177/002205742309801902 · Cited by 85
Scaffolding vs. Hints in the Assistment System
Razzaq, L. M., & Heffernan, N. T. (2006) · Lecture notes in computer science · doi:10.1007/11774303_63 · Cited by 66
Self-citationAnozie, N. O., & Junker, B. W. (2006) · Cited by 65
Student modeling in the ACT Programming Tutor.
Corbett, A. T., Anderson, J. R., & O'Brien, A. T. (1995) · Cited by 57
Automatically Assessing Oral Reading Fluency in a Computer Tutor that Listens
Beck, J. E., Jia, P., & Mostow, J. (2004) · Cited by 38
Do Performance Goals Lead Students to Game the System
Baker, R. S., Roll, I., Corbett, A. T., & Koedinger, K. R. (2005) · Cited by 37
Self-citationTowards Live Informing and Automatic Analyzing of Student Learning: Reporting in ASSISTment System
Feng, M., & Heffernan, N. T. (2007) · Cited by 23
Self-citationWhat Level of Tutor Interaction is Best
Razzaq, L. M., Heffernan, N. T., & Lindeman, R. W. (2007) · Cited by 22
Self-citationCan we Predict which Groups of Questions Students will Learn from
Feng, M., Heffernan, N. T., Beck, J. E., & Koedinger, K. R. (2008) · Cited by 12
Self-citationThe Concept of Formative Assessment
Boston, C. (2002) · University of Massachusetts Amherst · doi:10.7275/kmcq-dj31
Cited by
Educational Data Mining and Learning Analytics
Baker, R. S., & Inventado, P. S. (2014) · Learning Analytics · doi:10.1007/978-1-4614-3305-7_4 · Cited by 1,029
Dynamic Key-Value Memory Networks for Knowledge Tracing
Zhang, J., Shi, X., King, I., & Yeung, D.-Y. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052580 · Cited by 862
Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012) · Cognitive Science · doi:10.1111/j.1551-6709.2012.01245.x · Cited by 772
“For example, student performance in the ASSISTment computer tutoring system (Feng et al., 2009) reveals that errors in simple fraction multiplication word problems (e.g., ''What is 3 ⁄ 4 of 1 ⁄ 2?'') are sometimes not about the math, but about vocabulary––follow-up questions indicate that students…”
BackgroundSelf-citationHeffernan, N. T., & Heffernan, C. L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0024-x · Cited by 616
Self-citationEnhancing the prediction of student performance based on the machine learning XGBoost algorithm
Asselman, A., Khaldi, M., & Aammou, S. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2021.1928235 · Cited by 490
Metadata
- Authors
- Feng, M., Heffernan, N., & Koedinger, K.
- Venue
- User Modeling and User-Adapted Interaction
- Published
- 2009-02-04
- Volume
- 19
- Issue
- 3
- Pages
- 243-266
- ISSN
- 0924-1868 (ISSN-L)
- 1573-1391
- OpenAlex
- W2066816090
- Cited by
- 502
- References
- 28
- FWCI
- 14.10
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.