The DOI matches its Crossref record; 71 of the 73 references it cites were found in a registry or an index.
A Conceptual Framework for Human–AI Hybrid Adaptivity in Education
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Continuously interwoven explanation of AI behavior and exploratory adaptation can improve both AI and human learning (Holstein et al. 2020).”
From Keeping the organization in the loop: a socio-technical extension of human-centered artificial intelligence (2023) · cites this paper as Holstein et al. 2020
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. (2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 25,268
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,825
(1991) · Choice Reviews Online · doi:10.5860/choice.28-5376 · Cited by 4,218
VanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982
An introduction to human factors engineering
Wickens, C. D. (1997) · Cited by 1,711
Artificial Intelligence: A New Synthesis
(1998) · Elsevier eBooks · doi:10.1016/c2009-0-27773-7 · Cited by 1,211
Intelligent Tutoring Goes To School in the Big City
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (2018) · doi:10.1184/r1/6470153 · Cited by 1,105
Fox, C. (2013) · Routledge eBooks · doi:10.4324/9781315009810-7 · Cited by 1,066
Vinkhuyzen, R. E., Verschure, P. F. M. J., & Newell, A. (1994) · The American Journal of Psychology · doi:10.2307/1422886 · Cited by 831
Heffernan, N. T., & Heffernan, C. L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0024-x · Cited by 616
Soller, A., Martínez‐Monés, A., Jermann, P., & Muehlenbrock, M. (2005) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2005-15(4)02 · Cited by 611
Schoenfeld, A. H. (2010) · Routledge eBooks · doi:10.4324/9780203843000 · Cited by 482
A review of recent advances in learner and skill modeling in intelligent learning environments
Desmarais, M. C., & Baker, R. S. (2011) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-011-9106-8 · Cited by 477
Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity
Holstein, K., McLaren, B. M., & Aleven, V. A. W. M. M. (2019) · Journal of Learning Analytics · doi:10.18608/jla.2019.62.3 · Cited by 429
Self-citationStupid tutoring systems, intelligent humans
Baker, R. S. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0105-0 · Cited by 425
Design for classroom orchestration
Dillenbourg, P. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.04.013 · Cited by 386
Help Helps, But Only So Much: Research on Help Seeking with Intelligent Tutoring Systems
Aleven, V. A. W. M. M., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0089-1 · Cited by 300
Self-citationHandbook of Research on Learning and Instruction
(2016) · Routledge eBooks · doi:10.4324/9781315736419 · Cited by 289
Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms
Holstein, K., McLaren, B. M., & Aleven, V. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93843-1_12 · Cited by 235
Self-citationIntelligent tutors as teachers' aides
Holstein, K., McLaren, B. M., & Aleven, V. A. W. M. M. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027451 · Cited by 194
Self-citationHolstein, K., McLaren, B. M., & Aleven, V. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-23204-7_14 · Cited by 189
Self-citationInstruction Based on Adaptive Learning Technologies
Aleven, V., McLaughlin, E. A., Glenn, R. A., & Koedinger, K. R. (2025) · Routledge eBooks · doi:10.4324/9781003484776-26 · Cited by 184
Self-citationSusan Bull, A., & Kay, J. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0090-8 · Cited by 180
Orchestrating technology enhanced learning: a literature review and a conceptual framework
Prieto, L. P., Dlab, M. H., Gutié, I., rrez, N. A., Abdulwahed, M. S., & Balid, W. (2011) · International Journal of Technology Enhanced Learning · doi:10.1504/ijtel.2011.045449 · Cited by 178
Example-Tracing Tutors: Intelligent Tutor Development for Non-programmers
Aleven, V. A. W. M. M., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M. A., & Koedinger, K. R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0088-2 · Cited by 153
Self-citationWilliams, J. J., Kim, J., Rafferty, A. N., Maldonado, S., Gajos, K. Z., Lasecki, W. S., & Heffernan, N. T. (2016) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2876034.2876042 · Cited by 147
Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration
Lakkaraju, H., Kamar, E. S., Caruana, R., & Horvitz, E. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10821 · Cited by 146
Adaptive Intelligent Support to Improve Peer Tutoring in Algebra
Walker, E., Rummel, N., & Koedinger, K. R. (2013) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-013-0001-9 · Cited by 142
Self-citationMathematics teacher noticing seeing through teachers' eyes
Sherin, M. G. (2011) · Cited by 140
The Evolution of Research on Digital Education
Dillenbourg, P. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0106-z · Cited by 136
What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations?
van Leeuwen, A., Rummel, N., & van Gog, T. (2019) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-019-09299-x · Cited by 105
Self-citationMartínez‐Maldonado, R., Clayphan, A. J., Yacef, K., & Kay, J. (2014) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2014.2365027 · Cited by 104
Developing Emotion-Aware, Advanced Learning Technologies: A Taxonomy of Approaches and Features
Harley, J. M., Lajoie, S. P., Frasson, C., & Hall, N. C. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0126-8 · Cited by 101
How Mastery Learning Works at Scale
RITTER, S., Yudelson, M. V., Fancsali, S. E., & Berman, S. R. (2016) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2876034.2876039 · Cited by 88
Teaching the Teacher: Tutoring SimStudent Leads to More Effective Cognitive Tutor Authoring
Matsuda, N., Cohen, W. W., & Koedinger, K. R. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0020-1 · Cited by 86
Tissenbaum, M., & Slotta, J. (2019) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-019-09306-1 · Cited by 79
Schofield, J. W., Eurich‐Fulcer, R., & Britt, C. L. (1994) · American Educational Research Journal · doi:10.3102/00028312031003579 · Cited by 72
Echeverria, V., Martinez-Maldonado, R., Buckingham Shum, S., Chiluiza, K., Granda, R., & Conati, C. (2018) · Journal of Learning Analytics · doi:10.18608/jla.2018.53.6 · Cited by 70
The TA Framework: Designing Real-time Teaching Augmentation for K-12 Classrooms
An, P., Holstein, K., d'Anjou, B., Eggen, B., & Bakker, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376277 · Cited by 69 · cited 8×
“…some technologies are designed to present information to humans with low-level, minimally pre-interpreted data (e.g., “number of help requests”) [4, 5, 16], several of the AIEd systems discussed above, including APTA , the Help Tutor , and Lumilo rely upon advanced student modeling techniques…”
BackgroundInfluentialSelf-citationCreem-Regehr, S. H., & Kunz, B. R. (2010) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.82 · Cited by 65
Rodríguez‐Triana, M. J., Prieto, L. P., Martínez‐Monés, A., Asensio-Pérez, J. I., & Dimitriadis, Y. A. (2018) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3170358.3170364 · Cited by 61
One framework to rule them all? Carrying forward the conversation started by Wise and Schwarz
Rummel, N. (2018) · International Journal of Computer-Supported Collaborative Learning · doi:10.1007/s11412-018-9273-2 · Cited by 49
Self-citationOrchestration tools to support the teacher during student collaboration: a review
van Leeuwen, A., & Rummel, N. (2019) · Unterrichtswissenschaft · doi:10.1007/s42010-019-00052-9 · Cited by 47
Self-citationAn, P., Bakker, S., Ordanovski, S., Taconis, R., Paffen, C. L. E., & Eggen, B. H. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300321 · Cited by 47
Design and evaluation of teacher assistance tools for exploratory learning environments
Mavrikis, M., Gutiérrez-Santos, S., & Poulovassilis, A. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883909 · Cited by 44
Automated detection of proactive remediation by teachers in reasoning mind classrooms
Miller, W. L., Baker, R. S., Labrum, M. J., Petsche, K., Liu, Y., & Wagner, A. Z. (2015) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2723576.2723607 · Cited by 43
Myers, G. E. (2017) · Yale University Press eBooks · doi:10.12987/9780300158045-006 · Cited by 43
Intelligent teaching assistant systems
Yacef, K. (2003) · International Conference on Computers in Education · doi:10.1109/cie.2002.1185885 · Cited by 42
Chen, N.-C., Suh, J., Verwey, J., Ramos, G., Drucker, S. M., & Simard, P. (2018) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3172944.3172950 · Cited by 40
Regulative Loops, Step Loops and Task Loops
VanLehn, K. A. (2015) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0056-x · Cited by 40
Studying Teacher Orchestration Load in Technology-Enhanced Classrooms
Prieto, L. P., Sharma, K., & Dillenbourg, P. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24258-3_20 · Cited by 39
Holstein, K., McLaren, B. M., & Aleven, V. (2017) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3027385.3027450 · Cited by 34
Self-citationVanLehn, K. A., Burkhardt, H., Cheema, S., Kang, S., Pead, D., Schoenfeld, A. H., & Wetzel, J. (2019) · Interactive Learning Environments · doi:10.1080/10494820.2019.1616567 · Cited by 33
Towards Hybrid Human-System Regulation
Molenaar, I., Horvers, A., & Baker, R. S. (2019) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3303772.3303780 · Cited by 31
Students’ Understanding of Their Student Model
Long, Y., & Aleven, V. A. W. M. M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21869-9_25 · Cited by 30
Self-citationKessler, A., Boston, M. D., & Stein, M. K. (2019) · Information and Learning Sciences · doi:10.1108/ils-07-2019-0075 · Cited by 26
The Apprentice Learner architecture: Closing the loop between learning theory and educational data.
MacLellan, C. J., Harpstead, E., Patel, R., & Koedinger, K. R. (2016) · Educational Data Mining · Cited by 24
Predicting Student Performance in a Collaborative Learning Environment.
Olsen, J. K., Aleven, V. A. W. M. M., & Rummel, N. (2015) · Grantee Submission · Cited by 21
Self-citationBarriers to ITS Adoption: A Systematic Mapping Study
Nye, B. D. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07221-0_74 · Cited by 20
Opening Up an Intelligent Tutoring System Development Environment for Extensible Student Modeling
Holstein, K., Yu, Z., Sewall, J., Popescu, O., McLaren, B. M., & Aleven, V. A. W. M. M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93843-1_13 · Cited by 19
Self-citationCognitive Tutor Use in Chile: Understanding Classroom and Lab Culture
Ogan, A., Yarzebinski, E., Fernández, P., & Casas, I. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_32 · Cited by 16
Towards a Framework for Smart Classrooms that Teach Instructors to Teach
Gerritsen, D., Zimmerman, J., & Ogan, A. (2018) · International Conference of Learning Sciences · Cited by 13
Some less obvious features of classroom orchestration systems
VanLehn, K. A., Cheema, S. A., Wetzel, J., & Pead, D. (2016) · Cited by 12
Intelligent Instructional Hand Offs.
Fancsali, S. E., Yudelson, M. V., Berman, S., & RITTER, S. (2018) · Educational Data Mining · Cited by 10
Design Perspectives of Learning at Scale
Kulkarni, C. E. (2019) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3330430.3333620 · Cited by 9
Orchestrating Combined Collaborative and Individual Learning in the Classroom
Olsen, J. K. (2017) · KiltHub Repository · doi:10.1184/r1/6720983 · Cited by 9
Self-Regulation of Learning with Multiple Representations in Hypermedia
Cromley, J. G., Azevedo, R., & Olson, E. (2005) · Cited by 8
Learning the Features Used To Decide How to Teach
Lee, M. H., Runde, J., Jibril, W., Wang, Z., & Brunskill, E. (2015) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2724660.2728707 · Cited by 4
Olsen, J. K., Rummel, N., & Aleven, V. (2018) · Cited by 4
Self-citationNick Chater, A., & Oaksford, M. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x17000073 · Cited by 4
Research works printed but not linked: 6
- Bransford, J.D., Brown, A.L., Cocking, R.R.: How People Learn, vol. 11. National Academy Press, Washington, DC (2000)
- De-Arteaga, M., Fogliato, R., Chouldechova, A.: A case for humans-in-the-loop: decisions in the presence of erroneous algorithmic scores (2020). arXiv preprint arXiv:2002.08035 · arXiv
- Holstein, K.: Designing real-time teacher augmentation to combine strengths of human and AI instruction. Unpublished doctoral dissertation, Carnegie Mellon University (2019)
- Roll, I., Wiese, E.S., Long, Y., Aleven, V., Koedinger, K.R.: Tutoring self-and co-regulation with intelligent tutoring systems to help students acquire better learning skills. In: Sottilare, R., Graesser, A., Hu, X., Holden, H. (eds.) Design Recommendations for Intelligent Tutoring Systems, vol. 2, pp. 169–182. US Army Research Laboratory, Orlando (2014)
- Simard, P.Y., et al.: Machine teaching: a new paradigm for building machine learning systems (2017). arXiv preprint arXiv:1707.06742 · arXiv
- Zhu, X., Singla, A., Zilles, S., Rafferty, A.N.: An overview of machine teaching (2018). arXiv preprint arXiv:1801.05927 · arXiv
Cited by
Sorted by citations
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13544 · Cited by 805
The impact of artificial intelligence on learner–instructor interaction in online learning
Seo, K., Tang, J., Roll, I., Fels, S., & Yoon, D. (2021) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-021-00292-9 · Cited by 788
Jeon, J., & Lee, S. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11834-1 · Cited by 696
The promise and challenges of generative AI in education
Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Hernandez-Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2025) · Behaviour & Information Technology · doi:10.1080/0144929x.2024.2394886 · Cited by 543
Bernacki, M. L., Greene, M. J., & Lobczowski, N. G. (2021) · Educational Psychology Review · doi:10.1007/s10648-021-09615-8 · Cited by 618
Metadata
- Authors
- Holstein, K., Aleven, V., & Rummel, N.
- Affiliations
- Carnegie Mellon University (US)
- Ruhr University Bochum (DE)
- Corresponding author
- Holstein, Kenneth
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2020
- Pages
- 240-254
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- PMC
- PMC7334162
- OpenAlex
- W3038925583
- Cited by
- 168
- References
- 73
- Citations by year
- 2026: 39 · 2025: 35 · 2024: 36 · 2023: 25 · 2022: 21 · 2021: 10 · 2020: 2
- FWCI
- 5.54
- Type
- conference paper
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Lecture notes in computer science (publishedVersion)
- PubMed Central (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- artificial intelligence in education, adaptive learning systems, human-AI collaboration, conceptual framework, human-AI augmentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
