Escape from the Skinner Box: The case for contemporary intelligent learning environments
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Intelligent Tutoring systems (ITSs) and Intelligent Learning Environments (ILEs) have been developed and evaluated over the last 40 years. Recent meta‐analyses show that they perform well enough to act as effective classroom assistants under the guidance of a human teacher. Despite this success, they have been criticised as embodying a retrograde behaviourist technology. They have also been caught up in broader controversies about the role of Artificial Intelligence in society and about the entry of big data companies into the education market and the harvesting of learner data. This paper concentrates on rebutting the criticisms of the pedagogy of ITSs and ILEs. It offers examples of how a much wider range of pedagogies are available than their critics claim. These wider pedagogies operate at both the screen level of individual systems, as well as at the classroom level within which the systems are orchestrated by the teacher. It argues that there are many ways that such systems can be integrated by the teacher into the overall experience of a class. Taken together, the screen‐level and orchestration‐level dramatically enlarge the range of pedagogies beyond what was possible with the “Skinner Box.”
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities
Palinscar, A. S., & Brown, A. L. (1984) · Cognition and Instruction · doi:10.1207/s1532690xci0102_1 · Cited by 5,186
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 1×
“This feature is based, in part, on the finding that the most effective tutoring is provided one-to-one by a skilled-human tutor who tailors the interaction to the specific capabilities, progress and needs of the tutee in order to achieve mastery at each stage (B. S. Bloom, 1984).”
BackgroundYeager, D. S., & Dweck, C. S. (2012) · Educational Psychologist · doi:10.1080/00461520.2012.722805 · Cited by 2,334
VanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 2×
“Many ILEs present an adaptive sequence of problem-statements and then help the learner to solve them, based on an AI-based analysis of what ought to be the next most useful problem for that particular student to solve (Anderson et al., 1995).”
BackgroundEffectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930 · cited 1×
“…of formal education, seven meta-reviews have compared the quality of the learning outcomes with ILES as compared to using more traditional methods (Kulik & Fletcher, 2016; Ma, Adesope, Nesbit, & Liu, 2014; Nesbit, Adesope, Liu, & Ma, 2014; Steenbergen-Hu & Cooper, 2013, 2014; VanLehn, 2011).”
MethodIntelligent tutoring systems and learning outcomes: A meta-analysis
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014) · Journal of Educational Psychology · doi:10.1037/a0037123 · Cited by 840 · cited 4×
“…of formal education, seven meta-reviews have compared the quality of the learning outcomes with ILES as compared to using more traditional methods (Kulik & Fletcher, 2016; Ma, Adesope, Nesbit, & Liu, 2014; Nesbit, Adesope, Liu, & Ma, 2014; Steenbergen-Hu & Cooper, 2013, 2014; VanLehn, 2011).”
BackgroundMethodInfluentialHeffernan, N. T., & Heffernan, C. L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0024-x · Cited by 616
Stupid tutoring systems, intelligent humans
Baker, R. S. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0105-0 · Cited by 425 · cited 1×
“R. S. Baker (2016) argues that the cost and time needed to develop ILEs has meant that the most widely-adopted systems have not incorporated some of the best features developed and evaluated in research labs.”
BackgroundDesign for classroom orchestration
Dillenbourg, P. (2013) · Computers & Education · doi:10.1016/j.compedu.2013.04.013 · Cited by 386 · cited 1×
“This addresses the issue of how the learners’ work with the system(s) is organised within the wider context of other learners, of their other activities and of the course they are undertaking, i.e. in formal educational settings (Dillenbourg, 2013).”
Background
Research works printed but not linked: 56
- 10.1016/j.chb.2014.03.053 · DOI
- Aleven V., 2017, Handbook of Research on Learning and Instruction
- 10.1145/2330601.2330666 · DOI · Paper page
- 10.1006/imms.1993.1013 · doi:10.1006/imms.1993.1013
- 10.1007/978-1-4419-5546-3 · DOI · Paper page
- Azevedo R., 2009, MetaTutor: A Metacognitive tool for enhancing self‐regulated learning
- Baker R., 2008, Why students engage in "gaming the system" behaviours in interactive learning environments, Journal of Interactive Learning Research, 19, 185
- 10.1007/s40593-015-0057-9 · DOI · Paper page
- 10.3102/0091732X004001003 · DOI · Paper page
- Bloom B. S., 1968, Learning for mastery. Instruction and curriculum. Regional education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1, Evaluation Comment, 1, n2
- Bloom B. S., 1971, Mastery learning, theory and practice, 47
- Bloom C. P., 1995, Roadblocks to successful ITS authoring in industry
- 10.1016/S0020-7373(79)80003-6 · DOI · Paper page
- Casas I., 2015, Culture‐oriented factors in the implementation of intelligent tutoring systems in Chile
- 10.1145/2851581.2892417 · DOI
- 10.1007/s40593-015-0083-7 · DOI
- 10.1016/S0020-7373(79)80004-8 · DOI · Paper page
- D’Mello S., 2011, Affective prospecting, 245
- D'Mello S., 2008, AutoTutor detects and responds to learners affective and cognitive states
- Boulay B., 2018, Tutoring and intelligent tutoring systems, 103
- Harsley R., 2017, Collaborative intelligent tutoring systems: Comparing learner outcomes across varying collaboration feedback strategies
- Herold B., 2017, The case(s) against personalized learning, Education Week, 37, 4
- Keller J. M., 2008, An integrative theory of motivation, volition, and performance, Technology, Inststruction, Cognition and Learning, 6, 79
- 10.1007/s40593-015-0082-8 · DOI · Paper page
- Koedinger K. R., 1997, Intelligent tutoring goes to school in the big city, International Journal of Artificial Intelligence in Education, 8, 30
- 10.1007/978-3-642-39112-5_32 · DOI
- 10.3233/IRG-2008-18(3)02 · DOI · Paper page
- Lehman B., 2013, Inducing and tracking confusion with contradictions during complex learning, International Journal of Artificial Intelligence in Education, 22, 85, 10.3233/JAI-130025 · DOI
- Maehr M. L., 2012, Encouraging a continuing personal investment in learning: Motivation as an instructional outcome
- Martin N. D., 2018, Fading distributed scaffolds: The importance of complementarity between teacher and material scaffolds, Instructional Science, 1
- Martinez‐Maldonado R., 2013, MTClassroom and MTDashboard: Supporting analysis of teacher attention in an orchestrated multi‐tabletop classroom
- 10.1007/s40593-014-0020-1 · DOI · Paper page
- 10.1016/j.chb.2015.08.038 · DOI
- 10.1007/s11257-016-9181-y · DOI
- 10.1109/ICALT.2014.38 · DOI · Paper page
- 10.1007/s40593-014-0029-5 · DOI · Paper page
- 10.1016/S0020-7373(79)80007-3 · DOI · Paper page
- 10.1007/s40593-014-0018-8 · DOI · Paper page
- 10.14742/ajet.4314 · DOI · Paper page
- 10.1177/2332858416673968 · DOI · Paper page
- Rosenberg R., 1987, A critical analysis of research on intelligent tutoring systems, Educational Technology, 27, 7
- 10.1177/0022487103257089 · DOI · Paper page
- Skinner B. F., 1968, The technology of teaching
- 10.1007/BF02504925 · DOI · Paper page
- 10.1037/a0032447 · DOI · Paper page
- Tabor N., 2018, Brooklyn students are protesting silicon valley's favorite education program, Intelligencer
- 10.1007/s11423-014-9343-8 · DOI
- VanLehn K., 2005, The Andes physics tutoring system: Lessons learned, International Journal of Artificial Intelligence in Education, 15, 147, 10.3233/IRG-2005-15(3)02 · DOI
- 10.1007/s40593-016-0100-5 · DOI
- 10.1142/S179320680900074X · DOI
- Watters A., 2015, Education technology and Skinner's box
- Watters A., 2017, Dunce's app: How silicon valley's brand of behaviorism has entered the classroom, The Baffler
- Williamson B., 2018, Learning lessons from data controversies
- 10.1080/17508487.2016.1186710 · DOI · Paper page
- Williamson B., 2018, The tech elite is making a power‐grab for public education
- 10.1108/IJILT-09-2016-0040 · DOI · Paper page
Cited by
A Review of Artificial Intelligence (AI) in Education from 2010 to 2020
Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J.-B., Yuan, J., & Li, Y. (2021) · Complexity · doi:10.1155/2021/8812542 · Cited by 1,364
“ID Authors Researchquestion Technology adoption Learning Subject Educational level Approach Effects 50 Cukurova et al.[69] REA Artificial intelligence and multimodal data Debating skills 127 questionnaires and 47 audio recordings from candidates who have applied to become a tutor SUR, DA OTH+ 51 Du Boulay [70] ADP Int…”
BackgroundState of the art and practice in AI in education
Holmes, W., & Tuomi, I. (2022) · European Journal of Education · doi:10.1111/ejed.12533 · Cited by 1,093
Artificial Intelligence in Education: The Three Paradigms
Ouyang, F., & Jiao, P. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100020 · Cited by 991
Historical threads, missing links, and future directions in AI in education
Williamson, B., & Eynon, R. (2020) · Learning, Media and Technology · doi:10.1080/17439884.2020.1798995 · Cited by 981
AI-enabled adaptive learning systems: A systematic mapping of the literature
Kabudi, T., Pappas, I., & Olsen, D. H. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100017 · Cited by 734
Power to the Teachers: An Exploratory Review on Artificial Intelligence in Education
Lameras, P., & Arnab, S. (2021) · Information · doi:10.3390/info13010014 · Cited by 291
“ITSs utilise AI techniques and prediction mechanisms to adapt and scaffold the experience of the individual student for improving the quality of learning as well as minimising the learning time (du Boulay [71]).”
MethodUnderstanding User Trust in Artificial Intelligence-Based Educational Systems: Evidence from China
Qin, F., Li, K., & Yan, J. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12994 · Cited by 204
The ethics of using AI in K-12 education: a systematic literature review
Gouseti, A., James, F., Fallin, L., & Burden, K. (2025) · Technology, Pedagogy and Education · doi:10.1080/1475939x.2024.2428601 · Cited by 141
Buckingham Shum, S., Lim, L.-A., Boud, D., Bearman, M., & Dawson, P. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00410-9 · Cited by 77
Wang, S., Yu, H., Hu, X., & Li, J. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12998 · Cited by 71
Effects and acceptance of precision education in an AI-supported smart learning environment
Hu, Y.-H. (2022) · Education and Information Technologies · doi:10.1007/s10639-021-10664-3 · Cited by 68
“The conduit for AI, if being applied in the learning environment, accentuates its statistical aspects of data mining to collect and analyze learners’ behaviors (Du Boulay, 2019).”
BackgroundFurlan, R., Gatti, M., Menè, R., Shiffer, D., Marchiori, C., Giaj Levra, A., Saturnino, V., Brunetta, E., & Dipaola, F. (2021) · JMIR Medical Informatics · doi:10.2196/24073 · Cited by 67
Tuomi, I. (2022) · European Journal of Education · doi:10.1111/ejed.12531 · Cited by 66
Human learning and learning analytics in the age of artificial intelligence.
Istenič Starčič, A. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12879 · Cited by 59
“…help comes in two ways: in the design phase when learning science informs AI design, and during the learning process where AI is a powerful tool for gathering and analysing data about learner (Du Boulay, 2019; Luckin & Cukurova, 2019; Mavrikis, Geraniou, Gutiérrez-Santos, & Poulovassilis, 2019).”
BackgroundReplay Enactments: Exploring Possible Futures through Historical Data
Holstein, K., Harpstead, E., Gulotta, R., & Forlizzi, J. (2020) · Proceedings of the 2020 ACM Designing Interactive Systems Conference · doi:10.1145/3357236.3395427 · Cited by 19
Elliott, S., Richter, M., Alozie, C., & Raidis, D. (2026) · Technology, Knowledge and Learning · doi:10.1007/s10758-025-09942-7 · Cited by 3
Metadata
- Authors
- du Boulay, B.
- Venue
- British Journal of Educational Technology
- Publisher
- Wiley
- Published
- 2019-07-15
- Volume
- 50
- Issue
- 6
- Pages
- 2902-2919
- ISSN
- 0007-1013 (ISSN-L)
- 1467-8535
- OpenAlex
- W2960651334
- Cited by
- 79
- References
- 16
- FWCI
- 3.57
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.