CoChat
Sign Up
Cited by 616Intelligent Tutoring Systems and Adaptive LearningOnline Learning and AnalyticsEducational Technology and E-Learning

The DOI matches its Crossref record; 68 of the 71 references it cites were found in a registry or an index.

The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching

Open access
Heffernan, Neil T., Heffernan, Cristina Lindquist·2014·International Journal of Artificial Intelligence in Education
DOI 10.1007/s40593-014-0024-x
Open this paper in CoChat — free account

Abstract

The ASSISTments project is an ecosystem of a few hundred teachers, a platform, and researchers working together. Development professionals help train teachers and get teachers to participate in studies. The platform and these teachers help researchers (sometimes explicitly and sometimes implicitly) simply by using content the teacher selects. The platform, hosted by Worcester Polytechnic Institute, allows teachers to write individual ASSISTments (composed of questions with answers and associated hints, solutions, web-based videos, etc.) or to use pre-built ASSISTments, bundle them together in a problem set, and assign these to students. The system gives immediate feedback to students while they are working and provides student-level data to teachers on any assignment. The word “ASSISTments” blends tutoring “assistance” with “assessment” reporting to teachers and students. While originally focused on mathematics, the platform now has content from many other subjects (e.g., science, English, Statistics, etc.). Due to the large library of mathematics content, however, it is mostly used by math teachers. Over 50,000 students used ASSISTments last school year (2013–4) and this number has been doubling each year for the last 8 years. The platform allows any user, mostly researchers, to create randomized controlled trials in the content, which has helped us use the tool in over 18 published and an equal number of unpublished studies. The data collected by the system has also been used in a few dozen peer-reviewed data mining publications. This paper will not seek to review these publications, but instead we will share why ASSISTments has been successful and what lessons were learned along the way. The first lesson learned was to build a platform for learning sciences, not a product that focused on a math topic. That is, ASSISTments is a tool, not a curriculum. A second lesson learned is expressed by the mantra “Put the teacher in charge, not the computer.” This second lesson is about building a flexible system that allows teachers to use the tool in concert with the classroom routine. Once teachers are using the tool they are more likely to want to participate in research studies. These lessons were born from the design decisions about what the platform supports and does not support. In conclusion, goals for the future will be presented.

Overview

Integrity signals

Registry

Crossref · 10.1007/s40593-014-0024-x

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 71 references

Research works printed but not linked: 8

  1. Heffernan, N. T (2001). Intelligent tutoring systems have forgotten the tutor: adding a cognitive model of human tutors dissertation. Computer Science Department, School of Computer Science, Carnegie Mellon University.
  2. Heffernan, N., Heffernan, C., Dietz, K., Soffer, D., Pellegrino, J. W., Goldman, S. R. & Dailey, M. (2012). Improving mathematical learning outcomes through automatic reassessment and relearning. Proceedings of AERA 2012. Vancouver, BC, Canada.
  3. Kelly, K., Heffernan, N., D’Mello, S., Namias, J., & Strain, A. (2013a). Adding teacher-created motivational video to an ITS. Florida artificial intelligence research society (FLAIRS 2013). pp. 503–508.
  4. Pardos, Z. & Heffernan, N. (2012). Tutor modeling vs. Student modeling. Proceedings of the twenty-fifth international Florida artificial intelligence research society conference. Invited talk. Florida Artificial Intelligence Research Society (FLAIRS 2012). pp 420–425.
  5. Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K. R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C., Choksey, S., Livak, T., Mercado, E.,Turner, T.E., Upalekar. R, Walonoski, J.A., Macasek. M.A. & Rasmussen, K.P. (2005). The assistment project: blending assessment and assisting. In C.K.Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th artificial intelligence in education, Amsterdam: ISO Press. pp. 555–562.
  6. Soffer, D., Das, V., Pellegrino, G., Goldman, S., Heffernan, N., Heffernan, C.,& Dietz, K. (2014) Improving long-term retention of mathematical knowledge through automatic reassessment and relearning. American educational research association (AERA 2014) Conference. Division C - learning and instruction/section 1c: mathematics. PDF, poster nominated for the best poster of the session.
  7. Whorton, S. (2013). Can a computer adaptive assessment system determine, better than traditional methods, whether students know mathematics skills? Masters thesis, computer science department. Retrieved on Jan 10, 2014 from http://www.wpi.edu/Pubs/ETD/Available/etd-041913-095912/
  8. WPI IRB (2014) Terms of use of using the platform to to run a study. Worcester polytechnic institute’s institutional review board. Retrieved Augusts 7th, 2014 from https://sites.google.com/site/assistmentstestbed/terms-of-use

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Heffernan, N. T., & Heffernan, C. L.
Affiliations
Worcester Polytechnic Institute (US)
Corresponding author
Heffernan, Neil T.
ORCID
Venue
International Journal of Artificial Intelligence in Education
Publisher
Elsevier BV
Published
2014-12
Volume
24
Issue
4
Pages
470-497
ISSN
1560-4292 (ISSN-L)
DOI
10.1007/s40593-014-0024-x
OpenAlex
W1995864528
Cited by
616
References
71
Citations by year
2026: 92 · 2025: 96 · 2024: 87 · 2023: 65 · 2022: 40 · 2021: 54 · 2020: 39 · 2019: 33 · 2018: 32 · 2017: 31 · 2016: 27 · 2015: 19 · 2014: 1
FWCI
25.68
Type
journal article
Language
en
Open access
Licence
Locations
International Journal of Artificial Intelligence in Education (publishedVersion, cc-by-nc-nd)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Information Systems › Educational Technology and E-Learning
Keywords
ASSISTments, online learning platforms, mathematics education, teacher participation in research, randomized controlled trials, student learning data, immediate feedback, teacher autonomy, learning sciences, educational technology design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review