CoChat
Sign Up
PreprintCited by 81Intelligent Tutoring Systems and Adaptive LearningOnline Learning and AnalyticsRecommender Systems and Techniques

The DOI matches its DataCite record.

RiPPLE: A Crowdsourced Adaptive Platform for Recommendation of Learning Activities

Open access
Khosravi, Hassan, Kitto, Kirsty, Williams, Joseph Jay·2019·arXiv
DOI 10.48550/arxiv.1910.05522arXiv 1910.05522
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper presents a platform called RiPPLE (Recommendation in Personalised Peer-Learning Environments) that recommends personalized learning activities to students based on their knowledge state from a pool of crowdsourced learning activities that are generated by educators and the students themselves. RiPPLE integrates insights from crowdsourcing, learning sciences, and adaptive learning, aiming to narrow the gap between these large bodies of research while providing a practical platform-based implementation that instructors can easily use in their courses. This paper provides a design overview of RiPPLE, which can be employed as a standalone tool or embedded into any learning management system (LMS) or online platform that supports the Learning Tools Interoperability (LTI) standard. The platform has been evaluated based on a pilot in an introductory course with 453 students at The University of Queensland. Initial results suggest that the use of the \name platform led to measurable learning gains and that students perceived the platform as beneficially supporting their learning.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1910.05522

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • RiPPLE has been deployed in higher education courses across multiple disciplines, and prior research has examined its effects on learning gains, engagement, and the quality of student-authored learning resources.

    “RiPPLE has been deployed in higher education courses across multiple disciplines, and prior research has examined its effects on learning gains, engagement, and the quality of student-authored learning resources [66].”

    From Making AI-Generated Feedback Matter: A Large-Scale Study of Feedback Workflows and Student Enactment (2026) · cites this paper as [66]

    The Introduction and §4 Evaluation report a RiPPLE pilot in one university relational-databases course and examine learning gains. The evaluation records student activity and ratings of resources, but does not establish deployment across multiple disciplines or examine RiPPLE’s effects on engagement and the quality of student-authored resources as claimed. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 54

  1. In C. F. Lynch, A. Merceron, M. Desmarais, & R. Nkambou (Eds.), Educational Data Mining 2019: 12th International Conference on Educational Data Mining (EDM2019), 2-5 July 2019, Montreal, Canada (pp. 228-233). International Educational Data Mining Society. Retrieved from http://educationaldatamining.org/edm2019/proceedings/
  2. Anderson, J. R., Boyle, C. F., & Reiser, B. J. (1985). Intelligent tutoring systems. Science, 228(4698), 456-462. https://dx.doi.org/10.1126/science.228.4698.456 · DOI
  3. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences, 4(2), 167-207. https://dx.doi.org/10.1207/s15327809jls0402 2 · DOI · Paper page
  4. Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399-424. https://dx.doi.org/10.1080/00273171.2011.568786 · DOI · Paper page
  5. Banks, J., Cochran-Smith, M., Moll, L., Richert, A., Zeichner, K., LePage, P., . . . McDonald, M. (2005). Teaching diverse learners. In L. Darling-Hammond & J. Bransford (Eds.), Preparing Teachers for a Changing World: What Teachers Should Learn and Be Able to Do (pp. 232-274). San Francisco, California, USA: Jossey-Bass.
  6. Barak, M., & Rafaeli, S. (2004). On-line question-posing and peer-assessment as means for web- based knowledge sharing in learning. International Journal of Human-Computer Studies, 61(1), 84-103. https://dx.doi.org/10.1016/j.ijhcs.2003.12.005 · DOI
  7. Bates, S. P., Galloway, R. K., & McBride, K. L. (2012). Student-generated content: Using PeerWise to enhance engagement and outcomes in introductory physics courses. AIP Conference Proceedings, 1413(1), 123-126. https://dx.doi.org/10.1063/1.3680009 · DOI
  8. Bates, S. P., Galloway, R. K., Riise, J., & Homer, D. (2014). Assessing the quality of a student- generated question repository. Physical Review Special Topics -Physics Education Research, 10(2), 020105. https://dx.doi.org/10.1103/physrevstper.10.020105 · DOI · Paper page
  9. Biggs, J. (2012). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 31(1), 39-55. https://dx.doi.org/10.1080/07294360.2012.642839 · DOI
  10. Bodily, R., Kay, J., Aleven, V., Jivet, I., Davis, D., Xhakaj, F., & Verbert, K. (2018). Open learner models and learn- ing analytics dashboards: A systematic review. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), 5-9 March, 2018, Sydney, Australia (pp. 41-50). New York, USA: ACM. https://dx.doi.org/10.1145/3170358.3170409 · DOI
  11. In P. J. Durlach & A. M. Les- gold (Eds.), Adaptive Technologies for Training and Education (p. 46-66). Cambridge University Press. https://dx.doi.org/10.1017/CBO9781139049580.006 · DOI
  12. Brusilovsky, P., & Millán, E. (2007). User models for adaptive hypermedia and adaptive educational systems. In P. Brusilovsky, A. Kobsa, & W. Nejdl (Eds.), The Adaptive Web: Methods and Strategies of Web Personalization. Lecture Notes in Computer Science (Vol. 4321, pp. 3-53). Springer Berlin Heidelberg. https://dx.doi.org/10.1007/978-3-540-72079-9 · DOI
  13. Bull, S., Ginon, B., Boscolo, C., & Johnson, M. (2016). Introduction of learning visualisations and metacognitive sup- port in a persuadable open learner model. In Proceedings of the Sixth International Conference on Learning An- alytics & Knowledge (LAK '16), 25-29 April, 2016, Edinburgh, Scotland (pp. 30-39). New York, USA: ACM. https://dx.doi.org/10.1145/2883851.2883853 · DOI
  14. Bull, S., & Kay, J. (2010). Open learner models. In R. Nkambou, J. Bourdeau, & R. Mizoguchi (Eds.), Advances in Intelligent Tutoring Systems (Vol. 308, pp. 301-322). Springer Berlin Heidelberg. https://dx.doi.org/10.1007/978-3-642-14363-2 1 · DOI
  15. Chambers, J. M., Cleveland, W. S., Kleiner, B., & Tukey, P. A. (1983). Graphical Methods for Data Analysis. Boca Raton, Florida, USA: CRC Press.
  16. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://dx.doi.org/10.1037/0033-2909.112.1.155 · DOI · Paper page
  17. Collins, A., & Halverson, R. (2018). Rethinking Education in the Age of Technology: The Digital Revolution and Schooling in America. New York, USA: Teachers College Press.
  18. Cooper, K., & Khosravi, H. (2018). Graph-based visual topic dependency models: Supporting assessment design and delivery at scale. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), 5-9 March, 2018, Sydney, Australia (pp. 11-15). New York, USA: ACM. https://dx.doi.org/10.1145/3170358.3170418 · DOI
  19. Denny, P., Hamer, J., & Luxton-Reilly, A. (2009). Students sharing and evaluating MCQs in a large first year engineering course. In 20th Annual Conference for the Australasian Association for Engineering Education (AAEE '09), 6-9 December 2009, The University of Adelaide, Adelaide, Australia (pp. 575-580). Barton, Australia: Engineers Australia.
  20. Denny, P., Hamer, J., Luxton-Reilly, A., & Purchase, H. (2008). PeerWise: Students sharing their multiple choice questions. In Proceedings of the Fourth International Workshop on Computing Education Research (ICER '08), 6-7 September 2008, Sydney, Australia (pp. 51-58). New York, USA: ACM. https://dx.doi.org/10.1145/1404520.1404526 · DOI · Paper page
  21. Draper, S. W. (2009). Catalytic assessment: Understanding how MCQs and EVS can foster deep learning. British Journal of Educational Technology, 40(2), 285-293. https://dx.doi.org/10.1111/j.1467-8535.2008.00920.x · DOI
  22. Essa, A. (2016). A possible future for next generation adaptive learning systems. Smart Learning Environments, 3(1), 16. https://dx.doi.org/10.1186/s40561-016-0038-y · DOI
  23. Falmagne, J.-C., Cosyn, E., Doignon, J.-P., & Thiéry, N. (2006). The assessment of knowledge, in theory and in practice. In R. Missaoui & J. Schmidt (Eds.), Formal Concept Analysis. Lecture Notes in Computer Science (Vol. 3874, pp. 61-79). Springer Berlin Heidelberg.
  24. Galloway, K. W., & Burns, S. (2015). Doing it for themselves: Students creating a high quality peer-learning environment. Chemistry Education Research and Practice, 16(1), 82-92. https://dx.doi.org/10.1039/c4rp00209a · DOI · Paper page
  25. Heffernan, N. T., & Heffernan, C. L. (2014). The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470-497. https://dx.doi.org/10.1007/s40593-014-0024-x · DOI · Paper page
  26. Heffernan, N. T., Ostrow, K. S., Kelly, K., Selent, D., Van Inwegen, E. G., Xiong, X., & Williams, J. J. (2016). The future of adaptive learning: Does the crowd hold the key? International Journal of Artificial Intelligence in Education, 26(2), 615-644. https://dx.doi.org/10.1007/s40593-016-0094-z · DOI
  27. Karataev, E., & Zadorozhny, V. (2017). Adaptive social learning based on crowdsourcing. IEEE Transactions on Learning Technologies, 10(2), 128-139. https://dx.doi.org/10.1109/TLT.2016.2515097 · DOI
  28. Khosravi, H., & Cooper, K. (2018). Topic dependency models: Graph-based visual analytics for communicating assessment data. Journal of Learning Analytics, 5(3), 136-153. https://dx.doi.org/10.18608/jla.2018.53.9 · DOI
  29. Khosravi, H., Cooper, K., & Kitto, K. (2017). RiPLE: Recommendation in peer-learning environments based on knowledge gaps and interests. Journal of Educational Data Mining, 9(1), 42-67. Retrieved from https://jedm.educationaldatamining.org/index.php/JEDM/article/view/239
  30. King, A. (1992). Facilitating elaborative learning through guided student-generated questioning. Educational Psychologist, 27(1), 111-126. https://dx.doi.org/10.1207/s15326985ep2701 8 · DOI · Paper page
  31. Kitto, K., Lupton, M., Davis, K., & Waters, Z. (2017). Designing for student-facing learning analytics. Australasian Journal of Educational Technology, 33(5), 152-168. https://dx.doi.org/10.14742/ajet.3607 · DOI
  32. Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901-918. https://dx.doi.org/10.1037/a0037123 · DOI · Paper page
  33. Matthews, K. E. (2017). Five propositions for genuine students as partners practice. International Journal for Students as Partners, 1(2). https://dx.doi.org/10.15173/ijsap.v1i2.3315 · DOI · Paper page
  34. May, M., George, S., & Prévôt, P. (2011). TrAVis to enhance online tutoring and learning activities: Real time visualization of students tracking data. Interactive Technology and Smart Education, 8(1), 52-69. https://dx.doi.org/10.1108/17415651111125513 · DOI
  35. Meer, N., & Chapman, A. (2014). Co-creation of marking criteria: Students as partners in the assessment process. Business and Management Education in HE, 1-15. https://dx.doi.org/10.11120/bmhe.2014.00008 · DOI
  36. Mojarad, S., Essa, A., Mojarad, S., & Baker, R. S. (2018). Studying adaptive learning efficacy using propensity score matching. In A. Pardo et al. (Eds.), Companion Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK '18), 5-9 March 2018, Sydney, Australia. Society for Learning Analytics Research.
  37. Morrison, B. B., & DiSalvo, B. (2014). Khan Academy gamifies computer science. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14), 5-8 March 2014, Atlanta, Georgia, USA (pp. 39-44). New York, USA: ACM. https://dx.doi.org/10.1145/2538862.2538946 · DOI
  38. Mulryan-Kyne, C. (2010). Teaching large classes at college and university level: Challenges and opportunities. Teaching in Higher Education, 15(2), 175-185. https://dx.doi.org/10.1080/13562511003620001 · DOI
  39. Oxman, S., Wong, W., DV X Innovations, & DeVry Education Group. (2014). White Pa- per: Adaptive Learning Systems. Integrated Education Solutions. Retrieved from http://kenanaonline.com/files/0100/100321/DVx A daptive L earning W hite P aper.pd f Park, O. c., & Lee, J. (2004). Adaptive instructional systems. In D. Jonassen (Ed.), Handbook of Research on Educational Communications and Technology (2nd ed., pp. 651-684). Mahwah, New Jersey, USA: Lawrence Erlbaum Associates Publishers.
  40. Pelánek, R. (2016). Applications of the Elo rating system in adaptive educational systems. Computers & Education, 98(C), 169-179. https://dx.doi.org/10.1016/j.compedu.2016.03.017 · DOI
  41. Pressley, M., Wood, E., Woloshyn, V. E., Martin, V., King, A., & Menke, D. (1992). Encouraging mindful use of prior knowledge: Attempting to construct explanatory answers facilitates learning. Educational Psychologist, 27(1), 91-109. https://dx.doi.org/10.1207/s15326985ep2701 7 · DOI
  42. Purchase, H., Hamer, J., Denny, P., & Luxton-Reilly, A. (2010). The quality of a PeerWise MCQ repository. In Proceedings of the Twelfth Australasian Conference on Computing Education (ACE '10), 1 January 2010, Brisbane, Australia (Vol. 103, pp. 137-146). Darlinghurst, Australia: Australian Computer Society, Inc.
  43. Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive Tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14(2), 249-255. https://dx.doi.org/10.3758/BF03194060 · DOI
  44. In O. Santos et al. (Eds.), Educational Data Mining 2015: 8th International Confer- ence on Educational Data Mining (EDM2015), 26-29 June 2015, Madrid, Spain. Retrieved from http://www.educationaldatamining.org/EDM2015/proceedings/edm2015 p roceedings.pd f
  45. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55. https://dx.doi.org/10.1093/biomet/70.1.41 · DOI · Paper page
  46. Solemon, B., Ariffin, I., Din, M. M., & Anwar, R. M. (2013). A review of the uses of crowdsourcing in higher education. International Journal of Asian Social Science, 3(9), 2066-2073.
  47. Sullivan, G. M. (2011). Getting off the "gold standard": Randomized controlled trials and education research. Journal of Graduate Medical Education, 3(3), 285-289. https://dx.doi.org/10.4300/JGME-D-11-00147. · DOI · Paper page
  48. Tackett, S., Raymond, M., Desai, R., Haist, S. A., Morales, A., Gaglani, S., & Clyman, S. G. (2018). Crowdsourcing for assessment items to support adaptive learning. Medical Teacher, 40(8), 838-841. https://dx.doi.org/10.1080/0142159X.2018.1490704 · DOI
  49. Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76(3), 467-481. https://dx.doi.org/10.1007/s10734-017-0220-3 · DOI · Paper page
  50. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197-221. https://dx.doi.org/10.1080/00461520.2011.611369 · DOI · Paper page
  51. Walsh, J. L., Harris, B. H., Denny, P., & Smith, P. (2018). Formative student-authored question bank: Perceptions, question quality and association with summative performance. Postgraduate Medical Journal, 94(1108), 97-103. https://dx.doi.org/10.1136/postgradmedj-2017-135018 · DOI
  52. Whitehill, J., Aguerrebere, C., & Hylak, B. (2019). Do learners know what's good for them? Crowdsourcing subjective ratings of OERs to predict learning gains. In C. F. Lynch, A. Merceron, M. Desmarais, & R. Nkambou (Eds.), Educational Data Mining 2019: 12th International Conference on Educational Data Mining (EDM2019), 2-5 June 2019, Montreal, Canada (pp. 462-467). Retrieved from http://educationaldatamining.org/edm2019/proceedings/
  53. Williams, J. J., Kim, J., Rafferty, A., Maldonado, S., Gajos, K. Z., Lasecki, W. S., & Heffernan, N. (2016). Axis: Generating explanations at scale with learnersourcing and machine learning. In Proceedings of the Third (2016) ACM Conference on Learning @ Scale (L@S 2016), 25-26 April 2016, Edinburgh, Scotland (pp. 379-388). New York, USA: ACM. https://dx.doi.org/10.1145/2876034.2876042 · DOI
  54. Yilmaz, B. (2017). Effects of Adaptive Learning Technologies on Math Achievement: A Quantitative Study of ALEKS Math Software (Unpublished doctoral dissertation). University of Missouri-Kansas City.

Cited by

Sorted by citations

Metadata

Authors
Khosravi, H., Kitto, K., & Williams, J. J.
Affiliations
  • The University of Queensland (AU)
  • University of Technology Sydney (AU)
  • University of Toronto (CA)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
Volume
6
Issue
3
ISSN
1929-7750 (ISSN-L)
DOI
10.48550/arxiv.1910.05522
arXiv
1910.05522
OpenAlex
W2980237866
Cited by
81
References
0
Citations by year
2026: 12 · 2025: 8 · 2024: 13 · 2023: 13 · 2022: 14 · 2021: 15 · 2020: 6
FWCI
3.99
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Learning Analytics (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • Cosmopolitan Civil Societies: An Interdisciplinary Journal (University of Technology Sydney) (submittedVersion, cc-by)
  • UTS ePRESS (University of Technology Sydney) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
Keywords
adaptive learning, peer learning, knowledge states, virtual learning environments, Learning Tools Interoperability, learning gains, student perceptions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review