CoChat
Sign Up
Cited by 0Online Learning and AnalyticsIntelligent Tutoring Systems and Adaptive LearningArtificial Intelligence in Education

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

What don’t you understand? Using large language models to identify and characterize student misconceptions about challenging topics

Open access
Parker, Michael J., Zavala-Cerna, María G.·2026·Education and Information Technologies
DOI 10.1007/s10639-026-13902-8
Open this paper in CoChat — free accountOpen PDF

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10639-026-13902-8

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 51

  1. Almeida, F., & Monteiro, J. (2021). The challenges of assessing and evaluating the students at distance. arXiv. https://doi.org/10.48550/arXiv.2102.04235 · DOI
  2. Carolan, K., Fennelly, L., & Smeaton, A. F. (2024). A review of multi-modal large language and vision models. arXiv. https://doi.org/10.48550/arXiv.2404.01322 · DOI
  3. Chi, M., VanLehn, K., Litman, D., & Jordan, P. (2011). Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Modeling and User-Adapted Interaction, 21(1–2), 137–180. https://doi.org/10.1007/s11257-010-9093-1 · DOI
  4. Coll, R. K., & Treagust, D. F. (2003). Investigation of secondary school, undergraduate, and graduate learners’ mental models of ionic bonding. Journal of Research in Science Teaching, 40(5), 464–486. https://doi.org/10.1002/tea.10085 · DOI
  5. Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119(2), 166.e7-166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036 · DOI · Paper page
  6. DeMars, C. (2010). Item response theory. Oxford University Press. · DOI
  7. Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5–6), 304–317. https://doi.org/10.1504/ijtel.2012.051816 · DOI
  8. Ferguson, R., Khosravi, H., Kovanović, V., Viberg, O., Aggarwal, A., Brinkhuis, M., Buckingham Shum, S., Chen, L. K., Drachsler, H., Guerrero, V. A., Hanses, M., Hayward, C., Hicks, B., Jivet, I., Kitto, K., Kizilcec, R., Lodge, J. M., Manly, C. A., Matz, R. L., … Yan, V. X. (2023). Aligning the goals of learning analytics with its research scholarship. Journal of Learning Analytics, 10(2), 14–50. https://doi.org/10.18608/jla.2023.8197 · DOI
  9. Glaser, R., Chudowsky, N., & Pellegrino, J. W. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. National Academies Press. https://doi.org/10.17226/10019 · DOI
  10. Gorgun, G., & Botelho, A. F. (2023). Enhancing the automatic identification of common math misconceptions using natural language processing. In I. I. Bittencourt, S. Isotani, C. Rueda, & M. J. Rodríguez-Triana (Eds.), Communications in Computer and Information Science: Vol. 1834. Artificial Intelligence in Education (pp. 302–307). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-36336-8_47 · DOI
  11. Gurel, D. K., Eryılmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students’ misconceptions in science. Eurasia Journal of Mathematics, Science and Technology Education, 11(5), 989–1008. https://doi.org/10.12973/eurasia.2015.1369a · DOI
  12. Haladyna, T. M., & Downing, S. M. (2011). Twelve steps for effective test development. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (2nd ed., pp. 17–40). Routledge. https://doi.org/10.4324/9780203874776 · DOI
  13. Halim, A. S., Finkenstaedt-Quinn, S. A., Olsen, L. J., Gere, A. R., & Shultz, G. V. (2018). Identifying and remediating student misconceptions in introductory biology via writing-to-learn assignments and peer review. CBE Life Sciences Education, 17(2), Article ar28. https://doi.org/10.1187/cbe.17-10-0212 · DOI
  14. Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30(3), 141–158. https://doi.org/10.1119/1.2343497 · DOI · Paper page
  15. Hoq, M., Vandenberg, J., Mott, B., Lester, J., Norouzi, N., & Akram, B. (2024, March 14–16). Towards attention-based automatic misconception identification in introductory programming courses. Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2 (pp. 1002–1009). ACM. https://doi.org/10.1145/3626253.3635575 · DOI
  16. Kaczmarczyk, L. C., Petrick, E. R., East, J. P., & Herman, G. L. (2010, March 10–13). Identifying student misconceptions of programming. Proceedings of the 41st ACM Technical Symposium on Computer Science Education (pp. 107–111). ACM. https://doi.org/10.1145/1734263.1734299 · DOI
  17. Kearns, L. R. (2012). Student assessment in online learning: Challenges and effective practices. Journal of Online Learning and Teaching, 8(3), 198–207.
  18. Kennedy, C., & Kraemer, E. T. (2018, November 15–17). What are they thinking? Eliciting student reasoning about troublesome concepts in introductory computer science. Proceedings of the 18th Koli Calling International Conference on Computing Education Research (pp. 1–10). ACM. https://doi.org/10.1145/3279720.3279728 · DOI
  19. Knight, S., Buckingham Shum, S., & Littleton, K. (2014). Epistemology, assessment, pedagogy: Where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23–47. https://doi.org/10.18608/jla.2014.12.3 · DOI
  20. Kökver, Y., Pektaş, H. M., & Çelik, H. (2025). Artificial intelligence applications in education: Natural language processing in detecting misconceptions. Education and Information Technologies, 30(3), 3035–3066. https://doi.org/10.1007/s10639-024-12919-1 · DOI
  21. Larkin, D. (2012). Misconceptions about “misconceptions”: Preservice secondary science teachers’ views on the value and role of student ideas. Science Education, 96(5), 927–959. https://doi.org/10.1002/sce.21022 · DOI
  22. Liu, M., Okuhara, T., Chang, X., Shirabe, R., Nishiie, Y., Okada, H., & Kiuchi, T. (2024). Performance of ChatGPT across different versions in medical licensing examinations worldwide: Systematic review and meta-analysis. Journal of Medical Internet Research, 26, Article e60807. https://doi.org/10.2196/60807 · DOI
  23. Liu, Y., Li, D., Wang, K., Xiong, Z., Shi, F., Wang, J., Li, B., & Hang, B. (2024). Are LLMs good at structured outputs? A benchmark for evaluating structured output capabilities in LLMs. Information Processing & Management, 61(5), Article 103809. https://doi.org/10.1016/j.ipm.2024.103809 · DOI
  24. Lo, S. L., Tan, K. W., & Ouh, E. L. (2021). Automated doubt identification from informal reflections through hybrid sentic patterns and machine learning approach. Research and Practice in Technology Enhanced Learning, 16, Article 1. https://doi.org/10.1186/s41039-021-00149-9 · DOI
  25. Meyer, J. H. F., & Land, R. (2003). Threshold concepts and troublesome knowledge: Linkages to ways of thinking and practising within the disciplines. In ISL10 Improving Student Learning: Theory and Practice Ten Years On (pp. 412–424). Oxford Brookes University. https://pressbooks.atlanticoer-relatlantique.ca/app/uploads/sites/803/2021/07/ETLreport4.pdf
  26. Michalenko, J. J., Lan, A. S., & Baraniuk, R. G. (2017). Data-mining textual responses to uncover misconception patterns. arXiv. https://doi.org/10.48550/arXiv.1703.08544 · DOI
  27. Modell, H., Michael, J., & Wenderoth, M. P. (2005). Helping the learner to learn: The role of uncovering misconceptions. The American Biology Teacher, 67(1), 20–26. https://doi.org/10.1662/0002-7685(2005)067[0020:HTLTLT]2.0.CO;2 · DOI
  28. Moskal, P. D., Dziuban, C. D., & Picciano, A. G. (2023). Data analytics and adaptive learning. Routledge. https://doi.org/10.4324/9781003244271 · DOI
  29. Motz, B. A., Bergner, Y., Brooks, C. A., Gladden, A., Gray, G., Lang, C., Li, W., Marmolejo-Ramos, F., & Quick, J. D. (2023). LAK of direction: Misalignment between the goals of learning analytics and its research scholarship. Journal of Learning Analytics, 10(2), 1–13. https://doi.org/10.18608/jla.2023.7913 · DOI
  30. Mustamin, R. (2024). The role of formative and summative assessment in improving learning quality and student learning outcomes. Journal of Strategy and Transformation in Educational Management, 4(1), 17–21. · DOI
  31. Nunn, S., Avella, J. T., Kanai, T., & Kebritchi, M. (2016). Learning analytics methods, benefits, and challenges in higher education: A systematic literature review. Online Learning, 20(2), 12–43. https://doi.org/10.24059/olj.v20i2.790 · DOI
  32. OldeBekkink, M., Donders, A. R., Kooloos, J. G., de Waal, R. M., & Ruiter, D. J. (2016). Uncovering students’ misconceptions by assessment of their written questions. BMC Medical Education, 16, Article 221. https://doi.org/10.1186/s12909-016-0739-5 · DOI
  33. Parker, M. J., Bunch, M., & Pike, A. (2025). How much is enough? Formative assessment dynamics: Quantifying the relationship between intermediate quiz performance and final exam scores. Journal of Learning Analytics. https://doi.org/10.18608/jla.2025.8753 · DOI
  34. Quadri, A., & Shukor, N. (2021). The benefits of learning analytics to higher education institutions: A scoping review. International Journal of Emerging Technologies in Learning, 16(23), 4–15. https://doi.org/10.3991/ijet.v16i23.27471 · DOI
  35. Ramos, A. R., & Castillo, G. C. (2024). Best practices and challenges in the assessment of online learning among BSEd freshmen. Journal of Tertiary Education and Learning, 2(1), 33–39. https://doi.org/10.54536/jtel.v2i1.2390 · DOI
  36. Rohrer, D. (2012). Interleaving helps students distinguish among similar concepts. Educational Psychology Review, 24(3), 355–367. https://doi.org/10.1007/s10648-012-9201-3 · DOI
  37. Shi, Y., Shah, K., Wang, W., Marwan, S., Penmetsa, P., & Price, T. (2021, April 12–16). Toward semi-automatic misconception discovery using code embeddings. LAK21: 11th International Learning Analytics and Knowledge Conference (pp. 606–612). ACM. https://doi.org/10.1145/3448139.3448205 · DOI · Paper page
  38. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004 · DOI · Paper page
  39. Slotta, J. D., & Chi, M. T. (2006). Helping students understand challenging topics in science through ontology training. Cognition and Instruction, 24(2), 261–289. https://doi.org/10.1207/s1532690xci2402_3 · DOI
  40. Smart, F., Bos, N. D., & Bos, J. T. (2024). Can large language models recognize and respond to student misconceptions? In R. A. Sottilare & J. Schwarz (Eds.), Lecture Notes in Computer Science: Vol. 14706. Adaptive Instructional Systems (pp. 288–299). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-60609-0_21 · DOI
  41. Suprapto, N. (2020). Do we experience misconceptions?: An ontological review of misconceptions in science. Studies in Philosophy of Science and Education, 1(2), 50–55. https://doi.org/10.46627/sipose.v1i2.24 · DOI
  42. Telenti, A., Auli, M., Hie, B. L., Maher, C., Saria, S., & Ioannidis, J. P. (2024). Large language models for science and medicine. European Journal of Clinical Investigation, 54(6), Article e14183. https://doi.org/10.1111/eci.14183 · DOI
  43. Terry, G., Hayfield, N., Clarke, V., & Braun, V. (2017). Thematic analysis. The SAGE Handbook of Qualitative Research in Psychology, 2(17–37), 25.
  44. Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10(2), 159–169. https://doi.org/10.1080/0950069880100204 · DOI · Paper page
  45. Verkade, H., Mulhern, T., Lodge, J., Elliott, K., Cropper, S., Rubinstein, B., Horton, A., Elliott, C., Espinosa, A., Dooley, L., Frankland, S., Mulder, R., & Livett, M. (2018). Misconceptions as a trigger for enhancing student learning in higher education: A handbook for educators. The University of Melbourne.
  46. Wandersee, J. H., Mintzes, J. J., & Novak, J. D. (1994). Research on alternative conceptions in science. In D. L. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 177–210). Macmillan Publishing Co.
  47. Wang, Z., Xia, M., He, L., Chen, H., Liu, Y., Zhu, R., Liang, K., Wu, X., Liu, H., Malladi, S., Chevalier, A., Arora, S., & Chen, D. (2024). CharXiv: Charting gaps in realistic chart understanding in multimodal LLMs. arXiv. https://doi.org/10.48550/arXiv.2406.18521 · DOI
  48. Wiegand, R. P., Bucci, A., Kumar, A. N., Albert, J., & Gaspar, A. (2021). Identifying informatively easy and informatively hard concepts. ACM Transactions on Computing Education (TOCE), 22(1), 1–28. Article 7. https://doi.org/10.1145/3477968 · DOI
  49. Wilson, M. (2023). Constructing measures: An item response modeling approach. Routledge. https://doi.org/10.4324/9781003286929 · DOI
  50. Wise, A. F. (2019). Learning analytics: Using data-informed decision-making to improve teaching and learning. In M. Khosrow-Pour (Ed.), Contemporary technologies in education (pp. 119–143). Springer International Publishing. https://doi.org/10.1007/978-3-319-89680-9_7 · DOI
  51. Xiao, J., & Zhang, Z. (2025). EduVQA: A multimodal visual question answering framework for smart education. Alexandria Engineering Journal, 122, 615–624. https://doi.org/10.1016/j.aej.2025.03.005 · DOI

Metadata

Authors
Parker, M. J., & Zavala-Cerna, M. G.
Affiliations
Beth Israel Deaconess Medical Center (US)
Corresponding author
Parker, Michael J.
ORCID
Venue
Education and Information Technologies
Publisher
Springer Science and Business Media LLC
Published
2026-04-06
Volume
31
Issue
12
Pages
4705-4744
ISSN
  • 1360-2357 (ISSN-L)
  • 1573-7608
DOI
10.1007/s10639-026-13902-8
OpenAlex
W7150844087
Cited by
0
References
9
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • Education and Information Technologies (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Education › Artificial Intelligence in Education
Keywords
student misconceptions, large language models, online learning, medical education, multiple-choice questions, generative AI, personalized learning interventions, student response patterns, lecture transcripts

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review