CoChat
Sign Up
Cited by 9Teaching and Learning ProgrammingArtificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive Learning

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Optimizing Large Language Models for Auto-Generation of Programming Quizzes

Kumar, Yulia, Manikandan, Anjana, Li, J. Jenny, Morreale, Patricia·2024·2024 IEEE Integrated STEM Education Conference (ISEC)
DOI 10.1109/isec61299.2024.10665141
Open this paper in CoChat — free account

Abstract

This study analyzes the use of Large Language Models (LLMs) like ChatGPT in creating quizzes for Java programming courses, specifically Object-Oriented Programming (CS1) and Data Structures (CS2). It aims to evaluate the accuracy of LLM-generated assessments, understand the benefits and drawbacks of using LLMs in CS education from educators' viewpoints, and identify effective prompt engineering strategies to enhance the quality of educational materials. The research compares quizzes made by LLMs against human-created content to assess their consistency with Java programming principles, alignment with CS1 and CS2 learning goals, and their impact on student engagement and comprehension, providing insights into LLMs' effectiveness in academic assessment creation for computer science education.

Overview

Integrity signals

Registry

Crossref · 10.1109/isec61299.2024.10665141

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 2 references

Research works printed but not linked: 20

  1. 10.1007/978-3-031-36336-8_108 · DOI
  2. Lu, 2021, Expert-authored and machine-generated short-answer questions, Educational Technology & Society
  3. Gan, 2019, AI-tutor: Generating tailored remedial questions and answers
  4. 10.1080/00131857.2020.1728732 · DOI · Paper page
  5. Denny, 2023, Can We Trust AI-Generated Educational Content?, arXiv preprint
  6. Clune, 2019, AI-GAs: AI-generating algorithms, arXiv preprint
  7. Speth, 2023, Investigating the Use of AI-Generated Exercises for Programming Courses
  8. 10.1080/08993400500150747 · DOI
  9. Orenstrakh, 2023, Detecting LLM -generated text in computing education, arXiv preprint
  10. Denny, 2023, Human vs Machine: Comparison of Student-generated and AI-generated Educational Content, arXiv preprint
  11. 10.1145/3629296.3629298 · DOI
  12. Sutherland, 2023, Exploring the Role of AI-Generated Feedback
  13. Thomas, 2023, Comparative analysis of learner sourced responses
  14. Aboalela, 2023, ChatGPT for generating questions and assessments, arXiv preprint
  15. 2024, Human-AI interaction regarding the assignments
  16. 2024, Human-AI interaction
  17. Patel, 2022, Implementing Inclusive Software Design in the CS Curriculum, 2, 1272
  18. 10.1109/ISEC57711.2023.10402302 · DOI
  19. 2024, OpenAI GitHub Repository
  20. 2024, Link to QuickStart Tutorial Instructions

Cited by

Sorted by citations

See all 9

Metadata

Authors
Kumar, Y., Manikandan, A., Li, J. J., & Morreale, P.
Affiliations
Kean University (US)
ORCID
Venue
2024 IEEE Integrated STEM Education Conference (ISEC)
Publisher
IEEE
Published
2024-03-09
Pages
1-5
DOI
10.1109/isec61299.2024.10665141
OpenAlex
W4402572069
Cited by
9
References
2
Citations by year
2026: 6 · 2025: 3
FWCI
3.95
Type
conference paper
Language
en
Open access
closed
Licence
Locations
IEEE Integrated STEM Education Conference (ISEC) (publishedVersion)
Funders
NSF (1834620,2137791)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
large language models, Java programming, CS1, carbon disulfide, object-oriented programming, data structures, prompt engineering, computer science education, student engagement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Optimizing Large Language Models for Auto-Generation of Programming Quizzes. https://app.cochat.ai/papers/10.1109/isec61299.2024.10665141
The paper
Kumar, Y., Manikandan, A., Li, J. J., & Morreale, P. (2024). Optimizing Large Language Models for Auto-Generation of Programming Quizzes. 2024 IEEE Integrated STEM Education Conference (ISEC). https://doi.org/10.1109/isec61299.2024.10665141