Assessing Learning of Computer Programing Skills in the Age of Generative Artificial Intelligence
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Generative artificial intelligence (AI) tools such as ChatGPT, Bard, and Claude have recently become a concern in the delivery of engineering education. For courses focused on computer coding, such tools are capable for creating working computer code across a range of computer languages and computing platforms. In a course for mechanical engineers focused on C++ coding for the Arduino microcontroller and coding engineering problems in Matlab, a new approach to assessment of programing homework assignments was developed. This assessment moved the focus of assigned points from the correctness of the code to the effort and understanding of the code demonstrated by the student during in-person grading. Students who participated fully in in-person grading did significantly better on a midterm exam. Relative to a previous semester, where grading was focused on correct code, students had a slightly higher average midterm exam score. This approach appears to be effective in supporting computational learning in the face of evolving tools that could be used to circumvent learning. Future work should examine how to also encourage responsible use of generative AI in computational learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Demystifying computational thinking
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017) · Educational Research Review · doi:10.1016/j.edurev.2017.09.003 · Cited by 1,466
Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023) · The International Journal of Management Education · doi:10.1016/j.ijme.2023.100790 · Cited by 1,261
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
Searching for explanations: How the Internet inflates estimates of internal knowledge.
Fisher, M., Goddu, M. K., & Keil, F. C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000070 · Cited by 330
Cited by
Cai, L., Msafiri, M. M., & Kangwa, D. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-13112-0 · Cited by 104
Liu, X., Guo, B., He, W., & Hu, X. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251329185 · Cited by 89
The Impact of AI on the Personal and Collaborative Learning Environments in Higher Education
Msambwa, M. M., Wen, Z., & Daniel, K. (2025) · European Journal of Education · doi:10.1111/ejed.12909 · Cited by 76
Daniel, K., Msambwa, M. M., & Wen, Z. (2025) · European Journal of Education · doi:10.1111/ejed.70036 · Cited by 71
Elshall, A. S., & Badir, A. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1596462 · Cited by 28
“For example, in data science education, student dependence on AI tools has the potential to diminish the development of basic coding and problem-solving skills necessary for their professional growth (Camarillo et al., 2024; Wilson and Nishimoto, 2024).”
Zhu, S., & Liu, Z. (2026) · Proceedings of the 2026 International Conference on Big Data and Informatization Education · doi:10.1145/3806980.3806985
Metadata
- Authors
- Wilson, S. E., & Nishimoto, M.
- Venue
- Journal of Biomechanical Engineering
- Publisher
- ASME International
- Published
- 2024-03-07
- Volume
- 146
- Issue
- 5
- ISSN
- 0148-0731 (ISSN-L)
- 1528-8951
- OpenAlex
- W4390143651
- Cited by
- 20
- References
- 6
- FWCI
- 4.62
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.