CoChat
Sign Up
PreprintCited by 60Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationIntelligent Tutoring Systems and Adaptive Learning

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Evaluating AI-Powered Learning Assistants in Engineering Higher Education: Student Engagement, Ethical Challenges, and Policy Implications

Open access
Sajja, Ramteja, Sermet, Yusuf, Fodale, Brian, Demir, Ibrahim·2025·arXiv
DOI 10.48550/arxiv.2506.05699arXiv 2506.05699PMC PMC12932848
Open this paper in CoChat — free accountOpen PDF

Abstract

As generative AI becomes increasingly integrated into higher education, understanding how students engage with these technologies is essential for responsible adoption. This study evaluates the Educational AI Hub, an AI-powered learning framework, implemented in undergraduate civil and environmental engineering courses at a large R1 public university. Using a mixed-methods design combining pre- and post-surveys, system usage logs, and qualitative analysis of students' AI interactions, the research examines perceptions of trust, ethics, usability, and learning outcomes. Findings show that students valued the AI assistant for its accessibility and comfort, with nearly half reporting greater ease using it than seeking help from instructors or teaching assistants. The tool was most helpful for completing homework and understanding concepts, though views on its instructional quality were mixed. Ethical uncertainty, particularly around institutional policy and academic integrity, emerged as a key barrier to full engagement. Overall, students regarded AI as a supplement rather than a replacement for human instruction. The study highlights the importance of usability, ethical transparency, and faculty guidance in promoting meaningful AI engagement. A total of 71 students participated across two courses, generating over 600 AI interactions and 100 survey responses that provided both quantitative and contextual insights into learning engagement.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2506.05699

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior research suggests that User Experience (UX) with such tools, including ease of use, sense of usefulness, and responsiveness, substantially influence engagement dimensions.

    “Prior research suggests that User Experience (UX) with such tools, including ease of use, sense of usefulness, and responsiveness, substantially influence these engagement dimensions by shaping how students interact with educational content and perceive their value (Sajja et al., 2025 ).”

    From Socratic Mind: Impact of a Novel GenAI-Powered Formative Assessment Tool on Student Learning and Higher-Order Thinking (2026) · cites this paper as Sajja et al., 2025

    Section 3.6 reports that comfort and perceived helpfulness were significantly associated with more frequent use of the AI assistant. It does not establish that UX causes engagement, and its convenience–usage correlation was not statistically significant. The cited findings do not show that responsiveness influences engagement. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Design features and User Experiences (UX) significantly influence the depth and quality of learner interaction.

    “Design features and User Experiences (UX) significantly influence the depth and quality of learner interaction (Chi & Wylie, 2014; D'Mello & Graesser, 2023 ; Sajja et al., 2025 ).”

    From Socratic Mind: Impact of a Novel GenAI-Powered Formative Assessment Tool on Student Learning and Higher-Order Thinking (2026) · cites this paper as Sajja et al., 2025

    Section 3.6 (page unavailable) reports significant associations between students’ comfort with the AI tool, perceived homework usefulness, and how frequently they used it. The Discussions section also recommends task-aligned features and scaffolding. These passages support a link between user experience, design, and engagement, but they do not establish a significant effect on the depth or quality of learner interaction. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. Almenara, J., Palacios-Rodríguez, A., Loaiza-Aguirre, M., & Abarca, P. (2024). The impact of pedagogical beliefs on the adoption of generative AI in higher education: predictive model from UTAUT2. Frontiers in Artificial Intelligence, 7. https://doi.org/10.3389/frai.2024.1497705 · DOI
  2. Altinay, Z., Altinay, F., Dagli, G., Shadiev, R., & Othman, A. (2024). Factors influencing AI learning motivation and personalisation among pre-service teachers in higher education. MIER Journal of Educational Studies Trends & Practices, 462-481. https://doi.org/10.52634/mier/2024/v14/i2/2714 · DOI
  3. Aravindh, K., & Singh, B. (2024). Applications of artificial intelligence in education. In Advances in educational technologies and instructional design book series (pp. 21-40). https://doi.org/10.4018/979-8-3693-7220-3. · DOI
  4. ch002 · DOI
  5. Bah, N. F. A. (2024). Artificial intelligence in education system. International Journal of Advanced Research in Science Communication and Technology, 439-443. https://doi.org/10.48175/ijarsct-22863 · DOI
  6. Cui, P., & Alias, B. S. (2024). Opportunities and challenges in higher education arising from AI: A systematic literature review (2020-2024). Journal of Infrastructure Policy and Development, 8(11), 8390. https://doi.org/10.24294/jipd.v8i11.8390 · DOI
  7. Dai, Y. (2024). Dual-contrast pedagogy for AI literacy in upper elementary schools. Learning and Instruction. https://doi.org/10.1016/j.learninstruc.2024.101899 · DOI
  8. Dai, Y., Lin, Z., Liu, A., & Wang, W. (2023). An embodied, analogical and disruptive approach of AI pedagogy in upper elementary education: An experimental study. Br. J. Educ. Technol., 55, 417-434. https://doi.org/10.1111/bjet.13371 · DOI
  9. Deri, M. N., Singh, A., Zaazie, P., & Anandene, D. (2024). Leveraging Artificial intelligence in Higher Educational Institutions: A Comprehensive Overview. Revista De Educación Y Derecho, 30. https://doi.org/10.1344/reyd2024.30.45777 · DOI
  10. Dilmi, C., & Sakri, Z. (2024). Artificial intelligence's Impact on Higher Education Quality. Science and Knowledge Horizons Journal, 4(01), 606-623. https://doi.org/10.34118/jskp.v4i01.3889 · DOI
  11. Doğan, M., Celik, A., & Arslan, H. (2024). AI In Higher Education: Risks and Opportunities From the Academician Perspective. European Journal of Education. https://doi.org/10.1111/ejed.12863 · DOI
  12. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 39(2), 175-191.
  13. Gao, Q. (2024). Decision Support Systems for Lifelong Learning: Leveraging Information Systems to Enhance Learning Quality in Higher Education. J. Internet Serv. Inf. Secur., 14, 121-143. https://doi.org/10.58346/jisis.2024.i4.007 · DOI
  14. Hamzah, H., Seman, M., & Ahmed, M. (2025). The impact of artificial intelligence in enhancing online learning platform effectiveness in higher education. Information Development. https://doi.org/10.1177/02666669251315842 · DOI
  15. Hashem, H. H., Saleh, N., & Wali, I. (2024). The effectiveness of using artificial intelligence in e-learning content design. Journal of Design Sciences and Applied Arts/Journal of Design Sciences and Applied Arts, 5(2), 316-326. https://doi.org/10.21608/jdsaa.2023.222169.1350 · DOI
  16. Ikhsan, I., Rangkuti, Z., & Supardi, S. (2025). Implementation of AI in LMS to Optimize Digital Learning Management Based on Real-Time Data. International Journal of Social and Human. https://doi.org/10.59613/gz1kcm56 · DOI
  17. Jain, V., Singh, I., Syed, M., Mondal, S., & Palai, D. R. (2024). Enhancing Educational Interactions: A comprehensive review of AI chatbots in learning environments. 2022 10th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO). https://doi.org/10.1109/icrito61523.2024.10522392 · DOI
  18. Jian, M. J. K. O. (2023). Personalized learning through AI. Advances in Engineering Innovation, 5(1), 16-19. https://doi.org/10.54254/2977-3903/5/2023039 · DOI
  19. Katonane Gyonyoru, K. I., & Katona, J. (2024). Student perceptions of AI-enhanced adaptive learning systems: A pilot survey. In 2024 IEEE 7th International Conference and Workshop Óbuda on Electrical and Power Engineering (CANDO-EPE) (pp. 93-98). IEEE. https://doi.org/10.1109/CANDO-EPE65072.2024.10772884 · DOI
  20. Katsamakas, E., Pavlov, O., & Saklad, R. (2024). Artificial intelligence and the transformation of higher education institutions. ArXiv, abs/2402.08143. https://doi.org/10.48550/arXiv.2402.08143 · DOI
  21. Kong, S., & Yang, Y. (2024). A Human-Centered Learning and Teaching Framework Using Generative Artificial Intelligence for Self-Regulated Learning Development Through Domain Knowledge Learning in K-12 Settings. IEEE Transactions on Learning Technologies, 17, 1588-1599. https://doi.org/10.1109/TLT.2024.3392830 · DOI · Paper page
  22. Leong, W. Y., Leong, Y. Z., & Leong, W. S. (2025). Artificial Intelligence in education. IET Conference Proceedings., 2024(22), 183-184. https://doi.org/10.1049/icp.2024.4341 · DOI
  23. Luo, Q. (2023). The Influence of AI-Powered Adaptive Learning Platforms on Student Performance in Chinese Classrooms. Journal of Education. https://doi.org/10.53819/81018102t4181 · DOI
  24. Mimoudi, A. (2024). AI, personalized education, and challenges. Proceedings of the International Conference on AI Research., 4(1), 271-280. https://doi.org/10.34190/icair.4.1.3133 · DOI
  25. Moleka, P. (2023). Exploring the Role of Artificial Intelligence in Education 6.0: Enhancing Personalized Learning and Adaptive Pedagogy. Preprints. https://doi.org/10.20944/preprints202309.0562.v1 · DOI
  26. Msambwa, M., Wen, Z., & Daniel, K. (2025). The Impact of AI on the Personal and Collaborative Learning Environments in Higher Education. European Journal of Education. https://doi.org/10.1111/ejed.12909 · DOI · Paper page
  27. Murdan, A. P., & Halkhoree, R. (2024). Integration of artificial intelligence for educational excellence and innovation in higher education institutions. In 2024 1st International Conference on Smart Energy Systems and Artificial Intelligence (SESAI) (pp. 1-6). IEEE. https://doi.org/10.1109/SESAI61023.2024.10599402 · DOI
  28. Ng, D., Lee, M., Tan, R., Hu, X., Downie, J., & Chu, S. (2022). A review of AI teaching and learning from 2000 to 2020. Education and Information Technologies, 28, 8445-8501. https://doi.org/10.1007/s10639-022-11491-w · DOI
  29. Niveditha, K. P., Choubey, K., Soni, K., Mishra, P., & Naz, A. (2023). Exploring the role of artificial intelligence in modern education. International Journal of Innovative Research in Computer and Communication Engineering, 12(05), 8147-8150. https://doi.org/10.15680/ijircce.2024.1205365 · DOI
  30. Pursnani, V., Sermet, Y., & Demir, I. (2025). A conversational intelligent assistant for enhanced operational support in floodplain management with multimodal data. International Journal of Disaster Risk Reduction, 122, 105422.
  31. Ramasamy, T. (2024). AI IN HIGHER EDUCATION. INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT. https://doi.org/10.55041/ijsrem35591 · DOI
  32. S Kumar, A., D, D., Dileepkumar, J., K V, N., Mathai Samuel, P., & K, S. (2025). A Comprehensive Survey on AI in Learning Management System. Preprints. https://doi.org/10.20944/preprints202501.0697.v1 · DOI
  33. Sajja, R., Pursnani, V., Sermet, Y., & Demir, I. (2025b). AI-assisted educational framework for floodplain manager certification: Enhancing vocational education and training through personalized learning. IEEE Access, 13, 42401-42413. https://doi.org/10.1109/ACCESS.2025.3548591 · DOI
  34. Sajja, R., Sermet, Y., & Demir, I. (2025a). End-to-end deployment of the Educational AI Hub for personalized learning and engagement: A case study on environmental science education. IEEE Access, 13, 55169-55186. https://doi.org/10.1109/ACCESS.2025.3554222 · DOI
  35. Sajja, R., Sermet, Y., & Demir, I. (2025c). An Open-Source Dual-Loss Embedding Model for Semantic Retrieval in Higher Education. arXiv preprint arXiv:2505.04916. · arXiv
  36. Sajja, R., Sermet, Y., Cikmaz, M., Cwiertny, D., & Demir, I. (2024). Artificial intelligence- enabled intelligent assistant for personalized and adaptive learning in higher education. Information, 15(10), 596.
  37. Sajja, R., Sermet, Y., Cwiertny, D., & Demir, I. (2023a). Platform-independent and curriculum- oriented intelligent assistant for higher education. International journal of educational technology in higher education, 20(1), 42.
  38. Sajja, R., Sermet, Y., Cwiertny, D., & Demir, I. (2023b). Integrating AI and learning analytics for data-driven pedagogical decisions and personalized interventions in education. arXiv preprint arXiv:2312.09548. · arXiv
  39. Santos, S. M. a. V., Da Silva, C. G., De Carvalho, I. E., De Castilho, L. P., Meroto, M. B. D. N., Tavares, P. R., Pires, R. D. R., & De Oliveira Rodrigues Moniz, S. S. (2024). The art of personalization of education: Artificial Intelligence on the stages of special education. Contribuciones a Las Ciencias Sociales, 17(2), e4971. https://doi.org/10.55905/revconv.17n.2-008 · DOI
  40. Saqr, R., Al-Somali, S., & Sarhan, M. (2023). Exploring the Acceptance and User Satisfaction of AI-Driven e-Learning Platforms (Blackboard, Moodle, Edmodo, Coursera and edX): An Integrated Technology Model. Sustainability. https://doi.org/10.3390/su16010204 · DOI
  41. Sari, H. E., Tumanggor, B., & Efron, D. (2024). Improving Educational Outcomes Through Adaptive Learning Systems using AI. International Transactions on Artificial Intelligence (ITALIC), 3(1), 21-31. https://doi.org/10.33050/italic.v3i1.647 · DOI
  42. Sermet, Y., & Demir, I. (2021). A semantic web framework for automated smart assistants: A case study for public health. Big Data and Cognitive Computing, 5(4), 57.
  43. Siddiqui, M., Mansoori, M., Siddiqui, M., & Yadav, A. (2025). AI-Enabled Pedagogy: Advancing Education Through Innovative Teaching Tools and the AI-TEACH Model. Journal of Informatics Education and Research. https://doi.org/10.52783/jier.v5i1.2261 · DOI
  44. Singh, P. (2024). Artificial intelligence and student engagement. In Advances in educational technologies and instructional design book series (pp. 201-232). https://doi.org/10.4018/979- 8-3693-5633-3. · DOI
  45. ch008 · DOI
  46. Supelano Londoño, M. L. (2024). Incidencia de la inteligencia artificial en el aprendizaje de los estudiantes universitarios. Plumilla Educativa, 33(2), 1-24. https://doi.org/10.30554/p.e.2.5153.2024 · DOI
  47. Vieira, D., Chinchilla, J. R., Liu, B. L., Yerovi, C., & Morales, D. (2022). Beyond the chatbot. In Advances in higher education and professional development book series (pp. 1-22). https://doi.org/10.4018/978-1-7998-9247-2. · DOI
  48. ch001 · DOI
  49. Wangdi, P. (2024). Integrating Artificial Intelligence in education: International Journal on Research in STEM Education, 6(2), 50-60. https://doi.org/10.33830/ijrse.v6i2.1722 · DOI
  50. Yue, M., Jong, M., & Dai, Y. (2022). Pedagogical Design of K-12 Artificial Intelligence Education: A Systematic Review. Sustainability. https://doi.org/10.3390/su142315620 · DOI · Paper page
  51. Zawacki-Richter, O., Marín, V., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education -where are the educators?. International Journal of Educational Technology in Higher Education, 16. https://doi.org/10.1186/s41239-019-0171-0 · DOI · Paper page
  52. Zhang, M., Zhu, L., Lin, S. Y., Herr, K., Chi, C. L., Demir, I., ... & Chi, N. C. (2023). Using artificial intelligence to improve pain assessment and pain management: a scoping review. Journal of the American Medical Informatics Association, 30(3), 570-587.

Cited by

Sorted by citations

Metadata

Authors
Sajja, R., Sermet, Y., Fodale, B., & Demir, I.
Affiliations
  • Tulane University (US)
  • University of Iowa (US)
Corresponding author
Sajja, Ramteja
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Volume
16
Issue
1
ISSN
2045-2322 (ISSN-L)
DOI
10.48550/arxiv.2506.05699
arXiv
2506.05699
PMC
PMC12932848
OpenAlex
W7128169455
Cited by
60
References
1
Citations by year
2026: 51 · 2025: 9
FWCI
263.45
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
University of Iowa
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
generative AI in education, student engagement, civil engineering education, environmental engineering education, academic integrity, AI ethics, trust in AI, usability, AI policy, educational technology adoption
MeSH terms
Academia, Generative Artificial Intelligence, Artificial Intelligence, Engineering, Female, Humans, Learning, Male, Surveys and Questionnaires, Students, Universities

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review