The Role of Instructional Guidance in Generative AI-Assisted Learning: Empirical Evidence from Construction Engineering Education
Open accessAbstract
Generative artificial intelligence (AI) is increasingly used to support self-directed learning, yet student interaction with such systems often remains unstructured, limiting engagement in deeper cognitive processes. This study examines how instructional guidance shapes student and AI interaction in construction education. A five-step prompting framework grounded in Generative Learning Theory (GLT) is introduced to guide learner interaction during review activities. A controlled experiment compares three learning conditions: slide-based learning, unprompted AI-supported learning, and prompted AI-supported learning. Learning performance is assessed using multiple-choice and open-ended tasks, and user experience is measured using the User Experience Questionnaire (UEQ). Performance differences are concentrated on tasks requiring explanation and reasoning. The prompted condition achieves higher open-ended scores, with an improvement of approximately 2 or 3 points on a scale of 18 (p < 0.01), while no significant differences are observed in multiple-choice performance. The unprompted condition remains comparable to slide-based learning. These findings indicate that the effectiveness of AI-supported learning depends on how interaction is structured. The proposed framework provides a basis for integrating learning science principles into generative AI systems for construction education.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Six experiments randomize how learners are steered to use the same AI tool.
“We identified six experiments that randomize how learners are steered to use the same AI tool: a tutoring prompt that withholds answers (Bastani et al., 2025), a scaffolded tutor (Bassner et al., 2026), a structured sequence of prompts (Hou et al., 2026), written guidance (Gallegos, 2026; Franco et al., 2026), and a note-taking requirement (Kreijkes et al., 2026).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Hou et al., 2026
Steering learners toward augmentation raises subsequent test scores in four of the six experiments.
“Steering learners toward augmentation raises subsequent test scores in four of the six experiments: by 0.18 SD in Bastani et al. (2025), 0.22 SD in Gallegos (2026), 0.78 SD in Hou et al. (2026), and 0.13 to 0.14 SD in Kreijkes et al. (2026).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Hou et al., 2026
Hou et al. (2026) found that steering learners toward augmentation raises subsequent test scores by 0.78 SD.
“Steering learners toward augmentation raises subsequent test scores in four of the six experiments: by 0.18 SD in Bastani et al. (2025), 0.22 SD in Gallegos (2026), 0.78 SD in Hou et al. (2026), and 0.13 to 0.14 SD in Kreijkes et al. (2026).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Hou et al., 2026
In Hou et al. (2026), guided AI use raised unassisted post-test total scores by 0.86 SD compared to lecture slides.
“guided AI use raised total scores by 0.86 SD, while unguided use produced a near-zero effect (0.09 SD).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Hou et al. (2026)
In Hou et al. (2026), unguided AI use produced a near-zero effect (0.09 SD) on unassisted post-test total scores.
“guided AI use raised total scores by 0.86 SD, while unguided use produced a near-zero effect (0.09 SD).”
From Experimental Evidence on the Learning Impact of Generative AI (2026) · cites this paper as Hou et al. (2026)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 23,433 · cited 1× · see citing works
“(Shrout and Fleiss 1979), indicating excellent agreement between raters (ICC = 0.91).”
InfluentialGenerative Processes of Comprehension
Wittrock, M. (1989) · Educational Psychologist · doi:10.1207/s15326985ep2404_2 · Cited by 978 · cited 1× · see citing works
“Generative Learning Theory conceptualizes learning as a constructive process in which learners organize information and relate it to prior knowledge to build coherent mental representations (Wittrock 1989).”
Self-Directed Learning in Problem-Based Learning and its Relationships with Self-Regulated Learning
Loyens, S. M. M., Magda, J., & Rikers, R. M. J. P. (2008) · Educational Psychology Review · doi:10.1007/s10648-008-9082-7 · Cited by 882 · see citing works
Opportunities and Challenges of Integrating Generative Artificial Intelligence in Education
AlAli, R. M., & Wardat, Y. (2024) · International Journal of Religion · doi:10.61707/8y29gv34 · Cited by 147 · cited 1× · see citing works
“Generative artificial intelligence (AI) is increasingly being incorporated into educational settings as a tool for learning support and instructional assistance (AlAli and Wardat 2024).”
Guo, L. (2022) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12650 · Cited by 141 · cited 1× · see citing works
“…studies in computer-based environments found that metacognitive prompts significantly improved both self-regulated learning activities (g = 0.50) and learning outcomes (g = 0.40), providing quantitative support for structured prompting as an effective instructional intervention (Guo 2022).”
InfluentialEffects of Reflection Prompts When Learning with Hypermedia
Bannert, M. (2006) · Journal of Educational Computing Research · doi:10.2190/94v6-r58h-3367-g388 · Cited by 136 · cited 1× · see citing works
“…in hypermedia and related digital settings report substantial variability in learners’ planning, monitoring, and evaluative behaviors, and many students demonstrate limited spontaneous use of self-regulatory strategies when instructional support is minimal (Azevedo and Cromley 2004; Bannert 2006).”
Fathi, T. E., Saad, A., Larhzil, H., Lamri, D., & Ibrahmi, E. M. A. (2025) · Disciplinary and Interdisciplinary Science Education Research · doi:10.1186/s43031-025-00125-z · Cited by 127 · cited 1× · see citing works
“…conversational agents has shown that AI-generated explanations can assist learners in articulating conceptual understanding, revisiting previously learned material, and reflecting on problem-solving strategies during independent study (Aluko et al. 2025; El Fathi et al. 2025; Mueller et al. 2025).”
Self-Regulated Learning in Technology Enhanced Learning Environments
Persico, D., & Steffens, K. (2017) · Springer eBooks · doi:10.1007/978-3-319-02600-8_11 · Cited by 74 · cited 1× · see citing works
“Research in technology-enhanced learning has extensively examined how learners regulate cognitive and metacognitive processes during self-directed study in computer-based environments (Persico and Steffens 2017; Sui, Yen and Chang 2024).”
Generative Learning: Principles and Implications for Making Meaning
Lee, H. W., Lim, K. Y., & Grabowski, B. L. (2007) · doi:10.4324/9780203880869.ch10 · Cited by 61 · cited 1× · see citing works
“Core generative processes include selecting relevant information, organizing relationships among ideas, and integrating new material with existing schemas (Fiorella and Mayer 2015) (Lee, Lim and Grabowski 2008).”
ChatGPT as an educational resource for civil engineering students
Uddin, S. M. J., Albert, A., Tamanna, M., Ovid, A., & Alsharef, A. F. (2024) · Computer Applications in Engineering Education · doi:10.1002/cae.22747 · Cited by 57 · cited 2× · see citing works
“…evidence shows that while AI-assisted explanations can enhance the clarity and depth of student responses, unstructured use often results in superficial engagement, reinforcing the need for instructional scaffolding that emphasizes critical evaluation over answer acceptance (Uddin et al. 2024).”
Wu, X.-Y., & Chiu, T. K. F. (2025) · Journal of New Approaches in Educational Research · doi:10.1007/s44322-025-00028-x · Cited by 53 · cited 1× · see citing works
“…have further reported that generative AI–supported learning environments can facilitate formative feedback and sustain learner engagement when students interact with conceptually complex or information-dense instructional content outside scheduled class time (Wu and Chiu 2025; Zhan et al. 2025).”
Hsu, Y., Wang, C.-Y., & Zhang, W. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.10.004 · Cited by 29 · cited 2× · see citing works
“Sequential analyses of inquiry-based learning tasks show that structured prompts influence learners’ regulatory patterns, with more successful learners demonstrating systematic monitoring and evaluation during prompted activities (Hsu, Wang and Zhang 2017).”
Investigating effects of perceived technology-enhanced environment on self-regulated learning
Sui, C.-J., Yen, M.-H., & Chang, C.-Y. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12270-x · Cited by 27 · cited 1× · see citing works
“Research in technology-enhanced learning has extensively examined how learners regulate cognitive and metacognitive processes during self-directed study in computer-based environments (Persico and Steffens 2017; Sui, Yen and Chang 2024).”
Wang, X., Mayer, R., Zhou, P., & Lin, L. (2021) · Journal of Educational Psychology · doi:10.1037/edu0000606 · Cited by 25 · cited 1× · see citing works
“Interactive graphic organizers are associated with deeper integrative processing and improved comprehension compared with static text or completed representations (Wang et al. 2021).”
Performance of a Large‐Language Model in scoring construction management capstone design projects
Castelblanco, G., Castro, L. M. C., & Yang, Z. (2024) · Computer Applications in Engineering Education · doi:10.1002/cae.22796 · Cited by 22 · see citing works
Aluko, H. A., Aluko, A., Offiah, G. A., Ogunjimi, F., Aluko, A. O., Alalade, F. M., Ukeje, I. O., & Nwani, C. H. (2025) · International journal of organizational analysis · doi:10.1108/ijoa-07-2024-4632 · Cited by 20 · cited 1× · see citing works
“…conversational agents has shown that AI-generated explanations can assist learners in articulating conceptual understanding, revisiting previously learned material, and reflecting on problem-solving strategies during independent study (Aluko et al. 2025; El Fathi et al. 2025; Mueller et al. 2025).”
Leveraging Generative AI for On-Demand Tutoring as a New Paradigm in Education
Leon, M. (2024) · International Journal on Cybernetics & Informatics · doi:10.5121/ijci.2024.130502 · Cited by 15 · cited 1×
“Prior studies have explored applications such as on-demand explanation generation, interactive question answering, and adaptive support that respond to learners’ evolving needs during review activities (Bernasconi, Redavid and Ferilli 2025; Leon 2024; Yogi, Chowdary and Santhoshi 2024).”
Bernasconi, E., Redavid, D., & Ferilli, S. (2025) · Electronics · doi:10.3390/electronics14071481 · Cited by 14 · cited 1× · see citing works
“Prior studies have explored applications such as on-demand explanation generation, interactive question answering, and adaptive support that respond to learners’ evolving needs during review activities (Bernasconi, Redavid and Ferilli 2025; Leon 2024; Yogi, Chowdary and Santhoshi 2024).”
Li, P., Wu, T., Wang, S. T., Lin, C., & Huang, Y. (2025) · Journal of Research on Technology in Education · doi:10.1080/15391523.2025.2586518 · Cited by 7 · cited 1× · see citing works
“…cognitive and metacognitive supports, commonly implemented through prompts that activate reflection and regulation, have been systematically investigated as instructional design mechanisms to promote deeper processing and improve learning outcomes (Hsu, Wang and Zhang 2017; Li et al. 2025).”
The Impact of Artificial Intelligence on Human Thought
Gesnot, R. (2025) · arXiv.org · doi:10.48550/arxiv.2508.16628 · Cited by 7 · cited 1×
“Empirical studies indicate that LLMs are most effective in facilitating conceptual understanding and interpretive reasoning, while their performance remains limited in advanced analytical and calculation-intensive problem-solving tasks (Gesnot 2025).”
Genç, O. (2025) · Journal of Asian Architecture and Building Engineering · doi:10.1080/13467581.2025.2574556 · Cited by 3 · cited 1× · see citing works
“Research on interaction design demonstrates that structured prompting strategies can significantly improve LLM performance on previously incorrect engineering exam questions, underscoring the role of students’ inquiry practices in mediating educational benefit (Genç 2025).”
Performance of AI Chatbots on the fundamentals of engineering civil exam
Oblitas, L., Adeeb, S., & Cruz-Noguez, C. A. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00646-6 · Cited by 2 · see citing works
Hou, X., Walavalkar, A., Xiao, B., Liu, H., & Mueller, S. T. (2026) · Journal of Management in Engineering · doi:10.1061/jmenea.meeng-7164 · Cited by 1 · cited 2× · see citing works
“Building on prior work in which a course-aligned generative AI platform was developed for construction education contexts (Hou et al. 2026), the present study shifts the focus from system feasibility to the pedagogical design of student–AI interaction during self-directed learning.”
Self-citation
Flagged first, then by citations
2 of the 25 references couldn’t be matched to a known paper.
Research works printed but not linked: 11
- Azevedo, R., and Cromley, J. G. (2004). "Does training on self-regulated learning facilitate students' learning with hypermedia?" Journal of Educational Psychology, 96(3), 523-535.
- Bisra, K., Liu, Q., Nesbit, J. C., Salimi, F., and Winne, P. H. (2018). "Inducing self-explanation: A meta-analysis." Educational Psychology Review, 30(3), 703-725.
- Engelmann, K., Bannert, M., and Melzner, N. (2021). "Do self-created metacognitive prompts promote short-and long-term effects in computer-based learning environments?" Research and Practice in Technology Enhanced Learning, 16(1), 3.
- Fiorella, L., and Mayer, R. E. (2015). Learning as a generative activity. Cambridge University Press.
- Fiorella, L., and Mayer, R. E. (2016). "Eight ways to promote generative learning." Educational Psychology Review, 28(4), 717-741.
- Jelodar, M. B. (2025). "Generative AI, large language models, and ChatGPT in construction education, training, and practice." Buildings, 15(6), 933.
- Maalek, R. (2024). "Integrating generative artificial intelligence and problem-based learning into the digitization in construction curriculum." Buildings, 14(11), 3642.
- Matzakos, N., and Moundridou, M. (2025). "Exploring large language models integration in higher education: A case study in a mathematics laboratory for civil engineering students." Computer Applications in Engineering Education, 33(3), e70049.
- Mueller, S. T., Carlson, K., Wang, Y., Liu, H., and Xiao, B. (2025). "Using LLMs, knowledge space representations and worked examples to support adaptive cybertraining in construction education." In Lecture Notes in Computer Science, Springer, 222-231.
- Schrepp, M., Hinderks, A., and Thomaschewski, J. (2017). "Construction of a benchmark for the User Experience Questionnaire (UEQ)." International Journal of Interactive Multimedia and Artificial Intelligence, 4(4), 40-44.
- Zhan, Y., Boud, D., Dawson, P., and Yan, Z. (2025). "Generative artificial intelligence as an enabler of student feedback engagement: A framework." Higher Education Research & Development, 44(5), 1289-1304.
Cited by
Sorted by citations
Experimental Evidence on the Learning Impact of Generative AI
Contractor, Z., & Reyes, G. (2026) · arXiv · doi:10.48550/arxiv.2607.08849
- Authors
- Hou, X., Xiao, B., Liu, H., & Mueller, S.
- Affiliations
- Michigan Technological University (US)
- Western Michigan University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2606.05509
- OpenAlex
- W7163696605
- Cited by
- 0
- References
- 25
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2417804)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Keywords
- instructional guidance, prompting framework, construction engineering education, student-AI interaction, self-directed learning, user experience, open-ended tasks, learning performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review