CoChat
Sign Up
PreprintCited by 70Problem and Project Based LearningArtificial Intelligence in EducationOnline Learning and Analytics

Charting the Future of AI in Project-Based Learning: A Co-Design Exploration with Students

Open access
Zheng, Chengbo, Yuan, Kangyu, Guo, Bingcan, Mogavi, Reza Hadi, Peng, Zhenhui, Ma, Shuai, Ma, Xiaojuan·2024·arXiv
DOI 10.48550/arxiv.2401.14915arXiv 2401.14915
Open this paper in CoChat — free accountOpen PDF

Abstract

The increasing use of Artificial Intelligence (AI) by students in learning presents new challenges for assessing their learning outcomes in project-based learning (PBL). This paper introduces a co-design study to explore the potential of students' AI usage data as a novel material for PBL assessment. We conducted workshops with 18 college students, encouraging them to speculate an alternative world where they could freely employ AI in PBL while needing to report this process to assess their skills and contributions. Our workshops yielded various scenarios of students' use of AI in PBL and ways of analyzing these uses grounded by students' vision of education goal transformation. We also found students with different attitudes toward AI exhibited distinct preferences in how to analyze and understand the use of AI. Based on these findings, we discuss future research opportunities on student-AI interactions and understanding AI-enhanced learning.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2401.14915

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 69 · Cited by 70

Research works printed but not linked: 73

  1. June Ahn, Fabio Campos, Maria Hays, and Daniela DiGiacomo. 2019. Designing in Context: Reaching beyond Usability in Learning Analytics Dashboard Design. Journal of Learning Analytics 6, 2 (2019), 70-85.
  2. Carlos Prieto Alvarez, Roberto Martinez-Maldonado, and Simon Bucking- ham Shum. 2020. LA-DECK: A card-based learning analytics co-design tool. In Proceedings of the tenth international conference on learning analytics & knowl- edge. 63-72.
  3. James Auger. 2010. Alternative Presents and Speculative Futures: Designing fictions through the extrapolation and evasion of product lineages. Negotiating futures-Design Fiction. 6 (2010), 42-57.
  4. Suyun Sandra Bae, Oh-Hyun Kwon, Senthil Chandrasegaran, and Kwan-Liu Ma. 2020. Spinneret: Aiding creative ideation through non-obvious concept associations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13.
  5. Brigid JS Barron, Daniel L Schwartz, Nancy J Vye, Allison Moore, Anthony Petrosino, Linda Zech, and John D Bransford. 1998. Doing with understanding: Lessons from research on problem-and project-based learning. Journal of the learning sciences 7, 3-4 (1998), 271-311.
  6. Stephanie Bell. 2010. Project-based learning for the 21st century: Skills for the future. The clearing house 83, 2 (2010), 39-43.
  7. Linda Birt, Suzanne Scott, Debbie Cavers, Christine Campbell, and Fiona Walter. 2016. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qualitative health research 26, 13 (2016), 1802-1811.
  8. Phyllis C Blumenfeld, Elliot Soloway, Ronald W Marx, Joseph S Krajcik, Mark Guzdial, and Annemarie Palincsar. 1991. Motivating project-based learning: Sustaining the doing, supporting the learning. Educational psychologist 26, 3-4 (1991), 369-398.
  9. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.
  10. Cheng-Huan Chen and Yong-Cih Yang. 2019. Revisiting the effects of project- based learning on students' academic achievement: A meta-analysis investigat- ing moderators. Educational Research Review 26 (2019), 71-81.
  11. EunJeong Cheon, Stephen Tsung-Han Sher, Šelma Sabanović, and Nor- man Makoto Su. 2019. I beg to differ: Soft conflicts in collaborative design using design fictions. In Proceedings of the 2019 on Designing Interactive Systems Conference. 201-214.
  12. EunJeong Cheon and Norman Makoto Su. 2018. Futuristic autobiographies: Weaving participant narratives to elicit values around robots. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. 388-397.
  13. Chris Elsden, Bettina Nissen, Andrew Garbett, David Chatting, David Kirk, and John Vines. 2016. Metadating: exploring the romance and future of personal data. In Proceedings of the 2016 chi conference on human factors in computing systems. 685-698.
  14. Pengyue Guo, Nadira Saab, Lysanne S Post, and Wilfried Admiraal. 2020. A review of project-based learning in higher education: Student outcomes and measures. International journal of educational research 102 (2020), 101586.
  15. Han L Han, Junhang Yu, Raphael Bournet, Alexandre Ciorascu, Wendy E Mackay, and Michel Beaudouin-Lafon. 2022. Passages: interacting with text across documents. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-17.
  16. Avneet Hira and Emma Anderson. 2021. Motivating online learning through project-based learning during the 2020 COVID-19 pandemic. IAFOR Journal of Education 9, 2 (2021), 93-110.
  17. Maurice Jakesch, Megan French, Xiao Ma, Jeffrey T Hancock, and Mor Naaman. 2019. AI-mediated communication: How the perception that profile text was written by AI affects trustworthiness. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-13.
  18. Martin Jonsson and Jakob Tholander. 2022. Cracking the code: Co-coding with AI in creative programming education. In Proceedings of the 14th Conference on Creativity and Cognition. 5-14.
  19. Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. 2023. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences 103 (2023), 102274.
  20. Majeed Kazemitabaar, Justin Chow, Carl Ka To Ma, Barbara J Ericson, David Weintrop, and Tovi Grossman. 2023. Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-23.
  21. Ahmed Kharrufa, Sally Rix, Timur Osadchiy, Anne Preston, and Patrick Olivier. 2017. Group Spinner: recognizing and visualizing learning in the classroom for reflection, communication, and planning. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 5556-5567.
  22. Taenyun Kim, Maria D Molina, Minjin Rheu, Emily S Zhan, and Wei Peng. 2023. One AI Does Not Fit All: A Cluster Analysis of the Laypeople's Perception of AI Roles. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-20.
  23. Dimitra Kokotsaki, Victoria Menzies, and Andy Wiggins. 2016. Project-based learning: A review of the literature. Improving schools 19, 3 (2016), 267-277.
  24. Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lo- rie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz- Candido, James Maningo, et al. 2023. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS digital health 2, 2 (2023), e0000198.
  25. Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q Vera Liao, Yunfeng Zhang, and Chenhao Tan. 2022. Human-ai collaboration via conditional delegation: A case study of content moderation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-18.
  26. Yen-Fen Lee, Gwo-Jen Hwang, and Pei-Ying Chen. 2022. Impacts of an AI- based cha bot on college students' after-class review, academic performance, self-efficacy, learning attitude, and motivation. Educational technology research and development 70, 5 (2022), 1843-1865.
  27. Ian Li, Anind Dey, Jodi Forlizzi, Kristina Höök, and Yevgeniy Medynskiy. 2011. Personal informatics and HCI: design, theory, and social implications. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. 2417-2420.
  28. Joseph CR Licklider. 1960. Man-computer symbiosis. IRE transactions on human factors in electronics 1 (1960), 4-11.
  29. Joseph Lindley and Paul Coulton. 2015. Back to the future: 10 years of design fiction. In Proceedings of the 2015 British HCI conference. 210-211.
  30. Siân E Lindley, Gavin Smyth, Robert Corish, Anastasia Loukianov, Michael Golembewski, Ewa A Luger, and Abigail Sellen. 2018. Exploring new metaphors for a networked world through the file biography. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-12.
  31. Duri Long and Brian Magerko. 2020. What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1-16.
  32. Michal Luria. 2023. Co-Design Perspectives on Algorithm Transparency Re- porting: Guidelines and Prototypes. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1076-1087.
  33. Shuai Ma, Taichang Zhou, Fei Nie, and Xiaojuan Ma. 2022. Glancee: An adaptable system for instructors to grasp student learning status in synchronous online classes. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-25.
  34. Reza Hadi Mogavi, Xiaojuan Ma, and Pan Hui. 2021. Characterizing student engagement moods for dropout prediction in question pool websites. arXiv preprint arXiv:2102.00423 (2021). · arXiv
  35. Emiliana Murgia, Zahra Abbasiantaeb, Mohammad Aliannejadi, Theo Huibers, Monica Landoni, and Maria Soledad Pera. 2023. ChatGPT in the Classroom: A Preliminary Exploration on the Feasibility of Adapting ChatGPT to Support Chil- dren's Information Discovery. In Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization. 22-27.
  36. Judith S Olson and Wendy A Kellogg. 2014. Ways of Knowing in HCI. Vol. 2. Springer.
  37. Zachary A Pardos, Matthew Tang, Ioannis Anastasopoulos, Shreya K Sheel, and Ethan Zhang. 2023. OATutor: An Open-source Adaptive Tutoring System and Curated Content Library for Learning Sciences Research. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-17.
  38. Zhenhui Peng, Xingbo Wang, Qiushi Han, Junkai Zhu, Xiaojuan Ma, and Huamin Qu. 2023. Storyfier: Exploring Vocabulary Learning Support with Text Genera- tion Models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1-16.
  39. Beatriz Pérez and Ángel L Rubio. 2020. A project-based learning approach for enhancing learning skills and motivation in software engineering. In Proceedings of the 51st ACM technical symposium on computer science education. 309-315.
  40. Lara Piccolo, Daniel Buzzo, Martin Knobel, Prasanna Gunasekera, and Tina Papathoma. 2023. Interaction Design as Project-Based Learning: Perspectives for Unsolved Challenges. In Proceedings of the 5th Annual Symposium on HCI Education. 59-67.
  41. Carlos G Prieto-Alvarez, Roberto Martinez-Maldonado, and Theresa Dirndorfer Anderson. 2018. Co-designing learning analytics tools with learners. In Learning analytics in the classroom. Routledge, 93-110.
  42. Carlos Gerardo Prieto-Alvarez, Roberto Martinez-Maldonado, and Simon Buck- ingham Shum. 2018. Mapping learner-data journeys: Evolution of a visual co-design tool. In Proceedings of the 30th Australian conference on computer- human interaction. 205-214.
  43. Junaid Qadir. 2023. Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education. In 2023 IEEE Global Engineering Education Conference (EDUCON). IEEE, 1-9.
  44. Tareq Rasul, Sumesh Nair, Diane Kalendra, Mulyadi Robin, Fernando de Oliveira Santini, Wagner Junior Ladeira, Mingwei Sun, Ingrid Day, Raouf Ah- mad Rather, and Liz Heathcote. 2023. The role of ChatGPT in higher education: Benefits, challenges, and future research directions. Journal of Applied Learning and Teaching 6, 1 (2023).
  45. Dan Richardson and Ahmed Kharrufa. 2020. We are the greatest showmen: Configuring a framework for project-based mobile learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-12.
  46. Danae Romrell, Lisa Kidder, and Emma Wood. 2014. The SAMR model as a framework for evaluating mLearning. Online Learning Journal 18, 2 (2014).
  47. Ethan Z Rong, Mo Morgana Zhou, Ge Gao, and Zhicong Lu. 2023. Understanding Personal Data Tracking and Sensemaking Practices for Self-Directed Learning in Non-classroom and Non-computer-based Contexts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-16.
  48. Jürgen Rudolph, Samson Tan, and Shannon Tan. 2023. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 6, 1 (2023).
  49. Juan Pablo Sarmiento and Alyssa Friend Wise. 2022. Participatory and co- design of learning analytics: An initial review of the literature. In LAK22: 12th international learning analytics and knowledge conference. 535-541.
  50. Douglas Schuler and Aki Namioka. 1993. Participatory design: Principles and practices. CRC Press.
  51. Chuhan Shi, Yicheng Hu, Shenan Wang, Shuai Ma, Chengbo Zheng, Xiaojuan Ma, and Qiong Luo. 2023. RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning. In Proceedings of the 2023 CHI Con- ference on Human Factors in Computing Systems. 1-20.
  52. Brett Smith. 2018. Generalizability in qualitative research: Misunderstandings, opportunities and recommendations for the sport and exercise sciences. Quali- tative research in sport, exercise and health 10, 1 (2018), 137-149.
  53. Sarah Sterman, Molly Jane Nicholas, Janaki Vivrekar, Jessica R Mindel, and Eric Paulos. 2023. Kaleidoscope: A Reflective Documentation Tool for a User Interface Design Course. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-19.
  54. John Thomas. 2000. A Review of Research on Project-Based Learning. (01 2000).
  55. Yao Tian, Chengwei Tong, Lik-Hang Lee, Reza Hadi Mogavi, Yong Liao, and Pengyuan Zhou. 2023. Last Week with ChatGPT: A Weibo Study on Social Perspective regarding ChatGPT for Education and Beyond. arXiv preprint arXiv:2306.04325 (2023). · arXiv
  56. Rama Adithya Varanasi and Nitesh Goyal. 2023. "It is currently hodgepodge": Examining AI/ML Practitioners' Challenges during Co-production of Responsi- ble AI Values. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-17.
  57. Dakuo Wang, Justin D Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-ai collaboration in data science: Exploring data scientists' perceptions of automated ai. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1-24.
  58. Qiaosi Wang, Shan Jing, and Ashok K Goel. 2022. Co-Designing AI Agents to Support Social Connectedness Among Online Learners: Functionalities, So- cial Characteristics, and Ethical Challenges. In Designing Interactive Systems Conference. 541-556.
  59. Qiaosi Wang, Michael Madaio, Shaun Kane, Shivani Kapania, Michael Terry, and Lauren Wilcox. 2023. Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-16.
  60. Qiaosi Wang, Koustuv Saha, Eric Gregori, David Joyner, and Ashok Goel. 2021. Towards mutual theory of mind in human-ai interaction: How language reflects what students perceive about a virtual teaching assistant. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1-14.
  61. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824-24837.
  62. Daniel Weitekamp, Erik Harpstead, and Ken R Koedinger. 2020. An interaction design for machine teaching to develop AI tutors. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1-11.
  63. Jordan Wirfs-Brock, Sarah Mennicken, and Jennifer Thom. 2020. Giving voice to silent data: Designing with personal music listening history. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-11.
  64. Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022. Ai chains: Transpar- ent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI conference on human factors in computing systems. 1-22.
  65. Meng Xia, Yuya Asano, Joseph Jay Williams, Huamin Qu, and Xiaojuan Ma. 2020. Using information visualization to promote students' reflection on" gaming the system" in online learning. In Proceedings of the Seventh ACM Conference on Learning@ Scale. 37-49.
  66. Meng Xia, Mingfei Sun, Huan Wei, Qing Chen, Yong Wang, Lei Shi, Huamin Qu, and Xiaojuan Ma. 2019. Peerlens: Peer-inspired interactive learning path planning in online question pool. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1-12.
  67. Kexin Bella Yang, Vanessa Echeverria, Zijing Lu, Hongyu Mao, Kenneth Holstein, Nikol Rummel, and Vincent Aleven. 2023. Pair-Up: Prototyping Human-AI Co- orchestration of Dynamic Transitions between Individual and Collaborative Learning in the Classroom. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-17.
  68. Nur Yildirim, Changhoon Oh, Deniz Sayar, Kayla Brand, Supritha Challa, Violet Turri, Nina Crosby Walton, Anna Elise Wong, Jodi Forlizzi, James McCann, et al. 2023. Creating Design Resources to Scaffold the Ideation of AI Concepts. In Proceedings of the 2023 ACM Designing Interactive Systems Conference. 2326- 2346.
  69. JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny can't prompt: how non-AI experts try (and fail) to design LLM prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-21.
  70. Ashley Ge Zhang, Yan Chen, and Steve Oney. 2023. VizProg: Identifying Misun- derstandings By Visualizing Students' Coding Progress. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-16.
  71. Chengbo Zheng, Dakuo Wang, April Yi Wang, and Xiaojuan Ma. 2022. Telling stories from computational notebooks: Ai-assisted presentation slides creation for presenting data science work. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-20.
  72. Chengbo Zheng, Yuheng Wu, Chuhan Shi, Shuai Ma, Jiehui Luo, and Xiaojuan Ma. 2023. Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-19.
  73. Barry J Zimmerman. 2002. Becoming a self-regulated learner: An overview. Theory into practice 41, 2 (2002), 64-70.

Authors
Zheng, C., Yuan, K., Guo, B., Mogavi, R. H., Peng, Z., Ma, S., & Ma, X.
Affiliations
  • Hong Kong University of Science and Technology (HK)
  • Sun Yat-sen University (CN)
  • University of Waterloo (CA)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Pages
1-19
DOI
10.48550/arxiv.2401.14915
arXiv
2401.14915
OpenAlex
W4396832913
Cited by
70
References
69
Citations by year
2026: 18 · 2025: 44 · 2024: 8
FWCI
236.54
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China
  • University of Waterloo
  • Natural Science Foundation for Young Scientists of Shanxi Province (62202509)
  • Young Scientists Fund
Topics
  • Social Sciences › Education › Problem and Project Based Learning
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
artificial intelligence in education, project-based learning, assessment of student learning, student attitudes toward AI, student-AI interaction, AI-assisted learning, educational goals, university students

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review