The DOI matches its Crossref record; 100 of the 101 references it cites were found in a registry or an index, and 1 are not checked yet; 10 of the 47 sources its claims cite were checked for support.
The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications
Open accessAbstract
A growing number of students in higher education are using ChatGPT for various educational purposes, ranging from seeking information to writing essays. Although many universities have officially banned the use of ChatGPT because of its potential harm and unintended consequences, it is still important to uncover how students leverage ChatGPT for learning, what challenges emerge, and how we can make better use of ChatGPT in higher education. Thus, we conducted focus group workshops and a series of participatory design sessions with thirty students who have actively interacted with ChatGPT for one semester in university and with other five stakeholders (e.g., professors, AI experts). Based on these, this paper identifies real opportunities and challenges of utilizing and designing ChatGPT for higher education.
Verified 2026-10-25 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
This CHI 2024 study used focus groups and participatory design with 30 undergraduates in a semester-long AI class plus 5 experts to examine ChatGPT in higher education. Students valued efficiency, personalization, and scalability, but reported hallucination, bias, transparency, recency, academic-integrity, social-interaction, and prompt-engineering challenges, leading to design proposals such as fine-tuning/embedding/fact-checking, prompt suggestions, learning maps, multi-agents, and social/metaverse features.
Key findings
- Students found ChatGPT usable and beneficial: all teams praised its speed/efficiency, 24/7 availability, interactivity, and concise explanations; they also valued personalized feedback, perceived objectivity/fairness, a sense of privacy, and scalability for diversity, equity, inclusion, and extensibility with other educational tools.
- Students identified algorithmic problems including hallucination, algorithmic bias (e.g., ChatGPT defaulting characters to specific races), lack of transparency/interpretability, and lack of recency; human/social problems including limited capability, decreased learning and academic integrity, reduced social interaction, lack of human ingenuity, and privacy concerns; and usability problems including difficult prompt engineering and lack of hands-on learning.
- In participatory design, students and experts proposed functions to address hallucination via fine-tuning, embedding, and external fact-checking algorithms; improve usability via prompt suggestions, tutorials, learning maps/navigation, and AI-helper agents; provide less biased but more diverse perspectives via multi-agents; and foster social interactions via one-to-many communication and metaverse/VR-style spaces.
- The paper argues that ChatGPT could complement intelligent tutoring systems: ITSs offer systematic dialogue, feedback, pumps, prompts, and scaffolding, while ChatGPT offers free-form, open-ended, student-directed interaction, but currently places high cognitive burdens on students.
Methods
- Design: semester-long, three-phase curriculum in an accredited undergraduate AI class at an American university on the East Coast, spring 2023, about one month after ChatGPT-3.5’s release; students were permitted to use AI tools including ChatGPT-3.5 and DALL·E2 for legitimate educational purposes.
- Sample: thirty undergraduate students (P01–P30) who actively participated in all curriculum phases, plus five external stakeholders/expert participants (P31–P35), including AI engineers and scholars; the study used focus group workshops, brainstorming, idea sketching, expert sessions, and participatory design sessions.
- Analysis: qualitative thematic analysis with an inductive and iterative process of student-generated content (reports, individual/team reports, sketches, final presentations, focus group design activities, contest submissions) and expert interviews; intercoder reliability was not measured, following similar qualitative and participatory design papers.
Limitations
- Stated: findings are based on English-speaking university students in a Western culture and may not reflect English-as-a-second-language students, students in other regions, or those unable to enter university; the study focused on undergraduates, so graduate students or those with different AI literacy may differ; classroom-based study may have made students uncomfortable sharing frank opinions in front of professors and peers; the study did not systematically observe or measure whether AI tools improved students’ learning; and it used ChatGPT-3.5, so newer versions such as ChatGPT-4 or Pro may yield different results.
- Evident from the text: the sample is small (30 students plus 5 experts) and drawn from a single class/university; no personal demographic information was collected due to de-anonymization concerns; and no intercoder reliability was measured, which the authors justify by citing similar qualitative and participatory design work.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 5 Not valid · 23 Not checked · 126 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“While researchers in HCI and AIEd have long accumulated relevant knowledge of educational AI and even established a basis for human-AI partnership in the classroom [98] beyond the development of successful AI tutors, little is known about how people should approach and prepare for LLM-based chatbots that are being used in all areas of education.”
What the sources say
- Supportedcites [98] [98] Kexin Bella Yang, Vanessa Echeverria, Zijing Lu, Hongyu Mao, Kenneth Holstein, Nikol Rummel, and Vincent Aleven. 2023. Pair-Up: Prototyping Human-AI Coorchestration of Dynamic Transitions between Individual and Collaborative Learning in the Classroom. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , 1-17. https://doi.org/10.1145/3544548.3581398 - source
The citation is attached to the claim about a basis for human–AI partnership in classrooms, not the following claim about LLM-based chatbots. The Introduction describes the paper’s classroom system as sharing orchestration control between teachers and AI; Section 2.1 also discusses human–AI co-orchestration in classrooms. Neither passage needs to establish the separate LLM claim. (LLM, conf=0.92)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The most critical problem of ChatGPT that our participants frequently mentioned was its hallucination”
What the sources say
- Supportedcites [64] [64] OpenAI. 2023. GPT-4 Technical Report. https://doi.org/10.48550/arXiv.2303. 08774 - source
The report identifies hallucination as a significant limitation of GPT models (5 Limitations, p. 10) and describes real-world ChatGPT responses flagged by users as not factual (3.1 Model Mitigations, p. 64). The statement about what the citing authors’ own participants mentioned is their finding, not a finding attributed to this source. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Among recent CHI studies [36, 68, 95, 96] that demonstrate the strong effects of social presence of working or learning in the metaverse”
What the sources say
- Not checked — no full text of the sourcecites [36] [36] Qiao Jin, Yu Liu, Svetlana Yarosh, Bo Han, and Feng Qian. 2022. How Will VR Enter University Classrooms? Multi-stakeholders Investigation of VR in Higher Education. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22), 1-17. https://doi.org/10.1145/3491102.3517542 - source
- Supportedcites [68] [68] Hyanghee Park, Daehwan Ahn, and Joonhwan Lee. 2023. Towards a Metaverse Workspace: Opportunities, Challenges, and Design Implications. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23), 1-20. https://doi.org/10.1145/3544548.3581306 - source
In “Increased Interactions and Collaborations” (page not provided), the paper reports that participants attributed increased productivity to the social presence of the metaverse workspace. It also describes stronger feelings of togetherness, improved relationships, and a greater sense of belonging. These findings support the claim’s working-in-the-metaverse branch; the citation need not establish a separate learning setting. (LLM, conf=0.88)
- Partly supportedcites [95] [95] Julie Williamson, Jie Li, Vinoba Vinayagamoorthy, David A. Shamma, and Pablo Cesar. 2021. Proxemics and Social Interactions in an Instrumented Virtual Reality Workshop. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , 1-13. https://doi.org/10.1145/3411764.3445729 - sourceExperimental — accuracy not yet measured
The paper’s opening summary (p. 1) reports that virtual workshop spaces influenced group formation, shared attention, and personal space; Results §5.1 (p. 10) describes differences in group cohesion across workshop activities. This supports the claim that social interaction matters in a virtual workshop, but the study does not establish strong effects of social presence on working or learning outcomes. (LLM, conf=0.82)
- Partly supportedcites [96] [96] Julie R. Williamson, Joseph O'Hagan, John Alexis Guerra-Gomez, John H Williamson, Pablo Cesar, and David A. Shamma. 2022. Digital Proxemics: Designing Social and Collaborative Interaction in Virtual Environments. In CHI Conference on Human Factors in Computing Systems , 1-12. https://doi.org/10. 1145/3491102.3517594 - sourceExperimental — accuracy not yet measured
The Procedure (section 4.3) describes a collaborative task in a virtual environment, and Social Signals and Attention (section 4.5.2) reports differences in participants’ social behavior between headset and desktop conditions. The paper does not establish strong effects of social presence on working or learning outcomes; its Conclusion (section 6) says further research is needed to understand the resulting experiences. Page numbers were not available in our copy. (LLM, conf=0.82)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Holstein et al. [28, 30], which discovered the least preferred feature of AI tutors among both teachers and students was providing automated emotional support to frustrated students.”
What the sources say
- Could not verifycites [28] [28] Kenneth Holstein and Vincent Aleven. 2022. Designing for human-AI complementarity in K-12 education. AI Magazine 43, 2: 239-248. https://doi.org/10. 1002/aaai.12058 - sourceExperimental — accuracy not yet measured
The Case Study Background describes exploring teachers’ information needs, and the Iterative Piloting section describes teachers offering encouragement that the AI tutor could not provide. Neither reports a preference ranking from both teachers and students or identifies automated emotional support for frustrated students as the least preferred feature (searched: least preferred, frustration, emotional support, and teacher and student preferences). (LLM, conf=0.72)
- Not checked — no full text of the sourcecites [30] [30] Kenneth Holstein, Bruce M. McLaren, and Vincent Aleven. 2018. Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. In Artificial Intelligence in Education: 19th International Conference, AIED 2018, London, UK, June 27-30, 2018, Proceedings, Part I 19 , 154-168. - source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“in another fi eld study of an AI-enhanced classroom, the authors observed a human teacher approach a student who struggled learning with the AI tutor and found out that the student's struggle did not result from interaction with AI but a break up with his significant other [28].”
What the sources say
- Partly supportedcites [28] [28] Kenneth Holstein and Vincent Aleven. 2022. Designing for human-AI complementarity in K-12 education. AI Magazine 43, 2: 239-248. https://doi.org/10. 1002/aaai.12058 - sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
64 more claims
Create a free account to see all 160 claims and to verify the 2 references still open.
- Probably not a substantial claim. 62
- It may state two separate claims as one. 50
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Several authors lament the current state of popular AI detection tools, such as iThenticate, MOSS, and Turnitin, stating they are inadequate for the needs of educators [4,9,10].”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reviewing support is typically provided through quantitative feedback, with a focus on evaluation rather than direct revision [95,132].”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [95]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Park and Ahn [9], for instance, reveal that while ChatGPT can enhance information accessibility and efficiency, it threatens academic integrity by potentially lowering learning engagement and standards.”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“Park and Ahn [9], for instance, reveal that while ChatGPT can enhance information accessibility and efficiency, it threatens academic integrity by potentially lowering learning engagement and standards.”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“As we all become more familiar with using generative AI, we are finding that some specific knowledge is necessary (e.g., basic prompting skills, [40]) but that even more important are general principles that guide usage, such as the need to prompt iteratively and encourage students to use their own agency to refine the AI output.”
From Incorporating generative AI into a writing-intensive undergraduate course without off-loading learning (2025) · cites this paper as [40]
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Kytle, J., & Bandura, A. (1978) · Contemporary Sociology · doi:10.2307/2065952 · Cited by 25,422
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 2×
“Kasneci et al. [41] has written a position paper that highlights how LLMs could potentially be utilized to produce educational content, personalize students’ learning, and induce deeper engagement or interaction from students.”
BackgroundTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 1×
“By leveraging massive text datasets and a human-involved feedback system known as ’Reinforcement Learning from Human Feedback’ (RLHF) [65], Chat-GPT has elevated the capabilities of NLP to unprecedented levels.”
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 2×
“Pinpointing the exact skill sets required in the AI era is challenging, but there is broad consensus among experts and scholars that critical thinking and AI literacy are key competencies [54].”
BackgroundArtificial intelligence and the future of work: Human-AI symbiosis in organizational decision making
Jarrahi, M. H. (2018) · Business Horizons · doi:10.1016/j.bushor.2018.03.007 · Cited by 2,772 · cited 1×
“AI is profoundly transforming our work environments [33, 34, 66, 67, 102], reshaping both the nature of work and the skills required.”
BackgroundVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323 · cited 7×
“Some have regarded ITSs as superior to earlier forms of learning technology, such as computer-assisted instruction or computer-based training [22, 80, 87].”
BackgroundResultInfluential
Research works printed but not linked: 8
- Safinah Ali, Blakeley H. Payne, Randi Williams, Hae Won Park, and Cynthia Breazeal. 2019. Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In International workshop on education in artificial intelligence k-12 (eduai’19), 1–4.
- Jill Barshay. 2023. PROOF POINTS: It's easy to fool ChatGPT detectors. The Hechinger Report. Retrieved September 9, 2023 from https://hechingerreport.org/proof-points-its-easy-to-fool-chatgpt-detectors/
- John W. Creswell and Cheryl N. Poth. 2016. Qualitative inquiry and research design: Choosing among five approaches. Sage publications.
- W. Lewis Johnson, Jeff W. Rickel, and James C. Lester. 2000. Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial intelligence in education 11, 1: 47–78.
- Will Summerlin. Productivity Gains Could Propel The AI Software Market To $14 Trillion By 2030 - ARK Invest. Retrieved September 7, 2023 from https://ark-invest.com/articles/analyst-research/productivity-gains-could-propel-the-ai-software-market-to-14-trillion-by-2030/
- J. World Economic Forum. 2020. The future of jobs report 2020. Retrieved from Geneva.
- 2014. Be More Human in the AI Age with Kevin Roose (Transcript). Steelcase. Retrieved September 7, 2023 from https://www.steelcase.com/research/articles/be-more-human-in-the-ai-age-with-kevin-roose-transcript/
- AI art: Artificial Intelligence won an art contest, and artists are furious. Retrieved September 7, 2023 from https://www.9news.com.au/world/ai-art-artificial-intelligence-won-an-art-contest-and-artists-are-furious/31d67f45-c0e7-4572-806f-f3ead1cde73d#
Cited by
Wang, J., & Fan, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04787-y · Cited by 322
“…and generate content to enhance students ’ academic performance, it lacks emotional intelligence (Birenbaum, 2023), limiting its ability to provide the kind of “ humanized ” interaction needed to establish emotional resonance with students or spark deeper learning interest (Park and Ahn, 2024).”
RetractedGenerative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda
Bittle, K., & El-Gayar, O. (2025) · Information · doi:10.3390/info16040296 · Cited by 305
“Park and Ahn [9], for instance, reveal that while ChatGPT can enhance information accessibility and efficiency, it threatens academic integrity by potentially lowering learning engagement and standards.”
InfluentialChatGPT Promises and Challenges in Education: Computational and Ethical Perspectives
Adel, A., Ahsan, A., & Davison, C. (2024) · Education Sciences · doi:10.3390/educsci14080814 · Cited by 201
Pang, R. Y., Schroeder, H., Smith, K. S., Barocas, S., Xiao, Z., Tseng, E., & Bragg, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713726 · Cited by 146
“For example, Park and Ahn [126] mentioned that their research is based only on English-speaking university students, so the result “ may not reflect students who speak English as a second language .”
Rapp, A., Lodovico, C. D., & Di, L. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103471 · Cited by 76
Metadata
- Authors
- Park, H., & Ahn, D.
- Affiliations
- Seoul National University (KR)
- University of Georgia (US)
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-21
- OpenAlex
- W4396831857
- Semantic Scholar
- aeb3963e323aceeedfb61dd1ef194ff8ee7cda5c
- Cited by
- 84
- References
- 83
- Citations by year
- 2026: 31 · 2025: 52 · 2024: 1
- FWCI
- 31.79
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- ChatGPT, large language models, higher education, student use of AI, artificial intelligence in education, generative AI in writing, participatory design, student perceptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-25, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-25 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications. https://app.cochat.ai/papers/10.1145/3613904.3642785
- The paper
- Park, H., & Ahn, D. (2024). The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642785
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.