CoChat
Sign Up
PreprintCited by 143Artificial Intelligence in EducationTeaching and Learning ProgrammingIntelligent Tutoring Systems and Adaptive Learning

The DOI matches its DataCite record.

Trust in Generative AI among students: An Exploratory Study

Open access
Amoozadeh, Matin, Daniels, David, Nam, Daye, Kumar, Aayush, Chen, Stella, Hilton, Michael, Ragavan, Sruti Srinivasa, Alipour, Mohammad Amin·2023·arXiv
DOI 10.48550/arxiv.2310.04631arXiv 2310.04631
Open this paper in CoChat — free accountOpen PDF

Abstract

Generative artificial systems (GenAI) have experienced exponential growth in the past couple of years. These systems offer exciting capabilities, such as generating programs, that students can well utilize for their learning. Among many dimensions that might affect the effective adoption of GenAI, in this paper, we investigate students' \textit{trust}. Trust in GenAI influences the extent to which students adopt GenAI, in turn affecting their learning. In this study, we surveyed 253 students at two large universities to understand how much they trust \genai tools and their feedback on how GenAI impacts their performance in CS courses. Our results show that students have different levels of trust in GenAI. We also observe different levels of confidence and motivation, highlighting the need for further understanding of factors impacting trust.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2310.04631

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ChatGPT can be an efficient tool to support CS learning for underrepresented students, potentially influencing their success and career paths in CS.

    “Also, ChatGPT can be an efficient tool to support CS (Computer Science) learning for underrepresented students, potentially influencing their success and career paths in CS (Amoozadeh et al., 2023).”

    From “ChatGPT seems too good to be true”: College students’ use and perceptions of generative AI (2024) · cites this paper as Amoozadeh et al., 2023

    Section 4.3.3 reports students’ perceptions that ChatGPT helped them learn programming concepts faster. The Discussion and Conclusion suggests that judicious use of GenAI could support first-generation students’ progression in CS programs. The paper does not establish an effect on career paths, and its evidence about underrepresented students concerns first-generation students specifically. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 39

  1. Matin Amoozadeh, David Daniels, Stella Chen, Daye Nam, Aayush Kumar, Michael Hilton, Mohammad Amin Alipour, and Sruti Srinivasa Ragavan. 2023. Towards Characterizing Trust in Generative Artificial Intelligence among Stu- dents. In Proceedings of the 2023 ACM Conference on International Computing Education Research-Volume 2. 3-4.
  2. Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Ben- netot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. In- formation fusion 58 (2020), 82-115.
  3. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. preprint arXiv:2302.04023 (2023). · arXiv
  4. Shraddha Barke, Michael B. James, and Nadia Polikarpova. 2023. Grounded Copilot: How Programmers Interact with Code-Generating Models. Proc. ACM Program. Lang. 7, OOPSLA1 (2023), 85-111. https://doi.org/10.1145/3586030 · DOI · Paper page
  5. Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. 2023. Programming Is Hard -Or at Least It Used to Be: Educational Opportunities and Challenges of AI Code Generation. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education, Volume 1, SIGCSE 2023, Toronto, ON, Canada, March 15-18, 2023. ACM, 500-506.
  6. Bettina Berendt, Allison Littlejohn, and Michael Blakemore. 2020. AI in education: learner choice and fundamental rights. Learning, Media and Technology 45 (2020), 312 -324.
  7. Ruijia Cheng, Ruotong Wang, Thomas Zimmermann, and Denae Ford. 2022. " It would work for me too": How Online Communities Shape Software Developers' Trust in AI-Powered Code Generation Tools. preprint arXiv:2212.03491 (2022). · arXiv
  8. Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research Techniques and Procedures for Developing Grounded Theory. Sage Publications, Inc.
  9. Paul Denny, James Prather, Brett A Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N Reeves, Eddie Antonio Santos, and Sami Sarsa. 2023. Computing Education in the Era of Generative AI. arXiv (2023). arXiv:2306.02608 · arXiv
  10. James Finnie-Ansley, Paul Denny, Brett A. Becker, Andrew Luxton-Reilly, and James Prather. 2022. The Robots Are Coming: Exploring the Implications of Ope- nAI Codex on Introductory Programming. In ACE '22: Australasian Computing Education Conference, Virtual Event, February 14 -18, 2022, Judy Sheard and Paul Denny (Eds.). ACM, 10-19. https://doi.org/10.1145/3511861.3511863 · DOI · Paper page
  11. Arto Hellas, Juho Leinonen, Sami Sarsa, Charles Koutcheme, Lilja Kujanpää, and Juha Sorva. 2023. Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests. arXiv (2023). https://doi.org/10.1145/3568813. 3600139 arXiv:2306.05715 · DOI · Paper page
  12. Alon Jacovi, Ana Marasović, Tim Miller, and Yoav Goldberg. 2021. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in AI. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 624-635.
  13. Alon Jacovi, Ana Marasovic, Tim Miller, and Yoav Goldberg. 2021. Formal- izing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Hu- man Trust in AI. In FAccT '21: 2021 ACM Conference on Fairness, Account- ability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021,
  14. Madeleine Clare Elish, William Isaac, and Richard S. Zemel (Eds.). ACM, 624-635. https://doi.org/10.1145/3442188.3445923 · DOI · Paper page
  15. Samia Kabir, David N Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2023. Who An- swers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions. preprint arXiv:2308.02312 (2023). · arXiv
  16. Majeed Kazemitabaar, Justin Chow, Carl Ka To Ma, Barbara J Ericson, David Weintrop, and Tovi Grossman. 2023. Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. arXiv (2023). https://doi.org/10.1145/3544548.3580919 arXiv:2302.07427 · DOI · Paper page
  17. Amy J Ko, Robert DeLine, and Gina Venolia. 2007. Information needs in collo- cated software development teams. In 29th International Conference on Software Engineering (ICSE'07). IEEE, 344-353.
  18. Moritz Körber. 2019. Theoretical considerations and development of a question- naire to measure trust in automation. In Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018). Springer, 13-30.
  19. Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. 2023. Comparing Code Explanations Created by Students and Large Language Models. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, ITiCSE 2023, Turku, Finland, July 7-12, 2023, Mikko-Jussi Laakso, Mattia Monga, Simon, and Judithe Sheard (Eds.). ACM, 124-130. https://doi.org/10.1145/3587102.3588785 · DOI · Paper page
  20. Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. 2023. Comparing Code Explanations Created by Students and Large Language Models. arXiv (2023). arXiv:2304.03938 · arXiv
  21. Jenny T. Liang, Chenyang Yang, and Brad A. Myers. 2023. Understanding the Usability of AI Programming Assistants. CoRR abs/2303.17125 (2023). https: //doi.org/10.48550/arXiv.2303.17125 arXiv:2303.17125 · DOI · Paper page
  22. Sebastian Linxen, Christian Sturm, Florian Brühlmann, Vincent Cassau, Klaus Opwis, and Katharina Reinecke. 2021. How weird is CHI?. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-14.
  23. Brady D Lund and Ting Wang. 2023. Chatting about ChatGPT: how may AI and GPT impact academia and libraries? Library Hi Tech News 40, 3 (2023), 26-29.
  24. Hussein Mozannar, Gagan Bansal, Adam Fourney, and Eric Horvitz. 2022. Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Program- ming. CoRR abs/2210.14306 (2022). https://doi.org/10.48550/arXiv.2210.14306 arXiv:2210.14306 · DOI · Paper page
  25. Annette M O'Connor, Guy Tsafnat, James Thomas, Paul Glasziou, Stephen B Gilbert, and Brian Hutton. 2019. A question of trust: can we build an evidence base to gain trust in systematic review automation technologies? Systematic reviews 8, 1 (2019), 1-8.
  26. Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2022. Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions. In 43rd IEEE Symposium on Security and Privacy, SP 2022, San Francisco, CA, USA, May 22-26, 2022. IEEE, 754-768. https://doi.org/ 10.1109/SP46214.2022.9833571 · DOI · Paper page
  27. Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2022. Do Users Write More Insecure Code with AI Assistants? CoRR abs/2211.03622 (2022). https://doi.org/10.48550/arXiv.2211.03622 arXiv:2211.03622 · DOI · Paper page
  28. Peter Pirolli and Stuart Card. 1999. Information foraging. Psychological review 106, 4 (1999), 643.
  29. James Prather, Brent N Reeves, Paul Denny, Brett A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. " It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers. preprint arXiv:2304.02491 (2023). · arXiv
  30. Steven I. Ross, Fernando Martinez, Stephanie Houde, Michael J. Muller, and Justin D. Weisz. 2023. The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development. In Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI 2023, Sydney, NSW, March 27-31, 2023. ACM, 491-514. https://doi.org/10.1145/3581641.3584037 · DOI
  31. Sami Sarsa, Paul Denny, Arto Hellas, and Juho Leinonen. 2022. Automatic Gen- eration of Programming Exercises and Code Explanations using Large Language Models. arXiv (2022). https://doi.org/10.48550/arxiv.2206.11861 arXiv:2206.11861 · DOI
  32. Patrick T Terenzini, Leonard Springer, Patricia M Yaeger, Ernest T Pascarella, and Amaury Nora. 1996. First-generation college students: Characteristics, ex- periences, and cognitive development. Research in Higher education 37 (1996), 1-22.
  33. Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI '22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022 -5 May 2022, Extended Abstracts. ACM, 332:1-332:7. https://doi.org/10.1145/3491101.3519665 · DOI · Paper page
  34. Oleksandra Vereschak, Gilles Bailly, and Baptiste Caramiaux. 2021. How to Evalu- ate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies. Proc. ACM Hum.-Comput. Interact. 5, CSCW2, Article 327 (oct 2021), 39 pages.
  35. Ruotong Wang, Ruijia Cheng, Denae Ford, and Thomas Zimmermann. 2023. Investigating and Designing for Trust in AI-powered Code Generation Tools. preprint arXiv:2305.11248 (2023). · arXiv
  36. Tianjia Wang, Daniel Vargas-Diaz, Chris Brown, and Yan Chen. 2023. Towards Adapting Computer Science Courses to AI Assistants' Capabilities. arXiv (2023). arXiv:2306.03289 · arXiv
  37. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. preprint arXiv:2206.07682 (2022). · arXiv
  38. Frank F. Xu, Bogdan Vasilescu, and Graham Neubig. 2022. In-IDE Code Genera- tion from Natural Language: Promise and Challenges. ACM Trans. Softw. Eng. Methodol. 31, 2 (2022), 29:1-29:47. https://doi.org/10.1145/3487569 · DOI · Paper page
  39. Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2022. Productivity assessment of neural code completion. In MAPS@PLDI 2022: 6th ACM SIGPLAN International Symposium on Machine Programming, San Diego, CA, USA, 13 June 2022, Swarat Chaudhuri and Charles Sutton (Eds.). ACM, 21-29. https://doi.org/ 10.1145/3520312.3534864 · DOI · Paper page

Cited by

Sorted by citations

Metadata

Authors
Amoozadeh, M., Daniels, D., Nam, D., Kumar, A., Chen, S., Hilton, M., Ragavan, S. S., & Alipour, M. A.
Affiliations
  • University of Houston (US)
  • Simon Fraser University (CA)
  • Carnegie Mellon University (US)
  • Indian Institute of Technology Kanpur (IN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
67-73
DOI
10.48550/arxiv.2310.04631
arXiv
2310.04631
OpenAlex
W4392542324
Cited by
143
References
0
Citations by year
2026: 41 · 2025: 72 · 2024: 30
FWCI
54.45
Type
conference paper
Language
en
Open access
Licence
Locations
  • ACM Technical Symposium on Computer Science Education (SIGCSE) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (2225373)
  • Universitas Brawijaya (2225373)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
generative AI, trust in AI, computer science education, student trust, artificial intelligence adoption, student self-confidence, student motivation, AI-generated code, academic performance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review