Chemistry Students’ Artificial Intelligence Literacy through their Critical Reflections of Chatbot Responses
The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
High Resolution Image Download MS PowerPoint Slide Instructors use of Artificial Intelligence (AI) language models (i.e., chatbots) as an educational resource will require an understanding of students’ AI literacy, namely their ability to critically reflect on the relevance, trustworthiness, and quality of these tools in the context of chemistry. This study sought to describe students' AI literacy via open-ended surveys of general chemistry I students and students in an upper-level chemistry elective. Thematic analysis was used to create a deeper understanding of chemistry students’ AI literacy when considering chatbots. Based on students’ responses, they were categorized as either with reservations toward chatbots or without reservations toward chatbots. Thematically, students tended to either reason with the utility/benefit of the tool or reason with concern toward the accuracy of the tool. Results suggest that students are more of a range between these two extremes. This new deeper understanding of AI literacy in chemistry can support instructional practices and inform future research efforts in AI literacy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
In AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038
Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919
Pelau, C., Dabija, D.-C., & Ene, I. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106855 · Cited by 889
Artificial intelligence literacy in higher and adult education: A scoping literature review
Laupichler, M. C., Aster, A., Schirch, J., & Raupach, T. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100101 · Cited by 658
Generative AI in Education and Research: Opportunities, Concerns, and Solutions
Alasadi, E. A., & Baiz, C. R. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00323 · Cited by 463
Leveraging ChatGPT for Enhancing Critical Thinking Skills
Guo, Y., & Lee, D. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00505 · Cited by 250
Chandra, Y., & Shang, L. (2019) · Springer eBooks · doi:10.1007/978-981-13-3170-1_8 · Cited by 233
Emenike, M. E., & Emenike, B. U. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00063 · Cited by 158
Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions
Clark, T. M. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00027 · Cited by 155
Exploring the Usage of ChatGPT in Higher Education: Frequency and Impact on Productivity
Firaina, R., & Sulisworo, D. (2023) · Buletin Edukasi Indonesia · doi:10.56741/bei.v2i01.310 · Cited by 150
Research works printed but not linked: 4
- 10.1080/10494820.2023.225385 · doi:10.1080/10494820.2023.225385
- Bhatia A., 2023, The New York Times
- Cisco Annual Internet Report, (2018–2023) White Paper
- Sahin I., 2006, Turkish Online Journal of Educational Technology-TOJET, 5, 14
Cited by
A systematic literature review of generative artificial intelligence (GenAI) literacy in schools
Park, J. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100487 · Cited by 145
The Dawn of Generative Artificial Intelligence in Chemistry Education
Yuriev, E., Wink, D. J., & Holme, T. A. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c00836 · Cited by 74
Cooper, M. M., & Klymkowsky, M. W. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c00765 · Cited by 48
Yuriev, E., Orgill, M., & Holme, T. A. (2025) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c01105 · Cited by 33
Artificial Intelligence Literacy and Its Associated Factors Among Nursing Students
Jiang, Z., Liu, Q., Jiang, N., Ning, M., & Yu, Q. (2025) · Nurse Educator · doi:10.1097/nne.0000000000001989 · Cited by 32
Sultan, Y., Dautova, G., & Dalle, J. (2025) · Journal of Ethnic and Cultural Studies · doi:10.29333/ejecs/2839 · Cited by 28
“Furthermore, activities based on error detection and justification conducted with chatbots can increase students' morphological and discursive awareness (Young et al., 2024).”
InfluentialYuriev, E., Burton, M. G., Knott, D., Terrill, A. E., Jackson, N. R. C., & Chen, S. (2025) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c01499 · Cited by 26
Ardyansyah, A., Yuwono, A. B., Rahayu, S. L., Alsulami, N. M., & Sulistina, O. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c00220 · Cited by 27
Ovi, J. A., Fierro, G., & Smith, C. E. (2025) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--55471 · Cited by 26
Fayzullina, A. R., Filippova, A. A., N.Yu., G., Astakhov, D. V., Александровна, К. Н., & Zaripova, Z. F. (2025) · Contemporary Educational Technology · doi:10.30935/cedtech/17519 · Cited by 27
Bilişim Alanında Öğrenim Gören Öğrencilerin Yapay Zekâ Okuryazarlığı Düzeylerinin Ölçülmesi
Güler, A., & Polatgıl, M. (2025) · Journal of Computer and Education Research · doi:10.18009/jcer.1549267 · Cited by 14
McAfee, S. C., & Rodriguez, J.-M. G. (2025) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c00958 · Cited by 9
Artificial Intelligence as a Catalyst for Promoting Utility Value Perceptions of Chemistry
Nayyar, P., Terán, Ó. Á. G., & Lewis, S. E. (2025) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c00625 · Cited by 10
Self-citationLi, Y., Tolosa, L. I., Echeverría, F. I. R., & Márquez, R. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c00819 · Cited by 8
Exploring the scope of artificial intelligence literacy in education: A bibliometric approach
Shoustikova, T. V., Zaripova, Z. F., Ибрагимов, Г. И., Zaitseva, N. A., Pashanova, O. V., & Lobuteva, A. V. (2026) · Contemporary Educational Technology · doi:10.30935/cedtech/17892 · Cited by 8
Sequeira, C. A., de Souza, R. S., & Borges, E. M. (2025) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c01414 · Cited by 6
A ChatGPT-Assisted Reading Protocol for Undergraduate Research Students
Sambar, M., Vázquez, G. R., Vázquez, A. V., & Vázquez, F. X. (2025) · The Biophysicist · doi:10.35459/tbp.2024.000281 · Cited by 5
Zheng, H., Han, F., Yang, H., Xu, J., Wu, Y., Wu, J., & Li, T. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2711024 · Cited by 3
Varju, B. R., Noman, M. E., Dicks, A. P., & Seferos, D. S. (2025) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c00790 · Cited by 2
Harjono, H., Sumarni, W., Sumarti, S. S., Munzil, Muntholib, Setiawan, B., Wicaksono, I. R., & Ramadhani, D. G. (2026) · Journal of Science Education and Technology · doi:10.1007/s10956-025-10286-1 · Cited by 1
“This written format encouraged critical thinking, allowing students to articulate their reasoning clearly, supporting a reflective learning approach (Young et al., 2024).”
Kadayifçi, H., Ekici, F., & Ulutaş, B. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c00869 · Cited by 1
Smith, D. K. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c01652 · Cited by 1
Abayeva, G. A., & Askarovna, B. L. (2026) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/18562 · Cited by 1
Development and Implementation of an AI Tutor To Scaffold Problem-Solving in Introductory Chemistry
Wang, K. D., Arens, J. T., Yu, C., Smith, J., Bartels, H., Haub, M., Fung, F. M., Haber, N., Salehi, S., & Poehlmann, J. S. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.6c00039 · Cited by 1
Beyond Automation: The Hidden Curriculum of AI and the Evolving Identity of Chemistry Educators
Reyes, R. L., Astorga, M. J., & Isleta, K. (2026) · International Journal of Innovation in Science and Mathematics Education · doi:10.30722/34.01.003 · Cited by 1
Teaching Chemistry with Technology
Kayima, F. (2025) · Springer texts in education · doi:10.1007/978-3-031-97122-8_8 · Cited by 1
Lo, S., Lea, T., Mavromoustakos, A., Joseph, B., Yii, M., Exintaris, B., Karunaratne, N., Pearson, D. L., Ritchie, T., Yuriev, E., et al. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1818781
Strategies for Self-Regulated Learning with AI in an Undergraduate Research Methods Course
Shao, A. A., Maram, S. S., Matuk, C. F., Shirshekar, S., Carstendottir, E., Melcer, E. F., Sutherland, S. C., El‐Nasr, M. S., & Harteveld, C. (2025) · Proceedings. · doi:10.22318/icls2025.100051
Munawwarah, M., Alqadri, Z., & Zubair, S. (2025) · Asian Journal of Science Education · doi:10.24815/ajse.v7i1.43492
Júnior, G. G., Morais, C., Nascimento-Júnior, N. M., & Miranda, P. C. M. L. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-02675-0_3
Pereira, N. C. P., Malschitzky, M. E. T., & Borges, E. M. (2025) · Preprints.org · doi:10.20944/preprints202511.2232.v1
Čolakovac, I., Barković, I., & Vlajčić, D. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-17272-3_11 · Cited by 1
A Review of AI Literacy Tools’ Contexts, Knowledge Dimensions, and Methodologies
Memarian, B., & Doleck, T. (2026) · AI-Enhanced Learning · doi:10.70725/168632elbfgi
Talanquer, V. (2026) · Royal Society of Chemistry eBooks · doi:10.1039/9781837072538-00245
Using chatbot in education – A systematic review and research agenda
Wut, T. M., Low, M. P., & Tam, K.-W. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.102956 · Cited by 1
Gelin, W. (2026) · UNILA Institutional Repository (Federal University for Latin American Integration)
Ramandha, M. E. P., & Fitri, A. M. (2026) · Jurnal Ilmiah Mandala Education · doi:10.58258/1agh6950
جابر, س. م. (2026) · Awraq Thaqafya · doi:10.67928/5649
Coimbra, D. W. D. R., Fox, R., Costa, A. M. S., Souto, W. M. S., Filho, J. S. M., Sales, R. D. S., Rocha, N. D. S., Paiva, A. M., Lemos, F. C. D. S., da Silva, W. F. B., et al. (2026) · Revista Tópicos · doi:10.70773/revistatopicos/790052529
Júnior, G. G., Chotolli, L. C., da Silva, V. L., Pedro, M. L. A., & Santos, M. D. C. (2026) · Revista da Sociedade Brasileira de Ensino de Química · doi:10.56117/resbenq.2026.v7.e072612
A Protocol to Identify Large Language Model Use in Undergraduate Chemistry Essays
Livingstone, K. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.5c01816
Linh, T. T. M., & Doan, H. H. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2027171
“Current understandings of AI literacy now place a strong emphasis on using AI systems responsibly; determining the reliability of output; mitigating prejudice and hallucination; and protecting authorship, agency, and academic integrity (Chan & Hu, 2023; Young et al., 2024).”
InfluentialDu, Z., Dong, T., & Wei, B. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2744398
Ahmad, Z., Latheef, N. A., Sultana, A., Siby, N., & Chiu, T. K. F. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2735920
Chen, P., Ravindran, L., Amini, M., & Arumugan, N. (2026) · Discover Education · doi:10.1007/s44217-026-02323-x
“Young et al. [62] like-wise demonstrated the critical-evaluative dimension of AI literacy through chemistry students’ reflections on chatbot responses, particularly their judgments of relevance, trustworthiness, quality, utility, and accuracy Hossain et al. [63] further documented that EFL…”
Empowering sustainable education: Unleashing the potential of artificial intelligence
Mulhem, A. A. A. (2026) · International Journal of Education and Practice · doi:10.18488/61.v14i4.5287
“In chemistry education, critical reflections of students upon chatbot responses were employed by Young, Dawood, and Lewis (2024) to support the advancement of AI literacy.”
Kalicheti, C., & Yik, B. J. (2026) · Journal of Chemical Education · doi:10.1021/acs.jchemed.6c01119
Metadata
- Authors
- Young, J. D., Dawood, L., & Lewis, S. E.
- Affiliations
- University of South Florida (US)
- Corresponding author
- Lewis, Scott E.
- Venue
- Journal of Chemical Education
- Publisher
- American Chemical Society (ACS)
- Published
- 2024-05-26
- Volume
- 101
- Issue
- 6
- Pages
- 2466-2474
- ISSN
- 0021-9584 (ISSN-L)
- 1938-1328
- OpenAlex
- W4399027682
- Cited by
- 35
- References
- 25
- Citations by year
- 2026: 16 · 2025: 16 · 2024: 3
- FWCI
- 13.16
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Journal of Chemical Education (publishedVersion)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Digital literacy in education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- AI literacy, chatbots, large language models, chemistry students, critical reflection, student attitudes toward AI, trustworthy AI, general chemistry, chemistry education
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.