AI or human? A study of university students’ awareness of library reference service agents
The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index.
Abstract
Purpose: This study aims to examine university students’ awareness of artificial intelligence (AI) acting as the agent in university libraries’ reference services. It also aims to identify and discover factors that influence students’ judgement. Design/methodology/approach: A within-participant design experiment was designed for this study. Five tasks covering five disciplines were developed. Two versions (ChatGPT versus subject librarian) of feedback were generated for each task. Participants were asked to judge the identity of the agent (a simplified Turing test) for each task. Think-aloud protocols were used to further analyse the factors affecting students’ judgements. Findings: This study indicates students’ limited ability to distinguish AI and human agents. They are more likely to mistake librarians for AI agents with complicated tasks. The perceived knowledge, ability and comprehensibility of the AI system and the provided information quality and expression approach, together with AI usage experience, significantly affect students’ awareness of AI. Students associate objective, thorough and expert knowledge with AI, while detailed, vivid and colloquial explanations to librarians. Originality/value: This study provides valuable insights into students’ awareness of AI versus human agents in reference services by synthesizing a theoretical model to explain students’ AI awareness, specifying components for each element and their relationships. It also benefits the effective integration of AI technology in libraries, especially in reference services, by emphasizing the importance of keeping pace with the development of AI and improving students’ AI literacy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Making sense of cronbach’s alpha.
Tavakol, M., & Dennick, R. (2011) · International Journal of Medical Education · doi:10.5116/ijme.4dfb.8dfd · Cited by 15,126
Terry, G., Hayfield, N., Clarke, V., & Braun, V. (2017) · The SAGE Handbook of Qualitative Research in Psychology · doi:10.4135/9781526405555.n2 · Cited by 2,448
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 1,435
Wang, B., Rau, P.-L. P., & Yuan, T. (2023) · Behaviour & Information Technology · doi:10.1080/0144929x.2022.2072768 · Cited by 1,036
Pelau, C., Dabija, D.-C., & Ene, I. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106855 · Cited by 889
Baird, A., & Maruping, L. M. (2021) · MIS Quarterly · doi:10.25300/misq/2021/15882 · Cited by 512
Automatic feedback in online learning environments: A systematic literature review
Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.-S., Gašević, D., & Mello, R. F. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100027 · Cited by 440
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363
AI Customer Service: Task Complexity, Problem-Solving Ability, and Usage Intention
Xu, Y., Shieh, C.-H., van Esch, P., & Ling, I.-L. (2020) · Australasian Marketing Journal · doi:10.1016/j.ausmj.2020.03.005 · Cited by 346
Okunlaya, R. O., Syed Abdullah, N., & Alias, R. A. (2022) · Library Hi Tech · doi:10.1108/lht-07-2021-0242 · Cited by 333
Research works printed but not linked: 9
- Braun, 2019, Qualitative Research in, 589
- Gyang, 2020, Library reference services based on artificial intelligence, Villanova Journal of Science, Technology and Management, 2
- Jdaitawi, 2024, Factors affecting students’ willingness to use artificial intelligence in university settings, International Journal of Information and Education Technology, 14
- Koohang, 2024, Students’ perceptions of benefits and opportunities of artificial intelligence (AI), Issues in Information Systems, 25.
- McNeal, 2013, Introducing Chatbots in libraries, Library Technology Reports, 49, 5
- Omehia, 2020, Pros and cons of artificial intelligence in 21st century library and information service delivery, International Journal of Scientific Research in Education, 13, 220
- Rzepka, 2018, User interaction with AI-enabled systems: a systematic review of IS research
- Van Someren, 1994, The Think Aloud Method: A Practical Approach to Modelling Cognitive Processes
- Vijayakumar, 2019, Applications of artificial intelligence in academic libraries, International Journal of Computer Sciences and Engineering, 7, 136
Cited by
Khan, A. R., Mahmood, S. M. F., Akbar, M., & Hossain, S. (2026) · The Electronic Library · doi:10.1108/el-07-2025-0328 · Cited by 6
Sultana, R., & Atikuzzaman, M. (2026) · Journal of Librarianship and Information Science · doi:10.1177/09610006261468357 · Cited by 2
Metadata
- Authors
- Wang, D., Guo, J., Zheng, K., & Deng, X.
- Affiliations
- Renmin University of China (CN)
- Corresponding author
- Wang, Di
- Venue
- The Electronic Library
- Publisher
- Emerald
- Published
- 2025-11-25
- Volume
- 44
- Issue
- 1
- Pages
- 136-154
- ISSN
- 0264-0473 (ISSN-L)
- 1758-616X
- OpenAlex
- W7106301202
- Cited by
- 2
- References
- 39
- Citations by year
- 2026: 2
- FWCI
- 0.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Electronic Library (publishedVersion)
- Funders
- Fundamental Research Funds for the Central Universities (24XNF041)
- Renmin University of China (24XNF041)
- National Natural Science Foundation of China (72104188)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Social Sciences › Library and Information Sciences › Web and Library Services
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Keywords
- artificial intelligence, university students, reference services, AI awareness, ChatGPT, Turing test, AI literacy, student perceptions, information quality, academic libraries
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AI or human? A study of university students’ awareness of library reference service agents. https://app.cochat.ai/papers/10.1108/el-06-2025-0230
- The paper
- Wang, D., Guo, J., Zheng, K., & Deng, X. (2026). AI or human? A study of university students’ awareness of library reference service agents. The Electronic Library. https://doi.org/10.1108/el-06-2025-0230
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.