CoChat

AI or human? A study of university students’ awareness of library reference service agents

Wang, Di, Guo, Jianting, Zheng, Kaiyang, Deng, Xizhou · 2026 · The Electronic Library

The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index.

Abstract

Purpose: This study aims to examine university students’ awareness of artificial intelligence (AI) acting as the agent in university libraries’ reference services. It also aims to identify and discover factors that influence students’ judgement. Design/methodology/approach: A within-participant design experiment was designed for this study. Five tasks covering five disciplines were developed. Two versions (ChatGPT versus subject librarian) of feedback were generated for each task. Participants were asked to judge the identity of the agent (a simplified Turing test) for each task. Think-aloud protocols were used to further analyse the factors affecting students’ judgements. Findings: This study indicates students’ limited ability to distinguish AI and human agents. They are more likely to mistake librarians for AI agents with complicated tasks. The perceived knowledge, ability and comprehensibility of the AI system and the provided information quality and expression approach, together with AI usage experience, significantly affect students’ awareness of AI. Students associate objective, thorough and expert knowledge with AI, while detailed, vivid and colloquial explanations to librarians. Originality/value: This study provides valuable insights into students’ awareness of AI versus human agents in reference services by synthesizing a theoretical model to explain students’ AI awareness, specifying components for each element and their relationships. It also benefits the effective integration of AI technology in libraries, especially in reference services, by emphasizing the importance of keeping pace with the development of AI and improving students’ AI literacy.

Overview

Integrity signals

Registry

Crossref · 10.1108/el-06-2025-0230

Cites

45 of 47 references verified1 citation errors · 1 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 47 references

Research works printed but not linked: 9

  1. Braun, 2019, Qualitative Research in, 589
  2. Gyang, 2020, Library reference services based on artificial intelligence, Villanova Journal of Science, Technology and Management, 2
  3. Jdaitawi, 2024, Factors affecting students’ willingness to use artificial intelligence in university settings, International Journal of Information and Education Technology, 14
  4. Koohang, 2024, Students’ perceptions of benefits and opportunities of artificial intelligence (AI), Issues in Information Systems, 25.
  5. McNeal, 2013, Introducing Chatbots in libraries, Library Technology Reports, 49, 5
  6. Omehia, 2020, Pros and cons of artificial intelligence in 21st century library and information service delivery, International Journal of Scientific Research in Education, 13, 220
  7. Rzepka, 2018, User interaction with AI-enabled systems: a systematic review of IS research
  8. Van Someren, 1994, The Think Aloud Method: A Practical Approach to Modelling Cognitive Processes
  9. Vijayakumar, 2019, Applications of artificial intelligence in academic libraries, International Journal of Computer Sciences and Engineering, 7, 136

Cited by

See all 2

Metadata

Authors
Wang, D., Guo, J., Zheng, K., & Deng, X.
Affiliations
Renmin University of China (CN)
Corresponding author
Wang, Di
Venue
The Electronic Library
Publisher
Emerald
Published
2025-11-25
Volume
44
Issue
1
Pages
136-154
ISSN
  • 0264-0473 (ISSN-L)
  • 1758-616X
OpenAlex
W7106301202
Cited by
2
References
39
Citations by year
2026: 2
FWCI
0.81
Type
journal article
Language
en
Open access
closed
Locations
The Electronic Library (publishedVersion)
Funders
  • Fundamental Research Funds for the Central Universities (24XNF041)
  • Renmin University of China (24XNF041)
  • National Natural Science Foundation of China (72104188)
Topics
  • Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
  • Social Sciences › Library and Information Sciences › Web and Library Services
  • Social Sciences › Library and Information Sciences › Library Science and Information Literacy
Keywords
artificial intelligence, university students, reference services, AI awareness, ChatGPT, Turing test, AI literacy, student perceptions, information quality, academic libraries

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: AI or human? A study of university students’ awareness of library reference service agents. https://app.cochat.ai/papers/10.1108/el-06-2025-0230
The paper
Wang, D., Guo, J., Zheng, K., & Deng, X. (2026). AI or human? A study of university students’ awareness of library reference service agents. The Electronic Library. https://doi.org/10.1108/el-06-2025-0230

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.