CoChat
Sign Up
Cited by 5AI in Service InteractionsOnline Learning and AnalyticsBlockchain Technology Applications and Security

Using Agentic AI to Enhance the Quality of Academic Libraries’ Responses for Student Queries

Open access
Wang, Ning‐Chiao, Chou, Yen‐Chen·2025·Proceedings of the Association for Information Science and Technology
DOI 10.1002/pra2.1518
Open this paper in CoChat — free accountOpen PDF

Abstract

ABSTRACT This study examines the application of agentic AI to evaluate the effectiveness, accuracy, and efficiency of academic libraries’ responses to student queries. Data were collected by analyzing frequently asked questions (FAQs) from the University of Wisconsin‐Milwaukee library website using OpenAI's large language models (LLMs), o4‐mini. The analysis focuses on identifying the types of questions that are most appropriate for AI‐assisted responses and how agentic AI can support librarians in delivering targeted and timely assistance. The findings reveal that agentic AI can significantly improve the quality of library services by providing more timely and targeted student support. This research aims to enhance the effectiveness of librarian responses, strengthen librarian‐student interactions, and support the meaningful integration of agentic AI into academic libraries.

Integrity signals

Registry

Crossref · 10.1002/pra2.1518

Cites

20 of 21 references verified1 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

2 of 13 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Large language models offered a promising solution by interpreting natural language, understanding context, and generating relevant responses.

    “Large language models (LLMs) offered a promising solution by interpreting natural language, understanding context, and generating relevant responses (Shah et al., 2023;Wang et al., 2024);”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Shah et al., 2023 source

      The Introduction and Section 3.3 describe using LLMs to analyze users’ natural-language requests and develop categories for understanding their intent, supporting the claim that LLMs offer a promising approach to interpreting requests. The paper also warns that chat logs may lack sufficient context, and it does not establish that LLMs generate relevant responses to those requests; that part of the attributed claim goes beyond its findings. No page numbers are available in our copy. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites Wang et al., 2024 source
  • Findingreasoning

    Well-structured sentences allowed the general assistant agent to interpret intent effectively and initiate appropriate follow-up questions.

    “Wellstructured sentences allowed the general assistant agent to interpret intent effectively and initiate appropriate followup questions.”
    Quote in the paperSubstantial claim
  • Findingown data

    Student queries with specific linguistic, structural, and contextual features significantly improved the performance of agentic AI.

    “For RQ2, the study found that student queries with specific linguistic, structural, and contextual features significantly improved the performance of agentic AI.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The 74 FAQs and anonymized student queries from the UWM library website (2022-2025) are representative of student queries in academic libraries generally.
    • Rests on an assumption: The single prompt example used for RQ1 is representative of school-specific AI-agent-potential queries.
  • Findingreasoning

    Queries that clearly stated the subject matter enabled the system to identify relevant resources more accurately.

    “Queries that clearly stated the subject matter (e.g.,”
    Quote in the paperSubstantial claim
  • Findingreasoning

    Contextual details (such as intended grade level or use case) further refined the AI's responses by guiding task delegation and resource filtering.

    “Contextual details (such as intended grade level or use case) further refined the AI's responses by guiding task delegation and resource filtering.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 3
  • May combine two claims. 3

  • The quote only partly states this claim.

    Findingown data

    The study categorized 74 FAQs and anonymized student queries from the UWM library website into three types: General_QA (n=10), School_Specific_with_Fact_Only (n=55), and School_Specific_with_AI_Agent_Potential (n=9).

    “This study analyzed 74 frequently asked questions (FAQs) and anonymized student queries from the University of Wisconsin-Milwaukee (UWM) library website (2022-2025) to explore opportunities for AI integration.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: OpenAI's o4-mini can reliably semantically analyze and categorize library FAQs and student queries into the three types.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Agentic AI introduces capabilities such as multi-agent collaboration, dynamic task decomposition, persistent memory and autonomous orchestration.

    “agentic AI introduces capabilities such as multi-agent collaboration, dynamic task decomposition, persistent memory and autonomous orchestration, yet despite its transformative potential remains underexplored in libraries, likely due to inherent complexity, and the need for oversight and institutional trust (Sapkota et al., 2025).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Sapkota et al., 2025

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    In academic libraries, LLMs can enhance virtual and digital reference services, automate content summarization, and support interactive learning.

    “In academic libraries these models can enhance virtual and digital reference services, automate content summarization, and support interactive learning (Ikwuanusi et al., 2023;Saeidnia, 2023;Soto et al., 2024).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Ikwuanusi et al., 2023 source
    • cites Saeidnia, 2023 source
    • cites Soto et al., 2024 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    LLMs' ability to deliver fast, context-aware responses stems from large-scale reinforcement learning (RL) and the integration of chain-of-thought (CoT) reasoning techniques.

    “Their ability to deliver fast, context-aware responses stems from large-scale reinforcement learning (RL) and the integration of chain-of-thought (CoT) reasoning techniques (OpenAI, 2025b).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites OpenAI, 2025b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    OpenAI's o4-mini represents a new generation of compact and efficient LLMs designed for real-time interaction and accessible deployment.

    “models such as OpenAI's o4-mini represent a new generation of compact and efficient LLMs designed for real-time interaction and accessible deployment (OpenAI, 2025a).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites OpenAI, 2025a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    With the assistance of agentic AI, librarians can clarify intent, locate and evaluate resources, provide context, and deliver precise, timely responses to students.

    “Overall, with the assistance of agentic AI, librarians can clarify intent, locate and evaluate resources, provide context, and deliver precise, timely responses to students.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Simulated catalog data used by Agents 3 and 4 adequately represent live library systems for evaluating agentic AI responses.
    • Rests on an assumption: The single prompt example used for RQ1 is representative of school-specific AI-agent-potential queries.
    • Rests on an assumption: The agentic AI system's performance on the selected queries reflects its potential in real academic library settings.
6 of 6

  • The paper relies on this without support.

    The 74 FAQs and anonymized student queries from the UWM library website (2022-2025) are representative of student queries in academic libraries generally.

    “This study analyzed 74 frequently asked questions (FAQs) and anonymized student queries from the University of Wisconsin-Milwaukee (UWM) library website (2022-2025) to explore opportunities for AI integration.”

  • The paper relies on this without support.

    Simulated catalog data used by Agents 3 and 4 adequately represent live library systems for evaluating agentic AI responses.

    “(Note: Agents 3 and 4 currently use simulated data but can be connected to live systems).”

  • The paper relies on this without support.

    OpenAI's o4-mini can reliably semantically analyze and categorize library FAQs and student queries into the three types.

    “Using OpenAI's o4-mini, questions were semantically analyzed and categorized into three types:”

  • The paper relies on this without support.

    The single prompt example used for RQ1 is representative of school-specific AI-agent-potential queries.

    “For RQ1, the study used a prompt drawn from FAQs of the University of Wisconsin-Milwaukee (UWM) library website:”

  • The paper relies on this without support.

    The agentic AI system's performance on the selected queries reflects its potential in real academic library settings.

    “Overall, with the assistance of agentic AI, librarians can clarify intent, locate and evaluate resources, provide context, and deliver precise, timely responses to students.”
5 of 5

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Mogali S.(2014 February).Artificial intelligence and its applications in libraries. InProceedings of the Bilingual International Conference on Information Technology: Yesterday Today and Tomorrow(pp. 1–10). Defence Scientific Information and Documentation Centre Ministry of Defence Delhi.
  2. Shavit Y., 2023, Practices for governing agentic AI systems, Research Paper.
  3. Verma M., 2023, Novel study on AI‐based chatbot (ChatGPT) impacts on the traditional library management, International Journal of Trend in Scientific Research and Development (IJTSRD), 7, 961

Cited by

Sorted by citations

Authors
Wang, N., & Chou, Y.
Affiliations
  • University of Wisconsin–Milwaukee (US)
  • Independent researcher
Corresponding author
Wang, Ning‐Chiao
ORCID
Wang, Ning‐Chiao
Venue
Proceedings of the Association for Information Science and Technology
Publisher
Wiley
Published
2025-10
Volume
62
Issue
1
Pages
1714-1716
ISSN
2373-9231 (ISSN-L)
DOI
10.1002/pra2.1518
OpenAlex
W4415303928
Cited by
4
References
19
Citations by year
2026: 4
FWCI
3.64
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor, 15 d)
Locations
Proceedings of the Association for Information Science and Technology (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in Service Interactions
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Information Systems › Blockchain Technology Applications and Security
Keywords
agentic AI, academic libraries, frequently asked questions, large language models, library services

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review