The DOI matches its Crossref record; 42 of the 43 references it cites were found in a registry or an index; 5 of the 16 sources its claims cite were checked for support.
Generative Agents Navigating Digital Libraries
Open accessPublished version of a preprint
Overview
CoChat Summary
TL;DR
Agent4DL is an LLM-based generative user simulator for digital library search, with profile, memory, and interaction modules, validated against real user data from EconBiz and Sowiport/SUSS. It generates realistic query, click, and stopping behavior and augmented data that improves IR ranking tasks, while showing limitations around metadata and scholarly search complexity.
Key findings
- For preference prediction, RoBERTa trained on Agent4DL data (3000 sessions) reached MRR 41.32, nDCG@1 22.66, and nDCG@3 35.19, outperforming BM25 (34.89, 15.12, 26.03) and RoBERTa trained on SUSS (34.15, 13.91, 26.45); even 1000 Agent4DL sessions reached 39.27, 20.37, 33.09.
- For relevance prediction, Agent4DL 3000 reached MRR 37.71, nDCG@1 18.90, and nDCG@3 32.78, compared with BM25 (32.11, 11.65, 24.21) and RoBERTa/SUSS (33.10, 12.36, 25.54).
- Query generation similarity was highest for Agent4DL: term overlap τ = 0.87 (±0.08), BLEU = 0.40 (±0.06), BERTScore = 0.83 (±0.04), above Popular, Random, and Discriminative Selection baselines.
- Click/stopping results: Agent4DL click accuracy 78.35, P@10 41.71, R@10 65.59, F1 50.99; stopping accuracy 84.14, P@10 82.90, R@10 39.89, F1 53.86. Augmenting SUSS with academic traits and research interests improved MRR from 59.12 to 64.37.
Methods
- Agent4DL uses LLM generative agents with three modules: profile (academic traits—depth, breadth, recency bias, interdisciplinarity—and research interests), memory (factual and emotional memories with retrieval, writing, reflection), and interaction (search-driven and emotion-driven actions), using ReAct and Chain-of-Thought prompting.
- Data come from EconBiz (August–November 2020, approximately 420,000 sessions, 15 GB) and Sowiport/SUSS (484,437 sessions, 179,796 queries, April 2014–April 2015); profiles are initialized from benchmark datasets and evaluated by training RoBERTa ranking models (5 epochs, batch size 128, learning rate 5e-6) with MRR, nDCG@1, nDCG@3, accuracy, P@10, R@10, and F1.
- Additional experiments simulate information overload over four rounds and use DirectLiNGAM to learn a causal graph for open access and resource usage.
Limitations
- Stated in the paper: Agent4DL relies heavily on rich metadata that is not uniformly available across digital libraries; its action space omits citation chaining, consultation with colleagues, and serendipitous browsing; and the LLM sometimes struggled with domain-specific or recent knowledge, producing anachronistic queries or misjudging cutting-edge relevance.
- Evident from the text: evaluation is limited to EconBiz and Sowiport/SUSS; Agent4DL does not effectively capture position bias and uses zero-shot learning; stopping behavior has lower recall and F1 than some baselines; and some components depend on ChatGPT/gpt-3.5-turbo-0125 and API-accessible digital libraries.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 9 Not checked · 55 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Simulating user search behaviors has emerged as a promising alternative, offering a controlled and repeatable way to study complex user interactions without the drawbacks of real-world studies [2].”
What the sources say
- Supportedcites [2] Balog, K., Zhai, C.: User simulation for evaluating information access systems. CoRR abs/2306.08550 (2023). https://doi.org/10.48550/ARXIV.2306.08550 source
The Introduction (p. 9) says simulation can enable repeatable, reproducible evaluations at low cost and show how performance changes under different conditions and user behaviours. The Overview (pp. 30–31) contrasts this with the cost and difficulty of reproducing experiments with real users, and says simulated users can be reused in a controlled manner. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When these highly-used items are incorporated into new training datasets, information retrieval systems tend to rank them higher in subsequent iterations, resulting in the open access citation advantage[13].”
What the sources say
- Partly supportedcites [13] He, Y., et al.: CausPref: causal preference learning for out-of-distribution recommendation. In: Laforest, F., et al. (eds.) WWW 2022: The ACM Web Conference 2022, Virtual Event, Lyon, France, 25–29 April 2022, pp. 410–421. ACM (2022). https://doi.org/10.1145/3485447.3511969 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) says popular items may have more chances to be exposed, and §4 EXPERIMENTS says unpopular items receive less exposure. The paper does not establish that incorporating highly used items into later training datasets raises their rankings, or that this produces an open-access citation advantage. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Incorporating user profiles and memory modules into agent systems allows them to display a range of behaviors similar to humans, based on their current observations [33].”
What the sources say
- Partly supportedcites [33] Yao, S., et al.: ReAct: synergizing reasoning and acting in language models. In: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, 1–5 May 2023. OpenReview.net (2023). https://openreview.net/pdf?id=WE_vluYUL-X sourceExperimental — accuracy not yet measured
Section 2 (p. 3) describes an agent acting on observations and reasoning over its current context; the Abstract (p. 1) describes human-like task-solving trajectories. Neither passage supports the claim that incorporating user profiles and memory modules produces those behaviors, and searches for those terms found no such account. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“This design is motivated by the need to monitor emotional states, particularly frustration with complex queries and satisfaction with search results, which play important roles in shaping search experiences and decision-making processes [25].”
What the sources say
- Could not verifycites [25] Park, J.S., O’Brien, J.C., Cai, C.J., Morris, M.R., Liang, P., Bernstein, M.S.: Generative agents: interactive simulacra of human behavior. In: Follmer, S., Han, J., Steimle, J., Riche, N.H. (eds.) Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST 2023, San Francisco, CA, USA, 29 October–1 November 2023, pp. 2:1–2:22. ACM (2023). https://doi.org/10.1145/3586183.3606763 sourceExperimental — accuracy not yet measured
Section 4.1 (Memory and Retrieval; page unavailable) discusses how simulated agents retrieve relevant memories in response to queries, not people's emotions during search. Searches for frustration, satisfaction with search results, and emotional states found no passage addressing their effects on search experiences or decision-making. (LLM, conf=0.76)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, Agent4DL demonstrates competitive performance compared to existing user search simulators such as SimIIR 2.0, particularly in its ability to generate more diverse and context-aware user behaviors.”
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
36 more claims
Create a free account to see all 69 claims and every source verdict.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 20
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4.3 Simulation Consistency · p. 5
“Agent4DL achieves a 87.6% Term Overlap Rate ( 𝜏 ) between LLM-generated and human queries”
§ 4.3 Simulation Consistency · p. 5
“| Agent4DL | 0.87 ( ± 0.08) | 0.40 ( ± 0.06) | 0.83 ( ± 0.04) |”
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 3×
“Popular Selection considers word weights, resulting in a higher BLEU score.”
InfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.18223 · Cited by 2,090
Cognitive neuroscience of emotional memory
LaBar, K. S., & Cabeza, R. (2006) · Nature Reviews Neuroscience · doi:10.1038/nrn1825 · Cited by 1,834 · cited 1×
“This dual approach aligns with cognitive research highlighting the influence of emotions on decision-making and personal history formation [17].”
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · cited 1×
“However, recent advancements have utilized Large Language Models (LLMs) to develop more dynamic and realistic simulations of user behavior in various contexts, such as web search, reasoning, and social interaction [25, 32, 34, 38].”
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Research works printed but not linked: 4
- Pearl, J., Glymour, M., Jewell, N.P.: Causal Inference in Statistics: A Primer. Wiley (2016)
- Ross, S.M.: Simulation. Statistical Modeling and Decision Science, 2nd edn. Academic Press (1997)
- Walsh, J.A., et al.: “The library is open!”: open data and an open API for the HathiTrust Digital Library. In: Sela, A., Jannidis, F., Romanowska, I. (eds.) Proceedings of the Computational Humanities Research Conference 2023, Paris, France, 6–8 December 2023. CEUR Workshop Proceedings, vol. 3558, pp. 703–714. CEUR-WS.org (2023). https://ceur-ws.org/Vol-3558/paper7875.pdf
- Zerhoudi, S., Granitzer, M., Seifert, C., Schlötterer, J.: Simulating user interaction and search behaviour in digital libraries. In: Nunzio, G.M.D., Portelli, B., Redavid, D., Silvello, G. (eds.) Proceedings of the 18th Italian Research Conference on Digital Libraries, Padua, Italy, 24–25 February 2022 (Hybrid Event). CEUR Workshop Proceedings, vol. 3160. CEUR-WS.org (2022). https://ceur-ws.org/Vol-3160/paper8.pdf
Cited by
Simulation of Interactive Information Retrieval: A Guided Tour
Zerhoudi, S., Roegiest, A., & Trippas, J. R. (2026) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3786304.3787892 · Cited by 4
“In addition, with the recent rise of large language models (LLMs) and generative models has introduced a new, powerful, but complex class of simulation techniques [4, 8, 10, 12].”
Self-citation
Metadata
- Authors
- Zerhoudi, S., & Granitzer, M.
- Affiliations
- University of Passau (DE)
- Corresponding author
- Zerhoudi, Saber
- Venue
- Lecture notes in computer science
- Publisher
- Springer Nature Singapore
- Published
- 2024-12-06
- Pages
- 171-188
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W4405025781
- Cited by
- 1
- References
- 34
- Citations by year
- 2026: 1
- FWCI
- 0.38
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- cc-by
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Library and Information Sciences › Library Science and Information
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Keywords
- digital libraries, large language models, generative AI agents, user profiles, search sessions, user interaction modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative Agents Navigating Digital Libraries. https://app.cochat.ai/papers/10.1007/978-981-96-0865-2_14
- The paper
- Zerhoudi, S., & Granitzer, M. (2025). Generative Agents Navigating Digital Libraries. Lecture notes in computer science. https://doi.org/10.1007/978-981-96-0865-2_14
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.