The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Describing archival photographs using multimodal LLMs: a case study on evaluating vision-language model performance for creating descriptive metadata
Abstract
Purpose: This paper aims to examine the efforts of librarians in Northeastern University Library’s Digital Production Services department to employ Gemini, a pre-trained multimodal large language model, for generating descriptive metadata for an archival photographic collection. Design/methodology/approach: The project comprised three phases: 1) researching and selecting a multimodal LLM that was both accurate and cost-effective for generating descriptive metadata for photographs; 2) developing an application to batch-submit photographs to the chosen model’s API and convert the results into a human-readable spreadsheet and 3) evaluating the output for completeness, accuracy, consistency and potential bias. Findings: Insights from model research guided the development and deployment of an application that queried the Gemini model to generate titles, abstracts and other key metadata. Iterative testing showed that although accuracy, completeness and consistency were imperfect, the output quality demonstrated strong potential for future implementation. Analysis of application results indicates that ongoing errors and biases may be reduced through strategic prompt engineering and systematic quality control measures. These testing methodologies will inform future efforts to operationalize computer vision workflows for processing archival photographs. Originality/value: While computer vision holds significant potential to improve cataloguing workflows for archival photograph collections, few case studies have explored methods for evaluating and operationalizing these workflows to produce suitable metadata records. This paper presents an evaluation process to assess completeness, accuracy, consistency and potential bias in the generated metadata. Crucially, this workflow is adaptable and can be repeated as the application, prompts and models evolve, ensuring ongoing reliability and improvement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Comprehensive Survey of Deep Learning for Image Captioning
Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H. (2019) · ACM Computing Surveys · doi:10.1145/3295748 · Cited by 879
Chow, E. H. C., Kao, T. J., & Li, X. (2024) · Cataloging & Classification Quarterly · doi:10.1080/01639374.2024.2394516 · Cited by 31
Research works printed but not linked: 21
- Adewojo, 2024, Exploring ethical considerations in AI-driven cataloguing and classification with ChatGPT, Business Information Review, 31, 130, 10.1177/02663821241264703 · DOI
- Al-Shamayleh, 2024, A comprehensive literature review on image captioning methods and metrics based on deep learning technique, Multimedia Tools and Applications, 83, 34219, 10.1007/s11042-024-18307-8 · DOI
- Bushey, 2024, Envisioning archival images with artificial intelligence, Archeion, 2024, 33, 10.4467/26581264arc.24.007.20202 · DOI
- deBoer, 2024, Marriott Reparative Metadata Assessment Tool (MaRMAT)
- Fornaro, 2021, Artificial intelligence for content and context metadata retrieval in photographs and image groups, Journal of Digital Media Management, 9, 297, 10.69554/lgqn4645 · DOI
- Gavrilis, 2015, Research and Advanced Technology for Digital Libraries (TPDL ’15), (Series Lecture Notes in Computer Science, 56, 10.1007/978-3-319-24592-8_5 · DOI
- Ghandi, 2023, Deep learning approaches on image captioning: a review, ACM Computing Surveys, 56, 1, 10.1145/3617592 · DOI
- Ghosh, 2020, Emerging Technology in Modelling and Graphics: Advances in Intelligent Systems and Computing, 171, 10.1007/978-981-13-7403-6_17 · DOI
- Greene, 2005, More product, less process: revamping traditional archival processing, The American Archivist, 68, 208, 10.17723/aarc.68.2.c741823776k65863 · DOI
- Hastings, 2025, AI chatbots and subject cataloging: a performance test, Library Resources and Technical Services, 69, 10.5860/lrts.69n2.8440 · DOI
- Kaldeli, 2021, CrowdHeritage: crowdsourcing for improving the quality of cultural heritage metadata, Information, 12, 64, 10.3390/info12020064 · DOI
- Kumpulainen, 2022, Struggling with digitized historical newspapers: Contextual barriers to information interaction in history research activities, Journal of the Association for Information Science and Technology, 73, 1012, 10.1002/asi.24608 · DOI
- Late, 2024, Image searching in open photograph archive: search tactics and faced barriers in historical research, International Journal on Digital Libraries, 25, 715, 10.1007/s00799-023-00390-1 · DOI
- Luke, 2024, Instituting a framework for reparative description, Archival Science, 24, 481, 10.1007/s10502-024-09435-z · DOI
- McCann, 2017, The whole story: news agency photographs in newspaper photo morgue collections, The American Archivist, 80, 163, 10.17723/0360-9081.80.1.163 · DOI
- Ma, 2024, Proceedings of the 38th AAAI Conference on Artificial Intelligence and 36th Conference on Innovative Applications of Artificial Intelligence and 14th Symposium on Educational Advances in Artificial Intelligence, 4089, 10.1609/aaai.v38i5.28203 · DOI
- Magnus, 2025, Metadata creation and enrichment using artificial intelligence at Meemoo, Journal of Digital Media Management, 13, 110, 10.69554/ngff5280 · DOI
- Mahmud, 2024, AI in automating library cataloging and classification, Library Hi Tech News,, 10.1108/lhtn-07-2024-0114 · DOI
- Rovner, 2024, Riot or Uprising? A Reflection on Race and Language in the Contested City
- Teper, 2004, Newspaper photo morgues – a survey of institutional holdings and practices, Library Collections, Acquisitions, and Technical Services, 28, 106, 10.1080/14649055.2004.10765978 · DOI
- Torabi, 2025, From captions to catalogues: a systematic review of GenAI models for image captioning and their role in image cataloguing, Cataloging and Classification Quarterly, 63, 436, 10.1080/01639374.2025.2539792 · DOI
Cited by
Sorted by citations
Research on Large Language Model-Based Bibliographic Cataloging Agent in the CNMARC Context
Tan, Z., Yang, X., Chen, Q., & Chen, T. (2026) · Publications · doi:10.3390/publications14010019 · Cited by 2
“Approximately 33,000 CNMARC bibliographic records in ISO 2709 (ISO, 2008) for‑ mat were obtained from a University Library, fully covering all 26 top‑level categories of the Chinese Library Classification (CLC).”
Influential
Metadata
- Authors
- Facklam, D., Sweeney, S. J., Majumdar, S., & Dmello Kamath, R. N.
- Affiliations
- Northeastern University (US)
- Northeastern University – Boston Campus Digital Production Services, , Boston, ,
- Corresponding author
- Sweeney, Sarah J.
- Venue
- The Electronic Library
- Publisher
- Emerald
- Published
- 2025-12-22
- Volume
- 44
- Issue
- 5
- Pages
- 919-942
- ISSN
- 0264-0473 (ISSN-L)
- 1758-616X
- OpenAlex
- W4417496510
- Cited by
- 5
- References
- 2
- Citations by year
- 2026: 5
- FWCI
- 5.24
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Electronic Library (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Library Science and Information Systems
- Social Sciences › Arts and Humanities › Conservation › Digital and Traditional Archives Management
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- historical photographs, descriptive metadata, multimodal large language models, Gemini, vision-language models, automated cataloguing, metadata evaluation, prompt engineering
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
