Comprehensive testing of large language models for extraction of structured data in pathology
Open accessAbstract
Background: Pathology departments generate large volumes of unstructured data as free-text diagnostic reports. Converting these reports into structured formats for analytics or artificial intelligence projects requires substantial manual effort by specialized personnel. While recent studies show promise in using advanced language models for structuring pathology data, they primarily rely on proprietary models, raising cost and privacy concerns. Additionally, important aspects such as prompt engineering and model quantization for deployment on consumer-grade hardware remain unaddressed. Methods: We created a dataset of 579 annotated pathology reports in German and English versions. Six language models (proprietary: GPT-4; open-source: Llama2 13B, Llama2 70B, Llama3 8B, Llama3 70B, and Qwen2.5 7B) were evaluated for their ability to extract eleven key parameters from these reports. Additionally, we investigated model performance across different prompt engineering strategies and model quantization techniques to assess practical deployment scenarios. Results: Here we show that open-source language models extract structured data from pathology reports with high precision, matching the accuracy of proprietary GPT-4 model. The precision varies significantly across different models and configurations. These variations depend on specific prompt engineering strategies and quantization methods used during model deployment. Conclusions: Open-source language models demonstrate comparable performance to proprietary solutions in structuring pathology report data. This finding has significant implications for healthcare institutions seeking cost-effective, privacy-preserving data structuring solutions. The variations in model performance across different configurations provide valuable insights for practical deployment in pathology departments. Our publicly available bilingual dataset serves as both a benchmark and a resource for future research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Traditional machine learning methods such as SVM, RF, and XGBoost perform well on structured clinical data and small to medium-sized datasets.
“Traditional ML methods, such as SVM, RF, and XGBoost, perform well on structured clinical data and small to medium-sized datasets [163].”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [163]
The Discussion describes SVMs and gradient boosting used to extract structured data from free-text pathology reports, reporting lower accuracy than LLMs for that task. It does not assess how SVM, RF, or XGBoost perform on already structured clinical data or on small to medium-sized datasets (searched for those terms and conditions). (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0252-y · Cited by 1,698 · see citing works
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045 · see citing works
Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial
Meskó, B. (2023) · Journal of Medical Internet Research · doi:10.2196/50638 · Cited by 709 · see citing works
Deep learning in cancer pathology: a new generation of clinical biomarkers
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. (2021) · British Journal of Cancer · doi:10.1038/s41416-020-01122-x · Cited by 667 · see citing works
Artificial intelligence in diagnostic pathology
Shafi, S., & Parwani, A. V. (2023) · Diagnostic Pathology · doi:10.1186/s13000-023-01375-z · Cited by 350 · see citing works
High-accuracy prostate cancer pathology using deep learning
Tolkach, Y., Dohmgörgen, T., Toma, M. I., & Kristiansen, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0200-7 · Cited by 183 · see citing works
Self-citationChoi, H. S., Song, J. Y., Shin, K. H., Chang, J. H., & Jang, B. (2023) · Radiation Oncology Journal · doi:10.3857/roj.2023.00633 · Cited by 133 · see citing works
Truhn, D., Loeffler, C. M., Müller‐Franzes, G., Nebelung, S., Hewitt, K. J., Brandner, S., Bressem, K. K., Foersch, S., & Kather, J. N. (2024) · The Journal of Pathology · doi:10.1002/path.6232 · Cited by 124 · see citing works
Le Guellec, B., Lefèvre, A., Geay, C., Shorten, L., Bruge, C., Hacein-Bey, L., Amouyel, P., Pruvo, J.-P., Kuchcinski, G., & Hamroun, A. (2024) · Radiology: Artificial Intelligence · doi:10.1148/ryai.230364 · Cited by 115 · see citing works
Effects of text length on lexical diversity measures: Using short texts with less than 200 tokens
Koizumi, R., & In’nami, Y. (2012) · System · doi:10.1016/j.system.2012.10.012 · Cited by 92 · see citing works
Sushil, M., Zack, T. I., Mandair, D., Zheng, Z., Wali, A., Yu, Y.-N., Quan, Y., Lituiev, D., & Butte, A. J. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae146 · Cited by 62 · see citing works
medBERT.de: A comprehensive German BERT model for the medical domain
Bressem, K. K., Papaioannou, J.-M., Grundmann, P., Borchert, F., Adams, L. C., Liu, L., Busch, F., Xu, L., Loyen, J. P., Niehues, S. M., et al. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121598 · Cited by 60 · see citing works
Mandelker, D. L., Marra, A., Mehta, N. N., Selenica, P., Yelskaya, Z., Yang, C., Somar, J., Mehine, M. M., Misyura, M., Baştürk, O., et al. (2023) · npj Precision Oncology · doi:10.1038/s41698-022-00342-z · Cited by 53 · see citing works
Chain-of-Verification Reduces Hallucination in Large Language Models
Dhuliawala, S., Komeili, M., Xu, J., Raileanu, R., Li, X., Celikyilmaz, A., & Weston, J. (2023) · arXiv · doi:10.48550/arxiv.2309.11495 · Cited by 42 · see citing works
Organizational preparedness for the use of large language models in pathology informatics
Hart, S. N., Hoffman, N. G., Gershkovich, P., Christenson, C., McClintock, D. S., Miller, L. J., Jackups, R., Azimi, V., Spies, N. C., & Brodsky, V. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100338 · Cited by 36 · see citing works
Swillens, J. E. M., Sluijter, C. E., Overbeek, L. I. H., Nagtegaal, I. D., & Hermens, R. (2019) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-019-02609-6 · Cited by 34 · see citing works
Measuring Lexical Diversity in Texts: The Twofold Length Problem
Bestgen, Y. (2024) · Language Learning · doi:10.1111/lang.12630 · Cited by 30 · see citing works
Reichenpfader, D., Müller, H., & Denecke, K. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-076865 · Cited by 27 · see citing works
Machine Learning Approaches for Extracting Stage from Pathology Reports in Prostate Cancer
Lenain, R., Seneviratne, M. G., Bozkurt, S., Blayney, D. W., Brooks, J. D., & Hernandez‐Boussard, T. (2019) · Studies in health technology and informatics · doi:10.3233/shti190515 · Cited by 15 · see citing works
Lee, D. T., Vaid, A., Menon, K. M., Freeman, R., Matteson, D. S., Marin, M. P., & Nadkarni, G. N. (2023) · medRxiv · doi:10.1101/2023.11.08.23298252 · Cited by 15 · see citing works
Sushil, M., Zack, T., Mandair, D., Zheng, Z., Wali, A., Yu, Y.-N., Quan, Y., & Butte, A. (2024) · Research Square · doi:10.21203/rs.3.rs-3914899/v1 · Cited by 10 · see citing works
Lu, Y., Srinivasan, G., Preum, S. M., Pettus, J. R., Davis, M., Greenburg, J., Vaickus, L. J., & Levy, J. (2023) · medRxiv · doi:10.1101/2023.09.10.23295318 · Cited by 5 · see citing works
Large Language Models for Extraction of Structured Data in Pathology
Grothey, B., & Tolkach, Y. (2024) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.13685448 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 7
- [6]Tolkach, Y. et al. An international multi-institutional validation study of the algorithm for prostate cancer detection and Gleason grading. npj Precis. Oncol. 7, 1–9 (2023). 37582946 PMC10427608
- [20]Deng, Y., Zhang, W., Chen, Z. & Gu, Q. Rephrase and respond: let large language models ask better questions for themselves. ArXiv:2311.04205 (2024). · arXiv
- [21]Touvron, H., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288 (2023). · arXiv
- [24]Yang, A. et al. Qwen2 technical report. arXiv:2407.10671 (2024). · arXiv
- [25]Jiang, A. Q. et al. Mistral 7B. arXiv:2310.06825 (2023). · arXiv
- [26]Almazrouei, E. et al. The Falcon series of open language models. arXiv:2311.16867 (2023). · arXiv
- [28]Sahoo, P. et al. A systematic survey of prompt engineering in large language models: techniques and applications. arXiv:2402.07927 (2024). · arXiv
Cited by
Sorted by citations
Sawma, T., Arghami, A., Schaff, H. V., Aslahishahri, M., Mangold, K. E., Dearani, J. A., Stulak, J. M., Bagameri, G., Villavicencio, M. A., Greason, K. L., et al. (2025) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2025.06.037 · Cited by 9 · see citing works
Jayasrilakshmi, S., & Mahapatra, A. (2026) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2026.111469 · Cited by 12 · see citing works
Li, L., Mathrani, A., & Sušnjak, T. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2025.10066 · Cited by 6 · see citing works
Xie, X., Wu, C., Huang, Z., Zhou, Y., Huang, J., Dao, F., Yan, D., Deng, K., Lyu, H., Ma, C., et al. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag083 · Cited by 4 · see citing works
“Traditional ML methods, such as SVM, RF , and XGBoost, perform well on structured clinical data and small to medium-sized datasets [163].”
Jo, E., Noh, T. I., & Joo, H. J. (2025) · Diagnostics · doi:10.3390/diagnostics15192474 · Cited by 4 · see citing works
Liu, M., Li, Y., Zhu, J., Yang, H., Huang, Y., Li, H., Chen, Y., Bai, X., Liao, Y., Qu, H., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05997-0_11 · Cited by 2 · see citing works
Vela, V., Sonay, A. Y., Limani, P., Graf, L., Sabani, B., Gjermeni, D., Rroku, A., Zela, A., Gorica, E., Biefer, H. R. C., et al. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14217555 · Cited by 4 · see citing works
Sheng, W., Wang, Y., Xiao, L., Guo, B., Zhang, Y., Qiao, L., Li, Q., Yang, L., & Zhang, Y. (2025) · European Radiology · doi:10.1007/s00330-025-12147-2 · Cited by 2 · see citing works
Carou‐Senra, P., Rodríguez‐Pombo, L., Alvarez‐Lorenzo, C., & Goyanes, Á. (2026) · International Journal of Pharmaceutics · doi:10.1016/j.ijpharm.2026.126587 · Cited by 2 · see citing works
The role of artificial intelligence in cancer epidemiology: Challenges and opportunities
Zhao, S., Yu, P., Xu, R., & Li, S. (2026) · Intelligent Oncology · doi:10.1016/j.intonc.2026.100070 · Cited by 2 · see citing works
MedPAO: A Protocol-Driven Agent for Structuring Medical Reports
Vaidya, S. S., Palani, G., Ramesh, S., Balasubramanian, V., Selvam, M., Srinivasaraja, G., & Krishnamurthi, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-06004-4_4 · Cited by 1 · see citing works
Fan, Z., Liang, R., Sun, C., Pan, J., Terry, R. S., & Xu, J. (2026) · JMIR Medical Informatics · doi:10.2196/84396 · Cited by 1 · see citing works
Computational pathology in bladder cancer: A scoping review
Superdock, M., Wobker, S. E., Carmicheal, I., & Kim, W. Y. (2026) · Bladder Cancer · doi:10.1177/23523735251413333 · Cited by 2 · see citing works
Chuan, A., & Nanda, M. (2026) · Anesthesiology · doi:10.1097/aln.0000000000005933 · Cited by 1 · see citing works
Chin, K. Y., Fujii, S., Ishida, S., & Terayama, K. (2026) · Scientific Reports · doi:10.1038/s41598-026-47277-0 · Cited by 1 · see citing works
Chow, N.-H., Chang, H.-C., Chen, H.-K., Lin, C.-Y., Liu, Y.-L., Tseng, P., Shiu, L.-J., Chu, Y.-W., Chung, P., & Chang, K.-P. (2026) · Diagnostics · doi:10.3390/diagnostics16111644 · Cited by 1 · see citing works
Malyala, R., Namekawa, T., Black, A. J., Gleave, M., & Mannas, M. P. (2026) · Urologic Oncology Seminars and Original Investigations · doi:10.1016/j.urolonc.2026.05.029 · Cited by 1 · see citing works
Abukhiran, I. M., Mansour, A., Caicedo, M., Minkowitz, J. M., & Pantanowitz, L. (2026) · Modern Pathology · doi:10.1016/j.modpat.2026.101049 · Cited by 1 · see citing works
Corso, F., Zec, A., Favali, M., Ligero, M., Hilgers, L., Invernizzi, L. M., Mazzeo, L., Morales, A. M., Müller‐Franzes, G. A., Machado, I. P., et al. (2026) · Cell Reports Medicine · doi:10.1016/j.xcrm.2026.102966 · Cited by 2 · see citing works
The consensus-based CINEX guideline for reporting clinical information extraction studies
Reichenpfader, D., Zaghir, J., Cécilia‐Joseph, E., Delange, B., Shaitarova, A., Sylvestre, E., Dennstädt, F., Zhang, B., Teodoro, D., Meystre, S. M., et al. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag136 · Cited by 2 · see citing works
Data Extraction From Oncology Imaging Reports by Large Language Models: A Comparative Accuracy Study
Passweg, L. P., Schwenke, J. M., Schönenberger, C., Locher, F., Picker, J., Dieterle, M., Thiele, B., Hasler, D., Danelli, A., Schmitt, A. M., et al. (2026) · JCO Clinical Cancer Informatics · doi:10.1200/cci-26-00002 · Cited by 1 · see citing works
Corso, F., Peppoloni, V., Mazzeo, L., Leone, G., Passos, L., Miskovic, V., Armanini, J., Ferrarin, A., Wiest, I. C., Wolf, F., et al. (2025) · medRxiv · doi:10.1101/2025.10.16.25337917 · Cited by 1 · see citing works
AI Streamlines Prostate Pathology Data Extraction
Basourakos, S. P., & Shoag, J. E. (2025) · NEJM AI · doi:10.1056/aie2500388 · Cited by 1 · see citing works
Pohyer, V., de Margerie‐Mellon, C., Perronne, L., Duron, L., Thibault, C., Oudard, S. M., Fournier, L. S., & Rance, B. (2025) · Research Square · doi:10.21203/rs.3.rs-7630430/v1
Chin, K. Y., Fujii, S., Ishida, S., & Terayama, K. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.10.29.685285
Clinical reasoning from real-world oncology reports using large language models
Park, J. H., Kim, S., & Heo, J. (2025) · Digital Health · doi:10.1177/20552076251394622
Maarseveen, T. D., Selani, D., Steinz, N., ten Brinck, R. M., Glas, H. K., Dieren, J. V.-V., Reinders, M. J. T., van den Akker, E. B., & Knevel, R. (2025) · Annals of the Rheumatic Diseases · doi:10.1016/j.ard.2025.11.001 · Cited by 1 · see citing works
Leveraging Large Language Models to Extract Prognostic Pathology Features in Ewing SarcomaPreprint
Huang, J., Batool, A., Gu, Z., Zhao, Z., Yao, B., Black, J. O., Davis, J. L., Al‐Ibraheemi, A., DuBois, S., Barkauskas, D. A., et al. (2026) · SSRN Electronic Journal · doi:10.64898/2026.02.20.707103
Lee, J. H., Ryu, B., & Kim, Y. K. (2026) · Research Square · doi:10.21203/rs.3.rs-8719534/v1
Abdulai, H., Manresa-Yee, C., Rexha, H., & Lafond, S. (2026) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ae5442
TaRA: Token-Aware Recalibration and Attention for Explainable Pathology Reports Classification
Mohamed, W., Abdelhalim, I., Soliman, T. H. A., AbdelAziz, A., Ali, A., El-Melegy, M., Ghazal, M., & El-Baz, A. S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11460938
Rao, A., Bali, A., Tran, V. V., Vasavada, S., Chu, J., Kanwal, F., Rusin, C., & El‐Serag, H. B. (2026) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2026.05.017
Tong, Q., Zhou, L., Liu, X., Liu, X.-Q., Wu, N.-F., Wang, Y., Luo, H.-Y., Yao, B., Tian, J., Han, D.-F., et al. (2026) · aBIOTECH · doi:10.1016/j.abiote.2026.100059
Ntinopoulos, V., Biefer, H. R. C., Rings, L., Rosalia, R. A., & Dzemali, O. (2026) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2025-101956
Benchmarking large language models for cell-free RNA diagnostic biomarker discovery
Gaudio, H., Bliss, A., Loy, C., Eweis-LaBolle, D., Gardella, A. E., & Vlaminck, I. D. (2026) · Nature Communications · doi:10.1038/s41467-026-74077-x
Renner, E., Patel, O., McOsker, S., Le, M., Lu, Y., Vaickus, L. J., Berg, A. H., Yao, K., & Levy, J. (2026) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2026.100687
Verdonschot, K. H. M., Vissers, P. A. J., van den Boezem, P. B., Overduin, C. G., Fütterer, J. J., de Wilt, J. H. W., Nagtegaal, I. D., & Stommel, M. W. J. (2026) · European Journal of Surgical Oncology · doi:10.1016/j.ejso.2026.111964
Kiermeyer, N., Lenfers, T., Dada, A., Friedrich, J., Khattab, S., Knop, E., Egger, J., Pauly, M., Jung, A., Montavon, G., et al. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001546
Neuro-Symbolic AI for Automated Pathology Quality MeasurementPreprint
Brann, F., Tadele, L., Skau, C., Bocsi, G. T., & Clarke, A. K. (2026) · medRxiv · doi:10.64898/2026.07.22.26358635
Kalkhoran, H. A., Martinot, T., Schonewille, L. H. N., Hüyük, M., Huigen, M. H. J., Guchelaar, H., Cohen, D., Smit, E. F., & Zwaveling, J. (2026) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00383
Govind, A., & Migerode, C. V. (2026) · Environment and Planning B Urban Analytics and City Science · doi:10.1177/23998083261471948
Exploring Cognitive Load Markers in Construction Fatality Reports: An LLM-Based Approach
Adjei, E. K., Jin, Z.-Y., & Gambatese, J. A. (2026) · Construction Research Congress 2026 · doi:10.1061/9780784486993.067
Posta, M., Figler, A., Dobolyi, Z., & Győrffy, B. (2026) · Journal of Medical Internet Research · doi:10.2196/95780
Standardizing longitudinal chest X-ray report evaluation via large language model annotation
Wang, X., Figueredo, G., Li, R., & Chen, X. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.134237
Ashrafi, A. F., Alhajj, R., & Rokne, J. G. (2026) · Applied Sciences · doi:10.3390/app16188962
Agentic AI workflows in anatomic pathology: Opportunities and challenges
Ngo, A., Tian, R., Mohammad, A., Weon, J. L., Arvisais‐Anhalt, S., Minor, J., & Youens, K. (2026) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2026.100718
From Cloud to Pocket: Frontier Edge Foundation Models and the Future of Clinical Informatics
Shankar, R. (2026) · Applied Clinical Informatics · doi:10.1055/a-2969-1166
Henn, J., Stoll, A., Feodorovici, P., Subramani, D., Röttgen, J., Arensmeyer, J. C., Gräff, I., Wulff, B., Kalff, J. C., & Matthaei, H. (2026) · Scientific Reports · doi:10.1038/s41598-026-73738-7
Koch, N., Hartmann, N., Männel, A., Baretton, G. B., Aust, D., & Hönscheid, P. (2026) · medRxiv · doi:10.64898/2026.10.07.26364402
Fine-Tuning of Small LLMs for Medical Data Extraction
Raykova, A., Tomov, L., & Markov, E. (2026) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-032-33123-6_15
Karlović, R., Sever, L. E., Miličević, M., Šegota, S. B., Mrzljak, V., & Lorencin, I. (2026) · Information · doi:10.3390/info17101020
Lee, S. Y. (2026) · Blood Research · doi:10.1007/s44313-026-00177-8
ThyroNet A Hybrid Deep Learning Model for Early Prediction of Recurrence in Thyroid Cancer PatientsPreprint
Shrotriya, N., & Janu, N. (2026) · Research Square · doi:10.21203/rs.3.rs-10983504/v1
- Authors
- Grothey, B., Odenkirchen, J., Brkic, A., Schömig-Markiefka, B., Quaas, A., Büttner, R., & Tolkach, Y.
- Affiliations
- University Hospital Cologne (DE)
- University of Cologne (DE)
- Corresponding author
- Grothey, Bastian
- Tolkach, Yuri
- Venue
- Communications Medicine
- Published
- 2025-03-31
- Volume
- 5
- Issue
- 1
- Pages
- 96-96
- ISSN
- 2730-664X (ISSN-L)
- PMC
- PMC11958830
- OpenAlex
- W4409011223
- Cited by
- 54
- References
- 25
- Citations by year
- 2026: 42 · 2025: 12
- FWCI
- 21.66
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Communications Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Kölner Universitäts PublikationsServer (Universität zu Köln) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Wilhelm Sander-Stiftung (Grand 2022.040.1)
- Bundesministerium für Bildung und Forschung (Project FED-PATH)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- large language models, pathology reports, structured data extraction, open-source language models, prompt engineering, model quantization, GPT-4 comparison, Llama 3, Qwen2.5, healthcare data privacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
