The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index; 31 of the 51 sources its claims cite were checked for support.
Generating dermatopathology reports from gigapixel whole slide images with HistoGPT
Open accessAbstract
Histopathology is the reference standard for diagnosing the presence and nature of many diseases, including cancer. However, analyzing tissue samples under a microscope and summarizing the findings in a comprehensive pathology report is time-consuming, labor-intensive, and non-standardized. To address this problem, we present HistoGPT, a vision language model that generates pathology reports from a patient’s multiple full-resolution histology images. It is trained on 15,129 whole slide images from 6705 dermatology patients with corresponding pathology reports. The generated reports match the quality of human-written reports for common and homogeneous malignancies, as confirmed by natural language processing metrics and domain expert analysis. We evaluate HistoGPT in an international, multi-center clinical study and show that it can accurately predict tumor subtypes, tumor thickness, and tumor margins in a zero-shot fashion. Our model demonstrates the potential of artificial intelligence to assist pathologists in evaluating, reporting, and understanding routine dermatopathology cases.
Overview
CoChat Summary
## TL;DR
HistoGPT is a vision-language model trained on 15,129 whole-slide images from 6,705 dermatology patients to generate dermatopathology reports from gigapixel WSIs. It produced report quality comparable to human-written reports for common, homogeneous lesions and showed strong zero-shot performance for tasks like tumor thickness, subtype, and margin prediction, but it performed worse on rare or underrepresented conditions.
## Key findings
- On the Munich test set, HistoGPT models captured about ~64% of dermatopathology keywords from original reports; HistoGPT-M-ER captured ~67%, versus ~44% for text-only BioGPT-1B.
- In blinded expert comparison on 100 report pairs, both dermatopathologists found the AI-generated and human reports equally good in ~45% of cases and preferred the AI report in ~15% of cases; for the five largest classes, inter-rater agreement was slight (Cohen’s κ = 0.09).
- For zero-shot tumor thickness prediction on 94 Munich samples, HistoGPT achieved RMSE = 1.8 mm and Pearson ρ = 0.52 (p = 9.7·10−8); slide-level contrastive baselines were worse (e.g., HistoCLIP RMSE = 4.35 mm, ρ = 0.006).
- For zero-shot BCC subtype classification on the Münster-3H cohort, HistoGPT-M achieved weighted F1 = 0.63, outperforming HistoCLIP (0.54) and HistoSigLIP (0.50); for tumor margins on 185 Münster-1K reports, HistoGPT-L reached F1 = 74% for positive margins and 41% for negative margins.
## Methods
- Retrospective, multi-cohort study using fully anonymized dermatopathology WSIs and reports; main training cohort was Munich with 15,129 WSIs from 6,705 patients and a 75/25 train-test split.
- Model: HistoGPT combines a vision encoder (CTransPath for small/medium, UNI for large) with BioGPT via a Perceiver Resampler and interleaved cross-attention; only the cross-attention blocks were trained from scratch, with causal language modeling and optional “Expert Guidance” / “Classifier Guidance.”
- Evaluation combined automatic metrics (keyword overlap/Jaccard, sentence similarity via BioBERT and GPT-3-ADA, BERTScore, BLEU/ROUGE/METEOR) with blinded expert review across internal and external cohorts, including Mayo, Münster, and Radboud.
## Limitations
- The paper states that HistoGPT was trained and tested only on dermatology cases, so it cannot yet be generalized to pan-cancer diagnosis.
- Performance is limited by severe class imbalance and by rare/long-tail classes; the paper explicitly says the model performs worse on inflammatory and minority classes and that rare diseases are harder because the training data are insufficient.
- Evident from the text: several external evaluation issues could affect results, including translation errors from German to English, different reporting styles/templates, and inter-observer variability among pathologists scoring the reports.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
19 Valid · 9 Not valid · 9 Not checked · 137 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“By design, these patch-level approaches ignore up to 99% of the total tissue, miss potentially diagnostically relevant areas, and cannot be applied to tasks that require the full context of the entire tissue sample (e.g., tumor thickness prediction).”
§ Generating dermatopathology reports from gigapixel whole slide images with HistoGPT > Introduction
What the sources say
- Disputedcites [11] Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018). sourceExperimental — accuracy not yet measured
The Results section (“A deep learning framework for the automatic analysis of histopathology images”; page unavailable) says the study divided whole-slide images into tens to thousands of non-overlapping patches per slide, rather than selecting a small fraction of tissue. The Statistical Analysis methods also describe aggregating predictions from the slide’s tiles. The paper does not report that its patch-level approach ignores up to 99% of tissue; its described approach runs contrary to that characterization. (LLM, conf=0.83)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“By design, these patch-level approaches ignore up to 99% of the total tissue, miss potentially diagnostically relevant areas, and cannot be applied to tasks that require the full context of the entire tissue sample (e.g., tumor thickness prediction).”
§ Generating dermatopathology reports from gigapixel whole slide images with HistoGPT > Introduction
What the sources say
- Could not verifycites [11] Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018). sourceExperimental — accuracy not yet measured
The Results section (‘A deep learning framework’; page unavailable) describes dividing whole-slide images into non-overlapping patches and producing tens to thousands of tiles per slide. The Online Methods (‘Tests on independent cohorts’; page unavailable) describe restricting some tests to pathologist-selected regions, but neither passage says patch-level approaches miss diagnostically relevant areas. I searched for missed diagnostic areas and tissue excluded by patch selection. (LLM, conf=0.74)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Contrastive vision language models in pathology, such as PLIP12 and CONCH13, align text and images at the patch-level.”
§ Generating dermatopathology reports from gigapixel whole slide images with HistoGPT > Introduction
What the sources say
- Not checked — no full text of the sourcecites [12] Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J. & Zou, J. A visual–language foundation model for pathology image analysis using medical Twitter. Nat. Med. 29, 2307–2316 (2023). source
- Supportedcites [13] Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nat. Med. 30, 863–874 (2024). source
The Abstract (page unavailable) says CONCH uses contrastive objectives to align images and text. Methods, ‘Supervised and weakly supervised classification experiments’ (page unavailable), says CONCH and PLIP use image representations aligned with text, with tiles serving as the images for whole-slide analysis. This supports patch-level alignment in the sense of histopathology tiles. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“By design, these patch-level approaches ignore up to 99% of the total tissue, miss potentially diagnostically relevant areas, and cannot be applied to tasks that require the full context of the entire tissue sample (e.g., tumor thickness prediction).”
§ Generating dermatopathology reports from gigapixel whole slide images with HistoGPT > Introduction
What the sources say
- Partly supportedcites [11] Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018). sourceExperimental — accuracy not yet measured
The Results section ('A deep learning framework for the automatic analysis of histopathology images'; page not provided) says the network used 512×512-pixel tiles because whole-slide images were too large for direct input. This supports the point that an individual patch lacks whole-sample context, but the paper does not say patch-level approaches cannot be applied to tasks requiring that context, or discuss tumor-thickness prediction. (LLM, conf=0.78)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In addition, our large model uses a three-dimensional factorized position embedding function28 to encode the x- and y-coordinates of each patch, as well as the z-coordinate indicating which slide it belongs to.”
§ Generating dermatopathology reports from gigapixel whole slide images with HistoGPT > Results > HistoGPT integrates vision and language to generate pathology reports
What the sources say
- Partly supportedcites [28] Dehghani, M. et al. Patch n’ Pack: NaViT, a vision transformer for any aspect ratio and resolution. In Proc. Thirty-seventh Conference on Neural Information Processing Systems (2023). sourceExperimental — accuracy not yet measured
In “Factorized & fractional positional embeddings” (page not specified), the source describes separate embeddings for patch x- and y-coordinates. It does not describe a three-dimensional embedding or a z-coordinate identifying a slide; those appear to be the citing authors’ extension of the cited method. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
80 more claims · 6 with their cited sources checked
Create a free account to see all 174 claims and every source verdict.
- Probably not a substantial claim. 35
- It may state two separate claims as one. 64
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“To overcome these limitations, new types of foundation models have recently been proposed for encoding entire WSIs into slide-level general-purpose feature representations57–72.”
From A multimodal whole-slide foundation model for pathology (2025) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“For example, specialist agents such as BrainGPT66 for 3D CT scans, HistoGPT67 for histopathology images, and scChat68 for single-cell sequencing data can be easily integrated into the MoMA workflow when these modalities are involved.”
From MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling (2025) · cites this paper as [67]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 1×
“The latter is a much larger (300 million parameters) Vision Transformer 21 trained at 0.5 mpp resolution on over 100,000 WSIs from 22 major tissue types using self-distillation and masked modeling 22 .”
InfluentialSwin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534 · cited 2×
“The former is a lightweight (30 million parameters) Swin Transformer 15 trained at a resolution of 1.0 micron per pixel (mpp) on over 32,000 WSIs from TCGA 16 and PAIP 19 using a semantically guided contrastive learning algorithm 20 .”
InfluentialLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
BioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 1×
“In particular,wecanassignthetumorsubtypepredictionasatextlabelto each patch and then aggregate them to obtain the prediction for the whole slide.”
Research works printed but not linked: 6
- Spronck, J. et al. nnUNet meets pathology: bridging the gap for application to whole-slide images and computational biomarkers. In Proc. Medical Imaging with Deep Learning Vol. 227 1859–1874 (PMLR, 2023).
- OpenAI et al. GPT-4 technical report. Preprint at arXiv [cs.CL] (2023).
- Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Proceedings of the 35th International Conference on Machine Learning (eds. Dy, J. & Krause, A.) Vol. 80 2127–2136 (PMLR, Stockholmsmässan, Stockholm Sweden, 2018).
- Kossard, S. et al. Basal cell carcinoma. In World Health Organization Classification of Tumours. Pathology & Genetics Skin Tumours (eds LeBoit, P. E. et al.) Vol. Skin Tumours 13–19 (IARC Press, 2006).
- Lin, C.-Y. ROUGE: a package for automatic evaluation of summaries. In Text Summarization Branches Out (eds Moens, M.-F. & Szpakowicz, S.) 74–81 (Association for Computational Linguistics, Barcelona, Spain, 2004).
- Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q. & Artzi, Y. BERTScore: Evaluating Text Generation with BERT. In Proc. International Conference on Learning Representations (2020).
Cited by
A multimodal whole-slide foundation model for pathology
Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03982-3 · Cited by 171
“We excluded all frozen tissue slides, slides without magnification information, metastatic or recurrent tumor slides, slides without tumor tissue and IHC slides.”
InfluentialSelf-citationCocuz, I. G., Niculescu, R., Popelea, M. C., Cocuz, M. E., Sabău, A.-H., Tinca, A. C., Cozac-Szőke, A. R., Chiorean, D. M., Budin, C. E., & Cotoi, O. S. (2025) · Diagnostics · doi:10.3390/diagnostics15172196 · Cited by 11
“[82] HistoGPT (Generative AI) Generating pathology reports/images, aiding education and diagnostics.”
Zhang, Y., Zhang, X., Qi, X., Wu, X., Chen, F., Yang, G., & Fu, H. (2025) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2025.3619086 · Cited by 16
“HistoGPT [67] successfully scales report generation to the whole-slide level, processing multiple gigapixel WSIs to produce narratives of a quality comparable to human experts and exhibiting emergent zero-shot capabilities.”
Foundation models: Insights and implications for gastrointestinal cancer
Shi, L., Huang, R., Zhao, L.-L., & Guo, A.-J. (2025) · World Journal of Gastroenterology · doi:10.3748/wjg.v31.i47.112921 · Cited by 4
Petzold, A., Wessely, A., Schliep, S., Jiang, H., Tran, M., Koch, E. A. T., Peng, T., Starz, H., Berking, C., Marr, C., et al. (2026) · The Journal of Pathology Clinical Research · doi:10.1002/2056-4538.70082 · Cited by 3
“In addition, we benchmarked our data against HistoGPT [14], which, to our knowledge, currently represents the largest dermatopathology-focused foundation model.”
Self-citation
Metadata
- Authors
- Tran, M., Schmidle, P., Guo, R. R., Wagner, S. J., Koch, V., Lupperger, V., Novotny, B., Murphree, D. H., Hardway, H. D., D’Amato, M., Lefkes, J., Geijs, D. J., Feuchtinger, A., Böhner, A., Kaczmarczyk, R., Biedermann, T., Amir, A. L., Mooyaart, A. L., Ciompi, F., Litjens, G., Wang, C., Comfere, N. I., Eyerich, K., Braun, S. A., Marr, C., & Peng, T.
- Affiliations
- Helmholtz Munich (DE)
- University of Freiburg (DE)
- Mayo Clinic (US)
- Munich Leukemia Laboratory (Germany) (DE)
- Radboud University Medical Center (NL)
- Technical University of Munich (DE)
- Erasmus MC (NL)
- University Hospital Münster (DE)
- Corresponding author
- Eyerich, Kilian
- Braun, Stephan A.
- Marr, Carsten
- Peng, Tingying
- ORCID
- Tran, Manuel
- Schmidle, Paul
- Guo, Ruifeng Ray
- Wagner, Sophia J.
- Koch, Valentin
- Lupperger, Valerio
- Novotny, Brenna
- Murphree, Dennis H.
- Hardway, Heather D.
- D’Amato, Marina
- Lefkes, Judith
- Geijs, Daan J.
- Feuchtinger, Annette
- Böhner, Alexander
- Kaczmarczyk, Robert
- Biedermann, Tilo
- Mooyaart, Antien L.
- Ciompi, Francesco
- Litjens, Geert
- Wang, Chen
- Comfere, Nneka I.
- Eyerich, Kilian
- Braun, Stephan A.
- Marr, Carsten
- Peng, Tingying
- Venue
- Nature Communications
- Published
- 2025-05-27
- Volume
- 16
- Issue
- 1
- Pages
- 4886-4886
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC12106639
- OpenAlex
- W4410734748
- Semantic Scholar
- 46b105f364292455106f7f68a8128debd84eebe1
- Cited by
- 36
- References
- 54
- Citations by year
- 2026: 28 · 2025: 8
- FWCI
- 14.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Communications (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, cc-by)
- Radboud Repository (Radboud University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- mediaTUM (Technical University of Munich) (submittedVersion)
- Funders
- European Commission (HORIZON2020, 866411)
- HORIZON EUROPE European Research Council (101113551)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
- Keywords
- dermatopathology, whole slide images, vision-language models, tumor thickness, tumor margin detection, zero-shot learning, AI-assisted pathology, skin cancer diagnosis
- MeSH terms
- Artificial Intelligence, Dermatology, Humans, Image Processing, Computer-Assisted, Natural Language Processing, Skin Neoplasms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generating dermatopathology reports from gigapixel whole slide images with HistoGPT. https://app.cochat.ai/papers/10.1038/s41467-025-60014-x
- The paper
- Tran, M., Schmidle, P., Guo, R. R., Wagner, S. J., Koch, V., Lupperger, V., Novotny, B., Murphree, D. H., Hardway, H. D., D’Amato, M., Lefkes, J., Geijs, D. J., Feuchtinger, A., Böhner, A., Kaczmarczyk, R., Biedermann, T., Amir, A. L., Mooyaart, A. L., Ciompi, F., . . . Peng, T. (2025). Generating dermatopathology reports from gigapixel whole slide images with HistoGPT. Nature Communications. https://doi.org/10.1038/s41467-025-60014-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.