CoChat

Handcrafted Histological Transformer (H2T): Unsupervised representation of whole slide images

Open access
Vu, Quoc Dang, Rajpoot, Kashif, Raza, Shan E. Ahmed, Rajpoot, Nasir · 2023 · Medical Image Analysis
Cited by 55

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Abstract

Diagnostic, prognostic and therapeutic decision-making of cancer in pathology clinics can now be carried out based on analysis of multi-gigapixel tissue images, also known as whole-slide images (WSIs). Recently, deep convolutional neural networks (CNNs) have been proposed to derive unsupervised WSI representations; these are attractive as they rely less on expert annotation which is cumbersome. However, a major trade-off is that higher predictive power generally comes at the cost of interpretability, posing a challenge to their clinical use where transparency in decision-making is generally expected. To address this challenge, we present a handcrafted framework based on deep CNN for constructing holistic WSI-level representations. Building on recent findings about the internal working of the Transformer in the domain of natural language processing, we break down its processes and handcraft them into a more transparent framework that we term as the Handcrafted Histological Transformer or H2T. Based on our experiments involving various datasets consisting of a total of 10,042 WSIs, the results demonstrate that H2T based holistic WSI-level representations offer competitive performance compared to recent state-of-the-art methods and can be readily utilized for various downstream analysis tasks. Finally, our results demonstrate that the H2T framework can be up to 14 times faster than the Transformer models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.media.2023.102743

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 19 references

Research works printed but not linked: 33

  1. Abbet, C., Zlobec, I., Bozorgtabar, B., Thiran, J.P., 2020. Divide-and-Rule: Self-Supervised Learning for Survival Analysis in Colorectal Cancer. In: MICCAI. · DOI
  2. Abels, 2019, Computational pathology definitions, best practices, and recommendations for regulatory guidance: A white paper from the digital pathology association, J. Pathol., 249, 10.1002/path.5331 · DOI · Paper page
  3. Andrews, S., Tsochantaridis, I., Hofmann, T., 2002. Support Vector Machines for Multiple-Instance Learning. In: NIPS, Vol. 2, No. 3. p. 7.
  4. Awan, 2017, Glandular morphometrics for objective grading of colorectal adenocarcinoma histology images, Sci. Rep., 7, 16852, 10.1038/s41598-017-16516-w · DOI · Paper page
  5. Azam, 2021, Diagnostic concordance and discordance in digital pathology: a systematic review and meta-analysis, J. Clin. Pathol., 74, 448, 10.1136/jclinpath-2020-206764 · DOI · Paper page
  6. Cao, 2016, Clinical impacts of a micropapillary pattern in lung adenocarcinoma: a review, OncoTargets Ther., 9, 149
  7. Chen, 2022, Deep learning for instance retrieval: A survey, IEEE Trans. Pattern Anal. Mach. Intell., 1
  8. Diao, 2021, Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes, Nature Commun., 12, 1613, 10.1038/s41467-021-21896-9 · DOI · Paper page
  9. Doersch, C., Zisserman, A., 2017. Multi-task self-supervised visual learning. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2051–2060. · DOI · Paper page
  10. Elston, 1991, Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up, Histopathology, 19, 403, 10.1111/j.1365-2559.1991.tb00229.x · DOI · Paper page
  11. Gentles, 2015, The prognostic landscape of genes and infiltrating immune cells across human cancers, Nature Med., 21, 10.1038/nm.3909 · DOI · Paper page
  12. Goldstraw, 2016, The IASLC lung cancer staging project: Proposals for revision of the TNM stage groupings in the forthcoming (eighth) edition of the TNM classification for lung cancer, J. Thorac. Oncol., 11, 39, 10.1016/j.jtho.2015.09.009 · DOI · Paper page
  13. Graham, 2020, Dense steerable filter cnns for exploiting rotational symmetry in histology images, IEEE Trans. Med. Imaging, 39, 4124, 10.1109/TMI.2020.3013246 · DOI
  14. Graham, 2019, Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images, Med. Image Anal., 10.1016/j.media.2019.101563 · DOI · Paper page
  15. Hamilton, 1994, Expert system support using Bayesian belief networks in the diagnosis of fine needle aspiration biopsy specimens of the breast, J. Clin. Pathol., 47, 329, 10.1136/jcp.47.4.329 · DOI
  16. Kandemir, 2015, Computer-aided diagnosis from weak supervision: A benchmarking study, Comput. Med. Imaging Graph., 42, 44, 10.1016/j.compmedimag.2014.11.010 · DOI
  17. Keenan, 2000, An automated machine vision system for the histological grading of cervical intraepithelial neoplasia (CIN), J. Pathol., 192, 351, 10.1002/1096-9896(2000)9999:9999<::AID-PATH708>3.0.CO;2-I · doi:10.1002/1096-9896(2000)9999:9999<::aid-path708>3.0.co;2-i
  18. Kim, D., Cho, D., Kweon, I.S., 2019. Self-supervised video representation learning with space-time cubic puzzles. In: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 01. pp. 8545–8552. · DOI · Paper page
  19. Koohbanani, 2021, Self-path: Self-supervision for classification of pathology images with limited annotations, IEEE Trans. Med. Imaging, 40, 2845, 10.1109/TMI.2021.3056023 · DOI · Paper page
  20. Kromp, 2020, An annotated fluorescence image dataset for training nuclear segmentation methods, Sci. Data, 7, 1, 10.1038/s41597-020-00608-w · DOI
  21. Kwak, 2017, Multiview boosting digital pathology analysis of prostate cancer, Comput. Methods Programs Biomed., 142, 91, 10.1016/j.cmpb.2017.02.023 · DOI
  22. Li, 2020, Deep learning methods for lung cancer segmentation in whole-slide histopathology images-the acdc@ lunghp challenge 2019, IEEE J. Biomed. Health Inf.
  23. Liu, 2012, Isolation-based anomaly detection, ACM Trans. Knowl. Discov. Data (TKDD), 6, 1, 10.1145/2133360.2133363 · DOI · Paper page
  24. Sertel, 2009, Computer-aided prognosis of neuroblastoma on whole-slide images: Classification of stromal development, Pattern Recognit., 42, 1093, 10.1016/j.patcog.2008.08.027 · DOI · Paper page
  25. Shaban, 2019, A novel digital score for abundance of tumour infiltrating lymphocytes predicts disease free survival in oral squamous cell carcinoma, Sci. Rep., 9, 1, 10.1038/s41598-019-49710-z · DOI · Paper page
  26. Shield, 2017, The global incidence of lip, oral cavity, and pharyngeal cancers by subsite in 2012, CA: Cancer J. Clin., 67, 51
  27. Solis, 2012, Histologic patterns and molecular characteristics of lung adenocarcinoma associated with clinical outcome, Cancer, 118, 2889, 10.1002/cncr.26584 · DOI
  28. Tabesh, 2007, Multifeature prostate cancer diagnosis and gleason grading of histological images, IEEE Trans. Med. Imaging, 26, 1366, 10.1109/TMI.2007.898536 · DOI · Paper page
  29. Tan, F., Yuan, J., Ordonez, V., 2021. Instance-level image retrieval using reranking transformers. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 12105–12115. · DOI
  30. Van der Maaten, 2008, Visualizing data using t-sne, J. Mach. Learn. Res., 9
  31. Verma, 2021, MoNuSAC2020: A multi-organ nuclei segmentation and classification challenge, IEEE Trans. Med. Imaging, 40, 3413, 10.1109/TMI.2021.3085712 · DOI · Paper page
  32. Vu, 2020, Unsupervised tumor characterization via conditional generative adversarial networks, IEEE J. Biomed. Health Inf., 25, 348, 10.1109/JBHI.2020.2993560 · DOI
  33. Wahab, 2021, Semantic annotation for computational pathology: Multidisciplinary experience and best practice recommendations, J. Pathol. Clin. Res.

Cited by

See all 55

Metadata

Authors
Vu, Q. D., Rajpoot, K., Raza, S. E. A., & Rajpoot, N.
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2023-04
Volume
85
Pages
102743
ISSN
1361-8415 (ISSN-L)
OpenAlex
W4317473799
Cited by
55
References
19
FWCI
4.83
Type
journal article
Language
en
Open access
hybrid
Licence
Funders
  • Medical Research Council (MR/P015476/1)
  • Royal Marsden NHS Foundation Trust
  • Royal Society
  • Alan Turing Institute
  • European Research Council (H2020 IMI2-RIA)
  • UK Research and Innovation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Handcrafted Histological Transformer (H2T): Unsupervised representation of whole slide images. https://app.cochat.ai/papers/10.1016/j.media.2023.102743
The paper
Vu, Q. D., Rajpoot, K., Raza, S. E. A., & Rajpoot, N. (2023). Handcrafted Histological Transformer (H2T): Unsupervised representation of whole slide images. Medical Image Analysis. https://doi.org/10.1016/j.media.2023.102743

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.