Cited by 75AI in cancer detectionMultimodal Machine Learning ApplicationsImage Retrieval and Classification Techniques

Multiple Instance Captioning: Learning Representations from Histopathology Textbooks and Articles

Open access
Gamper, Jevgenij, Rajpoot, Nasir·2021·2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Abstract

We present ARCH, a computational pathology (CP) multiple instance captioning dataset to facilitate dense supervision of CP tasks. Existing CP datasets focus on narrow tasks; ARCH on the other hand contains dense diagnos-tic and morphological descriptions for a range of stains, tissue types and pathologies. Using intrinsic dimensionality estimation, we show that ARCH is the only CP dataset to (ARCH-)rival its computer vision analog MS-COCO Captions. We conjecture that an encoder pre-trained on dense image captions learns transferable representations for most CP tasks. We support the conjecture with evidence that ARCH representation transfers to a variety of pathology sub-tasks better than ImageNet features or representations obtained via self-supervised or multi-task learning on pathology images alone. We release our best model and invite other researchers to test it on their CP tasks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr46437.2021.01628

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 8 · arXiv
  2. R. Mormont, P. Geurts, and R. Marée. Multi-task pre- training of deep neural networks for digital pathology. IEEE Journal of Biomedical and Health Informatics, 2020. 3, 5, 6
  3. D. Tellez, D. Hoppener, C. Verhoef, D. Grunhagen, P. Nierop, M. Drozdzal, J. van der Laak, and F. Ciompi. Ex- tending unsupervised neural image compression with super- vised multitask learning. arXiv preprint arXiv:2004.07041, 2020. 3 · arXiv
  4. D. Tellez, G. Litjens, J. van der Laak, and F. Ciompi. Neural image compression for gigapixel histopathology image anal- ysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. 2

Cited by

Sorted by citations

Authors
Gamper, J., & Rajpoot, N.
Affiliations
University of Warwick (GB)
Venue
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2021-06
Pages
16544-16554
ISSN
1063-6919 (ISSN-L)
OpenAlex
W3135141466
Cited by
75
References
58
Citations by year
2026: 6 · 2025: 16 · 2024: 30 · 2023: 16 · 2022: 5 · 2021: 2
FWCI
6.12
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Keywords
computational pathology, histopathology, representation transfer, intrinsic dimensionality estimation, histopathological image classification, morphological description

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review