Deep learning identifies inflamed fat as a risk factor for lymph node metastasis in early colorectal cancer
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
The spread of early‐stage (T1 and T2) adenocarcinomas to locoregional lymph nodes is a key event in disease progression of colorectal cancer (CRC). The cellular mechanisms behind this event are not completely understood and existing predictive biomarkers are imperfect. Here, we used an end‐to‐end deep learning algorithm to identify risk factors for lymph node metastasis (LNM) status in digitized histopathology slides of the primary CRC and its surrounding tissue. In two large population‐based cohorts, we show that this system can predict the presence of more than one LNM in pT2 CRC patients with an area under the receiver operating curve (AUROC) of 0.733 (0.67–0.758) and patients with any LNM with an AUROC of 0.711 (0.597–0.797). Similarly, in pT1 CRC patients, the presence of more than one LNM or any LNM was predictable with an AUROC of 0.733 (0.644–0.778) and 0.567 (0.542–0.597), respectively. Based on these findings, we used the deep learning system to guide human pathology experts towards highly predictive regions for LNM in the whole slide images. This hybrid human observer and deep learning approach identified inflamed adipose tissue as the highest predictive feature for LNM presence. Our study is a first proof of concept that artificial intelligence (AI) systems may be able to discover potentially new biological mechanisms in cancer progression. Our deep learning algorithm is publicly available and can be used for biomarker discovery in any disease setting. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
QuPath: Open source software for digital pathology image analysis
Bankhead, P., Loughrey, M. B., Fernández, J. A., Dombrowski, Y., McArt, D. G., Dunne, P. D., McQuaid, S., Gray, R. T., Murray, L. J., Coleman, H. G., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-17204-5 · Cited by 8,908 · cited 2×
“We presented this information in an unbiased way to a human expert using two approaches: by isolating the image tiles that achieved the highest tile-based prediction scores (Figures 3 and 4) and by an interactive analysis of prediction maps in the open source software QuPath (Figure 5) [41].”
BackgroundMethodEhteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Cellular Fatty Acid Metabolism and Cancer
Currie, E., Schulze, A., Zechner, R., Walther, T. C., & Farese, R. V. (2013) · Cell Metabolism · doi:10.1016/j.cmet.2013.05.017 · Cited by 2,123
A method for normalizing histology slides for quantitative analysis
Macenko, M., Niethammer, M., Marron, J. S., Borland, D., Woosley, J. T., Guan, X., Schmitt, C., & Thomas, N. E. (2009) · 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro · doi:10.1109/isbi.2009.5193250 · Cited by 1,482 · cited 1×
“the images, it is important to normalize the colour of the tiles before training, which was achieved by Macenko normalization [34].”
MethodKather, J. N., Pearson, A. T., Halama, N., Jäger, D., Krause, J., Loosen, S. H., Marx, A., Boor, P., Tacke, F., Neumann, U. P., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0462-y · Cited by 1,458
Self-citationREporting recommendations for tumour MARKer prognostic studies (REMARK).
for the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics, McShane, L. M., Altman, D. G., Sauerbrei, W., Taube, S. E., Gion, M., & Clark, G. M. (2005) · British Journal of Cancer · doi:10.1038/sj.bjc.6602678 · Cited by 1,394
Kather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002730 · Cited by 1,082 · cited 1×
“In gastrointestinal tumours, deep learning can be used to detect clinically relevant patterns such as tumour budding [14–16] and has been used to predict molecular subtypes [10,17,18] and survival [19,20].”
BackgroundSelf-citationDeep learning in cancer pathology: a new generation of clinical biomarkers
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. (2021) · British Journal of Cancer · doi:10.1038/s41416-020-01122-x · Cited by 667 · cited 1×
“Previous studies have shown that the performance of end-to-end deep learningderived biomarkers depends on three factors: the strength of the phenotype (stronger phenotypes are better); the total number of patients in the training cohort (more patients are better); and the class balance (balanced is better) [44].”
BackgroundSelf-citationDeep learning based tissue analysis predicts outcome in colorectal cancer
Bychkov, D., Linder, N., Turkki, R., Nordling, S., Kovanen, P. E., Verrill, C., Walliander, M., Lundin, M., Haglund, C., & Lundin, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-21758-3 · Cited by 644 · cited 1×
“In CRC, various studies have attempted LNM prediction using immunohistochemical stains [24] or tissue microarrays [25].”
BackgroundDeep learning for prediction of colorectal cancer outcome: a discovery and validation study
Skrede, O.-J., De Raedt, S., Kleppe, A., Hveem, T. S., Liestøl, K., Maddison, J., Askautrud, H. A., Pradhan, M., Nesheim, J. A., Albregtsen, F., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(19)32998-8 · Cited by 627
Yamashita, R., Long, J., Longacre, T., Peng, L., Berry, G., Martin, B., Higgins, J., Rubin, D. L., & Shen, J. (2021) · The Lancet Oncology · doi:10.1016/s1470-2045(20)30535-0 · Cited by 391 · cited 1×
“While prediction of LNM status in CRC has not been reported before, other studies including our own have shown that microsatellite instability or deficient mismatch repair (MSI/dMMR) in CRC can be predicted with high performance from histology images [35,42].”
ResultClinical-Grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning
Echle, A., Grabsch, H. I., Quirke, P., van den Brandt, P. A., West, N. P., Hutchins, G. G. A., Heij, L. R., Tan, X., Richman, S. D., Krause, J., et al. (2020) · Gastroenterology · doi:10.1053/j.gastro.2020.06.021 · Cited by 376
Self-citationThe impact of site-specific digital histology signatures on deep learning model accuracy and bias
Howard, F. M., Dolezal, J., Kochanny, S., Schulte, J., Chen, H., Heij, L., Huo, D., Nanda, R., Olopade, O. I., Kather, J. N., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-24698-1 · Cited by 330
Self-citationPathologic Predictors of Microsatellite Instability in Colorectal Cancer
Greenson, J. K., Huang, S.-C., Herron, C., Moreno, V., Bonner, J. D., Tomsho, L. P., Ben-Izhak, O., Cohen, H. I., Trougouboff, P., Bejhar, J., et al. (2009) · American Journal of Surgical Pathology · doi:10.1097/pas.0b013e31817ec2b1 · Cited by 286 · cited 1×
“Again, dMMR was used as a positive control because the histopathological patterns associated with MSI/dMMR are known [43].”
MethodInterpretable survival prediction for colorectal cancer using deep learning
Wulczyn, E., Steiner, D. F., Moran, M., Plass, M., Reihs, R., Tan, F., Flament-Auvigne, I., Brown, T., Regitnig, P., Chen, P.-H. C., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00427-2 · Cited by 231 · cited 1×
“For example, in CRC, deep learning has been used to identify tumour– adipose clusters as a negative prognostic feature in advanced tumours [28].”
MethodMorphological and molecular breast cancer profiling through explainable machine learning
Binder, A., Bockmayr, M., Hägele, M., Wienert, S., Heim, D., Hellweg, K., Ishii, M., Stenzinger, A., Hocke, A., Denkert, C., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00303-4 · Cited by 156
Zaytseva, Y. Y., Harris, J. W., Mitov, M. I., Kim, J. T., Butterfield, D. A., Lee, E. Y., Weiss, H. L., Gao, T., & Evers, B. M. (2015) · Oncotarget · doi:10.18632/oncotarget.3783 · Cited by 153
Muti, H. S., Heij, L. R., Keller, G., Kohlruss, M., Langer, R., Dislich, B., Cheong, J.-H., Kim, Y.-W., Kim, H., Kook, M.-C., et al. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00133-3 · Cited by 126 · cited 1×
“To predict LNM status and dMMR status, we used our own open-source algorithm [10] that has been extensively validated in colorectal cancer [35], bladder cancer [36], and gastric cancer [37].”
MethodSelf-citationvan Rijthoven, M., Balkenhol, M., Siliņa, K., van der Laak, J., & Ciompi, F. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101890 · Cited by 16
Cited by
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology
Shmatko, A., Ghaffari Laleh, N., Gerstung, M., & Kather, J. N. (2022) · Nature Cancer · doi:10.1038/s43018-022-00436-4 · Cited by 446
“Highly scoring image tiles can be visualized 19,20 , allowing human experts to check the plausibility of AI systems and also discover new features with the help of AI 21,22 (Fig.”
BackgroundSelf-citationMultistain deep learning for prediction of prognosis and therapy response in colorectal cancer
Foersch, S., Glasner, C., Woerl, A.-C., Eckstein, M., Wagner, D.-C., Schulz, S., Kellers, F., Fernandez, A., Tserea, K., Kloth, M., et al. (2023) · Nature Medicine · doi:10.1038/s41591-022-02134-1 · Cited by 251
Self-citationPathologist validation of a machine learning–derived feature for colon cancer risk stratification
L’Imperio, V., Wulczyn, E., Plass, M., Müller, H., Tamini, N., Gianotti, L., Zucchini, N., Reihs, R., Corrado, G. S., Webster, D. R., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.54891 · Cited by 57
One label is all you need: Interpretable AI-enhanced histopathology for oncology
Tavolara, T. E., Su, Z., Gurcan, M. N., & Niazi, M. K. K. (2023) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2023.09.006 · Cited by 28
Metadata
- Authors
- Brockmoeller, S., Echle, A., Ghaffari Laleh, N., Eiholm, S., Malmstrøm, M. L., Plato Kuhlmann, T., Levic, K., Grabsch, H. I., West, N. P., Saldanha, O. L., Kouvidi, K., Bono, A., Heij, L. R., Brinker, T. J., Gögenür, I., Quirke, P., & Kather, J. N.
- Venue
- The Journal of Pathology
- Publisher
- Wiley
- Published
- 2021-12-28
- Volume
- 256
- Issue
- 3
- Pages
- 269-281
- ISSN
- 0022-3417 (ISSN-L)
- 1096-9896
- OpenAlex
- W3210482027
- Cited by
- 78
- References
- 22
- FWCI
- 5.65
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.