Artificial Intelligence Algorithms to Assess Hormonal Status From Tissue Microarrays in Patients With Breast Cancer
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Importance: Immunohistochemistry (IHC) is the most widely used assay for identification of molecular biomarkers. However, IHC is time consuming and costly, depends on tissue-handling protocols, and relies on pathologists' subjective interpretation. Image analysis by machine learning is gaining ground for various applications in pathology but has not been proposed to replace chemical-based assays for molecular detection. Objective: To assess the prediction feasibility of molecular expression of biomarkers in cancer tissues, relying only on tissue architecture as seen in digitized hematoxylin-eosin (H&E)-stained specimens. Design, Setting, and Participants: This single-institution retrospective diagnostic study assessed the breast cancer tissue microarrays library of patients from Vancouver General Hospital, British Columbia, Canada. The study and analysis were conducted from July 1, 2015, through July 1, 2018. A machine learning method, termed morphological-based molecular profiling (MBMP), was developed. Logistic regression was used to explore correlations between histomorphology and biomarker expression, and a deep convolutional neural network was used to predict the biomarker expression in examined tissues. Main Outcomes and Measures: Positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristics curve measures of MBMP for assessment of molecular biomarkers. Results: The database consisted of 20 600 digitized, publicly available H&E-stained sections of 5356 patients with breast cancer from 2 cohorts. The median age at diagnosis was 61 years for cohort 1 (412 patients) and 62 years for cohort 2 (4944 patients), and the median follow-up was 12.0 years and 12.4 years, respectively. Tissue histomorphology was significantly correlated with the molecular expression of all 19 biomarkers assayed, including estrogen receptor (ER), progesterone receptor (PR), and ERBB2 (formerly HER2). Expression of ER was predicted for 105 of 207 validation patients in cohort 1 (50.7%) and 1059 of 2046 validation patients in cohort 2 (51.8%), with PPVs of 97% and 98%, respectively, NPVs of 68% and 76%, respectively, and accuracy of 91% and 92%, respectively, which were noninferior to traditional IHC (PPV, 91%-98%; NPV, 51%-78%; and accuracy, 81%-90%). Diagnostic accuracy improved given more data. Morphological analysis of patients with ER-negative/PR-positive status by IHC revealed resemblance to patients with ER-positive status (Bhattacharyya distance, 0.03) and not those with ER-negative/PR-negative status (Bhattacharyya distance, 0.25). This suggests a false-negative IHC finding and warrants antihormonal therapy for these patients. Conclusions and Relevance: For at least half of the patients in this study, MBMP appeared to predict biomarker expression with noninferiority to IHC. Results suggest that prediction accuracy is likely to improve as data used for training expand. Morphological-based molecular profiling could be used as a general approach for mass-scale molecular profiling based on digitized H&E-stained images, allowing quick, accurate, and inexpensive methods for simultaneous profiling of multiple biomarkers in cancer tissues.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis
Litjens, G., Sánchez, C. I., Timofeeva, N., Hermsen, M., Nagtegaal, I., Kovacs, I., Hulsbergen - van de Kaa, C., Bult, P., van Ginneken, B., & van der Laak, J. (2016) · Scientific Reports · doi:10.1038/srep26286 · Cited by 1,069
Hellström (2019) · doi:10.21037/atm · Cited by 798
Research works printed but not linked: 31
- Lin, 2014, Standardization of diagnostic immunohistochemistry: literature review and Geisinger experience., Arch Pathol Lab Med, 138, 1564, 10.5858/arpa.2014-0074-RA · DOI
- Gown, 2008, Current issues in ER and HER2 testing by IHC in breast cancer., Mod Pathol, 21, S8, 10.1038/modpathol.2008.34 · DOI · Paper page
- Robb, 2014, A call to standardize preanalytic data elements for biospecimens., Arch Pathol Lab Med, 138, 526, 10.5858/arpa.2013-0250-CP · doi:10.5858/arpa.2013-0250-cp
- Engel, 2011, Effects of preanalytical variables on the detection of proteins by immunohistochemistry in formalin-fixed, paraffin-embedded tissue., Arch Pathol Lab Med, 135, 537, 10.5858/2010-0702-RAIR.1 · DOI · Paper page
- Langer, 2015, Computer-aided diagnostics in digital pathology: automated evaluation of early-phase pancreatic cancer in mice., Int J Comput Assist Radiol Surg, 10, 1043, 10.1007/s11548-014-1122-9 · DOI
- Romo-Bucheli, 2017, A deep learning based strategy for identifying and associating mitotic activity with gene expression derived risk categories in estrogen receptor positive breast cancers., Cytometry A, 91, 566, 10.1002/cyto.a.v91.6 · doi:10.1002/cyto.a.v91.6
- Djuric, 2017, Precision histology: how deep learning is poised to revitalize histomorphology for personalized cancer care., NPJ Precis Oncol, 1, 22, 10.1038/s41698-017-0022-1 · DOI · Paper page
- Beck, 2011, Systematic analysis of breast cancer morphology uncovers stromal features associated with survival., Sci Transl Med, 3, 108ra113, 10.1126/scitranslmed.3002564 · DOI · Paper page
- Hammond, 2010, American Society of Clinical Oncology/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer (unabridged version)., Arch Pathol Lab Med, 134, e48, 10.5858/134.7.e48 · DOI · Paper page
- Troxell, 2017, Comparison of estrogen and progesterone receptor antibody reagents using proficiency testing data., Arch Pathol Lab Med, 141, 1402, 10.5858/arpa.2016-0497-OA · DOI
- Cheang, 2006, Immunohistochemical detection using the new rabbit monoclonal antibody SP1 of estrogen receptor in breast cancer is superior to mouse monoclonal antibody 1D5 in predicting survival., J Clin Oncol, 24, 5637, 10.1200/JCO.2005.05.4155 · DOI · Paper page
- Bogina, 2012, Comparison of anti-estrogen receptor antibodies SP1, 6F11, and 1D5 in breast cancer: lower 1D5 sensitivity but questionable clinical implications., Am J Clin Pathol, 138, 697, 10.1309/AJCPLX0QJROV2IJG · doi:10.1309/ajcplx0qjrov2ijg
- Tang, 2016, Immunohistochemical surrogates for molecular classification of breast carcinoma: a 2015 update., Arch Pathol Lab Med, 140, 806, 10.5858/arpa.2015-0133-RA · DOI
- Achanta, 2012, SLIC superpixels compared to state-of-the-art superpixel methods., IEEE Trans Pattern Anal Mach Intell, 34, 2274, 10.1109/TPAMI.2012.120 · DOI · Paper page
- Bhattacharyya, 1946, On a measure of divergence between two multinomial populations., Sankhya, 7, 401
- Marinelli, 2008, The Stanford tissue microarray database., Nucleic Acids Res, 36, D871
- Xing, 2014, Automatic Ki-67 counting using robust cell detection and online dictionary learning., IEEE Trans Biomed Eng, 61, 859, 10.1109/TBME.2013.2291703 · DOI · Paper page
- Saha, 2017, An advanced deep learning approach for Ki-67 stained hotspot detection and proliferation rate scoring for prognostic evaluation of breast cancer., Sci Rep, 7, 3213, 10.1038/s41598-017-03405-5 · DOI · Paper page
- Nadji, 2005, Immunohistochemistry of estrogen and progesterone receptors reconsidered: experience with 5,993 breast cancers., Am J Clin Pathol, 123, 21, 10.1309/4WV79N2GHJ3X1841 · doi:10.1309/4wv79n2ghj3x1841 · Paper page
- Barnes, 1996, Immunohistochemical determination of oestrogen receptor: comparison of different methods of assessment of staining and correlation with clinical outcome of breast cancer patients., Br J Cancer, 74, 1445, 10.1038/bjc.1996.563 · DOI · Paper page
- Regan, 2006, Re-evaluating adjuvant breast cancer trials: assessing hormone receptor status by immunohistochemical versus extraction assays., J Natl Cancer Inst, 98, 1571, 10.1093/jnci/djj415 · DOI · Paper page
- Harvey, 1999, Estrogen receptor status by immunohistochemistry is superior to the ligand-binding assay for predicting response to adjuvant endocrine therapy in breast cancer., J Clin Oncol, 17, 1474, 10.1200/JCO.1999.17.5.1474 · DOI · Paper page
- Bianchini, 2010, Molecular anatomy of breast cancer stroma and its prognostic value in estrogen receptor-positive and-negative cancers., J Clin Oncol, 28, 4316, 10.1200/JCO.2009.27.2419 · DOI · Paper page
- Haslam, 2003, Host microenvironment in breast cancer development: epithelial-cell–stromal-cell interactions and steroid hormone action in normal and cancerous mammary gland., Breast Cancer Res, 5, 208, 10.1186/bcr615 · DOI
- Gupta, 2007, Systemic stromal effects of estrogen promote the growth of estrogen receptor-negative cancers., Cancer Res, 67, 2062, 10.1158/0008-5472.CAN-06-3895 · DOI · Paper page
- Knight, 1977, Estrogen receptor as an independent prognostic factor for early recurrence in breast cancer., Cancer Res, 37, 4669
- Pasic, 1990, Comparison of sex steroid receptor determinations in human breast cancer by enzyme immunoassay and radioligand binding., J Clin Lab Anal, 4, 430, 10.1002/(ISSN)1098-2825 · doi:10.1002/(issn)1098-2825
- Mascalzoni, 2015, International charter of principles for sharing bio-specimens and data., Eur J Hum Genet, 23, 721, 10.1038/ejhg.2014.197 · DOI · Paper page
- Kalpathy-Cramer, 2014, Quantitative imaging network: data sharing and competitive AlgorithmValidation leveraging the cancer imaging archive., Transl Oncol, 7, 147, 10.1593/tlo.13862 · DOI
- Gecer, 2018, Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks., Pattern Recognit, 84, 345, 10.1016/j.patcog.2018.07.022 · DOI · Paper page
- Cruz-Roa, 2018, High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: application to invasive breast cancer detection., PLoS One, 13, e0196828, 10.1371/journal.pone.0196828 · DOI · Paper page
Cited by
Digital pathology and artificial intelligence in translational medicine and clinical practice
Baxi, V., Edwards, R., Montalto, M., & Saha, S. (2022) · Modern Pathology · doi:10.1038/s41379-021-00919-2 · Cited by 592
The impact of site-specific digital histology signatures on deep learning model accuracy and bias
Howard, F. M., Dolezal, J., Kochanny, S., Schulte, J., Chen, H., Heij, L., Huo, D., Nanda, R., Olopade, O. I., Kather, J. N., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-24698-1 · Cited by 330
Carter, S. M., Rogers, W., Win, K. T., Frazer, H., Richards, B., & Houssami, N. (2020) · The Breast · doi:10.1016/j.breast.2019.10.001 · Cited by 312
Naik, N., Madani, A., Esteva, A., Keskar, N. S., Press, M. F., Ruderman, D., Agus, D. B., & Socher, R. (2020) · Nature Communications · doi:10.1038/s41467-020-19334-3 · Cited by 258
Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine
Ahn, J. S., Shin, S., Yang, S.-A., Park, E. K., Kim, K. H., Cho, S. I., Ock, C.-Y., & Kim, S. (2023) · Journal of Breast Cancer · doi:10.4048/jbc.2023.26.e45 · Cited by 221
“Additionally, some AI models are promising in predicting ER and PgR status using only H&E-stained slides, thereby eliminating the need for specific IHC staining [119-121].”
BackgroundMethodArtificial intelligence in digital breast pathology: Techniques and applications
Ibrahim, A., Gamble, P., Jaroensri, R., Abdelsamea, M. M., Mermel, C. H., Chen, P.-H. C., & Rakha, E. A. (2020) · The Breast · doi:10.1016/j.breast.2019.12.007 · Cited by 202
“[51] built a deep learning system to predict the statuses of 19 biomarkers including ER and PR.”
BackgroundShamai, G., Livne, A., Polónia, A., Sabo, E., Cretu, A., Bar-Sela, G., & Kimmel, R. (2022) · Nature Communications · doi:10.1038/s41467-022-34275-9 · Cited by 193
Self-citationDetermining breast cancer biomarker status and associated morphological features using deep learning
Gamble, P., Jaroensri, R., Wang, H., Tan, F., Moran, M., Brown, T., Flament-Auvigne, I., Rakha, E. A., Toss, M., Dabbs, D. J., et al. (2021) · Communications Medicine · doi:10.1038/s43856-021-00013-3 · Cited by 154
Li, F., Yang, Y., Wei, Y., He, P., Chen, J., Zheng, Z., & Bu, H. (2021) · Journal of Translational Medicine · doi:10.1186/s12967-021-03020-z · Cited by 113
“Indeed, published data support the idea that morphological features of breast cancer can provide the subtype information [46].”
BackgroundDeep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images
Rawat, R. R., Ortega, I., Roy, P., Sha, F., Shibata, D., Ruderman, D., & Agus, D. B. (2020) · Scientific Reports · doi:10.1038/s41598-020-64156-4 · Cited by 100
A pyramid architecture-based deep learning framework for breast cancer detection
Sui, D., Liu, W., Chen, J., Zhao, C., Ma, X., Guo, M., & Tian, Z. (2021) · BioMed Research International · doi:10.1155/2021/2567202 · Cited by 29
Multistain Pretraining for Slide Representation Learning in Pathology
Jaume, G., Vaidya, A., Zhang, A., H. Song, A., J. Chen, R., Sahai, S., Mo, D., Madrigal, E., Phi Le, L., & Mahmood, F. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73414-4_2 · Cited by 27
Ding, Y., Dhawan, G., Jones, C., Ness, T., Nichols, E., Krasnogor, N., & Reynolds, N. J. (2023) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.18726 · Cited by 9
Shamai, G., Cohen, S., Binenbaum, Y., Sabo, E., Cretu, A., Mayer, C., Barshack, I., Goldman, T., Bar-Sela, G., Polónia, A., et al. (2025) · doi:10.1101/2025.05.15.25327686 · Cited by 3
Self-citation
Metadata
- Authors
- Shamai, G., Binenbaum, Y., Slossberg, R., Duek, I., Gil, Z., & Kimmel, R.
- Venue
- JAMA Network Open
- Publisher
- American Medical Association (AMA)
- Published
- 2019-07-26
- Volume
- 2
- Issue
- 7
- Pages
- e197700
- ISSN
- 2574-3805 (ISSN-L)
- OpenAlex
- W2964047652
- Cited by
- 133
- References
- 4
- FWCI
- 6.91
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.