CoChat

AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Abstract

The lack of publicly available ground-truth data has been identified as the major challenge for transferring recent developments in deep learning to the biomedical imaging domain. Though crowdsourcing has enabled annotation of large scale databases for real world images, its application for biomedical purposes requires a deeper understanding and hence, more precise definition of the actual annotation task. The fact that expert tasks are being outsourced to non-expert users may lead to noisy annotations introducing disagreement between users. Despite being a valuable resource for learning annotation models from crowdsourcing, conventional machine-learning methods may have difficulties dealing with noisy annotations during training. In this manuscript, we present a new concept for learning from crowds that handle data aggregation directly as part of the learning process of the convolutional neural network (CNN) via additional crowdsourcing layer (AggNet). Besides, we present an experimental study on learning from crowds designed to answer the following questions. 1) Can deep CNN be trained with data collected from crowdsourcing? 2) How to adapt the CNN to train on multiple types of annotation datasets (ground truth and crowd-based)? 3) How does the choice of annotation and aggregation affect the accuracy? Our experimental setup involved Annot8, a self-implemented web-platform based on Crowdflower API realizing image annotation tasks for a publicly available biomedical image database. Our results give valuable insights into the functionality of deep CNN learning from crowd annotations and prove the necessity of data aggregation integration.

Overview

Integrity signals

Registry

Crossref · 10.1109/tmi.2016.2528120

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 38 references

Research works printed but not linked: 1

  1. howe, 2006, The rise of crowdsourcing, Wired Mag, 14, 1

Cited by

See all 613

Metadata

Authors
Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., & Navab, N.
Affiliations
  • German Center for Neurodegenerative Diseases (DE)
  • Technical University of Munich (DE)
  • University of Oxford (GB)
  • Johns Hopkins University (US)
Venue
IEEE Transactions on Medical Imaging
Published
2016-05
Volume
35
Issue
5
Pages
1313-1321
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
OpenAlex
W2339885376
Cited by
606
References
38
Citations by year
2026: 14 · 2025: 24 · 2024: 59 · 2023: 59 · 2022: 60 · 2021: 92 · 2020: 89 · 2019: 80 · 2018: 73 · 2017: 46 · 2016: 10
FWCI
57.40
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • DZNE Pub (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Keywords
mitosis detection, breast cancer histopathology, crowdsourced annotation, noisy annotations, data aggregation, convolutional neural networks, CrowdFlower
MeSH terms
Machine Learning, Breast Neoplasms, Female, Histocytochemistry, Humans, Image Interpretation, Computer-Assisted, Mitosis, Neural Networks, Computer, Video Games, Internet, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images. https://app.cochat.ai/papers/10.1109/tmi.2016.2528120
The paper
Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., & Navab, N. (2016). AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/tmi.2016.2528120

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.