The DOI matches its Crossref record; 37 of the 40 references it cites were found in a registry or an index; none of the 3 sources its claims cite is checked for support yet.
Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer
Open accessAbstract
Importance: Application of deep learning algorithms to whole-slide pathology images can potentially improve diagnostic accuracy and efficiency. Objective: Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists' diagnoses in a diagnostic setting. Design, Setting, and Participants: Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). Exposures: Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. Main Outcomes and Measures: The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. Results: The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P < .001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC). Conclusions and Relevance: In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
38 Own result · 16 Own method · 10 Own interpretation · 4 No basis given · 3 Not checked yet
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the slide level in task 2, the pathologist WOTC achieved a sensitivity of 93.8% (95% CI, 86.9%-100.0%), a specificity of 98.7% (95% CI, 96.0%-100.0%),and an AUC of 0.966 (95% CI, 0.927-0.998).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the slide level, the leading algorithms performed better than the pathologists WTC in the simulation exercise.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The simulated time-constrained exercise mimics routine diagnostic pathology workflow.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Pathologist WOTC | 0.724 | 0.966 | | | | Expert pathologist who assessed”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“They achieved a mean sensitivity of 62.8% (95% CI, 58.9%-71.9%)with a mean specificity of 98.5% (95% CI, 97.9%-99.1%).The mean AUC was 0.810 (range, 0.738-0.884)”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No false-positives were produced in task 1 (ie, nontumorous tissue indicated as metastasis) by the pathologist WOTC”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This AUC exceeded the mean performance of the pathologists WTC (mean AUC, 0.810 [range, 0.738-0.884]) in the diagnostic simulation exercise (P < .001,calculated using MRMC ROC analysis 33 ) (Table 2).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Task 2: Whole-Slide Image Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Algorithms based on manually engineered features performed less well.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the slide level in task 2, the pathologist WOTC achieved a sensitivity of 93.8% (95% CI, 86.9%-100.0%), a specificity of 98.7% (95% CI, 96.0%-100.0%),and an AUC of 0.966 (95% CI, 0.927-0.998).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This algorithm achieved an overall AUC of 0.976 (95% CI, 0.941-0.999),and yielded the highest AUC in detecting macrometastases (AUC, 1.0).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Task 2: Whole-Slide Image Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 16
- The quote only partly states this claim. 3
- It may state two separate claims as one. 11
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All of the 32 algorithms submitted to CAMELYON16 used a discriminative learning approach to identify metastases in whole-slide images.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Mean pathologists WTC | | 0.810 | | | | The mean performance of 11”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, deep learning-based algorithms performed significantly better than other methods: the 19 topperforming algorithms in both tasks all used deep convolutional neural networks as the underlying methodology (Table 2).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Algorithm Performance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“An earlier submission by this team, HMS and MGH I, achieved an overall AUC of 0.964 (95% CI, 0.928-0.989)and ranked third.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Task 2: Whole-Slide Image Classification
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“but 27.6% of individual metastases were not identified (lesion level, truepositive fraction, 72.4% [95% CI, 64.3%-80.4%])that manifested when IHC staining was performed.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The pathologist did not produce any false-positives and achieved a true-positive fraction of 0.724 for detecting and localizing metastatic regions.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In cross-sectional analyses that evaluated 32 algorithms submitted as part of a challenge competition, 7 deep learning algorithms showed greater discrimination than a panel of 11 pathologists in a simulated time-constrained diagnostic setting, with an area under the curve of 0.994 (best algorithm) vs 0.884 (best pathologist).”
- Also stated in: § Task 2: Whole-Slide Image Classification · plain
“Overall, 7 of the 32 submitted algorithms had significantly higher AUCs than the pathologists WTC (see Table 2 for the individual P values calculated using MRMC ROC analysis).”
- Also stated in: § Discussion · boosted
“The CAMELYON16 challenge demonstrated that some deep learning algorithms were able to achieve a better AUC than a panel of 11 pathologists WTC participating in a simulation exercise for detection of lymph node metastases of breast cancer.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Key Points
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Task 2: Whole-Slide Image Classification · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The best algorithm achieved similar true-positive fraction as the pathologist WOTC when producing a mean of 1.25 false-positive lesions in 100 whole-slide images and performed better when allowing for slightly more falsepositive findings.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The set of images was randomly divided into a training (n = 270) and a test set (n = 129; details in Table 1).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Reference Standard
plainn = 270- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“system generalization and performance seemed more important.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Downloaded From: by a Radboud University Nijmegen User on 12/12/2017
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“Previous studies on diagnostic imaging tasks in which deep learning reached human-level performance, such as detection of diabetic retinopathy in retinal fundus photographs, used a reference standard based on the consensus of human experts.3”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Downloaded From: by a Radboud University Nijmegen User on 12/12/2017
What the sources say
- Not checked yetcites [3]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The test data set on which algorithms and pathologists were evaluated was enriched with cases containing metastases and, specifically, micrometastases and, thus, is not directly comparable with the mix of cases pathologists encounter in clinical practice.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Most sentinel lymph nodes do not contain metastases.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“pathologists may rely on IHC staining and the knowledge that all hematoxylin and eosin-slides with negative findings will undergo additional review with the use of IHC.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Rests on an assumption: In most hospitals, pathologists request additional immunohistochemical staining in equivocal cases.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Assessment without time constraint does not yield a fair measure of the accuracy of the routine diagnostic process.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Panel of Pathologists With Time Constraint
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“staining, yielding an independent reference against which human pathologists could also be compared.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Downloaded From: by a Radboud University Nijmegen User on 12/12/2017
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Less time constraint on task completion may increase the accuracy of SLN diagnostic review.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“algorithm runtime was not recorded nor included as a factor in the evaluation, but it might influence the suitability for use in, for example, frozen section analysis.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“the simulation exercise invited pathologists WTC to review a series of 129 hematoxylin and eosinstained slides in about 2 hours to determine the presence of macroscopic or microscopic SLN metastasis.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Thepercentilebootstrapmethod was used to construct 95% CIs for FROC true-positive fraction scores (FROC scores) and AUCs.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The funders and sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Funding
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The pathologists WTC assessed the same set of glass slides used for testing the algorithms (which used digitized whole-slide images of these glass slides).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Panel of Pathologists With Time Constraint
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“IHC is the most accurate method for metastasis evaluation and has little interpretation variability. [7][8]][9]”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Reference Standard
What the sources say
- Not checked yetcites [7][8]][9]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“algorithms were specifically trained to discriminate between normal and cancerous tissue in the background of lymph node histological architecture, but they might be unable to identify rare events such as co-occurring pathologies (eg, lymphoma, sarcoma, or infection).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The detection of other pathologies in the SLN, which is relevant in routine diagnostics, was not included in this study.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“No prior data were available for the performance of algorithms in this task.Therefore, no power analysis was used to predetermine the sample size.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Statistical Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: No prior data were available for algorithm performance in this task.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In this study, every pathologist was given 1 single hematoxylin and eosin-stained slide per patient to determine the presence or absence of breast cancer metastasis.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The P values were adjusted for multiple comparisons using the Bonferroni correction, in which the P values are multiplied by the number of comparisons (32; comparison of the 32 submitted algorithms with the panel of pathologists).”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Conclusions
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although feasible in the context of this simulation, this does not represent the work pace in other settings.”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The clinical value of isolated tumor cells in a sentinel lymph node is disputed.
“Because the clinical value of having only isolated tumor cells in an SLN is disputed, we did not include such slides in our study”
The paper relies on this without support.
The simulated time-constrained exercise mimics routine diagnostic pathology workflow.
“To mimic routine diagnostic pathology workflow, we asked 11 pathologists to independently assess the 129 slides in the test set in”
The paper relies on this without support.
The panel of 11 pathologists is representative of pathologists in routine practice.
“The panel of the 11 pathologists (mean age, 47.7 years [range, 31-61]) included 1 resident pathologist (3-year resident) and 10 practicing pathologists”
The paper relies on this without support.
Comparing algorithms on whole-slide images with pathologists on glass slides is valid despite the different modalities.
“The panel of 11 pathologists with time constraint (pathologists WTC), however, did their assessment on the corresponding glass slides for those images because diagnosing is most commonly done using a microscope in pathology labs.”
The paper relies on this without support.
In the Netherlands, use of immunohistochemistry is mandatory for cases with negative findings on H&E-stained slides.
“The setting resembled diagnostic practice in the Netherlands, where use of IHC is mandatory for cases with negative findings on hematoxylin and eosin-stained slides.”
The paper relies on this without support.
No prior data were available for algorithm performance in this task.
“No prior data were available for the performance of algorithms in this task.Therefore, no power analysis was used to predetermine the sample size.”
The paper relies on this without support.
Most sentinel lymph nodes do not contain metastases.
“Given the reality that most SLNs do not contain metastases, the data set curation was needed to achieve a well-rounded representation of what is encountered in clinical practice without including an exorbitant number of slides.”
The paper relies on this without support.
In real clinical settings, sections from multiple levels are evaluated for every lymph node.
“In a real clinical setting, sections from multiple levels are evaluated for every lymph node.”
The paper relies on this without support.
In most hospitals, pathologists request additional immunohistochemical staining in equivocal cases.
“in most hospitals pathologists request additional IHC staining in equivocal cases.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Medical research is no exception, as CNN has achieved expert-level performances in various fields. Gulshan et al. [4], Esteva et al. [5], and Ehteshami Bejnordi et al. [6] demonstrated the potential of deep learning for diabetic retinopathy screening, skin lesion classification, and lymph node metastasis detection, respectively.”
From Convolutional neural networks: an overview and application in radiology (2018) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“In a notable study, Bejnordi et al. (2017) found that certain deep learning algorithms surpassed a group of 11 pathologists in detecting lymph node metastases in breast cancer patients.”
From A perspective on integrating digital pathology, proteomics, clinical data and AI analytics in cancer research (2025) · cites this paper as [24]
In the citing paper
“This evaluation includes comparison with a reference panel of pathologists working on the same data set with predefined time constraints designed to mimic routine pathology workflows, where many of the models in the challenge exhibited higher performance than the panel of pathologists.”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [45]
In the citing paper
“The second option is deidentification and public release of data, usually via a consortium model.”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Downloaded From: by a Radboud University Nijmegen User on 12/12/2017
“Task 2 was measured on the 129 whole-slide images (for algorithms and the pathologist WTC) and corresponding glass slides (for 11 pathologists WOTC) in the test data set”
§ Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer > Task 1: Identification of Individual Metastases
“Algorithms assessed whole-slide images, as did the pathologist WOTC.The panel of 11 pathologists with time constraint (pathologists WTC), however, did their assessment on the corresponding glass slides”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565
Textural Features for Image Classification
Haralick, R. M., Shanmugam, K., & Dinstein, I. (1973) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1973.4309314 · Cited by 22,886
Cited by
Sorted by citations
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
Convolutional neural networks: an overview and application in radiology
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018) · Insights into Imaging · doi:10.1007/s13244-018-0639-9 · Cited by 4,682
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Precision medicine, AI, and the future of personalized health care
Johnson, K. B., Wei, W., Weeraratne, D., Frisse, M. E., Misulis, K., Rhee, K., Zhao, J., & Snowdon, J. L. (2021) · Clinical and Translational Science · doi:10.1111/cts.12884 · Cited by 1,854
“87 AI-mediated image analysis aimed at improving disease risk prediction and diagnosis will likely continue to increase in use for detection of diabetic retinopathy 88 and metastasis in cancer, 89 as well as for identification of benign melanoma.”
BackgroundArtificial intelligence in healthcare: transforming the practice of medicine
Bajwa, J., Munir, U., Nori, A., & Williams, B. (2021) · Future Healthcare Journal · doi:10.7861/fhj.2021-0095 · Cited by 1,856
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0252-y · Cited by 1,698
Artificial Intelligence and Deep Learning in Ophthalmology
Ting, D. S. W., Pasquale, L. R., Peng, L., Campbell, J. P., Lee, A., Raman, R., Tan, G. S. W., Schmetterer, L. F., Keane, P. A., & Wong, T. Y. (2018) · Springer eBooks · doi:10.1007/978-3-030-64573-1_200 · Cited by 1,660
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560
Self-citationRajpurkar, P., Irvin, J., Ball, R. L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C. P., et al. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002686 · Cited by 1,449
Transformers in medical imaging: A survey
Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102802 · Cited by 1,523
“TransMIL shows state-of-the-art performance on three different computational pathology datasets CAMELYON16 (breast) [250], TCGA-NSCLC (lung) [242], and TCGA-R (kidney) [251].”
BackgroundMethodZhou, S. K., Greenspan, H. K., Davatzikos, C. A., Duncan, J. S., van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,373
“[218] discuss the diagnostic assessment of DL algorithms for detection of lymph node metastases in women with breast cancer, as part of the CAMELYON16 challenge.”
BackgroundSelf-citationKather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002730 · Cited by 1,082
Ahmed, Z., Mohamed, K., Zeeshan, S., & Dong, X. (2020) · Database · doi:10.1093/database/baaa010 · Cited by 1,035
“radiology (71) (early diagnosis, enhance visualization of pathologies and predicting emergency situations (64,73,71)), oncology (diagnosis of breast (75), skin (76), lung cancer (77)), cardiology (interpreting electrocardiogram readings, echocardiography with 3D cardiac imaging, cardiac CT angiography for calcificatio…”
BackgroundLi, B., Li, Y., & Eliceiri, K. W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01409 · Cited by 957
Tumor-infiltrating lymphocytes in the immunotherapy era
Paijens, S. T., Vledder, A., de Bruyn, M., & Nijman, H. W. (2020) · Cellular and Molecular Immunology · doi:10.1038/s41423-020-00565-9 · Cited by 914
“The application of deep learning algorithms was also suggested for the detection of lymph node metastases in breast cancer.(189) Overall, recent developments support the implementation of immune scores as a new component in the classification of cancer and advocate for the development and use of automatic machine lear…”
BackgroundA visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966
Deep learning in histopathology: the path to the clinic
van der Laak, J., Litjens, G., & Ciompi, F. (2021) · Nature Medicine · doi:10.1038/s41591-021-01343-4 · Cited by 902
Self-citationA Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 1,184
“ut (Fashion-)MNIST, CIFAR-10, STL-10, ImageNet Synthetic MNIST-C [413], ImageNet-C [414], ImageNet-P [414], ImageNet-O [419] Real-world Industrial: MVTec-AD [189], PCB [420] Medical: CAMELYON16 [61], [421], NIH Chest Xray [61], [422], MOOD [423], HCP/BRATS [52], Neuropathology [60], [424] Security: Credit-card-fraud […”
Park, S. H., & Han, K. (2018) · Radiology · doi:10.1148/radiol.2017171920 · Cited by 894
Automatic diagnosis of the 12-lead ECG using a deep neural network
Ribeiro, A. H., Ribeiro, M. H., Paixão, G. M. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15432-4 · Cited by 894
Predicting cancer outcomes with radiomics and artificial intelligence in radiology
Bera, K., Braman, N., Gupta, A., Velcheti, V., & Madabhushi, A. (2022) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-021-00560-7 · Cited by 819
“This knowledge enables the clinician to identify patients with poor prognosis who might be candidates for escalation of many cases their diagnostic performance has been shown to match or even surpass that of human experts in multiple different cancer types 6,7 .”
ResultIntroduction to artificial intelligence in medicine
Mintz, Y., & Brodie, R. (2019) · Minimally Invasive Therapy & Allied Technologies · doi:10.1080/13645706.2019.1575882 · Cited by 867
Artificial Intelligence in Cancer Research and Precision Medicine
Bhinder, B., Gilvary, C., Madhukar, N. S., & Elemento, O. (2021) · Cancer Discovery · doi:10.1158/2159-8290.cd-21-0090 · Cited by 781
“org restricted to 40× magnification only), or (2) they retrained their model MSINet on the same training set as used by Kather and colleagues and applied it to their internal dataset (n = 100): here they report an AUC of 0.”
MethodArtificial intelligence in retina
Schmidt-Erfurth, U., Sadeghipour, A., Gerendas, B. S., Waldstein, S. M., & Bogunović, H. (2018) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2018.07.004 · Cited by 779
Deep neural network models for computational histopathology: A survey
Srinidhi, C. L., Ciga, O., & Martel, A. L. (2021) · Medical Image Analysis · doi:10.1016/j.media.2020.101813 · Cited by 749
Automatic diagnosis of the 12-lead ECG using a deep neural network
Ribeiro, A. H., Ribeiro, M. H., Paixao, G. M., Oliveira, D. M., Gomes, P. R., Canazart, J. A., Ferreira, M. P. S., Andersson, C. R., Macfarlane, P. W., Meira, W., et al. (2020) · RePEc: Research Papers in Economics · doi:10.5281/zenodo.3625006 · Cited by 722
Transformer-based unsupervised contrastive learning for histopathological image classification
Wang, X., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., & Han, X. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102559 · Cited by 751
Han, S. S., Kim, M. S., Lim, W., Park, G., Park, I., & Chang, S. E. (2018) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2018.01.028 · Cited by 740
“There have been several reports that advanced algorithms are as accurate as experienced specialists (Bejnordi et al., 2017; Esteva et al., 2017; Gulshan et al., 2016; Rajpurkar et al., 2017).”
BackgroundAlgorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750
“The CAMELYON16/17 challenge, which was used in validating the first ‘clinical-grade’ AI models for the detection of lymph-node metastases from diagnostic pathology images, was sourced entirely from the Netherlands 131,132 .”
MethodBulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J., Hulsbergen-van de Kaa, C., & Litjens, G. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30739-9 · Cited by 705
Self-citationRodriguez-Ruiz, A., Lång, K., Gubern-Merida, A., Broeders, M., Gennaro, G., Clauser, P., Helbich, T. H., Chevalier, M., Tan, T., Mertelmeier, T., et al. (2019) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djy222 · Cited by 692
“In medical imaging, deep learning-based AI is also rapidly closing the gap between humans and computers (15,16).”
BackgroundBrown, J. M., Campbell, J. P., Beers, A., Chang, K., Ostmo, S., Chan, R. V. P., Dy, J., Erdogmus, D., Ioannidis, S., Kalpathy-Cramer, J., et al. (2018) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2018.1934 · Cited by 657
Deep learning for prediction of colorectal cancer outcome: a discovery and validation study
Skrede, O.-J., De Raedt, S., Kleppe, A., Hveem, T. S., Liestøl, K., Maddison, J., Askautrud, H. A., Pradhan, M., Nesheim, J. A., Albregtsen, F., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(19)32998-8 · Cited by 627
Tellez, D., Litjens, G., Bándi, P., Bulten, W., Bokhorst, J.-M., Ciompi, F., & van der Laak, J. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101544 · Cited by 649
“This popular method has been used by numerous researchers in recent public chal-6 lenges (B´andi et al. (2019); Bejnordi et al. (2017)).”
BackgroundMethodInfluentialSelf-citationRajula, H. S. R., Verlato, G., Manchia, M., Antonucci, N., & Fanos, V. (2020) · Medicina · doi:10.3390/medicina56090455 · Cited by 606
“Bejnordi et al. have recently evaluated the performance of automated deep learning algorithms at identifying metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and related it with pathologists’ diagnoses in a diagnostic setting [37].”
BackgroundStröm, P., Kartasalo, K., Olsson, H., Solorzano, L., Delahunt, B., Berney, D. M., Bostwick, D. G., Evans, A. J., Grignon, D. J., Humphrey, P. A., et al. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30738-7 · Cited by 620
Self-citationSecure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59
“Clinical medicine has emerged as a exciting application area for ML/DL models, and these models have already achieved humanlevel performance in clinical pathology [10], radiology [11],”
BackgroundDigital pathology and artificial intelligence in translational medicine and clinical practice
Baxi, V., Edwards, R., Montalto, M., & Saha, S. (2022) · Modern Pathology · doi:10.1038/s41379-021-00919-2 · Cited by 592
Bandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Ehteshami Bejnordi, B., Lee, B., Paeng, K., Zhong, A., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2867350 · Cited by 569
Self-citationKrause, J., Gulshan, V., Rahimy, E., Karth, P., Widner, K., Corrado, G. S., Peng, L., & Webster, D. R. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.01.034 · Cited by 564
“It has also been applied recently in medical imaging, producing highly accurate algorithms for a variety of classification tasks, including melanoma [11], breast cancer lymph node metastasis [12,13], and DR [14–16] Because the network is trained to predict labels that have been paired with the images, it is imperative…”
Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning
Chen, R. J., Chen, C., Li, Y., Chen, T. Y., Trister, A. D., Krishnan, R. G., & Mahmood, F. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01567 · Cited by 621
“Moreover, WSIs also exhibit a hierarchical structure of visual tokens at varying image resolutions at 20× magnification: the 16 × 16 images encompass the bounding box of cells and other fine-grained features (stroma, tumor cells, lymphocytes) [22, 36], 256 × 256 images capture local clusters of cell-to-cell interactio…”
BackgroundA foundation model for clinical-grade computational pathology and rare cancers detection
Vorontsov, E., Bozkurt, A., Casson, A., Shaikovski, G., Zelechowski, M., Severson, K., Zimmermann, E., Hall, J., Tenenholtz, N., Fusi, N., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03141-0 · Cited by 572
Xu, L., Sanders, L., Li, K., & Chow, J. C. L. (2021) · JMIR Cancer · doi:10.2196/27850 · Cited by 571
“Studies have shown that the interpretation of medical images for the diagnosis of tumors performs equally well or better with AI compared with experts [53-56].”
Artificial Intelligence and Digital Pathology: Challenges and Opportunities
Tizhoosh, H. R., & Pantanowitz, L. (2018) · Journal of Pathology Informatics · doi:10.4103/jpi.jpi_53_18 · Cited by 544
A pathology foundation model for cancer diagnosis and prognosis prediction
Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., Zhang, J., Li, R., Tang, H., Wang, K., Li, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07894-z · Cited by 580
fastMRI: An Open Dataset and Benchmarks for Accelerated MRIPreprint
Žbontar, J., Knöll, F., Sriram, A., Murrell, T., Huang, Z., Muckley, M. J., Defazio, A., Stern, R., Johnson, P., Bruno, M. T., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08839 · Cited by 613
“Datasets such as BraTS [24], LUNA [37], ChestX-ray [50], DeepLesion [55], and Camelyon [1] are widely used in research with clinical applications.”
BackgroundSelf-citationArtificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge
Bulten, W., Kartasalo, K., Chen, P.-H. C., Ström, P., Pinckaers, H., Nagpal, K., Cai, Y., Steiner, D. F., van Boven, H., Vink, R., et al. (2022) · Nature Medicine · doi:10.1038/s41591-021-01620-2 · Cited by 640
“GG 1 852 (17) 1,814 (33) 24 (12) 48 (24) 48 (14) 53 (25) 247 (33) 65 (20) 3,151 (25)”
Self-citationKandel, I., & Castelli, M. (2020) · ICT Express · doi:10.1016/j.icte.2020.04.010 · Cited by 597
“The dataset used in this experiment was the PatchCamelyon [16], [17] a public dataset that contains 220,000 binary labeled images to train the CNN.”
MethodArtificial Intelligence in Anesthesiology
Hashimoto, D. A., Witkowski, E. R., Gao, L., Meireles, O. R., & Rosman, G. (2019) · Anesthesiology · doi:10.1097/aln.0000000000002960 · Cited by 591
Kim, H.-E., Kim, H. H., Han, B., Kim, K., Han, K., Nam, H., Lee, E. H., & Kim, E. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30003-0 · Cited by 556
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., & Huang, J. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101789 · Cited by 522
“The diagnosis is extremely laborious and highly dependent on expertise which requires pathologists to carefully examine the biopsies under the microscope [8].”
BackgroundHow to Read Articles That Use Machine Learning
Liu, Y., Chen, P.-H. C., Krause, J., & Peng, L. H. (2019) · JAMA · doi:10.1001/jama.2019.16489 · Cited by 547
A new era: artificial intelligence and machine learning in prostate cancer
Goldenberg, S. L., Nir, G., & Salcudean, S. E. (2019) · Nature Reviews Urology · doi:10.1038/s41585-019-0193-3 · Cited by 536
Artificial intelligence: Implications for the future of work
Howard, J. (2019) · American Journal of Industrial Medicine · doi:10.1002/ajim.23037 · Cited by 522
Deep Learning and Medical Diagnosis: A Review of Literature
Bakator, M., & Radosav, D. (2018) · Multimodal Technologies and Interaction · doi:10.3390/mti2030047 · Cited by 520
“[43] CNN Images produced with a digital slider Lymph node metastases detection in breast cancer; the best performing algorithm achieved performance comparable to a pathologist.”
Nagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495
Steiner, D. F., MacDonald, R., Liu, Y., Truszkowski, P., Hipp, J. D., Gammage, C., Thng, F., Peng, L., & Stumpe, M. C. (2018) · American Journal of Surgical Pathology · doi:10.1097/pas.0000000000001151 · Cited by 489
“10 15 Although immunohistochemistry (IHC) is often utilized to improve diagnostic sensitivity, such stains are costly and time consuming, and institution-speci fi c practices for the use of IHC vary considerably.”
BackgroundAdvances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 557
Abels, E., Pantanowitz, L., Aeffner, F., Zarella, M. D., van der Laak, J. A. W. M., Bui, M. M., Vemuri, V. N. P., Parwani, A. V., Gibbs, J., Agosto‐Arroyo, E., et al. (2019) · The Journal of Pathology · doi:10.1002/path.5331 · Cited by 469
Self-citationMachine learning: applications of artificial intelligence to imaging and diagnosis
Nichols, J., Chan, H. H., & Baker, M. A. B. (2018) · Biophysical Reviews · doi:10.1007/s12551-018-0449-9 · Cited by 509
“These outperformed pathologists when time constraints were in place, and performed comparably to pathologists without time constraints (Ehteshami Bejnordi et al. 2017).”
BackgroundZhang, H., Meng, Y., Zhao, Y., Qiao, Y., Yang, X., Coupland, S. E., & Zheng, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01824 · Cited by 546
“To further explore the proposed instance probability derivation, we train a classic AB-MIL model, and generate the heatmaps of 5 sub-fields of 5 slides from CAMELYON-16.”
MethodInfluentialArtificial intelligence as the next step towards precision pathology
Ács, B., Rantalainen, M., & Hartman, J. (2020) · Journal of Internal Medicine · doi:10.1111/joim.13030 · Cited by 497
“The CAMYLEON16 challenge was the first major challenge on computer-aided diagnosis in histopathology using whole slide images [86].”
BackgroundMethodA multimodal generative AI copilot for human pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Zhao, M., Chow, A. K., Ikemura, K., Kim, A., Pouli, D., Patel, A., et al. (2024) · Nature · doi:10.1038/s41586-024-07618-3 · Cited by 461
Hwang, E. J., Park, S., Jin, K.-N., Kim, J. I., Choi, S. Y., Lee, J. H., Goo, J. M., Aum, J., Yim, J.-J., Cohen, J. G., et al. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.1095 · Cited by 447
Emerging role of deep learning‐based artificial intelligence in tumor pathology
Jiang, Y., Yang, M., Wang, S., Li, X., & Sun, Y. K. (2020) · 癌症：英文版 · doi:10.1002/cac2.12012 · Cited by 460
Rotation Equivariant CNNs for Digital Pathology
Veeling, B. S., Linmans, J., Winkens, J., Cohen, T., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00934-2_24 · Cited by 476
Deep learning for cardiovascular medicine: a practical primer
Krittanawong, C., Johnson, K. W., Rosenson, R. S., Wang, Z., Aydar, M., Baber, U., Min, J. K., Tang, W. H. W., Halperin, J. L., & Narayan, S. M. (2019) · European Heart Journal · doi:10.1093/eurheartj/ehz056 · Cited by 452
“Deep learning has been used to classify images in many medical specialties, 52–54 recently extended to cardiac imaging ( Table 2 ).”
MethodArtificial intelligence-based multi-omics analysis fuels cancer precision medicine
He, X., Liu, X., Zuo, F., Shi, H., & Jing, J. (2022) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2022.12.009 · Cited by 494
Bellemo, V., Lim, Z. W., Lim, G., Nguyen, Q. D., Xie, Y., Yip, M. Y. T., Hamzah, H., Ho, J., Lee, X. Q., Hsu, W., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30004-4 · Cited by 419
Neural network analysis of sleep stages enables efficient diagnosis of narcolepsy
Stephansen, J. B., Olesen, A. N., Olsen, M., Ambati, A., Leary, E. B., Moore, H. E., Carrillo, O., Lin, L., Han, F., Yan, H., et al. (2018) · Nature Communications · doi:10.1038/s41467-018-07229-3 · Cited by 455
Song, Z., Zou, S., Zhou, W., Huang, Y., Shao, L., Yuan, J., Gou, X., Jin, W., Wang, Z., Chen, X., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18147-8 · Cited by 444
Self-citation1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset
Litjens, G. J. S., Bándi, P., Bejnordi, B. E., Geessink, O. G. F., Balkenhol, M. C. A., Bult, P., Halilović, A., Hermsen, M., van de Loo, R., Vogels, R. J., et al. (2018) · GigaScience · doi:10.1093/gigascience/giy065 · Cited by 441
“For example, a comparison of algorithms competing in the CAMELYON16 challenge to pathologists in clinical practice was recently published [29].”
Self-citationRETRACTED ARTICLE: Artificial intelligence and machine learning in precision and genomic medicine
Quazi, S. (2022) · Medical Oncology · doi:10.1007/s12032-022-01711-1 · Cited by 466
RetractedDeep Learning Models for Histopathological Classification of Gastric and Colonic Epithelial Tumours
Iizuka, O., Kanavati, F., Kato, K., Rambeau, M., Arihiro, K., & Tsuneki, M. (2020) · Scientific Reports · doi:10.1038/s41598-020-58467-9 · Cited by 412
Liu, Y., Kohlberger, T., Norouzi, M., Dahl, G. E., Smith, J. L., Mohtashamian, A., Olson, N. H., Peng, L. H., Hipp, J., & Stumpe, M. C. (2018) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2018-0147-oa · Cited by 413
“pathologists without time constraint, and significantly more accurate than pathologists in a simulated (1 minute per slide) environment.(14) Computer algorithms may significantly improve a pathologist’s workflow.”
BackgroundMethodInfluentialPhysician Confidence in Artificial Intelligence: An Online Mobile Survey
Oh, S., Kim, J. H., Choi, S.-W., Lee, H. J., Hong, J., & Kwon, S. H. (2019) · Journal of Medical Internet Research · doi:10.2196/12422 · Cited by 387
Pathology image analysis using segmentation deep learning algorithms
Wang, S., Yang, D. M., Rong, R., Zhan, X., & Xiao, G. (2019) · The American Journal of Pathology · doi:10.1016/j.ajpath.2019.05.007 · Cited by 382
“For example, deep learning algorithms achieved performance comparable to pathologists in interpreting whole slide images for the detection of tumor regions and lymph node metastases.(10) Although this comparable performance may not generalize to all task domains, advanced methodology is anticipated to solve or aid in…”
MethodArtificial intelligence in oncology
Shimizu, H., & Nakayama, K. I. (2020) · Cancer Science · doi:10.1111/cas.14377 · Cited by 388
Li, J., Chen, J., Tang, Y., Wang, C., Landman, B. A., & Zhou, S. K. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102762 · Cited by 454
Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images
Chen, R. J., Lu, M. Y., Weng, W.-H., Chen, T. Y., Williamson, D. F., Manz, T., Shady, M., & Mahmood, F. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00398 · Cited by 398
“…occupies a tiny proportion of the total image ( i.e. - a needle-in-a-haystack problem such as differentiating normal tissue vs micro-metastases) [4, 3, 5, 45], or 2) spanning the entire composition of a WSI and dependent on the interactions of its components ( i.e. - a fine-grained visual…”
BackgroundState-of-the-Art Deep Learning in Cardiovascular Image Analysis
Litjens, G. J. S., Ciompi, F., Wolterink, J. M., de Vos, B. D., Leiner, T., Teuwen, J., & Išgum, I. (2019) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2019.06.009 · Cited by 399
Self-citationArtificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge
Wouter, B., Kimmo, K., Cameron, C. P.-H., Peter, S., Hans, P., Kunal, N., Yuannan, C., Steiner, D. F., van Boven, H., Robert, V., et al. (2022) · UTUPub (University of Turku) · Cited by 375
Artificial intelligence for clinical oncology
Kann, B. H., Hosny, A., & Aerts, H. J. W. L. (2021) · Cancer Cell · doi:10.1016/j.ccell.2021.04.002 · Cited by 399
HistoQC: An Open-Source Quality Control Tool for Digital Pathology Slides
Janowczyk, A. R., Zuo, R., Gilmore, H. L., Feldman, M. D., & Madabhushi, A. (2019) · JCO Clinical Cancer Informatics · doi:10.1200/cci.18.00157 · Cited by 357
Peng, Y., Dharssi, S., Chen, Q., Keenan, T. D. L., Agrón, E., Wong, W. T., Chew, E. Y., & Lu, Z. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.11.015 · Cited by 363
Breast cancer detection using deep learning: Datasets, methods, and challenges ahead
Din, N. M. U., Dar, R. A., Rasool, M., & Assad, A. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106073 · Cited by 379
Will artificial intelligence solve the human resource crisis in healthcare?
Meskó, B., Hetényi, G., & Győrffy, Z. (2018) · BMC Health Services Research · doi:10.1186/s12913-018-3359-4 · Cited by 360
Precision medicine in the era of artificial intelligence: implications in chronic disease management
Subramanian, M., Wojtusciszyn, A., Favre, L., Boughorbel, S., Shan, J., Letaief, K. B., Pitteloud, N., & Chouchane, L. (2020) · Journal of Translational Medicine · doi:10.1186/s12967-020-02658-5 · Cited by 362
Cen, L., Ji, J., Lin, J., Ju, S.-T., Lin, H.-J., Li, T.-P., Wang, Y., Yang, J.-F., Liu, Y.-F., Tan, S., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-25138-w · Cited by 368
Self-citationDigital Pathology: Advantages, Limitations and Emerging Perspectives
Jahn, S. W., Plass, M., & Moinfar, F. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9113697 · Cited by 372
“Examples include the Grand Challenge on BreAst Cancer Histology (BACH) [65], and the Cancer Metastases in Lymph Nodes Challenge [66].”
BackgroundUsing Digital Health Technology to Better Generate Evidence and Deliver Evidence-Based Care
Sharma, A., Harrington, R. A., McClellan, M., Turakhia, M. P., Eapen, Z. J., Steinhubl, S. R., Mault, J. R., Majmudar, M. D., Roessig, L., Chandross, K. J., et al. (2018) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.03.523 · Cited by 332
Chen, P.-H. C., Gadepalli, K., MacDonald, R., Liu, Y., Kadowaki, S., Nagpal, K., Kohlberger, T., Dean, J., Corrado, G. S., Hipp, J. D., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0539-7 · Cited by 342
Graham, S., Chen, H., Gamper, J., Dou, Q., Heng, P., Snead, D. R. J., Tsang, Y. W., & Rajpoot, N. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.12.001 · Cited by 352
Self-citationDeep Learning for Whole Slide Image Analysis: An Overview
Dimitriou, N., Arandjelović, O., & Caie, P. D. (2019) · Frontiers in Medicine · doi:10.3389/fmed.2019.00264 · Cited by 334
Artificial intelligence for digital and computational pathology
Song, A. H., Jaume, G., Williamson, D. F. K., Lu, M. Y., Vaidya, A., Miller, T. R., & Mahmood, F. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00096-8 · Cited by 356
Designing deep learning studies in cancer diagnostics
Kleppe, A., Skrede, O.-J., Raedt, S. D., Liestøl, K., Kerr, D. J., & Danielsen, H. E. (2021) · Nature reviews. Cancer · doi:10.1038/s41568-020-00327-9 · Cited by 323
Metadata
- Authors
- Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., Balkenhol, M., Geessink, O., Stathonikos, N., van Dijk, M. C., Bult, P., Beca, F., Beck, A. H., Wang, D., Khosla, A., Gargeya, R., Irshad, H., Zhong, A., Dou, Q., Li, Q., Chen, H., Lin, H.-J., Heng, P.-A., Haß, C., Bruni, E., Wong, Q., Halici, U., Öner, M. Ü., Cetin-Atalay, R., Berseth, M., Khvatkov, V., Vylegzhanin, A., Kraus, O., Shaban, M., Rajpoot, N., Awan, R., Sirinukunwattana, K., Qaiser, T., Tsang, Y.-W., Tellez, D., Annuscheit, J., Hufnagl, P., Valkonen, M., Kartasalo, K., Latonen, L., Ruusuvuori, P., Liimatainen, K., Albarqouni, S., Mungal, B., George, A., Demirci, S., Navab, N., Watanabe, S., Seno, S., Takenaka, Y., Matsuda, H., Ahmady Phoulady, H., Kovalev, V., Kalinovsky, A., Liauchuk, V., Bueno, G., Fernandez-Carrobles, M. M., Serrano, I., Deniz, O., Racoceanu, D., & Venâncio, R.
- Affiliations
- Diagnostic Image Analysis Group, Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
- Eindhoven University of Technology (NL)
- Department of Pathology, University Medical Center Utrecht, Utrecht, the Netherlands
- Radboud University Medical Center (NL)
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
- University Medical Center Utrecht (NL)
- Rijnstate Hospital, Arnhem, the Netherlands
- BeckLab, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts
- Beth Israel Deaconess Medical Center (US)
- PathAI (United States) (US)
- Harker School, San Jose, California
- Harvard University (US)
- Chinese University of Hong Kong (HK)
- Chinese University of Hong Kong, Hong Kong, China
- ExB Research and Development GmbH, Munich, Germany
- Munich Business School (DE)
- Department of Electrical and Electronics Engineering, Middle East Technical University, Ankara, Turkey
- Middle East Technical University (TR)
- Cancer System Biology Laboratory, Graduate School of Informatics, Middle East Technical University, Ankara, Turkey
- NLP LOGIX, Jacksonville, Florida
- Smart Imaging Technologies, Houston, Texas
- University of Toronto (CA)
- University of Warwick (GB)
- University Hospital Coventry (GB)
- Qatar University (QA)
- HTW Berlin - University of Applied Sciences (DE)
- Tampere University of Technology (FI)
- Tampere University (FI)
- Technical University of Munich (DE)
- The University of Osaka (JP)
- University of South Florida, Tampa, Florida
- National Academy of Sciences of Belarus (BY)
- University of Castilla-La Mancha (ES)
- Visilab, University of Castilla-La Mancha, Ciudad Real, Spain
- Inserm (FR)
- Sorbonne Université (FR)
- ORCID
- Veta, Mitko
- Karssemeijer, Nico
- Litjens, Geert
- Hermsen, Meyke
- Manson, Quirine F
- Balkenhol, Maschenka
- Geessink, Oscar
- Bult, Peter
- Beck, Andrew H
- Irshad, Humayun
- Zhong, Aoxiao
- Dou, Qi
- Li, Quanzheng
- Chen, Hao
- Heng, Pheng-Ann
- Öner, Mustafa Ümit
- Kraus, Oren
- Shaban, Muhammad
- Rajpoot, Nasir
- Awan, Ruqayya
- Sirinukunwattana, Korsuk
- Qaiser, Talha
- Tsang, Yee-Wah
- Tellez, David
- Annuscheit, Jonas
- Hufnagl, Peter
- Kartasalo, Kimmo
- Latonen, Leena
- Ruusuvuori, Pekka
- Liimatainen, Kaisa
- Albarqouni, Shadi
- Navab, Nassir
- Seno, Shigeto
- Matsuda, Hideo
- Kovalev, Vassili
- Liauchuk, Vitali
- Bueno, Gloria
- Serrano, Ismael
- Racoceanu, Daniel
- Venue
- JAMA
- Publisher
- American Medical Association (AMA)
- Published
- 2017-12-12
- Volume
- 318
- Issue
- 22
- Pages
- 2199
- ISSN
- 0002-9955 (ISSN-L)
- 0098-7484
- PMC
- PMC5820737
- OpenAlex
- W2772723798
- Semantic Scholar
- ba913e2c03ece1c75f0af4d16dd11c7ffbc6e3ba
- Cited by
- 3,293
- References
- 40
- Citations by year
- 2026: 301 · 2025: 388 · 2024: 430 · 2023: 414 · 2022: 429 · 2021: 457 · 2020: 421 · 2019: 321 · 2018: 119 · 2017: 4
- FWCI
- 141.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- other-oa
- Locations
- JAMA (publishedVersion, other-oa)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- HAL (submittedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- OpenMETU (Middle East Technical University) (submittedVersion, cc-by-nc-nd)
- TU/e Research Portal (submittedVersion)
- Qatar University QSpace (Qatar University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (submittedVersion)
- Funders
- European Commission (601040)
- Japan Society for the Promotion of Science (15K00403)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- lymph node metastasis, breast cancer, whole slide images, deep learning, computer-aided diagnosis, H&E staining, diagnostic accuracy, immunohistochemistry validation, breast cancer staging, lymph node staging, false positives, diagnostic confidence
- MeSH terms
- Machine Learning, Pathologists, Algorithms, Breast Neoplasms, Female, Humans, Lymphatic Metastasis, Pathology, Clinical, ROC Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
