CoChat

Integration of Artificial Intelligence Decision Aids to Reduce Workload and Enhance Efficiency in Thyroid Nodule Management

Open access
Tong, Wen-Juan, Wu, Shao-Hong, Cheng, Mei-Qing, Huang, Hui, Liang, Jin-Yu, Li, Chao-Qun, Guo, Huan-Ling, He, Dan-Ni, Liu, Yi-Hao, Xiao, Han, Hu, Hang-Tong, Ruan, Si-Min, Li, Ming-De, Lu, Ming-De, Wang, Wei · 2023 · JAMA Network Open
Cited by 52

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

Abstract

Importance: To optimize the integration of artificial intelligence (AI) decision aids and reduce workload in thyroid nodule management, it is critical to incorporate personalized AI into the decision-making processes of radiologists with varying levels of expertise. Objective: To develop an optimized integration of AI decision aids for reducing radiologists’ workload while maintaining diagnostic performance compared with traditional AI-assisted strategy. Design, Setting, and Participants: In this diagnostic study, a retrospective set of 1754 ultrasonographic images of 1048 patients with 1754 thyroid nodules from July 1, 2018, to July 31, 2019, was used to build an optimized strategy based on how 16 junior and senior radiologists incorporated AI-assisted diagnosis results with different image features. In the prospective set of this diagnostic study, 300 ultrasonographic images of 268 patients with 300 thyroid nodules from May 1 to December 31, 2021, were used to compare the optimized strategy with the traditional all-AI strategy in terms of diagnostic performance and workload reduction. Data analyses were completed in September 2022. Main Outcomes and Measures: The retrospective set of images was used to develop an optimized integration of AI decision aids for junior and senior radiologists based on the selection of AI-assisted significant or nonsignificant features. In the prospective set of images, the diagnostic performance, time-based cost, and assisted diagnosis were compared between the optimized strategy and the traditional all-AI strategy. Results: The retrospective set included 1754 ultrasonographic images from 1048 patients (mean [SD] age, 42.1 [13.2] years; 749 women [71.5%]) with 1754 thyroid nodules (mean [SD] size, 16.4 [10.6] mm); 748 nodules (42.6%) were benign, and 1006 (57.4%) were malignant. The prospective set included 300 ultrasonographic images from 268 patients (mean [SD] age, 41.7 [14.1] years; 194 women [72.4%]) with 300 thyroid nodules (mean [SD] size, 17.2 [6.8] mm); 125 nodules (41.7%) were benign, and 175 (58.3%) were malignant. For junior radiologists, the ultrasonographic features that were not improved by AI assistance included cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, and nodules smaller than 5 mm, whereas for senior radiologists the features that were not improved by AI assistance were cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, very hypoechoic nodules, nodules taller than wide, lobulated or irregular nodules, and extrathyroidal extension. Compared with the traditional all-AI strategy, the optimized strategy was associated with increased mean task completion times for junior radiologists (reader 11, from 15.2 seconds [95% CI, 13.2-17.2 seconds] to 19.4 seconds [95% CI, 15.6-23.3 seconds]; reader 12, from 12.7 seconds [95% CI, 11.4-13.9 seconds] to 15.6 seconds [95% CI, 13.6-17.7 seconds]), but shorter times for senior radiologists (reader 14, from 19.4 seconds [95% CI, 18.1-20.7 seconds] to 16.8 seconds [95% CI, 15.3-18.3 seconds]; reader 16, from 12.5 seconds [95% CI, 12.1-12.9 seconds] to 10.0 seconds [95% CI, 9.5-10.5 seconds]). There was no significant difference in sensitivity (range, 91%-100%) or specificity (range, 94%-98%) between the 2 strategies for readers 11 to 16. Conclusions and Relevance: This diagnostic study suggests that an optimized AI strategy in thyroid nodule management may reduce diagnostic time-based costs without sacrificing diagnostic accuracy for senior radiologists, while the traditional all-AI strategy may still be more beneficial for junior radiologists.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamanetworkopen.2023.13674

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 52

Metadata

Authors
Tong, W.-J., Wu, S.-H., Cheng, M.-Q., Huang, H., Liang, J.-Y., Li, C.-Q., Guo, H.-L., He, D.-N., Liu, Y.-H., Xiao, H., Hu, H.-T., Ruan, S.-M., Li, M.-D., Lu, M.-D., & Wang, W.
Venue
JAMA Network Open
Published
2023-05-16
Volume
6
Issue
5
Pages
e2313674
ISSN
2574-3805 (ISSN-L)
OpenAlex
W4376641653
Cited by
52
References
8
FWCI
8.60
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.