The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens
Open accessAbstract
In clinical scenarios, multi-specialist consultation could significantly benefit the diagnosis, especially for intricate cases. This inspires us to explore a “multi-expert joint diagnosis” mechanism to upgrade the existing “single expert” framework commonly seen in the current literature. To this end, we propose METransformer, a method to realize this idea with a transformer-based backbone. The key design of our method is the introduction of multiple learnable “expert” tokens into both the transformer encoder and decoder. In the encoder, each expert token interacts with both vision tokens and other expert tokens to learn to attend different image regions for image representation. These expert tokens are encouraged to capture complementary information by an orthogonal loss that minimizes their over-lap. In the decoder, each attended expert token guides the cross-attention between input words and visual tokens, thus influencing the generated report. A metrics-based expert voting strategy is further developed to generate the final report. By the multi-experts concept, our model enjoys the merits of an ensemble-based approach but through a manner that is computationally more efficient and supports more sophisticated interactions among experts. Experimental results demonstrate the promising performance of our proposed model on two widely used benchmarks. Last but not least, the framework-level innovation makes our work ready to incorporate advances on existing “single-expert” models to further improve its performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Such references can be removed to reduce hallucinations when prior studies are not available, 9,14,18 or used in conjunction with prior images to enable descriptions of change.17,19,20”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [14]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 14×
“Specifically, we compare METransformer with 5 state-of-the-art (SOTA) image captioning methods, including Show-tell [32], AdaAtt [24], Att2in [1], Transformer [7] and M2transformer [6].”
MethodInfluentialAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472 · cited 3×
“1, our encoder comprises a vision transformer (ViT) [9] encoder and an expert bilinear transformer encoder.”
MethodInfluentialSqueeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · cited 1×
“A squeeze-excitation operation [10] to B mid is performed to obtain channel-wise attention Bilinear Encoder Layer.”
MethodPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“We implement our model using Pytorch [27] and Pytorch-lightning library 2 with two NVIDIA GeForce RTX 3090 GPU cards.”
MethodLayer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762 · cited 2×
“Layer-norm(LN) [2] is applied before every block and residual connections are applied after every block.”
BackgroundMethod(2004) · Cited by 9,718
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Research works printed but not linked: 1
- su, 2019, Vl-bert: Pre-training of generic visuallinguistic representations, ArXiv Preprint
Cited by
Sorted by citations
R2GenGPT: Radiology Report Generation with frozen LLMs
Wang, Z., Liu, L., Wang, L., & Zhou, L. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100033 · Cited by 253
Self-citationExplainable Deep Learning Methods in Medical Image Classification: A Survey
Patrício, C., Neves, J. C., & Teixeira, L. F. (2023) · ACM Computing Surveys · doi:10.1145/3625287 · Cited by 207
“Table 3 conveys the results in terms of a set of NLP evaluation metrics (BLEU score, ROUGE, METEOR, and CIDEr) for each of the selected methods.”
BackgroundPromptMRG: Diagnosis-Driven Prompts for Medical Report Generation
Jin, H., Che, H., Lin, Y., & Chen, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28038 · Cited by 181
Automated Radiology Report Generation: A Review of Recent Advances
Sloan, P., Clatworthy, P. L., Simpson, E., & Mirmehdi, M. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3408456 · Cited by 106
“Chen et al. [51]’s method has been utilised to assess an ARRG models performance by various researchers [11], [34], [49], [50], [67], [103], [123], [129].”
InfluentialVision-Language Models in medical image analysis: From simple fusion to general large models
Li, X., Li, L., Jiang, Y., Wang, H., Qiao, X., Feng, T., Luo, H., & Zhao, Y. J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102995 · Cited by 93
Metadata
- Authors
- Wang, Z., Liu, L., Wang, L., & Zhou, L.
- Affiliations
- The University of Sydney (AU)
- The University of Adelaide (AU)
- University of Wollongong (AU)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 11558-11567
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386076536
- Semantic Scholar
- 4d90382730eefbe8639e64f6855f206657758cc7
- Cited by
- 184
- References
- 51
- Citations by year
- 2026: 58 · 2025: 82 · 2024: 37 · 2023: 7
- FWCI
- 19.62
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- Research Online (University of Wollongong) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- radiology report generation, vision-language transformers, cross-attention
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
