The DOI matches its Crossref record; 79 of the 80 references it cites were found in a registry or an index.
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
Open accessAbstract
Self-supervised learning in vision-language processing (VLP) exploits semantic alignment between imaging and text modalities. Prior work in biomedical VLP has mostly relied on the alignment of single image and report pairs even though clinical notes commonly refer to prior images. This does not only introduce poor alignment between the modalities but also a missed opportunity to exploit rich self-supervision through existing temporal content in the data. In this work, we explicitly account for prior images and reports when available during both training and fine-tuning. Our approach, named BioViL-T, uses a CNN-Transformer hybrid multi-image encoder trained jointly with a text model. It is designed to be versatile to arising challenges such as pose variations and missing input images across time. The resulting model excels on downstream tasks both in single- and multi-image setups, achieving state-of-the-art (SOTA) performance on (I) progression classification, (II) phrase grounding, and (III) report generation, whilst offering consistent improvements on disease classification and sentence-similarity tasks. We release a novel multi-modal temporal benchmark dataset, MS-CXR-T, to quantify the quality of vision-language representations in terms of temporal semantics. Our experimental results show the advantages of incorporating prior images and reports to make most use of the data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Comparison to previous imaging studies is crucial for tracking the development of disease or impact of treatment, and references to prior studies are frequent in radiology reporting. 16,17”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Such references can be removed to reduce hallucinations when prior studies are not available, 9,14,18 or used in conjunction with prior images to enable descriptions of change.17,19,20”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“Comparison to previous imaging studies is crucial for tracking the development of disease or impact of treatment, and references to prior studies are frequent in radiology reporting. 16,17”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“It is not possible to quantitatively compare to models trained to generate other sections, such as Impression 17 or both Findings and Impression together.15,58”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“It is not possible to quantitatively compare to models trained to generate other sections, such as Impression 17 or both Findings and Impression together.15,58”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [58]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Empirically, in the MIMIC-CXR dataset, when the Comparison section is equivalent to 'No comparison available', references to prior studies are rarely observed, in contrast to the background rate of 40% in the full MIMIC-CXR dataset.17”
From MAIRA-2: Grounded Radiology Report Generation (2024) · cites this paper as [17]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
Layer NormalizationPreprint
Ba, J., Kiros, J., & Hinton, G. E. (2016) · arXiv.org · doi:10.48550/arxiv.1607.06450 · Cited by 13,762
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
(2004) · Cited by 9,718
Representation Learning with Contrastive Predictive Coding
Oord, A. V. D., Li, Y., & Vinyals, O. (2018) · arXiv · doi:10.48550/arxiv.1807.03748 · Cited by 6,242 · cited 3×
“…from image–text pairs has enabled the development of flexible general-purpose vision–language models both in the general domain [40, 53, 77] and for specialised domains such as biomedicine and radiology [9, 32, 81] often use only a single image for contrastive learning (e.g., InfoNCE [49]).”
BackgroundAvants, B., Epstein, C. L., Grossman, M., & Gee, J. C. (2007) · Medical Image Analysis · doi:10.1016/j.media.2007.06.004 · Cited by 5,924
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,668
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision
Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q. V., Sung, Y., Li, Z., & Duerig, T. (2021) · arXiv · doi:10.48550/arxiv.2102.05918 · Cited by 1,977
Fully Convolutional Siamese Networks for Change Detection
Daudt, R. C., Saux, B. L., & Boulch, A. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451652 · Cited by 1,872
Li, J., Li, D., Xiong, C., & Hoi, S. (2022) · arXiv · doi:10.48550/arxiv.2201.12086 · Cited by 1,741
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
CoCa: Contrastive Captioners are Image-Text Foundation ModelsPreprint
Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., & Wu, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.01917 · Cited by 1,413
Preparing a collection of radiology examinations for distribution and retrieval
Demner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, L., Antani, S., Thoma, G. R., & McDonald, C. J. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv080 · Cited by 1,164
Finetuned Language Models Are Zero-Shot Learners
Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Li, J., Selvaraju, R. R., Gotmare, A. D., Joty, S., Xiong, C., & Hoi, S. (2021) · arXiv · doi:10.48550/arxiv.2107.07651 · Cited by 1,078
Quantifying Attention Flow in Transformers
Abnar, S., & Zuidema, W. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.385 · Cited by 995
End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++
Peng, D., Zhang, Y., & Guan, H. (2019) · Remote Sensing · doi:10.3390/rs11111382 · Cited by 805
VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-ExpertsPreprint
Bao, H., Wang, W.-H., Dong, L., Liu, Q., Mohammed, O., Aggarwal, K., Som, S., Piao, S.-H., & Wei, F. (2021) · Neural Information Processing Systems · doi:10.48550/arxiv.2111.02358 · Cited by 803
Self-citationLowekamp, B., Chen, D. T., Ibáñez, L., & Blezek, D. J. (2013) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2013.00045 · Cited by 797
Generating Wikipedia by Summarizing Long SequencesPreprint
Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, Ł., & Shazeer, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10198 · Cited by 784
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782
Optimization of mutual information for multiresolution image registration
Unser, M. A., & Thévenaz, P. (2000) · IEEE Transactions on Image Processing · doi:10.1109/83.887976 · Cited by 694
Generating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693
FLAVA: A Foundational Language And Vision Alignment Model
Singh, A., Hu, R., Goswami, V., Couairon, G., Galuba, W., Rohrbach, M., & Kiela, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01519 · Cited by 678
Deep Learning Predicts Lung Cancer Treatment Response from Serial Medical Imaging
Xu, Y., Hosny, A., Zeleznik, R., Parmar, C., Coroller, T. P., Franco, I. “., Mak, R. H., & Aerts, H. J. W. L. (2019) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-2495 · Cited by 658
Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges
Shi, W., Zhang, M., Zhang, R., Chen, S., & Zhan, Z. (2020) · Remote Sensing · doi:10.3390/rs12101688 · Cited by 629
FILIP: Fine-grained Interactive Language-Image Pre-TrainingPreprint
Yao, L., Huang, R., Hou, L., Lu, G., Niu, M., Xu, H., Liang, X., Li, Z., Jiang, X., & Xu, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07783 · Cited by 595
Huang, S.-C., Shen, L., Lungren, M. P., & Yeung, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00391 · Cited by 489
Self-citationContrastive Learning of Medical Visual Representations from Paired Images and TextPreprint
Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., & Langlotz, C. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00747 · Cited by 465
Debiased Contrastive LearningPreprint
Chuang, C.-Y., Robinson, J., Lin, Y.-C., Torralba, A. B., & Jegelka, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00224 · Cited by 425
VLMo: Unified Vision-Language Pre-Training with Mixture-Of-Modality-Experts
Bao, H., Wang, W., Dong, L., Liu, Q., Mohammed, O. K., Aggarwal, K., Som, S., Piao, S., & Wei, F. (2022) · neural information processing systems · doi:10.52202/068431-2384 · Cited by 420
Self-citationRethinking and Improving Relative Position Encoding for Vision Transformer
Wu, K., Peng, H., Chen, M., Fu, J., & Chao, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00988 · Cited by 419
An Empirical Study of Training End-to-End Vision-and-Language Transformers
Dou, Z.-Y., Xu, Y., Gan, Z., Wang, J., Wang, S., Wang, L., Zhu, C., Zhang, P., Yuan, L., Peng, N., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01763 · Cited by 401
CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison
Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301590 · Cited by 392
Self-citationVirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01101 · Cited by 376
How Do Vision Transformers Work?
Park, N., & Kim, S. (2022) · arXiv · doi:10.48550/arxiv.2202.06709 · Cited by 356 · cited 18×
“Table 6 shows that the proposed framework greatly increases the sensitivity of sentence embeddings to temporal content whilst better capturing the static content (RadNLI).”
BackgroundInfluentialShih, G., Wu, C. C., Halabi, S. S., Kohli, M. D., Prevedello, L. M. S., Cook, T. S., Sharma, A., Amorosa, J. K., Arteaga, V. A., Galperin-Aizenberg, M., et al. (2019) · Radiology Artificial Intelligence · doi:10.1148/ryai.2019180041 · Cited by 341
Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing
Boecking, B., Usuyama, N., Bannur, S., Castro, D. C., Schwaighofer, A., Hyland, S., Wetscherek, M., Naumann, T., Nori, A., Alvarez-Valle, J., et al. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20059-5_1 · Cited by 317 · cited 10×
“Despite this, the existing VLP work to date considers alignment between only single images and reports [9,32,46,81], going so far as to remove temporal content from reports in training data to prevent ‘hallucinations’ in downstream report generation [54].”
BackgroundMethodInfluentialSelf-citationSelf-supervised Co-training for Video Representation Learning
Han, T., Xie, W., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.09709 · Cited by 292
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A., & Lungren, M. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.117 · Cited by 262
Self-citationMoon, J. H., Lee, H., Shin, W., Kim, Y., & Choi, E. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3207502 · Cited by 238
Chest x-ray findings and temporal lung changes in patients with COVID-19 pneumonia
Rousan, L. A., El-Obeid, E., Karrar, M., & Khader, Y. S. (2020) · BMC Pulmonary Medicine · doi:10.1186/s12890-020-01286-5 · Cited by 210
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation
Miura, Y., Zhang, Y., Tsai, E., Langlotz, C., & Jurafsky, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.416 · Cited by 149
Exploring the Limits of Out-of-Distribution Detection
Fort, S., Ren, J., & Lakshminarayanan, B. (2021) · arXiv · doi:10.48550/arxiv.2106.03004 · Cited by 142
Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP
Esmaeilpour, S., Liu, B., Robertson, E., & Shu, L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i6.20610 · Cited by 134
Durrleman, S., Pennec, X., Trouvé, A., Braga, J., Gerig, G., & Ayache, N. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0592-x · Cited by 125
Biomedical and clinical English model packages for the Stanza Python NLP library
Zhang, Y., Zhang, Y., Qi, P., Manning, C. D., & Langlotz, C. P. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab090 · Cited by 112
Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A. Y., & Lungren, M. P. (2020) · arXiv · doi:10.48550/arxiv.2004.09167 · Cited by 103
Self-citationBroaden Your Views for Self-Supervised Video Learning
Recasens, A., Luc, P., Alayrac, J.-B., Wang, L., Strub, F., Tallec, C., Malinowski, M., Pătrăucean, V., Altché, F., Valko, M., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00129 · Cited by 98
Clinical-BERT: Vision-Language Pre-training for Radiograph Diagnosis and Reports Generation
Yan, B., & Pei, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20204 · Cited by 94
Robust Contrastive Learning against Noisy Views
Chuang, C.-Y., Hjelm, R. D., Wang, X., Vineet, V., Joshi, N., Torralba, A. B., Jegelka, S., & Song, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01617 · Cited by 90
RadGraph: Extracting Clinical Entities and Relations from Radiology Reports
Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Duong, D. N., Bui, T., Chambon, P., Zhang, Y., Lungren, M. P., Ng, A. Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.14463 · Cited by 66
Self-citationInfluence of prior radiologic information on the interpretation of radiographic examinations
Aideyan, U. O., Berbaum, K. S., & Smith, W. L. (1995) · Academic Radiology · doi:10.1016/s1076-6332(05)80165-5 · Cited by 66
Vu, Y. N. T., Wang, R., Balachandar, N., Liu, C., Ng, A. Y., & Rajpurkar, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10663 · Cited by 63
Wang, C., Rimner, A., Hu, Y., Tyagi, N., Jiang, J., Yorke, E. D., Riyahi, S., Mageras, G. S., Deasy, J. O., & Zhang, P. (2019) · Medical Physics · doi:10.1002/mp.13765 · Cited by 50
COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image PredictionPreprint
Sriram, A., Muckley, M. J., Sinha, K., Shamout, F. E., Pineau, J., Geras, K. J., Azour, L., Aphinyanaphongs, Y., Yakubova, N., & Moore, W. H. (2021) · PubMed · doi:10.48550/arxiv.2101.04909 · Cited by 50
Longitudinal Detection of Radiological Abnormalities with Time-Modulated LSTM
Santeramo, R., Withey, S., & Montana, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00889-5_37 · Cited by 38
Long Short View Feature Decomposition via Contrastive Video Representation Learning
Behrmann, N., Fayyaz, M., Gall, J., & Noroozi, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00911 · Cited by 30
CheXRelNet: An Anatomy-Aware Model for Tracking Longitudinal Relationships Between Chest X-Rays
Karwande, G., Mbakwe, A. B., Wu, J. T.-Y., Celi, L. A., Moradi, M., & Lourentzou, I. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16431-6_55 · Cited by 25
Contrastive Learning of Global and Local Video Representations
Ma, S., Zeng, Z., McDuff, D., & Song, Y. (2021) · neural information processing systems · Cited by 23
Data Efficient Masked Language Modeling for Vision and Language
Bitton, Y., Elhadad, M., Stanovsky, G., & Schwartz, R. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.259 · Cited by 20
A nonparametric-based rib suppression method for chest radiographs
Lee, J.-S., Wang, J.-W., Wu, H.-H., & Yuan, M.-Z. (2012) · Computers & Mathematics with Applications · doi:10.1016/j.camwa.2012.03.084 · Cited by 18
Ramesh, V., Chi, N. A., & Rajpurkar, P. (2022) · arXiv · doi:10.48550/arxiv.2210.06340 · Cited by 17
Longitudinal Change Detection on Chest X-rays Using Geometric Correlation Maps
Oh, D. Y., Kim, J., & Lee, K. J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_83 · Cited by 14
Time Is MattEr: Temporal Self-supervision for Video TransformersPreprint
Yun, S., Kim, J., Han, D., Song, H., Ha, J.-W., & Shin, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.09067 · Cited by 12
Leveraging Time Irreversibility with Order-Contrastive Pre-trainingPreprint
Agrawal, M., Lang, H., Offin, M. D., Gazit, L., & Sontag, D. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02599 · Cited by 4
Zeng, D., Kheir, J. N., Zeng, P., & Shi, Y. (2021) · International Conference on Computer Aided Design · doi:10.1109/iccad51958.2021.9643443 · Cited by 4
Research works printed but not linked: 6
- brown, 2020, Language models are few-shot learners Advances in neural information processing systems, 33, 1877
- boecking, 2022, MS-CXR: Making the most of text semantics to improve biomedical vision-language processing (version 0.1), PhysioNet
- 2020, American College of Radiology (ACR), ACR practice guideline for communication of diagnostic imaging findings
- johnson, 2019, MIMIC-CXR database (version 2 0 0) PhysioNet
- wu, 2021, Chest imagenome dataset (version 1.0.0), PhysioNet
- endo, 2021, Retrieval-based chest x-ray report generation using a pre-trained contrastive language-image model, Machine Learning for Health, 209
Cited by
Sorted by citations
Advances in medical image analysis with vision Transformers: A comprehensive review
Azad, R., Kazerouni, A., Heidari, M., Aghdam, E. K., Molaei, A., Jia, Y., Jose, A., Roy, R., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103000 · Cited by 557
Towards Generalist Biomedical AI
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang, P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I., et al. (2024) · NEJM AI · doi:10.1056/aioa2300138 · Cited by 483
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443
Self-citationVision-language models for medical report generation and visual question answering: a review
Hartsock, I., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1430984 · Cited by 298
Collaboration between clinicians and vision–language models in radiology report generation
Tanno, R., Barrett, D. G. T., Sellergren, A., Ghaisas, S., Dathathri, S., See, A., Welbl, J., Lau, C., Tu, T., Azizi, S., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03302-1 · Cited by 186
Metadata
- Authors
- Bannur, S., Hyland, S., Liu, Q., Pérez-García, F., Ilse, M., Castro, D. C., Boecking, B., Sharma, H., Bouzid, K., Thieme, A., Schwaighofer, A., Wetscherek, M., Lungren, M. P., Nori, A., Alvarez-Valle, J., & Oktay, O.
- Affiliations
- Microsoft (United States) (US)
- Microsoft Health Futures
- ORCID
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 15016-15027
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386075916
- Cited by
- 227
- References
- 80
- Citations by year
- 2026: 93 · 2025: 85 · 2024: 40 · 2023: 9
- FWCI
- 24.28
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- temporal structure, self-supervised learning, clinical reports, phrase grounding, report generation, chest radiography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
