CoChat
Sign Up
PreprintCited by 5Multimodal Machine Learning ApplicationsArtificial Intelligence in Healthcare and EducationGenerative Adversarial Networks and Image Synthesis

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Vision-Language Generative Model for View-Specific Chest X-ray Generation

Open access
Lee, Hyungyung, Lee, Da Young, Kim, Wonjae, Kim, Jin-Hwa, Kim, Tackeun, Kim, Jihang, Sunwoo, Leonard, Choi, Edward·2023·arXiv
DOI 10.48550/arxiv.2302.12172arXiv 2302.12172
Open this paper in CoChat — free accountOpen PDF

Abstract

Synthetic medical data generation has opened up new possibilities in the healthcare domain, offering a powerful tool for simulating clinical scenarios, enhancing diagnostic and treatment quality, gaining granular medical knowledge, and accelerating the development of unbiased algorithms. In this context, we present a novel approach called ViewXGen, designed to overcome the limitations of existing methods that rely on general domain pipelines using only radiology reports to generate frontal-view chest X-rays. Our approach takes into consideration the diverse view positions found in the dataset, enabling the generation of chest X-rays with specific views, which marks a significant advancement in the field. To achieve this, we introduce a set of specially designed tokens for each view position, tailoring the generation process to the user's preferences. Furthermore, we leverage multi-view chest X-rays as input, incorporating valuable information from different views within the same study. This integration rectifies potential errors and contributes to faithfully capturing abnormal findings in chest X-ray generation. To validate the effectiveness of our approach, we conducted statistical analyses, evaluating its performance in a clinical efficacy metric on the MIMIC-CXR dataset. Also, human evaluation demonstrates the remarkable capabilities of ViewXGen, particularly in producing realistic view-specific X-rays that closely resemble the original images.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2302.12172

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Rethinking Attention with Performers

    Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · cited 6×

    “To achieve these, we introduce a set of special tokens tailored to each view position, including posterior-anterior (PA), anterior-posterior (AP), and lateral views, and employ a simplified architectural design by combining VQ-GAN (Esser et al., 2021) and Performer (Choromanski et al., 2020), which is an efficient Tra…”
    MethodInfluential

Flagged first, then by citations

Research works printed but not linked: 24

  1. Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier Van der Sluijs, Ma lgorzata Po lacin, Juan Manuel Zambrano Chaves, Tan- ishq Mathew Abraham, Shivanshu Purohit, Cur- tis P Langlotz, and Akshay Chaudhari. Roent- gen: Vision-language foundation model for chest x-ray generation. arXiv preprint arXiv:2211.12737, 2022a. · arXiv
  2. Pierre Chambon, Christian Bluethgen, Curtis P Langlotz, and Akshay Chaudhari. Adapting pretrained vision-language foundational models to medical imaging domains. arXiv preprint arXiv:2210.04133, 2022b. · arXiv
  3. Joseph Paul Cohen, Joseph D Viviano, Paul Bertin, Paul Morrison, Parsa Torabian, Matteo Guarrera, Matthew P Lungren, Akshay Chaudhari, Rupert Brooks, Mohammad Hashir, et al. Torchxrayvision: A library of chest x-ray datasets and models. In International Conference on Medical Imaging with Deep Learning, pages 231-249. PMLR, 2022.
  4. Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xi- aodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. Unified language model pre- training for natural language understanding and generation. Advances in Neural Information Pro- cessing Systems, 32, 2019.
  5. Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image syn- thesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873-12883, 2021.
  6. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adver- sarial networks. Communications of the ACM, 63 (11):139-144, 2020.
  7. Jonathan Ho, Ajay Jain, and Pieter Abbeel. De- noising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840- 6851, 2020.
  8. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degen- eration. arXiv preprint arXiv:1904.09751, 2019. · arXiv
  9. Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected con- volutional networks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 4700-4708, 2017.
  10. Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yi- fan Yu, Silviana Ciurea-Ilcus, Chris Chute, Hen- rik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radio- graph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 590-597, 2019.
  11. Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. Mimic-cxr, a de-identified publicly available database of chest radiographs with free- text reports. Scientific data, 6(1):1-8, 2019.
  12. Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In Inter- national Conference on Learning Representations, 2020.
  13. Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, and Nigel Collier. Self-alignment pretraining for biomedical entity representations. arXiv preprint arXiv:2010.11784, 2020. · arXiv
  14. Jong Hak Moon, Hyungyung Lee, Woncheol Shin, Young-Hak Kim, and Edward Choi. Multi-modal understanding and generation for medical images and text via vision-language pre-training. IEEE Journal of Biomedical and Health Informatics, 26 (12):6070-6080, 2022.
  15. Kai Packhäuser, Lukas Folle, Florian Thamm, and Andreas Maier. Generation of anonymous chest ra- diographs using latent diffusion models for training thoracic abnormality classification systems. arXiv preprint arXiv:2211.01323, 2022. · arXiv
  16. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic eval- uation of machine translation. In Proceedings of the 40th annual meeting of the Association for Compu- tational Linguistics, pages 311-318, 2002.
  17. Elizabeth Puddy and Catherine Hill. Interpretation of the chest radiograph. Continuing Education in Anaesthesia, Critical Care and Pain, 7(3):71-75, 2007.
  18. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022a. · arXiv
  19. Vignav Ramesh, Nathan A Chi, and Pranav Ra- jpurkar. Improving radiology report generation systems by removing hallucinated references to non-existent priors. In Machine Learning for Health, pages 456-473.
  20. PMLR, 2022b.
  21. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
  22. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022. · arXiv
  23. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  24. Changhan Wang, Kyunghyun Cho, and Jiatao Gu. Neural machine translation with byte-level sub- words. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9154-9160, 2020.

Cited by

Sorted by citations

Metadata

Authors
Lee, H., Lee, D. Y., Kim, W., Kim, J.-H., Kim, T., Kim, J., Sunwoo, L., & Choi, E.
Affiliations
  • Korea Advanced Institute of Science and Technology (KR)
  • Seoul National University (KR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2302.12172
arXiv
2302.12172
OpenAlex
W7202010457
Cited by
5
References
1
Citations by year
2025: 1 · 2024: 4
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • KAIST Institutional Repository (KAIST) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Research Foundation
  • Korea Health Industry Development Institute
  • National Research Foundation of Korea (2019-0-00075, NRF-2020H1D3A2A03100945)
  • Ministry of Science and ICT, South Korea (2019-0-00075, No.2019-0-00075)
  • Samsung (IO201211-08109-01)
  • Institute for Information and Communications Technology Promotion (2019-0-00075)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords
radiology reports, MIMIC-CXR, synthetic health data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review