Explainable multi-modal deep learning with cross-modal attention for diagnosis of dyssynergic defecation using abdominal X-ray images and symptom questionnaire
Open accessAbstract
Dyssynergic defecation (DD) is a type of functional constipation that requires a specialized test for diagnosis. However, these tests are only accessible in tertiary care because they require devices that are not available elsewhere. In this work, we present explainable multi-modal deep learning models that can pre-screen patients with DD, using affordable data accessible in small hospitals i.e. abdominal X-ray images and symptom questionnaires; the output classifies whether DD is present or not. To enhance the model’s performance, we apply cross-modal attention to help the model find meaningful interactions between the two modalities. A convolution block attention module (CBAM) is added to obtain more important semantic and spatial features from the images. Masking augmentation is implemented to ignore irrelevant backgrounds in images. Both explainable AI techniques like gradient-weighted class activation mapping (Grad-CAM) and deep shapley additive explanations (DeepSHAP) are also used to explain the important parts of images and the symptom data for each patient. In our experiments, all models are run on 3 patient-based bootstraps. Our model is compared with single-modal models and human experts. Results demonstrate that our multi-modal model outperforms the single-modal model and achieves the highest in terms of sensitivity, specificity, F1, and accuracy (87.37%, 77.01%, 82.17%, and 82.27%), respectively. In addition, our model outperforms human experts, which shows its ability to assist human experts in diagnosing DD. This model is a novel clinical tool that combines symptom and image data for a more accurate diagnosis of DD.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some papers studied in this review attempt to incorporate unimodal explainable AI methods, like Shapley values and Grad-CAM.
“Some papers studied in this review indeed attempt to incorporate (unimodal) explainable AI methods, like Shapley values (Kolk et al., 2024; Sangnark et al., 2024; Teuho et al., 2024; Xu et al., 2022a; Parvin et al., 2024) and Grad-CAM (Sangnark et al., 2024; Teuho et al., 2024).”
From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Sangnark et al., 2024
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939785 · Cited by 54,569 · see citing works
A Threshold Selection Method from Gray-Level Histograms
Otsu, N. (1979) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/tsmc.1979.4310076 · Cited by 43,748 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494 · see citing works
Automated detection of COVID-19 cases using deep neural networks with X-ray images
Öztürk, T., Talo, M., Yildirim, E. A., Baloğlu, U. B., Yıldırım, Ö., & Acharya, U. R. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103792 · Cited by 2,607 · see citing works
Attention mechanisms in computer vision: A survey
Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0271-y · Cited by 2,566 · cited 2× · see citing works
“Attention mechanisms are novel techniques that are widely applied to deep learning models for various tasks of computer vision i.e. image classification, object detection, and image segmentation [35].”
BackgroundMethodHuang, S.-C., Pareek, A., Seyyedi, S., Banerjee, I., & Lungren, M. P. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00341-z · Cited by 934 · see citing works
Autoencoders and their applications in machine learning: a survey
Berahmand, K., Daneshfar, F., Salehi, E. S., Li, Y., & Xu, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10662-6 · Cited by 666 · see citing works
Mechanisms, evaluation, and management of chronic constipation
Bharucha, A. E., & Lacy, B. E. (2020) · Gastroenterology · doi:10.1053/j.gastro.2019.12.034 · Cited by 655 · cited 1× · see citing works
“Nowadays, one of the most common gastrointestinal disorder symptoms is chronic constipation, which affects 10-15% of the worldwide population [1].”
BackgroundHeart Disease Identification Method Using Machine Learning Classification in E-Healthcare
Li, J., Haq, A. U., Din, S. U., Khan, J., Khan, A., & Saboor, A. (2020) · IEEE Access · doi:10.1109/access.2020.3001149 · Cited by 654 · see citing works
Machine learning-based prediction of COVID-19 diagnosis based on symptoms
Zoabi, Y., Deri-Rozov, S., & Shomron, N. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00372-6 · Cited by 530 · see citing works
Multimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511 · see citing works
An approach to the diagnosis and management of Rome IV functional disorders of chronic constipation
Aziz, I., Whitehead, W. E., Palsson, O. S., Törnblom, H., & Simrén, M. (2020) · Expert Review of Gastroenterology & Hepatology · doi:10.1080/17474124.2020.1708718 · Cited by 413 · cited 1× · see citing works
“According to Rome IV criteria, functional defecation disorder is a common cause of outlet dysfunction [3].”
BackgroundTabNet: Attentive Interpretable Tabular Learning
Arık, S. Ö., & Pfister, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16826 · Cited by 372 · see citing works
Diagnosis and Treatment of Dyssynergic Defecation
Rao, S. S. C., & Patcharatrakul, T. (2016) · Journal of Neurogastroenterology and Motility · doi:10.5056/jnm16060 · Cited by 285 · see citing works
Self-citationPacheco, A. G. C., & Krohling, R. A. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3062002 · Cited by 268 · see citing works
Multimodal Attention-Based Deep Learning for Alzheimer’s Disease Diagnosis
Golovanevsky, M., Eickhoff, C., & Singh, R. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac168 · Cited by 165 · see citing works
A multimodal transformer to fuse images and metadata for skin disease classification
Cai, G., Zhu, Y. M., Wu, Y., et al. (2022) · The Visual Computer · doi:10.1007/s00371-022-02492-4 · Cited by 162 · see citing works
Tao, R., Wei, Y., Jiang, X., Li, H., Qin, H., Wang, J., Ma, Y., Zhang, L., & Liu, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01074 · Cited by 152 · see citing works
AVNC: Attention-Based VGG-Style Network for COVID-19 Diagnosis by CBAM
Wang, S., Fernandes, S. L., Zhu, Z., et al. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3062442 · Cited by 139 · see citing works
Dai, G., Tian, Z., Fan, J., et al. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.108481 · Cited by 136 · see citing works
Multi-Modal Retinal Image Classification With Modality-Specific Attention Network
He, X., Deng, Y., Fang, L., & Peng, Q. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3059956 · Cited by 134 · see citing works
Zhang, Y., Zhang, Z., Zhang, X., & Wang, S. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.06.021 · Cited by 113 · see citing works
An Improved Densenet Deep Neural Network Model for Tuberculosis Detection Using Chest X-Ray Images
Huy, V. T. Q., & Lin, C. (2023) · IEEE Access · doi:10.1109/access.2023.3270774 · Cited by 113 · see citing works
Diverse Sample Generation: Pushing the Limit of Generative Data-Free Quantization
Qin, H., Ding, Y., Zhang, X., Wang, J., Liu, X., & Lu, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3272925 · Cited by 113 · see citing works
Tanzi, L., Vezzetti, E., Rodrigo Moreno, A., Aprato, A., Audisio, A., & Massè, A. (2020) · European Journal of Radiology · doi:10.1016/j.ejrad.2020.109373 · Cited by 110 · see citing works
Update on the Pathophysiology and Management of Anorectal Disorders
Patcharatrakul, T., & Rao, S. S. C. (2017) · Gut and Liver · doi:10.5009/gnl17172 · Cited by 108 · see citing works
Self-citationSunil, C. K., Jaidhar, C. D., & Patil, N. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120381 · Cited by 106 · see citing works
Multimodal fusion models for pulmonary embolism mortality prediction
Cahan, N., Klang, E., Marom, E. M., Soffer, S., Barash, Y., Burshtein, E., Konen, E., & Greenspan, H. K. (2023) · Scientific Reports · doi:10.1038/s41598-023-34303-8 · Cited by 62 · see citing works
Secure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59 · see citing works
Tanner, S. E., Chaudhry, A., Goraya, N., Badlani, R., Jehangir, A., Shahsavari, D., Malik, Z. A., & Parkman, H. P. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10092027 · Cited by 45 · see citing works
Schilcher, J., Nilsson, A., Andlid, O., & Eklund, A. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107704 · Cited by 41 · see citing works
Dyssynergic Defecation and Other Evacuation Disorders
Sharma, A., Herekar, A., Yan, Y., Karunaratne, T. B., & Rao, S. S. C. (2022) · Gastroenterology Clinics of North America · doi:10.1016/j.gtc.2021.10.004 · Cited by 38 · see citing works
TReC: Transferred ResNet and CBAM for Detecting Brain Diseases
Xiao, Y., Yin, H., Wang, S., & Zhang, Y. (2021) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2021.781551 · Cited by 36 · see citing works
Park, S., Khemani, D., Nelson, A. D., Eckert, D. J., & Camilleri, M. (2016) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2016.11.013 · Cited by 34 · see citing works
Kwon, G., Ryu, J., Oh, J., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-74653-1 · Cited by 29 · see citing works
Huang, Z.-H., Liu, Y., Wu, W.-J., & Huang, K.-W. (2023) · Bioengineering · doi:10.3390/bioengineering10080970 · Cited by 27 · see citing works
Enhanced text classification through an improved discrete laying chicken algorithm
Daneshfar, F., & Aghajani, M. J. (2024) · Expert Systems · doi:10.1111/exsy.13553 · Cited by 26 · see citing works
Artzi, M., Redmard, E., Tzemach, O., Zeltser, J., Gropper, O., Roth, J., Shofty, B., Kozyrev, D. A., Constantini, S., & Ben‐Sira, L. (2021) · IEEE Access · doi:10.1109/access.2021.3085771 · Cited by 22 · see citing works
Oba, Y., Tezuka, T., Sanuki, M., & Wagatsuma, Y. (2021) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa53316.2021.9564151 · Cited by 16 · see citing works
BiMatting: Efficient Video Matting via Binarization
Qin, H., Ke, L., Ma, X., et al. (2023) · neural information processing systems · doi:10.52202/075280-1876 · Cited by 14 · see citing works
The Use of Standardized Questions in Identifying Patients with Dyssynergic Defecation
Parker, C. H., TOMLINSON, G. A., Correia, A. J., et al. (2018) · Journal of the Canadian Association of Gastroenterology · doi:10.1093/jcag/gwy010 · Cited by 13 · see citing works
Levy, J., Navas, C. M., Chandra, J. A., Christensen, B. C., Vaickus, L. J., Curley, M. A., Chey, W. D., Baker, J. R., & Shah, E. D. (2022) · Digestive Diseases and Sciences · doi:10.1007/s10620-022-07759-3 · Cited by 12 · see citing works
Lembo, A. J. (2023) · Elsevier eBooks · doi:10.1016/b978-0-443-13911-6.00009-8 · Cited by 10 · see citing works
Deeb, A., Debow, A., Mansour, S., & Shkodyrev, V. P. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105285 · Cited by 10 · see citing works
Diagnostic performance of dynamic MR defecography in assessment of dyssynergic defecation
Thanaracthanon, P., Sasiwimonphan, K., Sunthornram, A., Harisinghani, M. G., & Chulroek, T. (2023) · Abdominal Radiology · doi:10.1007/s00261-023-04010-z · Cited by 7 · see citing works
Liu, Y., Lu, H.-P., & Lai, C.-H. (2022) · IEEE Access · doi:10.1109/access.2022.3158952 · Cited by 5 · see citing works
Poovongsaroj, S., Rattanachaisit, P., Patcharatrakul, T., et al. (2022) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse54890.2022.9836301 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Schouten, D., Nicoletti, G., Dille, B., Chia, C., Vendittelli, P., Schuurmans, M., Litjens, G., & Khalili, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103621 · Cited by 108 · see citing works
- Authors
- Sangnark, S., Rattanachaisit, P., Patcharatrakul, T., & Vateekul, P.
- Affiliations
- Chulalongkorn University (TH)
- Thai Red Cross Society (TH)
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 78132-78147
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4399282649
- Cited by
- 8
- References
- 51
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 1
- FWCI
- 1.18
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Research Council of Thailand (N42A640330)
- Rachadapiseksompoch Fund for the Center of Excellence in Neurogastroenterology and Motility, Faculty of Medicine, Chulalongkorn University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Gastroenterology › Gastrointestinal motility and disorders
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- dyssynergic defecation, functional constipation, symptom questionnaire, multimodal deep learning, cross-modal attention, convolution block attention module, Grad-CAM, DeepSHAP, explainable AI, prescreening, diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
