Robustness in deep learning models for medical diagnostics: security and adversarial challenges towards robust AI applications
Open accessThe DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Abstract
The current study investigates the robustness of deep learning models for accurate medical diagnosis systems with a specific focus on their ability to maintain performance in the presence of adversarial or noisy inputs. We examine factors that may influence model reliability, including model complexity, training data quality, and hyperparameters; we also examine security concerns related to adversarial attacks that aim to deceive models along with privacy attacks that seek to extract sensitive information. Researchers have discussed various defenses to these attacks to enhance model robustness, such as adversarial training and input preprocessing, along with mechanisms like data augmentation and uncertainty estimation. Tools and packages that extend the reliability features of deep learning frameworks such as TensorFlow and PyTorch are also being explored and evaluated. Existing evaluation metrics for robustness are additionally being discussed and evaluated. This paper concludes by discussing limitations in the existing literature and possible future research directions to continue enhancing the status of this research topic, particularly in the medical domain, with the aim of ensuring that AI systems are trustworthy, reliable, and stable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507
A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
Lee, K., Lee, K., Lee, H., & Shin, J. (2018) · arXiv · doi:10.48550/arxiv.1807.03888 · Cited by 2,450
Understanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203
Ensembling neural networks: Many could be better than all
Zhou, Z.-H., Wu, J., & Tang, W. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(02)00190-x · Cited by 1,964
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Self-citationImproving predictive inference under covariate shift by weighting the log-likelihood function
Shimodaira, H. (2000) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(00)00115-4 · Cited by 1,678
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., et al. (2018) · arXiv · doi:10.48550/arxiv.1810.01943 · Cited by 1,581
Big data and machine learning algorithms for health-care delivery
Ngiam, K. Y., & Khor, I. W. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30149-4 · Cited by 1,424
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065
Feature denoising for improving adversarial robustness
Xie, C., Wu, Y., Maaten, L. V. D., Yuille, A. L., & He, K. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00059 · Cited by 940
Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility?
Naik, N., Hameed, B. M. Z., Shetty, D. K., Swain, D., Shah, M., Paul, R., Aggarwal, K., Ibrahim, S., Patil, V., Smriti, K., et al. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.862322 · Cited by 931
AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilović, A., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942287 · Cited by 884
Albahri, A. S., Duhaim, A. M., Fadhel, M. A., Alnoor, A., Baqer, N. S., Alzubaidi, L., Albahri, O. S., Alamoodi, A. H., Bai, J., Salhi, A., et al. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.03.008 · Cited by 716
Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges
Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-65965-3_28 · Cited by 677
Adversarial Robustness Toolbox v1.0.0
Nicolae, M.-I., Sinn, M., Tran, M. N., Buesser, B., Rawat, A., Wistuba, M., Zantedeschi, V., Baracaldo, N., Chen, B., Ludwig, H., et al. (2018) · arXiv · doi:10.48550/arxiv.1807.01069 · Cited by 594
Privacy-preserving artificial intelligence in healthcare: Techniques and applications
Khalid, N., Qayyum, A., Bilal, M., Al-Fuqaha, A., & Qadir, J. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106848 · Cited by 581
Recent Advances in Adversarial Training for Adversarial Robustness
Bai, T., Luo, J., Zhao, J., Wen, B., & Wang, Q. (2021) · arXiv · doi:10.48550/arxiv.2102.01356 · Cited by 490
Recent Advances in Adversarial Training for Adversarial Robustness
Bai, T., Luo, J., Zhao, J., Wen, B., & Wang, Q. (2021) · Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/591 · Cited by 490
Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge
Campello, V. M., Gkontra, P., Izquierdo, C., Martin-Isla, C., Sojoudi, A., Full, P. M., Maier-Hein, K., Zhang, Y., He, Z., Ma, J., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3090082 · Cited by 481
Harnessing the power of synthetic data in healthcare: innovation, application, and privacy
Giuffrè, M., & Shung, D. L. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00927-3 · Cited by 392
Explainable, trustworthy, and ethical machine learning for healthcare: A survey
Rasheed, K., Qayyum, A., Ghaly, M., Al-Fuqaha, A., Razi, A., & Qadir, J. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106043 · Cited by 372
On the Robustness of Interpretability Methods
Alvarez-Melis, D., & Jaakkola, T. S. (2018) · arXiv · doi:10.48550/arxiv.1806.08049 · Cited by 310
Ahmad, K., Maabreh, M., Ghaly, M., Khan, K., Qadir, J., & Al-Fuqaha, A. (2022) · Computer Science Review · doi:10.1016/j.cosrev.2021.100452 · Cited by 291
Čartolovni, A., Tomičić, A., & Lazić Mosler, E. (2022) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104738 · Cited by 267
Masud, M., Gaba, G. S., Alqahtani, S., Muhammad, G., Gupta, B. B., Kumar, P., & Ghoneim, A. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3047662 · Cited by 258
Ethics of using and sharing clinical imaging data for artificial intelligence: a proposed framework
Larson, D. B., Magnus, D. C., Lungren, M. P., Shah, N. H., & Langlotz, C. P. (2020) · Radiology · doi:10.1148/radiol.2020192536 · Cited by 199
Robustness and Explainability of Artificial Intelligence
(2020) · Joint Research Centre (European Commission) · doi:10.2760/57493 · Cited by 155
Fairness GAN: Generating datasets with fairness properties using a generative adversarial network
Sattigeri, P., Hoffman, S. C., Chenthamarakshan, V., & Varshney, K. R. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2945519 · Cited by 134
Alsarhan, A., Alauthman, M., Alshdaifat, E., Al-Ghuwairi, A.-R., & Al-Dubai, A. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-02963-x · Cited by 123
Explainable AI-based Alzheimer’s prediction and management using multimodal data
Jahan, S., Abu Taher, K., Kaiser, M. S., Mahmud, M., Rahman, M. S., Hosen, A. S. M. S., & Ra, I.-H. (2023) · PLOS ONE · doi:10.1371/journal.pone.0294253 · Cited by 122
Mok, T. C. W., & Chung, A. C. S. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-11723-8_7 · Cited by 95
Measure and Improve Robustness in NLP Models: A Survey
Wang, X., Wang, H., & Yang, D. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.339 · Cited by 90
Amini, M., Pedram, M. M., Moradi, A., & Ouchani, M. (2021) · Computational and Mathematical Methods in Medicine · doi:10.1155/2021/5514839 · Cited by 84
Amugongo, L. M., Kriebitz, A., Boch, A., & Lütge, C. (2025) · AI and Ethics · doi:10.1007/s43681-023-00331-3 · Cited by 68
Must-have Qualities of Clinical Research on Artificial Intelligence and Machine Learning
Koçak, B., Cuocolo, R., Santos, D. P. D., Stanzione, A., & Ugga, L. (2023) · Balkan Medical Journal · doi:10.4274/balkanmedj.galenos.2022.2022-11-51 · Cited by 64
A systematic collection of medical image datasets for deep learning
Li, J., Zhu, G., Hua, C., Feng, M., BasheerBennamoun, Li, P., Lu, X., Song, J., Shen, P., Xu, X., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.12864 · Cited by 61
Secure and Robust Machine Learning for Healthcare: A Survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2020) · arXiv · doi:10.48550/arxiv.2001.08103 · Cited by 59
Secure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59
Silva: Interactively Assessing Machine Learning Fairness Using Causality
Yan, J. N., Gu, Z., Lin, H., & Rzeszotarski, J. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376447 · Cited by 54
Blagec, K., Kraiger, J., Frühwirt, W., & Samwald, M. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104274 · Cited by 47
Domain Shifts in Machine Learning Based Covid-19 Diagnosis From Blood Tests
Roland, T., Böck, C., Tschoellitsch, T., Maletzky, A., Hochreiter, S., Meier, J., & Klambauer, G. (2022) · Journal of Medical Systems · doi:10.1007/s10916-022-01807-1 · Cited by 42
Ethical frameworks for machine learning in sensitive healthcare applications
Javed, H., Muqeet, H. A., Javed, T., Rehman, A. U., & Sadiq, R. (2024) · IEEE Access · doi:10.1109/access.2023.3340884 · Cited by 35
Self-citation
Cited by
RHAD: A Reinforced Heterogeneous Anomaly Detector for Robust Industrial Control System Security
Han, X., Niu, Y., Cao, Z., Zhou, D., & Liu, B. (2025) · Electronics · doi:10.3390/electronics14122440 · Cited by 5
El Arab, R. A., Mustafa, M. H., Almagharbeh, W. T., Saleem, N. H., Al Abdulmohsen, S., Boathab, R., & Bu Washl, M. (2026) · Healthcare · doi:10.3390/healthcare14111459 · Cited by 8
“Other reviews similarly highlighted persistent obstacles, including bias, opacity, weak accountability, implementation barriers, and limited real-world validation, and noted that continuously evolving systems create governance demands that extend beyond initial assessment [34,38].”
Metadata
- Authors
- Javed, H., El-Sappagh, S., & Abuhmed, T.
- Venue
- Artificial Intelligence Review
- Published
- 2024-11-08
- Volume
- 58
- Issue
- 1
- ISSN
- 0269-2821 (ISSN-L)
- 1573-7462
- OpenAlex
- W4404127725
- Cited by
- 180
- References
- 56
- FWCI
- 26.64
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.