The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions
Open accessAbstract
Increasing quality and performance of artificial intelligence (AI) in general and machine learning (ML) in particular is followed by a wider use of these approaches in everyday life. As part of this development, ML classifiers have also gained more importance for diagnosing diseases within biomedical engineering and medical sciences. However, many of those ubiquitous high-performing ML algorithms reveal a black-box-nature, leading to opaque and incomprehensible systems that complicate human interpretations of single predictions or the whole prediction process. This puts up a serious challenge on human decision makers to develop trust, which is much needed in life-changing decision tasks. This paper is designed to answer the question how expert companion systems for decision support can be designed to be interpretable and therefore transparent and comprehensible for humans. On the other hand, an approach for interactive ML as well as human-in-the-loop-learning is demonstrated in order to integrate human expert knowledge into ML models so that humans and machines act as companions within a critical decision task. We especially address the problem of Semantic Alignment between ML classifiers and its human users as a prerequisite for semantically relevant and useful explanations as well as interactions. Our roadmap paper presents and discusses an interdisciplinary yet integrated Comprehensible Artificial Intelligence (cAI)-transition-framework with regard to the task of medical diagnosis. We explain and integrate relevant concepts and research areas to provide the reader with a hands-on-cookbook for achieving the transition from opaque black-box models to interactive, transparent, comprehensible and trustworthy systems. To make our approach tangible, we present suitable state of the art methods with regard to the medical domain and include a realization concept of our framework. The emphasis is on the concept of Mutual Explanations (ME) that we introduce as a dialogue-based, incremental process in order to provide human ML users with trust, but also with stronger participation within the learning process.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bruckert et al. shed light on the difficulties that are presented when rendering ML models explainable for healthcare purposes, highlighting that the implementation of such systems requires overlap between different disciplines and professions.
“Additionally, Bruckert et al. [137] shed light on the difficulties that are presented when rendering ML models explainable for healthcare purposes, highlighting that the implementation of such systems requires overlap between different disciplines and professions.”
From Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review (2021) · cites this paper as [137]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · see citing works
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885 · see citing works
Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A. B. H., Thomas, L., Enk, A., et al. (2018) · Annals of Oncology · doi:10.1093/annonc/mdy166 · Cited by 1,571 · see citing works
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395 · see citing works
Keil, F. C. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.57.102904.190100 · Cited by 1,350 · see citing works
Can machine-learning improve cardiovascular risk prediction using routine clinical data?
Weng, S. F., Reps, J., Kai, J., Garibaldi, J. M., & Qureshi, N. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174944 · Cited by 1,346 · see citing works
Coherence in Thought and Action
Thagard, P. R. (2000) · The MIT Press eBooks · doi:10.7551/mitpress/1900.001.0001 · Cited by 956 · see citing works
Interactive machine learning for health informatics: when do we need the human-in-the-loop?
Holzinger, A. (2016) · Brain Informatics · doi:10.1007/s40708-016-0042-6 · Cited by 906 · see citing works
A user-centric evaluation framework for recommender systems
Pu, P., Chen, L., & Hu, R. J. (2011) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/2043932.2043962 · Cited by 867 · see citing works
TNM Klassifikation maligner Tumoren
Cancer, U. I. U. A. (1987) · Springer eBooks · doi:10.1007/978-3-662-10565-8 · Cited by 775 · see citing works
Schaefer, K. E., Chen, J. Y. C., Szalma, J. L., & Hancock, P. A. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816634228 · Cited by 768 · see citing works
The epistemology of a rule-based expert system —a framework for explanation
Clancey, W. J. (1983) · Artificial Intelligence · doi:10.1016/0004-3702(83)90008-5 · Cited by 712 · see citing works
What do we need to build explainable AI systems for the medical domain?
Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017) · arXiv · doi:10.48550/arxiv.1712.09923 · Cited by 636 · see citing works
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633 · see citing works
Systematicity and Surface Similarity in the Development of Analogy
Gentner, D., & Toupin, C. (1986) · Cognitive Science · doi:10.1016/s0364-0213(86)80019-2 · Cited by 625 · see citing works
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · see citing works
Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Gao, L., Zhu, X., Song, J., Zhao, Z., & Shen, H. T. (2019) · doi:10.24963/ijcai.2019 · Cited by 538 · see citing works
Using Neural Network Rule Extraction and Decision Tables for Credit-Risk Evaluation
Baesens, B., Setiono, R., Mues, C., & Vanthienen, J. (2003) · Management Science · doi:10.1287/mnsc.49.3.312.12739 · Cited by 536 · see citing works
Systematicity and Surface Similarity in the Development of Analogy
Gentner, D., & Toupin, C. (1986) · Cognitive Science · doi:10.1207/s15516709cog1003_2 · Cited by 423 · see citing works
Hu, L., Bell, D. A., Antani, S., Xue, Z., Yu, K., Horning, M. P., Gachuhi, N., Wilson, B., Jaiswal, M. S., Befano, B., et al. (2018) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djy225 · Cited by 401 · see citing works
Feature Centrality and Conceptual Coherence
Sloman, S. A., Love, B. C., & Ahn, W. (1998) · Cognitive Science · doi:10.1207/s15516709cog2202_2 · Cited by 383 · see citing works
On the Robustness of Interpretability Methods
Alvarez-Melis, D., & Jaakkola, T. S. (2018) · arXiv · doi:10.48550/arxiv.1806.08049 · Cited by 310 · see citing works
Understanding algorithm aversion: When is advice from automation discounted?
Prahl, A., & Van Swol, L. (2017) · Journal of Forecasting · doi:10.1002/for.2464 · Cited by 308 · cited 1× · see citing works
“Explanations can provide a valuable basis for providing transparency and comprehensibility regarding systems’ decisions and therefore can lead to increased trust of ML users (Pu et al., 2011; Prahl and Swol, 2017; Miller, 2019).”
BackgroundExplanatory Interactive Machine Learning
Teso, S., & Kersting, K. (2019) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314293 · Cited by 276 · see citing works
Causal Status as a Determinant of Feature Centrality
Ahn, W., Kim, N. S., Lassaline, M. E., & Dennis, M. J. (2000) · Cognitive Psychology · doi:10.1006/cogp.2000.0741 · Cited by 254 · see citing works
Lewis, D. D. (1987) · doi:10.1093/0195036468.003.0007 · Cited by 206 · see citing works
Hägele, M., Seegerer, P., Lapuschkin, S., Bockmayr, M. L., Samek, W., Klauschen, F., Müller, K.-R., & Binder, A. (2020) · Scientific Reports · doi:10.1038/s41598-020-62724-2 · Cited by 185 · see citing works
Madhavan, P., & Wiegmann, D. A. (2007) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872007x230154 · Cited by 176 · see citing works
Argument based machine learning
Možina, M., Žabkar, J., & Bratko, I. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2007.04.007 · Cited by 165 · see citing works
Ultra-Strong Machine Learning: comprehensibility of programs learned with ILP
Muggleton, S., Schmid, U., Zeller, C., Tamaddoni‐Nezhad, A., & Besold, T. R. (2018) · Machine Learning · doi:10.1007/s10994-018-5707-3 · Cited by 108 · see citing works
Self-citationExplanations, belief revision and defeasible reasoning
Falappa, M. A., Kern-Isberner, G., & Simari, G. R. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(02)00258-8 · Cited by 107 · see citing works
Buetti‐Dinh, A., Galli, V., Bellenberg, S., Ilie, O., Herold, M., Christel, S., Boretska, M., Pivkin, I. V., Wilmes, P., Sand, W., et al. (2019) · Biotechnology Reports · doi:10.1016/j.btre.2019.e00321 · Cited by 91 · see citing works
Learning From Explanations Using Sentiment and Advice in RL
Krening, S., Harrison, B. E., Feigh, K. M., Isbell, C. L., Riedl, M. O., & Thomaz, A. L. (2016) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2016.2628365 · Cited by 83 · see citing works
FairML : ToolBox for diagnosing bias in predictive modeling
Adebayo, J. (2016) · Cited by 70 · see citing works
Identifying Clinical Terms in Medical Text Using Ontology-Guided Machine Learning
Arbabi, A., Adams, D. R., Fidler, S., & Brudno, M. (2019) · JMIR Medical Informatics · doi:10.2196/12596 · Cited by 66 · see citing works
Enriching Visual with Verbal Explanations for Relational Concepts – Combining LIME with Aleph
Rabold, J., Deininger, H., Siebers, M., & Schmid, U. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-43823-4_16 · Cited by 59 · see citing works
Self-citationPlanning interactive explanations
Cawsey, A. J. (1993) · International Journal of Man-Machine Studies · doi:10.1006/imms.1993.1009 · Cited by 54 · see citing works
Holzinger, A. (2018) · Informatik-Spektrum · doi:10.1007/s00287-018-1102-5 · Cited by 54 · see citing works
Mutual Explanations for Cooperative Decision Making in Medicine
Schmid, U., & Finzel, B. (2020) · Künstliche Intell. · doi:10.1007/s13218-020-00633-2 · Cited by 41 · see citing works
Self-citationGenerating interactive explanations
Cawsey, A. J. (1991) · Cited by 33 · see citing works
Binder, A., Bockmayr, M. L., Hägele, M., Wienert, S., Heim, D., Hellweg, K., Stenzinger, A., Parlow, L., Budczies, J., Goeppert, B. F., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.11178 · Cited by 26 · see citing works
Elicitation of neurological knowledge with argument-based machine learning
Groznik, V., Guid, M., Sadikov, A., Možina, M., Georgiev, D., Kragelj, V., Ribarič, S., Pirtošek, Z., & Ivan Bratko, A. (2012) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2012.08.003 · Cited by 21 · see citing works
Please delete that! Why should I?
Siebers, M., & Schmid, U. (2018) · Künstliche Intell. · doi:10.1007/s13218-018-0565-5 · Cited by 20 · see citing works
Self-citationOpening the machine learning black box with Layer-wise Relevance Propagation
Lapuschkin, S. (2019) · DepositOnce · doi:10.14279/depositonce-7942 · Cited by 16 · see citing works
Balancing Explicability and Explanations in Human-Aware Planning
Chakraborti, T., Sreedharan, S., & Subbarao Kambhampati, B. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/185 · Cited by 15 · see citing works
A Characterization of Monotone Influence Measures for Data Classification.Preprint
Śliwiński, J., Strobel, M., & Zick, Y. (2017) · arXiv (Cornell University) · Cited by 11 · see citing works
Selection Bias, Label Bias, and Bias in Ground Truth
Søgaard, A., Plank, B., & Hovy, D. (2014) · Research at the University of Copenhagen (University of Copenhagen) · Cited by 9 · see citing works
Liu, Q., & Xia, F. (2014) · International Conference on Computational Linguistics · Cited by 9 · see citing works
Arguments in Interactive Machine Learning
Možina, M. (2018) · Informatica · Cited by 5 · see citing works
European Conference on Data Analysis 2013. Book of Abstracts.
Lausen, B., Krolak‐Schwerdt, S., & Böhmer, M. (2013) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Biran O. Cotton C. (2017). “Explanation and justification in machine learning: a survey,” in IJCAI-17 Workshop on Explainable AI (XAI) Proceedings (Melbourne). Available online at: https://ijcai-17.org/workshop-program.html.
- Burri T. (2016). “Machine learning and the law: five theses,” in Conference on Neural Information Processing Systems (NeurIPS) (Short Paper) (Barcelona). Available online at: https://nips.cc/Conferences/2016
- Dennett D. (1987). The Intentional Stance. Cambridge MA: MIT Press.
- Pearl J. (2009). Causality. Cambridge: Cambridge University Press.
Cited by
Sorted by citations
Longo, L., Brčić, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 648 · see citing works
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629 · see citing works
Schwalbe, G., & Finzel, B. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00867-8 · Cited by 381 · see citing works
Self-citationExplainable AI for clinical and remote health applications: A survey on tabular and time series data
Di Martino, F., & Delmastro, F. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10304-3 · Cited by 216 · see citing works
Holzinger, A., Dehmer, M., Emmert‐Streib, F., Cucchiara, R., Augenstein, I., Ser, J. D., Samek, W., Jurišica, I., & Díaz-Rodríguez, N. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.10.007 · Cited by 215 · see citing works
Explainable Artificial Intelligence for Tabular Data: A Survey
Sahakyan, M., Aung, Z., & Rahwan, T. (2021) · IEEE Access · doi:10.1109/access.2021.3116481 · Cited by 135 · see citing works
Khan, F. S., Mazhar, S. S., Mazhar, K., AlSaleh, D., & Mazhar, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11215-9 · Cited by 101 · see citing works
Salvi, M., Seoni, S., Campagner, A., Gertych, A., Acharya, U. R., Molinari, F., & Cabitza, F. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105846 · Cited by 84 · see citing works
The grammar of interactive explanatory model analysis
Baniecki, H., Parzych, D., & Biecek, P. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00924-w · Cited by 50 · see citing works
Comprehensible Artificial Intelligence on Knowledge Graphs: A survey
Schramm, S., Wehner, C., & Schmid, U. (2023) · Journal of Web Semantics · doi:10.1016/j.websem.2023.100806 · Cited by 43 · see citing works
Self-citationArtificial intelligence for dementia research methods optimization
Bucholc, M., James, C. R., Khleifat, A. A., Badhwar, A., Clarke, N., Dehsarvi, A., Madan, C. R., Marzi, S. J., Shand, C., Schilder, B. M., et al. (2023) · Alzheimer s & Dementia · doi:10.1002/alz.13441 · Cited by 42 · see citing works
Beyond Supervised Learning for Pervasive Healthcare
Gu, X., Deligianni, F., Han, J., Liu, X., Chen, W., Yang, G., & Lo, B. P. L. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2023.3296938 · Cited by 41 · see citing works
Evans, R. P., Bryant, L. D., Russell, G., & Absolom, K. L. (2024) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2024.105342 · Cited by 38 · see citing works
Harada, Y., Katsukura, S., Kawamura, R., & Shimizu, T. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18115562 · Cited by 27 · see citing works
Eisenberg, L., Brossette, C., Rauch, J., Grandjean, A., Ottinger, H. D., Rissland, J., Schwarz, U., Graf, N. M., Beelen, D. W., Kiefer, S., et al. (2022) · American Journal of Hematology · doi:10.1002/ajh.26671 · Cited by 28 · see citing works
Tenório, K., & Romeike, R. (2023) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3631802.3631829 · Cited by 28 · see citing works
Kiefer, S. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.014 · Cited by 27 · see citing works
What is Missing in XAI So Far?
Schmid, U., & Wrede, B. (2022) · Künstliche Intell. · doi:10.1007/s13218-022-00786-2 · Cited by 26 · see citing works
Self-citationArtificial intelligence in routine blood tests
Santos-Silva, M. A., Sousa, N., & Sousa, J. C. (2024) · Frontiers in Medical Engineering · doi:10.3389/fmede.2024.1369265 · Cited by 24 · see citing works
Trustworthy AI Guidelines in Biomedical Decision-Making Applications: A Scoping Review
Mora‐Cantallops, M., García‐Barriocanal, E., & Sicilia, M. (2024) · Big Data and Cognitive Computing · doi:10.3390/bdcc8070073 · Cited by 23 · see citing works
Shulha, M., Hovdebo, J., D’Souza, V., Thibault, F., & Harmouche, R. (2024) · JMIR Formative Research · doi:10.2196/50475 · Cited by 22 · see citing works
The Co-12 Recipe for Evaluating Interpretable Part-Prototype Image Classifiers
Nauta, M., & Seifert, C. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_21 · Cited by 20 · see citing works
Explanation as a Process: User-Centric Construction of Multi-level and Multi-modal Explanations
Finzel, B., Tafler, D. E., Scheele, S., & Schmid, U. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87626-5_7 · Cited by 20 · see citing works
Self-citationMachine learning's role in personalized medicine & treatment optimization
Kumar, D., Pawar, P. P., Gonaygunta, H., Nadella, G. S., Meduri, K., & Singh, S. (2024) · World Journal of Advanced Research and Reviews · doi:10.30574/wjarr.2024.21.2.0641 · Cited by 18 · see citing works
Göbel, K., Niessen, C., Seufert, S., & Schmid, U. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.919534 · Cited by 16 · see citing works
Self-citationTrustworthy Artificial Intelligence: Comprehensible, Transparent and Correctable
Schmid, U. (2023) · Springer eBooks · doi:10.1007/978-3-031-45304-5_10 · Cited by 16 · see citing works
Self-citationFinzel, B. (2025) · Pflügers Archiv - European Journal of Physiology · doi:10.1007/s00424-025-03067-7 · Cited by 15 · see citing works
Self-citationMahdavi, M., Choubdar, H., Rostami, Z., Niroomand, B., Levine, A. T., Fatemi, A., Bolhasani, E., Vahabie, A., Lomber, S. G., & Merrikhi, Y. (2023) · Scientific Reports · doi:10.1038/s41598-023-29334-0 · Cited by 14 · see citing works
Understanding imbalanced data: XAI & interpretable ML framework
Dablain, D. A., Bellinger, C., Krawczyk, B., Aha, D. W., & Chawla, N. V. (2024) · Machine Learning · doi:10.1007/s10994-023-06414-w · Cited by 14 · see citing works
Leserer, S., Graf, T., Franke, M., Bogdanov, R., Arrieta‐Bolaños, E., Buttkereit, U., Leimkühler, N. B., Fleischhauer, K., Reinhardt, H. C., Beelen, D. W., et al. (2023) · Frontiers in Immunology · doi:10.3389/fimmu.2023.1082727 · Cited by 12 · see citing works
Finzel, B., Saranti, A., Angerschmid, A., Tafler, D. E., Pfeifer, B., & Holzinger, A. (2022) · Künstliche Intell. · doi:10.1007/s13218-022-00781-7 · Cited by 12 · see citing works
Self-citationEsteghamati, M. Z., Wang, J., Wang, X., Paal, S. G., Murphy, J. D., Namin, A., Salman, A., Sabari, A. A., Ibrar, M. A., Mazumder, R. K., et al. (2026) · Journal of Structural Engineering · doi:10.1061/jsendh.steng-15222 · Cited by 11 · see citing works
Xue, P., Si, M., Qin, D., Wei, B., Seery, S., Ye, Z., Chen, M., Wang, S., Song, C., Zhang, B., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/43832 · Cited by 10 · see citing works
XAI Method Properties: A (Meta-)study.Preprint
Schwalbe, G., & Finzel, B. (2021) · arXiv (Cornell University) · Cited by 10 · see citing works
Self-citationWhen performance is not enough—A multidisciplinary view on clinical decision support
Roller, R., Burchardt, A., Samhammer, D., Ronicke, S., Duettmann, W., Schmeier, S., Möller, S., Dabrock, P., Budde, K., Mayrdorfer, M., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282619 · Cited by 9 · see citing works
Gopalan, R., Onniyil, D., Viswanathan, G., & Samdani, G. (2025) · World Journal of Advanced Engineering Technology and Sciences · doi:10.30574/wjaets.2025.15.2.0635 · Cited by 8 · see citing works
Finzel, B. (2024) · Künstliche Intell. · doi:10.1007/s13218-024-00835-y · Cited by 8 · see citing works
Self-citationUnsupervised Anomaly Detection for Financial Auditing with Model-Agnostic Explanations
Kiefer, S., & Pesch, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87626-5_22 · Cited by 7 · see citing works
Explainable AI (XAI) for Object Detection and Application to Satellite Imagery
Roy, A., Gaikwad, K. D., Hude, A. J., Shinde, J. S., Parekh, J. J., & Nagrani, H. R. (2025) · IEEE Access · doi:10.1109/access.2025.3624198 · Cited by 6 · see citing works
Kapsner, L. A., Feißt, M., Purbojo, A., Prokosch, H., Ganslandt, T., Dittrich, S., Mang, J. M., & Wällisch, W. (2024) · Diagnostics · doi:10.3390/diagnostics14222587 · Cited by 6 · see citing works
Blondeau, J. M., Charles, M., Loo, V. G., Adam, H. J., Vecchio, M. G.-D., Ghakis, C., O’Callaghan, E., & Ali, R. H. (2023) · Scientific Reports · doi:10.1038/s41598-023-40012-z · Cited by 6 · see citing works
Losch, D., Jaschke, S., Michaeli, T., Opel, S., Schmid, U., Seegerer, S., & Stechert, P. (2025) · Informatik-Spektrum · doi:10.1007/s00287-024-01584-w · Cited by 6 · see citing works
Self-citationContrastive Learning for Generating Optical Coherence Tomography Images of the Retina
Kaplan, S., & Lensu, L. T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16980-9_11 · Cited by 5 · see citing works
Toward trustworthy AI with integrative explainable AI frameworks
Finzel, B. (2025) · it - Information Technology · doi:10.1515/itit-2025-0007 · Cited by 4 · see citing works
Self-citationExplanation of Student Attendance AI Prediction with the Isabelle Infrastructure Framework
Kammüller, F., & Satija, D. (2023) · Information · doi:10.3390/info14080453 · Cited by 4 · see citing works
Knowledge Based Recommender System for Disease Diagnostic and Treatment Using Adaptive Fuzzy-Blocks
Navin, K. S., & B., M. K. M. (2024) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2024.02.002 · Cited by 4 · see citing works
Kiefer, S., Hoffmann, M., & Schmid, U. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4040050 · Cited by 4 · see citing works
Self-citationEvaluation of XAI Methods in a FinTech Context
Gawantka, F., Just, F., Ullrich, M., Savelyeva, M., & Lässig, J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49552-6_13 · Cited by 3 · see citing works
Gawantka, F., Schulz, A., Lässig, J., & Just, F. (2022) · IEEE International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) · doi:10.1109/iccicc57084.2022.10101657 · Cited by 3 · see citing works
Pattern recognition of hematological profiles of tumors of the digestive tract: an exploratory study
Santos-Silva, M. A., Sousa, N., Majar, M., Machado, M., Reis, J. S., & Sousa, J. C. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1208022 · Cited by 3 · see citing works
Towards a Visualizable, De-identified Synthetic Biomarker of Human Movement Disorders
Hu, H., Xiao, D., Rhodin, H., & Murphy, T. H. (2022) · Journal of Parkinson s Disease · doi:10.3233/jpd-223351 · Cited by 2 · see citing works
Sofos, F. D., Rouka, E. C., Triantafyllia, V., Andreakos, E., Gourgoulianis, K. I., Karakasidis, E., & Karakasidis, T. E. (2024) · Health and Technology · doi:10.1007/s12553-024-00886-z · Cited by 2 · see citing works
Interactive System for Similarity-Based Inspection and Assessment of the Well-Being of mHealth Users
Prakash, S., Unnikrishnan, V., Pryss, R. C., Kraft, R., Schobel, J., Hannemann, R., Langguth, B., Schlee, W., & Spiliopoulou, M. (2021) · Entropy · doi:10.3390/e23121695 · Cited by 2 · see citing works
Chandrashekar, M., Tschida, J., Strawn, J. R., Hanson, H. A., Santel, D., Goethert, I., Kapadia, A., Glauser, T. A., Pestian, J. P., & Agasthya, G. A. (2025) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2025.114735 · Cited by 1 · see citing works
HiMLEdge – Energy-Aware Optimization for Hierarchical Machine Learning
Wissing, J., Scheele, S., Mohammed, A., Kolossa, D., & Schmid, U. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-20316-9_2 · Cited by 1 · see citing works
Self-citationArgasiński, J. K., Grabska-Gradzińska, I., Przystalski, K., Ochab, J. K., & Walkowiak, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63775-9_20 · Cited by 1 · see citing works
Explainable machine learning for decision support in healthcare: A scoping review (Preprint)Preprint
Shulha, M., Rahimi, S. A., Sandhu, A., Sharma, G., D’Souza, V., Harmouche, R., & Hovdebo, J. (2022) · doi:10.2196/preprints.39196
Kaidashova, Y., Finzel, B., & Schmid, U. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02813-6_3
Self-citationVertrauenswürdige Künstliche Intelligenz
Schmid, U. (2025) · Springer eBooks · doi:10.1007/978-3-662-71567-3_10
Self-citationContext-dependent Effects of Explanations on Multiple Layers of Trust
Ismail-Tsaous, E., Uhl, M., Spannagl, C., Krügel, S., & Schmid, U. (2025) · doi:10.64136/dryx3176
Self-citation16913 Explainable Artificial Intelligence in biomedical engineering
Fonck, S., & Stollenwerk, A. (2026) · De Gruyter eBooks · doi:10.1515/9783111586458-013
Ethical Perspectives on Artificial Intelligence in Biomedical Engineering
Manzeschke, P. D. T. H. A., & Wittenberg, T. C. (2026) · De Gruyter eBooks · doi:10.1515/9783111586458
Few-shot learning, a data-centric approach for adaptation
Lafargue, R. (2024) · theses.fr (ABES) · doi:10.70675/4f90d466z2cd1z4e11z9996z1615a464b603
Schmid, U., Hüllermeier, E., Braun, T., & Görz, G. (2026) · doi:10.1515/9783111386690-001
Self-citationExplainable AI Methods for Object Detection in Safety-CriticalApplications: A Systematic ReviewPreprint
Kramer, M., Lindelauf, R. H. A., Safar-Postma, M., & Timmermans, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6585565
Explainable and Human-Centered AIoT: A Systematic Review of Integration, Interaction, and Impact
Rosas, A. A. J., Miranda, M. F., Pazos, G. L. P., Panta, E. E. G., Reyes, C. A. R., de Chang, M. D. P. C., Valdiviezo, J. H. C., Chirinos, O. P. G., & Aguirre, C. E. E. (2026) · Future Internet · doi:10.3390/fi18060303
SemAS - Semantic Alignment Score for XAI Applications in Clinical Decision Support
Bergomi, L., Michalowski, M., Szymon Wilk, A., Carrier, M., Gal, G. L., Wang, T., & Michalowski, W. J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30710-1_3
Explainable Artificial Intelligence in Healthcare: Progress, Challenges, and Future Directions
Ali, S. R. M., Nisha, D. A. M., Sathik, K. A., & Niasi, K. S. K. (2026) · International Journal of Integrative Studies (IJIS) · doi:10.63856/ijis/v2i7/07
Ricci, P., Luca, M. D., Ferraro, G., & Russo, A. (2023) · Journal of Health Informatics and Digital Systems · doi:10.68159/z771636283
- Authors
- Bruckert, S., Finzel, B., & Schmid, U.
- Affiliations
- University of Bamberg (DE)
- Corresponding author
- Bruckert, Sebastian
- Venue
- Frontiers in Artificial Intelligence
- Publisher
- Frontiers Media SA
- Published
- 2020-09-24
- Volume
- 3
- Pages
- 507973-507973
- ISSN
- 2624-8212 (ISSN-L)
- PMC
- PMC7861251
- OpenAlex
- W3088652003
- Cited by
- 66
- References
- 54
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 13 · 2023: 17 · 2022: 11 · 2021: 9
- FWCI
- 0.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Artificial Intelligence (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Bundesministerium für Bildung und Forschung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- clinical decision support, model interpretability, explainable AI, black-box models, semantic alignment, human-in-the-loop learning, interactive machine learning, trust in AI, human-machine collaboration, diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
