Explainable Machine Learning for Scientific Insights and Discoveries
Open accessThe DOI matches its Crossref record; 135 of the 137 references it cites were found in a registry or an index.
Abstract
Machine learning methods have been remarkably successful for a wide range of application areas in the extraction of essential information from data. An exciting and relatively recent development is the uptake of machine learning in the natural sciences, where the major goal is to obtain novel scientific insights and discoveries from observational or simulated data. A prerequisite for obtaining a scientific outcome is domain knowledge, which is needed to gain explainability, but also to enhance scientific consistency. In this article, we review explainable machine learning in view of applications in the natural sciences and discuss three core elements that we identified as relevant in this context: transparency, interpretability, and explainability. With respect to these core elements, we provide a survey of recent scientific works that incorporate machine learning and the way that explainable machine learning is used in combination with domain knowledge from the application areas.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Several powerful AI algorithms are employing a so-called “black box” approach, where it is difficult or even impossible to understand how the obtained results have been achieved.66”
From Addressing bias in big data and AI for health care: A call for open science (2021) · cites this paper as [66]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“image or a video. Lerer et al. [2016] and Li et al. [2016] use video simulations to learn intuitive physics, for example about the stability of wooden block towers. Lerer et al. [2016] use ResNet-34 [He et al., 2016] and Googlenet [Szegedy et al., 2015] to predict the fall of towers of wooden blocks, as well as DeepMa…”
MethodGradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“o clips containing 101 objects of dierent characteristics, which was built for the task of deriving physical parameters such as velocity and mass. In their work, they use the LeNet CNN architecture [LeCun et al., 1998] to capture visual as well as physical characteristics while explicitly integrating physical laws bas…”
MethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
(2016) · Cited by 11,679
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“Further details, types of interpretation, and specific realization can be found in recent surveys [7], [8], [20].”
BackgroundDeep learning and process understanding for data-driven Earth system science
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat (2019) · Nature · doi:10.1038/s41586-019-0912-1 · Cited by 5,673
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Research works printed but not linked: 3
- 10.1037/0033-2909.110.3.499 · doi:10.1037/0033-2909.110.3.499 · Paper page
- frigg, 2018, Scientific representation, The Stanford Encyclopedia of Philosophy
- 10.1145/3233231 · doi:10.1145/3233231 · Paper page
Cited by
Items 401–500 of 999
Improved Prediction of Ligand–Protein Binding Affinities by Meta-modeling
Lee, H., Emani, P. S., & Gerstein, M. (2024) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.4c01116 · Cited by 14
Role of machine learning for predictive modeling for Industry 5.0 and Society 5.0
Bamakan, S. M. H., Dehghan, F., & Akbarpour, S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-33498-6.00008-x · Cited by 11
Hein, I., & Diefenbach, S. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2502983 · Cited by 13
“…as a multifaceted term comprises especially that users understand what the technology is doing, how it gets to its result, and why (Miller, 2019; Roscher et al., 2020), which may include an understanding of the technology’s processes, performance, and plans (Chen et al., 2014; Mohseni et al.,…”
TRANSPARÊNCIA VERSUS EXPLICAÇÃO: O PAPEL DA AMBIGUIDADE NA IA JURÍDICA
Esposito, E. (2022) · Revista Direito Mackenzie · doi:10.5935/2317-2622/direitomackenzie.v16n315705 · Cited by 9
All-sky longwave radiation modelling based on infrared images and machine learning
Zhao, C., Zhang, L., & Zhang, Y. (2023) · Building and Environment · doi:10.1016/j.buildenv.2023.110369 · Cited by 10
Learning interpretable descriptors for the fatigue strength of steels
He, N., Ouyang, R., & Qian, Q. (2021) · AIP Advances · doi:10.1063/5.0045561 · Cited by 13
Alamin, M. A. A., & Uddin, G. (2021) · IEEE International Conference on Autonomous Systems (ICAS) · doi:10.1109/icas49788.2021.9551151 · Cited by 22
Challenges and Opportunities in Using Data Science for Homelessness Service Provision
Chelmis, C., Qi, W., & Lee, W. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442442.3453454 · Cited by 14
“In data science, explainability refers to peoples’ ability to explain the output of a machine learning model [43].”
BackgroundHolistic Explainability Requirements for End-to-End Machine Learning in IoT Cloud Systems
Nguyen, M.-L., Phung, T., Ly, D.-H., & Truong, H. (2021) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew53955.2021.00034 · Cited by 11
“The work in [12] discussed having domain knowledge to obtain explainability and suggest differentiating between scientific and algorithmic explanations.”
BackgroundThe machine-learned radii of atoms
Nikolaienko, T. Y., & Булавін, Л. А. (2021) · Computational and Theoretical Chemistry · doi:10.1016/j.comptc.2021.113389 · Cited by 10
Gene-centric multi-omics integration with convolutional encoders for cancer drug response prediction
Lee, M., Kim, P., Joe, H., & Kim, H. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106192 · Cited by 10
Papacharalampous, G., Tyralis, H., Markonis, Y., Máca, P., & Hanel, M. (2023) · Progress in Earth and Planetary Science · doi:10.1186/s40645-023-00574-y · Cited by 12
“…seasonal and monthly time scales at different parts of the world (see, for example, the analyses by Nigam and Ruiz-Barradas 2006; Ljungqvist et al. 2016; PAGES Hydro2k Consortium 2017; Thomas and Nigam 2018; Papacharalampous et al. 2021, and the review by Koster et al. 2017) is undoubtedly large.”
BackgroundMapping and interpretability of aftershock hazards using hybrid machine learning algorithms
Liu, B., Wen, H., Di, M., Huang, J., Liao, M., Yu, J. Y., & Xiang, Y. (2024) · Journal of Rock Mechanics and Geotechnical Engineering · doi:10.1016/j.jrmge.2024.09.015 · Cited by 13
Cao, N., Farrag, A., Won, D., & Yoon, S. W. (2024) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2024.09.009 · Cited by 10
Eberle, O., Büttner, J., El-Hajj, H., Montavon, G., Müller, K., & Valleriani, M. (2024) · Science Advances · doi:10.1126/sciadv.adj1719 · Cited by 23
“ML-assisted insight discovery has primarily found its application in the natural sciences [158], but other fields have also begun to explore the potential of ML techniques to push existing boundaries of their respective domains.”
Confidence-Aware Ship Classification Using Contour Features in SAR Images
Hinai, A. A. A., & Raffaella Guida, A. (2025) · Remote Sensing · doi:10.3390/rs17010127 · Cited by 11
What makes a good BIM design? Quantifying the link between design behavior and quality
Ni, X.-R., Pan, P., & Lin, J. (2025) · Automation in Construction · doi:10.1016/j.autcon.2025.105992 · Cited by 15
NaLin, Ding, K., Tan, L., Li, B., Yang, K., Wang, C., Wang, B., Li, N., & Yang, R. (2025) · Advances in Space Research · doi:10.1016/j.asr.2025.03.067 · Cited by 17
Záhonyi, P., Fekete, D., Szabó, E., Nagy, Z. K., & Nagy, B. (2025) · European Journal of Pharmaceutical Sciences · doi:10.1016/j.ejps.2025.107173 · Cited by 11
Explainable few-shot learning workflow for detecting invasive and exotic tree species
Gevaert, C. M., Pedro, A. A., Ku, O., Cheng, H., Chandramouli, P., Javan, F. D., Nattino, F., & Georgievska, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-05394-2 · Cited by 10
“Here, we follow other publications in Earth Observation, which relate explainability to the ability to understand how models derive a prediction through the lens of specific domain knowledge 18 .”
InfluentialDigitalisierung souverän gestalten II: Handlungsspielräume in digitalen Wertschöpfungsnetzwerken
Hartmann, E. (2021) · Springer eBooks · doi:10.1007/978-3-662-64408-9 · Cited by 11
Basic Issues and Challenges on Explainable Artificial Intelligence (XAI) in Healthcare Systems
Oladipo, I. D., Awotunde, J. B., AbdulRaheem, M., & Salihu, S. A. (2022) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-6684-3791-9.ch011 · Cited by 10
Investor preference analysis: An online optimization approach with missing information
Hu, X., Chen, Y., Ren, L., & Xu, Z. (2023) · Information Sciences · doi:10.1016/j.ins.2023.03.066 · Cited by 9
Interpretable Graph-Network-Based Machine Learning Models via Molecular Fragmentation
Collins, E. M., & Raghavachari, K. (2023) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c01308 · Cited by 12
Vu, M. T., & Jardani, A. (2023) · Advances in Water Resources · doi:10.1016/j.advwatres.2023.104463 · Cited by 9
Exploring influential factors of CO2 emissions in China’s cities using machine learning techniques
Xiang, K., Yu, H., Du, H. Y., Hasan, M. H., Wei, S., & Xiang, X. (2023) · Environmental Science and Pollution Research · doi:10.1007/s11356-023-28285-3 · Cited by 12
RNN-LSTM model for reliable optical transmission in flexible switching network systems
Almawgani, A. H. M. (2023) · Wireless Networks · doi:10.1007/s11276-023-03599-9 · Cited by 9
“In [6] comprehensively described the functionalities of machine learning and the need for domain knowledge to use its advantages effectively.”
Farkhadov, M., Eliseev, A., & Петухова, Н. В. (2020) · IEEE International Conference on Application of Information and Communication Technologies (AICT) · doi:10.1109/aict50176.2020.9368576 · Cited by 10
“Summarizing, we can say that transparency is a consequence of how we choose our model, interpretability considers the model along with data, and explainability is determined by the model, data and human participation [11].”
BackgroundModel Simplification of Deep Random Forest for Real-Time Applications of Various Sensor Data
Kim, S., Ko, B. C., & Nam, J.-Y. (2021) · Sensors · doi:10.3390/s21093004 · Cited by 12
“In addition, in terms of transparency in a machine learning approach, the choice of hyperparameters such as the learning rate and batch size has a more heuristic, non-transparent algorithmic nature [23].”
BackgroundMethodInfluentialMei, F., Li, X., Zheng, J., Sha, H., & Li, D. (2022) · IET Generation Transmission & Distribution · doi:10.1049/gtd2.12385 · Cited by 9
Nagpal, S., Pinna, N. K., Pant, N., Singh, R. P., Srivastava, D., & Mande, S. S. (2022) · Journal of Molecular Biology · doi:10.1016/j.jmb.2022.167684 · Cited by 12
“We adopted a graduated approach of building multiple machine learning classifiers to gauge the predictive power of SARS-CoV-2 mutation signatures towards prognosis of a Covid-19 infection.”
Explainable Drug Repurposing Approach From Biased Random Walks
Castiglione, F., Nardini, C., Onofri, E., Pedicini, M., & Tieri, P. (2022) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2022.3191392 · Cited by 12
“Explainability in Artificial Intelligence methods (see [25]) like recommendation systems is particularly useful since it allows,”
BackgroundDefining factors in hospital admissions during COVID-19 using LSTM-FCA explainable model
Saleh, N. I. M., Ghani, H. A., & Jilani, Z. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102394 · Cited by 9
“In general, mathematical formalisms are applied to make the model more explainable, as per described by [4], since domain knowledge is an essential part of explainability.”
Inverse Estimation of Elastic Modulus Using Physics-Informed Generative Adversarial NetworksPreprint
Warner, J. E., Cuevas, J., Bomarito, G., Leser, P. E., & Leser, W. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05791 · Cited by 9
“This most commonly comes in the form of leveraging domain-specific knowledge in machine learning work flows [19].”
BackgroundInterpretability of machine-learning models in physical sciencesPreprint
Ghiringhelli, L. M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.10443 · Cited by 14
“The literature on interpretability is vast[1-7], but the field is pre-paradigmatic, i.”
BackgroundMonitoring bias and fairness in machine learning models: A reviewPreprint
Bacelar, M. (2021) · doi:10.14293/s2199-1006.1.sor-.pp59wrh.v1 · Cited by 12
Satellite Image and Machine Learning Based Knowledge Extraction in the Poverty and Welfare DomainPreprint
Hall, O., Ohlsson, M., & Rognvaldsson, T. S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4102620 · Cited by 9
Melekoğlu, E., Kocabıçak, Ü., Uçar, M. K., Bílgín, C., Bozkurt, M. R., & Çunkaş, M. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.1188 · Cited by 12
“In addition, these algorithms are suitable for transferring to embedded systems (Roscher et al., 2020; Santos, Moreno & Estombelo-Montesco, 2019; Saguil & Azim, 2019).”
MethodMa, F.-Q., He, C., Yang, H.-R., Hu, Z., Mao, H.-R., Fan, C.-Y., Qi, Y., Zhang, J., & Xu, B. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02192-6 · Cited by 9
“However, most ML studies worked hard to improve performance by increasing the model complexity, leading to uncertainties in the way how ML operates and makes decisions [24–26].”
BackgroundSelection of powerful radio galaxies with machine learning
Carvajal, R., Matute, I., Afonso, J., Norris, R. P., Luken, K., Sanchéz-Sáez, P., Cunha, P. A. C., Humphrey, A., Messias, H., Amarantidis, S., et al. (2023) · Astronomy and Astrophysics · doi:10.1051/0004-6361/202245770 · Cited by 11
Using Decision Trees for Interpretable Supervised Clustering
Kokash, N., & Махнист, Л. П. (2024) · SN Computer Science · doi:10.1007/s42979-023-02590-7 · Cited by 10
“Importantly, the decision tree can be linearized into decision rules, where the outcome is the contents of the leaf node, and the conditions along the path form a conjunction in the if-clause : This property highlights decision trees as the most prominent approach for explaining the outcomes of…”
Deep Sky Objects Detection with Deep Learning for Electronically Assisted Astronomy
Parisot, O., & Jaziri, M. (2024) · Astronomy · doi:10.3390/astronomy3020009 · Cited by 10
“Nowadays, these tools are seen as a tool for scientific discovery [38], in particular for understanding physical laws by comparing observations and AI predictions [39].”
BackgroundExplainable machine learning model of disorganisation in swarms of drones
Gackowska, M., & Cofta, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-73220-2 · Cited by 8
“As rightly noted by Roscher et al. 28 , reliance on an algorithmic interpretation of a model for individual data will not provide an explanation hat allows the user to understand the model’s decisions.”
BackgroundHuman bias and CNNs’ superior insights in satellite based poverty mapping
Sarmadi, H., Wahab, I., Hall, O., Rognvaldsson, T. S., & Ohlsson, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-74150-9 · Cited by 11
“Additionally, these insights can guide the development of new experimental designs or data collection strategies, aiming to further test and validate the findings initiated by machine learning analyses 13 .”
Parente, F. R., Rodrigues, E. B., & Mattos, C. L. C. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2025.103971 · Cited by 13
Chen, J., Chao, W., Wang, Y., Ren, N., & Lü, L. (2025) · ACS ES&T Engineering · doi:10.1021/acsestengg.5c00100 · Cited by 9
Machine Learning Efforts That Enhance Personalized Patient Care and Chronic Disease Management
Adeyinka, A. A., Lamina, Y., Tawo, O. E., Adeyeye, Y. I., & Minkah, A. Y. (2022) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst2302551 · Cited by 13
Aggregation Techniques in Federated Learning: Comprehensive Survey, Challenges and Opportunities
Sah, M. P., & Singh, A. (2022) · International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) · doi:10.1109/icacite53722.2022.9823702 · Cited by 18
Controlled Multi-modal Image Generation for Plant Growth Modeling
Miranda, M., Drees, L., & Roscher, R. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956115 · Cited by 8
“This is in line with the increasing demand for explainable machine learning to assure the value of a prediction [10].”
BackgroundSelf-citationDoctors in Medical Data Sciences: A New Curriculum
Cussat‐Blanc, S., Castets-Renard, C., & Monsarrat, P. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010675 · Cited by 10
“Future practitioners should understand the entire data analysis pipeline, how and where the data have been stored and annotated, how they are pre-processed and transformed into a dataset, how machine learning algorithms are trained and what biases can be introduced, and how predictions should be interpreted and explai…”
BackgroundShapley Additive Explanations for Knowledge Discovery via Surrogate Models
Palar, P. S., Zuhal, L. R., Shimoyama, K., Dwianto, Y. B., & Morlier, J. (2023) · AIAA SciTech Forum · doi:10.2514/6.2023-0332 · Cited by 10
Off-policy and on-policy reinforcement learning with the Tsetlin machine
Gorji, S. R., & Granmo, O. (2023) · Applied Intelligence · doi:10.1007/s10489-022-04297-3 · Cited by 8
“However, their black-box nature raises concerns, making methods for explaining black-box models a main focus of contemporary research [17, 29].”
BackgroundBhowmick, S., & Nagarajaiah, S. (2023) · Engineering Structures · doi:10.1016/j.engstruct.2023.116231 · Cited by 11
Zeng, J., Xu, H. J., Chen, Y., & Zhang, D. (2023) · Computational Geosciences · doi:10.1007/s10596-023-10244-z · Cited by 12
Stomberg, T. T., Leonhardt, J., Weber, I., & Roscher, R. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1278118 · Cited by 10
“Roscher et al. (2020) highlight the usefulness of such models being interpretable and explainable.”
BackgroundMethodSelf-citationMir, H. E., & Perinpanayagam, S. (2021) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc52595.2021.9594374 · Cited by 10
Miller, E., Ceballos, H. G., Engelmann, B., Schiffler, A., Batres, R., & Schmitt, J. (2021) · Machine Learning-Driven Digital Technologies for Educational Innovation Workshop · doi:10.1109/ieeeconf53024.2021.9733776 · Cited by 8
Revealing interactions between HVDC cross-area flows and frequency stability with explainable AI
Pütz, S., Schäfer, B., Witthaut, D., & Kruse, J. (2022) · Energy Informatics · doi:10.1186/s42162-022-00241-4 · Cited by 13
“2020), thus gaining additional scientific insights (Roscher et al. 2020).”
ScrutinAI: A visual analytics tool supporting semantic assessments of object detection models
Haedecke, E., Möck, M., & Akila, M. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.06.010 · Cited by 13
Zhang, T., Chai, H., Wang, H., Guo, T., Zhang, L., & Zhang, W. (2023) · Frontiers in Earth Science · doi:10.3389/feart.2023.1217384 · Cited by 10
Santos, J. I., Martín, Ó., Ahedo, V., Tiedra, P. D., & Galán, J. M. (2022) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-022-10444-4 · Cited by 9
Non-Uniform Interpolation in Integrated Gradients for Low-Latency Explainable-AI
Bhat, A., & Raychowdhury, A. (2023) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas46773.2023.10181829 · Cited by 13
“tasks [2] which require transparency and interpretability [3].”
BackgroundLiu, L., Zhou, F., & Cao, W. D. (2024) · Journal of Geochemical Exploration · doi:10.1016/j.gexplo.2024.107478 · Cited by 10
Prediction of bike-sharing station demand using explainable artificial intelligence
Ngeni, F., Kutela, B., Chengula, T. J., Ruseruka, C., Musau, H., Novat, N., Indah, D. A., & Kasomi, S. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100582 · Cited by 14
Sahoo, B. B., Najafzadeh, M., Santosh, D. T., Jithendra, T., Panigrahi, B., Mishra, S. M., Gupta, S. K., & Bhushan, M. (2025) · Journal of Irrigation and Drainage Engineering · doi:10.1061/jidedh.ireng-10441 · Cited by 7
Kost, L., Lier, S. K., & Breitner, M. H. (2025) · Energy and AI · doi:10.1016/j.egyai.2025.100648 · Cited by 13
Alnamasi, K., Paramasivam, P., Arun, B. K., & Kanti, P. K. (2025) · Case Studies in Thermal Engineering · doi:10.1016/j.csite.2025.107566 · Cited by 10
Variant Approach for Identifying Spurious Relations That Deep Learning Models Learn
Tesch, T., Kollet, S. J., & Garcke, J. (2021) · Frontiers in Water · doi:10.3389/frwa.2021.745563 · Cited by 8
Self-citationBonavita, M., Schneider, R., Baz, M. E., Arcucci, R., Bocquet, M., Chrust, M., Pourshamsi, M., Cocchiara, C. M., Carrassi, A., Raoult, N., et al. (2023) · npj Climate and Atmospheric Science · doi:10.1038/s41612-023-00387-2 · Cited by 12
Kawata, N., Iwao, Y., Matsuura, Y., Suzuki, M., Ema, R., Sekiguchi, Y., Sato, H., Nishiyama, A., Nagayoshi, M., Takiguchi, Y., et al. (2023) · Japanese Journal of Radiology · doi:10.1007/s11604-023-01466-3 · Cited by 8
“To interpret DL models, gradient-weighted class activation mapping (Grad-Cam) for the image input [30] and local interpretable model-agnostic (LIME) were used for the table data [31].”
MethodAsteroids co-orbital motion classification based on Machine Learning
Ciacci, G., Barucci, A., Ruzza, S. D., & Alessi, E. M. (2023) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stad3603 · Cited by 9
Huang, Y., & Chen, Y. (2024) · Water Air & Soil Pollution · doi:10.1007/s11270-024-06913-z · Cited by 7
“Machine learning uncovers concealed data patterns, enabling clustering and regression analysis tasks (Roscher et al., 2020).”
Zhidkikh, D., Heilala, V., Petegem, C. V., Dawyndt, P., Järvinen, M., Viitanen, S., Wever, B. D., Mesuere, B., Lappalainen, V., Kettunen, L., et al. (2024) · Journal of Learning Analytics · doi:10.18608/jla.2024.7979 · Cited by 14
“Moreover, one must consider the primary audience for these analytic results: diverse visualizations and analytics artifacts will have varying levels of explainability for researchers, educators, and students (Roscher et al., 2020).”
BackgroundQuantifying the Risk of Unmapped Associations for Mobile Robot Localization Safety
Chen, Y., PERVAN, B. S., & Spenko, M. J. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3401093 · Cited by 8
“…a system’s component’s risk to be strictly quantifiable, as mentioned earlier, any deep learning-based feature extractors, such as those that rely on neural networks, are not applicable [25] because trust in their information analytically cannot be guaranteed, a must in life-critical applications.”
BackgroundBeyond Cross-Validation—Accuracy Estimation for Incremental and Active Learning Models
Limberg, C., Wersing, H., & Ritter, H. (2020) · Machine Learning and Knowledge Extraction · doi:10.3390/make2030018 · Cited by 10
A predictive and user-centric approach to Machine Learning in data streaming scenarios
Carneiro, D. R., Guimarães, M., Silva, F., & Nováis, P. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.100 · Cited by 8
Lin, I., Tseng, J. Y.-C., Tung, H.-T., Hu, Y., & You, Z.-H. (2022) · Healthcare · doi:10.3390/healthcare10040667 · Cited by 7
“Generally, the XAI methods follow the following three principles: transparency, interpretability, and explainability [46].”
MethodNguyen, X. V., Dikici, E., Candemir, S., Ball, R. L., & Prevedello, L. M. S. (2022) · Tomography · doi:10.3390/tomography8040151 · Cited by 7
“The topic has gained significant importance in recent years [23].”
BackgroundA Model Distillation Approach for Explaining Black-Box Models for Hyperspectral Image Classification
Taşkın, G. (2022) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss46834.2022.9884727 · Cited by 9
“This has recently become an issue in remote sensing applications [5, 6, 7].”
BackgroundAn XAI Approach to Predictive Analytics of Pancreatic Cancer
Srinidhi, B. K., & Bhargavi, M. S. (2023) · doi:10.1109/icit58056.2023.10225991 · Cited by 8
“[16] distinguished between transparency, interpretability and explainability by identifying unique ideas from the various meanings of several descriptive terms.”
BackgroundAI for Technoscientific Discovery: A Human-Inspired Architecture
Tsao, J. Y., Abbott, R. G., Crowder, D. C., Desai, S., Dingreville, R., Fowler, J. E., Garland, A., Iyer, P. P., Murdock, J., Steinmetz, S., et al. (2024) · Journal of Creativity · doi:10.1016/j.yjoc.2024.100077 · Cited by 12
Adaptive risk prediction system with incremental and transfer learning
Koivu, A., Sairanen, M., Airola, A., Pahikkala, T., Leung, W.-C., Lo, T. K., & Sahota, D. S. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104886 · Cited by 8
Interpretation and Simplification of Deep ForestPreprint
Kim, S., Jeong, M., & Ko, B. C. (2020) · doi:10.36227/techrxiv.11661246.v1 · Cited by 7
Khademi, A., & Honavar, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.00357 · Cited by 14
“, features) [76,90].”
Taveekitworachai, P., Chanmas, G., Paliyawan, P., Thawonmas, R., Nukoolkit, C., Dajpratham, P., & Thawonmas, R. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e32930 · Cited by 8
“Similarly, ML-based techniques, which suffer from interpretation issues due to its black-box nature, should also consider incorporating explainable ML techniques [43].”
BackgroundDiefenbach, S., Ullrich, D., Lindermayer, T., & Isaksen, K. R. (2024) · Information · doi:10.3390/info15080460 · Cited by 8
“In consequence, users are less able to understand the processes, performance, intentions, or plans of the technologies [28,29], what they are currently doing, how they have got to a certain result, or why [30,31].”
BackgroundWang, X., Xie, L., Wang, H., Xing, X., Wan, W., Wu, Z., Ma, X., & Li, Q. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3487974 · Cited by 9
Kohler, O., & Imtiaz, M. H. (2025) · Electronics · doi:10.3390/electronics14050963 · Cited by 6
Gagliardi, R. V., & Andenna, C. (2025) · Atmosphere · doi:10.3390/atmos16050491 · Cited by 11
Machine learning and partial differential equations: benchmark, simplify, and discover
Koumoutsakos, P. (2025) · Data-Centric Engineering · doi:10.1017/dce.2025.15 · Cited by 8
“The intervention character of MADRL may be a gateway to explainable ML (Roscher et al., 2020) and effective closures for PDEs.”
Na, R., Gantumur, B., Du, W., Bayarsaikhan, S., Yu, S., Mu, Q., Bao, Y., Tegshjargal, N., & Vandansambuu, B. (2025) · Fire · doi:10.3390/fire8070273 · Cited by 7
Matas-Bustos, J. B., Mora, A. M., Lora, M. D. H., Nieto-Alarcón, A., & González-Férnandez, F. T. (2025) · PLoS ONE · doi:10.1371/journal.pone.0327960 · Cited by 9
A Note on Spurious Correlations and Explainable Machine Learning in Digital Soil Mapping
Rentschler, T., & Scholten, T. (2025) · European Journal of Soil Science · doi:10.1111/ejss.70172 · Cited by 10
“Both elements embed and expand the common black-box with input, model and output (Figure 5; Roscher et al. 2020; Wadoux, Minasny, et al. 2020).”
Mohammed, K., Kpienbaareh, D., Kerr, R. B., Wang, J., Luginaah, I., Lupafya, E., Dakishoni, L., & Mkandawire, M. (2025) · Environmental Impact Assessment Review · doi:10.1016/j.eiar.2025.108245 · Cited by 9
Wen, Y., Guo, R., Duan, Z., Tong, Y. S., Tang, X., Pan, T., & Fu, C. (2026) · Scientific Reports · doi:10.1038/s41598-026-36692-y · Cited by 15
Physics-constrained neural networks for high-fidelity composite failure envelopes
Zhang, R., Zhao, D., Zhang, P., & Tang, K. (2026) · Composite Structures · doi:10.1016/j.compstruct.2026.120358 · Cited by 6
Machine Learning with Multi-Source Data to Predict and Explain Marine Pilot Occupational Accidents
Camlıyurt, G., Park, Y. S., Kim, D., Kang, W. S., & Park, S. (2023) · Journal of marine science and technology · doi:10.51400/2709-6998.2709 · Cited by 11
Kunz, J. (2024) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180754712 · Cited by 8
X-Detect: explainable adversarial patch detection for object detectors in retail
Hofman, O., Giloni, A., Hayun, Y., Morikawa, I., Shimizu, T., Elovici, Y., & Shabtai, A. (2024) · Machine Learning · doi:10.1007/s10994-024-06548-5 · Cited by 11
“…points are selected by examining their surrounding pixels’ gradients after applying varied levels of blurring; and 4) The use of class prototypes [40] – an explainability technique that identifies data samples that best represent a class, i.e., the samples that are most related to a given class…”
BackgroundInteractive preference analysis: A reinforcement learning framework
Hu, X., Kang, S., Ren, L., & Zhu, S. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2024.06.033 · Cited by 8
Metadata
- Authors
- Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J.
- Affiliations
- University of Bonn (DE)
- University of Massachusetts Amherst (US)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 42200-42216
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2945526235
- Cited by
- 1,040
- References
- 137
- Citations by year
- 2026: 152 · 2025: 178 · 2024: 168 · 2023: 184 · 2022: 174 · 2021: 130 · 2020: 47
- FWCI
- 52.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Fraunhofer Cluster of Excellence Cognitive Internet Technologies
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
- Keywords
- explainable machine learning, scientific discovery, transparency, interpretability, explainability, domain knowledge, observational data, simulated data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.