The DOI matches its Crossref record; 35 of the 37 references it cites were found in a registry or an index.
Enhancing Explainability of Neural Networks Through Architecture Constraints
Open accessAbstract
Prediction accuracy and model explainability are the two most important objectives when developing machine learning algorithms to solve real-world problems. Neural networks are known to possess good prediction performance but suffer from a lack of model interpretability. In this article, we propose to enhance the explainability of neural networks through the following architecture constraints: 1) sparse additive subnetworks; 2) projection pursuit with orthogonality constraint; and 3) smooth function approximation. It leads to an enhanced explainable neural network (ExNN) with a superior balance between prediction performance and model interpretability. We derive sufficient identifiability conditions for the proposed ExNN model. The multiple parameters are simultaneously estimated by a modified minibatch gradient descent method based on the backpropagation algorithm for calculating the derivatives and the Cayley transform for preserving the projection orthogonality. Through simulation study under six different scenarios, we compare the proposed method to several benchmarks, including least absolute shrinkage and selection operator, support vector machine, random forest, extreme learning machine, and multilayer perceptron. It is shown that the proposed ExNN model keeps the flexibility of pursuing high prediction accuracy while attaining improved interpretability. Finally, a real data example is employed as a showcase application.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, in tightly controlled domains like IoT systems, avionics, finance, banking, autonomous systems, and medical services, integrating AI and its interoperability with legacy applications is much more onerous [27], [47].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [47]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Leading to significant biases and prejudices in the training set, black-box models may still make poor predictions even in the absence of adversarial attacks [28], [46], [47].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [47]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
(2010) · Cited by 14,134
Extreme learning machine: Theory and applications
Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.126 · Cited by 13,285
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Lu, Z. Q. J. (2010) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,916
Brown, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Biometrics · doi:10.2307/2532174 · Cited by 8,263
Koyak, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Journal of the American Statistical Association · doi:10.2307/2290538 · Cited by 7,717
Research works printed but not linked: 1
- brainard, 2018, What Are We Learning About Artificial Intelligence in Financial Services?
Cited by
Sorted by citations
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Kufel, J., Bargieł-Łączek, K., Kocot, S., Koźlik, M., Bartnikowska, W., Janik, M., Czogalik, Ł., Dudek, P., Magiera, M., Lis, A., et al. (2023) · Diagnostics · doi:10.3390/diagnostics13152582 · Cited by 476
“Deep neural networks, particularly with multiple hidden layers, which can be considered as black boxes, lack interpretability [86,87].”
BackgroundA Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352
“For instance, the European Union’s GDPR framework, which was been passed in 2016 and went into effect in May 2018, established criteria to support the ‘‘entitlement of an explanation’’ premise for users [47].”
BackgroundMethodIslam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 366
Degas, A., Islam, M. R., Hurter, C., Barua, S., Rahman, H., Poudel, M., Ruscio, D., Ahmed, M. U., Begum, S., Rahman, M. A., et al. (2022) · Applied Sciences · doi:10.3390/app12031295 · Cited by 168
A Survey of Binary Code Similarity
Haq, I. U., & Caballero, J. (2021) · ACM Computing Surveys · doi:10.1145/3446371 · Cited by 184
“, [129]) will become an important issue for applications like malware lineage and malware attribution where a human may need to understand the similarity captured by the models.”
BackgroundYang, Z., Zhang, A., & Sudjianto, A. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108192 · Cited by 144
Self-citationA Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3485275 · Cited by 137
“Thus, there exists an open research problem related to being able to explain how a given model was capable of predicting the target function, specifically as it relates to SE data [12, 59, 174, 178, 182].”
BackgroundReinforcement learning in spacecraft control applications: Advances, prospects, and challenges
Tipaldi, M., Iervolino, R., & Massenio, P. R. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.07.004 · Cited by 119
Huang, C., Huang, Q., Huang, X., Wang, H., Li, M., Lin, K.-J., & Chang, Y. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3418098 · Cited by 116
“Similarly, Yang et al. [61] presented a structured neural network that balances prediction performance and model interpretability through architecture constraints.”
BackgroundBrasse, J., Broder, H. R., Förster, M., Klier, M., & Sigler, I. (2023) · Electronic Markets · doi:10.1007/s12525-023-00644-5 · Cited by 121
“…XAI methods extract rules that reveal the functioning of an automatic diagnosis system to medical experts (Barakat et al., 2010; Seera & Lim, 2014) or, in the context of electronic markets, showcase central factors for loan approval on peer-to-peer lending platforms (Yang et al., 2021) (Fig.”
A novel ensemble-based conceptual-data-driven approach for improved streamflow simulations
Sikorska‐Senoner, A. E., & Quilty, J. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2021.105094 · Cited by 77
“Most previous data-driven approaches for streamflow simulation have used DDMs to explicitly simulate streamflow and i this way they provide an alternative to a HM.”
BackgroundMethodInfluentialPsychological foundations of explainability and interpretability in artificial intelligence
Broniatowski, D. A. (2021) · doi:10.6028/nist.ir.8367 · Cited by 76
Deep learning for pH prediction in water desalination using membrane capacitive deionization
Son, M., Yoon, N., Jeong, K., Abbas, A., Logan, B. E., & Cho, K. H. (2021) · Desalination · doi:10.1016/j.desal.2021.115233 · Cited by 75
“However, it is also known to be less explainable because of the presence of hidden layers [46,47].”
BackgroundInterpretable generalized additive neural networks
Kraus, M., Tschernutter, D., Weinzierl, S., & Zschech, P. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.06.032 · Cited by 59
Bottieau, J., Wang, Y., Grève, Z. D., Vallée, F., & Toubeau, J. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3195970 · Cited by 62
“…approach, which consist in analyzing an already trained (black-box) model by, e.g., interpretable local surrogates, gradient-based or perturbation-based methods [35], and ii) an intrinsic approach, in which the architecture of the model is directly designed with interpretable components [36], [37].”
A Survey of Multimodal Sensor Fusion for Passive RF and EO Information Integration
Vakil, A., Liu, J., Zulch, P. A., Blasch, E., Ewing, R. L., & Li, J. (2021) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/maes.2020.3006410 · Cited by 57
Kruschel, S., Hambauer, N., Weinzierl, S., Zilker, S., Kraus, M., & Zschech, P. (2026) · Business & Information Systems Engineering · doi:10.1007/s12599-024-00922-2 · Cited by 50
Zhang, Z., & Wei, X. (2023) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2023.02.005 · Cited by 41
Explainable Hierarchical Imitation Learning for Robotic Drink Pouring
Zhang, D., Li, Q., Zheng, Y., Wei, L., Zhang, D., & Zhang, Z. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2021.3138280 · Cited by 38
“However, these methods cannot achieve comparable performance as deep learning based methods to pursue accurate prediction [24]–[26].”
Cherrat, E. A., Raj, S., Kerenidis, I., Shekhar, A., Wood, B., Dee, J., Chakrabarti, S., Chen, R., Herman, D., Hu, S., et al. (2023) · Quantum · doi:10.22331/q-2023-11-29-1191 · Cited by 37
“These quantum layers, first defined in [21], naturally provide models that have orthogonal features, improving interpretability [22], allowing for deeper architectures, and providing theoretical and practical benefits in generalization [23].”
BackgroundGünther, J., Reichensdörfer, E., Pilarski, P. M., & Diepold, K. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243320 · Cited by 39
“As neural networks introduce a black box to the system [13], their use for PID parameter tuning is not yet widely accepted.”
Wu, X. H., Liu, S., & Wang, H. (2022) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2022.106694 · Cited by 30
Yan, T., Xing, X., Xia, T., & Wang, D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109046 · Cited by 26
Explainable Machine Learning for Improving Logistic Regression Models
Yang, Y., & Wu, M. (2021) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin45523.2021.9557392 · Cited by 26
Metrics for evaluating the performance of complex engineering system health monitoring models
Lewis, A. D., & Groth, K. M. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108473 · Cited by 20
Designing Inherently Interpretable Machine Learning ModelsPreprint
Sudjianto, A., & Zhang, A. (2021) · arXiv.org · doi:10.48550/arxiv.2111.01743 · Cited by 39
“Here we follow the interpretability constraint paradigm from [17] and attempt to propose a qualitative template for model-based interpretability assessment.”
BackgroundMethodInfluentialSelf-citationPosterior Regularized Bayesian Neural Network incorporating soft and hard knowledge constraints
Huang, J., Pang, Y., Liu, Y., & Yan, H. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110043 · Cited by 19
Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and SimplificationPreprint
Sudjianto, A., Knauth, W., Singh, R., Yang, Z., & Zhang, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04041 · Cited by 57
Self-citationYao, L., Yang, Z., Zhang, Z., Tang, S., Shen, B., & Zeng, J. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3323006 · Cited by 18
“For instance, the explainable boosting machine (EBM) [22] is developed from the standard gradient boosting model (GBM) with powerful approximation ability, where the fit model is interpretable in terms of the main inputs and input interactions [23].”
BackgroundSelf-citationUniversal Approximation Abilities of a Modular Differentiable Neural Network
Wang, J., Wu, S. M., Zhang, H., Yuan, B., Dai, C., & Pal, N. R. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3378697 · Cited by 18
“Inadequate interpretability usually leads to poor performance of the applications deduced from the models [40], [41], [42].”
Cost of Explainability in AI: An Example with Credit Scoring Models
Dessain, J., Bentaleb, N., & Vinas, F. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_26 · Cited by 17
CAeSaR: An Online Payment Anti-Fraud Integration System With Decision Explainability
Wang, C., Chai, S., Zhu, H., & Jiang, C. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3186733 · Cited by 16
“The existing methods integrate various models or features for better performance but lack the consideration of explainability [39], [40], [41].”
MethodA Self-Learning Channel Modeling Approach Based on Explainable Neural Network
Xue, P., & Zhao, Y. (2023) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2023.3272974 · Cited by 15
“At the same time, to strengthen the explainability of the neural network, some constraints are added to limit the structure of the neural network [12].”
BackgroundMethodHwang, E., Park, Y., Kim, J.-Y., Park, S.-H., Kim, S., & Kim, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3273187 · Cited by 15
“2) [33], [34], [35].”
A Survey of Neural Trees: Co-Evolving Neural Networks and Decision Trees
Li, H., Song, J., Xue, M., Zhang, H., & Song, M. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3446891 · Cited by 23
“There have been significant efforts to explore model interpretability and explainable AI [147], [148], [149], [150].”
On Understanding of Spatiotemporal Prediction Model
Huang, X., Li, X., Ye, Y., Feng, S., Luo, C., & Zhang, B. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3232889 · Cited by 13
“Various methods have been proposed to explore the internal mechanism and decision basis of ANNs [7], [8], [9], [10], [11], [12], [13].”
MethodRondinella, F., Daneluz, F., Vacková, P., Valentin, J., & Baldo, N. (2023) · Materials · doi:10.3390/ma16031017 · Cited by 13
“Although neural networks can successfully understand and model even highly nonlinear phenomena producing very accurate predictions [31–37], the difficulties related to the best hyperparameters’ identification and the lack of sufficient interpretability [38] could make them not preferable.”
Towards a Reliable Evaluation of Local Interpretation Methods
Li, J., Lin, D., Wang, Y., Xu, G., & Ding, C. (2021) · Applied Sciences · doi:10.3390/app11062732 · Cited by 11
“Despite the success of deep learning methods in these tasks, the opacity of this method has made it criticized [8].”
BackgroundEmerging role of artificial intelligence in nuclear medicine
Jha, A. K., Mithun, S., Rangarajan, V., Wee, L., & Dekker, A. L. A. J. (2021) · Nuclear Medicine Communications · doi:10.1097/mnm.0000000000001381 · Cited by 14
“The latter degree of acceptability is much harder to quantify and define; however, it relates directly to another major topic of research in artificial intelligence, that is, the degree of ‘explainability’ (or lack thereof) in an artificial intelligence-based decision support system [77].”
BackgroundMonotonic Neural Additive Models: Pursuing Regulated Machine Learning Models for Credit Scoring
Chen, D., & Ye, W. (2022) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3533271.3561691 · Cited by 12
“An alternative direction simplifies ML architectures to increase transparency [2, 5, 8, 24, 25].”
BackgroundIs Deep Learning ready to satisfy Industry needs?
Tripicchio, P., & D’Avella, S. (2020) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.10.167 · Cited by 12
Joy, M. S., Yadav, P. K., Jha, P., Bansal, T., Hằng, H. T., Mallick, J., & Abdo, H. G. (2026) · Geocarto International · doi:10.1080/10106049.2026.2671574 · Cited by 11
Using Decision Tree as Local Interpretable Model in Autoencoder-based LIME
Ranjbar, N., & Safabakhsh, R. (2022) · International Computer Conference, Computer Society of Iran (CSICC) · doi:10.1109/csicc55295.2022.9780503 · Cited by 11
Fink, O., Sharma, V., Nejjar, I., Krannichfeldt, L. V., Garmaev, S., Zhang, Z., Wei, A., Frusque, G., Forest, F., Zhao, M., et al. (2026) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2026.112213 · Cited by 20
Asset pricing via fused deep learning with visual clues
Tan, J., Deveci, M., Li, J., & Zhong, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102049 · Cited by 10
Chengjun, W., Hao, D., & Long, L. (2024) · Robotica · doi:10.1017/s0263574723001881 · Cited by 10
“In contrast, deep learning-based control algorithms have been widely adopted in various robotics applications due to their higher accuracy and faster speed, which can effectively improve the control accuracy and fast model prediction [19]–[20].”
BackgroundBias, Fairness, and Accountability with AI and ML AlgorithmsPreprint
Zhou, N., Zhang, Z., Nair, V. N., Singhal, H., Chen, J., & Sudjianto, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.06558 · Cited by 10
Self-citationDoes Deep Learning Have Epileptic Seizures? On the Modeling of the Brain
Depannemaecker, D., Pio‐Lopez, L., & Gauld, C. (2023) · Cognitive Computation · doi:10.1007/s12559-023-10113-y · Cited by 10
Liu, J., Wang, W., Xia, H., Yu, Y., Lei, X., & Pei, H. (2024) · Results in Physics · doi:10.1016/j.rinp.2024.107716 · Cited by 9
Deger, Z., Taskin, G., & Wallace, J. W. (2024) · Bulletin of Earthquake Engineering · doi:10.1007/s10518-024-01968-z · Cited by 11
“Some approaches developed based on the GAM framework include Neural additive model (NAM) (Agarwal et al. 2021), GAMI-Net (Yang et al. 2021), and Enhanced explainable neural network (ExNN) (Yang et al. 2020).”
Model Simplification of Deep Random Forest for Real-Time Applications of Various Sensor Data
Kim, S., Ko, B. C., & Nam, J.-Y. (2021) · Sensors · doi:10.3390/s21093004 · Cited by 11
“Yang et al. [22] proposed the use of an explain-able NN (xNN) subject to interpretability constraints in terms of the additivity, sparsity, orthogonality, and smoothness.”
BackgroundFuzzy-Based, Noise-Resilient, Explainable Algorithm for Regression
Viaña, J., & Cohen, K. D. O. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-82099-2_42 · Cited by 8
A Spatiotemporal Dynamic Wavelet Network for Infrared Thermography-Based Machine Prognostics
Jiang, Y., Xia, T., Wang, D., Xu, Y., Li, R., Pan, E., & Xi, L. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3321746 · Cited by 8
“Compared to passively explaining learned features and network outputs by post-hoc methods, ante-hoc methods are more attractive in diagnosis or prognostics [13].”
MethodSparse-enhanced additive interaction neural network for interpretable credit decision
Lan, X., Fan, H., Liu, W., Xia, M., & Guo, K. (2025) · Decision Support Systems · doi:10.1016/j.dss.2025.114507 · Cited by 9
Interpretation and Simplification of Deep ForestPreprint
Kim, S., Jeong, M., & Ko, B. C. (2020) · doi:10.36227/techrxiv.11661246.v1 · Cited by 7
FAME: Introducing Fuzzy Additive Models for Explainable AI
Gökmen, Ö. B., Güven, Y., & Kumbasar, T. (2025) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz62266.2025.11152211 · Cited by 7
Benchmarking alternative interpretable machine learning models for corporate probability of default
Jacobs, M. R., & Ph.D, C. (2024) · Data Science in Finance and Economics · doi:10.3934/dsfe.2024001 · Cited by 6
On marginal feature attributions of tree-based models
Filom, K., Miroshnikov, A., Kotsiopoulos, K., & Kannan, A. R. (2024) · Foundations of Data Science · doi:10.3934/fods.2024021 · Cited by 9
“The benefit of symmetry to interpretability observed in the present article is reminiscent of a general philosophy that intrinsic interpretability should be induced from certain constraints [64].”
Creating Unbiased Machine Learning Models by Design
Breeden, J. L., & Leonova, E. (2021) · Journal of risk and financial management · doi:10.3390/jrfm14110565 · Cited by 6
Baldo, N., Rondinella, F., Daneluz, F., Vacková, P., Valentin, J., Gajewski, M. D., & Król, J. (2023) · CivilEng · doi:10.3390/civileng4040059 · Cited by 6
“However, the difficulty of interpreting neural models coupled with the challenges associated with the optimization of the multiple hyperparameters involved [35] make them somewhat difficult to handle.”
BackgroundInterpretable Machine Learning Based on Functional ANOVA Framework: Algorithms and Comparisons
Hu, L., Nair, V. N., Sudjianto, A., Zhang, A., Chen, J., & Yang, Z. (2025) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.2916 · Cited by 6
Self-citationManifoldron: Direct Space Partition via Manifold Discovery
Wang, D., Fan, F., Hou, B., Zhang, H., Jia, Z., Zhang, B., Lai, R., Yu, H., & Wang, F. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3486252 · Cited by 6
“…machine learning algorithms: KNN [42], SVM with a linear kernel (Linear-SVM), SVM with a RBF kernel (RBF-SVM) [43], C4.5 [44], RF [45], NN [1], AdaBoost [18], Naive Bayes [46], QDA [47], XGBoost [48], LightGBM [21], ExNN [49], DFMNN [50], SNN [51], QNN [52], SwitchTab [53], and PTaRL [54].”
Instructive artificial intelligence (AI) for human training, assistance, and explainability
Kantack, N., Cohen, N., Bos, N., Lowman, C., Everett, J., & Endres, T. (2021) · Defense + Commercial Sensing · doi:10.1117/12.2618616 · Cited by 6
“13,14 In the case of the JHU/APL Hanabi agent, an expected reward for each possible action is computed based on the expected effect the action will have on the HPFs.”
Using phidelta diagrams to discover relevant patterns in multilayer perceptrons
Armano, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-76517-0 · Cited by 4
Analyzing User Characteristics of Hate Speech Spreaders on Social Media
Geissler, D., Maarouf, A., & Feuerriegel, S. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714502 · Cited by 6
“In addition, we design the following variants of our debiasing framework that we also use as baselines: (2) DF-ExNN where we replace the EBM with an explainable neural network [52].”
Explaining Explainability: Interpretable machine learning for the behavioral sciencesPreprint
Kennedy, B., Reimer, N. K., & Dehghani, M. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/9h6qr · Cited by 4
Alaoui, E. A. A., Sahlaoui, H., Sallah, A., & Nayyar, A. (2026) · Information Sciences · doi:10.1016/j.ins.2026.123128 · Cited by 4
Artificial Intelligence as the Next Visionary in Liquid Crystal Research
Astam, M. O., L'Karkouri, O., & Liu, D. (2026) · Advanced Functional Materials · doi:10.1002/adfm.202529976 · Cited by 3
“Asecondmajorconcernrevolvesaroundexplainability,especially for deep learning AI models with a multitude of ‘layers’, where the process that the input is subjected to becomes intangible for the researcher, despite claiming high predictive accuracy [88–90].”
Li, L. (2024) · Nonlinear Engineering · doi:10.1515/nleng-2024-0047 · Cited by 3
“In the experiment, this method improved the interpretability of the model while also making the predictive performance comparable to those of multiple benchmark models [10].”
Jurišica, I. (2022) · Springer eBooks · doi:10.1007/978-3-031-00119-2_13 · Cited by 3
Traversing the Local Polytopes of ReLU Neural Networks: A Unified Approach for Network VerificationPreprint
Xu, S., Vaughan, J., Chen, J., Zhang, A., & Sudjianto, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08922 · Cited by 3
“Built upon this framework, multiple studies [31], [34], [35] tried to explain the behavior of ReLU NNs and to improve their interpretability.”
Self-citationWang, K., Zhu, M., Chen, Z., Weng, J., Li, M., Yiu, S., Ding, W., & Gu, T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3359269 · Cited by 2
“Yang et al. [40] enhances the interpretability of the model through architectural constraints.”
Linear iterative feature embedding: an ensemble framework for an interpretable model
Sudjianto, A., Qiu, J., Li, M., & Chen, J. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08204-w · Cited by 2
“…researchers made efforts to build inherently interpretable model, such as explainable neural network (xNN) proposed by (Vaughan et al. (2018) [26]), adaptive explainable neural networks (AxNNs) by (Chen et al. (2020) [8]), and explainable neural networks with constraint (Yang et al. (2020) [28]).”
BackgroundSelf-citationA distributed and unified API service for machine learning models
Nandigramwar, H., Mittal, A., Bhatnagar, A., & Rashid, M. U. (2021) · International Conference on Intelligent Engineering and Management (ICIEM) · doi:10.1109/iciem51511.2021.9445348 · Cited by 4
“Different architectures extract different features and the attention to these features also differs [8, 12].”
BackgroundSudjianto, A., Yang, Z., & Zhang, A. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3126615 · Cited by 2
“Some recent efforts towards this line of research can be referred to, for instance, causal decision trees [9], the enhanced explainable neural network [10], and the explainable neural network based on generalized additive models and structured pairwise interactions [11].”
BackgroundMethodSelf-citationGuo, Y., Su, Y., Yang, Z., & Zhang, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08196 · Cited by 2
“Such a statistically interpretable model is a special case of additive index model (AIM) that has been successfully re-deployed into the explainable neural network (xNN) architectures (Vaughan et al., 2018; Yang et al., 2020) through univariate-input subnetwork parametrization of h(·) in (1).”
InfluentialSelf-citation2021 Index IEEE Transactions on Neural Networks and Learning Systems Vol. 32
(2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3134132 · Cited by 2
Less Discriminatory Alternative and Interpretable XGBoost Framework for Binary ClassificationPreprint
Pangia, A. C., Sudjianto, A., Zhang, A., & Khan, T. R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4998508 · Cited by 2
Self-citationGAM-GCN: Decomposing Graph Convolutional Networks via Generalized Additive Models
Liu, C., Yu, L., Han, K., & Yin, G. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00033 · Cited by 1
“Recently, it draws extensive attentions that GAM can also be reparametrized using neural network architectures [17], which shares the same idea as that in explainable neural networks [18], [19].”
周武彦, Liu, X., Yu, J., Li, Y., Cheng, L., Wang, G., & Jiang, F. (2026) · The Saudi Dental Journal · doi:10.1007/s44445-026-00160-0 · Cited by 1
Harnessing Orthogonality to Train Low-Rank Neural Networks
Coquelin, D., Flügel, K., Weiel, M., Kiefer, N., Debus, C., Streit, A., & Götz, M. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240729 · Cited by 1
“1”
Layme, R. C., Flores, J. G., & Segura, G. W. Z. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-70518-2_6 · Cited by 1
Pairwise networks for feature ranking of a geomagnetic storm model
Beukes, J., Lotz, S., & Davel, M. H. (2020) · South African Computer Journal · doi:10.18489/sacj.v32i2.860 · Cited by 1
“GAMI-Net (GAMs with structured Interactions) (Yang et al., 2020b) essentially adds pairwise interactions to GANNs, approximating: g(E[y]) = β + ∑ fi(xi) + ∑ fi,j(xi, xj) (4) Neural Interaction Transparency (NIT) (Tsang et al., 2018) is a framework that produces the same model as GAMI-Net, but by disentangling interact…”
MethodNeural Network Feature Explanation Using Neuron Activation Rate Based Bipartite Graph
Kuttichira, D. P., Azam, B., Verma, B., Rahman, A., Wang, L., & Sattar, A. (2023) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz61134.2023.10343968 · Cited by 1
“Imposing certain constraints on these networks is another approach to explain the decisions made by these models[15, 4].”
BackgroundHu, B.-G., & Qu, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06156 · Cited by 4
“For understanding AI machines as a rigorous science [30], more investigations have been reported, such as [31]– [35], in together with review papers, like [36]–[39].”
BackgroundInterpretation and Simplification of Deep ForestPreprint
Kim, S., Jeong, M., & Ko, B. C. (2020) · doi:10.36227/techrxiv.11661246 · Cited by 1
Deep Learning Approach for Interference Mitigation in MIMO-FBMC/OQAM Systems
Bedoui, A., & Et‐tolba, M. (2021) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-77185-0_2
Cheng, L., Zang, H., Han, H., Wei, Z., & Sun, G. (2022) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci51031.2022.10022193
A Sequential Stein’s Method for Faster Training of Additive Index Models
Zhang, H., Yang, Z., Sudjianto, A., & Zhang, A. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3273474
“In the synthetic experiments, we generate data from six cases with reference to [5].”
BackgroundMethodInfluentialSelf-citationNovel Automatic Deep Learning Feature Extractor with Target Class Specific Feature Explanations
Kuttichira, D. P., Verma, B., Rahman, A., & Wang, L. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191143
“Many works try to achieve explanations for these models by imposing constraints on these deep neural network model [7], [9].”
BackgroundJacobs, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4583014
Forcing the Network to Use Human Explanations in Its Inference Process
Viaña, J., & Vanderburg, A. M. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-46778-3_12
“Other researchers have considered adding constrains to the learning process [43, 44, 45] but it has not been done with the intention of forcing the machine to think or replicate human reasoning.”
BackgroundExplicable Integration Techniques: Relative Temporal Position Taxonomy
Wang, C. (2023) · Springer eBooks · doi:10.1007/978-981-99-5257-1_4
Mulvey, D., Foh, C. H., Imran, M. A., & Tafazolli, R. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3373101
“A recent paper by Yang et al. [13] describes an implementation approach in line with the second strategy which accepts architectural constraints on the NN in return for greater explainability.”
xNN-SF: An Explainable Neural Network Inspired by Stochastic Frontier Model
Zhao, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5588-2_37
A novel tree-based method for interpretable reinforcement learning
Li, Y., Qi, S., Wang, X., Zhang, J., & Cui, L. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3695464
“Although many studies have focused on exploring the theory of decision-making using neural networks and designing explainable XAI approaches [19, 51, 61], not all have focused on explaining RL agents.”
Understanding the Role of Neurons and Layers in One-Dimensional Three-Class Classifiers
Emerald, S. C., & Vengattaraman, T. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-9196-8_29
Interpreting atmospheric neural networks: insights from explainable AI
Vojtekova, A., Νικολάου, Ν., Yip, K. H., Vénot, O., Al-Refaie, A. F., Merín, B., & Waldmann, I. P. (2026) · RAS Techniques and Instruments · doi:10.1093/rasti/rzag002
FPENet: an explainable frequency prior embedded deep neural network for intelligent fault diagnosisPreprint
Li, S., Li, T., Sun, C., Yan, R., Chen, X., & Zio, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6271684
Metadata
- Authors
- Yang, Z., Zhang, A., & Sudjianto, A.
- Affiliations
- University of Hong Kong (HK)
- Wells Fargo (United States) (US)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2021-06
- Volume
- 32
- Issue
- 6
- Pages
- 2610-2621
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W3045825034
- Cited by
- 106
- References
- 37
- Citations by year
- 2026: 13 · 2025: 6 · 2024: 19 · 2023: 22 · 2022: 15 · 2021: 22
- FWCI
- 6.68
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Big Data Project Fund of The University of Hong Kong through Dr. Patrick Poon’s Donation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- explainable neural networks, projection pursuit, orthogonality constraints, smooth function approximation, identifiability conditions, Cayley transform, backpropagation, model interpretability, prediction accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
