Enhancing Explainability of Neural Networks Through Architecture Constraints
Open accessAbstract
Prediction accuracy and model explainability are the two most important objectives when developing machine learning algorithms to solve real-world problems. Neural networks are known to possess good prediction performance but suffer from a lack of model interpretability. In this article, we propose to enhance the explainability of neural networks through the following architecture constraints: 1) sparse additive subnetworks; 2) projection pursuit with orthogonality constraint; and 3) smooth function approximation. It leads to an enhanced explainable neural network (ExNN) with a superior balance between prediction performance and model interpretability. We derive sufficient identifiability conditions for the proposed ExNN model. The multiple parameters are simultaneously estimated by a modified minibatch gradient descent method based on the backpropagation algorithm for calculating the derivatives and the Cayley transform for preserving the projection orthogonality. Through simulation study under six different scenarios, we compare the proposed method to several benchmarks, including least absolute shrinkage and selection operator, support vector machine, random forest, extreme learning machine, and multilayer perceptron. It is shown that the proposed ExNN model keeps the flexibility of pursuing high prediction accuracy while attaining improved interpretability. Finally, a real data example is employed as a showcase application.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In tightly controlled domains such as IoT systems, avionics, finance, banking, autonomous systems, and medical services, integrating AI and its interoperability with legacy applications is much more onerous.
“However, in tightly controlled domains like IoT systems, avionics, finance, banking, autonomous systems, and medical services, integrating AI and its interoperability with legacy applications is much more onerous [27], [47].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [47]
The Introduction (p. 2) says that limited interpretability can restrict neural-network applications in medicine and finance, and discusses credit scoring in banking. It does not address interoperability with legacy applications or the claimed integration difficulty across IoT, avionics, and autonomous systems. (LLM, conf=0.80)
Experimental — accuracy not yet measuredBlack-box models may still make poor predictions even in the absence of adversarial attacks due to significant biases and prejudices in the training set.
“Leading to significant biases and prejudices in the training set, black-box models may still make poor predictions even in the absence of adversarial attacks [28], [46], [47].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [47]
The Introduction (p. 2) discusses black-box models’ limited interpretability and generally strong predictive performance, but not poor predictions caused by biased training data. Searches for training-set bias, prejudice, and adversarial attacks found no passage addressing the attributed claim. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
(2010) · Cited by 14,134 · see citing works
Extreme learning machine: Theory and applications
Huang, G.-B., Zhu, Q.-Y., & Siew, C.-K. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.126 · Cited by 13,285 · see citing works
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Lu, Z. Q. J. (2010) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,904 · see citing works
Brown, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Biometrics · doi:10.2307/2532174 · Cited by 8,262 · see citing works
Koyak, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Journal of the American Statistical Association · doi:10.2307/2290538 · Cited by 7,717 · see citing works
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091 · see citing works
Spline Models for Observational Data.
Müller, H., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,992 · see citing works
Spline Models for Observational Data
Wahba, G. (1990) · doi:10.1137/1.9781611970128 · Cited by 4,184 · see citing works
Sparse Principal Component Analysis
Zou, H., Hastie, T., & Tibshirani, R. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x113430 · Cited by 3,152 · see citing works
Friedman, J. H., & Stuetzle, W. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477729 · Cited by 2,116 · see citing works
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068 · see citing works
Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861 · see citing works
Do Deep Nets Really Need to be Deep?Preprint
Ba, J. L., & Caruana, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.6184 · Cited by 1,592 · see citing works
Visualizing the effects of predictor variables in black box supervised learning models
Apley, D. W., & Zhu, J. (2020) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12377 · Cited by 1,535 · see citing works
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360 · see citing works
Semiparametric least squares (SLS) and weighted SLS estimation of single-index models
Ichimura, H. (1993) · Journal of Econometrics · doi:10.1016/0304-4076(93)90114-k · Cited by 1,224 · see citing works
A feasible method for optimization with orthogonality constraints
Wen, Z., & Yin, W. (2012) · Mathematical Programming · doi:10.1007/s10107-012-0584-1 · Cited by 897 · see citing works
A Note on the Use of Principal Components in Regression
Jolliffe, I. T. (1982) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2348005 · Cited by 856 · see citing works
Prediction by Supervised Principal Components
Bair, E., Hastie, T. J., Paul, D., & Tibshirani, R. J. (2006) · Journal of the American Statistical Association · doi:10.1198/016214505000000628 · Cited by 822 · see citing works
Ravikumar, P., Lafferty, J., Liu, H., & Wasserman, L. (2009) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.1467-9868.2009.00718.x · Cited by 548 · see citing works
Barshan, E., Ghodsi, A., Azimifar, Z., & Jahromi, M. Z. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.12.015 · Cited by 343 · see citing works
Regression modeling in back-propagation and projection pursuit learning
Hwang, J., Lay, S.-R., Maechler, M., Martin, R. D., & Schimert, J. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.286906 · Cited by 285 · see citing works
Visualizing the Effects of Predictor Variables in Black Box Supervised Learning ModelsPreprint
Apley, D. W., & Zhu, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08468 · Cited by 247 · see citing works
A Novel Pruning Algorithm for Smoothing Feedforward Neural Networks Based on Group Lasso Method
Wang, J., Xu, C., Xi-feng, Y., & Żurada, J. M. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2748585 · Cited by 99 · see citing works
Generalized Additive and Index Models with Shape Constraints
Chen, Y., & Samworth, R. J. (2015) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12137 · Cited by 80 · see citing works
Fernández‐Navarro, F., Hervás‐Martínez, C., Sánchez‐Monedero, J., & Gutiérrez, P. A. (2011) · Neurocomputing · doi:10.1016/j.neucom.2010.11.032 · Cited by 74 · see citing works
Explainable Neural Networks based on Additive Index ModelsPreprint
Vaughan, J., Sudjianto, A., Brahimi, E., Chen, J., & Nair, V. N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01933 · Cited by 74 · see citing works
Self-citationLocally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)Preprint
Hu, L., Chen, J., Nair, V. N., & Sudjianto, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00663 · Cited by 38 · see citing works
Self-citationOn the identifiability of additive index models
Yuan, M. (2011) · Statistica Sinica · doi:10.5705/ss.2008.117 · Cited by 33 · see citing works
Liu, X., Chen, J., Vaughan, J., Nair, V. N., & Sudjianto, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07216 · Cited by 5 · see citing works
Self-citation
Flagged first, then by citations
2 of the 37 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983 · see citing works
Kufel, J., Bargieł-Łączek, K., Kocot, S., Koźlik, M., Bartnikowska, W., Janik, M., Czogalik, Ł., Dudek, P., Magiera, M., Lis, A., et al. (2023) · Diagnostics · doi:10.3390/diagnostics13152582 · Cited by 401 · see citing works
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352 · see citing works
“For instance, the European Union’s GDPR framework, which was been passed in 2016 and went into effect in May 2018, established criteria to support the ‘‘entitlement of an explanation’’ premise for users [47].”
BackgroundMethodIslam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 339 · see citing works
A Survey of Binary Code Similarity
Haq, I. U., & Caballero, J. (2021) · ACM Computing Surveys · doi:10.1145/3446371 · Cited by 158 · see citing works
Degas, A., Islam, M. R., Hurter, C., Barua, S., Rahman, H., Poudel, M., Ruscio, D., Ahmed, M. U., Begum, S., Rahman, M. A., et al. (2022) · Applied Sciences · doi:10.3390/app12031295 · Cited by 168 · see citing works
Yang, Z., Zhang, A., & Sudjianto, A. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108192 · Cited by 144 · see citing works
Self-citationA Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3485275 · Cited by 137 · see citing works
Reinforcement learning in spacecraft control applications: Advances, prospects, and challenges
Tipaldi, M., Iervolino, R., & Massenio, P. R. (2022) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2022.07.004 · Cited by 113 · see citing works
Huang, C., Huang, Q., Huang, X., Wang, H., Li, M., Lin, K.-J., & Chang, Y. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3418098 · Cited by 110 · see citing works
Brasse, J., Broder, H. R., Förster, M., Klier, M., & Sigler, I. (2023) · Electronic Markets · doi:10.1007/s12525-023-00644-5 · Cited by 101 · see citing works
A novel ensemble-based conceptual-data-driven approach for improved streamflow simulations
Sikorska‐Senoner, A. E., & Quilty, J. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2021.105094 · Cited by 77 · see citing works
Psychological foundations of explainability and interpretability in artificial intelligence
Broniatowski, D. A. (2021) · doi:10.6028/nist.ir.8367 · Cited by 76 · see citing works
Deep learning for pH prediction in water desalination using membrane capacitive deionization
Son, M., Yoon, N., Jeong, K., Abbas, A., Logan, B. E., & Cho, K. H. (2021) · Desalination · doi:10.1016/j.desal.2021.115233 · Cited by 73 · see citing works
Interpretable generalized additive neural networks
Kraus, M., Tschernutter, D., Weinzierl, S., & Zschech, P. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.06.032 · Cited by 59 · see citing works
Bottieau, J., Wang, Y., Grève, Z. D., Vallée, F., & Toubeau, J. (2022) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2022.3195970 · Cited by 57 · see citing works
A Survey of Multimodal Sensor Fusion for Passive RF and EO Information Integration
Vakil, A., Liu, J., Zulch, P. A., Blasch, E., Ewing, R. L., & Li, J. (2021) · IEEE Aerospace and Electronic Systems Magazine · doi:10.1109/maes.2020.3006410 · Cited by 57 · see citing works
Kruschel, S., Hambauer, N., Weinzierl, S., Zilker, S., Kraus, M., & Zschech, P. (2025) · Business & Information Systems Engineering · doi:10.1007/s12599-024-00922-2 · Cited by 52 · see citing works
Zhang, Z., & Wei, X. (2023) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2023.02.005 · Cited by 41 · see citing works
Explainable Hierarchical Imitation Learning for Robotic Drink Pouring
Zhang, D., Li, Q., Zheng, Y., Wei, L., Zhang, D., & Zhang, Z. (2021) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2021.3138280 · Cited by 38 · see citing works
Cherrat, E. A., Raj, S., Kerenidis, I., Shekhar, A., Wood, B., Dee, J., Chakrabarti, S., Chen, R., Herman, D., Hu, S., et al. (2023) · Quantum · doi:10.22331/q-2023-11-29-1191 · Cited by 37 · see citing works
Günther, J., Reichensdörfer, E., Pilarski, P. M., & Diepold, K. J. (2020) · PLoS ONE · doi:10.1371/journal.pone.0243320 · Cited by 31 · see citing works
Wu, X. H., Liu, S., & Wang, H. (2022) · Communications in Nonlinear Science and Numerical Simulation · doi:10.1016/j.cnsns.2022.106694 · Cited by 30 · see citing works
Yan, T., Xing, X., Xia, T., & Wang, D. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109046 · Cited by 26 · see citing works
Explainable Machine Learning for Improving Logistic Regression Models
Yang, Y., & Wu, M. (2021) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin45523.2021.9557392 · Cited by 22 · see citing works
Metrics for evaluating the performance of complex engineering system health monitoring models
Lewis, A. D., & Groth, K. M. (2022) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2022.108473 · Cited by 20 · see citing works
Designing Inherently Interpretable Machine Learning ModelsPreprint
Sudjianto, A., & Zhang, A. (2021) · arXiv.org · doi:10.48550/arxiv.2111.01743 · Cited by 39 · see citing works
Self-citationPosterior Regularized Bayesian Neural Network incorporating soft and hard knowledge constraints
Huang, J., Pang, Y., Liu, Y., & Yan, H. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110043 · Cited by 19 · see citing works
Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and SimplificationPreprint
Sudjianto, A., Knauth, W., Singh, R., Yang, Z., & Zhang, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04041 · Cited by 19 · see citing works
Self-citationYao, L., Yang, Z., Zhang, Z., Tang, S., Shen, B., & Zeng, J. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3323006 · Cited by 18 · see citing works
Self-citationUniversal Approximation Abilities of a Modular Differentiable Neural Network
Wang, J., Wu, S. M., Zhang, H., Yuan, B., Dai, C., & Pal, N. R. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3378697 · Cited by 18 · see citing works
Cost of Explainability in AI: An Example with Credit Scoring Models
Dessain, J., Bentaleb, N., & Vinas, F. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_26 · Cited by 17 · see citing works
CAeSaR: An Online Payment Anti-Fraud Integration System With Decision Explainability
Wang, C., Chai, S., Zhu, H., & Jiang, C. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3186733 · Cited by 16 · see citing works
A Self-Learning Channel Modeling Approach Based on Explainable Neural Network
Xue, P., & Zhao, Y. (2023) · IEEE Wireless Communications Letters · doi:10.1109/lwc.2023.3272974 · Cited by 15 · see citing works
Hwang, E., Park, Y., Kim, J.-Y., Park, S.-H., Kim, S., & Kim, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3273187 · Cited by 15 · see citing works
A Survey of Neural Trees: Co-Evolving Neural Networks and Decision Trees
Li, H., Song, J., Xue, M., Zhang, H., & Song, M. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3446891 · Cited by 14 · see citing works
On Understanding of Spatiotemporal Prediction Model
Huang, X., Li, X., Ye, Y., Feng, S., Luo, C., & Zhang, B. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3232889 · Cited by 13 · see citing works
Rondinella, F., Daneluz, F., Vacková, P., Valentin, J., & Baldo, N. (2023) · Materials · doi:10.3390/ma16031017 · Cited by 13 · see citing works
Towards a Reliable Evaluation of Local Interpretation Methods
Li, J., Lin, D., Wang, Y., Xu, G., & Ding, C. (2021) · Applied Sciences · doi:10.3390/app11062732 · Cited by 13 · see citing works
Emerging role of artificial intelligence in nuclear medicine
Jha, A. K., Mithun, S., Rangarajan, V., Wee, L., & Dekker, A. L. A. J. (2021) · Nuclear Medicine Communications · doi:10.1097/mnm.0000000000001381 · Cited by 13 · see citing works
Monotonic Neural Additive Models: Pursuing Regulated Machine Learning Models for Credit Scoring
Chen, D., & Ye, W. (2022) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3533271.3561691 · Cited by 12 · see citing works
Is Deep Learning ready to satisfy Industry needs?
Tripicchio, P., & D’Avella, S. (2020) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.10.167 · Cited by 12 · see citing works
Joy, M. S., Yadav, P. K., Jha, P., Bansal, T., Hằng, H. T., Mallick, J., & Abdo, H. G. (2026) · Geocarto International · doi:10.1080/10106049.2026.2671574 · Cited by 11 · see citing works
Using Decision Tree as Local Interpretable Model in Autoencoder-based LIME
Ranjbar, N., & Safabakhsh, R. (2022) · International Computer Conference, Computer Society of Iran (CSICC) · doi:10.1109/csicc55295.2022.9780503 · Cited by 11 · see citing works
Fink, O., Sharma, V., Nejjar, I., Krannichfeldt, L. V., Garmaev, S., Zhang, Z., Wei, A., Frusque, G., Forest, F., Zhao, M., et al. (2026) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2026.112213 · Cited by 10 · see citing works
Asset pricing via fused deep learning with visual clues
Tan, J., Deveci, M., Li, J., & Zhong, K. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102049 · Cited by 10 · see citing works
Chengjun, W., Hao, D., & Long, L. (2024) · Robotica · doi:10.1017/s0263574723001881 · Cited by 10 · see citing works
Bias, Fairness, and Accountability with AI and ML AlgorithmsPreprint
Zhou, N., Zhang, Z., Nair, V. N., Singhal, H., Chen, J., & Sudjianto, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.06558 · Cited by 10 · see citing works
Self-citationDoes Deep Learning Have Epileptic Seizures? On the Modeling of the Brain
Depannemaecker, D., Pio‐Lopez, L., & Gauld, C. (2023) · Cognitive Computation · doi:10.1007/s12559-023-10113-y · Cited by 9 · see citing works
Liu, J., Wang, W., Xia, H., Yu, Y., Lei, X., & Pei, H. (2024) · Results in Physics · doi:10.1016/j.rinp.2024.107716 · Cited by 9 · see citing works
Deger, Z., Taskin, G., & Wallace, J. W. (2024) · Bulletin of Earthquake Engineering · doi:10.1007/s10518-024-01968-z · Cited by 11 · see citing works
Model Simplification of Deep Random Forest for Real-Time Applications of Various Sensor Data
Kim, S., Ko, B. C., & Nam, J.-Y. (2021) · Sensors · doi:10.3390/s21093004 · Cited by 9 · see citing works
Fuzzy-Based, Noise-Resilient, Explainable Algorithm for Regression
Viaña, J., & Cohen, K. D. O. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-82099-2_42 · Cited by 8 · see citing works
A Spatiotemporal Dynamic Wavelet Network for Infrared Thermography-Based Machine Prognostics
Jiang, Y., Xia, T., Wang, D., Xu, Y., Li, R., Pan, E., & Xi, L. (2023) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3321746 · Cited by 8 · see citing works
Sparse-enhanced additive interaction neural network for interpretable credit decision
Lan, X., Fan, H., Liu, W., Xia, M., & Guo, K. (2025) · Decision Support Systems · doi:10.1016/j.dss.2025.114507 · Cited by 7 · see citing works
Interpretation and Simplification of Deep ForestPreprint
Kim, S., Jeong, M., & Ko, B. C. (2020) · doi:10.36227/techrxiv.11661246.v1 · Cited by 7 · see citing works
FAME: Introducing Fuzzy Additive Models for Explainable AI
Gökmen, Ö. B., Güven, Y., & Kumbasar, T. (2025) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz62266.2025.11152211 · Cited by 7 · see citing works
Benchmarking alternative interpretable machine learning models for corporate probability of default
Jacobs, M. R., & Ph.D, C. (2024) · Data Science in Finance and Economics · doi:10.3934/dsfe.2024001 · Cited by 6 · see citing works
On marginal feature attributions of tree-based models
Filom, K., Miroshnikov, A., Kotsiopoulos, K., & Kannan, A. R. (2024) · Foundations of Data Science · doi:10.3934/fods.2024021 · Cited by 6 · see citing works
Creating Unbiased Machine Learning Models by Design
Breeden, J. L., & Leonova, E. (2021) · Journal of risk and financial management · doi:10.3390/jrfm14110565 · Cited by 6 · see citing works
Baldo, N., Rondinella, F., Daneluz, F., Vacková, P., Valentin, J., Gajewski, M. D., & Król, J. (2023) · CivilEng · doi:10.3390/civileng4040059 · Cited by 6 · see citing works
Interpretable Machine Learning Based on Functional ANOVA Framework: Algorithms and Comparisons
Hu, L., Nair, V. N., Sudjianto, A., Zhang, A., Chen, J., & Yang, Z. (2025) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.2916 · Cited by 6 · see citing works
Self-citationManifoldron: Direct Space Partition via Manifold Discovery
Wang, D., Fan, F., Hou, B., Zhang, H., Jia, Z., Zhang, B., Lai, R., Yu, H., & Wang, F. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3486252 · Cited by 6 · see citing works
Instructive artificial intelligence (AI) for human training, assistance, and explainability
Kantack, N., Cohen, N. E., Bos, N., Lowman, C., Everett, J., & Endres, T. (2022) · Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications · doi:10.1117/12.2618616 · Cited by 5 · see citing works
Using phidelta diagrams to discover relevant patterns in multilayer perceptrons
Armano, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-76517-0 · Cited by 4 · see citing works
Analyzing User Characteristics of Hate Speech Spreaders on Social Media
Geissler, D., Maarouf, A., & Feuerriegel, S. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714502 · Cited by 4 · see citing works
Explaining Explainability: Interpretable machine learning for the behavioral sciencesPreprint
Kennedy, B., Reimer, N. K., & Dehghani, M. (2021) · doi:10.31234/osf.io/9h6qr · Cited by 4 · see citing works
Alaoui, E. A. A., Sahlaoui, H., Sallah, A., & Nayyar, A. (2026) · Information Sciences · doi:10.1016/j.ins.2026.123128 · Cited by 4 · see citing works
Artificial Intelligence as the Next Visionary in Liquid Crystal Research
Astam, M. O., L'Karkouri, O., & Liu, D. (2026) · Advanced Functional Materials · doi:10.1002/adfm.202529976 · Cited by 3 · see citing works
Li, L. (2024) · Nonlinear Engineering · doi:10.1515/nleng-2024-0047 · Cited by 3 · see citing works
Jurišica, I. (2022) · Springer eBooks · doi:10.1007/978-3-031-00119-2_13 · Cited by 3 · see citing works
Traversing the Local Polytopes of ReLU Neural Networks: A Unified Approach for Network VerificationPreprint
Xu, S., Vaughan, J., Chen, J., Zhang, A., & Sudjianto, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08922 · Cited by 3 · see citing works
Self-citationWang, K., Zhu, M., Chen, Z., Weng, J., Li, M., Yiu, S., Ding, W., & Gu, T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3359269 · Cited by 2 · see citing works
Linear iterative feature embedding: an ensemble framework for an interpretable model
Sudjianto, A., Qiu, J., Li, M., & Chen, J. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08204-w · Cited by 2 · see citing works
Self-citationA distributed and unified API service for machine learning models
Nandigramwar, H., Mittal, A., Bhatnagar, A., & Rashid, M. U. (2021) · International Conference on Intelligent Engineering and Management (ICIEM) · doi:10.1109/iciem51511.2021.9445348 · Cited by 2 · see citing works
Sudjianto, A., Yang, Z., & Zhang, A. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3126615 · Cited by 2 · see citing works
Self-citationGuo, Y., Su, Y., Yang, Z., & Zhang, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08196 · Cited by 2 · see citing works
Self-citation2021 Index IEEE Transactions on Neural Networks and Learning Systems Vol. 32
(2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3134132 · Cited by 2 · see citing works
Less Discriminatory Alternative and Interpretable XGBoost Framework for Binary ClassificationPreprint
Pangia, A. C., Sudjianto, A., Zhang, A., & Khan, T. R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4998508 · Cited by 2 · see citing works
Self-citationGAM-GCN: Decomposing Graph Convolutional Networks via Generalized Additive Models
Liu, C., Yu, L., Han, K., & Yin, G. (2024) · IEEE International Conference on Knowledge Graph (ICKG) · doi:10.1109/ickg63256.2024.00033 · Cited by 1 · see citing works
周武彦, Liu, X., Yu, J., Li, Y., Cheng, L., Wang, G., & Jiang, F. (2026) · The Saudi Dental Journal · doi:10.1007/s44445-026-00160-0 · Cited by 1 · see citing works
Harnessing Orthogonality to Train Low-Rank Neural Networks
Coquelin, D., Flügel, K., Weiel, M., Kiefer, N., Debus, C., Streit, A., & Götz, M. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240729 · Cited by 1 · see citing works
Layme, R. C., Flores, J. G., & Segura, G. W. Z. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-70518-2_6 · Cited by 1 · see citing works
Pairwise networks for feature ranking of a geomagnetic storm model
Beukes, J., Lotz, S., & Davel, M. H. (2020) · South African Computer Journal · doi:10.18489/sacj.v32i2.860 · Cited by 1 · see citing works
Neural Network Feature Explanation Using Neuron Activation Rate Based Bipartite Graph
Kuttichira, D. P., Azam, B., Verma, B., Rahman, A., Wang, L., & Sattar, A. (2023) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz61134.2023.10343968 · Cited by 1 · see citing works
Hu, B.-G., & Qu, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06156 · Cited by 1 · see citing works
Interpretation and Simplification of Deep ForestPreprint
Kim, S., Jeong, M., & Ko, B. C. (2020) · doi:10.36227/techrxiv.11661246 · Cited by 1 · see citing works
Deep Learning Approach for Interference Mitigation in MIMO-FBMC/OQAM Systems
Bedoui, A., & Et‐tolba, M. (2021) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-77185-0_2
Cheng, L., Zang, H., Han, H., Wei, Z., & Sun, G. (2022) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci51031.2022.10022193
A Sequential Stein’s Method for Faster Training of Additive Index Models
Zhang, H., Yang, Z., Sudjianto, A., & Zhang, A. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3273474
Self-citationNovel Automatic Deep Learning Feature Extractor with Target Class Specific Feature Explanations
Kuttichira, D. P., Verma, B., Rahman, A., & Wang, L. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191143
Jacobs, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4583014
Forcing the Network to Use Human Explanations in Its Inference Process
Viaña, J., & Vanderburg, A. M. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-46778-3_12
Explicable Integration Techniques: Relative Temporal Position Taxonomy
Wang, C. (2023) · Springer eBooks · doi:10.1007/978-981-99-5257-1_4
Mulvey, D., Foh, C. H., Imran, M. A., & Tafazolli, R. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3373101
xNN-SF: An Explainable Neural Network Inspired by Stochastic Frontier Model
Zhao, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5588-2_37
A novel tree-based method for interpretable reinforcement learning
Li, Y., Qi, S., Wang, X., Zhang, J., & Cui, L. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3695464
Understanding the Role of Neurons and Layers in One-Dimensional Three-Class Classifiers
Emerald, S. C., & Vengattaraman, T. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-9196-8_29
Interpreting atmospheric neural networks: insights from explainable AI
Vojtekova, A., Νικολάου, Ν., Yip, K. H., Vénot, O., Al-Refaie, A. F., Merín, B., & Waldmann, I. P. (2026) · RAS Techniques and Instruments · doi:10.1093/rasti/rzag002
FPENet: an explainable frequency prior embedded deep neural network for intelligent fault diagnosisPreprint
Li, S., Li, T., Sun, C., Yan, R., Chen, X., & Zio, E. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6271684
- Authors
- Yang, Z., Zhang, A., & Sudjianto, A.
- Affiliations
- University of Hong Kong (HK)
- Wells Fargo (United States) (US)
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2021-06
- Volume
- 32
- Issue
- 6
- Pages
- 2610-2621
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W3045825034
- Cited by
- 106
- References
- 37
- Citations by year
- 2026: 13 · 2025: 6 · 2024: 19 · 2023: 22 · 2022: 15 · 2021: 22 · 2020: 9
- FWCI
- 6.68
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Big Data Project Fund of The University of Hong Kong through Dr. Patrick Poon’s Donation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- explainable neural networks, projection pursuit, orthogonality constraints, smooth function approximation, identifiability conditions, Cayley transform, backpropagation, model interpretability, prediction accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
