The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Opening the black box of neural networks: methods for interpreting neural network models in clinical applications
Open accessAbstract
Artificial neural networks (ANNs) are powerful tools for data analysis and are particularly suitable for modeling relationships between variables for best prediction of an outcome. While these models can be used to answer many important research questions, their utility has been critically limited because the interpretation of the "black box" model is difficult. Clinical investigators usually employ ANN models to predict the clinical outcomes or to make a diagnosis; the model however is difficult to interpret for clinicians. To address this important shortcoming of neural network modeling methods, we describe several methods to help subject-matter audiences (e.g., clinicians, medical policy makers) understand neural network models. Garson's algorithm describes the relative magnitude of the importance of a descriptor (predictor) in its connection with outcome variables by dissecting the model weights. The Lek's profile method explores the relationship of the outcome variable and a predictor of interest, while holding other predictors at constant values (e.g., minimum, 20th quartile, maximum). While Lek's profile was developed specifically for neural networks, partial dependence plot is a more generic version that visualize the relationship between an outcome and one or two predictors. Finally, the local interpretable model-agnostic explanations (LIME) method can show the predictions of any classification or regression, by approximating it locally with an interpretable model. R code for the implementations of these methods is shown by using example data fitted with a standard, feed-forward neural network model. We offer codes and step-by-step description on how to use these tools to facilitate better understanding of ANN.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ggplot2: Elegant Graphics for Data Analysis
Ginestet, C. (2011) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 13,726
Approximation capabilities of multilayer feedforward networks
Hornik, K. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90009-t · Cited by 6,091
Artificial intelligence in medicine
Hamet, P., & Tremblay, J. (2017) · Metabolism · doi:10.1016/j.metabol.2017.01.011 · Cited by 2,334
caret: Classification and Regression Training
Kuhn, M. (2007) · CRAN: Contributed Packages · doi:10.32614/cran.package.caret · Cited by 1,786
Image reconstruction by domain-transform manifold learning
Zhu, B., Liu, J. Z., Cauley, S. F., Rosen, B. R., & Rosen, M. S. (2018) · Nature · doi:10.1038/nature25988 · Cited by 1,694
Interpreting neural-network connection weights
Garson, G. D. (1991) · AI Expert archive · doi:10.5555/129449.129452 · Cited by 1,472
Gevrey, M., Dimopoulos, I., & Lek, S. (2003) · Ecological Modelling · doi:10.1016/s0304-3800(02)00257-0 · Cited by 1,248
pdp: An R Package for Constructing Partial Dependence Plots
Greenwell, B. (2017) · The R Journal · doi:10.32614/rj-2017-016 · Cited by 1,189
Back-propagation neural networks for modeling complex systems
Goh, A. T. C. (1995) · Artificial Intelligence in Engineering · doi:10.1016/0954-1810(94)00011-s · Cited by 1,075
Application of neural networks to modelling nonlinear relationships in ecology
Lek, S., Delacoste, M., Baran, P., Dimopoulos, I., Lauga, J., & Aulagnier, S. (1996) · Ecological Modelling · doi:10.1016/0304-3800(95)00142-5 · Cited by 716
Cited by
Explainable artificial intelligence: a comprehensive review
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10088-y · Cited by 983
Big data and artificial intelligence modeling for drug discovery
Zhu, H. (2020) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010919-023324 · Cited by 455
Ethical considerations about artificial intelligence for prognostication in intensive care
Beil, M., Proft, I., van Heerden, D., Sviri, S., & van Heerden, P. V. (2019) · Intensive Care Medicine Experimental · doi:10.1186/s40635-019-0286-6 · Cited by 124
Jiang, Z., Bo, L., Xu, Z., Song, Y., Wang, J., Wen, P., Wan, X., Yang, T., Deng, X., & Bian, J. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106040 · Cited by 86
Hu, C.-A., Chen, C.-M., Fang, Y.-C., Liang, S.-J., Wang, H.-C., Fang, W.-F., Sheu, C.-C., Perng, W.-C., Yang, K.-Y., Kao, K.-C., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-033898 · Cited by 106
Metadata
- Authors
- written on behalf of AME Big-Data Clinical Trial Collaborative Group, Zhang, Z., Beck, M. W., Winkler, D. A., Huang, B., Sibanda, W., & Goyal, H.
- Venue
- Annals of Translational Medicine
- Publisher
- AME Publishing Company
- Published
- 2018-06
- Volume
- 6
- Issue
- 11
- Pages
- 216-216
- ISSN
- 2305-5839 (ISSN-L)
- 2305-5847
- OpenAlex
- W2807036076
- Cited by
- 293
- References
- 17
- FWCI
- 15.70
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Opening the black box of neural networks: methods for interpreting neural network models in clinical applications. https://app.cochat.ai/papers/10.21037/atm.2018.05.32
- The paper
- written on behalf of AME Big-Data Clinical Trial Collaborative Group, Zhang, Z., Beck, M. W., Winkler, D. A., Huang, B., Sibanda, W., & Goyal, H. (2018). Opening the black box of neural networks: methods for interpreting neural network models in clinical applications. Annals of Translational Medicine. https://doi.org/10.21037/atm.2018.05.32
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.