A Framework for Considering Comprehensibility in Modeling
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Comprehensibility in modeling is the ability of stakeholders to understand relevant aspects of the modeling process. In this article, we provide a framework to help guide exploration of the space of comprehensibility challenges. We consider facets organized around key questions: Who is comprehending? Why are they trying to comprehend? Where in the process are they trying to comprehend? How can we help them comprehend? How do we measure their comprehension? With each facet we consider the broad range of options. We discuss why taking a broad view of comprehensibility in modeling is useful in identifying challenges and opportunities for solutions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Toward measuring visualization insight
North, C. (2006) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2006.70 · Cited by 456 · cited 1×
“For example, insight quantification approaches 30,31 could measure how well a model supports hypothesis formulation (building theory).”
A survey of Knowledge Discovery and Data Mining process models
KURGAN, L. A., & MUSILEK, P. (2006) · The Knowledge Engineering Review · doi:10.1017/s0269888906000737 · Cited by 391
Research works printed but not linked: 28
- 10.1109/TVCG.2013.120 · DOI · Paper page
- Huysmans J Baesens B Vanthienen J. Using rule extraction to improve the comprehensibility of predictive models. SSRN 2006. Available at: http://dx.doi.org/10.2139/ssrn.961358 · DOI
- 10.1371/journal.pone.0144439 · DOI
- 10.1007/978-3-319-10590-1_53 · DOI · Paper page
- Munzner T. Visualization Analysis and Design. Boca Raton FL CRC Press 2014.
- 10.1109/TVCG.2013.124 · DOI · Paper page
- Craven M. Extracting comprehensible models from trained neural Networks. Ph.D. Dissertation University of Wisconsin–Madison 1996.
- 10.5951/mathteacher.108.6.0446 · doi:10.5951/mathteacher.108.6.0446
- 10.5772/6438 · doi:10.5772/6438
- Gleicher M. Position paper: Towards comprehensible predictive modeling. In: Visualization for Predictive Analytics Workshop, Paris, France, 2014.
- 10.1109/TVCG.2013.157 · DOI
- 10.1089/big.2015.0033 · DOI
- 10.1109/ICIF.2005.1592040 · DOI · Paper page
- Erhan D, , Bengio Y, , Courville A, , Vincent P. Visualizing higher-layer features of a deep network. Technical Report, Department IRO, University of Montreal, 2009.
- 10.1109/ICASSP.2012.6288863 · DOI · Paper page
- 10.1207/s15516709cog0000_76 · DOI · Paper page
- 10.1109/VIS.2005.73 · DOI · Paper page
- 10.1109/CIDM.2011.5949423 · DOI · Paper page
- 10.1162/neco.2006.18.7.1527 · DOI · Paper page
- 10.4236/psych.2014.519211 · DOI
- Chuang J, , Socher R. Interactive visualizations for deep learning. In: Visualization for Predictive Analytics Workshop, Paris, France, 2014.
- 10.1109/VAST.2014.7042493 · DOI
- 10.1109/TVCG.2014.2346578 · DOI · Paper page
- 10.1007/978-3-540-30204-9_9 · DOI
- 10.1145/2424563.2424571 · DOI
- 10.1109/TVCG.2005.53 · DOI · Paper page
- 10.1111/cgf.12373 · DOI
- Gotz D, , Sun J. Visualizing accuracy to improve predictive model performance. In: Visualization for Predictive Analytics Workshop, Paris, France, 2014.
Cited by
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
“• Comprehensibility: when conceived for ML models, comprehensibility refers to the ability of a learning algorithm to represent its learned knowledge in a human understandable fashion [19, 20, 21].”
Schwalbe, G., & Finzel, B. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00867-8 · Cited by 328
Benchmarking and survey of explanation methods for black box models
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00933-9 · Cited by 264
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
Metadata
- Authors
- Gleicher, M.
- Venue
- Big Data
- Publisher
- SAGE Publications
- Published
- 2016-06
- Volume
- 4
- Issue
- 2
- Pages
- 75-88
- ISSN
- 2167-6461 (ISSN-L)
- 2167-647X
- OpenAlex
- W2415277563
- Cited by
- 63
- References
- 3
- FWCI
- 4.98
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.