The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Towards better analysis of machine learning models: A visual analytics perspective
Open accessAbstract
Interactive model analysis, the process of understanding, diagnosing, and refining a machine learning model with the help of interactive visualization, is very important for users to efficiently solve real-world artificial intelligence and data mining problems. Dramatic advances in big data analytics have led to a wide variety of interactive model analysis tasks. In this paper, we present a comprehensive analysis and interpretation of this rapidly developing area. Specifically, we classify the relevant work into three categories: understanding, diagnosis, and refinement. Each category is exemplified by recent influential work. Possible future research opportunities are also explored and discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
Wold, S., Esbensen, K., & Geladi, P. (1987) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(87)80084-9 · Cited by 11,606
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Improving generalization with active learning
Cohn, D., Atlas, L., & Ladner, R. (1994) · Machine Learning · doi:10.1007/bf00993277 · Cited by 1,591
Active Learning with Statistical Models
Cohn, D. A., Ghahramani, Z., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.295 · Cited by 1,317
ImageNet classification with deep convolutional neural networks
Krizhevsky (2012) · Medical Entomology and Zoology · Cited by 799
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766
Research works printed but not linked: 8
- Chen, J., Zhu, J., Wang, Z., Zheng, X., Zhang, B. 2013. Scalable inference for logistic-normal topic models. In: NIPS
- Cui, 2014, How hierarchical topics evolve in large text corpora, IEEE TVCG, 20, 2281
- Dou, 2016, Topic-and time-oriented visual text analysis, IEEE Comput. Graph. Appl., 36, 8, 10.1109/MCG.2016.73 · doi:10.1109/mcg.2016.73
- Fekete, 2013, Visual analytics infrastructures: From data management to exploration, Computer, 46, 22, 10.1109/MC.2013.120 · doi:10.1109/mc.2013.120
- Liu, 2012, TIARA: Interactive, topic-based visual text summarization and analysis, ACM TIST, 3, 25:1
- Tzeng, F.Y., Ma, K.L. 2004. Acluster-space visual interface for arbitrary dimensional classification of volume data. In: Sixth Joint Eurographics - IEEE TCVG Conference on Visualization, pp. 17–24. http://dx.doi.org/10.2312/VisSym/VisSym04/017-024 · doi:10.2312/vissym/vissym04/017-024
- Wang, X., Liu, S., Song, Y., Guo, B. 2013. Mining evolutionary multi-branch trees from text streams. In: KDD, pp. 722–730 · DOI
- Wei, F., Li, W., Lu, Q., He, Y. 2008. Query-sensitive mutual reinforcement chain and its application in query-oriented multi-document summarization. In: SIGIR, pp. 283–290 · doi:10.1145/1390334.1390384
Cited by
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
“Examples of lossless visualisation methods include dimensionality reduction techniques [135, 136] (such as PCA or t-SNE), scatter plots, and heat maps.”
MethodAli, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
The International Journal of Logistics Management
(2023) · The International Journal of Logistics Management · doi:10.1108/ijlm · Cited by 672
Vellido, A. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04051-w · Cited by 635
Notions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Explainable machine learning in materials science
Zhong, X., Gallagher, B., Liu, S., Kailkhura, B., Hiszpanski, A., & Han, T. Y.-J. (2022) · npj Computational Materials · doi:10.1038/s41524-022-00884-7 · Cited by 501
Self-citationExplaining anomalies detected by autoencoders using Shapley Additive Explanations
Antwarg, L., Miller, R. M., Shapira, B., & Rokach, L. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115736 · Cited by 496
Explainable Deep Learning: A Field Guide for the Uninitiated
Ras, G., Xie, N., Van Gerven, M., & Doran, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13200 · Cited by 446
Visual Analytics for Explainable Deep Learning
Choo, J., & Liu, S. (2018) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2018.042731661 · Cited by 270
Self-citationA survey of visual analytics techniques for machine learning
Yuan, J., Chen, C., Yang, W., Liu, M., Xia, J., & Liu, S. (2021) · Computational Visual Media · doi:10.1007/s41095-020-0191-7 · Cited by 258
Self-citationRulematrix: Visualizing and understanding classifiers with rules
Ming, Y., Qu, H., & Bertini, E. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864812 · Cited by 243
explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
Spinner, T., Schlegel, U., Schafer, H., & El-Assady, M. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934629 · Cited by 254
Chowdhury, S., Joel-Edgar, S., Dey, P. K., Bhattacharya, S., & Kharlamov, A. (2023) · The International Journal of Human Resource Management · doi:10.1080/09585192.2022.2066981 · Cited by 140
What can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92
Explainable Artificial Intelligence: a Systematic Review
Vilone, G., & Longo, L. (2020) · arXiv · doi:10.48550/arxiv.2006.00093 · Cited by 269
Wu, T., Weld, D. S., & Heer, J. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3319616 · Cited by 39
Jaigirdar, F. T., Rudolph, C., Oliver, G., Watts, D., & Bain, C. (2020) · 2020 IEEE 6th International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic50333.2020.00030 · Cited by 18
An Interactive Approach to Bias Mitigation in Machine Learning
Wang, H., Mukhopadhyay, S., Xiao, Y., & Fang, S. (2021) · 2021 IEEE 20th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) · doi:10.1109/iccicc53683.2021.9811333 · Cited by 12
Metadata
- Authors
- Liu, S., Wang, X., Liu, M., & Zhu, J.
- Venue
- Visual Informatics
- Publisher
- Elsevier BV
- Published
- 2017-03
- Volume
- 1
- Issue
- 1
- Pages
- 48-56
- ISSN
- 2468-502X (ISSN-L)
- OpenAlex
- W2587299461
- Cited by
- 397
- References
- 46
- FWCI
- 14.60
- Type
- journal article
- Language
- en
- Update notices
- erratum (2021-03-01)
- Open access
- diamond
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.