The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow
Abstract
We present a design study of the TensorFlow Graph Visualizer, part of the TensorFlow machine intelligence platform. This tool helps users understand complex machine learning architectures by visualizing their underlying dataflow graphs. The tool works by applying a series of graph transformations that enable standard layout techniques to produce a legible interactive diagram. To declutter the graph, we decouple non-critical nodes from the layout. To provide an overview, we build a clustered graph using the hierarchical structure annotated in the source code. To support exploration of nested structure on demand, we perform edge bundling to enable stable and responsive cluster expansion. Finally, we detect and highlight repeated structures to emphasize a model's modular composition. To demonstrate the utility of the visualizer, we describe example usage scenarios and report user feedback. Overall, users find the visualizer useful for understanding, debugging, and sharing the structures of their models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,708
The complexity of theorem-proving procedures
Cook, S. A. (1971) · Proceedings of the third annual ACM symposium on Theory of computing - STOC '71 · doi:10.1145/800157.805047 · Cited by 6,385
Bostock, M., Ogievetsky, V., & Heer, J. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.185 · Cited by 3,165
Dryad: distributed data-parallel programs from sequential building blocks
Isard, M., Budiu, M., Yu, Y., Birrell, A., & Fetterly, D. (2007) · Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 · doi:10.1145/1272996.1273005 · Cited by 2,432
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
Chen, T., Li, M., Li, Y., Lin, M., Wang, N., Wang, M., Xiao, T., Xu, B., Zhang, C., & Zhang, Z. (2015) · arXiv · doi:10.48550/arxiv.1512.01274 · Cited by 1,819
Methods for Visual Understanding of Hierarchical System Structures
Sugiyama, K., Tagawa, S., & Toda, M. (1981) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1981.4308636 · Cited by 1,223
Visualizing and Understanding Recurrent Networks
Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845
A technique for drawing directed graphs
Gansner, E. R., Koutsofios, E., North, S. C., & Vo, K.-P. (1993) · IEEE Transactions on Software Engineering · doi:10.1109/32.221135 · Cited by 784
A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication
Christen, P. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2011.127 · Cited by 679
Layout Adjustment and the Mental Map
Misue, K., Eades, P., Lai, W., & Sugiyama, K. (1995) · Journal of Visual Languages & Computing · doi:10.1006/jvlc.1995.1010 · Cited by 543
Towards Better Analysis of Deep Convolutional Neural Networks
Liu, M., Shi, J., Li, Z., Li, C., Zhu, J., & Liu, S. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598831 · Cited by 498
VisTrails: visualization meets data management
Callahan, S. P., Freire, J., Santos, E., Scheidegger, C. E., Silva, C. T., & Vo, H. T. (2006) · Proceedings of the 2006 ACM SIGMOD international conference on Management of data · doi:10.1145/1142473.1142574 · Cited by 480
Graphviz and Dynagraph — Static and Dynamic Graph Drawing Tools
Ellson, J., Gansner, E. R., Koutsofios, E., North, S. C., & Woodhull, G. (2004) · Mathematics and Visualization · doi:10.1007/978-3-642-18638-7_6 · Cited by 438
Torch: a modular machine learning software library
Collobert, R., Samy Bengio, A., & Mariéthoz, J. (2002) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 435
FlumeJava: easy, efficient data-parallel pipelines
Chambers, C., Raniwala, A., Perry, F., Adams, S., Henry, R. R., Bradshaw, R., & Weizenbaum, N. (2010) · Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1806596.1806638 · Cited by 331
ASK-GraphView: A Large Scale Graph Visualization System
Abello, J., van Ham, F., & Krishnan, N. (2006) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2006.120 · Cited by 272
5 Demand Characteristics and the Concept of Quasi-Controls
Orne, M. T. (2009) · Artifacts in Behavioral Research · doi:10.1093/acprof:oso/9780195385540.003.0005 · Cited by 247
CIEL: a universal execution engine for distributed data-flow computing
Murray, D. G., Schwarzkopf, M., Smowton, C., Smith, S. M., Madhavapeddy, A., & Hand, S. (2011) · doi:10.5555/1972457.1972470 · Cited by 225
Dunne, C., & Shneiderman, B. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466444 · Cited by 195
Visualization of structural information: automatic drawing of compound digraphs
Sugiyama, K., & Misue, K. (1991) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/21.108304 · Cited by 193
Interactive Visualization of Small World Graphs
van Ham, F., & van Wijk, J. J. (2005) · IEEE Symposium on Information Visualization · doi:10.1109/infvis.2004.43 · Cited by 170
GrouseFlocks: Steerable Exploration of Graph Hierarchy Space
Archambault, D., Munzner, T., & Auber, D. (2008) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2008.34 · Cited by 153
Topological Fisheye Views for Visualizing Large Graphs
Gansner, E. R., Koren, Y., & North, S. C. (2005) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2005.66 · Cited by 136
Declarative interaction design for data visualization
Satyanarayan, A., Wongsuphasawat, K., & Heer, J. (2014) · ACM Symposium on User Interface Software and Technology · doi:10.1145/2642918.2647360 · Cited by 133
Self-citationOpening the Black Box - Data Driven Visualization of Neural Networks
Tzeng, F.-Y., & Ma, K. (2006) · Visualization · doi:10.1109/visual.2005.1532820 · Cited by 104
Opening the Black Box - Data Driven Visualization of Neural Network
Tzeng, F.-Y., & Ma, K. (2006) · doi:10.1109/vis.2005.73 · Cited by 91
Direct-Manipulation Visualization of Deep NetworksPreprint
Smilkov, D., Carter, S., Sculley, D., Viégas, F. B., & WATTENBERG, M. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.03788 · Cited by 84
Self-citationLevel-of-detail visualization of clustered graph layouts
Balzer, M., & Deußen, O. (2007) · doi:10.1109/apvis.2007.329288 · Cited by 75
Exact and heuristic algorithms for 2-layer straightline crossing minimization
Jünger, M., & Mutzel, P. (1996) · Lecture notes in computer science · doi:10.1007/bfb0021817 · Cited by 54
Visual Analysis of Hidden State Dynamics in Recurrent Neural NetworksPreprint
Strobelt, H., Gehrmann, S., Huber, B., Pfister, H., & Rush, A. M. (2016) · RePEc: Research Papers in Economics · Cited by 46
DIG-COLA: Directed Graph Layout through Constrained Energy Minimization
Dwyer, T., & Koren, Y. (2006) · IEEE Symposium on Information Visualization · doi:10.1109/infovis.2005.10 · Cited by 42
Drawing directed graphs using quadratic programming
Dwyer, T., Koren, Y., & Marriott, K. (2006) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2006.67 · Cited by 31
Constrained graph layout by stress majorization and gradient projection
Dwyer, T., Koren, Y., & Marriott, K. (2008) · Discrete Mathematics · doi:10.1016/j.disc.2007.12.103 · Cited by 25
Perfopticon: Visual Query Analysis for Distributed Databases
Moritz, D., Halperin, D., Howe, B., & Heer, J. (2015) · Computer Graphics Forum · doi:10.1111/cgf.12619 · Cited by 23
Dig-CoLa: directed graph layout through constrained energy minimization
Dwyer, T., & Koren, Y. (2005) · IEEE Symposium on Information Visualization · doi:10.1109/infvis.2005.1532130 · Cited by 22
Cited by
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Evaluating the quality of machine learning explanations: a survey on methods and metrics
Zhou, J., Gandomi, A. H., Chen, F., & Holzinger, A. (2021) · Electronics · doi:10.3390/electronics10050593 · Cited by 606
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Kwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294
“Especially for vision applications where convolutional neural networks (CNNs) have enjoyed a great success, many visualization methods such as heatmaps [85], blurring [79], dimensionality reduction [57, 66] of the filters and activation maps obtained during computation and backpropagation, and visualizing the model st…”
Method
Metadata
- Authors
- Wongsuphasawat, K., Smilkov, D., Wexler, J., Wilson, J., Mane, D., Fritz, D., Krishnan, D., Viegas, F. B., & Wattenberg, M.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2018-01
- Volume
- 24
- Issue
- 1
- Pages
- 1-12
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W2753713840
- Cited by
- 331
- References
- 39
- FWCI
- 17.86
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.