The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Quality Metrics in High-Dimensional Data Visualization: An Overview and Systematization
Abstract
In this paper, we present a systematization of techniques that use quality metrics to help in the visual exploration of meaningful patterns in high-dimensional data. In a number of recent papers, different quality metrics are proposed to automate the demanding search through large spaces of alternative visualizations (e.g., alternative projections or ordering), allowing the user to concentrate on the most promising visualizations suggested by the quality metrics. Over the last decade, this approach has witnessed a remarkable development but few reflections exist on how these methods are related to each other and how the approach can be developed further. For this purpose, we provide an overview of approaches that use quality metrics in high-dimensional data visualization and propose a systematization based on a thorough literature review. We carefully analyze the papers and derive a set of factors for discriminating the quality metrics, visualization techniques, and the process itself. The process is described through a reworked version of the well-known information visualization pipeline. We demonstrate the usefulness of our model by applying it to several existing approaches that use quality metrics, and we provide reflections on implications of our model for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1998) · Sage eBooks · Cited by 38,631
Readings in Information Visualization: Using Vision to Think
Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999) · Cited by 3,935
The Visual Display of Quantitative Information.
Fayers, P., & Tufte, E. R. (1990) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983122 · Cited by 1,936
The KDD process for extracting useful knowledge from volumes of data
Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996) · Communications of the ACM · doi:10.1145/240455.240464 · Cited by 1,934
A Projection Pursuit Algorithm for Exploratory Data Analysis
Friedman, J. H., & Tukey, J. W. (1974) · IEEE Transactions on Computers · doi:10.1109/t-c.1974.224051 · Cited by 1,652
Automating the design of graphical presentations of relational information
Mackinlay, J. (1986) · ACM Transactions on Graphics · doi:10.1145/22949.22950 · Cited by 1,610
Narrative Visualization: Telling Stories with Data
Segel, E., & Heer, J. (2010) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2010.179 · Cited by 1,334
Eastman, J. R., & Bertin, J. (1986) · Economic Geography · doi:10.2307/143508 · Cited by 1,097
Toward a Deeper Understanding of the Role of Interaction in Information Visualization
Yi, J. S., Kang, Y. A., Stasko, J., & Jacko, J. A. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.70515 · Cited by 1,031
The Visual Display of Quantitative Information
Reidhaar, J. W., & Tufte, E. R. (1986) · Design Issues · doi:10.2307/1571646 · Cited by 781
Rao, R., & Card, S. K. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191776 · Cited by 669
Parallel coordinates: a tool for visualizing multi-dimensional geometry
Inselberg, A., & Dimsdale, B. (2002) · Proceedings of the First IEEE Conference on Visualization: Visualization `90 · doi:10.1109/visual.1990.146402 · Cited by 574
Visual Analytics: Scope and Challenges
Keim, D. A., Mansmann, F., Schneidewind, J., Thomas, J., & Ziegler, H. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-71080-6_6 · Cited by 514
Self-citationA taxonomy of visualization techniques using the data state reference model
Chi, E. H. (2002) · IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings · doi:10.1109/infvis.2000.885092 · Cited by 466
Getting Things in Order: An Introduction to the R Package seriation
Hahsler, M., Hornik, K., & Buchta, C. (2008) · Journal of Statistical Software · doi:10.18637/jss.v025.i03 · Cited by 385
A Taxonomy of Clutter Reduction for Information Visualisation
Ellis, G., & Dix, A. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.70535 · Cited by 375
Cognitive Measurements of Graph Aesthetics
Ware, C., Purchase, H. C., Colpoys, L., & McGill, M. (2002) · Information Visualization · doi:10.1057/palgrave.ivs.9500013 · Cited by 374
The Visual Design and Control of Trellis Display
Becker, R. A., Cleveland, W. S., & Shyu, M.-J. (1996) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1996.10474701 · Cited by 330
Dynamic Graphics for Statistics
Buckingham, J. (1990) · Technometrics · doi:10.1080/00401706.1990.10484731 · Cited by 277
Clutter Reduction in Multi-Dimensional Data Visualization Using Dimension Reordering
Peng, W., Ward, M. O., & Rundensteiner, E. A. (2004) · Cited by 251
Similarity clustering of dimensions for an enhanced visualization of multidimensional data
Ankerst, M., Berchtold, S., & Keim, D. A. (2002) · Proceedings IEEE Symposium on Information Visualization (Cat. No.98TB100258) · doi:10.1109/infvis.1998.729559 · Cited by 250
Self-citationWilkinson, L., Anand, A., & Grossman, R. (2006) · Proceedings of the 2005 IEEE Symposium on Information Visualization (INFOVIS'05) · doi:10.1109/infovis.2005.14 · Cited by 214
Selecting good views of high‐dimensional data using class consistency
Sips, M., Neubert, B., Lewis, J. P., & Hanrahan, P. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01467.x · Cited by 206
A Rank-by-Feature Framework for Interactive Exploration of Multidimensional Data
Seo, J., & Shneiderman, B. (2005) · Information Visualization · doi:10.1057/palgrave.ivs.9500091 · Cited by 199
The Perception of Correlation in Scatterplots
Rensink, R. A., & Baldridge, G. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01694.x · Cited by 181
Yang, J., Peng, W., Ward, M. O., & Rundensteiner, E. A. (2003) · IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714) · doi:10.1109/infvis.2003.1249015 · Cited by 172
Rethinking visualization: A high-level taxonomy
Tory, M., & Möller, T. (2004) · Cited by 165
Clutter Reduction in Multi-Dimensional Data Visualization Using Dimension Reordering
Peng, W., Ward, M. O., & Rundensteiner, E. A. (2005) · IEEE Symposium on Information Visualization · doi:10.1109/infvis.2004.15 · Cited by 164
Wilkinson, L., Anand, A., & Grossman, R. L. (2005) · IEEE Symposium on Information Visualization · doi:10.1109/infvis.2005.1532142 · Cited by 154
Pargnostics: Screen-Space Metrics for Parallel Coordinates
Dasgupta, A., & Kosara, R. (2010) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2010.184 · Cited by 140
Tatu, A., Albuquerque, G., Eisemann, M., Schneidewind, J., Theisel, H., Magnor, M. A., & Keim, D. A. (2009) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2009.5332628 · Cited by 139
Self-citationRethinking Visualization: A High-Level Taxonomy
Tory, M. K., & Möller, T. B. (2005) · IEEE Symposium on Information Visualization · doi:10.1109/infvis.2004.59 · Cited by 137
THE VISUAL DISPLAY OF QUANTITATIVE INFORMATION
Tufte, E. R. (1985) · Journal for Healthcare Quality · doi:10.1097/01445442-198507000-00012 · Cited by 130
Exploring N-dimensional databases
LeBlanc, J., Ward, M. O., & Wittels, N. (2002) · doi:10.1109/visual.1990.146386 · Cited by 130
Interactive Dimensionality Reduction Through User-defined Combinations of Quality Metrics
Johansson, S. M., & Johansson, J. (2009) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2009.153 · Cited by 124
Rao, R., & Card, S. K. (1994) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/259963.260391 · Cited by 118
Judging Correlation from Scatterplots and Parallel Coordinate Plots
Li, J., Martens, J.-B., & van Wijk, J. J. (2008) · Information Visualization · doi:10.1057/ivs.2008.13 · Cited by 115
Seo, J., & Shneiderman, B. (2004) · University Libraries (University of Maryland) · Cited by 112
Wilkinson, L., Anand, A., & Grossman, R. L. (2006) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2006.94 · Cited by 105
Pixel Bar Charts: A Visualization Technique for Very Large Multi-Attribute Data Sets
Keim, D. A., Hao, M., Dayal, U., & Hsu, M. (2002) · Information Visualization · doi:10.1057/palgrave.ivs.9500003 · Cited by 101
Self-citationEnabling Automatic Clutter Reduction in Parallel Coordinate Plots
Ellis, G. P., & Dix, A. J. (2006) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2006.138 · Cited by 101
Guo, D. (2003) · Information Visualization · doi:10.1057/palgrave.ivs.9500053 · Cited by 98
Improving the visual analysis of high-dimensional datasets using quality measures
Albuquerque, G., Eisemann, M., Lehmann, D. J., Theisel, H., & Magnor, M. A. (2010) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2010.5652433 · Cited by 74
Visual quality metrics and human perception
Tatu, A., Bak, P., Bertini, E., Keim, D. A., & Schneidewind, J. (2010) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/1842993.1843002 · Cited by 70
Self-citationBertini, E., & Lalanne, D. (2010) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1809400.1809404 · Cited by 70
Self-citationA Note on Space-Filling Visualizations and Space-Filling Curves
WATTENBERG, M. P. (2006) · IEEE Symposium on Information Visualization · doi:10.1109/infovis.2005.1 · Cited by 69
Yang, J., Hubball, D., Ward, M. O., Rundensteiner, E. A., & Ribarsky, W. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.1010 · Cited by 54
Pixnostics: Towards Measuring the Value of Visualization
Schneidewind, J., Sips, M., & Keim, D. A. (2006) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2006.261423 · Cited by 49
Self-citationMetrics for effective information visualization
Brath, R. (1997) · doi:10.1109/infvis.1997.636794 · Cited by 47
Pairwise Display of High-Dimensional Information via Eulerian Tours and Hamiltonian Decompositions
Hurley, C., & Oldford, R. W. (2010) · Journal of Computational and Graphical Statistics · doi:10.1198/jcgs.2010.09136 · Cited by 46
Visual Boosting in Pixel‐based Visualizations
Oelke, D., Janetzko, H., Simon, S., et al. (2011) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2011.01936.x · Cited by 38
Exploring high-D spaces with multiform matrices and small multiples
MacEachren, A., Xiping, D., Hardisty, F., Guo, D., & Lengerich, G. (2004) · IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714) · doi:10.1109/infvis.2003.1249006 · Cited by 37
Bertini, E., & Santucci, G. (2006) · doi:10.1145/1168149.1168159 · Cited by 37
Self-citationFerdosi, B. J., Buddelmeijer, H., Trager, S. C., Wilkinson, M. H. F., & Roerdink, J. B. T. M. (2010) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2010.5652450 · Cited by 32
A Screen Space Quality Method for Data Abstraction
Johansson, J., & Cooper, M. (2008) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2008.01240.x · Cited by 24
Cited by
Visualizing structure and transitions in high-dimensional biological data
Moon, K. R., van Dijk, D., Wang, Z., Gigante, S., Burkhardt, D. B., Chen, W. S., Yim, K., Elzen, A. V. D., Hirn, M. J., Coifman, R. R., et al. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0336-3 · Cited by 1,065
Visualizing high-dimensional data: advances in the past decade
Liu, S., Maljovec, D., Wang, B., Bremer, P.-T., & Pascucci, V. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2640960 · Cited by 427
Knowledge Generation Model for Visual Analytics
Sacha, D., Stoffel, A., Stoffel, F., Kwon, B. C., Ellis, G., & Keim, D. A. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346481 · Cited by 387
Self-citationA survey on information visualization: recent advances and challenges
Liu, S., Cui, W., Wu, Y., & Liu, M. (2014) · The Visual Computer · doi:10.1007/s00371-013-0892-3 · Cited by 373
Visualization and Visual Analysis of Multifaceted Scientific Data: A Survey
Kehrer, J., & Hauser, H. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.110 · Cited by 363
Metadata
- Authors
- Bertini, E., Tatu, A., & Keim, D.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2011-12
- Volume
- 17
- Issue
- 12
- Pages
- 2203-2212
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W2157379557
- Cited by
- 257
- References
- 55
- FWCI
- 16.22
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.