The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
An Insight-Based Methodology for Evaluating Bioinformatics Visualizations
Abstract
High-throughput experiments, such as gene expression microarrays in the life sciences, result in very large data sets. In response, a wide variety of visualization tools have been created to facilitate data analysis. A primary purpose of these tools is to provide biologically relevant insight into the data. Typically, visualizations are evaluated in controlled studies that measure user performance on predetermined tasks or using heuristics and expert reviews. To evaluate and rank bioinformatics visualizations based on real-world data analysis scenarios, we developed a more relevant evaluation method that focuses on data insight. This paper presents several characteristics of insight that enabled us to recognize and quantify it in open-ended user tests. Using these characteristics, we evaluated five microarray visualization tools on the amount and types of insight they provide and the time it takes to acquire it. The results of the study guide biologists in selecting a visualization tool based on the type of their microarray data, visualization designers on the key role of user interaction techniques, and evaluators on a new approach for evaluating the effectiveness of visualizations for providing insight. Though we used the method to analyze bioinformatics visualizations, it can be applied to other domains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cluster analysis and display of genome-wide expression patterns
Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.25.14863 · Cited by 16,184
The visual display of quantitative information
(1985) · General Pharmacology: The Vascular System · doi:10.1016/0306-3623(85)90230-7 · Cited by 5,070
The eyes have it: a task by data type taxonomy for information visualizations
Shneiderman, B. (2002) · Proceedings 1996 IEEE Symposium on Visual Languages · doi:10.1109/vl.1996.545307 · Cited by 4,535
Readings in Information Visualization: Using Vision to Think
Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999) · Cited by 3,935
(2013) · Information Visualization · doi:10.1016/b978-0-12-381464-7.00018-1 · Cited by 3,742
Baechler, E. C., Batliwalla, F. M., Karypis, G., Gaffney, P. M., Ortmann, W. A., Espe, K. J., Shark, K. B., Grande, W. J., Hughes, K. M., Kapur, V., et al. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0337679100 · Cited by 2,240
The Visual Display of Quantitative Information.
Zelterman, D., & Tufte, E. R. (1984) · Journal of the American Statistical Association · doi:10.2307/2288294 · Cited by 2,059
Microarray data normalization and transformation
Quackenbush, J. (2002) · Nature Genetics · doi:10.1038/ng1032 · Cited by 1,975 · cited 1×
“Since our study is focused on the interactive visualization portion of data analysis, the data sets were preprocessed, normalized, prefiltered, and converted to the required formats (as discussed in [31] and [32]) in advance.”
MethodThe Visual Display of Quantitative Information.
Fayers, P., & Tufte, E. R. (1990) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983122 · Cited by 1,936
Expression profiling using cDNA microarrays
Duggan, D. J., Bittner, M., Chen, Y., Meltzer, P., & Trent, J. M. (1999) · Nature Genetics · doi:10.1038/4434 · Cited by 1,657
Cited by
A Nested Model for Visualization Design and Validation
Munzner, T. (2009) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2009.111 · Cited by 880
Design Study Methodology: Reflections from the Trenches and the Stacks
Sedlmair, M., Meyer, M., & Munzner, T. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.213 · Cited by 826
Visualization of omics data for systems biology
Gehlenborg, N., O'Donoghue, S. I., Baliga, N. S., Goesmann, A., Hibbs, M. A., Kitano, H., Kohlbacher, O., Neuweger, H., Schneider, R., Tenenbaum, D., et al. (2010) · Nature Methods · doi:10.1038/nmeth.1436 · Cited by 620
Toward measuring visualization insight
North, C. (2006) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2006.70 · Cited by 456
Self-citationCasual Information Visualization: Depictions of Data in Everyday Life
Pousman, Z., Stasko, J., & Mateas, M. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.70541 · Cited by 421
Knowledge Generation Model for Visual Analytics
Sacha, D., Stoffel, A., Stoffel, F., Kwon, B. C., Ellis, G., & Keim, D. A. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346481 · Cited by 387
Evaluating Information Visualizations
Carpendale, S. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-70956-5_2 · Cited by 364
The Effects of Interactive Latency on Exploratory Visual Analysis
Liu, Z., & Heer, J. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346452 · Cited by 326
Collaborative visualization: Definition, challenges, and research agenda
Isenberg, P., Elmqvist, N., Scholtz, J., Cernea, D., Ma, K.-L., & Hagen, H. (2011) · Information Visualization · doi:10.1177/1473871611412817 · Cited by 299
Yi, J. S., Kang, Y.-A., Stasko, J. T., & Jacko, J. A. (2008) · Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization · doi:10.1145/1377966.1377971 · Cited by 175
“The recent growth of visual analytics [29] and the emergence of insight-based evaluation [23,24] elevated the level of interest in insight as a proper evaluation measure of InfoVis.”
BackgroundSupporting Accessible Data Visualization Through Audio Data Narratives
Siu, A., S-H Kim, G., O'Modhrain, S., & Follmer, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517678 · Cited by 64
A Critical Reflection on Visualization Research: Where Do Decision Making Tasks Hide?
Dimara, E., & Stasko, J. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114813 · Cited by 60
Bernard, J., Sessler, D., Kohlhammer, J., & Ruddle, R. A. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2803829 · Cited by 55
Troubling Collaboration: Matters of Care for Visualization Design Study
Akbaba, D., Lange, D., Correll, M., Lex, A., & Meyer, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581168 · Cited by 40
Freitas, C. M. D. S., Pimenta, M. S., & Scapin, D. L. (2014) · Handbook of Human Centric Visualization · doi:10.1007/978-1-4614-7485-2_12 · Cited by 28
Visualizations at First Sight: Do Insights Require Training?
Smuc, M., Mayr, E., Lammarsch, T., Bertone, A., Aigner, W., Risku, H., & Miksch, S. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89350-9_19 · Cited by 16
Metadata
- Authors
- Saraiya, P., North, C., & Duca, K.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2005-07
- Volume
- 11
- Issue
- 4
- Pages
- 443-456
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W2130774862
- Cited by
- 343
- References
- 35
- FWCI
- 17.15
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.