The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
VLAT: Development of a Visualization Literacy Assessment Test
Abstract
The Information Visualization community has begun to pay attention to visualization literacy; however, researchers still lack instruments for measuring the visualization literacy of users. In order to address this gap, we systematically developed a visualization literacy assessment test (VLAT), especially for non-expert users in data visualization, by following the established procedure of test development in Psychological and Educational Measurement: (1) Test Blueprint Construction, (2) Test Item Generation, (3) Content Validity Evaluation, (4) Test Tryout and Item Analysis, (5) Test Item Selection, and (6) Reliability Evaluation. The VLAT consists of 12 data visualizations and 53 multiple-choice test items that cover eight data visualization tasks. The test items in the VLAT were evaluated with respect to their essentialness by five domain experts in Information Visualization and Visual Analytics (average content validity ratio = 0.66). The VLAT was also tried out on a sample of 191 test takers and showed high reliability (reliability coefficient omega = 0.76). In addition, we demonstrated the relationship between users' visualization literacy and aptitude for learning an unfamiliar visualization and showed that they had a fairly high positive relationship (correlation coefficient = 0.64). Finally, we discuss evidence for the validity of the VLAT and potential research areas that are related to the instrument.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1991) · Cited by 15,554
A QUANTITATIVE APPROACH TO CONTENT VALIDITY1
LAWSHE, C. H. (1975) · Personnel Psychology · doi:10.1111/j.1744-6570.1975.tb01393.x · Cited by 7,313
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Test Theory: A Unified Treatment
Zwick, R., & McDonald, R. P. (2000) · Journal of the American Statistical Association · doi:10.2307/2669496 · Cited by 3,603
Dunn, T. J., Baguley, T., & Brunsden, V. (2014) · British Journal of Psychology · doi:10.1111/bjop.12046 · Cited by 3,596
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
(1999) · Cited by 2,653
Uses and abuses of coefficient alpha
Schmitt, N. (1996) · Psychological Assessment · doi:10.1037/1040-3590.8.4.350 · Cited by 2,588
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · cited 1×
“Thus, systematic ways to gauge the participants’ attention are necessary [26].”
BackgroundVisualization Analysis and Design
Munzner, T. (2014) · A K Peters/CRC Press eBooks · doi:10.1201/b17511 · Cited by 1,697
Semiology of Graphics: Diagrams, Networks, Maps
Bertin, J., & Berg, W. J. (2010) · Cited by 1,555
Measurement and Evaluation in Psychology and Education.
S., R., Thorndike, R. L., & Hagen, E. (1961) · Journal of the American Statistical Association · doi:10.2307/2282039 · Cited by 1,230
Tree-Maps: A Space Filling Approach to the Visualization of Hierarchical Information Structures
Johnson, B., & Shneiderman, B. (1991) · Cited by 1,121
Toward a Deeper Understanding of the Role of Interaction in Information Visualization
Yi, J. S., Kang, Y. A., Stasko, J., & Jacko, J. A. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.70515 · Cited by 1,031
Ullman, S. (1984) · Cognition · doi:10.1016/0010-0277(84)90023-4 · Cited by 1,027
Psychological Testing and Assessment: An Introduction to Tests and Measurement
Cohen, R. J., & Swerdlik, M. E. (1992) · Cited by 992
Huff, C., & Tingley, D. (2015) · Research & Politics · doi:10.1177/2053168015604648 · Cited by 903
Making Sense of Graphs: Critical Factors Influencing Comprehension and Instructional Implications
Friel, S. N., Curcio, F. R., & Bright, G. W. (2001) · Journal for Research in Mathematics Education · doi:10.2307/749671 · Cited by 819 · cited 2×
“They defined graph comprehension as reading and interpreting a graph [24], and they proposed the three levels of graph comprehension framework [7, 13, 49]: (1) the first level is the elementary level in which a graph reader can read a specific value in a graph (e.”
BackgroundA Multi-Level Typology of Abstract Visualization Tasks
Brehmer, M., & Munzner, T. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.124 · Cited by 747
Tree-maps: a space-filling approach to the visualization of hierarchical information structures
Johnson, B., & Shneiderman, B. (2002) · Proceeding Visualization '91 · doi:10.1109/visual.1991.175815 · Cited by 611
Educational and Psychological Measurement and Evaluation
Hopkins, K. D., Stanley, J. C., & Hopkins, B. R. (1972) · Medical Entomology and Zoology · Cited by 565
Implications of Evidence‐Centered Design for Educational Testing
Mislevy, R. J., & Haertel, G. D. (2006) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2006.00075.x · Cited by 543
Low-Level Components of Analytic Activity in Information Visualization
Amar, R., Eagan, J., & Stasko, J. (2006) · Proceedings of the 2005 IEEE Symposium on Information Visualization (INFOVIS'05) · doi:10.1109/infovis.2005.24 · Cited by 512
DeMars, C. (2010) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195377033.001.0001 · Cited by 497
Comprehension of Mathematical Relationships Expressed in Graphs
Curcio, F. R. (1987) · Journal for Research in Mathematics Education · doi:10.2307/749086 · Cited by 405
Fan, X. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058003001 · Cited by 389
Knowledge Generation Model for Visual Analytics
Sacha, D., Stoffel, A., Stoffel, F., Kwon, B. C., Ellis, G., & Keim, D. A. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346481 · Cited by 387
Self-citationGraph Literacy: A Cross-Cultural Comparison
Galesic, M., & Garcia-Retamero, R. (2011) · Medical Decision Making · doi:10.1177/0272989x10373805 · Cited by 361
Low-level components of analytic activity in information visualization
Amar, R., Eagan, J., & Stasko, J. (2005) · IEEE Symposium on Information Visualization, 2005. INFOVIS 2005. · doi:10.1109/infvis.2005.1532136 · Cited by 343
A Principled Way of Assessing Visualization Literacy
Boy, J., Rensink, R. A., Bertini, E., & Fekete, J.-D. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346984 · Cited by 278
Understanding Graphs and Tables
Wainer, H. (1992) · Educational Researcher · doi:10.3102/0013189x021001014 · Cited by 235
The Persuasive Power of Data Visualization
Pandey, A. V., Manivannan, A., Nov, O., Satterthwaite, M., & Bertini, E. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346419 · Cited by 217
Börner, K., Maltese, A., Balliet, R. N., & Heimlich, J. (2016) · Information Visualization · doi:10.1177/1473871615594652 · Cited by 203
Bar and Line Graph Comprehension: An Interaction of Top‐Down and Bottom‐Up Processes
Shah, P., & Freedman, E. G. (2011) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2009.01066.x · Cited by 186
Lee, S., Kim, S.-H., Hung, Y.-H., Lam, H., Kang, Y.-A., & Yi, J. S. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467195 · Cited by 161
Self-citationChoosing Specifiers: An Evaluation of the Basic Tasks Model of Graphical Perception
Carswell, C. M. (1992) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089203400503 · Cited by 125
Diamond, J. J., & Evans, W. (1973) · Review of Educational Research · doi:10.2307/1169935 · Cited by 122
Data Visualization Literacy: Investigating Data Interpretation Along the Novice–Expert Continuum
Maltese, A. V., Harsh, J. A., & Svetina, D. (2015) · The journal of college science teaching · doi:10.2505/4/jcst15_045_01_84 · Cited by 112
Formula Scoring of Multiple‐Choice Tests (Correction for Guessing)
Frary, R. B. (1988) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.1988.tb00434.x · Cited by 107
Strategies for crowdsourcing social data analysis
Willett, W., Heer, J., & Agrawala, M. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207709 · Cited by 107
An Inquiry into Language-switching in Second Language Composing Processes
Qi, D. (1998) · Canadian Modern Language Review/ La Revue canadienne des langues vivantes · doi:10.3138/cmlr.54.3.413 · Cited by 102
A Comparative Evaluation on Online Learning Approaches using Parallel Coordinate Visualization
Kwon, B. C., & Lee, B. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858101 · Cited by 73
Self-citationSuggested Interactivity: Seeking Perceived Affordances for Information Visualization
Boy, J., Eveillard, L., Détienne, F., & Fekete, J. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467201 · Cited by 72
Toward effective insight management in visual analytics systems
Chen, Y., Yang, J., & Ribarsky, W. (2009) · IEEE Pacific Visualization Symposium · doi:10.1109/pacificvis.2009.4906837 · Cited by 60
delMas, R. C., Garfield, J., & Ooms, A. (2005) · Research Repository (Kingston University London) · Cited by 57
Data Visualization Literacy: Investigating Data Interpretation along the Novice-Expert Continuum
Maltese, A. V., Harsh, J. A., & Svetina, D. (2015) · The journal of college science teaching · Cited by 49
Visual analytic roadblocks for novice investigators
Kwon, B. C., Fisher, B., & Yi, J. S. (2011) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2011.6102435 · Cited by 38
Self-citationUnderstand users’ comprehension and preferences for composing information visualizations
Yang, H., Li, Y., & Zhou, M. X. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2541288 · Cited by 36
Cohen, J., & Wiebe, G. D. (1955) · American Psychologist · doi:10.1037/h0049373 · Cited by 17
Cited by
The Science of Visual Data Communication: What Works
Franconeri, S. L., Padilla, L. M., Shah, P., Zacks, J. M., & Hullman, J. (2021) · Psychological Science in the Public Interest · doi:10.1177/15291006211051956 · Cited by 360
Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments
Börner, K., Bueckle, A., & Ginda, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1807180116 · Cited by 316
Task-based effectiveness of basic visualizations
Saket, B., Endert, A., & Demiralp, C. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2829750 · Cited by 178
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations
Abdul, A., von der Weth, C., Kankanhalli, M., & Lim, B. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376615 · Cited by 96
Charitopoulos, A., Rangoussi, M., & Koulouriotis, D. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-020-00200-8 · Cited by 89
Metadata
- Authors
- Lee, S., Kim, S.-H., & Kwon, B. C.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2017-01
- Volume
- 23
- Issue
- 1
- Pages
- 551-560
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W2513780624
- Cited by
- 290
- References
- 49
- FWCI
- 6.85
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.