A Principled Way of Assessing Visualization Literacy
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
We describe a method for assessing the visualization literacy (VL) of a user. Assessing how well people understand visualizations has great value for research (e. g., to avoid confounds), for design (e. g., to best determine the capabilities of an audience), for teaching (e. g., to assess the level of new students), and for recruiting (e. g., to assess the level of interviewees). This paper proposes a method for assessing VL based on Item Response Theory. It describes the design and evaluation of two VL tests for line graphs, and presents the extension of the method to bar charts and scatterplots. Finally, it discusses the reimplementation of these tests for fast, effective, and scalable web-based use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Item-Response Theory Analysis of Self-Report Measures of Adult Attachment
Fraley, R. C., Waller, N. G., & Brennan, K. A. (2000) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.78.2.350 · Cited by 3,385 · cited 1×
“Traditionally, IRT has been used to assess examinees’ abilities via predefined tests and surveys in areas such as education [24], social sciences [14], and medicine [29].”
Hambleton, R. K., & Swaminathan, H. (1985) · doi:10.1007/978-94-017-1988-9 · Cited by 1,777
ltm: An R Package for Latent Variable Modeling and Item Response Theory Analyses
Rizopoulos, D. (2006) · Journal of Statistical Software · doi:10.18637/jss.v017.i05 · Cited by 1,242
Making Sense of Graphs: Critical Factors Influencing Comprehension and Instructional Implications
Friel, S. N., Curcio, F. R., & Bright, G. W. (2001) · Journal for Research in Mathematics Education · doi:10.2307/749671 · Cited by 819
Crowdsourcing graphical perception: using mechanical turk to assess visualization design
Heer, J., & Bostock, M. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753357 · Cited by 816
A theory of graph comprehension.
Pinker, S. A. (1990) · Cited by 477
Comprehension of Mathematical Relationships Expressed in Graphs
Curcio, F. R. (1987) · Journal for Research in Mathematics Education · doi:10.2307/749086 · Cited by 405
A model of the perceptual and conceptual processes in graph comprehension.
Carpenter, P. A., & Shah, P. (1998) · Journal of Experimental Psychology: Applied · doi:10.1037//1076-898x.4.2.75 · Cited by 373
Experimental components for the evaluation of interactive information retrieval systems
Borlund, P. (2000) · Journal of Documentation · doi:10.1108/eum0000000007110 · Cited by 307
The Perception of Correlation in Scatterplots
Rensink, R. A., & Baldridge, G. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01694.x · Cited by 181
Self-citationVariables on Scatterplots Look More Highly Correlated When the Scales Are Increased
Cleveland, W. S., Diaconis, P., & McGill, R. (1982) · Science · doi:10.1126/science.216.4550.1138 · Cited by 173
Toward a Model of Knowledge-Based Graph Comprehension
Freedman, E. G., & Shah, P. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46037-3_3 · Cited by 125
Baer, J. D., Kutner, M. A., Sabatini, J., & White, S. (2009) · National Center for Education Statistics · Cited by 118
Comparing averages in time series data
Correll, M., Albers, D., Franconeri, S., & Gleicher, M. L. (2012) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208556 · Cited by 93
Toward a Comprehensive Model of Graph Comprehension: Making the Case for Spatial Cognition
Trickett, S. B., & Trafton, J. G. (2006) · Lecture notes in computer science · doi:10.1007/11783183_38 · Cited by 71
A Test of Graphicacy in Children
Wainer, H. (1980) · Applied Psychological Measurement · doi:10.1177/014662168000400305 · Cited by 70
A Study on Dual-Scale Data Charts
Isenberg, P., Bezerianos, A., Dragicevic, P., & Fekete, J. (2011) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.160 · Cited by 58
Self-citationTHE PERCEPTION OF PEARSON PRODUCT MOMENT CORRELATIONS FROM BIVARIATE SCATTERPLOTS
Bobko, P., & Karren, R. J. (1979) · Personnel Psychology · doi:10.1111/j.1744-6570.1979.tb02137.x · Cited by 56
Correlation estimates as perceptual judgments.
Meyer, J., Taieb, M., & Flascher, I. (1997) · Journal of Experimental Psychology Applied · doi:10.1037/1076-898x.3.1.3 · Cited by 39
Extracting Explicit and Implict Information from Complex Visualizations
Trafton, J. G., Marshall, S. P., Mintz, F. E., & Trickett, S. B. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46037-3_22 · Cited by 37
Shedding light on the graph schema: Perceptual features versus invariant structure
Ratwani, R. M., & Trafton, J. G. (2008) · Psychonomic Bulletin & Review · doi:10.3758/pbr.15.4.757 · Cited by 32
New Models of Deep Comprehension
Graesser, A. C., Swamer, S. S., Baggett, W. B., & Sell, M. A. (2014) · doi:10.4324/9781315806143-6 · Cited by 28
“Reading” scientific diagrams: Characterising components of skilled performance
Lowe, R. (1988) · Research in Science Education · doi:10.1007/bf02356586 · Cited by 20
A Model of the Cognitive and Perceptual Processes in Graphical Display Comprehension
Shah, P. (1997) · Cited by 19
Research works printed but not linked: 14
- 10.1037/h0042245 · doi:10.1037/h0042245
- 1984, OTA, Computerized Manufacturing Automation Employment Education and the Workplace
- abilock, 2008, Visual information literacy: Reading a documentary photograph, Knowledge Quest, 36
- 1989, Join, ACRL and Council, Chapters, Presidential Committee on Information Literacy Final Report
- knutson, 0, Applying the Rasch model to measure first-year students perceptions of college academic readiness, Paper presented at the annual meeting of the MWERA Annual Meeting
- 10.1002/j.2333-8504.2001.tb01867.x · DOI
- lowe, 1989, Distributed by ERIC Clearinghouse, Scientific Diagrams How Well Can Students Read Them? [microform What Research Says to the Science and Mathematics Teacher Number 3 /Richard K Lowe
- wu, 2007, Applying the Rasch model to psycho-social measurement [electronic resource] A practical approach / Margaret Wu and Ray Adams
- bertin, 1973, Semiologie graphique
- 10.1037/1076-898X.4.2.75 · doi:10.1037/1076-898x.4.2.75 · Paper page
- bristor, 1994, Linking the language arts and content areas through visual technology, T H E Journal, 22, 74
- tversky, 2004, Semantics Syntax and Pragmatics of graphics, 141
- taylor, 2003, New kinds of literacy, and the world of visual information, Literacy
- sir, 1959, Crowther, The Crowther Report, 1
Cited by
Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco
Moritz, D., Wang, C., Nelson, G. L., Lin, H., Smith, A. M., Howe, B., & Heer, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865240 · Cited by 317
Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments
Börner, K., Bueckle, A., & Ginda, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1807180116 · Cited by 316
VLAT: Development of a Visualization Literacy Assessment Test
Lee, S., Kim, S.-H., & Kwon, B. C. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598920 · Cited by 290
Visualization Literacy at Elementary School
Alper, B., Riche, N. H., Chevalier, F., Boy, J., & Sezgin, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025877 · Cited by 163
“Visual information literacy [1, 57, 61], or visualization literacy — the more concise term proposed in our community — is gaining increasing attention [8].”
BackgroundInfluentialSelf-citationMillecamp, M., Htun, N. N., Conati, C., & Verbert, K. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302313 · Cited by 148
Metadata
- Authors
- Boy, J., Rensink, R. A., Bertini, E., & Fekete, J.-D.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2014-12-31
- Volume
- 20
- Issue
- 12
- Pages
- 1963-1972
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W2050674259
- Cited by
- 278
- References
- 24
- FWCI
- 10.98
- Type
- journal article
- Language
- en
- Open access
- green
- Funders
- Google Research Award
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.