Criteria for Rigor in Visualization Design Study
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
We develop a new perspective on research conducted through visualization design study that emphasizes design as a method of inquiry and the broad range of knowledge-contributions achieved through it as multiple, subjective, and socially constructed. From this interpretivist position we explore the nature of visualization design study and develop six criteria for rigor. We propose that rigor is established and judged according to the extent to which visualization design study research and its reporting are INFORMED, REFLEXIVE, ABUNDANT, PLAUSIBLE, RESONANT, and TRANSPARENT. This perspective and the criteria were constructed through a four-year engagement with the discourse around rigor and the nature of knowledge in social science, information systems, and design. We suggest methods from cognate disciplines that can support visualization researchers in meeting these criteria during the planning, execution, and reporting of design study. Through a series of deliberately provocative questions, we explore implications of this new perspective for design study research in visualization, concluding that as a discipline, visualization is not yet well positioned to embrace, nurture, and fully benefit from a rigorous, interpretivist approach to design study. The perspective and criteria we present are intended to stimulate dialogue and debate around the nature of visualization design study and the broader underpinnings of the discipline.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Constructing Grounded Theory. A Practical Guide Through Qualitative Analysis
(2006) · QMiP Bulletin · doi:10.53841/bpsqmip.2006.1.2.36 · Cited by 11,828
Design Science in Information Systems Research1
Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004) · MIS Quarterly · doi:10.2307/25148625 · Cited by 10,707 · cited 1×
“e context of developing technology for and within organizations [97]. Influenced by design science – a problem-solving paradigm that seeks to analyze, design, implement, and manage information systems [54,112] – Sein et al. propose guiding principles through their action design research [97] methodology. These principl…”
BackgroundDesign science in information systems research
(2004) · Journal of the Association for Information Systems · doi:10.5555/2017212.2017217 · Cited by 8,254
Qualitative Quality: Eight “Big-Tent” Criteria for Excellent Qualitative Research
Tracy, S. J. (2010) · Qualitative Inquiry · doi:10.1177/1077800410383121 · Cited by 7,536 · cited 10×
“And Tracy provocatively offers eight universal criteria for quality in qualitative studies [110], which are much used and routinely critiqued [102].”
BackgroundMethodResultInfluentialAction Research and Minority Problems
Lewin, K. (1946) · Journal of Social Issues · doi:10.1111/j.1540-4560.1946.tb02295.x · Cited by 6,554 · cited 1×
“The action design research method, and underlying principles, stem more broadly from action research, an approach that relies on action as a means for developing knowledge [51,64].”
MethodWicked Problems in Design Thinking
Buchanan, R. (1992) · Design Issues · doi:10.2307/1511637 · Cited by 3,539 · cited 4×
“These solutions cannot be assessed as true or false, but rather as good or bad [11].”
BackgroundInfluentialThe Nature of Theory in Information Systems1
Gregor, S. (2006) · MIS Quarterly · doi:10.2307/25148742 · Cited by 3,306 · cited 3×
“The implication that the general is more valuable than the particular is often subtle, but sometimes explicit: “We do not regard a collection of facts, or knowledge of an individual fact or event, as theory” [45].”
BackgroundSmith, B., & McGannon, K. R. (2018) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2017.1317357 · Cited by 3,074 · cited 9×
“And Tracy provocatively offers eight universal criteria for quality in qualitative studies [110], which are much used and routinely critiqued [102].”
BackgroundInfluentialBurawoy, M. (1998) · Sociological Theory · doi:10.1111/0735-2751.00040 · Cited by 2,916
Cited by
Criteria for Good Qualitative Research: A Comprehensive Review
Yadav, D. (2022) · The Asia-Pacific Education Researcher · doi:10.1007/s40299-021-00619-0 · Cited by 217
From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics
Martinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G., & Buckingham Shum, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376148 · Cited by 123
“Human-centered design, and its tensions The adoption of human-centered design is an obvious way to address concerns such as the above [63].”
BackgroundTroubling Collaboration: Matters of Care for Visualization Design Study
Akbaba, D., Lange, D., Correll, M., Lex, A., & Meyer, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581168 · Cited by 40
Self-citation
Metadata
- Authors
- Meyer, M., & Dykes, J.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2019
- Pages
- 1-1
- ISSN
- 1077-2626 (ISSN-L)
- 1941-0506
- 2160-9306
- OpenAlex
- W2969940513
- Cited by
- 140
- References
- 25
- FWCI
- 6.93
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Criteria for Rigor in Visualization Design Study. https://app.cochat.ai/papers/10.1109/tvcg.2019.2934539
- The paper
- Meyer, M., & Dykes, J. (2019). Criteria for Rigor in Visualization Design Study. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/tvcg.2019.2934539
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.