The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Design Study Methodology: Reflections from the Trenches and the Stacks
Abstract
Design studies are an increasingly popular form of problem-driven visualization research, yet there is little guidance available about how to do them effectively. In this paper we reflect on our combined experience of conducting twenty-one design studies, as well as reading and reviewing many more, and on an extensive literature review of other field work methods and methodologies. Based on this foundation we provide definitions, propose a methodological framework, and provide practical guidance for conducting design studies. We define a design study as a project in which visualization researchers analyze a specific real-world problem faced by domain experts, design a visualization system that supports solving this problem, validate the design, and reflect about lessons learned in order to refine visualization design guidelines. We characterize two axes - a task clarity axis from fuzzy to crisp and an information location axis from the domain expert's head to the computer - and use these axes to reason about design study contributions, their suitability, and uniqueness from other approaches. The proposed methodological framework consists of 9 stages: learn, winnow, cast, discover, design, implement, deploy, reflect, and write. For each stage we provide practical guidance and outline potential pitfalls. We also conducted an extensive literature survey of related methodological approaches that involve a significant amount of qualitative field work, and compare design study methodology to that of ethnography, grounded theory, and action research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Constructing Grounded Theory. A Practical Guide Through Qualitative Analysis
(2006) · QMiP Bulletin · doi:10.53841/bpsqmip.2006.1.2.36 · Cited by 11,828
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731
Handbook of Qualitative Research
(1994) · Journal of Environmental Psychology · doi:10.1016/s0272-4944(05)80230-0 · Cited by 5,715
Ericsson (1993) · doi:10.1037/0033-295x.87.3.215 · Cited by 4,539
The eyes have it: a task by data type taxonomy for information visualizations
Shneiderman, B. (2002) · Proceedings 1996 IEEE Symposium on Visual Languages · doi:10.1109/vl.1996.545307 · Cited by 4,535
(2013) · Information Visualization · doi:10.1016/b978-0-12-381464-7.00018-1 · Cited by 3,742
Contextual Design: Defining Customer-Centered Systems
Beyer, H. R., & Holtzblatt, K. (1997) · Cited by 2,748
Research methods in Anthropology: Qualitative and quantitative approaches
van Arsdale, P. W. (1996) · Evaluation Practice · doi:10.1016/s0886-1633(96)90044-6 · Cited by 2,526
Interaction Design: Beyond Human-Computer Interaction
Rogers, Y., Sharp, H., & Preece, J. J. (2002) · Open Research Online - ORO (The Open University) · Cited by 1,843
Illuminating the Path: The Research and Development Agenda for Visual Analytics
Thomas, J. J., & Cook, K. A. (2005) · Cited by 1,770
Cited by
Kwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294
Temporal Event Sequence Simplification
Monroe, M., Lan, R., Lee, H., Plaisant, C., & Shneiderman, B. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.200 · Cited by 266
Degas, A., Islam, M. R., Hurter, C., Barua, S., Rahman, H., Poudel, M., Ruscio, D., Ahmed, M. U., Begum, S., Rahman, M. A., et al. (2022) · Applied Sciences · doi:10.3390/app12031295 · Cited by 168
Extending the Nested Model for User-Centric XAI: A Design Study on GNN-based Drug Repurposing
Wang, Q., Huang, K., Chandak, P., Zitnik, M., & Gehlenborg, N. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209435 · Cited by 82
How Does Automation Shape the Process of Narrative Visualization: A Survey of Tools
Chen, Q., Cao, S., Wang, J., & Cao, N. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3261320 · Cited by 71
Metadata
- Authors
- Sedlmair, M., Meyer, M., & Munzner, T.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2012-12
- Volume
- 18
- Issue
- 12
- Pages
- 2431-2440
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W1970569592
- Cited by
- 826
- References
- 71
- FWCI
- 33.44
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Design Study Methodology: Reflections from the Trenches and the Stacks. https://app.cochat.ai/papers/10.1109/tvcg.2012.213
- The paper
- Sedlmair, M., Meyer, M., & Munzner, T. (2012). Design Study Methodology: Reflections from the Trenches and the Stacks. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/tvcg.2012.213
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.