The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco
Open accessAbstract
There exists a gap between visualization design guidelines and their application in visualization tools. While empirical studies can provide design guidance, we lack a formal framework for representing design knowledge, integrating results across studies, and applying this knowledge in automated design tools that promote effective encodings and facilitate visual exploration. We propose modeling visualization design knowledge as a collection of constraints, in conjunction with a method to learn weights for soft constraints from experimental data. Using constraints, we can take theoretical design knowledge and express it in a concrete, extensible, and testable form: the resulting models can recommend visualization designs and can easily be augmented with additional constraints or updated weights. We implement our approach in Draco, a constraint-based system based on Answer Set Programming (ASP). We demonstrate how to construct increasingly sophisticated automated visualization design systems, including systems based on weights learned directly from the results of graphical perception experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
ggplot2: Elegant Graphics for Data Analysis
Ginestet, C. (2011) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2010.00676_9.x · Cited by 13,726
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
John Lu, Z. Q. (2010) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2010.00646_6.x · Cited by 12,390
(1988) · Cited by 3,452
Richardson, M., & Domingos, P. (2006) · Machine Learning · doi:10.1007/s10994-006-5833-1 · Cited by 2,676
Learning to rank for information retrieval
Liu, T.-Y. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835676 · Cited by 1,856
Cleveland, W. S., & McGill, R. (1984) · Journal of the American Statistical Association · doi:10.2307/2288400 · Cited by 1,796
Cleveland, W. S., & McGill, R. (1984) · Journal of the American Statistical Association · doi:10.1080/01621459.1984.10478080 · Cited by 1,788
Visualization Analysis and Design
Munzner, T. (2014) · A K Peters/CRC Press eBooks · doi:10.1201/b17511 · Cited by 1,697
Learning Logical Definitions from Relations
Quinlan, J. R. (1990) · Machine Learning · doi:10.1023/a:1022699322624 · Cited by 1,644 · cited 1×
“The AI community uses inductive logic programming methods to infer logic programs from databases of positive and negative examples [48].”
MethodAutomating the design of graphical presentations of relational information
Mackinlay, J. (1986) · ACM Transactions on Graphics · doi:10.1145/22949.22950 · Cited by 1,610
Learning to Rank for Information Retrieval
Liu, T.-Y. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000016 · Cited by 1,568
Semiology of Graphics: Diagrams, Networks, Maps
Bertin, J., & Berg, W. J. (2010) · Cited by 1,555
Constraint logic programming: a survey
Jaffar, J., & Maher, M. J. (1994) · The Journal of Logic Programming · doi:10.1016/0743-1066(94)90033-7 · Cited by 1,312
Answer set programming at a glance
Brewka, G., Eiter, T., & Truszczyński, M. (2011) · Communications of the ACM · doi:10.1145/2043174.2043195 · Cited by 931
Vega-Lite: A Grammar of Interactive Graphics
Satyanarayan, A., Moritz, D., Wongsuphasawat, K., & Heer, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2599030 · Cited by 883
Self-citationCrowdsourcing graphical perception: using mechanical turk to assess visualization design
Heer, J., & Bostock, M. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753357 · Cited by 816
Self-citationWilkinson, L. (2010) · WIREs Computational Statistics · doi:10.1002/wics.118 · Cited by 712
Constraint-directed search : a case study of job-shop scheduling
Fox, M. S. (2018) · doi:10.1184/r1/6552371 · Cited by 526
Low-Level Components of Analytic Activity in Information Visualization
Amar, R., Eagan, J., & Stasko, J. (2006) · Proceedings of the 2005 IEEE Symposium on Information Visualization (INFOVIS'05) · doi:10.1109/infovis.2005.24 · Cited by 512
Voyager: Exploratory Analysis via Faceted Browsing of Visualization Recommendations
Wongsuphasawat, K., Moritz, D., Anand, A., Mackinlay, J., Howe, B., & Heer, J. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467191 · Cited by 507
Self-citationWhat is answer set programming
Lifschitz, V. (2008) · National Conference on Artificial Intelligence · Cited by 494
Show Me: Automatic Presentation for Visual Analysis
Mackinlay, J., Hanrahan, P., & Stolte, C. (2007) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2007.70594 · Cited by 480
Potassco: The Potsdam Answer Set Solving Collection
Gebser, M., Kaufmann, B., Kaminski, R., Ostrowski, M., Schaub, T., & Schneider, M. (2011) · AI Communications · doi:10.3233/aic-2011-0491 · Cited by 456
Support vector learning for ordinal regression
Herbrich, R. (1999) · 9th International Conference on Artificial Neural Networks: ICANN '99 · doi:10.1049/cp:19991091 · Cited by 448
Answer set programming and plan generation
Lifschitz, V. (2002) · Artificial Intelligence · doi:10.1016/s0004-3702(02)00186-8 · Cited by 442
Learning Explanatory Rules from Noisy Data
Evans, R., & Grefenstette, E. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5714 · Cited by 426
Conflict-driven answer set solving
Gebser, M., Kaufmann, B., Neumann, A., & Schaub, T. H. (2007) · publish.UP (University of Potsdam) · Cited by 374
Entity Resolution with Markov Logic
Singla, P., & Domingos, P. (2006) · Sixth International Conference on Data Mining (ICDM'06) · doi:10.1109/icdm.2006.65 · Cited by 359
Task-analytic approach to the automated design of graphic presentations
Casner, S. M. (1991) · ACM Transactions on Graphics · doi:10.1145/108360.108361 · Cited by 345
Learning the structure of Markov logic networks
Kok, S., & Domingos, P. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102407 · Cited by 298
Voyager 2: Augmenting Visual Analysis with Partial View Specifications
Wongsuphasawat, K., Qu, Z., Moritz, D., Chang, R., Ouk, F., Anand, A., Mackinlay, J., Howe, B., & Heer, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025768 · Cited by 295
Self-citationAnswer Set Solving in Practice
Gebser, M., Kaminski, R., Kaufmann, B., & Schaub, T. H. (2012) · Synthesis lectures on artificial intelligence and machine learning · doi:10.2200/s00457ed1v01y201211aim019 · Cited by 288
A Principled Way of Assessing Visualization Literacy
Boy, J., Rensink, R. A., Bertini, E., & Fekete, J.-D. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346984 · Cited by 278
Poon, H., & Domingos, P. (2009) · Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing Volume 1 - EMNLP '09 · doi:10.3115/1699510.1699512 · Cited by 253
Discriminative training of Markov logic networks
Singla, P., & Domingos, P. (2005) · Cited by 245
DataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization
Gao, T., Dontcheva, M., Adar, E., Liu, Z., & Karahalios, K. G. (2015) · Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology · doi:10.1145/2807442.2807478 · Cited by 242
An A-Prolog Decision Support System for the Space Shuttle
Nogueira, M. L., Balduccini, M., Gelfond, M., et al. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45241-9_12 · Cited by 232
Eviza: A Natural Language Interface for Visual Analysis
Setlur, V., Battersby, S. E., Tory, M., Gossweiler, R., & Chang, A. X. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984588 · Cited by 231
Clingo = ASP + Control: Preliminary Report
Gebser, M., Kaminski, R., Kaufmann, B., & Schaub, T. (2014) · arXiv · doi:10.48550/arxiv.1405.3694 · Cited by 230
The Elements of Statistical Learning: Data Mining, Inference, and Prediction 2nd Edition
Hastie, T., Tibshirani, R. J., & Friedman, J. H. (2020) · Journal of International Crisis and Risk Communication Research · Cited by 229
Ranking Visualizations of Correlation Using Weber's Law
Harrison, L., Yang, F., Franconeri, S., & Chang, R. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346979 · Cited by 212
Modeling Color Difference for Visualization Design
Szafir, D. A. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744359 · Cited by 207
Data characterization for intelligent graphics presentation
Roth, S. F., & Mattis, J. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97273 · Cited by 201
Answer Set Programming for Procedural Content Generation: A Design Space Approach
Smith, A. M., & Mateas, M. (2011) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2011.2158545 · Cited by 201
Self-citationHow Deceptive are Deceptive Visualizations? An Empirical Analysis of Common Distortion Techniques
Pandey, A. V., Rall, K., Satterthwaite, M. L., Nov, O., & Bertini, E. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702608 · Cited by 181
Task-based effectiveness of basic visualizations
Saket, B., Endert, A., & Demiralp, C. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2829750 · Cited by 178 · cited 2×
“[51] analyze how effectiveness varies by task.”
MethodInstance‐based prediction of real‐valued attributes
Kibler, D. F., Aha, D. W., & Albert, M. K. (1989) · Computational Intelligence · doi:10.1111/j.1467-8640.1989.tb00315.x · Cited by 154
Four types of ensemble coding in data visualizations
Szafir, D. A., Haroz, S., Gleicher, M. L., & Franconeri, S. (2016) · Journal of Vision · doi:10.1167/16.5.11 · Cited by 142
Qu, Z., & Hullman, J. R. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744198 · Cited by 133
Complex optimization in answer set programming
Gebser, M., Kaminski, R., & Schaub, T. H. (2011) · Theory and Practice of Logic Programming · doi:10.1017/s1471068411000329 · Cited by 124
Towards a general-purpose query language for visualization recommendation
Wongsuphasawat, K., Moritz, D., Anand, A., Mackinlay, J. D., Howe, B., & Heer, J. (2016) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/2939502.2939506 · Cited by 123
Self-citationA case study of expressively constrainable level design automation tools for a puzzle game
Smith, A. M., Andersen, E., Mateas, M., & Popović, Z. (2012) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/2282338.2282370 · Cited by 116
Self-citationPerception of Average Value in Multiclass Scatterplots
Gleicher, M. L., Correll, M., Nothelfer, C. E., & Franconeri, S. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.183 · Cited by 110
Assessing Effects of Task and Data Distribution on the Effectiveness of Visual Encodings
Kim, Y.-H., & Heer, J. (2018) · Computer Graphics Forum · doi:10.1111/cgf.13409 · Cited by 98
Self-citationGraphScape: A Model for Automated Reasoning about Visualization Similarity and Sequencing
Kim, Y.-H., Wongsuphasawat, K., Hullman, J. R., & Heer, J. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025866 · Cited by 97
Self-citationGebser, M., Harrison, A., Kaminski, R., Lifschitz, V., & Schaub, T. H. (2015) · Theory and Practice of Logic Programming · doi:10.1017/s1471068415000150 · Cited by 90
Describing complex charts in natural language: a caption generation system
Mittal, V. O., Carenini, G., Moore, J. D., & Roth, S. F. (1998) · Cited by 88
Extracting and Retargeting Color Mappings from Bitmap Images of Visualizations
Poco, J., Mayhua-Quispe, A., & Heer, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744320 · Cited by 85
Self-citationSrinivasan, A., & Stasko, J. (2017) · Eurographics · doi:10.2312/eurovisshort.20171133 · Cited by 63
Constraint Programming - What is behind?
Barták, R. (1999) · Cited by 27
Cited by
Wu, T., Terry, M., & Cai, C. J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517582 · Cited by 483
Agency plus automation: Designing artificial intelligence into interactive systems
Heer, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1807184115 · Cited by 279
“This underlying model is adaptive: In response to user actions or new perception experiments, we can retrain our model of preference weights to generate refined chart design recommendations (19).”
MethodSelf-citationCalliope: Automatic Visual Data Story Generation from a Spreadsheet
Shi, D., Xu, X., Sun, F., Shi, Y., & Cao, N. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2020.3030403 · Cited by 182
Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizations
Srinivasan, A., Nyapathy, N., Lee, B., Drucker, S. M., & Stasko, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445400 · Cited by 58
“, chart type), detect attributes in input utterances with toolkits like NL4DV [18], and then determine the most perceptually efective bindings for the classifed chart type’s encoding channels using tools like CompassQL [29] and Draco [17].”
MethodData Player: Automatic Generation of Data Videos with Narration-Animation Interplay
Shen, L., Zhang, Y., Zhang, H., & Wang, Y. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327197 · Cited by 45
Metadata
- Authors
- Moritz, D., Wang, C., Nelson, G. L., Lin, H., Smith, A. M., Howe, B., & Heer, J.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2019-01
- Volume
- 25
- Issue
- 1
- Pages
- 438-448
- ISSN
- 1077-2626 (ISSN-L)
- 1941-0506
- 2160-9306
- OpenAlex
- W2888611489
- Cited by
- 317
- References
- 62
- FWCI
- 15.95
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.