CoChat
Sign Up
Cited by 77Data Visualization and AnalyticsInnovative Human-Technology InteractionDigital Media and Visual Art

Where are we so far? Understanding data storytelling tools from the perspective of human-AI collaboration

Open access
Li, Haotian, Wang, Yun, Qu, Huamin·2024·Proceedings of the CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3613904.3642726
Open this paper in CoChat — free accountOpen PDF

Abstract

Data storytelling is powerful for communicating data insights, but it requires diverse skills and considerable effort from human creators. Recent research has widely explored the potential for artificial intelligence (AI) to support and augment humans in data storytelling. However, there lacks a systematic review to understand data storytelling tools from the perspective of human-AI collaboration, which hinders researchers from reflecting on the existing collaborative tool designs that promote humans’ and AI’s advantages and mitigate their shortcomings. This paper investigated existing tools with a framework from two perspectives: the stages in the storytelling workflow where a tool serves, including analysis, planning, implementation, and communication, and the roles of humans and AI in each stage, such as creators, assistants, optimizers, and reviewers. Through our analysis, we recognize the common collaboration patterns in existing tools, summarize lessons learned from these patterns, and further illustrate research opportunities for human-AI collaboration in data storytelling.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642726

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI has significantly impacted various creative domains, reshaping fields such as art and design.

    “Advancements in AI technology have significantly impacted various creative domains [3, 59], reshaping fields such as art [19] and design [58].”

    From Understanding Screenwriters' Practices, Attitudes, and Future Expectations in Human-AI Co-Creation (2025) · cites this paper as [59]

    The Introduction and Conclusion describe AI-assisted data storytelling tools and increasing research interest in them. This supports an impact on data storytelling, but the paper does not establish significant impact across various creative domains. The separate citations attached to art and design are not assessed here. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 128 · Cited by 77

Research works printed but not linked: 142

  1. Hiroshi Akiba, Chaoli Wang, and Kwan-Liu Ma. 2009. AniViz: A Template- Based Animation Tool for Volume Visualization. IEEE Computer Graphics and Applications 30, 5 (2009), 61-71.
  2. Saleema Amershi, Daniel S. Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi T. Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human- AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-13.
  3. Fereshteh Amini, Nathalie Henry Riche, Bongshin Lee, Andres Monroy- Hernandez, and Pourang Irani. 2016. Authoring Data-Driven Videos with DataClips. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 501-510.
  4. Leilani Battle and Alvitta Ottley. 2022. A Programmatic Definition of Visualiza- tion Insights, Objectives, and Tasks. arXiv preprint arXiv:2206.04767 (2022). · arXiv
  5. Leilani Battle and Alvitta Ottley. 2023. What Exactly is an Insight? A Literature Review. arXiv preprint arXiv:2307.06551 (2023). · arXiv
  6. Matthew Brehmer and Robert Kosara. 2021. From Jam Session to Recital: Syn- chronous Communication and Collaboration Around Data in Organizations. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 1139- 1149.
  7. Matthew Brehmer, Bongshin Lee, Nathalie Henry Riche, David Tittsworth, Kate Lytvynets, Darren Edge, and Christopher White. 2019. Timeline Storyteller: The Design & Deployment of an Interactive Authoring Tool for Expressive Timeline Narratives. In Proceedings of the Computation+Journalism Symposium, Vol. 2.
  8. Chris Bryan, Kwan-Liu Ma, and Jonathan Woodring. 2016. Temporal Summary Images: An Approach to Narrative Visualization via Interactive Annotation Generation and Placement. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 511-520.
  9. Yining Cao, Jane L E, Zhutian Chen, and Haijun Xia. 2023. DataParticles: Block- based and Language-oriented Authoring of Animated Unit Visualizations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-15.
  10. Qing Chen, Shixiong Cao, Jiazhe Wang, and Nan Cao. 2023. How Does Au- tomation Shape the Process of Narrative Visualization: A Survey of Tools. IEEE Transactions on Visualization and Computer Graphics (2023).
  11. Qing Chen, Fuling Sun, Xinyue Xu, Zui Chen, Jiazhe Wang, and Nan Cao. 2021. VizLinter: A Linter and Fixer Framework for Data Visualization. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 206-216.
  12. Yang Chen, Scott Barlowe, and Jing Yang. 2010. Click2Annotate: Automated Insight Externalization with Rich Semantics. In Proceedings of 2010 IEEE Sympo- sium on Visual Analytics Science and Technology. IEEE, 155-162.
  13. Yang Chen, Jing Yang, Scott Barlowe, and Dong H Jeong. 2010. Touch2Annotate: Generating Better Annotations with Less Human Effort on Multi-touch Inter- faces. In Extended Abstracts of the 2010 CHI Conference on Human Factors in Computing Systems. 3703-3708.
  14. Zhutian Chen, Yun Wang, Qianwen Wang, Yong Wang, and Huamin Qu. 2019. Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timeline. IEEE Transactions on Visualization and Computer Graph- ics 26, 1 (2019), 917-926.
  15. Zhutian Chen, Qisen Yang, Xiao Xie, Johanna Beyer, Haijun Xia, Yingcai Wu, and Hanspeter Pfister. 2022. Sporthesia: Augmenting Sports Videos Using Natural Language. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 918-928.
  16. Zhutian Chen, Shuainan Ye, Xiangtong Chu, Haijun Xia, Hui Zhang, Huamin Qu, and Yingcai Wu. 2021. Augmenting Sports Videos with VisCommentator. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 824-834.
  17. Hao Cheng, Junhong Wang, Yun Wang, Bongshin Lee, Haidong Zhang, and Dongmei Zhang. 2022. Investigating the Role and Interplay of Narrations and Animations in Data Videos. Computer Graphics Forum 41, 3 (2022), 527-539.
  18. Fanny Chevalier, Nathalie Henry Riche, Catherine Plaisant, Amira Chalbi, and Christophe Hurter. 2016. Animations 25 Years Later: New Roles and Opportuni- ties. In Proceedings of the International Working Conference on Advanced Visual Interfaces. 280-287.
  19. Fanny Chevalier, Melanie Tory, Bongshin Lee, Jarke van Wijk, Giuseppe San- tucci, Marian Dörk, and Jessica Hullman. 2018. From Analysis to Communi- cation: Supporting the Lifecycle of a Story. In Data-Driven Storytelling. AK Peters/CRC Press, 151-183.
  20. Noptanit Chotisarn, Junhua Lu, Libinzi Ma, Jingli Xu, Linhao Meng, Bingru Lin, Ying Xu, Xiaonan Luo, and Wei Chen. 2021. Bubble Storytelling with Automated Animation: A Brexit Hashtag Activism Case Study. Journal of Visualization 24 (2021), 101-115.
  21. Darius Coelho and Klaus Mueller. 2020. Infomages: Embedding Data into Thematic Images. Computer Graphics Forum 39, 3 (2020), 593-606.
  22. Matthew Conlen, Megan Vo, Alan Tan, and Jeffrey Heer. 2021. Idyll Studio: A Structured Editor for Authoring Interactive & Data-Driven Articles. In Pro- ceedings of the 34th Annual ACM Symposium on User Interface Software and Technology. 1-12.
  23. Anamaria Crisan and Brittany Fiore-Gartland. 2021. Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-15.
  24. Anamaria Crisan, Brittany Fiore-Gartland, and Melanie Tory. 2020. Passing the Data Baton: A Retrospective Analysis on Data Science Work and Workers. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1860-1870.
  25. Weiwei Cui, Jinpeng Wang, He Huang, Yun Wang, Chin-Yew Lin, Haidong Zhang, and Dongmei Zhang. 2021. A Mixed-Initiative Approach to Reusing Infographic Charts. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 173-183.
  26. Weiwei Cui, Xiaoyu Zhang, Yun Wang, He Huang, Bei Chen, Lei Fang, Haidong Zhang, Jian-Guan Lou, and Dongmei Zhang. 2019. Text-to-Viz: Automatic Gen- eration of Infographics from Proportion-Related Natural Language Statements. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 906-916.
  27. Sebastian Deterding, Jonathan Hook, Rebecca Fiebrink, Marco Gillies, Jeremy Gow, Memo Akten, Gillian Smith, Antonios Liapis, and Kate Compton. 2017. Mixed-Initiative Creative Interfaces. In Extended Abstracts of the 2017 CHI Con- ference on Human Factors in Computing Systems. 628-635.
  28. Victor Dibia. 2023. LIDA: A Tool for Automatic Generation of Grammar- Agnostic Visualizations and Infographics using Large Language Models. arXiv preprint arXiv:2303.02927 (2023). · arXiv
  29. Rui Ding, Shi Han, Yong Xu, Haidong Zhang, and Dongmei Zhang. 2019. Quick- Insights: Quick and Automatic Discovery of Insights from Multi-Dimensional Data. In Proceedings of the 2019 ACM International Conference on Management of Data. 317-332.
  30. Veronika Domova and Katerina Vrotsou. 2022. A Model for Types and Levels of Automation in Visual Analytics: A Survey, a Taxonomy, and Examples. IEEE Transactions on Visualization and Computer Graphics (2022).
  31. Christina Elmer, Jonathan Schwabish, and Benjamin Wiederkehr. 2018. Orga- nizing the Work of Data-Driven Visual Storytelling. In Data-Driven Storytelling. AK Peters/CRC Press, 185-208.
  32. Mica R Endsley. 1987. The Application of Human Factors to the Development of Expert Systems for Advanced Cockpits. In Proceedings of the Human Factors Society Annual Meeting, Vol. 31. SAGE Publications Sage CA: Los Angeles, CA, 1388-1392.
  33. Arlen Fan, Yuxin Ma, Michelle Mancenido, and Ross Maciejewski. 2022. An- notating Line Charts for Addressing Deception. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-12.
  34. Zezheng Feng, Huamin Qu, Shuang-Hua Yang, Yulong Ding, and Jie Song. 2022. A Survey of Visual Analytics in Urban Area. Expert Systems 39, 9 (2022), e13065.
  35. Xin Fu, Yun Wang, Haoyu Dong, Weiwei Cui, and Haidong Zhang. 2019. Visu- alization Assessment: A Machine Learning Approach. In Proceedings of the 2019 IEEE Visualization Conference. IEEE, 126-130.
  36. Tong Gao, Jessica R Hullman, Eytan Adar, Brent Hecht, and Nicholas Diakopou- los. 2014. NewsViews: An Automated Pipeline for Creating Custom Geovisual- izations for News. In Proceedings of the 2014 CHI Conference on Human Factors in Computing Systems. 3005-3014.
  37. Tong Ge, Bongshin Lee, and Yunhai Wang. 2021. CAST: Authoring Data-Driven Chart Animations. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-15.
  38. Samuel Gratzl, Alexander Lex, Nils Gehlenborg, Nicola Cosgrove, and Marc Streit. 2016. From Visual Exploration to Storytelling and Back Again. Computer Graphics Forum 35, 3 (2016), 491-500.
  39. Brian D Hall, Lyn Bartram, and Matthew Brehmer. 2022. Augmented Chironomia for Presenting Data to Remote Audiences. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1-14.
  40. Jeffrey Heer. 2019. Agency Plus Automation: Designing Artificial Intelligence into Interactive Systems. Proceedings of the National Academy of Sciences 116, 6 (2019), 1844-1850.
  41. Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proceedings of the 1999 CHI Conference on Human Factors in Computing Systems. 159-166.
  42. Youyang Hou and Dakuo Wang. 2017. Hacking with NPOs: Collaborative Analytics and Broker Roles in Civic Data Hackathons. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1-16.
  43. Kevin Hu, Michiel A Bakker, Stephen Li, Tim Kraska, and César Hidalgo. 2019. VizML: A Machine Learning Approach to Visualization Recommendation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.
  44. Jessica Hullman, Nicholas Diakopoulos, and Eytan Adar. 2013. Contextifier: Automatic Generation of Annotated Stock Visualizations. In Proceedings of the 2013 CHI Conference on Human Factors in Computing Systems. 2707-2716.
  45. Jessica Hullman, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher, and Eytan Adar. 2013. A Deeper Understanding of Sequence in Narrative Visualization. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2406-2415.
  46. Jialun Aaron Jiang, Kandrea Wade, Casey Fiesler, and Jed R Brubaker. 2021. Sup- porting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-23.
  47. DaYe Kang, Tony Ho, Nicolai Marquardt, Bilge Mutlu, and Andrea Bianchi. 2021. ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-14.
  48. Nam Wook Kim, Nathalie Henry Riche, Benjamin Bach, Guanpeng Xu, Matthew Brehmer, Ken Hinckley, Michel Pahud, Haijun Xia, Michael J McGuffin, and Hanspeter Pfister. 2019. DataToon: Drawing Dynamic Network Comics With Pen + Touch Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.
  49. Nam Wook Kim, Hyejin Im, Nathalie Henry Riche, Alicia Wang, Krzysztof Gajos, and Hanspeter Pfister. 2019. DataSelfie: Empowering People to Design Personalized Visuals to Represent Their Data. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.
  50. Nam Wook Kim, Eston Schweickart, Zhicheng Liu, Mira Dontcheva, Wilmot Li, Jovan Popovic, and Hanspeter Pfister. 2016. Data-Driven Guides: Supporting Expressive Design for Information Graphics. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 491-500.
  51. Younghoon Kim and Jeffrey Heer. 2021. Gemini 2 : Generating Keyframe-Oriented Animated Transitions Between Statistical Graphics. In Proceedings of the 2021 IEEE Visualization Conference. IEEE, 201-205.
  52. Younghoon Kim, Kanit Wongsuphasawat, Jessica Hullman, and Jeffrey Heer. 2017. GraphScape: A Model for Automated Reasoning about Visualization Similarity and Sequencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2628-2638.
  53. David Kouril, Ondrej Strnad, Peter Mindek, Sarkis Halladjian, Tobias Isenberg, Eduard Groeller, and Ivan Viola. 2021. Molecumentary: Adaptable Narrated Doc- umentaries Using Molecular Visualization. IEEE Transactions on Visualization and Computer Graphics (2021).
  54. Shahid Latif, Zheng Zhou, Yoon Kim, Fabian Beck, and Nam Wook Kim. 2021. Kori: Interactive Synthesis of Text and Charts in Data Documents. IEEE Trans- actions on Visualization and Computer Graphics 28, 1 (2021), 184-194.
  55. Bongshin Lee, Rubaiat Habib Kazi, and Greg Smith. 2013. SketchStory: Telling More Engaging Stories with Data through Freeform Sketching. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2416-2425.
  56. Bongshin Lee, Nathalie Henry Riche, Petra Isenberg, and Sheelagh Carpendale. 2015. More Than Telling a Story: Transforming Data into Visually Shared Stories. IEEE Computer Graphics and Applications 35, 5 (2015), 84-90.
  57. Haotian Li, Yun Wang, Q Vera Liao, and Huamin Qu. 2023. Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling. arXiv preprint arXiv:2304.08366 (2023). · arXiv
  58. Haotian Li, Yong Wang, Aoyu Wu, Huan Wei, and Huamin Qu. 2022. Structure- aware Visualization Retrieval. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-14.
  59. Haotian Li, Min Xu, Yong Wang, Huan Wei, and Huamin Qu. 2021. A Visual Analytics Approach to Facilitate the Proctoring of Online Exams. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-17.
  60. Lucy Li and David Bamman. 2021. Gender and Representation Bias in GPT-3 Generated Stories. In Proceedings of the Third Workshop on Narrative Under- standing. 48-55.
  61. Wenchao Li, Sarah Schöttler, James Scott-Brown, Yun Wang, Siming Chen, Huamin Qu, and Benjamin Bach. 2023. NetworkNarratives: Data Tours for Visual Network Exploration and Analysis. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-15.
  62. Wenchao Li, Zhan Wang, Yun Wang, Di Weng, Liwenhan Xie, Siming Chen, Haidong Zhang, and Huamin Qu. 2023. GeoCamera: Telling Stories in Geo- graphic Visualizations with Camera Movements. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-15.
  63. Joseph CR Licklider. 1960. Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics 1 (1960), 4-11.
  64. Yanna Lin, Haotian Li, Aoyu Wu, Yong Wang, and Huamin Qu. 2023. DMiner: Dashboard Design Mining and Recommendation. IEEE Transactions on Visual- ization and Computer Graphics (2023).
  65. Yanna Lin, Haotian Li, Leni Yang, Aoyu Wu, and Huamin Qu. 2023. InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computa- tional Notebooks. arXiv preprint arXiv:2307.07922 (2023). · arXiv
  66. Dongyu Liu, Di Weng, Yuhong Li, Jie Bao, Yu Zheng, Huamin Qu, and Yingcai Wu. 2017. SmartAdP: Visual Analytics of Large-scale Taxi Trajectories for Selecting Billboard Locations. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 1-10.
  67. Shuqi Liu, Mingtian Tao, Yifei Huang, Changbo Wang, and Chenhui Li. 2022. Image-Driven Harmonious Color Palette Generation for Diverse Information Visualization. IEEE Transactions on Visualization and Computer Graphics (2022).
  68. Paul Benjamin Lowry, Aaron Curtis, and Michelle René Lowry. 2004. Building a Taxonomy and Nomenclature of Collaborative Writing to Improve Interdis- ciplinary Research and Practice. The Journal of Business Communication 41, 1 (2004), 66-99.
  69. Junhua Lu, Wei Chen, Hui Ye, Jie Wang, Honghui Mei, Yuhui Gu, Yingcai Wu, Xiaolong Luke Zhang, and Kwan-Liu Ma. 2021. Automatic Generation of Unit Visualization-based Scrollytelling for Impromptu Data Facts Delivery. In Proceedings of the 2021 IEEE Pacific Visualization Symposium. IEEE, 21-30.
  70. Junhua Lu, Jie Wang, Hui Ye, Yuhui Gu, Zhiyu Ding, Mingliang Xu, and Wei Chen. 2020. Illustrating Changes in Time-Series Data With Data Video. IEEE Computer Graphics and Applications 40, 2 (2020), 18-31.
  71. Todd Lubart. 2005. How can Computers be Partners in the Creative Process: Classification and Commentary on the Special Issue. International Journal of Human-Computer Studies 63, 4-5 (2005), 365-369.
  72. Yaoli Mao, Dakuo Wang, Michael Muller, Kush R Varshney, Ioana Baldini, Casey Dugan, and Aleksandra Mojsilović. 2019. How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question? Proceedings of the ACM on Human-Computer Interaction 3, GROUP (2019), 1-23.
  73. Andreas Mathisen, Tom Horak, Clemens Nylandsted Klokmose, Kaj Grønbæk, and Niklas Elmqvist. 2019. InsideInsights: Integrating Data-Driven Reporting in Collaborative Visual Analytics. Computer Graphics Forum 38, 3 (2019), 649-661.
  74. Liam McNabb and Robert S Laramee. 2017. Survey of Surveys (SoS) -Map- ping The Landscape of Survey Papers in Information Visualization. Computer Graphics Forum 36, 3 (2017), 589-617.
  75. Ronald Metoyer, Qiyu Zhi, Bart Janczuk, and Walter Scheirer. 2018. Coupling Story to Visualization: Using Textual Analysis as a Bridge Between Data and Interpretation. In Proceedings of the 23rd International Conference on Intelligent User Interfaces. 503-507.
  76. Yao Ming, Huamin Qu, and Enrico Bertini. 2018. RuleMatrix: Visualizing and Understanding Classifiers with Rules. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 342-352.
  77. Eric Mörth, Stefan Bruckner, and Noeska N Smit. 2022. ScrollyVis: Interactive Visual Authoring of Guided Dynamic Narratives for Scientific Scrollytelling. IEEE Transactions on Visualization and Computer Graphics (2022).
  78. Humphrey O Obie, Caslon Chua, Iman Avazpour, Mohamed Abdelrazek, John Grundy, and Tomasz Bednarz. 2020. Authoring Logically Sequenced Visual Data Stories with Gravity. Journal of Computer Languages 58 (2020), 100961.
  79. Humphrey O Obie, Dac Thanh Chuong Ho, Iman Avazpour, John Grundy, Mohamed Abdelrazek, Tomasz Bednarz, and Caslon Chua. 2022. Gravity++: A Graph-based Framework for Constructing Interactive Visualization Narratives. Journal of Computer Languages 71 (2022), 101125.
  80. OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023). · arXiv
  81. Joon Sung Park, Joseph C O'Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. arXiv preprint arXiv:2304.03442 (2023). · arXiv
  82. Ilona R Posner and Ronald M Baecker. 1992. How People Write Together (Groupware). In Proceedings of the 25th Hawaii International Conference on System Sciences, Vol. 4. IEEE, 127-138.
  83. Xiaoying Pu, Sean Kross, Jake M Hofman, and Daniel G Goldstein. 2021. Data- mations: Animated Explanations of Data Analysis Pipelines. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-14.
  84. Chunyao Qian, Shizhao Sun, Weiwei Cui, Jian-Guang Lou, Haidong Zhang, and Dongmei Zhang. 2020. Retrieve-Then-Adapt: Example-based Automatic Gener- ation for Proportion-related Infographics. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 443-452.
  85. Xuedi Qin, Yuyu Luo, Nan Tang, and Guoliang Li. 2020. Making Data Visu- alization More Efficient and Effective: A Survey. The VLDB Journal 29 (2020), 93-117.
  86. Donghao Ren, Matthew Brehmer, Bongshin Lee, Tobias Höllerer, and Eun Ky- oung Choe. 2017. ChartAccent: Annotation for Data-Driven Storytelling. In Proceedings of the 2017 IEEE Pacific Visualization Symposium. 230-239.
  87. Donghao Ren, Bongshin Lee, and Matthew Brehmer. 2018. Charticulator: Interac- tive Construction of Bespoke Chart Layouts. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 789-799.
  88. Pengkun Ren, Yi Wang, and Fan Zhao. 2023. Re-understanding of Data Sto- rytelling Tools from a Narrative Perspective. Visual Intelligence 1, 1 (2023), 11.
  89. Arvind Satyanarayan and Jeffrey Heer. 2014. Authoring Narrative Visualizations with Ellipsis. Computer Graphics Forum 33, 3 (2014), 361-370.
  90. Edward Segel and Jeffrey Heer. 2010. Narrative Visualization: Telling Stories with Data. IEEE Transactions on Visualization and Computer Graphics 16, 6 (2010), 1139-1148.
  91. Rita Sevastjanova, Mennatallah El-Assady, Adam Bradley, Christopher Collins, Miriam Butt, and Daniel Keim. 2021. VisInReport: Complementing Visual Discourse Analytics Through Personalized Insight Reports. IEEE Transactions on Visualization and Computer Graphics 28, 12 (2021), 4757-4769.
  92. Leixian Shen, Yizhi Zhang, Haidong Zhang, and Yun Wang. 2023. Data Player: Automatic Generation of Data Videos with Narration-Animation Interplay. arXiv preprint arXiv:2308.04703 (2023). · arXiv
  93. Danqing Shi, Fuling Sun, Xinyue Xu, Xingyu Lan, David Gotz, and Nan Cao. 2021. AutoClips: An Automatic Approach to Video Generation from Data Facts. Computer Graphics Forum 40, 3 (2021), 495-505.
  94. Danqing Shi, Xinyue Xu, Fuling Sun, Yang Shi, and Nan Cao. 2020. Calliope: Automatic Visual Data Story Generation from a Spreadsheet. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 453-463.
  95. Yang Shi, Tian Gao, Xiaohan Jiao, and Nan Cao. 2023. Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Liter- ature Review. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1-35.
  96. Yang Shi, Pei Liu, Siji Chen, Mengdi Sun, and Nan Cao. 2022. Supporting Expressive and Faithful Pictorial Visualization Design with Visual Style Transfer. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 236-246.
  97. Minjeong Shin, Joohee Kim, Yunha Han, Lexing Xie, Mitchell Whitelaw, Bum Chul Kwon, Sungahn Ko, and Niklas Elmqvist. 2022. Roslingifier: Semi- Automated Storytelling for Animated Scatterplots. IEEE Transactions on Visual- ization and Computer Graphics (2022).
  98. Ben Shneiderman. 2007. Creativity Support Tools: Accelerating Discovery and Innovation. Commun. ACM 50, 12 (2007), 20-32.
  99. Ben Shneiderman and Pattie Maes. 1997. Direct Manipulation vs. Interface Agents. Interactions 4, 6 (1997), 42-61.
  100. Xinhuan Shu, Aoyu Wu, Junxiu Tang, Benjamin Bach, Yingcai Wu, and Huamin Qu. 2020. What Makes a Data-GIF Understandable? IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1492-1502.
  101. Arjun Srinivasan, Steven M Drucker, Alex Endert, and John Stasko. 2018. Aug- menting Visualizations with Interactive Data Facts to Facilitate Interpretation and Communication. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 672-681.
  102. Manuel Stein, Halldor Janetzko, Andreas Lamprecht, Thorsten Breitkreutz, Philipp Zimmermann, Bastian Goldlücke, Tobias Schreck, Gennady Andrienko, Michael Grossniklaus, and Daniel A Keim. 2017. Bring It to the Pitch: Combining Video and Movement Data to Enhance Team Sport Analysis. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 13-22.
  103. Hariharan Subramonyam and Jessica Hullman. 2023. Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research. arXiv preprint arXiv:2308.06290 (2023). · arXiv
  104. Nicole Sultanum, Fanny Chevalier, Zoya Bylinskii, and Zhicheng Liu. 2021. Leveraging Text-Chart Links to Support Authoring of Data-Driven Articles with VizFlow. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-17.
  105. Nicole Sultanum and Arjun Srinivasan. 2023. DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articles. arXiv preprint arXiv:2308.04076 (2023). · arXiv
  106. Mengdi Sun, Ligan Cai, Weiwei Cui, Yanqiu Wu, Yang Shi, and Nan Cao. 2022. Erato: Cooperative Data Story Editing via Fact Interpolation. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 983-993.
  107. Bo Tang, Shi Han, Man Lung Yiu, Rui Ding, and Dongmei Zhang. 2017. Ex- tracting Top-K Insights from Multi-dimensional Data. In Proceedings of the 2017 ACM International Conference on Management of Data. 1509-1524.
  108. Tan Tang, Junxiu Tang, Jiewen Lai, Lu Ying, Yingcai Wu, Lingyun Yu, and Peiran Ren. 2022. SmartShots: An Optimization Approach for Generating Videos with Data Visualizations Embedded. ACM Transactions on Interactive Intelligent Systems 12, 1 (2022), 1-21.
  109. John R Thompson, Zhicheng Liu, and John Stasko. 2021. Data Animator: Au- thoring Expressive Animated Data Graphics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-18.
  110. Alice Thudt, Jagoda Walny, Theresia Gschwandtner, Jason Dykes, and John Stasko. 2018. Exploration and Explanation in Data-Driven Storytelling. In Data-Driven Storytelling. AK Peters/CRC Press, 59-83.
  111. Chao Tong, Richard C. Roberts, Rita Borgo, Sean P. Walton, Robert S. Laramee, Kodzo Wegba, Aidong Lu, Yun Wang, Huamin Qu, Qiong Luo, and Xiaojuan Ma. 2018. Storytelling and Visualization: An Extended Survey. Information 9, 3 (2018), 65.
  112. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288 (2023). · arXiv
  113. Anjul Tyagi, Jian Zhao, Pushkar Patel, Swasti Khurana, and Klaus Mueller. 2022. Infographics Wizard: Flexible Infographics Authoring and Design Exploration. Computer Graphics Forum 41, 3 (2022), 121-132.
  114. Marialena Vagia, Aksel A Transeth, and Sigurd A Fjerdingen. 2016. A Literature Review on the Levels of Automation during the Years. What are the Different Taxonomies that have been Proposed? Applied Ergonomics 53 (2016), 190-202.
  115. Dakuo Wang, Elizabeth Churchill, Pattie Maes, Xiangmin Fan, Ben Shneiderman, Yuanchun Shi, and Qianying Wang. 2020. From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That Can Work Together with People. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1-6.
  116. Fengjie Wang, Xuye Liu, Oujing Liu, Ali Neshati, Tengfei Ma, Min Zhu, and Jian Zhao. 2023. Slide4N: Creating Presentation Slides from Computational Note- books with Human-AI Collaboration. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-18.
  117. Junpeng Wang, Shixia Liu, and Wei Zhang. 2023. Visual Analytics For Machine Learning: A Data Perspective Survey. arXiv preprint arXiv:2307.07712 (2023). · arXiv
  118. Qianwen Wang, Zhutian Chen, Yong Wang, and Huamin Qu. 2021. A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization. IEEE Transactions on Visualization and Computer Graphics 28, 12 (2021), 5134-5153.
  119. Xumeng Wang, Ziliang Wu, Wenqi Huang, Yating Wei, Zhaosong Huang, Min- gliang Xu, and Wei Chen. 2023. VIS+AI: Integrating Visualization with Artificial Intelligence for Efficient Data Analysis. Frontiers of Computer Science 17, 6 (2023), 1-12.
  120. Xingbo Wang, Haipeng Zeng, Yong Wang, Aoyu Wu, Zhida Sun, Xiaojuan Ma, and Huamin Qu. 2020. VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-12.
  121. Yun Wang, Yi Gao, Ray Huang, Weiwei Cui, Haidong Zhang, and Dongmei Zhang. 2021. Animated Presentation of Static Infographics with InfoMotion. Computer Graphics Forum 40, 3 (2021), 507-518.
  122. Yun Wang, Leixian Shen, Zhengxin You, Xinhuan Shu, Bongshin Lee, John Thompson, Haidong Zhang, and Dongmei Zhang. 2023. WonderFlow: Narration- Centric Design of Animated Data Videos. arXiv preprint arXiv:2308.04040 (2023). · arXiv
  123. Yun Wang, Zhida Sun, Haidong Zhang, Weiwei Cui, Ke Xu, Xiaojuan Ma, and Dongmei Zhang. 2019. DataShot: Automatic Generation of Fact Sheets from Tabular Data. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 895-905.
  124. Yun Wang, Haidong Zhang, He Huang, Xi Chen, Qiufeng Yin, Zhitao Hou, Dongmei Zhang, Qiong Luo, and Huamin Qu. 2018. InfoNice: Easy Creation of Information Graphics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-12.
  125. Zijie J Wang, Alex Kale, Harsha Nori, Peter Stella, Mark E Nunnally, Duen Horng Chau, Mihaela Vorvoreanu, Jennifer Wortman Vaughan, and Rich Caruana. 2022. Interpretability, Then What? Editing Machine Learning Models to Reflect Hu- man Knowledge and Values. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4132-4142.
  126. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned Language Models are Zero-Shot Learners. In Proceedings of the International Conference on Learning Representations.
  127. James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fer- nanda Viégas, and Jimbo Wilson. 2019. The What-If Tool: Interactive Probing of Machine Learning Models. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 56-65.
  128. Florian Wolf and Edward Gibson. 2005. Representing Discourse Coherence: A Corpus-Based Study. Computational Linguistics 31, 2 (2005), 249-287.
  129. Kanit Wongsuphasawat, Zening Qu, Dominik Moritz, Riley Chang, Felix Ouk, Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. 2017. Voyager 2: Augmenting Visual Analysis with Partial View Specifications. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2648-2659.
  130. Aoyu Wu, Yun Wang, Xinhuan Shu, Dominik Moritz, Weiwei Cui, Haidong Zhang, Dongmei Zhang, and Huamin Qu. 2021. AI4VIS: Survey on Artificial In- telligence Approaches for Data Visualization. IEEE Transactions on Visualization and Computer Graphics (2021).
  131. Haijun Xia, Nathalie Henry Riche, Fanny Chevalier, Bruno De Araujo, and Daniel Wigdor. 2018. DataInk: Direct and Creative Data-Oriented Drawing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1-13.
  132. Leni Yang, Cindy Xiong, Jason K Wong, Aoyu Wu, and Huamin Qu. 2021. Explaining with Examples: Lessons Learned from Crowdsourced Introductory Description of Information Visualizations. IEEE Transactions on Visualization and Computer Graphics (2021).
  133. Leni Yang, Xian Xu, XingYu Lan, Ziyan Liu, Shunan Guo, Yang Shi, Huamin Qu, and Nan Cao. 2021. A Design Space for Applying the Freytag's Pyramid Structure to Data Stories. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 922-932.
  134. Yilin Ye, Rong Huang, and Wei Zeng. 2022. VISAtlas: An Image-based Explo- ration and Query System for Large Visualization Collections via Neural Image Embedding. IEEE Transactions on Visualization and Computer Graphics (2022).
  135. Lu Ying, Yun Wang, Haotian Li, Shuguang Dou, Haidong Zhang, Xinyang Jiang, Huamin Qu, and Yingcai Wu. 2023. Reviving Static Charts into Live Charts. arXiv preprint arXiv:2309.02967 (2023). · arXiv
  136. Lin-Ping Yuan, Ziqi Zhou, Jian Zhao, Yiqiu Guo, Fan Du, and Huamin Qu. 2021. InfoColorizer: Interactive Recommendation of Color Palettes for Infographics. IEEE Transactions on Visualization and Computer Graphics 28, 12 (2021), 4252- 4266.
  137. Jiayi Eris Zhang, Nicole Sultanum, Anastasia Bezerianos, and Fanny Chevalier. 2020. DataQuilt: Extracting Visual Elements from Images to Craft Pictorial Visualizations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-13.
  138. Jian Zhao, Shenyu Xu, Senthil Chandrasegaran, Chris Bryan, Fan Du, Aditi Mishra, Xin Qian, Yiran Li, and Kwan-Liu Ma. 2021. ChartStory: Automated Partitioning, Layout, and Captioning of Charts into Comic-Style Narratives. IEEE Transactions on Visualization and Computer Graphics 29, 2 (2021), 1384-1399.
  139. Zhenpeng Zhao and Niklas Elmqvist. 2023. The Stories We Tell About Data: Sur- veying Data-Driven Storytelling Using Visualization. IEEE Computer Graphics and Applications (2023).
  140. Chengbo Zheng, Dakuo Wang, April Yi Wang, and Xiaojuan Ma. 2022. Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Cre- ation for Presenting Data Science Work. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-20.
  141. Sujia Zhu, Guodao Sun, Qi Jiang, Meng Zha, and Ronghua Liang. 2020. A Survey on Automatic Infographics and Visualization Recommendations. Visual Informatics 4, 3 (2020), 24-40.
  142. Jonathan Zong, Josh Pollock, Dylan Wootton, and Arvind Satyanarayan. 2022. Animated Vega-Lite: Unifying Animation with a Grammar of Interactive Graph- ics. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 149-159.

Authors
Li, H., Wang, Y., & Qu, H.
Affiliations
  • Hong Kong University of Science and Technology (HK)
  • Microsoft Research Asia (China) (CN)
ORCID
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-19
DOI
10.1145/3613904.3642726
OpenAlex
W4396832564
Cited by
77
References
128
Citations by year
2026: 24 · 2025: 38 · 2024: 15
FWCI
15.89
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
Research Grants Council, University Grants Committee (16210722)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
Keywords
data storytelling, human-AI collaboration, data transmission

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review