CoChat
Sign Up
Cited by 21Explainable Artificial Intelligence (XAI)Design Education and PracticeInnovative Human-Technology Interaction

Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation

Open access
Wang, Yunlong, Venkatesh, Priyadarshini, Lim, Brian Y·2022·CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3491102.3517551
Open this paper in CoChat — free accountOpen PDF

Abstract

Feedback in creativity support tools can help crowdworkers to improve their ideations. However, current feedback methods require human assessment from facilitators or peers. This is not scalable to large crowds. We propose Interpretable Directed Diversity to automatically predict ideation quality and diversity scores, and provide AI explanations — Attribution, Contrastive Attribution, and Counterfactual Suggestions — to feedback on why ideations were scored (low), and how to get higher scores. These explanations provide multi-faceted feedback as users iteratively improve their ideations. We conducted formative and controlled user studies to understand the usage and usefulness of explanations to improve ideation diversity and quality. Users appreciated that explanation feedback helped focus their efforts and provided directions for improvement. This resulted in explanations improving diversity compared to no feedback or feedback with scores only. Hence, our approach opens opportunities for explainable AI towards scalable and rich feedback for iterative crowd ideation and creativity support tools.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3491102.3517551

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 81

  1. Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y Lim, and Mohan Kankan- halli. 2018. Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. In CHI 2018. https://doi.org/10.1145/3173574. 3174156 · DOI · Paper page
  2. Leonard Adelman, James Gualtieri, and Suzanne Stanford. 1995. Examining the effect of causal focus on the option generation process: An experiment using protocol analysis. Organizational Behavior and Human Decision Processes (1995). https://doi.org/10.1006/obhd.1995.1005 · DOI
  3. Elena Agapie, Bonnie Chinh, Laura R Pina, Diana Oviedo, Molly C Welsh, Gary Hsieh, and Sean Munson. 2018. Crowdsourcing Exercise Plans Aligned with Expert Guidelines and Everyday Constraints. In CHI 2018. ACM, 324. https: //doi.org/10.1145/3173574.3173898 · DOI
  4. Faez Ahmed, Sharath Kumar Ramachandran, Mark Fuge, Samuel Hunter, and Scarlett Miller. 2019. Interpreting idea maps: Pairwise comparisons reveal what makes ideas novel. Journal of Mechanical Design 141, 2 (2019). https://doi.org/ 10.1115/1.4041856 · DOI
  5. Andrew Anderson, Jonathan Dodge, Amrita Sadarangani, Zoe Juozapaitis, Evan Newman, Jed Irvine, Souti Chattopadhyay, Matthew Olson, Alan Fern, and Mar- garet Burnett. 2020. Mental Models of Mere Mortals with Explanations of Rein- forcement Learning. ACM Transactions on Interactive Intelligent Systems 10, 2 (2020). https://doi.org/10.1145/3366485 · DOI
  6. Robert K. Atkinson, Sharon J. Derry, Alexander Renkl, and Donald Wortham. 2000. Learning from examples: Instructional principles from the worked examples research. Review of Educational Research 70, 2 (2000), 181-214. https://doi.org/ 10.3102/00346543070002181 · DOI
  7. Suyun Sandra Bae, Oh-hyun Kwon, Senthil Chandrasegaran, and Kwan-liu Ma. 2020. Spinneret : Aiding Creative Ideation through Non-Obvious Concept Asso- ciations. In CHI 2020. 1-13. https://doi.org/10.1145/3313831.3376746 · DOI
  8. Baptiste Barbot. 2018. The dynamics of creative ideation: Introducing a new assessment paradigm. Frontiers in Psychology (2018). https://doi.org/10.3389/ fpsyg.2018.02529 · DOI
  9. Michael S Bernstein, Greg Little, Robert C Miller, Björn Hartmann, Mark S Ackerman, David R Karger, David Crowell, and Katrina Panovich. 2010. Soylent: A Word Processor with a Crowd Inside. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 313-322. https://doi.org/10.1145/1866029.1866078 · DOI
  10. Aditya Bharadwaj, Pao Siangliulue, Adam Marcus, and Kurt Luther. 2019. Crit- ter: Augmenting Creative Work with Dynamic Checklists, Automated Qual- ity Assurance, and Contextual Reviewer Feedback. In CHI '19. 1-12. https: //doi.org/10.1145/3290605.3300769 · DOI
  11. Alexander Binder, Sebastian Bach, Gregoire Montavon, Klaus Robert Müller, and Wojciech Samek. 2016. Layer-wise relevance propagation for deep neural network architectures. In Information science and applications (ICISA) 2016, Vol. 376. 913- 922. https://doi.org/10.1007/978-981-10-0557-2_87 · DOI
  12. Osvald M Bjelland and Robert Chapman Wood. 2008. An Inside View of IBM's 'Innovation Jam'. MIT Sloan management review 50, 1 (2008), 32.
  13. Margaret A Boden. 1996. Chapter 9 -Creativity. In Artificial Intelligence, Mar- garet A Boden (Ed.). Academic Press, 267-291. https://doi.org/10.1016/B978- 012161964-0/50011-X · DOI
  14. Margaret A Boden. 2004. The creative mind: Myths and mechanisms. Routledge.
  15. Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Ce- likyilmaz, and Yejin Choi. 2020. CoMET: Commonsense transformers for auto- matic knowledge graph construction. ACL 2019 -57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (2020), 4762-4779. https://doi.org/10.18653/v1/p19-1470 arXiv:1906.05317 · DOI · Paper page
  16. Paula Phillips Carson and Kerry D. Carson. 1993. Managing Creativity Enhance- ment Through Goal-Setting and Feeddback. Journal of Creative Behavior 27, 1 (1993), 36-45. https://doi.org/10.1002/j.2162-6057.1993.tb01385.x · DOI
  17. Daniel Cer, Yinfei Yang, Sheng yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Céspedes, Steve Yuan, Chris Tar, Yun Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder. EMNLP 2018 -Conference on Empirical Methods in Natural Lan- guage Processing: System Demonstrations, Proceedings (2018), 169-174. https: //doi.org/10.18653/v1/d18-2029 arXiv:arXiv:1803.11175 · DOI
  18. Joel Chan, Pao Siangliulue, Denisa Qori McDonald, Ruixue Liu, Reza Moradinezhad, Safa Aman, Erin T Solovey, Krzysztof Z Gajos, and Steven P Dow. 2017. Semantically far inspirations considered harmful? accounting for cognitive states in collaborative ideation. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. 93-105. https://doi.org/10.1145/3059454.3059455 · DOI
  19. Jesse Chandler, Cheskie Rosenzweig, Aaron J. Moss, Jonathan Robinson, and Leib Litman. 2019. Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behavior Research Methods 51, 5 (oct 2019), 2022-2038. https://doi.org/10.3758/s13428-019-01273-7 · DOI · Paper page
  20. Robert B. Cialdini, Wilhelmina Wosinska, Daniel W. Barrett, Jonathan Butner, and Malgorzata Gornik-Durose. 1999. Compliance with a request in two cultures: The differential influence of social proof and commitment/consistency on collectivists and individualists. Personality and Social Psychology Bulletin 25, 10 (1999), 1242- 1253. https://doi.org/10.1177/0146167299258006 · DOI · Paper page
  21. Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A. Smith. 2018. Creative writing with a machine in the loop: Case studies on slogans and stories. In International Conference on Intelligent User Interfaces, Proceedings IUI. 329-340. https://doi.org/10.1145/3172944.3172983 · DOI · Paper page
  22. Sven Coppers, Jan Van Den Bergh, Kris Luyten, Karin Coninx, Iulianna Van Der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. Intellingo: An intelligible translation environment. In CHI 2018. 1-13. https://doi.org/10. 1145/3173574.3174098 · DOI
  23. Samuel Rhys Cox, Yunlong Wang, Ashraf Abdul, Christian Von Der Weth, and Brian Y. Lim. 2021. Directed diversity: Leveraging language embedding distances for collective creativity in crowd ideation. In CHI'21. https://doi.org/10.1145/ 3411764.3445782 · DOI
  24. Devleena Das and Sonia Chernova. 2020. Leveraging rationales to improve human task performance. In International Conference on Intelligent User Interfaces, Pro- ceedings IUI. 510-518. https://doi.org/10.1145/3377325.3377512 arXiv:2002.04202 · DOI · Paper page
  25. Roelof A J de Vries, Khiet P Truong, Sigrid Kwint, Constance H C Drossaert, and Vanessa Evers. 2016. Crowd-Designed Motivation: Motivational Messages for Exercise Adherence Based on Behavior Change Theory Roelof. In CHI 2016. ACM, 297-308. https://doi.org/10.1145/2858036.2858229 · DOI
  26. Michael R.P. Dougherty, Charles F. Gettys, and Eve E. Ogden. 1999. MINERVA- DM: A memory processes model for judgments of likelihood. Psychological Review (1999). https://doi.org/10.1037/0033-295X.106.1.180 · DOI
  27. Steven P. Dow, Alana Glassco, Jonathan Kass, Melissa Schwarz, Daniel L. Schwartz, and Scott R. Klemmer. 2010. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Transactions on Computer-Human Interaction 17, 4 (2010). https://doi.org/10.1145/1879831. 1879836 · DOI · Paper page
  28. Steven P. Dow, Kate Heddleston, and Scott R. Klemmer. 2009. The efficacy of prototyping under time constraints. In Proceedings of the 2009 ACM SIGCHI Conference on Creativity and Cognition. 165-174. https://doi.org/10.1145/1640233. 1640260 · DOI
  29. Alejandra Duque-Estrada. 2014. Voyant: Generating Structured Feedback on Visual Designs Using a Crowd of Non-Experts. In CSCW 2014. 1433-1444. https: //doi.org/10.5377/cultura.v24i74.8893 · DOI
  30. Upol Ehsan, Brent Harrison, Larry Chan, and Mark O. Riedl. 2018. Rationaliza- tion: A Neural Machine Translation Approach to Generating Natural Language Explanations. AIES 2018 -Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (2018), 81-87. https://doi.org/10.1145/3278721.3278736 arXiv:1702.07826 · DOI
  31. Jonas Frich, Michael Mose Biskjaer, and Peter Dalsgaard. 2018. Twenty Years of Creativity Research in Human-Computer Interaction: Current State and Future Directions. In Proceedings of the 2018 Designing Interactive Systems Conference (DIS '18). Association for Computing Machinery, New York, NY, USA, 1235-1257. https://doi.org/10.1145/3196709.3196732 · DOI
  32. Mahak Gambhir and Vishal Gupta. 2017. Recent automatic text summarization techniques: a survey. Artificial Intelligence Review 47, 1 (2017), 1-66. https: //doi.org/10.1007/s10462-016-9475-9 · DOI
  33. Vinod Goel. 2010. Neural basis of thinking: Laboratory problems versus real- world problems. Wiley Interdisciplinary Reviews: Cognitive Science (2010). https: //doi.org/10.1002/wcs.71 · DOI
  34. Adam E. Green, David J.M. Kraemer, Jonathan A. Fugelsang, Jeremy R. Gray, and Kevin N. Dunbar. 2012. Neural correlates of creativity in analogical reasoning. Journal of Experimental Psychology: Learning Memory and Cognition (2012). https: //doi.org/10.1037/a0025764 · DOI
  35. Simo Hosio, Niels van Berkel, Jonas Oppenlaender, and Joorge Goncalves. 2020. Crowdsourcing Personalized Weight Loss Diets. Computer January (2020), 63-71. https://doi.org/10.1109/MC.2019.2902542 · DOI
  36. Gaoping Huang and Alexander J Quinn. 2017. BlueSky: Crowd-Powered Uniform Sampling of Idea Spaces. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. Association for Computing Machinery, New York, NY, USA, 119-130. https://doi.org/10.1145/3059454.3059481 · DOI
  37. L. Robin Keller and Joanna L. Ho. 1988. Decision Problem Structuring: Generating Options. IEEE Transactions on Systems, Man and Cybernetics (1988). https: //doi.org/10.1109/21.21599 · DOI
  38. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. 2018. Interpretability beyond feature attribution: Quan- titative Testing with Concept Activation Vectors (TCAV). In 35th International Conference on Machine Learning, ICML 2018, Vol. 6. 4186-4195. arXiv:1711.11279 · arXiv
  39. Ana Cristina Bicharra Klein, Mark, and Garcia. 2015. High-speed idea filtering with the bag of lemons. Decision Support Systems 78 (2015), 39-50. https: //doi.org/10.1016/j.dss.2015.06.005 · DOI
  40. Janin Koch, Nicolas Taffin, Michel Beaudouin-Lafon, Markku Laine, Andrés Lucero, and Wendy E. MacKay. 2020. ImageSense: An Intelligent Collaborative Ideation Tool to Support Diverse Human-Computer Partnerships. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1-27. https: //doi.org/10.1145/3392850 · DOI
  41. Rafal Kocielnik and Gary Hsieh. 2017. Send Me a Different Message: Utilizing Cognitive Space to Create Engaging Message Triggers. In CSCW. 2193-2207. https://doi.org/10.1145/2998181.2998324 · DOI
  42. Brian Y. Lim and Anind K. Dey. 2011. Design of an intelligible mobile context- aware application. Mobile HCI 2011 -13th International Conference on Human- Computer Interaction with Mobile Devices and Services (2011), 157-166. https: //doi.org/10.1145/2037373.2037399 · DOI
  43. Brian Y. Lim and Anind K. Dey. 2013. Evaluating intelligibility usage and use- fulness in a context-aware application. In International Conference on Human- Computer Interaction. 92-101. https://doi.org/10.1007/978-3-642-39342-6_11 · DOI
  44. Ryan Louie, Andy Coenen, Cheng Zhi Huang, Michael Terry, and Carrie J Cai. 2020. Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models. In CHI 2020. 1-13. https://doi.org/10.1145/3313831.3376739 · DOI · Paper page
  45. Todd I Lubart. 2001. Models of the creative process: Past, present and future. Creativity research journal 13, 3-4 (2001), 295-308. https://doi.org/10.1207/ S15326934CRJ1334_07 · DOI
  46. Scott M Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In 31st Conference on Neural Information Processing Systems (NIPS 2017). 4768-4777. https://doi.org/10.5555/3295222.3295230 · DOI
  47. Kurt Luther, Nathan Hahn, Steven P Dow, and Aniket Kittur. 2015. Crowdlines : Supporting Synthesis of Diverse Information Sources through Crowdsourced Out- lines. In The Third AAAI Conference on Human Computation and Crowdsourcing (HCOMP-15). https://ojs.aaai.org/index.php/HCOMP/article/view/13239
  48. Joke Meheus. 2000. Analogical Reasoning in Creative Problem Solving Processes: Logico-Philosophical Perspectives. In Metaphor and Analogy in the Sciences. https://doi.org/10.1007/978-94-015-9442-4_2 · DOI
  49. Tim Miller. 2019. Explanation in artificial intelligence : Insights from the social sciences. Artificial Intelligence 267 (2019), 1-38. https://doi.org/10.1016/j.artint. 2018.07.007 · DOI · Paper page
  50. Fabio Del Missier, Mimì Visentini, and Timo Mäntylä. 2015. Option generation in decision making: Ideation beyond memory retrieval. Frontiers in Psychology (2015). https://doi.org/10.3389/fpsyg.2014.01584 · DOI
  51. Bernard A. Nijstad and Wolfgang Stroebe. 2006. How the Group Affects the Mind: A Cognitive Model of Idea Generation in Groups. Personality and Social Psychology Review 10, 3 (aug 2006), 186-213. https://doi.org/10.1207/s15327957pspr1003_1 · DOI
  52. Bernard A. Nijstad, Wolfgang Stroebe, and Hein F.M. Lodewijkx. 2002. Cognitive stimulation and interference in groups: Exposure effects in an idea generation task. Journal of Experimental Social Psychology (2002). https://doi.org/10.1016/S0022- 1031(02)00500-0 · DOI
  53. Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. 2017. Feature Visual- ization. Distill (2017). https://doi.org/10.23915/distill.00007 · DOI · Paper page
  54. Jonas Oppenlaender and Simo Hosio. 2019. Design Recommendations for Aug- menting Creative Tasks with Computational Priming. In Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia (MUM '19). Associ- ation for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/ 3365610.3365621 · DOI
  55. Jonas Oppenlaender and Simo Hosio. 2019. Supporting Creative Work with Crowd Feedback Systems. In DC2S2 '19:Workshop on Designing Crowd-powered Creativity Support Systems. arXiv:arXiv:2004.09204v1 · arXiv
  56. Jonas Oppenlaender and Elina Kuosmanen. 2021. Hardhats and bungaloos: Comparing crowdsourced design feedback with peer design feedback in the classroom. CHI '21 (2021). https://doi.org/10.1145/3411764.3445380 · DOI
  57. Jonas Oppenlaender, Thanassis Tiropanis, and Simo Hosio. 2020. CrowdUI : Supporting Web Design with the Crowd. Proc. ACM Hum.-Comput. Interact. 4, EICS (2020). https://doi.org/10.1145/3394978 · DOI
  58. Zhenhui Peng, Qingyu Guo, Ka Wing Tsang, and Xiaojuan Ma. 2020. Exploring the Effects of Technological Writing Assistance for Support Providers in Online Mental Health Community. In CHI '20. 1-15. https://doi.org/10.1145/3313831. 3376695 · DOI · Paper page
  59. Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. 2020. Language models as knowledge bases? EMNLP-IJCNLP 2019 -2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference (2020), 2463-2473. https://doi.org/10. 18653/v1/d19-1250 arXiv:1909.01066 · DOI · Paper page
  60. Arkalgud Ramaprasad. 1983. On the definition of feedback. Behavioral Science 28, 1 (1983), 4-13. https://doi.org/10.1002/bs.3830280103 · DOI · Paper page
  61. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?" Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining -KDD '16. ACM Press, New York, New York, USA, 1135-1144. https://doi.org/10.1145/2939672.2939778 · DOI · Paper page
  62. Matthew Richardson, Amit Prakash, and Eric Brill. 2006. Beyond PageRank: Machine learning for static ranking. In Proceedings of the 15th International Conference on World Wide Web. 707-715. https://doi.org/10.1145/1135777.1135881 · DOI
  63. C. Riedl, I. Blohm, J. M. Leimeister, and H. Krcmar. 2010. Rating scales for collective intelligence in innovation communities: Why quick and easy decision making does not get it right. In Thirty First International Conference on Information Systems. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1714524
  64. Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2020. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. International Journal of Com- puter Vision 128, 2 (2020), 336-359. https://doi.org/10.1007/s11263-019-01228-7 arXiv:1610.02391 · DOI · Paper page
  65. Pararth Shah, Dilek Hakkani-Tür, Gokhan Tür, Abhinav Rastogi, Ankur Bapna, Neha Nayak, and Larry Heck. 2018. Building a conversational agent overnight with dialogue self-play. arXiv preprint arXiv:1801.04871 (2018). · arXiv
  66. Ben Shneiderman. 2007. CREATIVITY SUPPORT TOOLS:Accelerating Discovery and Innovation. Commun. ACM 50, 12 (2007), 20-32. http://dl.acm.org/citation. cfm?id=1323689
  67. Pao Siangliulue, Kenneth C Arnold, Krzysztof Z Gajos, and Steven P Dow. 2015. Toward Collaborative Ideation at Scale-Leveraging Ideas from Others to Generate More Creative and Diverse Ideas. In CSCW. https://doi.org/10.1145/2675133. 2675239 · DOI
  68. Pao Siangliulue, Joel Chan, Steven P Dow, and Krzysztof Z Gajos. 2016. Idea- Hound: Improving Large-scale Collaborative Ideation with Crowd-Powered Real- time Semantic Modeling. In UIST 2016 -Proceedings of the 29th Annual Sym- posium on User Interface Software and Technology (UIST '16). ACM, 609-624. https://doi.org/10.1145/2984511.2984578 · DOI
  69. Steven M. Smith. 2010. The Constraining Effects of Initial Ideas. In Group Creativity: Innovation through Collaboration. https://doi.org/10.1093/acprof: oso/9780195147308.003.0002 · DOI
  70. Robert Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI '17). 4444-4451. https: //doi.org/10.5555/3298023.3298212 · DOI
  71. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In 34th International Conference on Machine Learning, ICML 2017, Vol. 7. 5109-5118. https://doi.org/10.5555/3305890.3306024 · DOI
  72. Joe Tullio, Anind K. Dey, Jason Chalecki, and James Fogarty. 2007. How it works: a field study of non-technical users interacting with an intelligent system. In CHI 2007. 31-40. https://doi.org/10.1145/1240624.1240630 · DOI
  73. Kurt Vanlehn, Randolph M Jones, Michelene T H Chi, Kurt Vanlehn, Randolph M Jones, and Michelene T H Chi. 2016. A Model of the Self-Explanation Ef- fect A Model of the Self-Explanation Effect. Journal of the Learning Sciences 8406, December (2016), 1-59. http://www.tandfonline.com/doi/abs/10.1207/ s15327809jls0201_1 · DOI · Paper page
  74. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In NIPS 2017. https://doi.org/10.1109/2943.974352 · DOI
  75. Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y Lim. 2019. Designing Theory-Driven User-Centric Explainable AI. In CHI '19. https://doi.org/10.1145/ 3290605.3300831 · DOI · Paper page
  76. Kai Wang and Jeffrey V. Nickerson. 2017. A literature review on individual creativity support systems. Computers in Human Behavior 74 (2017), 139-151. https://doi.org/10.1016/j.chb.2017.04.035 · DOI
  77. Jingjing Xu, Xuancheng Ren, Junyang Lin, and Xu Sun. 2018. DP-GAN: Diversity- Promoting Generative Adversarial Network for Generating Informative and Diversified Text. arXiv (2018). arXiv:1802.01345 http://arxiv.org/abs/1802.01345 · arXiv
  78. Kexin Bella Yang, Tomohiro Nagashima, Junhui Yao, Joseph Jay Williams, Kenneth Holstein, and Vincent Aleven. 2021. Can Crowds Customize Instructional Mate- rials with Minimal Expert Guidance? Exploring Teacher-guided Crowdsourcing for Improving Hints in an AI-based Tutor. Proceedings of the ACM on Human- Computer Interaction 5, CSCW1 (2021), 1-24. https://doi.org/10.1145/3449193 · DOI
  79. Yu-Chun Grace Yen, Steven P Dow, Elizabeth Gerber, and Brian P Bailey. 2017. Listen to Others, Listen to Yourself: Combining Feedback Review and Reflection to Improve Iterative Design. In Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. ACM, New York, NY, USA, 158-170. https://doi. org/10.1145/3059454.3059468 · DOI
  80. Lixiu Yu and Jeffrey V. Nickerson. 2011. Cooks or cobblers?. In CHI 2011. 1393- 1402. https://doi.org/10.1145/1978942.1979147 · DOI
  81. Wencan Zhang and Brian Y. Lim. 2022. Towards Relatable Explainable AI with the Perceptual Process. In CHI '22. Association for Computing Machinery. https: //doi.org/10.1145/3491102.3501826 arXiv:2112.14005 · DOI · Paper page

Cited by

Sorted by citations

Authors
Wang, Y., Venkatesh, P., & Lim, B. Y.
Affiliations
  • National University of Singapore (SG)
  • University College London (GB)
ORCID
Venue
CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2022-04-29
Pages
1-28
DOI
10.1145/3491102.3517551
OpenAlex
W3199894449
Cited by
21
References
13
Citations by year
2026: 4 · 2025: 8 · 2024: 3 · 2023: 4 · 2022: 2
FWCI
2.35
Type
conference paper
Language
en
Open access
Licence
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Ministry of Education - Singapore (T1 251RES1804)
  • Institute for Health Innovation and Technology, National University of Singapore
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
model explanation, explainable AI, automated feedback, creativity support tools, crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review