CoChat
Sign Up
Cited by 83Usability and User Interface DesignInnovative Human-Technology InteractionSoftware Engineering Research

Generating Automatic Feedback on UI Mockups with Large Language Models

Open access
Duan, Peitong, Warner, Jeremy, Li, Yang, Hartmann, Bjoern·2024·Proceedings of the CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3613904.3642782
Open this paper in CoChat — free account

Abstract

Feedback on user interface (UI) mockups is crucial in design. However, human feedback is not always readily available. We explore the potential of using large language models for automatic feedback. Specifically, we focus on applying GPT-4 to automate heuristic evaluation, which currently entails a human expert assessing a UI’s compliance with a set of design guidelines. We implemented a Figma plugin that takes in a UI design and a set of written heuristics, and renders automatically-generated feedback as constructive suggestions. We assessed performance on 51 UIs using three sets of guidelines, compared GPT-4-generated design suggestions with those from human experts, and conducted a study with 12 expert designers to understand fit with existing practice. We found that GPT-4-based feedback is useful for catching subtle errors, improving text, and considering UI semantics, but feedback also decreased in utility over iterations. Participants described several uses for this plugin despite its imperfect suggestions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642782

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Most prior work on AI-supported UI evaluation-including the single-pass baseline condition-treats models as automated judges: systems that take a screen as input and output a fixed list of issues, with users interacting mainly through coarse-grained, post-hoc prompts to revise it.

    “Most prior work on AI-supported UI evaluation-including our single-pass baseline condition-treats models as automated judges : systems that take a screen as input and output a fi xed list of issues, with users interacting mainly through coarse-grained, post-hoc prompts to revise it [17, 85, 93].”

    From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [17]

    Split into 2 atoms — 0 confirmed, 1 partially supported, 1 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Collapsing perception, interpretation, and evaluation into a single step constrains an AI system's ability to reason in a grounded, context-sensitive manner, often yielding generic, weakly anchored feedback that is difficult to adapt to task or context.

    “For the AI, collapsing perception, interpretation, and evaluation into a single step constrains its ability to reason in a grounded, context-sensitive manner, often yielding feedback that is generic, weakly anchored in the concrete screen, and difficult to adapt to task or context [17, 48].”

    From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [17]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Prior work reports frequent misrecognition of UI structure, limited sensitivity to design intent, and weakly actionable critiques in current AI UI evaluation systems.

    “Prior work reports frequent misrecognition of UI structure [34, 94], limited sensitivity to design intent [17, 57], and weakly actionable critiques [48].”

    From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [17]

    The comparison with human evaluators (p. 13) reports that GPT-4 missed more issues requiring understanding of a UI’s purpose and context, supporting limited sensitivity to design intent. The qualitative results (p. 12) report that five of twelve participants found its suggestions too vague about how to fix violations. The cited passages do not establish frequent misrecognition of UI structure, so that part of the sentence is unsupported by this paper. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Recent generative models can offload parts of UI evaluation: metrics quantify visual properties, predictive models simulate user behaviors, and LLMs detect heuristic violations from screenshots.

    “Metrics can quantify visual properties [71], predictive models can simulate user behaviors [17, 91], and Large Language Models (LLMs) can detect heuristic violations directly from screenshots [17, 85, 93].”

    From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [17]

    The Introduction (p. 2) says GPT-4 can automatically evaluate some UI-design heuristics. But the Implementation section (p. 5) says the tool evaluates a JSON representation of the mockup, not a screenshot; it also reports that a multimodal model performed considerably worse for this task. The paper does not present its model as simulating user behavior. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Our sample of 26 exceeds the power analysis bound and aligns with prior HCI work on AI-supported evaluation and critique.

    “Our sample of 26 exceeds this bound and aligns with prior HCI work on AI-supported evaluation and critique [11, 17, 88].”

    From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [17]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 52

  1. [3][3] Ali Borji and Mehrdad Mohammadian. 2023. Battle of the Wordsmiths: Comparing ChatGPT, GPT-4, Claude, and Bard. https://doi.org/10.2139/ssrn.4476855 - · DOI
  2. [2][2] Rukshan Alexander, David Murray, and Nik Thompson. 2017. Cross-Cultural Web Design Guidelines. In Proceedings of the 14th International Web for All Conference (Perth, Western Australia, Australia) (W4A '17) . Association for Computing Machinery, New York, NY, USA, Article 10, 4 pages. https://doi.org/10.1145/ 3058555.3058574 -
  3. [6][6] Chun-Fu (Richard) Chen, Marco Pistoia, Conglei Shi, Paolo Girolami, Joseph W. Ligman, and Yong Wang. 2017. UI X-Ray: Interactive Mobile UI Testing Based on Computer Vision (IUI '17) . Association for Computing Machinery, New York, NY, USA, 245-255. https://doi.org/10.1145/3025171.3025190 - · DOI
  4. [5][5] Lukas Brückner, Luis A. Leiva, and Antti Oulasvirta. 2022. Learning GUI Completions with User-Defined Constraints. ACM Transactions on Interactive Intelligent Systems (TiiS) 12, 1, Article 6 (mar 2022), 40 pages. https: //doi.org/10.1145/3490034 - · DOI
  5. [7][7] Chin-Yi Cheng, Forrest Huang, Gang Li, and Yang Li. 2023. PLay: Parametrically Conditioned Layout Generation using Latent Diffusion. arXiv:2301.11529 [cs.LG] - · arXiv
  6. [9][9] André de Lima Salgado, Sandra Souza Rodrigues, and Renata Pontin M. Fortes. 2016. Evolving Heuristic Evaluation for Multiple Contexts and Audiences: Perspectives from a Mapping Study. In Proceedings of the 34th ACM International Conference on the Design of Communication (Silver Spring, MD, USA) (SIGDOC '16) . Association for Computing Machinery, New York, NY, USA, Article 19, 8 pages. https://doi.org/10.1145/2987592.2987617 - · DOI
  7. [8][8] Ruijia Cheng, Ziwen Zeng, Maysnow Liu, and Steve W. Dow. 2020. Critique Me. Proceedings of the ACM on Human-Computer Interaction 4 (2020), 1-24. https://api.semanticscholar.org/CorpusID:224804650 -
  8. [10][10] Peitong Duan, Björn Hartmann, Karina Nguyen, Yang Li, Marti Hearst, and Meredith Ringel Morris. 2023. Towards Semantically-Aware UI Design Tools: Design, Implementation and Evaluation of Semantic Grouping Guidelines. In ICML 2023 Workshop on Artificial Intelligence and Human-Computer Interaction . https://research.google/pubs/pub52594/ -
  9. [12][12] Frank Elavsky, Cynthia Bennett, and Dominik Moritz. 2022. How accessible is my visualization? Evaluating visualization accessibility with Chartability. In Computer Graphics Forum , Vol. 41. Wiley Online Library, 57-70. -
  10. [11][11] Peitong Duan, Casimir Wierzynski, and Lama Nachman. 2020. Optimizing User Interface Layouts via Gradient Descent. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20) . Association for Computing Machinery, New York, NY, USA, 1-12. https: //doi.org/10.1145/3313831.3376589 - · DOI
  11. [15][15] C. Ailie Fraser, Tricia J. Ngoon, Ariel S. Weingarten, Mira Dontcheva, and Scott Klemmer. 2017. CritiqueKit: A Mixed-Initiative, Real-Time Interface For Improving Feedback. In Adjunct Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST '17 Adjunct) . Association for Computing Machinery, New York, NY, USA, 7-9. https://doi.org/10.1145/3131785.3131791 - · DOI
  12. [14][14] Camilo Fosco, Vincent Casser, Amish Kumar Bedi, Peter O'Donovan, Aaron Hertzmann, and Zoya Bylinskii. 2020. Predicting Visual Importance Across Graphic Design Types. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST '20) . Association for Computing Machinery, New York, NY, USA, 249-260. https://doi.org/10.1145/ 3379337.3415825 -
  13. [18][18] Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023. Evaluating Large Language Models in Generating Synthetic HCI Research Data: A Case Study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23) . Association for Computing Machinery, New York, NY, USA, Article 433, 19 pages. https://doi.org/10.1145/3544548.3580688 - · DOI · Paper page
  14. [19][19] Tae Soo Kim, DaEun Choi, Yoonseo Choi, and Juho Kim. 2022. Stylette: Styling the Web with Natural Language. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22) . Association for Computing Machinery, New York, NY, USA, Article 5, 17 pages. https: //doi.org/10.1145/3491102.3501931 - · DOI
  15. [20][20] Chinmay E. Kulkarni, Michael S. Bernstein, and Scott R. Klemmer. 2015. PeerStudio: Rapid Peer Feedback Emphasizes Revision and Improves Performance. In Proceedings of the Second (2015) ACM Conference on Learning @ Scale (Vancouver, BC, Canada) (L@S '15) . Association for Computing Machinery, New York, NY, USA, 75-84. https://doi.org/10.1145/2724660.2724670 - · DOI · Paper page
  16. [21][21] Sri Kurniawan and Panayiotis Zaphiris. 2005. Research-Derived Web Design Guidelines for Older People. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Baltimore, MD, USA) (Assets '05) . Association for Computing Machinery, New York, NY, USA, 129-135. https://doi.org/10.1145/1090785.1090810 - · DOI
  17. [23][23] Chunggi Lee, Sanghoon Kim, Dongyun Han, Hongjun Yang, Young-Woo Park, Bum Chul Kwon, and Sungahn Ko. 2020. GUIComp: A GUI Design Assistant with Real-Time, Multi-Faceted Feedback. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20) . Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/ 3313831.3376327 -
  18. [25][25] Chen Ling and Gavriel Salvendy. 2009. Effect of evaluators' cognitive style on heuristic evaluation: Field dependent and field independent evaluators. International Journal of Human-Computer Studies 67, 4 (2009), 382-393. https: //doi.org/10.1016/j.ijhcs.2008.11.002 - · DOI
  19. [26][26] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruction Tuning. arXiv:2304.08485 [cs.CV] - · arXiv
  20. [27][27] Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. 2023. Chatting with GPT-3 for Zero-Shot HumanLike Mobile Automated GUI Testing. arXiv:2305.09434 [cs.SE] - · arXiv
  21. [29][29] Kurt Luther, Amy Pavel, Wei Wu, Jari-lee Tolentino, Maneesh Agrawala, Björn Hartmann, and Steven P. Dow. 2014. CrowdCrit: Crowdsourcing and Aggregating Visual Design Critique. In Proceedings of the Companion Publication of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA) (CSCW Companion '14) . Association for Computing Machinery, New York, NY, USA, 21-24. https://doi.org/10.1145/2556420.2556788 - · DOI
  22. [31][31] Jennifer Mankoff, Anind K. Dey, Gary Hsieh, Julie Kientz, Scott Lederer, and Morgan Ames. 2003. Heuristic Evaluation of Ambient Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03) . Association for Computing Machinery, New York, NY, USA, 169-176. https://doi.org/10.1145/642611.642642 - · DOI
  23. [30][30] Kurt Luther, Jari-Lee Tolentino, Wei Wu, Amy Pavel, Brian P. Bailey, Maneesh Agrawala, Björn Hartmann, and Steven P. Dow. 2015. Structuring, Aggregating, and Evaluating Crowdsourced Design Critique. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW '15) . Association for Computing Machinery, New York, NY, USA, 473-485. https://doi.org/10.1145/2675133.2675283 - · DOI
  24. [32][32] Andrii Matviienko, Maria Rauschenberger, Vanessa Cobus, Janko Timmermann, Heiko Müller, Jutta Fortmann, Andreas Löcken, Christoph Trappe, Wilko Heuten, and Susanne Boll. 2015. Deriving Design Guidelines for Ambient Light Systems. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia (Linz, Austria) (MUM '15) . Association for Computing Machinery, New York, NY, USA, 267-277. https://doi.org/10.1145/2836041.2836069 - · DOI
  25. [34][34] Aliaksei Miniukovich, Antonella De Angeli, Simone Sulpizio, and Paola Venuti. 2017. Design Guidelines for Web Readability. In Proceedings of the 2017 Conference on Designing Interactive Systems (Edinburgh, United Kingdom) (DIS '17) . Association for Computing Machinery, New York, NY, USA, 285-296. https://doi.org/10.1145/3064663.3064711 - · DOI
  26. [33][33] Na Mi, Lora Anne Cavuoto, Kenneth Benson, Tonya L. Smith-Jackson, and Maury A. Nussbaum. 2014. A heuristic checklist for an accessible smartphone interface design. Universal Access in the Information Society 13 (2014), 351-365. https://api.semanticscholar.org/CorpusID:16948811 -
  27. [35][35] Christine Murad, Cosmin Munteanu, Leigh Clark, and Benjamin R. Cowan. 2018. Design Guidelines for Hands-Free Speech Interaction. In Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (Barcelona, Spain) (MobileHCI '18) . Association for Computing Machinery, New York, NY, USA, 269-276. https://doi.org/10.1145/3236112. 3236149 - · DOI
  28. [36][36] Tricia J. Ngoon, C. Ailie Fraser, Ariel S. Weingarten, Mira Dontcheva, and Scott Klemmer. 2018. Interactive Guidance Techniques for Improving Creative Feedback. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18) . Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/3173574.3173629 - · DOI
  29. [37][37] Tricia J. Ngoon, Joy O Kim, and Scott Klemmer. 2021. Shöwn: Adaptive Conceptual Guidance Aids Example Use in Creative Tasks. In Proceedings of the 2021 ACM Designing Interactive Systems Conference (Virtual Event, USA) (DIS '21) . Association for Computing Machinery, New York, NY, USA, 1834-1845. https://doi.org/10.1145/3461778.3462072 - · DOI
  30. [38][38] Jakob Nielsen. 1992. Finding Usability Problems through Heuristic Evaluation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Monterey, California, USA) (CHI '92) . Association for Computing Machinery, New York, NY, USA, 373-380. https://doi.org/10.1145/142750.142834 - · DOI
  31. [40][40] Jakob Nielsen and Rolf Molich. 1990. Heuristic Evaluation of User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Washington, USA) (CHI '90) . Association for Computing Machinery, New York, NY, USA, 249-256. https://doi.org/10.1145/97243.97281 - · DOI · Paper page
  32. [42][42] OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] - · arXiv
  33. [41][41] Jasper O'Leary, Holger Winnemöller, Wilmot Li, Mira Dontcheva, and Morgan Dixon. 2018. Charrette: Supporting In-Person Discussions around Iterations in User Interface Design. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18) . Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/ 3173574.3174109 -
  34. [43][43] Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. arXiv:2304.03442 [cs.HC] - · arXiv
  35. [45][45] Amy Pavel, Dan B. Goldman, Björn Hartmann, and Maneesh Agrawala. 2016. VidCrit: Video-Based Asynchronous Video Review. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST '16) . Association for Computing Machinery, New York, NY, USA, 517-528. https://doi.org/10.1145/2984511.2984552 - · DOI
  36. [44][44] Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2022. Social Simulacra: Creating Populated Prototypes for Social Computing Systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology (Bend, OR, USA) (UIST '22) . Association for Computing Machinery, New York, NY, USA, Article 74, 18 pages. https://doi.org/10.1145/3526113.3545616 - · DOI · Paper page
  37. [46][46] Savvas Petridis, Michael Terry, and Carrie Jun Cai. 2023. PromptInfuser: Bringing User Interface Mock-Ups to Life with Large Language Models. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA '23) . Association for Computing Machinery, New York, NY, USA, Article 237, 6 pages. https://doi.org/10.1145/3544549.3585628 - · DOI
  38. [48][48] D Royce Sadler. 1989. Formative assessment and the design of instructional systems. Instructional science 18 (1989), 119-144. -
  39. [47][47] David Pinelle, Nelson Wong, and Tadeusz Stach. 2008. Heuristic Evaluation for Games: Usability Principles for Video Game Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI '08) . Association for Computing Machinery, New York, NY, USA, 1453-1462. https://doi.org/10.1145/1357054.1357282 - · DOI
  40. [49][49] Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. 2022. Multitask Prompted Training Enables Zero-Shot Task Generalization. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . https://openreview.net/forum?id=9Vrb9D0WI4 -
  41. [51][51] Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366 [cs.AI] - · arXiv
  42. [50][50] Eldon Schoop, Xin Zhou, Gang Li, Zhourong Chen, Bjoern Hartmann, and Yang Li. 2022. Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI '22) . Association for Computing Machinery, New York, NY, USA, Article 36, 21 pages. https: //doi.org/10.1145/3491102.3517497 - · DOI
  43. [52][52] Amanda Swearngin and Yang Li. 2019. Modeling Mobile Interface Tappability Using Crowdsourcing and Deep Learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19) . Association for Computing Machinery, New York, NY, USA, 1-11. https://doi.org/10.1145/3290605.3300305 - · DOI
  44. [53][53] Amanda Swearngin, Chenglong Wang, Alannah Oleson, James Fogarty, and Amy J. Ko. 2020. Scout: Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints . Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3313831.3376593 - · DOI
  45. [54][54] Kashyap Todi, Daryl Weir, and Antti Oulasvirta. 2016. Sketchplore: Sketch and Explore with a Layout Optimiser. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (Brisbane, QLD, Australia) (DIS '16) . Association for Computing Machinery, New York, NY, USA, 543-555. https://doi.org/10.1145/ 2901790.2901817 -
  46. [56][56] Bryan Wang, Gang Li, and Yang Li. 2023. Enabling Conversational Interaction with Mobile UI Using Large Language Models. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI '23) . Association for Computing Machinery, New York, NY, USA, Article 432, 17 pages. https://doi.org/10.1145/3544548.3580895 - · DOI · Paper page
  47. [55][55] Norman G. Vinson. 1999. Design Guidelines for Landmarks to Support Navigation in Virtual Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI '99) . Association for Computing Machinery, New York, NY, USA, 278-285. https://doi.org/10.1145/302979.303062 - · DOI
  48. [57][57] Jeremy Warner, Amy Pavel, Tonya Nguyen, Maneesh Agrawala, and Bjoern Hartmann. 2023. SlideSpecs: Automatic and Interactive Presentation Feedback Collation. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI '23) . Association for Computing Machinery, New York, NY, USA, 695-709. https://doi.org/10.1145/3581641.3584035 - · DOI
  49. [59][59] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 [cs.CL] - · arXiv
  50. [58][58] Dominik Weber, Sven Mayer, Alexandra Voit, Rodrigo Ventura Fierro, and Niels Henze. 2016. Design Guidelines for Notifications on Smart TVs. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video (Chicago, Illinois, USA) (TVX '16) . Association for Computing Machinery, New York, NY, USA, 13-24. https://doi.org/10.1145/2932206.2932212 - · DOI
  51. [60][60] Dongwook Yoon, Nicholas Chen, François Guimbretière, and Abigail Sellen. 2014. RichReview: Blending Ink, Speech, and Gesture to Support Collaborative Document Review. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST '14) . Association for Computing Machinery, New York, NY, USA, 481-490. https://doi.org/10. 1145/2642918.2647390 -
  52. [61][61] Panayiotis Zaphiris, Mariya Ghiawadwala, and Shabana Mughal. 2005. AgeCentered Research-Based Web Design Guidelines. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (Portland, OR, USA) (CHI EA '05) . Association for Computing Machinery, New York, NY, USA, 1897-1900. https://doi.org/10.1145/1056808.1057050 · DOI

Cited by

Sorted by citations

Authors
Duan, P., Warner, J., Li, Y., & Hartmann, B.
Affiliations
  • University of California, Berkeley (US)
  • Google (United States) (US)
  • EECS, UC Berkeley, United States
ORCID
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-20
DOI
10.1145/3613904.3642782
OpenAlex
W4393064140
Cited by
83
References
7
Citations by year
2026: 39 · 2025: 40 · 2024: 4
FWCI
26.75
Type
conference paper
Language
en
Open access
Licence
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Keywords
automated feedback, large language models, GPT-4, heuristic evaluation, design guidelines, user interface design, expert designers, usability evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review